Wiley Encyclopedia of Clinical Trials

1. Abbreviated New Drug Application (ANDA) 2. Absolute Risk Reduction 3. Accelerated Approval 4. Adverse Drug Reaction Reporting 5. Adverse Event Evaluation 6. Adverse Event Report System (AERS) 7. Advisory Committees 8. AIDS Clinical Trials Group (ACTG) 9. Algorithm-Based Designs 10. Aligned Rank Test 11. Allocation Concealment 12. Alpha-Spending Function 13. Analysis of Variance ANOVA 14. Analysis Population 15. Application of New Designs in Phase I Trials 16. ASCOT Trial 17. Assay Sensitivity 18. Assessment Bias 19. Assessment of Health-Related Quality of Life 20. Audit 21. Audit Certificate 22. Audit Report 23. Bayesian Dose-Finding Designs in Healthy Volunteers 24. Benefit/Risk Assessment in Prevention Trials 25. Biased-Coin Randomization 26. Bioequivalence (BE) Testing for Generic Drugs 27. Biological Assay, Overview 28. Block Randomization 29. Bootstrap 30. Cardiac Arrhythmia Suppression Trial (CAST) 31. Categorical Response Models 32. Causal Inference 33. Cell Line 34. Censored Data 35. Center for Devices and Radiological Health (CDRH) 36. Center for Drug Evaluation and Research (CDER) 37. Central Nervous System (CNS) 38. CFR 21 Part 11 39. Change/Percent Change From Baseline 40. Chemistry, Manufacturing and Controls (CMC) 41. Citizen Petition 42. Clinical Data Coordination 43. Clinical Data Management 44. Clinical Hold Decision 45. Clinical Significance 46. Clinical Trial Misconduct 47. Clinical Trials, Early Cancer and Heart Disease 48. Clinical Trials to Support Prescription to Over-the-Counter Switches 49. Cluster Randomization 50. Code of Federal Regulations (CFR)

51. Coherence in Phase I Clinical Trials 52. Cohort vs. Repeated Cross-Sectional Survey Designs 53. Collinearity 54. Combination Therapy 55. Committee for Medicinal Product for Human Use (CHMP) 56. Common Technical Document (CTD) 57. Community-Based Breast and Cervical Cancer Control Research in Asian Immigrant Populations 58. Compliance and Survival Analysis 59. Composite Endpoints in Clinical Trials 60. Computer-Assisted Data Collection 61. Conditional Power in Clinical Trial Monitoring 62. Confidence Interval 63. Confidence Intervals and Regions 64. Confirmatory Trials 65. Confounding 66. CONSORT 67. Contract Research Organization (CRO) 68. Control Groups 69. Cooperative North Scandinavian Enalapril Survival Study (CONSENSUS) 70. Cooperative Studies Program, US Department of Veterans Affairs 71. Coordinating Committee 72. Coordinating Investigator 73. Coronary Drug Project 74. Correlation 75. Cost-Effectiveness Analysis 76. Covariates 77. Cox Proportional Hazard Model 78. Cronbach's Alpha 79. Crossover Design 80. Crossover Trials 81. Data Mining 82. Data Monitoring Committee 83. Data Safety and Monitoring Board (DSMB) 84. Data Standards 85. Dermatology Trials 86. Designs with Randomization Following Initial Study Treatment 87. Diagnostic Studies 88. Discriminant Analysis 89. Disease Trials for Dental Drug Products 90. Disease Trials in Reproductive Diseases 91. Disease Trials on Pediatric Patients 92. DNA Bank 93. Dose Escalation and Up-and-Down Designs 94. Dose Escalation Guided by Graded Toxicities 95. Dose Finding Studies 96. Dose Ranging Crossover Designs 97. Double-Dummy 98. Drift (For Interim Analysis) 99. Drug Development

100. Drug Packaging 101. Drug Registration and Listing System (DRLS) 102. Drug Supply 103. Eastern Cooperative Oncology Group (ECOG) 104. Eligibility and Exclusion Criteria 105. Emergency Use Investigational New Drug (IND) 106. End of Phase II Meeting 107. End-of-Phase I Meeting 108. Enrichment Design 109. Environmental Assessments (EAs) 110. Equivalence Trials and Equivalence Limits1 Essential Documents 111. 112. Ethical Challenges Posed by Cluster Randomization 113. Ethical Issues in International Research 114. European Medicines Agency (EMEA) 115. European Organization for Research and Treatment of Cancer (EORTC) 116. Factor Analysis: Confirmatory 117. Factorial Designs in Clinical Trials 118. Fast Track 119. FDA Division of Pharmacovigilance and Epidemiology (DPE) 120. FDA Modernization Act (FDAMA) of 1997 121. Federal Food, Drug and Cosmetic Act 122. Federal Register 123. Fileable New Drug Application (NDA) 124. Financial Disclosure 125. Fisher's Exact Test 126. Flexible Designs 127. Food and Drug Administration (FDA) 128. Frailty Models 129. Futility Analysis 130. Generalized Estimating Equations 131. Generalized Linear Models 132. Generic Drug Review Process 133. Gene Therapy 134. Genetic Association Analysis 135. Global Assessment Variables 136. Gold Standard 137. Good Clinical Practice (GCP) 138. Good Laboratory Practice (GLP) 139. Goodness of Fit 140. Group-Randomized Trials 141. Group Sequential Designs 142. Hazard Rate 143. Hazard Ratio 144. Heritability 145. Historical Control 146. Hypothesis 147. Hypothesis Testing

148. Identifying the Most Successful Dose (MSD) in Dose-Finding Studies 149. Imaging Science in Medicine 150. Imputation 151. Incompetent Subjects and Proxy Consent 152. Independent Ethics Committee (IEC) 153. Inference, Design-Based vs. Model-Based 154. Informed Consent Process, Forms, and Assent 155. Institution 156. Institutional and Independent Review Boards 157. Institutional Review Boards (IRB) 158. Integrated Database 159. Intention-to-Treat Analysis 160. Interaction Model 161. Interim Analyses 162. Interim Clinical Trial/Study Report 163. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) 164. International Studies of Infarct Survival (ISIS) 165. Interrater Reliability 166. Interval Censored 167. Intraclass Correlation Coefficient 168. Intrarater Reliability 169. Investigational Device Exemption (IDE) 170. Investigational New Drug Application Process (IND) 171. Investigational Product 172. Investigator 173. Investigator/Institution 174. Investigator's Brochure 175. Kaplan–Meier Plot 176. Kappa 177. Kefauver–Harris Drug Amendments 178. Lan-DeMets Alpha-Spending Function 179. Large Simple Trials 180. Linear Model 181. Logistic Regression 182. Logrank Test 183. Longitudinal Data 184. Manual of Policies and Procedures (MaPPs) 185. Masking 186. Maximum Duration and Information Trials 187. Maximum Tolerable Dose (MTD) 188. Metadata 189. Methods for Conduct of Rigorous Group-Randomization 190. Microarray 191. Minimum Effective Dose (MinED) 192. Ministry of Health, Labour and Welfare (MHLW, Japan) 193. Min Test 194. Missing Data Monitoring 195. 196. Monotherapy

197. Mother to Child Human Immunodeficiency Virus Transmission Trials 198. Multicenter Trial 199. Multinational (Global) Trial 200. Multiple Comparisons 201. Multiple Endpoints 202. Multiple Evaluators 203. Multiple Risk Factor Intervention Trial (MRFIT) 204. Multiple Testing in Clinical Trials 205. Multistage Genetic Association Studies 206. National Cancer Institute (NCI) 207. National Center for Toxicological Research (NCTR) 208. National Cooperative Gallstone Study 209. National Eye Institute (NEI) 210. National Heart, Lung, and Blood Institute (NHLBI) 211. National Human Genome Research Institute (NHGRI) 212. National Institute of Allergy and Infectious Disease (NIAID) 213. National Institute of Arthritis and Musculoskeletal and Skin Diseases (NIAMS) 214. National Institute of Biomedical Imaging and Bioengineering (NIBIB) 215. National Institute of Child Health and Human Development (NICHD) 216. National Institute of Dental and Craniofacial Research (NIDCR) 217. National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) 218. National Institute of Environmental Health Science (NIEHS) 219. National Institute of General Medical Sciences (NIGMS) 220. National Institute of Mental Health (NIMH) 221. National Institute of Neurological Disorders and Stroke (NINDS) 222. National Institute of Nursing Research (NINR) 223. National Institute on Aging (NIA) 224. National Institute on Alcohol Abuse and Alcoholism (NIAAA) 225. National Institute on Deafness and Other Communication Disorders (NIDCD) 226. National Institute on Drug Abuse (NIDA) 227. National Institutes of Health (NIH) 228. National Institutes of Health Stroke Scale (NIHSS) 229. National Library of Medicine (NLM) 230. New Drug Application (NDA) Process 231. N of 1 Randomized Trial 232. Non-Compartmental Analysis 233. Noncompliance 234. Non-Inferiority Trial 235. Nonparametric Methods Non-Randomized Trial 236. 237. Objectives 238. Office of Orphan Products Development (OOPD) 239. Office of Pharmacoepidemiology and Statistical Science (OPaSS) 240. Office of Regulatory Affairs (ORA) 241. One-Sided Versus Two-Sided Tests 242. Open-Labeled Trial 243. Optimal Biological Dose for Molecularly-Targeted Therapies

244. 245. 246. 247. 248. 249. 250. 251. 252. 253. 254. 255. 256. 257. 258. 259. 260. 261. 262. 263. 264. 265. 266. 267. 268. 269. 270. 271. 272. 273. 274. 275. 276. 277. 278. 279. 280. 281. 282. 283. 284. 285. 286. 287. 288. 289. 290. 291. 292. 293.

Optimizing Schedule of Administration in Phase I Clinical Trials Orphan Drug Act (ODA) Orphan Drugs Orphan Products Grant Program Outliers Over dispersion Over-the-Counter (OTC) Drug Product Review Process Over the Counter (OTC) Drugs Overview of Anti-Infective Drug Development Overview of Safety Pharmacology Paired t Test Parallel Track Partially Balanced Designs “Patients” vs. “Subjects” Permutation Tests in Clinical Trials Pharmacoepidemiology, Overview Pharmacovigilance Phase 2/3 Trials Phase I/II Clinical Trials Phase III Trials Phase II Trials Phase I Trials Phase I Trials in Oncology Phase IV Trials Physicians' Health Study (PHS) Placebos Planning A Group-Randomized Trial Poisson Regression Population Pharmacokinetic and Pharmacodynamic Methods Postmenopausal Estrogen/Progestin Intervention Trial (PEPI) Power Transformations Predicting Random Effects in Community Intervention Preference Trials Premarket Approval (PMA) Premarket Notification 510(k) Premature Termination or Suspension Pre-NDA Meeting Prescription Drug User Fee Act (PDUFA) Prescription Drug User Fee Act (PDUFA) II Prescription Drug User Fee Act (PDUFA) IV Prescription Drug User Fee Act III Prevention Trials Primary Efficacy Endpoint Priority Review Prognostic Variables in Clinical Trials Propensity Score Proportional Odds Model Proportions, Inferences, and Comparisons Prostate Cancer Prevention Trial Protocol

294. Protocol Amendment 295. Protocol Deviators 296. Protocol Violators 297. Publication Bias 298. P Value 299. Quality Assessment Of Clinical Trials 300. Quality Assurance 301. Quality Control 302. Quality of Life Query Management: The Route to a Quality Database 303. 304. Question-Based Review (QbR) 305. Randomization-Based Nonparametric Analysis of Covariance 306. Randomization Procedures 307. Randomization Schedule 308. Rank-Based Nonparametric Analysis of Covariance Record Access 309. 310. Refuse to File Letter 311. Registration of Drug Establishment Form 312. Regression 313. Regression Models to Incorporate Patient Heterogeneity 314. Regression to the Mean 315. Regulatory Authorities 316. Regulatory Definitions 317. Relative Risk Modeling 318. Reliability Study 319. Repeatability and Reproducibility 320. Repeated Measurements 321. Repository 322. Response Adaptive Randomization 323. Response Surface Methodology 324. Risk Assessment 325. Robust Two-Stage Model-Guided Designs for Phase I Clinical Studies 326. Run-In Period 327. Safety Information 328. Sample Size Calculation for Comparing Means 329. Sample Size Calculation for Comparing Proportions 330. Sample Size Calculation for Comparing Time-to-Event Data 331. Sample Size Calculation for Comparing Variabilities 332. Sample Size Considerations for Morbidity/Mortality Trials 333. Screening, Models of 334. Screening Trials 335. Secondary Efficacy Endpoints 336. Sensitivity, Specificity and Receiver Operator Characteristic (ROC) Methods 337. Sequential Analysis 338. Serious Adverse Event (SAE) 339. Simple Randomization 340. Software for Genetics/Genomics 341. Sponsor 342. Sponsor-Investigator

343. Spontaneous Reporting System (SRS) 344. Stability Analysis 345. Stability Study Designs 346. Standard Operating Procedures (SOP) 347. Statins 348. Stepped Wedge Design 349. Stopping Boundaries 350. Stratification 351. Stratified Designs Stratified Randomization 352. 353. Subgroup 354. Subgroup Analysis 355. Subinvestigator 356. Superiority Trial Surrogate Endpoints 357. 358. Survival Analysis, Overview 359. Suspension or Termination of IRB Approval 360. The Belmont Report 361. The Carotene and Retinol Efficacy Trial (Caret) 362. The Center for Biologics Evaluation and Research 363. The Cochrane Collaboration 364. The Community Intervention Trial for Smoking Cessation (COMMIT) 365. The FDA and Regulatory Issues 366. Therapeutic Dose Range 367. Therapeutic Equivalence 368. Therapeutic Index 369. TNT Trial 370. Treatment-by-Center Interaction 371. Treatment Interruption 372. Treatment Investigational New Drug (IND) 373. Trial Site 374. True Positives, True Negatives, False Positives, False Negatives 375. UGDP Trial 376. Update in Hyperlipidemia Clinical Trials 377. Using Internet in Community Intervention Studies1 378. Vaccine 379. Vaccine Adverse Event Report System (VAERS) Web Based Data Management System 380. 381. Wei-Lin-Weissfeld Method for Multiple Times to Events 382. Wilcoxon–Mann–Whitney Test 383. Wilcoxon Signed-Rank Test 384. Women's Health Initiative: Statistical Aspects and Selected Early Results 385. Women's Health Initiative Dietary Trial 386. Women's Health Initiative Hormone Therapy Trials 387. World Health Organization (WHO): Global Health Situation

ABBREVIATED NEW DRUG APPLICATION (ANDA)

patent protection for the new medicines they developed to make up for time lost while their products were going through the FDA’s approval process. Brand-name drugs are subject to the same bioequivalence tests as generics upon reformulation.

An Abbreviated New Drug Application (ANDA) contains data that when submitted to the U.S. Food and Drug Administration’s (FDA) Center for Drug Evaluation and Research, Office of Generic Drugs provides for the review and ultimate approval of a generic drug product. Once approved, an applicant may manufacture and market the generic drug product to provide a safe, effective, low-cost alternative to the American public. Generic drug applications are termed ‘‘abbreviated’’ because they are generally not required to include preclinical (animal) and clinical (human) data to establish safety and effectiveness. Instead, generic applicants must scientifically demonstrate that their product is bioequivalent (i.e., performs in the same manner as the innovator drug). One way scientists demonstrate bioequivalence is to measure the time it takes the generic drug to reach the bloodstream in 24 to 36 healthy volunteers. This gives them the rate of absorption, or bioavailability, of the generic drug, which they can then compare with that of the innovator drug. The generic version must deliver the same amount of active ingredients into a patient’s bloodstream in the same amount of time as the innovator drug. Using bioequivalence as the basis for approving generic copies of drug products was established by the ‘‘Drug Price Competition and Patent Term Restoration Act of 1984,’’ also known as the Waxman-Hatch Act. This Act expedites the availability of less costly generic drugs by permitting the FDA to approve applications to market generic versions of brand-name drugs without conducting costly and duplicative clinical trials. At the same time, the brand-name companies can apply for up to 5 additional years longer This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/regulatory/applications/ ANDA.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

ABSOLUTE RISK REDUCTION

studies or in cross-sectional designs. (Some hypothesis testing procedures such as the Fisher exact test are conditional also on the total number of ‘‘successes’’ in the two groups combined. This alternative conditioning is inappropriate for confidence intervals for a difference of proportions; in particular in the event that no successes are observed in either group, this approach fails to produce an interval.) The outcome variable is binary: 1 if the event of interest occurs, 0 if it does not. (We do not consider here the case of an integer-valued outcome variable; typically, this involves the number of episodes of relapse or hospitalization, number of accidents, or similar events occurring within a defined follow-up period. Such an outcome would instead be modeled by the Poisson distribution.) We observe that r1 subjects in group 1 and r2 subjects in group 2 experience the event of interest. Then the proportions having the event in the two groups are given by p1 = r1 /n1 and p2 = r2 /n2 . If responses in different individuals in each group are independent, then the distribution of the number of events in each group is binomial. Several effect size measures are widely used for comparison of two independent proportions:

ROBERT G. NEWCOMBE Cardiff University Cardiff, Wales, United Kingdom

Many response variables in clinical trials are binary: the treatment was successful or unsuccessful; the adverse effect did or did not occur. Binary variables are summarized by proportions, which may be compared between different arms of a study by calculating either an absolute difference of proportions or a relative measure, the relative risk or the odds ratio. In this article we consider several point and interval estimates for the absolute difference between two proportions, for both unpaired and paired study designs. The simplest methods encounter problems when numerators or denominators are small; accordingly, better methods are introduced. Because confidence interval methods for differences of proportions are derived from related methods for the simpler case of the single proportion, which itself can also be of interest in a clinical trial, this case is also considered in some depth. Illustrative examples relating to data from two clinical trials are shown. 1

Difference of proportions p1 – p2 Ratio of proportions (risk ratio or relative risk) p1 /p2 Odds ratio (p1 /(1 – p1 ))/(p2 /(1 – p2 ))

PRELIMINARY ISSUES

In most clinical trials, the unit of data is the individual, and statistical analyses for efficacy and safety outcomes compare responses between the two (or more) treatment groups. When subjects are randomized between these groups, responses of subjects in one group are independent of those in the other group. This leads to unpaired analyses. Crossover and split-unit designs require paired analyses. These have many features in common with the unpaired analyses and will be described in the final section. Thus, we study n1 individuals in group 1 and n2 individuals in group 2. Usually, all analyses are conditional on n1 and n2 . Analyses conditional on n1 and n2 would also be appropriate in other types of prospective

In this article we consider in particular the difference between two proportions, p1 – p2 , as a measure of effect size. This is variously referred to as the absolute risk reduction, risk difference, or success rate difference. Other articles in this work describe the risk ratio or relative risk and the odds ratio. We consider both point and interval estimates, in recognition that ‘‘confidence intervals convey information about magnitude and precision of effect simultaneously, keeping these two aspects of measurement closely linked’’ (1). In the clinical trial context, a difference between two proportions is often referred to as an

Wiley Encyclopedia of Clinical Trials, Copyright © 2007, John Wiley & Sons, Inc.

1

2

ABSOLUTE RISK REDUCTION

absolute risk reduction. However, it should be borne in mind that any term that includes the word ‘‘reduction’’ really presupposes that the direction of the difference will be a reduction in risk—such terminology becomes awkward when the anticipated benefit does not materialize, including the nonsignificant case when the confidence interval for the difference extends beyond the null hypothesis value of zero. The same applies to the relative risk reduction, 1 – p1 /p2 . Whenever results are presented, it is vitally important that the direction of the observed difference should be made unequivocally clear. Moreover, sometimes confusing labels are used, which might be interpreted to mean something other than p1 – p2 ; for example, Hashemi et al. (2) refer to p1 – p2 as attributable risk. It is also vital to distinguish between relative and absolute risk reduction. In clinical trials, as in other prospective and cross-sectional designs already described, each of the three quantities we have discussed may validly be used as a measure of effect size. The risk difference and risk ratio compare two proportions from different perspectives. A halving of risk will have much greater population impact for a common outcome than for an infrequent one. Schechtman (3) recommends that both a relative and an absolute measure should always be reported, with appropriate confidence intervals. The odds ratio is discussed at length by Agresti (4). It is widely regarded as having a special preferred status on account of its role in retrospective case-control studies and in logistic regression and meta-analysis. Nevertheless, it should not be regarded as having gold standard status as a measure of effect size for the 2 × 2 table (3, 5). 2 POINT AND INTERVAL ESTIMATES FOR A SINGLE PROPORTION Before considering the difference between two independent proportions in detail, we first consider some of the issues that arise in relation to the fundamental task of estimating a single proportion. These issues have repercussions for the comparison of proportions because confidence interval methods for p1 – p2 are generally based closely on those

for proportions. The single proportion is also relevant to clinical trials in its own right. For example, in a clinical trial comparing surgical versus conservative management, we would be concerned with estimating the incidence of a particular complication of surgery such as postoperative bleeding, even though there is no question of obtaining a contrasting value in the conservative group or of formally comparing these. The most commonly used estimator for the population proportion π is the familiar empirical estimate, namely, the observed proportion p = r/n. Given n, the random variable R denoting the number of subjects in which the response occurs has the binomial B(n,p) distribution, with Pr[R = r] = {n!/r!(n – r!)} pr qn – r where q = 1 – p. The simple empirical estimator is also the maximum likelihood estimate for the binomial distribution, and it is unbiased— in the usual statistical sense that the expectation of R given n, E[R|n] = π. However, when r = 0, many users of statistical methods are uneasy with the idea that p = 0 is an unbiased estimate. The range of possible values for π is the interval from 0 to 1. Generally, this means the open interval 0 < π < 1, not the closed interval 0 ≤ π ≤ 1, as usually it would already be known that the event sometimes occurs and sometimes does not. As the true value of π cannot then be negative or zero but must be greater than zero, the notion that p = 0 should be regarded as an unbiased estimate of π seems highly counterintuitive. Largely with this issue in mind, alternative estimators known as shrinkage estimators are available. These generally take the form pψ = (r + ψ)/(n + 2ψ) for some ψ > 0. The quantity ψ is known as a pseudo-frequency. Essentially, ψ observations are added to the number of ‘‘successes’’ and also to the number of ‘‘failures.’’ The resulting estimate pψ is intermediate between the empirical estimate p = r/n and 1/2, which is the midpoint and center of symmetry of the support scale from 0 to 1. The degree of shrinkage toward 1/2 is great when n is small and minor for large n. Bayesian analyses of proportions lead naturally to shrinkage estimators, with ψ = 1

ABSOLUTE RISK REDUCTION

and 1/2 corresponding to the most widely used uninformative conjugate priors, the uniform prior B(1,1) and the Jeffreys prior B( 1/2, 1/2). It also is important to report confidence intervals, to express the uncertainty due to sampling variation and finite sample size. The simplest interval p ± z × SE(p), where √ SE(p) = (pq/n), remains the most commonly used. This is usually called the Wald interval. Here, z denotes the relevant quantile of the standard Gaussian distribution. Standard practice is to use intervals that aim to have 95% coverage, with 2.5% noncoverage in each tail, leading to z = 1.9600. Unfortunately, confidence intervals for proportions and their differences do not achieve their nominal coverage properties. This is because the sample space is discrete and bounded. The Wald method for the single proportion has three unfavorable properties (6–9). These can all be traced to the interval’s simple symmetry about the empirical estimate. The achieved coverage is much lower than the nominal value. For some values of π , the achieved coverage probability is close to zero. The noncoverage probabilities in the two tails are very different. The location of the interval is too distal—too far out from the center of symmetry of the scale, 1/2. The noncoverage of the interval is predominantly mesial. The calculated limits often violate the boundaries at 0 and 1. In particular, when r = 0, a degenerate, zero-width interval results. For small non-zero values of r (1, 2, and sometimes 3 for a 95% interval), the calculated lower limit is below zero. The resulting interval is usually truncated at zero, but this is unsatisfactory as the data tells us that 0 is an impossible value for π . Corresponding anomalous behavior at the upper boundary occurs when n – r is 0 or small. Many improved methods for confidence intervals for proportions have been developed. The properties of these methods are evaluated by choosing suitable parameter space points (here, combinations of n and π ), using these to generate large numbers of simulated random samples, and recording how often the resulting confidence interval includes the true value π . The resulting coverage probabilities are then summarized by

3

calculating the mean coverage and minimum coverage across the simulated datasets. Generally, the improved methods obviate the boundary violation problem, and improve coverage and location. The most widely researched options are as follows. A continuity correction may be incorpo√ rated: p ± {z (pq/n) + 1/(2n)}. This certainly improves coverage and obviates zerowidth intervals but increases the incidence of boundary overflow. The Wilson score method (10) uses the theoretical value π , not the empirical estimate p, in the formula for the standard error of p. Lower and upper limits are obtained as the two solutions of the equation p = π ± √ z × SE(π ) = π ± z × (π (1 – π )/n), which reduces to a quadratic in π . The two roots are given in closed form as √ {2p + z2 ± z (z2 + 4rq)}/{2(n + z2 )}. It is easily demonstrated (7) that the resulting interval is symmetrical on the logit scale—the other natural scale for proportions—by considering the product of the two roots for π , and likewise for 1 – π . The resulting interval is boundary respecting and has appropriate mean coverage. In contrast to the Wald interval, location is rather too mesial. The midpoint of the score interval, on the ordinary additive scale, is a shrinkage estimator with ψ = ( 1/2)z2 , which is 1.92 for the default 95% interval. With this (and also Bayesian intervals) in mind, Agresti and Coull (8) proposed a pseudo-frequency method, which adds ψ = 2 to the numbers of successes (r) and failures (n – r) before using the ordinary Wald formula. This is also a great improvement over the Wald method, and is computationally and conceptually very simple. It reduces but does not eliminate the boundary violation problem. A variety of alternatives can be formulated, with different choices for ψ, and also using something other than n + 2ψ as the denominator of the variance. Alternatively, the Bayesian approach described elsewhere in this work may be used. The resulting intervals are best

4

ABSOLUTE RISK REDUCTION

referred to as credible intervals, in recognition that the interpretation is slightly different from that of frequentist confidence intervals such as those previously described. Bayesian inference starts with a prior distribution for the parameter of interest, in this instance the proportion π . This is then combined with the likelihood function comprising the evidence from the sample to form a posterior distribution that represents beliefs about the parameter after the data have been obtained. When a conjugate prior is chosen from the beta distribution family, the posterior distribution takes a relatively simple form: it is also a beta distribution. If substantial information about π exists, an informative prior may be chosen to encapsulate this information. More often, an uninformative prior is used. The simplest is the uniform prior B(1,1), which assumes that all possible values of π between 0 and 1 start off equally likely. An alternative uninformative prior with some advantages is the Jeffreys prior B( 1/2, 1/2). Both are diffuse priors, which spread the probability thinly across the whole range of possible values from 0 to 1. The resulting posterior distribution may be displayed graphically, or may be summarized by salient summary statistics such as the posterior mean and median and selected centiles. The 2 1/2 and 97 1/2 centiles of the posterior distribution delimit the tail-based 95% credible interval. Alternatively, a highest posterior density interval may be reported. The tail-based interval is considered preferable because it produces equivalent results when a transformed scale (e.g., logit) is used (11). These Bayesian intervals perform well in a frequentist sense (12). Hence, it is now appropriate to regard them as confidence interval methods in their own right, with theoretical justification in the Bayesian paradigm but empirical validation from a frequentist standpoint. They may thus be termed beta intervals. They are readily calculated using software for the incomplete beta function, which is included in statistical packages and also spreadsheet software such as Microsoft Excel. As such, they should now be regarded as computationally of ‘‘closed form,’’ though less transparent than Wald methods.

Many statisticians consider that a coverage level should represent minimum, not average, coverage. The Clopper-Pearson ‘‘exact’’ or tail-based method (13) achieves this, at the cost of being excessively conservative; intervals are unnecessarily wide. There is a trade-off between coverage and width; it is always possible to increase coverage by widening intervals, and the aim is to attain good coverage without excessive width. A variant on the ‘‘exact’’ method involving a mid-P accumulation of tail probabilities (14, 15) aligns mean coverage closely with the nominal 1 – α. Both methods have appropriate location. The Clopper-Pearson interval, but not the mid-P one, is readily programmed as a beta interval, of similar form to Bayes intervals. A variety of shortened intervals have also been developed that maintain minimum coverage but substantially shrink interval length (16, 17). Shortened intervals are much more complex, both computationally and conceptually. They also have the disadvantage that what is optimized is the interval, not the lower and upper limits separately; consequently, they are unsuitable when interest centers on one of the limits rather than the other. Numerical examples illustrating these calculations are based on some results from a very small randomized phase II clinical trial performed by the Eastern Cooperative Oncology Group (18). Table 1 shows the results for two outcomes, treatment success defined as shrinkage of the tumor by 50% or more, and life-threatening treatment toxicity, for the two treatment groups A and B. Table 2 shows 95% confidence intervals for both outcomes for treatment A. These examples show how Wald and derived intervals often produce inappropriate limits (see asterisks) in boundary and near-boundary cases. 3 AN UNPAIRED DIFFERENCE OF PROPORTIONS We return to the unpaired difference case. As described elsewhere in this work, hypothesis testing for the comparison of two proportions takes a quite different form according to whether the objective of the trial is to

ABSOLUTE RISK REDUCTION

5

Table 1. Some Results from a Very Small Randomized Phase II Clinical Trial Performed by the Eastern Cooperative Oncology Group Treatment A

Treatment B

14 0 2

11 0 1

Number of patients Number with successful outcome: tumor shrinkage by ≥ 50% Number with life-threatening treatment toxicity Source: Parzen et al. J Comput Graph Stat. 2002: 11; 420–436.

Table 2. 95% Confidence Intervals for Proportions of Patients with Successful Outcome and With Life-Threatening Toxicity on Treatment A in the Eastern Cooperative Oncology Group Trial Outcome

Successful tumor shrinkage

Life-threatening toxicity

0 0 to 0* < 0* to 0.0357 0 to 0.2153 0.1111 < 0* to 0.2563 0 to 0.2180 0 to 0.1616 0 to 0.2316 0 to 0.1926

0.1429 < 0* to 0.3262 < 0* to 0.3619 0.0401 to 0.3994 0.2222 0.0302 to 0.4143 0.0433 to 0.4046 0.0309 to 0.3849 0.0178 to 0.4281 0.0247 to 0.3974

Empirical estimate Wald interval Wald interval with continuity correction Wilson score interval Agresti-Coull shrinkage estimate Agresti-Coull interval Bayes interval, B(1,1) prior Bayes interval, B( 1/2, 1/2) prior Clopper-Pearson ‘‘exact’’ interval Mid-P interval

Note: Asterisks denote boundary violations. Source: Parzen et al. J Comput Graph Stat. 2002: 11; 420–436.

ascertain difference or equivalence. When we report the contrast between two proportions with an appropriately constructed confidence interval, this issue is taken into account only when we come to interpret the calculated point and interval estimates. In this respect, in comparison with hypothesis testing, the confidence interval approach leads to much simpler, more flexible patterns of inference. The quantity of interest is the difference between two binomial proportions, π 1 and π 2 . The empirical estimate is p1 – p2 = r1 /n1 – r2 /n2 . It is well known that, when comparing means, there is a direct correspondence between hypothesis tests and confidence intervals. Specifically, the null hypothesis is rejected at the conventional two-tailed α = 0.05 level if and only if the 100(1 – α) = 95% confidence interval for the difference excludes the null hypothesis value of zero. A similar property applies also to the comparison of proportions—usually, but not invariably. This is because there are several options for constructing a confidence interval for the difference of proportions, which have

different characteristics and do not all correspond directly to purpose-built hypothesis tests. The Wald interval is calculated as p1 – √ p2 ± z (p1 q1 /n1 + p2 q2 /n2 ). It has poor mean and minimum coverage and fails to produce an interval when both p1 and p2 are 0 or 1. Overshoot can occur when one proportion is close to 1 and the other is close to 0, but this situation is expected to occur infrequently in practice. Use of a continuity correction improves mean coverage, but minimum coverage remains low. Several better methods have been developed, some of which are based on specific mathematical models. Any model for the comparison of two proportions necessarily involves both the parameter of interest, δ = π 1 −π 2 , and an additional nuisance parameter γ . The model may be parametrized in terms of δ and π 1 + π 2 , or δ and (π 1 + π 2 )/2, or δ and π 1 . We will define the nuisance parameter as γ = (π 1 + π 2 )/2. Some of the better methods substitute the profile estimate γ δ , which is the maximum likelihood estimate of γ conditional on a

6

ABSOLUTE RISK REDUCTION

hypothesized value of δ. These include scoretype asymptotic intervals developed by Mee (19) and Miettinen and Nurminen (20). Newcombe (21) developed tail-based exact and mid-P intervals involving substitution of the profile estimate. All these intervals are boundary respecting. The ‘‘exact’’ method aligns the minimum coverage quite well with the nominal 1 – α; the others align mean coverage well with 1 – α, at the expense of fairly complex iterative calculation. Bayesian intervals for p1 – p2 and other comparative measures may be constructed (2, 11), but they are computationally much more complex than in the single proportion case, requiring use of numerical integration or computer-intensive methodology such as Markov chain Monte Carlo (MCMC) methods. It may be more appropriate to incorporate a prior for p1 – p2 itself rather than independent priors for p1 and p2 (22). The Bayesian formulation is readily adapted to incorporate functional constraints such as δ ≥ 0 (22). Walters (23) and Agresti and Min (11) have shown that Bayes intervals for p1 – p2 with uninformative beta priors have favorable frequentist properties. Two computationally simpler, effective approaches have been developed. Newcombe (21) also formulated square-and-add intervals for differences of proportions. The concept is a very simple one. Assuming independence, the variance of a difference between two quantities is the sum of their variances. In other words, standard errors ‘‘square and add’’—they combine in the same way that differences in x and in y coordinates combine to give the Euclidean distance along the diagonal, as in Pythagoras’ theorem. This is precisely how the Wald interval for p1 – p2 is constructed. The same principle may be applied starting with other, better intervals for p1 and p2 separately. The Wilson score interval is a natural choice as it already involves square roots, though squaring and adding would work equally effectively starting with, for instance, tail-based (24) or Bayes intervals. It is easily demonstrated that the square-and-add process preserves the property of respecting boundaries. Thus, the square-and-add interval is obtained as follows. Let (li , ui ) denote the

score interval for pi , for i = 1,2. Then the square-and-add limits are p1 − p2 − p1 − p2 +

√ √

{(p1 − l1 )2 + (u2 − p2 )2 }, {(u1 − p1 )2 + (p2 − l2 )2 }.

This easily computed interval aligns mean coverage closely with the nominal 1 – α. A continuity correction is readily incorporated, resulting in more conservative coverage. Both intervals tend to be more mesially positioned than the γ δ -based intervals discussed previously. The square-and-add approach may be applied a second time to obtain a confidence interval for a difference between differences of proportions (25); this is the linear scale analogue of assessing an interaction effect in logistic regression. Another simple approach that is a great improvement over the Wald method is the pseudo-frequency method (26, 27). A pseudofrequency ψ is added to each of the four cells of the 2 × 2 table, resulting in the shrinkage estimator (r1 + ψ)/(n1 + 2ψ) – (r2 + ψ)/(n2 + 2ψ). The Wald formula then produces the limits √ pψ1 − pψ2 ± z {pψ1 (1 − pψ1 )/(n1 + 2ψ) + pψ2 (1 − pψ2 )/(n2 + 2ψ)}, where pψi = (ri + ψ)/(ni + 2ψ) i = 1, 2. Agresti and Caffo (27) evaluated the effect of choosing different values of ψ, and they reported that adding 1 to each cell is optimal here. So here, just as for the single proportion case, in total four pseudo-observations are added. This approach also aligns mean coverage effectively with 1 – α. Interval location is rather too mesial, very similar to that of the square-and-add method. Zero-width intervals cannot occur. Boundary violation is not ruled out but is expected to be infrequent. Table 3 shows 95% confidence intervals calculated by these methods, comparing treatments A and B in the ECOG trial (18).

ABSOLUTE RISK REDUCTION

7

Table 3. 95% Confidence Intervals for Differences in Proportions of Patients with Successful Outcome and with Life-Threatening Toxicity between Treatments A and B in the Eastern Cooperative Oncology Group Trial Outcome

Successful tumor shrinkage

Life-threatening toxicity

0 0* to 0* −0.2588 to 0.2153 −0.2667 to 0.2223 −0.2849 to 0.2316 −0.2384 to 0.1926 −0.2198 to 0.1685 −0.1768 to 0.1361 −0.2588 to 0.2153 −0.0144 −0.2016 to 0.1728

0.0519 −0.1980 to 0.3019 −0.2619 to 0.3312 −0.2693 to 0.3374 −0.2721 to 0.3514 −0.2539 to 0.3352 −0.2432 to 0.2986 −0.2288 to 0.3008 −0.2524 to 0.3192 0.0337 −0.2403 to 0.3076

Empirical estimate Wald interval Mee interval Miettinen-Nurminen interval Tail-based ‘‘exact’’ interval Tail-based mid-P interval Bayes interval, B(1,1) priors for p1 and p2 Bayes interval, B( 1/2, 1/2) priors for p1 and p2 Square-and-add Wilson interval Agresti-Caffo shrinkage estimate Agresti-Caffo interval

Note: Asterisks denote boundary violations. Source: Parzen et al. J Comput Graph Stat. 2002: 11; 420–436.

4

NUMBER NEEDED TO TREAT

In the clinical trial setting, it has become common practice to report the number needed to treat (NNT), defined as the reciprocal of the absolute risk difference: NNT = 1/(p1 – p2 ) (28, 29). This measure has considerable intuitive appeal, simply because we are used to assimilating proportions expressed in the form of ‘‘1 in n,’’ such as a 1 in 7 risk of lifethreatening toxicity for treatment A in Table 1. The same principle applies to differences of proportions. These tend to be small decimal numbers, often with a leading zero after the decimal point, which risk being misinterpreted by the less numerate. Thus if p1 = 0.35 and p2 = 0.24, we could equivalently report p1 – p2 = 0.11, or as an absolute difference of 11% or an NNT of 9. The latter may well be an effective way to summarize the information when a clinician discusses a possible treatment with a patient. As always, we need to pay careful attention to the direction of the difference. By default, NNT is read as ‘‘number needed to treat for (one person to) benefit,’’ or NNTB. If the intervention of interest proves to be worse than the control regime, we report the number needed to harm (NNTH). A confidence interval for the NNT may be derived from any good confidence interval method for p1 – p2 by inverting the two limits. For example, Bender (30) suggests an

interval obtained by inverting square-andadd limits (21). But it is when we turn attention to confidence intervals that the drawback of the NNT approach becomes apparent. Consider first the case of a statistically significant difference, with p1 – p2 = +0.25, and 95% confidence interval from +0.10 to +0.40. Then an NNT of 4 is reported, with 95% confidence interval from 2.5 to 10. This has two notable features. The lower limit for p1 – p2 gives rise to the upper limit for the NNT and vice versa. Furthermore, the interval is very skewed, and the point estimate is far from the midpoint. Neither of these is a serious contraindication to use of the NNT. But often the difference is not statistically significant— and, arguably, reporting confidence intervals is even more important in this case than when the difference is significant. Consider, for example, p1 – p2 = +0.10, with 95% confidence interval from –0.05 to +0.25. Here, the estimated NNT is 1/0.10 = +10. Inverting the lower and upper confidence limits for p1 − p2 gives −20 and +4. This time, the two limits do not change places apparently. But there are two problems. The point estimate, +10, is not intermediate between −20 and +4. Moreover, the interval from −20 to +4 does not comprise the values of the NNT that are compatible with the data, but rather the ones that are not compatible with it. In fact, the confidence region for the NNT in this case consists of two intervals that extend to infinity, one from + 4 to +

8

ABSOLUTE RISK REDUCTION

∞ in the direction of benefit, the other from –20 to –∞ in the direction of harm. It could be a challenge to clinicians and researchers at large to comprehend this singularity that arises when a confidence interval spanning 0 is inverted (31). Accordingly, it seems preferable to report absolute risk reductions in percentage rather than reciprocal form. The most appropriate uses of the NNT are in giving simple bottomline figures to patients (in which situation, usually only the point estimate would be given), and in labeling a secondary axis on a graph. 5

A PAIRED DIFFERENCE OF PROPORTIONS

Crossover and split-unit trial designs lead to paired analyses. Regimes that aim to produce a cure are generally not suitable for evaluation in these designs, because in the event that a treatment is effective, there would be a carryover effect into the next treatment period. For this reason, these designs tend to be used for evaluation of regimes that seek to control symptomatology, and thus most often give rise to continuous outcome measures. Examples of paired analyses of binary data in clinical trials include comparisons of different antinauseant regimes administered in randomized order during different cycles of chemotherapy, comparisons of treatments for headache pain, and split-unit studies in ophthalmology and dermatology. Results can be reported in either risk difference or NNT form, though the latter appears not to be frequently used in this context. Other examples in settings other than clinical trials include

longitudinal comparison of oral carriage of an organism before and after third molar extraction, and twin studies. Let a, b, c, and d denote the four cells of the paired contingency table. Here, b and c are the discordant cells, and interest centers on the difference of marginals: p1 − p2 = (a + b)/n − (a + c)/n = (b − c)/n. Hypothesis testing is most commonly performed using the McNemar approach (32), using either an asymptotic test statistic expressed as z or chi-square, or an aggregated tail probability. In both situations, inference is conditional on the total number of discordant pairs, b + c. Newcombe (33) reviewed confidence interval methods for the paired difference case. Many of these are closely analogous to unpaired methods. The Wald interval performs poorly. So does a conditional approach, based on an interval for the simple proportion b/(b + c). Exact and tail-based profile methods perform well; although, as before, these are computationally complex. A closedform square-and-add approach, modified to take account of the nonindependence, also aligns mean coverage with 1 – α, provided that a novel form of continuity correction is incorporated. Tango (34) developed a score interval, which is boundary respecting and was subsequently shown to perform excellently (35). Several further modifications were suggested by Tang, Tang, and Chan (36). Agresti and Min (11) proposed pseudo-frequency methods involving adding ψ = 0.5 to each cell

Table 4. Crossover Trial of Home Versus Hospital Physiotherapy: Treating Physiotherapist’s Assessment of Whether the Patient Benefited from Either Type of Treatment Number of patients benefiting from: Treatment sequence Home versus hospital Hospital versus home Difference Half

Difference (first minus second treatment)

Both

First

Second

Neither

Estimate

95% confidence interval

11 9

6 4

1 5

3 1

+ 0.2381 −0.0526 + 0.2907 + 0.1454

−0.0127 to + 0.4534 −0.3372 to + 0.2434 −0.0973 to + 0.6475 −0.0486 to + 0.3238

Source: R. G. Newcombe, Estimating the difference between differences: measurement of additive scale interaction for proportions. Stat Med. 2001; 20: 2885–2893. Reproduced with permission.

ABSOLUTE RISK REDUCTION

and demonstrated good agreement of mean coverage with 1 – α. However, overshoot can occasionally occur. The above methods are appropriate for a paired difference of proportions. But for crossover and simultaneous split-unit studies, a slightly different approach is preferable. Thus, in a crossover study, if the numbers of subjects who get the two treatment sequences AB and BA are not identical, the simple difference of marginals contains a contribution from period differences. A more appropriate analysis is based on the analysis of differences of paired differences described by Newcombe (25). The example in Table 4 relates to a crossover trial of home versus hospital physiotherapy for chronic multiple sclerosis (37). Twenty-one patients were randomized to receive home physiotherapy followed by hospital physiotherapy, and 19 to receive these treatments in the reverse order. Following Hills and Armitage (38) and Koch (39), the treatment effect is estimated as half the difference between the within-subjects period differences in the two treatment order groups. The resulting estimate, + 0.1454 and 95% confidence interval, –0.0486 to + 0.3238, are very similar but not identical to those obtained by direct application of the modified square-and-add approach (33), + 0.1500 and –0.0488 to + 0.3339. REFERENCES 1. K. Rothman, Modern Epidemiology. Boston: Little, Brown, 1986. 2. L. Hashemi, B. Nandram, and R. Goldberg, Bayesian analysis for a single 2 × 2 table. Stat Med. 1997; 16: 1311–1328. 3. E. Schechtman, Odds ratio, relative risk, absolute risk reduction, and the number needed to treat—which of these should we use? Value Health. 2002; 5: 431–436. 4. A. Agresti, Categorical Data Analysis, 2nd ed. Hoboken, NJ: Wiley, 2002. 5. R. G. Newcombe, A deficiency of the odds ratio as a measure of effect size. Stat Med. 2006; 25: 4235–4240. 6. S. E. Vollset, Confidence intervals for a binomial proportion. Stat Med. 1993; 12: 809–824. 7. R. G. Newcombe, Two-sided confidence intervals for the single proportion: comparison of seven methods. Stat Med. 1998; 17: 857–872.

9

8. A. Agresti and B. A. Coull, Approximate is better than ‘‘exact’’ for interval estimation of binomial proportions. Am Stat. 1998; 52: 119–126. 9. L. D. Brown, T. T. Cai, and A. DasGupta, Interval estimation for a proportion. Stat Sci. 2001; 16: 101–133. 10. E. B. Wilson, Probable inference, the law of succession, and statistical inference. J Am Stat Assoc. 1927; 22: 209–212. 11. A. Agresti and Y. Min, Frequentist performance of Bayesian confidence intervals for comparing proportions in 2 × 2 contingency tables. Biometrics. 2005; 61: 515–523. 12. B. P. Carlin and T. A. Louis, Bayes and Empirical Bayes Methods for Data Analysis. London: Chapman & Hall, 1996. 13. C. J. Clopper and E. S. Pearson, The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika. 1934; 26: 404–413. 14. H. O. Lancaster, The combination of probabilities arising from data in discrete distributions. Biometrika. 1949; 36: 370–382. 15. G. Berry and P. Armitage, Mid-P confidence intervals: a brief review. Statistician. 1995; 44: 417–423. 16. H. Blaker, Confidence curves and improved exact confidence intervals for discrete distributions. Can J Stat. 2000; 28: 783–798. 17. J. Reiczigel, Confidence intervals for the binomial parameter: some new considerations. Stat Med. 2003: 22; 611–621. 18. M. Parzen, S. Lipsitz, J. Ibrahim, and N. Klar, An estimate of the odds ratio that always exists. J Comput Graph Stat. 2002: 11; 420–436. 19. R. W. Mee, Confidence bounds for the difference between two probabilities. Biometrics. 1984; 40: 1175–1176. 20. O. S. Miettinen and M. Nurminen, Comparative analysis of two rates. Stat Med. 1985; 4: 213–226. 21. R. G. Newcombe, Interval estimation for the difference between independent proportions: comparison of eleven methods. Stat Med. 1998; 17: 873–890. 22. R. G. Newcombe, Bayesian estimation of false negative rate in a clinical trial of sentinel node biopsy. Stat Med. 2007; 26: 3429–3442. 23. D. E. Walters, On the reliability of Bayesian confidence limits for a difference of two proportions. Biom. J. 1986; 28: 337–346. 24. T. Fagan, Exact 95% confidence intervals for differences in binomial proportions. Comput

10

ABSOLUTE RISK REDUCTION

Biol Med. 1999; 29: 83–87. 25. R. G. Newcombe, Estimating the difference between differences: measurement of additive scale interaction for proportions. Stat Med. 2001; 20: 2885–2893. 26. W. W. Hauck and S. Anderson, A comparison of large-sample confidence interval methods for the difference of two binomial probabilities. Am Stat. 1986; 40: 318–322. 27. A. Agresti and B. Caffo, Simple and effective confidence intervals for proportions and differences of proportions result from adding 2 successes and 2 failures. Am Stat. 2000; 54: 280–288. 28. A. Laupacis, D. L. Sackett, and R. S. Roberts, An assessment of clinically useful measures of the consequences of treatment. N Engl J Med. 1988; 318: 1728–1733. 29. D. G. Altman, Confidence intervals for the number needed to treat. BMJ. 1998; 317: 1309–1312. 30. R. Bender, Calculating confidence intervals for the number needed to treat. Control Clin Trials. 2001; 22: 102–110. 31. R. G. Newcombe, Confidence intervals for the number needed to treat—absolute risk reduction is less likely to be misunderstood. BMJ. 1999; 318: 1765. 32. Q. McNemar, Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika. 1947; 12: 153–157. 33. R. G. Newcombe, Improved confidence intervals for the difference between binomial proportions based on paired data. Stat Med. 1998; 17: 2635–2650. 34. T. Tango, Equivalence test and CI for the difference in proportions for the paired-sample design. Stat Med. 1998; 17: 891–908. 35. R. G. Newcombe, Confidence intervals for the mean of a variable taking the values 0, 1 and 2. Stat Med. 2003; 22: 2737–2750. 36. M. L. Tang, N. S. Tang, and I. S. F. Chan, Confidence interval construction for proportion difference in small sample paired studies. Stat Med. 2005; 24: 3565–3579. 37. C. M. Wiles, R. G. Newcombe, K. J. Fuller, S. Shaw, J. Furnival-Doran, et al., Controlled

randomised crossover trial of physiotherapy on mobility in chronic multiple sclerosis. J Neurol Neurosurg Psych. 2001; 70: 174–179. 38. M. Hills and P. Armitage, The two-period cross-over clinical trial. Br J Clin Pharmacol. 1979; 8: 7–20. 39. G. G. Koch, The use of non-parametric methods in the statistical analysis of the twoperiod change-over design. Biometrics. 1972; 28: 577–584.

FURTHER READING Microsoft Excel spreadsheets that implement chosen methods for the single proportion, unpaired and paired difference, and interaction cases can be downloaded from the author’s website: http://www.cardiff.ac.uk/medicine/ epidemiology statistics/research/statistics/ newcombe The availability of procedures to calculate confidence intervals for differences of proportions is quite patchy in commercial software. StatXact (Cytel Statistical Software) includes confidence intervals for differences and ratios of proportions and odds ratios. These are ‘‘exact’’ intervals, designed to guarantee minimum coverage 1 – α. The resulting intervals are likely to be relatively wide compared with methods that seek to align the mean coverage approximately with 1 – α.

CROSS-REFERENCES Estimation Confidence interval Categorical variables Comparing proportions Sample size for comparing proportions (superiority and noninferiority) Relative risk Odds ratio Bayesian approach Equivalence analysis Noninferiority analysis

ACCELERATED APPROVAL

In 1987, four major initiatives were drafted to speed drug delivery, cut cost, and make drugs available sooner. Two of these initiatives, ‘‘Treatment IND’’ and ‘‘Parallel Track,’’ focused on expanding access of potentially life-saving drugs to specific patient populations prior to approval of the drug. In contrast, the ‘‘Subpart E’’ and ‘‘Accelerated Approval’’ programs focused on decreasing the amount of clinical and regulatory review time needed to approve a life-saving drug. Subpart E provides a regulatory framework to grant approval to certain drugs after an extended Phase II trial. Accelerated approval allows for a drug to be approved with restrictions on distribution or with the use of unvalidated surrogate endpoints or measures of indirect clinical benefits to determine efficacy (1). The terminology and regulatory implications for various FDA programs can be confusing and are often mislabeled in literature. In understanding the terms, it is worth mentioning that ‘‘Subpart E,’’ which is also known as ‘‘Expedited Development,’’ refers to the sub-part of title 21 in the Code of Federal Regulations (CFR). Likewise, ‘‘Accelerated Approval’’ is sometimes referred to as ‘‘Subpart H,’’ as that is the subpart of 21 CFR that created the regulations allowing for this process by the FDA. Since the 1990s, more initiatives such as Fast Track designation, Priority Review, and Rolling Review have been implemented to facilitate the path of life-saving drugs to market, through separate initiatives.

LOUIS A. CABANILLA M.S. CHRISTOPHER P. MILNE D.V.M, M.P.H, J.D Tufts University Center for the Study of Drug Development, Boston, Massachusetts

The development and approval of a new drug is a complex and time-consuming procedure, which takes many years to accomplish. By the time a drug receives FDA approval, extensive lab and clinical work must be performed to ensure that the drug is both safe and effective, and months or years spent by the FDA reviewing the drug’s application. Although this level of quality control is a great benefit to people who use the drug, the extensive amount of time spent on development and regulatory submission review represents a barrier for a person with a serious or life threatening disease for which no treatment exists, or for a person for whom available treatments have failed. In such cases, a speedy development and regulatory review are of the utmost importance. A patient with no or few therapeutic options is often willing to accept a higher risk to benefit ratio including the use of a treatment whose efficacy is predicated on indirect measures of the expected clinical benefit. 1 ACCELERATED DEVELOPMENT VERSUS EXPANDED ACCESS IN THE U.S.A. In the United States, an emphasis has been placed on making potentially life saving drugs available as soon as possible. In some instances, this process involves making experimental drugs available to patients who are not enrolled in clinical trials, but more typically it involves programs designed to decrease the time to market for these important drugs. This movement took shape in the middle to late 1980s when AIDS drugs were being developed but were not made available to the wider patient population quickly enough, which caused outrage among those who risked dying while waiting for the FDA to approve these drugs for marketing.

2 SORTING THE TERMINOLOGY—WHICH FDA INITIATIVES DO WHAT? • Treatment IND. ‘‘TIND’’ provides for

early access of promising drugs to patients with serious or life-threatening illnesses who have no treatment options, or for patients who have failed available therapies. Drug companies are allowed to recoup the cost of producing the drug for these patients who are not involved in the clinical trials.

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

ACCELERATED APPROVAL • Parallel Track. This program is a more

• Priority Review. This review allows the

extensive version of the TIND program aimed at people who do not qualify for participation in clinical trials. The program was a response to the AIDS epidemic, and it is generally intended for HIV/AIDS treatments or related problems. Drug companies collect basic safety data from these patients and are allowed to recoup the cost of producing the drug.

FDA to allocate more resources on the review of a ‘‘priority’’ drug (i.e., one that represents a significant advance over those currently on the market). • Rolling Review.. Under rolling review, companies can submit sections or ‘‘Reviewable Units (RUs)’’ of a NDA or BLA as they complete them, to be reviewed by the FDA. Although an RU may be complete, marketing rights are not granted until all RUs are submitted and approved (1,2).

• Expedited Development (Subpart E).

This provision allows for extensive interaction and negotiation with the FDA to move a drug to the market as quickly as possible. This process includes the possibility of expanding a Phase II trial and using that data for a new drug application (NDA)/biologic licensing application (BLA) submission, and then focusing on postmarketing safety surveillance.

3 ACCELERATED APPROVAL REGULATIONS: 21 C.F.R. 314.500, 314.520, 601.40 The FDA has several programs to expedite the development and approval of lifesaving drugs. Because some similarities are observed among the programs that pertain to the eligibility requirements and regulatory language, they are sometimes confused with each other. Although some overlap does occur, they generally affect different segments of the development and application review timeline (see Fig. 1).

• Accelerated Approval (Subpart H). This

provision allows drugs to gain approval with distribution restrictions, or more commonly, based on unvalidated surrogate endpoints or an indirect clinical benefit. It allows lifesaving drugs to become available while the company completes a long-term Phase IV study. • Fast Track. This program originated as

4 STAGES OF DRUG DEVELOPMENT AND FDA INITIATIVES

a provision of the second Prescription Drug User Fee Act (PDUFA II), and it allows the FDA to facilitate the development of and to expedite the review for drugs intended to treat serious or lifethreatening illness that have the potential to address unmet medical needs.

IND Pre-IND

End of Phase I

End of Phase II

End of Phase III

The intended effect of FDA expedited development programs is to speed development and approval; however, the respective programs focus on disparate aspects of the development and regulatory process. For example,

NDA Submission

NDA Approval Marketing

Priority Rating Rolling Review

Accelerated Approval Sub-Part E Fast Track

Figure 1. Stages of drug development and FDA initiatives. The progressive stages of drug development, and where the initiatives begin/end.

ACCELERATED APPROVAL

3

Accelerated Approval (1992–2005) and... 50% 45% 40% Percentage

35% 30% 25% 20% 15% 10% Figure 2. Accelerated Approval and . . . ’’ The overlap between accelerated approval and various other initiatives.

5% 0%

accelerated approval is often confused with fast track designation and priority review; however, they are quite different. The central mechanism of the accelerated approval program is ‘‘conditional’’ approval based on the use of unvalidated surrogate endpoints or indirect clinical endpoints as evidence of efficacy, or restricted distribution. Fast track designation provides the opportunity for intensive scientific interaction with the FDA and acts as a threshold criterion for rolling review. Priority review is an administrative prioritization scheme implemented by the FDA to give precedence to applications for drugs that represent an improvement to the currently marketed products for a particular disease or condition (also applies to eligible diagnostics and preventatives). Moreover, it should be recognized that the same drug could be a part of accelerated approval, fast track, and other programs at the same time (see Fig. 2). Many accelerated approvals have also benefited from other programs aimed at decreasing the time and resources required to bring crucial drugs to market. In addition, many accelerated approvals are also orphan drugs. Orphan drugs are treatments for rare diseases and conditions, and the FDA designation provides certain economic and regulatory incentives for a drug company to develop them.

Fast Track

Subpart E

Rolling NDA Orphan Drug

5 ACCELERATED APPROVAL REGULATIONS: 21 CFR 314.500, 314.520, 601.40 Accelerated approval regulations were promulgated December 11, 1992. The law stipulates the following (3): FDA may grant marketing approval for a new drug product on the basis of adequate and well-controlled clinical trials establishing that the drug product has an effect on a surrogate endpoint that is reasonably likely, based on epidemiologic, therapeutic, pathophysiologic, or other evidence, to predict clinical benefit or on the basis of an effect on a clinical endpoint other than survival or irreversible morbidity.

Conditions for approval: • Possible FDA restrictions or on distri-

bution and use by facility physician, a mandated qualifying test, or procedural administration requirement • Promotional materials must be submit-

ted to, and approved by the FDA • Streamlined withdrawal mechanisms if • Anticipated benefit is not confirmed

in Phase IV trials • Sponsor fails to exercise due dili-

gence in performing Phase IV trials • Restrictions on use prove insuffi-

cient to ensure safe usage • Violations on restrictions on use

and distribution

4

ACCELERATED APPROVAL • Promotional materials are false or

misleading • Other evidence that the drug is not safe or effective (3) An FDA advisory committee can recommend a drug for accelerated approval based on the set criteria for qualification, which includes the soundness of evidence for the surrogate markers. Following an advisory committee recommendation, the FDA can then review the application and approve the drug for marketing. Drugs must be intended for patients with serious or life-threatening illnesses. Moreover, the data used for the approval must show effectiveness of an unvalidated surrogate endpoint that is ‘‘reasonably likely to predict a clinical effect.’’ In contrast, a validated surrogate endpoint would proceed though the normal approval process. If a company seeks accelerated approval based on restricted distribution, then it must have clear distribution restriction practices and provider/user education programs for the drugs to gain approval. An NDA submission with unvalidated surrogate endpoints must still stand up to the scrutiny of the NDA review process, and it can be rejected for any of the reasons a traditional NDA could be rejected such as safety, efficacy, or concern about product quality. Beyond the issues of approval, all promotional materials for an accelerated approval drug must be submitted to the FDA for approval, and they must be periodically reviewed. This review is another method of ensuring the appropriate understanding and availability of these drugs for both doctors and patients. The initial approval process for a drug by the FDA pertains to a particular or limited set of indications for a particular Sub-population of patients. However, it is common that a drug can later be found to have a benefit for multiple populations and multiple indications. For a drug that is already approved for use, the expansion for a subsequent indication requires a less comprehensive supplemental NDA (sNDA) or supplemental BLA (sBLA). A supplement can be eligible for accelerated approval status, whether the first indication was a traditional approval or not. Over the past 5 years more sNDAs have been granted accelerated approval. In the 2000s,

23 sNDA accelerated approvals were granted by the FDA, whereas in the 1990s (1993–1999) only five sNDAs were approved (4). Most accelerated approvals have been granted to sponsors of small molecule drugs (i.e., chemicals), but a significant minority, which has increased over time, have been granted to large molecule drugs (i.e., biologics). This latter group is generally composed of monoclonal antibodies, although many more designations have been granted for other biologics such as vaccines or recombinant proteins. Nearly 31% of all accelerated approvals have been for biologics, most for oncology, although several accelerated approvals have been granted for rare diseases such as multiple sclerosis and Fabry’s disease (5). Increasingly, accelerated approvals have been given to speed the development and availability of preventative vaccines, such as the influenza vaccine for the 2006–2007 flu season (6) (Fig. 3). 6 ACCELERATED APPROVAL WITH SURROGATE ENDPOINTS The major advantage of the accelerated approval designation is that it can decrease the complexity of late stage clinical trials using an unvalidated surrogate endpoint, which may decrease the number of patients who must be enrolled, decrease the amount of data being collected, and most of all decrease the time required to conduct the necessary studies (7). For example, in 1992 ddI (also called Videx or didanosine) was approved based on CD4 levels rather than the survival rate for HIV infected patients, which greatly reduced the Phase III trial time. However, this endpoint has become validated; as a result, new drugs that use CD4 levels as a clinical measurement now proceed through the normal NDA process. In terms of speed to market, drugs that are designed to increase the survival rate or slow disease progression can take years to properly test, which is generally the case for illnesses such as cancer, HIV, or multiple sclerosis. In such cases, accelerated approval is beneficial in making a drug available to patients while long-term data are collected.

ACCELERATED APPROVAL

The ability to use easily observable and quantifiable data as evidence of clinical effectiveness allows a drug to reach the market much more quickly than using traditional measurement such as overall survival rate (7). Surrogate endpoints are a subset of biological markers, which are believed to have a correlation for endpoints in a clinical trial (8). Although it is not guaranteed that an actual relationship exists, surrogate endpoints are ‘‘based on scientific evidence for clinical benefit based on epidemiologic, therapeutic, pathophysiologic or other scientific evidence’’ (8). A clinical trial testing the efficacy of a statin, for example, might use a reduction in cholesterol as a surrogate for a decrease in heart disease. Elevated cholesterol levels are linked to heart disease, so this correlation is likely. The benefit of using cholesterol levels is that heart disease often takes decades to develop and would require a time-consuming and expensive trial to test. With this in mind, the FDA might approve the statin for marketing contingent on the sponsor company’s agreement to complete postmarketing commitment(s) to test for a clinical effect (Phase III b or Phase IV studies). Phase IIIb and Phase IV studies are distinct from each other in that the former begins after the application submission, but before approval, and it typically continues into the postmarketing period; a Phase IV study is a postmarketing study.

6.1 What Is a Surrogate Endpoint? A surrogate endpoint is a laboratory or physical sign that is used in therapeutic trials as a substitute for a clinically meaningful endpoint (or biomarker) that is a direct measure of how a patient feels, functions, or survives and that is expected to predict the effect of the therapy [8]. 6.2 What Is a Biomarker? A biomarker is a characteristic that is measured and evaluated objectively as an indicator of normal biologic or pathogenic processes or pharmacological responses to a therapeutic intervention. Surrogate endpoints are a subset of biomarkers [8]. 7 ACCELERATED APPROVAL WITH RESTRICTED DISTRIBUTION Although accelerated approval drugs are generally approved on the basis of surrogate endpoints, however, section 21 C.F.R. Section 314.520 allows for approval of a drug based on restrictions on distribution. These restrictions can refer to certain facilities or physicians who are allowed to handle and prescribe the drug, or to certain requirements and tests or medical procedures, which must be performed prior to use of the drug. The drug thalidomide has a restricted accelerated approval for the treatment of Erythema

Product Type of Accelerated Approvals: 1992–2005 80% 70%

Percentage

60% 50% 40% 30% 20% 10% 0%

5

Small Molecule

Biologics Type of Drug

Figure 3. Product type of accelerated approvals. Comparison between small molecule and biologic drugs approved by FDA through the Subpart H.

6

ACCELERATED APPROVAL

Nodosum Leprosum because the drug is associated with a high risk of birth defects for infants whose mothers are exposed to the drug. To mollify this risk, doctors must complete an educational program on the risks and safe usage of thalidomide before they are certified to prescribe the drug. Few accelerated approvals are based on this restriction, although they have become more common over time. The number of restricted distribution approvals averaged one per year between 2000 and 2005. However, in the first half of 2006 there were four restricted approvals, all of which were supplemental approvals (Fig. 4) (4). 8 PHASE IV STUDIES/POSTMARKETING SURVEILLANCE Although a drug that is given accelerated approval might reach the market, the company that sponsors the drug is required to complete the necessary research to determine efficacy with a Phase IV study. These studies are intended to increase the quality of the application dossier to the scientific standards of drugs approved through the traditional NDA/BLA process. Sec. 314.510 (9) stipulates: Approval under this section will be subject to the requirement that the applicant study the drug further, to verify and describe its clinical benefit, where there is uncertainty as to

the relation of the surrogate endpoint to clinical benefit, or of the observed clinical benefit to ultimate outcome. Postmarketing studies would usually be studies already underway. When required to be conducted, such studies must also be adequate and well controlled. The applicant shall carry out any such studies with due diligence.

Some ambiguity exists in this section of the law as no actual time frame was given, until recent years, for companies to complete their Phase IV trials and submit their findings to the FDA. Rather, it was expected that companies would use ‘‘due diligence’’ in completing their research. This nebulous definition has lead to some contention about the level of compliance by the drug industry. This question will be considered in more depth in a subsequent section. 9 BENEFIT ANALYSIS FOR ACCELERATED APPROVALS VERSUS OTHER ILLNESSES Drugs that qualify for accelerated approval are typically intended to treat serious conditions and diseases such as pulnanary Arterial hypertension, HIV, malignancies, Fabry’s disease, and Crohn’s disease. Given that the average median total development time for all FDA-approved drugs during the 1990s and 2000s was just over 7 years (87 months), drugs under the accelerated approval program have spent considerably less time in

Approval Type: 1992–2005 Number of Approvals

90 80 70 60 50

sNDAs/sBLAs NDA/BLA

40 30 20 10 0

Surrogate

Restricted Basis

Figure 4. Accelerated Approval information through 06/06. Approval type: A comparison between Subpart H approvals with restricted distribution versus surrogate basis.

ACCELERATED APPROVAL

the journey from bench to bedside in such critical therapeutic areas as HIV/AIDS and cancer (Fig. 5). To qualify for accelerated approval, a drug must offer a meaningful therapeutic benefit over available therapy, such as: • Greater efficacy • A more favorable safety profile • Improved patient response over existing

therapies (9) Given the severity of the illnesses and the potential for a ‘‘meaningful therapeutic benefit,’’ the FDA is generally willing to accept a higher risk-to-benefit ratio for a potential therapy. A drug with a comparatively high risk of side effects or with less certainty of proof of efficacy can be approved. For example, a drug with a traditional approval for the treatment of insomnia would have an acceptable overall safety and efficacy profile. Conversely, an accelerated approval drug for multiple sclerosis could gain approval with a likely clinical effect, based on surrogate endpoints. This change in approval occurred because patient advocacy groups and public opinion have made it clear that patients in such a situation are willing to bear additional risks. For patients with no other options, a new therapy often represents the only possibility for increasing the length or quality of their life. Likewise, a drug designated as a firstline treatment for a malignancy would have

7

a more rigorous approval process, given the currently available treatments, than a drug designated as a third-line treatment for patients who have failed all conventional treatment options. Within any patient population, a certain percentage will fail to respond to first- or second-line treatments or they may become resistant to therapy. This finding is particularly true with malignancies that have high relapse rates and infections such as HIV, in which the virus gains immunity to treatments over time. In situations such as these, creating the incentive for drug companies to develop second-, third-, and even fourth-line treatments with new mechanisms of action is critical.

10 PROBLEMS, SOLUTIONS, AND ECONOMIC INCENTIVES Accelerated approval is intended to increase the speed to market for important drugs. Although this timeline is a benefit to patients, it is also a benefit to the company that sponsors the drug. The use of surrogate endpoints can greatly decrease the length and complexity of a clinical trial. This use allows a drug company to begin recouping sunk costs earlier and provides returns on sales for a longer period. Without accelerated approval, it is possible that many of these drugs would have been viewed as too risky or expensive to develop in terms of return on investment for

Median Development and Approval Times for Accelerated Approval Drugs: 1992–2005 140

Time (months)

120 100 80 60 40 20

Figure 5. Median development and approval times for accelerated approval drugs. Development and regulatory times by indication for Subpart H.

0 HIV/AIDS

Cancer

Cardiovascular Related Diseases

Other

Median Approval Time Median Development Time

8

ACCELERATED APPROVAL Table 1. 2004 Sales figures for drugs first approved under accelerated approval: sales figures for various accelerated approval drugs in billions of dollars (US) Viracept Biaxin Cipro Eloxatin Taxotere Levaquin Remicade Celebrex Erbitux Temodar Thalomid Casodex Norvir Arimidex Epivir Gleevec Betaseron Kaletra Camptosar Viread

nelfinavir mesylate clarithromycin ciprofloxacin hcl oxaliplatin docetaxel levofloxacin infliximab celecoxib cetuximab temozolomide thalidomide bicalutamide ritonavir anastrozole lamivudine imatinib mesylate interferon beta-1b lopinavir/ritonavir irinotecan hcl tenofovir disoproxil fumarate

drug companies. Even after approval, sponsor companies have some degree of uncertainty. It is possible that some practitioners, patients, and third-party payers may consider such approvals to be conditional and avoid their use until no alternative is available. Moreover, insurance companies may refuse to reimburse experimental drugs, thus the financial burden for the treatment may also place the drugs out of reach of patients. However, these drugs can be very profitable. In 2004, 20 drugs had accelerated approval among the top 200 best-selling drugs, with combined sales of $12.54 billion (Table 1) (10). A prerequisite for accelerated approval designation is that the drug company will complete a Phase IV trial with ‘‘due diligence’’ following the marketing of the drug. Once approved, evidence from a Phase IV trial that does not support the efficacy of the drug may lead to the drug being withdrawn. The original text has proven problematic in that ‘‘due diligence’’ is ambiguous, and it is understood differently by the various stakeholders. Moreover, despite the wording of the provisions for drug withdrawal, it is difficult to do so once market rights have

$0.23 $0.27 $0.27 $0.78 $0.94 $1.70 $1.80 $2.70 $0.24 $0.24 $0.27 $0.28 $0.30 $0.30 $0.31 $0.34 $0.38 $0.45 $0.48 $0.53

been granted. A congressional staff inquiry found that as of March 2005, 50% of outstanding accelerated approval postmarketing studies had not begun, whereas most of the other 50% of studies were underway. Overall, a significant proportion (26%) of trials is taking longer than expected to complete (11). The incentive for a company to perform a Phase IV trial is minimal given that the sponsor company has already received marketing approval, and the completion of the trial is expensive and time consuming. As of 2006, no drug had been withdrawn for failure to complete a Phase IV trial. However, proposed regulations and guidance within the FDA could change this (12). The biggest change to the original procedure for accelerated approval came in 2003 when the FDA reinterpreted the laws regarding the number of accelerated approvals granted per oncology indication. Whereas the old practice allowed only one accelerated approval per indication, the new interpretation viewed a medical need as ‘‘unmet’’ until the sponsoring company completes a Phase IV trial (13). This change provides an incentive for drug companies to complete Phase IV studies quickly to avoid market competition

ACCELERATED APPROVAL

from other drugs. More recently, the FDA has released a guidance that suggests accelerated approvals could be based on interim analysis, which will continue until a final Phase IV report is completed, rather than on separate Phase IV trials (14). This guidance would potentially decrease the cost of a Phase IV study as well as alleviate the issue of patient enrollment, which is often cited as a cause for delays by drug companies. The idea of granting ‘‘conditional’’ approval contingent on Phase IV completion for ‘‘full’’ approval has also been mentioned (15). Increasing Phase IV follow-through is important to the proper functioning of the accelerated approval program. Although the failure to complete a Phase IV trial might hurt the patient population by leaving some uncertainty as to a drug’s safety or efficacy, it also hurts the drug companies in the long run. In 2005, an advisory committee refused to grant the drug Zarnestra, (Johnson & Johnson, New Brunswick, NJ) accelerated approval and cited the FDA’s failure to enforce Phase IV obligations as a reason (16). This decision was an example of the impact that the lack of confidence in FDA enforcement, and sponsor compliance could have on the future performance of the program. A final issue, which the FDA must address, is that of public perception and understanding of exactly what accelerated approval is and is not. There is often the misconception about the conditions under which accelerated approval is granted, that it is better than a full approval, or that it is the same as a fast track approval. In the interest of transparency, that the public must understand that an accelerated approval is less certain than a traditional approval and that less than a full quantum of evidence was used as the basis for efficacy. However, an accelerated approval should not be viewed as inherently more risky, when in fact it has been demonstrated to be more beneficial for certain patients to take the drug than not based on consideration of all the risks and benefits currently known. 11

FUTURE DIRECTIONS

Will there be changes to the accelerated approval program? On one hand, some

9

patient groups believe that accelerated approval is not delivering on its promise for bringing more therapeutic options to patients with diseases for which few or no adequate treatments are available. They question the arduousness of the process, the increasing trepidation of the FDA regarding potential ill effects, as well as political influences being brought to bear on a program that was borne out of an initiative inspired by patients, medical practitioners, and FDA in response to a public health emergency. On the other hand, there is increasing discomfort among some consumer watchdog groups as well as patient groups concerning the diligence with which the confirmatory post-marketing studies are being pursued by biopharmaceutical firms. Addressing this issue was a key focus of the public debates surrounding the reauthorization of the PDUFA in 2007. Will the concept of accelerated approval go global? Research on biomarkers is becoming a global endeavor and the need to get lifesaving drugs to patients more expeditiously is a worldwide concern. It would seem likely that the regulatory agencies in countries that are major centers of biopharmaceutical research and development such as Japan and the European Union would follow the United State’s lead as they did in setting up their orphan drug programs 10 and 20 years after the United States, respectively (with piggy-back programs emerging in other industrialized countries such as Australia and Singapore). However, the EU already has a program that is similar to accelerated approval, which is termed marketing authorization ‘‘under exceptional circumstances’’ for products that do not have, and are likely to never have full safety and efficacy data (17). The EU also recently enacted ‘‘conditional approval’’ that allows for approval based on a less complete approval package than is normally required (18). It is possible that some symbiosis may occur between the EU and U.S. program. The FDA may become more comfortable with the concept of a conditional approval (i.e., annual status review and restrictive marketing until confirmatory studies are complete), whereas the EU may become more expansive in employing conditional approval based on the increasing availability of surrogate biomarkers as

10

ACCELERATED APPROVAL

adequate indicators of safety and efficacy in ‘‘exceptional circumstances.’’ The Japanese regulatory body has also adopted a conditional approval process, which includes provisions for the use of restricted access, and surrogate endpoints. However, they do not have a program that is specific to accelerated approvals. This program, which encompasses many initiatives, has been an effective means of decreasing development and regulatory time (19). Overall, the success of the accelerated approval program to date in bringing some 100 drug products to market months or years faster than would have happened through the traditional process, together with advances in the science and technology of the clinical evaluation of biopharmaceuticals and the more powerful voice of patient advocacy in determining regulatory agency policy, ought to ensure that accelerated approval will expand its reach both therapeutically and geographically.

REFERENCES 1. S. S. Shulman and J. S. Brown. The Food and Drug Administration’s early access and fast-track approval initiatives: how have they worked. Food Drug Law J. 1995; 50: 503–531. 2. Guidance for Industry: Fast Track Drug Development Programs – Designation, Development, and Application Review. January 2006, Procedural, Revision 2. U.S. Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research (CDER) & Center for Biologics Evaluation and Research (CBER). 3. New Drug, Antibiotic, and Biological Drug Product Regulations; Accelerated Approval. Final Rule. 57 Federal Register 58942 (December 11, 1992). Department of Health and Human Services, Food and Drug Administration. 4. Accelerated Approvals under 21 CFR 314 Subpart H (drugs) & 21 CFR 601 Subpart E (biologics). U.S. Food and Drug Administration. CDER website. Available at http://www.fda. gov/cder/rdmt/accappr.htm. Last accessed 09/15/ 2006. 5. Unpublished Data, Marketed Database, September 2006. Tufts Center for the Study of Drug Development. Boston, MA: Tufts University.

6. FDA News: FDA Approves Additional Vaccine for Upcoming Influenza Season. October 5, 2006. U.S. Food and Drug Administration. [Press Release] Available at: http://www.fda. gov/hhs/topics/NEWS/2006/NEW01478.html. 7. R. L. Schilsky, Hurry up and wait: is accelerated approval of new cancer drugs in the best interests of cancer patients? J. Clin. Oncol. 2003; 21: 3718–3720. 8. A. G. Chakravarty, Surrogate markers – their role in the regulatory decision process. FDA Workshop, American Statistical Association, Toronto, Canada, 2004. 9. H. Subpart, Accelerated Approval of New Drugs for Serious or Life-Threatening Illnesses. 21 CFR 314.50 (Revised April 1, 2004). 10. Top 200 Drugs for 2004 by U.S. Sales. Available at: http://www.drugs.com/top200 2004. html. 11. Conspiracy of Silence: How the FDA Allows Drug Companies to Abuse the Accelerated Approval Process. Responses by the Food and Drug Administration and the Securities and Exchange Commission to Correspondence from Rep. Edward J. Markey (D-MA). Congressional staff summary, June 1, 2005. 12. A. Kaspar, Accelerated approval should be ‘‘conditional’’ panel says. Pink Sheet Daily, June 17, 2005. 13. FDA revising accelerated approval standard in Oncology; biomarkers sought. The Pink Sheet June 9, 2003;65: 23. 14. Susman E. Accelerated approval seen as triumph and roadblock for cancer drugs. J. Natl. Cancer Inst. 2004; 96: 1495–1496. 15. Fox JL. Conflicting signals on US accelerated approvals. Nature Biotechnology, published online, 31 August 2005. Available at: http://www. nature.com/news/2005/050829/full/nbt09051027.html. 16. Iressa casts shadow over Zarnestra advisory committee review. The Pink Sheet May 16, 2005;67: 18. 17. Guideline on procedures for the granting of a marketing authorization under exceptional circumstances, pursuant to article 14 (8) of regulation (EC) No 726/2004. London, 15 December 2005. Doc. Ref. EMEA/357981/2005. European Medicines Agency (EMEA). 18. Guideline on the scientific application and the practical arrangements necessary to implement commission regulation (EC) No 507/2006 on the conditional marketing authorization for medicinal products for human use

ACCELERATED APPROVAL falling within the scope of regulation (EC) No 726/2004. London, 5 December 2006. Doc. Ref. EMEA/509951/2006. European Medicines Agency (EMEA). 19. Japan regulators increase use of ‘‘conditional approvals’’ on drugs. Pacific Bridge Medical: Asian Medical Newsletter 2006; 6: 1–2.

FURTHER READING Federal Register Final Rules, Title 21, Chapter 1, Subchapter D- Drugs for Human Use, Subpart H- Accelerated Approval of New Drugs for Serious or Life Threatening Illnesses. §314 and 600. Guidance for industry: Providing Clinical Evidence of Effectiveness for Human Drug and Biological Products, U.S Department of Health and Human Services Food and Drug Administration, May 6, 1998. Guidance for Industry: Fast Track Drug Development Programs- Designation, Development, and Application Review, U.S Department of Health and Human Services Food and Drug Administration, January 2006.

11

ADVERSE DRUG REACTION REPORTING The sponsor should expedite the reporting to all concerned investigator(s)/institutions(s), to the Institutional Review Board(s) (IRB)/Independent Ethics Committee(s) (IEC), where required, and to the regulatory authority(ies) of all adverse drug reactions (ADRs) that are both serious and unexpected. Such expedited reports should comply with the applicable regulatory requirement(s) and with the ICH Guideline for Clinical Safety Data Management: Definitions and Standards for Expedited Reporting. The sponsor should submit to the regulatory authority(ies) all safety updates and periodic reports, as required by applicable regulatory requirement(s).

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

ADVERSE EVENT EVALUATION

at the time of the data collection, it may be extremely difficult to assess the causality of a single, specific event, and it is only through the review of aggregated data, or after a series of apparently unrelated events have occurred, that a relationship becomes apparent. For example, falls are not uncommon in an elderly population, and it may seem unremarkable that an older study participant has a fall resulting in minor injuries during participation in a study, and that this does not really constitute an adverse event. However, if several subjects in the study seem to be having falls, at a rate higher than would be expected, additional investigation may demonstrate that the investigational product causes dizziness and that the risk of falling is a true safety issue. Although it is often easiest to think of safety assessment in terms of adverse events occurring during a study in which the intervention is a pharmaceutical product, adverse events may also occur in studies in which there is a different type of intervention. For example, emotional distress or nightmares may result from a study in which a new technique for counseling is used in the treatment of survivors of childhood trauma.

LINDSAY A. MCNAIR, M.D., Vertex Pharmaceuticals, Cambridge, MA Boston University School of Public Health Department of Epidemiology Boston, Massachusetts

1

INTRODUCTION

Although much of the focus in later stage clinical trials is often on study design and the evaluation of the efficacy of an intervention or pharmaceutical product, the appropriate collection and assessment of safety data is a critical part of any clinical study. In early stage trials, the primary objective of the study may be the assessment of safety (or the identification of a maximum tolerated dose). For the assessment of safety data, several key factors must be collected for any adverse event. This article will provide information for the identification of adverse events and the data to be collected, to provide the necessary tools for safety assessment in clinical studies. 2

IDENTIFYING AN ADVERSE EVENT 3 INFORMATION COLLECTED FOR ADVERSE EVENTS

An adverse event is defined as any untoward medical occurrence in a patient or clinical investigation subject administered a pharmaceutical product and that does necessarily not have to have a causal relationship with the treatment. It may include worsening of a preexisting disease or condition that occurs during the study period that is outside the normally expected variations (1). Adverse events may also include the abnormal or clinically significant results of laboratory tests or medical procedures (see Adverse Event Definitions). It is important to note that the definition of an adverse event does NOT state or imply any relationship to the intervention or investigational product. Therefore, any medical event regardless of whether the investigator thinks it may have been related to the study intervention must be collected as an adverse event. The rationale for this is that

The first piece of information collected is the adverse event name. The exact term used by the investigator to name the event (e.g., hypertension or diarrhea) is called the verbatim term. Some medical events can be named in many different ways, but they are all referring to the same clinical finding or diagnosis (e.g., hypertension, high blood pressure, elevated blood pressure, increased BP, etc.). For the analysis of adverse events, the verbatim terms will be interpreted through a coding dictionary. The coding dictionary will match each reported verbatim as closely as possible to the appropriate medical term, which will then be grouped with other like terms that all describe the same clinical event. The dictionary includes a hierarchy of terms, so that the more specific, lower level terms can be then grouped into disorders of specific

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

ADVERSE EVENT EVALUATION

mechanisms, diagnoses, and body systems (see Coding Dictionaries).

4

RELATIONSHIP TO THE INTERVENTION

Assessing relationship to the intervention or the investigational product is often one of the more difficult aspects of safety data collection and interpretation. Clinical study subjects often have comorbidities, physical findings related to the disease under study, side effects from other concomitant medications, or even normal variations in health status that complicate the decision about whether a particular adverse event is related to the intervention being studied. Even in early phase studies of healthy volunteers, there are reports of adverse events of alanine aminotransferase elevations in subjects receiving placebo (2), which may be from the change in daily habits during study participation, or to fluctuations in normal laboratory values when watched more closely than is standard clinical practice. In determining the relationship, there are several questions the investigator must consider. What was the timing of the adverse event (start date/time, and stop date/time)? What was the relationship to the administration of the intervention? Depending on the type of investigational product and the pharmacokinetics, this may be related to the actual dosing time, dosing route (for example, an injection site reaction), a maximal plasma concentration, or a cumulative exposure over time. If the intervention or investigational product is dose reduced or withdrawn, does the event improve? If the dose is increased or the intervention is restarted, does the event recur? Are there other comorbid conditions or other medications that may have caused the event? Has this event or this type of event been seen before with this intervention? Is there a reasonable biologic mechanism through which this event could have been caused? After determining the likelihood that the adverse event is related to the intervention, the investigator must choose one of several options that reflects the clinical assessment. The options usually include no relationship,

an unlikely relationship, a possible relationship, a probable relationship, and a definite relationship. The specific wording of the options may vary (‘‘unlikely’’ is sometimes called ‘‘remote’’). There are rarely more options for the relationship categories, but there are often fewer; sometimes the options are collapsed into four, three, or even two categories (related/not related). The CDISC standard, which will be standard for all data used to support regulatory approvals by 2010 (3), includes four options (not related/unlikely related/possibly related/related). 5 ASSESSING SEVERITY The investigator must assess the severity, sometimes called the intensity, of the adverse event. For the interpretation of the data collected, it is important that investigators are using the same guidelines to determine severity, so that an event considered moderate in intensity by one investigator is reasonably similar to the same type of event considered moderate by a different investigator. For this reason, there are many standard guidelines for the assessment of the severity of adverse events. Perhaps the best known of the standard guidelines is the National Cancer Institute’s Common Toxicity Criteria (NCI CTC) (4). This guideline, now in Version 3.0 and more extensive with each version, includes hundreds of possible adverse events with brief descriptions for what would be considered appropriate for each grade of severity, from grade 1 through grade 4. In general, grade 1 events are considered mild, grade 2 events are moderate, grade 3 events are severe, and grade 4 events are life-threatening. Any event that results in death is considered grade 5. For example, nausea is grade 1 if it includes loss of appetite but no real change in eating habits, grade 2 if there is decreased oral intake but no weight loss or malnutrition and IV fluids are needed for less than 24 hours, and grade 3 if oral intake is so low that parenteral nutrition or hydration is required for more than 24 hours. Results of laboratory tests are graded as well, usually by the proportional change from the reference range for the laboratory (i.e., 0.5 times

ADVERSE EVENT EVALUATION

lower limit of normal range) rather than by absolute numbers. The NCI CTC is well known, extensive, and commonly used, but it is not appropriate for all clinical studies. The NCI CTC is designed for use in clinical studies in oncology, and in other populations (particularly healthy volunteers), the grading of events may be considered too lenient, with significant clinical signs and symptoms before an event is considered to be severe. In the design of a new clinical study, it is necessary to review different grading guidelines to find the one best suited to the study and the study population—or, in some cases, to develop a new grading guideline, particularly if there are safety events that are expected or that are of particular interest. In general, for events for which there is not specific grading guidance, the usual recommendation is to consider the impact on the activities of daily life for the study participant, with a mild event having little or no impact, a moderate having some impact, and a severe event having substantial impact. 6

DETERMINING EXPECTEDNESS

The determination of when a specific event is considered to be ‘‘Expected’’ for a specific investigational product is a formal process. In general, an event is considered expected if events of the same nature and severity are described in the product label (or in the case of investigational products, the Investigator’s Brochure). The formal determination of expectedness is important, as it is a significant factor in the requirements for the reporting of adverse events to regulatory authorities. In general, serious adverse events that are considered expected do not have to be reported in the same expedited manner (within 7–15 days, depending on other factors) to investigators and regulatory agencies. Therefore, the persons who are responsible for maintaining the Investigator’s Brochure usually make the decision about what is considered ‘‘Expected.’’ In addition to considering the documented safety profile for the intervention, the investigator may use an informal interpretation of expectedness also taking into account his or her own experience. The expectedness should

3

be considered as part of the determination of relationship, but the investigator will not be asked to specify the expectedness of an event.

7

ACTIONS TAKEN

In addition to collecting the event name, start and stop dates (and sometimes times), the relationship, and the severity, it is also important to know what actions were taken in response to a specific event. Did the dose of the investigational drug have to be decreased, or did dosing have to be interrupted (with possible impact on efficacy)? Did the administration of the intervention have to be stopped completely? Did the subject discontinue from the study? Did they require addition medications to treat the event, or hospitalization? Although in general, the more dramatic actions (stopping administration, etc.) may parallel the event severity, this may not always be the case. Adverse events that do not have significant clinical safety impact may still impact the tolerability of the investigational product, especially if it will be used over a long period of time. Because of the potential effects of tolerability on treatment adherence, this information must be analyzed as part of the overall safety profile.

REFERENCES 1. European Medicines Agency, ICH E2A. Clinical Safety Data Management, Definitions and Guidelines for Expedited Reporting. (March 1995). Available: http://www.fda.gov/cder/ guidance/iche2a.pdf, accessed January 2008. 2. Rosenzweig P, Miget N, Broheir S. Transaminase elevation on placebo in Phase 1 Clinical Trials: Prevalence and significance. J. Clin. Pharmacol. 1999; 48:19–23. 3. Clinical Data Interchange Standards Consortium; mission statement and strategic plan summary. Available: http://www.cdisc.org/ about/index.html, accessed January 2008. 4. National Cancer Institute Common Toxicity Criteria, Version 3.0. Available: http://ctep. cancer.gov/reporting/ctc v30.html, accessed January 2008.

4

ADVERSE EVENT EVALUATION

FURTHER READING Guidance for Industry: Adverse Event Reporting—Improving Human Subject Protection. (Issued in draft, April 2007). Available: http://www.fda.gov/cber/gdlns/advreport.pdf, accessed January 2008. Guidance for Industry: Premarketing Risk Assessment. (Issued March 2005). Available: http:// www.fda.gov/cder/guidance/6357fnl.pdf, accessed January 2008.

CROSS-REFERENCES Coding dictionaries Regulatory definitions Safety assessments Adverse event reporting system

ADVERSE EVENT REPORT SYSTEM (AERS)

labeling information, sending out a ‘‘Dear Health Care Professional’’ letter, or reevaluating an approval decision.

The Adverse Event Reporting System (AERS) is a computerized information database designed to support the U.S. Food and Drug Administration’s (FDA) postmarketing safety surveillance program for all approved drug and therapeutic biologic products. The ultimate goal of the AERS is to improve the public health by providing the best available tools for storing and analyzing safety reports. The FDA receives adverse drug reaction reports from manufacturers as required by regulation. Health-care professionals and consumers send reports voluntarily through the MedWatch program. These reports become part of a database. The structure of this database is in compliance with the International Conference on Harmonisation (ICH) international safety reporting guidelines (ICH E2B). The guidelines describe the content and format for the electronic submission of reports from manufacturers. The FDA codes all reported adverse events using the standardized international terminology of MedDRA (the Medical Dictionary for Regulatory Activities). Among the AERS system features are the on-screen review of reports, searching tools, and various output reports. The FDA staff use AERS reports in conducting postmarketing drug surveillance and compliance activities and in responding to outside requests for information. The AERS reports are evaluated by clinical reviewers in the Center for Drug Evaluation and Research (CDER) and the Center for Biologics Evaluation and Research (CBER) to detect safety signals and to monitor drug safety. They form the basis for further epidemiologic studies when appropriate. As a result, the FDA may take regulatory actions to improve product safety and protect the public health such as updating a product’s This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/aers/default.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

ADVISORY COMMITTEES Advisory committees provide the U.S. Food and Drug Administration (FDA) with independent advice from outside experts on issues related to human and veterinary drugs, biological products, medical devices, and food. In general, advisory committees include a Chair, several members, plus a consumer, industry, and sometimes a patient representative. Additional experts with special knowledge may be added for individual meetings as needed. Although the committees provide advice to the FDA, final decisions are made by the Agency. Nominations for scientific members, consumer, industry, and patient representatives originate from professional societies, industry, consumer and patient advocacy groups, the individuals themselves, or other interested persons. Candidates are asked to provide detailed information regarding financial holdings, employment, research grants and contracts, and other potential conflicts of interest that may preclude membership. Persons nominated as scientific members must be technically qualified experts in their field (e.g., clinical medicine, engineering, biological and physical sciences, biostatistics, and food sciences) and have experience interpreting complex data. Candidates must be able to analyze detailed scientific data and understand its public health significance.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/oc/advisory/vacancies/ acvacfaq.html) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

AIDS CLINICAL TRIALS GROUP (ACTG)

recognizing that a CD4 cell is infected and destroying it. After a burst of viral replication that occurs immediately after people are initially infected, viral loads drop to a lower ‘‘set point.’’ The effects on the immune system are gradual, so individuals often are unaware they are infected for years. If left untreated, the disruption of the immune system by HIV infection is catastrophic. This leaves people vulnerable to a long list of opportunistic infections (OIs), wasting, neurologic damage, and cancers that people with an intact immune system do not normally have. When this state is reached, people are considered to have AIDS. HIV was initially identified in 1981 when a small but growing number of individuals in cities in the United States and Europe were diagnosed with unusual and rapidly fatal infections and cancers. Once the syndrome was described and when a blood test for antibody to it became available in 1983, it became apparent that this infection already existed worldwide at pandemic proportions. In 2005, it was estimated that 33.4 to 46.0 million people were living with HIV worldwide, 3.4–6.2 million became newly infected, and 2.4–3.3 million people had already died from HIV/AIDS (1).

JANET W. ANDERSEN ACTG Statistics and Data Management Center, Harvard School of Public Health, Boston, Massachusetts,

This article provides a brief overview of HIV and AIDS, a short history of the founding of the ACTG, an overview of the structure of the ACTG, and sections that outline the major areas covered by the agenda of the ACTG, including clinical trial design issues that developed in the context of the studies and science that was developed to answer this agenda. The ACTG website (www.aactg.org) provides brief overviews of Group studies open to enrollment, information about the ACTG and clinical trials in general, and links to a vast array of sites with information about HIV and AIDS. 1

A BRIEF PRIMER ON HIV/AIDS

Acquired immunodeficiency syndrome (AIDS) is a collapse of the immune system caused by infection with the human immunodeficiency virus (HIV). HIV is a retrovirus; a retrovirus is a virus whose genetic material is RNA that is read into DNA (or reverse transcribed to DNA) when the virus enters the target cell. This DNA incorporates itself into the DNA in the host cell’s nucleus, and then it is replicated to produce new retrovirus particles. HIV is passed from person to person primarily by direct blood transfer from transfusions or needles, by unprotected sexual activity, or from a mother to her child at birth or through breast-feeding. HIV’s target cell in humans is the CD4 or helper T-cell. CD4 cells are the part of the blood’s lymphocytes or white blood cells that are responsible for organizing and arousing (activating) the other cells of the immune system to produce antibodies against an invader or to attack a foreign cell directly. HIV infection of the CD4 cells results in their destruction, either directly by the virus or from the immune system

2

ACTG OVERVIEW

2.1 History In 1986, the National Institutes of Allergy and Infectious Diseases (NIAID) established the Division of AIDS (DAIDS). In that same year, NIAID established the AIDS Treatment Evaluation Units, and, in 1987, they established the AIDS Clinical Studies Groups to conduct studies of treatment for AIDS and HIV. In 1988, the AIDS Clinical Trials Group (ACTG) was formed by the merger of these two groups. Two original units had a primary focus in the treatment of children with HIV. This component of the ACTG agenda grew as the Network grew. In 1997, NIAID funded the Pediatric AIDS Clinical Trials Group (PACTG) that was constituted from the sites in the ACTG that specialized in studies in pediatric HIV and prevention of mother-tochild transmission (MTCT). At that time, the

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

AIDS CLINICAL TRIALS GROUP (ACTG)

ACTG was renamed the Adult AIDS Clinical Trials Group (AACTG) to reflect the change in its focus (2). In 2006, the NIAID and DAIDS initiated a wide-ranging reorganization of the AIDS Networks in the United States including the vaccine and prevention networks as well as those like the AACTG that focused on treatment of the disease. In the grant period between 1999 and 2006, the AACTG had increased its agenda to include international membership engaged in both common and country-specific clinical research. In the reorganization of the networks, the AACTG defined and expanded this international effort, restructured its scientific committees, and integrated the Oral HIV/AIDS Research Alliance (OHARA) that studies oral manifestations and complications of HIV. To reflect this revised and refocused agenda, the AACTG reassumed the name ‘‘AIDS Clinical Trials Group’’ and the acronym, ACTG. This overview will refer to the Group as the ACTG throughout.

•

•

2.2 ACTG Structure The infrastructure required for the success of a network of the size of the ACTG is massive, interlocking, and complex. The general structure of the ACTG has been very stable over time, consisting of the following: • The Group Chair, Vice-Chair, and Exec-

utive Committee. The ACTG Executive Committee (AEC) is constituted of Group leadership and representatives from the Scientific Committees, community, sites, Operations Center, Statistical and Data Management Center (SDMC), and DAIDS. Subcommittees of the AEC include one that evaluates the performance of the Group and its sites, and the Scientific Agenda Steering Committee that coordinates and oversees the scientific agenda of the Group. The Principal Investigators of each site are an essential component of the leadership of the Group. • The Operations Center that is sited at Social and Scientific Systems (Silver Springs, MD). This center provides administrative support and coordination for

•

•

leadership and scientific activities, protocol development and implementation, site establishment, regulatory activities, and contract management. The SDMC that is constituted of the Statistical and Data Analysis Center (SDAC) sited at the Harvard School of Public Health (Boston, MA) and the Data Management Center (DMC) sited at the Frontier Science Technology and Research Foundation, Inc. (Amherst, NY). SDAC members provide leadership and expertise in protocol design and implementation, and they prepare both interim and final analyses of studies. SDAC members also conduct methodological research essential to HIV studies. DMC members design the study forms, monitor data and specimen submission and quality, conduct training for sites, and work closely with sites on data issues. The Network Laboratory Committee that is constituted of laboratories studying virology (quantification of HIV and other viruses; testing of in vitro and genetic drug resistance), immunology (measurement of cytokines, lymphocyte proliferation, and cellular immune responses), and pharmacology (measurement of parameters of drug distribution and elimination and drug bioavailability). The laboratories also develop innovative assays and methodology in this changing field. The Network Community Advisory Board and the community at large bring community perspectives and input to the Group’s agenda as a whole and to issues related to specific research areas and protocols. Community representatives are members of ACTG leadership groups as well as of all protocol teams. The sites, site investigators and staff, and subjects who are the backbone of the research.

3 ACTG SCIENTIFIC ACTIVITIES The scientific agenda of the ACTG is carried out through its scientific committees. It is

AIDS CLINICAL TRIALS GROUP (ACTG)

impossible to detail the activity of the ACTG since its inception in this brief overview. A few studies will be mentioned to illustrate the nature and breadth of the integrated research conducted by the Group. References are provided where results are given but not for every study number cited. A search engine such as PubMed (3) can be used to identify ACTG publications by topic and/or study number. Information on current studies is on the ACTG website. 4 DEVELOPMENT OF POTENT ANTIRETROVIRAL THERAPY (ART) Treatment of HIV ultimately depends on preventing components of the retroviral lifecycle. Thus far, no evidence suggests the eradication of progressive HIV disease. However, the development of drugs and combination ART has resulted in prolonged survival and profound improvements in quality of life when drugs are taken as prescribed. The numbers and types of drugs available have mirrored the increasing understanding of the structure and functions of the virus and its components. Although active in early drug development, an essential role of a large, multicenter network like the ACTG is the ability to complete large comparative (i.e., Phase III and IV) studies quickly to make the results about optimal treatments available to the clinical community. Once HIV was identified as a retrovirus, nucleosides such as zidovudine (ZDV), didanosine (ddI), lamivudine (3TC), and zalcitabine (ddC) were developed that interfere with the reverse transcription of the viral RNA into DNA. The class of drugs is called nucleoside reverse transcriptase inhibitor (NRTI). The ACTG conducted pivotal studies that led to the approval of these drugs as monotherapy and also identified the emergence of drug resistance that limits long-term efficacy. To illustrate the scope of the epidemic, ACTG 019 that tested ZDV versus placebo enrolled 3222 subjects in 2 years. ACTG 175 (4) was a randomized, doubleblind, placebo-controlled study of ZDV, ddI, ZDV + ddI, or ZDV + ddC, which demonstrated the superiority of combination NRTIs over monotherapy. Numerous studies investigated other NRTIs and combinations.

3

A second class of drugs, the nonnucleosides or NNRTIs such as nevirapine (NVP) and efavirenz (EFV), was then developed. ACTG 241 (5) was the first study of a 2-class 3-drug regimen (ZDV, ddI, and NVP) that showed the superiority of multiclass therapy. ACTG 364 (6) extensively evaluated EFV + NVP + 2 NRTIs, which showed this arm to be superior to three-drug regimens in subjects with prior NRTI experience. For HIV to replicate, a long protein must be cleaved by an enzyme (a protease) encoded by its genes. The protease inhibitors (PIs) were the next class of effective drugs to be developed. ACTG 320 (7) was a turning point in HIV research. 3TC or 3TC plus the PI indinavir (IDV) were added to ZDV in ZDV-experienced subjects. A data safety monitoring board (DSMB) halted the study early because of the superiority of the IDV arm in preventing AIDS-defining events (OIs, cancer, etc.) or death. Data and experience from this trial, which followed both clinical events and HIV viral loads with a newly developed assay, also provided convincing evidence that quantitative measurement of the HIV virus was a highly relevant and reliable early surrogate endpoint. Subsequent studies investigated increasingly active combination therapy. For example, ACTG 384 was a partially blinded, randomized, 2 × 3 factorial design in which the first factor was two different NRTI backbones, and the second factor was the use of nelfinavir (NFV, a PI) or EFV or both. This study specified second-line regimens to be used if the initial failed; primary endpoints included both the results of the initial regimen and comparisons of consecutive treatments (8). This study demonstrated the superiority of one of the six arms (ZDV/3TC + EFV), that ZDV/3TC in combinations was more effective and safer than ddI + d4T, and that a four-drug regimen was not better than the three-drug regimen (9,10). These findings led to changes in U.S. ART guidelines. Some Phase III studies evaluate the equivalence or noninferiority of new drugs or drug combinations compared with standard regimens. A5095 investigated three PI-sparing regimens hoping to preserve the class for those who failed first-line therapy. It compared a control arm of 3 NRTIs,

4

AIDS CLINICAL TRIALS GROUP (ACTG)

including abacavir (ABC), with two arms that contained NRTIs + EFV. The primary aims were to demonstrate: the noninferiority of ABC/3TC/ZDV to 3TC/ZDV/EFV, the superiority of ABC/3TC/ZDV/EFV to 3TC/ZDV/EFV, and the superiority of ABC/3TC/ZDV/EFV to ABC/3TC/ZDV. At a prespecified review and based on the statistical design that included guidelines for early evaluation of the noninferiority endpoint (11), the DSMB recommended discontinuation of the three-NRTI arm because of inferior results and suggested continuation of the EFV-containing arms (12). This study illustrates numerous issues involved in the design and management of equivalence and noninferiority studies, and it is an example of efficient study design. As the number of drugs in all classes and the surety of HIV control in a high proportion of ART-na¨ıve subjects increased, tablets that combine highly active drugs were formulated, including a ‘‘one pill once-a-day’’ regimen. A5175 is a three-arm, noninferiority study of a once-daily PI-containing and once-daily and twice-daily PI-sparing regimen, which are both primarily targeted for resource limited settings (refrigeration not required) including sites in Africa, India, South America, and Thailand, although sites in the United States are also participating. A5202 is a four-arm equivalence study of once-daily regimens. Although the arms form a factorial design (PI vs. NNRTI as one factor, and the NRTI backbone as the other), the sample size is fully powered for between-arm equivalence comparisons to guard against a between-factor interaction that could limit the interpretability of the results. Studies conducted internationally must consider numerous other factors, such as differences in host and viral genomics; socioeconomic issues, including communication, transportation and laboratory capabilities; malnutrition; the social stigma of HIV infection; and co-infections such as tuberculosis (TB) and malaria. 4.0.1 Side Effects. No regimen is without adverse effects. Some side effects of ART, such as nausea, diarrhea, reduced blood counts, and impaired liver and kidney function, can be managed with dose and

drug changes as well as supportive care. Others are more complex, including ARTrelated neuropathy and metabolic complications, such as disfiguring lipoatrophy and fat redistribution, mitochondrial toxicity, and extremely high cholesterol levels. The first challenges for the ACTG were developing definitions of these new conditions and understanding their etiologies (13). In fact, some of the first investigations involved establishing whether the metabolic complications were a direct effect of ART or a result of dysregulation from the longer-term survival with HIV that ART was making possible. The ACTG has conducted numerous studies to investigate the underlying cause of these conditions and also to prevent and treat them. The use of substudies, which are intensive investigations on a subset of subjects in a large study, is very powerful. For example, A5097s investigated the neuropsychological effects of EFV in A5095 (14), and early effects were found such as ‘‘bad dreams’’ during the first week of therapy. 4.0.2 Viral Load Data, Complex Endpoints. Both the development of sensitive assays to measure the HIV viral load and the advent of fully suppressive ART resulted in the need for innovative methods for study design and analysis. All tests for viral loads as well as many other markers have a lower limit of reliable quantification; a result below this limit does not mean virus has been eliminated. SDAC members have been at the forefront of development for methods for such ‘‘left censored’’ data (15), combined correlated endpoints (16), and the use of surrogate endpoints in clinical trials (17). 4.0.3 Drug Resistance. HIV replication is highly error prone, which results in a high rate of viral mutation. A change in just one base in HIV’s RNA is sufficient to confer resistance not only to a specific drug but also to other drugs from the same class. ACTG researchers have provided breakthroughs in understanding, identifying, and quantifying drug resistance, including extensive modeling to uncover complex patterns of mutations that affect viral control and the use of predicted drug susceptibility to guide regimen choice. Salvage studies such as A5241

AIDS CLINICAL TRIALS GROUP (ACTG)

and treatment options after virologic failure included in front-line studies like A5095 base treatment assignment on an individual’s virus’ drug resistance profile. Additionally, the possibility of resistance developing in the mother’s virus from brief treatment with ART during pregnancy to prevent MTCT and implications for the future treatment of the mother are considered in several studies (A5207, A5208). 4.0.4 ALLRT. As survival with HIV has increased, long-term follow-up of subjects with standardized data and sample collection beyond participation in an individual study became vital. The ACTG mounted the ALLRT (AIDS Longitudinally Linked Randomized Trials or A5001) in 2000 with an accrual target of at least 4500 enrollees. Subjects in targeted, large, randomized ART studies are followed on the ALLRT during and far beyond participation in their initial study. Data available from any concurrently enrolled ACTG study are merged with data collected specifically for the ALLRT, which results in a complete longitudinal database. Numerous analyses that involve long-term response, immunologic recovery, quality of life, complications of HIV and therapy, co-infection, and epidemiologic design have resulted or are underway. 4.1 Drug Development, Pharmacology (PK), and Pharmacogenomics A large component of the ACTG’s activities is the initial evaluation of new therapies for HIV, from the earliest days of ddI, to intensive investigations of novel modalities such as entry inhibitors that target one specific molecule on the surface of the CD4 cell (A5211). Some of the earliest stages of drug development involve establishing the pharmacology of a drug: How fast is it absorbed and eliminated? Even brief monotherapy quickly results in viral drug resistance as noted earlier, so studies of individual drugs are performed in HIV-uninfected volunteers. Subjects with HIV are taking at least three antiretrovirals in their ART. Thus, any PK study in HIV-infected subjects is ultimately a study of drug–drug interactions, either comparing the PK of the target drug between

5

monotherapy in HIV uninfected volunteers and when added to ART in HIV-infected subjects, or comparing the PK of one or more of the ART drugs before and after receipt of the new agent. These studies are essential because one drug can cause another either to be ineffective or to reach toxic levels at normal dosing. Subjects with HIV also receive numerous medications to prevent and treat AIDSdefining illnesses and side effects of ART. Thus, the ACTG also conducts essential studies of interactions between antiretrovirals and other medications. For example, simvastatin is often prescribed to combat ARTrelated dislipidemia. A5047 (18) provided crucial evidence of a life-threatening increase in simvastatin levels when given with RTV plus saquinivir, which is a PI, whereas A5108 (19) established that it was safe when given with EFV. As well as population-based analyses, determination of the genetic basis of drug metabolism as well as other biologic systems is essential both to individualize treatment and also to explain and ultimately predict outcome. In 2002, the ACTG launched a unique study, which is called A5128 (20), to collect informed consent and samples specifically for human genetic data to be linked to clinical data in a highly confidential manner. An example of the power of both A5128 and the interrelationship of ACTG studies is a merge of data from A5128, A5095, and A5097s that determined that a specific allelic variant in a liver enzyme gene that is more common in African Americans is associated with much greater EFV exposure (21). The ACTG conducts some classic doseescalating Phase I studies, but it has performed numerous studies that test the efficacy of new drugs or, perhaps more importantly, identify inefficacious or problematic drugs and combinations. ACTG 290 and 298 confirmed laboratory findings that ZDV and d4T are antagonistic, which led to ART guidelines that they not be used together. ACTG 307 and A5025 demonstrated that hydroxyurea was inefficacious. Additionally, the latter also showed that the risk of pancreatitis was increased when ddI and d4T were given together, which led to the recommendation not to combine them. Phase II

6

AIDS CLINICAL TRIALS GROUP (ACTG)

evaluations of strategies for multidrug class experienced subjects lead to studies of ‘‘class sparing’’ front-line regimens, and research on studies such as ACTG 398 demonstrated the importance to drug resistance of ‘‘minority’’ variants in the HIV genome. 4.2 Immunology, Pathogenesis, and Translational Research Ultimately, central issues in HIV infection are the failure of the individual’s immunity to mount an effective defense when the person first became infected and the inability of the immune system to clear the virus over time. The ACTG’s research in this area includes investigations in immunity and its regulation, immune-based therapies to enhance anti-HIV immunity after infection, immune restoration with ART, and special issues such as reservoirs where virus can lie latent and unique challenges and opportunities in specific populations. 4.2.1 HIV-Specific Immunity. It was not clear whether the inability of the immune system to recognize and destroy the HIV virus was caused solely by the progressive loss of both na¨ıve (unprimed) and memory (lymphocytes that remain after an immune response) antigen-reactive subsets of the CD4 cells, or whether wider intrinsic or HIVrelated defects occurred in cellular immunity. Clinical and basic research informs each other and both were critical in understanding the interaction of complex biological systems involved in the immune disregulation in this disease. Studies in the pre-ART era (e.g., ACTG 137 and 148) found some immunological effect but no clinical benefit of vaccination with proteins derived from the virus’ outer coating, which is a strategy effective in other diseases. Once CD4 counts were being restored with ART, the logical step was to investigate strategies to boost HIV-specific immunity as an adjunct to the initiation of ART and in synergy with successful ART. This strategy required the ACTG to pursue innovative basic research in the underlying biological mechanisms of cellular immunity; the nature of the defects, which includes both those caused by the virus and those lingering despite viral suppression and the level

of immunocompetency required for any given immune-based strategy to be successful. This research also required an intensive effort in the development of specialized laboratories, standardization and validation of research assays, and development of a central specimen repository to ensure proper storage and that samples could be retrieved long after study closure. 4.2.2 Immune Restoration. Once reliable suppressive ART had been defined, it became important to study whether the immunodeficiency caused by HIV is reversible and what factors are associated with the degree and pace of reconstitution. ACTG studies 315 and 375 followed subjects over 6 years of successful ART, finding that most failed to restore normal CD4 counts and function (22). Intensive studies of clinical (A5014, A5015), immunologic (DACS 094, DACS 202), and human genetic (NWCS 233) parameters could not identify strong predictors of who would and would not achieve immunologic reconstitution among subjects who had sustained virologic control on ART. Studies such as ACTG 889 found that recovery of immunity to pathogens other than HIV is also incomplete, which indicated not only that patients might still be susceptible to OIs despite a CD4 increase, but also that the effectiveness of anti-HIV immune therapy might need to be investigated in populations with a relatively intact immune system. Although consistent predictors of immunologic reconstitution have not yet been identified, the Group has produced extensive cutting-edge research in immune activation and modulation, CD4 cell subsets, pathogenspecific immunity, thymic activity, and HIV-1 pathogenesis. 4.2.3 Immune-Based Therapy. Because HIV control alone is insufficient to restore a normal immune system, the ACTG has investigated immune-based treatments as adjuncts to ART including immunomodulators (e.g., growth factors, cyclosporin A, IL-2, and IL12) and therapeutic vaccination with more general polyvalent peptide vaccines (A5176) and also those based on viral vectors (A5197). Future studies might include infusions of autologous cells primed ex vivo

AIDS CLINICAL TRIALS GROUP (ACTG)

with autologous HIV RNA. The ‘‘analytical treatment interruption’’ (ATI) is a unique and powerful tool for evaluating immunomodulation of HIV. After establishment of sustained viral suppression and with careful criteria for CD4 levels, all ARVs are discontinued. HIV viral rebound is closely monitored and ART is restarted if rebound is rapid or the CD4 count drops. Successful immunomodulation provides a substantial period of time with minimal viral replication and with sustained CD4 counts. ATI provides rapid readout to test vaccines and other immune-based modalities. 4.2.4 Latent Reservoirs, Special Populations. Understanding HIV has required innovative research into underlying differences between populations. An intensive study (A5015) included measurement of thymic volume as well as serial quantification of numerous T-cell and serum markers of immune function. Subjects were 13–30 years of age or at least 45 years old. Not only did the study identify age-associated differences in immune restoration with ART, but also numerous new findings were made in the biology of immune restoration. Genomic research is covered elsewhere, but it is an important component of understanding subpopulations. PK studies in women consider interactions between ARVs and contraceptives, and other studies evaluate other aspects of gender differences (e.g., do women and men with the same body mass index have different PK?). HIV can lie latent in resting cells, and several intensive studies consider viral reservoirs other than the blood such as lymph nodes, cerebrospinal fluid, mucosal tissue, and the genital tracts of men and women to understand how to eradicate all HIV viruses to prevent resurgence. 4.3 OIs and other AIDS-Related Complications As noted above, the immune system collapse caused by prolonged infection with HIV makes individuals susceptible to a long list of diseases that are very rare in people with an intact immune system and that often have an unusually rapid course with AIDS. The Centers for Disease Control, the World

7

Health Organization, and other groups have developed a list of AIDS-defining conditions (23). These conditions include the following: fungal infections (candidiasis, Pneumocystis carinii pneumonia, and cryptococcus); bacterial infections [Mycobacterium avium complex (MAC), and TB]; viral infections (cytomegalovirus); wasting; neurological complications [progressive multifocal leukoencephalopathy (PML), dementia, peripheral neuropathy (PN)]; and cancers [Kaposi’s Sarcoma (KS), lymphomas, cervical cancer].

4.3.1 OIs. Before effective ART was developed, numerous trials in the ACTG were performed to study prevention and treatment of these life-threatening diseases and conditions. An example of the prevalence of these unusual conditions in AIDS is ACTG study 196 that tested treatment for MAC, which enrolled over 1200 subjects in 9 months from late 1993 to early 1994. The role of the ACTG in drug development is evident in sequences like ACTG 196 that demonstrated the superiority of clarithromycin over standard therapy (24) followed by ACTG 223 that demonstrated the superiority of clarithromycin plus two other drugs (25). Similar sequences of studies in other OIs lead to enhanced quality of life and prolonged survival for subjects waiting successful drugs for HIV. The development of highly effective ART in the late 1990s produced a paradigm shift. Several OI studies were halted or completely refocused when it became apparent that immune reconstitution with ART dramatically reduced OI incidence. For example, a planned interim analysis of ACTG 362, which is a large three-arm study of prevention of MAC, found only two events when many were anticipated. The study was continued for extended follow-up to focus on cardiovascular, metabolic, and neurologic changes, and the rates of AIDS-defining events in these subjects who once had a very low CD4 count (C/A means more than C toxicity incidences out of A patients. Similarly, the notation E/(A + B)

Escalate to Dose Level l + 1* Dose level de-escalate to l − 1**

≤ E/(A + B)

E/(A + B)

>D/A

Dose level l − 1

Dose de-escalate to the lower dose level**

* At the last dose level n

Enroll A patients

>D/A

≤D/A

Add B patients To dose level n − 1

>E/(A + B)

≤E/(A + B) No MTD can be identified

** if l − 1 = 0

No MTD can be identified

as compared with the standard A + B design in two case studies. They concluded that the modified accelerated design and the standard design select a dose level as the MTD with

Figure 1. Diagram of modified accelerated design.

similar probabilities. However, the modified accelerated design requires fewer expected numbers of patients because of its ability to accelerate dose escalation through low-dose

6

ALGORITHM-BASED DESIGNS

levels and to switch to the standard design when DLT is encountered. Under certain dose toxicity profiles such as a dose-toxicity response curve that is gradually increasing, the savings in patient resources and time may be tremendous, translating into much shorter studies. For trials with high starting-dose toxicity or with only a few dose levels, the modified accelerated design may not save the number of patients because it rushes through the initial doses to the higher doses and may require dose de-escalation a few more levels down than the standard design. Their modified accelerated design can also be extended easily to allow the flexibility to start a trial at a dose level that may be several levels above the studies prescribed minimum dose. This flexibility gives researchers increased freedom to expedite a trial as needed while having the option to move downward to the lower doses when necessary. 3 MODEL-BASED APPROACH IN THE ESTIMATION OF MTD In cancer clinical trials, a single universal TTL does not appear to exist based on the literature (5, 12, 21, 26). Depending on the type and severity of a disease, a TTL of approximately 33% is suitable for many cancer trials, and a TTL of 50% may also be reasonable when the toxicities are viewed to be less serious (or more acceptable) relative to the severity of the disease being treated. Among the variety of possible TTLs, it would be difficult to use the algorithm-based approach to derive the MTD of certain TTLs as MTD observed from the data from an algorithmbased design has no interpretation as an estimate of the dose that yields a prespecified rate of target toxicity (6). 3.1 Evaluation of the Traditional Algorithm-based 3 + 3 Design The common perception by most clinicians is that the traditional algorithm-based 3 + 3 designs produce a 33% toxicity rate (5). This is not true based on discussions by Lin and Shih (10) and Kang and Ahn’s simulation study (27, 28). Kang and Ahn showed that the expected toxicity rate at the MTD is between

19% and 22% if the dose-toxicity relationship is assumed to be logistic or hyperbolic tangent functions. He et al. (29) also conducted a simulation study to investigate the properties of the algorithm-based 3 + 3 design. Instead of assuming a particular functional form of the dose-toxicity relationship, they only allowed the dose-toxicity relationship to be monotonically nondecreasing. They found that the expected toxicity levels are in the range of 19% to 24% rather than the anticipated 33%, depending on the number of prescribed doses in a study. They observed that the estimate of the expected toxicity level and its associated standard deviation decrease with the increasing number of dose levels prescribed for a study. This is most likely for the following reasons. For the algorithm-based 3 + 3 designs, the MTD is designed as the highest dose level (≥1st dose level) in which either six patients are treated with at most one patient experiencing the DLT or three patients are treated with no patient experiencing the DLT and the immediate higher dose level having at least two patients experiencing the DLT out of a cohort of three or six patients. The expected toxicity level at the MTD that is identified under this design scenario is more likely associated with DLTs between 0% (0/3) to 16.7% (1/6) because at most zero out of three patients or one out of six patients experiences DLT at the MTD level. With increasing number of prescribed dose levels in a study, the accuracy and precision of identifying a MTD that is associated with an expected toxicity level approaching this limit increase. Because it has been shown that the traditional 3 + 3 algorithm-based designs cannot provide accurate estimates of the MTD when TTLs are set high (27–29), He et al. (29) proposed a model-based approach in the estimation of the MTD following an algorithm-based design. 3.2 Model-based Approach in the Estimation of MTD He et al. (29) assumed a one-parameter family model to depict the dose-toxicity relationship and denoted it by ψ(d1 ,a). Let the dose range be D = {d1 , . . ., dk }, where k is the number of dose levels in the study; let S =

ALGORITHM-BASED DESIGNS

{s1 , . . ., sm } be the dose level in the dose escalation or de-escalation steps during the trial, where m is the total number of steps of dose escalation and/or de-escalation before the study stops; and let Y = {y1 , . . ., ym } be the number of patients experiencing DLT events at the corresponding S = {s1 , . . ., sm } steps, and the elements in Y may take on values (0, 1, 2, 3). Under traditional 3 + 3 designs, at each step i three patients are studied at dose level si , where dose level si may or may not be equal to dose level sj at step j (i = j). They derived the likelihood function as follows: L(a) =

m


Pr(yl |sl ) ∝

l=1

m
{ψ(sl , a)}yl l=1

{1 − ψ(sl , a)}

(3−yl )

.

(1)

These derivations can be easily extended to general traditional A + B algorithm-based designs. Let v = {v1 , . . . , vm } be the number of patients studied at the corresponding S = {s1 , . . . , sm } steps, and the elements in v may take on values (A, B) for the traditional A + B designs. Based on the likelihood function in equation 1, the likelihood function for a traditional A + B design can be written as L(a) ∝

m
{ψ(sl , a)}yl {1 − ψ(sl , a)}vl −yl ) .

(2)

l=1

Because the likelihood function in equation 2 is derived based on the data from each step of dose escalation or de-escalation, this likelihood function can also apply to the modified accelerated 1 + A + B designs mentioned in the previous section. In the case of modified accelerated designs, v = {v1 , . . . , vm } may take on values (1, A, B). Further, for any algorithm-based designs, be they an A + B design or an accelerated design or an M1−M3 A + B design (7), the likelihood function can be written as equation 2, with v = {v1 , . . . , vm } taking on possible integer values in (1 to A) or (1 to B). He et al. (29) employed a Bayesian approach for the model inference. They conducted simulations to investigate the properties of their model-based approach in the estimation of the MTD as compared with the MTD identified from a traditional

7

algorithm-based 3 + 3 design. The simulation results demonstrated that their modelbased approach produced much less biased estimates of the MTDs compared with the estimates obtained from the traditional 3 + 3 designs. Although the traditional 3 + 3 designs produce estimates of the MTD that are generally associated with expected toxicity levels of 19% to 24% regardless of any prespecified target toxicity levels, their model-based approach enables a substantially less biased estimate of the MTD based on the data as compared with the traditional 3 + 3 designs. Their method is applicable in any trial that was conducted using an algorithm-based design, when the intended target toxicity level was much higher than 20%, potentially salvaging the trial and probably the whole development program by bringing forward to later phase trials a therapeutic dose for confirmatory efficacy studies. 4 EXPLORING ALGORITHM-BASED DESIGNS WITH PRESPECIFIED TARGETED TOXICITY LEVELS 4.1 Rationale for Exploring the Properties of Algorithm-based Designs As mentioned in the previous section, the chief reason for the broad use of algorithmbased designs is due to their simplicity in application, even though many alternative model-based designs have been proposed and advocated in recent years. Correspondingly, the chief complaint about these designs is that the recommended MTD dose observed from the data from an algorithm-based design has no interpretation as an estimate of the dose that yields a prespecified rate of target toxicity. The properties of the traditional algorithm-based 3 + 3 designs have been studied extensively by many (27–29). Simulation studies have shown that the traditional algorithm-based 3 + 3 designs are associated with expected toxicity levels in the range of 19% to 24% or even less, depending on the actual algorithm and number of prescribed doses in a trial. Other algorithmbased designs have not been studied, and it is not certain what design properties, especially the expected toxicity level, an identified MTD

8

ALGORITHM-BASED DESIGNS

may be associated with at the conclusion of an algorithm-based trial. In the following section, we study the expected toxicity levels of a few A + B and 1 + A + B designs with varying design parameters by simulations.

4.2 Simulation Study Design and Results Suppose that k dose levels from the dose range D = {d1 , . . . , dk } are chosen for a study. Let θ i be the probability of DLT that corresponds to the dose level di (i = 1, . . . , k). We assume that θ i satisfies θ i = ψ(di ), where ψ( ) is any monotonic function with the constraint θ 1 ≤ θ 2 ≤ . . . θ k . Denote ri = P(MTD = di ) as the probability that dose level di is declared as the MTD. Based on the exact formulas of Lin and Shih (10) and exact formulas of Liu et al. (25), ri can be derived for traditional A + B designs with and without dose de-escalation and the modified accelerated 1 + A + B designs. Once ri is obtained for each dose level di , we can derive the expected toxicity level (ETL) at the MTD as

and the corresponding standard deviation as  k−1 k−1   θi2 ri ri − (ETL)2 . SD =  i=1

The ETL is different from the target toxicity level (TTL) in that TTL is an acceptable toxicity level at the MTD prespecified by investigators at the onset of a trial, whereas ETL is the expected toxicity level at the MTD achieved after the conclusion of a trial. For dose levels d1 , . . . , dk , instead of calculating the ETL for one set of fixed θ 1 , . . . , θ k , we generated N = 4000 sets of the corresponding probabilities of the DLT as θ 1 , . . . , θ k from the uniform distribution (0,1) with θ 1 ≤ θ 2 ≤ . . . ≤ θ k through simulation. Then for each of the N sets of the θ 1 , . . . , θ k , ETL is obtained by the exact computation using the formulas by Lin and Shih (10) and Liu et al. (25), respectively, for A + B and 1 + A + B designs. For our simulations, k = 10 is used. The results of the simulations are then pooled together to evaluate the parameter estimates with the following summary statistics:

ETL = P(toxicity at MTD | dose 1 ≤ MTD < does k) =

k−1  i=1

θi ri

k−1 

i=1

ˆ = ETL

N 

 ˆ ETLl /N,

l=1

ri , ˆ l represents the expected toxicity where ETL

i=1

Table 1. Estimated Expected Toxicity Levels Based On Algorithm-Based Designs: Commonly Used Algorithm-Based Designs Design Parameters

Designs

A

B

C

D

E

A+B

1+A+B

3 3 3 3 3 3 3 3 3 3 3 3 3 3

3 3 3 3 3 6 6 6 6 6 6 6 6 6

1 1 1 2 2 1 1 1 2 2 2 2 2 2

1 1 2 2 2 1 1 1 2 2 2 2 3 3

1 2 2 2 3 2 3 4 2 3 4 5 4 5

20% 27% 30% 33% 42% 21% 26% 30% 25% 31% 38% 45% 39% 48%

21% 29% 32% 32% 42% 22% 27% 32% 25% 32% 38% 45% 39% 47%

ALGORITHM-BASED DESIGNS

level at the lth (l = 1, . . . , N) simulation. The ˆ l is defined as standard deviation of ETL N ˆ = SD(ETL)

ˆ

ˆ 2 − ETL) . N−1

l=1 (ETLl

Using the same definitions for A, B, C, D, E, C/A, >C/A, and 0 α(γ , t) = α[(1 − e−γ t )/(1 − e−γ )],

for γ # 0

The increment α(tj ) – α(tj −1) represents the additional amount of alpha or Type I error probability that can be used at the jth analysis. In general, to solve for the boundary values Zc (j), we need to obtain the multivariate distribution of Z(1), Z(2), . . . , Z(J). In the cases to be discussed, the distribution is asymptotically multivariate normal with covariance structure  = (σ jk ) where σjk = cov(Z(j), Z(k)) √ √ = (tj /tk ) = (iJ /iK ) for j ≤ k where ij and ik are the amount of information available at the j-th and k-th data monitoring, respectively. Note that at the jth data monitoring, ij and ik are observable and σ jk is known even if I (total information) is unknown. However, if I is not known during interim analysis, we must estimate I by Iˆ and tj by tj = Ij /Iˆ so that we can estimate α(tj ) by (tˆ). If these increments have an independent distributional structure, which is often the case, then derivation of the values of the Zc (j) from the chosen form of α(t) is relatively straightforward using Equation (1) and the methods of Armitage et al. (21,22). If the sequentially computed statistics do not have an independent increment structure, then the derivation of the Zc (j) involves a more complicated numerical integration and sometimes is estimated by simulation. However, as discussed later, for the most frequently used test statistics, the independent increment structure holds.

This formulation of the alpha spending function provides two key flexible features. Neither the timing nor the total number of interim analyses have to be fixed in advance. The critical boundary value at the j-th analysis only depends on the information fraction tj and the previous j-1 information fractions, t1 , t2 , . . . , tj−1 , and the specific spending function being used. However, once an alpha spending function has been chosen before the initiation of the trial, that spending function must be used for the duration of the trial. A DMC can change the frequency of the interim analyses as trends emerge without appreciably affecting the overall α level (23,24). Thus, it is difficult to abuse the flexibility of this approach. The timing and spacing of interim analyses using the alpha spending function approach have been examined (19, 25–27). For most trials, two early analyses with less than 50% of the information fraction are adequate. An early analysis say at 10% is often useful to make sure that all of the operational and monitoring procedures are in order. In rare cases, such early interim reviews can identify unexpected harm such as in the Cardiac Arrhythmia Suppression Trial (28) that terminated early for increased mortality at 10% of the information fraction using an alpha spending function. A second early analysis at 40 or 50% information fraction can also identify strong convincing trends of benefit as in two trials that evaluated beta blocker drugs in chronic heart failure (29,30). Both trials terminated early at approximately 50% of the information fraction with mortality benefits. Computation of the alpha spending function can be facilitated by available software on the web (www.biostat.wisc.edu/landemets) or by commercial software packages (www .cytel.com/Products/East/default.asp). 3 APPLICATION OF THE ALPHA SPENDING FUNCTION Initial development of group sequential boundaries was for comparison of proportions or means (15,16,26). In these cases, the increments in information are represented by additional groups of subjects and their responses to the intervention. For comparing means or proportions, the information

ALPHA-SPENDING FUNCTION

fraction t can be estimated by the n/N, the observed sample size divided by the expected sample size. However, later work expanded the use to other common statistical procedures. Tsiatis and colleagues (31,32) demonstrated that sequential logrank test statistics and the general class of rank statistics used in censored survival data had the independent increment structure that made the application to group sequential boundary straightforward. Later, Kim and Tsiatis (33) demonstrated that the alpha spending function approach for sequential logrank tests was also appropriate. In this case, the information fraction is approximated by d/D, the number of observed events or deaths divided by the expected or design for number of events or deaths (34). Application of the alpha spending function for logrank tests has been used in several clinical trials (e.g., 28–30). Group sequential procedures including the alpha spending function have also been applied to longitudinal studies using a linear random effects model (35,36). Longitudinal studies have also been evaluated using generalized estimating equations (37). In a typical longitudinal clinical trial, subjects are added over time, and more observations are gathered for each subject during the course of the trial. One statistic commonly used is to evaluate the rate of change by essentially computing the slope of the observations for each subject and then taking a weighted average of these slopes over the subjects in each intervention arm. The sequential test statistics for comparison of slopes using the alpha spending function must take into account their distribution. If the information fraction is defined in terms of the Fisher information (i.e., inverse of the variance for the slopes), then the increments in the test statistic are independent, and the alpha function can be applied directly (38). The total expected information may not be known exactly, but it often can be estimated. Wu and Lan (36) provide other approaches to estimate the information fraction in this setting. Scharfstein and Tsiatis (39) demonstrated that any class of test statistics that satisfies specific likelihood function criteria will have this property and thus can be used directly in this group sequential setting.

5

4 CONFIDENCE INTERVALS AND ESTIMATION Confidence intervals for an unknown parameter θ following early stopping can be computed by using the same ordering of the sample space described by Tsiatis et al. (32) and by using a process developed by Kim and DeMets (25,40) for the alpha spending function procedures. The method can be briefly summarized as follows: A 1–γ lower confidence limit is the smallest value of θ for which an event at least as extreme as the one observed has a probability of at least γ . A similar statement can be made for the upper limit. For example, if the first time the Z-value exists the boundary at tj with the observed Z*(j) ≥ Zc (j), then the upper θ U and lower θ L confidence limits are θ U = sup {θ : Pθ {Z(1) ≥ Zc (1), or · · · , or Z(j − 1) > Zc (j − 1), or Z(j) ≥ Z∗ (j)} ≤ 1 − γ }} and θ L = inf {θ : Pθ {Z(1) ≥ Zc (1), or · · · , or Z(j − 1) ≥ Zc (j − 1), or Z(j) ≥ Z1 (j)} ≥ γ }} Confidence intervals obtained by this process will have coverage closer to 1 – γ than na¨ıve ˆ confidence intervals using θˆ ±Zγ /2 SE(θ). As an alternative to computing confidence intervals after early termination, Jennison and Turnbull (41) have allocated the calculation of repeated confidence intervals. This calculation is achieved by inverting a sequential test to obtain the appropriate coefficient Z∗α/2 in the general form for the confidence ˆ This inversion can be interval, θˆ ± Z∗α/2 SE(θ). achieved when the sequential test is based on an alpha spending function. If we compute the interim analyses at the tj , obtaining corresponding critical values Zc (j), then the repeated confidence intervals are of the form θˆk ± Zc (j)SE(θˆj ) where θˆj is the estimate for the parameter θ at the j-th analysis. Methodology has also been developed to obtain adjusted estimates for the intervention effect (42–47). Clinical trials that terminate early are prone to exaggerate the

6

ALPHA-SPENDING FUNCTION

magnitude of the intervention effect. These methods shrink the observed estimate closer to the null. The size of the adjustments may depend on the specific sequential boundary employed. Conservative boundaries such as that proposed by Peto or O’Brien and Fleming generally require less adjustment, and the na¨ıve point estimate and confidence intervals may be quite adequate. Another issue is the relevance of the estimate to clinical practice. The population sample studied is usually not a representative sample of current and future practice. Subjects were those who passed all of the inclusion and exclusion criteria and volunteered to participate. Early subjects may differ with later subjects as experience is gained with the intervention. Thus, the intervention effect estimate may represent populations like the one studied, the only solid inference, but not as relevant to how the intervention will be used. Thus, complex adjustments may not be as useful. 5

TRIAL DESIGN

If any trial is planning to have interim analyses for monitoring for benefit or harm, then that plan must be taken into account in the design. The reason is that group sequential methods will impact the final critical value, and thus power, depending on which boundary is used. For the alpha spending function approach, the specific alpha function must be chosen in advance. In addition, for planning purposes, the anticipated number of interim analyses must be estimated. This number does not have to be adhered to in the application, but it is necessary for the design. Variance with this number in the application will not practically affect the power of the trial. Thus, the design strategy for the alpha spending function is similar to that strategy described by Pocock for the initial group sequential methods (15). The key factor when the sample size is computed is to take into consideration the critical value at the last analysis when the information fraction is 1.0. One simple approach is to use this new critical value in the standard sample size formula. This estimate will reasonably approximate a more exact approach described below.

To illustrate, consider a trial that is comparing failure rates of successive groups of subjects. Here, H0 :pC − pT = 0 HA :pC − pT = δ # 0 where pC and pT denote the unknown response rates in the control and newtreatment groups, respectively. We would estimate the unknown parameter by pˆ C and pˆ T , the observed event rates in our trial. For a reasonably large sample size, we often use the following test statistics, Z=


pˆ C − pˆ T p(1 ˆ − p)(1/m ˆ C + 1/mT

to compare event rates where pˆ is the combined event rate across treatment groups. For sufficiently large n where n = mC = mT , this statistics has an approximate standard normal distribution with mean and unit variance under the null hypothesis H0 : = 0. In this case, assuming equal sample size (n) per group in each arm,
√ = n(pC − pT / 2p(1 − p)
√ = nδ/ 2p(1 − p) where p = (pC + pT )/2 It follows that n=

2 2 p(1 − p) δ2

To design our studies, we evaluate the previous equation for n, the sample size per treatment per sequential group. Because the plan is to have J groups each of size 2n, the total sample size 2N equals 2nJ. Now, to obtain the sample size in the context of the alpha spending function, we proceed as follows: 1. For planning purposes, estimate the number of planned interim analyses J at equally spaced increments of information (i.e., 2n subjects). It is also possible to specify unequal increments, but equal space is sufficient for design purposes.

ALPHA-SPENDING FUNCTION

2. Obtain the boundary values of the K interim analyses under the null hypothesis H0 to achieve a prespecified overall alpha level, α, for a specific spending function α(t). 3. For the boundary obtained, obtain the value of to achieve a desired power (1–β). 4. Determine the value of n that determines the total sample size 2N = 2nJ. 5. Having computed these design parameters, one may conduct the trial with interim analysis to be done based on the information fraction tj approximated by

6

7

CONCLUSIONS

The alpha spending function approach for group sequential interim analysis has provided the necessary flexibility that allows data monitoring committees to fulfill their task. DMCs can adjust their analysis as data accumulates and trends emerge. As long as the alpha spending function is specified in advance, there is little room for abuse. Many trials sponsored by industry and government have successfully used this approach. Although the decision to terminate any trial early, for benefit or harm, is a very complex decision process, the alpha spending function can be an important factor in that process.

tj = Number of subjects observed/2N at the jth analysis (38). The number of actual interim analyses may not be equal to J, but the alpha level and the power will be affected only slightly (26). As a specific example, consider using an O’Brien–Fleming-type alpha spending function α 1 (t) with a one-sided 0.025 alpha level and 0.90 power at equally spaced increments at t = 0.2, 0.4, 0.6, 0.8 and 1.0. Using previous publications (16) or available computer software, we obtain boundary values 4.56, 3.23, 2.63, 2.28, and 2.04. Using these boundary values and available software, we find that = 1.28 provides the desired power of 0.90. If we specify pC = 0.6, pT = 0.4 (p = 0.5) under the alternative hypothesis, then we can obtain a sample size as follows. For = 1.28, n=

2(1.28)2 (0.5)(0.5) = 20.5, (0.2)2

and we have a total sample size of 2(21)5 = 210 subjects. We can then proceed to conduct interim analysis times at information fraction tj equal to the observed number of subjects divided by 210. Similar formulations can be developed for the comparison of means, repeated measures, and survival analysis (48). However, for most applications, the standard sample size formulas with the new alpha spending function final critical value will be a very good approximation.

REFERENCES 1. L. Friedman, C. Furberg, and D. DeMets, Fundamentals of Clinical Trials. Littleton, MA: John Wright – PSG Inc., 1981. 2. S. J. Pocock, Clinical Trials: A Practical Approach, New York: Wiley, 1983. 3. Heart Special Project Committee. Organization, review and administration of cooperative studies (Greenberg Report): a report from the Heart Special Project Committee to the National Advisory Council, May 1967. Control. Clin. Trials 1988; 9: 137–48. 4. P. L. Canner, Monitoring treatment differences in long–term clinical trials. Biometrics 1977; 33: 603–615. 5. Coronary Drug Project Research Group. Practical aspects of decision making in clinical trials: The Coronary Drug Project as a case study. Control. Clin. Trials 1982; 9: 137–148. 6. D. L. DeMets, L. Friedman, C. D. Furberg, Data Monitoring in Clinical Trials: A Case Studies Approach. 2005. Springer Science + Business Media, New York, NY. 7. S. Ellenberg, T. Fleming and D. DeMets, Data Monitoring Committees in Clinical Trials: A Practical Perspective. West Sussex, UK: John Wiley & Sons, Ltd., 2002. 8. ICH Expert Working Group: International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH Harmonised Tripartite Guideline. Statistical principles for clinical trials. Stats. Med. 1999; 18: 1905–1942. 9. U.S. Department of Health and Human Services.Food and Drug Administration.

8

ALPHA-SPENDING FUNCTION Docket No. 01D–0489. Guidance for Clinical Trial Sponsors on the Establishment and Operations of Clinical Trial Data Monitoring Committees. Federal Register 66: 58151–58153, 2001. Available: http://www.fda.gov/OHRMS/DOCKETS/98fr/ 112001b.pdf.

10. F. J. Anscombe, Sequential medical trials. Journal of the American Statistical Association 1963; 58: 365–383.

24.

25.

26.

11. I. Bross, Sequential medical plans. Biometrics 1952; 8: 188–205. 12. P. Armitage, Sequential Medical Trials, 2nd ed. New York: John Wiley and Sons, 1975. 13. P. Armitage, C. K. McPherson, and B. C. Rowe, Repeated significance tests on accumulating data. J. Royal Stat. Soc. Series A 1969; 132: 235–244. 14. R. Peto, M. C. Pike, P. Armitage, et al., Design and analysis of randomized clinical trials requiring prolonged observations of each patient. 1. Introduction and design. Br. J. Cancer 1976; 34: 585–612.

27.

28.

29.

15. S. J. Pocock: Group sequential methods in the design and analysis of clinical trials. Biometrika 1977; 64: 191–199. 16. P. C. O’Brien and T. R. Fleming, A multiple testing procedure for clinical trials. Biometrics 1979; 35: 549–556.

30.

17. Beta–Blocker Heart Attack Trial Research Group. A randomized trial of propranolol in patients with acute myocardial infarction. I Mortality results. J. Amer. Med. Assoc. 1982; 247: 1707–1714. 18. K. K. G. Lan and D. L. DeMets, Discrete sequential boundaries for clinical trials. Biometrika 1983; 70: 659–663. 19. K. Kim and D. L. DeMets. Design and analysis of group sequential tests based on the type I error spending rate function. Biometrika 1987; 74: 149–154. 20. I. K. Hwang, W. J. Shih, and J. S. DeCani, Group sequential designs using a family of type I error probability spending function. Stats. Med. 1990; 9: 1439–45. 21. K. K. G. Lan and D. L. DeMets, Group sequential procedures: Calendar versus information time. Stats. Med. 1989; 8: 1191–1198. 22. D. M. Reboussin, D. L. DeMets, K. M. Kim, and K. K.G. Lan, Computations for group sequential boundaries using the Lan–DeMets spending function method. Control. Clin. Trials 2000; 21: 190–207. 23. M. A. Proschan, D. A. Follman, and M. A. Waclawiw. Effects of assumption violations

31.

32.

33.

34.

35.

36.

on type I error rate in group sequential monitoring. Biometrics 1992; 48: 1131–1143. K. K. G. Lan and D. L. DeMets, Changing frequency of interim analyses in sequential monitoring. Biometrics 1989 Sept; 45(3): 1017–1020. K. Kim and D. L. DeMets, Confidence intervals following group sequential tests in clinical trials. Biometrics 1987; 4: 857–864. K. Kim and D. L. DeMets, Sample size determination for group sequential clinical trials with immediate response. Stats. Med. 1992; 11: 1391–1399. Z. Li and N. L. Geller, On the choice of times for date analysis in group sequential trials. Biometrics 1991; 47: 745–750. Cardiac Arrhythmia Suppression Trial (CAST) Investigators. Preliminary report: Effect of endainide and flecainide on mortality in a randomized trial of arrhythmia suppression after myocardial infarction. N. Engl. J. Med. 1989; 321: 406–412. MERIT–HF Study Group. Effect of metoprolol CR/XL in chronic heart failure: Metoprolol CR/XL randomised intervention trial in congestive heart failure. Lancet 1999; 353: 2001–2007. M. Packer, A. J. S. Coats, M. B. Fowler, H. A. Katus, H. Krum, P. Mohacsi, J. L. Rouleau, M. Tendera, A. Castaigne, C. Staiger, et al. for the Carvedilol Prospective Randomized Cumulative Survival (COPERNICUS) Study Group. Effect of Carvedilol on survival in severe chronic heart failure. New Engl. J. Med. 2001; 334: 1651–1658. A. A. Tsiatis, Repeated significance testing for a general class of statistics used in censored survival analysis. J. Am. Stat. Assoc. 1982; 77: 855–861. A. A. Tsiatis, G. L. Rosner, and C. R. Mehta, Exact confidence intervals following a group sequential test. Biometrics 1984; 40: 797–803. K. Kim and A. A. Tsiatis, Study duration for clinical trials with survival response and early stopping rule. Biometrics 1990; 46: 81–92. K. K. G. Lan and J. Lachin, Implementation of group sequential logrank tests in a maximum duration trial. Biometrics 1990; 46: 759–770. J. W. Lee and D. L. DeMets, Sequential comparison of changes with repeated measurement data. J. Am. Stat. Assoc. 1991; 86: 757–762. M. C. Wu and K. K. G. Lan, Sequential monitoring for comparison of changes in a response variable in clinical trials. Biometrics 1992; 48: 765–779.

ALPHA-SPENDING FUNCTION 37. S. J. Gange and D. L. DeMets, Sequential monitoring of clinical trials with correlated categorical responses. Biometrika 1996; 83: 157–167. 38. K. K. G. Lan, D. M. Reboussin, D. L. DeMets: Information and information fractions for design and sequential monitoring of clinical trials. Communicat. Stat.– Theory Methods 1994; 23: 403–420. 39. D. O. Scharfstein, A. A. Tsiatis, and J. M. Robins, Semiparametric efficiency and its implication on the design and analysis of group–sequential studies. J. Am. Stat. Assoc. 1997; 92: 1342–1350. 40. K. Kim, Point estimation following group sequential tests. Biometrics 1989; 45: 613–617. 41. C. Jennison and B. W. Turnbull: Interim analyses: The repeated confidence interval approach. J. Royal Stat. Soc., Series B 1989; 51: 305–361. 42. S. S. Emerson and T. R. Fleming, Parameter estimation following group sequential hypothesis testing. Biometrika 1990; 77: 875–892. 43. M. D. Hughes and S. J. Pocock, Stopping rules and estimation problems in clinical trials. Stats. Med. 1981; 7: 1231–1241. 44. Z. Li and D. L. DeMets, On the bias of estimation of a Brownian motion drift following group sequential tests. Stat. Sinica 1999; 9: 923–937. 45. J. C. Pinheiro and D. L. DeMets: Estimating and reducing bias in group sequential designs with Gaussian independent structure. Biometrika 1997; 84: 831–843. 46. D. Siegmund, Estimation following sequential tests. Biometrika 1978; 65: 341–349. 47. J. Whitehead, On the bias of maximum likelihood estimation following a sequential test. Biometrika 1986; 73: 573–581. 48. D. L. DeMets and K. K. G. Lan, The alpha spending function approach to interim data analyses. In: P. Thall (ed.), Recent Advances in Clinical Trial Design and Analysis. Dordrecht, The Netherlands: Kluver Academic Publishers, 1995, pp. 1–27.

FURTHER READING M. N. Chang and P. C. O’Brien, Confidence intervals following group sequential tests. Control. Clin. Trials 1986; 7: 18–26.

9

T. Cook and D. L. DeMets, Statistical Methods in Clinical Trials. Boca Raton, FL: CRC Press/Taylor & Francis Co., 2007. D. L. DeMets, Data monitoring and sequential analysis – An academic perspective. J. Acq. Immune Def. Syn. 1990; 3(Suppl 2):S124–S133. D. L. DeMets, R. Hardy, L. M. Friedman, and K. K. G. Lan, Statistical aspects of early termination in the Beta–Blocker Heart Attack Trial. Control. Clin. Trials 1984; 5: 362–372. D. L. DeMets and K. K. G. Lan, Interim analysis: the alpha spending function approach. Stats. Med. 1994; 13: 1341–1352. D. L. DeMets, Stopping guidelines vs. stopping rules: A practitioner’s point of view. Communicat. Stats.–Theory Methods 1984; 13: 2395–2417. T. R. Fleming and D. L. DeMets, Monitoring of clinical trials: issues and recommendations. Control. Clin. Trials 1993; 14: 183–197. J. L. Haybittle, Repeated assessment of results in clinical trials of cancer treatment. Brit. J. Radiol. 1971; 44: 793–797. K. K. G. Lan, W. F. Rosenberger, and J. M. Lachin: sequential monitoring of survival data with the Wilcoxon statistic. Biometrics 1995; 51: 1175–1183. J. W. Lee, Group sequential testing in clinical trials with multivariate observations: a review. Stats. Med. 1994; 13: 101–111. C. L. Meinert, Clinical Trials: Design, Conduct, and Analysis. New York: Oxford University Press, 1986. S. Piantadosi. Clinical Trials: A Methodologic Perspective, 2nd ed. Hoboken, NJ: John Wiley and Sons, Inc., 2005. S. J. Pocock. Statistical and ethical issues in monitoring clinical trials. Stats. Med. 1993; 12: 1459–1469. S. J. Pocock. When to stop a clinical trial. Br. Med. J. 1992; 305: 235–240. E. Slud and L. J. Wei, Two–sample repeated significance tests based on the modified Wilcoxon statistic. J. Am. Stat. Assoc. 1982; 77: 862–868. A. Wald, Sequential Analysis. New York: John Wiley and Sons, 1947.

ANALYSIS OF VARIANCE ANOVA

The basis for generalizability of a successful clinical trial is strengthened when the coverage of a study is as broad as possible with respect to geographical area, patient demographics, and pre-treatment characteristics as well as other factors that are potentially associated with the response variables. At the same time, heterogeneity among patients becomes more extensive and conflicts with the precision of statistical estimates, which is usually enhanced by homogeneity of subjects. The methodology of the ANOVA is a means to structure the data and their validation by accounting for the sources of variability such that homogeneity is regained in subsets of subjects and heterogeneity is attributed to the relevant factors. The method ANOVA is based on the use of sums of squares of the deviation of the observations from respective means (→ Linear Model). The tradition of arraying sums of squares and resulting F-statistics in an ANOVA table is so firmly entrenched in the analysis of balanced data that the extension of the analysis to unbalanced data is necessary. For unbalanced data, many different sums of squares can be defined and then be used in the numerators of F-statistics providing it tests for a wide variety of hypotheses. In order to provide a practically relevant and useful approach, the ANOVA through the cell means model will be introduced. The concept of the cell means model was introduced by Searle (2, 3), Hocking and Speed (4), and Hocking (5) to resolve some of the confusion associated with ANOVA models with unbalanced data. The simplicity of such a model is readily apparent: No confusion exists on which functions are estimable, what their estimators are, and what hypotheses can be tested. The cell means model is conceptually easier, it is useful for understanding the ANOVA models, and it is, from the sampling point of view, the appropriate model to use. In many applications, the statistical analysis is characterized by the fact that a number of detailed questions need to be answered. Even if an overall test is significant, further

¨ Dr. JORG KAUFMANN

AG Schering SBU Diagnostics & Radiopharmaceuticals Berlin, Germany

1

INTRODUCTION

The development of analysis of variance (ANOVA) methodology has in turn had an influenced on the types of experimental research being carried out in many fields. ANOVA is one of the most commonly used statistical techniques, with applications across the full spectrum of experiments in agriculture, biology, chemistry, toxicology, pharmaceutical research, clinical development, psychology, social science, and engineering. The procedure involves the separation of total observed variation in the data into individual components attributable to various factors as well as those caused by random or chance fluctuation. It allows performing hypotheses tests of significance to determine which factors influence the outcome of the experiment. However, although hypothesis testing is certainly a very useful feature of the ANOVA, it is by no means the only aspect. The methodology was originally developed by Sir Ronald A. Fisher (1), the pioneer and innovator of the use and applications of statistical methods in experimental design who coined the name ‘‘Analysis of Variance – ANOVA.’’ For most biological phenomena, inherent variability exists within the response processes of treated subjects as well as among the conditions under which treatment is received, which results in sampling variability, meaning that results for a subject included in a study will differ to some extent from those of other subjects in the affected population. Thus, the sources of variability must be investigated and must be suitably taken into account when data from comparative studies have to be evaluated correctly. Clinical studies are in particular a fruitful field for the application of this methodology.

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

ANALYSIS OF VARIANCE ANOVA

analyses are, in general, necessary to assess specific differences in the treatments. The cell means model provides within the ANOVA framework the appropriate model for a correct statistical inference and provides such honest statements on statistical significance in a clinical investigation. 2

FACTORS, LEVELS, EFFECTS, AND CELLS

One of the principal uses of statistical models is to explain variation in measurements. This variation may be caused by the variety of factors of influence, and it manifests itself as variation from one experimental unit to another. In well-controlled clinical studies, the sponsor deliberately changes the levels of experimental factors (e.g., treatment) to induce variation in the measured quantities to lead to a better understanding of the relationship between those experimental factors and the response. Those factors are called independent and the measured quantities are called dependent variables. For example, consider a clinical trial in which three different diagnostic imaging modalities are used on both men and women in different centers. Table 1 shows schematically how the resulting data could be arrayed in a tabular fashion. The three elements used for classifications (center, sex, and treatment) identify the source of variation of each datum and are called factors. The individual classes of the classifications are the levels of the factor (e.g., the three different treatments T1, T2, and T3 are the three levels of the factor treatment). Male and female are the two levels of the

Table 1. Factors, Levels and Cells (ijk)

Treatment (k) Center (i)

Sex (j)

1

male female

2

male female

3

male female

T1

T2

cell (2 1 2)

T3

factor sex, and center1, center2, center3 are the three levels of the factor center. A subset of the data present for a ‘‘combination’’ of one level of each factor under investigation is considered a cell of the data. Thus, with the three factors, center (3 levels), sex (2 levels), and treatment (3 levels), 3 × 2 × 3 = 18 cells numbered by triple indexing i j k exist. Repeated measurements in one cell may exist, which they usually do. Unbalanced data occur when the number of repeated observations per cell nijk are different for at least some of the indices (i, j, k). In clinical research, this occurrence is rather the role than the exception. One obvious reason could be missing data in an experiment. Restricted availability of patients for specific factor combination is another often experienced reason. 3 CELL MEANS MODEL A customary practice since the seminar work of R. A. Fisher has been that of writing a model equation as a vehicle for describing ANOVA procedures. The cell means model is now introduced via a simple example in which only two treatments and no further factors are considered. Suppose that yir , i = 1, 2, r = 1, . . . , ni represents a random sample of two normal populations with means µ1 and µ2 and common variance σ 2 . The data point yir denotes the rth observation on the ith population of size ni and its value assumed to follow a Gaussian normal distribution: yir ∼ N(µi , σ 2 ). The fact that the sizes n1 and n2 of the two populations differ indicates that a situation of unbalanced data exists. In linear model form, it is written yir = µi + eir , i = 1, 2; r = 1, . . . , ni

(1)

where the errors eir are identically independent normal N(0, σ 2 ) distributed (i.i.d. variables). Notice, a model consists of more than just a model equation: It is an equation such as Equation (1) plus statements that describe the terms of the equation. In the example above, µi is defined as the population mean of the ith population and is equal to the expectation of yir E( yir ) = µi , r = 1, . . . , ni , for i = 1, 2

(2)

ANALYSIS OF VARIANCE ANOVA

The difference yir − E( yir ) = yir − µi = eir is the deviation of the observed yir value from the expected value E( yir ). This deviation denoted eir is called the error term or residual error term, and from the introduction of the model above, it is a random variable with expectation zero and variance v(eir ) = σ 2 . Notice, in the model above, the variance is assumed to be the same for all eir s. The cell means model can now be summarized as follows:

yir = µi + eir

(3)

E( yir ) = µi E(eir ) = 0 v(eir ) = σ 2 for all i and r

Notice that Equation (3) does not assume explicitly the Gaussian normal distribution. In fact, one can formulate the cell means model more generally by only specifying means and variances. Below, however, it is restricted to the special assumption eir ≈ i.i.d. N(0, σ 2 ). It will be shown that the cell means model can be used to describe any of the models that are classically known as ANOVA models.

4

ONE–WAY CLASSIFICATION

Let us begin with the case of a cell means model in which the populations are identified by a single factor with i levels and ni observations at the ith level for i = 1, . . . , i. 4.1 Example 1 A clinical study was conducted to compare the effectiveness of three different doses of a new drug and placebo for treating patients with high blood pressure. For the study, 40 patients are included. To control for unknown sources of variation, 10 patients each were assigned at random to the four treatment groups. As response, one considered the difference in diastolic blood pressure measurement between baseline (pre-value) and the measurement four weeks after administration of treatment. The response measurements yir , sample means yi , and sample variances s2i are shown in Table 2. The cell means model to analyze the data of the example 1 is then given as yir = µi + eir i = 1, 2, 3, 4; r = 1, . . . , 10

dose 1

dose 2

dose 3

1.6 0.1 −0.3 1.9 −1.3 0.6 −0.6 1.0 −1.1 1.1

1.7 4.7 0.2 3.5 1.7 2.6 2.3 0.2 0.3 0.7

3.6 2.3 2.1 1.6 1.8 3.3 4.3 1.2 1.5 2.4

3.1 3.5 5.9 3.0 5.2 2.6 3.7 5.3 3.4 3.6

where µi defines the population mean at the ith level as E( yir ) = µi and σ 2 is the common variance of the eir .

yi.

0.30

1.79

2.41

3.93

sample means

s2i

1.24

2.31

1.01

1.26

sample variances

σˆ 2 = 1.46

(4)

eir ≈ i.i.d. N(0, σ 2 )

Table 2. Example 1, Data for a Dose Finding Study

Placebo

3

estimated common error variance of the model

4

5

ANALYSIS OF VARIANCE ANOVA

PARAMETER ESTIMATION

The µi s in Equation (4) are usually estimated by the methods of least squares. The estimator for µi (i = 1, . . . , i) is then given by µˆ i =

1
yi =: yi. nir r

(5)

2 σˆ =

1

( yir − µˆ i )2 n.- I r

(6)

i

where n. =




ni

i

The sample means and sample variances for each treatment and the estimated common error variance σ 2 are given in Table 2.

SST =

6 THE R(.) NOTATION – PARTITIONING SUM OF SQUARES


( yi − y.)2

(7)

with 1
yi ni

y2i r

(10)

r

The error sum of squares after fitting the model E( yir ) = µi is SSE(µi ) =



i

i

y. =



i

The ANOVA procedure can be summarized as follows: Given n observations yi s, one defines the total sum of squared deviations from the mean by Total SS =

error sum of squares, respectively. Thus, Total SS = Model SS + Error SS. The Total SS always has the same value for a given set of data because it is nothing other than the sums of squares of all data points relative to the common mean. However, the partitioning into Model SS and Error SS depends on model selection. Generally, the addition of a new factor to a model will increase the Model SS and, correspondingly, reduce the Error SS. When two models are considered, each sum of squares can be expressed as the difference between the sums of squares of the two models. Therefore, the approach related to given sum of squares allows the comparison of two ANOVA models very easily. In the one-way classification in Equation (4), the total sum of squares of each observation is

( yir − µˆ i )2

(11)

r

R(µi ) = SST − SSE(µi ) is denoted the reduction in sum of squares because of fitting the model E( yir ) = µi

i

The ANOVA technique partitions the variation among observations into two parts: the sum of squared deviations from the model to the overall mean
( yˆ i − y.)2 (8) Model SS = i

and the sum of squared deviations from the observed values yi values to the model Error SS = SSE =




( yi − yˆ i )2

(9)

i

These two parts are called the sum of squares because of the model and the residual or

Fitting the simplest of all linear models, the constant mean model E( yir ) = µ, the estimate of E( yir ) would be yˆ ir = µˆ = y and the error sum of squares results as SSE(µ) =



i

( yir − µ) ˆ 2

r

R(µ) = SST − SSE(µ) is denoted the reduction in sum of squares because of fitting the model E( yir ) = µ. The two models E( yir ) = µi and E( yir ) = µ can now be compared in terms of their respective reductions in sum of

ANALYSIS OF VARIANCE ANOVA

5

Table 3a. ANOVA—Partioning the Total Sum of Squares 1-way Classification, Cell Means Model

Source of variation

df

Sum of square

Mean square

Model µ

1

R(µ)

R(µ)

Model µi

I−1

R(µi /µ)

R(µi /µ) I−1

Residual

n. − I

SSE = SST − R(µi )

SST − R(µi ) n. − I

Total

n.

F statistic

R(µi /µ) (I − 1)



SST − R(µi ) n. − I

SST

Table 3b. Partitioning the Total Sum Sqares Adjusted for Mean

Source of variation

df

Sum of square

Mean square

Model µi

I−1

R(µi /µ)

R(µi /µ) I−1

Residual

n. − I

SSE = SST − R(µi )

SST − R(µi ) n. − I

Total a.f.m

n. − 1

SST − R(µ)

squares given by R(µi ) and R(µ). The difference (R(µi ) − R(µ) is the extent to which fitting E( yir ) = µi brings about a greater reduction in sum of squares than does fitting E( yir ) = µ. Obviously, the R(.) notation is a useful mnemonic for comparing different linear models in terms of the extent to which fitting each accounts for a different reduction in the sum of squares. The works of Searle (2, 3), Hocking (5), and Littel et al. (6) are recommended for deeper insight. It is now very easy to partition the total sum of squares SST into terms that develop in the ANOVA. Therefore, the identity SST = R(µ) + (R(µi ) − R(µ)) + (SST − R(µi )) (12) = R(µ) + R(µi /µ) + SSE(µi ) or SST − R(µ) = R(µi /µ) + SSE(µi ) are used with R(µi /µ) = R(µi ) − R(µ). The separation of the Table 3a and Table 3b is

F statistic  R(µi /µ) SST − R(µi ) (I − 1) n. − I

appropriate to Equation (12) the first and last line. Table 3a displays the separation into the components atributable to the model µ in the first line, to the model µi extent µ in the second line, and, in the third line to the error term, the last line to the total sum of squares. Table 3b displays only the separation into the two components attributable to the model µi extent µ and in the second line to the error term. 7

ANOVA–HYPOTHESIS OF EQUAL MEANS

Consider the following inferences about the cell means. This analysis includes the initial null hypothesis of equal means (global hypotheses, all means simultaneous by the same) so-called ANOVA hypothesis contingent with pairwise comparisons, contrasts, and other linear function comprising either hypothesis tests or confidence intervals. In starting off with the model E( yij ) = µI , the global null hypothesis H0 : µ1 = µ2 = . . . = µI

6

ANALYSIS OF VARIANCE ANOVA

Table 3c. ANOVA Example 1

Source of variation

df

Sum of square

Mean square

F statistic

Pr > F

Model µi Residual Total a.f.m.

3 36 39

67.81 52.52 120.33

22.60 1.46

15.5

Fα ) < 0.0001. As this probability is less then the type I error α = 0.05, the hypothesis Ho can be rejected in favor of the alternative µi = µj for at least one pair i and j of the four treatments. Rejection of the null hypothesis H0 indicates that differences exist among treat ments, but it does not show where the differences are located. Investigators’ interest is rarely restricted to this overall test, but rather to comparisons among the doses of the new drug or placebo. As a consequence, multiple comparisons comparing the three classes with placebo are required. 8

MULTIPLE COMPARISONS

In many clinical trials, often more than two drugs or more than two levels of one drug are considered. Having rejected the global

hypothesis of equal treatment means (e.g., when the probability of the F-statistic in Table 3c, last column, is less then 0.05), questions related to picking out drugs that are different from others or determining what dose level is different from the others and placebo have to be attacked. These analyses generally constitute making many (multiple) further comparisons among the treatments in order to detect effects of prime interest to the researcher. The excessive use of multiple significance tests in clinical trials can greatly increase the chance of false-positive findings. A large amount of statistical research has been devoted to multiple comparison procedures and the control of false-positive results caused by multiple testing. Each procedure usually has the objective of controlling the experiment-wise or family-wise error rate. A multiple test controls the experiment-wise or family-wise multiple level α, if the probability to reject at least one of the true null hypotheses does not exceed α, irrespective of how many hypotheses and which of them is in fact true. For ANOVA, several types of multiple comparison procedures exist that adjust the critical value of the test statistic. For example, the Scheffe -, Tukey-, and Dunnetttest, and procedures that adjust the compari son-wise P-values (e.g., the Bonferroni-Holm procedure), and the more general closed-test procedures exist (7, 8). Marcus et al. (9) introduced those so-called closed multiple test procedures, which keeps the family-wise multiple level α under control. The closed-test principle requires a special structure among the set of null hypotheses, and it can be viewed as a general tool for deriving a multiple test. In general, the use of an appropriate multiple-comparison test for inference concerning treatment comparisons is indicated (10):

ANALYSIS OF VARIANCE ANOVA

1. to make an inference concerning a particular comparison that has been selected on the basis of how the data have turned out; 2. to make an inference that requires the simultaneous examination of several treatment comparisons, for example, the minimum effected dose in dose findings studies; and 3. to perform so-called ‘‘data dredging,’’ namely, assembling the data in various ways in the hope that some interesting differences will be observable.

9

At the same time, the more extensive heterogeneity among patients conflicts with the precision of statistical estimates, which is usually enhanced by requiring homogeneity of subjects (12). Two study design strategies are followed: 1. Stratified assignment of treatments to subjects who are matched for similarity on one or more block factors, such as gender, race, age, initial severity, or on strata, such as region or centers. Separately and independently within each stratum, subjects are randomly assigned to treatment groups in a block randomization design (11). When a stratified assignment is used, treatment comparisons are usually based on appropriate within stratum/center differences (13). 2. A popular alternative to the block randomization design is the stratification of patients according to their levels on prognostic factors after complete randomization in the analysis phase. This alternative strategy, termed poststratification, leads exactly to the same kind of statistical analysis, but it has disadvantages compared with prestratification. Pre-stratification guards by design against unlikely but devastating differences between the groups in their distributions on the prognostic factors and in sample sizes within stratum. With pre-stratification, these will be equal—or least close to equal—by design. With post-stratification due to randomization, they will be equal only in a long-term average sense (14).

TWO–WAY CROSSED CLASSIFICATION

Two basic aspects of the design of experiments are the error control and the optimal structuring of the treatment groups. A general way of reducing the effect of uncontrolled variations on the error of treatment comparisons is grouping the experimental units (patients) into sets of units being alike (uniform) as much as possible. All comparisons are then made within and between sets of similar units (11). Randomization schemes (e.g., as complete randomization, randomized blocks) are part of the error control of the experimental design. On the other hand, the structure of treatments, what factor(s) and factor levels are to be observed is called the treatment design (6). The factorial treatment design is one of the most important and widely used treatment structures. Thereby one distinguishes between treatment and classification factors. The factorial design can be used with any randomization scheme. Factorial experiments can be compared with the one factor at a time approach, and they have the advantage of giving greater precision in the estimating of overall factor effects. They enable the exploration of interactions between different factors, which are being and allow an extension of the range of validity of the conclusions by the insertion of additional factors. Factorial designs enhance the basis for any generalizability of the trial conclusion with respect to geographical area, patient demographics, and pre-treatment characteristics as well as other factors that are potentially associated with the response variable.

7

10

BALANCED AND UNBALANCED DATA

When the number of observations in each cell is the same, the data shall be described as balanced data. They typically come from well-designed factorial trials that have been executed as planned. The analysis of balanced data is relatively easy and have been extensively described (15). When the number of observations in the cells is not uniform, the data shall be described as unbalanced data.

8

ANALYSIS OF VARIANCE ANOVA

Table 4. Liver Encym (Log Transform)—2-way Crossed Classification Treatment (T), Liver Impairment (B) Case T B X Case T B X

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

2 2 1 2 1 1 1 1 1 2 1 2 1 2 2 1 2 1 2 1

1 1 2 1 1 2 1 1 1 1 1 2 1 1 2 1 1 1 1 1

0.85 1 1.32 0.95 0.90 1.43 1.30 1.08 1 1.08 1.15 1.60 1.34 0.95 1.36 0.95 0.90 1.28 0.95 0.85

T = 1 contrast media 1 T = 2 contrast media 2

21 22 23 24 25 26 27 28 29 30 31 33 34 35 36 37 38 39 40

1 2 2 2 2 1 2 2 1 2 1 2 1 1 2 2 1 2 1

1 1 1 1 1 1 1 1 1 1 1 1 2 1 1 1 2 1 1

1.11 1.28 0.90 1 1 0.85 1 0.90 0.90 1 1 1.04 1.49 0.85 1.08 0.90 1.64 1.26 1.08

B = 1 liver impairment no B = 2 liver impairment yes

Case

T

B

X

Case

T

B

X

101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120

2 1 1 1 2 1 1 1 1 2 2 2 2 2 1 2 1 1 1 2

1 1 2 1 1 2 1 1 2 2 2 1 2 2 1 1 2 1 1 1

1.15 1.11 1.15 0.95 0.90 1.52 1.36 1.18 1.15 1.46 1.18 1.04 1.34 1 0.95 0.70 1.20 0.85 1.30 1.15

121 122 123 124 125 127 128 129 130 131 132 133 134 135 136 137 138 139 140

2 1 1 1 1 2 2 2 1 2 2 2 1 2 1 1 1 2 2

1 1 2 2 2 1 1 1 1 1 1 2 1 1 1 1 2 1 1

1 0.95 1.04 1.43 1.53 1.04 1 1 1.26 1 0.90 1.15 0.95 1.11 1.32 0.95 1.40 0.90 1.04

ANALYSIS OF VARIANCE ANOVA

This fact will be illustrated using a numerical example. Suppose interest exists in the comparison of two contrast media, T = 1, 2, regarding a liver enzym and a second factor B, with categories B = 1, 2, describing the initial severity of the liver impairment (B = 1 not impaired, B = 2 impaired). Table 4 shows the data of a clinical trial designed as a result. Let xijk be the kth observation on treatment i for impaired liver at severity j, where i = 1, 2, j = 1, 2, and k = 1, . . . , nij , with nij being the number of observations on treatment i and impairment j. For the sake of generality, treatment is referred to as row–factor A with levels i = 1, . . . , i and liver impairment as column-factor B with levels j = 1, . . . , j. Notice the data of Table 4 are represented as logarithms yijk = log(xijk ) of the liver enzyme data xijk . The cell means model is as follows: yijk = µij + eijk , i = 1, . . . , I j = 1, . . . , J, k = 1, . . . , nij

(13)

A normal (Gaussian) distribution is assumed for the log-transformed measurements

An important difference between balanced and unbalanced data is the definition of a mean over rows or columns, respectively. For example, the row mean of the cell means µij in row i is straightforward µi . = 1/J

eijk ≈ i.i.d. N(0, σ 2 )

It follows as for the one-way classification that the cell means µij are estimated by the mean of the observation in the appropriate cells (i, j) yijk =: yij.

(14)

k

and the residual error variance is estimated as σˆ 2 = SSE/(N − s)

(15)

where




SSE =

i

j

( yijk − µˆ ij )2

k

and where  s = I × J − (number of empty nij is the total sample size. cells). N = i

j

µij

(16)

µˆ ij

(17)

j

and its estimator is ˆ i . = 1/J µ


j

with variance estimate ˆ i .) = 1/J 2 1/nij σˆ 2 v(µ ˆ 1. = 1.22 and µ ˆ 2. = 1.15 is For example 2, µ obtained. A different row mean of the cell means µij in row i, sometimes thought to be an interesting alternative to µi ., is the weighted mean with the weights nij /ni. : µi . =


nij j

ˆ i . = µ







ni.

µij

(18)

estimated by the mean over all observation in row i,

yijk = log xijk

µˆ ij = 1/nij

9


nij ni.

µˆ ij = yi.

(19)

with variance estimate 

ˆ i ) = σˆ 2 /ni. v(µ ˆ 1. = 1.16 and µ ˆ 2. = 1.05 is For example 2, µ obtained. The difference between these two cell mean estimates µi. and µi. consists in the fact that µi. is independent from the number of the data points in the cells, whereas µi. depends on the number of observations in each cell. As the sample sizes of participating centers in a multi-center study are mostly different, a weighted mean over the centers is a better overall estimator for treatment comparisons (13, 14, 16). If it can be ascertained that the population in a stratum or block is representative of a portion of the general population of interest, for example, in post-stratification, then it is natural to use

10

ANALYSIS OF VARIANCE ANOVA

a weighted mean for an overall estimator of a treatment effect. Many clinical trials result in unbalanced two-way factorial designs. Therefore, it is appropriate to define row means or column respectively as weighted means resulting in different sum of squares. If weights other than these two possibilities are wanted (e.g., Cochran-MantelHaenszel weights), one may consider a general weighting of the form
µi . = tij µij (20) j

with  j



ˆ i . = tij = 1 and the estimator µ

j



ˆ i ) = σˆ 2 tij yij. with v(µ

 j

t2ij /ni .

11 INTERACTION BETWEEN ROWS AND COLUMNS Consider an experiment with two factors A (with two levels) and B (with three levels). The levels of A may be thought of as two different hormone treatments and the three levels of B as three different races. Suppose that no uncontrolled variation exists and that observations as of Fig 1a are obtained. These observations can be characterized in various equivalent ways: 1. the difference between the observations corresponding to the two levels of A is the same for all three levels of B; 2. the difference between the observations for any two levels of B is the same for the two levels of A; and mij

3. the effects of the two factors are additive. For levels i and i of factor A and levels j and j of factor B, consider the relation among means given by µij − µij = µi j − µi j

(21)

If this case holds, the difference in the means for level i and i of factor A is the same for levels j and j of factor B and, vice versa, the change from level j to level j of factor B is the same for both levels of factor A. If Equation (21) holds for all pairs of levels of factor A and all pairs of levels of factor B, or when the conditions a), b), or c) are satisfied, one can say factor A does not interact with factor B: No interaction exists between factor A and B. If the relation in Equation (21) or the conditions a), b), or c) are not satisfied, then an interaction exists between factor A and factor B. Many ways exist in which interaction can occur. A particular method is shown in Fig 1b. The presence of an interaction can mislead conclusions about the treatment effects in terms of row effects. It is therefore advisable to assess the presence of interaction before making conclusions, which can be done by testing an interaction hypothesis. The test may indicate that the data are consistent with the absence of interaction but may not prove that no real interaction exists.

mij A1

16

16

14

A2

14

12

12

10

10

8

8

6

6 1

2

3

Factor B

Figure 1a. Two-way classification, no interaction 2 rows, 3 columns.

A2

A1 1

2

3

Factor B

Figure 1b. Two-way classification, interaction 2 rows, 3 columns.

ANALYSIS OF VARIANCE ANOVA

Table 5. Sample Size, Means

T1 T2

12

B1

B2

(27) 1.07 (32) 1.00

(12) 1.36 (7) 1.30

ANALYSIS OF VARIANCE TABLE

ANOVA tables have been successful for balanced data. They are generally available, widely documented, and ubiquitously accepted (15). For unbalanced data, often no unique, unambiguous method for presenting the results exists. Several methods are available, but often not as easily interpretable as methods for balanced data. In the context of hypothesis testing, or of arraying sums of squares in an ANOVA format, a variety of sums of squares are often used. The problem in interpreting the output of computerspecific programs is to identify those sums of squares that are useful and those that are misleading. The information for conducting tests for row effects, column effects, and interactions between rows and columns is summarized in an extended ANOVA table.

Various computational methods exist for generating sums of squares in an ANOVA table (Table 5) since the work of Yates (17). The advantage of using the µij -model notation introduced above is that all the µij are clearly defined. Thus, a hypothesis stated in terms of the µij is easily understood. Speed et al. (18), Searle (2, 3), and Pendleton et al. (16) gave the interpretations of four different types of sums of squares computed (e.g., by the SAS, SPSS, and other systems). To illustrate the essential points, use the model in Equation (13), assuming all nij > 0. For reference, six hypotheses of weighted means are listed in Table 6 that will be related to the different methods (16, 18). A typical method might refer to H1 , H 2 , or H 3 as ‘‘main effect of A,’’ row effect. The hypotheses H 4 and H 5 are counterparts of H 2 and H 3 generally associated with ‘‘main effect B,’’ column effect. The hypothesis of no interaction is H 6 , and it is seen to be common to all methods under the assumption nij < 0. The hypotheses H 1 , H 2 , and H 3 agree in the balanced case (e.g., if nij = n for all i and j but not otherwise). The hypothesis H3 does not depend on the nij . All means have the same weights 1/j and are easy to interpret. As it states, no difference exists in the levels of the factor A when averaged over all levels of factor B [Equation (16)].

Table 6. Cell Means Hypotheses

Hypothesis

Main Effect

Weighted Means   j nij µij /ni. = j nij µij /ni.    j nij µij = i j nij nij µij /n.j

H1 ⎫ ⎬ H2 ⎭ H3 H  4

Factor B

µi = µi    i nij µij = j i nij nij µij /ni.

H5

columns

µ.j = µ.j

H6

Interaction A × B

µij − µij − µij + µij = 0

Factor A rows

11

For all i, i, j, j, i?i, j?j H1 weighted mean equation (18) H2 Chochran Mantel-Haenszel weights equation (20) for t = nij nij /n.j H3 weighted mean equation (16) H4 counterpart of H2 with factor B H5 counterpart of H3 with factor B H6 interaction between factor A and factor B

12

ANALYSIS OF VARIANCE ANOVA

The hypotheses H1 and H 2 represent comparisons of weighted averages [Equations (18) and (20)] with the weights being a function of the cell frequencies. A hypothesis weighted by the cell frequencies might be appropriate if the frequencies reflected population sizes but would not be considered as the standard hypothesis. Table 7 specifies the particular hypothesis tested each type of ANOVA sums of squares. Table 8 shows three different analyses of variance tables for the example 2 ‘‘liver encym’’ computed with PROC GLM SAS. The hypotheses for the interaction term TxB is given by H6 . The test is the same for Type I, II, and III sums of squares. In this example, no interaction exists between treatment and liver impairment, P-value Pr > F = 0.94.

The hypothesis for ‘‘main effect A’’—the treatment comparison—is given by H1 , H 2 , or H 3 , corresponding with the different weighting means (Table 6 and Table 7). The range of the results for the treatment comparison are different (Table 8, source T) 1. for Type I, the P-value, Pr > F = 0.0049, is less then 0.005 (highly significant) 2. for Type II, the P-value, Pr > F = 0.074, is between 0.05 < P < 0.1 (not significant) 3. for Type III, the P-value, Pr > F = 0.142, is greater then 0.1 (not significant). The hypothesis H2 (i.e., Type II) is appropriate for treatment effect in the analysis of this

Table 7. Cell Means Hypotheses Being Tested

Sum of Sqares

Row effect, Factor A Column effect, Factor B Interaction, A×B

Typ I

Typ II

Typ III

H1 H4 H6

H2 H4 H6

H3 H5 H6

Typ I, Type II, and Type III agree when balanced data occur Table 8. Liver encyms -Two way classification with interaction term Treatment (T), impairment (B), interaction (T*B)

Source T B T*B Source T B T*B Source T B T*B

DF

Typ I SS

Mean Square

F-Value

Pr > F

1 1 1

0.20615 1.22720 0.00015

0.20615 1.22720 0.00015

8.41 50.08 0.01

0.0049 0.0001 0.9371

DF

Typ III

Mean Square

F-Value

Pr > F

1 1 1

0.05413 1.19127 0.00015

0.05413 1.19127 0.00015

2.21 48.61 0.01

0.1415 0.0001 0.9371

DF

Typ II

Mean Square

F-Value

Pr > F

1 1 1

0.08038 1.22720 0.00015

0.08038 1.22720 0.00015

3.28 50.08 0.01

0.0742 0.0001 0.9371

ANALYSIS OF VARIANCE ANOVA

example—a two-way design with unbalanced data. No interaction exists between treatment and liver impairment. The different test results for H 1 , H 2 , and H 3 result from the unbalanced data and factor—liver impairment. Any rules for combining centers, blocks, or stratas in the analysis should be set up prospectively in the protocol. Decisions concerning this approach should always be taken blind to treatment. All features of the statistical model to be adopted for the comparison of treatments should be described in advance in the protocol. The hypothesis H2 is appropriate in the analysis of multi-centers trials when treatment differences over all centers (13, 14, 19) are considered. The essential point emphasized here is that the justification of a method should be based on the hypotheses being tested and not on heuristic grounds or computational convenience. In the presence of a significant interaction, the hypotheses of main effects may not be of general interest and more specialized hypotheses might be considered. With regard to missing cells, the hypotheses being tested can be somewhat complex for the various procedures or types of ANOVA tables. Complexities associated with those models are simply aggravated when dealing with models for more than two factors. How many factors exist or how many levels each factor has, the mean of the observations in each filled cell, is an estimator of the population mean for that cell. Any linear hypothesis about cell means of any non empty cell is testable; see the work of Searle (3: 384–415). REFERENCES 1. R. A. Fisher, Statistical Methods for Research Workers. Edinburgh: Oliver & Boyd, 1925. 2. S. R. Searle, Linear Models. New York: John Wiley & Sons, 1971. 3. S. R. Searle, Linear Models for Unbalanced Data. New York: John Wiley & Sons, 1987. 4. R. R. Hocking and F. M. Speed, A full rank analysis of some linear model problems. JASA 1975; 70: 706–712. 5. R. R. Hocking, Methods and Applications of Linear Models-Regression and the Analysis

13

of Variance. New York: John Wiley & Sons, 1996. 6. R. C. Littel, W. W. Stroup, and R. J. Freund, SAS for Linear Models. Cary, NC: SAS Institute Inc., 2002. 7. P. Bauer, Multiple primary treatment comparisons on closed tests. Drug Inform. J. 1993; 27: 643–649. 8. P. Bauer, On the assessment of the performance of multiple test procedures. Biomed. J. 1987; 29(8): 895–906. 9. R. Marcus, E. Peritz, and K. R. Gabriel, On closed testing procedures with special reference to ordered analysis of variance. Biometrica 1976; 63: 655–660 10. C. W. Dunnett and C. H. Goldsmith, When and how to do multiple comparison statistics in the pharmaceutical industry. In: C. R. Buncker and J. Y. Tsay, eds. Statistics and Monograph, vol. 140. New York: Dekker, 1994. 11. D. R. Cox, Planning of Experiments. New York: John Wiley & Sons, 1992. 12. G. G. Koch and W. A. Sollecito, Statistical considerations in the design, analysis, and interpretation of comparative clinical studies. Drug Inform. J. 1984; 18: 131–151. 13. J. Kaufmann and G. G. Koch, Statistical considerations in the design of clinical trials, weighted means and analysis of covariance. Proc. Conference in Honor of Shayle R. Searle, Biometrics Unit, Cornell University, 1996. 14. J. L. Fleiss, The Design and Analysis of Clinical Experiments. New York: John Wiley & Sons, 1985. 15. H. Sahai and M. I. Ageel, The Analysis of Variance – Fixed, Random and Mixed Models. ¨ Boston: Birkhauser, 2000. 16. O. J. Pendleton, M. von Tress, and R. Bremer, Interpretation of the four types of analysis of variance tables in SAS. Commun. Statist.Theor. Meth. 1986; 15: 2785–2808. 17. F. Yates, The analysis of multiple classifications with unequal numbers in the different classes. JASA 1934; 29: 51–56. 18. F. M. Speed, R. R. Hocking, and O. P. Hackney, Methods of analysis of linear models with unbalanced data. JASA 1978; 73: 105–112. 19. S. Senn, Some controversies in planning and analysing multi-centre trials. Stat. Med. 1998; 17: 1753–1765.

• Three patients in the A arm and eight

ANALYSIS POPULATION

patients in the B arm were lost to followup during the 2-year duration of the study. • Four patients were missing a lab measure at baseline that could serve as an important covariate. • At least three patients in the A arm were known to be taking medication B, perhaps in addition to A. • During the trial, five patients moved to another state, and although no lab measures or visit data were missing, lab measures for these patients have now been conducted at different facilities, with unknown quality-control procedures.

ANDREW S. MUGGLIN JOHN E. CONNETT Division of Biostatistics, University of Minnesota, Minneapolis, Minnesota

Most applied statisticians, at some time, have been given the request: ‘‘Please go analyze the data and then report back.’’ But hidden in the instruction, ‘‘ . . . go analyze the data’’ are at least three major issues, each of which has many subissues. Exactly which data should be analyzed? What hidden biases exist in the way that these data were collected? Which analysis technique is the most appropriate for these data? None of these questions has a trivial answer, but appropriate, high-quality analysis and interpretation depend on careful attention to these questions. In this article, we examine specifically the question of which data should be analyzed. In data sets that emanate from clinical trials, this task can be surprisingly challenging. 1

Even in a simple setting such as this, it is not at all clear how to ‘‘go analyze the data.’’ The problems only grow when a trial contains thousands of patients, longer follow-up, and more complicated data collection schemes. One simple and common approach to this setting might be to take all patients who have complete datasets and analyze them as they were actually treated. On the surface, this approach sounds appealing on two fronts: First, the notion of ‘‘complete datasets’’ presents no analytical difficulties (statistical techniques are typically well defined on welldefined datasets). Second, to analyze patients as they were actually treated seems to imply that we are studying the actual scientific consequence of treating patients in a specified way. However, these approaches have several major problems. What does ‘‘actually treated’’ mean when patients either forget or refuse to take their medication precisely as prescribed? What kinds of systematic factors lurk behind a patient’s or caregiver’s decision not to comply completely with a specified treatment regimen? And if only complete datasets are analyzed, what systematic factors influence whether a patient’s dataset is incomplete? For instance, if sicker patients tend to have more missing values or tend not to take their medication faithfully, an analysis that ignores incomplete datasets and handles patients according to their actual

APPRECIATING THE DIFFICULTIES

Consider the following hypothetical example. Suppose we have a trial of 100 patients, randomized equally into two study arms: oral medication A and oral medication B. Suppose that the following plausible scenarios all occur: • Three patients who violated the inclu-

sion/exclusion criteria of the trial were enrolled—one patient because of a clerical mistake, one because of an erroneous lab value, and one because of a clinician’s judgment about the intent of the trial. • Because of clerical error, one patient that should have been assigned to A was actually assigned to B. • Three patients assigned to take A refused to take it and dropped out of the study.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

ANALYSIS POPULATION

treatment regimen can produce biased and misleading conclusions. Another approach might be to take all enrolled patients—regardless of whether they had data missing—and analyze them in the groups to which they were assigned by randomization. In this instance, the analyst immediately faces the problem of how to handle the missing values. Is it best to impute the worst possible value for these? Or observed values from some other patients? Or to impute these values via some mathematical model? And considering assignment by randomization (assuming the trial is randomized), analyzing patients according to how they were randomized results in a bias toward no treatment difference in the study arms. (The term ‘‘bias’’ in clinical trials is not intended to be limited to intentional or fraudulent bias but is meant to include the more common situation of hidden influences that systematically affect an outcome.) If the aim of the study is to show that a new treatment is better than an older one, then it is generally deemed to be a small evil, because the bias only makes it harder to demonstrate the superiority of the new treatment. But if the aim of the study is to show that the new treatment is equivalent or noninferior to an existing one, then analyzing patients by assigned treatment can be dangerous—especially if compliance to treatment is low—because the bias may falsely cause an inferior treatment to seem equivalent to a superior one. 2

DEFINITIONS OF COMMON TERMS

Several terms are commonly used to describe a particular analysis data set and the principles by which it was chosen. We describe some of them here.

patient is how they were randomized. (Exceptions and difficult cases have occurred; for example, a patient is accidentally assigned to the wrong treatment, or a patient somehow gets enrolled but is never randomized.) In nonrandomized studies, this principle is more difficult to implement but is probably most likely to be implemented as the intended course of treatment when a patient was initially enrolled, if that can be determined. Intention-to-treat is often the default analysis philosophy because it is conceptually simple and because it is conservative in the sense that lack of perfect compliance to the prescribed therapy tends to bias the treatment effects toward each other, which makes claims of superiority of one therapy over another more difficult to achieve. However, as mentioned previously, in the setting of equivalence or noninferiority trials, this bias can be an anticonservative bias. 2.2 As-Randomized This term is often used interchangeably with intention-to-treat, but it has a few differences. Obviously, the term ‘‘as-randomized’’ does not apply in a nonrandomized study. More subtle differences include situations where a patient is incorrectly assigned to a treatment. For instance, if randomizations are preprinted in a sequence of envelopes, and someone opens the wrong envelope for a given patient, this patient may be randomized to a treatment other than what was intended. Also, if patients are enrolled and intended to be randomized but for some reason are never randomized, an intention-totreat analysis may place those patients into whichever arm seems to be have the most conservative effect, whereas an as-randomized analysis may omit them. 2.3 As-Treated

2.1 Intention-to-Treat (or Intent-to-Treat, or ITT) This principle dictates that patients be divided into analysis groups based on how the investigator intended to treat them, rather than on how they were actually treated. This principle is easiest to apply in a study where patients are randomized to treatments, because in most cases the intended way to treat a

This approach assigns patients to analysis groups based on how they were actually treated. A patient randomized to therapy A but who receives therapy B will be analyzed as belonging to the therapy B group. This approach attempts to study the more purely scientific effect of the therapies as actually implemented, which avoids the bias that occurs under intention to treat. It has

ANALYSIS POPULATION

other biases, however. If patients who take an incorrect therapy do so for some systematic reason (e.g., the assigned therapy has failed, or sicker patients tend to switch therapies because of side effects), this will bias the interpretation in unpredictable ways. Moreover, it can be difficult in practice to determine how a patient is actually treated. What, for example, does one do with data from a patient who has taken the prescribed medication intermittently? Or who took only half the prescribed dose? Or one who, after taking therapy A for awhile, discontinues it and begins taking therapy B? 2.4 Per Protocol Sometimes called an ‘‘adherers only’’ analysis, this term is used to denote an analysis where patients’ data are included in the analysis dataset only if the patients adhered to the requirements of the protocol. This analysis typically means that patients are randomized correctly to a therapy, have met the inclusion and exclusion criteria for the study, and have followed the treatment regimen correctly. Patients who drop out of a treatment arm or who drop in to a different arm (whether or not they discontinue the assigned treatment) are excluded, at least for the periods of time where they were not properly complying with the protocol. This approach attempts to answer the scientific question of how the treatment behaves under ideal conditions, while avoiding the biases inherent in analyzing patients as they were actually treated (see above). However, some biases still exist, because patients who are excluded from the analysis may be eliminated for some systematic reason (e.g., elderly people had more trouble complying with therapy B than with therapy A and thus more elderly people are excluded from the B arm).

3

finished the entire prescribed course of therapy or some minimum amount of the entire course of therapy (making it more of an astreated or per-protocol analysis). It can help with understanding of the long-term effects of therapy, especially among compliers. It is subject to a very serious bias, however, because it ignores patients who drop out of the study or who do not comply. It is dangerous to assume that noncompliance is a random occurrence. 2.6 Evaluable Population This term is not very informative in how it handles patients (e.g., as randomized?), but the intention is that ‘‘nonanalyzable’’ data are excluded. Data can be classified as ‘‘nonanalyzable’’ for a variety of reasons: They can be missing (e.g., a patient has missed a follow-up visit, a serum assay was lost or not performed), technically inadequate (as occurs in certain echocardiographic or other high-tech measurements), or in some cases important endpoints cannot be adjudicated reliably (e.g., whether a death was specific to a certain cause). In some instances, the term seems to refer to patients who have taken the therapy at some minimum level of adherence or had a minimum level of follow-up (making it essentially a completer analysis). This approach is one of convenience. It is subject to many biases, and data interpretation can be difficult. Authors who use this term should take care to define it, and readers should pay careful attention to the authors’ definition. 3

MAJOR THEMES AND ISSUES

In selecting the analysis dataset, certain issues and questions are generated frequently. In this section, we address several of them.

2.5 Completer Analysis Set

3.1 Should We Analyze Data as Intention-To-Treat or Not?

This term is less common and seems to be used in different ways. Some use it to describe an analysis set that includes only patients who have finished the specified minimum follow-up time. When contrasted in publication with an intention-to-treat analysis, it sometimes represents those patients who

The reflexive answer to this question is often affirmative, but the decision is sometimes not easy. Cases where it is easy are settings in which an investigator is trying to prove that an experimental treatment is superior to a standard treatment or placebo. As mentioned previously, the rationale here is that

4

ANALYSIS POPULATION

biases tend to work against the experimental therapy. If patients cannot comply well with the experimental regimen, then it will tend to work against the finding that the experimental treatment is superior. If a large amount of drop-in or drop-out of the experimental arm occurs (e.g., control subjects getting the experimental therapy, or experimental arm patients who follow the control regimen), then it will have the same effect. This conservative bias is also an incentive to investigators to be vigilant in executing the trial according to plan, because mistakes such as accidentally randomizing or assigning patients to treatments will tend to hurt the goals of the trial. Another compelling reason to analyze patients by intention to treat is that assignment of patients to analysis groups is usually straightforward (although it may be challenging to conduct the analysis, such as in the case of missing data). In some settings, however, this approach may not be the best. In an equivalence trial, where the goal is to acquire evidence to establish that two treatments are nearly the same, poor compliance will bias the trial in favor of the desired finding. In some settings, particularly pilot studies, the question of most interest regards the treatment effect when implemented correctly (‘‘efficacy’’), rather than its effect in actual practice (‘‘effectiveness’’). In other settings, an investigator may opt for an intention-to-treat analysis when studying a therapy’s efficacy or effectiveness but not for its safety. It is good practice to think through these issues and document one’s approach prior to knowing the results for any particular analysis. This practice defends against the criticism that a particular cohort was selected simply because it yields the most favorable interpretation. 3.2 What If Inclusion or Exclusion Criteria Are Violated? In trials of moderate or large size, it is common that a few patients are enrolled that should not have been enrolled. Reasons range from simple mistakes, to lab values that were not known when the patient enrolled, to investigator fraud. Hopefully in a wellexecuted trial, this group of patients will be small enough that results do not depend

on whether these patients are included. But there are no guarantees, and in the end, one analysis must be selected as the official trial result. Should false inclusions be contained in this analysis? One argument to exclude data from such patients is that they do not come from the intended study population. It is difficult to know whether their inclusion in the analysis represents a conservative or anticonservative bias. An argument to include them is that this sort of mistake will inevitably happen in real life, so it is best to analyze data that way. Senn (1, p. 157) and Friedman et al. (2, ch. 16) give several other arguments on each side. Once again, it is important that investigators think through their specific cases prior to knowing the analysis results. In our opinion, it is generally best to analyze according to the principle that ‘‘if enrolled, then analyzed.’’ A variant on this question is, ‘‘If I discover a false inclusion during the trial, should follow-up on that patient be discontinued even though I intend to analyze the data I have thus far collected?’’ This question has almost no good answer. If the answer is ‘‘Yes,’’ then potential late adverse events may be lost. If the answer is ‘‘No,’’ then the investigator may be exposing the patient to an inappropriate risk inherent in the therapy. In our opinion, this situation must be handled on a case-by-case basis, with investigators wrestling with the issues and erring, if necessary, on the side of the patient’s best interests and the most disclosure about the nature of the mistake. 3.3 What Do We Do if Data Are Missing? Missing data are inevitable in a study that involves humans. Data go missing for many reasons: patients drop out or withdraw consent, patients miss scheduled visits, urine samples are lost, records are lost, required tests are not conducted, test results are found to be technically inadequate, memory banks in recording devices become full, and so on. Furthermore, many different data items can be missing, and they are not always in the same patients. A patient who relocates and loses contact with the study center will have all data missing after a certain time point. A patient may have complete follow-up data

ANALYSIS POPULATION

but be missing an important baseline lab measure. A patient cannot make a certain scheduled visit but may have full information from all other visits. If data were always missing because of random causes, then it would be sensible to analyze the data that are actually collected and ignore missing cases. Most statistical software packages do this by default. But what assurance is there that the nonmissing data are representative of the missing data? In particular, if the sickest patients are dropping out or skipping scheduled visits because of their condition, or if caregivers are not prescribing certain tests because it would put a very sick patient under excessive stress, then the nonmissing data values are not representative of the missing ones, and any analyses that ignore missingness will be biased. Much statistical research has focused on the topic of statistical adjustment for missing data (3,4). But the fundamental reality is that missing data are still missing, and any statistical adjustment is only as good as the assumptions employed in the adjustment. It is important to assess and disclose the potential impact of missing data on study conclusions through sensitivity analyses. It is even more important to expend maximal energy to prevent the data from going missing in the first place. 3.4 What Is the Value of Subgroup Analyses or Responder Analyses? At the conclusion of a trial, investigators often wish to know if a therapy was especially effective for particular types of patients. A subgroup analysis is one where specific subclasses of patients are selected and their data are analyzed. Typical subgroups are defined by gender, race, geographical location, or many disease-specific categorizations. Here, the analysis is typically well defined (it is usually the primary objective analysis applied to a subset of patients), but the number of subgroups that could be analyzed is practically limitless. Subgroup analyses are often presented graphically, with confidence intervals for the treatment effect in each subgroup presented together in one graph. It is not surprising that the treatment effect is not the same in all

5

subgroups. Inevitably, some will have larger confidence intervals than others (because of varying subgroup sizes or different subgroup variances), and some can seem to be different from the others in the location of the confidence interval. It is important to remember that if enough subgroups are tested, probably every possible finding will be observed, so results should be interpreted with caution, and researchers should examine other studies to corroborate any apparent findings. A related analysis is a post-hoc responder analysis. Here, patients are identified as being a ‘‘responder’’ to a therapy according to some measure (e.g., still alive, or blood pressure adequately controlled), and the task of the analyst is to determine whether any subgroupings, baseline measures, or early indicators significantly predict who the responders will be. This method can be helpful in generating hypotheses for additional research. But because of the potentially limitless number of covariates that can be analyzed, these results should be viewed with extreme caution. 3.5 How Should Analyses be Handled If There is Poor Compliance to Treatment Regimen? In some studies, such as certain medical device studies, compliance is nearly perfect (e.g., if a pacemaker is implanted, it does what it is programmed to do; noncompliance is rare and might be caused by device malfunction or physician’s programming error but not by patient cooperation). In other studies, compliance is much poorer (e.g., when a patient must remember to take a combination of poorly tolerated medications on different dosing schedules). It is often difficult or impossible to determine the extent of the noncompliance accurately, although this fact should not discourage an investigator from attempting its assessment. This assessment is more for the purpose of defending trial conduct and interpretation than to decide whether to analyze any particular patient’s data. At one extreme, the data from all patients are analyzed regardless of compliance, which is consistent with the intention-to-treat approach. This method usually makes sense whenever the intention-to-treat analysis app-

6

ANALYSIS POPULATION

roach makes sense. At the other extreme, absolutely perfect or at least very good compliance is required to be included in an analysis. This analysis makes sense if the aim is to determine whether a treatment works under ideal circumstances or to identify adverse effects of a treatment when applied exactly as intended. It is usually better to analyze more data than less.

4

RECOMMENDATIONS

Hopefully, it is clear at this point that the resolution to most issues in choosing an analysis dataset is seldom simple and that experts can and do disagree. The following sections represent our opinions on some approaches that we have found useful in determining the makeup of an analysis dataset from a clinical trial. 4.1 Wrestle with the Issues It is not possible to eliminate or control all biases in a clinical investigation. It is usually not possible to estimate their magnitudes, and in some cases, it is impossible even to know the direction of bias. But it is possible to understand many of them and take steps to mitigate them, especially in trial conduct but also in data analysis. It can be hard work, but the result of this effort is a great enhancement in the quality and validity of the results. 4.2 Prespecify Wherever Possible Not all issues can be anticipated, but it should not be an excuse for not trying. Identify and document your planned approach to selecting an analysis cohort in advance.

4.4 Know That What Is Conservative in One Setting May Not Be In Another As previously mentioned, the intention-totreat principle is usually conservative in a trial of superiority, although it is not conservative in a trial of equivalence or noninferiority. At times, what is conservative for an efficacy objective may not be conservative for a safety objective. 4.5 Favor General Applicability Over Pure Science Several legitimate reasons can be cited to study a therapy under ideal conditions and also under more realistic conditions. When in doubt, we have found it helpful to favor the realistic side and to attempt to include rather than exclude data. 4.6 Conduct Sensitivity Analyses Check the robustness of your conclusions to varying definitions of the analysis cohort, as well as missing data assumptions and analysis techniques. Disclose in the final report or manuscript whether the conclusions change under reasonable deviations from your planned approach. 4.7 Account Painstakingly For How the Analysis Dataset Was Selected Account carefully in any final report for every patient that is included in or excluded from each analysis. In a journal article, we have less room to describe this in detail, but it is still good practice to account for patients’ data in summary fashion. The CONSORT statement (5) provides a checklist and example statements to illustrate how journal articles should report on methods in general (including specifically the numbers analyzed) for randomized clinical trials. Many of these principles apply to nonrandomized studies as well.

4.3 Take the Conservative Course

4.8 Illustrative Example

When biases cannot be controlled, seek to align them against the conclusion that you wish to reach, if possible. Then, if you achieve statistical significance regarding your desired conclusion, you have firm evidence.

The Lung Health Study (6) provides an excellent example of several issues presented above. The intent of this randomized trial was to study three treatments for chronic obstructive pulmonary disease, which is a

ANALYSIS POPULATION

major cause of morbidity and mortality that occurs almost exclusively in smokers. The three treatments were 1) smoking intervention plus an inhaled bronchodilator, 2) smoking intervention plus placebo, and 3) no intervention. The primary goal was to determine whether the treatments differentially influence the rate of decline in forced expiratory volume in 1 second (FEV1 ). It was intended that 6000 patients be enrolled and followed at annual visits for 5 years. The manuscript in Reference 6 provides an unusually detailed description of the study methods, which includes paragraphs on design, entry criteria, patient recruitment, spirometry, compliance monitoring, follow-up procedures, quality control, data management, and statistical methods. In a table in the Results section, the investigators report the follow-up rates at each annual visit by treatment group, with a maximum follow-up rate of 95% in the intervention plus bronchodilator group at year 5, and a minimum follow-up rate of 88% that occurred at one time in each of the three arms. In year 5, follow-up rates were at least 94% in all three groups. The generally high follow-up rates and especially the 94–95% follow-up rates in year 5 were in part the result of extra efforts by the investigators to ensure attendance at the fifth annual visit by as many surviving participants as possible. This result is important because the fifth annual visit data had the greatest influence on the computation of rates of change of the outcome variable and gave the most accurate estimate of the participant’s lung function status at the time they exited the study. It is possible to estimate the rate of decline in pulmonary function if a person has at least two annual visits (i.e., baseline and at least one follow-up measurement). The implicit assumption in such an analysis is that if a visit is missed, then the outcome data are missing at random (i.e., the missingness of the measurement is independent of the value of the outcome variable itself). This assumption is unlikely to be true in practice. A person with seriously impaired lung function is more likely to be sick or incapacitated, and therefore more likely to miss a visit, than a similar person with better lung function. In this case, no adjustments for

7

nonrandom missingness were applied in the estimates of rates of decline in lung function. Similarly, data for participants who died during the course of the study were included for all lung function measurements made prior to the time of death, and the data were entered into the analysis of the outcome. The manuscript points out several details that bolster the reader’s understanding of the analysis cohort. Among these are statements such as the following: ‘‘At the end of the study, the whereabouts and vital status of only 21 participants (0.4%) were unknown.’’ ‘‘Of the 5887 participants, 149 died during the 5-year follow-up period.’’ In captions in several figures, statements disclose how missing data values were handled: ‘‘Nonattenders of follow-up visits are counted as smokers’’ and ‘‘Participants not attending the visit were classified as noncompliant.’’ In an assessment of patient compliance to the bronchodilation regimen, the investigators report that 20–30% of patients who attended follow-up visits did not bring in their medication canisters for weighing, and this missingness probably represents a source of bias. The investigators discuss this bias, along with overestimates and underestimates of the bias, and report that by either measure, inhaler compliance did not differ between the intervention plus bronchodilator group and the intervention plus placebo group. This example demonstrates several principles found earlier in this article. First, analysis was by intention to treat, which was conservative in this case. (The manuscript words it as a ‘‘comparison of the randomly assigned groups.’’) Strong efforts were made to prevent losses to follow-up, with a high degree of success. Researchers had an understanding of and presented a disclosure of biases involved in missing data items, as well as a sensitivity analysis to how the missing compliance data were handled. And finally, researchers accounted carefully for patients at various time points and how it impacted each analysis. 4.9 Summary The process of selecting a dataset to represent the official results of a clinical trial is far from trivial. The principles involved are not

8

ANALYSIS POPULATION

universally accepted, and even when there is general agreement, there are often difficulties in specific data items that make implementing those principles difficult or even impossible. Nevertheless, some common terms can be used by researchers to describe their approaches to data selection and analysis, and common situations can develop wherein researchers can learn from each other. If this article can be boiled down to its simplest essence, it is this: Selecting a clinical trial analysis dataset is much more difficult than it seems. Therefore, think hard about the issues and potential biases, take a conservative course when you are unsure, and openly disclose how the final datasets were selected.

The difficulty involved in selecting an analysis dataset is just another of the many challenges— legal, ethical, scientific, practical—involved in medical research on humans. But in carefully adhering to good scientific principles and in paying close attention to biases and limitations—which include the selection of data for analysis—the impact of medical research is substantially enhanced. REFERENCES 1. S. Senn, Statistical Issues in Drug Development. New York: John Wiley & Sons, 1997. 2. L. M. Friedman, C. D. Furberg, D. L. DeMets, Fundamentals of Clinical Trials, 3rd ed. New York: Springer-Verlag, 1998. 3. D. B. Rubin, Multiple Imputation for Nonresponse in Surveys. New York: John Wiley & Sons, 1987. 4. J. L. Schafer, Analysis of Incomplete Multivariate Data. New York: Chapman and Hall, 1997. 5. D. Moher, K. F. Schulz, and D. G. Altman, for the CONSORT Group, The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. Lancet 2001; 357: 1191–1194. 6. N. R. Anthonisen, J. E. Connett, J. P. Kiley, M. D. Altose, W. C. Bailey, AS. Buist, W. A. Conway Jr, P. L. Enright, R. E. Kanner, P. O’Hara, G. R. Ownes, P. D. Scanlon, D. P. Tashkin, and R. A. Wise, for the Lung Health Study Research Group, Effects of smoking

intervention and the use of an anticholinergic bronchodilator on the rate of decline of FEV1. JAMA 1994; 272: 1497–1505.

FURTHER READING P. Armitage, Exclusions, losses to follow-up, and withdrawals in clinical trials. In: S. H. Sharpiro and T. A. Louis (eds.), Clinical Trials: Issues and Approaches, New York: Marcel Dekker, 1983, pp. 99–113. S. Piantadosi, Clinical Trials: A Methodologic Perspective. New York: John Wiley & Sons, 1997. The ICH Guidelines are also useful references. See especially E3: International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use, ICH Harmonized Tripartite Guideline, Structure and Content of Clinical Study Reports (E3), 1995. Available in Federal Register on July 17, 1996 (61 FR 37320) and at: http://www.ich.org/LOB/media/MEDIA479.pdf and also at: http://www.fda.gov/cder/guidance/ iche3.pdf.

CROSS-REFERENCES Bias Missing Values Intention-to-Treat Analysis Equivalence Trial

APPLICATION OF NOVEL DESIGNS IN PHASE I TRIALS

development in cancer research where cytotoxic drugs were developed, the optimal dose was considered the maximally tolerated dose (MTD): the highest dose that has an acceptable level of toxicity. But there are other optimality criteria, such as minimally effective dose (MinED): the minimum dose that shows sufficient efficacy.

ELIZABETH GARRETT-MAYER Sidney Kimmel Comprehensive Cancer Center Johns Hopkins University Baltimore, Maryland,

Phase I dose finding studies have become increasingly heterogeneous in regards to the types of agents studied, the types of toxicities encountered, and degree of variability in outcomes across patients. For these reasons and others, when designing a phase I study it is especially important to select a design that can accommodate the specific nature of the investigational agent. However, most current phase I studies use the standard 3 + 3 design, which has been demonstrated to have poor operating characteristics, allows little flexibility, and cannot properly accommodate many of current dose-finding objectives. Novel phase I trials have been introduced over the past 20 years that have superior properties and are much more flexible than the 3 + 3 design, yet they have not been fully embraced in practice. These newer designs are in general not cumbersome and provide a more accurate and precise estimate of the appropriate dose to pursue in phase II and later studies. 1

2 STANDARD DESIGNS AND THEIR SHORTCOMINGS 2.1 The Standard 3 + 3 Design The standard 3 + 3 is the most commonly used and well-known phase I design (1). However, it is also probably the most inappropriate in most cases. It is an algorithmic design where a set of rules are followed for dose escalation or de-escalation based on observed dose limiting toxicities (DLTs); at the end of the trial, a maximally tolerated dose (MTD) is declared without any analysis of the data. The way the algorithm works is as follows. Enter three patients on dose level k: 1. If 0 of 3 patients have a DLT, dose escalate to dose level k + 1. 2. If 2 or 3 of 3 patients have a DLT, de-escalate to dose level k – 1. 3. If 1 of 3 patients has a DLT, add an additional 3 patients at dose level k. (a) If 1 of the 6 patients treated at k have a DLT, escalate to dose level k + 1. (b) If 2 or more of the 6 patients at k have a DLT, de-escalate to dose level k – 1.

OBJECTIVES OF A PHASE I TRIAL

It is important to be reminded of the goals of the phase I trial when selecting a design. Phase I studies are generally dose-finding studies, where the goal of the trial is to identify the optimal dose to take forward for further testing (usually for efficacy testing). For ethical reasons, patients cannot be randomized to doses: one cannot treat patients at high doses until lower ones have been explored. The standard approach, therefore, has been to administer a low dose of an agent to a small number of patients and, if the dose appears to be well-tolerated within those patients, to administer a higher dose to another small number of patients. Historically, due to the roots of phase I trial

When de-escalations occur, an additional 3 patients are usually treated at the dose unless 6 patients have already been treated at that dose. The MTD is then defined as the highest dose at which 0 or 1 of 6 patients has experienced a DLT. Note that there are slight variations to this design, but the way it is described here is one of the more common implementations (1). Many agents currently under evaluation in phase I studies do not have the same toxi-

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

APPLICATION OF NOVEL DESIGNS IN PHASE I TRIALS

city concerns that cytotoxic agents have had in the past. The standard 3 + 3 phase I design makes several assumptions that are not practical for many current phase I studies. Some of these assumptions are the following: 1. As dose increases, the chance of a response increases (i.e., there is a monotonic increasing association between dose and response). 2. As dose increases, the chance of toxicity increases (i.e., there is a monotonic increasing association between dose and toxicity). 3. The dose that should be carried forward should be the highest dose with acceptable toxicity. 4. Acceptable toxicity is approximately 15% to 30% (regardless of the definition of a toxic event). These assumptions are very limiting and are not practical in many of our current settings. For example, angiogenesis inhibitors or colony-stimulating factors may have very low probability of toxicity, and a dose-finding study may be more concerned with evaluating pharmacodynamic response over an increasing range of doses. As such, we would not want to escalate until we saw toxicity: we would want to escalate until we saw sufficient pharmacodynamic response. We may also assume that there may be a range of doses that provides a pharmacodynamic response, but doses below or above this range may be ineffective. This invalidates the assumption of monotonicity between dose and response. 2.1.1 What Is Wrong with the Standard 3 + 3 Design? There are a number of flaws that can be identified in the standard 3 + 3 design, and a few are highlighted here. One of the most problematic issues with this design is that it does not allow for the desired toxicity rate to be user-specified. For example, in certain cancers with very high mortality rates (e.g., brain cancer), the acceptable toxicity rate (i.e., the proportion of patients who experience a DLT) is higher than in some other more curable and less deadly cancers. Also, acceptable toxicity rates for short-term treatments (e.g., several cycles

of chemotherapy) will likely be higher than for long-term treatments (e.g., a daily dose of tamoxifen for 5 years for breast cancer survivors). The standard 3 + 3 design is prespecified in such a way that these considerations cannot be accommodated. The target toxicity rate for these trials is not well-defined (although it is often assumed that they target a rate between 15% and 30%) because it depends heavily on the spacing between prespecified dose levels and how many dose levels are included. In many cases, the 3 + 3 design will significantly underestimate or overestimate the desired toxicity rate. Another major shortcoming of the standard 3 + 3 is that, when choosing the MTD, it ignores any toxicity information observed at lower doses and has very poor precision. This is a common feature of algorithmic designs, but it is still a serious inefficiency. Newer designs estimate a dose-toxicity curve after all the data have been observed to not only help in choosing the most appropriate MTD, but also to provide an estimate of precision. By using the data that have accrued throughout the trial at all doses, there will be greater precision in the estimate of the MTD. Even if the target toxicity rate is approximately 20% and the goal of the trial is to find the MTD, the 3 + 3 will be a less efficient and more error-prone way of identifying a dose than model-based approaches described later in this chapter. Lastly, the 3 + 3 design will not achieve the objectives of a phase I trial when the primary outcome of interest for dose selection is not the maximally tolerated dose. When identifying the MinED, escalating based on DLTs is illogical. However, the standard 3 + 3 only allows for toxicity-based outcomes. 2.2 The Accelerated Titration Design There have been some improvements made to the standard 3 + 3 approach. Simon et al. (2) introduced the accelerated titration design, which has two major advantages over the standard design: (1) it treats only one or two patients per dose level until a DLT is observed; and (2) at the end of the trial, all of the data are incorporated into a statistical model for determining the appropriate dose to recommend for phase II testing. These

APPLICATION OF NOVEL DESIGNS IN PHASE I TRIALS

designs also allow for intrapatient dose escalation. It is clearly a more ethical and more appealing design to patients (patients are less likely to be treated on ineffective doses and can be escalated if they tolerate the treatment). This design still has the shortcoming that escalation is determined based on the same rules as the 3 + 3. However, this is mitigated to some extent by choosing the MTD based on a fitted statistical model. The accelerated titration design is also limited in that it is only appropriate for dose escalation that is toxicity based: for agents where toxicity is not the major outcome for dose finding, the accelerated titration design should not be used. 3

SOME NOVEL DESIGNS

In the past 20 years, a number of designs have been introduced for increased efficiency in phase I trials, but, unfortunately, most of these designs have not been adopted into mainstream use despite their significant improvements as compared with the standard algorithmic designs. Part of the reason for this is that they are based on statistical models and those without a strong background in statistics may be wary of them. There is a misconception that these novel designs are not as safe as the standard 3 + 3, but the evidence is quite to the contrary: many investigators have shown that, for example, the continual reassessment method is safer than the standard 3 + 3 because it exposes fewer patients to toxic doses and requires a smaller sample size (3–7). Even so, proponents of the 3 + 3 continue to promote to it with the idea that such a simple and standard design is safer than the alternatives and that its historic presence is a testament to its utility. An adaptive design is one in which certain decisions about how to proceed (e.g., what doses to give patients, how many patients to treat, which treatment to apply) are made for future patients in the trial based on results observed on all patients treated earlier in the trial. Adaptive phase I designs are designs in which the dose for a future patient is chosen based on the results from patients already treated on the study. There are several ways

3

that adaptive designs can be used in phase I studies, and just a few are described here. These designs have well-documented advantages over nonadaptive approaches and are becoming more commonly seen in many phase I clinical trials (3–7). 3.1 The Continual Reassessment Method The continual reassessment method (CRM) was first introduced by O’Quigley et al. in 1990 (8) and has undergone many modifications in the past 16 years. It is fundamentally different from the standard 3 + 3 design and most other ‘‘up and down’’ designs because (1) it relies on a mathematical model, (2) it assumes that dose is continuous (i.e., doses are not from a discrete set), and (3) it allows and requires the user to select a target toxicity rate. The original CRM used a Bayesian framework and had relatively few safety considerations. Newer and more commonly used versions have included several safety concerns (e.g., not allowing dose to increase by more than 100%) and a simpler estimation approach (i.e., maximum likelihood estimation). Although there are a number of CRMs currently used in practice, the model of Piantadosi et al. (9) will be the approach described here because of its simplicity. For other modifications, see Goodman et al. (4), Faries et al. (10), and Moller (11). The design requires several user-specified inputs: 1. The target dose-limiting toxicity rate 2. A dose that is expected to have a low level of toxicity (e.g., 5% or 10%) 3. A dose that is expected to have a high level of toxicity (e.g., 90% or 95%) 4. The number of patients per dose level (usually 1, 2, or 3). The number of patients per cohort usually will depend on the anticipated accrual. The mathematical model used is prespecified; with the knowledge of the three items listed above, the dose for the first cohort of patients can be calculated. This is demonstrated in Figure 1A where three assumptions are made: a 5% DLT rate at a dose of 100 mg, a 95% DLT rate at a dose of 900 mg, and a logistic dose-toxicity model. (For more

4

APPLICATION OF NOVEL DESIGNS IN PHASE I TRIALS

detail on the logistic dose-toxicity model, see Piantadosi et al. [9]). The dose-toxicity curve is drawn according to this information, with a horizontal line at the target toxicity rate (25%). The dose corresponding to a 25% DLT rate is 343 mg, and this is the dose for our first cohort of patients. Let us assume that two patients are treated at dose level 1 (dose = 343 mg). The DLT information observed has three possibilities: (1) neither patient has a DLT, (2) one patient has a DLT, or (3) both patients have a DLT. We add this information to our presumed information described in the previous paragraph. In Figures 1B to 1D, the updated

dose-toxicity curves are drawn using this new information, corresponding to the three possible outcomes, with the toxicity outcomes included in the graphs (0 = no DLT, 1 = DLT). If neither patient has a DLT, the dose is escalated to 418 mg (see Figure 1B); if one patient has a DLT, the dose is decreased to 218 mg (see Figure 1C); if both have a DLT, then dose is decreased to 147 mg for the next cohort (see Figure 1D). The approach continues where, after each cohort of two patients is observed, the additional data are incorporated to update the estimated dose-toxicity curve. The CRM is considered ‘‘adaptive’’ because we are using

600

800

1.0 0.8 0.6 0.4

1000

0

200

400

600

800

1000

DOSE (MG)

C. 1/2 DLTs: next dose is 233mg

D. 2/2 DLTs: next dose is 147mg

600

800

1000

0.8 0.6 0.4 0.2

PROBABILITY OF DLT 400

DOSE (MG)

0.0

0.8 0.6 0.4 0.2

200

1.0

DOSE (MG)

0.0 0

0.2

PROBABILITY OF DLT 400

0.0

1.0 0.8 0.6 0.4 0.2

200

1.0

0

PROBABILITY OF DLT

B. 0/2 DLTs: next dose is 418mg

0.0

PROBABILITY OF DLT

A. Starting dose: 343mg

0

200

400

600

800

1000

DOSE (MG)

Figure 1. (A) Estimated dose-toxicity curve based on a priori information about likely low and high toxicity doses. The starting dose is 343 mg. (B) Updated dose-toxicity curve if no dose-limiting toxicities (DLTs) are seen in two patients treated at 343 mg. The updated dose for next cohort would be 418 mg. (C) Updated dose-toxicity curve if one DLT is seen in two patients treated at 343 mg. The updated dose for next cohort would be 233 mg. (D) Updated dose-toxicity curve if two DLTs are seen in two patients treated at 343 mg. The updated dose for next cohort would be 147 mg.

APPLICATION OF NOVEL DESIGNS IN PHASE I TRIALS

the data collected as we proceed to determine the doses for future cohorts. And, unlike the algorithmic designs, when choosing a dose for the next cohort, we use information collected from all previous cohorts and not just the last cohort of patients. There are a variety of approaches that may be used to determine when the CRM trial should stop. Suggested approaches include when the recommended dose differs by no more than 10% from the current dose (9), defining a prespecified number of patients (4), or based on a measure of precision of the curve (12). For a detailed discussion of sample size choice, see Zohar and Chevret (13). Although in theory CRMs are markedly better than the standard 3 + 3 design, they do have several potential limitations. First, they require a statistician to be intimately involved with the daily coordination of the trial. The dose for a new patient depends on the data observed up until the new patient is enrolled and on the statistical model that has been chosen. Determining the next dose is not a trivial endeavor and requires that a statistician be available and familiar with the trial to perform these calculations on short notice. Second, the mathematical model that is assumed may not be flexible enough to accommodate the true or observed dosetoxicity relationship. The approach that we have highlighted here by Piantadosi et al. (9) is more flexible than some others, but in some cases, the chosen model may not be robust. 3.2 Extensions of the CRM for Efficacy Outcomes Although the CRM was developed to be used for dose-toxicity relationships, dose-finding based on dose-efficacy relationships can also be explored. This is most appropriate in the setting where a targeted therapy is being evaluated and, for example, there is a desired level of inhibition of a particular target. A target effectiveness rate could be defined as the dose that achieves inhibition of the target in at least 80% of patients. Pharmacokinetic parameters (e.g., area under the curve) could also be the basis for the design of a CRM study.

5

3.3 Bayesian Adaptive Designs Bayesian adaptive designs comprise a wide range of designs that are all based on the same general principles. For phase I studies, Bayesian adaptive designs have many similarities to the CRM described in the previous sections. They tend to be more complicated mathematically, so many of the details of these designs are beyond the scope of this article. However, the basic elements required for Bayesian decision making are (1) a statistical model, (2) a prior distribution that quantifies the presumed information about the toxicity of doses before the trial is initiated, (3) a set of possible actions to take at each look at the data, and (4) a ‘‘gain’’ function. We will consider each of these in turn.

1. The statistical model. This is a mathematical representation of the relationship between dose and toxicity. In the CRM we have described, we assumed a logistic model, but there are many possible models. 2. A prior distribution. This is similar to the quantification of toxicity a priori that is shown in Figure 1A. In the Bayesian approach, this is more formally described using an actual probability distribution but will take a form similar to that shown in Figure 1A. Often these are taken to be quite conservative by assuming that high doses tend to be very toxic so that the dose escalation is less likely to investigate high doses unless there is relatively strong evidence that they are safe, based on the accumulated information at lower doses. 3. Possible actions. These are the actions that can be taken when new patients are enrolled and include all of the possible doses that could be given to them. 4. Gain function. This is an attribute of the Bayesian approach that is quite different from the CRM. The gain function reflects what is best for the next cohort of patients. This is particularly relevant in phase I trials because of the tradeoff of efficacy and toxicity: a higher dose likely means an increase in both, and

6

APPLICATION OF NOVEL DESIGNS IN PHASE I TRIALS

the gain function allows us to include that tradeoff in our decision making about the next cohort. However, the gain function can take many forms and need not represent the tradeoff between toxicity and efficacy.

mathematical details needed to fully describe the approach, but the general idea is shown in Figure 2. Each curve (i.e., ‘‘contour’’) in Figure 2 represents a set of efficacy–toxicity tradeoffs that would be considered equally desirable. The user must elicit information from the clinical investigator to determine what a target contour would be. Dose is escalated based on the combined outcomes of toxicity and efficacy, and new doses are determined as a function of previously observed results. Another adaptive design recently proposed uses a three category outcome variable to describe both efficacy and toxicity (22). This approach has many nice properties, such as maximizing efficacy with constraints on toxicity instead of allowing them equal importance. This may be more practical in many settings, especially when it is expected that the agent under investigation has very low toxicity.

These designs are described in detail by Gatsonis and Greenhouse (14), Whitehead (15), and Haines et al. (16). Descriptions aimed at clinical investigators are provided by Zhou (17) and Whitehead et al. (18). Zohar et al. (19) have developed publicly available software for conducting Bayesian dose-finding studies. 3.4 Efficacy and Toxicity for Dose Finding Recently developed dose escalation designs have formally accounted for the desire to find doses that have both acceptable toxicity and maximal efficacy. This is in response to the trend in recent years of applying standard designs to find MTDs and subsequently looking at efficacy outcomes to see which, if any, of the doses explored showed efficacy either based on clinical outcomes, such as tumor response, or based on correlative outcomes, such as modulation of a genetic marker known to be in the cancer pathway. A more efficient approach is to simultaneously maximize efficacy while minimizing toxicity. Thall and Cook (20, 21) have proposed an adaptive Bayesian method that chooses the optimal dose based on both efficacy and toxicity. The principles are like those described in the previous section, but with the added complexity of two outcomes. There are many

4 DISCUSSION

0.0

Probability of Toxicity 0.2 0.4 0.6 0.8 1.0

In recent years, there have been quite a few novel designs proposed for phase I trials, many of which are Bayesian and most of which are adaptive. The trend has been to steer away from the old-fashioned algorithmic designs that have been shown repeatedly to be inefficient. Chevret (23) has recently published a book on the dose-finding methods that addresses many of the issues discussed here in more depth and detail. There are a number of approaches that could not be discussed in this chapter due to space considerations. Readers interested

0.0

0.2

0.4 0.6 Probability of Efficacy

0.8

1.0

Figure 2. An example of contours for Thall and Cook’s adaptive Bayesian design using both efficacy and toxicity. The target contour is shown by the thick line. Several other contours are shown with thinner lines.

APPLICATION OF NOVEL DESIGNS IN PHASE I TRIALS

in learning more about novel designs should also consider investigating the curve-free method (24) and the biased coin up-and-down design with isotonic regression (25) that have been introduced in recent years. REFERENCES 1. L. Edler, Overview of phase I trials In: J. Crowley (ed.), Handbook of Statistics in Clinical Oncology. New York: Marcel Dekker, 2001, pp. 1–34. 2. R. M. Simon, B. Freidlin, L. V. Rubinstein, S. Arbuck, J. Collins, and M. Christian, Accelerated titration design for phase I clinical trials in oncology. J Natl Cancer Inst. 1997; 89: 1138–1147. 3. B. Storer, An evaluation of phase I clinical trial designs in the continuous dose-response setting. Stat Med. 2001; 20: 2399–2408. 4. S. N. Goodman, M. L. Zahurak, and S. Piantadosi, Some practical improvements in the continual reassessment method for phase I studies. Stat Med. 1995; 14: 1149–1161. 5. E. Garrett-Mayer, The continual reassessment method for dose-finding: a tutorial. Clin Trials. 2006; 3: 57–71. 6. J. O’Quigley, Another look at two phase I clinical trial designs. Stat Med. 1999; 18: 2683–2690. 7. C. Ahn, An evaluation of phase I cancer clinical trial designs. Stat Med. 1998; 17: 1537–1549. 8. J. O. O’Quigley, M. Pepe, and L. Fisher, Continual reassessment method: a practical design for phase I clinical trials in cancer. Biometrics. 1990; 46: 33–48. 9. S. Piantadosi, J. D. Fisher, and S. Grossman, Practical Implementation of a modified continual reassessment method for dose-finding trials. Cancer Chemother Pharmacol. 1998; 41: 429–436. 10. D. Faries, Practical modifications of the continual reassessment method for phase I cancer clinical trials. J Biopharm Stat. 1994; 4: 147–164. 11. S. Moller, An extension of the continual reassessment methods using a preliminary up-and-down design in a dose finding study in cancer patients, in order to investigate a greater range of doses. Stat Med. 1995; 14: 911–922. 12. N. Ishizuka and Y. Ohashi, The continual reassessment method and its applications: a

7

Bayesian methodology for phase I cancer clinical trials. Stat Med. 2001; 20: 2661–2681. 13. S. Zohar and S. Chevret, The continual reassessment method: comparison of Bayesian stopping rules for dose-ranging studies. Stat Med. 2001; 20: 2827–2843. 14. C. Gatonis and J. Greenhouse, Bayesian methods for phase I clinical trials. Stat Med. 1992; 11: 1377–1389. 15. J. Whitehead, Using Bayesian decision theory in dose-escalation studies In: S. Chevret (ed.), Statistical Methods in Dose-Finding Experiments. Chicester, UK: Wiley, 2006, pp. 149–171. 16. L. M. Haines, I. Perevozskaya, and W. F. Rosenberger, Bayesian optimal designs for phase I clinical trials. Biometrics. 2003; 59: 591–600. 17. Y. Zhou, Choice of designs and doses for early phase trial. Fundam Clin Pharmacol. 2004; 18: 1–7. 18. J. Whitehead, Y. Zhou, S. Patterson, N. D. Webber, and S. Francis, Easy-to-implement Bayesian methods for dose-escalation studies in healthy volunteers. Biostatistics. 2001; 2: 47–61. 19. S. Zohar, A. Latouche, M. Tacconet, and S. Chevret, Software to compute and conduct sequential Bayesian phase I and II doseranging clinical trials with stopping rules. Comput Methods Program Biomed. 2003; 72: 117–125. 20. P. F. Thall and J. D. Cook, Dose-finding based on efficacy-toxicity trade-offs. Biometrics. 2004; 60: 684–693. 21. P. F. Thall and J. D. Cook, Using both efficacy and toxicity for dose-finding In: S. Chevret (ed.), Statistical Methods in DoseFinding Experiments. Chicester, UK: Wiley, 2006, pp. 275–285. 22. W. Zhang, D. J. Sargent, S. Mandrekar, An adaptive dose-finding design incorporating both toxicity and efficacy. Stat Med. 2006; 25: 9243–9249. 23. S. Chevet, ed. Statistical Methods for DoseFinding Experiments. Chicester, UK: Wiley, 2006. 24. M. Gasparini and J. Eisele, A curve-free method for phase I clinical trials. Biometrics. 2000; 56: 609–615. 25. M. Stylianou and N. Flourny, Dose-finding using the biased coin up-and-down design and istonic regression. Biometrics. 2002; 58: 171–177.

8

APPLICATION OF NOVEL DESIGNS IN PHASE I TRIALS

FURTHER READING

CROSS-REFERENCES

J. O’Quigley, Dose-finding designs using the continual reassessment method In: J. Crowley (ed.), Handbook of Statistics in Clinical Oncology. New York: Marcel Dekker, 2001, pp. 35–72. B. Storer, Choosing a phase I design In: J. Crowley (ed.), Handbook of Statistics in Clinical Oncology. New York: Marcel Dekker, 2001, pp. 73–92.

Continual reassessment method Dose escalation design Phase I trials Bayesian approach

ASCOT TRIAL

trial of a lipid-lowering agent (lipid-lowering arm, ASCOT-LLA, 5). Between 1998 and 2000, patients were recruited to an independent, investigator-inititated, investigatorled, multicenter, prospective, randomizedcontrolled trial. Patients were eligible for ASCOT-BPLA if they were aged 40–79 years at randomization, and had either untreated hypertension with a systolic blood pressure of 160 mm Hg or more, diastolic blood pressure of 100 mm Hg or more, or both, or they were treated for hypertension with systolic blood pressure of 140 mm Hg or more, diastolic blood pressure of 90 mm Hg or more, or both. In addition, the patients had to have at least three other cardiovascular risk factors and they were to be free of any previous cardiac events or current cardiac disease

JESPER MEHLSEN Frederikesberg Hospital—Clinical Physiology & Nuclear Medicine Frederiksberg, Denmark

One of the first randomized, placebocontrolled trials in hypertensive subjects (1) demonstrated that whereas treatment dramatically reduced mortality and the incidence of strokes, it did not prevent myocardial infarction. This paradox was confirmed by metaanalyses of randomized trials—all of which used standard diuretic and/or betablocker therapy—showing treatment effects similar to those predicted by prospective, observational studies on stroke incidence but not on the occurrence of coronary heart disease (CHD; 2). It has been speculated that adverse effects of the older antihypertensive drugs on serum lipids, glucose, and potassium could blunt the cardioprotection conferred by lowering of blood pressure. 1

2.1 Endpoints Primary objectives were as follows: 1) to compare the effects on the combined outcome of non-fatal myocardial infarction (MI) and fatal CHD of a beta-blocker-regimen (atenolol) + if necessary a diuretic (bendroflumethiazide-K) with a calcium channel blocker-based regimen (amlodipine) + if necessary an ACE inhibitor (perindopril); 2) to compare the effect on the combined outcome of non-fatal MI and fatal CHD of a statin (atorvastatin) with that of placebo among hypertensive patients with total cholesterol < 6.5 mmol/L.

OBJECTIVES

The rationale for the ASCOT study (3) was thus to answer the issue of whether a newer combination of antihypertensive agents, a dihydropyridine calcium channel blocker (CCB) and an angiotensin converting enzyme (ACE) inhibitor, would produce greater benefits in terms of reducing CHD events than the standard beta-blocker/diuretic combination. The second main issue of ASCOT was whether lipid lowering with a statin would provide additional beneficial effects in those hypertensive patients with average or below average levels of serum cholesterol. 2

2.2 Drug Treatment The ASCOT-BPLA used forced titration of the study drugs with specific add-on drugs, the first of which was the alfa-blocker doxazosin, gastrointestinal transport system (GITS). The target for antihypertensive treatment was 140/90 mm Hg for nondiabetics and 130/80 mm Hg for patients with diabetes mellitus. The ASCOT-LLA used a standard dose of atorvastatin 10 mg for all patients assigned to active treatment without any specific target level for cholesterol.

STUDY DESIGN

ASCOT involved two treatment comparisons in a factorial design—a prospective, randomized, open, blinded endpoint design comparing two antihypertensive regimens (blood pressure-lowering arm, ASCOT-BPLA, 4) and, in a subsample of those hypertensives studied, a double-blind, placebo-controlled

2.3 Sample Size Assuming an adjusted yearly rate of nonfatal myocardial infarction and fatal CHD events

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

ASCOT TRIAL

of 1.42% and an intention-to-treat effect of 15% reduction in risk, it was estimated that a sample size of 18,000 was required to generate 1150 primary endpoints. This sample size would provide 80% power to detect such an effect. In the ASCOT-LLA, a 30% reduction in cholesterol was expected and assumed to translate into a reduction in nonfatal MI and fatal CHD of 30%. Under these conditions, a sample of 9000 patients would have 90% power to detect such an effect. 2.4 Data Safety Monitoring Plan The Data Safety Monitoring Committee (DSMC) did unblinded interim analysis during the trial and used the symmetric Haybittle–Peto statistical boundary (critical value Z = 3) as a guideline for deciding to recommend early termination of the trial. 3

RESULTS

In the Nordic countries, 686 family practices randomized patients, and in the United Kingdom and Ireland, 32 regional centers to which patients were referred by their family doctors recruited patients. A total of 19,342 patients were randomized, but 85 patients had to be excluded because of irregularities in blood-pressure measurements resulting in 19,257 evaluable patients. Participants were well matched between groups; over 80% were on previous antihypertensive treatment, they were mainly white and male, and had a mean age of 63 years, a mean body mass index (BMI) of almost 29 kg/m2 , a mean total cholesterol of 5.9 mmol/L, and a mean baseline sitting blood pressure of 164/95 mm Hg. The average number of the additional cardiovascular risk factors required for inclusion in the trial was 3.7. From the main study, 10,305 patients were found to be eligible for inclusion in ASCOT-LLA. In 2002, the data safety monitoring board (DSMC) recommended that ASCOT-LLA be stopped on the grounds that atorvastatin had resulted in a highly significant reduction in the primary endpoint of CHD events compared with placebo. All patients in the ASCOT-LLA were offered atorvastatin 10 mg daily to be continued to the end of ASCOTBPLA. In 2004, the DSMB recommended

the ASCOT-BPLA to be stopped as those patients allocated the atenolol-based regimen had significantly higher mortality as well as worse outcomes on several other secondary endpoints than those allocated the amlodipine-based regimen. ASCOT-LLA accumulated 33,041 patient years of follow-up (median, 3.3 years). At the close of follow-up for the lipid-lowering arm, complete information was obtained on 10,186 (98.8%) of the patients originally randomized. By the end of the study, 87% of patients originally assigned atorvastatin were still taking a statin, and 9% of those in the placebo group had been prescribed open-label statins. Comparing atorvastatin treatment to placebo, total and LDL-cholesterol were lowered by 1.0 mmol/L and 1.0 mmol/L (19% and 29%), respectively, and triglycerides were reduced by 14%. Changes in HDL-cholesterol concentrations were minimal and blood pressure control throughout the trial was similar in the two groups. The primary endpoint of nonfatal myocardial infarction and fatal CHD was significantly lower by 36% (hazard ratio 0.64 [95% CI 0.50–0.83], P = 0.0005) in the atorvastatin group than in the placebo group. There were also significant reductions in four of seven secondary endpoints, some of which incorporated the primary endpoint: total cardiovascular events including revascularization procedures (21%); total coronary events (29%); the primary endpoint excluding silent myocardial infarction (38%); and fatal and nonfatal stroke (27%). Effects of statin on the secondary endpoints of heart failure or cardiovascular mortality did not differ significantly from those of placebo. ASCOT-BPLA accumulated 106,153 patient years of follow-up (median, 5.5 years). In October, 2004, the DSMC recommended the trial be stopped on the grounds that compared with those allocated the amlodipinebased regimen, those allocated the atenololbased regimen had significantly higher mortality as well as worse outcomes on several other secondary endpoints. Complete endpoint information was collected at the end of the study for 18,965 patients (99%). On average, blood pressure dropped from a mean of 164.0/94.7 (SD 18.0/10.4) mm Hg to a mean of 136.9/78.3 (16.7/9.8) mm Hg. At the trial close

ASCOT TRIAL

out, 32% of patients with diabetes and 60% of those without had reached both the systolic and the diastolic blood pressure targets. Compared with those allocated the atenolol-based regimen, blood pressure values were lower in those allocated the amlodipine-based regimen and the average difference throughout the trial was 2.7/1.9 mm Hg. By the end of the trial, as intended by design, most patients (78%) were taking at least two antihypertensive agents. The primary endpoint of nonfatal myocardial infarction plus fatal CHD was nonsignificantly lowered by 10% in those allocated the amlodipine-based regimen compared with those allocated the atenolol-based regimen. There were significant reductions in most of the secondary endpoints (except fatal and nonfatal heart failure): nonfatal myocardial infarction (excluding silent myocardial infarction) and fatal CHD (reduced by 13%); total coronary events (13%); total cardiovascular events and procedures (16%); allcause mortality (11%); cardiovascular mortality (24%); and fatal and nonfatal stroke (23%). The difference in all-cause mortality was caused by the significant reduction in cardiovascular mortality, with no apparent difference in noncardiovascular mortality. Of the tertiary endpoints, there were significant reductions associated with the amlodipinebased regimen for unstable angina (32%), peripheral arterial disease (35%), development of diabetes (30%), and development of renal impairment (15%). Twenty-five percent of patients stopped therapy because of an adverse event, with no significant difference between the allocated treatment groups. 4

DISCUSSION & CONCLUSIONS

The lipid-lowering arm of ASCOT (ASCOTLLA) showed that cholesterol lowering with atorvastatin compared with placebo conferred a significant reduction in nonfatal myocardial infarction and in fatal CHD in hypertensive patients at moderate risk of developing cardiovascular events. Observational data have indicated a relatively weak association between serum cholesterol and the risk of stroke (6), but previous randomized trials of statin use have shown significant reductions in stroke events of the

3

same order of magnitude as in ASCOT-LLA (7). There were no significant adverse effects on any of the prespecified secondary or tertiary endpoints in association with the use of atorvastatin. The relative magnitude of the benefits in ASCOT-LLA are notably larger for CHD prevention than are the effects of blood-pressure lowering in randomized, placebo-controlled trials (2), whereas the relative reduction in stroke seems somewhat smaller. However, the results show the benefits of statin treatment are additional to those of good blood-pressure control. The findings support the concept that strategies aimed at reducing cardiovascular disease should depend on global assessment of risk, and that benefits of lipid lowering are present across the whole range of serum cholesterol concentrations. Subsequent economic analysis has indicated that adopting the treatment strategy used in ASCOT-LLA would be costeffective (8). ASCOT-BPLA showed that amlodipine-based treatment was superior to an atenolol-based therapy in hypertensive patients at moderate risk of developing cardiovascular events in terms of reducing the incidence of all types of cardiovascular events and all-cause mortality, and in terms of risk of subsequent new-onset diabetes. The effective blood pressure lowering achieved by the amlodipinebased regimen, particularly in the first year of follow-up, is likely to have contributed to the differential cardiovascular benefits. However, the systolic blood pressure difference observed would, based on previous randomized trials (2) and by observational studies (9), be expected to generate a difference in coronary events and in strokes far below that achieved. A large substudy in ASCOT, the Conduit Artery Function Evaluation (CAFE) Study (10), showed that the two drug regimens had substantially different effects on central aortic pressures and hemodynamics despite a similar impact on brachial blood pressure in those included. The study indicated that differences in central aortic pressures could be a potential mechanism to explain the different clinical outcomes between the two blood pressure treatment arms in ASCOT.

4

ASCOT TRIAL

Another explanation for the difference observed could possibly be found in a prespecified assessment of whether any synergistic effects were apparent between the lipid-lowering and blood-pressure-lowering regimens (11). This analysis revealed that atorvastatin reduced the relative risk of CHD events by 53% (P < 0.0001) among those allocated the amlodipine-based regimen, but nonsignificantly (16%) among those allocated the atenolol-based regimen (P < 0.025 for heterogeneity). A significant excess of new-onset diabetes was observed in those allocated the atenololbased regimen and is compatible with the results of previous studies (12). The effect on short-term cardiovascular outcomes of individuals who became diabetic during the course might not be apparent compared with those who did not develop diabetes, although adverse outcomes associated with type 2 diabetes could reasonably be expected with extended follow-up. The ASCOT-BPLA reaffirm that most hypertensive patients need at least two agents to reach recommended blood pressure targets, and that most can reach current targets if suitable treatment algorithms are followed. Economic analysis has indicated that amlodipine-based therapy would be costeffective when compared with the atenololbased therapy (13). Both arms of the ASCOT trial have impacted on current recommendations for the treatment of hypertension. The European guidelines (14) now recommend the addition of a statin to the antihypertensive treatment in hypertensive patients aged less than 80 years who have an estimated 10 years risk of cardiovascular disease of more than 20% or of cardiovascular death of 5% or more. The British National Institute for Health and Clinical Excellence (15) no longer recommends beta-blockers as a first-line therapy, and the European guidelines express several reservations regarding the use of beta-blockers in hypertension particularly in combination with a thiazide diuretic. REFERENCES 1. VA Cooperative Study Group, Effects of treatment on morbidity in hypertension. II. Results

in patients with diastolic blood pressure averaging 90 through 114mm Hg. JAMA 1970; 213: 1143–1152. 2. Blood Pressure Lowering Treatment Trialists’ Collaboration, Effects of different blood pressure lowering regimens on major cardiovascular events: Results of prospectively designed overviews of randomised trials. Lancet 2003; 362: 1527–1535. 3. Sever PS, Dahl¨of B, Poulter NR, et al. Rationale, design, methods and baseline demography of participants of the Anglo-Scandinavian Cardiac Outcomes Trial. J. Hypertens. 2001; 6: 1139–1147. 4. Dahl¨of B, Sever PS, Poulter NR, et al. Prevention of cardiovascular events with an antihypertensive regimen of amlodipine adding perindopril as required versus atenolol adding bendroflumethiazide as required, in the Anglo-Scandinavian Cardiac Outcomes Trial-Blood Pressure Lowering Arm (ASCOTBPLA): A multicentre randomised controlled trial. Lancet 2005; 366: 895–906. 5. Sever PS, Dahlof B, Poulter NR, et al. Prevention of coronary and stroke events with atorvastatin in hypertensive patients who have average or lower-than-average cholesterol concentrations, in the Anglo-Scandinavian Cardiac Outcomes Trial-Lipid Lowering Arm (ASCOT-LLA): A multicentre randomised controlled trial. Lancet 2003; 361: 1149–1158. 6. Eastern Stroke and Coronary Heart Disease Collaborative Research Group. Blood pressure, cholesterol, and stroke in eastern Asia. Lancet 1998; 352: 1801–1807. 7. Crouse JR III, Byington RP, Furberg CD. HMG-CoA reductase inhibitor therapy and stroke risk reduction: An analysis of clinical trials data. Atherosclerosis 1998; 138: 11–24. 8. Lindgren P, Buxton M, Kahan T, et al. Costeffectiveness of atorvastatin for the prevention of coronary and stroke events: an economic analysis of the Anglo-Scandinavian Cardiac Outcomes Trial–lipid-lowering arm (ASCOTLLA). Eur. J. Cardiovasc. Prev. Rehabil. 2005; 12: 29–36. 9. Lewington S, Clarke R, Qizilbash N, et al. Age-specific relevance of usual blood pressure to vascular mortality: A meta-analysis of individual data for one million adults in 61 prospective studies. Lancet 2002; 360: 1903–1913. 10. Williams B, Lacy PS, Thom SM, et al. Differential Impact of Blood Pressure–Lowering Drugs on Central Aortic Pressure and Clinical Outcomes: Principal Results of the Conduit

ASCOT TRIAL Artery Function Evaluation (CAFE) Study. Circulation 2006; 113: 1213–1225. 11. Sever P, Dahl¨of B, Poulter N, et al. Potential synergy between lipid-lowering and bloodpressure-lowering in the Anglo-Scandinavian Cardiac Outcomes Trial. Eur. Heart J. 2006; 27: 2982–2988. 12. Opie LH, Schall R. Old antihypertensives and new diabetes. J. Hypertens. 2004; 22: 1453–1458. 13. Lindgren P, Buxton M, Kahan K, et al. Economic Evaluation of ASCOT-BPLA: Antihypertensive treatment with an amlodipinebased regimen is cost-effective compared to an atenolol-based regimen. Heart Online, October 2007. 14. The Task Force for the Management of Arterial Hypertension of the European Society of Hypertension (ESH) and of the European Society of Cardiology (ESC). 2007 Guidelines for the Management of Arterial Hypertension. J. Hypertens. 2007; 25: 1105–1187. 15. National Institute of Clinical Excellence. Hypertension: management of hypertension in adults in primary care. 2006; Available from: http://www.nice.org.uk/ nicemedia/pdf/CG034NICEguideline.pdf.

5

ASSAY SENSITIVITY

estimate for the primary endpoint, one can calculate the power at any effect size, yielding the power function. This power function takes a value equal to the type I error rate when the effect size is 0 and a value equal to the desired power when the effect size is
. Because the goal of a superiority trial is to detect a difference, high assay sensitivity will help a trial achieve its objective in the presence of a true treatment difference. In such a trial, the power function can help provide some quantification for assay sensitivity. The higher the power to detect a treatment effect is, the greater the assay sensitivity is. For a superiority trial, several factors could reduce the true treatment effect or increase the variability, leading to a smaller signal-to-noise ratio when comparing an effective treatment with a less effective or an ineffective one. These factors include, but are not limited to:

CHRISTY CHUANG-STEIN Pfizer Global Research and Development Kalamazoo, Michigan

A good source for assay sensitivity is the International Conference on Harmonisation of Technical requirements for Registration of Pharmaceuticals for Human Use (ICH) guidance on Choice of Control Group and Related Issues in Clinical Trials (E10) (1). According to ICH E10, the assay sensitivity of a clinical trial is defined as the trial’s ability to distinguish an effective treatment from a less effective or ineffective one. Since it is always important for a trial to tell treatments of different effects apart, assay sensitivity is an important consideration when designing and conducting a trial. The lack of adequate assay sensitivity has different implications for trials designed to demonstrate superiority and trials designed to demonstrate noninferiority as we will discuss below. 1

1. High measurement or ascertainment errors caused by the use of multiple equipments, multiple technicians, and multiple raters that produce different results. 2. Poor compliance with therapy such as study medication administration, use of concomitant medications, visit schedule, sample collection, and data recording. 3. Large patient dropout rate that makes analysis challenging and renders findings suspicious. 4. The enrollment population is not what the effective treatment will benefit the most. 5. The dose or the administration schedule is not optimal, which results in a suboptimal exposure. 6. A large placebo effect in the enrolled population makes it hard to demonstrate a treatment benefit. 7. Poorly or inconsistently applied diagnostic criteria that result in the inclusion of subjects who might not have the disorder under investigation. 8. Treatment randomization that fails to take into consideration crucial baseline covariates, leading to imbalance

SUPERIORITY TRIAL

The primary objective of a superiority trial is to demonstrate the superiority of a treatment over a concurrent comparator. The comparator could be a placebo or another treatment. The primary endpoint could be related to efficacy, safety, or benefit/risk. When determining the sample size for a superiority study, a trialist needs to consider the type I error (false-positive) rate and the power (one minus the false-negative rate) at a particular effect size. The allowable type I error rate, typically set to be 0.05 for a two-sided test in a confirmatory trial, determines the level at which statistical significance will be claimed. As for statistical power, a prespecified requirement (80% or 90%) at a prespecified effect size (
) gives the probability that we will conclude a treatment effect difference if the treatment effect difference is truly of the amount
. Given a sample size, the type I error rate, and the variability associated with the

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

ASSAY SENSITIVITY

between treatment groups with respect to important covariates. 9. Lack of clarity in the protocol, leaving investigators to interpret the requirements themselves and leading to discrepancies in implementation. When a superiority trial concludes a treatment effect difference, the finding demonstrates that the trial has assay sensitivity. If the trial fails to conclude a treatment effect difference, it could be because of the absence of a true treatment effect difference. It could also be that the study is lacking adequate assay sensitivity because of one or more of the reasons listed above. Since the latter is something that a trialist could control with proper steps, it is important that a trialist ensures adequate assay sensitivity through careful design and excellent execution. By comparison, it is hard to change the innate property of a drug candidate. 2

NONINFERIORITY TRIALS

For convenience, we will focus our discussion on noninferiority trials (2) even though many of the comments are equally applicable to equivalence trials. Here, we assume that a noninferiority decision will be based on a confidence interval constructed for the treatment effect difference. Furthermore, we assume that a positive value for the difference signals a beneficial treatment effect (for the new treatment). Noninferiority will be concluded if the lower confidence limit exceeds a predetermined quantity. When noninferiority in efficacy is being investigated, the hope is to confirm the efficacy of the new treatment over a placebo based on the proven efficacy of the active control (over a concurrent placebo) observed in previous trials. The idea is that if the efficacy of the new treatment is ‘‘reasonably’’ close to that of the active control and the latter is believed to have a better efficacy than a placebo, then one could conclude the efficacy of the new treatment over a placebo if a placebo had been included in the trial. To make the above work, it is critical that the active control successfully and consistently demonstrated a benefit over a concurrent placebo in past trials. In other words,

historical evidence of sensitivity to drug effects is available showing that similarly designed trials regularly distinguished the active control from a placebo. This condition, although sounds simple, is not always true. In fact, we know this condition often fails for disorders such as depression. For depression, a non-negligible chance exists that an effective treatment fails to demonstrate efficacy over a placebo comparator in a given trial. Even when historical evidence for assay sensitivity exists, the effect over the concurrent placebo is not necessarily similar from trial to trial. When this happens, no assurance is given that similarly designed noninferiority trials involving the active control will have similar assay sensitivity. All trials should be planned carefully. A trialist should consider all things that could possibly go wrong and make previsions for them at the design stage. On the other hand, the appropriateness of trial conduct could only be fully evaluated after the trial is completed. To extrapolate the efficacy of the active control observed in previous trials to the current one, the current trial should be similar to those from which historical evidence has been drawn. For example, we should have similar inclusion/exclusion criteria, similar allowance for concomitant medications, similar measurement process, and similar follow-up and data collection/analysis methods. Since historical evidence for assay sensitivity only applies to similarly designed trials, the current trial should be conducted with the same high quality as the previous ones. 3 IMPACT OF SLOPPY TRIAL EXECUTION Careful conduct of a clinical trial according to the protocol has a major impact on the credibility of the results (3). Sloppy trial conduct affects both noninferiority (4,5) and superiority trials, albeit differently. Since the primary objective of a noninferiority trial is to demonstrate the absence of a difference, the lack of assay sensitivity could reduce the treatment effect difference and increase the chance of erroneously concluding noninferiority. On the other hand, the variability could be increased as a result of increased sources

ASSAY SENSITIVITY

of variability. The variability contributes to a greater confidence interval width, which in turn could reduce our chance to conclude noninferiority correctly. The above discussion is further complicated in the case of a binary endpoint (6). Use the case of an antibiotic trial where the success rate of the new treatment and the positive control is expected to be around 70%. If the observed rates are close to 50%, this would result in a higher than expected variability when constructing the confidence interval for the difference in the success rates. Several factors could play against each other in regard to the noninferiority decision. Clearly, the observed treatment effect is a major factor. However, the magnitude of the change in the point estimate for treatment effect needs to be evaluated against a likely increase in the variability for their combined effect on the final conclusion. Koch and Rohmel (7) conducted a simulation study to evaluate the impact of sloppy study conduct on noninferiority studies. They showed that sloppy study conduct could affect the rate of false-positive conclusions in noninferiority trials. In their investigations, Koch and Rohmel examined the following four factors: (1) the objective of the trial; (2) the population analyzed and the mechanism for imputation of nonevaluable measurements; (3) the mechanism generating nonevaluable measurements; and (4) the selected measure for treatment effect. They encouraged researchers to perform simulation studies routinely to investigate the effect of these often counteracting factors in practical situations. By comparison, sloppy study conduct in a superiority trial generally leads to a decrease in assay sensitivity, and therefore, it reduces our chance to conclude superiority even if the treatment of interest is better than the comparator. 4

DESIGNS INCLUDING THREE ARMS

A design using three arms is used increasingly to help gauge assay sensitivity in a trial (8). This applies to trials designed from the superiority or noninferiority perspective. The three arms discussed here are placebo, an active control, and a new treatment.

3

In a superiority trial, the inclusion of an active control helps decide whether a study has adequate assay sensitivity. If the study can conclude a significant treatment effect between the active control (known to have an effect) and the placebo, one can conclude that the study has adequate assay sensitivity. If the new treatment fails to demonstrate a significant effect over the placebo in the same study, one could conclude that the new treatment, as applied to the patients in the study, is not likely to possess the kind of treatment effect used to size the study at the design stage. The three-arm design has gained popularity for proof of concept or dose-ranging studies also. Since most product candidates entering into clinical testing will fail, many drug development programs are designed so that the sponsors could terminate a program early if the candidate is not likely to meet the required commercial profile. To this end, the inclusion of an active control in an early trial (proof of concept or dose-ranging) helps answer the question of whether the observed effect of the new treatment is reasonably close to the truth based on the observed response to the active control. Similarly, for situations where product differentiation is important, including an active control allows the sponsor to compare the new treatment head-to-head with the active control after efficacy of the new treatment is confirmed. In some cases, a sponsor might cease further development of a new treatment because it does not meet the commercial requirements relative to the active control even if the new treatment demonstrates efficacy over a placebo. A classic noninferiority trial designed from the efficacy perspective does not include a placebo. The basis for this is typically because of ethical consideration. For example, in trials involving patients with life-threatening conditions for which treatments are available, it will be unethical to treat patients with a placebo. For confirmatory trials that do contain a placebo and an active control, the primary analysis is typically to compare the new treatment with the placebo (9). A secondary analysis is to compare the new treatment with the active control for product differentiation.

4

ASSAY SENSITIVITY

In some situations, the primary objective of the trial is to compare the new treatment with a placebo on safety to rule out a safety signal. The design is basically a noninferiority design. The hope is to conclude that the primary safety endpoint associated with the new treatment is within a certain limit of that associated with the placebo. This is the case with the ‘‘thorough QT/QTc’’ study discussed in ICH E14 (10). ICH E14 states that the ‘‘thorough QT/QTc’’ study is intended to determine whether a drug candidate under development has a threshold pharmacologic effect on cardiac repolarization as detected by QT/QTc prolongation. Such a study, which is typically conducted early in a clinical development program, is intended to provide maximum guidance for the collection of ECG data in later trials. For example, a negative ‘‘thorough QT/QTc’’ study will often allow the collection of on-therapy ECGs in accordance with the current practices in each therapeutic area. On the other hand, a positive ‘‘thorough QT/QTc study’’ will almost always lead to an expanded ECG safety evaluation during later stages of drug development. Because of the critical role of this study, the confidence in the study’s findings can be greatly enhanced by the use of a concurrent positive control (pharmacological or nonpharmacological) to establish assay sensitivity. Detecting the positive control’s effect in the study will establish the ability of the study to detect an effect similar to that of the positive control. A common choice of the positive control is a drug or an intervention that has an effect on the mean QT/QTc interval of about 5 ms. Positive controls play a similar role in the assessment of mutagenicity studies. According to Hauschke et al. (11), the classification of an experiment as being negative or positive should be based also on the magnitude of the responses in the positive control. In such studies, the determination of the maximum safe dose is often done by incorporating a biologically meaningful threshold value, which is expressed as a fraction of the difference between positive and vehicle control. Therefore, the inclusion of a positive control not only determines the quality of the experiment qualitatively, but it is also part of the summary results involving a new compound in a quantitative manner.

In summary, assay sensitivity is a critical factor to the success of a trial, whether the trial is designed from the efficacy or the safety perspective. REFERENCES 1. International Conference on Harmonisation E10. (2000). Choice of control group and related issues in clinical trials. Step 5. Available: http://www.ich.org/cache/compo/276254-1.html. 2. Hung HMJ, Wang SJ, O’Neill R. (2006). Noninferiority trials. Wiley Encyclopedia of Clinical Trials. 3. International Conference on Harmonisation E9. (1998). Statistical principles for clinical trials. Step 5. Available: http://www.ich.org/ cache/compo/276-254-1.html. 4. Jones B, Jarvis P, Lewis JA, Ebbutt AF. Trials to assess equivalence: the importance of rigorous methods. Br. Med. J. 1996; 313: 36–39. 5. Sheng D, Kim MY. The effect of noncompliance on intent-to-treat analysis of equivalence trials. Stat Med. 2006; 25: 1183–1199. 6. Chuang-Stein C. Clinical equivalence—a clarification. Drug Information J. 1999; 33: 1189–1194. 7. Koch A, Rohmel J. The impact of sloppy study conduct on non-inferiority studies. Drug Information J. 2002; 36: 3–6. 8. Chuang C, Sanders C, Snapinn S. An industry survey on current practice in the design and analysis of active control studies. J. Biopharm. Stat. 2004; 14(2): 349–358. 9. Koch A, Rohmel J. Hypothesis testing in the ‘‘gold standard’’ design for proving the efficacy of an experimental treatment relative to placebo and a reference. J. Biopharm. Stat. 2004; 14(2): 315–325. 10. International Conference on Harmonisation E14. (2005). The clinical evaluation of QT/QTc interval prolongation and proarrhythmic potential for non-antiarrhythmic drugs. Step 4 document. Available: http://www.ich.org/ cache/compo/276-254-1.html. 11. Hauschke D, Slacik-Erben R, Hensen S, Kaufmann R. Biostatistical assessment of mutagenicity studies by including the positive control. Biometrical J. 2005; 47: 82–87.

ASSESSMENT BIAS

when they assessed the effect openly but not when they assessed the effect blindly in the same patients (4). Some outcomes can only be meaningfully evaluated by the patients (e.g., pain and well-being). Unfortunately, blinding patients effectively can be very difficult, which is why active placebos are sometimes used. The idea behind an active placebo is that patients should experience side effects of a similar nature as when they receive the active drug, although it contains so little of a drug that it can hardly cause any therapeutic effect. As lack of blinding can lead to substantial bias, it is important in blinded trials to test whether the blinding has been compromised. Unfortunately, this test is rarely done (Asbjørn Hr´objartsson, unpublished observations), and, in many cases, double-blinding is little more than window dressing. Some outcome assessments are not made until the analysis stage of the trial (see below). Blinding should, therefore, also be used during data analysis, and it should ideally be preserved until two versions of the manuscript— written under different assumptions which of the treatments is experimental and which is control—have been approved by all the authors (5).

PETER C. GØTZSCHE Nordic Cochrane Centre Rigshospitalet, København Ø, Denmark

1

INTRODUCTION

Assessment bias in a clinical trial occurs if bias in the assessment of the outcome exists. It is also called ascertainment bias, diagnostic bias, or detection bias (1). A major cause of assessment bias is lack of blinding. Other problems relate to differential identification of harmless or false-positive cases of disease, bias in assessment of disease-specific mortality, the use of composite outcomes, competing risks, timing of the assessments, and bias in assessment of harms. 2

LACK OF BLINDING

One of the most important and most obvious causes of assessment bias is lack of blinding. In empirical studies, lack of blinding has been shown to exaggerate the estimated effect by 14%, on average, measured as odds ratio (2). These studies have dealt with a variety of outcomes, some of which are objective and would not be expected to be influenced by lack of blinding (e.g., total mortality). When patient reported outcomes are assessed, lack of blinding can lead to far greater bias than the empirical average. An example of a highly subjective outcome is the duration of an episode of the common cold. A cold does not stop suddenly, and awareness of the treatment received could therefore bias the evaluation. In a placebo-controlled trial of Vitamin C, the duration seemed to be shorter when an active drug was given, but many participants had guessed they received the vitamin because of its taste (3). When the analysis was restricted to those who could not guess what they had received, the duration was not shorter in the active group. Assessments by physicians are also vulnerable to bias. In a trial in multiple sclerosis, neurologists found an effect of the treatment

3 HARMLESS OR FALSE-POSITIVE CASES OF DISEASE Assessment bias can occur if increased diagnostic activity leads to increased diagnosis of true, but harmless cases of disease. Many stomach ulcers are silent (i.e., they come and go and give no symptoms). Such cases could be detected more frequently in patients who receive a drug that causes unspecific discomfort in the stomach. Similarly, if a drug causes diarrhea, it could lead to more digital, rectal examinations, and therefore also to the detection of more cases of prostatic cancer, most of which would be harmless, because many people die with prostatic cancer but rather few die from prostatic cancer. Assessment bias can also be caused by differential detection of false-positive cases

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

ASSESSMENT BIAS

of disease. Considerable observer variation often exists with common diagnostic tests. For gastroscopy, for example, a kappa value of 0.54 has been reported for the interobserver variation in the diagnosis of duodenal ulcers (6), which usually means that rather high rates of both false-positive findings and false-negative findings occur. If treatment with a drug leads to more gastroscopies because ulcers are suspected, one would therefore expect to find more (false) ulcers in patients receiving that drug. A drug that causes unspecific, non-ulcer discomfort in the stomach could therefore falsely be described as an ulcer-inducing drug. The risk of bias can be reduced by limiting the analysis to serious cases that would almost always become known (e.g., cases of severely bleeding ulcers requiring hospital admission or leading to death). 4

DISEASE-SPECIFIC MORTALITY

Disease-specific mortality is very often used as the main outcome in trials without any discussion of how reliable it is, even in trials of severely ill patients in which it can be difficult to ascribe particular causes for the deaths with acceptable error. Disease-specific mortality can be highly misleading if a treatment has adverse effects that increases mortality from other causes. It is only to be expected that aggressive treatments can have such effects. Complications to cancer treatment, for example, cause mortality that is often ascribed to other causes, although these deaths should have been added to the cancer deaths. A study found that deaths from other causes than cancer were 37% higher than expected and that most of this excess occurred shortly after diagnosis, suggesting that many of the deaths were attritutable to treatment (7). The use of blinded endpoint committees can reduce the magnitude of misclassification bias, but cannot be expected to remove it. Radiotherapy for breast cancer, for example, continues to cause cardiovascular deaths even 20 years after treatment (8), and it is not possible to distinguish these deaths from cardiovascular deaths from other causes. Furthermore, to work in an unbiased

way, death certificates and other important documents must have been completed and patients and documents selected for review without awareness of status, and it should not be possible to break the masking during any of these processes, including review of causes of death. This result seems difficult to obtain, in particular because those who prepare excerpts of the data should be kept blind to the research hypothesis (1). Fungal infections in cancer patients with neutropenia after chemotherapy or bonemarrow transplantation is another example of bias in severely ill patients. Not only is it difficult to establish with certainty that a patient has a fungal infection and what was the cause of death, but evidence also exists that some of drugs (azole antifungal agents) may increase the incidence of bacteriaemias (9). In the largest placebocontrolled trial of fluconazole, more deaths were reported on drug than on placebo (55 vs 46 deaths), but the authors also reported that fewer deaths were ascribed to acute systemic fungal infections (1 vs 10 patients, P = 0.01) (10). However, if this subgroup result is to be believed, it would mean that fluconazole increased mortality from other causes (54 vs 36 patients, P = 0.04). Bias related to classification of deaths can also occur within the same disease. After publication of positive results from a trial in patients with myocardial infarction (11), researchers at the U.S. Food and Drug Administration found that the causeof-death classification was ‘‘hopelessly unreliable’’ (12). Cardiac deaths were classified into three groups: sudden deaths, myocardial infarction, or other cardiac event. The errors in assigning cause of death nearly all favored the conclusion that sulfinpyrazone decreased sudden death, the major finding of the trial. 5 COMPOSITE OUTCOMES Composite outcomes are vulnerable to bias when they contain a mix of objective and subjective components. A survey of trials with composite outcomes found that when they included clinician-driven outcomes, such as hospitalization and initiation of new antibiotics, in addition to objective outcomes such

ASSESSMENT BIAS

as death, it was twice as likely that the trial reported a statistically significant effect (13). 6

COMPETING RISKS

Composite outcomes can also lead to bias because of competing risks (14), for example, if an outcome includes death as well as hospital admission. A patient who dies cannot later be admitted to hospital. This bias can also occur in trials with simple outcomes. If one of the outcomes is length of hospital stay, a treatment that increases mortality among the weakest patients who would have had long hospital stays, may spuriously appear to be beneficial. 7

given in the protocol, the conversion of these data into publishable bits of information can be difficult and often involves subjective judgments. Particularly vulnerable to assessment bias is exclusion of reported effects because they are not felt to be important, or not felt to be related to the treatment. Trials that have been published more than once illustrate how subjective and biased assessment of harms can be. Both number of adverse effects and number of patients affected can vary from report to report, although no additional inclusion of patients or follow-up have occurred, and these re-interpretations or reclassifications sometimes change an insignificant difference into a significant difference in favor of the new treatment (16).

TIMING OF OUTCOMES

Timing of outcomes can have profound effects on the estimated result, and the selection of time points for reporting of the results is often not made until the analysis stage of the trials, when possible treatment codes have been broken. A trial report of the anti-arthritic drug, celecoxib, gave the impression that it was better tolerated than its comparators, but the published data referred to 6 months of follow-up, and not to 12 and 15 months, as planned, when little difference existed; in addition, the definition of the outcome had changed, compared with what was stated in the trial protocol (15). Trials conducted in intensive care units are vulnerable to this type of bias. For example, the main outcome in such trials can be total mortality during the stay in the unit, but if the surviving patients die later, during their subsequent stay at the referring department, little may be gained by a proven mortality reduction while the patients were sedated. A more relevant outcome would be the fraction of patients who leave the hospital alive. 8

3

ASSESSMENT OF HARMS

Bias in assessment of harms is common. Even when elaborate, pretested forms have been used for registration of harms during a trial, and guidelines for their reporting have been

REFERENCES 1. A. R. Feinstein, Clinical Epidemiology. Philadelphia: Saunders, 1985. ¨ 2. P. Juni, D. G. Altman, and M. Egger, Systematic reviews in health care: assessing the quality of controlled clinical trials. BMJ 2001; 323: 42–46. 3. T. R. Karlowski, T. C. Chalmers, L. D. Frenkel, A. Z. Kapikian, T. L. Lewis, and J. M. Lynch, Ascorbic acid for the common cold: a prophylactic and therapeutic trial. JAMA 1975; 231: 1038–1042. 4. J. H. Noseworthy et al., The impact of blinding on the results of a randomized, placebocontrolled multiple sclerosis clinical trial. Neurology 1994; 44: 16–20. 5. P. C. Gøtzsche, Blinding during data analysis and writing of manuscripts. Controlled Clin. Trials 1996; 17: 285–290. 6. T. Gjørup et al., The endoscopic diagnosis of duodenal ulcer disease. A randomized clinical trial of bias and interobserver variation. Scand. J. Gastroenterol. 1986; 21: 561–567. 7. B. W. Brown, C. Brauner, and M. C. Minnotte, Noncancer deaths in white adult cancer patients. J. Natl. Cancer Inst. 1993; 85: 979–987. 8. Early Breast Cancer Trialists’ Collaborative Group, Favourable and unfavourable effects on long-term survival of radiotherapy for early breast cancer: an overview of the randomised trials. Lancet 2000; 355: 1757–1770. 9. P. C. Gøtzsche and H. K. Johansen, Routine versus selective antifungal administration for

4

ASSESSMENT BIAS

control of fungal infections in patients with cancer (Cochrane Review). In: The Cochrane Library, Issue 3. Oxford: Update Software, 2003. 10. J. L. Goodman et al., A controlled trial of fluconazole to prevent fungal infections in patients undergoing bone marrow transplantation. N. Engl. J. Med. 1992; 326: 845–851. 11. The Anturane Reinfarction Trial Research Group, Sulfinpyrazone in the prevention of sudden death after myocardial infarction. N. Engl. J. Med. 1980; 302: 250–256. 12. R. Temple and G. W. Pledger, The FDA’s critique of the anturane reinfarction trial. N. Engl. J. Med. 1980; 303: 1488–1492. 13. N. Freemantle, M. Calvert, J. Wood, J. Eastaugh, and C. Griffin, Composite outcomes in randomized trials: greater precision but with greater uncertainty? JAMA 2003; 289: 2554–2559. 14. M. S. Lauer and E. J. Topol, Clinical trials - multiple treatments, multiple end points, and multiple lessons. JAMA 2003; 289: 2575–2577. ¨ 15. P. Juni, A. W. Rutjes, and P. A. Dieppe, Are selective COX 2 inhibitors superior to traditional non steroidal anti-inflammatory drugs? BMJ 2002; 324: 1287–1288. 16. P. C. Gøtzsche, Multiple publication in reports of drug trials. Eur. J. Clin. Pharmacol. 1989; 36: 429–432.

ASSESSMENT OF HEALTH-RELATED QUALITY OF LIFE

(7) Role-Emotional, and (8) Mental Health. These eight domains can be further summarized by two summary scales: Physical Component Summary (PCS) and Mental Component Summary (MCS) scales. This article is intended to provide an overview of assessment of HRQOL in clinical trials. For more specific details on a particular topic mentioned in this article, the readers should consult the cited references. The development of a new HRQOL questionnaire and its translation into various languages are separate topics and are not covered in this article.

C. S. WAYNE WENG Department of Biomedical Engineering Chung Yuan Christian University Chungli, Taiwan

1

INTRODUCTION

Randomized clinical trials are the gold standard for evaluating new therapies. The primary focus of clinical trials has traditionally been evaluation of efficacy and safety. As clinical trials evolved from traditional efficacy and safety assessment of new therapies, clinicians were interested in an overall evaluation of the clinical impact of these new therapies on patient daily functioning and well-being as measured by health-related quality of life (HRQOL). As a result, HRQOL assessments in clinical trials have risen steadily throughout the 1990s and continue into the twentyfirst century. What is HRQOL? Generally, quality of life encompasses four major domains (1): 1. 2. 3. 4.

2

CHOICE OF HRQOL INSTRUMENTS

HRQOL instruments can be classified into two types: generic instruments and diseasespecific instruments. The generic instrument is designed to evaluate general aspects of a person’s HRQOL, which should include physical functioning, mental functioning, and social well-being. A generic instrument can be used to evaluate the HRQOL of a group of people in the general public or a group of patients with a specific disease. As such, data collected with a generic instrument allow comparison of HRQOL among different disease groups or against a general population. As a generic instrument, is designed to cover a broad range of HRQOL issues and it may be less sensitive regarding important issues for a particular disease or condition. Disease-specific instruments focus assessment in a more detailed manner for a particular disease. A more specific instrument allows detection of changes in disease-specific areas that a generic instrument is not sufficiently sensitive to detect. For example, the Health Assessment Questionnaire (HAQ) is developed to measure functional status of patients with rheumatic disease. The HAQ assesses the ability to function in eight areas of daily life: dressing and grooming, arising, eating, walking, hygiene, reach, grip, and activities. Table 1 (3–30) includes a list of generic and disease-specific HRQOL instruments for common diseases or conditions.

Physical status and functional abilities Psychological status and well-being Social interactions Economic or vocational status and factors

World Health Organization (WHO) defines ‘‘health’’ (2) as a ‘‘state of complete physical, mental, and social well-being and not merely the absence of infirmity and disease.’’ HRQOL focuses on parts of quality of life that are related to an individual’s health. The key components of this definition of HRQOL include (1) physical functioning, (2) mental functioning, and (3) social wellbeing, and a well-balanced HRQOL instrument should include these three key components. For example, the Medical Outcomes Study Short Form-36 (SF-36), a widely used HRQOL instrument, includes a profile of eight domains: (1) Physical Functioning, (2) Role-Physical, (3) Bodily Pain, (4) Vitality, (5) General Health, (6) Social Functioning,

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

ASSESSMENT OF HEALTH-RELATED QUALITY OF LIFE

Table 1. Generic and Disease-Specific HRQOL Instruments for Common Diseases or Conditions Generic HRQOL Instruments

Reference

Short Form-36 (SF-36) Sickness Impact Profile Nottingham Health Profile Duke Health Profile McMaster Health Index Questionnaire Functional Status Questionnaire WHO Quality of Life Assessment

3–5 6 7 8 9 10 11

Disease-Specific HRQOL Instruments: Disease Instrument

Reference

Pain

Brief Pain Inventory McGill Pain Questionnaire Visual Analogue Pain Rating scales

Depression

Beck Depression Inventory Center for Epidemiologic Studies Depression Scale (CES-D) Hamilton Rating Scale for Depression The Hospital Anxiety and Depression Questionnaire Zung Self-Rating Depression Scale WHO Well-Being Questionnaire Health Assessment Questionnaire (HAQ)

Rheumatic Disease: • Rheumatoid Arthritis • Osteoarthritis • Ankylosing Spondylitis • Juvenile Rheumatoid Arthritis Inflammatory Bowel Disease Asthma Airway Disease Seasonal Allergic Rhinitis Parkinson’s Disease

Cancer Both have tumor-specific modules

Inflammatory Bowel Disease Questionnaire (IBDQ) Asthma Quality of Life Questionnaire (AQLQ) St. George’s Respiratory Questionnaire (SGRQ) Rhinoconjunctivitis Quality of Life Questionnaire (RQLQ) Parkinson’s disease Questionnaire −39 item (PDQ-39) Parkinson’s Disease Quality of Life Questionnaire (PDQL) EORTC QLQ-C30 Functional Assessment of Cancer Therapy (FACT)

A comprehensive approach to assessing HRQOL in clinical trials can be achieved

12 13 Various authors; see Reference 14, p. 341 15 16 17 18 19 20 21

22, 23 24 25 26 27 28 29 30

relevant HRQOL components or a ‘‘module’’ approach, which includes a core measure of HRQOL domains supplemented in

using a battery of questionnaires when a single questionnaire does not address all

the same questionnaire by a disease- or treatment-specific set of items. The battery

ASSESSMENT OF HEALTH-RELATED QUALITY OF LIFE

approach combines a generic HRQOL instrument plus a disease-specific questionnaire. For example, in a clinical trial on rheumatoid arthritis (RA), one can include the SF-36 and HAQ to evaluate treatment effect on HRQOL. The SF-36 allows comparison of RA burden on patients’ HRQOL with other diseases as well as the general population. The HAQ, being a disease-specific instrument, measures patients’ ability to perform activities of daily life and is more sensitive to changes in a RA patient’s condition. The module approach has been widely adopted in oncology as different tumors impact patients in different ways. The most popular cancer-specific HRQOL questionnaires, EORTC QLQ-C3O and FACT, both include core instruments that measure physical functioning, mental functioning, and social well-being as well as common cancer symptoms, supplemented with a list of tumor- and treatment-specific modules. In certain diseases, a disease-specific HRQOL instrument is used alone in a trial because the disease’s impact on general HRQOL is so small that a generic HRQOL instrument will not be sensitive enough to detect changes in disease severity. For example, disease burden of allergic rhinitis on generic HRQOL is relatively small compared with the general population. Most published HRQOL studies in allergic rhinitis use Juniper’s Rhinoconjunctivitis Quality of Life Questionnaire (RQLQ), a disease-specific HRQOL questionnaire for allergic rhinitis. 3 ESTABLISHMENT OF CLEAR OBJECTIVES IN HRQOL ASSESSMENTS A clinical trial is usually designed to address one hypothesis or a small number of hypotheses, evaluating a new therapy’s efficacy, safety, or both. When considering whether to include HRQOL assessment in a study, the question of what additional information will be provided by the HRQOL assessment must be asked. As estimated by Moinpour (31), the total cost per patient is $443 to develop an HRQOL study, monitor HRQOL form submission, and analyze HRQOL data. Sloan et al. (32) have revisited the issue of the cost of HRQOL assessment in a number of settings including clinical trials and

3

suggest a wide cost range depending on the comprehensiveness of the assessment, which is not a trivial sum of money to be spent in a study without a clear objective in HRQOL assessment. The objective of HRQOL assessment is usually focused on one of the four possible outcomes: (1) improvement in efficacy leads to improvement in HRQOL, (2) treatment side effects may cause deterioration in HRQOL, (3) the combined effect of (1) and (2) on HRQOL, and (4) similar efficacy with an improved side effect profile leads to improvement in HRQOL. After considering possible HRQOL outcomes, one will come to a decision whether HRQOL assessment should be included in the trial. In many published studies, HRQOL was included in the studies without a clear objective. These studies generated HRQOL data that provided no additional information at the completion of the studies. Goodwin et al. (33) provide an excellent review of HRQOL measurement in randomized clinical trials in breast cancer. They suggest that, given the existing HRQOL database for breast cancer, it is not necessary to measure HRQOL in every trial, at least until ongoing trials are reported. An exception is interventions with a psychosocial focus where HRQOL has to be the primary outcome. 4

METHODS FOR HRQOL ASSESSMENT

The following components should be included in a study protocol with an HRQOL objective: – Rationale for assessing HRQOL objective(s) and for the choice of HRQOL instrument(s): – To help study personnel understand the importance of HRQOL assessment in the study, inclusion of a clear and concise rationale for HRQOL assessment is essential, along with a description of the specific HRQOL instrument(s) chosen. – HRQOL hypotheses: – The study protocol should also specify hypothesized HRQOL outcomes with respect to general and specific domains. It is helpful to identify

4

ASSESSMENT OF HEALTH-RELATED QUALITY OF LIFE

the primary domain and secondary domains for HRQOL analysis in the protocol. – Frequency of HRQOL assessment: – In a clinical trial, the minimum number of HRQOL assessments required is two, at baseline and the end of the study for studies with a fixed treatment duration, and most patients are expected to complete the treatment. One or two additional assessments should be considered between baseline and study endpoint depending on the length of the study so that a patient’s data will still be useful if endpoint data was not collected. More frequent assessments should be considered if the treatment’s impact on HRQOL may change over time. Three or more assessments are necessary to characterize patterns of change for individual patients. In oncology trials, it is common to assess HRQOL on every treatment cycle, as patients’ HRQOL is expected to change over time. However, assessment burden can be minimized if specific time points associated with expected clinical effects are of interest and can be specified by clinicians (e.g., assess HRQOL after the minimum number of cycles of therapy required to observe clinical activity of an agent). Another factor to be considered for the frequency of HRQOL assessment is the recall period for a particular HRQOL instrument. The recall period is the duration a subject is asked to assess his/her responses to an HRQOL questionnaire. The most common recall periods are one week, two weeks, and four weeks. – Administering HRQOL questionnaires: – To objectively evaluate HRQOL, one needs to minimize physician and study nurse influence on patient’s response to HRQOL questions. Therefore, the protocol should indicate that patients are to complete the HRQOL questionnaire in a quiet place in the doctor’s office at the beginning of his/her office

visit, prior to any physical examination and clinical evaluation by the study nurse and physician. – Specify the magnitude of difference in HRQOL domain score that can be detected with the planned sample size: – This factor is especially important when the HRQOL assessment is considered a secondary endpoint. As the sample size is based on the primary endpoint, it may provide only enough power to detect a relatively large difference in HRQOL scores. The question of whether to increase the sample size to cover HRQOL assessment often depends on how many more additional patients are needed and the importance of the HRQOL issue for the trial. To collect HRQOL data when power will be insufficient to detect effects of interest is a waste of clinical resources and the patient’s time. – Specify how HRQOL scores are to be calculated and analyzed in the statistical analysis section: – Calculation of HRQOL domain scores should be stated clearly, including how missing items will be handled. As a result of the nature of oncology studies, especially in late-stage disease, patients will stop treatment at different time points because of disease progression, intolerance to treatment side effects, or death and therefore fail to complete the HRQOL assessment schedule. For example, if data are missing because of deteriorating patient health, the study estimates of effect on HRQOL will be biased in favor of better HRQOL; the term ‘‘informative missing data’’ is the name for this phenomenon and must be handled with care. Fairclough (34) has written a book on various longitudinal methods to analyze this type of HRQOL data. However, analyzing and interpreting HRQOL data in this setting remain a challenge. – Strategies to improve HRQOL data collection:

ASSESSMENT OF HEALTH-RELATED QUALITY OF LIFE

↑ Education at the Investigators’ meeting and during site initiation visit: It is important to have investigators and study coordinators committed to the importance of HRQOL assessment. Without this extra effort, HRQOL assessment is likely to be unsuccessful, simply because collecting HRQOL data is not part of routine clinical trial conduct. – Emphasize the importance of the HRQOL data: ↑ Baseline HRQOL forms should be required in order to register a patient to a trial. Associate grant payment per patient with submission of a patient’s efficacy data. Specifying some portion of the grant payment with the submission of HRQOL form has significantly increased the HRQOL completion rate in the author’s clinical experience. • Establish a prospective reminder system for upcoming HRQOL assessments and a system for routine monitoring of forms at the same time clinical monitoring is being conducted. The following checklist (Table 2) may be helpful when considering inclusion of HRQOL assessment in a clinical trial protocol. 5

HRQOL AS THE PRIMARY ENDPOINT

To use HRQOL as the primary endpoint in a clinical trial, prior information must demonstrate at least comparable efficacy of a study treatment to its control. In this context, to design a study with HRQOL as the primary endpoint, the sample size will have to be large enough to assure adequate power to detect meaningful differences between treatment groups in HRQOL. Another context for a primary HRQOL endpoint is in the setting of treatment palliation. In this case, treatment efficacy is shown by the agent’s ability to palliate disease-related symptoms and overall HRQOL without incurring treatmentrelated toxicities. For example, patient report

5

of pain reduction can document the achievement of palliation [e.g., see Tannock et al. (35) example below]. A HRQOL instrument usually has several domains to assess various aspects of HRQOL. Some HRQOL instruments also provide for an overall or total score. The HRQOL endpoint should specify a particular domain, or the total score, as the primary endpoint of the HRQOL assessment in order to avoid multiplicity issues. If HRQOL is included as a secondary endpoint, it is a good practice to identify a particular domain as the primary focus of the HRQOL assessment. This practice forces specification of the expected outcomes of HRQOL assessments. Some investigators have applied multiplicity adjustments to HRQOL assessments. The approach may be statistically prudent, but it does not provide practical value. The variability of HRQOL domain scores is generally large. With multiple domains being evaluated, only a very large difference between groups will achieve the required statistically significance level. When evaluating HRQOL as a profile of a therapy’s impact on patients, clinical judgment of the magnitude of HRQOL changes should be more important than the statistical significance. However, this ‘‘exploratory’’ analysis perspective should also be tempered with recognition that some significant results may be marginally significant and subject to occurrence by chance. 6

INTERPRETATION OF HRQOL RESULTS

Two approaches have been used to interpret the meaningfulness of observed HRQOL differences between two treatment groups in a clinical trial: distribution-based and anchor-based approaches. The most widely used distribution-based approach is the effect size, among other methods listed in Table 3 (36–45). Based on the effect size, an observed difference is classified into (1) 0.2 = a small difference, (2) 0.5 = a moderate difference, and (3) 0.8 = a large difference. To advocate using the effect size to facilitate the interpretation of HRQOL data, Sloan et al. (46) suggested a 0.5 standard deviation as

6

ASSESSMENT OF HEALTH-RELATED QUALITY OF LIFE

Table 2. Checklist for HRQOL Assessment • • • • •

Rationale to assess HRQOL and the choice of HRQOL instrument(s) Hypothesis in terms of expected HRQOL outcomes Frequency of HRQOL assessment Procedures for administering HRQOL questionnaires Specify the magnitude of difference in HRQOL domain score that can be detected with the planned sample size • Specify in the statistical analysis section how HRQOL scores are to be calculated and analyzed • Strategies to improve HRQOL data collection

a reasonable benchmark for a 0–100 scale to be clinically meaningful. This suggestion is consistent with Cohen’s (47) suggestion of one-half of a standard deviation as indicating a moderate effect and therefore clinically meaningful. The anchor-based approach compares observed differences relative to an external standard. Investigators have used this approach to define the minimum important difference (MID). For example, Juniper and Guyatt (26) suggested that a 0.5 change in RQLQ be the MID (RQLQ score ranges from 1 to 7). Osoba (48) suggested that a 10-point change in EORTC QLQ-C30 questionnaire would be a MID. Both these two MIDs are group average scores. How these MIDs apply to individual patients is still an issue. Another issue in using MID is related to the starting point of patients’ HRQOL scores. Guyatt et al. (49) provides a detailed overview of various strategies to interpret HRQOL results. 7

EXAMPLES

7.1 HRQOL in Asthma To evaluate salmeterol’s effect on quality of life, patients with nocturnal asthma were enrolled into a double-blind, parallel group, placebo-controlled, multicenter study (50). The study rationale was that patients with nocturnal asthma who are clinically stable have been found to have poorer cognitive performance and poorer subjective and objective sleep quality compared with normal, healthy patients. To assess salmeterol’s effect on reducing the impact of nocturnal asthma on patients’ daily functioning and well-being, patients were randomized to

receive salmeterol, 42 µg, or placebo twice daily. Patients were allowed to continue theophylline, inhaled corticosteroids, and ‘‘asneeded’’ albuterol. Treatment duration was 12 weeks, with a 2-week run-in period. The primary study objective was to assess the impact of salmeterol on asthma-specific quality of life using the validated Asthma Quality of Life Questionnaire (24) (AQLQ). Patients were to return to the clinic every 4 weeks. Randomized patients were to complete an AQLQ at day 1; weeks 4, 8, 12; and at the time of withdrawal from the study for any reason. Efficacy (FEV1, PEF, nighttime awakenings, asthma symptoms, and albuterol use) and safety assessments were also conducted at these clinic visits. Scheduling HRQOL assessment prior to efficacy and safety evaluations at office visits minimizes investigator bias and missing HRQOL evaluation forms. The AQLQ is a 32-item, self-administered, asthma-specific instrument that assesses quality of life over a 2-week time interval. Each item is scored using a scale from 1 to 7, with lower scores indicating greater impairment and higher scores indicating less impairment in quality of life. Items are grouped into four domains: (1) activity limitation (assesses the amount of limitation of individualized activities that are important to the patient and are affected by asthma); (2) asthma symptoms (assesses the frequency and degree of discomfort of shortness of breath, chest tightness, wheezing, chest heaviness, cough, difficulty breathing out, fighting for air, heavy breathing, difficulty getting a good night’s sleep); (3) emotional function (assesses the frequency of being afraid of not having medications, concerned about medications, concerned about

ASSESSMENT OF HEALTH-RELATED QUALITY OF LIFE

7

Table 3. Common Methods Used to Measure a Questionnaire’s Responsiveness to Change Method

Formula


Relative change
Effect size

Meantest1 − Meantest2 SDtest1

36 

Square of Effect Sizedimension Effect Sizestandard
 Meantest1 − Meantest2 SDdifference
 Meantest1 − Meantest2 SDstable group 
Meantest1 − Meantest2 SEdifference

Relative efficiency

Standardized response mean

Responsiveness statistic

Paired t statistic SE of measurement

Meantest1 − Meantest2 Meantest1

Reference 




SDtest × Square Root  1 − Reliability Coefficienttest

37, 38 39

40

41, 42

43 44, 45

Reprinted with permission

having asthma, frustrated); and (4) environmental exposure (assesses the frequency of exposure to and avoidance of irritants such as cigarette smoke, dust, and air pollution). Individual domain scores and a global score are calculated. A change of 0.5 (for both global and individual domain scores) is considered the smallest difference that patients perceive as meaningful (51). To achieve 80% power to detect a difference of 0.5 in AQLQ between two treatment arms would only require 80 patients per arm at a significance level of 0.05. However, this study was designed to enroll 300 patients per arm so that it could also provide 80% power to detect differences in efficacy variables (e.g., FEV1, nighttime awakening) between two treatment arms at a significance level of 0.05. A total of 474 patients were randomly assigned to treatment. Mean change from baseline for the AQLQ global and each of the four domain scores was significantly greater (P 0.005) with salmeterol compared with placebo, first observed at week 4 and continuing through week 12. In addition, differences between salmeterol and placebo groups were greater than 0.5 at all visits except at week 4 and week 8 for the environmental exposure domain. At week 12, salmeterol significantly (P < 0.001 compared

with placebo) increased mean change from baseline in FEV1 , morning and evening PEF, percentage of symptom-free days, percentage of nights with no awakenings due to asthma, and the percentage of days and nights with no supplemental albuterol use. This study demonstrated that salmeterol’s effect in improving patients’ asthma symptoms had a more profound effect on improving patients’ daily activity and well-being. 7.2 HRQOL in Seasonal Allergy Rhinitis A randomized, double-blind, placebocontrolled study was conducted to evaluate the effects on efficacy, safety, and quality of life of two approved therapies (fexofenadine HCI 120 mg and loratadine 10 mg) for the treatment of seasonal allergy rhinitis (SAR), (52). Clinical efficacy was based on a patient’s evaluation of SAR symptoms: (1) sneezing; (2) rhinorrhea; (3) itchy noses, palate, or throat; and (4) itchy, watery, or red eyes. The primary efficacy endpoint was the total score for the patient symptom evaluation, defined as the sum of the four individual symptom scores. Each of the symptoms was evaluated on a 5-point scale (0 to 4), with higher scores indicating more severe symptoms. Treatment duration was 2 weeks, with a run-in period of

8

ASSESSMENT OF HEALTH-RELATED QUALITY OF LIFE

3–7 days. After randomization at study day 1, patients were to return the clinic every week. During these visits, patients were to be evaluated for the severity of SAR symptoms and to complete a quality of life questionnaire. Patient-reported quality of life was evaluated using a validated disease-specific questionnaire— Rhinoconjunctivitis Quality of Life Questionnaire (RQLQ) (26). The RQLQ is a 28-item instrument that assesses quality of life over a 1-week time interval. Each item is scored using a scale from 0 (i.e., not troubled) to 6 (i.e., extremely troubled), with lower scores indicating greater impairment and higher scores indicating less impairment in quality of life. Items are grouped into seven domains: (1) sleep, (2) practical problems, (3) nasal symptoms, (4) eye symptoms, (5) non-nose/eye symptoms, (6) activity limitations, and (7) emotional function. Individual domain scores and an overall score are calculated. A change of 0.5 (for both global and individual domain scores) is considered the smallest difference that patients perceive as meaningful (53). The RQLQ assessment was a secondary endpoint. No sample size and power justification was mentioned in the published paper. A total of 688 patients were randomized to receive fexofenadine HCI 120 mg, loratadine 10 mg, or placebo once daily. Mean 24-hour total symptom score (TSS) as evaluated by the patient was significantly reduced by both fexofenadine HCI and loratadine from baseline (P ≤ 0.001) compared with placebo. The difference between fexofenadine HCI and loratadine was not statistically significant. For overall quality of life, a significant improvement from baseline occurred for all three treatment groups (mean improvement was 1.25, 1.00, and 0.93 for fexofenadine HCI, loratadine, and placebo, respectively). The improvement in the fexofenadine HCI group was significantly greater than that in either the loratadine (P ≤ 0.03) or placebo (P ≤ 0.005) groups. However, the magnitude of differences among the treatment groups was less than the minimal important difference of 0.5. The asthma example demonstrates that salmeterol not only significantly improved patients’ asthma-related symptoms, both statistically and clinically, but also relieved their

asthma-induced impairments on daily functioning and well-being. On the other hand, the SAR example demonstrates that fexofenadine HCI and loratadine both were effective in relief of SAR symptoms. The difference between fexofenadine HCI and loratadine in HRQOL was only statistically significant, but not clinically. However, Hays and Wooley (54) have cautioned investigators about the potential for oversimplication when applying a single minimal clinically important difference (MCID). 7.3 Symptoms Relief for Late-Stage Cancers Although the main objective for the treatment of early-stage cancers is to eradicate the cancer cells and prolong survival, it may not be achievable in late-stage cancers. More often, the objective for the treatment of latestage cancers is palliation, mainly through relief of cancer-related symptoms. As the relief of cancer-related symptoms represents clinical benefit to patients, the objective of some clinical trials in late-stage cancer was relief of a specific cancer-related symptom such as pain. To investigate the benefit of mitoxantrone in patients with symptomatic hormone-resistant prostate cancer, hormonerefractory patients with pain were randomized to receive mitoxantrone plus prednisone or prednisone alone (35). The primary endpoint was a palliative response defined as a two-point decrease in pain as assessed by a six-point pain scale completed by patients (or complete loss of pain if initially 1+) without an increase in analgesic medication and maintained for two consecutive evaluations at least 3 weeks apart. Palliative response was observed in 23 of 80 patients (29%; 95% confidence interval; range 19–40%) who received mitoxantrone plus prednisone, and in 10 of 81 patients (12%; 95% confidence interval; range 6–22%) who received prednisone alone (P = 0.01). No difference existed in overall survival. In another study assessing gemcitabine effect on relief of pain (55), 162 patients with advanced symptomatic pancreatic cancer completed a lead-in period to characterize and stabilize pain and were randomized to receive either gemcitabine 1000 mg/m2

ASSESSMENT OF HEALTH-RELATED QUALITY OF LIFE

weekly × 7 followed by 1 week of rest, then weekly × 3 every 4 weeks thereafter, or to fluorouracil (5-FU) 600 mg/m2 once weekly. The primary efficacy measure was clinical benefit response, which was a composite of measurements of pain (analgesic consumption and pain intensity), Karnofsky performance status, and weight. Clinical benefit required a sustained (≥4 weeks) improvement in at least one parameter without worsening in any others. Clinical benefit response was experienced by 23.8% of gemcitabine-treated patients compared with 4.8% of 5-FU-treated patients (P = 0.0022). In addition, the median survival durations were 5.65 and 4.41 months for gemcitabinetreated and 5-FU-treated patients, respectively (P = 0.0025). Regarding the use of composite variables, researchers have urged investigators to report descriptive results for all components so that composite results do not obscure potential negative results for one or more of the components of the composite (56, 57). In a third study example, although symptom assessment was not the primary endpoint, it was the main differentiating factor between the two study arms in study outcomes. As second-line treatment of small cell lung cancer (SCLC), topotecan was compared with cyclophosphamide, doxorubicin, and vincristine (CAV) in 211 patients with SCLC who had relapsed at least 60 days after completion of first-line therapy (58). Response rate and duration of response were the primary efficacy endpoints. Patientreported lung-cancer-related symptoms were also evaluated as secondary endpoints. Similar efficacy in response rate, progression-free survival, and overall survival was observed between topotecan and CAV. The response rate was 26 of 107 patients (24.3%) treated with topotecan and 19 of 104 patients (18.3%) treated with CAV (P = 0.285). Median times to progression were 13.3 weeks (topotecan) and 12.3 weeks (CAV) (P = 0.552). Median survival was 25.0 weeks for topotecan and 24.7 weeks for CAV (P = 0.795). However, the proportion of patients who experienced symptom improvement was greater in the topotecan group than in the CAV group for four of eight lung-cancer-related symptoms evaluated, including dyspnea, anorexia,

9

hoarseness, and fatigue, as well as interference with daily activity (P ≤ 0.043). 8

CONCLUSION

Although HRQOL assessment in clinical trials has increased steadily over the years, a substantial challenge remains in interpretation of HRQOL results and acceptance of its value in clinical research. Both issues will require time for clinicians and regulators to fully accept HRQOL assessments. To help build acceptance, existing HRQOL instruments should be validated in each therapeutic area, rather than develop new instruments. The most urgent need in HRQOL research is to increase HRQOL acceptance by clinicians and regulators so that pharmaceutical companies will continue to include financial support for HRQOL assessments in new and existing drug development programs. 9

ACKNOWLEDGMENT

The author is deeply indebted to Carol M. Moinpour for her numerous suggestions and to Carl Chelle for his editorial assistance. REFERENCES 1. J. A. Cramer and B. Spilker, Quality of Life and Pharmacoeconomics: An Introduction. Philadelphia: Lippincott-Raven, 1998. 2. World Health Organization, The First Ten Years of the World Health Organization. Geneva: World Health Organization, 1958, p. 459. 3. J. E. Ware, Jr. and C. D. Sherbourne, The MOS 36-Item Short-Form Health Survey (SF36). I. Conceptual framework and item selection. Med. Care 1992; 30: 473–483. 4. C. A. McHorney, J. E. Ware, Jr., and A. E. Raczek, The MOS 36-Item Short-Form Health Survey (SF-36): II. Psychometric and clinical tests of validity in measuring physical and mental health constructs. Med. Care 1993; 31: 247–263. 5. C. A. McHorney et al., The MOS 30-Item Short-Form Health Survey (SF-36): III. Tests of data quality, scaling assumptions, and reliability across diverse patient groups. Med. Care 1994; 32: 40–66.

10

ASSESSMENT OF HEALTH-RELATED QUALITY OF LIFE

6. M. Bergner et al., The Sickness Impact Profile: development and final revision of a health status measure. Med. Care 1981; 19: 787–805.

21.

7. S. M. Hunt, J. McEwen, and S. P. McKenna, Measuring Health Status. London: Croom Helm, 1986. 8. G. R. Parkerson, Jr., W. E. Broadhead, and C. K. Tse, The Duke Health Profile. A 17-item measure of health and dysfunction. Med. Care 1990; 28: 1056–1072.

22.

9. L. W. Chambers, The McMaster Health Index Questionnaire (MHIQ): Mehtodologic Documentation and Report of the Second Generation of Investigations. Hamilton, Ontario, Canada: McMaster University, Department of Clinical Epidemiology and Biostatistics, 1982.

23.

10. A. M. Jette et al., The Functional Status Questionnaire: reliability and validity when used in primary care. J. Gen. Intern. Med. 1986; 1: 143–149. 11. S. Szabo (on behalf of the WHOQOL Group), The World Health Organization Quality of Life (WHOQOL) assessment instrument. In: B. Spilker (ed.), Quality of Life and Pharmacoeconomics in Clinical Trials. Philadelphia: Lippincott-Reven, 1996. 12. C. S. Cleeland, Measurement of pain by subjective report. In: C. R. Chapman and J. D. Loeser (eds.), Issue in Pain Measurement. New York: Raven Press, 1989.

24.

25.

26.

27.

28.

13. R. Melzack, The McGill Pain Questionnaire: major properties and scoring methods. Pain 1975; 1: 277–299. 14. I. McDowell and C. Newell, Measuring Health: A Guide to Rating Scales and Questionnaires, 2nd ed. New York: Oxford University Press, 1996.

29.

15. A. T. Beck et al., An inventory for measuring depression. Arch. Gen. Psychiat. 1961; 4: 561–571.

30.

16. L. S. Radloff, The CES-D scale: a self-report depression scale for research in the general population. Appl. Psychol. Measure. 1977; 1: 385–401.

31.

17. M. Hamilton, Standised assessment and recording of depressive symptoms. Psychiat. Neurol. Neurochir. 1969; 72: 201–205.

32.

18. A. Zigmond and P. Snaith, The Hospital Anxiety and Depression Questionnaire. Acta Scand. Psychiat. 1983; 67: 361–368. 19. W. W. K. Zung, A self-rating depression scale. Arch. Gen. Psychiat. 1965; 12: 63–70. 20. P. Bech et al., The WHO (Ten) Well-Being Index: validation in diabetes. [comment]. [Clinical Trial. Journal Article. Multicenter

33.

Study. Randomized Controlled Trial]. Psychother. Psychosomat. 1996; 65: 183–190. J. F. Fries et al., The dimension of health outcomes: the Health Assessment Questionnaire, disability and pain scales. J. Rheumatol. 1982; 9: 789–793. G. H. Guyatt, A. Mitchell, E. J. Irvine et al., A new measure of health status for clinical trials in inflammatory bowel disease. Gastroenterology 1989; 96: 804–810. E. J. Irvine, B. Feagan et al., Quality of life: a valid and reliable measure of therapeutic efficacy in the treatment of inflammatory bowel disease. Gastroenterology 1994; 106: 287–296. E. F. Juniper, G. H. Guyatt, P. J. Ferrie, and L. E. Griffith, Measuring quality of life in asthma. Am. Rev. Respir. Dis. 1993; 147: 832–838. P. W. Jones, F. H. Quirk, and C. M. Baveystock, The St. Geroge’s Respiratory Questionnaire. Respiratory Med. 1991; 85: 25–31. E. F. Juniper and G. H. Guyatt, Development and testing of a new measure of health status for clinical trials in rhinoconjunctivitis. Clin. Exp. Allergy 1991; 21: 77–83. V. Peto et al., The development and validation of a short measure of functioning and well being for individuals with Parkinson’s disease. Qual. Life Res. 1995; 4(3): 241–248. A. G. E. M. De Boer et al., Quality of life in patients with Parkinson’s disease: development of a questionnaire. J. Neurol. Neurosurg. Psychiat. 1996; 61(1): 70–74. N. K. Aaronson et al., The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology. J. Natl. Cancer Inst. 1993; 85: 365–376. D. F. Cella et al., The Functional Assessment of Cancer Therapy scale: development and validation of the general measure. J. Clin. Oncol. 1993; 11: 570–579. C. M. Moinpour, Costs of quality-of-life research in Southwest Oncology Group trials. J. Natl. Cancer Inst. Monogr. 1996; 20: 11–16. J. A. Sloan et al. and the Clinical Significance Consensus Meeting Group, The costs of incorporating quality of life assessments into clinical practice and research: what resources are required? Clin. Therapeut. 2003; 25(Suppl D). P. J. Goodwin et al., Health-related quality-oflife measurement in randomized clinical trails in breast cancer—taking stock. J. Natl. Cancer Inst. 2003; 95: 263–281.

ASSESSMENT OF HEALTH-RELATED QUALITY OF LIFE

11

34. D. L. Fairclough, Design and Analysis of Quality of Life Studies in Clinical Trials. Boca Raton, FL: Chapman & Hall/CRC Press, 2002.

47. J. Cohen, Statistical Power Analysis for the Behavioral Sciences, 2nd ed. London: Academic Press, 1988.

35. I. F. Tannock et al., Chemotherapy with mitoxantrone plus prednisone or prednisone alone for symptomatic hormone-resistant prostate cancer: a Canadian randomized trial with palliative end points. J. Clin. Oncol. 1996; 14: 1756–1764.

48. D. Osoba, G. Rodrigues, J. Myles, B. Zee, and J. Pater, Interpreting the significance of changes in health-related quality of life scores. J. Clin. Oncol. 1998; 16: 139–144.

36. R. A. Deyo and R. M. Centor, Assessing the responsiveness of functional scales to clinical change: an analogy to diagnostic test performance. J. Chronic Dis. 1986; 39: 897–906. 37. Kazis et al., 1989. 38. R. A. Deyo, P. Diehr, and D. L. Patrick, Reproducibility and responsiveness of health status measures: statistics and strategies for evaluation. Control Clin. Trials 1991; 12(4 Suppl): 142S–158S. 39. C. Bombardier, J. Raboud, and Auranofin Cooperating Group, A comparison of healthrelated quality-of-life measures for rheumatoid arthritis research. Control Clin. Trials 1991; 12(4 Suppl): 243S–256S. 40. J. N. Katz et al., Comparative measurement sensitivity of short and longer health status instruments. Med. Care 1992; 30: 917–925. 41. G. H. Guyatt, S. Walter, and G. Norman, Measuring change over time: assessing the usefulness of evaluative instruments. J. Chronic Dis. 1987; 40: 171–178. 42. G. H. Guyatt, B. Kirshner, and R. Jaeschke, Measuring health status: what are the necessary measurement properties? J. Clin. Epidemiol. 1992; 45: 1341–1345. 43. M. H. Liang et al., Comparative measurement efficiency and sensitivity of five health status instruments for arthritis research. Arthritis Rheum. 1985; 28: 542–547. 44. K. W. Wyrwich, N. A. Nienaber, W. M. Tierney, and F. D. Wolinsky, Linking clinical relevance and statistical significance in evaluating intra-individual changes in health-related quality of life. Med. Care 1999; 37: 469–478.

49. G. H. Guyatt et al. and the Clinical Significance Consensus Meeting Group, Methods to explain the clinical significance of health status measures. Mayo Clin. Proc. 2002; 77: 371–383. 50. R. F. Lockey et al., Nocturnal asthma – effect of salmeterol on quality of life and clinical outcomes. Chest 1999; 115: 666–673. 51. E. F. Juniper et al., 1994. 52. P. Van Cauwenberge and E. F. Juniper, The Star Study Investigating Group. Comparison of the efficacy, safety and quality of life provided by fexofenadine hydrochloride 120 mg, loratadine 10 mg and placebo administered once daily for the treatment of seasonal allergic rhinitis. Clin. Exper. Allergy 2000; 30: 891–899. 53. E. F. Juniper et al., 1996. 54. R. D. Hays and J. M. Woolley, The concept of clinically meaningful difference in healthrelated quality-of-life research. How meaningful is it? Pharmacoeconomics 2000; 18: 419–423. 55. H. A. Burris, III et al., Improvements in survival and clinical benefit with gemcitabine as first-line therapy for patients with advanced pancreas cancer: a randomized trial. J. Clin. Oncol. 1997; 15: 2403–2413. 56. N. Freemantle, M. Calvert, J. Wood, J. Eastaugh, and C. Griffin, Composite outcomes randomized trials. Greater precision but with greater uncertainty? JAMA 2003; 289: 2554–2559. 57. M. S. Lauer and E. J. Topol, Clinical trials—Multiple treatment, multiple end points, and multiple lessons. JAMA 2003; 289: 2575–2577. 58. J. von Pawel et al., Topotecan versus cyclophosphamide, doxorubicin, and vincristine for the treatment of recurrent smallcell lung cancer. J. Clin. Oncol. 1999; 17: 658–667.

45. K. W. Wyrwich, W. M. Tierney, and F. D. Wolinsky, Further evidence supporting an SEM-based criterion for identifying meaningful intra-individual changes in health-related quality of life. J. Clin. Epidemiol. 1999; 52: 861–873.

FURTHER READING

46. J. A. Sloan et al., Randomized comparison of four tools measuring overall quality of life in patients with advanced cancer. J. Clin. Oncol. 1998; 16: 3662–3673.

B. Spilker (ed.), Quality of Life and Pharmacoeconomics in Clinical Trials. Philadelphia: Lippincott-Reven, 1996.

12

ASSESSMENT OF HEALTH-RELATED QUALITY OF LIFE

M. A. G. Sprangers, C. M. Moinpour, T. J. Moynihan, D. L. Patrick, and D. A. Revicki, Clinical Significance Consensus Meeting Group. assessing meaningful change in quality of life over time: a users’ guide for clinicians. Mayo Clin. Proc. 2002; 77: 561–571. J. E. Ware et al., SF-36 Health Survey: Manual and Interpretation Guide. Boston: The Health Institute, New England Medical Center, 1993. World Health Organization, International Classification of Impairments, Disabilities, and Handicaps. Geneva: World Health Organization, 1980.

AUDIT

of legal proceedings or investigations. Where required by applicable law or regulation, the sponsor should provide an audit certificate.

An Audit is a systematic and independent examination of trial-related activities and documents to determine whether the evaluated trial-related activities were conducted, and the data were recorded, analyzed, and reported accurately according to the protocol, sponsor’s Standard Operating Procedures (SOPs), Good Clinical Practice (GCP), and applicable regulatory requirement(s). The purpose of a sponsor’s audit, which is independent of and separate from routine monitoring or quality control functions, should be to evaluate trial conduct and compliance with the protocol, SOPs, GCP, and applicable regulatory requirements. The sponsor should appoint individuals who are independent of the clinical trial/data collection system(s) to conduct audits. The sponsor should ensure that the auditors are qualified by training and by experience to conduct audits properly. An auditor’s qualifications should be documented. The sponsor should ensure that the auditing of clinical trials/systems is conducted in accordance with the sponsor’s written procedures on what to audit, how to audit, the frequency of audits, and the form and content of audit reports. The sponsor’s audit plan and procedures for a trial audit should be guided by the importance of the trial to submissions to regulatory authorities, the number of subjects in the trial, the type and complexity of the trial, the level of risks to the trial subjects, and any identified problem(s). The observations and findings of the auditor(s) should be documented. To preserve the independence and the value of the audit function, the regulatory authority(ies) should not request the audit reports routinely. Regulatory authority(ies) may seek access to an audit report on a case-by-case basis, when evidence of serious GCP noncompliance exists, or in the course This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

AUDIT CERTIFICATE An Audit Certificate is the declaration of confirmation by the auditor that an audit has taken place. Where required by applicable law or regulation, the sponsor should provide an audit certificate.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

AUDIT REPORT An Audit Report is the written evaluation by the sponsor’s auditor of the results of the audit. To preserve the independence and the value of the audit function, the regulatory authority(ies) should not request the audit reports routinely. Regulatory authority(ies) may seek access to an audit report on a case-by-case basis, when evidence of serious Good Clinical Practice (GCP) noncompliance exists, or in the course of legal proceedings or investigations.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

BAYESIAN DOSE-FINDING DESIGNS IN HEALTHY VOLUNTEERS

The majority of phase I studies are conducted in healthy volunteers. Current doseescalation designs for healthy volunteer studies are usually crossover designs in which each subject receives a dose in each of a series of consecutive treatment periods separated by washout periods. Such a design is illustrated in Table 1. Groups of subjects, known as cohorts or panels, some on active doses and others on placebo, are treated simultaneously. The lowest dose, d1 , is normally regarded as the only ‘‘safe’’ dose to be used for the first cohort of subjects. Safety data along with pharmacokinetic/pharmacodynamic (PK /PD) data from each treatment period are collected then summarized in tables, listings, and graphs for a safety review committee to make decisions regarding the doses to be given during the next period. The committee will normally assign doses according to some predefined dose-escalation scheme, but they may alter the scheme by repeating the previous doses or de-escalating one dose level; they may even stop the trial if its safety is in question. This is essentially a PK/PD ‘‘clinical judgment’’– guided design. Response variables are analyzed either by simple analysis of variance (ANOVA) approaches (2) or by repeated measures analysis of variance. The PK/PD models may also be estimated and presented at the end of a trial. Under conventional dose-escalation rules, it is likely that healthy volunteers will be treated at subtherapeutic doses. Consequently, information gathered from a trial is mostly not relevant for identifying the optimal dose for phase II studies. Bayesian decision-theoretic designs, motivated by statistical-model–based designs for cancer trials, have been proposed to enhance the precision of the optimal dose for phase II studies, to increase the overall efficiency of the dose-escalation procedure while maintaining the safety of subjects (3–6).

YINGHUI ZHOU Medical and Pharmaceutical Statistics Research Unit The University of Reading Reading, Berkshire, United Kingdom

Phase Istudies have unique characteristics that distinguish them from other phases of clinical research. This is the phase when new experimental drugs are given to human subjects at the first time: a more explicit name for such trials is ‘‘first-into-man’’ (FIM) studies. Although intensive toxicologic studies have been carried out at preclinical trials, the primary concern for FIM studies is always the safety of the participating subjects (1). Safety can be assessed by the incidence of doselimiting events (DLE), such as moderate or severe adverse events, clinically significant changes in cardiac function, exceeding the maximum exposure limits of pharmacokinetic profiles such as the area under the curve (AUC) or the maximum concentration (Cmax ), and changes in pharmacodynamic parameters such as blood flow or heart rate. The majority of FIM studies are conducted on healthy volunteers for whom dose-limiting events should generally be avoided. Only a small proportion of FIM studies is conducted on late-stage patients in cancer trials. For these patients, standard therapies have failed, and low, safe doses will not achieve therapeutic effects, while high, unsafe doses will cause toxicity. A small risk of a DLE is therefore permitted to gain some therapeutic effect. The primary objective of FIM studies is to find an optimal dose, both safe and potentially efficacious, for later phases of clinical research. This involves a dose-escalation scheme: a fixed range of discrete doses, d1 < . . . < dk for some integer k, is predefined by the investigators to be administered in turn to subjects to assess the safety and tolerability of the compound. For healthy volunteer studies, the optimal dose is the maximum safe dose leading to an acceptable concentration of drug in plasma or to an adequate description of the biological effects.

1 A BAYESIAN DECISION-THEORETIC DESIGN In healthy volunteer studies, several PK/PD measurements are normally monitored and recorded. The methodology described here,

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

BAYESIAN DOSE-FINDING DESIGNS IN HEALTHY VOLUNTEERS Table 1. An example of a crossover design Cohort

Dosing intervals 1

1 2 3 . ..

2

X

3 X

X

5

X

(1)

The logarithm of the j-th active dose received by the i-th subject is denoted by ij . The term si is a random effect relating to the i-th subject. The si and εij are modeled as mutually independent, normally distributed random variables with mean zero and variances τ 2 and σ 2 , respectively. The correlation, ρ, between two responses on the same subject can is equal to τ 2 /(σ 2 + τ 2 ). Placebo administrations are ignored in this model because there will be no drug detected in plasma.

6

7

X X

however, focuses on a single pharmacokinetic variable derived from the curve relating the concentration of the drug in plasma to the time since administration (7). Commonly used summaries such as area under the curve (AUC) or peak drug concentration (Cmax ) are often modeled by the normal distribution after a logarithmic transformation (8). The mean value of y, denoting either log(AUC) or log(Cmax ), is modeled as linear in log dose: E(y) = θ 1 + θ 2 log(dose), where θ 1 and θ 2 are unknown population parameters. A maximum exposure level, L, is defined before the start of the study based on the toxicity profile for the compound observed in the most sensitive animal species. Any concentration of the drug leading to an AUC or Cmax in excess of this level will be unacceptable. As each subject receives more than one dose, the multiple responses of each subject are correlated. Furthermore, different subjects vary considerably in terms of pharmacokinetics. Therefore, both random subject effects and random errors are included in the model. A log–log mixed effects model (3–6) is fitted to yij , the response following the j-th dose to the i-th subject: yij = θ1 + θ2 lij + si + εij

4

X X

X

Bayesian decision theory supplies a general framework for making decisions under uncertainty, which can be applied in many scientific and business fields (9–11). Let θ = (θ 1 , θ 2 )
be the vector of the unknown parameters. Some information or expert opinion about θ , from either preclinical data or experience of similar compounds, will be available before the dose-escalation study begins. This information can be formulated as a probability density function for θ , denoted by h0 (θ ) and known as the ‘‘prior density.’’ Let x denote the data collected in the doseescalation study, and f(x | θ ) be its likelihood function of the data. Then, the posterior density function of θ , h(θ | x), can be derived using Bayes’ theorem: h (θ |x) =


h0 (θ ) f (x|θ ) . h0 (ϕ) f (x|ϕ) dϕ

This posterior represents the opinion about θ formed by combining the prior opinion with the data. Potentially, one could treat θ 1 , θ , τ 2 , and σ 2 all as unknown parameters. However, the more parameters that are modeled, the less accurate the resulting estimates, especially as sample sizes are small in phase I trials. One way to solve the problem is to give some parameters fixed values as if they are known. In particular, the within-subject correlation, ρ, might be set to a value such as 0.6. Hence, there will be three unknown parameters in the model (1) instead of four parameters, namely θ 1 , θ 2 , and σ 2 . Fixing ρ is convenient, and the effect of doing so has been studied by Whitehead et al. (5) by taking the alternative strategy of specifying discrete priors on ρ. In the resulting analyses, ρ is underestimated by the Bayesian procedure, and

BAYESIAN DOSE-FINDING DESIGNS IN HEALTHY VOLUNTEERS

the estimate of σ 2 is overestimated. Consequently, it was concluded that fixing ρ was a reasonable policy for practical use. Conjugate priors have been proposed for θ 1 , θ 2 , and ν (3–6), where ν is the withinsubject precision ν = σ −2 . In particular, the conditional distribution of θ given ν, and the marginal distribution of ν can be taken to be   θ |ν ∼ N µ0 , (νQ0 )−1 ; ν ∼ Ga (α0 , β0 ) , (2) where N denotes a normal distribution, Ga a gamma distribution, and the values of µ0 , Q0 , α 0 , and β 0 are chosen to represent prior knowledge. Being a conjugate prior, the posterior distribution shares the same form. Suppose that n subjects have been treated. The i-th subject has received pi periods of treatment, i = 1, . . . , n, and so a total of p1 + · + pn = p observations are available. Let the p-dimensional y denote the vector of responses with elements yij ordered by subject and by period within subject. The (p × 2) design matrix X is made up of rows of the form (1, ij ), ordered in the same way as y. The (p × n) matrix U, the design matrix of the random subject effect, is defined as having a 1 in the i-th column of rows p1 + · + pi1 + 1, . . . , p1 + . . . + pi for i = 1, . . . , n, and zeros elsewhere. The identity matrix is denoted by I. The (p × p) matrix P is defined as P = [I + ρ/(1 – ρ)UU
]−1 . Posterior distributions for θ and ν are   (3) θ |v ∼ N µ, (vQ)−1 ; v ∼ Ga (α, β) , where α = α 0 + p/2, β = β0 + (y
Py + µ0Q0µ0−µ
Qµ)/2, µ = (Q0 + X
PX)−1 (Q0
µ0 + X
Py), and Q = (Q0 + X
PX). Priors reflect not only one’s opinions, but also how strongly they are held. Here, a small value for α 0 represents a weak opinion. Consequently, dose escalation may be quick at the beginning as a few safe observations will soon overcome any prior reservations. A bigger value for α 0 represents a strong prior, and the resulting dose escalation will be conservative as prior concerns will only be removed by a clear demonstration of safety. A safety constraint can be used to help control overdosing (3–6). This requires that no dose be given if the predicted response at

3

this dose is likely to exceeds the safety limit L. Mathematically this can be expressed as   (4) P yij > log L|dij , y ≥ π0 , where π 0 , the tolerance level, can be set at a low value such as 0.05 or 0.20. The dose at which the above probability is equal to π 0 is called the maximum safe dose for the i-th subject following the j-th observation. The maximum safe dose is subject related, and posterior estimates may differ among subjects who have already been observed, being lower for a subject who previously had absorbed more drug than average and higher if the absorption was less. After each treatment period of the dose-escalation procedure, the posterior predictive probability that a future response will lie above the safety limit is updated. The decision of which dose to administer to each subject in each dosing period is made using a predefined criterion. This criterion can be based on safety; for example, one could use the maximum safe dose as the recommended dose. It can also be based on accuracy of estimates of unknown parameters; for example, the optimal choice of doses is that which will maximize the determinant of the posterior variance-covariance matrix of the joint posterior distribution or minimize the posterior variance of some key parameter. 2 AN EXAMPLE OF DOSE ESCALATION IN HEALTHY VOLUNTEER STUDIES An example, described in detail by Whitehead et al. (5), in which the principal pharmacokinetic measure was Cmax will be outlined briefly here. The safety cutoff for this response was taken to be yL = log(200). Seven doses were used according to the schedule: 2.5, 5, 10, 25, 50, 100, 150 µg. The actual trial was conducted according to a conventional design, and the dosing structure and the resulting data are listed in Table 2. From a SAS PROC MIXED analysis, maximum likelihood estimates of the parameters in model (1) are θ1 = 1.167, θ2 = 0.822, σ 2 = 0.053, and τ2 = 0.073.

(5)

4

BAYESIAN DOSE-FINDING DESIGNS IN HEALTHY VOLUNTEERS

Note that it follows that the within-subject correlation is estimated as 0.579. As an illustration, the Bayesian method described above will be applied retrospectively to this situation. Conjugate priors must first be expressed. It is assumed prior expert opinion suggested that the Cmax values would be 5.32 and 319.2 pg/mL at the doses 2.5 and 150 µg respectively. This prior indicates that the highest dose 150 µg is not a safe dose because the predicted Cmax exceeds the safety limit of 200 pg/mL. The value for ρ is set as 0.6. This forms the bivariate normal distributions for θ ,  N

θ |v ∼    0.756 0.940 − 0.109 , . 1 −0.109 0.04

The value for α 0 is set as 1, as suggested in Whitehead et al. (5). The value for β 0 can be found via the safety constraint: P(y01 > logL | d01 = 2.5) = 0.05. This implies that dose 2.5 µg will be the maximum safe dose for all new subjects at the first dosing period in the first cohort. Therefore, ν ∼ Ga(1, 0.309). To illustrate Bayesian dose escalation, data are simulated based on the parameters found from the mixed model analysis given by Whitehead et al. (5). Thus, Cmax values are generated from three cohorts of eight healthy volunteers, each treated in four consecutive periods and receiving three active doses and one randomly placed placebo. A simulated dose escalation with doses chosen according to the maximum safe dose criterion is shown in Table 3. The first six subjects received the lowest dose, 2.5 µg. All subjects at the next dosing period received dose 10 µg, in which 5 µg dose was skipped for subjects 1 to 4. Subjects 7 and 8, who were on placebo in the first dosing period, skipped two doses, 2.5 and 5 µg, to receive 10 µg in the second dosing period. If this dosing proposal was presented to a safety committee in a real trial, the committee members might wish to alter this dose recommendation. The Bayesian approach provides a scientific recommendation for dose escalations. However, the final decision on which doses are given should come from a safety committee. The procedure would be able to make use of results

from any dose administered. In Table 3, it is shown that the maximum safe dose for a new subject at the beginning of the second cohort is 25 µg. All subjects in the second cohort received 50 µg at least twice. Subjects 17 to 22 in the first dosing period of the final cohort received 50 µg. However, they all had a high value of Cmax . The Bayesian approach then recommended a lower dose, 25 µg for subjects 17 to 20 (subjects 21 and 22 were on placebo). This shows that the Bayesian approach can react to different situations quickly: when escalation looks safe, dose levels can be skipped; when escalation appears to be unsafe, lower doses are recommended. Two high doses that were administered in the real trial, 100 and 150 µg, were never used in this illustrative run. The posterior distributions for θ at the end of the third cohort are

 N

θ |v ∼    1.376 0.024 − 0.006 , , 0.790 −0.006 0.002

and ν ∼ Ga(37, 2.625). Figure 1 shows the doses administered to each subject and the corresponding responses. Table 4 gives the maximum likelihood estimates from the real data in Table 2 that were used as the true values in the simulation, together with the maximum likelihood estimates from the simulated data in Table 3. Results show that σ 2 and τ 2 were underestimated from the simulated data, with there being no evidence of between-subject variation. Consequently, the estimated correlation from the simulated data is zero, in contrast to the true value of 0.579 used in the simulation. This is a consequence of the small dataset, and illustrates the value of fixing ρ during the escalation process. Different prior settings will result in different dose escalations. For example, if the value for α 0 is changed from 1.0 to 0.1, then the dose-escalation will be more rapid. Table 5 summarizes the recommended doses and simulated responses from another simulation run where α 0 = 0.1. In the second dosing period of the first cohort, subjects 1, 2, and 4 skipped two doses, 5 and 10 µg. Subjects 3, 7, and 8 skipped three doses in that cohort.

BAYESIAN DOSE-FINDING DESIGNS IN HEALTHY VOLUNTEERS

5

Table 2. Real data from a healthy volunteer trial in Whitehead et al. (2006) Period Subject

1 Dose

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

2.5 2.5 2.5 10

2 Cmax

Cmax

5 5

14.5

24.1 2.5 2.5 5 5

2.5 2.5 2.5 10 10 10 10 10

3

Dose

29.0 15.0 21.8 23.0

50

54.7

50 50 50 50 50

161.5 101.4 194.1 61.6 81.7

10.4 13.5

10 25

14.6 34.1

25 25 25 10 2.5 100 50

34.9 70.6 40.7 16.6 125.6 79.4

100 100

131.3 232.5

100 50

84.2 113.8

4

Dose

Cmax

10

11.0

5 25 5 5 10

10.0 33.3 10.2 20.6 28.5

5 25 50 25

19.4 71.6 37.1

50 25 5

73.4 53.0 14.4

100

108.1

100 150 100

104.5 217.6 178.9

Dose

Cmax

10 10 50 10

20.4 19.7 74.6 29.7

10 10 50

14.6 27.0 47.7

50 50 50

59.2 76.1 166.6

50 10

87.1 17.8

Source: Whitehead et al. Stat Med. 2006; 25: 433–445.

The starting dose for all subjects in the first dosing period of the second and third cohorts was 50 µg (25 µg and 50 µg were the doses in Table 3). All subjects but one occasion repeatedly received 50 µg during the second cohort.Inthe third cohort, the dose of 100 µg was used quite frequently. The highest dose, 150 µg, was never used. This example shows that different prior settings will affect dose-escalation procedures. Multiple simulations should therefore be conducted to gain a better understanding of the properties of a design. Different scenarios should be tried to ensure that the procedure has good properties, whether the drug is safe, unsafe, or only safe for some lower doses. 3

DISCUSSION

Bayesian methods offer advantages over conventional designs. Unlike conventional

designs, more doses within the predefined dose range, or even outside of the predefined dose range, can be explored without necessarily needing extra dosing time as dose level skipping can be permitted. From the simulation runs in Tables 3 and 5, more doses, such as 40, 60, and 80 µg, could perhaps be included in the dose scheme. Simulations have shown that skipping dose levels does not affect either safety or accuracy; on the contrary, it improves safety or accuracy as a wider range of doses is used (12). Providing a greater choice of doses, while allowing dose skipping, leads to procedures that are more likely to find the target dose and to learn about the dose-response relationship curve efficiently. Ethical concerns can be expressed through cautious priors or safety constraints. Dose escalation will be dominated by prior opinion at the beginning, but it will soon

6

BAYESIAN DOSE-FINDING DESIGNS IN HEALTHY VOLUNTEERS Table 3. A simulated dose-escalation Period Subject

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

1

2

3

4

Dose

Cmax

Dose

Cmax

Dose

Cmax

2.5 2.5 2.5 2.5 2.5 2.5

9.9 12.4 10.4 11.1 6.5 7.7

10 10 10 10

19.7 27.0 35.6 35.7

25 25

64.1 76.4

25 25 25 25 25 25

50 50 50 50 50 50

10 10 50 50 50 50

38.9 41.4 42.5 53.4 50.8 48.7

50 50 25 25 25 25

138.2 194.4 173.2 139.9 181.6 117.9

50 50

be influenced more by the accumulating real data. Before a Bayesian design is implemented in a specific phase I trial, intensive simulations should be used to evaluate different prior settings, safety constraints, and dosing criteria under a range of scenarios. Investigators can then choose a satisfactory design based on the simulation properties that they

21.3 24.5 84.4 71.0 87.1 67.2

84.4 82.2 73.6 57.9 76.6 43.1

93.6 97.5

25 25 50 25 50 50

50 50 50 50 25 25

25 50 50 50

33.7 62.9 93.2 52.8 53.7 58.6

60.4 66.5 89.4 75.5 46.5 69.2

47.2 87.7 75.5 79.2

Dose

Cmax

25 25 50 50 50 50

44.8 27.1 58.8 57.9 58.0 58.1

50 50 50 50 50 50

92.7 91.9 79.6 110.6 62.6 76.4

25 50 50 50 50 50

37.4 80.2 79.2 63.9 82.5 65.9

are interested in. For instance, they may look for a design that gives the smallest number of toxicities or the most accurate estimates. Until recently, such prospective assessment of design options was not possible, but now that the necessary computing tools are available, it would appear inexcusable not to explore any proposed design before it is implemented.

Table 4. Maximum likelihood estimates (MLE) and Bayesian modal estimates of simulated data in Table 3 (with standard errors or standard deviations)

Truth for simulations (MLE from real data) Final MLE (exclude the prior information) Bayesian prior modal estimates Bayesian posterior modal estimates

θ1

θ2

σ2

τ2

ρ

d∗f

1.167 (0.158) 1.568 (0.143) 0.759 (0.967) 1.376 (0.156)

0.822 (0.046) 0.741 (0.041) 1.00 (0.192) 0.790 (0.042)

0.053

0.073

0.579

73.42

0.090

0.000

0.000

69.45

0.309

0.463

0.6

2.5

0.071

0.106

0.6

58.18

BAYESIAN DOSE-FINDING DESIGNS IN HEALTHY VOLUNTEERS

7

Dose 157 135 113 91 69 47 25 3 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Subject

Cmax ∈ (0, 0.2L);

Key: Cmax 210

Cmax ∈ (0.2L, 0.5L);

Cmax ∈ (0, 0.5L, L).

safty limit

180 150 120 50% safty limit 90 60 20% safty limit

30 3 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

24

Subject

Figure 1. An illustration of a simulated dose escalation (using data from Table 3).

Although the methodology described here is presented only for a single pharmacokinetic outcome, the principles are easily generalized for multiple endpoints. Optimal doses can be found according to the safety limits for each of the endpoints, and then the lowest of these doses can be recommended. The Bayesian decision-theoretic approach has also been extended for application to an attention deficit disorder study

(13), where a pharmacodynamic response (heart rate change from baseline), a pharmacokinetic response (AUC), and a binary response (occurrence of any dose-limiting events) are modeled. A one-off simulation run indicates that the Bayesian approach controls unwanted events, dose-limiting events, and AUC levels exceeding the safety limit, while achieving more heart rate changes within the therapeutic range.

8

BAYESIAN DOSE-FINDING DESIGNS IN HEALTHY VOLUNTEERS Table 5. A simulated dose-escalation (the value for α 0 is 0.1) Period Subject

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

1

2

Dose

Cmax

2.5 2.5 2.5 2.5 2.5 2.5

8.2 9.9 6.2 8.4 8.0 6.4

50 50 50 50 50 50

50 50 50 50 50 50

88.9 104.4 74.9 98.3 86.2 89.4

58.5 48.3 70.5 45.3 55.3 49.1

3

4

Dose

Cmax

Dose

Cmax

25 25 50 25

53.4 40.4 118.5 61.1

50 50

66.1 78.9

25 25 50 50 50 50

50 50 100 100 50 100

100 50

50 50 50 50 50 50

65.1 37.7 64.4 49.5 64.7 84.0

50 50 50 50 50 100

109.9 115.0 172.8 140.7 73.8 89.7

100 100 100 50

157.6 72.8

Bayesian methodology only provides scientific dose recommendations. These should be treated as additional information for guidance of the safety committee of a trial rather than dictating final doses to be administered. Statisticians and clinicians need to be familiar with this methodology. Once the trial starts, data need to be available quickly and presented unblinded with dose recommendations to the safety committee. They can then make a final decision, based on the formal recommendations, together with all of the additional safety, laboratory, and other data available.

4.

REFERENCES

6.

3.

5.

1. U.S. Food and Drug Administration. 1997. Guidance for Industry: General considerations for the clinical evaluation of drugs. 2. K. Gough, M. Hutchison, O. Keene, B. Byrom, S. Ellis S, et al., Assessment of dose proportionality—Report from the

7.

92.7 74.1 60.9 98.2 46.6 46.2

48.4 56.7 46.6 68.5 60.5 98.2

142.0 115.6 102.5 63.9

Dose

Cmax

50 50 50 50 50 50

76.7 84.2 101.7 83.0 78.2 72.0

50 50 100 50 50 50

57.9 72.6 83.6 60.8 59.2 93.4

50 100 100 100 100 100

92.7 119.8 106.5 101.5 156.5 145.3

Statisticians in the Pharmaceutical Industry/Pharmacokinetic UK Joint Working Party. Drug Inf J. 1995; 29: 1039–1048. S. Patterson, S. Francis, M. Ireson, D. Webber, and J. Whitehead, A novel Bayesian decision procedure for early-phase dose finding studies. J Biopharm Stat. 1999; 9: 583–597. J. Whitehead, Y. Zhou, S. Patterson, D. Webber, and S. Francis, Easy-to-implement Bayesian methods for dose-escalation studies in healthy volunteers. Biostatistics. 2001; 2: 47–61. J. Whitehead, Y. Zhou, A. Mander, S. Ritchie, A. Sabin, and A. Wright, An evaluation of Bayesian designs for dose-escalation studies in healthy volunteers. Stat Med. 2006; 25: 433–445. Y. Zhou, Dose-escalation methods for phase I healthy volunteer studies. In: S. Chevret (ed.), Statistical Methods for DoseFinding Experiments. Chichester, UK: Wiley, 2006, pp. 189–204. S. C. Chow and J. P. Liu, Design and Analysis of Bioavailability and Bioequivalence Studies. Amsterdam: Dekker, 1999.

BAYESIAN DOSE-FINDING DESIGNS IN HEALTHY VOLUNTEERS 8. W. J. Westlake, Bioavailability and bioequivalence of pharmaceutical formulations. In: K. E. Peace (ed.), Biopharmaceutical Statistics for Drug Development. New York: Dekker, 1988, pp. 329–352. 9. D. V. Lindley, Making Decisions. London: Wiley, 1971. 10. J. Q. Smith, Decision Analysis: A Bayesian Approach. London: Chapman & Hall, 1988. 11. J. O. Berger, Statistical Decision Theory and Bayesian Analysis, New York: Springer, 1985. 12. Y. Zhou and M. Lucini, Gaining acceptability for the Bayesian decision-theoretic approach in dose escalation studies. Pharm Stat. 2005; 4: 161–171. 13. L. Hampson, Bayesian Methods to Escalate Doses in a Phase I Clinical Trial. M.Sc. dissertation. School of Applied Statistics, University of Reading, Reading, United Kingdom, 2005.

CROSS-REFERENCES Phase I trials Pharmacodynamic study Pharmacokinetic study Crossover design Analysis of variance (ANOVA) Placebo Bayesian approach

9

BENEFIT/RISK ASSESSMENT IN PREVENTION TRIALS

relatively few will receive a preventive benefit from the therapy. In this setting, the use of B/RAs provides an additional mechanism to ensure that all participants comprehend the full extent of potential benefits and risks, and that they make a well-informed decision about the interchange of benefits and risks they are willing to accept by participating in the trial. The use of B/RA in prevention trials also provides a method to evaluate the global effect of the therapy as a safeguard against subjecting trial participants to an unforeseen harmful net effect of treatment. Once the results of the trial are known and the true levels of benefits and risks of the therapy have been established, the individualized B/RA employed in the trial can become the basis for the development of a B/RA methodology that could be used in the clinical setting to facilitate the decision-making process for individuals and their health care providers who may be considering the use of preventive therapy. The trial results can also be used to develop a population-based B/RA to identify changes in patient loads for the outcomes affected by the preventive therapy that would be anticipated as health care professionals incorporate the use of the preventive therapy into their clinical practice. This information could in turn be used for decision-making regarding the planning for and use of health care resources.

JOSEPH P. COSTANTINO University of Pittsburgh, Pittsburgh, PA, USA

Benefit/risk assessment (B/RA) is a mathematical procedure to estimate the probability of detrimental outcomes, beneficial outcomes and the net-effect anticipated from exposure to a given agent. B/RAs of health-related outcomes are used for public health planning, decision-making regarding health care financing and therapeutic decision-making in clinical practice (12,24). Information obtained from B/RAs based on findings from controlled clinical trials, particularly those with doublemasking of treatment, are most informative for health care planning and decision-making because such information is less likely to be biased than is information obtained from observational studies (20,23,26). Thus, B/RAs in prevention trials are an excellent source of information to use as the basis for the types of health care planning and decision-making mentioned above. However, in a prevention trial a B/RA is primarily performed as a supplement for planning, monitoring and analyzing the trial. It is designed to provide a global assessment of all potential beneficial and harmful effects that may occur as a result of a treatment that is being evaluated as a means to reduce the incidence of some particular disease or condition. The Women’s Health Initiative (WHI), the Breast Cancer Prevention Trial (BCPT) and the Study of Tamoxifen and Raloxifene (STAR) are examples of large-scale, multicenter prevention trials which included B/RA as part of the trial methodology (7,8,22,28,29). Compared with treatment trials, the need for the type of information provided by a B/RA may be greater in prevention trials. This situation exists because prevention trials usually involve healthy persons among whom only a small proportion may develop the disease of primary interest during the course of the trial (5–713,22,29,30). As such, all participants are subjected to the risks of therapy during the course of the trial, but

1 TYPES OF B/RAs PERFORMED IN PREVENTION TRIALS In a prevention trial a B/RA can take one of three forms which can be classified according to the nature of the population that constitutes the basis for the assessment. These include assessments based on the general population, those based on the trial cohort and those based on an individual trial participant. Each of these forms of B/RA is

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

BENEFIT/RISK ASSESSMENT IN PREVENTION TRIALS

performed for specific purposes, namely to support various aspects of the conduct of the trial. A B/RA based on the general population is often performed pre-trial as part of the justification for initiating the trial. The purpose of this form of assessment is to demonstrate the potential net health benefit to society that could be obtained if the therapy being evaluated in the trial actually exhibits the efficacy that is anticipated. This type of assessment is the most generalized form. It is usually accomplished by estimating effects on a national basis assuming the therapy is administered to all susceptible individuals or to a subset of high-risk individuals and demonstrating that there is a significant net benefit when comparing the number of cases prevented with the estimates for the number of additional cases of detrimental outcomes that may be caused as a side-effect of the therapy. A B/RA based on the trial cohort is performed during the course of trial implementation as part of the safety monitoring effort. It can be accomplished in a regimented fashion as part of the formal plan for the interim monitoring of the trial or as an informal tool used by the data monitoring committee to assess the overall safety of the therapy being evaluated. This type of assessment is not usually necessary during a trial if the anticipated effects from the therapy involve only a few outcomes or if the anticipated beneficial effects substantially outweigh the anticipated detrimental effects. However, in complex situations where the anticipated outcomes affected by the therapy involve multiple diseases or conditions and/or the magnitude of the anticipated net benefit may not be large, a B/RA based on the trial cohort can be a very useful supplement for trial surveillance as a method of monitoring the global effect of all beneficial and detrimental outcomes combined. A notable difference between a B/RA based on the general population and one based on the study cohort is in the nature of the measures that are provided by these two forms of assessment. A risk assessment based on a general population provides a measure of the theoretical net effect of the therapy from estimates of

anticipated beneficial and detrimental outcomes. In contrast, a risk assessment based on the trial cohort determines the observed net effect of therapy based on outcomes actually experienced by the cohort during the course of the trial. A B/RA based on an individual trial participant is similar to that of the populationbased assessment in that it is also a theoretical estimate. In this case the assessment is not made for the general population, but instead for a specific subpopulation of persons who have the same risk factor profile (age, sex, race, medical history, family history, etc.) for the anticipated beneficial and detrimental outcomes as that of a particular individual participating in the trial. Information from this type of assessment is used to facilitate the communication to each potential trial participant of the nature of the benefits and risks that are anticipated for them as a result of taking therapy during trial participation. This type of individualized B/RA is used in prevention trials when the nature of anticipated effects is complex and benefit/risk communication is a more difficult task due to the interplay of multiple beneficial and detrimental outcomes. When it is used in this manner, it becomes an integral part of the process to obtain informed consent for each individual’s participation in the trial. 2 ALTERNATIVE STRUCTURES OF THE BENEFIT/RISK ALGORITHM USED IN PREVENTION TRIALS The core components of a B/RA are the measures of the treatment effect for each of the health outcomes that may be affected by the therapy being assessed. In this instance the treatment effect is defined as the difference between the probability that the outcome will occur among individuals who do not receive the therapy being evaluated (p0 ) and the probability that the outcome will occur among those who do receive the therapy (p1 ). For outcomes beneficially affected by therapy, the treatment effect (p0 − p1 ) will have a positive sign, representing cases prevented by therapy. For outcomes detrimentally affected by therapy, the treatment effect will have a

BENEFIT/RISK ASSESSMENT IN PREVENTION TRIALS

negative sign, representing cases caused by therapy. In its simplest structure, the benefit/risk analysis is summarized by an index of net effect (
) as the summation of treatment effects for all outcomes affected. If there are I number of outcomes affected by therapy, then the basic algorithm for the B/RA is defined as:


1 =

I


(p0,i − p1,i ).

(1)

i=1

When the sign of the index of net effect is positive, the therapy exhibits an overall beneficial health effect. When the sign is negative, the therapy has an overall detrimental effect. When dealing with a B/RA based on the trial cohort, the probabilities of (1) are obtained directly from the observations in the trial. When dealing with assessments based on the general population or the individual trial participant, the probabilities utilized as anticipated values among those who do not receive therapy (p0 ) are usually taken from some type of national database or from prospective studies of large populations that included measurements of the outcomes of interest. The probabilities used in these latter types of assessments as anticipated values among those who receive therapy are determined by multiplying the anticipated probability among those not treated by the relative risk (untreated to treated) anticipated as the treatment effect. For example, if we anticipate that treatment will reduce the incidence of a particular outcome by 35% then the anticipated relative risk would be 0.65 and the value used for p1 would be 0.65 p0 . If we anticipate that treatment will increase the incidence of an outcome by 30%, then the anticipated relative risk would be 1.30 and the value used for p1 would be 1.30 p0 . Estimates of the anticipated treatment effects for each outcome are taken from the literature dealing with pharmacokinetics, animal studies and studies in humans undertaken as preliminary investigations of the therapy as an agent to prevent the disease, or from human studies in which the therapy was being used as an agent for the treatment of disease.

3

In the prevention trial setting it is often advantageous to utilize structures of the benefit/risk algorithm other than that defined in (1). Since a B/RA based on the trial cohort is meant to be performed as part of the effort to monitor safety during the trial, an alternative structure of the benefit/risk algorithm can be used to facilitate this effort. This structure incorporates a standardization of the differences between the probabilities among those receiving and not receiving the therapy being evaluated. In this situation the index of net effect is defined as: I
(p0,i − p1,i )


2 =

i=1



I
s.e. (p0,i − p1,i )

.

(2)

i=1

In this structure, the index of net-effect (
2 ) becomes a standardized value with an N(0,1) distribution. As such, the standardized values are Z-scores. Critical values of this index of net effect in the form of Z and −Z can then be used as cut-points for global monitoring indicating that there is a significant net effect that is beneficial or detrimental, respectively. In addition to that for the standardized score, there are other structures of the algorithm used in the prevention trial setting. Instead of expressing the differences between those treated and not treated in terms of the probabilities of the anticipated outcomes, an alternative structure of the algorithm is that based on differences between treatment groups in terms of the number of cases of the outcomes. The structure of the algorithm based on the difference in terms of the number of cases is defined as:
3 =

I
(n0,i − n1,i ),

(3)

i=1

where n0 is the number of cases occurring among those who do not receive the therapy being evaluated and n1 is the number of cases among those who do receive the therapy. This structure of the algorithm is that which is utilized to perform B/RAs based on the general population. This type of assessment is meant

4

BENEFIT/RISK ASSESSMENT IN PREVENTION TRIALS

to justify the need for a trial by demonstrating the potential health benefit to society. The net effect to society is more effectively communicated to a greater proportion of individuals when it is expressed as the number of cases prevented from (3) than when it is expressed as the probability from (1). This facilitation of risk communication is also the reason that (3) is preferred over (1) for B/RAs based on individual trial participants where the specific goal of the assessment is to enhance the individual’s comprehension of benefits and risks that may be experienced as a result of trial participation. For a population-based assessment, the numbers of cases in (3) are determined by multiplying the anticipated probabilities p0,i and p1,i by the number of persons in the general population, frequently that of the total US, to obtain an estimate of the number of cases that may be prevented or caused by treatment for each outcome on an annual basis. For an individual participant-based assessment, the numbers of cases in (3) are determined by multiplying the anticipated probabilities by a fixed sample size (N) of theoretical individuals who all have a risk factor profile similar to that of the individual being assessed. A fixed period of follow-up time (t) is assumed to obtain the number of cases prevented or caused by treatment in t years among N individuals. In scenarios where the length of follow-up is long and/or the population is of older age, the estimation of n0,i and n1,i should incorporate the competing risk of mortality that would be anticipated. If d is the probability of dying and RR is the relative risk anticipated for the outcome of interest, the adjusted expected number of cases among those not treated can be calculated as:   p0,i [1 − exp{−t(p0,i + di )}], n0,i = N (p0,i + di ) (4) and the adjusted expected number of cases among those treated can be calculated as:   RRp0,i n1,i = N (RRp0,i + di ) ×[1 − exp{−t(RRp0,i + di )}].

(5)

In most prevention trials the outcomes that are potentially affected by the therapy

being evaluated encompass a wide range of severity. A simple adding together of the risks of beneficial and detrimental effects without including a consideration of the relative severity of the outcomes may not be appropriate or desirable. For example, suppose a therapy is anticipated to prevent breast cancer and hip fractures, but may cause an increase in uterine cancer and cataracts. Is it appropriate to equate one case of breast cancer prevented to one case of cataracts caused or equate one case of hip fracture prevented to one case of uterine cancer caused? In situations where it is important to include a consideration of the relative severity of the outcomes affected by the therapy, the equations described above for determining the index of net effect can be modified to incorporate a weighting of the outcomes. If wi is used to represent the weight for each of the I outcomes, then the modification to (3) to incorporate weighting of the outcomes is:
4 =

I


wi (n0,i − n1,i ).

(6)

i=1

Equations (1) and (2) can be modified in a similar fashion by including wi as a multiplier of the quantity of difference in the probabilities. 3 METHODOLOGICAL AND PRACTICAL ISSUES WITH B/RA IN PREVENTION TRIALS There are several issues to be faced when performing a B/RA in a prevention trial. These issues concern the variability of the index of net effect, weighting the outcomes by severity, estimating the index of net effect for individuals with specific profiles of risk factors and communicating the findings of a B/RA to individual participants. Some discussion of each of these issues is presented below. The estimates of p0,i and p1,i used in a B/RA have a variability associated with them in terms of the strength of evidence supporting the treatment effect and in terms of the precision of the treatment effect. If this variability is substantial, then it may be necessary to incorporate consideration of the variability into the B/RA. Freedman et al. (8,25) have described a Bayesian approach

BENEFIT/RISK ASSESSMENT IN PREVENTION TRIALS

to incorporating a measure of variability into the index of net effect when it is measured in the form of weighted, standardized probabilities. They assume a skeptical prior distribution based on the strength of the preliminary evidence used as the anticipated treatment effect for each outcome potentially affected by therapy. Gail et al. (10) have described a method to incorporate a measure of variability into the estimate of the index of net effect measured in the form of a weighted number of cases. Their method involves bootstrapping, based on the 95% confidence intervals of the anticipated relative risk associated with treatment for each outcome, to determine the probability that the net number of cases is greater than zero. The values used for weighting the differences between those treated and not treated can be based on a utility function related to the severity of the outcome, preferences in terms of levels of risk acceptability or other considerations. However, the best choice of a utility function is not always obvious. A measure of mortality such as the case-fatality ratio is one possible utility. If this type of weighting is used, then the choice of the one-year, five-year or ten-year case-fatality ratios would be an issue because the relative weighting of the outcomes could likely be very different depending on which time period for case-fatality is used. Also, weights based on case-fatality would eliminate the consideration of any nonfatal outcome, which would not be preferable if there were several nonfatal outcomes of interest or if a nonfatal outcome has a significant impact on morbidity. Issues also arise with the use of rankings based on the impact on quality of life or preferences regarding the acceptability of risk (1,11). The issues with these utilities arise because the rankings are often subjective in nature, based on the opinions of a relatively small panel of individuals, and it is possible that the rankings of outcomes could differ substantially depending on the population from whom the opinions are ascertained (2,15,16). In light of these issues, attempting to identify a basis for weighting a B/RA is a practical problem that can be difficult to resolve. The preferred choice for any particular trial could differ from one group of individuals to another. As such, if a B/RA is planned as part of

5

trial monitoring, it is essential that the data monitoring committee reviews and reaches a consensus regarding the proposed weighting before it initiates review of the outcome data. To accomplish the individualization desired for B/RAs based on individual trial participants, it is necessary to provide estimates of effect specific to the individual’s full spectrum of risk factors for each of the outcomes expected to be affected by the therapy of interest. A problem likely to be faced when performing individualized assessments is the unavailability of probability estimates specific to the individual’s full set of risk factors. For outcomes with several relevant risk factors to be considered or for outcomes that have not been studied in diverse populations, estimates of the outcome probabilities for a specific category of risk factor profiles may not exist. In some cases, multivariate regression models are available that can be used to predict probabilities of outcomes for specific risk factor profiles from data based on the general population. Examples of such models include those for heart disease, stroke and for breast cancer (4,9,1417–19). However, the models currently available are primarily limited to those for the more common diseases and are not generally applicable to all race and sex populations. Also, relatively few of these models have been well validated. Thus, in practice it is often necessary to use estimates of outcome probabilities for individualized B/RAs that are taken from populations that are more representative of the general population than of the population specific to the risk factor profile of the individual being assessed. When this is the case, the limitations of the methodology need to be recognized and used in this light. Nonetheless, a B/RA that has been individualized to the extent possible is more informative to a trial participant than one based on the general population. Additional discussions of the limitations of individualized B/RAs can be found in presentations concerning individualized B/RAs for the use of tamoxifen to reduce breast cancer risk (3,4,27). Communicating the results of a B/RA to an individual is a skilled task. An effort must be made to provide information in a manner that facilitates the individual’s comprehension (21). Tools are needed to facilitate this

6

BENEFIT/RISK ASSESSMENT IN PREVENTION TRIALS

Table 1. Example of Data Presentation Tool for Communicating the Benefits and Risks of Tamoxifen Therapy The information below provides the number of certain events that would be expected during the next five years among 10 000 untreated women of your age (ageX ), race (raceY ) and five-year breast cancer risk (riskZ ). To help you understand the potential benefits and risks of treatment, these numbers can be compared with the numbers of expected cases that would be prevented or caused by five years of tamoxifen use Severity of event Life-threatening events

Other severe events

Other events

Expected number of cases among 10 000 untreated women

Expected effect among 10 000 women if they all take tamoxifen for five years

Invasive breast cancer Hip fracture

N0,1 cases expected N0,2 cases expected

Potential benefits N1,1 of these cases may be prevented N1,2 of these cases may be prevented

Endometrial cancer Stroke Pulmonary embolism

N0,3 cases expected N0,4 cases expected N0,5 cases expected

Potential risks N1,3 more cases may be caused N1,4 more cases may be caused N1,5 more cases may be caused

In situ breast cancer

N0,6 cases expected

Potential benefit N1,6 of these cases may be prevented

Deep vein thrombosis

N0,7 cases expected

Potential risk N2,7 more cases may be caused

Potential benefits:

Tamoxifen use may reduce the risk of a certain type of wrist fracture called Colles’ fracture by about 39%, and also reduce the risk from fractures of the spine by about 26%. Tamoxifen use may increase the occurrence of cataracts by about 14%.

Type of event

Potential risk:

effort. These tools must be developed before the initiation of the trial and included as part of the protocol approved by the Institutional Review Board. Relatively little work has been done in the area of developing tools for communicating the benefits and risks of participation in a prevention trial. However, some tools have been developed that serve as examples for future development. Tools to enhance the communication of B/RA information to women screened for participation were developed for use in the BCPT (7,22). Since the conclusion of this trial, the tools were refined for use in the STAR trial (28). Table 1 provides an example of the type of tool used in the STAR trial to inform potential participants regarding their individualized B/RA. This tool was developed based on the principles put forth by the participants of the National Cancer Institute’s workshop convened to develop information to assist in counseling women about the benefits and risks of tamoxifen when used to reduce

the risk of breast cancer. This workshop and the specific methodology used for the B/RA are described by Gail et al. (10). There were several key working premises that guided the development of the STAR trial tool displayed in Table 1. The premises were considerations of form and format to facilitate the participant’s comprehension of their individualized B/RA. These working premises were to: (1) avoid the use of probabilities and relative risk as these concepts are not readily understood by the nonstatistician; (2) provide information for each outcome anticipated to be affected by therapy; (3) group the information presented by severity of the outcomes; (4) provide detailed information for the outcomes with more severe consequences and provide an estimate of effects among those not treated so the individual can understand the context in which to place the expected treatment effects; and (5) limit the tool to one page of data presentation to reduce the amount of data overload perceived by the individual.

BENEFIT/RISK ASSESSMENT IN PREVENTION TRIALS

The precise considerations involved in any prevention trial may differ; however, working premises of this nature designed to enhance comprehension should always be employed when developing tools to communicate B/RA information to potential trial participants. REFERENCES 1. Bennett, K. J. & Torrance, G. W. (1996). Measuring health state preferences and utilities: ratings scale, time trade-offs and standard gamble techniques, in Quality of Life and Pharmacoeconomics in Clinical Trials, 2nd Ed., B. Spilker, ed. Lippincott-Raven, Philadelphia, pp. 253–265. 2. Boyd, N. F., Sutherland, H. J., Heasman, K. Z., TritchlerD. L. & Cummings B. J. (1990). Whose utilities for decision analysis?, Medical Decision Making 1, 58–67. 3. Costantino, J. P. (1999). Evaluating women for breast cancer risk-reduction therapy, in ASCO Fall Education Book. American Society of Clinical Oncology, pp. 208–214. 4. Costantino, J. P., Gail, M. H., Pee, D., Anderson, S., Redmond, C. K. & Benichou, J. (1999). Validation studies for models to project the risk of invasive and total breast cancer incidence, Journal of the National Cancer Institute 91, 1541–1548. 5. Cummings, S. R., Echert, S., Krueger, K. A., Grady, D., Powles, T. J., Cauley, J. A., Norton, L., Nickelsen, T., Bjarnason, N. H., Morrow M., Lippman M. E., Black, D., Glusman, J. E. & Jordan, V. C. (1999). The effect of raloxifene on risk of breast cancer in postmenopausal women: results from the MORE randomized trial, Journal of the American Medical Association 281, 2189–2197. 6. Ettinger, B., Black, D. M., Mitlak B. H., Knickerbocker, R. K., Nickelsen, T., Genant, H. K., Christiansen, C., Delmas, P. D., Zanchetta, J. R., Stakkestad, J., Gluer, C. C., Krueger, K., Cohen, F. J., Eckert, S., Ensrud, K. E., Avioli, L. V., Lips, P. & Cummings, S. R. (1999). Reduction of vertebral fracture risk in postmenopausal women with osteoporosis treated with raloxifene: results from a 3-year randomized clinical trial, Journal of the American Medical Association 282, 637–645. 7. Fisher, B., Costantino, J. P., Wickerham, D. L., Redmond, C. K., Kavanah, M., Cronin, W. M., Vogel, V., Robidoux, A., Dimitrov, N., Atkins, J., Daly, M., Wieand, S., Tan-Chiu, E., Ford, L. & Wolmark, N. (1998). Tamoxifen

7

for prevention of breast cancer: report of the National Surgical Adjuvant Breast and Bowel Project P-1 study, Journal of the National Cancer Institute 90, 1371–1388. 8. Freedman, L., Anderson, G., Kipnis, V., Prentice, R., Wang, C. Y., Rousouw, J., Wittes, J. & DeMets, D. (1996). Approaches to monitoring the results of long-term disease prevention trials: examples from the Women’s Health Initiative, Controlled Clinical Trials 17, 509–525. 9. Gail, M. H., Brinton, L. A., Byar, D. P., Corle, D. K., Green, S. B., Schairer, C. & Mulvihill, J. J. (1989). Projecting individualized probabilities of developing breast cancer for white females who are being examined annually, Journal of the National Cancer Institute 81, 1879–1886. 10. Gail, M. H., Costantino, J. P., Bryant, J., Croyle, R., Freedman, L., Helzlsouer, K. & Vogel V. (1999). Weighing the risks and benefits of tamoxifen for preventing breast cancer, Journal of the National Cancer Institute 91, 1829–1846. 11. Guyatt, G., Feeny, D. & Patrick, D. (1993). Measuring health-related quality of life, Annuals of Internal Medicine 118, 622–629. 12. Haynes, R. B., Sackett, D. L., Gray, J. A. M., Cook, D. J. & Guyatt, G. H. (1996). Transferring evidence from research to practice: 1. The role of clinical care research evidence in clinical decisions, APC Journal Club 125, A14–A15. 13. Hulley, S., Grady, D., Bush, T., Furberg, C., Herrington, D., Riggs, B. & Vittinghoff, E. (1998). Randomized trial of estrogen plus progestin for secondary prevention of coronary heart disease in postmenopausal women. Heart and Estrogen/Progestin Replacement Study (HER) Research Group, Journal of the American Medical Association 280, 605–613. 14. Liao, Y., McGee, D. L., Cooper, R. S. & Sutkowski, M. B. (1999). How generalizable are coronary risk prediction models? Comparison of Framingham and two other national cohorts, American Heart Journal 137, 837–845. 15. Llewellyn-Thomas, H. A. (1995). Patients’ health care decision making: a framework for descriptive and experimental investigations, Medical Decision Making 15, 101–106. 16. Llewellyn-Thomas, H. A., Naylor, C. D., Cohen, M. N., Baskiniski, A. S., Ferris, L. E. & Williams, J. E. (1992). Studying patients’ preferences in health care decision making,

8

17.

18.

19.

20.

21.

22.

BENEFIT/RISK ASSESSMENT IN PREVENTION TRIALS Canadian Medical Association Journal 147, 859–864. Lloyd-Jones, D. M., Larson, M. G., Beiser, A. & Levy, D. (1999). Lifetime risk of developing coronary heart disease, Lancet 353, 89–92. Manolio, T. A., Kronmal, R. A., Burke, G. L., O’Leary, D. H. & Price, T. R. (1996). Shortterm predictors of incidence stroke in older adults. The Cardiovascular Health Study, Stroke 27, 1479–1486. Menotti, A., Jacobs, D. R., Blackburn, H., Krombout, D., Nissinen, A., Nedeljkovic, S., Buzina, R., Mohacek, I., Seccareccia, F., Giampaoli, S., Dontas, A., Aravanis, C. & Toshima, H. (1996). Twenty-five year prediction of stroke deaths in the seven countries study: the role of blood pressure and its changes, Stroke 27, 381–387. Pocock, S. J. & Elbourne, D. R. (2000). Randomized trials or observational tribulations?, The New England Journal of Medicine 342, 1907–1909. Redelmeier, D. A., Rozin, P. & Kahneman D. (1993). Understanding patients’ decision–cognitive and emotional perspectives, Journal of the American Medical Association 270, 72–76. Redmond, C. K. & Costantino, J. P. (1996). Design and current status of the NSABP Breast Cancer Prevention Trial, Recent Results in Cancer Research 140, 309–317.

23. Sackett, D. L. (1997). Bias in analytical research, Journal of Chronic Diseases 32, 51–63. 24. Simon, G., Wagner, E. & VonKorff, M. (1995). Cost-effectiveness comparisons using ‘‘real world’’ randomized trials: the case of the new

antidepressant drugs, Journal of Clinical Epidemiology 48, 363–373. 25. Spiegelhalter, D. J., Freedman, L. & Parmar, M. K. B. (1994). Bayesian approaches to randomization clinical trials, Journal of the Royal Statistical Society, Series A 157, 357–416. 26. Steineck, G. & Ahlbom, A. (1992). A definition of bias founded on the concept of the study base, Epidemiology 3, 477–482. 27. Taylor, A. L., Adams-Cambell, L. & Wright, J. T. (1999). Risk/benefit assessment of tamoxifen to prevent breast cancer—still a work in progress, Journal of the National Cancer Institute 19, 1792–1973. 28. Wolmark, N., Wickerham, D. L., Costantino, J. P. & Cronin, W. (1999). NSABP Protocol P2: Study of Tamoxifen and Raloxifene (STAR) for the Prevention of Breast Cancer. National Surgical Breast and Bowel Project, Pittsburgh, Pennsylvania. 29. Women’s Health Initiative Study Group (1998). Design of the Women’s Health Initiative clinical trial and observational study, Controlled Clinical Trials 19, 61–109. 30. Writing Group for the PEPI Trial (1995). Effects of estrogen/progestin regimens on heart disease risk factors in postmenopausal women: the Post-menopausal Estrogen/Progestin Intervention (PEPI) Trial, Journal of the American Medical Association 273, 199–208.

BIASED COIN RANDOMIZATION

is b, and the maximum number of consecutive subjects with the same treatment is 2b. This method is most commonly used in clinical trial practice. However, it may be possible to predict or guess future treatments for subjects in the later part of each block with knowledge of the treatment for patients earlier in the block. For example, with two treatment groups and a block size of 4, an initial allocation of two patients to treatment A will mean that the next two subjects will receive treatment B. As such, the treatment allocation is deterministic for patients toward the end of blocks. In blinded trials, the block size is usually not disclosed because of the potential for unblinding and bias. However, even this strategy may not afford total protection from selection bias. If study staff are able to make a good guess at the previous treatment—based on frequent occurrence of known treatmentrelated adverse events or strong efficacy results (sometimes called functional unblinding)— then selection bias may still occur in PBR. The aim of this article is to describe a class of randomization procedures called biased coin randomization procedures, which are designed to eliminate or substantially reduce the problems of the CR and PBR procedures by

MIKE D. SMITH Clinical Statistics Pfizer Global Research & Development New London, Connecticut

1 RANDOMIZATION STRATEGIES FOR OVERALL TREATMENT BALANCE In a ‘‘simple’’ or ‘‘complete’’ randomization (CR) scheme, subjects are allocated to treatment groups based on a fixed probability without any regard to previous allocation of subjects or level of current imbalance. In the case of equal randomization to two groups, this procedure is equivalent to tossing a coin. Because it is based on the inability to predict or guess the treatment for future patients, this procedure is free from selection bias (1) , which is the bias introduced into a trial from awareness of a patient’s treatment allocation. The CR procedure does not guarantee overall treatment balance at the end of the trial, and there is a potential during the trial for a long run of one treatment versus another. For these reasons, the CR procedure is rarely used. Avoiding imbalance at specific time points during the trial is particularly important in the implementation of adaptive designs based on interim analysis of data. An early alternative to CR that addressed the issue of potential treatment imbalance is the nonrandom systematic design (e.g., ABBAABBAA), which is not discussed here as it contains no element of randomization. An alternative randomization procedure that aims to limit imbalance throughout the trial is ‘‘permuted block randomization’’ (PBR) (2). For a fixed number of treatments groups, fixed-sized blocks are generated that contain all possible treatment permutations. The full randomization list is then generated by randomly selecting from this full set of treatment blocks until the required sample size is achieved. Under PBR, the magnitude of treatment imbalance and the maximum number of consecutive subjects receiving the same treatment are both limited. In the case of two treatment groups, with a block size of 2b the maximum treatment imbalance

• minimizing the size any treatment

imbalance in a trial (and within each strata) and reducing the chance of long runs of one treatment group. • removing any deterministic treatment allocation for patients to treatment and so eliminating selection bias. The randomization procedures discussed here, including biased coin randomization and others, are described in detail in Rosenberger and Lachin (3), and summarized in Chow and Chang (4). 2 THE BIASED COIN RANDOMIZATION PROCEDURE Efron (5) first described a biased coin randomization procedure for two treatment groups

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

BIASED COIN RANDOMIZATION

(A and B) where equal treatment allocation over a whole study is required. We can consider this random allocation as tossing a coin. If the current treatment allocation is balanced, then the coin used will be a fair unbiased coin. However, if there is any imbalance in the current treatment allocation, then the coin will have a fixed bias favoring the treatment with the smaller number of subjects. Thus, the bias is independent of the size of treatment imbalance. Let nA and nB be the current number of subjects assigned to treatment A and B, respectively. The probability of the next subject being assigned to treatment A in the trial is   1 − p nA > nB nA = nB Probability (treatment A) = 0.5  p nA < nB , where p represents the bias in the randomization ‘‘coin’’ (1 ≥ p ≥ 0.5). This procedure is labeled BCD(p). Complete randomization is equivalent to using BCD(0.5). A block randomization using a block size of 2 is equivalent to using BCD(1), which is not usually considered as it has deterministic allocation for each alternate subject with the associated potential for selection bias. Efron (5) favored the value BCD(2/3) to provide a clear bias for the coin while maintaining an adequate random element. Pocock (6) suggested that BCD(3/4) is too predictable in the presence of imbalance, BCD(2/3) is appropriate for a relatively small trial; and BCD(3/5) is adequate for a larger trial with at least 100 patients. Operationally, the BCD(p) procedure uses three distinct fixed-randomization schedules, using the probability of being assigned to treatment A. For an investigator-blinded trial or a trial with multiple investigator sites, the three fixed randomization lists would be

administered centrally and not at individual sites. Within a stratified randomization (with one stratification factor), it would be proposed that each stratum be treated as a separate trial. Therefore, an attempt is made to achieve treatment balance for each stratum and thereby over the whole trial. A discussion of a more complex stratification procedure is described later.

3 PROPERTIES The BCD(p) procedure produces a Markov Chain random walk process for the difference DN = nA − nB . Under the assumption that both positive and negative imbalances are equally important to avoid, then the properties of the BCD(p) procedure can be evaluated more simply by considering |DN |. The transition matrix for |DN | is formed by considering the probability of moving from the current state to all possible states for the next patient. If balance is achieved at any stage in the study, the probability of an imbalance of 1 following the next subject is 1. If the study is imbalanced at any stage in the study, the probability of greater imbalance will be 1 − p and the probability of lesser imbalance will be p. The limiting probabilities (N = nA + nB → ∞) of imbalance of size j can be calculated from the transition matrix for |DN | (3). These are shown in Table 1 for even and odd N. The likelihood of perfect balance (for even N) or an imbalance of 1 (for odd N) tends to 1 as r → ∞ (p → 1); however, the deterministic nature of treatment allocation will increase, and so increase the potential for selection bias. An illustration of the limiting probabilities of imbalance for BCD(0.60), BCD(0.75) and BCD(0.90) is given in Figure 1A and B.

Table 1. Probability of imbalance of size j for the BCD(p) in terms of r = p/(1 – p) Even N

Odd N

1 − 1/r (j = 0)

r2 −1 (odd rj+1

r2 −1 (even rj+1

j, j ≥ 2)

j, j ≥ 1)

1.0

1.0

0.8

0.8 Probability

Probability

BIASED COIN RANDOMIZATION

0.6 0.4 0.2

3

0.6 0.4 0.2

0.0

0.0 0

1

2

3

4

5

6

7

8

9

10 11

0

1

2

3

4

Imbalance

5

6

7

8

9

10 11

Imbalance

Figure 1. Probability of imbalance for (A) even and (B) odd total number of subjects using biased coin design with p = 0.60 (blue), 0.75 (red), 0.90 (green).

30 Difference (Dn = nA–nB)

Difference (Dn = nA–nB)

30 20 10 0 −10 −20 −30 0

100 Subject Number

200

20 10 0 −10 −20 −30 0

100 Subject Number

200

Difference (Dn = nA–nB)

30 20 10 0 −10 −20 −30 0

100 Subject Number

200

Figure 2. Difference in number of subjects by additional recruited subjects (A) BCD(0.60), (B) BCD(0.75), and (C) BCD(0.90) with starting imbalance of +20 (red line), 0 (blue line), –20 (green line).

An example of this procedure would be an ongoing clinical trial where a further 200 subjects will be randomized. Let the current level of imbalance in this example be Dn = +20, 0, and −20 shown as the red, blue, and green curves, respectively, in Figure 2A–C. We can

see the difference DN = nA − nB for the additional 200 randomized subjects using the BCD(0.60), BCD(0.75), and BCD(0.90) procedures (as A, B, and C, respectively). The plots indicate that as p increases we achieve faster convergence to Dn = 0, given initial

4

BIASED COIN RANDOMIZATION

imbalance and a lower level of imbalance around Dn = 0. Many investigators use the expected number of correct guesses of future treatment assignment as a surrogate measure of potential selection bias (5). Wei (7) showed that the number of correct guesses of the next treatment assignment is much lower for BCD than for PBR. The CR procedure has the minimum expected number of correct guesses, and so (as expected) it minimizes the potential section bias. The number of correct guess is greater for BCD compared with CR; however, this difference decreases as studies get larger.

4 EXTENSIONS TO THE BIASED COIN RANDOMIZATION 4.1 Unequal Randomization Ratio A simple extension to the BCD procedure is to allow for unequal treatment allocation. If the required randomization ratio (A:B) is k, the probability of assigning the next subjects to treatment A could be Probability (treatment A)  k   k+1 − p nA > knB k = k+1 nA = knB   k + p nA < knB . k+1 This procedure will further modify the bias of the originally biased coin. The initial coin k and this bias is either increased or bias is, k+1 decreased by p in cases of imbalance from the planned randomization ratio, subject to the   k constraint 1 − k+1 > p > 0 (where k > 1). 4.2 More Than Two Treatments The BCD procedure can be easily modified to >2 treatment groups. Under equal randomization (with t treatment groups and a total of N subjects), the expected number of subjects in each treatment group will be N/t. In this procedure, the coin bias is applied to each treatment group and is established by comparing the current individual treatment allocation again the expected level.

We define the probability of assigning the next patient to treatment i(i = 1, . . . , t) as follows: Probability (treatment i)   c(1/t − p) ni > N/t = pi = c(1/t) ni = N/t   c(1/t + p) ni < N/t, where ni is the current number of subjects for treatment i, 1/t > p > 0, and c is a rescaling value such that the probabilities sum to 1( ti=1 pi = 1) 1. As before, larger values of p lead to faster convergence with N/t and subsequently to a lower level of imbalance around N/t. Figure 3(A and B) shows an example of the proportion of subjects assigned to each of four treatment groups for an intended randomization ratio of 1:1:1:1, using values of p equal to 0.05 and 0.15 for 200 subjects. As with before in the two treatment case, a larger value of p will tend the proportion of subjects allocated to each treatment more quickly toward the expected value (here N/t) and maintain a smaller imbalance around the expected value. Over a large number of subjects, the proportion of subjects allocated to each treatment will tend to N/t regardless of the choice of p. However, the choice of p does greatly influence the proportion of subjects allocated to each treatment over a small or moderate number of subjects. The choice of p will also influence the chance of long runs of one treatment group, regardless of the number of subjects in the trial. 5 ADAPTIVE BIASED COIN RANDOMIZATION The coin bias in the original BCD procedure is constant, regardless of the size of the treatment imbalance (DN = nA − nB ) or the current size of the study (N = nA + nB ). For example, the probability of the next subject being assigned to treatment A is the same, regardless of whether DN is 1 or 100, or whether N is 20 or 2000. An extension to the simple BCD(p) procedure is to modify the coin bias according to the current size of the imbalance. We will describe this using

1.0

1.0

0.8

0.8 Proportion

Proportion

BIASED COIN RANDOMIZATION

0.6 0.4

5

0.6 0.4 0.2

0.2

0.0

0.0 0

50

100 150 Subject Number

0

200

50

100

150

200

Subject Number

Figure 3. Proportion of subjects allocated to each of four treatment groups using (A) p = 0.05 and (B) p = 0.15. Table 2. Probability of imbalance of size j for the Big Stick Rule

Even c Odd c

Even N

Odd N

1/c (j = 0 or c) 2/c (even j, c–2 ≥ j ≥ 2) 1/c (j = 0) 2/c (even j, c – 1 ≥ j ≥ 2)

2/c (odd j, c–1 ≥ j ≥ 1) 1/c (j = c) 2/c (odd j, c – 2 ≥ j ≥ 1)

both step and continuous functions for the coin bias. A further extension is to also modify the coin bias based on the current size of the study, and this will be described for continuous coin bias functions.

on whether N and c are odd or even. The

5.1 Step Functions for Coin Bias

ing the deterministic nonrandom treatment

Rosenberger and Lachin (3) describe two modified BCD procedures using a step function for coin bias based on DN. The first is called the Big Stick Rule (8) and uses an unbiased coin for treatment allocation where the current imbalance |DN | < c for some predefined critical value of c. Where the imbalance reaches this critical value, the probability of being assigned to treatment A is 0 (DN = c) or 1 (DN = −c). The imbalance between treatment groups is limited to c, and the maximum consecutive allocation of one treatment group is 2c + 1. There is a chance that treatment allocation is determined in a nonrandomized manner when the maximum imbalance is reached, and the frequency of this depends on the choice of c. A value of c = ∞ gives the BCD(p) design. Using the random walk process, we can calculate the long-term probabilities of imbalance of size j, which depend

allocation is 1/c (when N and c are either both

long-term probability of imbalance if size j for the Big Stick Rule is shown in Table 2. Therefore, the long-term probability of apply-

odd or both even), and it will decrease as c increases. The second modified BCD procedure described by Rosenberger and Lachin (3) using a step function for coin bias is the ‘‘biased coin design with imbalance intolerance’’ (9). This procedure is a mixture of Efron’s original BCD procedure and the Big Stick Rule. The treatment allocation is deterministic where the imbalance exceeds some critical value as in the Big Stick Rule; however, the biased coin is still used for any imbalance below the critical level as in Efron’s

6

BIASED COIN RANDOMIZATION

procedure. This rule is shown here:

and p. Therefore, the treatment assignment in this procedure would be:

Probability (treatment A)   0 Dn = c      1 − p c > Dn > 0 = 1/2 Dn = 0   p 0 > Dn > −c     1 Dn = −c,

  1 − p DN = c Probability (treatment A) = 1/2 |DN | < c   p DN = −c.

where 1 ≥ p ≥ 0.5 (a value of 0.5 gives the big stick design, and a value of 1 gives the big stick design with c = 1). Chen (9) gives the probability of imbalance of size j. In particular, the probability of imbalance of size c (and so the probability of applying the deterministic   c−1  treatment allocac−1 q , where q = 1 − p. tion) is (p−q)p c c p −q pc−1 This is always less than 1/c, so the likelihood of reaching the imbalance boundary is less with this procedure compared with the Big Stick Rule. Similarly even  N), the  (for c−1 (p−q)p , which is probability of balance is pc −qc always greater than 1/c, so the likelihood of achieving balance is greater with this procedure compared with the Big Stick Rule. Both the Big Stick Rule and the biased coin design with imbalance intolerance procedures use nonrandom allocation of treatment to patients when imbalance has reached a predefined limit, which creates hard boundaries of −c and c for Dn . Pocock (6) describes a modified procedure that avoids the deterministic allocation in the Big Stick Rule by replacing the probabilities of 0 and 1 with 1 − p

A similar modification to the biased coin design with imbalance intolerance design might be to replace 0 and 1 with 1 − p2 and p2 (such that 1/2 < p < p2 < 1). This would give greater balance compared with Pocock’s procedure, but at the expense of a more complex strategy now requiring the definition of two levels of coin bias rather than one. In this procedure, the imbalance is now not limited by a hard boundary of c but rather by what might be viewed as a ‘‘soft’’ limit. An illustration of this procedure (with p = 0.60, p2 = 0.90, and a soft limit of 3) is compared with Chen’s design (with p = 0.60 and a hard limit of 3) in Figure 4. 5.2 Continuous Functions for Coin Bias A continuous function can be defined for the treatment allocation coin bias such that it is both (1) directly proportional to the size of the current treatment imbalance and (2) inversely proportional to the current size of the study. This provides the benefit that changes to treatment balance are greatest earlier in the trial and/or where the treatment imbalance is greatest. The use of a continuous function for coin bias means that

Difference (Dn = nA–nB)

5 3

0

−3 −5 0

25

50 Subject Number

75

100

Figure 4. Difference in number of subjects (A-B) by subject number for adaptive step function with hard (red) and soft (blue) limits of 3 using p = 0.60 and p2 = 0.90.

BIASED COIN RANDOMIZATION

the arbitrary and subjective cut-off points in the step functions described previously do not need to be defined. Two examples of a continuous function for coin bias are shown here. As before, for two treatment groups, and shown in terms of the probability of assigning the next subject to treatment A, n2B n2B

+ n2B

(10) and ω + βnB 2ω + β(nA + nB ) for some ω,β ≥ 0(11). Wei (7, 12) describes a general class of functions for coin bias and defines a set of properties that this class of function should meet. The CR, PBR, and BCD procedures are shown to fit within this general framework. For two treatment groups, the probability of assigning the next subject to treatment A would be  nA − nB , p(Dn /N) = p nA + nB where p(x) is a nonincreasing function satisfying p(x) + p(−x) = 1 for −1≥x≥1. This class of functions need not be continuous (e.g., the step functions for coin bias described in section 5.1 would satisfy these conditions). However, Wei (7, 12) shows that if the function p(x) is continuous at x = 0 then selection bias will tend to 0 as the sample size increases. ρ The function p(x) = (1+x)(1−x) ρ +(1−x)ρ with x = Dn /N, yields the following function for the probability of assigning the next subject to treatment A: ρ

ρ nA

nB ρ + nB

(13). This function is a more general form of the rule by Atkinson (10). The function p(x) = (1 − x)/2 with x = Dn /N gives the allocation rule for the probability of being assigned to treatment A as: nB nA + nB

7

(12). In these procedures, as for the previous BCD strategies, the coin bias will favor randomization toward the under represented treatment group. However, by using these continuous functions for coin bias, the coin bias is affected by the size of the imbalance. Additionally, as the denominator in all these functions includes nA and nB , the coin bias will tend to 0 as the study gets larger; thus, these procedures will tend to CR as the sample size increases. Another general class of continuous coin bias functions, given by Baldi Antognini and Giovagnoli (14), is the adjustable biased coin designs (ABCDs). The ABCDs are based on Dn rather than Dn /N. In these functions, the coin bias does not necessarily tend to 0 as the study gets larger. This property can be viewed as either an advantage (14) or a disadvantage (12). One ABCD function highlighted by Baldi Antognini and Giovagnoli (14) is  |D |a n   |Dn |a +1 Dn < 0 Probability(treatment A) = 0.5 Dn = 0   1 D n>0 |Dn |a +1 where a ≥ 0, a function labeled as ABCD(Fa). Smaller values of a will increase the chance of some treatment imbalance but will decrease selection bias (the special case a = 0 would give CR with no selection bias). The function ABCD(Fa) will tend to the big stick design with boundaries of ± 2 as a → ∞. The effect of a can be dramatic even when the treatment balance is not large. For example, the probability of being assigned to treatment A is r ≈ 2 1 −  r

3 (nA + nB )



(3). Note that this probability is independent of both ω and β. There is a direct equivalence between the BCD and the P´olya UD, where the UD(ω,β) is equivalent to the BCD using the allocation probability defined  by the continuous  ω+βnB (11). function for coin bias of 2ω+β(n +n ) A B The special case UD(0,1) is equivalent to the BCD with allocation probability proportional to the number ofsubjects  allocated to treatnB ment group A of n +n . A B Wei and Lachin (17) show the following properties of the UD(ω,β) as compared with other randomization procedures: • The chance of imbalance for UD(ω,β) is

far less likely compared with CR.

• The potential for selection bias for UD

(ω,β) is less compared with BCD(p) and PBR, and it tends to the minimum selection bias possible (i.e., under CR) as the study size gets larger. • As the study size gets larger, UD(ω,β) is free from accidental bias (which may be caused by an imbalance of a known or unknown prognostic factor). The extension of this design to more than two treatments is simple (19). The initial urn contains ω distinct balls for each treatment group. A ball is removed from the urn at random and replaced. The next patient is allocated the treatment corresponding to this drawn ball. The urn is updated by adding β balls for all the other treatment groups. The probability of the next subject receiving treatment i(i = 1, . . . , t) is 

ω + β(N − ni ) tω + β(t − 1)N

,

where N is the total number of subjects (N = n1 + . . . + nt ). The probability of imbalance for treatment group i versus perfect balance

10

BIASED COIN RANDOMIZATION

is 

 Pr ni − N/t > r   

 t2 (t + 1)  . ≈ 2 1 −  r N(t − 1)2

Again the long-term probability of imbalance is independent of both ω and β. The UD has a particularly simple application to response-adaptive randomization and is described later. 6.2 Urn Model without Replacement The urn design UD(N/2,0) using sampling from the urn without replacement is equivalent to the random allocation rule (21). In this design, treatment balance at the end of the trial is guaranteed, given that the total number of subjects required for the trial is known and fixed in advance. The major criticism of this procedure is that once all N/2 subjects are allocated for either group then the allocation for the remaining subjects must all be assigned to a single treatment group, which may increase both selection and accidental bias, the latter due to time trends in patient characteristics, for example. Another criticism is that this procedure may still lead to imbalances during the trial or a long series of one treatment group. The former might be an important consideration if the trial uses interim analyses or is terminated early. 6.3 Friedman’s Urn Model Friedman (18) introduced a more general urn model whereby a ball is removed at random from the urn (initially containing ω balls of two colors as before). The ball is then replaced together with α balls of the same color and β balls of the opposite color. Treatment allocation for the next patient is chosen from the color of the selected ball. This design will be labeled as UD(ω,α,β). The model UD(ω,0,β) is the P´olya Urn Model discussed previously, which is usually described using two parameters, UD(ω,β). To achieve treatment balance, we require α, β > 0 and also β > α (the case where β = α is CR). Wei (19) confirms that the UD(ω,α,β)

tends to the CR procedure as the study size gets larger as for the simpler UD(ω,β). The UD(ω,α,β) also has the same properties of asymptotic balance and is free from selection and accidental bias. The UD(ω,α,β) tends to balance more quickly as β/α gets larger (19), so the P´olya design of UD(ω,0,β) will tend to balance more quickly that the generalized Friedman design with α > 0. This model is equivalent to a BCD, with an adaptive probability of the next subject being assigned to treatment A as 

ω + αnA + βnB 2ω + (α + β)(nA + nB )



The extension of the Friedman urn model to more than two treatment groups (t > 2) is described by Wei (19), and it follows the same procedure as for the simpler P´olya urn model. Following the random selection and replacement of a ball from the urn, the urn is then updated by adding β balls for each of the other treatment groups and α balls for the selected treatment group. The probability of the next subject receiving treatment is  i (i = 1, . . . , t) is

ω + αni + β(N − ni ) tω + (α + β(t − 1))N

,

where N is the total number of subjects (N = n1 + . . . + nt ). The probability of imbalance for treatment group i versus perfect balance for large N (where (t + 1) β > α) is 

 Pr ni − N/t > r   

 2 ((t + 1)β − α) t  . ≈ 2 1 −  r N(t − 1)(α + (t − 1)β)

7 TREATMENT BALANCE FOR COVARIATES In this section we discuss amendments to the biased coin designs that attempt to balance the treatment allocation not only over the whole study, but also within levels of important covariates or prognostic factors. In a study with treatment imbalance with respect to prognostic factors, the estimated treatment differences will still be unbiased as long as the prognostic factor is accounted for

BIASED COIN RANDOMIZATION

in the analysis (this is not true if the factor is ignored in the analysis). Therefore, the real benefit of maintaining treatment balance within strata is to reduce the variability of the estimated treatment difference, and so to improve the efficiency of the study (22). The BCD procedures aim to bias the treatment allocation for future subjects based on the treatment allocation of previous subjects. As we have seen, the potential for overall treatment imbalance is greatly reduced in the BCD compared with CR. However, the BCD procedures will not guarantee treatment balance within levels of any important prognostic factor. A common procedure to achieve treatment balance within strata is stratified randomization, and the most common method of this is stratified PBR. This method will help improve balance within strata across the prognostic factors. However, it is possible that the overall study could still have some sizeable imbalance across the strata, particularly where the total study size is small and number of strata is large. In this case, many blocks may be unfilled and potentially imbalanced at any point in the trial. The potential for selection bias would also still exist in this procedure. Another simple procedure to achieve treatment balance within strata is minimization, which attempts to achieve balance within the strata of each main prognostic factor (not in strata across factors as in stratified randomization. For example, we can create balance for both gender and for all study sites, but not at each gender-site level. Minimization has been proposed using: • A deterministic nonrandom allocation

rule (23). • A procedure using a combination of both

nonrandom and random allocation rules, the former for higher imbalance, the latter for no or mild imbalance (2). • A rule maintaining random allocation (10, 24). The latter approaches can be viewed as biased coin randomization procedures, for example, where the coin bias is weighted toward allocation that gives the least treatment imbalance in the marginal covariate

11

factors for the next patient (24), or toward allocation that minimizes the variances of the treatment contrasts under the design containing the current state of the treatment allocation and covariate information and the covariate factors for the next patient (D-optimality [10, 25]). Within the BCD and UD procedures previously described, a common simple solution is simply to treat each strata as a separate experiment. The bias in the coin is then calculated using the current state for the single urn (strata) relevant for the next patient (e.g., the single urn for males, age < 45 years, in study site 10). Again, where the numbers of strata are large and/or the sample size is small, this procedure may not protect against overall treatment imbalance or even imbalance within strata. In this case, Wei (19) suggests balancing the marginal urns, either using some composite score calculated from the imbalance over all relevant urns (e.g., the three urns for males, age < 45 years, and study site 10) or using only the currently most imbalanced marginal urn. This procedure would not aim to achieve balance within cross-factor strata, but it is more likely to achieve balance in the marginal factor levels and across the study. Therefore, the BCD procedure to achieve marginal-factor urn balance is a minimization procedure maintaining the random allocation rule. 8 APPLICATION OF BIASED COIN DESIGNS TO RESPONSE-ADAPTIVE RANDOMIZATION The aim of the various bias coin designs is to use a randomization strategy that tends the treatment allocation toward an intended allocation ratio (most often balanced) while reducing allocation bias and other biases in the trial. These strategies can also be extended to the treatment allocation within important strata (covariate-adaptive randomization). A quite distinct class of techniques are those for response adaptive randomization (RAR); as their name suggests, these techniques allocate subjects to treatment groups based on the known responses of previous subjects, rather than on just the treatment group or covariate values. Broadly, the aim of RAR techniques could be:

12

BIASED COIN RANDOMIZATION • To optimize some key parameter of inter-

est. • To tend the treatment allocation toward the more ‘‘successful’’ treatment groups and away from the inferior groups (with respect to efficacy, safety or toleration, or some combination). Both of these objectives can be viewed as creating a more ethical design for the patients recruited into the clinical trial, for example, by minimizing variance of the treatment contrasts (to increase power or reduce overall sample size) or reducing the number of patients subjected to a clearly inferior treatment. Both the BCD and UD techniques may be used for RAR, whether the aim is to optimize some parameter or to bias allocation toward successful treatment groups. The application is easiest for binary response data as in for the randomized ‘‘play the winner’’ rule (RPW), the most well-known RAR technique (26). In this urn model, the initial composition is ω balls for each treatment group (each treatment identified by different colored balls). The first patient is allocated the treatment according to a ball drawn (with replacement) from the initial urn, and the response is noted. If the subject has a treatment response, β colored balls for that treatment are added into the urn, and α balls for each of the other treatments. If the subject does not have a treatment response, α and β balls are added for the current and each of the other treatments, respectively. The study then continues to recruit subjects, to collect the patient responses, and to update the urn. This design is labeled as the RPW(ω,α,β), where β > α ≥ 0 (CR is a special case where β = α). In the two-group case (treatments A and B) for RPW(ω,0,β), the ratio of balls in the urn and the treatment allocation ratio (A/B) will tend to qB /qA , where qi = 1 − pi is the nonresponse rate for treatment i. For the more general RPW(ω,α,β), the treatment allocation ratio will tend to a function based on α and β. These results show that the BCD, using a continuous function for coin bias based on the responses in each treatment group, could also be used as a strategy for a RAR design with the same objective as for the RPW design.

Many modifications have been made to the initial RPW urn model for RAR. These include strategies for more than two treatment groups, for categorical and continuous outcomes, for delayed responses, for treatment elimination from the randomization, and for targeting response percentiles (3, 4, 15, 27).

REFERENCES 1. D. Blackwell and J. L. Hodges, Design for the control of selection bias. Ann Math Stat. 1957; 28: 449–460. 2. M. Zelen, The randomization and stratification of patients to clinical trials. J Chron Dis. 1974; 27: 365–375. 3. W. F. Rosenberger and J. M. Lachin, Randomization in Clinical Trials Theory and Practice. New York: Wiley, 2002. 4. S. C. Chow and M. Chang, Adaptive Design Methods in Clinical Trials. Boca Raton: Chapman & Hall, 2007. 5. B. Efron, Forcing a sequential experiment to be balanced. Biometrika. 1971; 58: 403–417. 6. S. J. Pocock, Clinical Trials: A Practical Approach. New York: Wiley, 1983. 7. L. J. Wei, A class of treatment assignment rules for sequential experiments. Commun Stat Theory Methods. 1978; A7: 285–295. 8. J. F. Soares and C. F. Wu, Some restricted randomization rules in sequential designs. Commun Stat Theory Methods. 1983; 12: 2017–2034. 9. Y. P. Chen, Biased coin design with imbalance intolerance. Commun Stat Stochastic Models. 1999; 15: 953–975. 10. A. C. Atkinson, Optimum biased coin designs for sequential clinical trials with prognostic factors. Biometrika. 1982; 69: 61–67. 11. L. J. Wei, A class of designs for sequential clinical trials. J Am Stat Assoc. 1977; 72: 382–386. 12. L. J. Wei, The adaptive biased coin design for sequential experiments. Ann Stat. 1978; 6: 92–100. 13. R. L. Smith, Sequential treatment allocation using biased coin designs. J R Stat Soc Ser B Methodol. 1984; 46: 519–543. 14. A. Baldi Antognini and A. Giovagnoli, A new ‘biased coin design’ for the sequential allocation of two treatments. Appl Stat. 2004; 53: 651–664.

BIASED COIN RANDOMIZATION 15. W. F. Rosenberger, Randomized urn models and sequential design. Sequential Analysis. 2002; 21: 1–28. ¨ 16. F. Eggenberger and G. P´olya, Uber die statis¨ tik verketteter vorgange. Zeitschrift fur ¨ angewandte mathematic and mechanik. 1923; 3: 279–289. 17. L. J. Wei and J. M. Lachin, Properties of the urn randomization in clinical trials. Control Clin Trials. 1988; 9: 345–364. 18. B. Friedman, A simple urn model. Commun Appl Math. 1949; 1: 59–70. 19. L. J. Wei, An application of an urn model to the design of sequential controlled clinical trials. J Am Stat Assoc. 1978; 73: 559–563. 20. N. L. Johnson and S. Kotz, Urn Models and Their Applications. New York: Wiley, 1977. 21. J. M. Lachin, Properties of simple randomization in clinical trials. Control Clin Trials. 1988; 9: 312–326. 22. S. Senn, Statistical Issues in Drug Development. Chichester, UK: Wiley, 1997.

13

23. D. R. Taves, Minimization: a new method of assigning patients to treatment and control groups. Clin Pharm Ther. 1974; 15: 443–453. 24. S. J. Pocock and R. Simon, Sequential treatment assignment with balancing for prognostic factors in the controlled clinical trial. Biometrics. 1975; 31: 103–115. 25. A. C. Atkinson, Optimum biased–coin designs for sequential treatment allocation with covariate information. Stat Med. 1999; 18: 1741–1752. 26. L. J. Wei and S. Durham, The randomized play-the-winner rule in medical trials. J Am Stat Assoc. 1978; 73: 840–843. 27. A. Ivanova and W. F. Rosenberger, A comparison of urn designs for randomized clinical trials of K > 2 treatments. J Biopharm Stat. 2000; 10: 93–107.

BIOEQUIVALENCE (BE) TESTING FOR GENERIC DRUGS To receive approval for an Abbreviated New Drug Application (ANDA), an applicant generally must demonstrate, among other things, that its product has the same active ingredient, dosage form, strength, route of administration, and conditions of use as the listed drug and that the proposed drug product is bioequivalent to the reference listed drug [21 United States Code (U.S.C.) 355(j)(2)(A); 21 Code of Federal Regulations (CFR) 314.94(a)]. Bioequivalent drug products show no significant difference in the rate and extent of absorption of the therapeutic ingredient [21 U.S.C. 355(j)(8); 21 CFR 320.1(e)]. Studies for BE are undertaken in support of ANDA submissions with the goal of demonstrating BE between a proposed generic drug product and its reference listed drug. The regulations governing BE are provided at 21 CFR Part 320.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/6772dft.pdf) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

BIOLOGICAL ASSAY, OVERVIEW

penetrating level of research focusing on the gene-environment interaction in conventional experiments with biological units, and thereby calling for drastically different statistical resolutions for bioassays. We include also a brief synopsis of these recent developments. Traditionally, in a bioassay, a test (new) and a standard preparation are compared by means of reactions that follow their applications to some biological units (or subjects), such as subhuman primates (or human) living tissues or organs; the general objective being to draw interpretable statistical conclusions on the relative potency of the test preparation with respect to the standard one. Usually, when a drug or a stimulus is applied to a subject, it induces a change in some measurable characteristic that is designated as the response variable. In this setup, the dose may have several chemically or therapeutically different ingredients while the response may also be multivariable. Thus the stimulus–response or dose–esponse relationship for the two preparations, both subject to inherent stochastic variability, are to be compared in a sound statistical manner (with adherence to biological standardization) so as to cast light on their relative performance with respect to the set objectives. Naturally, such statistical procedures may depend on the nature of the stimulus and response, as well as on other extraneous experimental (biological or therapeutical) considerations. As may be the case with some competing drugs for the treatment of a common disease or disorder, the two (i.e. test and standard) preparations may not have the same chemical or pharmacological constitution, and hence, statistical modeling may be somewhat different than in common laboratory experimentation. Nevertheless, in many situations, the test preparation may behave (in terms of the response/tolerance distribution) as if it is a dilution or concentration of the standard one. For this reason, often, such bioassays are designated to compare the relative performance of two drugs under the dilution–concentration postulation, and are thereby termed dilution assays.

PRANAB K. SEN Chapel Hill, NC, USA

This article mainly emphasizes the classical aim of biological assay (or bioassay), to estimate relative potency, arising out of a need for biological standardization of drugs and other products for biological usage. There is a basic difference between biological and chemical endpoints or responses: the former exhibits greater (bio)variability and thereby requires in vivo or in vitro biological assays wherein a standard preparation (or a reference material) is often used to have a meaningful interpretation of relative potency. However, the term bioassay, has also been used in a wider sense, to denote an experiment, with biological units, to detect possible adverse effects such as carcinogenicity or mutagenicity; Mutagenicity Study). In the context of environmental impact on biosystems, toxicodynamic and toxicokinetic (TDTK) models as well as physiologically based pharmacokinetic (PBPK) models have been incorporated to expand the domain of bioassays; structure– activity relationship information (SARI) is often used to consolidate the adoption of bioassays in a more general setup; the genesis of dosimetry (or animal studies) lies in this complex. The use of biomarkers in studying environmental toxic effects on biological systems, as well as in carcinogenicity studies, has enhanced the scope of bioassays to a greater interdisciplinary field; we need to appraise, as well, bioassays in this broader sense. Further, recent advances in bioinformatics have added new frontiers to the study of biological systems; bioassay models are gaining more popularity in the developing area of computational biology. Our appraisal of bioassay would remain somewhat incomplete without an assessment of the role of Pharmacogenomics as well as Toxicogenomics in establishing a knowledge base of the chemical effects in biological systems. The developments in genomics during the past eight years, have opened the doors for a far more

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

BIOLOGICAL ASSAY, OVERVIEW

Dilution assays are classified into two broad categories: Direct dilution and indirect dilution assays. In a direct assay, for each preparation, the exact amount of dose needed to produce a specified response is recorded, so that the response is certain while the dose is a nonnegative random variable that defines the tolerance distribution. Statistical modeling of these tolerance distributions enables us to interpret the relative potency in a statistically analyzable manner, often in terms of the parameters associated with the tolerance distributions. By contrast, in an indirect assay, the dose is generally administered at some prefixed (usually nonstochastic) levels, and at each level, the response is observed for subjects included in the study. Thus, the dose is generally nonstochastic and the stochastic response at each level leads to the tolerance distributions that may well depend on the level of the dose as well as the preparation. If the response is a quantitative variable, we have an indirect quantitative assay, while if the response is quantal in nature (i.e. all or nothing), we have a quantal assay. Both of these indirect assays are more commonly addressed in statistical formulations. Within this framework, the nature of the dose–response regression may call for suitable transformations on the dose variable (called the dosage or dose-metameter) and/or the response variable, called the response-metameter. The basic objective of such transformations is to achieve a linear dosage-response regression, which may induce simplifications in statistical modeling and analysis schemes. In view of the original dilution structure, such transformations may lead to different designs for such assays, and the two most popular ones are (i) parallel-line assays and (ii) slope-ratio assays. Within each class, there is also some variation depending on the (assumed) nature of tolerance distributions, and within this setup, the probit (or normit) and logit transformations, based on normal and logistic distributions respectively, are quite popular in statistical modeling and analysis of bioassays. Bliss 2 contains an excellent account of the early developments in this area, while the various editions of Finney 6 capture more up-to-date developments, albeit with a predominantly parametric flavor. We refer to

these basic sources for extensive bibliography of research articles, particularly in the early phase of developments where biological considerations often dominated statistical perspectives. In this framework, it is also possible to include bioassays that may be considered for bioavailability and bioequivalence studies, though basically there are some differences in the two setups: Bioassays for assessing relative potency relate to clinical therapeutic equivalence trials, while in bioequivalence trials, usually, the relative bioavailability of different formulations of a drug are compared. Thus, in bioequivalence studies, the pharmacologic results of administrating essentially a common drug in alternative forms, such as a capsule versus a tablet, or a liquid dose of certain amount, capsules (tablets) or liquid forms of larger dose versus smaller dose with increased frequency of prescription, or even the administration of a drug at different time of the day, such as before breakfast or sometime after a meal, and so on, are to be assessed in a valid statistical manner. In this sense, the active ingredients in the drug in such alternative forms may be essentially the same, and differences in bioavailability reflect the form and manner of administration. We shall see later on that these basic differences in the two setups call for somewhat different statistical formulations and analysis schemes.

1 DIRECT DILUTION ASSAYS As an illustrative example, consider two toxic preparations (say, S and T), such that a preparation is continuously injected into the blood stream of an animal (say, cat) until its heart stops beating. Thus, the response (death) is certain, while the exact amount of the dose (X) required to produce the response is stochastic. Let X S and X T stand for the dose (variable) for the standard and test preparation, and let F S (x) and F T (x), x ≥ 0, be the two tolerance distributions. The fundamental assumption of a direct dilution assay is the following: FT (x) = FS (ρx),

for all x ≥ 0,

(1)

BIOLOGICAL ASSAY, OVERVIEW

where ρ(>0) is termed the relative potency of the test preparation with respect to the standard one. Standard parametric procedures for drawing statistical conclusions on ρ are discussed fully in the classical text of Finney 6, where other references are also cited in detail. If F S (.) is assumed to be a normal distribution function, then ρ is characterized as the ratio of the two means, as well as the ratio of the two standard deviations. Such simultaneous constraints on means and variances vitiate the simplicity of achieving optimality of parametric procedures (in the sense of maximum likelihood estimators and related likelihood ratio tests). On the other hand, if we use the log-dose transformation on the two sets of doses, and the resulting dosage distributions, denoted by FS∗ (.)andFT∗ (, )respectively, are taken as normal, then they have the same variance, while the difference of their means define logρ. Interestingly enough, in the first case, the estimator of ρ is the ratio of the sample arithmetic means, while in the other case, it turns out as the ratio of the sample geometric means. A different estimator emerges when one uses a power-dosage (as is common in slope-ratio assays). Thus, in general, these estimators are not the same, and they depend sensibly on the choice of a dosage. This explains the lack of invariance property of such parametric estimates (as well as associated test statistics) under monotone dosage transformations. From an operational point of view, an experimenter may not have the knowledge of the precise dosage, and hence, it may not be very prudent to assume the normality, lognormality or some other specific form of the tolerance distribution. Therefore, it may be reasonable to expect that an estimator of the relative potency should not depend on the chosen dosage as long as the latter is strictly monotone. For example, if the true tolerance distribution is logistic while we assume it to be (log)normal, the sample estimator may not be unbiased and fully efficient. Even when the two tolerance distributions are taken as normal, the ratio of the sample means is not unbiased for ρ. In this respect, such parametric procedures for the estimation of the relative potency (or allied tests for the fundamental assumption) are

3

not so robust, and any particular choice of a dosage may not remain highly efficient over a class of such chosen tolerance distributions. Nonparametric procedures initiated by Sen 17–19 and followed further by Shorack 26, and Rao and Littell 14, among others, eliminate this arbitrariness of dosage selection and render robustness to a far greater extent. Basically, we may note that ranks are invariant under strictly monotone (not necessarily linear) transformations on the sample observations. As such, a test for the fundamental assumption in (1) based on appropriate rank statistic remains invariant under such transformations. Similarly, if an estimator of the relative potency is based on suitable rank statistics, it remains invariant under such strictly monotone dosage transformations. Both the Wilcoxon–Mann–Whitney twosample rank-sum test and the (Brown–Mood) median test statistics were incorporated by Sen 17 for deriving nonparametric estimators of relative potency, and they also provide distribution-free confidence intervals for the same parameter. If there are m observations X S1 , . . . , X Sm for the standard preparation and n observations X T1 , . . . , X Tn , for the test preparation, we define the differences Yij = XSi − XTj , for i = 1, . . . , m; j = 1, . . . , n.

(2)

We arrange the N(= mn) observations Y ij in ascending order of magnitude, and let Y˜ N bethe median of these N observations. If N is even, we take the average of the two central order statistics. Then Y˜ N isthe Wilcoxon score estimator of logρ, and it is a robust and efficient estimator of logρ. The estimator is invariant under any strictly monotone transformation on the dose. Similarly, the confidence interval for logρ can be obtained in terms of two specified order statistics of the Y ij , and this is a distribution-free and robust procedure. A similar procedure works out for the median procedure; for general rank statistics, generally, an iterative procedure is needed to solve for such robust R-estimators. Rao and Littell 14 incorporated the twosample Kolmogorov–Smirnov test statistics in the formulation of their estimator. For computational convenience, because of

4

BIOLOGICAL ASSAY, OVERVIEW

the invariance property, it is simpler to work with the log-dose dosage, and in that way, the estimators of the log-relative potency correspond to the classical rank estimators in the two-sample location model. These direct dilution assays require the measurement of the exact doses needed to produce the response; this may not be the case if there are some latent effects. For example, the time taken by the toxic preparation to traverse from the point of infusion to the heart multiplied by the infusion rate may account for such a latent effect. In general, the situation may be much more complex. This naturally affects the fundamental assumption in (1), and variations in the modeling and statistical analysis to accommodate such effects have been discussed in 6 and 17 in the parametric and nonparametric cases respectively. 2

INDIRECT DILUTION ASSAYS

As an example, consider two drugs, A and B, each administered at k(≥ 2) prefixed levels (doses) d1 , . . . , dk . Let X Si and Y Ti be the response variable for the standard and test preparation respectively. These drugs may not have the same chemical ingredients, and may not have the same dose levels. It is not necessary to have the same doses for both the preparations, but the modifications are rather straightforward, and hence we assume this congruence. We assume first that both X Si and Y Ti are continuous (and possibly nonnegative) random variables. Suppose further that there exist some dosage xi = ξ (di ), i = 1, . . . , k and response-metameter X* = g(X), Y* = g(Y), for some strictly monotone g(.), such that the two dosage–response regressions may be taken as linear, namely, that ∗ ∗ = αT + βT xi + eTi , XSi YTi

= αS + βS xi + eSi ,

(3)

for i = 1, . . . , k, where for statistical inferential purposes, certain distributional assumptions are needed for the error components eTi and eSi , i = 1, . . . , k. Generally, in the context of log-dose transformations, we have a parallel-line assay, while slope-ratio assays

arise typically for power transformations. Thus, in a parallel-line assay, the two dose– response regression lines are taken to be parallel, and further that the errors eTi and eSi have the same distribution (often taken as normal). In this setup, we have then β S = β T = β (unknown), while α T = α S + β log ρ, where ρ is the relative potency of the test preparation with respect to the standard one. This leads to the basic estimating function log ρ =

{αT − αS } , β

(4)

so that if the natural parameters β, α S and α T are estimated from the acquired bioassay dataset, statistical inference on logρ (and hence ρ) can be drawn in a standard fashion. For normally distributed errors, the whole set of observations pertains to a conventional linear model with a constraint on the two slopes β S , β T , so that the classical maximum likelihood estimators and allied likelihood ratio tests can be incorporated for drawing statistical conclusions on the relative potency or the fundamental assumption of parallelism of the two regression lines. However, the estimator of logρ involves the ratio of two normally distributed statistics, and hence, it may not be unbiased; moreover, generally, the classical Fieller’s theorem [6] is incorporated for constructing a confidence interval for logρ (and hence, ρ), and it is known that this may result in an inexact coverage probability. Because of this difference in setups (with that of the classical linear model), design aspects for such parallel-line assays need a more careful appraisal. For equispaced (log —)doses, a symmetric 2k-point design has optimal information contents, and are more popularly used in practice. We refer to 6 for a detailed study of such bioassay designs in a conventional normally distributed errors model. Two main sources of nonrobustness of such conventional inference procedures are the following: 1. Possible nonlinearity of the two regression lines (they may be parallel but yet curvilinear); 2. Possible nonnormality of the error distributions.

BIOLOGICAL ASSAY, OVERVIEW

On either count, the classical normal theory procedures may perform quite nonrobustly, and their (asymptotic) optimality properties may not hold even for minor departures from either postulation. However, if the two dose–response regressions (linear or not) are not parallel, the fundamental assumption of parallel-line assays is vitiated, and hence, statistical conclusions based on the assumed model may not be very precise. In a slope-ratio assay, the intercepts α S and α T are taken as the same, while the slopes β S and β T need not be the same and their ratio provides the specification of the relative potency ρ. In such slope-ratio assays, generally, a power transformation: dosage = (dose)λ , for some λ > 0 is used, and we have
ρ=

βT βS

1/λ ,

(5)

which is typically a nonlinear function of the two slopes β T and β S , and presumes the knowledge of λ. In such a case, the two error components may not have the same distribution even if they are normal. This results in a heteroscedastic linear model (unless ρ = 1), where the conventional linear estimators or allied tests may no longer possess validity and efficiency properties. Moreover, as ρ λ is a ratio of two slopes, its conventional estimator based on usual estimators of the two slopes is of the ratio-type. For such ratio-type estimators, again the well-known Fieller Theorem (6) is usually adopted to attach a confidence set to ρ or to test a suitable null hypothesis. Such statistical procedures may not have the exact properties for small to moderate sample sizes. Even for large sample sizes, they are usually highly nonrobust for departures from the model-based assumptions (i.e. linearity of regression, the fundamental assumption, and normality of the errors). Again the design aspects for such slope-ratio assays need a careful study, and (6) contains a detailed account of this study. Because of the common intercept, usually a 2k + 1 point design, for some nonnegative integer k is advocated here. The articles on Parallel-line Assay and Slope-ratio Assay should be consulted for further details. The primary emphasis in these articles is on standard parametric

5

methods, and hence we discuss briefly here, the complementary developments of nonparametric and robust procedures for such assays. These were initiated in (21, 22) and also systematically reviewed in (23). First, we consider a nonparametric test for the validity of the fundamental assumption in a parallelline assay. This is essentially a test for the equality of slopes of two regression lines, and as in (21), we consider an aligned test based on the Kendall τ statistic. For each preparation with the set of dosages as independent variate and responses as dependent variable, one can define the Kendall tau statistic in the usual manner. We consider the aligned observations Y Ti − bxi and xi , and denote the corresponding Kendall τ (in the summation but not average form) as K T (b), and for the standard preparation, an aligned Kendall’s τ statistic is defined by K S (b), where we allow b to vary over the entire real line. Let then K ∗ (b) = KT (b) + KS (b), −∞ < b < ∞. (6) Note then that K T (b), K S (b), and hence K*(b) are all nonincreasing in b and have finitely many step-down discontinuities. Equating K*(b) to 0 (20), we obtain the pooled estimator ˆ βofβ. Let us then write L=

ˆ 2 + [KS (β)] ˆ 2} {[KT (β)] , Vn

(7)

where V n is the variance of the Kendall τ statistic under the hypothesis of no regression (and is a known quantity). This statistic has, under the hypothesis of homogeneity of β T and β S , closely central chi-square distribution with one degree of freedom. Lisused as a suitable test statistic for testing the validity of the fundamental assumption of a parallel-line assay where the normality of the error components is not that crucial. In that sense it is a robust test. Moreover, ˆ having obtained the pooled estimator βofβ, under the hypothesis of homogeneity of the slopes, we consider the residuals ˆ i , Yˆ Si = YSi − βx ˆ i, Yˆ Ti = YTi − βx

(8)

for different i, and treating them as two independent samples, as in the case of dilution direct assays, we use the

6

BIOLOGICAL ASSAY, OVERVIEW

Figure 1. Graphical procedure for obtaining a nonparametric confidence interval for the log potency ratio in a parallel-line assay

parameters, and obtain similar robust estimation and testing procedures. It is also possible to use general (aligned) M-statistics for this purpose. In general, such solutions are to be obtained by iterative methods, and hence, for simplicity and computational ease, we prescribe the use of the Kendall tau and twosample rank- sum statistics for the desired statistical inference. Next, we proceed to the case of sloperatio assays, and consider first a nonparametric test for the validity of the fundamental assumption (of a common intercept but possibly different slopes). We define the Kendall tau statistics K T (b) and K S (b) as in the case of the parallel-line assay, and equating them to 0, we obtain the corresponding estimates of β T and β S , which are denoted by βˆT and βˆS respectively. Consider then the residuals Y˜ Ti = YTi − βˆT xi , Y˜ Si = YSi − βˆS xi , ∀i. (9)

Wilcoxon–Mann–Whitney rank-sum test statistic to estimate the difference of the intercepts α T − α S in a robust manner. As in the direct dilution assay, this estimator is the median of the differences of all possible pairs of residuals from the test and standard preparation respectively. A robust, consistent and asymptotically normally distributed estimator of logρ is then obtained by dividing this estimator by the pooled estimaˆ tor β. For drawing a confidence interval for logρ (and hence, ρ), we can then use the Fieller Theorem by an appeal to the asymptotic normality of the estimator, or as in (21), consider a rectangular confidence set for β and α T − α S by computing a coordinate-wise confidence interval for each with coverage probability 1 − γ /2, and as in Figure 1, draw a robust confidence set for logρ with coverage probability 1 − γ. Though this does not have an exact coverage probability, it is quite robust and works out well even for quite nonnormal error distributions. In the above setup, instead of the Kendall τ and the two-sample rank-sum statistics, we may use a general linear rank statistic for regression and a two-sample linear rank statistic for difference of location

We pool all these residuals into a combined set, and use the signed-rank statistic to derive the corresponding rank estimator of the hypothesized common value of the intercept; this estimator, denoted by α,isthe ˜ median of all possible midranges of the set of residuals listed above. Let then Yˆ Ti = ˜ Yˆ Si = Y˜ Si − α, ˜ ∀i,and for each prepaY˜ Ti − α, ration, based on these residuals, we consider the Wilcoxon signed-rank statistic. These are ˆ S respectively. As in the ˆ T and W denoted by W case of parallel-line assays, here we consider a test statistic for testing the validity of the fundamental assumption as L=

ˆ 2} ˆ 2 +W {W T S , Vn

(10)

where V n is the variance of Wilcoxon signedrank statistic under the hypothesis of symmetry of the distribution around 0 (and is a known quantity). When the fundamental assumption holds, the distribution of L isclose to the central chi-square distribution with 1 degree of freedom, and hence a test can be carried out using the percentile point of this chi-square law. This test is quite robust and the underlying normality of the errors may not be that crucial in this context. Note that for the slope-ratio assay, granted the

BIOLOGICAL ASSAY, OVERVIEW

fundamental assumption of a common intercept, a natural plug-in estimator of ρ is given by  ρˆ =

βˆT βˆS

1/λ .

(11)

We may use the Fieller Theorem under an asymptotic setup to construct a confidence interval for ρ. Alternatively, as in the case of a parallel line assay, for a given γ (0 < γ < 1), we may consider a distribution-free confidence interval of coverage probability 1 − γ /2 for each of the two slopes β T and β S , and obtain a confidence interval for ρ λ (and hence ρ). The situation is quite comparable to the Figure for the parallel-line assay, excepting that β T and β S are taken for the two axes. Here also, instead of the Kendall tau statistics and the Wilcoxon signed-rank statistics, general regression rank statistics and (aligned) signed-rank statistics (or even suitable M-statistics) can be used to retain robustness of the procedures without sacrificing much efficiency. However, the solutions are generally to be obtained by iterative procedures, and hence, we prefer to use the simpler procedures considered above. 3

INDIRECT QUANTAL ASSAYS

In this type of (indirect) assays, the response is quantal (i.e. all or nothing) in nature. For each preparation (T or S) and at each level of administered dose, among the subjects, a certain number manifest the response while the others do not; these frequencies are stochastic in nature and their distribution depends on the dose level and the preparation. Thus, for a given dosage x, we denote by F T (x) and F S (x) the probability of the response for the test and standard preparation respectively. It is customary to assume that both F T (x) and F S (x) are monotone increasing in x, and for each α(0 < α < 1), there exits unique solutions of the following FT (ξTα ) = α,

and FS (ξSα ) = α,

(12)

so that ξ Tα and ξ Sα are the α-quantile of the test and standard preparation; they are

7

termed the 100α% effective dosage. In particular, for α = 1/2, they are termed the median effective dosage. Whenever the response relates to death (as is usually the case with animal and toxicologic studies), the ξTα , ξSα are also termed 100α%-lethal dosage. In many studies, generally, low dosages are contemplated so that α is chosen to be small. This is particularly the case with radioimmunoassays, and we shall comment on that later on. Estimation of the ξ Tα and ξ Sα with due attention to their interrelations is the main task in a quantal assay. The concept of parallel-line and slope-ratio assays, as laid down for indirect quantitative assays, is also adoptable in quantal assays, and a detailed account of the parametric theory based on normal, lognormal, logistic, and other notable forms of the distribution F T (x) is available with Finney [6, Chapter 17]. In this context, the probit and logit analyses are particularly notable, and we shall discuss them as well. To set the ideas, we consider a single preparation at k(≥ 2)- specified dosage d1 , . . . ,dk , where d1 < d2 < . . . < dk . Suppose that the dosage di has been administered to ni subjects, out of which ri respond positively while the remaining ni − ri do not, for i = 1, . . . , k. In this setup, the di , ni are nonstochastic, while the ri are random. The probability of a positive response at dosage di , denoted by π (di ), is then expressed as π (di ) = π (θ + βdi ),

i = 1, . . . , k,

(13)

where θ and β are unknown (intercept and regression) parameters, and π (x), −∞ < x < ∞, is a suitable distribution function. In a parametric mold, the functional form of π (.) is assumed to be given, while in nonparametrics, no such specific assumption is made. Note that the joint probability law of r1 , . . . , rk is given by k    ni i=1

ri

π (θ + βdi )ri [1 − π (θ + βdi )]ni −ri , (14)

so that the likelihood function involves only two unknown parameters θ and β. The loglikelihood function or the corresponding estimating equations are not linear in the parameters, and this results in methodological as well as computational complications.

8

BIOLOGICAL ASSAY, OVERVIEW

If π (.) is taken as a logistic distribution, that is, π (x) = {1 + e−x }−1 , then we have from the above discussion 
π (di ) = θ + βdi , i = 1, . . . , k. log [1 − π (di )] (15) This transformation, known as the logit transformation, relates to a linear regression on the dosage, and simplifies related statistical analysis schemes. Thus, at least intuitively, we may consider the sample logits
 ri − ri ) , i = 1, . . . , k, (16) Zi = log ni and attempt to fit a linear regression of the Zi on di . In passing, we may remark that technically ri could be equal to zero or ni (with a positive probability), so that Zi would assume the values − ∞ and +∞ with a positive probability, albeit for large ni , this probability converges to zero very fast. As in practice, the ni may not be all large; to eliminate this impasse, we consider the Anscombe correction to a binomial variable, and in (16), modify the Zi as   (ri + 38 ) Zi = log , i = 1, . . . , k. (17) (ni − ri + 38 ) Though the ri have binomial distributions, the Zi have more complex probability laws, and computation of their exact mean, variance, and so on, is generally highly involved. For large values of the ni , we have the following √

ni (Zi − θ − βdi )

D

→ N(0, {π (di )[1 − π (di )]}−1 ),

(18)

for each i = 1, . . . , k, where the unknown π (di ) can be consistently estimated by the sample proportion pi = ri /ni . Thus, using the classical weighted least squares estimation (WLSE) methodology, we may consider the quadratic norm Q(θ , β) =

k  i=1

and minimize this with respect to θ , β to obtain the WLS estimators. Although the logit transformation brings the relevance of generalized linear models (GLM), the unknown nature of their variance functions makes the WLSE approach more appropriate for the suggested statistical analysis. In any case, the asymptotic flavor should not be overlooked. If π (x) is taken as the standard normal distribution function (x), whose density function is denoted by φ(x), then we may consider the transformation   ri , i = 1, . . . , k, (20) Zi = −1 ni known as the probit or normit transformation. Here also, it would be better to modify Zi as Zi =

−1

(ni + 12 )

,

i = 1, . . . , k. (21)

Note that by assumption, −1 (π (di )) = θ + βdi , i = 1, . . . , k, and this provides the intuitive appeal for a conventional linear regression analysis. However, the likelihood approach based on the product-binomial law encounters computational difficulties and loses its exactness of distribution theory to a greater extent. Here also, we would have complications in the computation of the exact mean, variance, or distribution of the Zi , and hence, as in the logit model, we consider a WLSE approach in an asymptotic setup where the ni are large. By virtue of √ ni (pi − the asymptotic normality of the π (di ))(where again we take pi = (ri + 3/8)/(ni + 1/2)), we obtain that for every i ≥ 1, √

ni [Zi − θ − βdi ]

D

→ N(0,

π (di )[1 − π (di )] ), φ 2 ( −1 (π (di )))

(22)

so that we consider quadratic norm in a WLSE formulation Q(θ , β) =

ni pi (1 − pi ){Zi − θ − βdi }2 , (19)

(ri + 38 )

k  ni φ 2 ( −1 (pi )) i=1

pi (1 − pi )

[Zi − θ − βdi ]2 , (23)

BIOLOGICAL ASSAY, OVERVIEW

and minimizing this with respect to θ , β, we arrive at the desired estimators. For both the logit and probit models, the resulting estimators of θ , β are linear functions of the Zi with coefficients depending on the ni and the pi . Therefore, the asymptotic normality and other properties follow by standard statistical methodology. Moreover the (asymptotic) dispersion matrix of these estimators, in either setup, can be consistently estimated from the observational data sets. Thus, we have the access to incorporate standard asymptotics to draw statistical conclusions based on these estimators. Let us then consider the case of quantal bioassays involving two preparations (S and T), and for each preparation, we have a setup similar to the single preparation case treated above. The related parameters are denoted by θ S , β S and θ T , β T respectively, and for modeling the response distributions, we may consider either the logit or probit model, as has been discussed earlier. If we have a parallel-line assay, as in the case of an indirect assay, we have then βT = βS = β unknown, and θT − θs = β log ρ, (24) so that based on the estimates θˆS , βˆS , θˆT and βˆT ,along with their estimated dispersion matrix, we can incorporate the WLSE to estimate the common slope β and the intercepts θ S and θ T . The rest of the statistical analysis is similar to the case of indirect assays. Moreover, this WLSE methodology is asymptotically equivalent to the classical likelihood-function–based methodology, so it can be regarded, computationally, as a simpler substitute for a comparatively complicated one. For a slope-ratio assay, we have similarly a common intercept while the ratio of the slopes provide the measure of the relative potency, and hence, the WLSE based on the individual preparation estimators can be adopted under this restriction to carryout the statistical analysis as in the case of an indirect assay. Besides the logit or probit method, there are some other quasi-nonparametric methods, of rather an ad hoc nature, and among these, we may mention of the following estimators of the median effective dosage:

9

¨ 1. The Spearman–Karber estimator; 2. The Reed–Muench estimator, and 3. The Dragstedt–Behrens estimator. These procedures are discussed in (7), p. 43. If the tolerance distribution is symmetric, ¨ the Spearman–Karber estimator estimates the median effective dosage closely; otherwise, it may estimate some other characteristic of this distribution. Miller 13 studied the relative (asymptotic) performance of these three estimators, casting light on their bias terms as well. From a practical point of view, none of these estimators appears to be very suitable. Rather, if the π (di ) do not belong to the extreme tails (i.e. are not too small or close to 1), the logit transformation provides a robust and computationally simpler alternative, and is being used more and more in statistical applications. In passing, we may remark that Finney [7, Chapter 10] contains some other techniques that incorporate modifications in the setup of usual quantal assays, such as the numbers ni being unknown and possibly random, multiple (instead of binary) classifications, errors in the doses. In the following chapter, he also introduced the case of doses in mixtures that require a somewhat extended model and more complex statistical designs and analysis schemes. We shall comment on these below. 4 STOCHASTIC APPROXIMATION IN BIOASSAY In the context of a quantal assay, we have the dosage-response model in terms of the tolerance distribution π (d), and the median effective (lethal) dosage, LD50, is defined by the implicit equation π (LD50) = 0.50. In this context, for each preparation (standard or test), corresponding to initial dosage levels d1 , . . . , dk , we have estimates p(d1 ), . . . , p(dk ) of the unknown π (d1 ), . . . , π (dk ). We may set pi = π (di ) + e(di ), i = 1, . . . , k,

(25)

where the errors are (for large ni , the number of subjects treated) closely normally distributed with zero mean and variance n−1 i π (di )[1 − π (di )].Onthe basis of this initial response data, we can choose an appropriate

10

BIOLOGICAL ASSAY, OVERVIEW

do for which the corresponding p(do ) is closest to 1/2. Then, we let d(1) = do + ao [p(do ) − 1/2], for some ao > 0, and recursively we set  1 , d(j+1) = d(j) + aj p(d(j) ) − 2 for some aj > 0; j ≥ 0. (26) The aim of this stochastic approximation procedure, due to Robbins and Monro 15, is to estimate the LD50 without making an explicit assumption on the form of the tolerance distribution π (d). But in this setup, the p(d(j) ) as well as the d(j) are stochastic elements, and for the convergence of this stochastic iteration procedure, naturally, some regularity conditions are needed on the {ai ;i ≥ 0} and π (d) around the LD50. First of all, in order that the iteration scheme terminates with a consistent estimator of the LD50, it is necessary that the ai converge to zero as i increases. More precisely, it is assumed in this context that   an diverges to + ∞, but a2n < +∞. n≥0

n≥0

in many toxicologic studies where a turn occurs at an unknown level. Often this is treated in a general change-point model framework. The main advantage of the stochastic approximation approach over the classical quantal assay approach is that no specific assumption is generally needed on the probability function π (d), so that the derived statistical conclusions remain applicable in a much wider setup. On the other hand, the stochastic iteration scheme generally entails a larger number of subjects on which to administer the study, and often that may run contrary to the practicable experimental setups, especially with respect to cost considerations. In this general context, a significant amount of methodological research work has been carried out during the past 40 years, and an extensive review of the literature on stochastic approximation is made by Ruppert (16) where the relevant bibliography has also been cited. The scope of stochastic approximation schemes is by no means confined to quantal assays; they are also usable for quantitative bioassays, and even to other problems cropping up in far more general setups.

(27) In addition, the continuity and positivity of the density function corresponding to the distribution function π (x) at the population LD50 is also a part of the regularity assumptions. Further assumptions are needed to provide suitable (stochastic) rates of convergence of the estimator of the LD50 and its asymptotic normality and related large sample distributional properties. Once the LD50 values are estimated for each preparation, we may proceed as in the case of a quantal assay, and draw conclusions about the relative potency and other related characteristics. It is not necessary to confine attention specifically to the LD50, and any LD100α, for α ∈ (0, 1) can be treated in a similar fashion. In fact, Kiefer and Wolfowitz (12) considered an extension of the Robbins–Monro stochastic approximation procedure that is aimed to locate the maximum (or minimum) of a dose–response function that is not necessarily (piecewise or segmented) linear but is typically nonmonotone, admitting a unique extremum (maximum or minimum) of experimental importance. Such dose–response regressions arise

5 RADIOIMMUNOASSAY In radioimmunoassays antigens are labeled with radioisotopes, and in immunoradiometric assays antibodies are labeled. For a broad range of antigens, such radioligand assays enable the estimation of potency from very small quantities of materials and usually with high precision. Radioligand assays are based upon records of radiation counts in a fixed time at various doses, so that potency estimation involves the relation between counts of radioactivity and dose, generally both at low levels (8). In many such studies, the regression function of the count of radioactivity on dose has been found to be satisfactorily represented by a logistic curve; however, the lower and upper asymptotes of such a curve are not necessarily equal to zero and one, but are themselves unknown parameters. This difference with the classical logistic distribution is reflected in a somewhat different form of the variance function of radiation counts. Unlike the Poisson process, the variance function may not be

BIOLOGICAL ASSAY, OVERVIEW

equal to the mean level of the radiation counts U(d) (i.e. their expectation at a given dose level d); in many studies, it has been experimentally gathered that the variance function V(d) behaves like [U(d)]λ , where λ(>0) typically lies between 1 and 2. For this reason, the usual Poisson regression model in generalized linear models (GLM) methodology may not be universally applicable in radioimmunoassays. Moreover, such radioligand assays may not be regarded as strictly bioassays, since they may not depend upon responses measured in living organisms or tissues. However, the advent of the use of biologic markers in mutagenesis studies and in molecular genetics, particularly during the past 20 years, has extended the domain of statistical perspectives in radioligand assays to a much wider setup of investigations, and strengthened the structural similarities between radioimmunoassays and the classical bioassays. They involve statistical modeling and analysis schemes of very similar nature, and in this sense, their relevance in a broader setup of bioassays is quite appropriate. 6

DOSIMETRY AND BIOASSAY

As has been noted earlier, a dose–response model exhibits the (mathematical) relationship between an amount of exposure or treatment and the degree of a biological or health effect, generally a measure of an adverse outcome. Bioassay and clinical trials are generally used in such dose–response studies. With the recent advances in pharmacoepidemiology as well as in risk analysis, bioassays have led to another broader domain of statistical appraisal of biological dose–response studies, known as dosimetry (or animal study). Pharmacoepidemiology rests on the basic incorporation of pharmacodynamics (PD) and pharmacokinetics (PK) in the development of the so called structure– activity relationship information (SARI). Though a PD model directly relates to a dose–response model, the PK actions of the exposure or drug needs to be taken into account in the dose–response modeling. This is now done more in terms of SARI where the structure refers to the dose factors

11

and activity refers to the biological reactions that follow the exposure (dose) to a specific species or organism. In a majority of cases, the target population is human, but owing to various ethical and other experimental constraints, human beings may not be usable to the full extent needed for such a dose–response modeling. As such, animal studies are often used to gather good background information, which is intended for incorporation in human studies in bioassay and clinical trials. Dosimetry pertains to this objective. Dosimetry models intend to provide a general description of the uptake and distribution of inhaled (or ingested or absorbed) toxics (or compounds having adverse health effects) on the entire body system. For judgment on human population, such dosimetric models for animal studies need to be extrapolated with a good understanding of the interspecies differences. SARI is a vital component in enhancing such statistical validation of pooling the information from various animal studies and extrapolating to the human population. Most dose–response relationships are studied and through well-controlled animal bioassays with exposure or dose levels generally much higher than typically perceived in human risk analysis. In this respect, dosimetry is directly linked to bioassay, though in dosimetry, the SARI is more intensively pursued to facilitate extrapolation. PDPK aspects not only may vary considerably from subhuman primates to human beings, but also there is much less of control in human exposure to such toxics. Also, metabolism in the human being is generally quite different from that in subhuman primates. An important element in this context is the environmental burden of disease (EBD) factor that exhibits considerable interspecies variation as well as geopolitical variation. Hence, ignoring the SARI part, a conventional dose–response model for a subhuman primate may not be of much help in depicting a similar model for human exposure. For the same reason, conventional statistical extrapolation tools may be of very limited utility in this interspecies extrapolation problems (25). Finally, in many carcinogenicity studies, it has been observed that xenobiotic effects underlie such dose–response

12

BIOLOGICAL ASSAY, OVERVIEW

relations, and this is outlined in a later section. 7

SEMIPARAMETRICS IN BIOASSAYS

The GLM methodology has been incorporated in a broad variety of statistical modeling and analysis schemes pertaining to a wide range of applications, and bioassays are no exceptions. Going back to the direct dilution assays, if we had taken both the distributions, F S and F T , as exponentials with respective means µS and µT , then the two distributions would have constant hazard rates 1/µS and 1/µT respectively, so that the relative potency ρ is equal to the ratio of the two hazard rates. Inspired by this observation, and by the evolution of the Cox (3) proportional hazard model (PHM), research workers have attempted to relate the two survival functions SS (x) = P{X S > x} and ST (x) = P{X T > x} as ST (x) = [SS (x)]ρ , x ≥ 0,

(28)

and interpret ρ as the relative potency of the test preparation with respect to the standard one. Though this representation enables one to import the PHM-based statistical analysis tools for the estimation of the relative potency, for distributions other than the exponential ones, the interpretation of ‘‘dilution assays’’ may no longer be tenable under such a PHM. There is an alternative interpretation in terms of the parallelism of the two log-hazard functions, but that may not fit well with the fundamental assumption in dilution assays. For some related statistical analysis of bioassays based on GLM methodologies, we refer to (24), where indirect bioassays have also been treated in the same manner along with the classical parametrics. 8

NONPARAMETRICS IN BIOASSAYS

The estimators of relative potency and tests for fundamental assumptions in dilution (direct as well as indirect) assays based on rank statistics, considered earlier, spark the first incorporation of nonparametrics in biological assays. However, these may be

characterized more in terms of semiparametrics, in the sense that the assumed linearity of dose–response regressions was essentially parametric in nature, while the unknown form of the underlying tolerance distribution constitutes the nonparametric component. Thus, together they form the so-called semiparametric models. It is possible to incorporate more nonparametrics in bioassays mostly through the nonparametric regression approach. For direct dilution assays, such nonparametric procedures are quite simple in interpretation and actual formulation. We consider the log-dose transformation, so that the dosage for the test and standard preparations have the distributions FT∗ (x) and FS∗ (x), respectively, where FT∗ (x) = FS∗ (x + log ρ), forall x. If we denote the p-quantile of FT∗ and FS∗ byQT (p) and QS (p) respectively, then we have QS (p) − QT (p) = log ρ, ∀ p ∈ (0, 1), (29) so that the well-known Q–Q plot for the two preparations results in a linear regression form, and this provides the statistical information to test for this fundamental assumption as well as to estimate the relative potency. A similar conclusion can also be drawn from a conventional P–P plot. The classical Kolmogorov–Smirnov statistics (in the two-sample case) can be used for drawing statistical conclusions, and we may refer to Rao and Littell (14) for some related work. The situation is a bit more complex with indirect assays. In the classical parametric setup, we work with the expected response at different dosages, assuming of course a linear regression. In a semiparametric approach, this linearity of dosage-response regression is taken as a part of the basic assumption, but the distribution of the errors is allowed to be a member of a wider class, so that robust procedures based on rank or M-statistics are advocated instead of the classical WLSE. In a pure nonparametric setup, the linearity of the dosage-response regression is not taken for granted. Therefore the two dosage-response regression functions may be of quite arbitrary nature, and yet parallel in an interpretable manner. The statistical task is therefore to assess this parallelism without imposing linearity or some other parametric forms. Here

BIOLOGICAL ASSAY, OVERVIEW

also, at a given dosage level, instead of the mean response level, we may consider median or a p-quantile, and based on such robust estimators, we draw statistical conclusions allowing the quantile functions to be of a rather arbitrary nature. Asymptotics play a dominant role in this context, and often this may require a relatively much larger sample size. On the other hand, in terms of robustness and validity, such pure nonparametric procedures have a greater scope than parametric or semiparametric ones. 9 BIOAVAILABILITY AND BIOEQUIVALENCE MODELS As has been explained earlier bioequivalence trials differ from conventional bioassays, as here, generally, the active substances in the drug are the same but the differences in bioavailability reflect the form and manner of administration. Such alternative modes may therefore call for additional restraints in the statistical formulation, and because of anticipated biological equivalence, there is less emphasis on relative potency and more on general equivalence patterns. For such reasons, regulatory requirements for establishing average bioequivalence of two preparations (that are variations of an essentially common drug) relate to a verification of the following: A confidence interval for the relative potency, having the confidence limits ρ L , ρ U , lies between two specified endpoints, say ρ o < 1 < ρ o , with a high coverage probability (or confidence coefficient) γ . Generally, γ is chosen close to 1 (namely, 0.95), and also ρ 0 = (ρ o )−1 is very close to one. These requirements in turn entail a relatively large sample size, and therefore, (group) sequential testing procedures are sometimes advocated (9). For general considerations underlying such bioequivalence trials, we refer to (1,11,29), where other pertinent references are cited. Generally, such statistical formulations are more complex than the ones referred to earlier. As has been mentioned earlier, the term bioassay is used in a more general form, and this is equally true for bioequivalence and bioavailability models. Kinetic measures of

13

bioavailability and pharmacokinetic parameters have been developed to meet the demand for such recent usage. We will illustrate this somewhat differently with pharmacogenomics, which is revolutionizing the field of bioinformatics and experiments with biological units, in general. 10 PHARMACOGENOMICS IN MODERN BIOASSAYS Following Ewens and Grant (5), we take bioinformatics to mean the emerging field of science growing from the application of mathematics, statistics, and information technology, including computers and the theory surrounding them, to study and analysis of very large biological and, in particular, genetic data sets. Having its genesis 50 years ago (28), the field has been fueled by the immense increase in the DNA data generation. Earlier interpretation of bioinformatics with emphasis on computational biology by Waterman (27) also merits serious considerations, while Durbin et al. (4) had a view point geared by computer algorithms along with some heuristic usage of hidden Markov models. At the current stage, gene scientists cannot scramble fast enough to keep up with the genomics, with developments emerging at a furious rate and in astounding detail. Bioinformatics, at least at this stage, as a discipline, does not aim to lay down some fundamental mathematical laws (which might not even exist in such a biological diversity). However, its utility is perceived in the creation of innumerable computer graphics and algorithms that can be used to analyze exceedingly large data sets arising in bioinformatics. In this context, naturally data mining and statistical learning tools (under the terminology Knowledge Discovery and Data Mining (KDDM)) are commonly used (10), though often in a heuristic rather than objective manner. There could be some serious drawbacks of statistical analysis based on such KDDM algorithms alone, and model selection has emerged as a challenging task in bioinformatics. Given the current status of bioinformatics as the information technology (advanced computing) based discipline of analyzing

14

BIOLOGICAL ASSAY, OVERVIEW

exceedingly high dimensional data with special emphasis on genomics, and that genomics looks at the vast network of genes, over time, to determine how they interact, manipulate, and influence biological pathways, networks, as well as physiology, it is quite natural to heed to genetic variation (or polymorphism) in most studies involving biological units. Moreover, because of the drug-response relationship, basic in bioassay, it is natural to appraise the role of pharmacogenomics in this setup. Pharmacology is the science of drugs including materia medica, toxicology, and therapeutics, dealing with the properties and reactions of drugs, especially with relation to their therapeutic values. In the same vein, pharmacodynamics, a branch of pharmacology, deals with reactions between drugs and living structures; pharmacokinetics relates to the study of the bodily absorption, distribution, metabolism, and excretion of drugs. In bioequivalence trials, these tools have already been recognized as fundamental. Pharmacogenetics deals with genetic variation underlying differential response to drugs as well as drug metabolism. The whole complex constitutes the discipline: Pharmacogenomics. In the same way, Toxicogenomics relates to the study of gene-environmental interactions in disease and dysfunction to cast light on how genomes respond to environmental stress or toxics. It is conceived that there are certain genes that are associated with disease phenotype, side effects, and drug efficacy. Also, because of inherent (genetic) variations and an enormously large number of genes as well as a very large pool of diseases and disorders, there is a genuine need of statistical methods to assess the genetic mapping of disease genes. Pharmaco-toxicogenomics is therefore destined to play a fundamental role in biological assays, in the years to come. 11 COMPLEXITIES IN BIOASSAY MODELING AND ANALYSIS There are generally other sources of variations, which may invalidate the use of standard statistical analysis schemes in bioassays to a certain extent. Among these factors, special mention may be made of the following:

1. Censoring of various types, 2. Differentiable / Nondifferentiable measurement errors, 3. Stochastic compliance of dose, 4. Correlated multivariate responses, and 5. Curse of dimensionality in genomics. It is generally assumed that censoring is usually of Type I (truncation of the experiment at a prefixed timepoint), Type II (truncation following a certain prefixed number or proportion of responses), and random, where the censoring time and response time are assumed to be stochastically independent, and moreover, the censoring is assumed to be noninformative, so that the censoring time distribution remains the same for both the preparations. In actual practice, this may not be generally true, and hence, effects of departures from such assumptions on the validity and efficacy of standard statistical procedures are therefore needed to be assessed. Measurement of the actual dose levels in quantal assays, or the response levels in an indirect assay may often be impaired to a certain extent by measurement errors. In statistical analysis, usually such measurement errors are assumed to be either differentiable or nondifferentiable type, and appropriate statistical models and related analysis schemes depend on such assumptions. In radioimmunoassays, dosimetric studies in pharmacokinetics, as well as in other types, not the full amount of a prescribed dose may go into the organ or experimental unit, and the actual consumption of the dose may be (often, highly) stochastic in nature. Therefore, the dose–response regression relation may be subject to nonidentifiability and overdispersion effects. This calls for more modifications of existing models and analysis schemes. Finally, when there are multiple endpoints with possibly binary or polytomous responses, a dimension reduction for the model-based parameters becomes necessary from statistical modeling and inference perspectives. Otherwise, an enormously large sample size may be needed to handle adequately, the full parameter model, and this may run contrary to the practical setup of an assay. The situation is worse when some of the responses are

BIOLOGICAL ASSAY, OVERVIEW

quantitative while the others are quantal or at best polychotomous. These naturally introduce more model complexities and call for more complicated statistical analysis tools.

REFERENCES 1. Anderson, S. & Hauck, W. W. (1990) Considerations of individual bioequivalence, Journal of Pharmacokinetics and Biopharmaceutics 18, 259–273. 2. Bliss, C. I. (1952) The Statistics of Bioassay. Academic Press, New York. 3. Cox, D. R. (1972) Regression models and life tables (with discussion), Journal of the Royal Statistical Society B 34, 187–220. 4. Durbin, R., Eddy, S., Krogh, A. & Mitchison, G. (1998) Biological Sequence Analysis: Probabilistic Models for Proteins and Nucleic Acids. Cambridge University Press, Cambridge, UK. 5. Ewens, W. J. & Grant, G. R. (2001) Statistical Methods in Bioinformatics: An Introduction. Springer-Verlag, New York. 6. Finney, D. J. (1964) Statistical Methods in Biological Assay, 2nd Ed. Griffin, London. 7. Finney, D. J. (1971) Probit Analysis, 3rd ed. University Press, Cambridge. 8. Finney, D. J. (1976) Radioligand assay, Biometrics 32, 721–730. 9. Gould, A. L. (1995) Group sequential extensions of a standard bioequivalence testing procedure, Journal of Pharmacokinetics and Biopharmaceutics 23, 57–86. 10. Hastie, T., Tibshirani, R. & Friedman, J. (2001) The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer-Verlag, New York. 11. Hochberg, Y. (1955) On assessing multiple equivalences with reference to bioequivalence, in Statistical Theory and Applications: Papers in Honor of H. A. David., H. N. Nagaraja, P. K. Sen & D. F. Morrison, eds. Springer-Verlag, New York, pp. 265–278.

15

15. Robbins, H. & Monro, S. (1951) A stochastic approximation method, Annals of Mathematical Statistics 22, 400–407. 16. Ruppert, D. (1991) Stochastic approximation, in Handbook of Sequential Analysis, B. K. Ghosh & P. K. Sen, eds. Marcel Dekker, New York, pp. 503–529. 17. Sen, P. K. (1963) On the estimation of relative potency in dilution (-direct) assays by distribution-free methods, Biometrics 19, 532–552. 18. Sen, P. K. (1964) Tests for the validity of fundamental assumption in dilution (-direct) assays, Biometrics 20, 770–784. 19. Sen, P. K. (1965) Some further applications of nonparametric methods in dilution (-direct) assays, Biometrics 21, 799–810. 20. Sen, P. K. (1968) Estimates of the regression coefficient based on Kendall’s tau, Journal of the American Statistical Association 63, 1379–1389. 21. Sen, P. K. (1971) Robust statistical procedures in problems of linear regression with special reference to quantitative bioassays, I, International Statistical Review 39, 21–38. 22. Sen, P. K. (1972) Robust statistical procedures in problems of linear regression with special reference to quantitative bioassays, II, International Statistical Review 40, 161–172. 23. Sen, P. K. (1984) Nonparametric procedures for some miscellaneous problems, in Handbook of Statistics, Vol. 4: Nonparametric Methods, P. R. Krishnaiah & P. K. Sen, eds. Elsevier, Holland, pp. 699–739. 24. Sen, P. K. (1997) An appraisal of generalized linear models in biostatistical applications, Journal of Applied Statistical Sciences 5, 69–85. 25. Sen, P. K. (2003) Structure-activity relationship information in health related environmental risk assessment, Environmetrics 14, 223–234. 26. Shorack, G. R. (1966) Graphical procedures for using distribution-free methods in the estimation of relative potency in dilution (-direct) assays, Biometrics 22, 610–619.

12. Kiefer, J. & Wolfowitz, J. (1952) Stochastic estimation of the maximum of a regression function, Annals of Mathematical Statistics 23, 462–466.

27. Waterman, M. S. (1995) Introduction to Computational Biology: Maps, Sequences and Genomes. Chapman & Hall, Cambridge.

13. Miller, R. G., Jr. (1973) Nonparametric estimators of the mean tolerance in bioassay, Biometrika 60, 535–542.

28. Watson, J. D. & Crick, F. H. C. (1953) Genetical implications of the structure of deoxyribonucleic acid, Nature 171, 964–967.

14. Rao, P. V. & Littell, R. (1976) An estimator of relative potency, Communication of Statistics Series A 5, 183–189.

29. Westlake, W. J. (1988) Bioavailability and bioequivalence of pharmaceutical formulations, in Biopharmaceutical Statistics for

16

BIOLOGICAL ASSAY, OVERVIEW Drug Developments, K. E. Peace, ed. Marcel Dekker, New York, pp. 329–352.

FURTHER READING Cox, C. (1992) A GLM approach to quantal response models for mixtures, Biometrics 48, 911–928. Moses, L. E. (1965) Confidence limits from rank tests, Technometrics 7, 257–260. Sen, P. K. (2002) Bioinformatics: statistical perspectives and controversies, in Advances in Statistics, Combinatorics and Related Areas, C. Gulati & S. N. Mishra, eds. World Science Press, London, pp. 275–293.

BLOCKED RANDOMIZATION

groups, and similarly there is no control of the balance of allocation to groups over time. Consider the issue of overall imbalance. Let NA and NB represent the numbers of subjects randomized to treatments A and B by simple randomization, respectively. Then the difference NA minus NB is asymptotically normal with a zero mean and variance N = NA + NB . This property can be used to derive the distribution of the difference. The probability that the absolute value of the difference exceeds D is approximately calculated as 2(1–φ(D/n0.5 )) where φ is the cumulative standard normal distribution function. For example, in a trial of 100 subjects, the 95% confidence interval for the difference is calculated as ±19.6, which we round up to 20. Thus, there is a 5% chance of a split that equals or is more extreme than 40 versus 60 or 60 versus 40. Particularly as the sample size increases, the loss of study power that is a consequence of such imbalances is minimal, but other considerations apply. Concerns may include the following: risk of low power at interim analyses, which will have smaller sample sizes than the final; trial credibility to journal or regulatory reviewers; and limited trial supplies. Temporal bias may be caused by too many subjects being assigned to one group early in the study and too few at later time points. If the type of patient enrolled changes over time, then simple randomization may lead to imbalances in the baseline characteristics of the treatment groups. Such changes are entirely feasible and could be caused by investigators who wish to acquire experience of the treatment and trial before they enter the sicker subjects. Similarly, temporal trends in patient outcomes could be caused by the learning curve associated with the trial and treatments. Although theoretically the temporal trends can be accounted for in the analysis, it may not be simple in the typical multicenter trial that starts up centers at different times. For both of the above reasons simple randomization is rarely used.

DAMIAN McENTEGART Head of Statistics, Clinphone Group Ltd, Nottingham, United Kingdom

Randomization techniques can be classified as static, in that allocation is made from a randomization list that is generated before the start of the trial, or they can be dynamic, in that the allocation depends on the current balance of allocated treatments either overall or within particular sub-groups. This article deals with static randomizations from a list composed of one or more blocks. Blocked randomization is the most commonly used randomization technique as judged by reports in medical journals (1) and my own experience (80% of trials on our database of over 1500 randomized trials use blocked randomization). Most theory and examples presented will relate to the case of randomization into two treatment groups with a desired equal allocation. This method is for simplicity of exposition and everything can be generalized to cover multiple treatments and/or unequal randomization ratios. 1

SIMPLE RANDOMIZATION

To understand the rationale for blocked randomization and what it entails, it is first necessary to define simple randomization as a basis for comparison. Simple randomization (alternatively called complete or unconstrained randomization) occurs when each treatment assignment is made at random, and the allocation is completely independent of previous allocations. It has the attractive property of complete unpredictability. One method of constructing a list in a trial of two treatments would be to toss a coin for each allocation. Two disadvantages of simple randomization are that the randomization may yield an imbalance in the overall numbers of subjects assigned to the treatment

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

BLOCKED RANDOMIZATION

2 RESTRICTED RANDOMIZATION THROUGH THE USE OF BLOCKS The alternative to simple randomization is to impose some form of restriction on the randomization. Restricted randomization is defined as any procedure that deviates from simple randomization and controls the randomization to achieve balance between groups in terms of the overall size of the groups and/or their characteristics. The focus of this article is blocked randomization methods, which are used to balance treatment groups overall and, if needed, for time trends and prognostic factors. We will deal with two types of blocked randomization. The first instance is in which the block size or length equals the required sample size (i.e., there is one block for the entire trial). Abel (2) terms this complete balanced randomization. We will devote a section to such blocks. The second is random permuted blocks, in which several blocks make up the randomization list. For convenience, we deal with the second type of scheme first. 2.1 Random Permuted Blocks The permuted blocks procedure involves randomizing subjects to treatment by constructing a randomization list partitioned into blocks. In the simplest case of a constant block size with two treatments and m subjects per treatment in each block, then the block size is 2m. If the list is composed of B blocks, then the total number of entries on the list is 2mB. Allocation should be made sequentially from the list that corresponds to the order that subjects are randomized; otherwise the process is subject to subversion. Thus, the list is enforcing a perfect balance of treatment allocations after every 2m subjects have been recruited. Although this method protects against temporal trends, the constraint of return to periodic balance has the disadvantage that the allocations become increasingly predictable toward the end of each block if the past allocations and block size are known. It may lead to investigator selection bias where the investigator may decide not to enter particular subjects in the trial to avoid the receiving treatments that are not favoured by the investigator (e.g., the

control) or delay their entry until the chances of them receiving a particular treatment (e.g., the experimental treatment) are better than average. We will refer to the topic of selection bias again. Permuted blocks are used when there is a perceived need to protect against time trends in patient characteristics and/or outcomes. Even in the perceived absence of time trends, it is often advantageous to ensure a reasonable degree of overall treatment allocation balance during the trial in case of interim analyses. An additional motivation for permuted blocks may be the need to balance treatment groups within prognostic strata. 2.2 Generation of Blocks Two methods of generating blocks are used: the random allocation rule and the truncated binomial design. Given the block size 2m, the random allocation rule is to generate the randomization probability for each treatment in accordance with the totals of each treatment that still have to be assigned to meet the overall balance requirement. Thus, for example, if we are randomizing the fifth patient in a block of eight, and the allocations in the block to date have been three for treatment A and two for treatment B, then the patient will be randomized to receive treatment A with probability 1/3 and treatment B with probability 2/3. The random allocation rule is equivalent to sampling from the population of all possible randomization orderings with equal probability. If multiple blocks are used, then generally the blocks are generated independently (i.e., sampled with replacement). The alternative method of the truncated binomial design (3) is effected by using simple randomization until m of one of the treatments have been assigned, then assigning the other treatment to the remaining subjects. The chance of a long run of predictable allocations at the end of truncated binomial design blocks has precluded their use in practice, and they will not be referred to again. 2.3 Stratified Randomization Using Permuted Blocks In the absence of fraud or selection bias, there will be no systematic reason for treatment imbalances on prognostic factors using

BLOCKED RANDOMIZATION

simple or blocked randomization but chance imbalances are possible, especially in smaller trials. Kernan et al. (4) investigate the chances of an imbalance for two treatment groups on a binary prognostic factor that is present in 30% of the population. The chance that the two treatment group proportions will arithmetically differ by more than 10% is 43% for a trial of 30 subjects, 38% for a trial of 50 subjects, 27% for a trial of 100 subjects, 9% for a trial of 200 subjects, and 2% for a trial of 400 subjects. These results relate to a single factor; the chance of at least one imbalance on multiple factors is magnified. Chance imbalances on important prognostic variables reduce the precision of the treatment estimate, particularly for smaller trials. But if the prognostic variables are accounted for in the analysis model (poststratification), then the loss in precision for trials of more than 50 subjects per treatment group is slight (5). Of more importance may be the attitude of regulators to chance imbalances. European regulators require sensitivity analyses ‘‘to demonstrate that any observed positive treatment effect is not solely explained by imbalances at baselines in any of the covariates’’ and further note that in the case of a ‘‘very strong baseline imbalance, no adjustment may be sufficiently convincing to restore the results’’ (6). Thus trialists may wish to protect the credibility of their trial by balancing on a few important prognostic factors. A list of other potential reasons for achieving balance is given by McEntegart (5). With a randomization list, balancing takes place within individual strata defined as the crossclassification of prognostic factor levels. For continuous scale variables, this procedure will involve some categorization; methods for deciding on cut-points are available (7). As the number of balancing factors and levels increases, the number of strata increases geometrically. Regulatory guidance suggests that the use of more than two or three balancing factors is rarely necessary (8). If the randomization is to be stratified, then in the absence of temporal trends, a single block (complete balanced randomization) within each stratum would be a possibility in which the numbers of patient to be recruited in each stratum can be fixed in advance. Fixed recruitment, however, is often undesirable

3

because of the implications for the duration of trial recruitment and the extra administrative burden [although interactive voice response (IVR)/web randomization provides a solution where recruitment to each stratum is automatically closed once the stratum limit has been reached (9)]. Alternatively, where stratum sizes are not fixed in advance, permuted blocks will have to be used. Blocks are generated and assigned to each stratum with the effect that a separate randomization list is observed for each stratum. The allocation of blocks to strata is normally done at the time of generating the randomization list. If an IVR/web system is being used to perform the allocation during the trial, then it is not necessary, and allocation of blocks can be performed dynamically as they are needed. This method can be advantageous when center or country is stratification factors. Consider the example of stratification by gender and site. For any particular gender* site stratum, when the first subject with the relevant characteristics enrolls in the study, the system allocates the first free unallocated block of randomization code to that stratum. The current subject is then assigned the treatment associated with the first randomization code in the block, and the block is reserved for future subjects in this stratum. Subsequent subjects who enroll into this stratum are assigned to the next available randomization entry in the block. As a block is completed, then the next available block is reserved if another subject enrolls into that stratum. This allocation saves code that may have some small benefits in terms of checking, maintenance, and system performance. More importantly, it allows flexibility in site recruitment including easily allowing new, unplanned sites to join the study; the system simply allocates blocks of code that follow the same process. When using such a process however, it is important that the randomization (sequence) number is not revealed to the investigator, or else he can deduce the block size from the step change in the randomization numbers when allocation is made from a new block. If it is desired to reveal the randomization number, then a scrambled number that differs from the sequence number on the list should be used. Arguably using dynamic allocation

4

BLOCKED RANDOMIZATION

of blocks makes unscrupulous monitoring by the study team more difficult because it takes more work to deduce the blocks used within each strata. In a stratified design with K strata, then potentially K incomplete (open) blocks are at the end of the trial, each with a maximum imbalance of m and a maximum imbalance for the trial as a whole of Km. If too many strata are relative to the number of subjects and treatments, then the trial is subject to what has been called over stratification (10). Balance may not be achieved for the study as a whole or for the individual strata levels if multiple stratifying factors exist. Hallstrom and Davies (11) provide methodology and formulae to evaluate the risk of an aggregate imbalance combined over strata. For a twotreatment group trial, an approximate 95% confidence interval for the difference between the treatment group allocations can be calculated as ±1.96* (K* (2m + 1)/6)1/2 . Extensions to other numbers of treatment groups and unequal allocation ratios are possible or simulation may be used. Kernan et al. (4) suggest that to avoid the worst effects of incomplete blocks in trials with interim analyses, the number of strata for a two-treatment group trial is limited to the number of subjects divided by four times the block size. This method is to allow for interim analyses. If the number of strata is too large, then it may be necessary to use a dynamic randomization technique such as minimization (12) or dynamic balancing (13). The merits and issues of using such techniques are discussed elsewhere (1,5). If it is desired to balance over a stratification factor at the study level in a multicenter trial, then a central randomization using IVR/web randomization will have to be used. Country is often an appropriate factor in multinational trials (6). Finally, we note that the logistics of organizing medication supplies for stratified randomization in multicenter trials are considerably simpler in IVR/web systems that separate the randomization step from the dispensing step (9). In a traditional trial, supplies are provided to sites in patient numbered packs; normally the patient number will match the number used on the case record form used to

collect the patient data. If the trial is stratified by any factor in addition to site, then typically separate sets of patient numbered packs are reserved for the lists used in each stratum at the site. Unless the numbers to be recruited to each stratum at each site are fixed, which rarely happens as this would slow recruitment, then considerable wastage of patient supplies will occur, as packs are provided that allow for flexibility of strata composition in recruitment. 2.4 Multicenter Trials and Blocking To meet recruitment timelines and to aid generalization of results, most clinical trials are conducted at more than one center. Although a special case of stratification, multicenter trials deserve a separate discussion with regard to blocking because it is the most common use of blocked randomization. European regulatory guidance (6) notes that ‘‘most multicentre trials are stratified by centre (or investigator) for practical reasons or because centre (or investigator) is expected to be confounded with other known or unknown prognostic factors.’’ This regulation reflects the ICH E9 Guidance (8), which states that ‘‘It is advisable to have a separate random scheme for each centre, i.e. to stratify by centre or to allocate several whole blocks to each centre.’’ Most pharmaceutical industry trials are stratified by center, and it seems to be true of all trials. In a survey of trials reported in a 3 month period in 1997 in leading journals, Assmann et al. (14) found that 18 of 25 trials gave sufficient detail stratified by site. One reason for using some form of site stratified, blocked randomization in multicenter trials that are not managed electronically develops from the logistical considerations surrounding supplies and the uncertainty surrounding the recruitment process. One way of organizing the randomization would be to construct a single block of randomization codes equal to the trial sample size and sequentially send several corresponding patient numbered packs to each site—the exact number depending on each site’s agreed recruitment target. If all sites filled their target, then randomization is completely balanced at the study level. But sites

BLOCKED RANDOMIZATION

rarely meet their recruitment target with a consequent high risk of substantial imbalance at the study level. To avoid these problems, randomization for multicenter trials are usually balanced at the site level; this target is most commonly achieved via a blocked randomization, which also guards against temporal trends within each site as well as simplifying the logistics. One way of stratifying by site in a trial with randomization managed electronically is to construct the site stratified blocks dynamically by use of a balancing constraint. This method is a variant of the balanced block randomization scheme first described by Zelen (15). Consider a trial with five treatment groups where it is required to balance at the site level. In the dynamic scheme, subject to the balance constraint, subjects are allocated the treatment that corresponds to the next unused randomization code from a central unstratified randomization list with a block size of five. The balance constraint is that allocation must not cause the treatment imbalance (maximum minus minimum) at site to exceed one. In effect, blocks of size five are being used at the site level with the blocks being dynamically constructed considering the study level balance. A consequence of this method is that the codes on the central list are not always allocated sequentially. The advantage of the scheme is that it allows site balancing without the risk of imbalance at the study level from too many open blocks. It is particularly relevant in studies with many treatments conducted at many sites in which the average recruitment is expected to be less than the number of treatments. If site stratification is employed when it may not otherwise have been (because of the risk of study level imbalance), then a marked supply savings can occur if a system that automatically manages site inventories is also used (16). Assuming subjects enter in a random order, then all randomization sequences within any given center are possible before the trial starts, which is a perfectly valid randomization scheme, albeit that the blocks are not being selected independently and the sample is restricted (for instance, it would not be possible to have all centers commencing the first block with the same treatment). However, the scheme technically violates the

5

ICH E9 Guidance (8), which states that ‘‘The next subject to be randomized into a trial should always receive the treatment corresponding to the next free number in the appropriate randomization list.’’ Wittes (17) has argued that block sizes can be varied by clinic with expected higher recruiting centers having higher block sizes. This option is rarely used because of the problems of predicting the high recruiting clinics. It has, however, been implemented on an ongoing basis in the light of actual experience within an IVR system. In this scheme, the size of block that was dynamically allocated by the system was determined by the calculated recruitment status of the center. 3 SCHEMES USING A SINGLE BLOCK FOR THE WHOLE TRIAL If no stratification occurs, then the simplest way to overcome the problem of unequal allocation to groups is to restrict the randomization list so that the numbers in each group are equal (or in the required proportions). This function is equivalent to using a single block for the whole trial. 3.1 Maximal Procedure Abel (2) described an alternative randomization scheme that constrains the size of the within block imbalance as follows. One specifies a measure of imbalance, I , and a maximum permitted imbalance I0 ; the simplest imbalance measure is the range of treatment allocations. A randomization list of the required length n1 is then generated using simple randomization. Compute the imbalance In for each n ≤ n1 . If, for some n, In > I0 then replace the list. The process is repeated until an acceptable list is found. Berger et al. (18) extended Abel’s concept through the maximal procedure, which also imposes the condition of so called terminal balance on the list. Thus, the maximal procedure retains the desirable characteristics of the randomized permuted block design namely by minimizing imbalance and temporal bias. The permuted block procedure is characterized by its forced returns to perfect balance at the end of each block. For a given block size, the imbalance cannot

6

BLOCKED RANDOMIZATION

exceed half the block size, and this balance is called the maximum tolerated imbalance (MTI). Forced returns to periodic imbalance at the end of every block do not substantially help control temporal bias, yet they do potentially increase selection bias. The maximal procedure considers schedules that satisfy the constraint of never having an imbalance between treatments that exceeds the given MTI and also satisfy the constraint of perfect balance at the end of the sequence. The scheme is therefore less subject to selection bias at the cost of a very small increase in expected imbalance if the trial does not run to completion. Sequences are generated using the random allocation rule until one is found that satisfies the MTI conditions. The technique can be used in stratified trials, but unless the numbers in the strata are fixed, there is a slight increase in the risk of imbalance at the study level. This result occurs because of the slightly higher expected imbalance caused by stopping part way through the maximal block. Nevertheless, it seems a small price to pay, and the technique should be considered in any site stratified trial where the possibility of selection bias is a concern. 3.2 Hadamard Matrix Procedure The maximal scheme has some similarities to Bailey and Nelson’s method (19). This method of generating blocks involves selecting a block at random from an array derived from the permuted columns of a Hadamard matrix (20) with a selection criterion based on the MTI at any point in the block. For instance, with a block size of 12 for a two treatment trial, the example scheme used in the paper has an MTI of 3, and there are 44 possible blocks to choose from. If multiple blocks are selected with replacement, then the resultant scheme is valid in the sense that if terms for treatment and block are included in the analysis model, the model-based estimator of the error variance is also unbiased in that its expectation over the randomization is correct no matter what the order of patient presentation. If multiple blocks are used in an unstratified trial, then we are enforcing periodic returns to balance; thus, the similarity with the maximal procedure ends here. Furthermore, the Hadamard scheme is

less flexible than the maximal scheme, more prone to selection bias, more restrictive (for example for a trial size of 12 and an MTI of 3, there are 792 potential maximal sequences and only 44 in the Hadamard example), and harder to generate for block sizes larger than 12. It does, however, avoid the need for a permutation test in the analysis (see later). 4 USE OF UNEQUAL AND VARIABLE BLOCK SIZES The issue of predictability and selection bias is particularly relevant in site-stratified trials that are not blinded and where the investigator knows the past treatment allocations. One strategy that is sometimes used in an attempt to reduce potential selection bias is to employ a variable block design with some degree of randomness in the block sizes. The block sizes could be chosen at random from a specified subset of block sizes or alternatively be determined as a random ordering of a defined number of each possible block size. The advantage of the latter is that it is possible to fix the exact number of records in randomization list, which is not possible with the former scheme. This strategy is implicitly encouraged in the International Conference on Harmonisation (ICH) E9 Guidance on statistical principles for clinical trials (8) which states that: Care should be taken to choose block lengths that are sufficiently short to limit possible imbalance, but that are long enough to avoid predictability towards the end of the sequence in a block. Investigators and other relevant staff should generally be blind to the block length; the use of two or more block lengths, randomly selected for each block, can achieve the same purpose. (Theoretically, in a double-blind trial predictability does not matter, but the pharmacological effects of drugs may provide the opportunity for intelligent guesswork.)

The considerations that surround such a strategy depend on the assumed type of selection bias. Under the Blackwell-Hodges model (3), the experimenter wishes to bias the study by selecting a patient with a more favorable expected outcome when he guesses that treatment A is the next treatment to

BLOCKED RANDOMIZATION

be allocated. Conversely, the experimenter selects a patient with a poorer prognosis when he guesses that treatment B is the next treatment to be allocated. Clearly the optimal strategy for the investigator is to guess treatment A when it has been allocated least to date, to guess treatment B if it has been allocated least to date, and to guess with equal probability in the case of a tie in prior allocations. This method is called the convergence strategy. Under the Blackwell-Hodges model, the use of random block sizes does not reduce the potential for or degree of selection bias as the investigator’s strategy and expected successes remains the same even if he is unmasked to the block sizes. Rosenberger and Lachin (21) show that a design employing multiple block sizes has an expected bias factor associated with the average block size. So under the Blackwell Hodges model, no advantage exists to the use of random block sizes. Indeed if the MTI condition is not relaxed, then variable blocking will lead to even more scope for prediction than fixed size blocks because of the greater number of blocks and the associated return to balance on the completion of each block (18). But this argument is overstated, as one would not generally know the size of the current block. Furthermore, in my experience the MTI requirement is usually relaxed when variable blocking is used (e.g., practitioners might consider mixed blocks of 4 and 6 rather than fixed blocks of size 4). From the above, one might conclude that there is little advantage in employing variable sized blocks. But it would not be a complete representation of the situation. Arguably with fixed blocks, investigators often can guess the block size from customary practice based on logistical and balance considerations. For example, in a site stratified trial, supplies may be sent to sites in single blocks (22,23). Also, overall study level balance considerations may dictate against the use of larger block sizes; in a site stratified trial of two treatments, the block size is almost always chosen to be four and investigators are aware of this. In this case, the potential for selection bias is a concern as the investigator can determine the tail allocations of the block with complete certainty. Matts and Lachin (24) formally investigate

7

the bias properties in this situation and show that the expected number of treatments that are completely predictable within a block of known length 2m containing m allocations to each of two treatments is (m + 1) – 1 . They investigate the scenario in which the sequence of block sizes is masked but the sizes employed are unmasked. As might be expected, the use of variable block sizes will substantially reduce the potential for prediction with certainty in this scenario. Dupin-Spriet et al. (25) quantify the degree of prediction with certainty in trials with more than two arms and in trials with an unequal allocation ratio. For example in a three-arm trial with balanced blocks of size 6, 20% of treatments assignments are predictable. In a later paper (26), the same authors devise methods to quantify prediction with certainty in the cases of series with two and three unequal sized blocks for schemes that they claim are of a common size (e.g., a fixed number of subjects per center of 8, 12, or 30). For instance in the case of two blocks, the calculations will quantify the probability of identifying a long block when it comes before a short one if it starts with a sequence incompatible with the content of a short block. Both the situations of known block lengths as well as concealed order and concealed block lengths and order are considered. The results are compared against the predictability of the maximal method in the cases where the MTI is known and unknown. For a blocked scheme, if the details of the scheme (block lengths and order) can be assumed to be unknown, then there is actually no reduction in predictability with certainty when using unequal length blocks. The best scheme is always the one with the shortest possible block length as the unknown length order decreases opportunities for deductions about the composition of the blocks. But of course, it is precisely this scheme that is the most likely to be guessed. Furthermore, the scenarios of fixed recruitment per center are relatively uncommon. So although it is interesting to consult this reference when devising a scheme, the use of variable blocking should not be dismissed solely based on this reference.

8

BLOCKED RANDOMIZATION

In summary, the ICH suggestion to use blocks of unequal size is not as straightforward as it might first seem. But it is to be encouraged in open label trials (or blinded trials that are subject to guesswork) where for logistical or other reasons, it is felt necessary to stratify at the site level. It is most relevant when trial designers can be persuaded to use a higher MTI than they would otherwise have considered or where there is no requirement for a fixed number of randomization numbers at each site. In the absence of the latter requirement, block sizes can truly be chosen at random from the specified set of allowable block sizes, which provides less scope for deduction. 4.1 Mixed Randomization The notion of variable blocking has been extended by Schulz and Grimes (27), who recommend their method of mixed randomization for non-double-blind trials. The mixed randomization procedure begins with an uneven block generated with a maximum disparity between the treatments. This procedure is then followed by series of random permuted blocks with randomly varied block sizes interspersed with a simple random sequence. If used in site stratified multicenter trials, the sample size per center would have to be large to allow for the practicalities of the scheme; and more scope for study level imbalances exists compared with conventional site stratification. Nevertheless, this method adds another tool to the armoury of trying to confuse the issue and may be useful. The concept of uneven blocks at the start of the sequence is one I have used on occasion. 5 INFERENCE AND ANALYSIS FOLLOWING BLOCKED RANDOMIZATION The theory underpinning inferences from analyses following the use of blocked randomization is provided by Rosenberger and Lachin (21). On most occasions, a randomization model is used as the basis for inference. Permutation tests based on the randomization model assume that under the null hypothesis, the set of observed patient responses are fixed irrespective of treatment assignment. Then, the observed difference

between the treatment groups depends only on the treatment assigned, and statistical testing can be based around it. Treating the observed patient outcomes as given, the chosen test statistic is computed for all possible permutations of the randomization sequence. The incidence of observing a test statistic that is equal to or is more extreme than the observed test statistic is enumerated, and the probability is calculated by dividing by the total number of possible randomizations. This calculation is assumption free. As enumeration of all possible randomizations may be onerous as the sample size increases, an adequate alternative is to perform a Monte Carlo simulation of 10,000 or 100,000 randomization sequences. Actually, there is no need to perform such analyses following conventional blocked randomization because conventional tests, such as the block-stratified Mantel-Haenszel test, or analysis of variance fitting a factor for the block effect, will provide tests that are asymptotically equivalent to the permutation test (21). To ensure that tests have a true false positive error rate that is close to the nominal rate (e.g., 0.05), the analysis must account for the blocking and any stratification employed. Consider the case of blocking in an unstratified trial. Rosenberger and Lachin (21) show that the correct analysis accounts for the blocks and that the bias of an analysis without inclusion of the blocks depends on the direction of the intrablock correlation coefficient. In addition, an analysis that does not take the blocking into account is likely to be conservative. Omitting the blocks from analyses is common practice because most practitioners doubt the prognostic significance of the blocking factor, and the large number of extra parameters can cause instability and other model-fitting difficulties; this practice is acknowledged by the regulators (28). The situation regarding stratification factors is not treated so liberally, because although an analysis that does not reflect the stratification should again be conservative, there is no excuse for not performing the appropriate test as required under the assumptions of the randomization model. Thus, the ICH E9 Guidance (8) states that ‘‘if one or more factors are used to stratify the

BLOCKED RANDOMIZATION

design, it is appropriate to account for these factors in the analysis,’’ and the European guidance (6) states that, The primary analysis should reflect the restriction on the randomization implied by the stratification. For this reason, stratification variables—regardless of their prognostic value – should usually be included as covariates in the primary analysis.

The European guidance recognizes the problem of low recruiting centers, and if center was used in the randomization but not adjusted for in the analysis then sponsors ‘‘should explain why and demonstrate through well explained and justified sensitivity analyses, simulations, or other methods that the trial conclusions are not substantially affected because of this.’’ But generally the intention of the regulators is clear—you should ‘‘analyse as you randomize.’’ Appropriate theory has not been developed for the other single-block methods described. Thus, to use conventional asymptotic analysis, one must appeal to the homogeneity of the patient population. If one is not prepared to make this assumption, then Kalish and Begg (29) argue that large time trends would need to be observed before there was substantial distortion of the test characteristics; such large trends would be observable and can be accounted for in the model. This issue remains a debate. 6 MISCELLANEOUS TOPICS RELATED TO BLOCKED RANDOMIZATION 6.1 Blocking in Crossover and Group Randomized Designs In certain trials, subjects will be scheduled to receive the treatments in different orders. In these crossover trials, each subject serves as his own control provided that the results from each treatment administration are independent of the preceding allocations; arguably then no need exists to constrain the randomization in any way (i.e., a simple randomization can be used for the sequence of treatment administrations to each patient). But carryover of effects from the previous treatment can never be ruled out, so it is conventional to balance the treatment sequences (e.g., in a

9

two treatment, two period crossover trial, the two possible sequences are usually generated in blocks). In trials with three treatments, the six possible orderings are grouped in a block. In trials with more than three treatments, Williams squares are used as blocks that are balanced Latin square designs constructed so that each treatment is preceded equally often by each other treatment (30,31). Bellamy et al. (32) consider the situation of randomizing subjects in cohorts of a predetermined size to one of two study treatments using blocking factors, the composition of which cannot be determined prior to assembling the cohorts for randomization. This situation develops in trials in which subjects are to be randomized as a group, but not all the subjects attend for the visit. The randomization is effected by alternation into arbitrarily labelled groups within each stratification factor level and then random assignment of these groups to treatment. 6.2 Generation of Blocks: Software and Practical Considerations Most computer statistical packages include pseudo-random number generators, which are so called because the numbers they generate are not the result of a random process but have properties similar to those generated via a random process. These numbers can be used to prepare randomized blocks using the random allocation rule or by generating the block contents and then using the random number generator to assign block position. The SAS procedure PROC PLAN (SAS Institute, Inc., Cary, NC) provides a simple and flexible module for generating randomization lists; Deng and Graz (33) give some useful examples of generating blocked randomizations using this procedure. A variety of freely available software is available for generating blocked randomizations. A recent and flexible package that overcomes the lack of flexibility of some earlier software is described by Saghaei (34). It is important that the trial statistician reviews the generation process used for the randomization list before the trial. Ideally, the trial statistician should not generate the list. My recommended process is that the statistician reviews a ‘‘dummy list’’ that is

10

BLOCKED RANDOMIZATION

created by a validated software package that, based on a set of input parameters, simultaneously generates two lists using different seeds. The dummy list can be used for data set-up, testing, and quality control procedures. Procedures should be in place to ensure secure storage and appropriate limited access to the list after generation. Generally, blocks will be generated independently of one another, that is, they are sampled with replacement, as this method underpins the theory for statistical inference. Generally, there is no need to restrict the number of possible randomization orderings unnecessarily. But marginal advantages for overall treatment balance may be obtained by ensuring a balance of block types within the list for center stratified schemes where incomplete blocks can be expected. No real objection to this practice will occur provided the details are kept away from staff involved in the monitoring of the trial. 6.3 Blocked Randomization also Used for Drug Packaging

kept in separate documents or appendices that are not seen by staff involved with the trial performance or conduct. Good Clinical Practice (36) requires that the protocol contain ‘‘a description of the measures taken to minimize/avoid bias, including randomization and blinding,’’ but it is not necessary to go into specific detail about the type of blocking and block sizes. In contrast after the trial has completed, reports of the trial should contain full details of the randomization procedures. The CONSORT group statement (37) asks for details of the method used to generate the random allocation sequence and details of blocking and stratification. REFERENCES 1. N. W. Scott, G. C. McPherson, C. R. Ramsay, and M. K. Campbell, The method of minimization for allocation to clinical trials: a review. Control. Clin. Trials 2002; 23: 662–674. 2. U. Abel, Modified replacement randomization. Statist. Med. 1987; 6: 127–135.

Trial material packaging is also done using randomized blocks. When the patient number corresponds to the randomization sequence number as it does in a traditional trial, then only one scheme is needed. If, however, the randomization and material dispensing steps are separated, as is often the case with IVR and web trials (9), then separate randomizations will have to be performed for randomization and the packaging list. To avoid unblinding through logical deduction, it is important that after the randomization has identified the appropriate treatment to allocate, that selection from the set of packs available at the site is performed randomly (35). This function is best achieved by use of a double randomized pack list; this block randomized pack list is then scrambled to remove any association between pack number and sequence number (35).

3. D. Blackwell and J. L. Hodges, Design for the control of selection bias. Ann. Math. Stats. 1957; 28: 449–460.

6.4 Documentation and Reporting

8. International Conference on Harmonisation, E-9 Document, Guidance on statistical principles for clinical trials. Federal Register 1998; 63: 49583–49598. Available: http://www.fda.gov/cder/guidance/91698.pdf.

From the preceding discussions, it is clear that the more that can be kept from investigators and monitors, the better it is for study integrity. Thus, before and during the trial conduct, specifics of randomization should be

4. W. N. Kernan, C. M. Viscoli, R. W. Makuch, L. M. Brass, and R. I. Horwitz, Stratified randomization for clinical trials. J. Clin. Epidemiol. 1999; 52: 19–26. 5. D. J. McEntegart, The pursuit of balance using stratified and dynamic randomisation techniques. Drug. Inf. J. 2003; 37: 293–308. Available: http://www.clinphone.com/files/ Stratfied%20Dynamic%20Randomization% 20Techniques.pdf. 6. Committee for Proprietary Medicinal Products. Points to Consider on Adjustment for Baseline Covariates. 2003. CPMP/EWP/283/ 99 Available: http://www.emea.eu.int/pdfs/ human/ewp/286399en.pdf. 7. S. C. Choi, T. P. Germanson, and T. Y. Barnes, A simple measure in defining optimal strata in clinical trials. Control. Clin. Trials 1995; 16: 164–171.

9. B. Byrom, Using IVRS in clinical trial management. Appl. Clin. Trials 2002; 10: 36–42.

BLOCKED RANDOMIZATION http://www.clinphone.com/files/Using IVRS in trial%20management.pdf 10. R. Simon, Patient subsets and variation in therapeutic efficacy. Br. J. Clin. Pharmacol. 1982; 14: 473–482. 11. A. I. Hallstrom and K. Davis, Imbalance in treatment assignments in stratified blocked randomization. Control. Clin. Trials 1988; 9: 375–382. 12. S. J. Pocock and R. Simon, Sequential treatment assignment with balancing for prognostic factors in the controlled clinical trial. Biometrics 1975; 31: 103–115. 13. S. Heritier, V. Gebski, P. Pillai, Dynamic balancing randomization in controlled clinical trials. Stat. Med. 2005; 24:3729-3741. 14. S. F. Assman, S. J. Pocock, L. E. Enos, L. E. Kasten, et al., Subgroup analysis and other (mis)uses of baseline data in clinical trials. Lancet 2000; 355: 1064–1069. 15. M. Zelen, The randomization and stratification of patients to clinical trials. J. Chronic. Dis. 1974; 27: 365–375. 16. D. McEntegart and B. O’Gorman, Impact of supply logistics of different randomisation and medication management strategies used within IVRS. Pharmaceut. Engin. 2005; 36–46. Available: http://www. clinphone.com/files/ISPE%20paper1.pdf. 17. J. Wittes, Randomized treatment assignment. P Armitage, T Colton (eds). Encyclopedia of Biostatistics. John Wiley, 1998.:3703-3711 18. V. W. Berger, A. Ivanova, and M. Deloria Knoll, Minimizing predictability while retaining balance through the use of less restrictive randomization procedures. Stat. Med. 2003; 22: 3017–3028. 19. R. A. Bailey and P. R. Nelson, Hadamard randomisation: a valid restriction of random permuted blocks. Biomet. J. 2003; 45: 554–560. 20. A. S. Hedayat and W. D. Wallis, Hadamard matrices and their applications. Ann. Statist. 1978; 6: 1184–1238. 21. W. F. Rosenberger and J. M. Lachin, Randomization in Clinical Trials. New York: John Wiley & Sons, 2002. 22. S. J. Kunselman, T. J. Armstrong, T. B. Britton, and P. E. Forand, Implementing randomisation procedures in the asthma clinical research network. Control. Clin. Trials 2001: 22:181S–195S. 23. D. McEntegart, Implementing randomization procedures in the Asthma Clinical Research Network. Control. Clin. Trials 2002; 23: 424–426.

11

24. J. P. Matts and J. M. Lachin, Properties of permuted-block randomization in clinical trials. Control. Clin. Trials 1988; 9: 327–344. 25. T. Dupin-Spriet, J. Fermanian, and A. Spriet, Quantification of predictability in block randomization. Drug. Inf. J. 2004; 38: 135–141. 26. T. Dupin-Spriet, J. Fermanian, and A. Spriet, Quantification methods were developed for selection bias by predictability of allocations with unequal block randomization. J. Clin. Epidemiol. 2005; 58: 1269–1270. 27. K. F. Schulz and D. A. Grimes, Unequal group sizes in randomised trials: guarding against guessing. Lancet 2002; 359: 966–970. 28. J-M. Grouin, S. Day, Lewis, Adjustment for baseline covariates: an introductory note. J. Stat. Med. 2004; 23: 697–699. 29. L. A. Kalish and C. B. Begg, The impact of treatment allocation procedures on nominal significance levels and bias. Control. Clin. Trials 1987; 8: 121–135. 30. E. J. Williams, Experimental designs balanced for the estimation of residual effects of treatments. Australian J. Sci. Res. 1949; 2A: 149–168. 31. R. G. Newcombe, Sequentially balanced threesquares cross-over designs. Stat. Med. 1996; 15: 2143–2147. 32. S. L. Bellamy, A dynamic block-randomization algorithm for group-randomized clinical trials when the composition of blocking factor is not known in advance. Contemp. Clin. Trials 2005; 26: 469–479. 33. C. Deng and J. Graz, Generating randomisation schedules using SAS programming. SUGI 27, April 14-17 2002 (SAS Users Group) Paper 267-27. Available: http://www2. sas.com/proceedings/sugi27/p267-27.pdf. 34. M. Saghaei, Random allocation software for parallel group randomized trials. BMC Med. Res. Methodol. 2004; 4: 26. Available: http:// www.biomedcentral.com/1471-2288/4/26. 35. M. Lang, R. Wood, and D. McEntegart, Double-randomised Packaging Lists in Trials managed by IVRS. Good Clin. Pract. J. 2005; 10–13. Available: http://www.clinphone.com/ files/GCPJ%20article%20final%20Nov%2020 05.pdf. 36. International Conference on Harmonisation, E-6 Document. Good Clinical Practice. Federal Register 1997; 62: 25691–25709. Available: http://www.fda.gov/ cder/guidance/iche6.htm. 37. CONSORT Group, D. G. Altman, et al., The revised CONSORT statement for reporting

12

BLOCKED RANDOMIZATION randomised trials: explanation and elaboration. Ann. Intern. Med. 2001; 134: 663–694.

FURTHER READING V. W. Berger, Selection Bias and Covariate Imbalances in Clinical Trials. New York: John Wiley & Sons, 2005.

CROSS-REFERENCES Blocking Randomization Randomization Codes Randomization Methods Randomization Procedures Randomization List Simple Randomization Stratified Randomization Interactive Voice Randomization System (IVRS)

to x (the difference is −0.047, to three decimal places). The bootstrap distribution looks normal, with some skewness. This amount of skewness is a cause for concern. This example may be counter to the intuition of many readers, who use normal probability plots to look at data. This bootstrap distribution corresponds to a sampling distribution, not raw data. This distribution is after the central limit theorem has had its one chance to work, so any deviations from normality here may translate into errors in inferences. We may quantify how badly this amount of skewness affects confidence intervals; we defer this to the section on Bootstrap confidence intervals. We first discuss the idea behind the bootstrap, and give some idea of its versatility.

BOOTSTRAP TIM HESTERBERG Insightful Corp., Seattle, Washington

1

INTRODUCTION

We begin with an example of the simplest type of bootstrapping in this section, then discuss the idea behind the bootstrap, implementation by random sampling, using the bootstrap to estimate standard error and bias, the central limit theorem and different types of bootstraps, the accuracy of the bootstrap, confidence intervals, hypothesis tests, planning clinical trials, the number of bootstrap samples needed and ways to reduce this number, and we conclude with references for additional reading. Figure 1 shows a normal quantile plot of Arsenic concentrations from 271 wells in Bangladesh, from http://www.bgs.ac.uk/ arsenic/bangladesh/Data/SpecialStudyData. csv referenced from statlib http://lib.stat.cmu. edu/datasets. The sample mean and standard deviation are x = 124.5 and s = 298, respectively. √The usual formula standard error is s/ n = 18.1, and the√usual 95% confidence interval x ± tα/2,n−1 s/ n is (88.8, 160.2). This interval may be suspect because of the skewness of the data, despite the reasonably large sample size. We may use the bootstrap for inferences for the mean of this dataset. We draw a bootstrap sample, or resample, of size n with replacement from the data, and compute the mean. We repeat this process many times, say 104 or more. The resulting bootstrap means comprise the bootstrap distribution, which we use to estimate aspects of the sampling distribution for X. Figure 2 shows a histogram and normal quantile plot of the bootstrap distribution. The bootstrap standard error is the standard deviation of the bootstrap distribution; in this case the bootstrap standard error is 18.2, which is close to the formula standard error. The mean of the bootstrap means is 124.4, which is close

2

PLUG-IN PRINCIPLE

The idea behind the bootstrap is the plug-in principle (1)—that if a quantity is unknown, then we plug in an estimate for it. This principle is used all the time in statistics. The standard deviation of a sample mean for i.i.d. observations from a √ population with standard deviation σ is σ/ n; when σ is unknown, we plug in an estimate s to obtain √ the usual standard error s/ n. What is different in the bootstrap is that we plug in an estimate for whole population, not just for a numerical summary of the population. Statistical inference depends on the sampling distribution. The sampling distribution depends on the following: 1. the underlying population(s), 2. the sampling procedure, and 3. the statistic, such as X. Conceptually, the sampling distribution is the result of drawing many samples from the population and calculating the statistic for each. The bootstrap principle is to plug in an estimate for the population, then mimic the real-life sampling procedure and statistic calculation. The bootstrap distribu-

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

BOOTSTRAP

1500 1000 0

500

As (ug/L)

2000

2

−2

−1

0

1

2

3

Figure 1. Arsenic concentrations in 271 wells in Bangladesh.

0.025

Quantiles of Standard Normal

180 160 Mean

0.015

100

120

0.010 0.0

80

0.005

Density

0.020

200

Observed Mean

140

−3

80

100

120

140

160

180

200

Mean

−4

−2

0

2

4

Quantiles of standard normal

Figure 2. Bootstrap distribution for arsenic concentrations.

tion depends on 1. an estimate for the population(s), 2. the sampling procedure, and 3. the statistic, such as X. The simplest case is when the original data are an i.i.d. sample from a single population, and we use the empirical distributhe population, where tion Fˆ n to estimate
Fˆ n (u) = (1/n) I(xi ≤ u). This equation gives the ordinary nonparametric bootstrap, which corresponds to drawing samples of size n without replacement from the original data.

2.1 How Useful is the Bootstrap Distribution? A fundamental question is how well the bootstrap distribution approximates the sampling distribution. We discuss this question in greater detail in the section on Accuracy of bootstrap distribution, but note a few key points here. For most common estimators (statistics that are estimates of a population parameter, e.g., X is an estimator for µ, whereas a l statistic is not an estimator), and under fairly general distribution assumptions, center: the center of the bootstrap distribution is not an accurate

BOOTSTRAP

approximation for the center of the sampling distribution. For example, the center of the bootstrap distribution for X is centered at approximately x, the mean of the sample, whereas the sampling distribution is centered at µ. spread: the spread of the bootstrap distribution does reflect the spread of the sampling distribution. bias: the bootstrap bias estimate (see below) does reflect the bias of the sampling distribution. skewness: the skewness of the bootstrap distribution does reflect the skewness of the sampling distribution. The first point bears emphasis. It means that the bootstrap is not used to get better parameter estimates because the bootstrap distributions are centered around statistics θˆ calculated from the data (e.g., x or regression ˆ rather than the unknown populaslope β) tion values (e.g., µ or β). Drawing thousands of boot-strap observations from the original data is not like drawing observations from the underlying population, it does not create new data.

Instead, the bootstrap sampling is useful for quantifying the behavior of a parameter estimate, such as its standard error, bias, or calculating confidence intervals. Exceptions do exist where bootstrap averages are useful for estimation, such as random forests (2). These examples are beyond the scope of this article, except that we give a toy example to illustrate the mechanism. Consider the case of simple linear regression, and suppose that a strong linear relationship exists between y and x. However, instead of using linear regression, one uses a step function the data are split into eight equal-size groups based on x, and the y values in each group are averaged to obtain the altitude for the step. Applying the same procedure to bootstrap samples randomizes the location of the step edges, and averaging across the bootstrap samples smooths the edges of the steps. This method is shown in Fig. 3. A similar effect holds in random forests, using bootstrap averaging of tree models, which fit higher-dimensional data using multivariate analogs of step functions. 2.2 Other Population Estimates Other estimates of the population may be used. For example, if there was reason to assume that the arsenic data followed a gamma distribution, we could estimate parameters for the gamma distribution, then draw

step function bootstrap average

Figure 3. Step function defined by eight equalsize groups, and average across bootstrap samples of step functions.

3

4

BOOTSTRAP

samples from a gamma distribution with those estimated parameters. In other cases, we may believe that the underlying population is continuous; rather than draw from the discrete empirical distribution, we may instead draw samples from a density estimated from the data, say a kernel density estimate. We return to this point in the section entitled ‘‘Bootstrap distributions are too narrow.’’ 2.3 Other Sampling Procedures When the original data were not obtained using an i.i.d. sample, the bootstrap sampling should reflect the actual data collection. For example, in stratified sampling applications, the bootstrap sampling should be stratified. If the original data are dependent, the bootstrap sampling should reflect the dependence; this information may not be straightforward. Some cases exist where the bootstrap sampling should differ from the actual sampling procedure, including: • regression (see ‘‘Examples’’ section) • planning clinical trials (Section 9), (see

‘‘Planning Clinical trials’’ section) • hypothesis testing (see ‘‘Hypothesis test-

ing’’ section), and • small samples (see ‘‘Bootstrap distributions are too narrow’’) 2.4 Other Statistics The bootstrap procedure may be used with a wide variety of statistics mean, median, trimmed mean, regression coefficients, hazard ratio, x-intercept in a regression, and others—using the same procedure. It does not require problem-specific analytical calculations. This factor is a major advantage of the bootstrap. It allows statistical inferences such as confidence intervals to be calculated even for statistics for which there are no easy formulas. It offers hope of reforming statistical practice—away from simple but nonrobust estimators like a sample mean or least-squares regression estimate, in favor of robust alternatives.

3 MONTE CARLO SAMPLING—THE ‘‘SECOND BOOTSTRAP PRINCIPLE’’ The second bootstrap ‘‘principle’’ is that the bootstrap is implemented by random sampling. This aspect is not actually a principle, but an implementation detail. Given that we are drawing i.i.d. samples of size n from the empirical distribution Fˆ n , there are at most nn possible samples. In small samples, we could create all possible bootstrap samples, deterministically. In practice, n is usually too large for that to be feasible, so we use random sampling. Let B be the number of bootstrap samples used (e.g., B = 104 ). The resulting B statistic values represent a random sample of size B with replacement from the theoretical bootstrap distribution that consists of nn values (including ties). In some cases, we can calculate the theoretical bootstrap distribution without simulation. In the arsenic example, parametric bootstrapping from a gamma distribution causes the theoretical bootstrap distribution for the sample mean to be another gamma distribution. In other cases, we can calculate some aspects of the sampling distribution without simulation. In the case of the nonparametric bootstrap when the statistic is the sample mean, the mean and standard deviation of the theoretical bootstrap distribution are x √ and σˆ Fˆ n / n, respectively, where σˆ F2ˆ = n−1 n



(xi − x)2 .

Note that this differs from the usual sample standard deviation in using a divisor of n instead of n − 1. We return to this point in the section ‘‘Bootstrap distribution are too narrow.’’ The use of Monte Carlo sampling adds additional unwanted variability, which may be reduced by increasing the value of B. We discuss how large B should be in the section ‘‘How many bootstrap samples are needed.’’ 4 BIAS AND STANDARD ERROR Let θ = θ (F) be a parameter of a population, such as the mean, or difference in regression

BOOTSTRAP

coefficients between subpopulations. Let θˆ be the corresponding estimate from the data, θˆ ∗ be the estimate from a bootstrap sample, ∗

θˆ = B−1

B 

ˆ EFˆ (θˆ ∗ ) − θ (F)

(2)

The Monte Carlo version in Equation (1) substitutes the sample average of bootstrap statistics for the expected value.

be average of B bootstrap estimates, and s2θˆ ∗ = (B − 1)−1

for population F. Substituting Fˆ for the unknown F in both terms yields the theoretical bootstrap analog

θˆb∗

b=1

5

B  ∗ (θˆb∗ − θˆ )2

5

EXAMPLES

b=1

the sample standard deviation of the bootstrap estimates. Some bootstrap calculations require that θˆ be a functional statistic, which is one that depends on the data only through the empirical distribution, not on n. A mean is a functional statistic, whereas the usual sample standard deviation s with divisor n − 1 is not—repeating each observation twice gives the same empirical distribution but a different s. The bootstrap bias estimate for a functional statistic is ∗

θˆ − θˆ

(1)

Relative risk bootstrap CI t CI

0.0

Proportion in high risk group

0.6 0.4 0.2

An outlier at 10.1 is omitted. Other large observations are indicated below.

0.020

0.030

A major study of the association between blood pressure and cardiovascular disease found that 55 of 3338 men with high blood pressure died of cardiovascular disease during the study period, compared with 21 out of 2676 patients with low blood pressure. The estimated relative risk is θˆ = pˆ 1 /pˆ 2 = 0.0165/0.0078 = 2.12. To bootstrap this, we draw samples of size n1 = 3338 with replacement from the first group, independently draw samples of size n2 = 2676 from the second group, and calculate the relative risk θˆ ∗ . In addition, we record the individual proportions pˆ ∗1 and pˆ ∗2 . The bootstrap distribution for relative risk is shown in the left panel of Fig. 4. It is

Observed Mean

0.0

Density

5.1 Relative Risk

0.010

0.8

Note how this example relates to the plugˆ − in principle. The bias of a statistic is E(θ) θ , which for a functional statistic may be ˆ − θ (F), the expected value expanded as EF (θ) ˆ of θ when sampling from F minus the value

In this section, we consider some examples, with a particular eye to standard error, bias, and normality of the sampling distribution.

1

2

3 4 5 Relative Risk

6

7

0.0

0.005

0.010

Proportion in low risk group Slope = Relative Risk

Figure 4. Bootstrap distribution for relative risk.

0.015

6

BOOTSTRAP

up and down 1.96 residual standard errors from the central point (the original data) to the circled points; the endpoints of the interval are the slopes of the lines from the origin to the circled points. A t interval would not be appropriate in the example because of the bias and skewness. In practice, one would normally do a t interval on a transformed statistic, for example, log of relative risk, or log-odds-ratio log(pˆ 1 (1 − pˆ 2 )/((1 − pˆ 1 )p2 )). Figure 5 shows a normal quantile plot for the bootstrap distribution of the log of relative risk. The distribution is much less skewed than is relative risk, but it is still noticeably skewed. Even with a log transformation, a t interval would only be adequate for work where accuracy is not required. We discuss confidence intervals even more in the section entitled ‘‘Bootstrap confidence intervals.’’ 5.2 Linear Regression The next examples for linear regression are based on a dataset from a large pharmaceutical company. The response variable is a pharmacokinetic parameter of interest, and candidate predictors are weight, sex, age, and dose (3 levels—200, 400, and 800). In all, 300 observations, are provided, one per subject. Our primary interest in this dataset will be to use the bootstrap to investigate the behavior of stepwise regression; however, first we consider some other issues.

1.5 1.0 0.5 0.0

Log Relative Risk

2.0

highly skewed, with a long right tail caused by a divisor relatively close to zero. The standard error, from a sample of 104 observations, is 0.6188. The theoretical bootstrap standard n n error is undefined because some of the n1 1 n2 2 ∗ bootstrap samples have θˆ undefined because the denominator pˆ ∗ is zero; this aspect is not important in practice. The average of the bootstrap replicates is larger than the original relative risk, which indicates bias. The estimated bias is 2.205 − 2.100 = 0.106, which is 0.17 standard errors. Although the bias does not seen large in the figure, this amount of bias can have a huge impact on inferences; a rough calculation suggests that the actual noncoverage of one side of a two-sided 95% confidence interval would be (0.17 + 1.96) = 0.0367 rather than 0.025, or 47% too large. The right panel of Fig. 4 shows the joint bootstrap distribution of pˆ ∗1 and pˆ ∗2 . Each point corresponds to one bootstrap sample, and the relative risk is the slope of a line between the origin and the point. The original data is at the intersection of horizontal and vertical lines. The solid diagonal lines exclude 2.5% of the bootstrap observations on each side; the slopes are the endpoint of a 95% bootstrap percentile confidence interval. The bottom and top dashed diagonal lines are the endpoints of a t interval with standard error obtained using the usual delta method. This interval corresponds to calculating the standard error of residuals above and below the central line (the line with slope θˆ ), going

−4

−2

0

2

Quantiles of Standard Normal

4

Figure 5. Bootstrap distribution for log of relative risk.

BOOTSTRAP

A standard linear regression using main effects gives:

(Intercept) wgt sex age dose

Value

Std. Error

t value

32.0819 0.2394 −7.2192 −0.1507 0.0003

4.2053 0.0372 1.2306 0.0367 0.0018

7.6290 6.4353 −5.8666 −4.1120 0.1695

Pr (>|t|) 0.0000 0.0000 0.0000 0.0000 0.8653

5.2.1 Prediction Intervals and NonNormality. The right panel also hints at the difference between a confidence interval (for mean response given covariates) and a prediction interval (for a new observation). With large n, the regression lines show little variation, but the variation of an individual point above and below the (true) line remains constant regardless of n. Hence as n increases, confidence intervals become narrow but prediction intervals do not. This example is reflected in the standard formulae for confidence intervals:  (3) yˆ ± tα 1/n + (x − x)2 /Sxx and prediction intervals in the simple linear regression case:  yˆ ± tα 1 + 1/n + (x − x)2 /Sxx (4)

60

As n → ∞ the terms inside the square root decrease to zero for a confidence interval but original bootstrap

PK Parameter 30 40

40

10

20

30 10

20

PK Parameter

covariate (age) to the model reduces residual variance. Note that the y values here are the actual data; they are not adjusted for differences between the actual sex and age and the base case. Adjusting the male observations would raise these valves. Adjusting both would make the apparent residual variation in the plot smaller to match the residual variance from the regression.

50

original bootstrap

50

60

The left panel of Fig. 6 contains a scatterplot of clearance versus weight, for the 25 females who received dose = 400, as well as regression lines from 30 bootstrap samples. This graph is useful for giving a rough idea of variability. A bootstrap percentile confidence interval for mean clearance given weight would be the range of the middle 95% of heights of regression lines at a given weight. The right panel shows all 300 observations and predictions for the clearance/weight relationship using (1) all 300 observations, (2) the main-effects model, and (3) predictions for the ‘‘base case’’, for females who received dose = 400. In effect, this graph uses the full dataset to improve predictions for a subset, ‘‘borrowing strength’’. Much less variability is observed than in the left panel, primarily because of the larger sample size and also because the addition of an important

40

50

60

70 Weight

80

90

100

7

40

50

60

70

80

90

100

Weight

Figure 6. Bootstrap regression lines. Left panel: 25 females receiving dose = 400. Right panel: all observations, predictions for females receiving dose = 400.

8

BOOTSTRAP

approach 1 for a prediction interval, the prediction interval approaches yˆ ± zα . Now, suppose that residuals are not normally distributed. Asymptotically and for reasonably large n the confidence intervals are approximately correct, but prediction intervals are not—the interval yˆ ± zα is only correct for normally distributed data. Prediction intervals should approach (ˆy ± Fˆ ε−1 (α/2), yˆ ± Fˆ ε−1 (1 − α/2)) as n → ∞, where Fˆ is the estimated residual distribution. In other words, no central limit theorem exists for prediction intervals. The outcome for a new observation depends primarily on a single random value, not an average across a large sample. Equation (4) should only be used after confirming that the residual distribution is approximately normal. And, in the opinion of this author, Equation (4) should not be taught in introductory statistics to students ill-equipped to understand that it should only be used if residuals are normally distributed. A bootstrap approach that takes into account both the shape of the residual distribution and the variability in regression lines is outlined below in the subsection entited ‘‘Prediction Intervals.’’ 5.2.2 Stepwise Regression. Now, consider the case of stepwise regression. We consider models that range from the intercept-only model to a full second-order model that includes all main effects, all interactions, and quadratic functions of dose, age, and weight. We use forward and backward stepwise regression, with terms added or subtracted to minimize the CP statistic, using the step function of S-PLUS (insightful Corp., Seattle, WA). The resulting coefficients and inferences are:

Value

Std. Error

t value

(Intercept) 12.8035 14.1188 0.9068 wgt 0.6278 0.1689 3.7181 sex 9.2008 7.1634 1.2844 age −0.6583 0.2389 −2.7553 I(ageˆ2) 0.0052 0.0024 2.1670 wgt: sex −0.2077 0.0910 −2.2814

Pr (>|t|) 0.3637 0.0002 0.1980 0.0055 0.0294 0.0218

The sex coefficient is retained even though it has small t value because main effects are included before interactions. We use the bootstrap here to check model stability, obtain standard errors, and check for bias. 6 MODEL STABILITY The stepwise procedure selected a six-term model. We may use the bootstrap to check the stability of the procedure under random sampling (does it consistently select the same model, or is there substantial variation) and to observe which terms are consistently included. Here, we create bootstrap samples by resampling subjects— whole rows of the data—with replacement. We sample whole rows instead of individual values to preserve covariances between variables. In 1000 bootstrap samples, only 95 resulted in the same model as for the original data; on average, 3.2 terms differed between the original model and the bootstrap models. The original model has six terms; the bootstrap models ranged from 4 to 12, with an average of 7.9, which is 1.9 more than for the original data. This result suggests that stepwise regression tends to select more terms for random data than for the corresponding population, which in turn suggests that the original six-term model may also be overfitted. Figure 7 shows the bootstrap distributions for two coefficients: dose and sex. The dose coefficient is usually zero, although it may be positive or negative. This graph suggests that dose is not very important in determining clearance. The sex coefficient is bimodal, with the modes on opposite sides of zero. It turns out that the sex coefficient is usually negative when the weight–sex interaction is included, otherwise it is positive. Overall, the bootstrap suggests that the original model is not very stable. For comparison, repeating the experiment with a more stringent criteria for variable inclusion—a modified Cp statistic with double the penalty—results in a more stable model. The original model has the same six

BOOTSTRAP

Density

0.03

0.04

Observed Mean

0

0.0

0.01

0.02

40 20

Density

60

Observed Mean

9

−0.05

0.0 Coef.dose

0.05

−40

−20

0 Coef.sex

20

40

Figure 7. Bootstrap distribution for dose and sex coefficients in stepwise regression.

terms. In all, 154 bootstrap samples yielded the same model, and on average the number of different terms was 2.15. The average number of terms was 5.93, which is slightly less than for the original data; this number suggests that stepwise regression may now be slightly underfitting. 6.0.2.1 Standard Errors. At the end of the stepwise procedure, the table of coefficients, standard errors, and t values was calculated, ignoring the variable selection process. In particular, the standard errors are calculated under the usual regression assumptions, and assuming that the model were fixed from the outset. Call these nominal standard errors. In each bootstrap sample, we recorded the coefficients and the nominal standard errors. For the main effects the bootstrap standard errors (standard deviation of bootstrap coefficients) and average of the nominal standard errors are:

Intercept wgt sex age dose

boot SE

avg.nominal SE

27.9008 0.5122 9.9715 0.3464 0.0229

14.0734 0.2022 5.4250 0.2137 0.0091

The bootstrap standard errors are much larger than the nominal standard errors.

That information is not surprising— the bootstrap standard errors reflect additional variability because of model selection, such as the bimodal distribution for the sex coefficient. This is not to say that one should use the bootstrap standard errors here. At the end of the stepwise variable selection process, it is appropriate to condition on the model and do inferences accordingly. For example, a confidence interval for the sex coefficient should be conditional on the weight–sex interaction being included in the model. But it does suggest that the nominal standard errors may be optimistic. Indeed they are, even conditional on the model terms, because the residual standard error is biased. 6.0.2.2 Bias. Figure 8 shows bootstrap distributions for R2 (unadjusted) and residual standard deviation. Both show very large bias. The bias is not surprising— optimizing generally gives biased results. Consider ordinary linear regression—unadjusted R2 biased. If it were calculated using the true βs ˆ it would not be biased. instead of estimated βs Optimizing βˆ to minimize residual squared error (and maximize R2 ) makes unadjusted R2 biased. In classic linear regression, with the model selected in advance, we commonly use adjusted R2 to counteract the bias. Similarly, we

10

BOOTSTRAP

Observed Mean

0.8 0.6

Density

0

0.0

0.2

2

0.4

4

Density

6

1.0

8

1.2

Observed Mean

0.25

0.35 Rsquare

0.45

6.5

7.0 7.5 Sigma

8.0

Figure 8. Bootstrap distributions for R2 and residual standard deviation in stepwise regression.

use residual variance calculated using a divisor of (n − p − 1) instead of n, where p is the number of terms in the model. But in this case, it is not only the values of the coefficients that are optimized, but which terms are included in the model. This result is not reflected in the usual formulae. As a result, the residual standard error obtained from the stepwise procedure is biased downward, even using a divisor of (n − p − 1). 6.0.3 Bootstrapping Rows or Residuals. Two basic ways exist to bootstrap linear regression models—to resample rows, or residuals (3). To resample
residuals, we fit the initial model yˆ i = βˆ0 + βˆj xij , calculate the residuals ri = yi − yˆ i , then create new bootstrap samples as y∗i = yˆ i + r∗i

(5)

for i = 1, . . . , n, where r∗i is sampled with replacement from the observed residuals {r1 , . . . , rn }. We keep the original x and yˆ values fixed to create new bootstrap y* values. Resampling rows corresponds to a random effects sampling design—in which x and y are both obtained by random sampling from a joint population. Resampling residuals corresponds to a fixed effects model, in which the xs are fixed by the experimental design and ys are obtained conditional on the xs.

So at first glance, it would seem appropriate to resample rows when the original data collection has random xs. However, in classic statistics we commonly use inferences derived using the fixed effects model, even when the xs are actually random. We do inferences conditional on the observed x values. Similarly, in bootstrapping we may resample residuals even when the xs were originally random. In practice, the difference matters most when there are factors with rare levels or interactions of factors with rare combinations. If resampling rows, it is possible that a bootstrap sample may have none of the level or combination, in which case the corresponding term cannot be estimated and the software may give an error. Or, what is worse, there may be one or two rows with the rare level, enough so the software would not crash, but instead quietly give garbage answers. That are imprecise because they are based on few observations. Hence, with factors with rare levels or small samples more generally, it may be preferable to resample residuals. Resampling residuals implicitly assumes that the residual distribution is the same for every x and that there is no heteroskedasticity. A variation on resampling residuals that allows heteroskedasticity is the wild bootstrap, which in its simplest form adds either plus or minus the original residual ri to each

BOOTSTRAP

fitted value,

y∗i = yˆ i ± ri

(6)

with equal probabilities. Hence the expected value of y∗i is yˆ i , and the standard deviation is proportional to ri . For more discussion see Ref. 3. Other variations on resampling residuals exist, such as resampling studentized residuals or weighted error resampling for nonconstant variance (3). 6.0.4 Prediction Intervals. The idea of resampling residuals provides a way to obtain more-accurate prediction intervals. To capture both variation in the estimated regression line and residual variation, we may resample both. Variation in the regression line may be obtained by resampling either ˆ∗ residuals or rows to generate random
β val∗ ˆ ˆ ues and corresponding yˆ = β0 + βj x0j , for predictions at x0 . Independently, we draw random residuals r* and add them to the yˆ *. After repeating this many times, the range of the middle 95% of the (ˆy* + r*) values gives a prediction interval. For more discussion and alternatives see Ref. 3. 6.1 Logistic Regression

observation to the prediction from another yields values anywhere between −1 and 2. Instead, we keep the xs fixed and generate y values from the estimated conditional distribution given x. Let pˆ i be the predicted probability that yi = 1 given xi . Then y∗i



1 with probability pˆ 0 with probability 1 − pˆ

(7)

The kyphosis dataset (4) contains observations on 81 children who had corrective spinal surgery, on four variables: Kyphosis (a factor indicating whether a postoperative deformity is present), Age (in months), Number (of vertebrae involved in the operation), and Start (beginning of the range of vertebrae involved). A logistic regression using main effects gives coefficients:

(Intercept) Age Start Number

Value

Std. Error

t value

−2.03693352

1.449574526 0.006446256 0.067698863 0.224860819

−1.405194

0.01093048 −0.20651005

0.41060119

1.695633 −3.050421

1.826024

1.0 0.8 0.0

0.2

0.4

Kyphosis

original bootstrap

0.6

1.0

which suggest that Start is the most important predictor. The left panel of Fig. 9 shows Kyphosis versus Start, together with predicted curve for the base case with Age = 87 (the median)

0.8 0.6 0.4 0.2 0.0

Kyphosis

In logistic regression, it is straightforward to resample rows of the data, but resampling residuals fails—the y values must be either zero or one, but adding the residual from one

5

10 Start

15

11

5

10 Start

Figure 9. Bootstrap curves for predicted kyphosis, for Age = 87 and Number = 4.

15

12

BOOTSTRAP

Figure 10. Bootstrap distributions for logistic regression coefficients.

and Number = 4 (the median). This graph is a sunflower plot (5,6), in which a flower with k > 2 petals represents k duplicate values. The right panel of of Fig. 9 shows predictions from 20 bootstrap curves. Figure 10 shows the bootstrap distributions for the four regression coefficients. All distributions are substantially non-normal. It would not be appropriate to use classic normal-based inferences. Indeed, the printout of regression coefficients above, from a standard statistical package (S-PLUS) includes t values but omits p values. Yet, it would be tempting for a package user to interpret the t coefficients as developing from a t distribution; the bootstrap demonstrates that it

would be improper. The distributions are so non-normal as to make the use of standard errors doubtful. The numerical bootstrap results are:

(Intercept) Age Start Number

Observed

Mean

Bias

−2.03693

−2.41216

−0.375224

SE

1.737216 0.01093 0.01276 0.001827 0.008017 −0.20651 −0.22991 −0.023405 0.084246 0.41060 0.48335 0.072748 0.274049

The bootstrap standard errors are larger than the classic (asymptotic) standard errors by 20–24%. The distributions are also ex-

BOOTSTRAP

tremely biased, with absolute bias estimates that range from 0.22 to 0.28 standard errors. These results are for the conditional distribution bootstrap, which is a kind of parametric bootstrap. Repeating the analysis with the nonparametric bootstrap (resampling observations) yields bootstrap distributions that are even longer-tailed, with larger biases and standard errors. This result reinforces the conclusion that classic normal-based inferences are not appropriate here. 7 ACCURACY OF BOOTSTRAP DISTRIBUTIONS How accurate is the bootstrap? This inquiry entails two questions: • How accurate is the theoretical boot-

strap? • How accurately does the Monte Carlo

implementation approximate the theoretical bootstrap? We begin this section with a series of pictures intended to illustrate both questions. We conclude this section with a discussion of cases where the theoretical bootstrap is not accurate, and remedies. In the section on ‘‘How many bootstrap samples are needed,’’ we return to the question of Monte Carlo accuracy. The treatment in this section is mostly not rigorous. Much literature examines the first question rigorously and asymptotically; we reference some of that work in other sections, particularly in the Section. About confidence intervals, and also refer the reader to (7,8) and some sections of (Ref. 3), as well as the references therein. Figure 11 shows a population, and five samples of size 50 from the population in the left column. The middle column shows the sampling distribution for the mean, and bootstrap distributions from each sample based on B = 1000 bootstrap samples. Each bootstrap distribution is centered at the statistic (x) from the corresponding sample rather than being centered at the population mean µ. The spreads and shapes of the bootstrap distributions vary a bit. This example informs what the bootstrap distributions may be used for. The bootstrap

13

does not provide a better estimate of the population parameter µ, because no matter how many bootstrap samples are used, they are centered at x (plus random variation), not µ. On the other hand, the bootstrap distributions are useful for estimating the spread and shape of the sampling distribution. The right column shows five more bootstrap distributions from the first sample; the first four using B = 1000 resamples and the final using B = 104 . These examples illustrate the Monte Carlo variation in the bootstrap. This variation is much smaller than the variation caused by different original samples. For many uses, such as quick and dirty estimation of standard errors or approximate confidence intervals, B − 1000 resamples is adequate. However, there is noticeable variability, particularly in the tails of the bootstrap distributions, so when accuracy matters, B = 104 or more samples should be used. Note the difference between using B = 1000 and B = 104 bootstrap samples. These examples correspond to drawing samples of size 1000 or 104 observations, with replacement, from the theoretical bootstrap distribution. Using more samples reduces random Monte Carlo variation, but it does not fundamentally change the bootstrap distribution— it still has the same approximate center, spread, and shape. Figure 12 is similar to Fig. 11, but for a smaller sample size, n = 9 (and a different population). As before, the bootstrap distributions are centered at the corresponding sample means, but now the spreads and shapes of the bootstrap distributions vary substantially because the spreads and shapes of the samples also vary substantially. As before, the Monte Carlo variation is small, and it may be reduced using B = 104 or more samples. It is useful to compare the bootstrap distributions to classic statistical inferences. With √ classic t intervals of the form x ± tα/2 s/ n, the confidence interval width varies substantially in small samples as the sample standard deviation s varies. Similarly, the classic √ standard error s/ n varies. The bootstrap is no different in this regard—bootstrap standard errors and widths of confidence intervals for the mean are proportional to s.

14

BOOTSTRAP Population mean = mu Sample mean = x Sampling distribution

Population

−3

mu

0

3

6

0

mu

3

x

0

x

3

3 0

x

Bootstrap distribution 3 for Sample 1

0

3

x

x

3

x

0

0

x

3

0

Bootstrap distribution 5 for Sample 1

x

3 0

3

x

B=10^4

Bootstrap distribution for Sample 5

Sample 5

3

x

3 0

x

Bootstrap distribution 4 for Sample 1

B=1000

Bootstrap distribution for Sample 4

3

x

B=1000

3 0

x

Sample 4

0

3 0

Bootstrap distribution for Sample 3

0

3

x

B=1000

Sample 3

0

Bootstrap distribution 2 for Sample 1

Bootstrap distribution for Sample 2

Sample 2

0

B=1000

Bootstrap distribution for Sample 1

Sample 1

0

3

Bootstrap distribution 6 for Sample 1

x

3

Figure 11. Bootstrap distribution for mean, n = 50. The left column shows the population and five samples. The middle column shows the sampling distribution and bootstrap distributions from each sample. The right column shows five more bootstrap distributions from the first sample, with B = 1000 or B = 104 .

Where the bootstrap does differ from classic inferences is how it handles skewness. The bootstrap percentile interval, and other bootstrap confidence intervals discussed below in the next Section, are in general asymmetrical with asymmetry depending on the sample. They estimate the population skewness from the sample skewness. In contrast, classic t intervals assume that the population is zero. In Bayesian terms, the bootstrap uses

a noninformative prior for skewness, whereas classic procedures use a prior with 100% of its mass on skewness = 0. Which method is preferred? In large samples, it is clearly the bootstrap. In small samples, the classic procedure may be preferred. If the sample size is small, then skewness cannot be estimated accurately from the sample, and it may be better to assume skewness

BOOTSTRAP Population

15

Population mean = mu Sample mean = x Sampling distribution

−3

mu

3

-3

mu

3

Bootstrap distribution for Sample 1

Sample 1

−3

x

3

Bootstrap distribution 2 for Sample 1

−3

x

Sample 2

−3

3

Bootstrap distribution for Sample 2

x

3

−3

x

3

Bootstrap distribution for Sample 3

Sample 3

−3

x

3

x

3

−3

x

3

x

3

−3

Bootstrap distribution 3 for Sample 1

−3

−3

x

3

−3

x

3

−3

3

B=1000

x

3

B=1000

x

3

B=10^4

Bootstrap distribution 6 for Sample 1

x

3

B=1000

Bootstrap distribution 5 for Sample 1

Bootstrap distribution for Sample 5

Sample 5

−3

−3

x

Bootstrap distribution 4 for Sample 1

Bootstrap distribution for Sample 4

−3

−3

B=1000

x

3

Figure 12. Bootstrap distribution for mean, n = 9. The left column shows the population and five samples. The middle column shows the sampling distribution and bootstrap distributions from each sample. The right column shows five more bootstrap distributions from the first sample, with B = 1000 or B = 104 .

= 0 despite the bias, rather than to use an estimate that has high variability. Now turn to Fig. 13, where the statistic is the sample median. Here, the bootstrap distributions are poor approximations of the sampling distribution. In contrast, the sampling distribution is continuous, but the bootstrap distributions are discrete, with the only possible values being values in the original

sample (here n is odd). The bootstrap distributions are very sensitive to the sizes of gaps among the observations near the center of the sample. The bootstrap tends not to work well for statistics such as the median or other quantiles that depend heavily on a small number of observations out of a larger sample.

16

BOOTSTRAP

−4

M

10

−4

M

10

Bootstrap distribution for Sample 1

Sample 1

−4

M

10

−4

M

10

M

10

−4

M

10

M

10

−4

M

10

M

10

−4

M

Sample 5

−4

M

10

M

10

−4

M

M

10

10

Bootstrap distribution 4 for Sample 1

−4

M

10

Bootstrap distribution 5 for Sample 1

−4

M

Bootstrap distribution for Sample 5

−4

10

Bootstrap distribution 3 for Sample 1

Bootstrap distribution for Sample 4

Sample 4

−4

−4

Bootstrap distribution for Sample 3

Sample 3

−4

Bootstrap distribution 2 for Sample 1

Bootstrap distribution for Sample 2

Sample 2

−4

Population median = M Sample median = m

Sampling distribution

Population

10

Bootstrap distribution 6 for Sample 1

−4

M

10

Figure 13. Bootstrap distribution for median, n = 15. The left column shows the population and five samples. The middle column shows the sampling distribution and bootstrap distributions from each sample. The right column shows five more bootstrap distributions from the first sample.

7.1 Systematic Errors in Bootstrap Distributions

Second, in many applications there is a relationship between the statistic and its standard error [‘‘acceleration’’ in the termi-

We note three ways that bootstrap distributions are systematically different than sampling distributions. First, as noted above, bootstrap distributions are centered at the statistic θˆ (plus bias) rather than at the parameter θ (plus bias).

nology of Efron (9)]. For example, the standard error of a binomial proportion  p(1 ˆ − p)/n ˆ depends on p. ˆ Similarly, when sampling from a gamma distribution, the variance of the sample mean depends on the

BOOTSTRAP

underlying mean. More generally when sampling the mean from positively skewed distributions, samples with larger means tend to give larger standard errors. When is acceleration occurs, the bootstrap standard error reflects the standard error ˆ not the true standard corresponding to θ, deviation of the sampling distribution (corresponding to θ ). Suppose the relationship is positive; when θˆ < θ , it tends to be true that the estimated standard error is also less than the true standard deviation of the sampling distribution, and confidence intervals tend to be too short. This is true for t intervals, whether using a formula or bootstrap standard error, and also to a lesser extent for bootstrap percentile intervals. The more accurate intervals discussed in the next section correct for acceleration. The third systematic error is that bootstrap distributions tend to be too narrow. 7.2 Bootstrap Distributions Are Too Narrow In small samples, bootstrap distributions tend to be too narrow. Consider the case of a sample mean from a single population; in this case, the
standard √ theoretical bootstrap error is σˆ / n where σˆ 2 = (1/n) (xi − x)2 . In contrast to the usual sample standard deviation s, this method uses a divisor of n rather than n − 1. The reason the distributions are too narrow relates to the plug-in principle; when plugging in the empirical distribution Fˆ n for use as the population, we are drawing samples from a population with standard deviation σˆ . The result is that bootstrap  standard errors are too small, by√a factor 1 − 1/n relative to the usual s/ n; the errors are about 5% too small when n = 10, about 1% too small when n = 50, and so on. In stratified bootstrap situations, the bias depends on the strata sizes rather than on the total sample size. Some easy remedies can be implemented. The first is to draw bootstrap samples of size n − 1, with replacement from the data of size n. The second, bootknife sampling (10), is a combination of jackknife and bootstrap sampling—first create a jackknife sample by omitting an observation, then draw a bootstrap sample of size n with replacement from

17

the n − 1 remaining observations. The omission can be random or systematic. A third remedy is the smoothed bootstrap. Instead of drawing random samples from the discrete distribution Fˆ n , we draw
from a kernel density estimate Fˆ h (x) = n−1 ((x − xi )/h), where  is the standard normal density (other densities may be used). The original motivation (11,12) was to draw samples from continuous distributions, but it can also be used to correct for narrowness (10). The variance of an observation from Fˆ h is σˆ 2 + h2 . Using h2 = s2 /n results in the theoretical bootstrap standard error matching the usual formula (10). For multidimensional data x the kernel covariance can be 1/n times the empirical covariance matrix. For non-normal data, it may be appropriate to smooth on a transformed scale; for example, for failure time data, take a log transform of the failure times, add normal noise, then transform it back to the original scale. 8

BOOTSTRAP CONFIDENCE INTERVALS

Several bootstrap confidence intervals have been proposed in the literature. Reviews of confidence intervals are found in Refs. 13–15. Here we focus on five examples: t intervals with either bootstrap or formula standard errors, bootstrap percentile intervals (16), bootstrap t intervals (17) bootstrap BCa intervals (9), and bootstrap tilting (18–21). Note that ‘‘t intervals with bootstrap standard errors’’ and ‘‘bootstrap t intervals’’ are different. Percentile and t intervals are quick-anddirty intervals that are relatively simple to compute, but they are not very accurate except for very large samples. They do not properly account for factors such as bias, acceleration, or transformations. They are first-order correct under fairly general circumstances (basically, for asymptoticallynormal statistics) the one-sided noncoverage levels √ for nominal (1√− α) intervals are α/2 + O(1/ n). The O(1/ n) errors decrease to zero very slowly. The BCa, tilting, and bootstrap t intervals are second-order correct, with coverage errors O(1/n).

18

BOOTSTRAP

The percentile, BCa, and tilting intervals are transformation invariant—they give equivalent results for different transformations of a statistic, for example, hazard ratio and log-hazard ratio, or relative risk and log relative risk. t intervals are not transformation invariant. Bootstrap t intervals are less sensitive to transformations than are t intervals; the use of different (smooth) transformations has coverage√effects of order O(1/n), compared with O(1/ n) for t intervals. Our focus is on one-sided errors because few practical situations are truly two-sided. A nominal 95% interval that misses 2% of the time on the left and 3% of the time on the right should not be considered satisfactory. It is a biased confidence interval—both endpoints are too low, so it gives a biased impressions about where the true parameter may be. The appropriate way to add one-sided coverage errors is by adding their absolute values, so the 2%/3% interval has a total coverage error of |2 − 2.5| + |3 − 2.5| = 1%, not 0%. 8.1 t Intervals A t interval is of the form θˆ ± tα/2,ν sθˆ

(8)

where sθˆ is a standard error computed using a formula or using the bootstrap, and ν is degrees of freedom, which is typically set to n − 1 (although other values would be better for non-normal distributions). The bootstrap standard error may be computed using the techniques in the section ‘‘Bootstrap distributions are too narrow’’— bootknife, sampling with reduced size, or smoothed bootstrap. This results in slightly wide intervals that are usually more accurate in practice. These techniques have an O(1/n) effect on one-sided coverage errors, which is unimportant for large samples but is important in small samples. For example, for a sample of independent identically distributed observations from a normal distribution, a nominal 95% t interval for the mean using a bootstrap standard error without these corrections would have one-sided coverages errors:

n 10 20 40 100

Non-coverage

Error

0.0302 0.0277 0.0264 0.0256

0.0052 0.0027 0.0014 0.0006

8.2 Percentile Intervals In its simplest form, a 95% bootstrap percentile interval is the range of the middle 95% of a bootstrap distribution. More formally, bootstrap percentile intervals are of the form ˆ −1 (1 − α/2)) ˆ −1 (α/2), G (G

(9)

ˆ is the estimated bootstrap distribuwhere G tion of θˆ ∗ . Variations are possible that improve finitesample performance. These examples have received little attention in the bootstrap literature, which tends to focus on asymptotic properties. In particular, the simple bootstrap percentile intervals tend to be too narrow, and the variations give wider intervals with better coverage. First, the bootknife or other techniques cited previously may be used. Second, the percentiles may be adjusted. In a simple situation, like the sample mean from a symmetric distribution, the interval is similar to the t interval in Equation (8) but using quantiles of a normal distribution rather than t distribution, zα/2 rather than tα/2,n−1 . As a result, the interval tends to be too narrow. A correction is to adjust the quantiles based on the difference between a normal and t distribution, ˆ −1 (1 − α  /2)) ˆ −1 (α  /2), G (G

(10)

−1 (α/2) and where  is where −1 (α  /2) = Ft,n−1 the standard normal distribution and F t,n−1 is the t distribution function with n − 1 degrees of freedom. This method gives wider intervals. Extensive simulations (not included here) show that this gives smaller coverage errors in practice, in a wide variety of applications. The effect on coverage errors is O(1/n), which is the same order as the

BOOTSTRAP

bootknife adjustment, but the magnitude of the effect is larger; for example, the errors caused by using z rather than t quantiles in a standard l interval for a normal population are as follows:

n

Non-coverage

Error

0.0408 0.0324 0.0286 0.0264

0.0158 0.0074 0.0036 0.0014

10 20 40 100

A third variation relates to how quantiles are calculated for a finite number B of bootstrap samples. Hyndman and Fan (22) give a family of definitions of quantiles for finite samples, governed by a parameter 0 ≤ ∗ is the (b δ ≤ 1. The bth order statistic θˆ(b) − δ)/(B + 1 − 2δ) quantile of the bootstrap distribution for b = 1, . . . , B. Linear interpolation between adjacent bootstrap statistics is used if the desired quantile is not of the form (b − δ)/(B + 1 − 2δ) for some integer b. For bootstrap confidence intervals δ = 0 is preferred, as other choices result in lower coverage probability. The effect on coverage errors is O(1/B).

The standard error may be calculated either by formula or bootstrap sampling; in the latter case, calculating each sθˆ ∗ requires a second level of bootstrap sampling, with secondlevel bootstrap samples drawn from each first-level bootstrap sample. Figure 14 shows the bootstrap distribution for the t statistic for means arsenic concentration, where t is the ordinary t statistic √ (x − µ)/(s/ n). In contrast to Fig. 2, in which the bootstrap distribution for the mean is positively skewed, the distribution for the t statistic is negatively skewed. The reason is that there is positive correlation between x* and s*, as seen observed in the right panel of Fig. 14, so that a negative numerator in Equation (12) tends to occur with a small denominator. The bootstrap t interval is based on the identity θˆ − θ < G−1 t (1 − α/2)) = 1 − α sθˆ (13) where Gt is the sampling distribution of t [Equation (11)]. Assuming that t* [Equation (12)] has approximately the same distribution as t, we substitute quantiles of the bootstrap distribution for t*; then solving for θ yields the bootstrap t interval P(G−1 t (α/2)
θ) borrows this idea. The idea behind bootstrap tilting is to create a one-parameter family of populations that includes the empirical distribution function to find the member of that family that has 2.5% (or 97.5%) of the bootstrap distribution exceeding the observed value. Let the left (right) endpoint of the interval be the parameter of interest calculated from that population. The family is restricted to have support on the empirical data with varying probabilities on the observations. For example, given i.i.d. observations (x1 , . . . , xn ) when the parameter of interest is the population mean, one suitable family is the exponential tilting family, which places probabilities Pi = c exp(τ xi )

n  (θˆ(i) − θ (i) )3

(19)

i=1

Davison and Hinkley (3) also give expressions for a in the case of stratified sampling, which includes two sample applications. For the arsenic data, z0 = 0.0438 (based on 100,000 replications) and a = 0.0484. The 95% interval is then the range from the 0.0436 to 0.988 quantiles of the bootstrap distribution. The BCa interval has greater Monte Carlo error than the ordinary percentile interval because Monte Carlo error in estimating z0 propagates into the endpoints and because typically one of the quantiles is farther in the tail than for the percentile interval. For example, here the 98.8% quantile is used instead of the 97.5% quantile. In the best case, that a = z0 = 0, this method requires a bit more than twice as many bootstrap samples as the percentile interval for comparable Monte Carlo accuracy. 8.5 Bootstrap Tilting Intervals In parametric statistics, the left endpoint of a confidence interval for a parameter θ is the value such that the sampling distribution has

21

(20)

on observation i, where τ is a tilting parameter, and c is a normalizing constant (depend
ing on τ ) such that i pi = 1. τ = 0 gives equal probabilities pi = 1/n, which corresponds to the empirical distribution function, and about half of the bootstrap distribution is below the observed x. τ < 0 places higher probabilities on smaller observations; sampling with these probabilities is more likely to give samples with smaller observations, and smaller bootstrap means, so more of the bootstrap distribution is below x. We find the values of τ for which only 2.5% of the bootstrap distribution is above x. The left endpoint of the confidence interval is the mean of the corresponding weighted population n  pi xi . θleft = i=1

Similarly, the right endpoint is n 

pi xi

i=1

when pi is computed using the τ that puts 97.5% of the bootstrap distribution to the right of x.

22

BOOTSTRAP

Another suitable family is the maximum likelihood family, with probability pi =

c 1 − τ (xi − x)

(21)

on observation i. 8.6 Importance Sampling Implementation Conceptually, finding the right value of τ requires trial and error; for any given τ , we calculate p = (p1 , . . . , pn ), draw bootstrap samples with those probabilities, calculate the bootstrap statistics, and calculate the fraction of those statistics that are above θˆ , then repeat with a different τ until the fraction is 2.5%. This method is expensive, and the fraction varies because of random sampling. In practice, we use an importance sampling implementation. Instead of sampling with unequal probabilities, we sample with equal probabilities, then reweight the bootstrap samples by the relative likelihood of the sample under weighted and ordinary bootstrap sampling. The likelihood for a bootstrap sample is l(x1∗ , . . . , xn∗ ) = wi∗

(22)

compared with (1/n)n for ordinary bootstrap sampling. Let wb = wi∗ /(1/n)n = nwi∗ be the relative likelihood for bootstrap sample b. We estimate the probability by Pˆ p (θˆ ∗ > θˆ ) = B−1

B 

wb I(θˆb∗ > θˆ ) = B−1

b=1



 b∈R

 b∈L

Ui (p) = lim ε−1 (θ (p + ε(δi − p)) − θ (p)) (24) →0

where δ i is the vector with 1 in position i and 0 elsewhere. When evaluated at p0 these derivatives are known as the empirical influence function, or infinitesimal jackknife. Four least-favorable families found in the tilting literature are: F1 : pi = c exp(τ Ui (p0 )) F2 : pi = c exp(τ Ui (p))

wb

F4 : pi = c(1 − τ Ui (p))−1

(25)

b∈R

wb + (1.2)



wb .

b∈E

Similar calculations are done for the τ used for the right endpoint; solve 0.025B =

8.6.1 Tilting for Nonlinear Statistics. The procedure can be generalized to statistics other than the mean using a least-favorable single-parameter family, one for which inference within the family is not easier, asymptotically, than for the original problem (18). This method is best done in terms of derivatives. Let F p denote a weighted distribution with probability pi on original data point xi , θ (p) = θ (F p ) be the parameter for the weighted distribution (e.g., weighted mean, or weighted regression coefficient), and p0 = (1/n, . . . , 1/n) correspond to the original equal-probability empirical distribution function. The gradient of θ (p) is

F3 : pi = c(1 − τ Ui (p))−1

(23) where R is the subset of {1, . . . , B} with θˆb∗ > θˆ . In practice we also worry about ties, on ˆ let E be the subset where cases with θˆ ∗ = θ; ˆ We numerically find τ to solve with θˆb∗ = θ. 0.025B =

In any case, after finding τ , the endpoint of the interval is the weighted mean for the empirical distribution with probabilities calculated using τ .

wb + (1/2)



wb

b∈E

where L is the subset of {1, . . . , B} with θˆb∗ < θˆ .

each indexed by a tilting parameter τ , where each c normalizes the probabilities to add to 1. F1 and F2 are well known as ‘‘exponential tilting,’’ and the coincide with Equation (20) if θ is a mean. Similarly F3 and F4 are maximum likelihood tilting and coincide with Equation (21) for a mean. F2 and F4 minimize the backward and forward Kullback-Leibler distances between p and p0 , respectively, subject to pi ≥ 0, pi = 1, and θ (p) = A; varying A results in solutions of the form given in Equation (25). F4 also maximizes the likelihood pi subject to the same constraints.

BOOTSTRAP

As in the case of the sample mean, having selected a family, we find the value of τ for which 2.5% (95%) of the bootstrap distribution is to the right of the observed θˆ ; the left (right) endpoint of the confidence interval is then the parameter calculated for the weighted distribution with probability pi on xi . All four families result in second-order accurate confidence intervals (19), but the finite-sample performance differs, sometimes dramatically for smaller samples. The fixedderivative versions F1 and F3 are easier to work with, but they have inferior statistical properties. They are shorter, have actual coverage probability lower than the nominal confidence, and for sufficiently high nominal confidence levels the actual coverage can decrease as the nominal confidence increases. The maximum likelihood version F4 gives the widest intervals with highest and usually most accurate coverage. The differences in coverage between the four families are O(1/n). The maximum likelihood family has better statistical properties, which produces wider confidence intervals with closer to the desired coverage levels. The exponential tilting family is more convenient numerically. 8.7 Confidence Intervals for Mean Arsenic Concentration Table 1 shows 95% confidence intervals for the mean arsenic concentration example described in the Introduction. The intervals vary dramatically, particularly in the degree of asymmetry. The t intervals are symmetric about x. The bootstrap t interval reaches much farther to the right, and it is much wider. The percentile interval is asymmetrical, longer on the right side, to a lesser extent than other asymmetrical intervals. Whereas it is asymmetrical, it is so haphazardly rather than by design, and the amount of asymmetry is too small for good accuracy. Although preferable to the t intervals, it is not as accurate as the second-order accurate procedures. The t intervals assume that the underlying population is normal, which is not true here. Still, the common practice with a sample size as large as 271 would be to use t intervals anyway. The bootstrap can help answer whether

23

Table 2. Actual Noncoverage of Nominal 95% t Intervals, as Estimated from Second-Order Accurate Intervals

bootstrap t BCa

Left

Right

0.0089 0.0061

0.062 0.052

The actual noncoverage should be 0.025 on each side. A t interval would miss more than twice too often on the right side.

that is reasonable, by giving an idea what the actual noncoverage is for a 95% t interval. Table 2 shows the what nominal coverage levels would be needed for the bootstrap t and BCa intervals to coincide with the actual endpoints of the t interval— in other words, what the bootstrap t and BCa intervals think is the actual noncoverage of the l intervals. The discrepancies are striking. On the left side, the t interval should miss 2.5% of the time; it actually misses only about a third or fourth that often, according to the bootstrap t and BCa intervals. On the right side, it should miss 2.5% of the time, but it actually misses somewhere between 5.2% and 6.2%, according to the BCa and bootstrap t procedures. This finding suggests that the t interval is severely biased, with both endpoints systematically lower than they should be. 8.8 Implications for Other Situations The t intervals are badly biased in the arsenic example. What does this finding imply for other situations? On the one hand, the arsenic data are skewed relative to most data observed in practice. On the other hand, the sample size is large. What can we say about other combinations of sample size and population skewness? For comparison, samples of size 47 from an exponential population are comparable with the arsenic data, in the sense that the sampling distribution for the mean is equally skewed. A quick simulation with 106 samples of exponential data with n = 47 shows that the actual noncoverage of 95% t intervals is 0.0089 on the left and 0.0567 on the right, comparable with the bootstrap estimates above. This example shows that for a distribution with only moderate skewness,

24

BOOTSTRAP

like the exponential distribution, n = 30 is not nearly enough to use t intervals; that even n = 47 results in noncoverage probabilities that are off by factors of about 3 and 2, on the two sides. Reducing the errors in noncoverage to a more reasonable 10% of the desired value, i.e. that the actual one-sided noncoverage probabilities are between 2.25% and 2.75% on each side for a nominal 95% interval would require around n = 7500 for an exponential distribution. Even for distributions that are not particularly skewed, say 1/4 the skewness of an exponential distribution (e.g., a gamma distribution with shape = 16), the sample size would need to be around 470 to reduce the errors in noncoverage to 10% of the desired values. To obtain reasonable accuracy for smaller sample sizes requires the use of more accurate confidence intervals, either a secondorder accurate bootstrap interval or a comparable second-order accurate nonbootstrap interval. Two general second-order accurate procedures that do not require sampling are ABC (23) and automatic percentile (24) intervals, which are approximations for BCa and tilting intervals, respectively. The current practice of statistics, using normal and t intervals with skewed data, systematically produces confidence intervals with endpoints that are too low (for positivelyskewed data). Similarly, hypothesis tests are systematically biased; for positively skewed data, they reject H 0 : θ = θ 0 too often for cases with θˆ < θ0 and too little for θˆ > θ0 . The primary reason is acceleration— when θˆ < θ0 then acceleration makes it likely that s < σ , and the t interval does not correct for this, so it improperly rejects H 0 . 8.9 Comparing Intervals t intervals and bootstrap percentile intervals are quick-and-dirty intervals that are suitable for rough approximations, but these methods should not be used where accuracy is needed. Among the others, I recommend the BCa in most cases, provided that the number of bootstrap samples B is very large. In my experience with extensive simulations, the bootstrap t is the most accurate in

terms of coverage probabilities. However, it achieves this result at a high cost—the interval is longer on average than the BCa and tilting intervals, often much longer. Adjusting the nominal coverage level of the BCa and tilting intervals upward gives comparable coverage to bootstrap t with shorter length. And the lengths of bootstrap t intervals vary much more than the others. I conjecture that this length difference occurs because bootstrap t intervals are sensitive to the kurtosis of the bootstrap distribution, which is hard to estimate accurately from reasonablesized samples. In contrast, BCa and tilting intervals depend primarily on mean, standard deviation, and skewness of the bootstrap distribution. Also, the bootstrap t is computationally expensive if the standard error is obtained by bootstrapping. If sθˆ is calculated by bootstrapping, then sθˆ ∗ is calculated using a second level of bootstrapping—drawing bootstrap samples from each first-level bootstrap sample (requiring a total of B + BB2 bootstrap samples, if B2 second-level bootstrap samples from each of B first-level bootstrap samples). The primary advantage of bootstrap tilting over BCa is that it requires many fewer bootstrap replications, typically by a factor of 37 for a 95% confidence interval. The disadvantages of tilting are that the small-sample properties of the fixed-derivative versions F1 and F3 are not particularly good, whereas the more rigorous F2 and F4 are harder to implement reliably. 9 HYPOTHESIS TESTING An important point in bootstrap hypothesis testing is that sampling should be performed in a way that is consistent with the null distribution. We describe here three bootstrap hypothesis testing procedures: pooling for two-sample tests, bootstrap tilting, and bootstrap t. The first is for two-sample problems, such as comparing two means. Suppose that the null hypothesis is that θ 1 = θ 2 , and that one is willing to assume that if the null hypothesis is true, then the two populations are the same. Then one may pool the data, draw samples of size n1 and n2 with replacement

BOOTSTRAP

from the pooled data, and compute a test statistic such as θˆ1 − θˆ2 or a t statistic. Let T* be the bootstrap test statistic, and led T 0 be the observed value of the test statistic. The P-value is the fraction of time that the T* exceeds T 0 . In practice, we add 1 to the numerator and denominator when computing the fraction—the one-sided P-value for the onesided alternative hypothesis θˆ1 − θˆ2 > 0 is

(26)

#(T ∗ > T0 ) + 1 . B+1

The lower one-sided P-value is #(T ∗ < T0 ) + 1 , B+1 and the twosided P-value is the two times the smaller of the one-sided P-values. This procedure is similar to the twosample permutation test, which pools the data and draws n1 observations without replacement for the first sample and allots the remaining n2 observations to be the second sample. The permutation test is preferred. For example, suppose one outlier exists in the combined sample; every pair of permutation samples has exactly one copy of the outlier, whereas the bootstrap samples may have 0, 1, 2, . . . copies. This result adds extra variability not present in the original data, and it detracts from the accuracy of the resulting P-values. Now suppose that one is not willing to assume that the two distributions are the same. Then bootstrap tilting hypothesis testing (3,25,26) may be suitable. Tilting may also be used in one-sample and other contexts. The idea is to find a version of the empirical distribution function(s) with unbreakequal probabilities that satisfy the null hypothesis (by maximizing likelihood or minimizing Kullback-Leibler distance subject to the null hypothesis), then draw samples from the unequal-probability empirical distributions, and let the P-value be the fraction of times the bootstrap test statistic exceeds the observed test statistic. As in the case of confidence intervals, importance sampling may

25

be used in place of sampling with unequal probabilities, see the section on ‘‘Bootstrap confidence intervals.’’ This method shares many close connections to empirical likelihood (27). Bootstrap tilting hypothesis tests reject H 0 if bootstrap tilting confidence intervals exclude the null hypothesis value. The third general-purpose bootstrap testing procedure is related to bootstrap t confidence intervals. A t statistic is calculated for the observed data, and the P-value for the statistic is calculated not by reference to the Students t distribution, but rather by reference to the bootstrap distribution for the t statistic. In this case, the bootstrap sampling need not be done consistently with the null hypothesis, because t statistics are approximately pivotal—their distribution is approximately the same independent of θ . 10

PLANNING CLINICAL TRIALS

The usual bootstrap procedure is to draw samples of size n from the empirical data, or more generally to plug in an estimate for the population and draw samples using the sampling mechanism actually used in practice. In planning clinical trials, we may modify this procedure in two ways: • try other sampling procedures, such as

different sample sizes or stratification, and/or • plug in alternate population estimates. For example, given training data of size n, to estimate standard errors or confidence interval width that would result from a possible clinical trial of size N, we may draw bootstrap samples of size N with replacement from the data. Similarly, we may estimate the effects of different sampling mechanisms, such as stratified sampling, or case-control allocation to arms, even if pilot data were obtained other ways. For example, we consider preliminary results from a clinical trial to evaluate the efficacy of maintenance chemotherapy for acute myelogenous leukemia (28,29). After achieving remission through chemotherapy,

26

BOOTSTRAP Table 3. Leukemia Data Group

Length of Complete Remission (in weeks)

Maintained Nonmaintained

9, 13, 13+ 18, 23, 28+ 31, 34, 45+ 48, 161+ 5, 5, 8, 8, 12, 16+ 23, 27, 30, 33, 43, 45

the patients were assigned to a treatment group that received maintenance chemotherapy and a control group that did not. The goal was to examine whether maintenance chemotherapy prolonged the time until relapse. The data are in Table 3. In all, 11 subjects were in the treatment group, and 12 subjects were in the control group. A Cox proportional hazards regression, using Breslow’s method of breaking ties, yields a log-hazard ratio of 0.904 and standard error 0.512: coef exp(coef) se(coef) z p group 0.904 2.47 0.512 1.77 0.078 An ordinary bootstrap with B = 104 resulted in 11 samples with complete separation— where the minimum observed relapse time in the treatment group exceeds the maximum observed relapse in the control group, which yields an infinite estimated hazard ratio. A stratified bootstrap reduces the number of samples with complete separation to three. Here, stratification is preferred (even if the original allocation were not stratified) to condition on the actual sample sizes and prevent imbalance in the bootstrap samples. Omitting the three observations results in a slightly long-tailed bootstrap distribution, with standard error 0.523, which is slightly larger than the formula standard error. Drawing 50 observations from each group results in a bootstrap distribution for loghazard ratio that is nearly exactly normal with almost no bias, no samples with separation (they are still possible, but unlikely), and a standard error of 0.221—about 14% less than would be obtained by dividing the formula standard error by two, 0.512/2 = 0.256. Similar results are obtained using Efron’s method for handling ties and from a smoothed bootstrap with a small amount of noise added to the remission times. The fact that the reduction in standard error was 14% greater than expected based on the usual

√ O(1/ n) rate may be because censored observations have a less-serious impact with larger sample sizes. 10.1 ‘‘What If’’ Analyses—Alternate Population Estimates In planning clinical trials, it may often be of interest to do ‘‘what if’’ analyses, perturbing various inputs. For example, how might the results differ under sampling from populations with a log hazard ratio of zero, or 0.5? This method should be done by reweighting observations (30,31). This method is a version of bootstrap tilting (18,20,30,32) and is closely related to empirical likelihood (33). Consider first a simple example—sampling the difference in two means, θˆ = x1 − x2 . To sample from populations with different values of θ , it is natural to consider perturbing the data, shifting one or both samples, for example, adding θ − θˆ to each value in sample 1. Perturbing the data does not generalize well to other situations. Furthermore, perturbing the data would often give incorrect answers. Suppose that the observations represent positive skewed observations such as survival times, with a mode at zero. Shifting one sample to the left would give negative times; a shift to the right would make the mode nonzero. More subtle, but very important, is that shifting ignores the mean– variance relationships for skewed populations—increasing the mean should also increase the variance. Instead, we use a weighted version of the empirical data, which maximizes the likelihood of the observed data subject to the weighted distributions that satisfy desired constraints. To satisfy µ1 − µ2 = θ 0 , for example, we maximize n1  i=1

w1i

n2  i=1

w2i

(27)

BOOTSTRAP

subject to constraints on weights (given here for two samples): w1i > 0, i = 1, . . . , n1

(28)

w2i > 0, i = 1, . . . , n2 n1 

w2i = 1

i=1

and the constraint specific to comparing two means: n1  i=1

w1i x1i −

n2 

w2i x2i = θ0

(29)

i=1

For other statistics we replace Equation (28) with the more general θ (Fˆ n,w ) = θ0

distribution. In this case, both bootstraps are nearly normal, and the standard errors are very similar—0.221 for the ordinary bootstrap and 0.212 for the weighted bootstrap, both with 50 observations per group. 11 HOW MANY BOOTSTRAP SAMPLES ARE NEEDED

w1i = 1

i=1 n2 

27

(30)

where Fˆ n,w is the weighted empirical distribution (with obvious generalization to multiple samples or strata). The computational tools used for empirical likelihood (33) and bootstrap tilting (18,20) are useful in determining the weights. The bootstrap sampling is from the weighted empirical distributions, that is, the data are sampled with unequal probabilities. Figure 15 shows this idea applied to the leukemia data. The top left shows KaplanMeier survival curves for the original data, and the top right shows the bootstrap distribution for the log hazard ratio, using 50 observations in each group. The bottom left shows weights chosen to maximize Equation (26), subject to Equation (28) and a log hazard ratio equal to 0.5. To reduce the ratio from its original value of 0.904, the treatment group gets high weights early and low weights later (the weighted distribution has a higher probability of early events) whereas the control group gets the converse. Censored observations get roughly the average weight of the remaining noncensored observations in the same group. The middle left shows the resulting weighted survival estimates, and the middle right shows the corresponding bootstrap

We suggested in the Section ‘‘Accuracy of Bootstrap definitions’’ that 1000 bootstrap samples is enough for rough approximations, but that more are needed for greater accuracy. In this section, we give details. The focus here is on Monte Carlo accuracy— how well the usual random-sampling implementation of the bootstrap approximates the theoretical bootstrap distribution. A bootstrap distribution based on B random samples corresponds to drawing B observations with replacement from the theoretical bootstrap distribution. Quantities such as the mean, standard deviation, or quantiles of the bootstrap distribution converge to their √ theoretical counterparts at the rate O(1/ B), in probability. Efron and Tibshirani (1) suggest that B = 200, or even as few as B = 25, suffices for estimating standard errors, and that B = 1000 is enough for confidence intervals. We argue that larger sizes are appropriate, on two grounds. First, those criterion were developed when computers were much slower; with faster, computers, it is much easier to take large samples. Second, those criteria were developed using arguments that combine the random variation caused by the original sample with the random variation caused by bootstrap sampling. For example, Efron and Tibshirani (1) indicate that  E( ) + 2 1/2 . ˆ ∞ )2 + ˆ B )= cv(se cv(se 4B where cv is coefficient of variation, cv(Y) = σ Y /E(Y), seB and se∞ are bootstrap standard errors using B or ∞ replications, respectively, and relates to the kurtosis of the bootstrap distribution; it is zero for normal distributions. Even relatively small values of ˆ ∞ ) not much ˆ B )/cv(se B make the ratio cv(se larger than 1.

28

BOOTSTRAP

Kaplan Meier curves 1.0

Ordinary bootstrap

0.5

1.0

Density

0.6 0.4

0.0

0.0

0.2

Proportion Surviving

0.8

1.5

Observed Mean

0

50

100

150

0.0

Weighted bootstrap

1.5

Observed Mean

Density

0.6

0.5

0.4

1.0

1.5

0.8

1.0

2.0

Weighted Kaplan Meier curves

0.0

0.0 0

50

100

0.0

150

0.5

Observation Weights

0.10

0.12

Maintained Control

0.08

⊕

0.06

⊕

⊕

0.02

0.04

⊕

0.0

Maintained/censored ⊕ Control/censored 0

50

100

1.0

Log Hazard Ratio

Survival Time in Weeks

Observation Weights

1.0

Log Hazard Ratio

0.2

Proportion Surviving

0.5

Survival Time in Weeks

43

150

Log Hazard Ratio

Figure 15. Perturbed bootstrap distribution for survival.

1.5

BOOTSTRAP

We feel more relevant is the variation in bootstrap answers conditional on the data. This finding is particularly true in clinical trial applications, where • reproducibility is important—two peo-

ple analyzing the same data should get (almost exactly) the same results, with random variation between their answers minimized, and • the data may be very expensive—there is little point in wasting the value of expensive data by introducing extraneous variation using too small B. Given the choice between reducing variation in the ultimate results by gathering more data or by increasing B, it would be cheaper to increase B, at least until B is large. .

ˆ B) =  Conditional on the data, cv(se (δ + 2)/(4B), where δ is the kurtosis of the theoretical bootstrap distribution (conditional on the data). When δ is zero (usually approximately √true), this equation simplifies . ˆ B )=1/ 2B. to cv(se To determine how large B should be, we consider the effect on confidence intervals. Consider a t interval of the form θˆ ± tα/2 seB . Suppose that such an interval using se∞ would be approximately correct, with onesided noncoverage α/2. Then the actual noncoverage using seB in place of se∞ would −1 (α/2)). For large n be Ft,n−1 ((seB /se∞ )Ft,n−1 and α = 0.05, to have the actually one-sided noncoverage fall within 10% of the desired value (between 0.0225 and 0.0275) requires that seB /se∞ be between −1 (0.025*1.1)/−1 (0.025) = 0.979 and −1 (0.025*0.9)/−1 (0.025) = 1.023. To have 95% confidence of no√more than 10% error requires that 1.96/ 2B ≤ 0.022, or B ≥ 0.5(1.96/0.022)2 = 3970, or about 4000 bootstrap samples. To satisfy the more stringent criterion of 95% confidence that the noncoverage error is less than 1% of 0.025 would require approximately 400,000 bootstrap samples. With modern computers, this number is not unreasonable, unless the statistic is particularly slow to compute. Consider also bootstrap confidence intervals based on quantiles. The simple bootstrap percentile confidence interval is the range

29

from the α/2 to 1 − α/2 quantiles of the bootstrap distribution. Let G−1 ∞ (c) be the c quantile of the theoretical bootstrap distribution, and the number of bootstrap statistics that fall below this quantile is approximately binomial with parameters B and c (the proportion parameter may differ slightly because the discreteness of the bootstrap distribution). For finite B, the one-sided  error has standard error approximately c(1 − c)/B. For c = 0.025, to reduce 1.96 standard errors to c/10 requires B ≥ (10/0.025)2 1.962 0.025*0.975 = 14980, about 15,000 bootstrap samples. The more stringent criterion of a 1% error would require approximately 1.5 million bootstrap samples. The bootstrap BCa confidence interval has greater Monte Carlo error because it requires estimating a bias parameter using the proportion of bootstrap samples that fall below the original θˆ (and  the variance of a binomial proportion p(1 − p)/B is greatest for . p = 0.5). It requires B about twice as large as the bootstrap percentile interval for equivalent Monte Carlo accuracy— 30,000 bootstrap samples to satisfy the 10% criterion. On the other hand, the bootstrap tilting interval requires about 17 times fewer bootstrap samples for the same Monte Carlo accuracy as the simple percentile interval, so that about 1000 bootstrap samples would suffice to satisfy the 10% criterion. In summary, to have 95% probability that the actual one-sided non-coverage for a 95% bootstrap interval falls within 10% of the desired value, between 0.0225 and 0.0275, conditional on the data, requires about 1000 samples for a bootstrap tilting interval, 4000 for a t interval using a bootstrap standard error, 15,000 for a bootstrap percentile interval, and 30,000 for a bootstrap BCa interval. Figure 16 shows the Monte Carlo variability of several bootstrap confidence interval procedures, for various combinations of sample size, statistic, and underlying data; these samples are representative of a larger collection of examples in a technical report (21). The panels show the variability caused by Monte Carlo sampling with a finite bootstrap sample-size B, conditional on the data.

30

BOOTSTRAP

Figure 16 is based on 2000 randomly generated datasets for each sample size, distribution, and statistic. For each dataset, and for each value of B, two sets of bootstrap samples are created and intervals calculated using all methods. For each method, a sample variance is calculated using the usual unbiased sample variance (based on two observations). The estimate of Monte Carlo variability is then the average across the 2000 datasets of these unbiased sample variances. The result is the ‘‘within-group’’ component of variance (caused by Monte Carlo variability) and excludes the ‘‘between-group’’ component (caused by differences between datasets). 11.1

Assessing Monte Carlo Variation

To assess Monte Carlo variation in practice, two options can be employed. The first is to use asymptotic formulae. For example, the bootstrap estimate of bias in Equation (1) depends on the sample mean of the bootstrap statistics; the usual formula √ for standard error of a sample mean is seB / B, in which seB is the sample standard deviation of the bootstrap statistics. The standard error of a  ˆ − p)/B. ˆ The bootstrap proportion pˆ is p(1 standard  error of a bootstrap standard error seB is seB (δ + 2)/(4B). The other alternative is to resample the bootstrap values. Given B i.i.d. observations θˆ1∗ , θˆ2∗ , . . . , θˆB∗ from the theoretical bootstrap distribution, and a summary statistic Q (e.g., standard error, bias estimate, or endpoint of a confidence interval), we may draw B2 bootstrap samples of size B from the B observations, and calculate the summary statistics Q∗1 , Q∗2 , . . . , Q∗B . The sample standard devia2 tion of the Q*s is the Monte Carlo standard error. 11.2

Variance Reduction

Several techniques can be used to reduce the Monte Carlo variation. The balanced bootstrap (34), in which each of the n observations is included exactly B times in the B bootstrap samples, is useful for bootstrap bias estimates but is of little value otherwise. Antithetic variates (35) is moderately helpful for bias estimation but is of little value otherwise.

Importance sampling (36,37) is particularly useful for estimating tail quantiles, as for bootstrap percentile and BCa intervals. For nonlinear statistics, one should used a defensive mixture distribution (38,39). Control variates (35,38,40,41) is moderately to extremely useful for bias and standard error estimation and can be combined with importance sampling (42). They are most effective in large samples for statistics that are approximately linear. Concomitants (41,43) are moderately to extremely useful for quantiles and can be combined with importance sampling (44). They are most effective in large samples for statistics that are approximately linear; linear approximations tailored to a tail of interest can dramatically improve the accuracy (45). Quasi-random sampling (46) can be very useful for small n and large B; the convergence rate is O(log(B)n B−1 ) compared with O(B−1/2 ) for Monte Carlo methods. Analytical approximations for bootstrap distributions are available in some situations, which include analytical approximations for bootstrap tilting and BCa intervals (19,23), and saddle-point approximations (47–51).

12 ADDITIONAL REFERENCES In this section we give some additional references. Hesterberg et al. (52) is an introduction to the bootstrap written for introductory statistics students. It is available free at http://bcs. whfreeman.com/pbs/cat 160/PBS18.pdf Efron and Tibshirani (1) is an introduction to the bootstrap written for upper-level undergraduate or beginning graduate students. Davison and Hinkley (3) is the best general-purpose reference to the bootstrap for statistical practitioners. Hall (7) looks at asymptotic properties of various bootstrap methods. Chernick (53) has an extensive bibliography, with roughly 1700 references related to the bootstrap.

BOOTSTRAP

0.025

Correlation, Normal, n = 80

0.020 0.015 0.010 0.005

Monte Carlo standard dev

empir BCa boot-t exp-tilt ml-tilt

0.0

0.01

0.02

0.03

empir BCa boot-t exp-tilt ml-tilt

0.0

Monte Carlo standard dev

Mean, Exponential, n = 40

100

140

200 300 500 Bootstrap sample size

1K

2K

100

200 300 500 Bootstrap sample size

1K

2K

1K

2K

0.08 0.06 0.04

Monte Carlo standard dev

0.10

empir BCa boot-t exp-tilt ml-tilt

0.0

0.02

0.005

0.015

0.025

140

Variance, Exponential, n = 20

empir BCa boot-t exp-tilt ml-tilt

0.0

Monte Carlo standard dev

Variance, Normal, n = 10

100

140

200 300 500 Bootstrap sample size

1K

2K

100

200 300 500 Bootstrap sample size

Ratio of Means, Exponential, n = 80

0.06 0.04

Monte Carlo standard dev

0.06 0.04

0.0

0.02 0.0

empir BCa boot-t exp-tilt ml-tilt

0.02

empir BCa boot-t exp-tilt ml-tilt

0.08

140

0.08

Ratio of Means, Normal, n = 10

Monte Carlo standard dev

31

100

140

200 300 500 Bootstrap sample size

1K

2K

100

45

140

200 300 500 Bootstrap sample size

Figure 16. Monte Carlo variability for confidence intervals.

1K

2K

32

BOOTSTRAP

The author’s website www.insightful.com/ Hesterberg/bootstrap has resources for teaching statistics using the bootstrap and for bootstrap software in S-PLUS. Some topics that are beyond the scope of this article include bootstrapping dependent data (time series, mixed effects models), cross-validation and bootstrap-validation (bootstrapping prediction errors and classification errors), Bayesian bootstrap, and bootstrap likelihoods. References 1 and 3 are good starting points for these topics, with the exception of mixed effects models. REFERENCES 1. B. Efron and R. J. Tibshirani, An Introduction to the Bootstrap. Boca Raton, FL: Chapman and Hall, 1993. 2. L. Breiman, Random forests. Mach. Learn. 2001; 45(1): 5–32. 3. A. Davison, and D. Hinkley, Bootstrap Methods and their Applications. Cambridge, UK: Cambridge University Press, 1997. 4. J. Chambers and T. Hastie, Statistical Models in S. Beimont, CA: Wadsworth, 1992. 5. J. M. Chambers, W. S. Cleveland, B. Kleiner, and P. A. Tukey, Graphical Methods for Data Analysis. Beimont, CA: Wadsworth, 1983. 6. A. Ruckstuhl, W. Stahel, M. Maechler, and T. Hesterberg, sunflower. 1993. Statlib: Availables: http://lib.stat.emu.edu/S. 7. P. Hall, The Bootstrap and Edgeworth Expansion. New York: Springer, 1992. 8. J. Shao and D. Tu, The Jackknife and Bootstrap. New York: Springer-Verlag, 1995. 9. B. Efron, Better bootstrap confidence intervals (with discussion). J. Am. Stat. Assoc., 1987; 82: 171–200. 10. T. C. Hesterberg, Unbiasing the Bootstrap Bootknife Sampling vs. Smoothing. In Proceedings of the Section on Statistics & the Environment, Alexandria, VA: American Statistical Association, 2004, pp. 2924–2930. 11. B. Silverman and G. Young, The bootstrap: to smooth or not to smooth. Biometrika 1987; 74: 469–479. 12. P. Hall, T. DiCiccio, and J. Romano, On smoothing and the bootstrap. Ann. Stat., 1989; 17: 692–704. 13. T. J. DiCiccio and J. P. Romano, A review of bootstrap confidence intervals (with discussion). J. Royal Stat. Soc. Series B, 1988; 50(3): 338–354.

14. P. Hall, Theoretical comparison of bootstrap confidence intervals (with discussion). Ann. Stat., 1988; 16: 927–985. 15. T. DiCiccio and B. Efron, Bootstrap confidence intervals (with discussion). Stat. Sci., 1996; 11(3): 189–228. 16. B. Efron, Bootstrap methods: another look at the jackknife (with discussion). Ann. of Stat., 1979; 7: 1–26. 17. B. Efron, The Jackknife, the Bootstrap and Other Resampling Plans. National Science Foundation–Conference Board of the Mathematical Sciences Monograph 38. Philadelphia, PA: Society for Industrial and Applied Mathematics, 1982. 18. B. Efron, Nonparametric standard errors and confidence intervals. Canadian J. Stat., 1981; 9: 139–172. 19. T. J. DiCiccio and J. P. Romano, Nonparametric confidence limits by resampling methods and least favorable families. Internat. Stat. Rev., 1990; 58(1): 59–76. 20. T. C. Hesterberg, Bootstrap tilting confidence intervals and hypothesis tests. In: K. Berk and M. Pourahmadi, (eds.), Computer Science and Statistics: Proceedings of the 31st Symposium on the Interface, vol. 31. Fairfax Station, VA: Interface Foundation of North America, 1999, pp. 389–393. 21. T. C. Hesterberg, Bootstrap Tilting Confidence Intervals. Research Department 81, Seattle, WA: MathSoft, Inc., 1999. 22. R. J. Hyndman and Y. Fan, Sample quantiles in statistical packages. Am. Stat., 1996; 50: 361–364. 23. T. DiCiccio and B. Efron, More accurate confidence intervals in exponential families. Biometrika, 1992; 79: 231–245. 24. T. J. DiCiccio, M. A. Martin, and G. A. Young, Analytic approximations to bootstrap distribution functions using saddlepoint methods. Technical Report 356. Palo Alto, CA: Department of Statistics, Stanford University, 1990. 25. B. Efron, Censored data and the bootstrap. J. Am. Stat. Assoc., 1981; 76(374): 312–319. 26. D. V. Hinkley, Bootstrap significance tests. Bull. Internat. Stat. Inst., 1989; 65–74. 27. A. Owen, Empirical likelihood ratio confidence intervals for a single functional. Biometrika 1988; 75: 237–249. 28. S. H. Embury, L. Elias, P. H. Heller, C. E. Hood, P. L. Greenberg, and S. L. Schrier, Remission maintenance therapy in acute

BOOTSTRAP myelogenous leukemia. Western J. Med. 1977; 126: 267–272. 29. Insightful. S-PLUS 8 Guide to Statistics. Seattle, WA: Insightful, 2007. 30. T. C. Hesterberg, Bootstrap tilting diagnostics, Proceedings of the Statistical Computing Section, 2001. 31. T. C. Hesterberg, Advances in Importance Sampling. PhD thesis, Statistics Department, Stanford University, Palo Alto, California, 1988. 32. T. C. Hesterberg, Resampling for Planning Clinical Trials-Using S + Resample. Paris: Statistical Methods in Biopharmacy, 2005. Available: http:// www.insightful.com/Hesterberg/ articles/Paris05-ResampleClinical.pdf. 33. P. Hall and B. Presnell, Intentionally biased bootstrap methods. J. Royal Stat. Soc., Series B, 1999; 61(1): 143–158. 34. A. Owen, (2001). Empirical Likelihood. Boca Raton− FL: Chapman & Hall/CRC Press, 2001. 35. J. R. Gleason, Algorithms for balanced bootstrap simulations. Am. Statistic., 1988; 42(4): 263–266. 36. T. M. Therneau, Variance reduction techniques for the bootstrap, PhD thesis, Department of Statistics, Stanford University, Palo Alto, California, 1983. Technical Report No. 200. 37. M. V. Johns, Importance sampling for bootstrap confidence intervals. J. Am. Stat. Assoc., 1988; 83(403): 701–714. 38. A. C. Davison, Discussion of paper by D. V. Hinkley. J. Royal Stat. Soc. Series B, 1986; 50: 356–57. 39. T. C. Hesterberg, Weighted average importance sampling and defensive mixture distributions. Technometrics 1995; 37(2): 185–194. 40. A. C. Davison, D. V. Hinkley, and E. Schechtman, Efficient bootstrap simulation. Biometrika, 1986; 73: 555–566. 41. B. Efron, More efficient bootstrap computations. J. Am. Stat. Assoc., 1990; 85(409): 79–89. 42. T. C. Hesterberg, Control variates and importance sampling for efficient bootstrap simulations. Stat. Comput., 1996; 6(2): 147–157. 43. K. A. Do and P. Hall, Distribution estimation using concomitants of order statistics, with application to Monte Carlo simulations for the bootstrap. J. Roy. Stat. Soc., Series B, 1992; 54(2): 595–607.

33

44. T. C. Hesterberg, Fast bootstrapping by combining importance sampling and concomitants. In: E. J. Wegmand and S. Azen, (ed.), Computing Science and Statistics: Proceedings of the 29th Symposium on the Interface, volume 29. Fairfax Station, VA: Interface Foundation of North America, 1997, pp. 72–78. 45. T. C. Hesterberg, Tail-specific linear approximations for efficient bootstrap simulations. J. Computat. Graph. Stat., 1995; 4(2): 113–133. 46. K. A. Do and P. Hall, Quasi-random sampling for the bootstrap. Stat. Comput., 1991; 1(1): 13–22. 47. M. Tingley and C. Field, Small-sample confidence intervals. J. Am. Stat. Assoc., 1990; 85(410): 427–434. 48. H. E. Daniels and G. A. Young, Saddlepoint approximation for the studentized mean, with an application to the bootstrap. Biometrika 1991, 78(1): 169–179. 49. S. Wang, General saddlepoint approximations in the bootstrap. Stat. Prob. Lett., 1992; 13: 61–66. 50. T. J. DiCiccio, M. A. Martin, and G. A. Young, Analytical approximations to bootstrap distributions functions using saddlepoint methods. Statistica Sinica 1994; 4(1): 281. 51. A. J. Canty and A. C. Davison, Implementation of saddlepoint approximations to bootstrap distributions, In L. Billard, and N. I. Fisher, (eds.), Computing Science and Statistics; Proceedings of the 28th Symposium on the Interface, vol. 28. Fairfax Station, VA: Interface Foundation of North America, 1997, pp. 248–253. 52. T. Hesterberg, S. Monaghan, D. S. Moore, A. Clipson, and R. Epstein, Bootstrap methods and permutation tests In: D. Moore, G. McCabe, W. Duckworth, and S. Sclore, The Practice of Business Statistics. New York: W. H. Freeman, 2003. 53. M. R. Chernick, Bootstrap Methods: A Practitioner’s Guide. New York: Wiley, 1999.

CARDIAC ARRHYTHMIA SUPPRESSION TRIAL (CAST)

1

OBJECTIVES

The major objective of CAST was to test whether post MI patients with left ventricular dysfunction (defined by reduced ejection fraction) who have either symptomatic or asymptomatic ventricular arrhythmias suppressed by antiarrhythmic agents would be less likely to experience an arrhythmic death over 3 years 5.

LEMUEL A. MOYE´ University of Texas School of Public Health

The Cardiac Arrhythmia Suppression Trial (CAST) was the National Heart, Lung, and Blood Institute-(NHLBI)-sponsored, randomized, controlled clinical trial designed to confirm the arrhythmia suppression hypothesis. Instead, it revealed a 2.5-fold increase (96% CI 1.6 to 4.5) in the mortality rate of patients assigned to either encainide or flecainide, which resulted in the early termination of the study. Its findings altered the development program of antiarrhythmic agents and demonstrated the value of objective evidence in formulating treatment guidance. After a myocardial infarction (MI), patients have an increased risk of death from arrhythmia and nonarrhythmic cardiac causes. Although many factors contribute to this risk, ventricular premature contractions (VPCs) that occur post MI confer an independent risk for both arrhythmic death and death from all cardiac causes (1,2). These patients are commonly treated with antiarrhythmic drugs 3. During the 1980s the available antiarrhythmic agents (quinidine and procainimide) were supplemented with the development of the three new agents, encainide, flecanide, and moricizine. Initial reports from case studies suggested that these drugs successfully suppressed ventricular arrhythmias 4. After numerous discussions, but in the absence of a clinical trial, these drugs were approved by the FDA for use in patients with serious ventricular arrhythmias, in the belief that the suppression of asymptomatic or mildly symptomatic ventricular arrhythmias would reduce the arrhythmic death rate. This theory became known as the arrhythmia suppression hypothesis. After FDA approval, the National Heart, Lung, and Blood Institute (NHLBI) designed a clinical trial to test this concept.

1.1 Primary Endpoint The single primary endpoint for CAST was death from arrhythmia. This definition included (1) witnessed instantaneous death in the absence of severe congestive heart failure or shock, (2) unwitnessed death with no preceding change in symptoms and for which no other cause was identified, and (3) cardiac arrest. 1.2 Secondary Endpoints Additional, prospectively declared endpoints were (1) new or worsened congestive heart failure, (2) sustained ventricular tachycardia, (3) recurrent myocardial infarction, (4) cardiac procedures, and (5) quality of life. 2

STUDY DESIGN

CAST was a randomized, double-blind, placebo-controlled clinical trial. 2.1 Patient Eligibility Patients were eligible for CAST screening if they had an MI between 6 days and 2 years before the screening. Patients whose MI was within 90 days before the screening Holter monitor examination had to have ejection fractions of 55% or less. Patients whose MI occurred more than 90 days before the qualifying Holter were required to have an ejection fraction of 40% or less. To be eligible, patients must have had a ventricular arrhythmia defined as at least six or more ventricular premature contractions per hour and, in addition, must have had documented sustained ventricular tachycardia

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

CARDIAC ARRHYTHMIA SUPPRESSION TRIAL (CAST)

to be randomized. However, patients whose ventricular ectopy caused either presyncope or syncope were excluded. In addition, patients who had no contraindications to the therapy or were poor compliers were also denied entry to the study. The protocol was approved by the institutional review board of each clinical center. 2.2 Titration Patients who met the above entry criteria then underwent on open titration period, receiving up to three drugs (encainide, flecanide, and moricizine) at two oral doses. The goal of the titration was to suppress the ventricular arrhythmia. Titration ceased when the arrhythmia was suppressed, defined by a greater than 60% reduction in premature ventricular contractions and at least a 90% reduction in runs of unsustained ventricular tachycardia. Subjects with ejection fractions less than 30% were not given flecainide. 135 patients (6%) had incomplete arrhythmia suppression. Patients whose arrhythmia was aggravated by the titration therapy or proved to be intolerant to the drugs were not randomized. The titration exercise produced 447 patients who either died or were intolerant to the drug. 2.3 Randomization and Follow-Up Patients who 1) met the inclusion and exclusion criteria and 2) had their ventricular ectopy suppressed during the titration phase were randomized by telephone call through the CAST Coordinating Center. Patients were randomized to receive either encainide, flecainide, moricizine, or placebo therapy. Randomization was stratified by clinical center, ejection fraction (either ≤ 30% or > 30%), and time between the MI and the qualifying Holter examination (< 90 days or ≥ 90 days). Follow-up examinations were scheduled at 4-month intervals. The expected duration of follow-up was anticipated to be 3 years. 2.4 Sample Size The primary study hypothesis was whether the use of antiarrhythmic agents (encainide, flecanide, and moricizine) would reduce the 3-year cumulative mortality rate from sudden death. Notably, CAST was designed as a

one-tailed test to assess whether drug therapy was beneficial or had no benefit. It was not designed with statistical rigor to demonstrate harm. The anticipated 3-year cumulative mortality rate from death because of arrhythmia was 11%. Treatment was anticipated to reduce this by 30%. The overall alpha error was 0.025 (one-tailed), and the desired power was 85%. These computations produced a sample size of 4400 patients for the trial. 2.5 Analysis Plan Baseline comparisons were performed by ttests and chi-square tests. Primary and secondary endpoint comparisons that involve time-to-event analyses were executed using log-rank tests. The significance levels for the individual drug comparisons were increased three-fold to adjust for multiple comparisons among the three drug groups. Confidence intervals were estimated by the method of Cornfield. The relative risk of treatment in clinically defined subgroups was calculated to evaluate the consistency of the effects of the drugs on the study endpoints across subgroups. 2.6 Monitoring Procedures The Data and Safety Monitoring Board met twice yearly to review the unblinded CAST results. This group approved the interim monitoring protocol in September 1988 before it evaluated any data. This protocol included a conservative boundary for stopping the study because of demonstrated benefit, a symmetric boundary for advising stopping the study because of adverse effects, and a stochastic curtailment boundary to permit stopping CAST for a low probability of demonstrating a beneficial effect. The total number of events anticipated in the trial was 425, but the data as of March 1989 indicated that less than 300 events would occur. 3 RESULTS 3.1 Screening As of March 30, 1989, 2309 patients had been recruited and either were undergoing or had completed the open-label titration phase of the study. Of these, 1727 patients met the

CARDIAC ARRHYTHMIA SUPPRESSION TRIAL (CAST)

arrhythmia requirement, had their arrhythmias suppressed, and completed randomization. Assignments to therapy were as follows: 1455 received either encainide, flecainide, or placebo; 272 received either moricizine or placebo. Overall, 730 patients were randomly assigned to encainide and flecainide, and 725 patients received placebo therapy. 3.2 Baseline Characteristics The baseline characteristics of patients randomized to encainide or flecainide were compared with patients recruited to placebo therapy. The average age was 61 years old, and 82% of participants were male. Placebo and treatment groups were similar with respect to all characteristics, including age, ejection fraction, time elapsed since myocardial infarction, and use of beta-blockers, calcium-channel blockers, digitalis, or diuretics at baseline. The mean left ventricular ejection fraction was 0.40 in patients treated with encainide or flecainide. Approximately 2.2% of patients had an ejection fraction below 0.20; 20% had an ejection fraction between 0.30 and 0.55. The mean frequency of VPCs was 127 per hour in drug-treated patients, and 20.6% of the patients had a least one run of unsustained ventricular tachycardia (≥ 120 beats per minute) during baseline Holter recording. 3.3 Compliance As of March 30, 1989, 8.4% of patients randomized to encainide or flecainide therapy had their therapy discontinued. More than half of the withdrawals exited because of protocol-defined reasons (e.g., major adverse events or symptoms). Placebo therapy was removed in 8.6% of patients. Of the patients still taking active therapy or placebo, 79% were taking at least 80% of their medication. Of the patients who died of arrhythmia or were resuscitated after a cardiac arrest, 88% were following the study regimen at the time of the fatal event. The average exposure to therapy was 295 days in the encainide/flecainide group and 300 days in the placebo group.

3

3.4 Mortality Findings 730 patients were recruited to encainide and flecainide, and 725 patients were recruited to placebo. 78 deaths occurred in the combined encainide/flecainide-placebo groups, 56 deaths in the encainide/flecainide group and 22 deaths in the placebo group. The total mortality relative risk for encainide or flecainide therapy when compared with placebo was 2.5 (95% CI 1.6 to 4.5). The relative risks of encainide or flecainide were indistinguishable (2.7 and 2.2, respectively). 33 deaths occurred because of either arrhythmia or cardiac arrests in the encainide/flecainide group, and 9 deaths occurred in the placebo group. The relative risk of death from arrhythmia for encainide or flecainide was 3.6 (95% CI 1.7–8.5). The relative risks of death from arrhythmia or cardiac arrest for patients receiving encainide or flecainide considered separately were not different (3.4 and 4.4, respectively). 14 deaths occurred for other cardiac reasons in the encainide/flecainide group compared with 6 deaths in the placebo group. 9 noncardiac deaths, unclassified deaths, or other cardiac arrest occurred in the encainide/ flecainide group, and 7 occurred in the placebo group. 3.5 Subgroup Analyses The effect of the therapy was consistent across subgroups. In all subgroups, patients treated with encainide or flecainide had higher rates of both total mortality and death because of arrhythmia than patients treated with placebo. The observed increased risk from either of these two agents was present regardless of age, prolonged QRS interval, or use of beta-blockers, calcium-channel blockers, digitalis, or diuretic at baseline. Notably, patients with an ejection fraction ≥ 30% in which use of encainide and flecainide were stratified had essentially equal relative risks (4.6 and 4.4, respectively) 3.6 Early Termination On April 16 and 17, 1989, the CAST Data and Safety Monitoring Board reviewed the data available as of March 30, 1989. An evaluation of the data for all randomized patients

4

CARDIAC ARRHYTHMIA SUPPRESSION TRIAL (CAST)

revealed that the interim monitoring statistic (Z = − 3.22) had crossed the lower advisory boundary for harm (Z = − 3.11). In addition, the conditional power for a beneficial effect (< 0.27) was well below the minimum conditional power established (0.55). The DSMB therefore recommended that the encainide and flecainide arms of the study be discontinued. 4

CONCLUSIONS

The scheduled termination of CAST was preempted by an unexpected increase in mortality, which prompted the termination of the encainide and flecainide limbs of the study. Given that millions of patients received prescriptions for approved antiarrhythmic agents during the conduct of CAST, the public health implications of its finding of antiarrhythmic therapy-induced harm were profound 6. The use of these drugs was sharply curtailed as the practicing community struggled to understand the staggering findings. No confounding factors that could explain the finding of harm were identified. In fact, the consistency of the excessive mortality risk of encainide and flecainide in all subgroups was a noteworthy feature of this study. The implications of CAST for the development of new antiarrhythmic drugs was profound. Its demonstration of hazard in the face of VPC suppression removes this suppression as an informative, predictive surrogate endpoint for clinical endpoints. In addition, the results had regulatory implications, as the FDA grappled with the correct combination of the number of trials, the type of patients, the sample size, and the goals of these trials to view the risk–benefit balance of antiarrhythmic therapy clearly. Finally, perhaps no point more clearly reveals the expectations of the CAST investigators than their acceptance of the onetailed statistical hypothesis test, specifically designed not to focus on harm. The demonstration of excess mortality overwhelmed the ability of the CAST analysis that was designed simply to demonstrate benefit versus lack of benefit. The use of one-tailed research continues to be debated in the clinical literature (7,8). The surprise results of CAST reinforced

the time-tested need for the study investigators not just to consider the possibility of harm, but also to weave this concern into the fabric of the study 9. Fortunately, their wise decision to include a placebo group not only distinguished CAST from the many preliminary studies that led to FDA approval of drugs subsequently shown to be harmful but also was the one essential characteristic of the study that permitted a clear demonstration of the damage stemming from the use of these drugs. REFERENCES 1. J. T. Bigger, Jr. et al., Multicenter PostInfarction Research Group, The relationships among ventricular arrhythmia, left ventricular dysfunction, and mortality in the 2 years afgter myocardial infarction. Circulation. 1984; 69: 250–258. 2. J. Mukharjil et al., Risk factors for sudden death afgter acute myocardial inarction; two year follow-up. Am J Cardiol. 1984; 54: 31–36. 3. S. Vlay, How the university cardiologist treats ventrcicular premature beats; a nationwide survey of 65 university medical centers. Am Heart J. 1985; 110: 904–912. 4. The CAPS Investigators, The Cardiac Arrhythmia Pilot Study. Am J Cardiol. 1986; 57: 91–95. 5. The CAPS Investigators, The CAST Investigators (1989) Preliminary Report: Effect of encainide and flecainide on mortality in a randomized trial of arrhythmia suppression after myocardial infarction. New England J Medicine. 1989; 3212: 406–412. 6. L. K. Hines, T. P. Gross, and D. L. Kennedy, Outpatient arrhythmic drug use from 1970 to 1986. Arch Intern Med. 1989; 149: 1524–1527. 7. J. A. Knottnerus and L. M. Bouter, (2001) Commentary: The ethics of sample size: Twosided testing and one-sided thinking. J Clinical Epidemiology. 2001; 54: 109–110. 8. L. A. Moy´e and A. Tita, Defending the Rationale for the Two-Tailed Test in Clinical Research. Circulation. 2002; 105: 3062–3065. 9. L. A. Moy´e, Statistical Reasoning in Medicine. The Intuitive P-value Primer. New York: Springer-Velag, 2006.

FURTHER READING T. Moore, Deadly Medicine. New York: Simon and Schuster, 1995.

CARDIAC ARRHYTHMIA SUPPRESSION TRIAL (CAST)

CROSS-REFERENCES Placebo controlled clinical trial Run in period Interim analyses Trial termination

5

Table 1. 2 × 2 Contingency Table for Treatment and Success

CATEGORICAL RESPONSE MODELS GERHARD TUTZ

Response

¨ Statistik Munich Institut Fur Germany Treatment

1

INTRODUCTION

Drug (Treatment 1) Control (Treatment 2)

In clinical trials, categorical data occur if the response is measured in categories. The simplest experimental design compares two treatment groups, usually a treatment and a control group, on a binary response that may be generally described by the categories of success or failure. Thus, data are given in a 2 × 2 table (see Table 1) where the marginals n1+ , n2+ denote the number of observations in treatment groups 1 and 2. With the binary response given by y = 1 for success and y = 0 for failure, and T denoting the treatment, investigation concerns the conditional probabilities of success in treatment group t

Failure 2

Marginals

n11

n12

n1+

n21

n22

n2+

The dependence of y on treatment group may be given in the form of a binary logit model log

πt1 = β0 + βT(t) πt2

(1)

where βT(t) is a parameter connected to treatment group t. As πt1 + πt2 = 1, only two probabilities, namely π11 and π21 , are free to vary. Therefore, the model has too many parameters and an additional restriction is needed. By setting βT(1) = βT , βT(2) = 0, one immediately obtains

πt1 = P(y = 1|T = t). The effect of the treatment is measured by comparing π11 and π12 (i.e., the probabilities of success in group 1 and 2) or functions thereof. Instead of comparing the probabilities themselves, it is often preferable to compare the odds, which for treatment group t are defined by

log

π11 = β0 + βT , π12

log

π21 = β0 . π22

Simple calculation shows that βT may be interpreted as the log odds ratio βT = log

P(y = 1|T = t) πt1 = πt2 P(y = 0|T = t)

π11 /π12 π21 /π22

and

where πt2 = 1 − πt1 = P(y = 0|T = t). Comparison of treatments may be based on the odds ratio (also called cross-product ratio) θ=

Success 1

eβT =

π11 /π12 π21 /π22

is the odds ratio. Thus, the case of no effect where the odds ratio equals (1) corresponds to βT = 0. For general designs, it is instructive to give the model in Equation (1) in the usual form of a regression model. With the restriction βT(2) = 0, Equation (1) is equivalent to the logit model

π11 /π12 π21 /π22

which is a directed measure of association between treatment and success. If θ = 1, the odds in both treatment groups are the same, and therefore treatment has no effect on the probability of success. If θ > 1, odds in treatment group 1 are larger than in treatment group 2; for 0 < θ < 1, treatment group 2 is superior to treatment group 1.

log

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

πt1 = β0 + xt βT πt2

(2)

2

CATEGORICAL RESPONSE MODELS

Table 2. m × 2 × 2 Contingency Table with Binary Response Response Center 1 2 · · m

Treatment

Success

Failure

Drug Control Drug Control · · · · Drug Control

n111 n121 n211 n221 · · · · nm11 nm21

n112 n122 n212 n222 · · · · nm12 nm22

n11+ n12+ n21+ n22+ · · · · nm1+ nm2+

where x1 = 1 (coding T = 1) and x2 = 0 (coding T = 2). 2 BINARY RESPONSE MODELS WITH COVARIATES

Often treatments are compared by using data from different sources. If, in a multi-center study, the success of treatment is investigated, one obtains count data as given in Table 2. As the effect of a treatment may depend on the specifics of a center, it is necessary to include the information where the data come from. With πst1 = P(y = 1|T = t, C = s) denoting the conditional probability of success given treatment t and center s, a logit model that uses center as covariate is given by πst1 = β0 + xt βT + βC(s) πst2

πs11 /πs12 πs21 /πs22 =

P(y = 1|T = 1, C = s)/P(y = 0|T = 1, C = s) P(y = 1|T = 2, C = s)/P(y = 0|T = 2, C = s)

= eβT which does not depend on the center. Thus, the effect of the treatment (measured by odds ratio) does not depend on the center, although centers might differ in efficiency. If the effect of treatment is modified by centers, a more complex model including interactions applies. 2.2 The Binary Logit Model with Covariates

2.1 Observations from Several Strata

log

odds. If βT = 0, variation of success might exist across centers but treatment is without effect. If Equation (3) holds, the association between treatment and response for center s is given by the odds ratio

(3)

where πst2 = 1 − πst1 = P(y = 0|T = t, C = s), and x1 = 1 (for T = 1) and x2 = 0 (for T = 2). The parameter βT specifies the effect of treatment whereas βC(s) represents the center effect. The effect on the odds is seen from the form πst1 = eβ0 · (eβT )xt · eβC(s) . πst2 Thus, eβ0 and eβC(s) represent the multiplicative effects of treatment and center on the

The multi-center study is just one example in which observations come from several strata that should be included as control variables. Alternative covariates that influence the response may be levels of age or severity of the condition being treated. If the covariates are categorical, as, for example, age levels, one again obtains a contingency table of the form given in Table 2. In the more general case where covariates also may be metric, as, for example, age in years, data are no longer given in contingency tables. Let yi with yi = 1 for success and yi = 0 for failure denote the response of the ith individual (or observed unit). Together with yi one has information about treatment (xT,i ) and a vector of covariates (zi ), which may contain age, gender, condition, and so on. With πi = P(yi = 1|xi , zi ), the corresponding binary logit model is given by log

πi = β0 + x
T,i βT + zi βz . 1 − πi

(4)

For two treatment groups, xT,i is a dummy variable as in Equation (3), given by xT,i = 1 if treatment 1 is applied on the ith observation and xT,i = 0 if treatment 2 is applied on the ith observation. It should be noted that i refers to observations not treatments.

CATEGORICAL RESPONSE MODELS

In the case with m treatment groups, xT,i represents a vector of dummy variables, x
T,i = (xT(1),i , . . . , xT(m−1),i ) where xT(s),i

⎧ ⎨1 if treatment s is used on individual i = ⎩ 0 otherwise.

log

πi = x
i β 1 − πi

exp(x
i β) . 1 + exp(x
i β)

The last form has the general structure πi = F(x
i β)

Estimation of parameters may be based on the concept of maximum likelihood. The loglikelihood for Equation (5) is given by n 

yi log(πi ) + (1 − yi ) log(1 − πi ) (6)

i=1

where πi is specified by πi = F(x
i β). Maximization of l(β) yields the maximum likeˆ In the general case, no lihood estimator β. explicit form of βˆ is available, therefore the estimator is computed iteratively [e.g., McCullagh & Nelder (2)]. For the accuracy of the maximum likelihood estimator, the Fisher or expected information matrix F(β) = E(−∂ 2 l(β)/∂β ∂β
) has to be computed. One obtains F(β) =

n 

F
(x
i β) xi − x
j − F(x
i β))

F
(x
i β)(1 i=1

where F
(η) is the first derivative F
(η) = ∂F(η)/∂η. Under regularity conditions [see Fahrmeir & Kaufmann (3)], one obtains asymptotic existence of the maximum likelihood estimate, consistency, and asymptotic normality, β˜ ∼ N(β, F(β)−1 ).

or equivalently πi =

2.3 Inference in Binary Response Models

l(β) =

The variables xT(s),i are dummy variables that code if treatment s is used or not by 0, 1 coding. Alternative coding schemes are given, for example, in Fahrmeir and Tutz (1). More generally, xT,i may contain dummy variables for treatment groups as well as metric variables referring to treatment, as, for example, dosage of drug. The same holds for the covariates zi . In a multi-center study, zi is a vector of dummy variables coding centers, but it may also contain metric variables. Equation (4) accounts for the control variables given in zi but still assumes that the effect of treatment is not modified by these variables. The model is a main effect model because it contains treatment as well as covariates as separate additive effects within the logit model. By collecting intercept, treatment, and control variables into one vector x
i = (1, x
T,i , z
i ) and defining β
= (β0 , βT
, βz
), one obtains the general form of the logit model

3

(5)

where F is the logistic distribution function F(η) = exp(η)/(1 + exp(η)). The logit model is a special case of a generalized linear model [compare Kauermann & Norris in EoCT and McCullagh & Nelder (2)], with F as the (inverse) link function. Alternative (inverse) link functions in use are based on the exponential distribution F(η) = 1 − exp(−η), the extreme minimal-value distribution F(η) = 1 − exp(− exp(η)), or the normal distribution function. The latter yields the so-called probit model [see McCullagh & Nelder (2)].

If only categorical covariates are present and therefore data are given in a contingency table, estimation may also be based on the weighted least squares approach suggested by Grizzle et al. (4). If the linear predictor ηi = x
i β is partitioned into treatment effects xi,T βT and covariates zi βz , the focus of interest is on the null hypothesis H0 : βT = 0, although the effect of covariates may also be investigated by considering the null hypothesis H0 : βz = 0. These hypotheses are special cases of the linear hypothesis H0 : Cβ = ξ

against

H1 : Cβ = ξ

where C is a specified matrix and ξ is a given vector.

4

CATEGORICAL RESPONSE MODELS

Let βˆ denote the maximum likelihood estimate for the model and β˜ denote the maximum likelihood estimate for the submodel under the additional restriction Cβ = ξ . Test statistics for the linear hypothesis are the likelihood ratio test ˜ − l(β)) ˆ λ = −2(l(β) where l(β) is the log-likelihood given in Equation (6), the Wald test ˆ
)(Cβˆ − ξ ) w = (Cβˆ − ξ )
(CF −1 (β)C and the score test

parameters than models in which ordinality of the response is ignored. For categorical data, parsimony of parameters is always recommended. In the following, let Y take values from ordered categories 1, . . . , k. 3.1 Cumulative Type Models A simple concept for treating the ordinal response is to use a model for binary responses in which the binary response corresponds to a split between category r and r + 1 yielding binary response categories {1, . . . , r}, {r + 1, . . . , k}. The corresponding binary response model in Equation (5) with linear predictor β0r + x
β is

˜ −1 (β)s( ˜ β) ˜ u = s
(β)F

P(Y ≤ r|x) = F(β0r + x
β).

where s(β) = ∂l(β)/∂β is the score function. With rg(C) denoting the rank of matrix C, all three test statistics, λ, w, and u, have the same limiting χ 2 -distribution

Although the resulting models for r = 1, . . . , k − 1 seem to be separate models, they are linked by the common response Y. A consequence is that the restriction β01 ≤ . . . ≤ β0,r+1 must hold because P(Y ≤ r) ≤ P(Y ≤ r + 1). Moreover, because β does not depend on the category in Equation (7), it is assumed that the effect of x is the same for all splits into categories {1, . . . , r}, {r + 1, . . . , k}. As in Equation (7), categories are cumulated, it is referred to as a cumulative response model. It may be motivated by considering Y as a coarser version of an underlying unobservable variable Y˜ = −x
β + with β01≤...≤β0,r+1 denoting the category boundaries on the latent continuum that defines the levels of Y. The link between observable Y and underlying variable Y˜ is specified by Y = r if γ0,r−1 < Y˜ ≤ γ0r where has distribution function F. The most common model is the proportional odds model that results from choosing F as the logistic distribution function. One obtains

a

λ, w, u ∼ χ 2 (rg(C)) under regularity conditions, which are similar to the conditions needed for asymptotic results for maximum likelihood estimation (5). An advantage of the Wald statistic is that only one fit is necessary. Although for likelihood ration and score test the restricted estimate β˜ has to be computed, the Wald test uses only the maximum likelihood estimate ˆ For testing single coefficients H0 : βT = 0, β. the Wald test is the square (w = t2 ) of the ‘‘t-value’’ βˆT t= √ aˆ rT where aˆ rT = v ar(βˆT ) is the rth diagonal element of the estimated covariance matrix ˆ Most program packages use t when F −1 (β). investigating the effect of single variables. 3

ORDINAL RESPONSES

In many cases, the response is measured in ordered categories; for example, patient condition may be given by good, fair, serious, or critical. The use of the ordering of categories allows for models that have less

log

(7)

P(Y ≤ r|x) = β0r + x
β, P(Y > T|x)

r = 1, . . . , k − 1. The model is extensively discussed in McCullagh (6). The name proportional odds is caused by the property that the cumulative odds are proportional, for example, for two values of explanatory variables x1 , x2 one obtains log

P(Y ≤ r|x1 )/P(Y > |x1 ) = (x
1 − x
2 )β P(Y ≤ r|x2 )/P(Y > |x2 )

CATEGORICAL RESPONSE MODELS

which does not depend on r. Therefore, the proportion of cumulative odds only depends on the explanatory variables. 3.2 Sequential Type Models For ordered categories, it may often be assumed that they are reached successively. The sequential model may be seen as a process model with start in category 1 and consecutive modeling of binary transitions to higher categories. The transition is specified by P(Y = r|Y ≥ r, x) = F(β0r + x
β) r = 1, . . . , k − 1. The binary model specifies the probability that the process stops in category r given category r is reached. In most common use is the sequential logistic model with F(η) = exp(η)/(1 + exp(η)). An advantage of the sequential model over the cumulative model is that no ordering restriction for the intercept is necessary. Moreover, it may be estimated by software that handles binary regression models. The relationship to ¨ a¨ cumulative models is investigated in La¨ ar and Matthews (7) and Greenland (8). 4

NOMINAL RESPONSE

For nominal response variable Y ∈ {1, . . . , k}, the most common model is the multinomial logit model P(Y = r|x) =

exp(x
βr ) , k−1 1 + s=1 exp(x
βs )

r = 1, . . . , k − 1, where k is the reference category with P(Y = k) = 1 − p(Y = r) − . . . − P(Y = k − 1). The alternative form log

P(Y = r|x) = x
βr P(Y = k|x))

(8)

shows the similarity to the binary logit model. It should be noted that when comparing P(Y = r|x) to P(Y = k|x), which is basically parameterized in Equation (8), the parameter β r has to depend on r. As Y is assumed to be on nominal scale level, the interpretation of parameters in

5

Equation (8) is similar to the binary logit model with β r referring to the effect of x on the logits between category r and the reference category k. The model is easily reparameterized if not k but k0 {1, . . . , k} is considered as the reference category by considering log

P(Y = r|x) = x
(βr − βk0 ) − x
β˜r P(Y = k0 |x)

where β˜r = βr − βk0 are the parameters corresponding to reference category k0 .

5 INFERENCE FOR MULTICATEGORICAL RESPONSE For data (Yi , xi , i = 1, . . . , n), the multicategorical models from Sections 3 and 4 may be written as multivariate generalized linear models πi = h(Xi β) where πi
= (πi1 , . . . , πi,k−1 ), πi,r = P(Yi = r|xi ), h is a link function depending on the model, and Xi is a design matrix. The multinomial model, for example, uses the block design matrix Xi = Diag(x
i , x
i , . . . , x
i ), β
= (β1
, . . . ,
), and link function h = (h1 , . . . , hk ) with βk−1  hr (η1 , . . . , ηk−1 ) = exp(ηr )/(1 + k−1 s=1 exp(ηs )). For more details, see, for example, Fahrmeir and Tutz (1). The log likelihood has the form

l(β) =

n  k−1 

yij log(πij /(1 − πi1 − . . . − πiq ))

i=1 j=1

+ log(1 − πi1 − . . . − πiq ), The Fisher matrix F(β) = E(−∂ 2 l/∂β∂β
) is given by F(β) =

n 

Xi
Wi (β)Xi

i=1

where Wi (β) = {

∂g (πi ) ∂g(π ) i (β) ∂π i }−1 ∂πi
i

with g de-

noting the inverse function of h, g = h−1 , and

6

CATEGORICAL RESPONSE MODELS

i (β) = Diag(πi ) − πi πi
denoting the covariance matrix. Under regularity conditions, one asymptotically obtains β˜ ∼ N(β, F(β)

−1

).

Table 3. Clinical Trial for Curing an Infection (16) Response Center Treatment Success Failure Odds Ratio

Linear hypotheses H0 : Cβ = ζ may be tested by likelihood ratio tests, Wald tests, or score tests in the same way as for binary response models.

2

6

4

3

FURTHER DEVELOPMENTS

The models in the previous sections use the linear predictor ηi = x
i β. This assumption may be weakened by assuming a more flexible additive structure ηi = f(1) (xi1 ) + . . . + f(p) (xip ) where f(j) are unspecified functions. Several methods have been developed to estimate the components f(j) ; see Hastie and Tibshirani (9) and Fan and Gijbels (10) for localizing approaches and Green and Silverman (11) for penalizing approaches. For small sample sizes, asymptotic inference may not be trustworthy. Exact inference, for example, in case control studies is considered in Agresti (12). A short overview on exact inference is found in Agresti (13). In medical research, often continuous variables are converted into categorical variables by grouping values into categories. By using models with flexible predictor structure, categorization may be avoided. Categorization always means loss of information (14), and the choice of cutpoints may yield misleading results [e.g., Altman et al. (15)]. 7

1

AN EXAMPLE

Beitler and Landis (16) considered a clinical trial in which two cream preparations, an active drug and a control, are compared on their success in curing an infection [see also Agresti (12)]. The data were collected at eight centers. The present analysis is restricted to four centers and the data are given in Table 3. By considering the centers as fixed (not as a random sample), the most general logistic model incorporates interactions between treatment (T) and centers (C). Testing the main effect model 1 + T + C against the

Drug Control Drug Control Drug Control Drug Control

11 10 16 22 14 7 2 1

25 27 4 10 5 12 14 16

1.19 1.82 4.80 2.29

Table 4. Estimated Effects for Main Effects Model

1 C2 C3 C4 T

estimate

std error

t-value

tail probability

−1.265 2.028 1.143 −1.412 0.676

0.325 0.419 0.423 0.622 0.340

−3.89 4.83 2.70 −2.133 1.98

.000090 .000001 .00681 .03295 .04676

model with interaction 1 + T + C + T.C yields λ = 2.66 on 4 degrees of freedom. Therefore, the interaction effect may be omitted. Further simplification does not seem advisable as the likelihood ratio test for the relevance of the main effect treatment yields λ = 4.06 on 1 degree of freedom and, for the main effect center, λ = 48.94 on 3 degrees of freedom. The estimates for the main effect model with center 1 as reference category are given in Table 4. The Wald test for the effect of treatment shows significance at the 0.05 level, confirming the result of the likelihood ratio test. REFERENCES 1. L. Fahrmeir and G. Tutz, Multivariate Statistical Modelling based on Generalized Linear Models, 2nd ed. New York: Springer, 2001. 2. P. McCullagh and J. A. Nelder, Generalized Linear Models, 2nd ed. New York: Chapman & Hall, 1989. 3. L. Fahrmeir and H. Kaufmann, Consistency and asymptotic normality of the maximum

CATEGORICAL RESPONSE MODELS likelihood estimator in generalized linear models. Ann. Stat. 1985; 13: 342–368. 4. J. E. Grizzle, C. F. Starmer, and G. G. Koch, Analysis of categorical data by linear models. Biometrika 1969; 28: 137–156. 5. L. Fahrmeir, Asymptotic likelihood inference for nonhomogeneous observations. Statist. Hefte (N.F.) 1987; 28: 81–116. 6. P. McCullagh, Regression model for ordinal data (with discussion). J. Royal Stat. Soc. B 1980; 42: 108–127. ¨ a¨ and J. N. Matthews, The equivalence 7. E. La¨ ar of two models for ordinal data. Biometrika 1985; 72: 206–207. 8. S. Greenland, Alternative models for ordinal logistic regression. Stat. Med. 1994; 13: 1665–1677. 9. T. Hastie and R. Tibshirani, Generalized Additive Models. London: Chapman & Hall, 1990. 10. J. Fan and I. Gijbels, Censored regression: local linear approximation and their applications. J. Amer. Statist. Assoc. 1994; 89: 560–570. 11. D. J. Green and B. W. Silverman, Nonparametric Regression and Generalized Linear Models: A Roughness Penalty Approach. London: Chapman & Hall, 1994. 12. A. Agresti, Categorical Data Analysis. New York: Wiley, 2002. 13. A. Agresti, A survey of exact inference for contingency tables. Stat. Sci. 1992; 7: 131–177. 14. J. Cohen, The cost of dichotomization. Appl. Psycholog. Meas. 1983; 7: 249–253. 15. D. G. Altman, B. Lausen, W. Sauerbrei, and S. Schumacher, Dangers of using ‘‘optimal’’ cutpoints in the evaluation of prognostic factors. J. Nat. Cancer Inst. 1994; 86: 829–835. 16. P. Beitler and J. Landis, A mixed-effects model for categorical data. Biometrics 1985; 41: 991–1000.

7

CAUSAL INFERENCE

Since it is not possible to observe the value of Y t (u) and Y c (u) at the same time, one cannot observe the causal effect of the experiment medicine on subject u. This is considered to be the Fundamental Problem of Causal Inference. To solve this problem, one turns to using a set of units with similar characteristics to infer the average causality. The SUTVA (stable unit treatment value assumption) is essential for ensuring the equivalence between the effect at a unit and the effect in the set of units (the Population). SUTVA assumes (1) no interference between units; and (2) no hidden version of treatment; i.e., no matter how unit u receives a treatment, the observed outcome would be Yt (u) or Y c (u). When special assumptions are added to the above model, causal effect can be inferred. Some of those assumptions are described below, again, in the context of clinical trials.

YILI L. PRITCHETT Abbott Laboratories Abbott Park, Illinois

Causal inference refers to making statistical inference with respect to causality. An effect of a cause is always defined in the context with respect to another cause. Cause is different from attributes; that is, causal relationships are different from associational relationships. Associational inferences use observed data to estimate how different outcomes from the same experimental unit are related. A typical example is that, correlation, the most commonly used statistic to quantify the relation between two variables, does not infer any causation. To evaluate causal effect, an experimental setting is the desired and the simplest setup. Ronald A. Fisher (1) and Jerzy Neyman (2) pioneered the design of experiments for causal inference. Fisher’s sharp null hypothesis testing, using P-values to quantify the strength of the evidence when rejecting a null hypothesis, and his exact test for 2 × 2 tables were among his many significant contributions to causal inference. Neyman first came up with the notation of potential outcome, which provided the foundation for a general framework developed later by Donald B. Rubin (3). Rubin’s causal model can be described in the setting of clinical trials as follows, where ‘‘treatment’’ is the cause of interest. Assume there is a population U of subjects. Let S be a binary variable of treatment assignment. S(u) = t indicates that the subject u receives the experimental medicine, and S(u) = c indicates the control medicine. S(u) for each subject could have been different. Let Y(u) be the outcome measure of the clinical trial. On each subject u, both Yt (u) and Yc (u) are potential outcomes. However, only one of those will be observed in the study, depending on which treatment the subject is exposed to. The causal effect of t on subject u is defined as Yt (u) − Yc (u)

1. Temporal Stability and Causal Transience Assumptions. Temporal stability assumes consistency of potential outcomes over time; that is, a subject’s outcome is independent from when the treatment is given. Causal transience assumes that the previous treatment effect under c is transient so that it has no impact on the potential outcome under current treatment of t, and vise versa. Under these two assumptions, it is plausible that one can expose a subject to different treatments in sequence, t and c, and observe outcomes under each treatment. The causal effect can then be estimated by Y t (u) − Yc (u). These two assumptions are the fundamental assumptions supporting the validity of using crossover design to evaluate causal effect. 2. Unit Homogeneity Assumption. Assume that Yt is homogeneous over different subjects, and so is Yc . Thus, the causal effect of t can be estimated as Yt (u1 ) and Yc (u2 ). Case-control studies are operated under this assumption (4). In those types of studies, to evaluate a cause effect, subjects with similar (if not identical) demographic and base-

(1)

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

CAUSAL INFERENCE

line characteristics are matched with only an exception for the existence of a condition, which is under investigation as a potential cause. The outcome measures are compared between these two groups of subjects, and the difference will be attributed to the only difference of possessing that specific condition. Unit homogeneity assumption implies a weaker assumption than the constant effect assumption, which assumes that the cause effect is a constant over the population. 3. Independence Assumption. Denote the average causal effect of T on population U as T = E(Yt − Yc ) = E(Yt ) − E(Yc )

(2)

If the average potential outcome of Yt and Yc on the population can be estimated using observed data, the average causal effect T can then be estimated. In parallel group design, at least half of Yt (u) and Yc (u) are missing. What one can estimate is E(Yt |S = t), which is the conditional average of outcome Yt over those who receive t. Similarly, it is also estimable for E(Yc |S = c). When subjects are assigned at random to either group by a certain physical mechanism, the determination of which treatment the subject is exposed to is regarded as statistically independent of all other variables over U, including outcomes Yt and Yc . That is, in a randomized clinical trial, if the randomization procedure is carried out correctly, it is plausible that S is independent of Yt and Yc . Therefore, we have E(Yt ) = E(Ys |S = t) E(Yc ) = E(Ys |S = c) The above equations imply

estimated by the between-difference in sample means as has been routinely performed in parallel design, randomized clinical trials. Holland (5) introduced the concept of prima facie causal effect, TPF , an associational parameter for the joint distribution of (YS , S), as TPF = E(YS |S = t) − E(YS |S = c)

(4)

In general, TPF and T are not equal, if the independence assumption is not true. Randomized experiments play an important role in causal inference with the reason as shown above. In nonrandomized experiments, Equation (3) no longer holds true, because the units exposed to one treatment generally differ systematically from the units exposed to the other treatment. That is, the sample means are confounded with selection bias, and they cannot be used directly to infer to causal effect. A solution to this problem is to create a balancing score b(X), which is a function of observed covariates X such that the treatment assignment is independent to X conditioning on b(X). Therefore, causal effect can be evaluated at each stratum characterized by b(X). The average effect over b(X) can be used to estimate the average cause effect on the population. In that approach, the unit-level probability of treatment assignment is called the propensity score (6). Causal inference highly relies on how the treatment is assigned to the experimental units. Using the concept of ‘‘assignment mechanism,’’ where the probability that a specific treatment is assigned to an experimental unit is modeled, randomized, nonrandomized, and response adaptive randomized experiments can be characterized within the same framework. To illustrate, let Pr(S = t|X,YS ,YC ) denote the assignment mechanism for a clinical trial and X the covariates, YS and YC are the potential outcomes, and YOBS are the accumulate observed outcomes. We have

T = E(Ye ) − E(Yc ) (3)

• Pr(S = t|X,YS ,YC ) = Pr(S = t), when this

That is, when the independence assumption holds true, the average causal effect T can be

• Pr(S = t|X,YS ,YC ) = Pr(S = t|X), when

= E[YS |S = t] − E[YS |S = c]

is a classic randomized design. this is a nonrandomized design.

CAUSAL INFERENCE • Pr(S = t|X,YS ,YC ) = Pr(S = t|YOBS ),

when this is a response-adaptive, randomized design. In this type of design, the probability of assigning treatment t is not a constant any more, but it is dynamically adapting based on the observed response data. Of note, causal inference is not the same in concept as causal modeling that has been extensively used in social science. In causal modeling, path diagrams are used to describe the relationships among variables that have plausible causal or associational relationships. For example, studying the relationship between age and job satisfaction (7), a plausible assumption was ‘‘the older the employee gets, the more satisfied he is with his job.’’ When other intervening factors were also taken into consideration, such as the responsibility and the income associated with the job, the relation between age and job satisfaction became more complicated. One could use a path diagram to put those plausible relationships together and could use regression approaches to find the answer. That is, the coefficients between variables estimated from regression models could be used to quantify the relationship from age to Job satisfaction on each potential pathway. The sum of the coefficients among several pathways provided a quantitative measure on the relationship between the two variables. One can see that the two variables of interests in this example were not causal in nature. This example demonstrated that the application of causal modeling could deliver answers for associational inference but not necessary for causal inference. However, if all variables included in the causal modeling are causal factors, then causal modeling can be used as one of the tools to deduct the causal effects. REFERENCES 1. R. A. Fisher, The Design of Experiments. Edinburgh: Oliver & Boyd, 1935. 2. J. Neyman, On the application of probability theory to agricultural experiments. In: Essay on Principles. Section 9 (in Polish). Reprinted in English in Stat. Sci. 1990; 5:463– 480.

3

3. D. B. Rubin, Estimating causal effects of treatments in randomized and nonrandomized studies. J. Educ. Psychol. 1974; 66:688– 701. 4. D. B. Rubin, Matched Sampling for Causal Effects. Cambridge, UK: Cambridge University Press, 2006. 5. P. W. Holland, Statistics and causal inference (with discussion). J. Am. Stat. Assoc. 1986; 81:945 − 970. 6. P. R. Rosenbaum and D. B. Rubin, The central role of the propensity score in observational studies for causal effects. Biometrika 1983; 70:41 − 55. 7. R. D. Retherford and M. K. Choe, Statistical Models for Causal Analysis. New York: Wiley, 1993.

CELL LINE

grow for long periods in culture if they are fed regularly under aseptic conditions (3). It was not until the 1940s that evidence began to develop to indicate that vertebrate cells could be maintained indefinitely in culture. Wilton Earle, at the National Cancer Institute, isolated single cells of the L cell line and showed that they form clones of cells in tissue culture (4). At around the same time, Gey et al. (5) were attempting to establish continuous cultures from rat and human tissue. One of their successes was the culture of continuous line of cells from human cervical carcinoma, the cell line that later become the well-known HeLa cell line (5). The cultivation of human cells received another stimulus by the development of different serum-free selective media as well as by the greater control of contamination with antibiotics and clean–air equipment.

THERU A. SIVAKUMARAN SUDHA K. IYENGAR Case Western Reserve University Department of Epidemiology and Biostatistics Cleveland, Ohio

Culturing of cells is a process whereby cells derived from various tissues are grown with appropriate nutrients under appropriate environmental conditions. Cell cultures and lines can be used as an alternative approach to the use of live animals for studying models of physiological function. A cell line, which is formed after first subculture of the primary culture, is a permanently established cell culture that can proliferate indefinitely under appropriate conditions. The formation of a cell line from primary culture implies an increase in the total number of cells over several generations as well as the ultimate predominance of the cells or cell lineage with the capacity for high growth, which results in a degree of uniformity in the cell population. Although all the cells in a cell line are very similar, they may have similar or distinct phenotypes. If one cell lineage is selected to have certain specific properties in the bulk of cells in the culture, this becomes cell strain. 1

2

PRIMARY CELL CULTURE

The primary culture is the stage that starts from isolation cells, is applied until the culture reaches confluence, and is split into multiple cultures for the first time. At this stage, the cells are usually heterogeneous but still represent the parent cell types and expression of tissue-specific properties. Primary cell culture can be obtained by one of three methods:

CELL CULTURE: BACKGROUND

2.1 Mechanical Disaggregation The tissue is disaggregated by chopping with a scalpel or by forcing the tissue through a mesh screen or syringe needle. The resulting suspension of cells is allowed to attach to an appropriate substrate to enable growth.

The concept of maintaining the live cell that was separated from original tissue was discovered in the nineteenth century. In late 1885, Wilhelm Roux (1) established the principle of tissue culture by demonstrating that embryonic chick cells can be maintained outside the animal body in a saline solution for several days. The first success in culturing animal cells is attributed to Harrison (2), who showed that the spinal cord from frog embryo could be cultured in a natural medium of clotted frog lymph; he also observed outgrowth of nerve fibers from the explant. The man who was responsible for establishing the idea of the cell culture was Alexis Carrel. He showed that cells could

2.2 Enzymatic Disaggregation Cell–cell and cell–substrate adhesion is generally mediated by three major classes of transmembrane proteins: the cell–cell adhesion molecules, cell–substrate interactions molecules, and transmembrane proteoglycans. Proteolytic enzymes, such as trypsin, collagenase, and so on, are usually added to the tissue fragments to dissolve the cement that holds the cells together, and enzymatic

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

CELL LINE

disaggreation is the most widely used method to isolate cells for primary culture. 2.3 Primary Explanation This method was originally developed by Harrison (2) for initiating cell culture, and it is useful when a small amount of tissue is available. The tissue is finely chopped and rinsed, and the small fragments are seeded onto the surface of the culture vessel so it will adhere. Adherence is facilitated by scratching the plastic surface or trapping the tissue under a coverslip, or by use of surface tension created by adding a small volume of medium with a high concentration of serum or clotted plasma. Once adhesion is achieved, the primary culture develops by outgrowth from the explant. The tendency of animal cells in vivo to interact with one another and with the surrounding extracellular matrix is mimicked in their growth in culture. After tissue aggregation or subculture, most normal cells, with the exception of hematopoietic cells, need to spread out on a substrate to proliferate, and the cells are said to be anchorage dependent. Cells that can grow in suspension such as hematopoietic cells, tumor cells, transformed cell lines, and so on, are known as adherent independent. Cell adhesion is known to be mediated by specific cell surface receptor molecules in the extracellular matrix, and it is likely that secretion of extracellular matrix protein and proteoglycans may precede spreading by the cells. These matrix components bind to the culture surface and function as a bridge between the surface and the cells. Hence, the culture surface can be conditioned by treating it with spent medium from another culture or by purified fibronectin or collagen. 3

SUBCULTURE

When cells in culture have grown and filled up the entire available surface, they must be transferred, subcultured, or passaged into fresh culture vessels to give them room for continued growth. The subculture process involves removal of the culture medium and addition of trypsin to dislodge the cells from the substrate and to disrupt the cell–cell

connections. When the primary cell culture is thus subcultured, it generates a secondary culture that becomes known as a cell line. Each cell line can be subcultured several times such that a secondary subculture can then yield to tertiary culture and so on; this nomenclature is seldom used beyond the tertiary culture. With each successive subculture, the components of the cell population with the ability to proliferate most rapidly will gradually predominate, and nonproliferating or slowly proliferating cells will be diluted out. This finding is apparent after first subculture, in which differences in proliferative capacity of cells is compounded by varying abilities and transfer. To assess when a cell line will require subculture or how long cells will take to reach a certain density, it is necessary to become familiar with the cell cycle for each cell line, as cells at different phases of the growth cycle proliferate at different rates and show differences in respiration, synthesis of a variety of products, enzyme activity, and so on. This growth cycle is typically divided into three phases (6). 3.1 The Lag Phase When cells are taken from a stationary culture, a lag occurs before growth begins (this lag is the time that follows reseeding after subculture) (Fig. 1). During this time, cells replace elements of the cell surface, and the extracellular matrix that is lost during trypsinization adheres to the substrate and spreads out. 3.2 The Log Phase The log phase is the phase when the cells grow and the population doubles. The length of the log phase depends on the density at which the cells were seeded (initial culture), the growth rate of the cells, and the final concentration that will inhibit additional cell proliferation. The log phase, when the growth rate is as high as 90–100%, is the optimal time for using cells for experiments as the culture is in its most reproducible form. 3.3 The Plateau Phase At the end of the log phase, the entire available growth surface is occupied, and every

CELL LINE

3

Saturation density 106

Cells/cm2

Cells/ml

105

105

Figure 1. Three different phases of the growth cycle of cultured cells. Reproduced from Freshney’s Culture of animal cells: A manual of basic technique (6).

104

cell is in contact with its surroundings; this phenomenon is known as confluence. As a result, the growth rate slows and eventually may stop; specialized proteins may now be synthesized faster than during other phases. The growth rate in the plateau phase drops to 0–10%. Because the transformed cells are deficient in contact inhibition of cell motility and density limitation of cell proliferation, they will continue to proliferate for several generations even after reaching confluence, and they will reach higher saturation densities at plateau. The plateau phase of these cultures is a balance between increased proliferation and cell loss from apoptosis, which results in a greater turnover than occurs in normal cells at plateau. The construction of a growth curve from cell counts made at regular intervals after initiation of the subculture enables the measurements of parameters, such as lag period or lag time as well as population doubling time and saturation density, that determine the characteristics of the cell line under a given set of culture conditions. When these parameters are consistent, they can be used to calculate variables such as split ratio. The latter is defined as the degree of dilution required to reinitiate a new growth cycle with a short lag period and to achieve the appropriate density for subculture at a convenient time in the future.

104

Doubling time

Seeding concentration Lag 0

4

2

4 6 8 Days from subculture

10

EVOLUTION OF CELL LINES

The cell lines that are derived from normal tissue can divide only for a limited number of times, and their lifespan is determined by the cell type as well as by the environmental conditions at the first subculture. If the medium, substrate, and other conditions are appropriate, then the cell line may progress through several serial subcultures (Fig. 2). Conversely, if the conditions are not adequate, then the cell line will die out within one or two subcultures. Even for cell lines grown under satisfactory conditions, the maximum number of attainable subcultures (passages) is fixed. After a few passages, cells enter a state where they no longer divide, which is known as replicative senescence. For example, normal human fibroblasts typically divide only 25–40 times in culture before they stop. Senescence is a genetic event determined by the inability of terminal DNA sequences in the telomeres of each chromosome to replicate at each cell division, which results in progressive shortening of telomeres, until the cell cannot divide any more (7). Human somatic cells have turned off the enzyme, called telomerase, that normally maintains the telomeres and that is why their telomeres shorten with each cell division. The cell lines that only provide a limited number of population doublings are called finite cell lines (6). Some cell lines, notably those from rodents and most tumors, are not limited by a finite

4

CELL LINE

Transformation

1020

Cumulative Cell Number

1018

Primary Culture

Continuous Cell Line

Finite Cell line

1016

1st Subculture 2nd Subculture

1014

Senescence and Cell Death

1012 1010

Subculture Interval 108 Serial Passage 106

0

Explantation

2

4

6 8 10 Weeks in Culture

12

14

100

Figure 2. Evolution of a cell line. Reproduced from Freshney’s Culture of animal cells: A manual of basic technique (6).

lifespan, and they can continue to divide indefinitely. These cell lines are called continuous cell lines. The ability of growing continuously is reflective of the action of specific genetic variations, such as deletions or mutations of the p53 gene and overexpression of the telomerase gene. 5 DEVELOPMENT OF IMMORTALIZED CELL LINES The use of normal human cells in biomedical research is restricted in part by the limited proliferative potential of primary cultures. Thus, researchers need to re-establish fresh cultures from explanted tissue frequently, which is a tedious process. To use the same cells throughout an ongoing research project, primary cells need to extend their replicative capacity or undergo immortalization. Some cells immortalize spontaneously by passing through replicative senescence and thus adapt easily to life in culture. However, these spontaneously immortalized cells invariably have unstable genotypes and are host to numerous genetic mutations, which renders them less reliable representatives of their starting tissue’s phenotype. Many applications would require having essentially unlimited numbers of carefully characterized cells

with the desired phenotypes. Therefore, the ideal immortalization protocol would produce cells that are capable not only of extended proliferation, but also possess the same genotype and tissue markers of their parental tissue. The most commonly used method to attain this objective is the transduction of normal cells, before they enter senescence, with genes from DNA tumor virus, such as simian virus 40 (SV40), a human papillomavirus (HPV), or Epstein-Barr virus (EBV). 5.1.1 SV40. The major region of SV40 oncoprotein encoded by early region DNA, which is the larger T antigen (LT) gene, inactivates the p53 and Rb proteins. Cells that express these oncoproteins proliferate beyond the point of senescence. Transduction of normal cells with the SV40 early region genes by transfection of an expression plasmid remains a very common immortalization technique for almost any human cell type (8). 5.1.2 HPV. The E6 and E7 oncogenes of high-risk oncogenic risk HPV strains, which are HPV types 16 and 18, have been shown to assist the immortalization of several different cell types. The E6 protein causes degradation of the p53 protein, it upregulates the

CELL LINE

c-myc expression, and it also partially activates the telomerase (9). The HPV16 E7 protein induces degradation of the Rb protein via ubiquitin-proteasome pathway, and this mechanism explains the immortalization ability of E7 gene independently of E6 gene (10). 5.1.3 EBV. This herpes virus has the unique ability of transforming B lymphocytes into permanent, latently infected lymphoblastoid cell lines. Every infected cell carries multiple extrachromosomal copies of the viral episome and constitutively expresses a limited set of viral gene products, which are called latent proteins, that comprise six EBV nuclear antigens (EBNAs 1, 2, 3A, 3B, 3 C, and -LP) and three latent membrane proteins (LMPs 1, 2A, and 2B). The latent membrane protein 1 and EBNA1 are likely to be required for EBV to immortalize B lymphocytes. They have been shown to mediate the increase in p53 levels via activation of the NF-kB transcription factor (11). The process of transfection or infection extends the proliferative capacity of cells to 20–30 population doublings before entering a state called crisis. In some cultures, very few cells (about 1 per 107 ) recommence proliferation after a variable period of crisis, probably because of additional genetic changes, and form an immortalized cell line. The limitation of transformation with DNA tumor viruses is that the proteins they encode may cause undesirable changes in the resulting immortalized cell lines (e.g., loss of some differentiated properties and loss of normal cellcycle checkpoint controls). These unwanted changes can be avoided by direct induction of telomerase activity. This activity is done by transfecting the cells with the telomerase gene htrt, which thereby extends the life span of the cell line such that a proportion of these cells become immortal but not malignantly transformed (7). 5.2 Characterization and Authentication of Cell Lines Research that involves the use of cell lines demands the precise knowledge of the purity and species of origin of the working cell lines. In the facilities where multiple cell

5

lines are maintained, the possibility of cross contamination can occur. In some cases, the contaminating colony of cells in the culture may be identified visually. In other cases, this unintentional coculture cannot be determined by visual inspection. Therefore, it is important to verify the identity and purity of cell cultures periodically. In the absence of such a monitoring system, it is possible that interspecies and intraspecies cell line contamination may occur, which results in the generation of false conclusions. Apart from this cross contamination, most human cell lines are prone to various sorts of genetic rearrangements, such as chromosomal aneuploidy, deletion of chromosomal regions, and so on, which affect the biochemical, regulatory, and other phenotypic features of cells during their cultivation. Therefore, it would be very useful to monitor the authenticity of the cell lines permanently and/or to have a comprehensive set of standard tests for confirming their cellular identity. Finally, instances occur when hybrid cells lines from two species are created, which are called somatic cell hybrids (e.g., culture lines that are developed with a mix of human and mouse cells through Sendai virus fusion). These types of hybrids predominantly retain the genome of one of the species but also keep some chromosomal complement from the alternate species. Herein, it would be necessary to determine which species contributed which element of the genomic complement. The standard approach that is generally followed is derivation of multiple immunological and genetic marker system profiles from each cell culture to be tested. Once an early molecular signature of the cells is obtained, it can later be compared with all future cell lines for identity validation. Some methods used to derive these profiles are described below. 5.2.1 Isoenzyme Analysis. Isoenzymes are enzymes that exhibit interspecies and intraspecies polymorphisms. Isoenzymes have similar catalytic properties but differ in their structure, and they can be detected based on their differences in electrophoretic mobility. By studying the isoenzymes present in the cell lines, it would be possible

6

CELL LINE

to identify the species from which the cell line was derived. 5.2.2 Chromosomal Analysis. Chromosome content is one of the best-defined criteria for differentiating cell lines derived from more than one species and sex. Chromosomal banding patterns (e.g., trypsin-Giemsa banding or G-banding) can distinguish between normal and malignant cells and can also distinguish between human and mouse chromosomes. Thus, this technology is used frequently in cell line characterization. Fluorescence in situ hybridization is another powerful and rapid method for detecting aneuploidy, chromosomal deletions, and so on. Using different combinations of chromosomeand species-specific probes to hybridize to individual chromosomes, it would be possible to identify chromosomal fragments in metaphases of interspecies hybrid cells. 5.2.3 DNA Profiling. This method is based on the existence of dispersed hypervariable regions of tandem-repetitive nucleotide sequences in the genome (12). The original approach is based on using multilocus Variable Number of Tandem Repeats (VNTR) probes and Southern blotting, which is a DNA analysis technique that works on the principle that smaller fragments of DNA electrophorese at rates faster than larger fragments of DNA. When the DNA from a cell line analyzed by Southern blotting with probes targeted to single locus, many unique size fragments characteristic of many different alleles encoding that particular locus are identified. Given the high degree of heterogeneity at every locus, each cell line produces two specific bands. After examining many such VNTR loci, application of multiple single-locus probes to a single enzyme yields a unique cell line profile. This profile is cell line specific unless more than one cell line is derived from the same individual or if highly inbred donor subjects have been used. More recently, other techniques that can produce profiles more rapidly have been advocated. Thus, the second approach is a more rapid method that involves amplification of microsatellite markers using species-specific primers by polymerase chain reaction; this method is used only for human cell lines. The

banding pattern produced by electrophoretic separation of amplified DNA products can be compared with archived profiles on a database, which is similar to what is done for forensic profiling. Like multilocus VNTR analysis, this technique can also be used to demonstrate continuity between master and working banks and to detect contamination and instability in cell lines. 5.3 Cell Banks Advances in cell technology enable scientists to isolate and cultivate any tissue cells from any species of interest. Alternatively, wellcharacterized cell lines can be obtained from cell banks. Several cell banks such as American Type Culture Collection (ATCC), European Collection for Animal Cell Cultures, Coriell Institute for Medical Research, and so on, acquire, preserve, authenticate, and distribute reference cell lines to the scientific communities. Currently, the ATCC house over 4000 cell lines derived from over 150 different species (Table 1). Once obtained, cell lines can be preserved in cell banks for future use.

REFERENCES 1. J. Paul, Achievement and challenge. In: C. Barigozzi (ed.), Progress in Clinical and Biological Research, vol. 26. New York: Alan R Liss, Inc., 1977, pp. 3–10. 2. A. G. Harrison, Observations on the living developing nerve fiber. Proc. Soc. Exp. Biol. Med. 1907; 4: 140–143. 3. A. Carrel, Artificial activation of the growth in vitro of connective tissue. J. Exp. Med. 1913; 17: 14–19. 4. W. R. Earle, E. L. Schilling, T. H. Stark, N. P. Straus, M. F. Brown, and E. Shelton, Production of malignacy in vitro; IV: the mouse fibroblast cultures and changes seen in the living cells. J. Natl. Cancer Inst. 1943; 4: 165–212. 5. G. O. Gey, W. D. Coffman, and M. T. Kubicek, Tissue culture studies of the proliferative capacity of cervical carcinoma and normal epithelium. Cancer Res. 1952; 12: 364–365. 6. R. I. Freshney, Culture of Animal Cells: A Manual of Basic Technique. New York: WileyLiss, 2000.

CELL LINE

7

Table 1. List of Some Commonly used Cell Lines Cell Line

Cell Type

Origin

Age

MRC-5 MRC-9 BHK21-C13 HeLa 293 3 T3-A31 CHO-K1 WI-38 ARPE-19 C2 BRL3A NRK49F A2780 A9 B16 MOG-G-CCM SK/HEP-1 Caco-2 HL-60 Friend ZR-75-1 C1300 HT-29 KB Vero

Fibroblast Fibroblast Fibroblast Epithelial Epithelial Fibroblast Fibroblast Fibroblast Epithelial Fibroblastoid Fibroblast Fibroblast Epithelial Fibroblast Fibroblastoid Epithelioid Endothelial Epithelial Suspension Suspension Epithelial Neuronal Epithelial Epithelial Fibroblast

Human lung Human lung Syrian hamster kidney Human cervix Human kidney Mouse BALB/c Chinese hamster ovary Human lung Human retina (RPE) Mouse Skeletal muscle Rat liver Rat kidney Human ovary Mouse subcutaneous Mouse melanoma Human glioma Human hepatoma Human colon Human myeloid leukemia Mouse spleen Human breast, ascites fluid Rat neuroblastoma Human colon Human oral Monkey kidney

Embryonic Embryonic Newborn Adult Embryonic Embryonic Adult Embryonic Adult Embryonic Newborn Adult Adult Adult Adult Adult Adult Adult Adult Adult Adult Adult Adult Adult Adult

7. A. G. Bodnar, M. Ouellette, M. Frolkis, S. E. Holt, C. P. Chiu, G. B. Morin, et al., Extension of life-span by introduction of telomerase into normal human cells. Science 1998; 279: 349–352. 8. L. V. Mayne, T. N. C. Price, K. Moorwood, and J. F. Burke, Development of immortal human fibroblast cell lines. In: R. I. Freshney and N. R. Cooper (eds.), Culture of Immortalized Cells. New York: Wiley-Liss, 1996, pp. 77–93. 9. A. J. Klingelhutz, S. A. Foster, and J. K. McDougall, Telomerase activation by the E6 gene product of human papillomavirus type 16. Nature 1996; 380: 79–82. 10. S. N. Boyer, D. E. Wazer, and V. Band, E7 protein of human papilloma virus-16 induces degradation of retinoblastoma protein through the ubiquitin-proteasome pathway. Cancer Res. 1996; 56: 4620–4624. 11. W. P. Chen and N. R. Cooper, Epstein-Barr virus nuclear antigen 2 and latent membrane protein independently transactivate p53 through induction of NF-kappa B activity. J. Virol. 1996; 70: 4849–4853. 12. A. J. Jeffreys, V. Wilson, and S. L. Thein, Individual-specific fingerprints of human

DNA. Nature 1985; 316: 76–79.

FURTHER READING J. P. Mather and D. Barnes, eds. Animal cell culture methods. In: Methods in Cell Biology, vol. 57. San Diego, CA: Academic Press, 1997.

CROSS-REFERENCES In vitro/in vivo correlation Pharmacokinetic study Toxicity Pharmacogenomics Microarray DNA Bank

CENSORED DATA

1. The presence of censoring may alter the hazard function of the lifetime X i , i.e. the conditional distribution of X i , given that i is alive at t(X i ≥ t) and uncensored at t (U i ≥ t), may be different from what it was in the uncensored case, i.e. just given X i ≥ t (dependent censoring). 2. The observed right-censoring times, U i , may contain information on θ (informative censoring).

PER KRAGH ANDERSEN University of Copenhagen, Copenhagen, Denmark

In classical statistics, the observations are frequently assumed to include independent random variables X 1 , . . . , X n , with X i having the density function

An example of a dependent censoring scheme would be if, in a clinical trial with survival times as the outcome variables, one removed patients from the study while still alive and when they appeared to be particularly ill (or particularly well), so that patients remaining at risk are not representative of the group that would have been observed in the absence of censoring. In other words, dependent censoring represents a dynamic version of what in an epidemiologic context would be termed a selection bias. An example is provided below (Example 1). Mathematical formulations of independent censoring (conditions on the joint distribution of X i and U i ) may be given, and it may be shown that several frequently used models for the generation of the times of right-censoring satisfy these conditions. The difficulty in a given practical context lies in the fact that the conditions may be impossible to verify, since they refer to quite hypothetical situations. The second concept mentioned, noninformative censoring, is simpler and relates to the fact that if censoring is informative, then a more efficient inference on θ may be obtained than the one based on (2); see below.

fiθ (x) = αiθ (x)Sθi (x), where αiθ (x) is the hazard function, Sθi (x) is the survival function, and θ is a vector of unknown parameters. Then inference on θ may be based on the likelihood function, L(θ ) =




fiθ (Xi ),

i

in the usual way. In survival analysis, however, one can rarely avoid various kinds of incomplete observation. The most common form of this is right-censoring where the observations are (X˜ i , Di ),

i = 1, . . . , n,

(1)

where Di is the indicator I{X˜ i = Xi }, and X˜ i = Xi , the true survival time, if the observation of the lifetime of i is uncensored and X˜ i = Ui , the time of right-censoring, otherwise. Thus, Di = 1 indicates an uncensored observation, Di = 0 corresponds to a right-censored observation. Other kinds of incomplete observation will be discussed below. Survival analysis, then, deals with ways in which inference on θ may be performed based on the censored sample (1). We would like to use the function Lc (θ ) =




1

The general definition of independent censoring given by Andersen et al. 2, Section III.2.2 for multivariate counting processes has the following interpretation for the special case of survival analysis with time-fixed covariates. The basic (uncensored) model is that conditional on covariates Z = (Z1 , . . . , Zn ) the lifetimes X 1 , . . . , X n are independent, X i

αiθ (X˜ i )Di Sθi (X˜ i )

i

=




fiθ (X˜ i )Di Sθi (X˜ i )1−Di

INDEPENDENT CENSORING

(2)

i

for inference, but there are two basic problems:

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

CENSORED DATA

having the hazard function αiθ (t|Zi ) ≈ Pθ φ (Xi ∈ Idt |Xi ≥ t, Z)/dt.

(3)

Here, Idt is the interval [t, t + dt) and Pθ φ is the joint distribution of X 1 , . . . , X n , Z and the censoring times. Note that the hazard function only depends on θ , i.e. φ is a nuisance parameter. Because of the conditional independence of X i it follows that Pθ φ (Xi ∈ Idt |Ft− ) ≈ αiθ (t|Zi )I{Xi ≥ t}dt, where the history Ft− contains Z and all information on X 1 , . . . , X n from the interval [0, t), i.e. values of X i for i with X i < t and the information that X j ≥ t for j with X j ≥ t. Let there now be given right-censoring times U 1 , . . . , U n and define the enlarged history Gt as the one containing Ft and all information on U 1 , . . . , U n from the interval [0, t], i.e. values of U i ≤ t and the information that U j ≥ t for those j where U j ≥ t. The condition for independent censoring is then that Pθ φ (Xi ∈ Idt |Ft− ) = Pθ φ (Xi ∈ Idt |Gt− ).

(4)

It follows that simple type I censoring, where all U i are equal to a fixed time, u0 , and simple type II censoring, where all U i are equal to the kth smallest lifetime X (k) for some k between 1 and n, are both independent, since the rightcensoring times in these cases give rise to no extra randomness in the model; that is, Ft = Gt . In some models, U 1 , . . . , U n are assumed to be independent given Z and Z1 , . . . , Zn are independent identically distributed (iid). Then the assumption (4) reduces to αiθ (t|Zi ) ≈ Pθ φ (Xi ∈ Idt |Xi ≥ t, Ui ≥ t, Z)/dt (5) and it is fulfilled, e.g. if U i and X i are independent given Zi . This is, for instance, the case in the simple random censorship model where U 1 , . . . , U n are iid and independent of X1, . . . , Xn. Some authors take the condition (5) (which is less restrictive than (4)) as the definition of independent censoring; see, for example, 6, p. 128. However, (4) may be generalized to other models based on counting processes

and both (4) and (5) cover the most frequently used mathematical models for the right-censoring mechanisms. These include both the models already mentioned, i.e. simple type I, type II and random censorship and various generalizations of these (e.g. progressive type I censorship (cf. Example 2, below), general random censorship, and randomized progressive type II censorship; see, [2, Section III.2.2]). Earlier contributions to the definition and discussion of independent censoring are the monographs by Kalbfleisch & Prentice 13, p. 120 and Gill 7, Theorem 3.1.1 and the papers by Cox 5, Williams & Lagakos 16, Kalbfleisch & MacKay 12 and Arjas & Haara 3, all of whom give definitions that are close or equivalent to (5). Another condition for independent censoring, stronger than (5) but different from (4), is discussed by Jacobsen 11. From (4) and (5) it is seen that censoring is allowed to depend on covariates as long as these are included in the model for the hazard function of the lifetime distribution in (3). Thus, an example of a dependent censoring scheme is one where the distribution of U i depends on some covariates that are not included there. This is illustrated in the following example. 1.1 Example 1: Censoring Depending on Covariates Suppose that iid binary covariates, Z1 , . . . , Zn , have Pθ φ (Zi = 1) = 1 − Pθ φ (Zi = 0) = φ, and that X 1 , . . . , X n are iid with survival function S(t). The Kaplan–Meier estima based on the X i then provides a tor S(t) consistent estimate of θ = S(·), the marginal distribution of X i . This may be written as S(t) = φS1 (t) + (1 − φ)S0 (t), where Sj (t), for j = 0, 1, is the conditional distribution given Zi = j. Note that these may be different, e.g. S1 (t) < S0 (t) if individuals with Zi = 1 are at higher risk than those with Zi = 0. Define now the right-censoring times U i by Ui = u0 , if Zi = 1,

Ui = +∞, if Zi = 0.

CENSORED DATA

Then, for t < u0 the Kaplan–Meier estimator will still consistently estimate S(t), while for 
S(u0 ) will estimate S0 (t)/S0 (u0 ). t > u0 , S(t)/ If, however, the covariate is included in the model for the distribution of X i , i.e. θ = [S0 (·), S1 (·)], then S j (t), the Kaplan–Meier estimator based on individuals with Zi = j, j = 0, 1, will consistently estimate the corresponding Sj (t), also based on the right-censored sample (though, of course, no information will be provided about S1 (t) for t > u0 ). It is seen that censoring is allowed to depend on the past and on external (in the sense of conditionally independent) random variation. This means that if, in a lifetime study, sex and age are included as covariates, then a right-censoring scheme, where, say, every year, one out of the two oldest women still alive and uncensored is randomly (e.g. by flipping a coin) chosen to be censored, is independent. However, a right-censoring scheme depending on the future is dependent. This is illustrated in the following example. 1.2 Example 2: Censoring Depending on the Future Suppose that, in a clinical trial, patients are accrued at calendar times T 1 , . . . , T n and that they have iid lifetimes X 1 , . . . , X n (since entry) independent of the entry times. The study is terminated at calendar time t0 and the entry times are included in the observed history, i.e. Zi = T i in the above notation. If, at t0 , all patients are traced and those still alive are censored (at times U i = t0 − T i ) and, for those who have died, their respective lifetimes, X i , are recorded, then this right-censoring is independent (being deterministic, given the entry times, so-called progressive type I censoring). Consider now, instead, the situation where patients are only seen, for instance, every year, i.e. at times T i + 1, . . . , T i + ki ≤ t0 and suppose that if a patient does not show up at a scheduled follow-up time, then this is because he or she has died since last follow-up and the survival time is obtained. Suppose, further, that for the patients who are alive at the time, T i + ki , of their last scheduled follow-up, and who die before time t0 , there is a certain probability, φ, of obtaining information on the failure, whereas for those

3

who survive past t0 nothing new is learnt. If these extra survival times are included in the analysis and if everyone else is censored at ki , then the right-censoring scheme is dependent. This is because the fact that patient i is censored at ki tells the investigator that this patient is likely not to die before t0 and the right-censoring, therefore, depends on the future. To be precise, if the average probability of surviving past t0 , given survival until the last scheduled follow-up time is 1 − π , then the probability of surviving past t0 , given censoring at the time of the last scheduled follow-up, is (1 − π )/[π (1 − φ) + 1 − π ], which is 1 if φ = 1, 1 − π if φ = 0, and between 1 − π and 1, otherwise. If, alternatively, everyone still alive at time T i + ki were censored at ki , then the censoring would be independent (again being deterministic given the entry times). Another censoring scheme that may depend on the future relative to ‘‘time on study’’, but not relative to calendar time, occurs in connection with testing with replacement, see, for example, (8). Let us finally in this section discuss the relation between independent rightcensoring and competing risks. A competing risks model with two causes of failure, d and c, is an inhomogeneous Markov process W(·) with a transient state 0 (‘‘alive’’), two absorbing states d and c and two causespecific hazard functions α 0d (t) and α 0c (t), e.g. Andersen et al. 1. This generates two random variables: X = inf[t : W(t) = d] and U = inf[t : W(t) = c], which are incompletely observed since the observations consist of the transition time  = X∧U and the state W(X)  = d or c reached X at that time. The elusive concept of ‘‘independent competing risks’’ (e.g. [13, Section 7.2]) now states that in a population where the risk c is not operating, the hazard function for d is still given by α 0d (t). This condition is seen to be equivalent to censoring by U being independent. However, since the population where a given cause of failure is eliminated is

4

CENSORED DATA

usually completely hypothetical in a biological context, this formal equivalence between the two concepts is of little help in a practical situation and, as is well known from the competing risks literature (e.g. 4,15, and [13, Chapter 7]), statistical independence of the random variables X and U cannot be tested  W(X)].  from the incomplete observations [X, What can be said about the inference on the parameter θ = α 0d (·) based on these data is that consistent estimation of θ may be obtained by formally treating failures from cause c as right-censorings, but that this parameter has no interpretation as the d failure rate one would have had in the hypothetical situation where the cause c did not operate. For the concept of independent censoring to make sense, the ‘‘uncensored experiment’’ described in the beginning of this section should, therefore, be meaningful. 2 LIKELIHOODS: NONINFORMATIVE CENSORING The right-censored data will usually consist of i , Di , Zi ; i = 1, . . . , n) (X and, under independent censoring, the likelihood can then be written using productintegral notation L(θ , φ) = Pθ φ (Z)



i

αiθ (t)Di (dt)

t0 θφ

× [1 − αiθ (t)dt]1−Di (dt) γi (t)Ci (dt) θφ

× [1 − γi (t)dt]1−Ci (dt) .

(6)

Here, Di (dt) = I{X i ∈ Idt }, Ci (dt) = I{U i ∈ Idt }, θφ and αiθ (t) and γi (t) are the conditional hazards of failure and censoring, respectively, given the past up until t- (including covariates). The likelihood (6) may be written as L(θ , φ) = Lc (θ )L∗ (θ , φ), with Lc (θ ) given by (2) and where the contributions from censoring and covariates are collected in L*(θ , φ). Thus, the function (2), which is usually taken as the standard censored data likelihood, is, under independent

censoring, a partial likelihood on which a valid inference on θ may be based. It is only the full likelihood for θ if L*(θ , φ) does not depend on θ , which is the case if censoring (and covariates) are noninformative. Thus, noninformative censoring is a statistical concept (while the concept of independent censoring is probabilistic) and means that the θφ conditional hazard of censoring γi (t) does, in fact, not depend on θ , the parameter of interest. An example of an informative rightcensoring scheme could be in a study with two competing causes of failure and where only one of the two cause-specific failure rates is of interest; if the two cause-specific failure rates are proportional (as in the so-called Koziol–Green model for random censoring, 14), then the failures from the second cause (the censorings) will carry information on the shape of the hazard function for the failure type of interest. It is, however, important to notice that even if the censoring is informative, then inference based on (2) will still be valid (though not fully efficient) and as it is usually preferable to make as few assumptions as possible about the distribution of the right-censoring times, the (partial) likelihood (2) is often the proper function to use for inference. 3 OTHER KINDS OF INCOMPLETE OBSERVATION When observation of a survival time, X, is right-censored, then the value of X is only known to belong to an interval of the form [U, + ∞). This is by far the most important kind of censoring for survival data, but not the only one. Thus, the observation of X is interval-censored if the value of X is only known to belong to an interval [U, V) and it is said to be left-censored if U = 0. It was seen above that under independent right-censoring a right-censored observation, U i , contributed to the partial likelihood function with a factor Sθ (U i ), which was also the contribution to the full likelihood under noninformative censoring. Similarly, concepts of independent and noninformative interval-censoring may be defined as leading to a contribution of Sθ (U i )–Sθ (V i ) to, respectively, the partial and the full likelihood.

CENSORED DATA

These concepts have received relatively little attention in the literature; however, this way of viewing censoring is closely related to the concept of coarsening at random. Formally, grouped data, where for each individual the lifetime is known only to belong to one of a fixed set of intervals [uk−1 , uk ) with 0 = u0 < u1 < · · · < um = +∞, are also interval-censored. However, the fact that the intervals are the same for everyone simplifies the likelihood to a binomial-type likelihood with parameters pθk = Sθ (uk−1 ) − Sθ (uk ), k = 1, . . . , m. Let us finally remark that while, following Hald [9,10, p. 144], censoring occurs when we are able to sample a complete population but individual values of observations above (or below) a given value are not specified, truncation corresponds to sampling from an incomplete population, i.e. from a conditional distribution. Left-truncated samples, where an individual is included only if his or her lifetime exceeds some given lower limit, also occur frequently in the analysis of survival data, especially in epidemiologic studies where hazard rates are often modeled as a function of age and where individuals are followed only from age at diagnosis of a given disease or from age at employment in a given factory. REFERENCES 1. Andersen, P.K., Abildstrom, S. & Rosthøj, S. (2002). Competing risks as a multistate model. Statistical Methods in Medical Research. 11, 203–215. 2. Andersen, P.K., Borgan, Ø., Gill, R.D. & Keiding, N. (1993). Statistical Models Based on Counting Processes. Springer-Verlag, New York. 3. Arjas, E. & Haara, P. (1984). A marked point process approach to censored failure data with complicated covariates, Scandinavian Journal of Statistics 11, 193–209. 4. Cox, D.R. (1959). The analysis of exponentially distributed life-times with two types of failure, Journal of the Royal Statistical Society, Series B 21, 411–421. 5. Cox, D.R. (1975). Partial likelihood, Biometrika 62, 269–276. 6. Fleming, T.R. & Harrington, D.P. (1991). Counting Processes and Survival Analysis. Wiley, New York.

5

7. Gill, R.D. (1980). Censoring and stochastic integrals, Mathematical Centre Tracts 124, Mathematisch Centrum, Amsterdam. 8. Gill, R.D. (1981). Testing with replacement and the product limit estimator, Annals of Statistics 9, 853–860. 9. Hald, A. (1949). Maximum likelihood estimation of the parameters of a normal distribution which is truncated at a known point, Skandinavisk Aktuarietidsskrift 32, 119–134. 10. Hald, A. (1952). Statistical Theory with Engineering Applications. Wiley, New York. 11. Jacobsen, M. (1989). Right censoring and martingale methods for failure time data, Annals of Statistics 17, 1133–1156. 12. Kalbfleisch, J.D. & MacKay, R.J. (1979). On constant-sum models for censored survival data, Biometrika 66, 87–90. 13. Kalbfleisch, J.D. & Prentice, R.L. (1980). The Statistical Analysis of Failure Time Data. Wiley, New York. 14. Koziol, J.A. & Green, S.B. (1976). A Cram´ervon Mises statistic for randomly censored data, Biometrika 63, 465–474. 15. Tsiatis, A.A. (1975). A nonidentifiability aspect of the problem of competing risks, Proceedings of the National Academy of Sciences 72, 20–22. 16. Williams, J.A. & Lagakos, S.W. (1977). Models for censored survival analysis: constant sum and variable sum models, Biometrika 64, 215–224.

CENTER FOR DEVICES AND RADIOLOGICAL HEALTH (CDRH)

the product’s safety. These specific regulations may include requirements for meeting performance standards recognized by the FDA, postmarket surveillance, patient registries, or other appropriate requirements. Class III: Premarket Approval. Class III devices are life supporting, life sustaining, or important in preventing impairment of human health. Because general controls may be insufficient to provide reasonable assurance of the device’s safety and effectiveness, FDA preapproval is required before it is marketed. Under Class III regulations, devices such as heart valves, breast implants, and cranial electrotherapy stimulators must be reviewed for safety and effectiveness before marketing.

The U.S. Food and Drug Administration’s Center for Devices and Radiological Health (CDRH) ensures the safety and effectiveness of medical devices and the safety of radiologic products. 1

MEDICAL DEVICES

The Food, Drug, and Cosmetic (FD&C) Act defines a medical device as any health-care product that does not achieve its principal intended purposes by chemical action or by being metabolized. Under this definition, a ‘‘device’’ can be as simple as a tongue depressor or a thermometer, or as complex as a kidney dialysis machine. Medical devices are classified and regulated according to their degree of risk to the public.

2

OBTAINING FDA APPROVAL

The CDRH works with both medical device and radiologic health industries. A manufacturer of a Class III device files a PreMarket Approval Application (PMA) to obtain FDA approval to market the product. Like the submission that is filed for the approval of a new drug, a PMA contains clinical and laboratory testing data to demonstrate safety and effectiveness. A Premarket Notification, also known as a 510(k), is an application submitted to the FDA to demonstrate that a medical device is substantially equivalent to (i.e., meaning just as safe and effective as) a legally marketed device that does not require premarket approval.

1.1 Regulatory Classes Because each device is different, the Food and Drug Administration (FDA) establishes three different regulatory classes to ensure that each device is subject to regulations that are appropriate. Class I: General Controls. Class I devices are subject to a set of general regulations that apply to all devices. General controls include the registration of manufacturers, general recordkeeping requirements, and compliance with Good Manufacturing Practice regulations. Class II: Special Controls. Class II devices are those for which general regulations are not enough to guarantee the safety of the device. A Class II device may be subject to specific regulations to ensure

3 GOOD MANUFACTURING PRACTICES (GMPS) The FDA further ensures the safety and effectiveness of medical devices by regulating their manufacture. As with drugs, the FDA has established Good Manufacturing

This article was modified from the website of the United States Food and Drug Administration (http://www.eduneering.com/fda/courses/FDATour3/tourFDA-frames-08.html) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

CENTER FOR DEVICES AND RADIOLOGICAL HEALTH (CDRH)

Practices for medical devices and regularly inspects manufacturers to ensure they comply with these regulations. 4

CONTINUOUS ANALYSIS

After approval of a medical device, the FDA continuously analyzes reports to ensure that products are safe and to watch for dangerous events related to the use of medical devices. The CDRH also monitors certain electronic products to protect the public from unnecessary exposure to radiation. Products that are monitored by the FDA include televisions, microwave ovens, X-ray machines, and devices employing lasers (including laser light shows). The FDA administers the law by setting and enforcing standards to limit unnecessary radiation emissions.

CENTER FOR DRUG EVALUATION AND RESEARCH (CDER)

comprise the single most important factor in the approval or disapproval of a new drug and are the basis for a new drug application (NDA). Once the NDA is approved, the product can be legally marketed in the United States. 2. Postmarket Drug Surveillance. The CDER continuously monitors the safety of drugs that have already been marketed. After the drug is approved and marketed, the FDA uses different mechanisms for ensuring that firms adhere to the terms and conditions of approval described in the application and that the drug is manufactured in a consistent and controlled manner. This is done by periodic unannounced investigations of drug production and control facilities by the FDA’s field investigators and analysts. The Center also ensures that prescription drug information provided by drug firms is truthful, balanced, and accurately communicated. In addition, the CDER follows up with companies on concerns related to medication errors, drug shortages, and ineffective or toxic drugs. 3. Generic Drug Review. Generic drug applications are termed ‘‘abbreviated’’ in that they are generally not required to include preclinical (animal) and clinical (human) data to establish safety and effectiveness. The Abbreviated New Drug Application (ANDA) provides for the review and ultimate approval of a generic drug product. After all components of the application are found to be acceptable, an approval or tentative approval letter is issued to the applicant. Tentative approvals require the manufacturer to delay marketing the generic drug until all patent/exclusivity issues have expired. 4. Over-the-Counter Drug Review. Overthe-counter (OTC) drug products are those that are available to consumers without a prescription, and 6 out of 10 medications bought by consumers are OTC drugs. The CDER’s OTC program establishes drug monographs for each

The U.S. Food and Drug Administration’s Center for Drug Evaluation and Research (CDER) ensures that safe and effective prescription, nonprescription, and generic drugs are made available as quickly as possible by overseeing the research, development, manufacture, and marketing of drugs. The CDER reviews the clinical trial evidence of the safety and effectiveness of new drugs before approving them for marketing and monitors their performance for unexpected health risks. The CDER also ensures that drug labeling, drug information for patients, and drug promotion are truthful, helpful, and not misleading. 1

CDER MISSION

The Center is organized into four main functional areas. 1. New Drug Development and Review . Drug development is a highly complicated, lengthy process. During preclinical drug development, a sponsor evaluates the drug’s toxic and pharmacologic effects through in vitro and in vivo laboratory animal testing. Next, the firm submits an investigational new drug (IND) application to the CDER to obtain U.S. Food and Drug Administration (FDA) approval that the new drug is sufficiently safe to allow clinical trials. During clinical trials, the investigational drug is administered to humans and is evaluated for its safety and effectiveness in treating, preventing, or diagnosing a specific disease or condition. The results of this testing will This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/about/default.htm and http://www.eduneering.com/fda/courses/FDA Tour3/tourFDA-frames-06.html) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

CENTER FOR DRUG EVALUATION AND RESEARCH (CDER)

class of product; these are like a recipe book, covering the acceptable ingredients, doses, formulations, and labeling. Products conforming to a monograph may be marketed without further FDA clearance, and those that do not must undergo separate review and approval through the new drug approval process.

FDA in 1 year or less; priority applications for breakthrough medications are usually approved in 6 months. The PDUFA user fees, however, do not cover the FDA’s expenses connected with generic and nonprescription drugs, plant inspections, postmarket surveillance, or monitoring of drug advertisements. 5 ACCELERATED APPROVAL

2

GOOD MANUFACTURING PRACTICES

To make sure that drugs are manufactured to the same high standards that are required for their approval, the FDA provides a set of regulations called Good Manufacturing Practices (GMPs). The law requires that the Office of Regulatory Affairs (ORA) perform periodic inspections of all drug firms for compliance with GMPs. 3

ADVERSE EVENT REPORTING

The FDA maintains several reporting systems that alert the Agency to side effects that were not detected during clinical trials but rather emerged when the product became widely used. One of these programs is the CDER’s MedWatch, which encourages health professionals to report serious adverse events involving any medical product (including drugs, devices, and biologics). If necessary, the FDA can take regulatory actions to protect consumers. Regulatory actions may include restrictions on the product’s use or its withdrawal from the market. About 1% to 3% of products approved each year must be removed later because of rare but serious side effects. 4

PRESCRIPTION DRUG USER FEE ACT

In the Prescription Drug User Fee Act of 1992 (PDUFA) the U.S. Congress, pharmaceutical industry, and FDA agreed on specific review goals for certain drugs and biologics, which are to be achieved with the help of user fees paid to the FDA by the products’ manufacturers. The program has been instrumental in reducing the FDA’s median drug-review times by more than half. As a result, typical drug applications are processed by the

Many of the drugs currently used to treat lifethreatening conditions such as cancer were approved through an accelerated FDA review process. In accelerated approval, the FDA approves the drug on the condition that the applicant will study and report the findings of the clinical benefit of the drug. The FDA continues to review new information and data about these drugs as the data become available; if the findings are negative, the appropriate actions are taken.

CENTRAL NERVOUS SYSTEM (CNS)

nutritional deficiencies, environmental toxicities, or cerebrovascular disorders, neurodegenerative conditions, headache syndromes, seizure disorders, movement disorders, or autoimmune-mediated diseases other than those listed. Finally, we restrict our discussion to RCTs and do not review epidemiological or observational studies.

DAVID B. SOMMER LARRY B. GOLDSTEIN Duke University Medical Center Durham, North Carolina

1

INTRODUCTION

2 CNS TRIALS: GENERAL INCENTIVES AND CONSTRAINTS

Clinical neurology has a rich history based on careful patient observation. Over the last quarter century, however, a major shift toward evidence-based medicine has occurred, and that shift includes the practice of neurology. Although many diagnostic and therapeutic approaches employed in neurology today have never been formally evaluated, clinicians, patients, and payers now seek informative results from well-designed and conducted clinical trials to help guide therapeutic decisions. The randomized controlled trial (RCT) is the gold standard for evaluating putative interventions. The scope of central nervous system (CNS) diseases is vast, and a comprehensive review would be lengthy. The number of published trials has nearly doubled since the last such report was published in 2001 (1). This article provides an overview of issues related to CNS clinical trials focusing on six selected conditions (ischemic stroke, Alzheimer’s disease, migraine headache, epilepsy, Parkinson’s disease, and multiple sclerosis) that primarily involve the CNS and about which an increasing number of trials have been published over the last few decades (Table 1). These conditions were chosen both because they are associated with significant morbidity (Fig. 1) (2) and because they illustrate the diversity of diseases affecting the CNS. It should be noted that each of these topics could support an extensive report and that, for the topics that are discussed, only selected studies are cited. We do not address trials of treatments for CNS effects of systemic illness, psychiatric diseases, infections of the CNS, sleep disorders, traumatic brain injury, CNS neoplasms, inborn errors of metabolism and congenital malformations,

A RCT can be conducted ethically only when clinical equipoise exists between the interventions being considered, the trial can provide an answer to the question being studied, and the rights of subjects can be preserved. RCTs are often expensive and therefore require appropriate levels of funding. In 2005, public funding for brain research in the United States was estimated at 6.1 billion euros. Private funding for brain research in the United States was 8.4 billion euros during the same period. In Europe, the totals were 0.9 billion and 3.3 billion euros, respectively (3). Public granting agencies generally fund clinical trials in CNS disease based on the quality of the science, but in the United States only about 15% of applications are currently funded, with costly clinical trials competing for funds with a variety of other types of research projects (4). Because of the nature of the peer-review process, great effort can be extended in developing successive versions of a proposal with no guarantee of success. Industry-funded trials are undertaken primarily to provide evidence of the efficacy and safety of a company’s product. Research investment is generally limited to interventions that are likely to be profitable. Economic analyses show that device trials have the highest return on investment because of the lower regulatory hurdles for governmental approval and the high market prices (5,6). Suggesting the roles of incentives and constraints other than potential public health impact, Figs. 1 and 2 illustrate the discor-

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

CENTRAL NERVOUS SYSTEM (CNS)

Table 1. Number of Randomized Controlled Trials in Selected Neurological Diseases Indexed in Medline Year of Publication

Stroke

Alzheimer’s Disease

Migraine Headache

Epilepsy

Parkinson’s Disease

Multiple Sclerosis

Before 1981 1981–1986 1987–1991 1992–1996 1997–2001 2002–2006 Total

5 16 35 49 281 674 1060

1 18 62 155 201 335 772

44 52 75 135 193 240 739

29 50 81 181 236 199 776

45 21 67 134 179 262 708

14 26 62 108 163 230 603

Source: PubMed search for articles indexed with major subject heading (MeSH) of Stroke, Alzheimer Disease, Migraine Disorders, Epilepsy, Parkinson Disease, and Multiple Sclerosis (searches limited to randomized controlled trials and by publication year).

Global Burden of Disease (Million Quality Adjusted Life Years) Multiple Sclerosis, 1.5 Parkinson's Disease, 1.6

Stroke, 49.2

Epilepsy, 7.3

Migraine, 7.7

Figure 1. Source: World Health Report 2002 (2).

Alzheimer's (and other dementias), 10.4

Number of Randomized Controlled Trials (1965-2006) Multiple Sclerosis, 603

Stroke, 1060

Parkinson's, 708

Alzheimer's, 772 Epilepsy, 776 Migraine, 739

Figure 2. Source: PubMed search for articles indexed with major subject heading (MeSH) of Stroke, Alzheimer Disease, Migraine Disorders, Epilepsy, Parkinson Disease, and Multiple Sclerosis (searches limited to randomized controlled trials and by publication year).

CENTRAL NERVOUS SYSTEM (CNS)

dance between the burdens of different groups of CNS diseases and the numbers of clinical trials conducted in each area. 3

CNS TRIALS: COMMON CHALLENGES

Adequate and timely subject enrollment and retention are challenges faced by all prospective clinical trials. Trials involving CNS diseases can be particularly difficult. CNS disease can affect cognition and/or communication and therefore impair the subject’s capacity to give informed consent. In these circumstances, investigators must obtain consent from a surrogate decision maker. This requirement is particularly challenging in clinical trials of interventions for acute stroke in which the onset of disease is abrupt and unexpected, and the time window for treatment is limited. Because many neurological diseases evolve over several years (e.g., clinical worsening in Alzheimer’s disease, disability in multiple sclerosis, evolution of Parkinson’s disease, etc.), long periods of observation may be necessary to detect a treatment effect. This need presents a particular challenge for patient retention, increases the problem of competing comorbidities, and raises trial costs. In the case of industry-developed treatments in which a period of patent protection is limited, the latter two issues may make a trial unfeasible. A short follow-up period also limits a trial’s capacity to detect delayed harm. The majority of CNS trials end at or before 24 months, yet many drugs used to treat CNS disease are taken for a lifetime. Another historic problem faced by clinical trials of CNS disease has been a lack of standardized disease definitions or reliable, validated outcome measures. For example, a 1987 review of all 28 articles including clinical assessments published in the journal Stroke in 1984 and 1985 found that no single neurological scale was used more than once (7). This problem was not unique to stroke research and has been at least partially addressed not only for stroke but also for a variety of other conditions as described for each disease reviewed below. Agreement on useful outcome measures has been challenging for many CNS diseases because of

3

the lack of readily available pathologic data (brain biopsy being generally considered the most invasive of monitoring procedure). Neuroimaging technologies [e.g., magnetic resonance imaging (MRI)] have been employed to provide a measure of disease activity in diseases such as multiple sclerosis, but their use as a primary surrogate endpoint remains controversial (8). No valid or practical biologic endpoints exist for many CNS diseases for which endpoints must be constructed from clinician assessments and/or patient/caregiver report. 4

ISCHEMIC STROKE—PREVENTION

Stroke has the greatest health impact of all neurological diseases (Fig. 1). About 80% of strokes are ischemic, and consequently, interventions that prevent or ameliorate ischemic stroke have the greatest potential cumulative impact of all CNS treatments. The advent of thrombolytic therapy for selected patients with acute ischemic stroke has done little to reduce the overall burden of the disease, and primary and secondary prevention remain the most important interventions from the public health standpoint (9). Numerous clinical trials have shaped current medical therapy for stroke prevention. The benefit of lowering blood pressure in patients with hypertension has been demonstrated in at least 25 controlled trials of a variety of antihypertensive drugs (10). Treatment with dose-adjusted warfarin reduces the risk of stroke by 60% in patients with nonvalvular atrial fibrillation, whereas antiplatelet drugs such as aspirin reduce the risk by approximately 20% (11). Antiplatelet therapy represents a cornerstone of secondary stroke prevention, and several studies inform current practice. The Canadian Cooperative Study was among the first RCTs to demonstrate a benefit of aspirin in secondary stroke prevention (12). Supported by numerous other trials, aspirin use is associated with an approximate 13% reduction in the risk of serious vascular events in patients with prior stroke or transient ischemic attack (TIA) (13). The role of clopidogrel is somewhat more controversial. The Clopidogrel versus Aspirin for the Prevention of Recurrent Ischemic Events (CAPRIE) trial found a reduction in

4

CENTRAL NERVOUS SYSTEM (CNS)

a composite endpoint of myocardial infarction, stroke, or vascular death in patients with a history of myocardial, infarction, or symptomatic peripheral arterial disease with clopidogrel as compared with aspirin, but because of sample size limitations, it was not clear whether patients with stroke benefited (14). A meta-analysis of available data in 2002 demonstrated similar degrees of risk reduction with ASA and clopidogrel (15). The Management of Atherothrombosis with Clopidogrel in High Risk Patients (MATCH) trial found that the combination of aspirin plus clopidogrel was associated with a higher risk of bleeding without a reduction in recurrent ischemic events in patients with cerebrovascular disease risk when compared to clopidogrel alone (16). The Clopidogrel for High Atherothrombotic Risk and Ischemic Stabilization, Management, and Avoidance Trial (CHARISMA) found no benefit for the combination as compared with aspirin alone in high-risk patients (17). In contrast, the European Stroke Prevention Study-2 (ESPS2) found that the combination of aspirin and sustained-release dypiridamole was more efficacious than aspirin alone, a result that now has been supported by the European/ Australasian Stroke Prevention in Reversible Ischemia Trial (ESPIRIT) (18,19). Negative RCTs are as important as positive ones in that they show which treatments are not efficacious or safe. The Warfarin–Aspirin Recurrent Stroke Study (WARSS) found no benefit for warfarin over aspirin for secondary prevention in patients with noncardioembolic stroke (20). A substudy of WARSS evaluating patients with cryptogenic stroke who had a patent foramen ovale (PFO) similarly found no benefit for warfarin as compared with aspirin (21). The Warfarin–Aspirin Symptomatic Intracranial Disease (WASID) trial found no benefit of warfarin over aspirin in patients with a symptomatic intracranial stenosis (22). Postmenopausal hormone replacement therapy was once thought to reduce the risk of stroke and other cardiovascular events and was widely prescribed, but RCTs now have found either no benefit or increased stroke risk for women with coronary heart disease (23) or prior stroke history (24) or for otherwise healthy post-menopausal women (25).

Trials in subjects with established coronary heart disease or certain risk factors show an approximate 20% reduction in the risk of stroke associated with the use of HMG-CoA reductase inhibitors (statins). Studies with combined vascular endpoints had demonstrated lower stroke rates with statins versus placebo in patients with vascular risk factors (9). The Stroke Prevention with Aggressive Reductions in Cholesterol Levels (SPARCL) trial also found a reduction in recurrent stroke and other vascular events associated with a statin in patients with a recent stroke or TIA, no known coronary heart disease, and a low-density lipoprotein cholesterol (LDL-C) level between 100 to 190 mg/dL (26). A randomized trial first drastically altered surgical intervention for stroke prevention in 1985 with the publication of the negative Extracranial– Intracranial Bypass Study (27). The role of carotid endarterectomy (CEA) in severe symptomatic carotid stenosis for stroke prevention now has been firmly established by RCTs (28–30). The Asymptomatic Carotid Atherosclerosis Study (ACAS) (31) and the Asymptomatic Carotid Surgery Trial (ACST) (32) also demonstrate a smaller benefit in asymptomatic patients with high-grade carotid stenosis. Several trials comparing endarterectomy with carotid angioplasty/stenting have been published (33–35), but the results are inconsistent and the procedure has only been approved in the United States for symptomatic patients deemed at high endarterectomy risk. Several interventions recommended for stroke prevention (e.g., smoking cessation, moderation of alcohol consumption, and diet and weight management) are based on epidemiologic data and have not been examined in an RCT, either because it would be unethical or impractical to do so. Ongoing stroke prevention trials sponsored by the National Institutes of Health (NIH)/NINDS or other public agencies include evaluation of aspirin versus aspirin plus clopidogrel for secondary prevention of small subcortical strokes (NCT00059306), evaluation of warfarin versus aspirin for primary stroke prevention in patients with a cardiac ejection fraction less than

CENTRAL NERVOUS SYSTEM (CNS)

35% (NCT00041938), PFO closure versus anticoagulation versus antiplatelet therapy in recurrent stroke with PFO (NCT00562289), endarterectomy versus carotid stenting in symptomatic carotid stenosis (NCT00004732), and multiple trials of cholesterol-lowering agents in primary prevention (additional information on these trials and other ongoing trials can be accessed at ClinicalTrials.gov by typing the ID number into the search box). Industrysponsored trials include two large Phase III trials of oral direct thrombin inhibitors compared with warfarin in the prevention of stroke in atrial fibrillation (NCT00403767 and NCT00412984), a large Phase III trial of aspirin for primary prevention in patients with moderate cardiovascular risk (NCT00501059), and evaluation of an endovascularly deployed device to prevent embolism from the left atrial appendage (NCT00129545). A large industrysponsored Phase III randomized trial is comparing carotid stenting versus endarterectomy for asymptomatic severe carotid stenosis (NCT00106938), and multiple industry-sponsored trials evaluate PFO closure devices for patients with cryptogenic stroke (e.g., NCT00289289 and NCT00201461). 5

ISCHEMIC STROKE—ACUTE TREATMENT

The outcomes to be measured in stroke prevention trials are relatively straightforward (i.e., the numbers of strokes or other vascular events prevented). The associated costs and impact of therapy on quality of life are secondary measures. In contrast, the primary outcomes of interest for trials of acute stroke treatments are functional measures of stroke severity and are reflected in scales that assess neurological impairments (e.g., the NIH Stroke Scale and NIHSS), disability (e.g., the Barthel Index), and handicap [e.g., the Modified Rankin Scale, (mRS)]. Outcomes of acute stroke interventions are generally assessed 90 days poststroke. The optimal way of analyzing functional outcome data remains a point of discussion. No single trial has had a greater impact in revolutionizing the medical approach to

5

acute ischemic stroke than the NINDS trial of tissue plasminogen activator (tPA) (36). For patients treated within 3 hours of symptom onset, a 13% absolute (32% relative) increase occurred in proportion of patients with little or no functional deficit after 3 months. Because of the limited time window for treatment and because the likelihood of benefit increases the sooner the drug can be given, the NINDS tPA trial has led to a major change in the approach to acute ischemic stroke (37). A pilot RCT suggested that transcranial ultrasound could augment the effectiveness of systemic tPA, however, this finding has not yet been replicated or adopted in practice (38). Trials of several neuroprotective agents, heparin, and abciximab for the treatment of acute stroke have all yielded disappointing results (37). Currently, the NINDS is sponsoring three Phase III trials in acute stroke management. Albumin in Acute Ischemic Stroke (ALIAS, NCT00235495) and Field Administration of Stoke Therapy–Magnesium (FAST–MAG, NCT00059332) are placebocontrolled RCTs that evaluate the potential neuroprotective effects of human albumin and magnesium sulfate, respectively. The Interventional Management of Stroke III Trial (NCT00359424) is randomizing patients presenting with acute stroke within 3 hours to standard therapy with intravenous tPA versus partial-dose intravenous tPA followed by angiography and individualized intravascular intervention in eligible candidates. Currently, industry is sponsoring Phase III neuroprotective trials with citicoline (NCT00331890); ONO-2506, a homolog of valproic acid (NCT00229177); and Tanakan, a standardized ginkgo biloba extract (NCT00276380). Industry is also conducting a placebo-controlled double-blinded evaluation of ancrod, an enzyme derived from snake venom, for the treatment of ischemic stroke presenting within 6 hours (NCT00141011). The manufacturers of the NeuroThera topical phototherapy system and the NeuroFlo partial aortic occlusion devises are sponsoring Phase III efficacy trials (NCT00419705 and NCT00119717, respectively).

6

6

CENTRAL NERVOUS SYSTEM (CNS)

ALZHEMIER’S DISEASE

Alzheimer’s disease (AD) is the most common cause of dementia and leads to a significant burden of disease for both patients and caregivers. Histopathology provides the gold standard diagnostic test for AD, but this test cannot be used to identify subjects for enrollment in clinical trials and must be interpreted in the context of the patient’s clinical status. Standardized clinical criteria for the diagnosis of AD have been developed that have a correlation of 80–100% as compared with histopathology and provide a means for identifying subjects for therapeutic trials (39). The National Institute of Neurological and Communicative Diseases and Stroke/Alzheimer’s Disease and Related Disorders Association (NINCDS–ADRDA) criteria are among the best validated and most widely used for identifying potential subjects with possible or probable AD (40). Objective cognitive assessments, clinician or caregiver subjective global assessments, and functional assessments are all used to measure outcome in AD trials. The MiniMental State Examination (MMSE) (41) is widely used in trials as a cognitive instrument, but the Alzheimer’s Disease Assessment Scale–Cognitive Portion (ADAS–Cog) (42) is a more robust measure and has been used in several clinical trials. Multiple disease-specific instruments are available for assessing functional status including the Disability Assessment for Dementia (DAD) (43) and the Alzheimer’s Disease Cooperative Study Activities of Daily Living inventory (ACDS–ADL) (44). In addition to clinical assessment, several groups are actively seeking to identify biomarkers that can be used to aid in the diagnosis and monitoring of AD. This finding might help in early identification of cases, when patients stand to benefit the most from treatments intended to slow the course of the disease (45). Despite sustained efforts at finding disease-modifying therapies for AD, none have been identified to date, and the available symptomatic therapies have limited clinical benefit (46). Tacrine was the first cholinesterase inhibitor shown to have beneficial effects in a randomized trial, but its use was limited by side effects (47). The

pivotal efficacy studies of donepezil, which had the major advantages of daily dosing and a favorable adverse effects profile, were published in 1998 (48,49). This was followed by publication of positive efficacy trials of rivastigmine (50) and galantamine (51). The N-methyl-D-aspartate (NMDA) receptor antagonist memantine has been shown to improve function compared with placebo in severe patients with AD (45,52). Publicly funded trials currently are evaluating the effectiveness of selenium and vitamin-E (NCT00040378) as well as celecoxib and naproxen (NCT00007189) for the primary prevention of AD. The National Institute on Aging (NIA) is also sponsoring trials of simvastatin (NCT00053599) and docosahexanoic acid (NCT00440050) to slow AD progression. Currently, industry is funding Phase III trials of 3 novel agents targeted at the beta amyloid protein, which are as follows: MCP-7869 (NCT00105547), 3APS (NCT00088673), and bapineuzumab (NCT00574132). Rosiglitazone is also being evaluated as a disease-modifying agent in AD (NCT00348309). 7 MIGRAINE Migraine headache is defined for clinical and research purposes by the International Headache Society (IHS) criteria (53). The prevalence of migraine in the United States and Western Europe has been estimated at 9% of the general population, and about one third of migraneurs have headaches severe enough to interfere with activities of daily living (52). Migraine headaches have long been the subject of RCTs—propranolol was first shown to be effective versus placebo for migraine prophylaxis in 1974 (54). Clinical trials have shaped current practice in both the acute treatment of migraines and the prevention of recurrent migraines. Outcomes in migraine trials rely entirely on subjective patient report. The IHS has recommended the following standardized outcomes for acute therapy trials: percentage of patients pain-free at 2 hours, sustained painfree state at 48 hours, intensity of headache rated on a categorical scale, percentage of patients with reduction in pain from

CENTRAL NERVOUS SYSTEM (CNS)

moderate–severe to none–mild, and functional outcome rated on a categorical scale. For prophylaxis trials, monthly frequency of migraine attacks is the IHS-recommended outcome (55). In 2000, the American Academy of Neurology reviewed the available evidence for the treatment of migraine. Convincing effectiveness data from RCTs in acute treatment existed for all of the serotonin agonists (triptans), dihydroergotamine (DHE) nasal spray, aspirin, ibuprofen, naproxen sodium, butorphanol nasal spray, and opiates. Convincing efficacy data from RCTs in migraine prophylaxis were available for divalproex sodium, amitriptyline, propranolol, timolool, and methysergide (56). Clinical trial data supporting the efficacy of gabapentin (57) and topiramate (58) for migraine prophylaxis have since become available. RCTs have continued to refine our understanding of migraine treatment by evaluation of combination therapy (59), by direct comparisons of different agents (60), and through rigorous testing of widely used but unproven therapies. A recent placebo-controlled trial of intravenous dexamethasone administered in the emergency department failed to find benefit in acute migraine (61). Although parenteral DHE is widely used and believed to be helpful, its efficacy has never been proven in an RCT (62). The Chilean government currently is sponsoring a direct comparison trial of pregabalin and valproate in migraine prophylaxis (NCT00447369). Ongoing industrysponsored trials include a Phase III placebocontrolled evaluation of botulinum toxin type A for migraine prophylaxis (NCT00168428) and evaluations of an implantable neurostimulator (NCT00286078), transcranial magnetic stimulation (NCT00449540), and a cardiac atrial septal repair device (NCT00283738). 8

EPILEPSY

Epilepsy affects nearly 1% of the population and has a cumulative incidence of 3% by age 74 years. Before 1990, only six drugs were available for the treatment of seizures, but the range of available options

7

has since more than doubled (63). Definitions for seizure types and epilepsy syndromes used in research are maintained and periodically updated by the International League Against Epilepsy (www.ilae.org). Outcomes in antiepileptic drug (AED) trials are generally based on patient/caregiver reported seizure frequency. A common practice is to dichotomize subjects into responders (at least 50% reduction in seizure frequency) and nonresponders. The proportion of subjects remaining seizure-free has also been reported. The development of novel AEDs has been spurred because some patients remain refractory to all available therapies. These patients usually comprise the subjects for efficacy trials of new AEDs because the new AED can be ethically tested against placebo as add-on therapy and because an effect is more likely to be demonstrated in a short period of time in a population with frequent seizures. To evaluate the efficacy of newer AEDs in nonrefractory patients, randomized trials with active comparison groups rather than placebo-controlled trials are required because it would be unethical to deprive nonrefractory patients of therapy with proven benefit. The first placebo-controlled RCTs of AEDs as add-on therapy in refractory patients were published in 1975. Carbamazepine (64) and valproate (65) were evaluated in 23 and 20 patients, respectively. In a typical modern add-on trial, levetiracetam was recently shown to reduce mean seizure frequency and have a higher responder rate than placebo in subjects with uncontrolled idiopathic generalized epilepsy (66). The 45% responder rate among placebo patients in this study illustrates the importance of randomized placebo-controlled trials in the evaluation of therapeutic efficacy. Comparative trials of AEDs are less common than add-on trials. This situation is unfortunate because comparative trials have greater value in clinical decision making because little RCT evidence is available to guide AED choices (67). One example is a trial comparing lamotrigine and carbamazepine in subjects with both new onset partial and generalized seizures (68). This trial found

8

CENTRAL NERVOUS SYSTEM (CNS)

no difference in efficacy but improved tolerability with lamotrigine. The trial was not powered to detect a subtle difference in efficacy. The only Phase III NIH-sponsored trial currently in progress is a randomized parallel assignment trial comparing ethosuximde, lamotrigine, and valproic acid in the treatment of childhood absence epilepsy (NCT00088452). Industry-sponsored trials include a comparison of zonisamide versus carbamazepine in newly diagnosed partial epilepsy (NCT00477295) and a comparison of pregabalin versus levetiracetam (NCT00537238). Ongoing placebocontrolled add-on trials of new agents include evaluations of rufinamide (NCT00334958), SPM 927 (NCT00136019), retigabine (NCT00235755), and RWJ-333369 (NCT00433667). 9

PARKINSON’S DISEASE

Parkinson’s Disease (PD) is a neurodegenerative disorder with cardinal motor manifestations of tremor, bradykinesia, and rigidity. Levodopa was found to be the first efficacious symptomatic therapy for PD in the 1960s, and several RCTs have since demonstrated that this and several other agents are efficacious in ameliorating PD symptoms. No therapy has yet been proven to alter the progressive natural history of the disease (69). PD is usually diagnosed for research purposes using the United Kingdom Parkinson’s Disease Brain Bank criteria (70). Disability stage is measured using the classic five-stage scale of Hoehn and Yahr (71), but disease progression now is usually measured using the more comprehensive and detailed Unified Parkinson’s Disease Rating Scale (UPDRS) (72). Although the combination of levodopa and carbidopa has long been used for the treatment of PD, its effect on the natural history of the disease was uncertain. A landmark RCT, published in 2004, randomized patients to placebo or 3 different doses of levodopa and demonstrated a dose-related improvement in function with treatment that partially persisted 2 weeks after discontinuation of drug (73). A stable formulation

of levodopa/carbidopa for continuous duodenal infusion was shown to be efficacious in advanced PD (74). The dopamine agonist bromocriptine was shown to offer symptomatic benefit over placebo in 1975 (75), and pergolide was proven efficacious in 1985 (76). Pergolide has since been removed from the market because of toxicity. In the late 1990s, the dopamine agonists pramipexole (77) and ropinirole (78) were proven efficacious in RCTs and have been widely adopted into practice. Transdermal rotigotine has also recently been adopted as an effective dopaminergic therapy (79). Subcutaneous injection of apomorphine has been shown to treat refractory ‘‘off periods’’ effectively (80). Compounds that alter the metabolism of dopamine, such as the monoamine oxidase inhibitor selegiline, can improve symptom control (81). More recently, rasagiline (82) and an orally disintegrating formulation of selegiline (83) have been reported to improve motor symptoms. The catechol O-methyl transferase (COMT) inhibitors entacapone (84) and tolcapone (85) have been shown to reduce motor fluctuations in PD, but the latter has had limited use because of hepatotoxicity. Amantadine has been suggested to be efficacious in several small randomized trials, each with fewer than 20 patients (86). A large NINDS trial of vitamin E for neuroprotection in PD was negative (87). A trial of coenzyme Q10 yielded intriguing but nonsignificant results (88). Surgical treatment of advanced PD with deep brain stimulation (DBS) has been shown to improve quality of life and motor symptoms compared with medical therapy alone (89). The NINDS currently is sponsoring a Phase III evaluation of creatine as a diseasemodifying therapy in PD (NCT00449865). Two novel compounds currently in Phase III industry-sponsored trials are as follows: E2007 (NCT00286897) and SLV308 (NCT00269516). Another industry-sponsored Phase III evaluation of continuous infusion duodenal levodopa-carbidopa gel is in progress (NCT00357994).

CENTRAL NERVOUS SYSTEM (CNS)

10

MULTIPLE SCLEROSIS

Multiple sclerosis (MS) is a multifocal autoimmune demyelinating disease of the CNS associated with axonal injury with a typical onset in early adulthood. MS can have a variable course but can lead to significant disability. MS is nosologically divided into relapsing remitting MS (RRMS), secondary progressive MS (SPMS), and primary progressive MS (PPMS). Before 1990, no treatments proven to alter the disease or its clinical manifestations were available (90). MS is commonly diagnosed for clinical trial purposes using the so-called revised McDonald criteria (91), which incorporate both clinical and imaging findings. Commonly employed clinical outcomes in MS studies are relapse rates (RRMS) and disability progression using the Expanded Disability Scale (EDS). Several short-term imaging endpoints have been employed, but their correlation to the more important clinical endpoints has been debated (92). Four injectable drugs now have been proven efficacious for reducing disability and relapse rates in patients with RRMS. RCT results were published for interferon (INF) beta 1-b (93) and glatriamer acetate (94) in 1995; INF beta 1-a in 1996 (95); and natalizumab in 2006 (96). The first three compounds now are commonly used in patients with RRMS, whereas natalizumab’s use has been limited because of associated cases of progressive multifocal leukoencephalopathy (PML). In 2002, a higher dose three times weekly formulation of INF beta 1-a was shown to be superior to the previously approved once weekly formulation (97) and is now commonly in use. Promising results of preliminary RCTs for two oral agents, fingolimod (98) and laquinimod (99), in RRMS have been published. Administration of high-dose oral steroids has been shown to be efficacious versus placebo in reducing short-term disability in acute MS exacerbations (100). A small RCT found no difference between treatment of acute exacerbations with high-dose oral versus intravenous methylprednisolone (101). Nonetheless, many clinicians avoid the use of oral steroids for exacerbations because of the results of the Optic Neuritis Treatment

9

Trial in which the group that received lowdose oral steroids fared worse than the group that received a placebo (102). Trials in the treatment of progressive disease (PPMS and SPMS) have been less encouraging. Trials of azathioprine, cladribine, cyclophosphamide, cyclosporine, and methotrexate all have demonstrated a lack of efficacy or overwhelming toxicity (103). Positive results have been published for mitoxantrone (104,105), and this agent is frequently used in refractory patients. Currently, the NINDS is sponsoring a trial comparing INF beta 1-a versus glatriamer acetate versus the combination of the two agents (NCT00211887). Other sources of public funding currently are sponsoring comparative evaluations of cyclophosphamide versus methylprednisolone in secondary progressive disease (NCT00241254) and intravenous versus high-dose oral steroids in acute exacerbations of RRMS (NCT00418145). Industry currently is sponsoring Phase III evaluations of three novel oral agents in the treatment of RRMS, as follows: laquinimod (NCT00509145), BG00012 (NCT00420212), and fingolimod (NCT00420212). All three of these trials have target enrollments of at least 1000 patients. Industry is also sponsoring Phase III evaluations of alemtuzumab as add-on therapy to INF beta 1-a in RRMS (NCT00530348), MBP8298 in SPMS (NCT00468611), mitoxantrone in SPMS (NCT00146159), and rituximab in PPMS (NCT00087529).

11

CONCLUSION

This brief overview of clinical trials for selected diseases or conditions primarily affecting the CNS serves as an introduction to some issues involved in their conduct and interpretation. Only a few example trials for a few selected conditions are discussed and underscore the breath of clinical trials being conducted in this area. Therapeutic trials for CNS diseases remain in their infancy, but they already have had a dramatic impact on the practice of clinical neurology and will likely play an evermore important role in the future.

10

CENTRAL NERVOUS SYSTEM (CNS)

REFERENCES 1. R. J. Guiloff (ed.), Clinical Trials in Neurology. London: Springer, 2001. 2. C. Murry and A. Lopez (eds.), The World Health Report 2002—Reducing Risks, Promoting Health Life. Geneva, Switzerland: World Health Organization, 2002. 3. P. Sobocki, I. Lekander, S. Berwick, J. Olesen, and B. J¨onsson, Resource allocation to brain research in Europe (RABRE). Eur. J. Neurosci., 24(10): 2691–2693, 2006. 4. NINDS Funding Strategy—FY 2007. Available: http://www.ninds.nih.gov/funding/ ninds funding strategy.htm. Accessed December 14, 2007. 5. H. Moses, 3rd, E. R. Dorsey, D. H. Matheson, and S. O. Thier, Financial anatomy of biomedical research. JAMA. 294(11):1333–1342, 2005. 6. L. B. Goldstein, Regulatory device approval for stroke: fair and balanced? Stroke. 38: 1737–1738, 2007. 7. K. Asplund, Clinimetrics in stroke research. Stroke. 18: 528–530, 1987. 8. D. S. Goodin, E. M. Frohman, G. P. Garmany, Jr., et al., Disease modifying therapies in multiple sclerosis: report of the Therapeutics and Technology Assessment Subcommittee of the American Academy of Neurology and the MS Council for Clinical Practice Guidelines. Neurology. 58(2): 169–178, 2002. 9. L. B. Goldstein, R. Adams, M. J. Alberts, et al., Primary prevention of ischemic stroke: a guideline from the American Heart Association/American Stroke Association Stroke Council. Stroke. 37: 1583–1633, 2006. 10. C. M. Lawes, D. A. Bennett, V. L. Feigin, and A. Rodgers, Blood pressure and stroke: an overview of published reviews. Stroke. 35: 1024, 2004. 11. R. G. Hart, L. A. Pearce, and M. I. Aguilar, Meta-analysis: antithrombotic therapy to prevent stroke in patients who have nonvalvular atrial fibrillation. Annals of Internal Medicine. 146: 857–867, 2007. 12. The Canadian Cooperative Study Group, A randomized trial of aspirin and sulfinpyrazone in threatened stroke. New Engl. J. Med. 299: 53–59, 1978. 13. A. Algra and J. van Gijn, Cumulative metaanalysis of aspirin efficacy after cerebral ischaemia of arterial origin. J. Neurol. Neurosurg. Psychiatry. 66: 255, 1999. 14. CAPRIE Steering Committee, A randomized, blinded, trial of Clopidogrel Versus Aspirin in

Patients at Risk of Ischemic Events (CAPRIE). Lancet. 348: 1329–1339, 1996. 15. Antithrombotic Trialists’ Collaboration, Collaborative meta-analysis of randomised trials of antiplatelet therapy for prevention of death, myocardial infarction, and stroke in high risk patients. BMJ. 324: 71–86, 2002. 16. H. C. Diener, J. Bogousslavsky, L. M. Brass, et al., Aspirin and clopidogrel compared with clopidogrel alone after recent ischaemic stroke or transient ischaemic attack in highrisk patients (MATCH): randomised, doubleblind, placebo-controlled trial. Lancet. 364: 331–337, 2004. 17. D. L. Bhatt, K. A. Fox, W. Hacke, et al., Clopidogrel and aspirin versus aspirin alone for the prevention of atherothrombotic events. N. Engl. J. Med. 354: 1706–1717, 2006. 18. H. C. Diener, L. Cunha, C. Forbes, J. Sivenius, P. Smets, A. Lowenthal, European Stroke Prevention Study, 2: dipyridamole and acetylsalicylic acid in the secondary prevention of stroke. J. Neurol. Sci. 143: 1–13, 1996. 19. ESPRIT Study Group, Aspirin plus dipyridamole versus aspirin alone after cerebral ischaemia of arterial origin (ESPRIT): randomised controlled trial. Lancet. 367: 1665–1673, 2006. 20. J. P. Mohr, J. L. Thompson, R. M. Lazar, et al., A comparison of warfarin and aspirin for the prevention of recurrent ischemic stroke. N. Engl. J. Med. 345: 1444–1451, 2001. 21. S. Homma, R. L. Sacco, M. R. Di Tullio, R. R. Sciacca, and J. P. Mohr, for the PFO in Cryptogenic Stroke Study (PICSS) Investigators, Effect of medical treatment in stroke patients with patent foramen ovale. Circulation. 105: 2625–2631, 2002. 22. M. I. Chimowitz, M. J. Lynn, H. HowlettSmith, et al., Comparison of warfarin and aspirin for symptomatic intracranial arterial stenosis. N. Engl. J. Med. 352: 1305–1316, 2005. 23. S. Hulley, D. Grady, T. Bush, et al., Heart and Estrogen/progestin Replacement Study (HERS) Research Group, Randomized trial of estrogen plus progestin for secondary prevention of coronary heart disease in postmenopausal women. JAMA. 280: 605–613, 1998. 24. C. M. Viscoli, L. M. Brass, W. N. Kernan, P. M. Sarrel, S. Suissa, and R. I. Horwitz, A clinical trial of estrogen-replacement therapy after ischemic stroke. N. Engl. J. Med. 345: 1243–1249, 2001.

CENTRAL NERVOUS SYSTEM (CNS) 25. J. E. Rossouw, G. L. Anderson, R. L. Prentice, et al., Risks and benefits of estrogen plus progestin in healthy postmenopausal women: principal results from the Women’s Health Initiative randomized controlled trial. JAMA. 288: 321–333, 2002. 26. P. Amarenco, J. Bogousslavsky, A. Callahan, 3rd, et al., Stroke Prevention by Aggressive Reduction in Cholesterol Levels (SPARCL) Investigators. High-dose atorvastatin after stroke or transient ischemic attack. N. Engl. J. Med. 355: 549–559, 2006. 27. The EC/IC Bypass Study Group, Failure of extracranial-intracranial arterial bypass to reduce the risk of ischemic stroke: results of an international randomized trial. N. Engl. J. Med. 313: 1191–1200, 1985. 28. North American Symptomatic Carotid Endarterectomy Trial Collaborators, Beneficial effect of carotid endarterectomy in symptomatic patients with high-grade carotid stenosis. N. Engl. J. Med. 325: 445–453, 1991. 29. European Carotid Surgery Trialists’ Collaborative Group, MRC European Carotid Surgery Trial: interim results for symptomatic patients with severe (70–99%) or with mild (0–29%) carotid stenosis. Lancet. 337: 1235–1243, 1991. 30. M. R. Mayberg, S. E. Wilson, F. Yatsu, et al., Carotid endarterectomy and prevention of cerebral ischemia in symptomatic carotid stenosis: Veterans Affairs Cooperative Studies Program. JAMA. 266: 3289–3294, 1991. 31. Executive Committee for the Asymptomatic Carotid Atherosclerosis Study, Endarterectomy for asymptomatic carotid artery stenosis. JAMA. 273: 1421–1428, 1995. 32. A. Halliday, A. Mansfield, J. Marro, et al., Prevention of disabling and fatal strokes by successful carotid endarterectomy in patients without recent neurological symptoms: randomised controlled trial. Lancet. 363: 1491–1502, 2004. 33. J. S. Yadav, M. H. Wholey, R. E. Kuntz, et al., Protected carotid-artery stenting versus endarterectomy in high risk patients. N. Engl. J. Med. 351: 1493–1501, 2004. 34. The SPACE Collaborative Group, 30 day results from the SPACE trial of stent-protected angioplasty versus carotid endarterectomy in symptomatic patients: a randomised non-inferiority trial. Lancet. 368: 1239–1247, 2006. 35. J. L. Mas, G. Chatellier, B. Beyssen, et al., Endarterctomy versus stenting in patients

11

with symptomatic severe carotid stenosis. N. Engl. J. Med. 355: 1660–1167, 2006. 36. The National Institute of Neurological Disorders and Stroke rt-PA Stroke Study Group, Tissue plasminogen activator for acute ischemic stroke. N. Engl. J. Med. 333: 1581–1587, 1995. 37. H. P. Adams, Jr., G. del Zoppo, M. J. Alberts, et al., Guidelines for the early management of adults with ischemic stroke. Stroke. 38(5): 1655–1711, 2007. 38. A. V. Alexandrov, C. A. Molina, J. C. Grotta, et al., Ultrasound-enhanced systemic thrombolysis for acute ischemic stroke. N. Engl. J. Med. 351: 2170–2178, 2004. 39. O. L. Lopez, I. Litvan, K. E. Catt, et al., Accuracy of four clinical diagnostic criteria for the diagnosis of neurodegenerative dementias. Neurology. 53: 1292–1299, 1999. 40. G. McKhann, D. A. Drachman, M. F. Folstein, R. Katzman, D. L. Price, and E. Stadlan, Clinical diagnosis of Alzheimer’s disease: report of the NINCDS-ADRDA Work Group under the auspices of the Department of Health and Human Services Task Force on Alzheimer’s disease. Neurology. 34: 939–944, 1984. 41. J. R. Cockrell and M. F. Folstein, Mini-Mental State Examination (MMSE). Psychopharmacol. Bull. 24: 689–692, 1988. 42. R. C. Mhos, The Alzheimer’s disease assessment scale. Int. Psychogeriatr. 8: 195–203, 1996. 43. I. Gelinas, L. Gauthier, M. McIntyre, and S. Gauthier, Development of a functional measure for persons with Alzheimer’s disease: the disability assessment for dementia. Am. J. Occup. Ther. 53: 471–481, 1999. 44. D. Galasko, D. Bennett, M. Sano, et al., An inventory to assess activities of daily living for clinical trials in Alzheimer’s disease. Alzheimer Dis. Assoc. Disord. Suppl. 2:S33–S39, 1997. 45. A. M. Fagan, C. A. Csernansky, J. C. Morris, D. M. Holtzman, The search for antecedent biomarkers of Alzheimer’s disease. J. Alzheimers Dis. 8(4): 347–358, 2005. 46. R. S. Doody, J. C. Stevens, C. Beck, et al., Practice parameter: management of dementia (an evidence-based review). Report of the Quality Standards Subcommittee of the American Academy of Neurology. Neurology. 56(9): 1154–1166, 2001. 47. W. K. Summers, L. V. Majovski, G. M. Marsh, K. Tachiki, and A. Kling, Oral tetrahydroaminoacridine in long-term treatment of

12

48.

49.

50.

51.

52.

53.

54.

55.

56.

57.

58.

59.

60.

CENTRAL NERVOUS SYSTEM (CNS) senile dementia, Alzheimer type. N. Engl. J. Med. 315: 1241–1245, 1986. S. Rogers, R. Doody, R. Mohs, et al., and the Donepezil Study Group, Donepezil improved cognition and global function in Alzheimer’s disease. Arch. Intern. Med. 158: 1021–1031, 1998. S. Rogers, M. Farlow, R. Doody, et al., and the Donepezil Study Group, A 24-week, doubleblind, placebo-controlled trial of donepezil in patients with Alzheimer’s disease. Neurology. 50: 136–145, 1998. M. Rosler, R. Anand, A. Cicin–Sain, et al., Efficacy and safety of Rivastigmine in patients with Alzheimer’s disease: international randomised controlled trial. B. Med. J. 318: 633–638, 1999. M. Raskind, E. Peskind, T. Wessel, et al., and the Galantamine Study Group, Galantamine in AD. A 6-month randomized, placebocontrolled trial with a 6-month extension. Neurology. 54: 2261–2268, 2000. J. L. Brandes, Global trends in migraine care: results from the MAZE survey. CNS Drugs. 16 Suppl. 1: 13–18, 2002. Headache Classification Committee of the International Headache Society, Classification and diagnostic criteria for headache disorders, cranial neuralgias and facial pain. Cephalalgia. 8 Suppl. 7: 1–96, 1998. TE Wideroe and T. Vigander, Propranolol in the treatment of migraine. Br. Med. J. 2: 699–701, 1974. P. Tfelt-Hansen, G. Block, C. Dahl¨of, et al., Guidelines for controlled trials of drugs in migraine: second edition. Cephalalgia. 20: 765–786, 2000. S. D. Silberstein, Practice parameter: evidence-based guidelines for migraine headache (an evidence-based review): report of the Quality Standards Subcommittee of the American Academy of Neurology. Neurology. 55: 754–762, 2000. N. T. Mathew, A. Rapoport, J. Saper, et al., Efficacy of gabapentin in migraine prophylaxis. Headache. 41: 119, 2001. J. L. Brandes, J. R. Saper, M. Diamond, et al., Topiramate for migraine prevention: a randomized controlled trial. JAMA. 291: 965, 2004. J. L. Brandes, D. Kudrow, S. R. Stark, et al., Sumatriptan-naproxen for acute treatment of migraine: a randomized trial. JAMA. 297: 1443, 2007. B. W. Friedman, J. Corbo, R. B. Lipton, et al., A trial of metoclopramide vs sumatriptan

for the emergency department treatment of migraines. Neurology. 64: 463, 2005. 61. B. W. Friedman, P. Greenwald, T. C. Bania, et al., Randomized trial of IV dexamethasone for acute migraine in the emergency department. Neurology. 69: 2038–2044, 2007. 62. I. Colman, M. D. Brown, G. D. Innes, E. Grafstein, T. E. Roberts, and B. H. Rowe, Parenteral dihydroergotamine for acute migraine headache: a systematic review of the literature. Ann. Emerg. Med. 45: 393–401, 2005. 63. J. A. French, A. M. Kanner, J. Bautista, et al., Efficacy and tolerability of the new antiepileptic drugs I: treatment of new onset epilepsy: report of the Therapeutics and Technology Assessment Subcommittee and Quality Standards Subcommittee of the American Academy of Neurology and the American Epilepsy Society. Neurology. 62: 1252–1260, 2004. 64. H. Kutt, G. Solomon, C. Wasterlain, H. Peterson, S. Louis, and R Carruthers, Carbamazepine in difficult to control epileptic out-patients. Acta. Neurol. Scand. Suppl. 60: 27–32, 1975. 65. A Richens and S. Ahmad, Controlled trial of sodium valproate in severe epilepsy. Br. Med. J. 4: 255–256, 1975. 66. S. F. Berkovic, R. C. Knowlton, R. F. Leroy, J. Schiemann, and U. Falter, Levetiracetam N01057 Study Group, Placebo-controlled study of levetiracetam in idiopathic generalized epilepsy. Neurology. 69: 1751–1760, 2007. 67. J. A. French and R. J. Kryscio, Active control trials for epilepsy: avoiding bias in head-tohead trials. Neurology. 66: 1294–1295, 2006. 68. M. J. Brodie, A. Richens, and A. W. Yuen, Double-blind comparison of lamotrigine and carbamazepine in newly diagnosed epilepsy. UK Lamotrigine/ Carbamazepine Monotherapy Trial Group. Lancet. 345: 476–479, 1995. 69. E. Tolosa and R. Katzenshclager, Pharmacological management of Parkinson’s disease. In: J Jankovic and E Tolosa (eds.), Parkinson’s Disease and Movement Disorders. Philidephia, PA: Lippincott Williams and Wilkins, 2007. 70. A. J. Hughes, S. E. Daniel, L. Kilford, et al., Accuracy of clinical diagnosis of idiopathic Parkinson’s disease: a clinico-pathological study of 100 cases. J. Neurol. Neurosurg. Psychiatry. 55: 181–184, 1992. 71. M. M. Hoehn, and M. D. Yahr, Parkinsonism: onset, progression and mortality. Neurology. 17: 427–442, 1967.

CENTRAL NERVOUS SYSTEM (CNS) 72. S. Fahn and R. Elton, Members of the UPDRS Development Committee, Unified Parkinson’s Disease Rating Scale. In: S. Fahn, C. D. Marsden, D. B. Calne, and M. Goldstein (eds.), Recent Developments in Parkinson’s Disease, vol. 2. Florham Park, NJ: MacMillan, 1987. 73. Parkinson Study Group, Levodopa and the progression of Parkinson’s disease. N. Engl. J. Med. 351: 2498–2508, 2004. 74. D. Nyholm, A. I. Nilsson Remahl, N. Dizdar, et al., Duodenal levodopa infusion monotherapy vs oral polypharmacy in advanced Parkinson disease. Neurology. 64(2): 216–223, 2005. 75. P. F. Teychenne, P. N. Leigh, J. L. Reid, D. B. Calne, J. K. Greenacre, A. Petrie, and A. N. Bamji, Idiopathic parkinsonism treated with bromocriptine. Lancet. 2(7933): 473–476, 1975. 76. J. I. Sage and R. C. Duvoisin, Pergolide therapy in Parkinson’s disease: a double-blind, placebo-controlled study. Clin. Neuropharmacol. 8(3): 260–265, 1985. 77. A. Lieberman, A. Ranhosky, and D. Korts, Clinical evaluation of pramipexole in advanced Parkinson’s disease: results of a double-blind, placebo-controlled, parallelgroup study. Neurology. 49: 162–168, 1997. 78. A. Lieberman, C. W. Olanow, K. Sethi, et al., A multicenter trial of ropinirole as adjunct treatment for Parkinson’s disease, Ropinirole Study Group, Neurology. 51: 1057–1062, 1998. 79. R. L. Watts, J. Jankovic, C. Waters, A. Rajput, B. Boroojerdi, and J. Rao, Randomized, blind, controlled trial of transdermal rotigotine in early Parkinson disease. Neurology. 68(4): 272–276, 2007. 80. R. B. Dewey, Jr., J. T. Hutton, P. A. LeWitt, and S. A. Factor, A randomized, double-blind, placebo-controlled trial of subcutaneously injected apomorphine for parkinsonian offstate events. Arch. Neurol. 58: 1385–1392, 2001. 81. B. Sivertsen, E. Dupont, B. Mikkelsen, P. Mogensen, C. Rasmussen, F. Boesen, and E. Heinonen, Selegiline and levodopa in early or moderately advanced Parkinson’s disease: a double-blind controlled short- and longterm study. Acta. Neurol. Scand. Suppl. 126: 147–152, 1989. 82. Parkinson Study Group, A controlled, randomized, delayed-start study of rasagiline in early Parkinson disease. Arch. Neurol. 61: 561–566, 2004. 83. C. H. Waters, K. D. Sethi, R. A. Hauser,

13

E. Molho, and J. M. Bertoni, Zydis selegiline reduces off time in Parkinson’s disease patients with motor fluctuations: a 3-month, randomized, placebo-controlled study. Mov. Disord. 19: 426–432, 2004. 84. Parkinson Study Group, Entacapone improves motor fluctuations in levodopatreated Parkinson’s disease patients. Ann. Neurol. 42: 747–755, 1997. 85. M. C. Kurth, C. H. Adler, M. S. Hilaire, et al., Tolcapone improves motor function and reduces levodopa requirement in patients with Parkinson’s disease experiencing motor fluctuations: a multicenter, double-blind, randomized, placebo-controlled trial, Tolcapone Fluctuator Study Group I. Neurology. 48(1): 81–87, 1997. ´ 86. F. P. da Silva-Junior, P. Braga-Neto, F. Sueli Monte, and V. M. de Bruin, Amantadine reduces the duration of levodopainduced dyskinesia: a randomized, doubleblind, placebo-controlled study. Parkinsonism Relat. Disord. 11(7): 449–452, 2005. 87. The Parkinson Study Group, Effects of tocopherol and deprenyl on the progression of disability in early Parkinson’s disease. N. Engl. J. Med. 328: 176–183, 1993. 88. C. W. Shults, D. Oakes, K. Kieburtz, et al., Effects of coenzyme Q10 in early Parkinson disease: evidence of slowing of the functional decline. Arch. Neurol. 59: 1541–1550, 2002. 89. G. Deuschl, C. Schade-Brittinger, P. Krack, et al., A randomized trial of deep-brain stimulation for Parkinson’s disease. N. Engl. J. Med. 355(9): 896–908, 2006. 90. M. J. Olek and D. M. Dawson, Multiple sclerosis and other inflammatory demyelinating diseases of the central nervous system. In: W. G. Bradley, R. B. Daroff, G. M. Fenichel, and J. Jankovic (eds.), Neurology in Clinical Practice. Philidelphia, PA: Butterworth Heinemann, 2004. 91. C. H. Polman, S. C. Reingold, G. Edan, et al., Diagnostic criteria for multiple sclerosis: 2005 revisions to the ‘‘McDonald Criteria’’. Ann. Neurol. 58(6): 840–846, 2005. 92. D. S. Goodin, E. M. Frohman, G. P. Garmany, Jr., et al., Disease modifying therapies in multiple sclerosis: report of the Therapeutics and Technology Assessment Subcommittee of the American Academy of Neurology and the MS Council for Clinical Practice Guidelines. Neurology. 58(2): 169–178, 2002. 93. The IFNB Multiple Sclerosis Study Group and the UBC MS/MRI Analysis Group, Interferon beta-1b in the treatment of MS: final outcome

14

CENTRAL NERVOUS SYSTEM (CNS)

of the randomized controlled trial. Neurology. 45: 1277–1285, 1995. 94. K. P. Johnson, B. R. Brooks, J. A. Cohen, et al., Copolymer 1 reduces relapse rate and improves disability in relapsing-remitting multiple sclerosis: results of a phase III multicenter, doubleblind, placebo-controlled trial. Neurology. 45: 1268–1276, 1995. 95. L. D. Jacobs, D. L. Cookfair, R. A. Rudick, et al., Intramuscular interferon beta-1a for disease progression in exacerbating remitting multiple sclerosis. Ann. Neurol. 39: 285–294, 1996. 96. C. H. Polman, P. W. O’Connor, E. Havrdova, et al., A randomized, placebo-controlled trial of natalizumab for relapsing multiple sclerosis. N. Engl. J. Med. 354: 899, 2006. 97. H. Panitch, D. S. Goodin, G. Francis, et al., Randomized, comparative study of interferon beta-1a treatment regimens in MS: The EVIDENCE Trial. Neurology. 59: 1496, 2002. 98. L. Kappos, J. Antel, G. Comi, et al., Oral fingolimod (FTY720) for relapsing multiple sclerosis. N. Engl. J. Med. 355: 1124, 2006. 99. C. Polman, F. Barkhof, M. SandbergWollheim, et al., Treatment with laquinimod reduces development of active MRI lesions in relapsing MS. Neurology. 64: 987, 2005. 100. F. Sellebjerg, J. L. Frederiksen, P. M. Nielsen, and J. Olesen, Double-blind, randomized, placebo-controlled study of oral, highdose methylprednisolone in attacks of MS. Neurology. 51: 529, 1998. 101. D. Barnes, R. A. Hughes, R. W. Morris, et al., Randomised trial of oral and intravenous methylprednisolone in acute relapses of multiple sclerosis. Lancet. 349: 902, 1997. 102. R. W. Beck, P. A. Cleary, M. M. Anderson, Jr., et al., the Optic Neuritis Study Group, A randomized, controlled trial of corticosteroids in the treatment of acute optic neuritis. N. Engl. J. Med. 326: 581, 1992. 103. M. J. Olek, Treatment of progressive multiple sclerosis in Adults. In: B. D. Rose (ed.), UpToDate. Waltham, MA:, 2007. 104. G. Edan, D. Miller, M. Clanet, et al., Therapeutic effect of mitoxantrone combined with methylprednisolone in multiple sclerosis: A randomised multicentre study of active disease using MRI and clinical criteria. J. Neurol. Neurosurg. Psychiatry. 62: 112, 1997. 105. H. P. Hartung, R. Gonsette, N. Konig, et al., Mitoxantrone in progressive multiple sclerosis: a placebo-controlled, double-blind, randomised, multicentre trial. Lancet. 360: 2018, 2002.

CFR 21 PART 11

is important to note that the predicate rules encompass a broad range of areas and apply not only to Good Clinical Practice, but also to good laboratory practices and good manufacturing practices (GMP). Records maintained or submitted to the agency in these areas are subject to Part 11. One consequence of Part 11 was to provide a framework for electronic data capture and record keeping in clinical trials. As clinical trials increase in number and complexity, electronic solutions claim a growing presence in the effort to improve data collection, transmission, storage, and retrieval. Sponsors are moving toward electronic alternatives to introduce speed and greater reliability into these data-driven processes that historically, have been paper based. Additionally, the capability to execute electronic signatures at the investigative site (electronic case report forms and Case Books) is accompanied by increased responsibilities for sponsors and technology providers to adhere to and provide solutions that are compliant with 21CFR11 and the current FDA guideline(s). Significant challenges lie ahead, most notably, how to implement electronic solutions so the resulting electronic records are considered equivalent to paper records, and electronic signatures are considered equal to handwritten ones. Since its effective date, Part 11 has had a significant impact on clinical research, and specifically clinical trials with regard to the implementation of technology, validation, data collection, data management, and regulatory submissions.

MICHAEL P. OWINGS and PAUL A. BLEICHER Phase Forward, Waltham, Massachusetts

One consequence of Part 11 was to provide a framework for electronic data capture and electronic record keeping in clinical trials; as such, it has had a significant impact on clinical research, and specifically clinical trials with regard to the implementation of technology, validation, data collection, data management, and regulatory submissions. Several guidances remain in effect for the interpretation of 21 CFR Part 11, specifically the Guidance for Computerized Systems Used in Clinical Trials and a Scope and Application Guidance. The latter guidance indicates that the FDA is in the process of reinterpreting, and possibly redefining 21 CFR Part 11 in line with a risk management approach. On March 20, 1997, The U. S. FDA issued an important regulation under the Code of Federal Regulations entitled 21CFR11: ELECTRONIC RECORDS; ELECTRONIC SIGNATURES, which is commonly referred to as ‘‘Part 11.’’ The purpose of the regulation is to provide criteria under which the FDA will consider electronic records to be equivalent to paper records, and electronic signatures equivalent to traditional handwritten signatures. Part 11 applies to persons who, in fulfillment of a requirement in a statute, have chosen to maintain records or submit designated information electronically to FDA, and to records in electronic form that are created, modified, maintained, archived, retrieved, or transmitted under any records requirements set forth in Agency regulations. Part 11 also applies to electronic records submitted to the Agency under the Federal Food, Drug, and Cosmetic Act (the Act) (3) and the Public Health Service Act (the PHS Act) (4), even if such records are not specifically identified in Agency regulations (§ 11.1). The underlying requirements set forth in the Act, PHS Act, and FDA regulations (other than part 11) are referred to as predicate rules (2). It

1

BACKGROUND

Part 11 was developed by the FDA in concert with other government initiatives that began in the early 1990s and continue to the present. The final rule is the result of a six-year process, which began in 1991, when members of the pharmaceutical industry met with the FDA to explore how they could accommodate paperless record systems

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

CFR 21 PART 11

Table 1. Selected Information Specified in Subpart A Related to Scope and Definitions 21CFR11 Subpart A—General Provisions §11.1 – Scope (a) The regulations in this part set forth the criteria under which the agency considers electronic records, electronic signatures, and handwritten signatures executed to electronic records to be trustworthy, reliable, and generally equivalent to paper records and handwritten signatures executed on paper. (b) This part applies to records in electronic form that are created, modified, maintained, archived, retrieved, or transmitted, under any records requirements set forth in agency regulations. However, this part does not apply to paper records that are, or have been, transmitted by electronic means. (c) Where electronic signatures and their associated electronic records meet the requirements of this part, the agency will consider the electronic signatures to be equivalent to full handwritten signatures, initials, and other general signings as required by agency regulations, unless specifically excepted by regulation(s) effective on or after August 20, 1997. (e) Computer systems (including hardware and software), controls, and attendant documentation maintained under this part shall be readily available for, and subject to, FDA inspection. §11.3 – Definitions (4) Closed system means an environment in which system access is controlled by persons who are responsible for the content of electronic records that are on the system (6) Electronic record means any combination of text, graphics, data, audio, pictorial, or other information representation in digital form that is created, modified, maintained, archived, retrieved, or distributed by a computer system. (7) Electronic signature means a computer data compilation of any symbol or series of symbols executed, adopted, or authorized by an individual to be the legally binding equivalent of the individual’s handwritten signature. (9) Open system means an environment in which system access is not controlled by persons who are responsible for the content of electronic records that are on the system.

under the current good manufacturing practice (cGMP) regulations in 21 CFR Parts 210 and 211. From of these meetings came a task force (a working sub-group), who developed a 1992 notice in the Federal Register that the FDA was considering the use of electronic identification and signatures, a proposed rule in 1994 and ultimately, the final rule in 1997. In addition to 21 CFR Parts 210 and 211, Part 11 was complimentary with other initiatives to eliminate paper that were already underway at the federal government, such as the Paperwork Reduction Act of 1995 (5) and later the Government Paperwork Elimination Act (GPEA) of 1998 (6). The purpose of the Paperwork Reduction Act of 1995 as stated is to minimize the paperwork burden for individuals, small businesses, educational and nonprofit institutions, Federal contractors, State, local and tribal governments, and other persons resulting from the collection of information by or for the Federal Government.

Part 11 and GPEA shared the combined purpose of eliminating paper from complex and costly processes and promoting the use of technology with the creation and management of electronic records and use of electronic signatures. This period was an inflection point as the U.S. Government began to encourage and even require the use of technology. GPEA was enacted on October 21, 1998, as title XVII of P.L. 105-277, and contains specific requirements in the use of technology for electronic records such as: the technology shall be compatible with standards and technology for electronic signatures that are generally used in commerce and industry . . . and shall ensure that electronic signatures are as reliable as is appropriate for the purpose in question and keep intact the information submitted.

As will be shown later in the article, these requirements are consistent with, and in some cases almost identical to, the basic tenets contained in Part 11.

CFR 21 PART 11

3

Table 2. Information Specified in Subpart B Related to the Manifestation of the Electronic Signature and Signature Linking 21CFR11 Subpart B–Controls for Closed Systems §11.10 - Controls for closed systems Persons who use closed systems to create, modify, maintain, or transmit electronic records shall employ procedures and controls designed to ensure the authenticity, integrity, and, when appropriate, the confidentiality of electronic records, and to ensure that the signer cannot readily repudiate the signed record as not genuine. Such procedures and controls shall include the following: (a) Validation of systems to ensure accuracy, reliability, consistent intended performance, and the ability to discern invalid or altered records. (b) The ability to generate accurate and complete copies of records in both human readable and electronic form suitable for inspection, review, and copying by the agency. (c) Protection of records to enable their accurate and ready retrieval throughout the records retention period. (d) Limiting system access to authorized individuals. (e) Use of secure, computer-generated, time-stamped audit trails to independently record the date and time of operator entries and actions that create, modify, or delete electronic records. Record changes shall not obscure previously recorded information. Such audit trail documentation shall be retained for a period at least as long as that required for the subject electronic records and shall be available for agency review and copying. §11.50 - Signature manifestations. (a) Signed electronic records shall contain information associated with the signing that clearly indicates all of the following: (1) The printed name of the signer; (2) The date and time when the signature was executed; and (3) The meaning (such as review, approval, responsibility, or authorship) associated with the signature. §11.70 - Signature/record linking. Electronic signatures and handwritten signatures executed to electronic records shall be linked to their respective electronic records to ensure that the signatures cannot be excised, copied, or otherwise transferred to falsify an electronic record by ordinary means.

2

ORGANIZATION

Part 11 is divided into three subparts: General Provisions, Electronic Records, and Electronic Signatures. Within these three sections, Part 11 states the scope of the rule; provides definitions; and states that individuals using the system(s) are to employ procedures and controls designed to ensure the authenticity, integrity, and when appropriate, confidentiality of electronic records. 2.1 General Provisions Subpart A; General Provisions of Part 11 contains three sections: 11.1—Scope, 11.2— Implementation, and 11.3—Definitions. In essence, these three sections establish a basic framework for which to employ electronic records and electronic signatures in lieu of paper records, and the criteria under which

the FDA consider electronic records and electronic signatures to be equivalent to paper records and signatures executed on paper. Additionally, the applicability of Part 11 is briefly addressed: . . . applies to records in electronic form that are created, modified, maintained, archived, retrieved, or transmitted, under any records requirements set forth in agency regulations. However, this part does not apply to paper records that are, or have been, transmitted by electronic means.

Thus, this document indicates that Part 11 would not apply to a facsimile copy of a paper record or signature required by the agency. The definitions section contains nine definitions related to ERES. Definitions are included for the following terms: Electronic Record, Electronic Signature, and Open and

4

CFR 21 PART 11

Table 3. A Selected Portion of the Signature General Requirements and Procedures Contained in Subpart C 21CFR11 Subpart C–Electronic Signatures §11.100 - General requirements (a) Each electronic signature shall be unique to one individual and shall not be reused by, or reassigned to, anyone else. (b) Before an organization establishes, assigns, certifies, or otherwise sanctions an individual’s electronic signature, or any element of such electronic signature, the organization shall verify the identity of the individual. (c) Persons using electronic signatures shall, prior to or at the time of such use, certify to the agency that the electronic signatures in their system, used on or after August 20, 1997, are intended to be the legally binding equivalent of traditional handwritten signatures. (2) Persons using electronic signatures shall, upon agency request, provide additional certification or testimony that a specific electronic signature is the legally binding equivalent of the signer’s handwritten signature. §11.200 - Electronic signature components and controls. (a) Electronic signatures that are not based upon biometrics shall: (1) Employ at least two distinct identification components such as an identification code and password. (2) Be used only by their genuine owners; and (3) Be administered and executed to ensure that attempted use of an individual’s electronic signature by anyone other than its genuine owner requires collaboration of two or more individuals. (b) Electronic signatures based upon biometrics shall be designed to ensure that they cannot be used by anyone other than their genuine owners. §11.300 - Controls for identification codes/passwords Persons who use electronic signatures based upon use of identification codes in combination with passwords shall employ controls to ensure their security and integrity. Such controls shall include: (a) Maintaining the uniqueness of each combined identification code and password, such that no two individuals have the same combination of identification code and password. (d) Use of transaction safeguards to prevent unauthorized use of passwords and/or identification codes, and to detect and report in an immediate and urgent manner any attempts at their unauthorized use to the system security unit, and, as appropriate, to organizational management.

Table 4. An Overview of the Topics Contained in the Guidance for Industry: Computerized Systems Used in Clinical Trials Overview–Guidance for Industry: Computerized Systems Used in Clinical Trials • General Principles • Standard Operating Procedures • Data Entry • System Features • Security • System Dependability • System Controls • Training of Personnel • Records Inspection •Certification of Electronic Signatures

Closed Systems. Although all definitions in this section should be noted, the definitions provided for Electronic Record, Electronic Signature, and Open and Closed Systems have historically played a very important role

in how practitioners have defined systems and approached implementation and validation of systems for clinical applications in an effort to comply with the rule. Validation and controls of the system are paramount,

CFR 21 PART 11

given that the General Provisions also state that ‘‘Computer systems (including hardware and software), controls, and attendant documentation maintained under this part shall be readily available for, and subject to, FDA inspection.’’. 2.2 Subpart B—Electronic Records Subpart B—Electronic Records contains four sections: 11.10—Controls for closed systems, 11.30—Controls for open systems, 11.50— Signature Manifestations, and 11.70— Signature and record linking. These sections address how applicable systems are controlled, how specific requirements are related to the manifestation of an electronic signature on an electronic record, linking of that signature to the electronic record, what information must be included, and the need to link the signature to all respective electronic records. The preamble of 21CFR Part11, which is essential to interpreting the final rule, is flexible on the method of linkage. It states, While requiring electronic signatures to be linked to their respective electronic records, the final rule affords flexibility in achieving that link through use of any appropriate means, including use of digital signatures and secure relational database references.

Part 11 Subpart B is designed such that the controls delineated under ‘‘Controls for Closed Systems’’ also apply to open systems. This subpart includes important topics such as: Validation ‘‘to ensure accuracy, reliability, consistent intended performance, and the ability to discern invalid or altered records’’, system access and security, copies for inspection, record retention, and the ‘‘use of secure, computer-generated, time-stamped audit trails to independently record the date and time of operator entries and actions that create, modify, or delete electronic records.’’ In addition, procedures and controls must be in place so that the signer cannot easily repudiate the signed record as not genuine. 2.3 Subpart C—Electronic Signatures Subpart C contains three sections: 11.100— General requirements, 11.200—Electronic

5

signature components and controls, and 11.300—Controls for identification codes/ passwords. It addresses electronic signatures and defines them as ‘‘a computer data compilation of any symbol or series of symbols, executed, adopted, or authorized by an individual to be the legally binding equivalent of the individual’s handwritten signature.’’ This section explains the components of electronic signatures and distinguishes between signatures that are based on nonbiometrics and those that are. It specifies that a signature must ‘‘employ at least two distinct identification components such as an identification code and password.’’ According to the rule, whether an electronic signature is based on biometrics, steps must be taken to ensure that it cannot be used by anyone but the owner of that signature. Biometrics refers to the use of automated methods to verify an individual’s identity based on measurement of that individual’s physical features and/or repeatable actions that are unique to that individual. Examples include fingerprints, voice patterns, retinal or iris scanning, facial recognition, temperature, or hand geometry. Electronic signature integrity and detection of unauthorized use is emphasized in this section also, as it requires the ‘‘use of transaction safeguards to prevent unauthorized use of passwords and/or identification codes,’’ and it allows detection and reporting of attempts at their unauthorized use to the system security unit as appropriate. In the event that an individual or company is going to employ the use of electronic signatures, the FDA requires that the party submit an affidavit to ‘‘certify to the agency that the electronic signatures in their system, used on or after August 20, 1997, are intended to be the legally binding equivalent of traditional handwritten signatures.’’ 3

PART 11 GUIDANCE

Following the issuance of Part 11 in 1997, significant discussions ensued among industry, system providers, and the Agency concerning the interpretation and the practicalities of complying with the new regulations. In particular, concerns were raised in the areas

6

CFR 21 PART 11

of validation, audit trails, record retention, record copying, and legacy systems. These concerns suggested the need for additional official guidance in the interpretation of Part 11 from the FDA. In an effort to address these concerns and to assist in implementation of the regulation, the FDA published a compliance policy guide in 1999, (CPG 7153.17: Enforcement Policy: 21 CFR Part 11; Electronic Records; Electronic Signatures) (7), and began to publish several Guidance for Industry documents that included the following: • 21 CFR Part 11; Electronic Records;

Electronic Signatures, Computerized Systems Used in Clinical Trials (Final Rule: April 1999) (8) • 21 CFR Part 11; Electronic Records;

Electronic Signatures, (Draft: August 2001) (9)

Validation

• 21 CFR Part 11; Electronic Records;

Electronic Signatures, Glossary Terms (Draft: August 2001) (10)

of

• 21 CFR Part 11; Electronic Records;

Electronic Signatures, Time Stamps (Draft: February 2002) (11) • 21 CFR Part 11; Electronic Records;

Electronic Signatures, Maintenance of Electronic Records (Draft: July 2002) (12) • 21 CFR Part 11; Electronic Records;

Electronic Signatures, Electronic Copies of Electronic Records (Draft: August 2002) (13) The stated purpose of these guidances is to describe the Agency’s current thinking on a topic; therefore, it should be noted that the FDA states that ‘‘guidance documents do not establish legally enforceable responsibilities . . . and should be viewed only as recommendations.’’ Computerized Systems Used in Clinical Trials (GCSUCT): A Guidance for Industry was originally released in April 1999 (8). ‘‘The Guidance’’ (as it is often referred to in the electronic clinical trials arena) provides additional detail in support of Part 11 and a commentary on implementation. According to the GCSUCT, it was developed

for two related purposes: to address requirements of 21 CFR Part 11 and to provide the agency’s current thinking on issues that pertain to computerized systems used to create, modify, maintain, archive, retrieve, or transmit clinical data intended for submission to the FDA. Although the original version of the GCSUCT is no longer in effect, a new updated guidance has been issued to replace the original guidance (see discussion below). The GCSUCT has been useful and pertinent as a basic guide to Part 11 compliance and implementation for clinical systems. 3.1 Re-Examination of Part 11 and the Scope and Application Guidance After release of the GCSUCT in 1999, the industry awaited additional guidances in the hope that they would provide greater clarity as to how the FDA expected the industry to comply with Part 11. The releases of each of five additional DRAFT guidances were followed by many comments, re-examinations, discussion, and some confusion within the industry. By the end of 2002, five years after Part 11 was released, industry personnel, system providers, and the FDA still found themselves struggling with the interpretation and implementation of Part 11. Many persons within the industry suggested that full compliance with Part 11 could lead to monumental additional costs for sponsors. This statement spurred the FDA to re-examine the progress of Part 11 to date and to discuss how to proceed. At the time, the FDA was also re-examining the CGMP regulation under an initiative called: Pharmaceutical CGMPs for the 21st Century: A Risk-Based Approach; A Science and Risk-Based Approach to Product Quality Regulation Incorporating an Integrated Quality Systems Approach. It only made sense to also re-examine Part 11. On February 20, 2003 the FDA released a new Draft Guidance 21 CFR Part 11; Electronic Records; Electronic Signatures: Scope and Application (14). Accompanying the guidance was the withdrawal of CPG.7153.17 Part 11 Compliance Policy and all previous Part 11 draft Guidance(s) (with the exception of the final GCSUCT guidance). This new FDA approach to 21CFR11 included

CFR 21 PART 11

the beginning of an overall re-examination of the rule. The FDA stated that this reexamination may lead to more modifications to Part 11 itself, which would be aligned with the FDA’s risk- and science-based approach to the GMP regulations announced in August of 2002 (Pharmaceutical cGMPs for the 21st century). It is important to note that the issuance of this guidance was not a general withdrawal of Part 11, but a repositioning on the part of FDA. In the Scope and Application guidance, the FDA attempts to narrow the scope and application of Part 11 by closely coupling it with the Predicate Rule. In effect, the FDA re-emphasized the original basis for Part 11, which for the clinical space is Good Clinical Practices. The Scope and Application Final Rule was issued into effect in the federal register in August 2003. The approach outlined in the guidance as stated by FDA was based on three main elements: • Part 11 will be interpreted narrowly

by the agency; we (the FDA) are now clarifying that fewer records will be considered subject to part 11. • For those records that remain subject to Part 11, we (the FDA) intend to exercise enforcement discretion with regard to part 11 requirements for validation, audit trails, record retention, and record copying in the manner described in this guidance and with regard to all part 11 requirements for systems that were operational before the effective date of part 11(known as legacy systems). • We (the FDA) will enforce all predicate rule requirements, which include predicate rule record and recordkeeping requirements.

7

systems under Part 11 requirements based on a documented risk assessment with relation to Safety, Efficacy, and Quality (SEQ) as opposed to arbitrary parameters.

The FDA refers to enforcement discretion throughout the Scope and Application guidance. The FDA expressed their intent to not take regulatory action to enforce compliance with Part 11 regarding validation, audit trail, record retention, and record copying requirements. The guidance states that the FDA intends to exercise enforcement discretion with regard to legacy systems that otherwise met predicate rule requirements prior to August 20, 1997—the date that the original Part 11 went into effect. The definition of legacy systems is not well defined in the guidance. For example, the guidance is not specific about systems in place in 1997, but the FDA is concerned about components or software that has been replaced subsequently. The language in the guidance suggests that the (legacy) system had to be in compliance with the predicate rule prior to 1997 to qualify for enforcement discretion under this guidance. Some persons in the industry take the stance that if the basic system was in place and functions as originally intended, then it would fall into the definition of a legacy system. A stricter interpretation would only allow systems that were implemented entirely prior to 1997 into the category of legacy systems. One interpretation of the guidance would suggest that it reduced the need for validation, but the majority interpretation was that this result was not what the FDA intended. Some basic tenets still hold true regarding validation and Part 11.

Several key points and concepts were introduced and discussed in the Scope and Application guidance; risk assessment was one of the more prominent topics. The FDA states:

1. Validation of is still required for systems covered by Part 11. The basic approach to validation set forth in the Predicate rule and previous guidances is not specific to 21 CFR Part 11 and had been in practice before Part 11.

We recommend that you base your approach on a justified and documented risk assessment and a determination of the potential of the system to affect product quality and safety and record integrity. This allows for narrowing the scope of

2. Although the CPG.7153.17 Part 11 Compliance Policy and all but one previous Part 11 DRAFT Guidance have been withdrawn, 21CFR11 is still in force and compliance is expected within

8

CFR 21 PART 11

the modified expectations of the Scope and Application Guidance. 3. Predicate rules have increased in importance for company management and compliance regarding Part 11, which presents an opportunity for industry to re-examine and refocus their efforts on the truly important aspects of compliance and the predicate rules regarding clinical systems and electronic records.

Computerized Systems Used in Clinical Investigations; Guidance for Industry (May 2007) (16) supplements Scope and Application guidance and serves to unify Part 11 guidances In May 2007, the FDA issued a new final guidance: Computerized Systems Used in Clinical Investigations. This guidance was based on the DRAFT guidance issued in 2004: Computerized Systems Used in Clinical Trials (15). As stated in the introduction, the guidance ‘‘provides to sponsors, contract research organizations (CROs), data management centers, clinical investigators, and institutional review boards (IRBs), recommendations regarding the use of computerized systems in clinical investigations’’, and additionally ‘‘supersedes the guidance of the same name dated April 1999; and supplements the guidance for industry on Part 11, Electronic Records; Electronic Signatures — Scope and Application and the Agency’s international harmonization efforts when applying these guidances to source data generated at clinical study sites.’’ After the Scope and Application guidance was issued, inconsistencies existed between the new guidance and the 1999 GCSUCT guidance that remained in effect. Namely, the areas identified by the FDA as areas they would exercise ‘‘enforcement discretion’’ were addressed explicitly and rather prescriptively in the GCSUCT. This information caused a misalignment in the guidelines and required the FDA to re-examine and revise the GCSUCT. The document titled, ‘‘Computerized Systems Used in Clinical Investigations’’ is the result of that work. The following are 11 key points to note in the guidance:

1. Application of the guidance; this guidance applies to: • ‘‘Computerized systems that contain any data that are relied on by an applicant in support of a marketing application, including computerized laboratory information management systems that capture analytical results of tests conducted during a clinical trial’’ • ‘‘Recorded source data transmitted from automated instruments directly to a computerized system (e.g., data from a chemistry autoanalyser or a Holter monitor to a laboratory information system)’’ • ‘‘Source documentation that is created in hardcopy and later entered into a computerized system, recorded by direct entry into a computerized system, or automatically recorded by a computerized system (e.g., an ECG reading)’’ The guidance states that it does not apply to ‘‘computerized medical devices that generate such data and that are otherwise regulated by FDA.’’ In addition, the guidance states that acceptance of data from clinical trials for decision-making purposes depends on the FDA’s ability to verify the quality and integrity of the data during FDA on-site inspections and audits. (21 CFR 312, 511.1(b), and 812)’’. 2. The introduction references and emphasizes the importance of ‘‘source data’’ to reconstruct the study. The word ‘‘source’’ is used 24 times throughout the document in reference to source data and/or documentation. This usage differs from previous guidance(s) in which ‘‘source data’’ is hardly mentioned in the electronic context. 3. The guidance focuses on electronic source data and reiterates previously stated tenets related to data integrity. ‘‘Such electronic source data and source documentation must meet the same fundamental elements of data quality (e.g., attributable, legible, contemporaneous, original, and accurate) that are expected of paper records and must

CFR 21 PART 11

comply with all applicable statutory and regulatory requirements.’’

changes that systems make automatically to adjust to daylight savings time.’’ 10. Under Section IV (e) on External Security Safeguards, the guidance states that ‘‘you should maintain a cumulative record that indicates, for any point in time, the names of authorized personnel, their titles, and a description of their access privileges. 11. Change Control: The guidance emphasizes the need to preserve data integrity during ‘‘security and performance patches, or component replacement.’’ In addition, it recommends ‘‘maintaining back up and recovery logs’’. The specificity of this section is more explicit than previous guidance(s).

4. The guidance requires that sponsors identify each step at which a computerized system will be used to create, modify, maintain, archive, retrieve, or transmit source data. 5. The recommendations section (section IV) states that: The computerized systems should be designed, ‘‘to prevent errors in data creation, modification, maintenance, archiving, retrieval, or transmission (e.g., inadvertently unblinding a study).’’ This statement is important to note; essentially the FDA is saying that computerized systems can and should be designed and used to improve the data quality and ensure data integrity. 6. Also under Section IV (c) regarding Source Documentation and Retention, the FDA explicitly states that ‘‘when original observations are entered into a computerized system, the electronic record is the source document’’ (eSource). Here the FDA makes clear their position regarding the interpretation, or definition of electronic source data. 7. Overall, the guidance is not that prescriptive, with exception section IV (d) on Internal Security Safeguards, which literally provides suggested system features and workflow on how the user should conduct a data entry session. 8. Audit Trails: The most significant point regarding audit trails is that the FDA explicitly state that ‘‘Audit trails . . . used to capture electronic record activities should describe when, by whom, and the reason changes were made to the electronic record.’’ Historically, it has been debated whether ‘‘reason for change’’ is a necessary system feature, this guidance makes clear the FDA position on ‘‘reason for change’’ as required. 9. Date and Time Stamps: The FDA states that ‘‘Any changes to date and time should always be documented . . . we do not expect documentation of time

9

4

CONCLUSION

21CFR11 has fulfilled its original intent to allow for electronic recording of data and electronic record keeping in all documents intended for submission to the FDA. The FDA’s intention on the scope and enforcement of the rule was clarified in the Scope and Application guidance of 2003. 21CFR11 is currently being ‘‘re-examined’’ by the FDA, but they have not provided a clear timeframe of when this re-examination will conclude or what the expected outcome of this initiative will be. Some industry insiders speculate that it will be sooner rather than later, and that the regulation will either be significantly changed or revoked. Others believe that the re-examination will result in simply incorporating the philosophy of the Scope and Application guidance and the recently updated GCSUCI guidance into the actual regulation. REFERENCES 1. 62 FR 13430, 21 CFR part 11; Electronic Records; Electronic Signatures. Published in the Federal Register, March 20, 1997. 2. Predicate Rule: These requirements include certain provisions of the Current Good Manufacturing Practice regulations (21 CFR Part 211), Current Good Manufacturing Practice in Manufacturing, Processing, Packing, or

10

CFR 21 PART 11 Holding of Drugs (21 CFR Part 210), the Quality System Regulation (21 CFR Part 820), the Good Laboratory Practice for Non clinical Laboratory Studies regulations (21 CFR Part 58), Investigator recordkeeping and record retention (21 CFR 312.62), IND safety reports (21 CFR 312.32), Protection of Human Subjects (21 CFR 50.27). Documentation of informed consent (21 CFR PART 56), Subpart D—Records and Reports, 56.115 IRB records; Subpart C—IRB Functions and Operations, 56.109 IRB review of research, 21 CFR PART 314—APPLICATIONS FOR FDA APPROVAL TO MARKET A NEW DRUG (NDA); Subpart B—Applications.

3. Federal Food, Drug, and Cosmetic Act (the Act). 4. Public Health Service Act (the PHS Act). 5. Paperwork Reduction Act of 1995. 6. Government Act (GPEA), 10/21/98.

Paperwork Title XVII,

Elimination P.L. 105-277,

7. CPG 7153.17: Enforcement Policy: 21 CFR Part 11; Electronic Records; Electronic Signatures. 8. Computerized Systems Used in Clinical Trials, (GCSUCT), A Guidance for Industry (Final Rule: April 1999). 9. 21 CFR Part 11; Electronic Records; Electronic Signatures, Validation (Draft: August 2001). 10. 21 CFR Part 11; Electronic Records; Electronic Signatures, Glossary of Terms (Draft: August 2001). 11. 21 CFR Part 11; Electronic Records; Electronic Signatures, Time Stamps (Draft: February 2002). 12. 21 CFR Part 11; Electronic Records; Electronic Signatures, Maintenance of Electronic Records (Draft: July 2002). 13. 21 CFR Part 11; Electronic Records; Electronic Signatures, Electronic Copies of Electronic Records (Draft: August 2002). 14. 21 CFR Part 11; Electronic Records; Electronic Signatures: Scope and Application (Final Rule: August 2003). 15. Computerized Systems Used in Clinical Trials, (GCSUCT), Guidance for Industry (Draft: September 2004). 16. Computerized Systems Used in Clinical Investigations, (GCSUCI), Guidance for Industry (May 2007).

FURTHER READING R. D. Kush, eClinical Trials, Planning and Implementation. Boston, MA: Thomson Centerwatch, 2003.

CROSS-REFERENCES Case Report Form Clinical Data Management Software for Data Management Software for Genetics/Genomics Clinical Trial/Study Conduct Electronic Submission of NDA EMEA Food and Drug Administration (FDA, USA) Good Clinical Practice (GCP) International Conference on Harmonization (ICH) Investigational New Drug Application (IND) Confidentiality HIPAA

• CHANGE: X − Y is compared by T.

ANALYZING CHANGE FROM BASELINE IN RANDOMIZED TRIALS

• FRACTION: (X − Y) ÷ X is compared

by T. • ANCOVA: Calculate Y = β 1 X + β 2 T + constant.

ANDREW J. VICKERS Memorial Sloan-Kettering Cancer Center, New York, New York

To illustrate the methods, we will use pain scores on a 0–10 scale as an example, making the common assumption that the scores can be treated as a continuous variable, rather than 11 ordered categories. Take the case of a patient with a baseline pain score of 5, which improves to a score of 3 after randomized treatment. The number entered into a between-group two-sample comparison, such as t-test or Mann-Whitney U test, will be 3 for the POST analysis, 2 for the CHANGE analysis, and 40% for the FRACTION analysis. In ANCOVA, 3 would be entered for the dependent variable (Y) and 5 for the covariate (X).

A very common scenario in medicine is for patients to present to a clinician with some sort of a chronic complaint and ask whether the severity of the complaint can be reduced by treatment. Accordingly, many clinical trials involve measuring the severity of a complaint before and after randomly allocated treatment: Patients with chronic headache, obesity, or hypertension are randomized to treatment or control to observe whether the treatment is effective for reducing pain, weight, or blood pressure. This article provides a nontechnical introduction to the statistical analysis of two-arm randomized trials with baseline and follow-up measurement of a continuous outcome.

1.1 Clinical Interpretation of Statistical Results For the sake of argument, assume that our illustrative patient represented the mean in the treatment group, and that there was a change in means from 5 to 4 in the control group. The results of the trial could be stated in the following terms:

1 METHODS FOR ANALYZING TRIALS WITH BASELINE AND FOLLOW-UP MEASURES In the simplest case, an outcome measure such as pain, weight, or blood pressure is measured only twice, once before randomization (‘‘baseline’’) and once at some point after randomization (‘‘follow-up’’), such as shortly after treatment is complete. We will use a simple notation for this introductory paper: X is the baseline measurement, Y is the followup assessment, and T is an indicator variable coded 1 for patients who receive treatment and coded 0 for controls. We will assume that only a single X and Y exist per patient so that, for example, if a weekly pain score is reported for 4 weeks after treatment, a summary statistic such as a mean is calculated and used as each patient’s datum. We will compare four different methods of analysis: POST, CHANGE, FRACTION, ANCOVA (analysis of covariance), which are defined as follows:

POST:

CHANGE:

FRACTION:

ANCOVA:

Mean pain at follow-up was 3 in the treatment group compared with 4 in controls Pain fell by a mean of 2 points in the treatment group compared with 1 point in controls Pain fell by a mean of 40% in the treatment group compared with 20% in controls As for CHANGE, where β 2 is the change score

The ease with which clinicians can understand each of these interpretations will vary with the endpoint. In the case of hypertension, where particular cut-points such as 140 mm Hg are considered important, clinicians will often want to know mean

• POST: Y is compared by T.

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

ANALYZING CHANGE FROM BASELINE IN RANDOMIZED TRIALS

posttreatment measure (POST); for obesity research, clinical questions are often expressed in terms of weight loss (CHANGE). Percentage change (FRACTION) is obviously of little relevance for weight or blood pressure, but it is often useful for a symptom score, such as pain or number of hot flashes, which would ideally be close to zero. 2 STATISTICAL POWER OF THE ALTERNATIVE ANALYSIS METHODS

1.5 1 .5

Relative Sample Size

2

Frison and Pocock (1) have published formulae that can be used to estimate the relative power of POST, CHANGE, and ANCOVA. Relative power is dependent on the correlation between the baseline and follow-up measure (ρ xy ). Assuming that the variances of the baseline and postrandomization measures are similar, CHANGE has higher power than POST if correlation is greater than 0.5 and lower power otherwise. This measure can be thought in terms of whether the baseline measure adds variation (correlation less than 0.5) or prediction (correlation greater than 0.5). ANCOVA has superior power to both POST and CHANGE irrespective of the correlation between baseline and postrandomization measure. However, if correlation is either very high or low, the power of ANCOVA is not importantly higher than either CHANGE in the former case and POST in the latter. These relationships are shown in Fig. 1.

A slight complication in the use of Frison and Pocock’s formulae is that they require data on correlation between baseline and follow-up for the data set in the randomized trial. This correlation will be different to the correlation observed in any pretrial data set because of the effects of treatment. For example, imagine that a group of researchers has a large database of pain scores taken before and after treatment with a standard analgesic agent, and the correlation between the baseline and follow-up (ρ xy ) is 0.5. In a randomized trial that compares a new drug to the standard, ρ xy will be close to 0.5 in the control arm as well as in the trial as a whole if the new drug and standard analgesic are have similar analgesic effects. However, if the new agent was completely effective, with pain scores of 0 for all patients, ρ xy would be 0 in the experimental arm, and therefore somewhere between 0 and 0.5 for the trial as a whole. However, for the moderate effect sizes observed in most clinical trials, the difference between ρ xy in the control arm and ρ xy for the whole trial will be small: For example, if ρ xy is 0.5 in the control arm, and there is an effect size of 0.5 standard deviations, then ρ xy will be approximately 0.485 for control and treatment group patients combined. No analytic formulae have been published for the power of FRACTION. Accordingly, the power of the four analytic approaches can best be compared by simulation. Our base

0 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Correlation between baseline and follow-up measures

1.0

Figure 1. Relative sample size required for different methods of analysis. Dashed line: POST. Solid grey line: CHANGE. Solid dark line: ANCOVA.

ANALYZING CHANGE FROM BASELINE IN RANDOMIZED TRIALS

scenario is a trial with a 0.5 standard deviation difference between groups and a total sample size of 60 equally divided between two arms. This sample size was given to yield a power close to 50% for the POST analysis. The correlation between baseline and follow-up measures within each group separately was varied between 0.2 and 0.8. Two other parameters were varied: whether the effect of treatment was a shift (e.g., treatment reduced scores by two points) or a ratio (e.g., treatment reduced scores by 40%), and whether change was observed in the control arm. The results are shown in the tables. Table 1 is similar to a previously published result (2). Power is highest for ANCOVA; the relative power of POST and CHANGE are as shown in Fig. 1. FRACTION has lower power than CHANGE. Table 2 is a result that has not been previously reported: The scores are reduced in the control group (e.g., the ‘‘placebo effect’’) but are reduced more in the treatment group. FRACTION has higher power than CHANGE; however, if scores increase in both groups (e.g., pain worsening over time), although less so in treated patients, the power of FRACTION is reduced. Clearly, the relative power of FRACTION and

3

CHANGE depends on the difference between baseline and follow-up measures in the control group. Tables 3 and 4 show the results of the simulation using a ratio treatment effect. The relative powers of the different methods of analysis are similar to those for a shift treatment effect. 2.1 Which Method to Use? The relative power of POST, CHANGE, and FRACTION seems to vary in complex ways depending on the correlation, type of treatment effect, and the degree of change in the treatment group. Nonetheless, the key message remains the same: ANCOVA has the greatest power irrespective of the scenario. Yet, as pointed out above, occasionally other methods may provide results of greater clinical relevance. A simple solution is based on the principle that presentation of statistical results need not be determined solely by the principal inferential analysis. The most obvious application of this principle is when outcome is measured on a continuous scale, but a binary outcome is used as an additional method to present trial data. For example, in a trial on headache pain (3), the primary outcome was a difference between days

Table 1. Power of Methods for Analyzing Trials with Baseline and Follow-up Measures where Treatment Effect is a Shift Correlation between baseline and follow-up Method POST CHANGE FRACTION ANCOVA

0.20 0.475 0.322 0.300 0.488

0.35 0.484 0.387 0.367 0.527

0.50 0.482 0.470 0.454 0.589

0.65 0.481 0.626 0.609 0.701

0.80 0.475 0.847 0.838 0.874

The trial parameters are 30 patients per group, no change between baseline and follow-up in the control group, and a mean 0.5 standard deviation decrease in the treatment group.

Table 2. Power of FRACTION for Analyzing Trials with Baseline and Follow-up Measures, Varying Change in the Control Group Correlation between baseline and follow-up Change in controls 1 SD decrease 2 SD decrease 3 SD decrease 1 SD increase

0.20 0.362 0.422 0.461 0.251

0.35 0.436 0.492 0.515 0.303

0.50 0.531 0.570 0.561 0.379

0.65 0.678 0.675 0.611 0.510

0.80 0.859 0.788 0.655 0.727

The trial parameters are 30 patients per group, and a 0.5 standard deviation decrease in the treatment group. Power for other methods is invariant to the change in controls; accordingly, Table (1) can be used as a comparison.

4

ANALYZING CHANGE FROM BASELINE IN RANDOMIZED TRIALS

Table 3. Power of Methods for Analyzing Trials with Baseline and Follow-up Measures where Treatment Effect is a Ratio Correlation between baseline and follow-up Method POST CHANGE FRACTION ANCOVA

0.20 0.513 0.337 0.307 0.525

0.35 0.524 0.402 0.370 0.570

0.50 0.520 0.488 0.457 0.630

0.65 0.518 0.646 0.608 0.743

0.80 0.513 0.860 0.829 0.903

The trial parameters are 30 patients per group and no change between baseline and follow-up in the control group. Scores in the treatment group are reduced by a ratio equivalent to a mean 0.5 standard deviation.

Table 4. Power of Methods for Analyzing Trials with Baseline and Follow-up Measures where Treatment Effect is a Ratio and Scores Improve in the Control Group Correlation between baseline and follow-up Method POST CHANGE FRACTION ANCOVA

0.20 0.523 0.340 0.378 0.537

0.35 0.534 0.406 0.448 0.579

0.50 0.530 0.491 0.543 0.640

0.65 0.527 0.649 0.688 0.754

0.80 0.522 0.862 0.871 0.908

The trial parameters are 30 patients per group and a 1 standard deviation change between baseline and follow-up in the control group. Scores in the treatment group are reduced by a ratio equivalent to a mean 0.5 standard deviation.

with headache, but the authors also reported the proportion of patients who experienced a 50% or greater reduction in headache days. Accordingly, one can report the results of the ANCOVA and then, if it would be helpful, give percentage change by dividing mean follow-up score in the control group by β 2 , which is the difference between treatments estimated by ANCOVA. Note that choice of ANCOVA as the method of analysis has implications for sample size calculation. Such calculations require an estimate of the correlation between baseline and follow-up measures (1). Also, note that sample size was not varied in the simulations: ANCOVA involves the loss of an additional degree of freedom, which can hurt power if sample sizes are very small (such as fewer than five patients per group). 2.2 Additional Advantages of ANCOVA ANCOVA has several advantages additional to statistical power in comparison with CHANGE, POST, or FRACTION. The first concerns chance imbalance between groups at baseline, for example, if pain scores at randomization were higher in the treatment group. Of course, all methods of analysis are

unconditionally unbiased, that is, their power (true positive rate) and size (false positive rate or Type I error) are close to nominal levels when averaged over a large number of trials, even if a chance baseline imbalance is observed in some trials. However, baseline imbalance may compromise any specific trial. In the case where the baseline measure is higher in the treatment group, a POST analysis will underestimate the true treatment effect and, because of regression to the mean, a CHANGE analysis will overestimate treatment benefit [empirical examples of this phenomenon have been published (4)]. ANCOVA provides a good estimate of treatment effect irrespective of chance baseline imbalance. The second advantage of ANCOVA is related to stratification. If a trial incorporates one or more stratification factors in randomization, then failure to incorporate these factors in analysis will inflate P values (5). Power can be returned to nominal levels by adding randomization strata as covariates in ANCOVA. For example, if a hypertension trial was randomized using age and weight as stratification variables, then analysis should use Y = β 1 X + β 2 T + β 3 age + β 4 weight + constant, where age and weight are entered

ANALYZING CHANGE FROM BASELINE IN RANDOMIZED TRIALS

in the same form as for stratification (e.g., as continuous variables for minimized randomization; as binary variables for randomization by stratified blocks). ANCOVA is extended to more complex designs and analyses more easily than twosample comparisons. These designs include trials that assess treatment effects that diverge over time, for example, where the effects of a treatment are cumulative, those with more than two arms (e.g., psychotherapy vs. relaxation therapy vs. no treatment), or analyses that examine whether treatment response differs by baseline severity. Although these topics are outside the scope of this article, it can be noted briefly that trials with divergent treatment effects can be analyzed either by an ANCOVA using the slope of follow-up measurements as the dependent variable or by a generalized estimating equations approach with terms for both time and time by treatment interaction. Multiarm trials can be analyzed by ANCOVA using dummy variables for treatments (e.g., ‘‘contact’’ and ‘‘talking therapy’’ would be coded 1, 1 for psychotherapy; 1, 0 for relaxation therapy; 0, 0 for controls); analyses that examine whether treatment response differs by baseline severity can use an interaction term such that Y = β 1 X + β 2 T + β 3 T.X + constant. 3 EFFECT OF NON-NORMAL DISTRIBUTIONS ON ANALYSIS ANCOVA is a parametric analysis, and it has become an article of faith that parametric statistics require either normally distributed data or sample sizes large enough to invoke the ‘‘Central Limit Theorem.’’ As an example, one popular statistics textbook states that, ‘‘Parametric methods require the observations in each group to have an approximately Normal distribution . . . if the raw data do not satisfy there conditions . . . a non-parametric method should be used’’ (6). Moreover, a considerable number of simulation studies have shown that although Type I errors of parametric methods such as the t-test are unaffected by the sample distribution, such methods are underpowered for the analysis of nonsymmetric data (7). Accordingly perhaps, the New England Journal of

5

Medicine currently includes the following in its instructions to authors. ‘‘For analysis of [continuous] measurements, nonparametric methods should be used to compare groups when the distribution of the dependent variable is not normal’’ (note that, at the time of writing, these recommendations are under revision). That said, for a linear regression, it is not the distribution of the dependent variable Y that is of interest, but the distribution of Y conditional on the independent variables. This statement means that, for an ANCOVA, the distribution of interest is not Y but X − Y, in other words, the change score. As it happens, if X and Y are both drawn from non-normal distributions, X − Y will often be distributed normally, because change scores are a linear combination and the Central Limit Theorem therefore applies. As a straightforward example, imagine that baseline and post-treatment measure were represented by a single throw of a die. The posttreatment measure has a flat (uniform) distribution, with each possible value having an equal probability. The change score has a more normal distribution: A peak is observed in the middle at zero; the chance of a zero change score is the same as the chance of throwing the same number twice, that is 1 in 6—with more rare events at the extremes—there is only a 1 in 18 chance of increasing or decreasing score by 5. A similar result can be demonstrated for a wide variety of distributions, which include uniform, asymmetric, and what is probably most common in medical research, positive skew. Distributions for baseline and follow-up measures may show extreme deviations from the normal, yet the change score tends toward normality (7). Simulation studies have shown that ANCOVA retains its power advantages over CHANGE and POST for data sampled from distributions other than the normal (7).

REFERENCES 1. L. Frison and S. J. Pocock, Repeated measures in clinical trials: analysis using mean summary statistics and its implications for design. Stat. Med. 1992; 11: 1685–1704.

6

ANALYZING CHANGE FROM BASELINE IN RANDOMIZED TRIALS

2. A. J. Vickers, The use of percentage change from baseline as an outcome in a controlled trial is statistically inefficient: a simulation study. BMC Med. Res. Methodol. 2001; 1: 6. 3. D. Melchart, A. Streng, A. Hoppe, et al., Acupuncture in patients with tension-type headache: randomised controlled trial. BMJ 2005; 331: 376–382. 4. A. J. Vickers, Statistical reanalysis of four recent randomized trials of acupuncture for pain using analysis of covariance. Clin. J. Pain 2004; 20: 319–323. 5. L. A. Kalish and C. B. Begg, The impact of treatment allocation procedures on nominal significance levels and bias. Control. Clin. Trials 1987; 8: 121–135. 6. D. G. Altman, Practical Statistics for Medical Research. London: Chapman and Hall, 1991. 7. A. J. Vickers, Parametric versus nonparametric statistics in the analysis of randomized trials with non-normally distributed data. BMC Med. Res. Methodol. 2005; 5: 35.

FURTHER READING L. J. Frison and S. J. Pocock, Linearly divergent treatment effects in clinical trials with repeated measures: efficient analysis using summary statistics. Stat. Med. 1997; 16: 2855–2872.

L. Frison and S. J. Pocock, Repeated measures in clinical trials: analysis using mean summary statistics and its implications for design. Stat. Med. 1992; 11: 1685–1704. Y. K. Tu, A. Blance, V. Clerehugh, and M. S. Gilthorpe, Statistical power for analyses of changes in randomized controlled trials. J. Dent. Res. 2005; 84: 283–287. A. J. Vickers, Analysis of variance is easily misapplied in the analysis of randomized trials: a critique and discussion of alternative statistical approaches. Psychosom. Med. 2005; 67: 652–655.

CROSS-REFERENCES Longitudinal data (study design Phase III) ANCOVA (methodology / analysis) Generalized estimating equations (methodology / analysis) Linear model (methodology / analysis)

CHEMISTRY, MANUFACTURING AND CONTROLS (CMC)

should be addressed to the appropriate clinical division (8) and directed to the document control center at the appropriate address indicated below.

NORMAN R. SCHMUFF 1. For drug products regulated by CDER, send the IND submission to the Central Document Room, Center for Drug Evaluation and Research, Food and Drug Administration, 5901–B Ammendale Rd., Beltsville, MD 20705–1266. 2. For biological products regulated by CDER, send the IND submission to the Central Document Room, Center for Drug Evaluation and Research, Food and Drug Administration, 5901–B Ammendale Rd., Beltsville, MD 20705–1266. 3. For biological products regulated by CBER, send the IND submission to the Document Control Center (HFM–99), Center for Biologics Evaluation and Research, Food and Drug Administration, 1401 Rockville Pike, Suite 200 N, Rockville, MD 20852–1448.

U.S. Food and Drug Administration Silver Spring, Maryland

DAVID T. LIN Biologics Consulting Group, Inc. Alexandria, Virginia

1

OVERVIEW

Formerly, the U.S. Food and Drug Administration (FDA) used the term ‘‘Chemistry, Manufacturing, and Controls’’ (CMC) to refer to what other countries and regions called the ‘‘Quality’’ portion of a new drug submission. The terms are now used interchangeably at FDA. CMC/Quality requirements for INDs are spelled out briefly in the Code of Federal Regulations (CFR) (1). Note that under FDA’s Good Guidance Practices (2,3), items that are obligatory are referred to as ‘‘requirements.’’ Requirements are contained either in laws or legislation (4) (e.g., The Food, Drug and Cosmetic Act (5) or the Public Health Service Act (6) for biological products) or regulation, which, in this case, is a formal FDA-drafted elaboration and interpretation of the legislation, contained in Title 21 of the CFR. These requirements comprise what sponsors and applicants ‘‘must’’ do, whereas recommendations, which are generally contained in guidance documents, are not mandatory. 2

Quality aspects of these applications are assessed either by the Office of New Drug Quality Assessment (ONDQA) or the Office of Biotechnology Products (OBP); however, sponsors should rely on the clinical division as their primary contact point. ONDQA currently handles all of the traditional small molecule and hormone protein (e.g., the insulins and growth hormones) products. OBP handles most of the remaining biological products under CDER’s review jurisdiction.

ADMINISTRATIVE ASPECTS 3

INDs for vaccines, blood products, and gene therapy are under the review jurisdiction of CBER products. CDER currently has jurisdiction over small molecule drugs and biological drugs derived from natural sources, whether chemically synthesized or from recombinant DNA technology. For CDER or CBER, electronic INDs can be sent via FDA’s secure electronic gateway (7) that was established in 2007. Paper-based IND applications and electronic submission on physical media

PRE-INDS

CDER offers a Pre-Investigational New Drug Application (pre-IND) consultation program to foster early communications between sponsors and new drug review divisions to provide guidance on the data necessary to be submitted in INDs. The review divisions are organized generally along therapeutic class and can each be contacted using the designated ‘‘Pre-IND Consultation List’’ (9).

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

4

CHEMISTRY, MANUFACTURING AND CONTROLS (CMC)

QUALITY FOR IND TYPES

4.1 IND Categories The two IND categories, both of which are standard INDs not distinguished by the IND regulations, are as follows: • Commercial • Research (noncommercial)

‘‘Commercial’’ INDs are filed by pharmaceutical firms intending to file subsequent New Drug Applications (NDAs)/BLAs to commercialize their product. IND filings from large research institutions, such as the National Institutes of Health are also considered to be in this category. ‘‘Research’’ INDs may be submitted by a physician who both initiates and conducts an investigation, and under whose immediate direction the investigational drug is administered or dispensed. A physician might submit a research IND to propose studying an unapproved drug or studying an approved product for a new indication or in a new patient population. The quality data expectations for these applications are the same and are described in general in 21 CFR 312 (See Code of Federal Regulation (CFR) article). 4.2 Special IND Types The three special IND types are as follows: • Emergency Use • Treatment IND • Exploratory IND

An ‘‘Emergency Use’’ IND allows the FDA to authorize use of an experimental drug in an emergency situation that does not allow time for submission of an IND in accordance with 21 CFR 312.23 or 21 CFR 312.34. It is also used for patients who do not meet the criteria of an existing study protocol or for cases in which an approved study protocol does not exist. Typically, these INDs involve single patients. In nearly all cases for quality data, the application references other existing applications.

A ‘‘Treatment IND’’ is submitted for experimental drugs that show promise in clinical testing for serious or immediately lifethreatening conditions, while the final clinical work is conducted and the FDA review takes place. (For additional information see Treatment IND article) In January of 2006, CDER issued ‘‘Guidance for Industry, Investigators, and Reviewers Exploratory IND Studies.’’ An ‘‘exploratory IND study’’ is defined as a clinical trial that • is conducted early in Phase 1, • involves very limited human exposure,

and • has no therapeutic nor diagnostic intent

(e.g., screening studies or microdose studies). The extent of CMC-related IND information is stated to be ‘‘similar to that described in current guidance for use of investigational products.’’ For CMC purposes, little difference is found in the data expectations for this study and a traditional Phase 1 study. 4.3 IND Phases The regulations at 21 CFR 312.23(a)(7)(i) emphasize the graded nature of manufacturing and controls information. Although in each phase of the investigation sufficient information should be submitted to ensure the proper ‘‘identity strength quality and purity’’ of the investigational drug, the amount of information needed to make that assurance will vary with the phase of the investigation, the proposed duration of the investigation, the dosage form, and the amount of information otherwise available. 5 QUALITY-BY-DESIGN AND INDS As part of the Pharmaceutical Quality for the 21st Century Initiative (10), FDA is encouraging a more systematic approach to pharmaceutical development and manufacturing that generally falls under the concept of Quality-by-Design (QbD). The QbD approach suggests that product quality should be built in from the ground up by designing each

CHEMISTRY, MANUFACTURING AND CONTROLS (CMC)

manufacturing step for optimal product performance at every stage of the development process. QbD is defined in the Step 2 ICH Annex to Q8 (11) as: A systematic approach to development that begins with predefined objectives and emphasizes product and process understanding and process control, based on sound science and quality risk management.

ONDQA Director Moheb Nasr described the QbD approach this way in his November 2006 presentation (12) at the American Association of Pharmaceutical Scientists Annual Meeting: In a Quality by Design system: • The product is designed to meet patient

needs and performance requirements • The process is designed to consistently

meet product critical quality attributes • The impact of starting raw materials

and process parameters on product quality is understood • The process is continually monitored, evaluated and updated to allow for consistent quality over time • Critical sources of process variability are identified and controlled • Appropriate control strategies are developed As described below, much of the QbD aspects of an IND or NDA/BLA will be included in the P.2 Pharmaceutical Development section of the application. Although QbD is not a statutory requirement, FDA believes this approach to be beneficial to both sponsors/applicants and to the public. 6

ELECTRONIC SUBMISSIONS FOR INDS

Submissions of INDs in the electronic Common Technical Document (eCTD) format (13) have been strongly encouraged, although not required. However, as of January 1, 2008, the only acceptable electronic submissions to FDA are submissions in the eCTD format (14). Advantages to FDA include less required storage space for paper submissions, expedited routing, and simultaneous

3

availability to reviewers and managers. An advantage to both sponsors and the agency is the ‘‘lifecycle’’ features built into the eCTD. These features allow for an historical view of changes within a given section. It is also noteworthy that submission redundancies can be eliminated (e.g., submitting the same study report to both an IND and an NDA/BLA) because subsequent applications, both IND and NDA/BLA, simply can include links to previously submitted files. In 2007, an FDA electronic gateway (15) was established to facilitate timely submission and eliminate the need for submissions on physical media. Note that for purposes of eIND submissions (but not NDAs/BLAs), it is not necessary to comply with the ‘‘Granularity’’ Annex (16) to ICH M4. That is, it would be acceptable to submit a single file for the Module 2 Quality Overall Summary (consistent with the ‘‘Granularity’’ document), a single file for the ‘‘S’’ drug substance section, and a single file for the ‘‘P’’ drug product section. These latter two options are in conflict with the ‘‘Granularity’’ Annex but nonetheless acceptable for eINDs. 7

GENERAL CONSIDERATIONS

For the proposed use of a U.S. marketed product in unmodified form and in its original packaging, the sponsors need only to indicate this use. For products that are modified (e.g., over-encapsulated tablets), the IND should include some evidence of equivalent performance of the original to the modified product for the duration of the proposed trial. For repackaged products, data should be included to demonstrate that the product is stable for the duration of the trial. For placebo-controlled trials, component and composition data for the placebo could be included in a separate drug product section. A brief rationale should be included for why the blinding is adequate; it should address, for example, issues relating to appearance, color, and taste. Foreign-sourced comparators present special challenges. Generally, a U.S.-sourced comparator product is preferred. The use of FDA-approved drug products provides assurance of drug quality. Where this use is not

4

CHEMISTRY, MANUFACTURING AND CONTROLS (CMC)

possible and local products are used, documentation should be provided to show that the drug product is comparable in quality to the U.S. product. This documentation could involve, for example, comparing impurity, dissolution profiles and content uniformity. Without an adequate showing of comparability, study results would be considered inconclusive and would likely not be adequate for support of approval of a new agent, or a new indication for an existing product. In certain cases, a showing of comparability might involve an in vivo bioequivalence study. For cases in which it is desired to use a foreign comparator, when no U.S. approved product is available, discussions with FDA should be occur at an early development stage before filing the IND. 8

CMC CLINICAL HOLD CONSIDERATIONS

A clinical hold is as follows: An order issued by FDA to the sponsor to delay a proposed clinical investigation or to suspend an ongoing investigation. The clinical hold order may apply to one or more of the investigations covered by an IND. When a proposed study is placed on clinical hold, subjects may not be given the investigational drug. When an ongoing study is placed on clinical hold, no new subjects may be recruited to the study and placed on the investigational drug; patients already in the study should be taken off therapy involving the investigational drug unless specifically permitted by FDA in the interest of patient safety (17)

Every effort is made by FDA to facilitate the initiation of clinical trials. When serious deficiencies are identified before the 30-day date, sponsors are generally notified by telephone or e-mail. It is especially important in this period to respond promptly and completely to FDA requests. It is advantageous to have the primary contact available during FDA working hours during this time. Any CMC deficiencies should be addressed through a formal submission to the IND. The previously mentioned FDA electronic gateway (18) was established to facilitate timely submission and reviewer availability of such IND amendments.

For all disciplines, including CMC, the review focus is on the safety of the proposed clinical trial. Consequently, issues relating to commercial manufacture are generally not pertinent. However, inadequate attention to product variability can be a safety issue and may compromise the results of a clinical trial. For ‘‘Research’’ INDs, the most common CMC reasons for a clinical hold relates to lack of administrative information, such as a reference to an existing IND, and a letter of authorization from that IND’s sponsor. Issues may also develop relating to the performance or stability of a repackaged or reformulated commercial product. Any manipulation of a sterile product is also of special concern. For ‘‘Commercial’’ INDs, quoting the Phase 1 guidance: The identification of a safety concern or insufficient data to make an evaluation of safety is the only basis for a clinical hold based on the CMC section. Reasons for concern may include, for example: 1) a product made with unknown or impure components; 2) a product possessing chemical structures of known or highly likely toxicity; 3) a product that cannot remain chemically stable throughout the testing program proposed; or 4) a product with an impurity profile indicative of a potential health hazard or an impurity profile insufficiently defined to assess a potential health hazard; or 5) a poorly characterized master or working cell bank.

Regarding impurities, it is critical that the material proposed for use in the clinical trial have a similar impurity profile to that used in nonclinical studies. For example, the same drug substance lot used in nonclinical studies might be used to manufacture clinical trial lots. If impurities appear only in the drug product, then it is imperative that these, too, should be appropriately qualified. 9 SUBMISSION FORMAT AND CONTENT The content, but not the format, of CMC section for INDs is prescribed by 21 CFR 312.23(a)(7). In 1995, CDER and CBER issued ‘‘Guidance for Industry: Content and Format of Investigational New Drug Applications (INDs) for Phase 1 Studies of Drugs,

CHEMISTRY, MANUFACTURING AND CONTROLS (CMC)

Including Well- Characterized, Therapeutic, Biotechnology-derived Products’’ (19). This predated work on the International Conference on Harmonisation (20) (ICH) topic M4, the Common Technical Document (CTD), which reached Step 4 of this FDA guidance in November of 2000. Although the content is still pertinent, the submission should now be formatted according to the CTD format as described below. In May of 2003, CDER issued ‘‘Guidance for Industry: INDs for Phase 2 and Phase 3 Studies Chemistry, Manufacturing, and Controls Information’’ (21). Like the Phase 1 guidance described above it includes no references are to the CTD. Nevertheless, as for Phase 1 submissions, these, too, should be submitted in the CTD format, preferably as an eCTD. 9.1 Module 1 Module 1 contains region-specific data, largely administrative, that falls outside the scope of the ICH CTD. For eCTD applications, the format and content of Module 1 is described by FDA (22), Japanese MHLW/PMDA (23), and European EMEA (24). Quality-specific data for FDA should include, if appropriate, a request for categorical exclusion from or submission of an environmental assessment (25). Also, this module should include a copy of all labels and labeling to be provided to each investigator (26). A mock-up or printed representation of the proposed labeling that will be provided to investigator(s) in the proposed clinical trial should be submitted. Investigational labels must carry a ‘‘caution’’ statement as required by 21 CFR 312.6(a). That statement reads: ‘‘Caution: New Drug Limited by Federal (or United States) law to investigational use.’’ Although INDs are generally exempt from expiration dating, where new drug products for investigational use are to be reconstituted at the time of dispensing, their labeling must bear expiration information for the reconstituted drug product (27). 9.2 Module 2 Unlike Efficacy, and Safety, Quality has a single Module 2 heading known as the ‘‘Quality Overall Summary’’ (QOS). FDA has issued

5

no guidance on IND expectations for the QOS or other Module 2 summaries. Consequently, the inclusion of a QOS in an IND can be considered optional. However, if submitting a QOS, it seems appropriate to limit its length to five or fewer pages. The format could be as described in M4-Q (28), or it might be abbreviated as that guidance applies to NDAs/BLAs and ANDAs. 9.3 Module 3 If a QOS is not included, then provide a summary of the application in Module 3 (29). The file should be submitted under eCTD (v 3.2) element ‘‘m3-quality.’’ Although it is not obvious from the eCTD specification, it is permissible to submit documents under this heading (or element, to use XML terminology). A look at the eCTD technical specification will reveal that this possibility also exists for many other headings with no content other than subordinate headings. As suggested in the Phase 1 guidance (30), the sponsor should state whether it believes that either 1) the chemistry of either the drug substance or the drug product, or 2) the manufacturing of either the drug substance or the drug product, presents any signals of potential human risk. If so, then these signals of potential risks should be discussed, and the steps proposed to monitor for such risks should be described, or the reasons why the signals should be dismissed should be discussed. In addition, sponsors should describe any chemistry and manufacturing differences between the drug substance/product proposed for clinical use and the drug substance/product used in the animal toxicology trials that formed the basis for the sponsor’s conclusion that it was safe to proceed with the proposed clinical study. How these differences might affect the safety profile of the drug product should also be discussed. If no differences are found, then this fact should be stated explicitly. 10

S DRUG SUBSTANCE

A short summary, a few paragraphs in length, describing the drug substance should be provided. If the IND proposes a U.S.-approved

6

CHEMISTRY, MANUFACTURING AND CONTROLS (CMC)

drug, then the entire drug substance section can be omitted. If it is proposed to use a U.S.approved drug manufactured by an alternate supplier, then complete CMC information should be included.

10.2.1 Manufacturer(s)P1 . Provide the complete name and street address for the drug substance manufacturer and its FDA Establishment Identifier number (FEI) and/ or Central File Number (CFN) if known.

10.1 General InformationP1 10.1.1 NomenclatureP1 . Provide all names (including, e.g., company code, INN, and USAN) by which the substance has been known. Pick one of these names and use it consistently throughout the application. If legacy reports use alternate names, then it will be helpful to the reviewer if you can modify these reports so that a single standard name is used across the application. Follow a similar practice for naming impurities. Also, include a systematic name, preferably the name used by the Chemical Abstracts Service (CAS). Include the CAS Registry Number (CAS RN) for the drug and any relevant salt forms.

10.2.2 Description of Manufacturing Process and Process ControlsP1 . A brief narrative description should be included along with a detailed flow diagram. The latter should indicate weights, yields, solvents, reagents, in-process controls, and isolated intermediates. Critical steps should be identified, if known. Include milling or size-reduction steps. More information may be needed to assess the safety of biotechnology-derived drugs or drugs extracted from human or animal sources. Such information includes the cell bank, cell culture, harvest, purification and modification reactions, and filling, storage, and shipping conditions. The general description of the synthetic and manufacturing process (e.g., fermentation and purification) provided to support the Phase 1 studies should be updated from a safety perspective if changes or modifications have been introduced. Reprocessing procedures and controls need not be described except for natural sourced drug substances when the steps affect safety (e.g., virus or impurity clearance). For sterile drug substances, updates on the Phase 1 manufacturing process should be submitted. The Phase 2 information should include changes in the drug substance sterilization process (e.g., terminal sterilization to aseptic processing). Information related to the validation of the sterilization process need not be submitted at this time.

10.1.2 StructureP1 . Provide a legible graphical structure and clearly indicate any relevant stereochemical features. Include the molecular mass. For biotechnological materials, provide schematic amino acid sequence and indicate secondary and tertiary structure, if known, and sites of posttranslational modifications. It may be helpful to provide a machine-readable representation of the structure, such as a SMILES (31) string, MDL MOLFILE (32), or InChI (33). 10.1.3 General PropertiesP1 . Include melting point, and other physico-chemical properties, such as log P, and a pH solubility profile. Indicate which polymorph is proposed for use, as appropriate. Also, include a short summary of biological activity. 10.2

ManufactureP1

After the initial IND submission, if changes have been made in manufacturing sites or processes, then provide a short summary in the following sections. Amend the IND only if such changes involve lots used for stability studies or preclinical or clinical testing. P1 These sections should be included in submissions

for Phase 1 studies.

10.2.3 Control of MaterialsP1 . Indicate the quality standards for the starting material, solvents, and reagents. For fermentation products or natural substances extracted from plant, human, or animal sources, the following information should be provided in Phase 1 submissions: (1) origin (e.g., country), source (e.g., pancreas), and taxonomy (e.g., family, genus, species, and variety) of the starting materials or strain of the microorganism, (2) details of appropriate screening procedures for adventitious agents, if relevant, and (3) information to support the

CHEMISTRY, MANUFACTURING AND CONTROLS (CMC)

safe use of any materials of microbial, plant, human, or animal origin (e.g., certification, screening, and testing). Any updates to the information submitted in Phase 1 and any new information to support the safety of materials of human or animal origin should be provided in fater submissions. 10.2.4 Controls of Critical Steps and Intermediates. The general strategy for identifying critical steps and the nature of controls for these steps, if identified, should be discussed briefly. For isolated intermediates, provide a tentative specification. To the extent possible in Phase 2, sponsors should provide information on controls of critical steps and intermediates and tentative acceptance criteria to ensure that the manufacturing process is controlled at predetermined points. Although controls of critical steps and intermediates still can be in development, information on controls for monitoring adventitious agents should be provided for fermentation and natural sourced (human or animal) drug substances, as appropriate. 10.2.5 Process Validation and/or EvaluationP1 . Generally this section is not needed in an IND submission. However, it might be appropriate, for example, to describe criteria used to assess clearance of adventitious agents. 10.2.6 Manufacturing Process Development. Although only safety-related aspects need to be reported for Phase 1 studies, this section provides an opportunity for a brief description of the sponsor’s ‘‘Quality-byDesign’’ approach. 10.3 CharacterizationP1 10.3.1 Elucidation of Structure and other Characteristics. Evidence to support the proposed chemical structure of the drug should be submitted. It is understood that the amount of structure information will be limited in the early stage of drug development. Data should be summarized in tabular form and indicate, for example, peak assignments and coupling constants for 1 H NMR spectra. Infrared and 1 H NMR spectra should be submitted. For the latter, provide expansions of regions of interest. For biotechnological

7

materials, information should be provided on the primary, secondary, and higher-order structure. In addition, information on biological activity, purity, and immunochemical properties (when relevant) should be included. 10.3.2 ImpuritiesP1 . This section should provide an HPLC that clearly indicates impurities and should provide structural identification when possible. If structures have not been assigned, then spectra and/or spectral data should be included. Although Q3A does not apply to drug substances for investigational studies, it is advisable follow its general principles in that thresholds should be established for reporting, identification, and qualification. Subsequently, new impurities (e.g., from a change in synthetic pathway) should be qualified, quantified, and reported, as appropriate. Procedures to evaluate impurities to support an NDA/BLA (e.g., recommended identification levels) may not be practical at this point in drug development. Suitable limits should be established based on manufacturing experience, stability data, and safety considerations. 10.4

Control of Drug SubstanceP1

10.4.1 SpecificationP1 . ICH Q6A (34) and Q6B (35) define specification as . . . a list of tests, references to analytical procedures, and appropriate acceptance criteria which are numerical limits, ranges, or other criteria for the tests described.’’ A tentative specification should include the ‘‘Universal Tests / Criteria’’ defined in Q6A as well as the appropriate ‘‘Specific Tests / Criteria.’’ An individual test and acceptance criteria should be in place for impurities above an identification threshold defined by the sponsor. The specification should also include an acceptance criterion for total impurities. 10.4.2 Analytical ProceduresP1 . Analytical procedures used to assure the identity, strength, quality, and purity of the drug product must be included. Except for compendial procedures, a general description of each test method should be provided.

8

CHEMISTRY, MANUFACTURING AND CONTROLS (CMC)

10.4.3 Validation of Analytical Procedures. Although data should support that the analytical procedures are suitable for their intended purpose, validation data ordinarily need not be submitted at the initial stage of drug development. However, for some well-characterized, therapeutic biotechnology-derived products, validation data may be needed in certain circumstances to ensure safety in Phase 1. For Phase 2/3 studies, the analytical procedure (e.g., high-pressure liquid chromatography) used to perform a test and to support the tentative acceptance criteria should be described briefly, and changes should be reported when the changes are such that an update of the brief description is warranted. A complete description of analytical procedures and appropriate validation data should be available for the analytical procedures that are not from an FDA-recognized standard reference (e.g., official compendium and AOAC International Book of Methods), and this information should be submitted on request. 10.4.4 Batch AnalysesP1 . Batch analyses for all relevant batches should be included. Where possible, test results should be given as numeric values, not just ‘‘pass’’ or ‘‘fail.’’ The use of each batch should be indicated, with a clear indication of whether the proposed clinical trial batch is the same as that used in preclinical testing. 10.4.5 Justification of Specification. Justification of the interim specification should rely on all relevant data, including, for example, batches used in preclinical testing and development. 10.5

Reference Standards or MaterialsP1

Some information should be provided on reference standards. For biological and biotechnological products, if the working reference standard is based on a national or international standard (e.g., World Health Organization), then information should be provided on that reference standard. 10.6

Container Closure SystemP1

A general brief description of the container closure system should be provided. Include,

if known, whether the components in direct contact with the drug substance comply with the indirect food additive regulations in 21 CFR 174.5. 10.7 StabilityP1 Sufficient information should be submitted to support the stability of the drug substance during the toxicologic studies and the proposed clinical studies (see stability analysis and stability study design chapters). 10.7.1 Stability Summary and ConclusionsP1 . Summarize the data from section S.7.3. 10.7.2 Post-approval Stability Protocol and Stability Commitment. This section is not appropriate to an IND and can be deleted. Note that in the eCTD, most sections (elements) are optional. If sections are deleted, then it is important, however, not to change any of the element names that include the section numbers (i.e., if a section like this one is not included, then do not renumber subsequent sections). 10.7.3 Stability DataP1 . Provide the stability data in tabular format. A graphical display of the data can be helpful. 11 DRUG PRODUCT Provide a short drug product summary here. A separate Drug Product section should be provided for any comparator products. If a placebo is used in the trial, then it, too, can be in a separate Drug Product section, but to facilitate comparisons, it is preferable to include this information in the same section as the drug product. If the IND is for a U.S.-approved drug, and the drug product is used in unmodified form in its original container, then this situation should be clearly indicated. If the product is modified (e.g., the marketed tablet is over-encapsulated) or packaged in something other than the marketed container, then information assuring equivalent product performance should be included in the appropriate section.

CHEMISTRY, MANUFACTURING AND CONTROLS (CMC)

11.1 Description and Composition of the Drug ProductP1 A description of the drug product and its composition should be provided. The information provided should include, for example: • Description of the dosage form, • Composition, for example, list of all com-

ponents of the dosage form and their amount on a per-unit basis (including overages, if any), the function of the components, and a reference to their quality standards (e.g., compendial monographs or manufacturer’s specifications), • Description of accompanying reconstitution diluent(s), and • Type of container and closure used for the drug product and accompanying reconstitution diluent, if applicable. Note that this section includes the components and composition of the unit dose but not the batch formula that is in section P.3.2. 11.2 Pharmaceutical Development This section provides an opportunity to describe Quality-by-Design efforts that may have been built into the drug product. The expectations for a Phase 1 IND are quite modest, and most information indicated in M4Q (R1) can be considered optional. For more extensive P.2 guidance, consult the ICH Q8 ‘‘Pharmaceutical Development’’ guidance (36). Note however that the foregoing documents strictly apply to NDAs, and BLAs. 11.2.1 Components of the Drug Product 11.2.1.1 Drug Substance At minimum, the compatibility of the drug substance with excipients listed in P.1 should be discussed. Likewise, for fixed dose products, containing more than one active ingredient the compatibility of drug substances with each other should be addressed. 11.2.1.2 Excipients The choice of excipients listed in P.1, their concentration, and their characteristics that can influence the drug product performance should be discussed relative to their respective functions.

9

11.2.2 Drug Product 11.2.2.1 Formulation Development A brief summary that describes the development of the drug product should be provided, taking into consideration the proposed route of administration and usage. 11.2.2.2 Overages Any overages in the formulation(s) described in P.1 should be justified. 11.2.2.3 Physico-Chemical and Biological Properties Parameters relevant to the performance of the drug product, such as pH, ionic strength, dissolution, redispersion, reconstitution, particle size distribution, aggregation, polymorphism, rheological properties, biological activity or potency, and/or immunological activity, should be addressed. 11.2.3 Manufacturing Process Development. A general discussion of the approach to process development could be included here, with any specifics on what has been done so far, and what is planned. 11.2.4 Container Closure System. A brief description of the suitability of the container closure for its intended purpose could be discussed, considering, for example, choice of materials, protection from moisture and light, compatibility of the materials of construction with the dosage form (including sorption to container and leaching), safety of materials of construction, and performance (such as reproducibility of the dose delivery from the device when presented as part of the drug product). 11.2.5 Microbiological Attributes. Where appropriate, the microbiological attributes of the dosage form should be discussed, as well as the selection and effectiveness of preservative systems in products that contain antimicrobial preservatives. If applicable, then include the rationale for not performing microbial limits testing or not including preservatives in, for example, multiple-use liquid products. 11.2.6 Compatibility. The compatibility of the drug product with reconstitution diluents or dosage devices (e.g., precipitation of drug substance in solution, sorption on injection vessels, and stability) should be addressed.

10

11.3

CHEMISTRY, MANUFACTURING AND CONTROLS (CMC)

ManufactureP1

11.3.1 Manufacturer(s)P1 . The name, address, and responsibility of each manufacturer, including contractors, and each proposed production site or facility involved in manufacturing and testing should be provided, as well as the Central File Number (CFN)/ Firm Establishment Identifier (FEI) identifying number. 11.3.2 Batch FormulaP1 . A batch formula should be provided that includes a list of all components of the dosage form to be used in the manufacturing process, their amounts on a per batch basis, including overages, and a reference to their quality standards. 11.3.3 Description of Manufacturing Process and Process ControlsP1 . A diagrammatic presentation and a brief written description of the manufacturing process should be submitted, including sterilization process for sterile products. Flow diagrams are suggested as the usual, most effective, presentations of this information. If known, then critical steps could be indicated. 11.3.4 Controls of Critical Steps and Intermediates. If critical steps are identified in P.3.3, then corresponding controls should be briefly identified. 11.3.5 Process Validation and/or Evaluation. Although process validation is not expected for drug product manufactured during the IND stages of development, information appropriate to the development stage should be available to demonstrate that each process step accomplishes its intended purpose. 11.4

Control of ExcipientsP1

Generally, a reference to the current USP is adequate in this section. For atypical use of compendial excipients (e.g., lactose in a dry-powder inhaler), control of other attributes may be appropriate. Quality standards for noncompendial excipients should be described briefly. If subsections of P.4 are not needed, then the corresponding headings can be deleted.

11.4.1 Specifications. This section need only be included if appropriate, as described above. 11.4.2 Analytical Procedures. This section need only be included if appropriate, as described above. 11.4.3 Validation of Analytical Procedures. This section need only be included if appropriate, as described above. 11.4.4 Justification of Specifications. This section need only be included if appropriate, as described above. 11.4.5 Excipients of Human or Animal OriginP1 . For excipients of human or animal origin, information should be provided regarding adventitious agents (e.g., sources, specifications; description of the testing performed; and viral safety data). 11.4.6 Novel ExcipientsP1 . For excipients used for the first time in a drug product or by a new route of administration, details of manufacture, characterization, and controls, with cross references to support safety data (nonclinical and/or clinical), should be provided directly in the IND or by reference to a DMF, another IND, or a NDA/BLA. 11.5 Control of Drug ProductP1 11.5.1 Specification(s)P1 . The ICH Q6A defines specification as ‘‘ . . . a list of tests, references to analytical procedures, and appropriate acceptance criteria which are numerical limits, ranges, or other criteria for the tests described.’’ A tentative specification should include the ‘‘Universal Tests / Criteria’’ defined in Q6A as well as the appropriate ‘‘Specific Tests / Criteria.’’ An individual test and acceptance criteria should be in place for degradation products above an identification threshold defined by the sponsor. The specification should also include an acceptance criterion for total impurities. 11.5.2 Analytical ProceduresP1 . The analytical procedures used for testing the drug product should be described briefly.

CHEMISTRY, MANUFACTURING AND CONTROLS (CMC)

11.5.3 Validation of Analytical Procedures. Validation data ordinarily need not be submitted at the initial stage of drug development. However, sponsors should have data in hand to demonstrate that the procedures are suitable for their intended use. For some well-characterized, therapeutic biotechnology-derived products, method qualification (37) data may be needed to ensure safety in Phase 1.

11

of the components in the container closure system should be provided. Additional information may be requested for atypical delivery systems such as metered dose inhalers and disposable injection devices. Include, if known, whether the components in direct contact with the drug substance comply with the indirect food additive regulations in 21 CFR 174.5. 11.8

StabilityP1

P1

11.5.4 Batch Analyses . A description of batches, their use (including study numbers), and results of batch analyses should be provided. The batch proposed for clinical use should be clearly identified. 11.5.5 Characterization of Impurities. Information on the characterization of impurities should be provided, if not previously provided in ‘‘S.3.2 Impurities.’’ 11.5.6 Justification of Specification. Justification for the proposed interim drug product specification should be provided. It is understood that the specification will evolve during the drug’s development. For degradants, although ICH Q3B does not apply to drug products for investigational studies, it is advisable to follow its general principles in that thresholds should be established for reporting, identification, and qualification. As with Q3B, acceptance criteria should be set for individual degradants that are present above the identification threshold. A general acceptance criterion for unspecified impurities should be set at less than or equal to the identification threshold.

(See also stability analysis and stability study design chapters.) 11.8.1 Stability Summary and ConclusionP1 . Summarize the data provided in section 8.3. Note that although expiry dating of IND products is not required, it is necessary to obtain sufficient data to support the stability of product for the duration of the clinical trial. However, where new drug products for investigational use are to be reconstituted at the time of dispensing, their labeling must bear expiration information for the reconstituted drug product. 11.8.2 Post-approval Stability Protocol and Stability Commitment. This section is not appropriate for an IND. 11.8.3 Stability DataP1 . For Phase 1, provide available stability data in tabular format. For Phase 2 and 3, provide data on clinical product used in Phase 1 and 2, respectively, in tabular format. 12 MEETINGS AND OTHER COMMUNICATIONS

11.6 Reference Standards or MaterialsP1 Some information should be provided on reference standards. For biological and biotechnological products, if the working reference standard is based on a national or international standard (e.g., World Health Organization), then information should be provided on that reference standard. 11.7 Container Closure SystemP1 The container closure system is defined as all packaging components that together contain and protect the product. A brief description

Both CFR sections (38) and FDA guidance (39) relate to meetings. Firms have generally been encouraged to hold CMC-specific meetings at the end–of-Phase 2 (EOP2). However, for pre-IND questions, a preference is expressed for written questions, which will receive written answers. Many times questions can be clarified by reference to existing guidance documents. Sometimes a follow-up telephone conference can clear up any unresolved issues. For matters that require extensive discussion, a meeting may be appropriate.

12

13

CHEMISTRY, MANUFACTURING AND CONTROLS (CMC)

GMPs FOR CLINICAL TRIALS

A direct final rule exempting most Phase 1 products, including biological products, from the GMP regulations (21 CFR 211) was issued on January 17, 2006 (40). A proposed rule with language similar to the direct final rule and a companion draft guidance (41) were issued on the same date. These documents provided FDA’s recommendations on approaches to complying with current good manufacturing practice as required in the 501(a)(2)(B) statute of the Food, Drug, and Cosmetic Act (FD&C Act). Although the final rule exempting Phase 1 products from the GMP regulations was withdrawn on May 2, 2006 (42), the approaches in the proposed rule and the draft guidance can be used until both documents are finalized.

14

OTHER REGIONS

Recently, extensive regulations (43) relating to clinical trials have been put in place in the European Union. One quality-specific aspect is that Directive 2001/83/EC (44) requires that a ‘‘qualified person’’ provide certain oversight functions, such as certifying individual batch release. Another recent European Union guideline (45), EMEA/CHMP/SWP/ 28367/07, entitled ‘‘Guideline on Strategies to Identify and Mitigate Risks for First-inHuman Clinical Trials with Investigational Medicinal Products,’’ includes a section (4.2) on mitigating quality-associated risks in clinical trials. The Japanese Pharmaceutical and Medical Device Agency has information on clinical trials on their English language website (46). The United Kingdom’s MHRA has a web page devoted to clinical trials (47). The web site for the Official Newspaper of the Government of Canada also has a page devoted to clinical trials (48).

REFERENCES 1. Code of Federal Regulations (CFR). Available: http://www.gpoaccess.gov/cfr/index. html.

2. Good Guidance Practices 21 CFR 10.115. Available: http://www.accessdata.fda.gov/ scripts/cdrh/cfdocs/cfCFR/CFRSearch.cfm? fr=10.115. 3. Federal Register Notice on Administrative Practices and Procedures; Good Guidance Practices. Available: http://www.fda. gov/OHRMS/DOCKETS/98fr/091900a.pdf. 4. Laws Enforced by the FDA and Related Statutes. Available: http://www.fda.gov/ opacom/laws/. 5. Federal Food, Drug, and Cosmetic Act. Available: http://www.fda.gov/opacom/laws/ fdcact/fdctoc.htm. 6. Public Health Service Act. Available: http:// www.fda.gov/opacom/laws/phsvcact/phsvcact. htm. 7. FDA Electronic Submissions Gateway. Available: http://www.fda.gov/esg/. 8. CDER Reference Guide. Available: http:// www.fda.gov/cder/directories/reference guide.htm. 9. Pre-IND Consultation Contacts. Available: http://www.fda.gov/cder/ode4/preind/PreINDConsultationList.pdf. 10. FDA’s Pharmaceutical Quality for the 21st Century. Available: http://www.fda.gov/ oc/cgmp/. 11. Pharmaceutical Development: Annex to Q8. Available: http://www.ich.org/cache/compo/ 363-272-1.html#Q8. 12. Office of New Drug Quality Assessment (ONDQA): Presentations. Available: http:// www.fda.gov/cder/Offices/ONDQA/presentations.htm. 13. eCTD Specification and Related Files. Available: http://estri.ich.org/ectd. 14. Memo regarding Docket Notice on eCTDs as the only acceptable electronic submission format. Available: http://www.fda.gov/ ohrms/dockets/dockets/92s0251/92s-0251m000034-vol1.pdf. 15. FDA Electronic Submissions Gateway. Available: http://www.fda.gov/esg/. 16. Guidance for Industry Granularity Document Annex to M4: Organization of the CTD. Available: http://www.fda.gov/cder/guidance/ 7042fnl.htm. 17. INDs: Clinical holds and requests for modification. Available: http://www.accessdata.fda. gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch.cfm? fr=312.42. 18. FDA Electronic Submissions Gateway. Available: http://www.fda.gov/esg/.

CHEMISTRY, MANUFACTURING AND CONTROLS (CMC)

13

19. Guidance for Industry: Content and Format of Investigational New Drug Applications (INDs) for Phase 1 Studies of Drugs, Including Well-Characterized, Therapeutic, Biotechnology-derived Products, Nov 1995. Available: http://www.fda.gov/cder/guidance/ clin2.pdf.

30. Guidance for Industry: Content and Format of Investigational New Drug Applications (INDs) for Phase 1 Studies of Drugs, Including Well-Characterized, Therapeutic, Biotechnology-derived Products, Nov 1995. Available: http://www.fda.gov/cder/guidance/ clin2.pdf.

20. International Conference on Harmonisation. Available: http://www.ich.org.

31. Simplified Molecular Input Line Entry Specification. Available: http://en.wikipedia.org/ wiki/Simplified molecular input line entry specification.

21. Guidance for Industry: INDs for Phase 2 and Phase 3 Studies Chemistry, Manufacturing, and Controls Information. Available: http://www.fda.gov/cder/guidance/3619fnl. pdf. 22. The eCTD Backbone Files Specification for Module 1. Available: http://www.fda.gov/cder/ regulatory/ersr/Module1Spec.pdf. 23. Pharmaceuticals and Medical Devices Agency [Japan]. Available: http://www.pmda.go.jp. In the search box on the home page, enter as a search term ‘‘0527004.’’ This retrieves eCTD related PDF files, including the Module 1 Schema: http://www.pmda.go.jp/ich/m/ m4 ectd toriatsukai 04 5 27.pdf. 24. Telematic Implementation Group for Electronic Submission and ICH Implementation. Available: http://esubmission.emea. eu.int/tiges. 25. a) IND content and format 21 CFR 312.23(a)(7)(iv)(e). Available: http://www. accessdata.fda.gov/scripts/cdrh/cfdocs/cfCFR/ CFRSearch.cfm?fr=312.23. b) Guidance for Industry: Environmental Assessment of Human Drug and Biologics Applications, July 1998. Available: http://www.fda.gov/cder/ guidance/1730fnl.pdf. 26. IND content and format 21 CFR 312.23(a)(7)(iv)(d). Available: http://www. accessdata.fda.gov/scripts/cdrh/cfdocs/cfCFR/ CFRSearch.cfm?fr=312.23. 27. CGMPs, Packaging and Labeling Control, Expiration Dating. Available: http:// www.accessdata.fda.gov/scripts/cdrh/cfdocs/ cfCFR/CFRSearch.cfm?fr=211.137.

32. Connation Table File (MDL MOLFILE). Available: http://en.wikipedia.org/wiki/MDL Molfile. 33. International Chemical Identifier. Available: http://en.wikipedia.org/wiki/International Chemical Identifier. The IUPAC International Chemical Identifier. Available: http:// iupac.org/inchi/index.html. 34. International Conference on Harmonisation; Guidance on Q6A Specifications: Test Procedures and Acceptance Criteria for New Drug Substances and New Drug Products: Chemical Substances. Available: http://www.fda.gov/ OHRMS/DOCKETS/98fr/122900d.pdf. 35. Guidance for Industry Q6B Specifications: Test Procedures and Acceptance Criteria for Biotechnological/Biological Products. Available: http://www.fda.gov/cder/guidance/ Q6Bfnl.PDF. 36. Guidance for Industry Q8 Pharmaceutical Development. Available: http://www.fda.gov /cder/guidance/6746fnl.pdf. 37. 37. N. Ritter, S. J. Advant, J. Hennessey, H. Simmerman, J. McEntire, A. Mire-Sluis, and C. Joneckis, What is test method qualification? Bioprocess Intern. 2004; 2(8): 32—46. Available:http://www.bioprocessintl.com/ default.asp? page=articles&issue=9%2F1%2F2004 (site requires registration). 38. 21 CFR 312.47 Investigational New Drug Application: Meetings. Available: http://www. accessdata.fda.gov/scripts/cdrh/cfdocs/cfCFR/ CFRSearch.cfm?fr=312.47.

28. a) Guidance for Industry: M4Q: The CTD—Quality. Available: http://www.fda. gov/cder/guidance/4539Q.PDF. b) Guidance for Industry: M4: The CTD—Quality Questions and Answers/ Location Issues. Available: http://www.fda.gov/cder/guidance/5951fnl. pdf.

39. a) Guidance for Industry: IND Meetings for Human Drugs and Biologics Chemistry, Manufacturing, and Controls Information, May 2001. Available: http://www.fda.gov/ cder/guidance/3683fnl.pdf. b) Guidance for Industry: Formal Meetings With Sponsors and Applicants for PDUFA Products. Available: http://www.fda.gov/cder/guidance/2125fnl. pdf.

29. In Module 3, the information that should be included to support a Phase 1 IND submission is noted by a superscript P1.

40. Current Good Manufacturing Practice Regulation and Investigational New Drugs. Federal Register. 2006; 71(10): 2458–2462.

14

CHEMISTRY, MANUFACTURING AND CONTROLS (CMC)

41. Guidance for Industry: INDs—Approaches to Complying with CGMP During Phase 1. Available: http://www.fda.gov/cder/guidance/ 6164dft.pdf. 42. 42. Current Good Manufacturing Practice Regulation and Investigational New Drugs; Withdrawal. Federal Register. 2006; 71(84): 25747. 43. EUDRALEX Volume 10—Clinical trials: Medicinal Products for human use in clinical trials (investigational medicinal products). Available: http://ec.europa.eu/enterprise/pharmaceuticals/eudralex/homev10.htm. 44. Directive 2001/83/EC of the European Parliament and of the Council of 6 November 2001 on the Community Code Relating to Medicinal Products for Human Use. Available: http://ec.europa.eu/enterprise/pharmaceuticals/eudralex/vol-1/consol 2004/human code. pdf. 45. Guideline on Strategies to Identify and Mitigate Risks for First-in-Human Clinical Trials with Investigational Medicinal Products. Available: http://www.emea.europa. eu/pdfs/human/swp/2836707enfin.pdf. 46. Pharmaceutical and Medical Device Agency: Clinical Trial Related Operations. Available: http://www.pmda.go.jp/english/clinical.html. 47. MHRA: Clinical Trials for Medicinal Products. Available: http://www.mhra.gov.uk/home/ idcplg?IdcService=SS GET PAGE&nodeId =101. 48. Canada Gazette: Regulations Amending the Food and Drug Regulations (1024–Clinical Trials). Available: http://canadagazette.gc.ca/ partII/2001/20010620/html/sor203-e.html.

CITIZEN PETITION Anyone may request or petition the U.S. Food and Drug Administration (FDA) to change or create an Agency policy or regulation under 21 Code of Federal Regulations (CFR) Part 10.30. Requests should be directed to FDA’s Dockets Management Branch. When submitting a petition, keep these points in mind: • Clearly state what problem you think

the Agency needs to address. • Propose specifically what the Agency’s

action should be. Your proposal should be based on sound, supportable facts. • Submit the petition, an original, and three (3) copies, unless otherwise stipulated in the Federal Register announcement, to: Food and Drug Administration Dockets Management Branch Room 1-23 12420 Parklawn Drive Rockville, MD 20857 827-6860 The FDA carefully considers every petition and must respond within 180 days by either approving or denying it (in whole or in part), or by providing a tentative response indicating why the FDA has been unable to reach a decision. All petitions will be subject to public examination and copying as governed by the rules in 21 CFR 10.20(j). If the FDA approves the petition, it may be published in the Federal Register. Any petition could eventually be incorporated into Agency policy.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/ora/fed state/Small business/ sb guide/petit.html) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

CLINICAL DATA COORDINATION

converting it into a high-quality database ‘‘. . .enabling the data to be efficiently analysed and displayed’’ (1, p. 69). The processes outlined are assumed for a regulatory study (i.e., a study intended for submission to a regulatory agency for approval of a drug or device) using a paper-based case report form. Data management activities are divided into the following phases:

CHERYL KIOUS 1

INTRODUCTION

The goal of clinical research trials is to generate data that proves the safety and effectiveness of a drug or device, leading to marketing approval by a regulatory authority. The acquisition, validation, and integration of this clinical data, known as clinical data management (CDM), are an integral component of the drug development process. CDM is the process of preparing the clinical data for statistical analysis; the conclusions from this analysis form the basis of a regulatory submission for approval and subsequent marketing. The discipline of CDM includes case report form design, clinical study database design and programming, data entry, data validation, coding, quality control, database finalization, and data archiving. CDM begins in the study design phase and ends when the clinical database is finalized and the data are considered ready for statistical analysis. CDM processes are greatly enhanced through the development of a comprehensive data management plan, adherence to good CDM practices, and by ensuring that quality steps are built into all stages of data handling. CDM is a comprehensive series of activities performed by a diverse team of people, ranging from data entry operators, database programmers, and clinical data coordinators to coding specialists. The core of the CDM team is the lead data manager (LDM), who serves as the lead clinical data coordinator and primary CDM contact for a clinical research study. The LDM is responsible for developing the data management plan, providing direction and leadership to the data management team, coordinating the data management activities, and ensuring that a high-quality database is provided for subsequent statistical data analysis. This article provides an overview of the role of the LDM, including best practices and considerations in the CDM process. The CDM process is responsible for taking the raw data provided by investigative sites and

• study initiation • study conduct • study closure

Best practices for the LDM to follow will be outlined for each data management phase. 2

STUDY INITIATION

Careful planning is one of the most important activities in the study initiation phase. ‘‘CDM need to ensure that they have the specific procedures and systems in place to handle the data when they arrive’’ (1, p. 73). It is important to set up good lines of communication within CDM and with other functional groups to allow for optimal data handling. Even with good planning, unexpected things happen during the course of the study. ‘‘If we plan for what is expected it will give us more time to devote to the unexpected and unplanned events’’ (1, p. 86). Good planning processes can facilitate the collection, processing, and reporting of high quality data in a timely manner (1, p. 87). The following activities occur in the study initiation phase: • case report form design • development of the data management

plan • design study database and program edit

checks 2.1 Case Report Form Design A well-designed case report form (CRF) can facilitate accurate data collection, analysis, and reporting. ‘‘Traditionally CRFs have been very extensive, capturing many thousands of

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

CLINICAL DATA COORDINATION

data points of which only 50-60% may actually be used. Quantity versus quality needs to be reconciled and only essential data collected and computerized’’ (1, p. 82). Data are collected for clinical studies for two primary purposes: (1) ‘‘To provide the answer to the question posed by the protocol’’ and (2) ‘‘to provide the required safety data regarding the drug being studied’’ (1, p. 134). A CRF should be designed to accomplish three basic purposes: the collection of raw data, ease in monitoring or auditing, and to facilitate processing of the data. It is challenging to optimally meet all of these purposes; compromises may need to be made. In that case, the investigative site’s perspective should take precedence over other groups’ needs (2). For detailed information and example CRFs, see Reference 2. The principles outlined below should be incorporated during the CRF development phase. 1. Document the procedures used for the creation, approval, and version control of the CRF. 2. Design the CRF to collect only the key safety and efficacy data specified in the protocol. The CRF can be drafted early in the protocol development phase because most of the data to be collected are known before the protocol is finalized (2). The CRF design process can help ‘‘. . .identify attributes of a protocol that are inappropriate or infeasible, requiring additional protocol development’’ (2, p. 4). The statistical analysis plan (SAP) can provide guidance on what data should be collected. If the SAP is not available during the CRF design phase, the statistical methods section of the protocol is a good reference to identify what data to include when developing the CRF. 3. Use standard forms whenever possible to facilitate integration of data across studies, such as the standards defined by the Clinical Data Interchange Standards Consortium (CDISC) Operational Data Model (ODM). Database setup can be achieved more efficiently and reliably with the use of standard CRF modules (1).

4. Obtain cross-functional input on the draft CRF. Representatives from clinical, biostatistics, and CDM programming departments should review the CRF to ensure it is adequate from their respective group’s view. Depending on the type of study and data being collected, other reviewers may need to be included, such as investigators and subject matter consultant experts. 5. If using a validated data collection instrument, such as rating scales or questionnaires, it should not be modified without consulting the author to ensure the validity of the tool is not compromised by the change (3). 6. Avoid collecting duplicate data whenever possible. If redundancy is needed to validate data, ensure the duplicate data are obtained through independent measurements (3). In some cases, some data may need to be collected to confirm other data, but ‘‘. . .it is essential that no one piece of data is asked for twice, either directly or indirectly’’ (1). 7. Include the investigator’s signature in the CRF to document the investigator’s review and approval of the recorded data. 8. Finalize the CRF in conjunction with the finalization of the protocol, ensuring each reviewer’s feedback is incorporated during each review cycle. Document approval of key reviewers on a CRF approval form, noting the version information on the approval form. 9. Ensure CRF completion instructions and training are provided to site personnel. All efforts should be made to ensure the CRF is available to the site before the first subject is enrolled in the study.

2.2 Development of the Data Management Plan A Data Management Plan (DMP) is a comprehensive document developed by the LDM to describe the data handling and data cleaning processes to be used during the study. The content for each section of the DMP is

CLINICAL DATA COORDINATION

suggested below or described in the corresponding section that follows. Each component of the DMP should have input, review, and approval from appropriate data management and cross-functional team members. The DMP should be created at the beginning of the study and updated to reflect any process decisions or changes as needed throughout the study. Each document in the DMP should, at a minimum, contain the following: • document title • version number or date • protocol identification • company identification (and sponsor

name, if study managed by a contract research organization) • author, reviewer, and authorizer names • distribution list • change history section

The following major components are recommended for the DMP: • data

management personnel: this section includes a list of key CDM team members, such as LDM, Database Programmer, Coding Lead, Data Entry Lead

• data management status reports: this

section includes samples of the status reports to be provided, the frequency, and distribution list • database design (see section below) • data tracking and entry (see section

below) • data validation (see section below) • coding (see section below) • external data processing (see section

below) • quality control (see section below) • serious adverse event (SAE) reconcilia-

tion (see section below) • data transfer: if applicable, this section

includes specifications for the transfer of data, such as from a CRO to the sponsor • data handling report (see section below)

3

• standard operating procedures (SOPs):

this section includes a list of applicable SOPs, including the dates they became effective for the study The relevant section of the DMP should be finalized before the corresponding activities begin. For example, data entry should not begin until the Data Tracking and Entry sections of the DMP are finalized. The foundation of the DMP is based on SOPs. SOPs identify what procedures are needed and why, when they occur, how procedures are implemented, and the roles or positions responsible for each procedure (1). The SOPs provide the core framework on which study-specific processes are documented in detail in the DMP. 2.3 Design Study Database and Program Edit Checks The focus of this section is the design and validation of a project-specific clinical database using a validated clinical data management system (CDMS). It does not address the requirements for software applications development and validation or installation and functional-level testing of a packaged CDM application. For further information, see References 4 and 5, which document on computerized systems in clinical trials and provide information on how ‘‘. . .expectations and regulatory requirements regarding data quality might be satisfied where computerized systems are being used to create, modify, maintain, archive, retrieve, or transmit clinical data’’ (5). Ensure that adequate procedures as specified in an SOP are in place to control user accounts and access within the CDMS. Granting and revoking of study access for each user should be documented in the data management study file. Access should be revoked for users who no longer need access to the study (e.g., not working on study, resignation from company). In the design of a project-specific clinical database, the database programmer should follow good programming practices, such as use of a header block, completion of informal development testing, and use of sound programming code. Whenever possible, use

4

CLINICAL DATA COORDINATION

standard data structures, such as CDISC ODM (6). Input from clinical and biostatistics are important during the database design phase ‘‘. . .as the more thought at the start about what is required at the end will result in a study that can be processed efficiently and effectively. It must be remembered that the computer is a tool and that the output from it is only as good as the ‘human’ instructions it receives’’ (1). A database should not be released into production until the study protocol and CRF are finalized. Database design activities consist of the following components: 1. Computer system validation (CSV) plan: The database programmer should create a clinical study database CSV plan to document the testing strategy for the database structure, data entry screens, edit checks, external data imports, and data transfers. The CSV plan outlines the scope of testing, methodology to be used, format for the test data, and documentation and resolution of errors in the testing process. 2. Database annotated CRF: Upon finalization of the CRF, the database programmer annotates the CRF with data file names, item names, code lists, derived items, and other relevant information for each data variable, including version information. 3. Database structure and data entry screens: Using the database annotated CRF, the programmer creates the database metadata and data entry screens. 4. Coding: The variables to be coded are set up as part of the database structure and entry screens and the programs used for auto-encoding are mapped to the appropriate dictionaries. The version of dictionaries used for coding should be referenced in the metadata. 5. Edit checks: Edit checks are programmed according to the edit check specifications document. Edit checks may consist of programmed edit checks that periodically run on the

data or programmed listings that report discrepant data based on specified criteria. 6. External data: Programs used to electronically load non-CRF data into the CDMS or system table (such as Oracle) should be tested to ensure that the import process does not affect the integrity of the data. 7. Data transfer: Programs used to export the data from the CDMS should be tested to assure the transfer process does not affect the integrity of the data being transferred. 8. Formal testing of the database may be done by the programmer or the LDM. Hypothetical or dummy subject data is entered to validate the database structure, data entry screens, coding setup, and edit specifications. All errors identified through the testing process, the action taken to correct the errors, and each correction retested to validate modifications should be documented. Once the database has undergone all testing and is considered ready for use, document the release of the database and edit checks to a production environment. Minimum testing requirements consist of: A. Database structure: verify each field on the CRF is present; each data variable’s type, name, length, ranges, and code lists are accurate; derivations calculate and derive to the targeted item correctly; confirm setup of auto encoding and the appropriate dictionary term populates correctly; and the flow of data entry screens are compatible with the CRF. B. Edit checks: both valid (clean) and invalid (unclean) should be entered to verify each check achieves the expected result and performs according to the edit specifications document. C. External data: completing a test import for each data source verifying the number of records matched the raw data file and all data fields mapped correctly to

CLINICAL DATA COORDINATION

the appropriate database variable in the CDMS or designated table (such as Oracle). If any modifications to the file specifications or programs used to load the dataoccur, the changes must be retested and documented. D. Data transfers: completing a test transfer verifying the number of records exported and that all data, data labels, code lists, formats, and calculated variables are exported as intended.

3.

4. 5.

6. 3

STUDY CONDUCT

The following activities occur in the study conduct phase: • data management documents tracking

and data entry • data validation • coding • serious adverse event (SAE) reconcilia-

7.

tion • external data processing • quality control

3.1 Data Management Documents Tracking and Data Entry The CDM documents that are expected to be received and the process and flow for tracking, entry, and filing should be defined at the start of the study and documented in the DMP. CDM documents may consist of CRFs, Data Clarification Forms (DCFs), sitebased DCFs (SDCFs), and source documents (for example, laboratory reports, ECGs, and subject diaries). These documents should be stored in a secure and controlled environment. It is recommended that the following information be documented: 1. Documents that will be received as a hard copy, fax, or other electronic method, such as PDF. 2. Documents that will be received as originals or copies. Define any hard copies of reports that will be received

8.

9.

5

that correspond with any electronically transferred data, such as laboratory reports. Identify whether any reconciliation of electronically transferred data with hard copy reports will occur. The process and systems to be used for any documents that will be tracked (manually or electronically). The filing organization of documents, within a paper file or electronically. The process for processing CRFs received in a language different than expected. The mechanism that will be used to ensure all expected CRFs and other data entry documents are received and entered in the CDMS. A program written within the CDMS to track pages once they are entered is an efficient and accurate method. Such a program can also provide reports for monitors of expected pages that are missing. The mechanism used to verify unscheduled pages have been received and entered, such as unscheduled visits, repeating pages of log-type forms such as concomitant medications and subject diaries. Any manual review that will be completed before data entry, such as verification for complete and correct subject identifiers, measurement of scales. Describe any data conventions that can be applied before entry. The method of data entry, such as whether it is interactive double data entry, independent double data entry with third party reconciliation, or single entry with 100% manual verification. ‘‘The great debate on whether single or double entry should be used to enter clinical data is still unresolved and it should be questioned if there is a right or wrong answer’’ (1). FDA and ICH regulations do not mandate a specific data entry method. ‘‘All clinical trial information should be recorded, handled, and stored in a way that allows its accurate reporting, interpretation, and verification’’

6

CLINICAL DATA COORDINATION

(7, p. 25695). The method can be evaluated based on factors such as the skill level of the entry operators, amount of time to complete the entry process, and the percent of acceptable errors in the data. 10. General and page-specific data entry guidelines to document handling of data, including date formats, subject initials, blank pages, missing pages, missing data fields, text fields, values recorded as not done or unknown, extraneous comments or marginalia, abbreviations, and illegible fields.

3.2 Data Validation Data validation or data cleaning are the processes used to validate the accuracy of data (3). Data validation can identify errors made while entering data from CRFs into the database and transcription errors made by the site when transcribing from source documents to the CRFs. These errors are identified through checks of the data that report inconsistencies within a subject’s data, medically invalid values, data values out of predefined ranges or formats, incorrect protocol procedures, spelling and legibility problems, and missing data. This check can be achieved through manual review of data listings and programmed edit checks that identify discrepant data. ‘‘The ultimate purpose of validation is to ensure that data meet a level of quality such that the inferences and conclusions drawn from any analyses may be correct’’ (1). Comprehensive data validation guidelines should be developed to check CRF and any other external data related to the study. ‘‘When data management locks a database that is still dirty, the data problems are passed off to the statisticians and programmers. The data problems do not go away. In fact, they become more expensive to fix’’ (8) Comprehensive data validation guidelines (DVGs) should be created consistent with the protocol requirements, CRF, and the statististical analysis plan (or key safety and efficacy data). Cross-functional input from clinical and biostatistics on the DVGs is important and will assure a sound data cleaning plan.

All data changes should be documented at the data management center and the investigative site. The DVGs will generally include the following components: 1. DCF process and flow: A. Frequency for running consistency checks and whether they will be run automatically or manually B. Whether DCF updates will be made to the CRF C. Timelines for DCF generation and receipt from investigative sites D. Method of DCF distribution to investigative sites E. Definitions of discrepancy and DCF statuses and audit trail reasons for change and their use F. Process for handling unsolicited comments on CRFs, for example, review by clinical or medical team members or issue a DCF to the investigative site G. Process for handling any duplicate DCFs 2. Edit specifications: A. Edit specifications can encompass pre-entry review, programmed consistency checks, and manual listings review. The edit specifications should be sufficiently detailed that the database programmer can use this document to program consistency checks and data listings. Examples include: a. Checks for sequential dates b. Checks for missing or required data c. Checks to assure data values are consistent with the data expected, such as character, numeric, within a predetermined range d. Checks across different types of data (cross-panel or crossmodule) e. Checks for protocol violations f. Checks to identify outliers, such as medically implausible

CLINICAL DATA COORDINATION

data, outside expected ranges for the study population B. Standardized DCF message text should be developed for both programmed consistency checks and manual queries issued from manual review of data listings. This process can increase consistency and efficiency during processing, especially when multiple CDCs are working on the study. 3. Data conventions A. Data cleaning conventions may be created for changes that can be made without the investigator’s approval on each individual change. These should be determined at the beginning of the study, before data cleaning activities begin, and only used for obvious errors. Data cleaning conventions should be made by trained data management personnel. B. Data conventions should not be applied to key safety or efficacy variables. C. The investigator should receive and acknowledge at least the first and final versions of data cleaning conventions. D. If certain protocol tests are being performed locally by the site and use normal reference ranges, such as a local laboratory, determine whether local reference ranges will be entered in the database and referenced in reporting or whether laboratory reference ranges will be normalized across investigative sites or studies.

3.3 Coding Clinical data, such as diseases, diagnoses, and drugs that are collected on CRFs, are coded using standard dictionaries to facilitate the analysis and reporting of the data. Coding of data provides enhanced data retrieval, flexible reporting capabilities, the ability to group consistent terms, and the capability

7

for different levels of hierarchical or crossreference reporting (1). ‘‘The aim of processing clinical trials data and safety information is to order and structure the data to allow effective reporting and retrieval of information for regulatory, safety, marketing and planning usage’’ (1). Data are typically coded using an auto-encoding system. Auto-encoders vary in complexity from exact verbatim text to dictionary term-matching to use of a thesaurus to manage synonyms that map to a dictionary term. When selecting an auto-encoder, capabilities to consider include the ability to manage synonyms, misspellings, mixed case, word variations, word order, and irrelevant verbatim information (3). Auto-encoders must be validated and comply with FDA regulations on electronic records (4). The International Conference on Harmonization (ICH) Medical Dictionary for Regulatory Activities (MedDRA) Terminology is recommended for coding clinical trial adverse event data. For guidance regarding the use of MedDRA and version updates, see the Maintenance and Support Services Organization (MSSO) website for publications entitled MedDRA Implementation and Versioning for Clinical Trials and MedDRA Term Selection: Points to Consider, Release 3.4 (http://www.meddramsso.com) (9, 10). Medications are coded to facilitate the analysis and reporting of possible drug interactions (3). The WHO Drug Dictionary is commonly used for coding medications (see Uppsala Monitoring Centre) (11). If adverse events and serious adverse events are maintained in two separate databases, ensure that the same dictionaries and versions are used, if possible, and that consistent coding guidelines are used to code serious adverse events. All versions of dictionaries used should remain available for reference. If changes or additions are needed to publish dictionaries, the organizations that maintain the dictionaries have procedures for submitting change requests (3). ‘‘All levels, codes or group assignments for the data should be stored’’ (3). This storing ensures all information is available if recoding is needed or if hierarchy or multi-axial information is needed to fully understand a particular coded term.

8

CLINICAL DATA COORDINATION

The following points should be considered for coding data and detailed coding guidelines should be developed for each study: 1. Identify the variables to be coded, the dictionaries, and the version used. 2. Define the process for terms that do not auto-encode, including misspellings, abbreviations, and irrelevant verbatim information. If coding guidelines allow modifications to be made to facilitate auto-encoding, the modifications should only be made to the corresponding modifiable term and the original verbatim term should remain unchanged in the database. Changes to the original verbatim term should only be made based on a DCF confirmed by the investigator. 3. Develop guidelines for how terms containing multiple concepts will be handled, whether they can be split per coding guidelines or issuing a DCF to investigator. In general, all adverse events reported as multiple events should be issued on a DCF to the investigator for clarification. 4. Any terms that cannot be coded should be documented in a data handling report. All serious adverse events should be coded. 5. Determine how dictionary updates that occur during the study will be managed. If updates will be applied, identify the frequency of planned updates and how updates will be applied (e.g., updates only or reload dictionary). 6. Define the process and frequency for review of coded terms for missing codes, consistency, and medical or clinical judgment. Define reports that will be used for coding review. For further information on dictionary management, the following are good resources. 1. Good Clinical Data Management Practices, Version 3 (3) 2. MedDRA maintenance and the Maintenance Support Services

Organization (MSSO), http://www. meddramsso.com (9, 10) 3. Uppsala Monitoring Centre, http:// www.who-umc.org (11)

3.4 Serious Adverse Event Reconciliation Serious adverse event (SAE) reconciliation is performed for studies in which SAE data and study adverse event data are maintained in two separate databases. SAE data are typically reported to a drug safety group responsible for maintaining the regulatory safety reporting database. Adverse events (AEs) are reported along with CRF data and entered into the clinical database. As some events are entered into both databases, a comparison of the databases for key safety data variables is performed to ensure consistency. When designing the AE and SAE forms, standardize the capture of SAE data variables for consistency in reporting and to facilitate reconciliation. Develop SAE reconciliation guidelines to include: 1. The systems and locations of the safety database and the clinical database. 2. The method for providing safety data between the two groups (i.e., paper or electronic). If electronic, the medium to be used and format of the data, such as ASCII or SAS. 3. The mechanism for comparing data (i.e., manually or programmatically). 4. The time intervals for when comparisons will be performed, including details for comparisons prior to interim analyses or safety data reporting. 5. The coding dictionaries and versions used to code diagnoses among the two databases. If using different dictionaries or a different version of the same coding dictionary, identify how differences in coded terms for the same event will be resolved. 6. Determine cut-off dates, such as database lock, after which no new SAEs will be added to the clinical database.

CLINICAL DATA COORDINATION

7. Identify the variables that will be reconciled and which items must be an exact match and which may be similar. The following items are examples of items that might be reconciled if they are present in both the safety and clinical databases: A. protocol B. investigator C. subject identification (e.g., subject number, randomization number, initials, date of birth, gender, race) D. diagnosis (i.e., verbatim or coded term) E. severity F. onset date G. date of death H. resolution date I. causality assessment J. action taken with drug K. outcome 8. Reconciliation can occur on events where all data to be reconciled have been entered and validated, no outstanding DCFs exist, and the event has been coded. It is recommended that discrepancies be documented consistently, such as on an SAE reconciliation form. 9. Determine the process for documenting and resolving discrepancies. Outline the communication process with the drug safety group and expected turn-around time for resolving discrepancies. Data clarification forms should be issued to the investigator to resolve discrepancies. The drug safety group should be notified as soon as possible of any events that are in the clinical database but not in the safety database. It is possible for events to exist in the safety database but not in the clinical database, reflecting CRFs that have not been retrieved and entered. 10. It is recommended to have written documentation from the drug safety lead and the LDM once all data have been received and reconciled. Any discrepancies that cannot be resolved should be documented in the DMP.

9

3.5 External Data Processing Use of a centralized vendor for processing and reporting of select data is very common in multicenter trials. Centralized vendors are used to standardize the methods of testing and reporting, using consistent reference values for all data collected across study sites. Many sponsors use one vendor for protocol procedures or tests such as automated randomization, clinical laboratory, ECG interpretation, pharmacokinetic data, and subject diaries. The results are created, entered, or processed and quality control checked by the vendor and typically sent as electronic file (or files) to the sponsor or CRO. The sponsor or CRO should ensure the vendor’s personnel, systems, and procedures are adequate and meet regulatory requirements. For further information, see Good Clinical Data Management Practices, Version 3, section Vendor Management (3). Once the vendor has been identified and qualified, it is important to work with a contact person at the vendor to develop transfer specifications, which are then documented. The transfer specifications and detailed guidelines for each type of external data should be developed as part of the DMP and include information, such as: 1. Contact names and phone numbers for both the sponsor or CRO and vendor. 2. Format of data to be received, such as ASCII or SAS, and version identification. 3. Method of data transfer, for example, disk, CD, data tape, Internet. If data are to be transferred through the Internet, ensure appropriate encryption mechanisms are used. 4. Schedule for data transfers. 5. Whether transfers will be incremental or cumulative. 6. Testing strategy, including completing a test transfer to validate the integrity of the data following the import process (see Design Study Database section). 7. Process for loading data, the programs used to manipulate data, if applicable, and loading logs.

10

CLINICAL DATA COORDINATION

8. Location of loaded data, such as CDMS, Oracle table, or SAS datasets. 9. Detailed file structure, including variable names, labels, attributes (size, format), code lists, mapping of external data variables to clinical database structure and position within file, required variables, how missing variables will be identified. 10. Details of normal ranges and process for conversion to different units, if applicable. The vendor should provide a list of reference ranges and effective dates. Ensure reference ranges are up-to-date throughout the study and prior to database lock. Identify how any reference range changes will be handled for reporting purposes. 11. Process for handling data corrections and additions. If an audit trail is not available for corrections at the data management site, the vendor should make all data changes and provide a corrected data file. 12. Process for handling repeat tests, duplicates, and partial records. 13. Process for handling special characters, such as symbols and quotation marks. 14. Mechanism to identify retransmission of corrected data. 15. Communication process with vendor to acknowledge receipt of data transfers, such as e-mail or hard copy form. 16. If external data contains information that would unblind the CDM team, identify the process for handling the data to ensure blind is maintained. Once transfer specifications have been agreed on with the vendor, the external data import program has been set up and tested, external data transfers may occur per the specifications. The following points should be considered in processing external data: 1. Acknowledge receipt of each data transfer. 2. Communicate any problems encountered with the data file and whether the data file was acceptable.

3. Obtain a list of checks that are performed on the data by the vendor before transfer to ensure CDM does not duplicate checks. 4. Identify in the edit specifications the checks that will be performed on the data following each transfer. Some checks to consider include: A. Verify a minimum number of demographic variables to ensure the data correspond to the intended subject, such as subject number, subject initials, and date of birth. B. Verify for each record in the data transfer file that a corresponding CRF record exists in the clinical database and the intended data variables match, such as visit identifier and date of sample or procedure. C. Verify for each CRF record in the clinical database that a corresponding record exists in the data transfer file and the intended data variables match as above. D. If the CRF and clinical database do not capture unscheduled tests or procedures, develop a process to ensure these records are present in the data transfer file and contain the appropriate data. It is recommended that the CRF capture, at a minimum, the date of each unscheduled test or procedure to ensure all expected data is received and reconciled in the transfer file. 5. To avoid duplication of information, transfer only those variables that are needed to accurately match the subject and associated visit with the CRF data in the clinical database. 6. Retain an electronic version of the data transfer file (i.e., each transfer file if incremental and the final transfer if cumulative). 3.6 Quality Control Quality control in the context of CDM are the procedures undertaken to ensure the

CLINICAL DATA COORDINATION

‘‘. . .data contains no errors, inconsistencies or omissions (that can be rectified) and that the data are coded correctly and transferred accurately from the case record form (CRF) to the computer database’’ (1). The ICH GCP guideline states, ‘‘Quality control should be applied to each stage of data handling to ensure that all data are reliable and have been processed correctly’’ (7, p. 25699). Quality is not a single measurement; quality is measured with individual steps of a total process (1). Quality can be influenced by the experience and judgment used by the CDM personnel reviewing discrepancies, ‘‘. . .while queries are usually issued in line with set procedures which broadly define what should and should not be queried, there will always be some discrepancies where an element of judgment is needed in order to arrive at the decision of whether a query is warranted’’ (1). The level of expertise and appropriate judgment used in determining whether to send a DCF to the investigative site can affect the overall quality of the study data. CDM staff responsible for data validation should receive adequate training and quality control of their work product until a reasonable level of accuracy is attained before working independently. Quality is typically measured by setting a certain standard for an allowable number of errors within a defined number of data values. Errors can be weighted differently according to the value of the data (e.g., key efficacy data may need to be error-free). Good data validation is time-consuming to define, develop, and implement. ‘‘It is important to assess the value of the effort put into validations against the resulting improvement in the data’’ (1). A quality control (QC) plan should be developed at the beginning of the study to document the QC procedures that will be performed during the study. QC reviews should consist of a comparison of the selected data variables and corresponding DCFs, site DCFs, and data conventions to ensure no entry, programming, or database update errors occurred in the data management process. The QC plan may provide for initial, ongoing, and final inspections. Initial inspections can provide immediate feedback in areas where investigative site or monitoring training is needed. Ongoing inspections

11

can be beneficial for studies of long duration or large size to ensure that no systematic processes resulting in errors have occurred. The final inspection is performed to provide an error rate between the CRF and the final database used for analysis. All errors identified in inspections should be documented and corrected in the clinical database. ‘‘Error detection methods that are applied to only a sample of the data can be used to obtain estimates of the distribution of undetected errors’’ (3). The Good Clinical Data Management Practices document suggests that the best practice for determining appropriate QC sample size is by statistical sampling (3, p. 82). CDM error rate standards vary across the industry. Errors should be expressed in the form of an error rate rather than a raw count of the number of errors. Error rates should be calculated by taking the number of errors identified divided by the total number of fields inspected. ‘‘The error rate gives a common scale of measurement for data quality’’ (3). Study-specific details should be documented in a QC plan, covering details that are not specified in the relevant QC SOP. Study-specific details that might be included in the QC plan are:

1. Identify any key safety and efficacy variables that are deemed critical for the analysis to undergo 100% QC and be error-free. These variables can be determined from the SAP (or the statistical methods section of the protocol if the SAP is not final) and in discussion with biostatistics. If the study has a very large volume of key safety and efficacy data, the LDM in conjunction with biostatistics determines an acceptable error rate for key variables while maintaining a statistical confidence interval. 2. If inspections will occur other than on the final, clean data, identify the timing and sampling methodology to be used.

12

4

CLINICAL DATA COORDINATION

STUDY CLOSURE

The following activities occur in the study closure phase: • final quality control procedures • database lock • data archiving

4.1 Final Quality Control Procedures Before database lock and when a specified percentage of the study data are considered clean, such as 90%, a final inspection is performed to estimate the error rate between the CRFs and the clinical database. The final inspection is to ensure data management processes resulted in high-quality data suitable for statistical analysis. 4.2 Database Lock Database lock occurs when all study data are considered clean and ready for statistical analysis. For some studies, databases may be locked for interim analyses to support study continuance for safety or efficacy reasons or for regulatory submissions. The locking of the database in the CDMS is completed to prevent inadvertent or unauthorized changes once analysis and reporting have begun (3). Written procedures should be in place for the locking of the database and removal of write access in the CDMS. Use of a database lock checklist is recommended to ensure all appropriate procedures have been completed. Potential items to include on a database lock checklist are mentioned later in the section. Once the database lock process is underway (such as initiation of signatures on a database lock authorization form), no additional changes should be made to the database. Instances occur when a database may need to be unlocked. A discrepancy may be identified during the statistical analysis and reporting phase that may warrant unlocking the database. A discrepancy should be discussed with project team members, including clinical, biostatistics, and data management, to determine any affect the discrepancy might have on the analysis. If the decision is made to unlock the database to make corrections

or query the investigator for additional information, the unlock process should be documented, including authorization signatures, date, time, and the names of individuals given access to the unlocked database. Once discrepancies are resolved or corrections are made to the database, formal database lock procedures should once again be followed, including completion of a database lock checklist. If the decision is made not to unlock the database, the rationale should be documented in the data management study file, preferably a data handling report or comparable document, and as errata in the clinical study report. In some cases, the discrepancy identified post-lock does not warrant unlocking because the data either do not effect or only minimally effect the statistical results. Before database lock, it is good practice to use a database lock checklist to ensure all items have been completed. The following should be considered in evaluating the database for readiness to lock: 1. All study data have been received or accounted for by data management 2. All data have been entered in the CDMS and all entry discrepancies have been resolved 3. All noncoded text fields and unsolicited comments have been reviewed and appropriate actions taken 4. All electronic external data have been received and reconciled with the clinical database and all discrepancies resolved 5. All data validation has been completed per the Data Management Plan and all queries have been received and updated 6. Coding has been completed and reviewed for consistency and accuracy 7. SAE reconciliation has been completed and all discrepancies have been resolved (applicable if AEs and SAEs are maintained in separate databases) 8. Reference ranges, such as laboratory normal ranges, are available in the database and have been verified to map appropriately to corresponding data values

CLINICAL DATA COORDINATION

9. Inspections specified in the QC plan have been completed and an acceptable error rate was achieved 10. All automatic batch jobs have been canceled 11. All deviations and unresolved data issues have been documented, such as in the Data Handling Report The concept of a Data Handling Report is to document deviations in processes defined in the DMP; deviations to SOPs; unresolved data issues, such as DCFs, missing CRF pages or other study data; terms that could not be coded; and any other information that will be useful for the statistical analysis and clinical study report. Write access to the clinical database should be revoked from all users once the database is locked. 4.3 Data Archiving Environmentally protected, secure, and accessible storage for study data documents is usually limited in most CDM organizations. Routine data archiving is good practice to allow space for ongoing studies and ensure the data are easily accessible by CDM personnel throughout the life of the study. Data archiving also ensures the documents are available for audits by independent auditing groups and regulatory authorities. All CDM organizations should have an SOP for archiving paper and electronic records based on at least the minimum requirements specified as essential documents ‘‘. . .those documents that individually and collectively permit evaluation of the conduct of a trial and the quality of the data produced. . .’’ in ICH Guideline, E6 Good Clinical Practice: Consolidated Guidance (7, p. 25705). In many organizations, paper archiving follows a master filing plan or template for all clinical study documents within which CDM documents are filed. A standard master filing plan ensures documents can be easily retrieved if needed for future reference. Study data documents should be retained according to applicable regulations of the country (or countries) where the drug is approved. According to ICH Guideline, ‘‘. . .essential documents should be retained

13

until at least 2 years after the last approval of a marketing application in an ICH region . . .’’ (7, p. 25700). Archiving consists of both paper and electronic records. Electronic data mediums are changing rapidly. ‘‘Media obsolescence can make orphans of data in the clinical world’’ (12). It is important for clinical trial data to have continued readability once archived. As technology advances, the hardware and software to read data files can quickly become obsolete. One strategy to keep data from becoming obsolete and unreadable is to migrate it to newer versions. This process can be time-consuming and costly to ensure the data is properly migrated and validated. The process of migrating data can have risks, such as losing data in the transfer or unintended transformation (12). Mostly recently, an industry-accepted standard for data archiving is the CDISC Operational Data Model (ODM). For further information, please see http://www.cdisc.org. The following should be considered when archiving CDM documents and electronic records: 1. Ensure electronic data are archived in an open format, such as Extensible Markup Language (XML) used in CDISC ODM, because it can be easily accessed, regardless of system software version used. 2. These items should be archived: A. Database metadata and program code B. Raw data, including CRF data and any externally loaded electronic data C. External data electronic files in original format, retaining each file if incremental data files received, or last file only if cumulative data files were received D. Copy of dictionary used to code data E. Each version of laboratory reference ranges used F. Audit trail G. Final data, including any derived data

14

CLINICAL DATA COORDINATION

H. Original documents, including CRFs, DCFs, DMP, projectspecific computer system validation documentation, and any other relevant study documentation, such as correspondence I. Discrepancy management data for data that failed edit checks J. SOPs used and any documented deviations K. Database closure documentation, such as database lock authorization, confirmation of database lock, and removal of access L. Electronic back-up copies 3. The recommended timing to archive data is subjective, but a standardized approach might be to archive once the clinical study report for a trial is finalized. 4.4 CDM’s Role in Training and Education CDM can play a significant role in the training and continuing education of clinical study site personnel and the clinical research associates who are responsible for monitoring the data. CDM input in all phases of the clinical study can improve the quality and integrity of the data. Examples of CDM input are: 1. CDM participates in the review or development of CRF completion guidelines. These guidelines provide detailed instructions to clinical study site personnel on how to complete each page of the CRF. 2. CDM participates in investigator meetings to present CRF completion guidelines and provide sample completed CRFs and DCFs. CDM provides instructions for correcting errors on CRFs on the original and how to handle errors for CRFs that have already been submitted to CDM. They communicate the flow of the data collection and cleaning process and identify potential errors in consistency across data collection forms. 3. CDM provides ongoing feedback on trends in CRF completion and frequency of data errors at clinical team

meetings or teleconferences, investigative site newsletters, and annual investigator meetings.

5 SUMMARY CDM is a crucial step in the drug development process. Good CDM processes can improve the integrity and quality of data when built on the foundation of a welldesigned clinical study and data collection tool and an adequate clinical monitoring plan. CDM is a complex process in which data from a clinical study are acquired, processed, validated according to predefined rules, and integrated into a comprehensive database for subsequent statistical analysis and reporting. Key to successful data management is a knowledgeable LDM, who applies the principles of good CDM practices in all stages of data handling. In the study initiation phase, it is important to proactively plan and document detailed processes in a Data Management Plan; develop a comprehensive data management timeline to ensure a complete and high-quality database within the required company or CRO timelines; lead the cross-functional team providing input on the design of the data collection tool following industry-accepted standards; and coordinate the design and testing of the database and edit checks before processing production data. In the next phase—study conduct—the LDM should assess the adequacy of the Data Management Plan and adapt processes to meet the requirements of the study; provide early and ongoing feedback to monitors and study sites on the quality of the data; ensure the data validation guidelines comprehensively address both anticipated and unanticipated data scenarios; and modify or add edit checks as needed. All of the planning efforts completed at the beginning of the study and the data validation steps that occur while the study is ongoing culminate in the study closure phase. During this final phase, QC procedures are carried out and the final steps to database lock are completed, including documenting

CLINICAL DATA COORDINATION

any deviations to SOPs or processes outlined in the Data Management Plan. Following database lock, completion of the statistical analysis, and a final clinical study report, the paper and electronic study data may be permanently archived. Well-planned and managed CDM processes that follow industry-accepted standards and practices can result in a timely, high-quality database. The conclusions drawn from high-quality databases can prove the safety and effectiveness of a drug or device, providing the foundation for a regulatory submission and subsequent approval and marketing. REFERENCES 1. R. K. Rondel, S. A. Varley, and C. F. Webb (eds.), Clinical Data Management. London: Wiley, 1993. 2. B. Spilker and J. Schoenfelder, Data Collection Forms in Clinical Trials. New York: Raven Press, 1991. 3. Society for Clinical Data Management, Inc., Good Clinical Data Management Practices (GCDMP), Version 3. Milwaukee, WI: SCDM, September 2003. 4. Food and Drug Administration, 21 CFR Part 11, Electronic Records; Electronic Signatures; Final Rule. Fed. Reg. 1997; 62(54): 13429–13466. 5. Food and Drug Administration, Guidance for Industry: Computerized Systems Used in Clinical Trials. Washington, DC: FDA, April 1999. 6. Clinical Data Interchange Standards Consortium (CDISC). Operational Data Model, Final Version 1.2.1. (online). Available: http://www.cdisc.org/models/odm/v1.2.1/ index.html. 7. International Conference on Harmonization, Good Clinical Practice (ICH GCP): Consolidated Guideline. Fed. Reg. 1997; 62(90). 8. K. L. Monti (2001). A statistician shows how to save time and money through data management. Appl. Clin. Trials (online). Available: http://www.actmagazine.com/ appliedclinicaltrials/article/articleDetail.jsp? id=92010. 9. Maintenance and Support Services Organization (MSSO), MedDRA Term Selection: Points to Consider, Release 3.4, November 18, 2004. (online). Available: http://www.meddramsso.com/NewWeb2003/

15

document library/9530-710%20ptc output% 20doc122204.pdf. 10. Maintenance and Support Services Organization (MSSO), Recommendation for MedDRA Implementation and Versioning for Clinical Trials. (online). Available: http://www.meddramsso.com/NewWeb2003/ Docs/clinicaltrialversioning.pdf. 11. Uppsala Monitoring Centre (2005). (online). Availabe: http://www.who-umc.org. 12. P. Bleicher (2002). Diamonds may be forever, but data? Appl. Clin. Trials (online). Available: http://www.actmagazine.com/ appliedclinicaltrials/article/articleDetail.jsp? id=87252.

CLINICAL DATA MANAGEMENT

data. As will be demonstrated in the articles that follow, the accuracy and completeness of data collected for clinical research is a primary focus of many of the activities surrounding clinical data management, whereas the focus for online retailers, airlines, and banking applications is optimization for transaction processing response and security. Clinical data management, then, encompasses the processes involved in transferring data to a computerized database; applying a variety of techniques to make sure that the data are as accurate and complete as possible; tracking all changes, modifications, additions, and deletions to the data; and providing a mechanism for delivering the data for statistical analysis.

RUTH MCBRIDE Axio Research, Seattle, Washington

Even before computers were used in clinical research, we have been collecting data, organizing it, and analyzing it. Since the advent of computers and specialized data management software, the area of clinical data management has matured to the point that clinical investigators can count on having complete, reliable, and current data for interim and final analyses. Clinical data management, as the name implies, is data management focused on clinical data, but not on all clinical data. Clinical data are collected for a variety of purposes, as part of electronic medical records to manage medical or dental practice, as part of billing information, and in the context that this section of the encyclopedia will discuss, to support clinical research and, in particular, clinical trials. In this context, there are some specialized requirements of a data management system or process. First of all, data collected for clinical trials research is primarily study participantcentric (or patient centric). This implies a hierarchical data structure where most data records are tied to participants. Within participant, data are further organized by ‘‘visit.’’ Contrast this data structure with databases designed to support, for example, banking or airlines. These databases are designed to support many transactions and are more typically relational in structure. Although commercial clinical data management systems are built on relational database servers (e.g., Oracle or SQL/Server), in order to retrieve and display data most efficiently, these systems take advantage of the hierarchical structure to build links and keys to optimize access. Second, the volume of data in clinical data management databases is relatively small compared with other commercial applications. A typical database for a phase III study might contain several thousand megabytes of information. Databases for large online retailers contain many terabytes of

1 HOW HAS CLINICAL DATA MANAGEMENT EVOLVED? 1.1 Coronary Drug Project and Early Trials As early as the 1960s computerized systems were being used for clinical data management in NIH-sponsored clinical trials. The Coronary Drug Project, which began collecting data in 1966 (1), describes data management processes that included computer files, keypunched data, and computer programs to scan for possible data problems. Originally, data for this trial were stored in fixed-length patient records on computer tape, indexed by a master patient directory. Each record contained variables or reserved space for each of the possible patient visits. Thus, a lot of precious file space was wasted. Data were keypunched onto 80 column punch cards and 100% verified before being loaded into the database. Text data were separately keyed and stored. Eventually, ‘‘key-to-disk’’ machines replaced the punch cards. Computer programs were written to check for a variety of potential data problems. The range of data edits, remarkably, was consistent with the data edits applied to more contemporary clinical databases: missing responses, where required; values outside expected limits; internal logical consistency; and data values indicating possible protocol deviations.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

CLINICAL DATA MANAGEMENT

Without intelligent data entry, however, these early data editing programs had to check to make sure that legitimate values were entered even for categorical responses. More contemporary programs will not allow an invalid entry for categorical responses such as ‘‘Yes’’/‘‘No’’ variables. In the late 1970s, the Coronary Artery Surgery Study (CASS), another NIH-sponsored clinical trial, initiated the use of distributed data entry or remote data entry (RDE). Computer workstations were installed at each of the clinical sites. The workstations allowed the clinical site staff to key study data directly to a computer diskette. Data entry programs checked for some possible data problems, generally univariate data edits. Double-entry verification was enforced. In the beginning, the computer diskettes were mailed to the Data Coordinating Center. Later during the trial, the data were electronically transmitted from the workstation at the clinical site to a similar workstation at the Coordinating Center. The key advantage to this system was that data were received at the Coordinating Center more quickly than if paper CRFs had been sent. Data editing programs could be run sooner and queries sent back to the clinical sites more quickly. A major premise driving this innovation is that the sooner that potential data errors or omissions are detected, the more likely they will be corrected. This was based on the conjecture that the clinic staff would be more likely to remember a patient visit and the medical chart would more likely still be easily available. The key-to-disk software was developed by staff at the Coordinating Center. Even at that time, there were few commercial programs to support clinical data management. The CASS Coordinating Center, like many other data coordinating centers involved in clinical trials at the time, developed its own clinical data management system (2). Separate computer programs were developed to scan the database for potential data problems. Query reports were periodically sent by mail to the clinical sites.

1.2 Development of Commercial Software to Support Clinical Data Management Until the 1980s, most clinical data management was done using ‘‘home-grown’’ software. Data Coordinating Centers at major universities supporting NIH-sponsored research, fledgling Contract Research Organizations, and pharmaceutical companies with their own data centers developed and deployed a wide variety of data systems. These early ‘‘home-grown’’ systems initially had to provide their own underlying file structure since database engines such as Oracle were not yet widely available. A few products emerged, such as the Scientific Information Retrieval System (3), which provided a database system tailored to clinical studies. SIR was a hierarchical data system optimizing for data linked together for cases and visits. It provided a robust storage format for clinical data, but at that time lacked many of the data entry and data cleaning tools of more contemporary systems. By the mid-1980s, a few commercial systems came to the market to support clinical data management. These systems added more features such as data entry, data cleaning, and reporting functions. Most of these systems presumed that paper CRFs would be sent to a central data center for entry and processing. The Internet was in its infancy, and Web-based applications were yet to come. By the mid-1990s, personal computers had penetrated the business world and the Web was quickly becoming a platform for secure business transactions. Clinical data management moved slowly toward adopting this new technology. A few university-based Data Coordinating Centers were experimenting with home-grown systems to allow data entry using Web applications. By the turn of the century, however, the market for Web-based clinical data management systems, primarily ‘‘electronic data capture’’ or eDC systems had emerged. Even into the 1990s, a big decision for any clinical data management group, however, was whether to acquire and implement a commercial software system or to develop a software system more closely tailored to internal processes.

CLINICAL DATA MANAGEMENT

2

ELECTRONIC DATA CAPTURE

Numerous articles have been written on the value of eDC systems over conventional paper CRFs. In the late 1990s, experts were predicting that by now well over half of clinical trials would be conducted using eDC systems. Granted, there has been tremendous growth in the adoption of eDC, reported by some to be as much as 23% per year (4). The number of eDC vendors has grown steadily in the intervening years, with a few systems capturing a sizeable portion of this market. However, at present, the proportion of trials that are managed using paper CRFs remains substantial. Several factors have limited the rate of adoption of eDC, including acceptance by clinical site staff. Although eDC clearly lessens the burden for storage of paper records, it does not necessarily lessen the burden on clinical site staff for collecting and transcribing that information from source documents to the CRF, whether it be paper or electronic. Not all clinics are equipped with sufficient numbers of properly configured PCs to allow research staff access to the eDC system when it would be most efficient. For example, most examination rooms do not yet have high-speed Internet access or wireless Internet access. 3 REGULATORY INVOLVEMENT WITH CLINICAL DATA MANAGEMENT For those clinical trials conducted in support of a new drug application or approval of a medical device, regulatory agencies such as the U.S. FDA, the European EMEA, and Health Canada have issued guidance documents and regulations that govern many aspects of clinical data management. The objective of these guidance documents and regulations is to assure that the information presented to the regulatory body is as complete and accurate as possible, since the safety of the patients who will use these new treatments will depend on its reliability. Perhaps the most far-reaching of these regulations is 21CFR11, Electronic Records and Signature. This regulation was issued in 1997 after extensive consultation with the pharmaceutical industry and other stakeholders. Work began on this regulation in

3

1991 with the FDA forming a Working Group in 1992 to investigate the various issues surrounding digital signature, in particular. A proposed rule was published in 1994 and resulted in a hailstorm of discussion. The final rule was issued in March 1997. The regulation, as its title implies, puts forth requirements for the use of both electronic data and electronic signatures for studies to be submitted to the U.S. FDA. The regulation’s primary focus is to make sure that electronic data are as reliable and attributable as data or information that would have been supplied on paper, and that if an electronic signature is used in place of a physical signature, that the electronic signature carries the same weight as a physical signature, is attributable, and is untamperable. Part 11 also addresses requirements for electronic systems that would support creation, storage, modification, and transmission of electronic records. These regulations lay out some fundamental requirements for clinical data management systems, such as complete electronic audit trail, attributability of data by automatic date/time stamp, and use of security measures such as passwords to prevent unauthorized access. In addition to 21CFR11, the FDA issued a guidance document in April 1999 on the use of computerized systems in clinical trials. This guidance document ‘‘applies to records in electronic form that are used to create, modify, maintain, archive, retrieve, or transmit clinical data required to be maintained, or submitted to the FDA’’ (5). The guidance was updated in May 2007 and contains extensive information about the agency’s expectations regarding the design and validation of computerized systems, such as clinical data management systems. 4

PROFESSIONAL SOCIETIES

The Society for Clinical Trials, founded in 1978 (www.sctweb.org), has always included professionals involved with clinical data management. Their annual meetings include numerous workshops and plenary sessions on various topics related to clinical data management. A supplement to the Society’s journal, Controlled Clinical Trials, published in

4

CLINICAL DATA MANAGEMENT

1995 (6) discusses the fundamentals of clinical data management. Similarly, the Drug Information Association (www.dia.org) supports professional interchange on clinical data management. Their annual meeting includes a specific track on data management, and they sponsor other meetings throughout the year with a particular focus on clinical data management. And the Society for Clinical Data Management (www.scdm.org) was founded to specifically ‘‘advance the discipline of clinical data management.’’ In addition to meetings, the SCDM offers a complete guide to ‘‘Good Clinical Data Management Practices’’ and a certification program for clinical data managers.

5

LOOK TO THE FUTURE

5.1 Standardization In recent years there has been a move toward standardizing the way that data are collected and exchanged. The Clinical Data Interchange Standards Consortium (CDISC, www. CDISC.org) was formed as a working group from the Drug Information Association. This now independent organization has developed several standards for data models that are becoming widely adopted, not just in the United States, but worldwide. The National Cancer Institute began an initiative to standardize the way that clinical data are captured with their Cancer Data Standards Repository (CaDSR) project (http://ncicb.nci.nih. gov/NCICB/infrastructure/cacore overview/ cadsr). This repository contains standardized data element definitions for a wide variety of domains involving both cancer studies and clinical studies in other areas, such as dentistry and cardiology. 5.2 Electronic Data Capture and Electronic Health Records As access to the Web continues to increase and as our society continues to move toward electronic rather than paper records, the proportion of studies conducted using electronic systems such as Web eDC systems is bound to increase. Whether eDC completely replaces paper CRFs is still a topic for debate. ‘‘Going

electronic’’ will require changes in the workflow model to take advantage of the efficiencies that an eDC system can offer. As long as the clinic staff are still recording clinical information on paper (either as clinic notes, source documentation, or paper worksheets), they are not realizing the full advantages of the eDC system since they will need to transcribe the information from paper, rather than recording it once electronically. The next level of efficiency to be gained will come from linking research needs with electronic health records (EHRs). The proportion of in-patient and out-patient facilities using electronic health records is increasing rapidly and has gained acceptance in Europe more than in the United States. A few groups have explored extracting data directly from the electronic health record to an electronic case report form. CDISC partnered with the Duke Clinical Research Institute in 2003 to demonstrate integration of electronic health records with clinical trial data (Starbrite project presented at DIA, www.cdisc.org/pdf/ss DIAJune2004lb.ppt). The objectives for EHR systems differ substantially from the objectives for eDC systems (7). Health records might contain data as discrete data points (e.g., laboratory values or blood pressure readings), or they may contain less-structured data such as progress notes or descriptions of symptoms. The purpose of the EHR is to store and convey to the health-care providers as much information as possible to assist in the diagnosis and treatment of a patient presenting at that health-care facility. The purpose of an eDC system is to collect and organize the information needed for a clinical trial, a much more focused need. The market for EHR systems is huge when compared with the market for eDC systems. Vendors of EHR systems will be driven by their market, in particular by large hospital systems. The number of EHR vendors is expected to be large, and thus, the formats for collecting data will be widely varied. The challenges are large, but the potential benefits are also large in terms of the savings in time and the decrease in transcription errors. Groups such as CDISC and PhRMA are collaborating on addressing issues related to

CLINICAL DATA MANAGEMENT

integrating clinical research with electronic health records (8,9). 6

CONCLUSION

The proliferation of the use of computers and the rapid penetration of the Internet as a means of secure exchange of data has had a profound impact on clinical data management. Where it might have taken months to collect, clean, and analyze data for a clinical trial 30 years ago, we now expect complete, relatively clean data almost as soon as a patient visit has been completed. Changes in technology have had a profound impact on the way that a trial is managed and in the role for data management staff (10). As technology has and continues to advance, the nature of clinical data management has changed, but the objective remains the same: to provide complete, accurate, and attributable data to support clinical studies, in particular, clinical trials. The articles that follow will describe various aspects of clinical data management and various approaches that have been taken to collect, organize, clean, and report clinical data. REFERENCES 1. C. L. Meinert, E. C. Heinz, and S. A. Forman, ‘‘The Coronary Drug Project: Role and Methods of the Coordinating Center’’ and other articles in this supplement. Control Clin Trials. 1983; 4(4): 355–375. 2. L. D. Fisher, M. J. Gillespie, M. Jones, and R. McBride, Design of clinical database management systems and associated software to facilitate medical statistical research. CRC Critical Reviews in Medical Informatics. Vol. I-4. 3. G. D. Anderson, E. Cohen, W. Gazdzik, and B. Robinson, ‘‘Scientific information retrieval system: A new approach to research data management’’, Proceedings of the 5th Annual ACM SIGUCCS Conference on User Services, 1977: 209–212. 4. K. A. Getz, The imperative to support site adoption of EDC. Appl. Clin. Trials. (Jan. 2006). Available: http://appliedclinical trialsonline.findpharma.com/appliedclinical trials/article/articleDetail.jsp?id=283027.

5

5. http://www.fda.gov/cder/guidance/7359fnl. pdf 6. R. McBride and S. W. Singer (eds.), Data management for multicenter studies: Methods and guidelines. Controlled Clin Trials. 1995; 16(2, suppl): 1–179. 7. P. Bleicher, Integrating EHR with EDC: When two worlds collide. Appl. Clin. Trials. (Mar. 2006). Available: http://appliedclinical trialsonline.findpharma.com/appliedclinical trials/IT/Integrating-EHR-with-EDC-WhenTwo-Worlds-Collide/ArticleStandard/Article/ detail/310798. 8. eClinical Forum and PhRMA, The Future Vision of Electronic Health Records as eSource for Clinical Research. Discussion document published Sept. 2006. Available: http://www.cdisc.org/publications/index. html. 9. CDISC, CDISC Standards and Electronic Source Data in Clinical Trials. Discussion document published Nov. 2006. Available: http://www.cdisc.org/publications/index. html. 10. F. Falk, Impact of eDC on clinical staff roles. Appl. Clin. Trials. (June 2007). Available: http://appliedclinicaltrialsonline.findpharma. com/appliedclinicaltrials/Feature+Article/ Impact-of-EDC-on-Clinical-Staff-Roles/ ArticleStandard/Article/detail/431922? contextCategoryId=35507.

CLINICAL HOLD DECISION A clinical hold is the mechanism that the U.S. Food and Drug Administration’s Center for Drug Evaluation and Research (CDER) uses when it does not believe, or cannot confirm, that the study can be conducted without unreasonable risk to the subjects/patients. If this occurs, the Center will contact the sponsor within the 30-day initial review period to stop the clinical trial. The CDER may either delay the start of an early-phase trial on the basis of information submitted in the Investigational New Drug Application (IND), or stop an ongoing study based on a review of newly submitted clinical protocols, safety reports, protocol amendments, or other information. When a clinical hold is issued, a sponsor must address the issue that is the basis of the hold before the order is removed. The CDER’s authority concerning clinical holds is outlined in federal regulations. The regulations specify the clinical hold criteria that CDER applies to various phases of clinical testing. In addition, all clinical holds are reviewed by upper management of CDER to ensure consistency and scientific quality in the Center’s clinical hold decisions.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/handbook/clinhold.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

CLINICAL SIGNIFICANCE

may be used by regulatory agencies for drug approval, by clinicians to decide between treatment alternatives, by patients to make informed decisions about treatment, by the health-care industry for formulary and reimbursement decisions, and by health-care policy makers to make policy decisions regarding resource allotment. Early evidence of the clinical implications of QOL is evident in the links between survival and QOL components such as patients’ fatigue levels, social support, and group counseling (14–17). Even a simple, single-item measure of patient global QOL can be related to patient survival (18). Changes in QOL scores can also be linked to positive economic (19, 20) and social (21) outcomes.

CELIA C. KAMATH and JEFFREY A. SLOAN Health Sciences Research Mayo Clinic Rochester, Minnesota

JOSEPH C. CAPPELLERI Pfizer, Inc. Global Research & Development Groton, Connecticut

1

INTRODUCTION

The field of patient-reported outcomes, particularly health-related quality of life (QOL), has burgeoned in the last few years (1, 2). The importance assigned to the study of these outcomes has been attributed to the aging of the population and consequently higher prevalence of chronic diseases, along with the reality that medical treatment often fails to cure the disease but may affect QOL (3). Health-related quality of life has gained attention in research and clinical trial settings (3, 4). The increasingly important role assigned by patients and clinicians to QOL’s role in medical decision-making has resulted in greater attention paid to the interpretation of QOL scores, particularly as it relates to clinical significance (5–7). Clinical significance relates to the clinical meaningfulness of inter-subject or intra-subject changes in QOL scores. Clinical significance has been difficult to determine, in part because of the development of a myriad of QOL instruments over the past decade (8, 9). Some of these developments have had little or no psychometric (1, 2, 6, 10, 11) or clinical validation (9, 12). Moreover, relative to traditional clinical endpoints like survival and systolic blood pressure, QOL as a clinical endpoint is relatively unfamiliar, especially in regard to interpretation and relevance of changes in QOL scores (13). Why is clinical significance of QOL scores important? It aids in the design of studies by helping to determine sample size calculations. Evidence of clinical significance

2

HISTORICAL BACKGROUND

Statistical significance as measured by a Pvalue is influenced by sample size and data variability. Although statistical significance can be considered a prerequisite for clinical significance, only clinical significance assigns meaning to the magnitude of effect observed in any study. Historically, Cohen (22) was responsible for proposing one of the earliest criteria for identifying important change, which can be construed as clinically significant. He suggested that a small ‘‘effect size’’ (defined later in the article) was 0.2 standard deviation units, a medium ‘‘effect size’’ was 0.5, and a large ‘‘effect size’’ was 0.8. Although his intention was to provide guidance for sample size calculations in the social and behavioral science, Cohen’s benchmarks have extended to health-care research to decide whether a change in QOL scores is important. Current research suggests that a moderate effect size of one-half a standard deviation unit (effect size = 0.5) is typically important (23). A more recent and popular definition of clinical significance uses an anchor-based approach based on an external standard that is interpretable and appreciably correlated to the target QOL measure in order to elucidate the meaning of change on the target QOL measure.

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

CLINICAL SIGNIFICANCE

Embedded under the rubric of clinical significance is the minimum important difference, a lower bound on clinical significance. One definition of a minimum important difference (MID) is ‘‘the smallest difference in score in the domain of interest which patients perceive as beneficial and which would mandate, in the absence of troublesome side-effects and excessive cost, a change in the patient’s management’’ (24). Some researchers prefer to use the term ‘‘minimally detectable difference’’ (25, 26); other definitions have sprouted (e.g., the ERES method) (27, 28). No single solution to the challenging topic of assessing clinical significance exists. Nevertheless, a series of proposals has engendered understanding and appreciation of the topic. Special issues in Statistics in Medicine (1999, Vol. 18) and the Journal of Consulting and Clinical Psychology (1999, Vol. 67) have been dedicated to the topic of clinical significance of QOL and other clinical measures. Proceedings from a meeting of an international group of about 30 QOL experts were published recently in a special issue of the Mayo Clinic Proceedings (2002, Vol. 77) (29–35), which provides practical guidance regarding the clinical significance of QOL measures.

3

ARTICLE OUTLINE

This article draws largely from recent scientific literature, including the Mayo Clinic Proceedings (29–35) and other sources (4), to provide an overview on the clinical significance of QOL measures. The following section on Design and Methodology covers the different perspectives and existing methods to determine clinical significance. The next section on Examples illustrates trials in which researchers attempted to define the concept on specific QOL measures. Then, the section on Recent Developments highlights new methods to determine clinical significance. Finally, the section on Concluding Remarks discusses some future directions for research.

4 DESIGN AND METHODOLOGY 4.1 Perspectives for Determining and Interpreting Clinical Significance Clinical significance involves assigning meaning to study results. The process of establishing such meaning can be conceptualized in two steps: (1) understanding what changes in score mean to the concerned stakeholder (e.g., patient, clinician, clinical researcher, policy maker) and (2) making results of clinical studies interpretable and comprehensible to such stakeholders or decision makers (30, 36). The term ‘‘clinical’’ in relation to significance has different meanings and implications for different stakeholders such as patients, clinicians, and society. From the patient’s perspective, clinical significance can be defined as the change in QOL scores that patients perceive as beneficial (or detrimental) and important and prompts them to seek health care or request changes in their treatment (33), or that induces patients to determine that the intervention has been successful (24). From the clinician’s perspective, it can be defined as the diagnosis of the clinician as to the amount of change in QOL scores that would mandate some form of clinical intervention (37). From the societal or population perspective, clinical significance is based on the values of the group surveyed, in which importance is defined by the outcomes that are deemed worthy of society’s resources. Any or all of these perspectives for defining clinical significance may be applicable, but they are not always in agreement (4). An equally important issue is the different perspectives for interpreting clinical meaningfulness of changes in reported QOL (35). For example, a clinician may use QOL data to explain the treatment alternatives to a patient, whereas a health-policy maker may describe to elected officials the financial impact on a patient population whose QOL has changed. Similarly, a regulatory agency and pharmaceutical company may ascertain the appropriate level of evidence for a successful research study (35). Thus, QOL results must be framed, analyzed, and presented in a way that is meaningful to the pertinent audience and its respective needs.

CLINICAL SIGNIFICANCE

Only then will the concept be meaningful and gain greater acceptance and use over time. 4.2 Methods to Explain the Clinical Significance of Health Status Measures Two common approaches used to establish the interpretability of QOL measures are termed anchor-based and distributionbased. The characteristics of each approach are described below. Several examples will be given later in the section on Examples. Interested readers are encouraged to read Crosby et al. (4) and Guyatt et al. (30) for an expanded discussion of the concepts presented here. Anchor-based approaches are used to determine clinically meaningful change via cross-sectional or longitudinal methods involve comparing measures of QOL to measures with clinical relevance (4). Crosssectional methods include several forms: (1) comparing groups that are different in terms of some disease-related criterion (38, 39); (2) linking QOL to some external benchmarking criteria (40–42); (3) eliciting preferencebased ratings on a pair-wise basis, where one person’s ratings state serves as an anchor to evaluate the other person’s ratings (43); and (4) using normative information from dysfunctional and functional populations (6). Longitudinal methods involve the comparison of changes in QOL scores across time with the use of (1) global ratings of change as ‘‘heuristics’’ to interpret changes in QOL scores (5, 24, 38, 44); (2) significant future medical events for establishing difference thresholds (45); and (3) comparisons of changes in HRQOL to other diseaserelated measures of outcome across time (46). Anchor-based methods are cataloged in Table 1 (4–6, 24, 38–41, 43–45, 47, 48). Anchor-based methods require two properties (30): (1) Anchors must be interpretable, otherwise they will hold no meaning to clinicians or patients and (2) anchors must share appreciable correlation with the targeted QOL measure. The biggest advantage of anchor-based approaches is the link with a meaningful external anchor (4), akin to establishing the construct validity of the measure (49). Potential problems, however, exist with this approach. These problems include

3

recall biases (50), low or unknown reliability and validity of the anchor measure (51), low correlation between anchor and actual QOL change score (52, 53, 54, 55), and complex relationships between anchors and QOL scores (56). Hays and Wooley (57) recommend caution in the indiscriminate dependence and use of a single minimum important difference (MID) measure. They also list several problems in estimating MIDs: The estimated magnitude could vary depending on the distributional index (57, 58), the external anchor (59), the direction of change (improvement vs. decline) (60), and the baseline value (61). In general, longitudinal methods are preferable because of their direct link with change (4). Distribution-based approaches for determining the importance of change are based on the statistical characteristics of the obtained sample, namely average scores and some measure variability in results. They are categorized as (1) those that are based on statistical significance using P-values (i.e., given no real change, the probability of observing this change or a more extreme change), which include the paired t-statistic (62) and growth curve analysis (63); (2) those that are based on sample variation (i.e., those that evaluate mean change in relation to average variation around a mean value), which include effect size (22, 64), standardized response mean (SRM) (44), and responsiveness statistic (65); and (3) those that are based on the measurement precision of the instrument (i.e., evaluate change in relation to variation in the instrument as opposed to variation of the sample), which includes the standard error of the mean (SEM) (7) and the reliable change index (RC) (6). Distributed-based methods are cataloged in Table 2 (4, 6, 7, 22, 44, 62–65). An advantage of the distribution-based methods is that they provide a way of establishing change beyond random variation and statistical significance. The effect size version of the distribution-based methods is useful to interpret differences at the group level and has benchmarks of 0.20 standard deviations units as a small effect, 0.50 as a moderate effect, and 0.80 as a large effect (22, 64, 66). The measures that seem most promising

4

CLINICAL SIGNIFICANCE

Table 1. Anchor-Based Methods of Determining Change Type

Method

Examples

HRQOL evaluated in relation to:

Advantages

Disadvantages

Crosssectional

Comparison to diseaserelated criteria

References 39 and 47

Disease severity or diagnosis

Can be standardized

May not reflect change

Easy to obtain

Groups may differ in other key variables May no reflect change

Comparison to nondiseaserelated criteria

References 40 and 41

Impact of life events

Easy to obtain

Provides external basis for interpretation

Preference rating

Longitudinal

Reference 43

Pairwise comparisons of health status

All health states are compared

Comparison to known populations

Reference 6

Functional or dysfunctional populations

Uses normative information

Global ratings of change

References 5, 24, 38, 44

Patients’ or clinicians’ ratings of improvement

Easy to obtain

Prognosis of future events

Changes in disease related outcome

Reference 45

Reference 48

Reprinted with permission from Crosby et al. (4).

Those experiencing and not experiencing some future event

Changes in clinical outcome

Best measure from individual perspective Can take into account a variety of information Prospective

Provides evidence of predictive validity Tied to objective outcome measure Known psychometric properties

Groups may differ on other key variables Relationship to HRQOL not clear May not reflect change Hypothetical, artificial Time Consuming Normative information not always available Amount of change needed not specified Does not consider measurement precision Unknown reliability

Influenced by specific rating scale and anchors Does not consider measurement precision Difficult to obtain

Does not consider measurement precision Assumes strong HRQOL-outcome correlation

CLINICAL SIGNIFICANCE

for the purpose of establishing clinical significance at the individual patient level are the SEM and the RC. These measures are based on the measurement precision of the instrument and incorporate the reliability of the instrument (e.g., Cronbach’s alpha or test-retest reliability), and the standard deviation of scores. In principle, SEM and RC are sample invariant. Researchers have favored Cronbach’s alpha over test-retest reliability to calculate reliability for the SEM (7, 30, 67). Distribution methods are particularly helpful when used together with meaningful anchors, which enhances validity and hence meaning to the QOL measure. Some encouragement exists to know that anchor-based measures appear to coincide with distribution-based methods. Researchers have found a correspondence between SEM and anchor-based determinant of a minimum important difference across difference diseases (7, 23, 67, 68). The 1 SEM benchmark corresponds with an effect size of approximately 0.5. Standard error of measurement is moderated by the reliability of the measure, where measures with higher reliability are ‘‘rewarded’’ by lowering the effect size needed to achieve a minimally important difference. A rationale for a SEM as a measure of MID is provided by Norman et al. (23) who assert that Miller’s theory (69) of the limits of human discernment is linked to the threshold of 0.5 standard deviation units. 5

EXAMPLES

This section provides examples of studies used to determine clinical significance and presents general advice for defining and interpreting clinical significance in clinical studies. Table 3 (5, 7, 24, 64, 67, 70–72) includes several examples on the use of both anchorbased methods and distribution-based methods to establish clinical significance across a wide range of QOL measures. These examples span several disease groups, instruments, and methods for determining clinical significance. Readers are encouraged to review the cited papers for further details on these studies. The authors begin with a classic paper by Jaeschke et al. (24), one of the first papers

5

on clinically meaningful differences determined through the anchor-based approach. The magnitude of difference considered minimally significant was an average of 0.5 per item on a 7-point scale, which was confirmed by Juniper et al. (5) on the Asthma Quality of Life Questionnaire (AQLQ). A third study, by Kazis et al. (64), examined the difference between statistical significance and clinical significance. Using several pain studies and the Pain Intensity numerical rating scale (PI-NRS), Farrar et al. (70) found a reduction of 2 points or 30% on the 11-point pain scale to be clinically significant. Using the Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) scale for osteoarthritis, Angst et al. (71) compared estimates derived from anchor-based and distribution-based approaches; they determined sample sizes for specified changes signifying worsening and, separately, improvement. Using the Chronic Heart Failure Questionnaire (CHQ), Wyrwich et al (7) also compared anchor- and distribution-based approaches in determining that 1 SEM equals the MCID of 0.5 per item determined by the anchor-based approach. Finally, using the Functional Assessment of Cancer Therapy-Lung (FACTL) questionnaire, Cella et al (72) showed the convergence of three different complementary approaches on clinical significance. Taken from Sprangers et al. (34), Table 4 provides a useful checklist of questions to help in interpretation of longitudinal, patient-derived QOL results presented in clinical trials and the clinical literature. These questions are based on the premise that detecting meaningful change depends on the adequacy of the research design, measurement quality, and data analysis. 6

RECENT DEVELOPMENTS

6.1 The

1 2

Standard Deviation Rule

It would be desirable to simply define, at least initially, what a clinical significant result is likely to be. Emerging research comparing anchor-based and distribution-based estimates provides an evolving standard as to what to use as an initial estimate (23). The anchor-based estimates averaging 0.5 per

6

CLINICAL SIGNIFICANCE

item on a 7-point scale appear to converge with an estimate of 12 standard deviation (SD) units. This latter estimate is derived through distribution-based methods such as the effect size approach (22, 64), the SEM (7, 67, 68), the Jacobson reliable change index (6), and the standardized response mean (73). Potential moderating factors that could impact these estimates upward or downward are the method used to determine minimum difference estimates and whether patients were suffering from acute or chronic conditions (23, 74). 6.2 Empirical Rule Effect Size Sloan et al. (27, 28) have taken this concept one step further in the form of the Empirical Rule Effect Size (ERES) by combining Cohen’s effect size categorization (22) with the empirical rule from statistical theory (75). The ERES is based on Tchebyschev’s Theorem and states that the distribution of any QOL tool is contained within 6 SDs of the observed values. The ERES entails the estimation of QOL change scores in terms of SD estimates, expressed as units on the theoretical range of a QOL instrument. Thus, small, moderate, and large effect sizes for comparing QOL treatment groups turn out to be 3%, 8%, and 13%, respectively, of the theoretical range of any QOL tool. This simple and intuitive rule to identify the magnitude of clinically significant changes is likely to be easy for clinical researchers to comprehend. The rule can facilitate the design of clinical trials in terms of sample size calculations and interim monitoring of clinical trials. The ERES framework for a priori establishment of effect sizes is sample-independent and thus an improvement over sample-dependent methods (5, 21, 76). However, the simplicity of the ERES method gives rise to some challenges and questions. The theoretical range of the instrument is rarely observed in its entirety, necessitating the modification of the theoretical range to more practical limits before calculating the ERES estimate for 1 SD as necessarily 16.7% (i.e., 1/6 of distribution of observed values) of the range. Similarly, truncated distributions, where the patient population is homogeneously ill or uniformly

healthy, can be accommodated by incorporating this knowledge into the definition of the appropriate range. These guidelines for clinical treatments can be used in the absence of other information but will need modification in their application to idiosyncratic or unique clinical settings. More research is needed to examine the generalizability of such benchmarks across baseline patient health, severity of illness, and disease groups. 6.3 Group Change vs. Individual Change Distinctions should be made in determining the significance of change at the group versus the individual level. Every individual in a group does not experience the same change in outcomes (group level outcomes are assigned a mean change value). Higher variability exists in individual responses than those of the group. Depending on the distribution of individual differences, the same group mean can have different implications for an individual (77). The traversing of group- and individuallevel QOL data entails procedures for moving from one level to the other involving two distinctive scientific traditions: deductive and inductive (31). A deductive approach is employed when one addresses the extent to which group data can be used to estimate clinical significance at the individual level. An inductive approach is used when one evaluates the extent to which individual change data can be brought to the group level to define clinical significance. Readers are advised to read Cella et al. (31) for a more detailed account. 6.4 Quality of Life as a ‘‘Soft’’ Endpoint The ‘‘softness’’ of QOL as an endpoint, relative to, say, survival and tumor response, is cited as a particular barrier to implementation and interpretation of results (13). However, methodological and conceptual strides made in defining and measuring QOL, and the growing familiarity with the interpretation and potential utility of QOL data, make those concerns increasingly outdated. Psychometric advances have been made in QOL assessment tools across disease areas (8, 78–81). Funding opportunities to study QOL endpoints have allowed for study designs

CLINICAL SIGNIFICANCE

that are large enough to have power to detect meaningful differences (13). Moreover, accumulated experience with analyzing QOL endpoints has resulted in the recognition that their statistical challenges are no different from those of ‘‘hard’’ endpoints. 7

CONCLUDING REMARKS

Several suggestions on clinical significance are offered. First, the application of multiple strategies for determining clinical significant is recommended. Doing so would enable better interpretability and validity of clinically significant change, would add to existing evidence of the magnitude of change that constitutes clinical significance, and would provide indicators of distributional parameters that create convergence or divergence in estimation of clinical significance. For example, Kolotkin et al. (46) found convergence between anchor-based and distribution-based methods at moderate level of impairment but wide disparities at mild and severe levels of impairment. Second, more research is needed to identify the effect psychometric properties—that is, reliability, validity, and responsiveness of QOL instruments— have in quantifying clinically meaningful change (4, 62, 82). Similarly, research into the psychometric properties of global rating and health transition scales used in anchor-based methods is also needed. Global ratings tend to be single item measures and may therefore fall short in terms of explaining complex QOL constructs. Anchoring assessment also tends to be positively correlated with post-treatment states but with near-zero correlation with pre-treatment states, suggesting a recall bias (83) or response shift (84). More research is needed to address the cognitive process used by patients to retrospectively assess changes in health over time (30). Third, baseline severity results in regression to the mean (RTM), an error-based artifact describing the statistical tendency of extreme scores to become less extreme at follow-up. Failure to take this tendency into account may lead to false conclusions that patients with severe impairments at baseline have shown clinically significant change

7

when, in fact, it was just RTM. RTM also has a greater impact on data when the measure is less reliable (4, 85). More research is also needed into the effect of baseline QOL impairment on magnitude of clinically meaningful change (4, 48, 66, 86, 87). Similar research is needed in terms of the generalizability of the standardized benchmarks for determining clinically meaningful change, especially for distribution-based methods (4, 66). Specifically, how satisfactory are the evolving benchmarks (effect sizes of 0.2, 0.5, and 0.8 for small, moderate, and large change, respectively) across different dimensions of QOL (e.g., mental versus physical), different disease groups (e.g., arthritis versus cancer), respondents (e.g., patients versus clinicians), measures (e.g., generic versus disease-specific), patient populations (e.g., older versus younger), or patient conditions (e.g., improving versus deteriorating)? Finally, care must be taken in presenting results of studies in a way that is familiar to the user of the information. For example, translating clinical significance into a number needed to treat (NNT) and a proportion of patients achieving various degrees of clinical benefit relative to the control may provide a desirable way to present study results (30). REFERENCES 1. N. Aaronson, Methodologic issues in assessing the quality of life of cancer patients. Cancer 1991; 67(3 Suppl): 844–850. 2. D. Cella and A. E. Bonomi, Measuring quality of life. Oncology 1995; 9(11 Suppl): 47–60. 3. R. Berzon, Understanding and Using HealthRelated Quality of Life Instruments within Clinical Research Studies. Quality of Life Assessment in Clinical Trials: Methods and Practice. Oxford, UK: 2000, pp. 3–15. 4. R. D. Crosby, R. L. Kolotkin, and G. R. Williams, Defining clinically meaningful change in health-related quality of life. J. Clin. Epidemiol. 2003; 56: 397–407. 5. E. F. Juniper, G. H. Guyatt, and A. Willan, Determining a minimal important change in a disease-specific quality of life questionnaire. J. Clin. Epidemiol. 1994; 47: 81–87. 6. N. S. Jacobson and P. Truax, Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J. Consult. Clin. Psychol. 1991; 59: 12–19.

8

CLINICAL SIGNIFICANCE 7. K. W. Wyrwich, W. M. Tiemey, and F. D. Wolinsky, Further evidence supporting an SEM-based criterion for identifying meaningful intra-individual changes in health related quality of life. J. Clin. Epidemiol. 1999; 52: 861–873. 8. D. Cella, Quality of life outcomes: measurement and validation. Oncology 1996; 10(11 Suppl): 233–246. 9. J. A. Sloan, J. R. O’Fallon, and V. J. Summan, Incorporating quality of life measurements in oncology clinical trials. Proceeding of the Biometrics Section of the American Statistical Association, 1998: 282–287.

10. B. Spilker, Quality of Life and Pharmacoeconomics in Clinical Trials. New York: Lippincott Raven, 1996. 11. D. Osoba, What has been learned from measuring health-related quality of life in clinical oncology. Eur. J. Cancer 1999; 35(11): 1565–1570. 12. J. A. Sloan and T. Symonds, Health-related quality of life measurement in clinical trials: when does a statistically significant change become relavant? unpublished manuscript, 2003. 13. M. H. S. J. Frost, Quality of life measures: a soft outcome - or is it? Amer. J. Managed Care 2002; 8(18 Suppl): S574–S579. 14. L. Degner and J. A. Sloan, Symptom distress in newly diagnosed ambulatory cancer patients as a preditor of survival in lung cancer. J. Pain Symptom Manag. 1995; 10(6): 423–431. 15. H. M. Chochinov and L. Kristjanson, Dying to pay: the cost of end-of-life care. J. Palliat. Care 1998; 14(4): 5–15. 16. R. A. Silliman, K. A. Dukes, and L. M. Sullivan, Breast cancer care in older women: sources of information, social support, and emotional health outcomes. Cancer 1998; 83(4): 706–711. 17. D. Spiegel, J. R. Bloom, and H. Kraemer, Psychological support for cancer patients. Lancet 1989; 2(8677): 1447. 18. J. A. Sloan, C. L. Loprinzi, and S. A. Kuross, Randomized comparison of four tools measureing overall quality of life in patients with advanced cancer. J. Clin. Oncol. 1998; 16: 3662–3673. 19. D. L. Patrick and P. Erickson, Applications of health status assessment to health policy. In: B. Spilker (ed.), Quality of Life and Pharmacoeconomics in Clinical Trials. New York: Lippincott Raven, 1996, pp. 717–727.

20. M. R. Gold, D. L. Patrick, and G. W. Torrance, Identifying and valuing outcomes. In: M. Gold et al. (eds.), Cost Effectiveness in Health and Medicine. New York: Oxford University Press, 1996, pp. 82–134. 21. E. F. Juniper, The value and quality of life in asthma. Eur. Resp. J. 1997; 7: 333–337. 22. J. Cohen, Statistical Power Analysis for the Behavioral Sciences, 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates, 1988. 23. G. R. Norman, J. A. Sloan, and K. W. Wyrwich, Interpretation of changes in health-related quality of life: the remarkable universality of half a standard deviation. Med. Care 2003; 41(5): 582–592. 24. R. Jaeschke, J. Singer, and G. H. Guyatt, Measurement of health status. Ascertaining the minimal clinically important difference. Control Clin. Trials 1989; 10(4): 407–415. 25. P. Jones, Interpreting thresholds for a clinically significant change in health status (quality of life) with treatment for asthma and COPD. Eur. Resp. J. 2002; 19: 398–404. 26. J. G. Wright, The minimally important difference: who’s to say what is important? J. Clin. Epidemiol. 1996; 49: 1221–1222. 27. J. A. Sloan et al., Detecting worms, ducks, and elephants: a simple approach for defining clinically relevant effects in quality of life measures. J. Cancer Integrative Med. 2003; 1(1): 41–47. 28. J. A. Sloan, T. Symonds, D. Vargas-Chanes, and B. Fridley, Practical guidelines for assessing the clinical significance of health-related QOL changes within clinical trials. Drug Inf. J. 2003; 37: 23–31. 29. J. A. Sloan et al., Assessing clinical significance in measuring oncology patient quality of life: introduction to the symposium, content overview, and definition of terms. Mayo Clin. Proc. 2002; 77: 367–370. 30. G. H. Guyatt et al., Methods to explain the clinical significance of health status measures. Mayo Clin. Proc. 2002; 77: 371–383. 31. D. Cella et al., Group vs individual approaches to understanding the clinical significance of differences or changes in quality of life. Mayo Clin. Proc. 2002; 77: 384–392. 32. J. A. Sloan et al., Assessing the clinical significance of single items relative to summated scores. Mayo Clin. Proc. 2002; 77: 479–487. 33. M. H. Frost et al., Patient, clinician, and population perspectives on determining the clinical significance of quality-of-life scores. Mayo Clin. Proc. 2002; 77: 488–494.

CLINICAL SIGNIFICANCE

9

34. M. A. G. Sprangers et al., Assessing meaningful change in quality of life over time: a users’ guide for clinicians. Mayo Clin. Proc. 2002; 77: 561–571.

47. R. A. Deyo et al., Physical and psychosocial function in rheumatoid arthritis: clinical use of a self-adminstered health status instrument. Arch. Intern. Med. 1992; 142: 879

35. T. Symonds et al., The clinical significance of quality-of-life results: practical considerations for specific audiences. Mayo Clin. Proc. 2002; 77: 572–583.

48. R. L. Kolotkin, R. D. Crosby, and G. R. Williams, Integrating anchor-based and distribution-based methods to determine clinically meaningful change in obesity-specific quality of life. Qual. Life Res. 2002; 11: 670.

36. M. A. Testa, Interpretation of quality-of-life outcomes issues that affect magnitude and meaning. Med. Care 2000; 38: II166–II174. 37. C. van Walraven, J. L. Mahon, D. Moher, C. Bohm, and A. Laupacis, Surveying physicians to determine the minimal important difference: implications for sample-size calculation. J. Clin. Epidemiol. 1999; 52: 717–723. 38. R. A. Deyo and T. S. Inui, Toward clinical application of health status measures: sensitivity of scales to clinically important changes. Health Serv. Res. 1984; 19: 278–289. 39. P. A. Johnson, L. Goldman, E. J. Orav et al., Comparison of the medical outcomes study short-form 36-item health survey in black patients and white patients with acute chest pain. Med. Care 1995; 33: 145–160. 40. J. E. Ware, R. H. Brook, A. Davies-Avery et al., Conceptualization and Measurement of Health for Adults in the Health Insurance Study, vol. 1. Model of Health and Methodology. Santa Monica, CA: Rand Corporation, 1979. 41. M. Testa and W. R. Lenderking, Interpreting pharmacoeconcomic and quality-of-life clinical trial data for use in therpeutics. Pharmacoeconomics 1992; 2: 107. 42. M. Testa and D. C. Simonson, Assessment of quality-of-life outcomes. N. Engl. J. Med. 1996; 28: 835–840. 43. H. A. Llewellyn-Thomas, J. I. Williams, and L. Levy, Using a trade-off techniques to assess patients’ treatment preferences for benign prostatic hyperplasia. Med. Decis. Making 1996; 16: 262–272. 44. G. Stucki, M. H. Liang, and A. H. Fossel, Relative responsiveness of condition specific and health status measures in degenerative lumbar spinal stenosis. J. Clin. Epidemiol. 1995; 48: 1369–1378. 45. J. M. Mossey and E. Shapiro, Self-rated health: a predictor of mortaility among the elderly. Amer. J. Public Health 1982; 72: 800–808. 46. R. L. Kolotkin, R. D. Crosby, and K. D. Kosloski, Development of a brief measure to assess quality of life in obesity. Obes. Res. 2001; 9: 102–111.

49. E. Lydick and R. S. Epstein, Interpretation of quality of life changes. Qual. Life Res. 1993; 2: 221–226. 50. N. Schwartz and S. Sudman, Autobiographical Memory and the Validity of Retrospective Reports. New York: Springer-Verlag, 1994. 51. K. W. Wyrwich, S. Metz, and A. N. Babu, The reliability of retrospective change assessments. Qual. Life Res. 2002; 11: 636. 52. B. Mozes, Y. Maor, and A. Shumueli, Do we know what global ratins of health-related quality of life measure? Qual. Life Res. 1999; 8: 269–273. 53. J. R. Kirwan, D. M. Chaput de Sainttonge, and C. R. B. Joyce, Clinical judgment in rheumatoid arthritis. III. British rheumatologists’ judgment of ‘change in response to therapy.’ Ann. Rheum. Dis. 1984; 43: 686–694. 54. D. Cella, E. A. Hahn, and K. Dineen, Meaningful change in cancer-specific quality of life scores: differences between improvement and worsening. Qual. Life Res. 2002; 11: 207–221. 55. G. H. Guyatt and R. Jaeschke, Reassessing quality of life instruments in the evaluation of new drugs. Pharmacoeconomics 1997; 12: 616–626. 56. F. Lydick and B. P. Yawn, Clinical interpretation of health-related quality of life data. In: M. J. Staquet et al. (eds.), Quality of Life Assessment in Clinical Trials: Methods and Practice. Oxford: Oxford University Press, 1998, pp. 299–314. 57. R. D. Hays and J. M. Wooley, The concept of clinically meaningful difference in healthrelated quality-of-life research. How meaningful is it? Pharmacoeconomics 2000; 18(5): 419. 58. J. Wright and N. L. Young, A comparison of different indices of responsiveness. J. Clin. Epidemiol. 1997; 50: 239–246. 59. B. Barber, N. C. Santanello, and R. S. Epstein, Impact of the global on patient perceivable change in an asthma specific QOL questionnaire. Qual. Life Res. 1996; 5: 117–122. 60. J. Ware, K. Snow, M. Kosinski et al., SF-36 Health Survey: Manual and Interpretation

10

61.

62.

63.

64.

65.

66.

67.

68.

69.

70.

71.

72.

CLINICAL SIGNIFICANCE Guide. Boston, MA: The Health Institute, 1993. D. W. Baker, R. D. Hays, and R. H. Brook, Understanding changes in health status: is the floor phenomenon merely the last step of the staircase? Med. Care 1997; 35: 1–15. J. A. Husted, R. J. Cook, and V. T. Farewll, Methods for assessing responsiveness: a critical review and recommendations. J. Clin. Epidemiol. 2000; 53: 459–468. D. C. Speer and P. D. Greenbaum, Five methods for computing significant individual client change and improvement rates: support for an individual growth curve approach. J. Consult. Clin. Psychol. 1995; 63: 1044–1048. L. Kazis, J. J. Anderson, and R. S. Meenan, Effect sizes for interpreting changes in health status. Med. Care 1989; 27(Suppl 3): S178–S189. G. H. Guyatt, C. Bombardier, and P. X. Tugwell, Measuring disease-specific quality of life in clinical trials. CMAJ 1986; 134: 889–895. G. Samsa, D. Edelman, and M. L. Rothman, Determining clinically important differences in health status measures: a general approach with illustration to the Health Utilities Index Mark II. Pharmacoeconomics 1999; 15: 41–55. K. W. Wyrwich, N. A. Nienaber, and W. M. Tiemey, Linking clinical relevance and statistical significance in evaluating intraindividual changes in health-related quality of life. Med. Care 1999; 37: 469–478. K. W. Wyrwich, W. M. Tiemey, and F. D. Wolinsky, Using the standard error of measurement to identify important changes on the Asthma Quality of Life Questionnaire. Qual. Life Res. 2002; 11: 1–7. G. G. Miller, The magic number seven plus or minus two: some limits on our capacity for processing information. Psychol. Rev. 1956; 63: 81–97. J. T. Farrar et al., Clinical importance of changes in chronic pain intensity measured on an 11-point numerical pain rating scale. Pain 2001; 94: 149–158. F. Angst, A. Aeschlimann, and G. Stucki, Smallest detectable and minimal clinically important differences of rehabilitation intervention with their implication for required sample sizes using WOMAC and SF-36 quality of life measurement instruments in patients with osteoarthritis of the lower extremities. Arthrit. Care Res. 2001; 45: 384–391. D. Cella et al., What is a clinically meaningful change on the Functional Assessment of Cancer Therapy-Lung (FACT-L) Questionnaire?

Results from Eastern Cooperative Oncology Group (ECOG) Study 5592. J. Clin. Epidemiol. 2002; 55: 285–295. 73. C. McHorney and A. Tarlov, Individualpatient monitoring in clinical practice: are available health status measures adequate? Qual. Life Res. 1995; 4: 293–307. 74. A. L. Stewart, S. Greenfield, and R. D. Hays, Functional status and well-being of patients with chronic conditions: results from the medical outcomes study. JAMA 1989; 262: 907–913. 75. F. Pukelsheim, The three sigma rule. Amer. Stat. 1994; 48: 88–91. 76. E. F. Juniper, G. H. Guyatt, and D. H. Feeny, Measuring quality of life in childhood asthma. Qual. Life Res. 1996; 5: 35–46. 77. G. Guyatt, E. F. Juniper, S. D. Walter, L. E. Griffith, and R. S. Goldstein, Interpreting treatment effects in randomized trials. BMJ 1998; 316: 690–693. 78. O. Chassany et al., Patient-reported outcomes: the example of health-related quality of life - a European guidance document for the improved integration of health-related quality of life assessment in the drug regulatory process. Drug Inf. J. 2002; 36: 209–238. 79. C. Speilberger, State-Trait Anxiety Inventory:STAI (Form Y). Palo Alto, CA: Consulting Psychologists Press, Inc., 1983. 80. L. Radloff, The CES-D scale: a self-report depression scale for research in the general population. Appl. Psychol. Meas. 1977; 1: 385–481. 81. D. M. McNair, M. Lorr, and L. F. Droppleman, Profile of mood states manual. San Diego, CA: EdiTS, 1992. 82. R. D. Hays, R. Anderson, and D. Revicki, Psychometric considerations in evaluating health-related quality of life measures. Qual. Life Res. 1993; 2: 441–449. 83. G. R. Norman, P. W. Stratford, and G. Regehr, Methodological problems in the retrospective computation of responsiveness to change: the lessons of Cronbach. J. Clin. Epidemiol. 1997; 50(8): 869–879. 84. C. E. Schwartz and M. A. G. Sprangers, Methodological approaches for assessing response shift in longitudinal health-related quality-of-life research. Social Sci. Med. 1999; 48: 1531–1548. 85. M. T. Moser, J. Weis, and H. H. Bartsch, How does regression to the mean affect thresholds of reliable change statistics? Simulations and examples for estimation of true change in

CLINICAL SIGNIFICANCE cancer-related quality of life. Qual. Life Res. 2002; 11: 669. 86. C. McHorney, Generic health measurement: past accomplishments and a measurement paradigm for the 21st century. Ann. Int. Med. 1997; 127: 743–750. 87. P. W. Stratford, J. Binkley, and D. L. Riddle, Sensitivity to change of the Roland-Morris Back Pain Questionnaire: part 1. Phys. Ther. 1998; 78: 1186–1196.

11

12

Table 2. Distribution-Based Methods of Determining Change Reference

HRQOL evaluated in relation to:

Paired t-statistic

62

Standard error of the mean change

Growth curve analysis

63

Standard error of the slope

Effect size

22, 64

Pre-test standard deviation

Calculation


x1 − x0  (di − d)2 n(n−1)

B √ V




x1 − x0  (x0 − x0 )2 n−1

Advantages

Disadvantages

None

Increases with sample size

Not limited to pre-test and post-test scores Uses all of the available data

Increases with sample size

Standardized units

Benchmarks for interpretation Independent of sample size Standardized response mean

Responsiveness statistic

44

65

Standard deviation of change

Standard deviation of change in a stable group







x1 − x0  (di − d)2 n−1

x1 − x0  (di stable − dstable )2 n−1

Standardized units

Requires large sample sizes Assumes data missing at random Decreases with increased baseline variability of sample Does not consider variability of change May vary widely among samples Varies as a function of effectiveness of treatment

Independent of sample size Based on variability of change Standardized units

More conservative than effect size Independent of sample size Takes into account spurious change due to measurement error

Data on stable subjects frequently not available

CLINICAL SIGNIFICANCE

Method

Table 2. (continued) Method Standard error of measurement

Reliable change index

Reference 7

6

HRQOL evaluated in relation to: Standard error measurement

Standard error of the measurement difference

Calculation


x1 − x0 

√ (x0 − x0 )2 ( 1 − r) (n − 1)

x1 − x0  2(SEM)2

Advantages

Disadvantages

Relatively stable across populations

Assumes measurement error to be constant across the range of possible scores

Takes into account the precision of the measure Cutoffs based on confidence intervals Relatively stable across populations

Reprinted with permission from Crosby et al. (4).

CLINICAL SIGNIFICANCE

Takes into account precision of measure Cutoffs based on confidence intervals

Assumes measurement error to be constant across the range of possible scores

13

14

CLINICAL SIGNIFICANCE

CLINICAL SIGNIFICANCE

15

16

CLINICAL SIGNIFICANCE

CLINICAL SIGNIFICANCE

17

18

– Is the questionnaire appropriate given the research objective and the rationale for QOL assessment? – Is the questionnaire appropriate given the domains included and in light of the disease and population characteristics? – Is the questionnaire reliable and valid? Is this information reported in the article? – Is the questionnaire responsive to change? Is this information reported in the article?

Is the QOL questionnaire relevant, reliable, valid, and responsive to change?

– What are their disease (e.g., tumor type), treatment (e.g., duration), sociodemographic and cultural (e.g., age, ethnicity), and behavioral (e.g., alcohol use) characteristics? – To what extent are the QOL data applicable to your patients? – Is actual QOL status of individual patients reported (e.g., by providing confidence intervals, standard deviations, subgroup data, individual data plots), thus documenting the amount of individual variation in response to treatment?

What are the characteristics of the population for whom changes in QOL are reported?

– Is the sample size appropriate for the research questions (e.g., by providing a power calculation)? – Is a rationale and/or source for the anticipated effect size specified?

Is the study adequately powered?

– Is QOL assessed at appropriate times to document treatment course, clinical events, and post-treatment effects? – Are standard research design procedures followed (e.g., avoidance of respondent burden, collection of data prior to treatment or consultation)? – Is the timing of the QOL assessments similar across treatment arms?

– Is a baseline assessment included? – Is QOL assessed at appropriate times for determining minimally important change given the natural course of the disease? – Is QOL assessed long enough to determine a clinical effect, taking disease stage into account?

Are the timing and frequency of assessments adequate?

– Are patients’ baseline QOL scores close to the extremes of the response scale? Do the treatment groups differ in baseline QOL?

– Is the questionnaire appropriate given practical considerations (e.g., regarding respondent burden and the availability of different language versions)?

Table 4. Checklist for Assessing Clinical Significance over Time in QOL

– Do the tabular and graphical presentations take the problems inherent in the data into account (e.g., presence of floor and ceiling effects, patient attrition)? – Are the data appropriately analyzed (e.g., are all time points included, are missing data taken into account, are pre-treatment co-variates included)? – Does the article provide sufficient information on the statistical models selected?

– Are the data presented in a meaningful and suitable way enabling an overview of QOL changes over time?

How are multiple time-points handled?

– Is the adopted approach of handling multiplicity explicitly described? – Which approach is taken: limiting the QOL outcomes, use of summary measures, adjustment of p-values, and/or multivariate statistical analysis and modeling? – Did the interpretation of the results take the problem of multiple outcomes into account?

How are multiple QOL outcomes addressed in analyses?

– Does the power calculation take into account: the scale range of the anticipated effect, the score distribution (i.e., magnitude and form), the number of outcome measures, and research hypothesis (i.e., equivalence versus difference)?

19

Reprinted with permission from Sprangers et al. (34).

– Does the article report the reasons for missing questionnaires? – Is there an association between patients’ health status and missing QOL data? – If patients with incomplete data are excluded from the analysis (e.g., by using complete case methods), does the article document that these are nonignorable missing data?

– Are missing data handled adequately? – Does the article indicate how missing items within a questionnaire are handled? – Does the article report the number of missing questionnaires at each scheduled assessment?

Can alternative explanations account for the observed change or lack of observed change? Are dissimilar baseline characteristics adequately accounted for? – Is the baseline QOL score used as a co-variate?

Table 4. (continued)

– Are changes in patient’s internal standards, values, and/or the conceptualization of QOL explicitly measured? – Are insignificant or small changes in QOL reported despite substantial changes in patient’s health status (i.e., deterioration or improvement)?

Did the patient’s QOL perspective change over time?

– Is observed survival difference combined with QOL in evaluating change? – If patients have died in the course of the study, is mortality accounted for in the evaluation of QOL? – Are summary indices (e.g., QALYs, Q-TWiST) or imputation techniques used?

– In cases of non-ignorable missing data, are several analytical approaches presented to address possible bias in conclusions based on this QOL data set?

– To what extent is the statement of clinical importance appropriate and empirically warranted?

– Does the article provide some guidance regarding the clinical importance of the observed change in QOL?

How is statistical significance translated into meaningful change?

– How likely is it that patients have changed their internal standards, values, and/or their conceptualization of QOL as a result of adaptation to deteriorating or improving health?

CLINICAL TRIAL MISCONDUCT

the present situation in the United States, where handling misconduct has become routine, and the United Kingdom, where efforts to establish consensus have been fragmentary and fitful, serves as a sharp reminder of the importance of facing up to and dealing with this unpleasant side of science. It also confirms that scientific institutions are unlikely to deal with the problem, except on a case-by-case basis, unless forced to do so by their governmental paymasters (5).

DRUMMOND RENNIE University of California Institute for Health Policy Studies San Francisco, California

1

THE SCOPE OF THIS ARTICLE

The purpose of clinical trials of interventions is to obtain unbiased, relevant, and reliable information on the value of the interventions (1). Many ethical problems can occur during a trial. It can be designed incorrectly, for example, so that the results of a placebo-controlled trial on a me-too drug are worthless to the clinician who wants to know whether it represents an improvement over the best already available. Randomization can be broken and concealment made inadequate. The trial can be under-powered so that the research participants are put at risk without any hope of obtaining results of value to mankind. The trial may be stopped early by the sponsor for financial reasons (2), the results of the trial can be buried, or the same results can be reported repeatedly by different authors without cross-referencing (3), resulting in publication bias or distortion of the published literature. These and other such actions are, to varying degrees, unethical, and all of them have at one time or another been labeled research or scientific misconduct (4), but in the United States at least, they fall outside the federal definition of research misconduct and no one would be found guilty of misconduct for such acts alone. It is particularly useful to look at the history and present situation of scientific misconduct in the United States to see how this came about, because it was in the United States that the first well-publicized cases occurred, and the United States was the first country to take decisive action to deal with it. This action took the part of forming a consensus that a problem to be solved existed; reaching a definition; developing a process to be followed when allegations arose; and modifying the process in response to experience and new case law. The contrast between

2 WHY DOES RESEARCH MISCONDUCT MATTER? Crooked research, clinical or otherwise, corrupts the record, leading other scientists down false trails. In clinical trials, it inevitably distorts the apparent efficacy of interventions, may falsely raise the expectations of patients and physicians alike, and, when uncovered, leads to public anger, skepticism, and loss of confidence in research in general and of all research institutions. Within trials and within institutions, misconduct, if not dealt with, leads to loss of morale, cynicism, and resignations of honest investigators. It often leads to the ostracism of whistleblowers, even if they are correct in their accusations, and to deep divisions within research teams. Clinical trials are exceedingly expensive, so efforts to correct tainted research may be thwarted because replication may be too costly to organize. Finally, the process of investigation is expensive and involves numerous busy people, with more pressing claims on their time, in extra work. 3

EARLY CASES

The modern history of research misconduct started in 1974 at Sloan-Kettering with Summerlin and his painted mouse (6–12). Over the next 15 years, numerous spectacular cases occurred at major U.S. research universities, later summarized by numerous books and articles. These cases were reported fully by the media, as were the responses of the scientific establishment. In general,

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

CLINICAL TRIAL MISCONDUCT

scientists, accustomed to trusting their colleagues, had great difficulty imagining that colleague would ever break that trust. They denied fraud could occur, or maintained, in the absence of any evidence, that those who committed fraud were extreme aberrations or sick individuals. Few seemed to accept the common sense view that some percent of scientists, as with any other profession, were likely to be fraudulent. ‘‘The response of each research institution varied but was only too often characterised by circling the wagons, denial, and cover up. Under the eyes of the press, each institution would hurriedly patch together its own process, assembling ad hoc panels, sometimes with glaring conflicts of interest. The results were frequently slow, bungled, idiosyncratic, and unfair to almost everyone’’ (5). Politicians became involved in 1981 when then Congressman Al Gore held the first of some dozen hearings at which delinquent scientists were called on the carpet, whistleblowers testified on how they had been abused, and important institutional administrators denied to openly skeptical audiences that any problem existed at all (5, 13–15). At the close of the first hearing, Gore could not ‘‘avoid the conclusion that one reason for the persistence of this type of problem is the reluctance of people high in the science field to take these matters seriously’’ (13). In the face of inertia, denial and opposition on the part of the scientific establishment and an increasingly rancorous atmosphere, Congress, using the justification that it was responsible for overseeing how tax dollars were spent, and that faked research was a fraud against the public purse, forced the setting up of an Office of Scientific Integrity (OSI) in the National Institutes of Health (NIH) in 1989 (6). Rules and a definition of misconduct were promulgated, and the process for investigation and adjudication laid down. Although governmental regulations did not apply to those on non-governmental money, the new regulations applied to all institutions receiving money from the government, so this effectively meant that all biomedical research institutions in the United States had to draw up institutional rules that complied with the government’s definition of misconduct and procedures for responding to it. Although

those funded by the National Science Foundation operated under a slightly different system, the governmental rules became, de facto, the universal rules within biomedicine. After several years, the initiative and the OSI were moved outside the NIH, but operated only within the Public Health Service. The OSI transmogrified into the more lawyerheavy Office of Research Integrity (ORI). It provoked bitter attacks from scientists, politicians, accused persons and their attorneys, whistleblowers, and the press. The procedures were said to be amorphous, inconsistent, illegal, and poorly articulated, the process too quick, too slow, too timid, too aggressive, and often bungled. The accused and whistleblower/accusers both complained of being deprived of their rights (6, 15). The OSI/ORI operated on a collegial ‘‘scientific dialog’’ model, intended to keep the process in the hands of scientists, rather than lawyers. But as cases accumulated and the OSI/ORI lost them when they were appealed to the courts, this was found to be legally flawed and unfair to the accused, and was abandoned for a process that followed the stricter rules of administrative law. Meanwhile, some whistleblowers, exasperated by the confusion, pursued an alternative route provided by the False Claims Act, which qualified them to share in a portion of any grant monies recovered through the courts rather than through the designated institutional routes (6, 15). 4 DEFINITION The process of developing the regulations had been greatly facilitated by a successful series of meetings, endorsed by the American Bar Association and scientific societies, notably the American Association for the Advancement of Science and attended by scientists, administrators, politicians, and lawyers. The most fundamental issue, the definition, was a bone of contention from the start. In general, scientists, well aware of the problems caused by breakdowns in the complex interpersonal relationships formed during team research, and afraid of the devastating effect of malicious and unfounded accusations, attempted to limit the definition to the relatively clear

CLINICAL TRIAL MISCONDUCT

crimes of fabrication, falsification, and plagiarism. Lawyers and politicians, who were mainly lawyers, were concerned to separate definitely criminal acts from uncouth and uncollegial behavior, (to separate, in Tina Gunsalus’ phrase, ‘‘the crooks from the jerks’’) and to separate those acts, such as sexual harassment, which could occur outside the scientific environment and for which laws already existed. Abused whistleblowers, (many of whom, although correct in their accusations, had ended up losing their jobs because of the universal tendency on the part of friends to rally around their accused colleagues, no matter what the evidence) often felt that their rights had been violated and usually sought a wider definition. The precise details of each of their cases differed, so there was pressure from them to include a large number of acts, from duplicate publication to impolite behavior, in the definition of research misconduct. Institutions were concerned that they have control over disciplining their faculty, while at the same time not being made to sink under the weight of elaborate new rules that were expensive to enforce. Everyone wanted clarity because, without a clear and universal definition, everyone had to invent their own rules, leading to confusion and unfairness. It was decided to use the term ‘‘misconduct’’ rather than ‘‘fraud,’’ because a finding of fraud, usually financial, needed several specific requirements to be proved, which would have been impossible to sustain in the case of purely scientific irregularities. The definition of scientific misconduct adopted by the OSI in 1989, was ‘‘Fabrication, falsification, plagiarism or other practices that seriously deviate from those that are commonly accepted within the scientific community for proposing, conducting, or reporting research’’ (16). Although everyone agreed that fabrication, falsification, and plagiarism were antithetical to good science, large scientific organizations immediately objected to the inclusion of ‘‘other practices . . . ’’ on the grounds that this would include, and so inhibit, breakthrough, unconventional science. They wanted the definition limited to fabrication, falsification, and plagiarism, what came to be called ‘‘FF&P.’’ Although this argument

3

seemed spurious to many, underlying the concern was the understandable fear that scientists would be found guilty of practices they had no idea could be wrong. Against this notion, others argued that many cases were not covered by FF&P, for example, stealing work during peer review. There could not be a definition that implied that such behaviors were not wrong. 5

INTENT

Gruber said: ‘‘The power and the beauty of science do not rest upon infallibility which it has not, but on corrigibility without which it is nothing’’ (17). No one could undertake the risky and uncertain business of science if every error were to be construed as misconduct. Distinguishing error from misconduct requires making a judgment about intent. ‘‘Misconduct’’ in law meals the ‘‘willful’’ trangression of a definite rule. Its synonyms are ‘‘misdemeanor, misdeed, misbehavior, delinquency, impropriety, mismanagement, offense, but not negligence or carelessness’’ (18), which is one reason why the definition decided on in 1999 at a consensus conference in Edinbugh, Scotland, (‘‘Behavior by a researcher, intentional or not, that falls short of good ethical and scientific standards.’’) is doomed (19). The law has a long history of judging intent, and that is what, in addition to deciding on the facts, panels looking into scientific misconduct must do. 6 WHAT SCIENTIFIC MISCONDUCT WAS NOT What became clear from the battle was that a great many examples of egregious conduct had to be dealt with, if at all, by different mechanisms. Thus, failure to obtain informed consent, or theft of laboratory funds, or mistreatment of laboratory animals, or accepting money to do fake research while billing the government for the costs, were serious offenses covered by laws concerning conduct of trials, laws against theft, laws governing animal research, or anti-kickback laws. Similarly, inserting the results of fake experiments as supporting evidence in grant applications to the government contravened the

4

CLINICAL TRIAL MISCONDUCT

law that, in the United States at least, it is illegal to lie intentionally to the government. For an excellent discussion of legal issues in research, the reader is referred to Kalb and Koehler (20). All sorts of other unethical behaviors were left to be dealt with by the institutions, the scientific community, and the scientific journals. So, for example, datadredging, duplicate publication, ghost and guest authorship, failure to publish, starting trials in the absence of equipoise, failure to share data, or failure to reveal massive financial conflicts of interest, and many other practices damaging to science and exasperating to scientists were either by implication condoned or left to the community to sanction. In this chapter, such issues or problems in clinical trials such as mismanagement of funds or improper and biased statistical analysis will not be addressed. 7

THE PROCESS

The 1989 regulations detailed a process to be followed whenever an allegation was received, either by an official at the research institution, or by the OSI/ORI (henceforth called the ORI), which stipulated an initial inquiry by the institution to see whether there might be any merit to the accusation, and, if likely, a full investigation. This investigation was to be carried out in the institutions by people with no conflict of interest, with the results to be forwarded to the ORI. In addition, the ORI was given a monitoring function to assure the public that the all research institutions were taking the matter seriously and complying with the regulations. The most important change, made in 1993, was the introduction of an appeals process before a Research Integrity Adjudication Panel appointed by the Departmental Appeals Board of the Department of Health and Human Services (DHHS). The process now is that the ORI reviews the findings and either (rarely) refers to the DHHS Office of Inspector General, or almost always, refers to the Assistant Secretary of Health for accepting or rejecting the institutions’ recommendations. If a finding of misconduct occurs, the ORI would impose sanctions or negotiate an agreement with the

accused. The accused could then appeal to the Departmental Appeals Board, where the accused scientist has a right to be represented by counsel, to discovery of all evidence used to convict, to cross examine witnesses, including the accuser, and to participate in the hearing (21). An independent analysis of policies and procedures at institutions approved by the ORI, done in 2000, has shown considerable variation between institutions in all phases of the handling of cases of suspected misconduct before the report goes to the ORI (22), and low levels of due process. For example, only 32% had policies requiring the accused have access to all the evidence, and only 21% the right to present witnesses (22). Mello and Brennan have criticized the process as providing insufficient safeguards and due process for accused scientists, for whom a finding of misconduct would be devastating (21). 8 THE PAST DECADE Despite an apparently clear and workable system, the controversy continued and, in the early 1990s, several high-profile cases were decided against government science agencies and their ways of proceeding. To try to restore calm, the U.S. Public Health Service set up a Commission on Research Integrity (named the Ryan Commission, after its chair, Kenneth J. Ryan of Harvard Medical School). The Commission heard testimony in 15 meetings across the United States from a large number of witnesses, including scientists and their organizations, whistleblowers, attorneys, institutions, the press, interested citizens, and government officials. Their report was completed in November 1995 (23). The Commission recommended that the definition of research misconduct should be ‘‘based on the premise that research misconduct is a serious violation of the fundamental principle that scientists be truthful and fair in the conduct of research and the dissemination of its results’’ (23). ‘‘Research misconduct is significant misbehavior that improperly appropriates the intellectual property or contributions of others, that intentionally impedes the progress

CLINICAL TRIAL MISCONDUCT

of research, or that risks corrupting the scientific record or compromising the integrity of scientific practices. Such behaviors are unethical and unacceptable in proposing, conducting, or reporting research or in reviewing the proposals or research reports of others’’ (23). The commission specifically included within this definition ‘‘misappropriation,’’ which included plagiarism and making use of another’s ideas and words during peer review; ‘‘interference’’ with another’s research; and ‘‘misrepresentation’’ by reporting scientific data falsely. Its members having the advantage of hearing many actual cases, the Commission also included obstruction of investigations of research misconduct, and noncompliance with research regulations (23). Finally, the Commission, recognizing that whistleblowers represented an invaluable quality assurance mechanism, but that whistleblowers had to receive protection from retaliation lest they suffer damage to their careers as a result of their actions, presented an appendix ‘‘Responsible Whistleblowing: a Whistleblower’s Bill of Rights.’’ The reaction to the report varied from enthusiastic (Nature and the Lancet) to angry. The President of the Federation of American Societies for Experimental Biology (FASEB) wrote to the Secretary for Health and Human Services that the ‘‘Commission’s report is so seriously flawed that it is useless as a basis for policy making and should be disavowed . . . we find the definition to be unworkable, and therefore unacceptable’’ (24). He was quoted in the press as calling the report ‘‘an attack on American science,’’ (25) surprising words given the reliance the commission had put on the National Academy of Science’s own report (26, 27). It seems likely that the cause of this excessive reaction was the Whistleblower’s Bill of Rights, the insertion of which was interpreted not as attention to the plight of the accuser, but as a failure to protect the rights of the already well-protected accused. Once again, the bogeyman that regulation beyond FF&P would inhibit groundbreaking research was raised, this time to attack the Commission, even though in thousands of actual cases this had never once happened. In response to court decisions, changes to the process were introduced to allow more

5

protections for the accused and to bring the entire process more into line with the procedures set down by administrative law. Institutions became used to handling cases, the process was seen to be fairer, and the late 1990s were characterized by a decrease in the shrillness of the debate. On December 6, 2000, the United States issued the new, government-wide regulations defining research misconduct and laying down the rules for investigation and adjudication of allegations of misconduct concerning research done with U.S. federal funds (Fig. 1) (28). As all important universities and research institutions receive such funds, these regulations, broadly speaking, became institutional rules, although institutions are allowed to have their own additional rules if they wish to impose a higher internal standard (28). The definition in the regulations (see Fig. 1) restricted the definition to FF&P; required intent to be taken into account; and stipulated that the allegation must be proven by a ‘‘preponderance of the evidence.’’ Relative quiet has descended on the community as institutions have successfully handled cases in an entirely routine fashion. The ORI now serves chiefly an educational and monitoring function, whereas the accused is now assured of the full protections of the law. 9

LESSONS FROM THE U.S. EXPERIENCE

Among the lessons to be gleaned from the turbulent experience in the United States are the following. Despite numerous very public scandals, and enormous publicity, building consensus on such an emotional topic is hard and takes many years. With notable exceptions, scientists are naturally loath to cede any of their authority, and good scientists often find it hard to imagine that anyone could break the bonds of trust that allow science to function. Unless pushed by those holding the purse strings, they will do little to police their own profession and tend to challenge, resist, and circumscribe attempts from outside to impose rules. They will tend to be suspicious of attempts to protect whistleblowers, who remain by far the most useful source for reporting misconduct and who often, perhaps usually, suffer for their efforts on behalf

6

CLINICAL TRIAL MISCONDUCT

I. Research § Misconduct Defined Research misconduct is defined as fabrication, falsification, or plagiarism in proposing, performing, or reviewing research, or in reporting research results. • Fabrication is making up data or results and recording or reporting them. • Falsification is manipulating research materials, equipment, or processes, or changing or omitting data or results such that the research is not accurately represented in the research record.‡ • Plagiarism is the appropriation of another person’s ideas, processes, results, or words without giving appropriate credit. • Research misconduct does not include honest error or differences of opinion. II. Findings of Research Misconduct A finding of research misconduct requires that: • There be a significant departure from accepted practices of the relevant research community; and • The misconduct be committed intentionally, or knowingly, or recklessly; and • The allegation be proven by a preponderance of evidence.
No rights, privileges, benefits, or obligations are created or abridged by issuance of this policy alone. The creation or abridgment of rights, privileges, benefits, or obligations, if any, shall occur only on implementation of this policy by the federal agencies. † Research, as used herein, includes all basic, applied, and demonstration research in all fields of science, engineering, and mathematics, which includes, but is not limited to, research in economics, education, linguistics, medicine, psychology, social sciences, statistics, and research involving human subjects or animals. ‡ The research record is the record of data or results that embody the facts resulting from scientific inquiry, and includes, but is not limited to, research proposals, laboratory records, both physical and electronic, progress reports, abstracts, theses, oral presentations, internal reports, and journal articles. § The term ‘‘research institutions’’ is defined to include all organizations using federal funds for research, including, for example, colleges and universities, intramural federal research laboratories, federally funded research and development centers, national user facilities, industrial laboratories, or other research institutes. Independent researchers and small research institutions are covered by this policy.

Figure 1. Federal Policy on Research Misconduct
,‡

of science. Scientists have learned that the same body must not be investigator, prosecutor, and judge. Lastly, although scientific misconduct has to be defined and assessed by scientists, the threats to the livelihood of the accused are so great that no solution will work unless it is seen to be fully in accord with the law, the workings of which they are often ill-informed.

10

OUTSIDE THE UNITED STATES

The most thoughtful national response to the issue of scientific misconduct has been made in Denmark (29, 30). What is particularly interesting is that those who organized this did so before Denmark, in sharp contradistinction to the United States and other countries, had any important cases or scandals with which to deal. Their effort was from the start aimed at prevention as well as at setting down procedures for responding to allegations. For a full description and analysis, the reader is referred to ‘‘Scientific Dishonesty & Good Scientific Practice’’ published by the Danish Medical Research Council in 1992 (29, 30).

In the United Kingdom, the situation still seems similar to that in the United States 20 years ago, few lessons having been learned from abroad. Certain granting agencies (31–33) have well-established rules—as in the United States, their authority based on the need to account for funds disbursed. But little enthusiasm has been expressed for any general response, despite energetic pushing from the editors of the BMJ and Lancet, who, exasperated, founded the Committee on Publication Ethics (COPE). Editors of general medical journals are quite frequently made aware of suspect research, and are then called on to investigate—something they ordinarily do not have the mandate, authority, money, legal standing, time, or expertise to do. In the United States, editors simply convey the allegation to the researcher’s institution and act as witnesses. In the United Kingdom, there may be no one and no system to investigate, adjudicate, and sanction. COPE, which has reported annually since 1998, serves as a sounding board, looks into ethical maters, and, where necessary and possible, refers to the appropriate authority. Although the editors have repeatedly called for a national

CLINICAL TRIAL MISCONDUCT

7

body, very little progress has been made (34, 35). Elsewhere, many other countries, which, like the United Kingdom and South Africa, have poorly defined procedures, are struggling to deal with the problem are sweeping it under the table. Germany has had to contend with long-running and widely publicized cases and is attempting to frame regulations (36). In China, Beijing University has issued broad regulations that include, for example, ‘‘intentionally exaggerating the academic value and economic and social results of a research finding’’ (37).

The team rapidly discovered serious protocol deviations, poor documentation, nonexistent patients, failure to obtain ethical approval for the study from the institution, and no evidence of informed consent. They concluded that Bezwoda’s study was invalid (43). Among the consequences of this behavior were disappointment for thousands of women and their physicians and loss of confidence in clinical research. It is unclear what part Bezwoda’s colleagues played, but the continued day-to-day involvement of colleagues in all aspects of this research would surely have prevented this misconduct.

11 SCIENTIFIC MISCONDUCT DURING CLINICAL TRIALS

12

Many of the early cases of scientific misconduct were perpetrated by physicians under contract to pharmaceutical companies to follow-up patients and to help with post-marketing surveillance. Wells and Brock have described several cases (38, 39). Pharmaceutical companies were left with a quandary. Their audits had revealed clear misconduct, but until the 1990s they did not wish to antagonize the physicianperpetrators or their colleagues by prosecuting (39). Now that the public recognizes that scientific misconduct exists, companies are backed up by the medical community, so their reluctance to pursue cases has diminished. As with misconduct in other settings, distinguished researchers in large medical centers may be found guilty (40). The difficulties of dealing with misconduct during a trial when no standard procedures exist is well illustrated by Hoeksema et al. (41). Of recent cases of misconduct within a clinical trial, the most notorious concerns the South African oncologist Dr. Werner Bezwoda of the University of Witwatersrand in Johannesburg. Bezwoda’s clinical trial was alone in reporting striking benefit from high-dose chemotherapy and peripheral blood stem cell rescue in women with high-risk breast cancer (42). His results, presented in 1999, were so at variance with three other trials reported of the same therapy that the National Cancer Institute sent an audit team to South Africa to investigate.

AUDIT

Audit is an inefficient and costly system for policing science and should not in general be used. Clinical trials are different. In 1978, well before there were standard methods for investigating misconduct, numerous researchers at Boston University Medical Center reported being pressured to falsify and fabricate data in clinical trials in oncology by their chief. He, on the other hand, alleged that the falsifications, later shown to involve 15% of the data, had been perpetrated by his juniors (7). He also maintained that ‘‘there are certain types of studies that are almost beyond the limits of absolute surveillance’’ as they are so complex (7). As a consequence of the wide publicity given this case, the National Cancer Institute mandated on-sight audits by their cooperative groups. Subsequently, in 1985, Shapiro and Charrow, examining the results of U.S. Food and Drug Administration (FDA) audits showed a high prevalence of bungled and fraudulent research (44). In 1993, Weiss et al. reported on their audit system for the Cancer and Leukemia Group B, and found a very low rate (45). However, the finding of one case of misconduct and one of ‘‘gross scientific error’’ was made under conditions where everyone had been put on notice that there would be regular audits; where one of those audited had received a suspended prison sentence for misconduct; where three large centers with adverse audit results had simply been dropped from the study, and where everyone involved had made a very strong commitment

8

CLINICAL TRIAL MISCONDUCT

to the expensive and time-consuming task of audit (46). It was a routine audit that revealed misconduct by one investigator, Dr. Roger Poisson of Montreal, in one of the most significant of trials, the National Surgical Adjuvant Breast and Bowel Project (NSABBP). The ORI, eventually alerted by the National Cancer Institute, discovered 115 well-documented cases of fabrication and falsification, for which Poisson took responsibility, as well as falsifications in other studies (47). This episode was particularly unfortunate because it took years for the audit results to become known to the journals that had published articles that had included Poisson’s data, and journals and public learned of it via the Chicago Tribune (47). The scientists who later asserted that Poisson’s falsifications did not materially affect the results of the trial, so that informing public and journals was unimportant, were missing an essential point. Research is paid for by the public, who are deeply interested in the results of important trials in breast cancer. The legitimacy of researchers and their trials depends on the trust of the public, which this incident, the fact that it was unearthed by a reporter, and the massive publicity it aroused, damaged severely. The result was doubly unfortunate, given that it was the trial’s own audit that had first revealed to its leaders that a problem existed. As Brock has pointed out, the knowledge that audit will be routine and thorough, and that misconduct will be reported and prosecuted is likely to have a deterrant effect (39). Steward et al. have suggested that peer reviewed journals consider publication of the results of trials only when a program of random audits exists (48). Experience has shown that in widely-flung enterprises like multicenter clinical trials, careful attention to audit, conducted by senior investigators involved with the trial, is necessary if trials are to be credible. 13

CAUSES

A great deal of unsupported opinion has been published on the causes. The usual favorite is that scientists are driven to misconduct

because of the ‘‘publish or perish’’ atmosphere of the research life. This facile explanation takes no account of the fact that no correlation, positive or negative, exists between publication and misconduct; that the vast majority of scientists are honest, whatever their publication rates; that the pressure to publish has been shown to be exaggerated; and that at least one prominent scientist found guilty of fabrication went out of his way to deny this was the cause (13). Brock concludes that the motive is often simple greed, which, associated with a desire for advancement, may be a powerful causative factor. We are far from understanding the psychological processes working here, as is evidenced by, for example, the case of Karen Ruggiero, whose extensive fabrications came as a complete shock to her psychologist superior colleague (49). It is to be hoped that the large amount of research stimulated by the ORI’s grants and research conferences will shed light on this topic. In the meantime, it seems likely that ignorance of scientific mores, poorly enunciated standards, poor or absent teaching, and inadequate individual supervision of young researchers are far more important. 14 PREVALENCE We do not know how commonly misconduct or misconduct in trials occurs. The publication of data from audit committees such as that by Weiss et al. (45) is useful, but such audits reflect only the experience of the trials with strict audit and the highest standards. Pharmaceutical companies seem to be increasing their audit of the performance of trials conducted in the community, but their combined experience has not been gathered and reported. The annual reports of the ORI (http://ori.hhs.gov) give some indication of the number of cases of misconduct, not confined to clinical trials, reported to the ORI and their disposition. In 2002, the following types of allegations were reported by institutions in the United States: fabrication: 45; falsification: 58; plagiarism: 27; ‘‘other’’: 33. During that year, there were 67 initial inquiries and 31 full investigations (50). No data exist to show that misconduct is more prevalent in one country than another.

CLINICAL TRIAL MISCONDUCT

It is simplest to assume that clinical scientists everywhere lie on a curve reaching from the obsessively honest through to the serially dishonest, and that as scientific or medical degrees do not come with any guarantee of probity, there will always be those who commit and report fraudulent research. The idea of an experimental, short-term, confidential audit, to be reported only in aggregate, in order to establish the prevalence of gross fabrication was first proposed in 1988 (51). But when the ORI recently funded a survey of misconduct and other forms of poor behavior, there were immediate and forceful protests from the same scientific bodies that had fought so hard to limit the definition of misconduct to FF&P (52, 53). There were legitimate concerns about the scientific quality of the projected survey. But in an editorial, the journal Nature described the defensiveness of the actions of the large scientific societies as ‘‘heads-in-the-sand,’’ and a ‘‘good impersonation of . . . . special interests with something to hide’’ (54). Whether this is true or not, given these official attitudes, it will be a long time before we have any clear idea of the prevalence of misconduct. 15

PEER REVIEW AND MISCONDUCT

A few cases of plagiarism of ideas and words have occurred during peer review. Given the privileged and confidential nature of manuscripts sent to reviewers, the possibility of plagiarism or breaking confidentiality is a particularly disturbing one, and one that editors must do their best to prevent by reminding reviewers of their responsibilities. Editors have a duty to report breaches of confidentiality and act as witnesses to investigatory panels. For a fuller account, the reader is referred to Reference 55. Peer review operates on the assumption that the authors are telling the truth, and is a very poor detector of fraudulent research, unless, as sometimes happens, the reviewer sees his or her own words plagiarized. 16

RETRACTIONS

The case of Poisson mentioned above, and a great many other serious cases, have amply

9

demonstrated that a single incident of misconduct should immediately mean that the validity of all the guilty scientist’s other research cannot be assumed, and all of it must be scrutinized carefully. The University of California San Diego (UCSD) set the standard after the discovery, in 1985, of duplicate data in two studies published by Robert A. Slutsky. Over the next year, investigating panels held Slutsky’s co-authors responsible for the validity of every part of every one of 137 articles published in 7 years (56). Overall, 77, including reviews, were judged valid, 48 were ‘‘questionable’’ (and therefore unciteable), and 12 were deemed at that time ‘‘fraudulent’’ (this event occurred before federal definitions had been promulgated). Nevertheless, most of the journals asked to retract the articles refused to do so, with the result that Slutsky’s fraudulent articles were being cited as if correct years later (57). Moreover, 2 years after a high-profile inquiry in Germany looking into the work of Friedhelm Herrmann and Marion Brach found 94 published papers to include manipulated data, 14 of the 29 journals publishing the articles had not published retraction notices concerning any of the articles (58). Journal editors tend to be cowed by threats either from the guilty scientist or from the innocent co-authors, each of whom has an interest in preventing the appearance of a retraction. The editor of Nature has written feelingly of the problems of retracting seven papers all sharing the first author, Jan Hendrik Sch¨on, who maintained throughout that his work should stand unless faced with hard evidence to the contrary (59). Despite this fact, it is everyone’s duty to correct the literature, and an increased resolve on the parts of editors, backed up by strong policies from editorial societies is in evidence. The International Committee of Medical Journal Editors policy reads: ‘‘The retraction or expression of concern, so labeled, should appear on a numbered page in a prominent section of the journal, be listed in the contents page, and include in its heading the title of the original article. It should not simply be a letter to the editor. Ideally, the first author should be the same in the retraction as in the article, although under certain

10

CLINICAL TRIAL MISCONDUCT

circumstances the editor may accept retractions by other responsible people. The text of the retraction should explain why the article is being retracted and include a bibliographic reference to it’’ (60). It is the duty of research institutions and a guilty scientist’s co-authors to check the scientist’s entire published work. It is also their duty, and that of the relevant journals, to issue retractions when the reports are found to contain fraudulent work (61, 62). The difficulties of doing this are shown in the case of Poehlman, who was sentenced to serve a year in prison for his misconduct (62). These difficulties are much greater when the work has originated in a country with no process for dealing with misconduct. 17

PREVENTION

From the start, it was recognized that many cases of misconduct were associated with apparent ignorance on the part of young researchers of the mores and standards of good research, and others with a complete breakdown in the assumed mentor-trainee relationship, resulting in ineffective supervision, monitoring, or training of young researchers. Add to that the fact that clinical trials demand rigorous adherence to sometimes elaborate protocols on the part of many individuals, some of whom, although quite senior, may be new to research, and many of whom are from different cultural backgrounds. The responsibility on trial leaders to educate, train, and monitor their colleagues is therefore considerable, but unavoidable. Every effort should be made not to put inordinate pressure on people such as clinical coordinators to recruit patients (63) or to distort the whole process with excessive monetary incentives. Audit and putting people on notice about the consequences of misconduct are necessary, but building up close relationships within a well-trained team may well turn out to be more important. REFERENCES 1. I. Chalmers, Unbiased, relevant, and reliable assessments in health care. BMJ 1998; 317: 1167–1168.

2. B. M. Psaty and D. Rennie, Stopping medical research to save money: a broken pact with researchers and patients. JAMA 2003; 289: 2128–2131. 3. D. Rennie, Fair conduct and fair reporting of clinical trials. JAMA 1999; 282: 1766–1768. 4. I. Chalmers, Underreporting research is scientific misconduct. JAMA 1990; 263: 1405–1408. 5. D. Rennie, Dealing with research misconduct in the United Kingdom. An American perspective on research integrity. BMJ 1998; 316(7146): 1726–1728. 6. D. Rennie and C. K. Gunsalus, Scientific misconduct. New definition, procedures, and office—perhaps a new leaf. JAMA 1993; 269(7): 915–917. 7. W. Broad and N. Wade, Betrayers of the Truth—Fraud and Deceit in the Halls of Science. New York: Simon & Schuster, 1982. 8. M. C. LaFollette, Stealing Into Print—Fraud, Plagiarism, and Misconduct in Scientific Publishing. Berkeley, CA: University of California Press, 1992. 9. S. Lock and F. Wells (eds.), Fraud and Misconduct in Biomedical Research. 1st ed. London: BMJ Publishing Group, 1993. 10. S. Lock and F. Wells (eds.), Fraud and Misconduct in Biomedical Research. 2nd ed. London: BMJ Publishing Group, 1996. 11. S. Lock, F. Wells, and M. Farthing (eds.), Fraud and Misconduct in Biomedical Research. 3rd ed. London: BMJ Publishing Group, 2001. 12. A. Kohn, False Prophets. Fraud and Error in Science and Medicine. Oxford: Basil Blackwell Ltd., 1986. 13. Fraud in Biomedical Research. In: Hearings Before the Subcommittee on Investigations and Oversight of the Committee on Science and Technology, 1981. 14. Fraud in NIH Grant Programs. In: Hearings Before the Subcommittee on Energy and Commerce, 1988. 15. D. Rennie and C. K. Gunsalus, Regulations on scientific misconduct: lessons from the US experience. In: S. Lock, F. Wells, and M. Farthing (eds.), Scientific Fraud and Misconduct. 3rd ed. BMJ Publishing Group, 2001, pp. 13–31. 16. U.S. Department of Health and Human Services, Public Health Service, Responsibilities of awardee and applicant institutions for dealing with and reporting possible misconduct

CLINICAL TRIAL MISCONDUCT in science: final rule. Fed. Reg. 1989; 54: 32446–32451. 17. New York Times. July 22, 1975. 18. B. Mishkin, The Investigation of scientific misconduct: some observations and suggestions. New Biologist 1991; 3: 821–823. 19. Joint Consensus Conference on Misconduct in Biomedical Research. In: Royal College of Physicians of Edinburgh, 1999. 20. P. E. Kalb and K. G. Koehler, Legal issues in scientific research. JAMA 2002; 287: 85–91. 21. M. M. Mello and T. A. Brennan, Due process in investigations of research misconduct. N. Engl. J. Med. 2003; 349: 1280–1286. 22. CHPS Consulting, Analysis of Institutional Policies for Responding to Allegations of Scientific Misconduct. Rockville, MD: Office of Research Integrity. CHPS Consulting, 2000. 23. Integrity and Misconduct in Research. Report of the Commission on Research Integrity to the Secretary of Health and Human Services, the House Committee on Commerce and the Senate Committee on Labor and Human Resources.(* the Ryan Commission). (1995). (online). Available: http://gopher.faseb.org/opar/cri.html;. 24. R. A. Bradshaw, Letter to Secretary of Health and Human Services Donna Shalala. January 4, 1996. 25. B. Goodman, Scientists are split over finding of Research Integrity Commission. The Scientist 1996;Sect. 1. 26. Responsible Science—Ensuring the Integrity of the Research Process, vol. I. Washington, DC: National Academy Press, 1992. 27. Responsible Science—Ensuring the Integrity of the Research Process, vol. II. Washington, DC: National Academy Press, 1993. 28. Office of Science and Technology Policy, Federal policy on research misconduct. Fed. Reg. 2000; 76260–76264. 29. D. Andersen, L. Attrup, N. Axelsen, and P. Riis, Scientific Dishonesty & Good Scientific Practice. Copenhagen: Danish Medical Research Council, 1992. 30. P. Riis, The concept of scientific dishonesty: ethics, value systems, and research. In: S. Lock, F. Wells, and M. Farthing (eds.), Fraud and Misconduct in Biomedical Research. 3rd ed. London: BMJ Publishing Group, 2001, p. 268. 31. The Wellcome Trust. (2002). Guidelines on Good Research Practice. (online). Available: http://www.wellcome.ac.uk/en/1/ awtvispolgrpgid.html.

11

32. R. Koenig, Wellcome rules widen the net. Science 2001; 293: 1411–1413. 33. I. Evans, Conduct unbecoming— the MRC’s approach. BMJ 1998; 316: 1728–1729. 34. R. Smith, The need for a national body for research misconduct—nothing less will reassure the public. BMJ 1998; 316: 1686–1687. 35. M. Farthing, R. Horton, and R. Smith, UK’s failure to act on research misconduct. Lancet 2000; 356: 2030. 36. A. Bostanci, Germany gets in step with scientific misconduct rules. Science 2002; 296: 1778. 37. D. Yimin, Beijing U. issues first-ever rules. Science 2002; 296: 448–449. 38. F. Wells, Counteracting research misconduct: a decade of British pharmaceutical industry action. In: S. Lock, F. Wells, and M. Farthing (eds.), Fraud and Misconduct in Biomedical Research. 3rd ed. London: BMJ Publishing Group, 2001, pp. 64–86. 39. P. Brock, A pharmaceutical company’s approach to the threat of research fraud. In: S. Lock, F. Wells, and M. Farthing (eds.), Fraud and Misconduct in Biomedical Research. 3rd ed. London: BMJ Publishing Group, 2001, pp. 89–104. 40. J. H. Tanne, FDA limits research of former AHA president for submitting false information. BMJ 2002; 325: 1377. 41. H. L. Hoeksema et al., Fraud in a pharmaceutical trial. Lancet 2000; 356: 1773. 42. R. Horton, After Bezwoda. Lancet 2000; 355: 942–943. 43. R. B. Weiss et al., High-dose chemotherapy for high-risk primary breast cancer: an on-site review of the Bezwoda study. Lancet 2000; 355: 999–1003. 44. M. F. Shapiro and R. P. Charrow, Scientific misconduct in investigational drug trials. N. Engl. J. Med. 1985; 312: 732–736. 45. R. B. Weiss et al., A successful system of scientific data audits for clinical trials. JAMA 1993; 270: 459–464. 46. D. Rennie, Accountability, audit, and reverence for the publication process. JAMA 1993; 270(4): 495–496. 47. D. Rennie, Breast cancer: how to mishandle misconduct. JAMA 1994; 271(15): 1205–1207. 48. W. P. Steward, K. Vantongelen, J. Verweij, D. Thomas, and A. T. Van Oosterom, Chemotherapy administration and data collection in an EORTC collaborative group—can we trust the results. Eur. J. Cancer 1993; 29A: 943–947.

12

CLINICAL TRIAL MISCONDUCT

49. C. Holden, Psychologist made up sex bias results. Science 2001; 294: 2457. 50. Report on 2002 Institutional Annual Report on Possible Research Misconduct. Washington, DC: Office of Research Integrity, August 2003. 51. D. Rennie (ed.), Mark, Dupe, Patsy, Accessory, Weasel, Flatfoot. In: Ethics and Policy in Scientific Publication. Bethesda, MD: Council of Biology Editors, Inc., 1990, pp. 155–174. 52. C. Holden, Planned misconduct surveys meet stiff resistance. Science 2002; 298: 1549. 53. D. S. Greenberg, Misconduct poll prompts fury among scientists. Lancet 2002; 360: 1669. 54. Soft responses to misconduct. Nature 2002; 240: 253. 55. D. Rennie, Misconduct and journal peer review. In: F. Godlee and T. Jefferson (eds.), Peer Review in Health Sciences. London: BMJ Books, 1999, pp. 90–99. 56. R. L. Engler et al., Misrepresentation and responsibility in medical research. N. Engl. J. Med. 1987; 317: 1383–1389. 57. W. P. Whitely, D. Rennie, and A. W. Hafner, The scientific community’s response to evidence of fraudulent publication. The Robert Slutsky case. JAMA 1994; 272(2): 170–173. 58. A. Abbott and J. Schwarz, Dubious data remain in print two years after misconduct inquiry. Nature 2002; 418: 113. 59. Retractions’ realities. Nature 2003; 422: 1. 60. International Committee of Medical Journal Editors. (2001). Uniform requirements for manuscripts submitted to biomedical journals. (online). Available: http://www.icmje.org/index.html#top. 61. E. Panel Marshall, Extensive Sudbø Fraud. Science 2006; 313: 29. 62. H. C. Sox, D. Rennie. Research Misconduct, Retraction and Cleansing the Medical Literature: Lessons from the Poehlman Case. Ann Intern Med. 2006; 144: 609–613. 63. P. A. Cola, Follow up to scientific misconduct. Clin. Researcher 2002; 2: 26–27.

CLINICAL TRIALS, EARLY CANCER AND HEART DISEASE

materials were effective and not overly toxic, ultimately lead to clinical trials in humans. By 1960, there was an annual screening of approximately 25 000–30 000 materials sponsored by NCI with only about 10—20 new agents having sufficient effectiveness in animal systems to merit consideration for testing in humans. Peter Armitage, of the London School of Hygiene and Tropical Medicine, was a visiting scientist at NCI in the late 1950s. His background in sequential statistical procedures quickly found direct application in the development of two- and three-stage screening procedures for animal tumor systems that permitted rejection of an agent at any stage but acceptance only at the final stage (3,43). The object was to determine quickly which new compounds should be considered for further study in man. In the late 1950s, optimism was high that this screening program would lead to a new chemotherapeutic treatment that would make large clinical trials unnecessary. Also, there was a belief that different forms of cancer were sufficiently similar so that an agent active in one form of the disease would also be active in another.

MARVIN A. SCHNEIDERMAN∗ EDMUND A. GEHAN Georgetown University Medical Center, Washington, DC, USA

Early developments in controlled clinical trials at the National Institutes of Health (NIH) took place mainly at the National Cancer Institute (NCI) and what was then the National Heart Institute (NHI) [subsequently the National Heart, Lung, and Blood Institute (NHLBI)] beginning in the 1950s. This article reviews the developments from the early 1950s to the late 1960s at both institutes, summarizing the early efforts in clinical trials, the organizations set up to conduct and monitor the clinical trials, and the developments in statistical methodology that have formed the basis for conducting many of the present day randomized controlled trials. The early history of clinical trials at these institutes has been reviewed in more detail at NCIbyGehan & Schneiderman and at NHLBI by Halperin et al. (28,32).

1.1 Early Efforts in Clinical Trials

1 DEVELOPMENTS IN CLINICAL TRIALS AT THE NATIONAL CANCER INSTITUTE (NCI)

Dr C. Gordon Zubrod came to NCI in 1954 at about the time that Dr James Holland departed for Roswell Park Memorial Institute in Buffalo, NY. Drs Emil Frei and E.J. Freireich arrived at NCI in 1955. Under the leadership of Zubrod, this formed the key group of clinicians who initiated the clinical trials program at NCI. When Zubrod was at Johns Hopkins University in the early 1950s, he indicated that there ‘‘were two streams of influence (relating to developments in clinical trials)—infectious disease chemotherapy and comparative studies of analgesics and hypnotic drugs’’ (52). Among those playing an important role in the conduct of clinical trials at Johns Hopkins were Dr James Shannon (later Director of the National Institutes of Health), the pharmacologist E.K. Marshall, Jr and W.G. Cochran. About this time, the studies of streptomycin in pulmonary tuberculosis by the Medical Research Council were

A major advance in the development of chemical agents for the treatment of cancer came from observations of the treatment of children with acute lymphocytic leukemia, which was a rapidly fatal disease until 1948 when Sidney Farber, in a nonrandomized study of methotrexate, observed complete remissions and longer survival among some pediatric patients (21). However, results did not meet with uniform acceptance and questions were raised about diagnosis, selection of patients, and reporting. There was a need for a more organized approach to treatment experimentation that would lead to unbiased evaluations of treatments. At about the same time, animal models of the major forms of cancer— sarcomas, carcinomas, and leukemias—were developed that could be used to screen candidate materials and, if the

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

CLINICAL TRIALS, EARLY CANCER AND HEART DISEASE

published and had a profound influence on the Johns Hopkins group (41). The first effort at a randomized trial was a comparison of the efficacy of tetracycline and penicillin in the treatment of lobar pneumonia (5). At the same time, the Veterans Administration began its first randomized controlled trials in tuberculosis (50). 1.2 The Organization of Trials In 1954, the US Congress created the Cancer Chemotherapy National Service Center (CCNSC) to stimulate research in the chemotherapy of cancer. A clinical panel was formed, headed by Dr I. Ravdin, and included among others Drs Zubrod and Holland. At an early meeting, the clinical panel reviewed a paper by Louis Lasagna, which enunciated five principles of the controlled clinical trial, including randomization and the statistical treatment of data (38). Over the next several years, the clinical panel of the CCNSC oversaw the organization of cooperative clinical trials groups for the conduct of clinical trials in cancer. By 1960, there were 11 cooperative clinical study groups (Table 1), each comprised of a number of universities and/or V.A. Hospitals and Medical Centers and a Statistical Coordinating Center (48). The cooperative groups were funded by the NCI through the Chairman and a Statistical Center. Zubrod recruited the chairman of each group and Marvin Schneiderman recruited the biostatisticians and statistical centers. One of the statisticians, W.J. Dixon, had two graduate students who were writing general statistical programs for the analysis of biomedical data. NCI awarded a contract to carry out this work that subsequently became the Biomedical Data Processing Program (BMDP) package of statistical programs. In the establishment of a clinical cooperative group, CCNSC agreed that there should be adherence to the following principles: combination of data from all institutions to accumulate rapidly the necessary number of patients; standard criteria of diagnosis, treatment, and measurement of effect; statistical design of the study, with a randomized assignment of patients to the groups to be compared; and statistical analysis and collaborative reporting of the results.

The clinical trials effort involved more types of clinical studies than randomized trials. There was a sequence of trials with differing objectives: Phase I—to determine the maximum tolerated dose of a regimen that can be used in looking for therapeutic effect; Phase II—to determine whether a particular dosage schedule of an agent is active enough to warrant further study; and Phase III—a comparative trial, usually randomized, to decide whether a new therapy is superior to a standard therapy. The primary objective of the clinical trials program was to provide a means of testing in humans new agents that had previously demonstrated effectiveness in animal tumor systems. 1.3 Some Early Trials Following some preliminary discussions between Dr Zubrod and Jerome Cornfield, a leading statistician at NIH, there was agreement that childhood leukemia was an ideal disease for testing some of the new agents objectively. The first randomized cooperative clinical trial in acute leukemia was planned in 1954, begun in 1955, and reported by Frei et al. in 1958 (23). The trial involved two regimens of combination chemotherapy—6-mercaptopurine and either intermittent or continuous methotrexate in 65 patients. The study had the following features: a uniform protocol at the four participating institutions uniform criteria of response adherence to the principles of the controlled clinical trial, especially the randomization of patients to therapies; and stratification of patients by age, type of leukemia, and history of prior therapy. Statistical methods used were a comparison of median survival times and median duration of remissions between therapies, confidence intervals, and Fisher’s exact test. The first randomized clinical trial in solid tumors was conducted by members of the Eastern Solid Tumor Group and reported by Zubrod et al. in 1960 (53). The trial involved a randomized comparison of two alkylating agents (thiotepa vs. nitrogen mustard) in patients with solid tumors. One objective was to ‘‘study the feasibility and usefulness of collaborative clinical research in cancer

CLINICAL TRIALS, EARLY CANCER AND HEART DISEASE

3

Table 1. Cooperative Clinical Study Groups in 1960 Group

Chairman Statistician

Acute leukemia, Group A Acute leukemia, Group B Eastern Solid Tumor Group Southeastern Group Western Group Southwestern Group Prostate Group Breast Group A Breast Group B V.A. Groups—various malignancies University Groups—lung, breast, stomach, ovary, colon

M. Lois Murphy E. Frei C.G. Zubrod R.W. Rundles F. Willett H.G. Taylor H. Brendler A. Segaloff G. Gordon J. Wolf et al. A. Curreri et al.

chemotherapy’’. The trial involved 258 randomized patients, and notable features were: blind evaluation of response by vote of clinical investigators objective procedures for measurement of tumors and determination of when a response began and ended; the importance of accounting for type I and type II statistical errors appropriate sample size for detection of differences between treatments and statistical analysis in the reporting of results. A subsequent trial demonstrated the value of combination chemotherapy in acute leukemia and the independent action of drugs to increase the probability that a patient achieves complete remission (24). Freireich et al. (25) reported a prospective, randomized, double-blind, placebo-controlled, sequential study of 6-mp vs. placebo in the maintenance of remissions in pediatric acute leukemia. This study established that 6-mp maintenance treatment leads to substantially longer remissions than placebo and was a forerunner to many adjuvant studies in other forms of cancer, such as breast cancer, in which treatments are administered when the patients are in a disease-free state (25). This study also was a motivation for the development of an extension of the Wilcoxon test for comparing survival distributions subject to censoring (27) and was used as an example by Cox in his, now classic, paper on regression models and life tables (16).

I. Bross M. Schneiderman M. Schneiderman B.G. Greenberg E. MacDonald D. Mainland M. Schneiderman M. Schneiderman M. Patno R. Stiver, G. Beebe, W. Dixon

1.4 Developments in Methodology In the clinical trials program at NCI prior to 1970, there were several developments in methodology that have influenced the conduct of subsequent clinical trials. Before 1960, the clinical testing of new agents often involved as few as five patients, with the agent discarded if no positive response was obtained in at least one patient. In 1961, Gehan proposed a plan for Phase II trials that determined the minimum number of consecutive patients to study when all patients are nonresponders, before one could reject a new agent for further study, at given levels of rejection error (26). This plan, or now more commonly Simon’s modification, continues in use today in Phase II studies (46). Several philosophical issues arose from the drug development program. The practice of human experimentation could be questioned by ‘‘Doesn’t a physician have an implied duty to give his patient the best treatment? If that is the case, how can one justify having the toss of coin (i.e. randomization) decide which treatment a patient should receive?’’ The reply was (and is), ‘‘If the physician really knows what is the best treatment for the patient, the patient must receive that treatment and not be randomized into a trial.’’ The question then becomes, ‘‘How and when does a physician know what is the best treatment for a specific patient?’’ The major ethical issue then becomes one of learning quickly (i.e. with a minimum number of patients) what is the best treatment. There have been several proposals for

4

CLINICAL TRIALS, EARLY CANCER AND HEART DISEASE

establishing what one ‘‘knows’’ while minimizing the number of patients who will receive the less effective treatment. Armitage proposed closed sequential procedures with paired patients on each treatment, and with the trial terminated as soon as one could establish the superiority of one of the treatments over the other (2). A feature of the plans was an upper limit on the number of patients one could enter. Schneiderman & Armitage later described a family of sequential procedures, called wedge plans because of the shape of the acceptance boundary, which provided a bridge between the open plans derived from Wald’s theory and the restricted procedures of Armitage (44,45). In the 6-mp vs. placebo study for maintaining remissions in pediatric leukemia, patients were paired according to remission status (complete or partial), one patient receiving 6-mp and the other placebo by a random allocation, and a preference was recorded for 6-mp or placebo depending upon the therapy which resulted in the longer remission. The trial was conducted sequentially according to one of the Armitage plans (2) and a sequential boundary favoring 6-mp was reached after 18 preferences had occurred— 15 for 6-mp and 3 for placebo. There were 12 patients still in remission at the time the study was terminated, although one could record a preference for one or the other treatment because the pair-mate had relapsed at an earlier time. It was clear that a more efficient analysis could be obtained by using the actual lengths of remission. Gehan, while working on an NCI fellowship with D.R. Cox at Birkbeck College in London, developed a generalization of the Wilcoxon test for the fixed sample size problem with each sample subject to arbitrary right censoring (27). Halperin had previously developed a generalization of the Wilcoxon test, when all times to censoring were equal to the longest observation time (30). Mantel noticed that one could utilize the chi-square test for comparison of survival data between two or more groups, assuming that one constructs a contingency table of deaths and survivors at each distinct failure time in the groups of patients under study. This chi-square test was appropriate when the risk of failure in one group was a constant multiple of that in the other; this

test was an extension of the earlier test developed by Mantel and Haenszel which measured the statistical significance of an observed association between a disease and a factor under study in terms of an increased relative risk of disease (39,40). This test subsequently became known variously as the Mantel–Haenszel test, the logrank test or the Cox–Mantel test, and has been studied by Cox and Peto, among others (16,42). Another development in the 1960s was the exponential regression model proposed by Feigl & Zelen (22). Dr Robert Levin of NCI was interested in studying the relationship of the survival time of leukemia patients to the concomitant variate of white blood count, separately according to the presence or absence of auer rods and/or significant granulature of leukemia cells in the bone marrow at diagnosis. Feigl & Zelen proposed a model in which an exponential survival distribution is postulated for each patient and the expected value of the survival time is linearly related to the patient’s white blood count. A more general loglinear model was subsequently given by Glasser (29), and there have been numerous subsequent developments in parametric regression models with censored survival data 17, Chapters 5 and 6, pp. 62–90. 2 DEVELOPMENTS IN CLINICAL TRIALS AT THE NATIONAL HEART, LUNG, AND BLOOD INSTITUTE (NHLBI) Prior to 1960, the National Heart Institute (NHI), subsequently to become NHLBI, had little involvement in multicenter clinical trials. In a trial designed in 1951, there was a comparison of ACTH, cortisone, and aspirin in the treatment of rheumatic fever and the prevention of rheumatic heart disease. A total of 497 children were enrolled in 12 centers in the UK, the US, and Canada. Felix Moore, then Chief of the Biometrics Section of NHI, was a statistical consultant. There were no differences in treatment effectiveness in the study, and no statistical or methodologic problems mentioned in the final report (8). Subsequently, there was a multicenter observational study of lipoproteins in

CLINICAL TRIALS, EARLY CANCER AND HEART DISEASE

atherosclerosis that had substantial impact on the methodology for coordinating studies performed at several sites (47). The Statistical Center was led by Felix Moore and Tavia Gordon at NHI. Careful quality control procedures and standardization of methods across centers were emphasized. 2.1 Early Efforts in Clinical Trials Jerome Cornfield joined the NHI in 1960 and strongly influenced the conduct of clinical trials at NHI and statistical research on methodologic issues arising in clinical trials. In the early 1960s, intensive planning for two clinical trials was begun at NHI to reduce risk factors for coronary heart disease—The Diet Heart Feasibility Study (DHFS) and the Coronary Drug Project (CDP) (14,20). These studies reflected the strong interest in both dietary and drug approaches to the prevention of coronary heart disease and the recurrence of myocardial infarction. For the DHFS, the NHI Biometrics Branch served as the statistical coordinating center, first under the supervision of Joseph Schachter and later of Fred Ederer. Max Halperin rejoined the NHI in 1966 and, upon Cornfield’s retirement in 1968, became Chief of the Biometrics Research Branch until his retirement in 1977. Four areas of clinical trials and methodology can be traced to these early studies and the individuals responsible for them. These areas are: organizational structure for clinical trials at NIH; methodology for the interim analysis of accumulating data, including the Bayesian approach, group sequential and stochastic curtailment methods; design and analysis of clinical trials, including the effects of patient noncompliance on power and the intention to treat principle; and methods for analysis of data from longitudinal clinical trials. 2.2 The Organization of NHLBI Trials The ‘‘NHLBI Model’’ for cooperative clinical trials evolved from discussion during the planning stage of the CDP among outside medical experts and NHI medical and statistical staff. In 1967, a report by a committee appointed by the National Advisory Heart Council and chaired by Bernard Greenberg

5

described this structure (35). The report, subsequently known as the ‘‘Greenberg Report’’, became the basis for a structure of nearly all subsequent NHLBI trials as well as for many other trials sponsored at NIH. The major components of the organizational structure include a Steering Committee, a Policy Advisory Board, a Data Monitoring Committee, and a Statistical or Data Coordinating Center, as well as individual clinics, central laboratories, and various other committees which served the needs of the trial. These might include committees to develop eligibility criteria, to assign cause of death, to define methodology and standards, or to oversee the preparation of manuscripts (for more details of organizational structure). From the biostatistical viewpoint, the Data Monitoring Committee has the responsibility of monitoring accumulating data on a periodic basis and analyzing results for evidence of early benefit or harm. Primary and secondary outcomes measures are reviewed, along with safety data, compliance to the protocol, and subgroup analyses which may identify particular risk groups. The Statistical Coordinating Center and the Data Monitoring Committee work closely together in performing the appropriate data analyses needed for fulfilling the Committee’s responsibilities. The Statistical and Data Coordinating Centers for early trials at the NHLBI are given in Table 2. Personnel at these coordinating centers have played an important role in the development of clinical trials and made numerous contributions to statistical methodology. 2.3 Developments in Methodology These are considered under three headings: data monitoring, design and analysis, and longitudinal studies. 2.3.1 Data Monitoring. Jerome Cornfield was involved in the planning and conduct of two clinical trials—the DHFS and the CDP. Both Cornfield and Halperin served on the Data and Safety Monitoring Committee of the CDP. At least partly motivated by his involvement in these trials, Cornfield

6

CLINICAL TRIALS, EARLY CANCER AND HEART DISEASE Table 2. Early NHLBI Coordinating Centers. University of Maryland/Maryland Research Institute Coronary Drug Project University of Texas School of Public Health Hypertension Detection and Follow-up Program University of North Carolina—Chapel Hill, School of Public Health Lipid Research Clinical Program University of Minnesota School of Public Health, Biometry Division Multiple Risk Factor Intervention Trial University of Washington School of Public Health, Biostatistics Department Coronary Artery Surgery Study George Washington University Biostatistics Center Intermittent Positive Pressure Breathing Trial NHLBI Biometrics Research Branch National Diet Heart Feasibility Study Urokinase Pulmonary Embolism Trial Urokinase Streptokinase Pulmonary Embolism Trial

published papers in 1966 on sequential trials, sequential analysis, and the likelihood principle, from a Bayesian perspective (9,10). In 1966, Max Halperin worked jointly with Cornfield and Samuel Greenhouse (then at the National Institute of Mental Health) to develop an adaptive allocation procedure that would assign an increasing proportion of patients to the better of two treatments as evidence accumulated (13). Their approach to the problem was Bayesian and generalized the earlier work of Anscombe and Colton (1,7). At around the same time, Cornfield published a general paper on the Bayesian approach that involved the use of a prior probability distribution with a mass of probability P at the null hypothesis, with a continuous density of total mass 1 − P over a set of alternative hypotheses (11). A key feature of Cornfield’s proposal was the rejection of the null hypothesis when the posterior odds (the relative betting odds or RBO) became small for H0 . The RBO was used in the CDP in the monitoring of mortality differences between the control and each of the drug treatment groups. Subsequently, Canner, of the CDP Coordinating Center, considered the determination of critical values for decision making at multiple time points during the conduct of the clinical trial from the Neyman–Pearson perspective (6). Later, curtailment and stochastic curtailment methods were developed and applied to trials of the NHLBI in the 1970s and early 1980s (19,31,34,37).

Statisticians working with the CDP were aware that, as the data accumulated, repeated testing for treatment differences using conventional statistical significance levels would increase the type I error over the nominal alpha level associated with that critical value. Armitage et al. evaluated the impact of repeated testing on the type I error and demonstrated that multiple tests could increase the type I error substantially (4). Interim analyses of clinical data are necessary for scientific and ethical reasons, but large type I errors are not acceptable. Canner developed a method for the CDP for determining the critical value at each interim analysis so that the overall type I error is close to the desired level (6). Statisticians involved with NHLBI trials developed group sequential methods and applied them to trials starting with the CDP. 2.3.2 Design and Analysis. In the DHFS, it was projected that a reduction in cardiovascular risk would result from a reduction in cholesterol level. The original sample size projection was for the entry of 8000 patients into several treatment arms. Although a special review committee suggested that this sample size might be too large, Cornfield argued that there were too many inconclusive small studies already in the literature. Several aspects of the trial required consideration, including noncompliance with the treatment regimen. It was presumed that the maximum effect on risk would occur only

CLINICAL TRIALS, EARLY CANCER AND HEART DISEASE

after some period of time on treatment and that failure to adhere to the treatment regimen could mean a return to higher risk levels. Halperin et al. (33) incorporated these considerations into the design of clinical trials by proposing methods for adjusting sample size for noncompliance in the treatment group. Studies were considered with a fixed period of observation and a comparison of proportions as the main analysis. Implicit in this paper is the ‘‘intention to treat’’ principle, i.e. analysis of all randomized patients in their assigned treatment group regardless of compliance. Ultimately, the report of the CDP recognized this point (15). Most primary and secondary prevention trials conducted by the NHLBI since 1970 have made use of sample size adjustments for noncompliance. The Framingham Heart Study was begun in 1948 and has had an important influence on methodologic research at the NHLBI and the design of prevention trials. Over 5000 adult residents of Framingham, Massachusetts, were entered into a longitudinal study with the objective of evaluating the effects of various risk factors on the development of subsequent cardiovascular disease. The study has clarified the roles of high blood pressure, elevated total serum cholesterol, and cigarette smoking on the risk of cardiovascular disease (18,36). Occurrence or not of a cardiovascular event in a 2-year follow-up period is a binary outcome. Cornfield considered a regression approach to deal with the binary outcome variables. The problem was closely related to the discrimination problem between two samples from multivariate normal distributions. For a specific prior probability of belonging or not to a disease group, the posterior probability could be represented as a logistic regression function that was closely related to what could be obtained from a conventional discriminant function analysis (49). Cornfield & Mitchell argued that one could use the logistic model to predict the impact on risk of specified changes in risk factors (12). Subsequently, this logistic model approach was used in the design of several NHLBI prevention trials. 2.3.3 Longitudinal Studies. A methodology for analysis of longitudinal data was needed for the Framingham Study which

7

could be considered both a cohort and a longitudinal study. Cohorts of individuals were followed to observe patterns of morbidity and mortality, and biennial measurements of cardiovascular risk factors provided an opportunity to study patterns relating to aging. Early reports of the Framingham study used simple graphical and descriptive methods to describe patterns of aging. During the 1980s, there was much work on methodology for longitudinal studies, Overview that ultimately led to NHLBI sponsorship of a workshop on methods for analysis of longitudinal and follow-up studies, whose proceedings have appeared as a special issue in Statistics in Medicine (51). REFERENCES 1. Anscombe, F. J. (1963). Sequential medical trials, Journal of the American Statistical Association 58, 365–383. 2. Armitage, P. (1957). Restricted sequential procedures, Biometrika 44, 9–26. 3. Armitage, P. & Schneiderman, M. (1958). Statistical problems in a mass screening program, Annals of the New York Academy of Science 76, 896–908. 4. Armitage, P., McPherson, C. K. & Rowe, B. C. (1969). Repeated significance tests on accumulating data, Journal of the Royal Statistical Society, Series A 132, 235–244. 5. Austrian, R., Mirick, G., Rogers, D., Sessoms, S. M., Tumulty, P. A., Vickers, W. H., Jr. & Zubrod, C. G. (1951). The efficacy of modified oral penicillin therapy of pneumococcal lobar pneumonia, Bulletin of Johns Hopkins Hospital 88, 264–269. 6. Canner, P. L. (1977). Monitoring treatment differences in long-term clinical trials, Biometrics 33, 603–615. 7. Colton, T. (1963). A model for selecting one of two medical treatments, Journal of the American Statistical Association 58, 388–400. 8. Cooperative Clinical Trial of ACTH, Cortisone and Aspirin in the Treatment of Rheumatic Fever and the Prevention of Rheumatic Heart Disease (October 1960). Circulation 22,. 9. Cornfield, J. (1966). Bayesian test of some classical hypotheses—with applications to sequential clinical trials, Journal of the American Statistical Association 61, 577–594. 10. Cornfield, J. (1966). Sequential trials, sequential analysis, and the likelihood principle, American Statistician 20, 18–23.

8

CLINICAL TRIALS, EARLY CANCER AND HEART DISEASE

11. Cornfield, J. (1969). The Bayesian outlook and its application, Biometrics 25, 617–657. 12. Cornfield, J. & Mitchell, S. (1969). Selected risk factors in coronary disease. Possible intervention effects, Archives of Environmental Health 19, 382–394. 13. Cornfield, J., Halperin, M. & Greenhouse, S. (1969). An adaptive procedure for sequential clinical trials, Journal of the American Statistical Association 64, 759–770. 14. Coronary Drug Project Research Group (1973). The Coronary Drug Project. Design, methods, and baseline results, Circulation 47, Supplement 1, 179. 15. Coronary Drug Research Group (1980). Influence of adherence to treatment and response of cholesterol on mortality in the Coronary Drug Project, New England Journal of Medicine 303, 1038–1041. 16. Cox, D. R. (1972). Regression models and life tables (with discussion), Journal of the Royal Statistical Society, Series B 34, 187–220. 17. Cox, D. R. & Oakes, D. (1984). Analysis of Survival Data. Chapman & Hall, London. 18. Dawber, T. R., Meadors, G. F. & Moor, F. E. (1951). Epidemiological approaches to heart disease: the Framingham Study, American Journal of Public Health 41, 279–286. 19. DeMets, D. L. & Halperin, M. (1981). Early stopping in the two-sample problem for bounded variables, Controlled Clinical Trials 3, 1–11. 20. Diet–Heart Feasibility Study Research Group (1968). The National Diet–Heart Study Final Report, Circulation 37, Supplement 1, 428. 21. Farber, S., Diamond, L. K., Mercer, R., Sylvester, R. F. Jr. & Wolff, J. A. (1948). Temporary remissions in children produced by folic acid antagonist aminopterin, New England Journal of Medicine 238, 787–793. 22. Feigl, P. & Zelen, M. (1965). Estimation of exponential survival probabilities with concomitant information, Biometrics 21, 826–838. 23. Frei, E., III, Holland, J. F., Schneiderman, M. A., Pinkel, D., Selkirk, G., Freireich, E. J., Silver, R. T., Gold, G. L. & Regelson, W. (1958). A comparative study of two regimens of combination chemotherapy in acute leukemia, Blood 13, 1126–1148. 24. Frei, E., III, Freireich, E. J., Gehan, E. A., Pinkel, D., Holland, J. F., Selawry, O., Haurani, F., Spurr, C. L., Hayes, D. M., James, W., Rothberg, H., Sodee, D. B., Rundles, W., Schroeder, L. R., Hoogstraten, B., Wolman, I. J., Tragis, D. G., Cooper, T., Gendel, B.

R., Ebaugh, F. & Taylor, R. (1961). Studies of sequential and combination antimetabolite therapy in acute leukemia: 6-mercaptopurine and methotrexate, Blood 18, 431–454. 25. Freireich, E. J., Gehan, E. A., Frei, E., III, Schroeder, L. R., Wolman, I. J., Anbari, R., Bergert, O., Mills, S. D., Pinkel, D., Selawry, O. S., Moon, J. H., Gendel, B. R., Spurr, C. L., Storrs, R., Haurani, F., Hoogstraten, B. & Lee, S. (1963). The effect of 6-mercaptopurine on the duration of steroid-induced remissions in acute leukemia: A model for evaluation of other potentially useful therapy, Blood 21, 699–716. 26. Gehan, E. A. (1961). The determination of the number of patients required in a preliminary and follow-up trial of a new chemotherapeutic agent, Journal of Chronic Diseases 13, 346. 27. Gehan, E. A. (1965). A generalized Wilcoxon test for comparing arbitrarily singly-censored samples, Biometrika 52, 203–223. 28. Gehan, E. A. & Schneiderman, M. A. (1990). Historical and methodological developments in clinical trials at the National Cancer Institute, Statistics in Medicine 9, 871–880. 29. Glasser, M. (1967). Exponential survival with covariance, Journal of the American Statistical Association 62, 561–568. 30. Halperin, M. (1960). Extension of the Wilcoxon-Mann-Whitney test to samples censored at the same fixed point, Journal of the American Statistical Association 55, 125–138. 31. Halperin, M. & Ware, J. (1974). Early decision in a censored Wilcoxon two-sample test for accumulating survival data, Journal of the American Statistical Association 69, 414–422. 32. Halperin, M., DeMets, D. L. & Ware, J. H. (1990). Early methodological developments for clinical trials at the National Heart Lung and Blood Institute, Statistics in Medicine 9, 881–882. 33. Halperin, M., Rogot, E., Gurian, J. & Ederer, F. (1968). Sample sizes for medical trials with special reference to long term therapy, Journal of Chronic Diseases 21, 13–24. 34. Halperin, M., Ware, J., Johnson, N. J., Lan, K. K. & Demets, D. (1982). An aid to data monitoring in long-term clinical trials, Controlled Clinical Trials 3, 311–323. 35. Heart Special Project Committee (1988). Organization, review, and administration of cooperative studies (Greenberg Report): A report from the Heart Special Project Committee to the National Advisory Heart Council, May 1967, Controlled Clinical Trials 9, 137–148.

CLINICAL TRIALS, EARLY CANCER AND HEART DISEASE 36. Kannel, W. B., Dawber, T. R., Kagan, A., Nevotskie, N. & Stokes, J. (1961). Factors of risk in the development of coronary heart disease—six year followup experience: the Framingham Study, Annals of Internal Medicine 55, 33–50. 37. Lan, K. K. G., Simon, R. & Halperin, M. (1982). Stochastically curtailed tests in long-term clinical trials, Communications in Statistics—Stochastic Models 1, 207–219. 38. Lasagna, L. (1955). The controlled clinical trial: theory and practice, Journal of Chronic Diseases 1, 353–358. 39. Mantel, N. (1966). Evaluation of survival data and two new rank order statistics arising in its consideration, Cancer Chemotherapy Reports 50, 163–170. 40. Mantel, N. & Haenszel, W. (1959). Statistical aspects of the analysis of data from retrospective studies of disease, Journal of the National Cancer Institute 22, 719–748. 41. Medical Research Council (1948). Streptomycin treatment of pulmonary tuberculosis, British Medical Journal 2, 769–783. 42. Peto, R. & Peto, J. (1972). Asymptotically efficient rank invariant test procedures (with discussion), Journal of the Royal Statistical Society, Series A 135, 185–206. 43. Schneiderman, M. A. (1961). Statistical problems in the screening search for anti-cancer drugs by the National Cancer Institute of the United States, in Quantitative Methods in Pharmacology. North-Holland, Amsterdam. 44. Schneiderman, M. A. & Armitage, P. (1962). A family of closed sequential procedures, Biometrika 49, 41–56. 45. Schneiderman, M. A. & Armitage, P. (1962). Closed sequential t-tests, Biometrika 49, 359–366. 46. Simon, R. (1989). Optimal two stage designs for Phase II trials, Controlled Clinical Trials 10, 1–10. 47. Technical Group and Committee on Lipoproteins and Atherosclerosis (1956). Evaluation of serum lipoproteins and cholesterol measurements as predictors of clinical complications of atherosclerosis, Circulation 14, 691–742. 48. The National Program of Cancer Chemotherapy Research (1960). Cancer Chemotherapy Reports 1, 5–34. 49. Truett, J., Cornfield, J. & Kannel, W. B. (1967). A multivariate analysis of the risk factors of coronary heart disease in Framingham, Journal of Chronic Diseases 20, 511–524.

9

50. Tucker, W. B. (1954). Experiences with controls in the study of the chemotherapy of tuberculosis, Transactions of the 13th Veterans Administration Conference on the Chemotherapy of Tuberculosis, Vol. 15. 51. Wu, M., Wittes, J. T., Zucker, D. & Kusek, J. eds (1988). Proceedings of the Workshop on Methods for Longitudinal Data Analysis in Epidemiological and Clinical Studies, Statistics in Medicine 7, 1–361. 52. Zubrod, C. G. (1982). Clinical trials in cancer patients: an introduction, Controlled Clinical Trials 3, 185–187. 53. Zubrod, C. G., Schneiderman, M., Frei, E., III, Brindley, C., Gold, G. L., Shnider, B., Oviedo, R., Gorman, J., Jones, R., Jr, Jonsson, U., Colsky, J., Chalmers, T., Ferguson, B., Dederick, M., Holland, J., Selawry, O., Regelson, W., Lasagna, L. & Owens, A. H., Jr (1960). Appraisal of methods for the study of chemotherapy of cancer in man: Comparative therapeutic trial of nitrogen mustard and thiophosphoramide, Journal of Chronic Diseases 11, 7–33.

CASE STUDIES: OVER-THE-COUNTER DRUGS

challenges. These questions must probe a consumer’s understanding of OTC label messages without cueing particular responses. Open-ended questions may introduce the least bias but may be difficult to objectively score. Some of these issues will be illustrated using data from the label development program supporting the prescription-to-OTC switch of omeprazole (Prilosec) as presented to the FDA’s Nonprescription Drugs Advisory Committee (7). An early version of the OTC Prilosec label included a warning that the consumer should ‘‘Ask a doctor before use if you . . . are taking . . . phenytoin (seizure medicine)’’ (7) because of concerns about a potential drug–drug interaction. In a label comprehension study, participants were provided the proposed label and were asked: ‘‘You suffer from seizures and are taking a medicine called Phenytoin to help control your seizures. You also routinely suffer from heartburn several times per week. You have just heard about this new product, Prilosec 1 for the prevention and relief of heartburn. If you were the person described in this situation and you wanted to use Prilosec 1 to prevent or treat your heartburn, what would you do now?’’ (7). This question, which was asked of 504 participants, includes a scenario designed to assess comprehension of the warning against concomitant use. To focus on this issue, the scenario provides a clinical indication for the use of Prilosec. Approximately 90% of participants responded that they would check with their doctor or would not use the Prilosec. Both of these answers are acceptable in that they are consistent with the intent of the warning. A low-literacy subset of the participants performed in a manner similar to the total study population. These results are encouraging and suggest that the label would prevent concomitant use of phenytoin and omeprazole based on effective communication of this specific message. However, the same participants were asked a similar question but with Prozac, a drug not mentioned on the label, as a possible concomitant medication. Over half the participants again indicated that they would not use the Prilosec or would consult with their

ERIC P. BRASS Harbor-UCLA Center for Clinical Pharmacology Torrance, California

Switching a drug from prescription to overthe-counter (OTC) status is based on evidence that consumers can use the drug safely and effectively in the absence of a healthcare professional (1). The OTC drug label is the key source of information for consumers when deciding whether to use a specific OTC drug and how to self-manage their condition when using the drug. Manufacturers must provide data supporting the claim that consumers will use the drug properly in the unsupervised OTC setting. Specifically, manufacturers submit clinical research data to regulatory authorities demonstrating that consumers can understand the OTC drug label and will heed the instructions contained in the label. Label comprehension studies evaluate the ability of consumers to understand key communication objectives as presented in the proposed OTC label (2, 3). Self-selection and actual use studies test the ability of consumers to decide if the OTC drug is appropriate for their use based on their individual health history and whether they can selfmanage the treatment course. The design, conduct, and interpretation of these studies pose unique challenges, some of which are illustrated in our case examples. Although some label development studies have been published (4–6), many examples are available from the deliberations of Food and Drug Administration (FDA) Advisory Committee meetings. Importantly, each OTC clinical program must be individually designed to meet the specific issues associated with the specific OTC candidate. 1

LABEL COMPREHENSION STUDIES

Construction of questions or instruments to assess label comprehension poses particular

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

CASE STUDIES: OVER-THE-COUNTER DRUGS

doctor first. This conservative response by consumers is reassuring from a public health perspective, but it suggests that the excellent response to the phenytoin scenario was less a measure of comprehension than of a more universal tendency to give a safe response in the testing scenario. The communication of the concomitant medication warnings for Prilosec was further studied in a group of individuals who had label contraindications for Prilosec use. These people were provided the label and asked, ‘‘If you were a heartburn sufferer and you wanted to relieve heartburn, would it be OK for you to use Prilosec 1 or not?’’ (7). Note that, in contrast to the phenytoin scenario, this question is very focused and applies to the individual participant. Over 40% of participants with a label contraindication responded that it would be OK to use, in contrast to the label’s intent. Attempts to use open-ended questions to elicit an understanding of why the incorrect responders felt that they could use the drug yielded answers ranged from ‘‘Don’t know’’ to ‘‘Warning labels don’t apply to me’’ (7). The Prilosec example illustrates the challenge in motivating consumers to heed clearly communicated label warnings. Comprehension of the core message is necessary but not sufficient for an OTC label to ensure proper drug use by consumers. Thus, selfselection and actual use studies are required to assess consumer behaviors. The poor intent-to-heed rate in the Prilosec case would represent a barrier to OTC approval if the phenytoin–omeprazole interaction would result in substantial risk to consumers. Moreover, it illustrates how label comprehension study questions must be designed to truly assess label communication as opposed to general consumer insights into expected responses (for example, ‘‘I’d ask my doctor’’). Ultimately, Prilsoec was approved for OTC marketing. The final label has a more limited concomitant medication warning excluding phenytoin and focusing on drugs where a more clinically relevant drug–drug interaction might occur.

2 ACTUAL USE STUDIES Most proposed OTC drugs require a unique set of self-selection and overall self-management behaviors by consumers. Thus, the design of each actual use trial must incorporate assessments designed to evaluate those aspects most relevant to the specific OTC drug being studied. 2.1 Orlistat Orlistat is a drug that was approved for prescription marketing in the United States for obesity management, including weight loss and weight maintenance. It works as an inhibitor of lipases in the gastrointestinal tract, and thus its proper use requires that it be taken with meals in patients compliant with a low-fat diet (8). Additionally, as it may inhibit absorption of other compounds from the intestine, including fat-soluble vitamins, it is recommended that orlistat users also take a multivitamin at least 2 hours before or after meals. Thus, when orlistat was proposed for a switch to OTC status, the self-selection and actual use trials were an important aspect of the FDA’s evaluation (9). The actual use study for orlistat was conducted at 18 pharmacies with a wide geographic distribution (9). Study participants were recruited through in-store and newspaper advertisements. Interested consumers were shown the orlistat package and told: ‘‘Imagine you are in a store and this is a new over-the-counter medicine. You can take as much time as you need to look at the packaging. Let me know when you are finished.’’ They were then asked the selfselection question: ‘‘Do you think this medication is appropriate for you to use?’’ After answering the self-selection question, they were asked, ‘‘The cost of this medication is $45 for a bottle of 90 capsules. Would you like to purchase the medicine today?’’ Each bottle would provide up to 30 days of treatment, and participants were allowed to buy up to three bottles at a time. Consumers who purchased orlistat were followed by periodic phone contact, which used a structured question-based interview. Additionally, participants could return to the study pharmacy to purchase additional orlistat. They were followed for up to 90 days of treatment.

CASE STUDIES: OVER-THE-COUNTER DRUGS

3

Figure 1. Flow of research subjects in orlistat OTC actual use study. Of the 703 participants screened at sites, 237 were included in the core analysis of longitudinal use behaviors. However, other cohorts provided useful information on the effectiveness of other label elements. Data from Feibus (9).

In the orlistat actual use study, of 681 eligible participants who examined the package, 543 said the product was appropriate for their use, 339 wanted to purchase the product, 262 actually purchased the product, and 237 of the purchasers had evaluable behavioral data (Figure 1). This numerical characterization of the population highlights an important aspect of data analysis for actual use studies: which denominator should be used? There is no generalized answer as the population should be the most relevant to the question/hypothesis being examined. Most often this is either the self-selection population or the evaluable purchasers. However, there may be exceptions. For example, if the label’s ability to deter use by consumers with a contraindication is being tested, the denominator may be the number of individuals with the contraindication who evaluated the product, and the numerator those that self-selected or purchased the drug. This is more meaningful than simply expressing the percentage of purchasers with the contraindication, which may be misleadingly low if a small percentage of the evaluating individuals had the contraindication. As in any study, omission

of a portion of the study population from the analysis for any reason raises questions as to the robustness of the data set. Understanding how orlistat would be used in an unsupervised setting was a major objective of the actual use trial. Key behaviors studied included whether orlistat was used with meals (>95% used with meals), whether the maximal daily dose was exceeded ( S(B2c − B2x ) (2) 2σ 2 (1 − α) (2 − α) This inequality can be used to determine situations in which a cohort design has lower MSE than a cross-sectional design. As the

4

COHORT VS. REPEATED CROSS-SECTIONAL SURVEY DESIGNS

cohort bias is usually larger than the crosssectional bias, the cross-sectional estimate will have lower MSE when S is large. Similarly, if ρ < α/(2 − α), then the left side of Equation (2) is negative, and the crosssectional design will be better. The equation can be solved for S*, the value of S for which the cohort and cross-sectional estimates have the same MSE. Notice that if (B2 c − B2 x ) is negative, dividing Equation (2) through by this term reverses the inequality; and if zero, the inequality is independent of S, and depends only on ρ and α. Continuing the smoking example, the correlation of smoking at T 0 and at T 1 is estimated as 0.80 and σ 2 = 1826.71. From Table 1, the attrition rate is 1 − 3280/5475 = 0.40. For 22 different health-related survey variables (including smoking), both the cohort and the cross section differed significantly from the stayers, with the stayers having less favorable results. The intertemporal correlation, ρ, ranged from 0.19 for being unemployed to 1.0 for race and gender (which do not change). For 19 of the 22 variables, the cohort bias was larger in absolute value than the cross-sectional bias, making the righthand side in Equations (2) and (3) positive. All variables but one had correlations larger than α/(2 − α) = 0.25, meaning that the lefthand term was usually positive. The values of S*, the total sample size at which the two MSEs are the same, ranged from 244 to infinity. A cross-sectional design may be best for variables with low S* for moderate and large numbers of surveys. A cohort estimate is better for the others unless very large samples can be afforded. 2.2 Comparison of Change in Two Communities Inequality (2) shows when a cohort is better than a cross-sectional design to estimate change in a single community. However, if the goal of the survey is to estimate the difference in change between two communities (usually a treatment versus a control community), it is possible that the biases in the treatment and control estimates are similar and will thus cancel out. A similar argument to that above shows that a cohort design estimate of the difference between the treatment

change and the control change has a lower mean squared error than a cross-sectional design estimate for the same fixed budget if 4σ 2


 α (2 − α) ρ− > S(βc2 − βx2 ) (3) (1 − α) (2 − α)

where β c = Bc,tx − Bc,ctrl (the difference in treatment and control cohort biases) and β x = Bx,tx − Bx,ctrl (the difference in treatment and control cross-sectional biases). Note that, even if Bc,tx and Bc,ctrl are large, β c may be small. Differences in costs and sample sizes can be incorporated into this equation. Of the 22 variables considered, the term [β 2 c − β 2 x ] was negative in only 5 instances, but S* was usually very large. That is, a cohort estimate has a lower mean squared error unless the samples are very large. If about 1000 surveys could be afforded, the cross-sectional estimate would be better for estimating the difference in changes for only 8 of the 22 variables. Equations (2) and (3) may be used to help plan community surveys when attrition is a factor, and other methods may be used when it is not (7, 8). The data used here for illustration had a high attrition rate, which should be favorable to repeated cross-sectional survey designs, but the cohort designs usually had lower MSE. Other articles cited here have different examples and somewhat different findings.

REFERENCES 1. D. C. Altman, A framework for evaluating community-based heart disease prevention programs. Soc. Sci. Med. 1986; 22: 479–487. 2. G. V. Glass, V. L. Willson, and J. M. Gottman, Design and Analysis of Time-Series Experiments; Boulder: Colorado Assoc. Univ., 1975. 3. S. Salvini, D. J. Hunter, L. Sampson, M. J. Stampfer, G. A. Colditz, et al., Food-based validation of dietary questionnaires: the effects of week-to-week variation in food consumption. Int. J. Epidemiol. 1989; 18: 858–867. 4. T. D. Koepsell, E. H. Wagner, A. C. Cheadle, et al., Selected methodological issues in evaluating community-based health promotion and disease prevention programs. Annu. Rev. Publ. Health. 1992; 13: 31–57.

COHORT VS. REPEATED CROSS-SECTIONAL SURVEY DESIGNS 5. J. T. Salonen, T. W. Kottke, D. R. Jacobs, and P. J. Hannan, Analysis of community-based studies-evaluation issues in the North Karelia Project and the Minnesota Heart Health Program. Int. J. Epidemiol. 1986; 15: 176–182. 6. L. S. Caplan, D. S. Lane, and R. Grimson, The use of cohort vs repeated cross-sectional sample survey data in monitoring changing breast cancer screening practices. Prev. Med. 1995; 24: 553–556. 7. S. M. McKinlay, Cost-efficient designs of cluster unit trials. Prev. Med. 1994; 23: 606–611. 8. H. A. Feldman and S. M. McKinlay, Cohort versus cross-sectional design in large field trials; precision, sample size, and a unifying model. Stat. Med. 1994; 13: 61–78. 9. P. Diehr, D. C. Martin, T. Koepsell, A. Cheadle, E. Wagner, and B. M. Psaty, Optimal survey design for community-intervention evaluations: cohort or cross-section? J. Clin. Epidemiol. 1995; 48: 1461–1472. 10. E. H. Wagner, T. D. Koepsell, C. Anderman et al., The evaluation of the Henry J Kaiser Family Foundation’s Community Health Promotion Grant Program: Design. J. Clin. Epidemiol. 1991; 44: 685–699. 11. M. H. Gail, D. P. Byar, T. F. Pechachek, and D. K. Corle, Aspects of statistical design for the Community Invervention Trial for Smoking Cessation (COMMIT). Controlled Clinical Trials. 1992; 13: 6–21.

5

COLLINEARITY

variable is, in turn, regarded as an outcome variable in a regression equation that includes the remaining p − 1 explanatory variables. Then, R2j represents the squared residual correlation obtained using explanatory variable j, j = 1, . . . , p, as the response. The VIF is then defined for each such regression as: 1 . VIFj = 1 − R2j

G. A. DARLINGTON Cancer Care Ontario, Toronto, Ontario, Canada

Collinearity (or ‘‘multicollinearity’’) refers to a high level of correlation within a set of explanatory variables. In a regression modeling situation, if explanatory variables are highly correlated, then regression coefficient estimates may become unstable and not provide accurate measures of the individual effects of the variables. The estimate of the precision of these coefficient estimates is also affected and therefore confidence intervals and hypothesis tests are, likewise, affected. For the estimation of regression coefficients, the columns of the design matrix must be linearly independent. At an extreme, if two explanatory variables are perfectly linearly associated (i.e. their correlation is equal to 1), then such collinearity is an example of linearly dependent columns in the design matrix, X. While two parameters require estimation (i.e. the regression coefficients for the two explanatory variables), information is not available in the design matrix to estimate both coefficients uniquely. The two individual effects cannot be distinguished as a result of this collinearity. While collinearity typically does not involve completely linearly related explanatory variables, high levels of correlation can still lead to difficulties in coefficient estimation. It should be noted that this issue pertains to the relationship among explanatory variables which, ultimately, affects the ability to investigate simultaneously the relationship between the response variable and the explanatory variables. Therefore, the identification of potential collinearity problems is usually addressed by examination of the relationships among explanatory variables. One simple technique for the identification of collinearity is presented in Kleinbaum et al. (1). The computation of the variance inflation factor (VIF) is suggested. If there are p explanatory variables, each explanatory

If there is a strong relationship between the explanatory variable j and the remaining p − 1 explanatory variables, then R2j is close to 1 and VIFj is large. It is suggested, in (1), that values of VIF greater than 10 indicate serious collinearity that will affect coefficient and precision estimation. Collinearity may also be indicated if coefficient estimates from fitting simple regression models of the response with each explanatory variable are substantially different from coefficient estimates from fitting a multiple regression model including all explanatory variables. Similarly, if the order in which certain terms are included in the model seriously affects the coefficient estimates for these terms, then collinearity is indicated. Of course, one of the primary purposes of multivariate regression models is to examine the role of explanatory variables having ‘‘adjusted’’ for other variables in the model so that such behavior is not necessarily a problem. However, serious collinearity problems may prohibit a multivariate model from being fitted at all. If two or more explanatory variables are highly correlated because they represent measurements of the same general phenomenon (e.g. highest attained level of education and current salary are both aspects of socioeconomic status), then collinearity can be addressed by choosing one variable thought to be the most relevant. This variable would then be included in any models and the remaining, so-called redundant, variables would be excluded. The identification of such redundant variables may be difficult, so, alternately, a new variable that combines information on the correlated variables can be derived. This aggregate variable would

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

COLLINEARITY

be included in models instead of all of the component variables. It is sometimes helpful, particularly when collinearity is created as a result of including polynomial terms (e.g. X and X2 are included in a model together) but also in general, to center the original explanatory variables. This is accomplished by computing new explanatory variables that are the original measurements with the means subtracted. Suppose there are n individuals and p explanatory variables measured on each individual, X ji , i = 1, . . . , n, j = 1, . . . , p. Then the new explanatory variables are Zji = X ji − X j . If a quadratic model is of interest, then one would include the terms Zji and Z2ji in the model. In (1), an example of the effectiveness of such an approach for correcting collinearity is presented.

When polynomial regression is being undertaken, then the further step of orthogonalization of the explanatory variables is also possible and frequently used in some settings. Orthogonalization of more general sets of explanatory variables is possible but not as widely used. REFERENCES 1. Kleinbaum, D. G., Kupper, L. L. & Muller, K. E. (1988). Applied Regression Analysis and Other Multivariate Methods, 2nd Ed. PWSKent, Boston.

COMBINATION THERAPY

as resistance rapidly emerged (2). The successive development of two- and three-drug regimens as more drugs became available, initially PAS (P) and isoniazid (H), led to effective treatments with regimens of three drugs for one to three months followed by two drugs for 12 to 18 months (SPH/PH) (3). The introduction of new drugs, particularly rifampicin (R) and pyrazinamide (Z), led to the development of more effective regimens, although it has proved impossible to reduce the duration below 6 months and maintain highly effective regimens (SHRZ/HR) (4). The need for new antituberculosis drugs is largely driven by the high incidence of resistance to one or more of these key drugs, but there would also be major benefits in combination regimens that would reduce the total duration to less than 6 months. A very similar process has led to the current standard therapies for HIV infection with trials demonstrating that, as they became available, two drugs were better than one (5) and three better than two (6). Now, although different combinations are compared in many trials, often only one of the drugs differs. For example, in the ACTG 384 trial, triple and quadruple regimens were compared, which all included two nucleoside analogue reverse transcriptase inhibitors (NRTIs) with either efavirenz (EFV), a non-nucleoside reverse transcriptase inhibitor (NNRTI), nelfinavir, a protease inhibitor (PI), or both drugs (7). This trial also explored, in a factorial design, two different nucleoside reverse transcriptase inhibitor (NRTI) combinations, didanosine (ddI) plus stavudine (d4 T) and ZDV plus lamivudine (3TC). As in tuberculosis, a need exists for new drugs and combinations for HIV infection as the current therapies have to be given for long periods and are associated with substantial failure rates, usually because of difficulties with adherence to therapy and the emergence of resistance. Once a combination has been shown to be effective and the optimal dosages of the drugs are clearly defined, advantages of combining the drugs in a single preparation exist, as this combination aids compliance by simplifying therapy. It also minimizes the risk of the

JANET DARBYSHIRE London, United Kingdom

1

DEFINITION

Combination therapy regimens most commonly combine two or more drugs that have additive or synergistic action to treat a disease more effectively, frequently infections or cancer, with the aim of producing more effective regimens and preventing the emergence of resistance of the micro-organism or the tumor to the drugs. However combinations of drugs are widely used to treat many other diseases such as hypertension, asthma, diabetes, and arthritis. Combinations of drugs and other modalities, such as surgery or radiotherapy, are also used in certain disease areas, such as cancer. Clinical trials may be designed to evaluate the combination therapy or to assess the effect of a new drug in addition to other drugs as part of a combination. In the first case, it is the combination that is being evaluated, and all the components may be different between the regimens or, indeed, a combination may be tested against a single drug (for example, the ICON 3 trial compared a combination of two drugs with either a regimen of 3 drugs or monotherapy) (1). Indeed, as the number of available drugs has increased for the treatment of diseases such as cancer, tuberculosis, or HIV infection, trials have successively evaluated combinations of two, three, or more drugs to achieve a more rapid or complete cure depending on the disease. The balance between increasing efficacy by increasing the number of drugs in a combination and the risk of increasing toxicity is key to the development of optimal regimens. The introduction of new drugs, especially with different modes of action, may lead to major changes in the therapy of a disease. Tuberculosis is a classic example of how the use of combination therapy changed over time. The first drug, streptomycin (S), was clearly highly effective, but only for a short time

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

COMBINATION THERAPY

development of resistance to a drug, in the case of antibiotics, as a result of the patients choosing to take only one of the drugs. However, disadvantages exist in terms of the lack of flexibility to relate dose to weight and the management of toxicity. Combinations of ethambutol and isoniazid, which are widely used for the treatment of tuberculosis, are available in a variety of formulations because of the dose-related toxicity of ethambutol and, therefore, the need to relate the dose to weight. Such combinations should be compared with the individual drugs to ensure that they are adequately absorbed and produce comparable pharmacokinetic profiles. If a trial is evaluating combination regimens that are available in combined preparations, a decision will have to be made as to whether the preparation should be used. If they are, and the comparator is not available as a combined preparation, it may overestimate the benefits of the combined preparation because it is more likely to be taken. However, if single drugs are given for both regimens, the potential benefits of the combined preparation cannot be adequately assessed. In a trial that is exploring experimental regimens, single preparations are more often used, but if the regimens are being assessed as they will be used in routine practice, then a case for using the combined preparation exists. The development of combination regimens is often built on evidence from laboratory studies of additive or synergistic activity of drugs and the need for combinations because no single drug is adequate to cure a disease in some or all patients (for example, cancer or tuberculosis) or to control it (for example, HIV infection, hypertension, or diabetes). In different diseases, the approach may be different according to availability of drugs or the disease course. In chronic diseases, such as hypertension, Type II diabetes, or epilepsy, the aim is to control the disease with the minimum therapy. New drugs may be added to a combination over time to achieve this result. Trials may be designed to compare aggressive therapy with this standard approach to assess the impact on long-term disease control. Trials of new drugs are likely to mimic this approach by adding a new drug or a standard drug compared with adding a placebo or,

alternatively, adding an existing drug, if this practice is standard. In some diseases, different aims from trials of combination therapies may exist. One approach is to try to improve the results of treatment by increasing the potency of the regimens, by adding more drugs to the initial regimen, ideally with a different mode of action. For example, the ACTG 384 trial compared a four-drug regimen with the two standard three-drug regimens for HIV infection (7). An alternative approach is to reduce the toxicity of the regimens by minimizing the exposure to drugs while maintaining control of the infection. Such an approach may be particularly important in diseases such as HIV infection where the drugs are likely to be given for long periods of time and have potentially serious long-term side effects. A number of ongoing trials are exploring different approaches, for example, comparing strict control of viral replication with less aggressive therapy based on immunological markers. Trials of combination therapies may be used to assess new drugs when they will only be given as part of such regimens. Two alternative approaches exist that may be appropriate in different circumstances. The first is to randomize to add the new drug or placebo to existing standard therapy (8). The second is to substitute the new drug for one of the drugs in a standard combination and to compare with the standard combination (9). The advantages of the former are that, theoretically, it is easier to demonstrate a difference from placebo, but the risk is that if the current therapy is highly effective, little if any benefit may exist from adding a new drug. The disadvantage of the second approach is that it is likely to be more difficult to demonstrate superiority or equivalence to an existing drug. Further, reluctance may exist to substitute a new drug for one that is known to be effective. In some areas, such as leukaemia or lymphoma where therapies are becoming more and more effective, and yet are still not uniformly successful, it is becoming increasingly difficult to assess new combination therapies as the improvements are likely to be small. The large trials needed to reliably assess such

COMBINATION THERAPY

small differences may not be feasible, especially for the types of disease that are more rare. The development of new drugs for diseases such as cancer and HIV brings new challenges, not the least of which is how best to evaluate the many potential combinations that can be selected by combining the new and old drugs. Novel trial designs are needed together with the development of better surrogate markers that can be used to select the best combinations to take forward into large trials to assess clinical benefits and risks. Two-stage designs, such as those reported by Royston and Parmar (10), are innovative approaches to this problem. Combination therapies are only needed if monotherapies are not potent enough. Ultimately, the aim is to provide a therapy that is effective, safe, and simple to take, and in many diseases, major advantages would exist in replacing regimens of three, four, or even more drugs by a monotherapy regimen. When the first trials of protease inhibitors demonstrated their high potency in HIV infection, some hope existed that they might be as effective as current two-drug combination regimens, but it soon became clear that they were not sufficiently potent to prevent the emergence of resistance on their own, although they had a major impact when added to the two-drug regimen. Combinations of drug and nondrug therapy may be used to treat diseases, such as cancer, where bone marrow or stem cell transplants require chemotherapy as part of the whole treatment. Similarly combinations of chemotherapy and radiotherapy are effective in some forms of tumor (referred to as chemoradiotherapy). Other multi-mode combinations, such as chemotherapy and immunotherapy, (with drugs or therapeutic vaccines) may be developed for infections. The approach to assessing all of these is similar to drug combinations but special issues often exist, such as the timing of the different interventions within the overall treatment plans to achieve maximum benefits and to minimize toxicity. These issues of timing may make the assessment of the regimens more complex, particularly if the timings differ between regimens.

3

REFERENCES 1. ICON Collaborators, Icon3: randomised trial comparing paclitaxel plus carboplatin against standard chemotherapyof either single agent carboplatin or CAP (Cyclophosphamide, doxirubicin, cisplatin) in women with ovarian cancer. Lancet 2002; 360: 505–515. 2. A five-year assessment of patients in a controlled trial of streptomycin in pulmonary tuberculosis. Quart. J. Med. 1954: 91: 347–366. 3. Medical Research Council, Long-term chemotherapy in the treatment of chronic pulmonary tuberculosis with cavitation. Tubercle 1962; 43: 201–267. 4. W. Fox, Whither short-course chemotherapy? Br. J. Dis. Chest 1981; 75: 331–357. 5. HIV Trialists’ Collaborative Group, Zidovudine, didanosine, and zalcitabine in the treatment of HIV infection: meta-analyses of the randomised evidence. Lancet 1999; 353: 2014–2025. 6. S. M. Hammer, K. E. Squires, M. D. Hughes, J. M. Grimes, L. M. Demeter, J. S. Cuyrrier et al., A controlled trial of two nucleoside analogues plus indinavir in persons with human immunodeficiency virus infection and CD4 cell counts of 200 per cubic millimeter or less. N. Engl. J. Med. 1997; 337(11): 725–733. 7. G. K. Robbins, V. De Gruttola, R. W. Shafer et al., Comparison of sequential three-drug regimens as initial therapy for HIV-1 infection. N. Engl. J. Med. 2003: 349; 2293–2363. 8. D. W. Cameron, M. Heath-Chiozzi, S. Danner, C. Cohen, S. Kravcik, C. Maurath et al., Randomised placebo controlled trial of ritonavir in advanced HIV-1 disease. Lancet 1998; 351(9102): 543–549. 9. S. Staszewski, J. Morales-Ramirez, K. T. Tashima, A. Rachlis, D. Skiest, J. Stanford, R. Stryker, P. Johnson, D. F. Labriola, D. Farina, D. J. Manion, and N. M. Ruiz, Efavirenz plus zidovudine and lamivudine, efavirenz plus indinavir, and indinavir plus zidovudine and lamivudine in the treatment of HIV1 infection in adults. NEJM 1999; 341(25): 1865–1873. 10. P. Royston, M. K. B. Parmar, and W. Qian, Novel designs for multi-arm clinical trials with survival outcomes with an application in ovarian cancer. Stat. Med. 2003; 22: 2239–2256.

COMMITTEE FOR MEDICINAL PRODUCT FOR HUMAN USE (CHMP)

in close cooperation with health-care professionals and the pharmaceutical companies themselves. The CHMP plays an important role in EU-wide pharmacovigilance by closely monitoring reports of potential safety concerns (Adverse Drug Reaction Reports, or ADRs) and, when necessary, by making recommendations to the European Commission regarding changes to a product’s marketing authorization or the product’s suspension/withdrawal from the market. In cases where there is an urgent requirement to modify the authorization of a medicinal product due to safety concerns, the CHMP can issue an Urgent Safety Restriction (USR) to inform health-care professionals about changes in how or under what circumstances the medication may be used. The CHMP publishes a European Public Assessment Report (EPAR) for every centrally authorized product that is granted a marketing authorization, which sets out the scientific grounds for the Committee’s opinion in favor of granting the authorization. A Summary of Product Characteristics (SPC) is also published with the labeling and packaging requirements for the product and details of the procedural steps taken during the assessment process. These EPARs are published on the EMEA’s website and are generally available in all official languages of the EU. Scientific assessment work conducted by the CHMP is subject to an internal peerreview system to safeguard the accuracy and validity of opinions reached by the Committee. The EMEA’s integrated qualitymanagement system ensures effective planning, operation, and control of the CHMP’s processes and records. Other important activities of the CHMP and its working parties include:

The Committee for Medicinal Products for Human Use (CHMP) is responsible for preparing the European Medicines Agency’s opinions on all questions concerning medicinal products for human use, in accordance with Regulation European Commission (EC) No 726/2004. The CHMP plays a vital role in the marketing procedures for medicines in the European Union (EU): • The CHMP is responsible for conduct-

ing the initial assessment of medicinal products for which a Community-wide marketing authorization is sought. The CHMP is also responsible for several postauthorization and maintenance activities, including the assessment of any modifications or extensions (variations) to the existing marketing authorization. • The CHMP arbitrates in cases of disagreement between member states over the marketing authorization of a particular medicinal product. The CHMP also acts in referral cases, initiated when there are concerns relating to the protection of public health or where other community interests are at stake. Assessments conducted by the CHMP are based on purely scientific criteria and determine whether the products concerned meet the necessary quality, safety, and efficacy requirements in accordance with EU legislation (particularly Directive 2001/83/EC). These processes ensure that once medicinal products reach the marketplace they have a positive risk–benefit balance in favor of the patients/users of the products. Subsequent monitoring of the safety of authorized products is conducted through the EU’s network of national medicines agencies,

• Assistance to companies in researching

and developing new medicines. • Preparation of scientific and regulatory

guidelines for the pharmaceuticals industry. • Cooperation with international partners on the harmonization of regulatory requirements for medicines.

This article was modified from the website of the European Medicines Agency (http://www.emea .europa.eu/htms/general/contacts/CHMP/CHMP. html) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

COMMON TECHNICAL DOCUMENT (CTD)

text. Acronyms and abbreviations should be defined the first time they are used in each module. References should be cited in accordance with the current edition of the Uniform Requirements. The CTD should be organized into five modules: Module 1 is region specific; modules 2, 3, 4, and 5 are intended to be common for all regions. Conformance with the CTD guidances should help ensure that these four modules are provided in a format acceptable to the regulatory authorities (see the figure and overall outline on the following pages).

Through the International Conference on Harmonisation (ICH) process, considerable harmonization has been achieved among the three regions (Japan, Europe, and the United States) in the technical requirements for the registration of pharmaceuticals for human use. However, until now, no harmonization of the organization of a submission has existed. Each region has its own requirements for the organization of the technical reports in the submission and for the preparation of the summaries and tables. In Japan, the applicants must prepare the GAIYO, which organizes and presents a summary of the technical information. In Europe, expert reports and tabulated summaries are required, and written summaries are recommended. The U.S. Food and Drug Administration (FDA) has guidance regarding the format and content of the new drug application submission. To avoid generating and compiling different registration dossiers, this guidance describes a harmonized format for the Common Technical Document (CTD) that will be acceptable in all three regions. Throughout the CTD, the display of information should be unambiguous and transparent to facilitate the review of the basic data and to help a reviewer quickly become oriented to the application contents. Text and tables should be prepared by using margins that allow the document to be printed on both A4 paper (E.U. and Japan) and 8.5 x 11’’ paper (U.S.). The left-hand margin should be sufficiently large that information is not obscured through binding. Font sizes for text and tables should be of a style and size that are large enough to be easily legible, even after photocopying. Times New Roman, 12-point font is recommended for narrative

Module 1. Administrative Information and Prescribing Information This module should contain documents specific to each region; for example, application forms or the proposed label for use in the region. The content and format of this module can be specified by the relevant regulatory authorities. For information about this module see the guidance for industry, General Considerations for Submitting Marketing Applications According to the ICH/CTD Format. Module 2. Common Technical Document Summaries Module 2 should begin with a general introduction to the pharmaceutical, including its pharmacologic class, mode of action, and proposed clinical use. In general, the introduction should not exceed one page. Module 2 should contain seven sections in the following order: • CTD Table of Contents • CTD Introduction • Quality Overall Summary • Nonclinical Overview • Clinical Overview • Nonclinical Written and Tabulated

Summaries • Clinical Summary

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cber/gdlns/m4ctd.pdf) by Ralph D’Agostino and Sarah Karl.

Because Module 2 contains information from the Quality, Efficacy, and Safety sections of the CTD, the organization of the

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

COMMON TECHNICAL DOCUMENT (CTD)

individual Module 2 summaries is discussed in three separate documents: • M4Q: The CTD – Quality • M4 S: The CTD – Safety • M4E: The CTD – Efficacy

Module 3. Quality Information on Quality should be presented in the structured format described in the guidance M4Q. Module 4. Nonclinical Study Reports The Nonclinical Study Reports should be presented in the order described in the guidance M4 S. Module 5. Clinical Study Reports The human study reports and related information should be presented in the order described in the guidance M4E. The CTD should be organized according to the following general outline. Module 1: Administrative Information and Prescribing Information. 1.1 Table of Contents of the Submission Including Module 1 1.2 Documents Specific to Each Region (for example, application forms and prescribing information) Module 2: Common Technical Document Summaries. 2.1 CTD Table of Contents 2.2 CTD Introduction 2.3 Quality Overall Summary 2.4 Nonclinical Overview 2.5 Clinical Overview 2.6 Nonclinical Written and Tabulated Summary Pharmacology Pharmacokinetics Toxicology 2.7 Clinical Summary Biopharmaceutics and Associated Analytical Methods Clinical Pharmacology Studies

Clinical Efficacy Clinical Safety Synopses of Individual Studies Module 3: Quality. 3.1 Module 3 Table of Contents 3.2 Body of Data 3.3 Literature References Module 4: Nonclinical Study Reports. 4.1 Module 4 Table of Contents 4.2 Study Reports 4.3 Literature References Module 5: Clinical Study Reports. 5.1 Module 5 Table of Contents 5.2 Tabular Listing of All Clinical Studies 5.3 Clinical Study Reports 5.4 Literature References

COMMUNITY-BASED BREAST AND CERVICAL CANCER CONTROL RESEARCH IN ASIAN IMMIGRANT POPULATIONS

cervical cancer screening are as follows: 70% of age-eligible women should have received mammography in the preceding two years and 90% of women should have received a Pap test within the preceding three years (9). Data from the 2000 National Health Interview Survey (NHIS) show wide variations in breast and cervical cancer screening rates by race and ethnicity (Table 1). In addition, the ‘‘Pathways Project’’ surveyed five San Francisco Bay populations in 1994; nearly all the white (99%) and black (98%) respondents reported at least one Pap smear, compared with 76% of Latina, 67% of Chinese, and 42% of Vietnamese respondents; similar patterns were observed for mammography (10). As the NHIS is conducted in English, it excludes individuals with limited English proficiency (11). The ‘‘Breast and Cervical Cancer Intervention Study’’ found breast and cervical cancer screening rates were significantly lower among non-English speaking than English speaking Chinese and Latina women (12). These findings support the importance of cancer screening interventions targeting immigrants, particularly those that are less acculturated (12, 13). This article focuses on community-based breast and cervical cancer control research in Asian populations. First, the authors address principles of community participatory research and several important program evaluation issues: sampling, survey methods, recruitment and retention, translation, data quality, and control group ‘‘contamination.’’ Second, the authors summarize communitybased studies that aimed to increase mammography or Pap testing levels among Asian women. For this summary, the authors used the same approach as Legler et al. (14) in their meta-analysis of interventions to promote mammography among women with historically low mammography rates. Specifically, the authors only considered studies in which the researchers used an experimental or quasi-experimental design to evaluate the effectiveness of their intervention, and reported intervention outcomes based on actual receipt of breast or cervical cancer screening (14).

VICTORIA M. TAYLOR Fred Hutchinson Cancer Research Center Division of Public Health Sciences Seattle, Washington University of Washington Department of Health Services Seattle, Washington

T. GREGORY HISLOP Cancer Control Research British Columbia Cancer Agency Vancouver, British Columbia, Canada

YUTAKA YASUI Fred Hutchinson Cancer Research Center Division of Public Health Sciences Seattle, Washington

SHIN-PING TU and J. CAREY JACKSON University of Washington Department of Medicine Seattle, Washington Harborview Medical Center Seattle, Washington,

1

INTRODUCTION

The number of Asian Americans in the United States increased from a little over one million in 1970 to nearly seven million in 1990, and reached more than 10 million by 2000 (1–3). Two-thirds (66%) of Asian Americans are foreign born and the majority (56%) do not speak English ‘‘very well.’’ Further, North America’s Asian population is heterogeneous and includes individuals from East Asia (e.g., mainland China), South Asia (e.g., India), Southeast Asia (e.g., Vietnam), and Island Asia (e.g., the Philippines) (4). The Presidential Initiative on Race aims to eliminate disparities in six areas of health status, including cancer screening, over the next decade (5, 6). Current American Cancer Society screening guidelines specify that women aged 40 and older should receive a screening mammogram annually and women aged 18 and older should receive Papanicolaou (Pap) testing every one to three years, depending on their risk for disease (7, 8). The Healthy People 2010 objectives for breast and

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

COMMUNITY-BASED BREAST AND CERVICAL CANCER CONTROL RESEARCH Table 1. Breast and Cervical Cancer Screening Rates by Race and Ethnicity—United States, 2000

Race/Ethnicity

Mammogram within the past two yearsa %

Pap test within the past three yearsb %

72 68 63 52 57

82 84 77 77 67

White Black Latina American Indian or Alaska Native Asian or Pacific Islander a b

Women aged 40 years and older Women aged 18 years or older

2 COMMUNITY PARTICIPATORY RESEARCH The importance of active community participation in research projects involving racial/ethnic minority groups is increasingly recognized (15, 16). Community-based research principles, developed by the University of Washington School of Public Health and Community Medicine, specify the following: Community partners should be involved in the earliest stages of the project and have real influence on project direction; research processes and outcomes should benefit the community; members of the community should be part of the analysis and interpretation of data; productive partnerships between researchers and community members should be encouraged to last beyond the life of the project; and community members should be empowered to initiate their own research projects that address needs they identify themselves (17). The University of California at San Francisco’s ongoing study ‘‘Reaching Vietnamese American Women: A Community Model for Promoting Cervical Cancer Screening’’ provides one example of successful communitybased research. The study collaborates with a community coalition that includes representatives from community-based organizations and health agencies that provide services to Vietnamese Americans; it uses community forums to both develop and evaluate program components; and it employs lay health workers from the targeted community. However, this research group has noted that principles of community-based research,

developed in the United States as a product of American culture, cannot always be applied to immigrants. For example, the researchers had to strike a balance between respecting the dominant cultural pattern of deferring to authority figures (e.g., local Vietnamese physicians) and encouraging community members to contribute their views. Additionally, the researchers were not able to involve non-English speaking community members in data collection because their Institutional Review Board would not allow individuals to engage in data collection activities unless they received human subjects certification (and no human subjects training materials are available in Vietnamese) (18).

3 EVALUATION ISSUES 3.1 Sampling Simple random sampling can only be used for studies targeting Asian-American populations when the group of interest is highly concentrated within a defined geographic boundary (19). For example, the ‘‘Pathways Project’’ successfully identified Chinese households using random digit dialing within San Francisco telephone prefixes that were at least 35% Chinese (20). In less concentrated populations, the following approaches have variously been used: purchasing marketing company lists for individuals of a particular Asian ethnicity, acquiring lists from organizations that serve one particular community (e.g., the Chinese Community Health Plan in Northern California), and convenience sampling

COMMUNITY-BASED BREAST AND CERVICAL CANCER CONTROL RESEARCH

(e.g., from Filipino and Korean church congregations and community organizations) (20–24). Certain Asian populations have characteristic surnames that can be applied to existing databases (e.g., telephone books) (25–29). However, the efficiency of this approach to sampling varies by Asian subgroup and is a function of population density (19, 27, 29). For example, it works well for Vietnamese who have a relatively small number of highly characteristic last names (it has been shown that 99% of Vietnamese households can be identified by using 37 surnames) (30). Surname lists work less well for other groups. It has been found to be an inefficient method of identifying Cambodian households, and it is doubtful that this approach would work for Filipinos who often have Spanish surnames (19, 31). Additionally, last name lists are not useful for identifying women from Asian subgroups with a high rate of interracial marriage (e.g., Japanese Americans) (19). Finally, for some Asian populations, a universal set of last names that can be used in research projects may not exist; rather, each locale and time period may require its own list of names. For example, some Chinese immigrant communities include a higher proportion of people with Mandarin (as opposed to Cantonese) last names than others (25). 3.2 Survey Methods Cancer control intervention studies often use surveys to identify individuals for randomized controlled trials or to evaluate intervention effectiveness. In-person interviews, conducted in the language of an interviewee’s choice, are believed to be the most effective method of enlisting cooperation from Asian immigrant groups, and can be cost-efficient when the sample is drawn from nondispersed communities (e.g., Vietnamese in the Tenderloin area of San Francisco); however, it is usually a cost-prohibitive approach for surveying dispersed Asian populations (e.g., Vietnamese in multiple Californian counties) (32, 33). Table 2 provides survey response rates from population-based breast and cervical cancer control projects, and shows that response rates vary by Asian subgroup and geographic area as well as method of survey administration (30, 31, 33–38).

3

3.3 Recruitment and Retention It can be difficult to obtain high response rates and low loss to follow-up in populations over-represented by immigrants for multiple reasons. Specifically, immigrants may not have working telephones, frequently have more than one job, may be concerned about safety and suspicious of strangers, often have multiple health and social problems, and tend to move frequently (19, 31, 39). Also, some Asian immigrants travel frequently between their original and adopted countries. A British research group identified South Asian women who were registered with general practitioners in England for a breast cancer intervention study, and found that 14% of the sample had recently moved while 12% were currently in Asia (40). Generally, recruitment and retention are enhanced by employing bilingual and bicultural staff members from the targeted community. Gender matching of research personnel and participants is also important, especially for members of traditional Asian cultures in which discussions of certain topics (e.g., cancer and gynecologic issues) are inappropriate with the opposite sex (41). 3.4 Translation As many Asian immigrants do not speak English, questionnaires have to be prepared in Asian languages, which is a somewhat cumbersome and time-consuming process that usually involves one of the following processes: translation, back-translation to ensure equivalence, and reconciliation of any discrepancies between the original and backtranslated English versions; or double forward translation into the relevant Asian language, and then review by a referee (41–44). Additionally, some Asian subgroups include individuals who speak multiple languages or dialects (e.g., Filipino American communities include Cebuano-, Ilaccano-, and Tagalog-speaking individuals) (45). Experience from the ‘‘Pathways Project’’ shows that Pap smear and mammogram, for example, cannot be translated into Chinese or Vietnamese. Therefore, the terms have to be stated in English and subsequently defined in the relevant Asian language (43, 44).

4

COMMUNITY-BASED BREAST AND CERVICAL CANCER CONTROL RESEARCH

Table 2. Response Rates from Surveys of Asian Populations Geographic area Taylor, 1999 Hislop, 2003

Cambodian Chinese

Taylor, 2002 Wismer, 1998

Chinese Korean

Bird, 1998

Vietnamese

Jenkins, 1999

Vietnamese

Nguyen, 2002

Vietnamese

Taylor, Submitted

Vietnamese

Seattle, Washington Vancouver, British Columbia Seattle, Washington Alameda/Santa Clara counties, California San Francisco, California Sacramento, California Alameda/Santa Clara counties, California Los Angeles/Orange counties, California Santa Clara county, California Harris county, Texas Seattle, Washington

3.5 Data Quality Most community-based studies to evaluate breast and cervical cancer screening intervention programs have used self-reported survey data to evaluate intervention effectiveness (46). However, increasing evidence exists that the quality of survey data may differ by race and ethnicity (46, 47). The ‘‘Pathways Project’’ surveyed five San Francisco Bay populations in 1994. Agreement between the baseline survey and a callback survey (conducted in a 10% randomly selected subsample) for the question ‘‘Have you ever had a mammogram?’’ was higher among white than racial/ethnic minority women. Specifically, the test-retest reliabilities among Chinese, Vietnamese, Latina, black, and white women were 0.90 (95% CI—0.74, 0.97), 0.74 (95% CI—0.51, 0.88), 0.90 (95% CI—0.79, 0.95), 0.93 (0.72—0.99), and 1.00 (95% CI—0.86, 1.00), respectively. Following a baseline telephone survey in multi-ethnic Alameda County, California, investigators from the ‘‘Pathfinders Project’’ examined medical records to validate breast and cervical cancer self-reports. The proportions of mammograms and Pap smears that

Year(s) survey conducted

Survey method

Response rate (%)

1997-98 1999

In-person In-person

72 55

1999 1994

In-person Telephone

64 80

1996

In-person

79 74

1996

Telephone

45

42

2000

Telephone

63

2002

In-person

54 82

could be validated were significantly lower among racial/ethnic minority than white women. Specifically, mammograms were validated for 89% of white women, 72% of black women, 72% of Latina women, 67% of Chinese women, and 76% of Filipina women. The corresponding proportions for Pap testing were 85%, 66%, 66%, 68%, and 67%, respectively (46). Several researchers have concluded that Asian Americans have a greater tendency than whites to provide socially desirable responses to survey questions, and have recommended using other methods of outcome ascertainment (e.g., medical record review), when possible (43, 46). 3.6 Control Group ‘‘Contamination’’ Program evaluation in Asian immigrant communities can be compromised by dissemination of the intervention to a study’s control group (24, 38, 48). Many Asian immigrant communities are relatively small and self-contained with strong social as well as extended family networks, and information is often quickly disseminated throughout the community. Although these communication channels serve the community well, they can

COMMUNITY-BASED BREAST AND CERVICAL CANCER CONTROL RESEARCH

compromise the methodological rigor needed for randomization protocols (48). 4

COMMUNITY-BASED STUDIES

4.1 Overview The authors identified nine studies that met inclusion criteria for review (i.e., the study design was experimental or quasiexperimental, and intervention outcomes were based on actual screening test receipt) (Table 3). These studies targeted Cambodian, Chinese, Filipina, Korean, South Asian, and Vietnamese women in Canada, the United States, and England. Overall, three of the studies randomized individual women to experimental or control status, two randomized groups of women, and four used a quasiexperimental (two-community) study design. 4.2 Individual Randomized Trials Taylor et al. (39) conducted a three-arm randomized controlled trial to evaluate cervical cancer screening interventions for Chinese American/Canadian women in Seattle, Washington, and Vancouver, British Columbia. Baseline survey respondents who under-utilized Pap testing were randomized to an outreach worker intervention (that included a home visit, use of a video and print materials, and logistic assistance accessing screening services), a direct mail intervention (that included a video and print materials), or control status. Outcome evaluation was based on results from a follow-up survey as well as medical record verification of selfreported Pap testing. Overall, 39% of the 129 women who received the outreach intervention, 25% of the 139 women who received the direct mail intervention, and 15% of the 134 controls reported Pap testing following randomization (outreach worker versus control P < 0.001, direct mail versus control P = 0.03, and outreach worker versus direct mail P = 0.02) (39). Investigators in Leicester, England, conducted a four-arm randomized controlled trial to evaluate the effects of health education on the uptake of Pap smears among women originally from the Indian subcontinent (Bangladesh, Pakistan, and India).

5

Nearly one-half (47%) of women who were shown a video during a home visit by an outreach worker adhered to screening recommendations, as did 37% of those who were visited and given a leaflet as well as a fact sheet. In contrast, only 5% of women who were not contacted and 11% of women who were sent print materials in the mail completed Pap testing (49). In another British study, researchers evaluated the effect of a home visit by an outreach worker on mammography participation by South Asian (Bangladeshi and Pakistani) women aged 50–64 years in Oldham. No difference existed in the proportion of intervention and control group women who subsequently responded to an invitation to Britain’s population-based mammography screening program (40). 4.3 Group Randomized Trials A Seattle research team conducted a grouprandomized controlled trial to evaluate a neighborhood-based outreach worker intervention to increase Pap testing among Cambodian refugees. Interventions were delivered by bicultural, bilingual Cambodian outreach workers; they included home visits and small group neighborhood meetings, use of a motivational video, and tailored logistic assistance accessing screening services. At baseline, 44% of the women in intervention neighborhoods and 51% of women in control neighborhoods reported Pap smear receipt in the past year. At follow-up, the proportions reporting a Pap test in the last 12months were 61% and 62% among intervention and control women, respectively. Increases in intervention group (17%) and control group (11%) cervical cancer screening rates were not significantly different (48). Maxwell et al. recently reported their results from a randomized controlled trial to increase breast and cervical cancer screening among Filipina American women in Los Angeles (24). Women aged 40 and older were recruited from community-based organizations as well as churches, and invited to attend a group session with a health educator. Groups were randomly assigned to receive a cancer screening module (intervention) or a physical activity module (control). Moderate increases in breast and cervical

6

COMMUNITY-BASED BREAST AND CERVICAL CANCER CONTROL RESEARCH

cancer screening rates were observed in both groups (9 to 12 percentage points). However, among recent immigrants (women who had spent less than 10 years in the United States), mammography screening increased significantly more in the intervention arm than in the control arm (a 27 versus a 6 percentage point increase, P < 0.05). 4.4 Quasi-Experimental Studies Wismer et al. (38) have reported interim evaluation data from a community-based project that aimed to increase breast and cervical cancer screening rates among Korean Americans. Lay health workers were trained to provide workshops and print materials were distributed. After an 18-month intervention period, no significant changes occurred in Pap testing rates in either the intervention community (Alameda County, California) or the control community (Santa Clara County, California). Observed mammography increases in both the intervention county and the control county were equivalent. The researchers concluded that competing programs in Santa Clara County, diffusion of the intervention from Alameda County to neighboring Santa Clara County, and secular trends for mammography may all have contributed to their negative findings (38). The ‘‘Vietnamese Community Health Promotion Project’’ in San Francisco has evaluated several breast and cervical cancer screening interventions for VietnameseAmerican women (32–34, 50–52). One study evaluated an outreach intervention to promote receipt and screening interval maintenance of mammography and Pap smears. Indigenous lay health workers conducted a series of three small group educational sessions with Vietnamese women in the Tenderloin district of San Francisco while women in Sacramento, California, served as controls. Pre- and post- intervention surveys showed that the proportions of women reporting at least one mammogram increased from 54% to 69% in the experimental area (P = 0.006). In contrast, rates remained constant in the control community. Similar results were obtained for previous Pap testing; rates increased from 46% to 66% (P = 0.001) in San Francisco, but did not increase in Sacramento (34).

Another ‘‘Vietnamese Community Health Promotion Project’’ study evaluated a medialed campaign that included use of television, newspaper, and billboard advertising as well as the distribution of audio-visual and print educational materials. Post-intervention, no difference existed in recent mammography or Pap testing use among women in two northern California experimental counties and those in two southern California control counties. However, women in the intervention area were more likely to be planning to have mammograms and Pap tests in the future than women in the control area (33). Finally, the ‘‘Vietnamese Community Health Promotion Project’’ used a two-community study design to evaluate a multi-faceted community intervention to increase mammography participation in a northern California community. No intervention effect was demonstrated in this study (52). 5 CONCLUSION The Indigenous Model has been proposed as an effective framework for the delivery of health education programs targeting immigrant groups. This model specifies that successful programs should be delivered by individuals who are acceptable and accessible to the target population, in convenient locations, and in multiple ways (53–55). The three interventions that were successful in promoting cancer screening among women of Asian descent have all used these principles (34, 39, 49). For example, Taylor et al. recruited Chinese women from the targeted communities to serve as outreach workers, delivered the intervention in women’s homes, and provided education through discussion sessions with outreach workers as well as audio-visual and print materials (39). Rychetnik et al. (56) believe that criticisms of the randomized controlled trial in public health research are based on a consideration of classic trials in which the intervention is standardized and the individual is the unit of randomization. They proposed the routine use of cluster trials, with groups as the unit of randomization and analysis, to evaluate complex public health interventions (56). Two studies of breast and cervical

COMMUNITY-BASED BREAST AND CERVICAL CANCER CONTROL RESEARCH

cancer interventions in Asian communities have used this approach with negative findings. In both cases, the investigators reported increases in screening levels among both the intervention and control groups, and proposed that control group contamination may have occurred (24, 48). As cancer control intervention programs often involve multi-faceted, community driven strategies, randomized controlled trials cannot always accommodate such programs (56). Consequently, investigators have used quasi-experimental designs with intervention and control communities (33, 34, 38, 52). Bird et al. (34) successfully used this study design to evaluate their breast and cervical cancer control intervention targeting Vietnamese women. A marked increase in breast and cancer screening rates among women in the experimental community with no change among women in the control community (together with the absence of any other promotional activities in the experimental community) provide compelling evidence for an intervention effect (34). However, other researchers have had difficulties evaluating breast and cervical cancer intervention programs, using a two-community design, because of unanticipated promotional activities in their control communities (38, 52, 57). This article addressed selected issues in the evaluation of public health interventions for Asian-American women. However, the authors did not address other important issues that are highly relevant to program development and delivery. For example, the importance of using qualitative methods during the development of interventions for immigrant groups and applying a culturally appropriate conceptual framework are not discussed (13, 41, 58, 59, 60, 61). In 1994, the Centers for Disease Control and Prevention published its report Chronic Disease in Minority Populations. The authors of the section about Asian Americans made the following recommendations: Special data collection efforts should focus on each Asian subgroup, attention should be given to low use of preventive measures, and intervention approaches should be culturally tailored to Asian communities (62). Almost a decade later, only a few studies have evaluated

7

breast and cervical cancer control interventions in Asian communities. However, these studies highlight the methodologic challenges in conducting evaluative public health research, particularly in racial/ethnic minority communities. 6

ACKNOWLEDGEMENTS

This work was supported by cooperative agreement #86322 and grant #82326 from the National Cancer Institute. REFERENCES 1. J. S. Lin-Fu, Asian and Pacific Islanders: an overview of demographic characteristics and health care issues. Asian Amer. Pacific Islander J. Health 1993; 1: 21–36. 2. U.S. Census Bureau, The Asian Population: 2000. Washington, DC: U.S. Department of Commerce, 2000. 3. U.S. Census Bureau, A Profile of the Nation’s Foreign-born Population from Asia (2000 update). Washington, DC: U.S. Department of Commerce, 2002. 4. U.S. Department of Commerce, We the Asian Americans. Washington, DC: U.S. Department of Commerce, 1993. 5. Department of Health and Human Services, Racial and Ethnic Disparities in Health. Washington, DC: Department of Health and Human Services, 1998. 6. D. Satcher, Eliminating racial and ethnic disparities in health: the role of the ten leading health indicators. J. Natl. Med. Assoc. 2000; 92: 315–318. 7. American Cancer Society, Cancer Prevention and Early Detection: Facts and Figures. Atlanta, GA: American Cancer Society, 2003. 8. D. Saslow et al., American Cancer Society guideline for the early detection of cervical neoplasia and cancer. CA Cancer J. Clinicians 2002; 52: 342–362. 9. Department of Health and Human Services, Healthy People 2010. Washington, DC: U.S. Government Printing Office, 2000. 10. R. A. Hiatt et al., Pathways to early cancer detection in the multiethnic population of the San Francisco Bay Area. Health Educ. Quart. 1996; 23: 10–27. 11. M. Kagawa-Singer and N. Pourat, Asian American and Pacific Islander breast and cervical carcinoma screening rates and Healthy

8

12.

13.

14.

15.

16.

17.

18.

19.

20.

21.

22.

23.

COMMUNITY-BASED BREAST AND CERVICAL CANCER CONTROL RESEARCH People 2000 objectives. Cancer 2000; 89: 696–705. R. A. Hiatt et al., Community-based cancer screening for underserved women: design and baseline findings from the Breast and Cervical Cancer Intervention Study. Prevent. Med. 2001; 33: 190–203. R. A. Hiatt and R. J. Pasick, Unsolved problems in early breast cancer detection: focus on the underserved. Breast Cancer Res. Treat. 1996; 40: 37–51. J. Legler et al., The effectiveness of interventions to promote mammography among women with historically lower rates of screening. Cancer Epidemiol. Biomark. Prevent. 2002; 11: 59–71. B. A. Israel et al., Review of community-based research: assessing partnership approaches to public health. Annu. Rev. Public Health 1998; 19: 173–202. P. M. Lantz et al., Can communities and academia work together on public health research? Evaluation results from a community-based participatory research partnership in Detroit. J. Urban Health 2001; 78: 495–507. University of Washington School of Public Health and Community Medicine. (2003). Community-based research principles. (online). Available: http://sphcm.washington. edu/research/community.htm. T. K. Lam et al., Encouraging VietnameseAmerican women to obtain Pap tests through lay health worker outreach and media education. J. Gen. Intern. Med. 2003; 18: 516–524. S. H. Yu and W. T. Lui, Methodologic issues. In: N. W. S. Zane et al., (eds.), Methodologic Issues. Thousand Oaks, CA: Sage Publications, 1994. M. Lee, F. Lee, and F. Stewart, Pathways to early breast and cervical cancer detection for Chinese American women. Health Educ. Quart. 1996; 23: 76–88. A. E. Maxwell, R. Bastani, and U. S. Warda, Breast cancer screening and related attitudes among Filipino-American women. Cancer Epidemiol. Biomark. Prevent. 1997; 6: 719–726. A. E. Maxwell, R. Bastani, and U. S. Warda, Mammography utilization and related attitudes among Korean-American women. Women Health 1998; 27: 89–107. A. E. Maxwell, R. Bastani, and U. S. Warda, Demographic predictors of cancer screening among Filipino and Korean immigrants in the United States. Amer. J. Prevent. Med. 2000; 18: 62–68.

24. A. E. Maxwell, R. Bastani, P. Vida, and U. S. Warda, Results of a randomized trial to increase breast and cervical cancer screening among Filipino American women. Prevent. Med. 2003; 37: 102–109. 25. B. C. K. Choi et al., Use of surnames to identify individuals of Chinese ancestry. Amer. J. Epidemiol. 1993; 138: 723–734. 26. B. K. Hage et al., Telephone directory listings of presumptive Chinese surnames: an appropriate sampling frame for a dispersed population with characteristic surnames. Epidemiology 1990; 1: 405–408. 27. D. S. Lauderdale and B. Kestenbaum, Asian American ethnic identification by surname. Population Res. Policy Rev. 2000; 19: 283–300. 28. A. Nicoll, K. Bassett, and S. J. Ulijaszek, What’s in a name? Accuracy of using surnames and forenames in ascribing Asian ethnic identity in English populations. J. Epidemiol. Community Health 1986; 40: 364–368. 29. E. Y. Tjam, How to find Chinese research participants: use of a phonologically based surname search method. Can. J. Public Health 2001; 92: 138–142. 30. T. Nguyen et al., Predictors of cervical Pap smear screening awareness, intention, and receipt among Vietnamese-American women. Amer. J. Prevent. Med. 2002; 23: 207–214. 31. V. M. Taylor et al., Cervical cancer screening among Cambodian-American women. Cancer Epidemiol. Biomark. Prevent. 1999; 8: 541–546. 32. S. J. McPhee et al., Pathways to early cancer detection for Vietnamese women: Suc Khoe La Vang! (Health is gold!). Health Educ. Quart. 1996; 23: 60–75. 33. C. N. Jenkins et al., Effect of a media-led education campaign on breast and cervical cancer screening among Vietnamese-American women. Prevent. Med. 1999; 28: 395–406. 34. J. A. Bird et al., Opening pathways to cancer screening for Vietnamese-American women: lay health workers hold a key. Prevent. Med. 1998; 27: 821–829. 35. T. G. Hislop et al., Facilitators and barriers to cervical cancer screening among Chinese Canadian women. Can. J. Public Health 2003; 94: 68–73. 36. V. M. Taylor et al., Pap testing adherence among Vietnamese American women. Cancer Epidemiol. Biomark. Prevent. 2004; 13: 613–619.

COMMUNITY-BASED BREAST AND CERVICAL CANCER CONTROL RESEARCH 37. V. M. Taylor et al., Cervical cancer screening among Chinese Americans. Cancer Detect. Prevent. 2002; 26: 139–145. 38. B. A. Wismer et al., Interim assessment of a community intervention to improve breast and cervical cancer screening among Korean American women. J. Public Health Manag. Pract. 2001; 7: 61–70. 39. V. M. Taylor et al., A randomized controlled trial of interventions to promote cervical cancer screening among Chinese women in North America. J. Natl. Cancer Inst. 2002; 94: 670–677. 40. T. Hoare et al., Can the uptake of breast screening by Asian women be increased? A randomized controlled trial of a linkworker intervention. J. Public Health Med. 1994; 16: 179–185. 41. M. Kagawa-Singer, Improving the validity and generalizability of studies with underserved US populations expanding the research paradigm. Ann. Epidemiol. 2000; 10: S92–S103. 42. J. Eyton and G. Neuwirth, Cross-cultural validity: ethnocentrism in health studies with special reference to the Vietnamese. Social Sci. Med. 1984; 18: 447–453. 43. R. J. Pasick et al., Problems and progress in translation of health survey questions: the Pathways experience. Health Educ. Quart. 1996; 23: S28–S40. 44. S. P. Tu et al., Translation challenges of crosscultural research and program development. Asian Amer. Pacif. Islander J. Health 2003; 10: 58–66. 45. M. R. McBride et al., Factors associated with cervical cancer screening among Filipino women in California. Asian Amer. Pacif. Islander J. Health 1998; 6: 358–367. 46. S. J. McPhee et al., Validation of recall of breast and cervical cancer screening by women in an ethnically diverse population. Prevent. Med. 2002; 35: 463–473. 47. R. J. Pasick et al., Quality of data in multiethnic health surveys. Public Health Reports 2001; 116: 223–243. 48. V. M. Taylor et al., Evaluation of an outreach intervention to promote cervical cancer screening among Cambodian American women. Cancer Detect. Prevent. 2002; 26: 320–327. 49. B. R. McAvoy and R. Raza, Can health education increase uptake of cervical smear testing among Asian women? Brit. J. Med. 1991; 302: 833–386.

9

50. S. J. McPhee, Promoting breast and cervical cancer screening among Vietnamese American women: two interventions. Asian Amer. Pacif. Islander J. Health 1998; 6: 344–350. 51. S. J. McPhee and T. T. Nguyen, Cancer, cancer risk factors, and community-based cancer control trials in Vietnamese Americans. Asian Amer. Pacif. Islander J. Health 2000; 8: 18–31. 52. T. Nguyen et al., Promoting early detection of breast cancer among Vietnamese-American women. Results of a controlled trial. Cancer 2001; 91: 267–273. 53. M. S. Chen, Jr. et al., Implementation of the indigenous model for health education programming among Asian minorities: beyond theory and into practice. J. Health Educ. 1992; 23: 400–403. 54. M. S. Chen, Jr., Cardiovascular health among Asian Americans/Pacific Islanders: an examination of health status and intervention approaches. Amer. J. Health Promot. 1993; 7: 199–207. 55. M. S. Chen, Jr. et al., An evaluation of heart health education for Southeast Asians. Amer. J. Health Promot. 1994; 10: 205–208. 56. L. Rychetnik et al., Criteria for evaluating evidence on public health interventions. J. Epidemiol. Community Health 2002; 56: 119–127. 57. L. Suarez et al., Why a peer intervention program for Mexican American women failed to modify the secular trend. Amer. J. Prevent. Med. 1997; 13: 411–417. 58. S. J. Curry and K. M. Emmons, Theoretical models for predicting and improving compliance with breast cancer screening. Ann. Behav. Med. 1994; 16: 302–316. 59. F. A. Hubbell et al., From ethnography to intervention: developing a breast cancer control program for Latinas. J. Natl. Cancer Inst. Monographs 1995; 109–115. 60. J. C. Jackson et al., Development of a cervical cancer control intervention program for Cambodian American women. J. Community Health 2000; 25: 359–375. 61. J. C. Jackson et al., Development of a cervical cancer control intervention for Chinese immigrants. J. Immigrant Health 2002; 4: 147–157. 62. B. I. Truman, J. S. Wing, and N. L. Keenan, Asians and Pacific Islanders. In: D. Satcher et al., (eds.), Chronic Disease in Minority Populations. Atlanta, GA: Centers for Disease Control, 1994.

10

COMMUNITY-BASED BREAST AND CERVICAL CANCER CONTROL RESEARCH

Table 3. Community-based Breast and Cervical Cancer Intervention Studies Breast and/or cervical cancer screening

Group

Taylor, 2002

Cervical

Taylor, 2002

Cervical

First author, year of publication

Assignment unit

Intervention strategies

Evaluation method(s)

Cambodian

Small area neighborhood

Survey 12 months after randomization Review of medical records

No effect

Chinese

Individual

Home visit by outreach worker Group education session Use of video Logistic assistance Group 1: Home visit by outreach worker Use of video and print materials Logistic assistance

Survey six months after randomization Review of medical records

Both interventions effective Outreach worker intervention more effective than direct mailing intervention

Survey 12 months after randomization

No overall effect for mammography or Pap testing Effective for mammography in recent immigrant sub-group No effect

Maxwell, 2003

Both

Filipina

Small group

Wismer, 2002

Both

Korean

Community

McAvoy, 1991

Cervical

South Asian

Individual

Hoare, 1994

Breast

South Asian

Individual

Group 2: Direct mailing of video and print materials Group education session Use of print materials

Workshops delivered by lay health workers Distribution of print materials Group 1: Home visit by outreach worker Use of video

Group 2: Home visit by outreach worker Use of print materials Group 3: Direct mailing of print materials Home visit by outreach worker

Pre- and postintervention cross-sectional surveys

Main finding(s)

Review of screening program computerized records four months after randomization

Home visits were effective

Review of screening program computerized records

No effect

COMMUNITY-BASED BREAST AND CERVICAL CANCER CONTROL RESEARCH

11

Table 3. (continued) Breast and/or cervical cancer screening

Group

Assignment unit

Intervention strategies

Evaluation method(s)

Bird, 1998

Both

Vietnamese

Community

Pre- and postintervention cross-sectional surveys

Effective for mammography and Pap testing

Jenkins, 1999

Both

Vietnamese

Community

Series of three group education sessions delivered by lay health workers Distribution of print materials Media campaign Distribution of audio-visual and print materials

Pre-and postintervention cross-sectional surveys

Nguyen, 2001

Breast

Vietnamese

Community

No effect on mammography or Pap testing behavior Increased mammography and Pap testing intentions No effect

First author, year of publication

Media campaign Group education sessions Distribution of audio-visual and print materials

Pre- and postintervention cross-sectional surveys

Main finding(s)

COMPLIANCE AND SURVIVAL ANALYSIS

assignment have no impact on outcome, is consistently tested irrespective of compliance levels. Under the alternative, we expect, however, different (smaller) intent-to-treat effects than the prescribed regime would create when it materializes. This happens as the treatment group becomes a mix of varying (lower) degrees of exposure (2,8). Estimation of the causal effect of actual dose timing becomes challenging, when observed exposure patterns are no longer randomized. (Un)measured patient characteristics and earlier experience may determine exposure levels that become confounded with the natural treatment-free hazard of the patient. The association between compliance which induces treatment exposure levels, and treatment-free hazards is often called a selection effect in line with missing data terminology. An ‘‘as-treated’’ analysis, such as a PH analysis, with the currently received treatment as a time-dependent covariate, compares hazards between differently treated groups at a given time and thus estimates a mix of selection and causal effects (11). An ‘‘on-treatment’’ analysis censors patients as soon as they go off the assigned treatment and thus generates informative censoring for the same reason. Structural accelerated failure time (SAFT) models and structural PH models have been designed to avoid these biases. We explain their key features and potential through a simple example first.

ELS GOETGHEBEUR Ghent University, Ghent, Belgium

1

COMPLIANCE: CAUSE AND EFFECT

Today, new treatments must prove their worth in comparative (double blind) randomized clinical trials, the gold standard design for causal inference. With noninformatively right-censored survival outcomes, a typical robust intention-to-treat analysis compares groups as randomized using the popular (weighted) logrank test. Accompanying Kaplan–Meier, curves describe nonparametrically how survival chances differ between arms. A one-parameter summary of the contrast follows from a semiparametric Cox proportional hazards (PH) model Accelerated Failure-Time Model (6). In general, and especially with long-term treatments, patients tend to deviate from their prescribed treatment regime. Varying patterns of observed exposure relative to the assigned are called ‘‘compliance (levels)’’ and recognized as a likely source of variation in treatment effect. Because deviations from prescribed regimes occur naturally in clinical practice, it is wise to learn about them within the trial context rather than restrict the study population to perfect compliers, an atypical and sometimes small and unobtainable subset of the future patient horizon (12). Treatments that are stopped or switched or are less dramatic lapses in dosing happen in response to a given assignment. Different exposure patterns between treatment arms therefore point to (perceived) differences following alternative assignments. Studying compliance patterns as an outcome can yield valuable insights (15). Of course, actual treatment regimes may also influence primary outcomes. From the intent-to-treat perspective, underdosing causes reduced power and a requirement for larger samples. Fortunately, the strong null hypothesis, where treatment and its

2

ALL-OR-NOTHING COMPLIANCE

In randomized studies that evaluate a oneshot treatment, such as surgery (7), vaccination (4), or an invitation to undergo screening (1), all-or-nothing compliance with experimental assignment arises naturally. Let Ri = 1(0) indicate whether individual i, with baseline covariates Xi , gets randomized to the experimental (control) arm. When experimental treatment is not available outside the treatment arm the control arm remains

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

COMPLIANCE AND SURVIVAL ANALYSIS

uncontaminated by experimental exposure and its outcomes can serve as reference outcomes for causal inference. Let T 0i denote such a potential survival time for individual i following a control trajectory free of experimental exposure. Let T 1i and Ci respectively be survival time and compliance following a possible treatment assignment. Ri operates independently of the vector (T 0i , T 1i , Ci , Xi ) but determines, which components are observed. Observed survival time and exposure are simply denoted T i , Ei for all. With an uncontaminated control arm, Ei = Ci Ri . One goal of causal inference is to estimate how the contrast between potential survival times T 1i and T 0i varies over the subpopulations determined by different Ci levels and their induced level of experimental exposure on the treatment arm, Ei . The sharp null hypothesis assumes it makes no difference what arm one is assigned to and hence: d|X T0i = i T1i ,

(1)

d|X where = i indicates equality in distribution conditional on Xi . The most obvious violation of (1) occurs when (some) patients on the experimental arm become exposed and exposure alters survival chances. This is called a direct causal effect of exposure (10). When an assignment influences survival through mechanisms of action operating independently from exposure levels, we have an indirect effect. Below, we consider a cancer clinical trial (7), where the experimental intervention consists of implanting an arterial device during surgical resection of metastases. A planned implant could lead to an operation scheduled earlier in the day and timing may create its own set of prognostic circumstances. In addition, the news that a planned implant did not happen could be depressing to the patient and diminish survival chances beyond what would have happened on the control arm. Both mechanisms can lead to an indirect (clinical) effect of exposure assignment. Double blind studies are carefully designed to avoid indirect {d|C =0, X } effects, so they satisfy: T0i i = i T1i and hence, P(T 1i > t—Ci = 0, Ri = 1, Xi ) = P(T 0i > t|Ci = 0, Ri = 0, Xi ), for all t. The contrast

between P(T 1i > t|Ci = e, Ri = 1, Xi ) and P(T 0i > t|Ci = e, Ri = 0, Xi ) then represents the causal effect of exposure level e. In general, however, this combines direct and indirect effects of assignment in the population with compliance level e. In what follows, we ignore Xi for simplicity, but stronger inference can be drawn when assumptions condition on Xi . To estimate P(T 0i > t|Ci = 1, Ri = 1), one can solve P(T 0i > t|Ci = 1|Ri = 1)P(Ci = 1, Ri = 1) + P(T 0i > t|Ci = 0, Ri = 1)P(Ci = 0|Ri = 1) = (P(T 0i > t|Ri = 1) =)P(T 0i > t|Ri = 0) after substituting empirical means or (Kaplan–Meier) estimates for the other unknown terms. Isotonic regression can turn the pointwise estimates in a monotone survival curve. To evaluate the treatment effect among the exposed, one comˆ 0i
t|Ci = ˆ 1i
t|Ci = 1, Ri = 1) with P(T pares P(T 1, Ri = 1). The selective nature of exposure is seen by contrasting treatment-free survival probabilities for the exposed and nonexˆ 0i
t|Ci = 1, Ri = 1) posed subpopulations: P(T ˆ 0i
t|Ci = 0, Ri = 1). and P(T 3 MORE GENERAL EXPOSURE PATTERNS A structural model parameterizes the shift in distribution from observed survival time T i to a reference time, T ei following a specific exposure regime e, in terms of observed (possibly time-dependent) exposures Ei and covariates Xi . One can thus perform parameter-specific (Ei , Xi )-dependent back transformations of observed survival times (or distributions). The parameter value that solves estimating equations demanding equality of estimated T ei distributions (conditional on baseline covariates) between arms is our point estimate. The procedure is illustrated in Figure 1 d|R

for the SAFT model Ti exp{−β0 Ei } = i T0i in our trial, where Ei indicates an actual implant of the arterial device. For timedependent implants Ei (t), we could have used
T d|R the SAFT model 0 i exp(−β0 Ei (u))du = i T0i . For technical points concerning the specific treatment of censored data, we refer the reader to (5,12). The left-hand panel shows ITT Kaplan–Meier curves in the standard and intervention arm. In the right-hand panel, the survival curve for the standard

COMPLIANCE AND SURVIVAL ANALYSIS

3

Figure 1. Estimation of structural parameters

arm is compared with KM-curves following the transformations T i exp{−βEi } with β = −1.5 and β = −0.36 on the intervention arm. Reducing treated failure times by the factor exp(−1.5) overcompensates for the observed harmful treatment effect as survival chances on the intervention arm are now higher than on the standard arm. This is confirmed by the logrank chi-squared value of 9.326, plotted in the middle panel. The survival curve corresponding to the point ˆ = 70%) is conestimate βˆ = −0.36 (exp(β) vincingly close to the observed survival in the standard arm. The middle panel reveals chisquared values for a range of hypothesized structural parameter values. Those that do not lead to significantly different curves at the 5% level form the 95% confidence interval [−1.07, 0.39] for β 0 . 4 OTHER STRUCTURAL MODELING OPTIONS One can propose many different maps of the observed into the treatment-specific survival distributions. This may happen on a PH scale

(7) or involve time-dependent measures of effect in the SAFT setting (12). Estimation methods, which rely on the instrument of randomization, protect the α –level just like the intent-to-treat test, but shift the point estimate away from a diluted average. To achieve this, they rely on the postulated structural model, which can sometimes be rejected by the data, but generally not confirmed owing to a lack of observed degrees of freedom. Special care is thus required when interpreting these models and their results. Some diagnostic procedures have been proposed and forms of sensitivity analyses (13,14). To explicitly account for measured timedependent confounders, structural nested failure-time models can be used as an alternative, or marginal structural models for T ei as in (3). The estimation process then relies on the assumption of ‘‘no residual confounding’’, ignores the instrument Ri , and loses its robust protection of the α –level. Structural modeling of failure time distributions has opened a world of practical and theoretical developments for the analysis of compliance and survival time. The field of

4

COMPLIANCE AND SURVIVAL ANALYSIS

research is very much alive today. Recent work (9), for instance, proposes to estimate optimal treatment regimes from compliance data. Our brief account can give but a flavor of this wealth.

5

ACKNOWLEDGMENTS

We thank Tom Loeys for drawing the figure.

REFERENCES 1. Baker, S. G. (1999). Analysis of Survival data from a randomized trial with all-or-nothing compliance: estimating the cost-effectiveness of a cancer screening program, Journal of the American Statistical Association 94, 929–934. 2. Frangakis, C. E. & Rubin, D. B. (1999). Addressing complications of intention-totreat analysis in the combined presence of all-or-none treatment-noncompliance and subsequent missing outcomes, Biometrika 80, 365–379. 3. Hernan, M. A., Brumback, B. & Robins, J. M. (2001). Marginal structural models to estimate the joint causal effect of nonrandomized treatments, Journal of the American Statistical Association 96, 440–448. 4. Hirano, K., Imbens, G., Rubin, D. & Zhou, X. H. (2000). Assessing the effect of an influenza vaccine in an encouragement design, Biostatistics 1, 69–88. 5. Joffe, M. M. (2001). Administrative and artificial censoring in censored regression models, Statistics in Medicine 20, 2287–2304. 6. Kalbfleisch, J. D. & Prentice, R. L. (2002). The Statistical Analysis of Failure Time Data, 2nd Ed. Wiley, New Jersey, Hoboken. 7. Loeys, T. & Goetghebeur, E. (2003). A causal proportional hazards estimator for the effect of treatment actually received in a randomized trial with all-or-nothing compliance, Biometrics 59, 100–105. 8. Mark, S. D. & Robins, J. M. (1993). A method for the analysis of randomized trials with compliance information: an application to the multiple risk factor intervention trial, Controlled Clinical Trials 14, 79–97. 9. Murphy, S. (2003). Optimal dynamic treatment regimes, Journal of the Royal Statistical Society, Series B 65, 331–355.

10. Pearl, J. (2001). Causal inference in the health sciences: a conceptual introduction, Health Services and Outcomes Research Methodology 2, 189–220. 11. Robins, J. M. & Greenland, S. (1994). Adjusting for differential rates of PCP prophylaxis in high-versus low-dose azt treatment arms in an aids randomized trial, Journal of the American Statistical Association 89, 737–749. 12. Robins, J. M. & Tsiatis, A. A. (1991). Correcting for non-compliance in randomized trials using rank preserving structural failure time models, Communications in Statistics, A 20, 2609–2631. 13. Scharfstein, D., Robins, J. M., Eddings, W. & Rotnitzky, A. (2001). Inference in randomized studies with informative censoring and discrete time-to-event endpoints, Biometrics 57, 404–413. 14. White, I. & Goetghebeur, E. (1998). Clinical trials comparing two treatment arm policies: which aspects of the treatment policies make a difference? Statistics in Medicine 17, 319–340. 15. White, I. & Pocock, S. (1996). Statistical reporting of clinical trials with individual changes from allocated treatment, Statistics in Medicine 15, 249–262.

CROSS-REFERENCES Noncompliance, Adjustment for Survival Analysis, Overview

COMPOSITE ENDPOINTS IN CLINICAL TRIALS

of type I error). Thus, the study is judged to have achieved statistically significant evidence of efficacy and to be a positive trial only if the test for the primary endpoint achieves the prespecified level of significance. The approach of relying on a single primary endpoint enables the investigators to control the overall probability of falsely declaring the experimental therapy to be efficacious, and it assists in the interpretation of analyses of the secondary endpoints, which will be helpful in delineating the nature of any treatment effect when the primary endpoint is significant. If the primary endpoint is not statistically significant, analyses of secondary endpoints are not used to definitively establish efficacy, but they may provide insights about treatment effects as well as information that may be helpful in designing future clinical trials. In performing secondary analyses, one must be concerned about the problem of multiple testing—that one or more secondary endpoints might reach statistical significance by chance. Because the overall error rate for the trial is controlled by specifying the primary endpoint, the usual approach to minimizing the problems with multiple testing is to limit the group of secondary endpoints to a relatively small number of the most important endpoints. Others are classified as tertiary endpoints. Because specification of the primary endpoint is critically important in the design of a clinical trial, it deserves serious consideration, a review of the relevant literature, and perhaps even preliminary studies. Five desirable properties of a primary endpoint have been proposed (1): it should be (1) clinically relevant, (2) accurate, (3) reproducible, (4) objective, and (5) quantitative. In practice, specification of a primary endpoint will also need to consider the nature of the disease and the manifestations under study as well as the availability of patients, investigators, and resources. The choice of the components of composite endpoint also should be related to the primary question of interest (2). The use of a composite endpoint, defined a priori, that comprises a variety of patient characteristics is often better suited to accom-

PETER C. O’BRIEN Division of Biostatistics Mayo Clinic College of Medicine Rochester, Minnesota

BARBARA C. TILLEY Department of Biostatistics, Bioinformatics, and Epidemiology Medical University of South Carolina Charleston, South Carolina

PETER J. DYCK Peripheral Nerve Lab Mayo Clinic College of Medicine Rochester, Minnesota

Composite endpoints have become increasingly useful as a measure of a patient’s disease status. They have been found useful in both clinical practice and medical research. Here, we shall examine the usefulness of composite endpoints as a primary endpoint in clinical trials. The basic rationale for their use is discussed and illustrated with examples, followed by a variety of procedures for forming composites including how they should be interpreted. In the section on global assessment variables, alternate approaches to conducting a primary analysis with multiple endpoints are considered and these methods are compared with the use of composite endpoints. 1 THE RATIONALE FOR COMPOSITE ENDPOINTS We consider the outcome measures used in the typical phase III clinical trial for efficacy, for simplicity assuming that an experimental therapy is being compared with a placebo. Most commonly, a protocol for such a study specifies a single primary endpoint (see the discussion of primary endpoints), various secondary endpoints (see the discussion of secondary endpoints), and often additional tertiary endpoints. The goal of categorizing study endpoints in this manner is to control the type I error rate (see the discussion

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

COMPOSITE ENDPOINTS IN CLINICAL TRIALS

plishing the objectives. In particular, by providing a more comprehensive assessment of the patient, it may be able to achieve a much higher level of clinical relevance. These considerations are illustrated with examples, including a detailed example involving clinical trials of drugs to treat diabetic neuropathy. 2 FORMULATION OF COMPOSITE ENDPOINTS Next, some useful methods for defining a composite endpoint for use as a primary endpoint in a clinical trial are considered. 2.1 Existing Scoring Systems Ideally, a suitable comprehensive composite endpoint has already been identified and extensively studied, such as the composite scores for diabetic sensory polyneuropathy. These types of composites seek to cumulate information about qualitatively different patient characteristics, all of which are directly relevant to efficacy. A commonly used type of composite endpoint consists of scores assigned to each of numerous individual questions, which are then summated to obtain the composite score. Many such scoring systems have been developed, validated, and widely used in clinical applications as well as in clinical trials. Their usefulness may have been studied by a group of experts in the field, and the appropriate use and interpretation may have been delineated by a consensus statement that will indicate what constitutes a meaningful effect size, among other things. The availability of such an existing composite endpoint that adequately measures patient status for the purposes of the specific clinical trial under consideration may be viewed as the ideal. If such a scoring system is available, a statistical approach for defining a composite endpoint is not needed. 2.2 Methods Based on Rankings In many situations, no well accepted scoring systems for combining variables are available. It may also be considered more meaningful to weight each of the endpoints equally

and without regard to their units of measure. A rank sum scoring approach (see the discussion of global assessment variables) may be useful in these situations. Specifically, one ranks all the subjects from best to worst separately for each variable. One then computes the sum of the ranks received for each subject to obtain rank sum scores measuring the relative performance of each subject. The relative rankings obtained are helpful in assessing the clinical relevance of the treatment effect and can be used to estimate the probability that the experiment will produce a better result than placebo (again, see the discussion of global assessment variables). 2.3 Methods for Endpoints That Are All Dichotomous or Censored In many trials, all the individual endpoints are censored. Cardiology trials, for example, are often interested in the time to any of several cardiac related events. An approach is to consider the time to the first event as the composite endpoint. Similarly, if all the variables are binary, a composite binary endpoint can be defined as the occurrence of any of the individual events. 3 EXAMPLES 3.1 Clinical Trials in Diabetes We can illustrate the use of composite versus individual measurement with the condition diabetic sensory polyneuropathy (DSPN) (3), which we have studied in detail in a crosssectional and longitudinal study (the Rochester Diabetic Neuropathy Study) and in several controlled clinical trials. Intuitively, we observe that DSPN is the sum total of a patient’s neuropathic symptoms; neurologic signs of weakness, sensory loss (different modalities), and autonomic nerve dysfunctions as well as overall impairments and disability. To track all of these diverse manifestations and appropriately assign a relative worth to each component is not presently feasible. Use of every measure might also be too intrusive, time consuming, and expensive. The problem is even more complex because a variety of composite scores are available (e.g., attributes of nerve conduction, neurologic

COMPOSITE ENDPOINTS IN CLINICAL TRIALS

signs, sensation tests, and so on). Without going into great detail, we found that no one attribute of nerve conduction or single clinical sign adequately represented the condition DSPN or was a suitable marker for its presence or severity. For controlled trials focusing on the development of DSPN, it is possible to use a single criterion such as that chosen for conduct of the Diabetes Control and Complications Trial study (4), which compared rigorous versus conventional glycemic control. The criterion for DSPN was: ≥2 abnormalities from among these three: (1) decreased or absent ankle reflexes, (2) decreased or absent vibration sensation of great toes, or (3) symptoms of polyneuropathy not attributable to other neuropathies. Although this score performed well for this purpose, it provided information only about frequency and not about severity of DSPN. For many purposes, knowledge about severity of DSPN would provide additional important information. We have developed several composite and continuous measures of severity
of DSPN. One such composite score is 5 NC nds. In this score, five attributes of nerve conduction (NC) of leg—chosen because they are distal nerves, representing different functional groups—are each expressed as a normal deviate (nd), from percentiles corrected for age, sex, height, and weight based on a study of healthy subjects and with all abnormalities expressed in the upper tail of the normal distribution, which are summed, divided by the number of measurable attributes, and multiplied by 5. We have found that this summated score can be used to provide an excellent measure not only of the presence but also the severity of polyneuropathy because it tracks severity of DSPN without a floor or ceiling effect, is sensitive at defined levels of specificity, is very reproducible, and correlates reasonably with clinical neurologic deficit. It has the advantage of being objective (patients cannot will the result). This measure has also been shown to have good monotonicity and is highly suitable for controlled trials. It is also our impression that
5 NC nds might have a low degree of inter–medical center variability when used by certified electromyographers whose training and choice of equipment would likely

3

result in a high degree of reproducibility of results. However, for judging severity of DSPN, other primary outcome measures might be used. For example, if the emphasis is to be on the effect on neurologic signs (i.e., leg weakness, reflex loss, or sensation loss), a summed score of neurologic signs (e.g., neuropathy impairment score of lower limb), or
summated quantitative sensation tests ( QST nds) might be used. Scales that judge the severity of a patient’s symptoms are needed when the drug to be tested has its putative action on the relief of symptoms (e.g., pain). Thus, in conditions such as headache, trigeminal or other neuralgias, fatigue states, anxiety, and the like, if composite clinical endpoints are chosen, these outcomes should include a measure of the frequency, duration, and severity of symptoms. In contrast to use of impairment measures in which an external physician or scientific observer makes observations, the patient’s judgment of quality of life, impairment for acts of daily living, or disability may also need to be considered. Scales of acts of daily living and quality of life are used to represent the patient’s perception of the impact of his or her illness on performance of life tasks and quality of life and whether an intervention improves the dysfunction or disability (5). Scoring of a single or a battery of motor functions or acts of daily living may also be used to assess overall severity of polyneuropathy. This approach has been extensively explored in assessment of therapy in immune polyneuropathies (6).

3.2 Clinical Trials in Cardiovascular Disease In cardiovascular disease (CVD) and other diseases where treatment is expected to affect both morbidity and mortality, it is common to choose an outcome representing failure of any one of a set of poor outcomes occur. For example, in assessing the benefits of aspirin for CVD prevention, outcomes such as vascular death, myocardial infarction, or major stroke; or death, myocardial infarction, or stroke; or death or reinfarction have been used as composite outcomes (7).

4

COMPOSITE ENDPOINTS IN CLINICAL TRIALS

3.3 Clinical Trials in Rheumatoid Arthritis The American College of Rheumatology criteria (ACR) for treatment success have been developed as a composite outcome. The ACR criteria were based on achieving success on at least some of the outcomes of interest. The composite outcome has been validated in a variety of settings and has been modified as background treatment has improved (8). 3.4 Other Comments In general, one can categorize the situations where multiple endpoints are needed to measure patient status in two ways: (1) each endpoint measures a qualitatively different patient attribute versus providing alternate measures of the same attribute, and (2) each endpoint considered by itself is sufficiently important that it could be used as a primary endpoint (but is not as informative and meaningful as the multiple measures taken together) versus a situation where one or more of the endpoints taken alone would not be sufficiently meaningful to provide a primary endpoint by itself but nonetheless adds important information in arriving at an overall assessment of the patient. Although these distinctions are helpful, they may be difficult to make in practice. For example, one might consider electromyographic measures of nerve conduction and measures of sensation to be assessing different attributes. However, for both endpoints, it is possible to distinguish whether the function of small or large nerve fibers is being assessed. Therefore, it could be argued that if both endpoints are evaluating small fibers then they are measuring the same attribute. The distinctions mentioned above also would be important in a disease that has qualitatively different manifestations. For example, Hunter syndrome affects primarily lung function in some patients and primarily ambulation in others, so focusing on only one of these attributes with a single primary endpoint would not provide a clinically comprehensive and meaningful assessment. Although the primary goal of a composite endpoint is to obtain the most clinically meaningful measure of patient status with which to measure efficacy, a more comprehensive measure will typically also have the

desirable statistical property of increasing the power of the trial. Combining multiple endpoints to provide a more comprehensive assessment of the patient may also be helpful in evaluating treatment safety. 4 INTERPRETING COMPOSITE ENDPOINTS The appropriate interpretation of a composite endpoint depends on how it was formed and its intended use. The most straightforward situation is the case of existing scoring systems that have been validated and widely used in clinical practice. In this case, there is a clear understanding of the clinical relevance of the magnitude of group differences observed in the trial. If the composite consists of a summation of many items, reporting of trial results often focuses only on the overall score and does not examine the individual components. This is unfortunate because one would expect that important additional information may be gleaned from determining whether the treatment effects were seen in only a few of the individual variables (in which case, the nature of any clustering may be important) or whether the effects were displayed consistently across all variables. One way to approach the interpretation of the composite score is to focus on inspection of the individual components. For example, with relatively few individual variables, it may be possible to prespecify what constitutes a clinically meaningful effect size for each endpoint. Individual statistical tests accompanying each endpoint may further assist in interpretation. Because the overall error rate has been controlled by the test for the composite endpoint, adjustment for these individual tests is not needed. Even if testing of individual outcomes is not conducted, at a minimum the results for each of the components of the composite outcome should be reported. As Wong (9) has commented, a review article on aspirin and CVD (7) that reports on only the composite outcome and myocardial infarction could mask the relationships between aspirin and stroke; if the individual outcomes as well as the composite outcome had been reported, this concern would have been addressed.

COMPOSITE ENDPOINTS IN CLINICAL TRIALS

Strategies for making decisions about clinical relevance with composite endpoints depend on the context. For example, if the overall test is significant and at least one of the endpoints reaches a clinically meaningful threshold, should the corresponding statistical test associated with that endpoint also be required to reach statistical significance? In our view, this depends on the circumstances. Alternatively, the overall test for the composite endpoint might reach statistical significance (demonstrating that the experimental treatment has a real beneficial effect), but none of the individual endpoints may reach the threshold for clinical importance or statistical significance. This could be likely to happen when the composite outcome is defined as the occurrence of any one of several events. Should the magnitude of the treatment effect be considered not clinically meaningful, or might the nature of the effect be diffuse but sufficiently consistent across endpoints that it cumulates to something clinically meaningful? Again, this depends on the circumstances, and clinical judgment is most important for arriving at a conclusion. It may be particularly important to examine the treatment effect on individual endpoints if the composite is the time to first event or the occurrence of any event because the type of events that occur soonest or most frequently may have an outsize effect on the composite. 5

CONCLUSIONS

Composite endpoints are valuable in clinical trials of efficacy, primarily because they provide a comprehensive and clinically meaningful assessment of the patient. The efficient use of multiple sources of information may also result in an increase in power over a single outcome. REFERENCES 1. P. C. O’Brien, Commentary to: A clinical trial endpoint based on subjective rankings, by Follmann D, Wittes J and Cutler JA. Stat Med. 1992; 11: 447–449. 2. N. Freemantle, M. Calvert, J. Wood, J. Eastaugh, and C. Griffen, Composite outcomes

5

in randomized trials: greater precision but with greater uncertainty? JAMA. 2003: 289: 2554–2559. 3. DCCT Research Group. The Diabetes Control and Complications Trial (DCCT): design and methodologic considerations for the feasibility phase. Diabetes. 1986; 35: 530–545. 4. DCCT Research Group. The effect of intensive treatment of diabetes on the development and progression of long-term complications in insulin-dependent diabetes mellitus. N Engl J Med. 1993; 329: 977–986. 5. P. J. Dyck and P. C. O’Brien, Polyneuropathy dysfunction scores. J Neurol Neurosurg Psychiatry. 2006; 77: 899–900. 6. Graham RC, Hughes RAC. A modified peripheral neuropathy scale: the Overall Neuropathy Limitations Scale. J Neurol Neurosurg Psychiatry. 2006; 77: 973–976. 7. C. L. Campbell, S. Smyth, G. Montalescot, and S. Steinhubl, Aspirin dose for the prevention of cardiovascular disease: a systematic review. JAMA. 2007. 297; 2018–2024. 8. D. T. Felson, J. J. Anderson, M. L. Lange, G. Wells, and M. P. LaValley, Should improvement in rheumatoid arthritis clinical trials be defined as fifty percent or seventy percent improvement in core set measures, rather than twenty percent? Arthritis Rheum. 1998; 41: 1564–1570. 9. G. K. Wong, Letter to the editor. JAMA. 2007; 298: 625.

COMPUTER-ASSISTED DATA COLLECTION

appearing for in-person data collection as well. Before then, research data from study participants was typically captured on paper forms. Key differences exist between computerbased data collection (or EDC) and traditional paper-based data collection. In a traditional data-collection setting, an interviewer reads question items and response options to the study participant and then captures the responses on paper. Alternatively, participants independently complete paper forms or self-administered questionnaires. Unlike a paper-based form, an EDC system can dynamically tailor question wording on the basis of previously entered responses, implement complex routing logic so that only applicable questions are asked of respondents, and incorporate real-time data edits to ensure that responses are within valid ranges and are consistent with prior responses. Moreover, the computer system can enforce response to questionnaire items (using a ‘‘don’t know’’ or ‘‘refuse to answer’’ response when necessary) to minimize or eliminate missing data. Also unlike paper data, data collected through EDC or computer-assisted interviewing can rapidly be made available to a data management team for quality control and preliminary results. Under typical computerassisted, in-person data collection, data are collected with use of a laptop or desktop computer and then retrieved from the computer as often as desired (generally by secure electronic transmissions initiated from each computer). By contrast, data captured on paper forms are less immediately available because the responses must be subjected to a data-entry and verification process. If the data are collected at multiple locations, there may be the further delay of shipping paper forms to a central data entry location or the added overhead of setting up multiple data entry sites. The advantages of EDC, or computer-assisted interviewing, over traditional penciland-paper methods can be summarized as follows: Reduction in time from last patient visit to database release

TIM J. GABEL , SUSAN D. PEDRAZZANI and ORIN A. DAY Survey and Computing Sciences RTI International Research Triangle Park, North Carolina

Computer-assisted data collection, or electronic data capture (EDC), has been used for more than two decades in the government survey and market research industries but has only in recent years begun gaining broad acceptance in the clinical research setting, which relies heavily on the use of paper-based case report forms (CRFs). In many clinical studies, participants typically use paper forms to provide self-reported data for the study. These forms are then converted to electronic format using data-entry or clinical data management systems suitable for EDC. An ever-increasing number of clinical studies are deploying computerassisted or EDC systems, choosing them over paper-and-pencil data collection as information on the advantages becomes more widely known. Driving this conversion from paper to computer-assisted data collection is the need to collect quality data in a more accurate, efficient, and cost-effective manner. Audio computer-assisted self-interviewing (A-CASI) and telephone audio computerassisted self-interviewing (T-ACASI) are just two of the numerous methods of computerassisted data collection that can be used in clinical trials. A-CASI and T-ACASI may be particularly applicable to the collection of high-quality patient-reported data, especially in Phase IV and postmarketing surveillance studies. 1 DESCRIPTION OF COMPUTER-ASSISTED INTERVIEWING Computer-assisted interviewing for telephone-based data collection has been in use since at least the early 1970s (1). With the introduction of personal computers in the 1980s, computer-assisted methods began

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

COMPUTER-ASSISTED DATA COLLECTION

Reduction in number of required queries (fewer data clarification forms) Faster and easier query resolution Reduction in site monitoring costs (improved and easier monitoring) Reduction in project management costs Efficiency gains from reuse of forms (development of standards library) Ability to perform interim or ad hoc analysis earlier Faster access to data, enabling better-informed decision making Elimination of paper handling Potential for more secure data with use of passwords, data encryption, and limitation of access to database Potential for more complicated questionnaire routing Ability to specify and computerize enforcement of logic checks and range checks Elimination of or reduction in missing data items Despite the advantages of computerassisted interviewing methods over paperbased data collection, however, often the necessary involvement of a live interviewer carries disadvantages that a self-administered instrument avoids. In 1990 Miller et al. (2) summarized some of the methodological challenges in conducting surveys concerned with acquired immunodeficiency syndrome (AIDS). Observing that participation and response were impacted by participants’ concerns about confidentiality, Miller and colleagues reported that ‘‘because of the sensitive and highly personal nature of these questions, virtually all of the surveys made some provision to permit respondents to reveal the details of their sexual behavior without undue embarrassment’’ (2). A-CASI and T-ACASI currently answer this kind of need for private response while retaining the advantages of EDC. 2 AUDIO COMPUTER-ASSISTED SELF-INTERVIEWING In 1994, O’Reilly et al. (3) described the concept of A-CASI as a new technology for

collecting data. With the use of A-CASI, instead of being read questions by an interviewer and responding verbally while the interviewer enters the responses into the computer, the respondent listens in privacy to digitally recorded audio questions delivered to headphones through the sound card of the computer. He or she then enters responses independently, using the computer keyboard or a touch-screen. In this way, an A-CASI instrument temporarily removes the interviewer from the process; in fact, an A-CASI instrument can offer modes in which questions are not even displayed on the screen so that interviews can be conducted in a completely confidential manner in an otherwise less than private setting. Building on the research by O’Reilly and colleagues, Turner et al. (4) reported on the outcomes of an A-CASI experiment embedded in the 1995 National Survey of Adolescent Males. The embedded A-CASI experiment demonstrated statistically significant differences in the reporting of ‘‘sensitive’’ behavior (in this case, male–male sexual contact within a national probability sample of males aged 15 to 19 years) when A-CASI methods were used instead of interviewer-administered survey questions. Since that time, A-CASI has gained widespread acceptance in governmentsponsored social science research. In more recent years, the A-CASI approach has been extended to telephone-based data collection efforts. Health-related studies collecting sensitive data have been implemented using Telephone A-CASI, or T-ACASI. This approach combines interactive voice-response technologies with interviewer-administered questions (5). With T-ACASI, the respondent listens to digitally recorded questions over the telephone and enters his or her response by using a touch-tone telephone keypad. Again, without a clinician or interviewer present to ask the questions or to enter the responses, the study participant can feel more comfortable answering sensitive questions in a confidential setting (6).

COMPUTER-ASSISTED DATA COLLECTION

3 COMPUTER-ASSISTED METHODS IN CLINICAL TRIALS These computer-assisted data collection methods apply especially well to the clinical research arena. The more prevalent EDC becomes in clinical trials, the more likely these methods will gain acceptance in the clinical community. However, careful consideration and planning regarding A-CASI and T-ACASI approaches are crucial for effective implementation and study outcomes. 3.1 Questionnaire Design Most clinical researchers recognize the importance of devoting adequate time and resources to the development of the questionnaire or clinical forms, but this is especially true for studies incorporating A-CASI or T-ACASI methods. Because these approaches involve self-administration by study participants who interact with a computer, all questionnaire items must be clear and understandable to the participants. We recommend that a forms design team develop and test the questions with the target population in mind, giving especially careful consideration to those items that will be self-administered. To design and test self-administered questions for requisite clarity, the design team must first assess study participant characteristics such as age, literacy, computer experience or access to a touch-tone telephone, culture, language, and disease or condition severity. Consideration of these characteristics will help determine, at the outset, whether A-CASI or T-ACASI can be a successful mode for interviewing a given population. For example, the population of older persons may require a large-point font in which to view an onscreen A-CASI application, a mentally challenged population may require the digitally recorded text to be administered at a much slower pace than is customary for other populations, or it may be necessary to record the text in multiple languages. The key is to build an A-CASI or T-ACASI application that the target population will understand and easily use, so that required data will truly inform the hypotheses. A clinical study that has successfully addressed the specific characteristics and

3

needs of a target population can fully benefit from one of the major advantages of using A-CASI and T-ACASI: computerized standardization of questionnaire administration. This standardization permits the collection of better-quality data than is possible with a live interviewer because the prerecorded questions are administered as audio files in exactly the same way to every respondent who completes the questionnaire. The voice, tone, inflection, speed, and text are identical for each respondent, no matter how many times a question is repeated for each respondent. Unlike clinician- or intervieweradministered questionnaires, A-CASI or T-ACASI prevents alteration of question inflection or of specific word or phrase emphasis that might compromise standardization. Because no interviewer or clinician is involved in the self-administered phase of the interview process, computerized questionnaires must be designed accordingly. One design feature common to most computerassisted questionnaires is provision for a ‘‘don’t know’’ response if the respondent does not understand the question or simply has no response. Typically, at the beginning of the self-administered portion of the questionnaire, some standard conventions are provided to the study participant. For example, the respondent may be told to press a specific key or button for a ‘‘don’t know’’ response, and a different key or button to hear the question repeated. Help screens or help text can also be built into the questionnaire to clarify a specific question or series of response categories. In keeping with the goal of maximizing study response participation when a live interviewer is absent, the design team for an A-CASI questionnaire may do well to consider whether the study participant will provide input by computer keyboard or by touch-screen monitor. Most adults and children in today’s society have become adept at using touch-screen systems such as airport kiosks, automatic teller machines, and grocery store checkout machines. There is some evidence to suggest that, when given a choice, clinical study participants prefer responding by touch-screen computer rather than by keyboard. For example, among a

4

COMPUTER-ASSISTED DATA COLLECTION

convenience sample of 108 patients at a Baltimore, Maryland, sexually transmitted disease (STD) clinic, nearly 70% indicated that using a touch-screen A-CASI application was easier than responding by computer keyboard (7). 3.2 Edit Checks Computer-assisted questionnaire edit checks can preempt the need for laborious data editing and cleaning on the back end. In clinical research, errors and anomalies occur despite careful study design and implementation of quality assurance and quality control strategies. To minimize the impact of such anomalies and errors on the study results, data editing and cleaning typically occur as part of the data management process once the data are collected. Data editing is the process of identifying ambiguous or erroneous data and diagnosing the source of the problem, and data cleaning is the process of changing the erroneous data shown to be incorrect. Typical clinical trial data editing procedures include a ‘‘query’’ process, whereby data anomalies are resolved by referring back to a CRF, and in some instances by interacting with staff at a clinical site. With A-CASI and T-ACASI, there is no CRF. In fact, there is no paper record at all for clarifying data issues. The patient-reported source data are initialized in electronic format, and real-time edits must be incorporated into the computerized application to ensure that high-quality data are captured. Data edits, such as logic and range checks, are routinely programmed into the questionnaire to ensure that data values are within legitimate ranges and to help ensure consistency across items. Although the ability to implement edit checks with the use of A-CASI or T-ACASI is an advantage over paper-and-pencil data collection, the questionnaire designer must still consider how often to use edit checks, when to use them, and whether to deploy ‘‘soft’’ edit checks, ‘‘hard’’ edit checks, or both during administration of the questionnaire. Use of edit checks involves balancing the need for the highest quality data possible with the added burden and potential frustration they pose for the respondent. With a triggered hard edit check, for example, a

message will appear—usually in a pop-up box on the same screen—requiring a change in a patient’s response. By contrast, the soft edit check, if triggered, will simply recommend that the respondent make a change in response but also will permit the response to remain unchanged and will allow the respondent to continue with the questionnaire. Among options for edit checks, one type of hard edit requires that the respondent actually enter a value into an item before moving on to the next question. This safeguard is particularly recommended for ACASI and T-ACASI because the clinician, being removed from the self-administered interviewing phase, cannot help prevent missing data. Good candidates for hard edit checks are questionnaire items that aim to collect data for critical or key study endpoints. Hard edits ensure that questions collecting key study hypotheses data will have little or no missing data, and that the data will be within expected and acceptable ranges. Soft edit candidates are items for which a logical range of values can be established but outlier values may be feasible. In such instances, the range edit check can be triggered when out-of-range values are entered, preventing data errors due to miskeyed values but allowing the respondent to reconfirm or reenter the out-of-range value and continue completing the computerized form. One way to establish optimal use of edit checks is to pretest computerized forms. Interviewing a small number of participants who have the same characteristics as the target population provides valuable feedback about where and when a questionnaire requires edit checks. For example, if more than half the pretest participants enter a particular question response outside the range of values the team was expecting, adding an edit check to that question may be wise. If the pretest indicates that many questionnaire items require an edit check, the team may want to consider prioritizing the questions and implementing edit checks for the more critical data items, and clarifying the other questions or adding help screens. Pretesting also provides an opportunity to judge the overall usability (e.g., clarity of question wording, placement of entry fields on the

COMPUTER-ASSISTED DATA COLLECTION

screen) of the computerized system by study participants. In developing any type of edit checks, the study team should consider how best to convey each ‘‘error’’ or notification message with minimal burden to the respondent. Ideally, an ‘‘error’’ or notification message is evident to the respondent and clearly specifies what the respondent is to do. For example, an error message regarding mis-keyed birth month might read, ‘‘Month of Birth must be between 1 and 12. Please reenter.’’ A well-designed error message is brief and clear; the subsequent instructions are simple and relatively easy to act upon. 3.3 Quality Assurance Although not the focus of this article, a brief discussion is essential to address clinical studies that because of sponsor requirements must follow specified regulations—such as studies that ultimately submit their data to the U.S. Food and Drug Administration (FDA) or other regulatory agencies. The conduct of these studies, including the development and documentation surrounding the computer application, must follow all relevant federal regulations and guidelines. One example of such a regulation is ‘‘Electronic Records; Electronic Signatures,’’ 21 C.F.R. pt. 11 (2003). Audit trails are a key component of such regulations and should be included in the computerized application. The FDA defines an audit trail as a ‘‘chronological record of system activities that is sufficient to enable the reconstruction, reviews, and examination of the sequence of environments and activities surrounding or leading to each event in the path of a transaction from its inception to output of final results’’ (8). Although most commercial survey systems were not originally designed to be used for clinical research, many of them (e.g., Blaise from Statistics Netherlands, www.blaise.com) include electronic audit trail features. The key is to know the requirements of the sponsor and any relevant governing agencies and to implement a system that meets all the requirements at study inception. Implementation includes creating and maintaining the system and the testing documentation according to those same requirements.

5

4 CLINICAL CASE STUDIES USING COMPUTER-ASSISTED METHODS In this section, selected case studies illustrate how A-CASI and T-ACASI have been used in clinical studies. Although the research described includes both federally and privately sponsored studies and covers a wide array of topics, it all shares one common thread: the need to collect sensitive or personal information in a private setting. Each of the four studies programmed its instrument to incorporate real-time data edits, with consistency checks across numerous questions. At the end of data collection, each study extracted a data file from the system to allow for further edits of the data. Because of the inclusion of the edit and consistency checks, and the ability to review interim data files throughout data collection, data managers performed very limited data cleaning before beginning data analyses. 4.1 Case 1 A seroprevalence study of herpes simplex virus-2 was conducted at 36 primary care physician (PCP) offices in suburban areas surrounding six cities in the United States in 2002 (9). Adults, aged 18 to 59 years, were asked to participate during a routine visit to the PCP office. After providing consent, patients were asked to complete a 15minute, computer-assisted self-interview and to provide a blood sample. The questionnaire included demographic questions as well as a series of questions designed to elicit sexual attitudes, behaviors, and symptom history relating to genital herpes. The A-CASI technology was selected to provide patients a confidential method to report sensitive information. The questionnaire was completed and a blood sample provided by 5433 patients. 4.2 Case 2 The survey of Male Attitudes Regarding Sexual Health (MARSH) was conducted in 2001 to determine the age- and race-specific prevalence of mild, moderate, and severe erectile dysfunction (ED) in African American, Caucasian, and Hispanic males aged 40 years or older (10). The survey was a nationally administered random-digit-dial, list-assisted

6

COMPUTER-ASSISTED DATA COLLECTION

representative telephone interview. A trained telephone interviewer administered the first part of the questionnaire, which included the screening, demographics, and general healthrelated questions, and then transferred the respondent to the T-ACASI section. The respondent then answered the remaining questions with the keypad on his touch-tone telephone. The T-ACASI method provided a private setting in which the more sensitive questions were asked (e.g., questions about sexual activity, sexual gratification, the number of sexual partners, ED treatment, and the discussion of erectile problems). The survey was conducted in both English and Spanish. Interviews were completed by 2173 respondents.

interventions aimed at reducing the number of infants there who are at increased risk of dying during their first year of life. One of the numerous protocols implemented under this initiative and supported by the National Institute on Alcohol Abuse and Alcoholism was the Prevention and Fetal Alcohol Effects Study, which used A-CASI technology to allow for a more private environment when disadvantaged, pregnant women were asked about their alcohol consumption. The study results demonstrated that using computerassisted technology to screen for alcohol use in disadvantaged pregnant populations is feasible and acceptable to the respondents (13).

4.3 Case 3

REFERENCES

The National Institute of Mental Health– sponsored Collaborative HIV/STD Prevention Trial began in 1999 to examine the efficacy of a community-level intervention to reduce HIV/STD incidence and high-risk behaviors in China, India, Peru, Russia, and Zimbabwe. The intervention sought to modify social norms at the community level to effect mass changes in HIV/STD risk behaviors. In each of the five countries, community popular opinion leaders were engaged as behaviorchange agents within their community of friends, neighbors, and coworkers. As part of this work, several pilot studies were conducted with use of A-CASI technology and developed in the respective languages of each country (11). The study selected this technology to test the feasibility of its use in developing countries and to allow for added privacy when sensitive and personal information is collected (e.g., risk behaviors). The study concluded that A-CASI appears to be feasible in these settings (12). 4.4 Case 4 The National Institute of Child Health and Human Development’s Initiative to Reduce Infant Mortality in Minority Populations in the District of Columbia began in 1993 to develop projects designed to better understand the factors that influence the high rate of infant mortality and morbidity in Washington, DC, and to design and evaluate

1. J. C. Fink, CATI’s first decade: the Chilton experience. Sociol Methods Res. 1983; 12: 153–168. 2. H. G. Miller, C. F. Turner, and L. E. Moses, eds. Methodological issues in AIDS surveys. In: H. G. Miller, C. F. Turner, and L. E. Moses (eds.), AIDS: The Second Decade. Washington, DC: National Academy Press, 1990, Chapter 6. 3. J. M. O’Reilly, M. L. Hubbard, J. T. Lessler, P. P. Biemer, and C. F. Turner, Audio and video computer-assisted self-interviewing: preliminary tests of new technologies for data collection. J Off Stat. 1994; 10: 197–214. 4. C. F. Turner,. L. Ku, F. L. Sonenstein, and J. H. Pleck, Impact of ACASI on reporting of male-male sexual contacts: preliminary results from the 1995 National Survey of Adolescent Males In: R. Warnecke (Ed.), Health Survey Research Methods. DHHS Pub. No. (PHS) 96-1013. Hyattsville, MD: National Center for Health Statistics, 1996, pp. 171–176. 5. P. C. Cooley, H. G. Miller, J. N. Gribble, and C. F. Turner, Automating telephone surveys: using T-ACASI to obtain data on sensitive topics. Comput Human Behav. 2000; 16: 1–11. 6. J. N. Gribble, H. G. Miller, J. A. Catania, L. Pollack, and C. F. Turner, The impact of T-ACASI interviewing on reported drug use among men who have sex with men. Subst Use Misuse. 2000; 35: 63–84. 7. P. C. Cooley, S. M. Rogers, C. F. Turner, A. A. Al-Tayyib, G. Willis, and L. Ganapathi, Using touch screen audio-CASI to obtain data on

COMPUTER-ASSISTED DATA COLLECTION sensitive topics. Comput Human Behav. 2001; 17: 285–293. 8. Office of Regulatory Affairs, U.S. Food and Drug Administration. Glossary of Computerized System and Software Development Terminology. Available at: http://www.fda.gov/ ora/inspect ref/igs/gloss.html 9. P. Leone, D. T. Fleming, A. Gilsenan, L. Li, and S. Justus, Seroprevalence of herpes simplex virus-2 in suburban primary care offices in the United States. Sex Transm Dis. 2004; 31: 311–316. 10. E. O. Laumann, S. West, D. Glasse, C. Carson, R. Rosen, and J. H. Kang, Prevalence and correlates of erectile dysfunction by race and ethnicity among men aged 40 or older in the United States: from the male attitudes regarding sexual health survey. J Sex Med. 2007; 4: 57–65.

7

11. L. C. Strader, Developing of a multi-lingual survey instrument in A-CASI. Paper presented at the American Public Health Association Annual Meeting; October 24, 2001, Atlanta, GA. Abstract 28287. 12. The NIMH Collaborative HIV/STD Prevention Trial Group. The feasibility of audio computer-assisted self-interviewing in international settings. AIDS. 2007; 21(Suppl 2): S49–58. 13. J. Thornberry, B. Bhaskar, C. J. Krulewitch, B. Weslet, M. L. Hubbard, et al., Audio computerized self-report interview in prenatal clinics: audio computer-assisted self-interview with touch screen to detect alcohol consumption in pregnant women: application of a new technology to an old problem. Comput Inform Nurs. 2002; 20: 46–52.

CONDITIONAL POWER

2

To introduce the concept with statistical rigor and clarity, consider the statistical framework of testing a normal mean into which many clinical trials can be formulated (2,3). We have H 0 : m £ 0 versus the alternative H1 : m > 0, where m denotes the treatment effect parameter and (for example) is the mean elevation of diastolic blood pressure above 90 mm Hg. Let m be the maximum sample size of the RNST and the current data be Sn . Then the conditional power is defined as

MING T. TAN Division of Biostatistics, University of Maryland School of Medicine and Greenebaum Cancer Center, College Park, Maryland

1

CONDITIONAL POWER

INTRODUCTION

For ethical, economical, and scientific reasons, clinical trials may be terminated early for a pronounced treatment benefit or the lack thereof. In addition to group sequential tests and type I error-spending functions, another approach is to address the question more directly by asking whether the accumulated evidence is sufficiently convincing for efficacy. Or, if evidence is lacking, one should ask whether the trial should continue to the planned end and whether the conventional (reference) nonsequential test (RNST) should be employed. Conditional power is one way to quantify this evidence. It is simply the usual statistical power of RNST conditional on the current data. Thus, it is the conditional probability that the RNST will reject the null hypothesis on completion of the trial given the data currently available at a given parameter value of the alternative hypothesis. Therefore, if the conditional power is too low or exceedingly high, then the trial may be terminated early for futility or for efficacy, respectively. The early stopping procedure derived based on conditional power is referred to as stochastic curtailing by which an ongoing trial is curtailed as soon as the trend based on current data becomes highly convincing (1). Therefore, conditional power serves two closely related purposes: (1) as a measure of trend reversal (e.g., futility index) and (2) as an interim analysis procedure (e.g., formally a group sequential method). However, this method can be readily communicated to nonstatisticians because it simply answers the question whether the evidence in the interim data is sufficient for making an early decision on treatment efficacy or the lack thereof given current data in reference to what the RNST concludes.

Pn (µ) = P(the RNST will reject H0 at m|Sn ). Furthermore, let X i be the observed elevation of diastolic blood pressure above 90 mm Hg of the ith subject. Thus, X i ∼ (µ, σ 2 ), i = 1, L, n, L, m. Then the current data are summarized by the sufficient statistic, the n
X i . Then Sn ∼ N(nµ, partial sum Sn = i=1

nσ 2 ), n = 1,2, L, m, where it is assumed that σ = 28.3 mm Hg based on preliminary data. We are interested in detecting a clinically important difference of 10 mm Hg at a significance level of α (for example, 0.025) with power 1 − β (for example, 0.90). The fixed sample design requires m = 86 subjects, and the null hypothesis would be√rejected if Zm > Zα = 1.96, where Zm = Sm / mσ and zα is the lower α-percentile of the standard normal curve, √ or equivalently Sm > s0 , where s0 = zα σ m = 554.68. Therefore, the conditional power is evaluated under the following conditional distribution:   Sm |µ, Sn ∼ N (m − n)µ + Sn , (m − n)σ 2 (1) and thus it is given by Pn (µ) = Pµ (RNST rejects H0 |Sn ) √ = Pµ (Sm > zα σ m|Sn ) √   Zα σ m − Sn − (m − n)µ √ . =1− σ m−n A similar derivation for two-sided tests can be found in Reference 3. Because the conditional power depends on the unknown true

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

CONDITIONAL POWER

0.2

Power 0.4 0.6

0.8

1.0

2

0.0

Usual Power xbar =8 xbar =2 0

2

4

6

8

10

12

mu

treatment effect parameter µ, different hypothetical values of µ have to be given to evaluate the conditional power. A common practice is to consider the three values: the null value, the value under the alternative hypothesis that the study is designed to detect, and the current estimate. One such current value is the average of the null and the alternative. If in the midcourse (n = 43), the average elevation of blood pressure (xbar in Fig. 1) is 8 mm Hg (i.e., Sn = 43*8 = 344) and the true treatment effect µ is 10 mm Hg, then the conditional power is 92%, which implies there is a good chance the null hypothesis will be rejected at the planned end of the trial. On the other hand, if the average elevation is 2, then the conditional power is only 49%. The conditional power has been used as a futility index. If the conditional power at the alternative hypothesis is too low (for example, less than 0.20), then the trial is not likely to reach statistical significance, which provides an argument for early termination of the trial for futility. Figure 1 gives the usual power function of the test as well as the conditional power that corresponds to the two average elevations of blood pressures (xbar values) at midcourse (n = 43) of the trial. Figure 2 gives the stochastic curtailing boundaries based on the conditional power. The difficulty is which µ to choose because it may be hard to anticipate a future trend. A stochastic curtailing procedure can be derived

14

Figure 1. Illustration of the conditional power.

using the conditional power of the RNST given Sn and some plausible values of the treatment effects. Then, we can derive a formal sequential test with upper boundary an and lower boundary bn where we reject H 0 the first time Zn ≥ an or accept H 0 the first time Zn ≤ bn . If the conditional power at µ = 0 is greater than γ 0 (for example, 0.80), then H 0 is rejected, and if the conditional power at µ = 10 is less than 1 − γ 1 (for example, 0.20), then H1 is accepted. Then, the sequential boundaries are √ an = zα  √ m/n + z1−γ0 (m − n)/n and bn = zα m/n −  √ z1−γ1 (m − n)/n − µ(m − n)/(σ n). It can be shown (1) that the derived curtailing procedure has a type I error no greater than α/γ 0 (0.0625) and type II error no greater than β/γ 1 (0.25). Figure 2 gives the boundaries of the two stochastic curtailing procedures based on conditional power with γ 0 = γ 1 = γ = 0.80 and 0.98. The extreme early conservatism of stochastic curtailing is apparent. It is now well known and widely utilized in the monitoring of clinical trials that the test statistic in most of the common phase III clinical trials can be formulated into the general Brownian motion framework (4). In other words, the test statistic can be rescaled √ into z-statistic Bt = Zn t (0 ≤ t ≤ 1) that follows approximately a Brownian motion with drift parameter µ. Thus µ may represent the pre–post change, or a log odds ratio or a log hazard ratio The primary goal is to test H0 :

20

CONDITIONAL POWER

3

Figure 2. Stopping boundaries of different stochastic curtailing procedures (DP denotes discordance probability).

−20

−10

0

Zn

10

gamma = 0.8 gamma = 0.98 Max DP = 0.02

m £ 0 against the alternative H1 : m > 0. Then, the conditional distribution of B1 given Bt is again normal with mean Bt + (1−t)µ and variance 1−t.Therefore, the conditional power in the general Brownian motion formulation is pt (µ) = Pµ (RNST rejects H0 |Bt ) = Pµ (B1 > Zα |Bt )   zα − Bt (1 − t)µ . =1− √ 1−t For a two-sided test, the conditional power is given by pt (µ) = Pµ (|S1 | > zα/2 |Bt )   zα/2 − Bt − (1 − t)µ = 1− √ 1−t   −zα/2 − Bt − (1 − t)µ . √ + 1−t Several authors (3–5) have documented in detail on how to formulate common clinical trials with various types of endpoints into the Brownian motion framework. For example, the sequentially computed log-rank statistic is normally distributed asymptotically with an independent increment structure (6,7). The conditional power is given in Reference 8 for comparing two proportions, in Reference 9 for censored survival time for log-rank or

0

20

40 n

60

80

weighted log-rank statistics, in Reference 10 for longitudinal studies, and in Reference 11 for models with covariates. In addition, several authors have used conditional power as an aid to extend an ongoing clinical trial to beyond the originally planned end for survival outcome (12) and in Brownian motion (13). More recently, the discordance probability is also extended and derived under the general Brownian motion framework (14,15) 3 WEIGHT-AVERAGED CONDITIONAL POWER Another way to avoid explicit choices of the unknown parameter is to use the weighted average of the conditional power with weights given by the posterior distribution of the unknown parameter µ given currently available data. Let the prior distribution of µ be π (µ) and its posterior be π (µ|Sn ). Then, the weight-averaged conditional power (also known as predictive power) for the one-sided hypothesis testing is given by  Pn = pn (µ)π (µ|Sn )dµ. If the improper as well as noninformative prior π (µ) = 1 is chosen, then the posterior of µ|Sn is normal with mean Sn /n and variance σ 2 /n. Then from Equation (1), the marginal distribution of Sm |Sn is again normal with

4

CONDITIONAL POWER

mean (m/n)Sn and variance σ 2 ((m − n)m/n). The predictive power is thus simply: √ Pn = P(Sm > zα σ m|Sn )  √ zα σ m − (m/n)Sn =1−  . σ (m − n)(m/n) Several authors have used the predictive power approach (16–18). Similar to conditional power, if Pn ≥ γ 0 , we consider rejecting the null, and if Pn ≤ 1 − γ 1 , we consider accepting the null. This criterion results in the following interim with  √ analysis procedure boundaries an = zα n/m + z1−γ0 (m − n)/m  √ and bn = zα n/m − z1−γ1 (m − n)/m. Unfortunately, no simple relationship relates the type I and II errors of the procedure with the predictive power. However, more informative usage of the predictive power may be through an informative prior. The data monitoring committee can make full use of the predictive power to explore the consequences of various prior believes on the unknown treatment effect parameter. 4 CONDITIONAL POWER OF A DIFFERENT KIND: DISCORDANCE PROBABILITY Based on the same principle of stopping a trial early as soon as the trend becomes inevitable, it is revealing to consider the conditional likelihood of the interim data given the reference test statistic at the planned end of the trial (m):  Sn |Sm ∼ N

 n n(m − n) 2 Sm , σ . m m

The distinct advantage of using this conditional probability is that it does not depend on the unknown parameter µ because conditioning is made on Sm , which is a sufficient statistic for µ. Using this conditional likelihood of the test statistic calculated at an interim time, we can derive a different kind of stochastic curtailing based on discordance probability defined as the probability the sequential test does not agree with the RNST in terms of accepting or rejecting the null hypothesis should the trial continue to the planned end (m). At a

given interim time point n, let an be the upper (rejection) boundary (i.e., if Sn ≥ an , reject the null hypothesis). Then Pµ (Sn ≥ an |Sm ≤ s0 ) is the probability that the decision to reject H 0 at n with Sn ≥ an is discordant with the decision to accept H0 at m when Sm ≤ s0 . Because for any µ, Pµ (Sn ≥ an |Sm ≤ s0 ) ≤ P(Sn ≥ an |Sm = s0 ). Then we can use P(Sn ≥ an |Sm = s0 ) to derive a sequential boundary. If this probability is smaller than ξ , then we stop the test and reject the null hypothesis; and, similarly, if P(Sn ≤ bn |Sm = s0 ) < ξ , then we stop the test and do not reject the null hypothesis. If we choose ξ (say, 0.05), that is, the same cut-off point for each n (n = 1, L, m), then we have P(Sn ≥ an |Sm = s0 )  an − ns0 /m =1−  ≤ ξ. n(m − n)/mσ √ Solving n = zα n/m  this equation we have a√ +  z1−ξ (m − n)/m and bn = zα n/m − z1−ξ (m − n)/m. It is worth noting again that the boundaries are derived using marginal probability ξ for each n(n = 1, L, m). Marginally, the stopping boundaries are the same as those from predictive power with noninformative prior. A more accurate statement may be by a global discordance probability defined as the probability that the sequential test on interim data does not agree with the acceptance/rejection conclusion of the RNST at the planned end (19). Xiong (19) derived the elegant sequential conditional probability ratio test (SCPRT) via a conditional likelihood ratio approach and obtained the boundaries of the same form. Most importantly, he derived the intricate relationship among the type I, II errors, the discordance probability, and he provided efficient algorithms to compute them. It shows that the sequential boundary can be derived such as it has virtually the same type I and II errors as the RNST and the probability that the rejection or acceptance of the null hypothesis based on interim data might be reversed is less than a given level ρ 0 (for example, 0.02) should the trial continues to the planned end. With instantaneous computation of the type I and

CONDITIONAL POWER

II errors and various discordance probabilities, a sharper monitoring boundaries can be derived (14,19–22). It is noted that similar boundaries themselves are also derived for Bernoulli series using the same parameter free approach in the context of reducing computation in a simulation study designed to evaluate the error rates of a bootstrap test (23). Figure 2 also gives the boundaries of the stochastic curtailing procedure based on the discordance probability with a maximum discordance probability (denoted DP in Fig. 2) less than 0.02. In contrast to the extreme early conservatism of stochastic curtailing based on conditional power, the three boundaries become closer as the trial approaches its end. Interestingly, in the last quarter of the information fraction of the trial, the curtailing procedure (with γ = 0.98) almost coincides with that of the SCPRT, whereas the boundary with γ = 0.80 becomes slightly tighter than that of the SCPRT, which results in an increase in discordance probability relative to that of the RFSST and reflecting the conservatism in the SCPRT. A more detailed comparison of the two curtailing approaches and the SCPRT with common group sequential procedures such as the O’Brien-Fleming, Pocock, and Haybittle-Peto procedures is given in References 21 and 22. 5

ANALYSIS OF A RANDOMIZED TRIAL

The Beta-Blocker Heart Attack Trial was a randomized double-blind trial to compare propranolol (n = 1916) with placebo (n = 1921) in patients who had recent myocardial infarction sponsored by the National Institute of Health. Patients were accrued from June 1978 to June 1980 with a 2-year follow-up period resulting in a 4-year maximum duration. The trial was terminated early for a pronounced treatment benefit. Aspects on the interim monitoring and early stopping of this trial have been summarized (24–26). The minimum difference of clinical importance to be detected is 0.26 in log hazard ratio derived based on projected 3-year mortality rates of 0.1746 for the placebo group and 0.1375 for the treatment group adjusting for compliance. Roughly 628 deaths are

5

required for a fixed sample size test to detect such a difference at a significance level of 5% with 90% power. Seven interim analyses that correspond to the times the Policy and Data Monitoring Board met were planned. The trial was stopped 9 months early at the sixth interim analysis with 318 deaths (183 in the placebo arm and 125 in the treatment arm) with the standardized z-statistic valued at 2.82, and the O’Brien-Fleming boundary was crossed. The conditional power can be evaluated for various expected deaths. For example, a linear interpolation of the life table based the current survival data suggests 80 additional deaths in the ensuing 9 months. Therefore, the information time at the sixth analysis is √ 318/(318 + 80) = 0.80, then Bt = 2.82 0.80 = 2.52. The conditional power p0.80 (0) is 0.89. If an additional 90 deaths are expected, then the conditional power p0.78 (0) is 0.87. Both suggest a rather high conditional power for a treatment effect. Assuming an additional 90 deaths, the SCPRT curtailing based on discordance probability can be derived (14), which gives a maximum discordance probability of 0.001. This finding implies that there is only 0.1% chance that the conclusion might be reversed had the trial continued to the planned end (14). If the 628 total deaths in the original design are used, and if an SCPRT is to be used as stated in the protocol, then a maximum discordant probability is 1%, which implies only a slight chance (1%) that the decision based on the SCPRT procedure in the protocol may be reversed had the trial continued to the planned end (21). Therefore, it is highly unlikely that the early stopping decision for efficacy would be reversed had the trial continued to the planned end by all three procedures. However, the SCPRT based curtailing provides a sharper stopping boundary for trend reversal as expected. 6 CONDITIONAL POWER: PROS AND CONS To put things in perspective, the conditional power approach attempts to assess whether evidence for efficacy or the lack of it based on the interim data is consistent with that

6

CONDITIONAL POWER

at the planned end of the trial by projecting forward. Thus, it substantially alleviates the major inconsistency in all other group sequential tests where different sequential procedures applied to the same data yield different answers. This inconsistency with the nonsequential test sets up a communication barrier in practice where we can claim a significant treatment effect via the nonsequential test, but we cannot to do so via the sequential test based on the same data set or we can claim significance with one sequential method but cannot do so with another. For example, in a clinical trial that compares two treatments at the 5% significance level where five interim analyses were planned, the nominal level at the fifth analysis for the Petcock procedure is 0.016 whereas the nominal level at the fifth interim analysis for the O’Brien and Fleming is 0.041. If the trial has a nominal P-value of 0.045 at the fifth analysis, then according to either of the group sequential designs, the treatment effect would not be significant, whereas investigators with the same data just carrying out a fixed sample size test would claim a significant difference. However, if the nominal P-value is 0.03, then the treatment effect is significant according to the O’Brien-Fleming procedure but not according to the Pocock procedure. The advantage of the conditional power approach for trial monitoring is its flexibility. It can be used for unplanned analysis and even analyses whose timing depends on previous data. For example, it allows inferences from over-running or under-running (namely, more data come in after the sequential boundary is crossed, or the trial is stopped before the stopping boundary is reached). Conditional power can be used to aid the decision for early termination of a clinical trial to complement the use of other methods or when other methods are not applicable. However, such flexibility comes with a price: a potentially more conservative type I and type II error bounds (α/γ 0 and β/γ 1 ) that one can report. The SCPRT based approach removes the unnecessary conservatism of the conditional power and can retain virtually the same type I and II errors with a negligible discordance probability by accounting for how the data pattern (sample path) is trended (traversed). The use of the SCPRT

especially in making decision in early stages has been explored by Freidlin et al. (27) and for one-sided test by Moser and George (28). The greatest advantage of predictive power is that it allows us to explore consequences of various prior believes about the unknown treatment effect parameter. Finally, conditional power has also been used to derive tests adaptive to the data in the first stage of the trial (see, Reference 13). More recently, the related reverse stochastic curtailing and the discordance probability are used to derive group tests adaptive to updated estimates of the nuisance parameter (14).

REFERENCES 1. K. Lan, R. Simon, and M. Halperin, Stochastically curtailed tests in long-term clinical trials. Sequent. Anal. 1982; 1: 207–219. 2. J. Whitehead, A unified therory for sequential clinical trials. Stat. Med. 1999: 2271–2286. 3. C. Jennison and B. W. Turnbull, Group Sequential Methods with Applications to Clinical Trials. New York: Chapman & Hall/CRC, 2000. 4. K. K. Lan and D. M. Zucker, Sequential monitoring of clinical trials: the role of information and Brownian motion. Stat. Med. 1993; 12: 753–765. 5. J. Whitehead, Sequential methods based on the boundaries approach for the clinical comparison of survival times. Stat. Med. 1994; 13: 1357–1368. 6. M. H. Gail, D. L. DeMets, and E. V. Slud, Simulation studies on increments of the twosample logrank score test for survival time data, with application to group sequential boundaries. In:: J. Crowley and R. A. Johnson, (eds.), Survival Analysis. Hayward, CA: Instititute of Mathematical Statistics, 1982, pp. 287–301. 7. A. A. Tsiatis, Repeated significance testing for a general class of statistics used in censored survival analysis. J. Am. Stat. Assoc. 1982; 77: 855–861. 8. M. Halperin, K. K. Lan, J. H. Ware, N. J. Johnson, and D. L. DeMets, An aid to data monitoring in long-term clinical trials. Control. Clin. Trials 1982; 3: 311–323. 9. D. Y. Lin, Q. Yao, Z. Ying, A general theory on stochastic curtailment for censored survival data. J. Am. Stat. Assoc. 1999; 94: 510–521.

CONDITIONAL POWER 10. M. Halperin, K. K. Lan, E. C. Wright, and M. A. Foulkes, Stochastic curtailing for comparison of slopes in longitudinal studies. Control. Clin. Trials 1987; 8: 315–326. 11. C. Jennison and B. W. Turnbull, Groupsequential analysis incorporating covariate information. J. Am. Stat. Assoc. 1997; 92: 1330–1341. 12. P. K. Andersen, Conditional power calculations as an aid in the decision whether to continue a clinical trial. Control. Clin. Trials 1986; 8: 67–74. 13. M. A. Proschan and S. A. Hunsberger, Designed extension of studies based on conditional power. Biometrics 1995; 51: 1315–1324. 14. X. Xiong, M. Tan, and J. Boyett, Sequential conditional probability ratio tests for normalized test statistic on information time. Biometrics 2003; 59: 624–631. 15. X. Xiong, M. Tan, and J. Boyett, A sequential procedure for monitoring clinical trials against historical controls. Stat. Med. 2007; 26: 1497–1511. 16. J. Herson, Predictive probability early termination plans for phase iI clinical trials. Biometrics 1979; 35:775–783. 17. S. C. Choi, P. J. Smith, and D. P. Becker, Early decision in clinical trials when the treatment differences are small. Experience of a controlled trial in head trauma. Control. Clin. Trials 1985; 6: 280–288. 18. D. J. Spiegelhalter, L. S. Freedman, and P. R. Blackburn, Monitoring clinical trials: Conditional or predictive power? Control. Clin. Trials 1986; 7: 8–17. 19. X. Xiong, A class of sequential conditional probability ratio tests. J. Am. Stat. Assoc. 1995; 90: 1463–1473. 20. X. Xiong, M. Tan, M. H. Kutner, Computational methods for evaluating sequential tests and post-test estimation via the sufficiency principle. Statist. Sin. 2002; 12: 1027–1041. 21. M. Tan, X. Xiong, M. H. Kutner, Clinical trial designs based on sequential conditional probability ratio tests and reverse stochastic curtailing. Biometrics 1998; 54: 682–695. 22. M. Tan and X. Xiong, Continuous and group sequential conditional probability ratio tests for phase II clinical trials. Stat. Med. 1996; 15: 2037–2051. 23. C. Jennison, Bootstrap tests and confidence intervals for a hazard ratio when the number of observed failures is small, with applications to group sequential survival studies. In: C. Page and R. LePage (eds.), Computing Science and Statistics: Twenty-second Symposium on

7

the Interface. Berlin: Springer-Verlag, 1992, pp. 89–97. 24. D. L. DeMets and K. K. Lan, Interim analysis: the alpha spending function approach. Stat. Med. 1994; 13: 1341–1352; discussion 1353–1346. 25. D. L. DeMets, R. Hardy, L. M. Friedman, and K. K. Lan, Statistical aspects of early termination in the beta-blocker heart attack trial. Control. Clin. Trials 1984; 5: 362–372. 26. K. K. Lan and D. L. DeMets, Changing frequency of interim analysis in sequential monitoring. Biometrics 1989; 45: 1018–1020. 27. B. Freidlin, E. L. Korn, and S. L. George, Data monitoring committees and interim monitoring guidelines. Control. Clin. Trials 1999; 20: 395–407. 28. B. K. Moser and S. L. George, A general formulation for a one-sided group sequential design. Clin. Trials 2005; 2: 519.

CROSS-REFERENCES Group Sequential Designs Interim Analysis

CONFIDENCE INTERVAL

sample value Q

JENO˝ REICZIGEL ´ University Szent Istvan Faculty of Veterinary Science Budapest, Hungary

“likely range” for the true value L

(95%)

U

Figure 1. Point estimate θˆ and confidence interval (L,U) for a parameter θ (prevalence, mean, difference, relative risk, etc.). θˆ represents the value experienced in the sample, whereas (L, U) defines a ‘‘reasonable range’’ or ‘‘likely range’’ for the unknown true (population) value θ

A confidence interval (CI) or interval estimate for an unknown parameter θ is a ‘‘from-to’’ range calculated from the sample so that it contains θ with a high—usually 90%, 95%, or 99%—probability (Fig. 1). As opposed to a point estimate θˆ , which is a single number calculated from the sample, the interval (L,U) gives a more expressive and easy-tounderstand picture about the uncertainty of the estimate. L and U are also called confidence limits for the parameter θ . Some examples of how CIs are usually communicated in papers are as follows:

the sample is meaningful at all, a CI is also meaningful. For curves like a regression line or a ROC curve, interval estimation results in a so-called confidence band (Fig. 2). Technically, a CI, as being a ‘‘fromto’’ range calculated from the sample, consists of two functions of the sample values L = L(x1 , x2 , . . . , xn ) and U = U(x1 , x2 , . . . , xn ), the lower and upper confidence limit, which satisfy P(L < θ < U) = 1 − α, where 1 − α is called coverage probability or level of confidence, and α is called error probability. (These probabilities may also depend on the true value of θ ; see below for details.) For symmetry, CIs are usually constructed so that they have an error probability of α/2 at each side, that is, P(θ < L) = P(θ > U) = α/2 (Fig. 3a), which are called symmetric or equaltailed CIs. In the asymmetric case, the sum of the error probabilities is α, but they do not need to be equal (Fig. 3b). One-sided CIs are intervals of the form (−∞, U) or (L, ∞) with the property P(θ < U) = 1 − α or P(θ > L) = 1 − α (Fig. 3c,d). It is easy to verify that given two 1 − α level one-sided CIs (−∞, U) and (L, ∞) for a certain parameter, then (L, U) forms a 1 − 2α level symmetric CI for the same parameter. For example, two 95% one-sided intervals define a 90% two-sided one. Similarly, if (L, U) is a 1 − α level equal-tailed CI, then (−∞, U) as well as (L, ∞) are 1 − α/2 level one-sided CIs for the same parameter. That is, a 95% level twosided symmetric interval can be converted into two 97.5% one-sided ones. In the frequentist (as opposite to Bayesian) view, the parameter θ is assumed to be a fixed

• ‘‘The estimated prevalence rate of the

disease in the study population was 8.3% (95% CI 5.8% to 11.7%).’’ • ‘‘Blood cholesterol fell by 0.76 mmol/l in group A vs 0.32 mmol/l in group B (difference 0.44 mmol/l, 95% CI 0.16 to 0.73).’’ • ‘‘Acute infections occurred in 25 of the patients on active treatment and in 36 of the patients on placebo (relative risk 0.69, 95% CI 0.55 to 0.87).’’ • ‘‘Treament failure was eight times more likely in the placebo than in the antibiotic group (13.1% vs 1.6%, odds ratio 8.2, 95% CI 1.9 to 34.7).’’ CIs can be given for unknown population parameters like the mean, variance, median, proportion, or for model parameters like a regression coefficient, odds ratio, correlation coefficient, or for functions of the abovementioned parameters like, for example, the variance/mean ratio, the difference of two proportions or two means, and so on (1). In clinical studies, CIs are typically given for the mean or median treatment effect, proportion of cure, difference in these outcome measures comparing two treatments, and so on. Practically for anything for which an estimate from

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

CONFIDENCE INTERVAL

(a)

Figure 2. Confidence band for a regression line (a) and for a ROC curve (b)

(b)

value, whereas L and U, as they depend on the sample, are random variables. Therefore, the statement P(L < θ < U) = 1 − α can only be interpreted in relation to drawing samples repeatedly and recalculating L and U each time. A small α corresponds to a low proportion of failures in repeated application of the method (i.e., it indicates that the method is good enough and makes mistakes as seldom as with a rate of α). However, considering one particular sample and the CI calculated from that sample, say when L = 12.1 and U = 13.5, the probability P(12.1 < θ < 13.5) is meaningless. Clearly, it is either true of false, given that also θ is a fixed value. To emphasize this fact, some authors suggest that in reporting the results one should avoid saying ‘‘the unknown parameter lies between 12.1 and 13.5 with 95% probability’’; one should rather say ‘‘with 95% confidence’’ or the CI should be reported as shown in the examples above. In the Bayesian model (in which even θ is regarded as a random variable), the corresponding interval estimates are the credible interval (i.e., the range of parameter values with the highest posterior probability) and Fisher’s fiducial interval. However, the calculation as well as the interpretation of these notions are different from each other, and quite different from that of a frequentist CI. If n unknown parameters to estimate exist (i.e., a parameter vector θ ∈ Rn ), one may want to construct a simultaneous confidence region (or confidence set) S ⊆ Rn with the property P(θ ∈ S) = 1 − α, rather than to give a CI for each parameter separately. Such questions come up most naturally in multiparameter models, for example in regression models. A basic example of a two-dimensional confidence region develops when one has

2.5%

95%

2.5%

(a) 2%

95%

3%

(b) 95%

5% (c)

5%

95% (d)

Figure 3. Symmetric (a), asymmetric (b), and onesided (c, d) confidence intervals

variance confidence region s 2obs

for (m, s 2)

mobs

mean

Figure 4. Joint confidence region (gray shaded area) for the mean and variance of a normal vari2 denote the sample mean and able (µobs and σobs variance)

a sample from a normal distribution with unknown µ and σ 2 , and one would like to construct a joint confidence set for the parameter pair (µ, σ 2 ), rather than two separate confidence intervals for µ and σ 2 (2). Such a confidence set (Fig. 4) can be used to derive CIs for functions of the parameters, for example, for the variance/mean ratio.

CONFIDENCE INTERVAL

In nonparametric inference, as no ‘‘parameters of the distribution’’ exist (in this case, neither the mean nor the variance are regarded as parameters of the distribution in the strict statistical sense), functionals of the distribution play the role of the above ‘‘unknown parameters.’’ For the nonstatistician user, however, it does not make any difference. CIs are closely related to statistical tests. Given a (1 − α)-level two-sided CI for a parameter θ , it is easy to convert it to a test for H0 : θ = θ0 , H1 : θ = θ0 , simply rejecting H 0 if θ 0 is not contained in the CI, that is, taking the CI as the acceptance region of the test (for a one-tailed test, a one-sided CI is needed). The resulted test has a Type I error rate of α. It works in the opposite direction as well, that is, tests can serve as the basis for CI construction (see the test inversion method below). 1 THE ROLE OF CONFIDENCE INTERVALS IN CLINICAL TRIALS In the last few decades, there has been a tendency toward wider use of CIs in reporting the results from clinical trials as well as from other studies (3–7). The main advantage of reporting CIs instead of (or even better, together with) P-values lies in better quantification of the effect size (the confidence limits are easy to compare with the clinically relevant effect size) and in direct indication of the uncertainty of the results (the wider the CI, the greater the uncertainty). Confidence intervals are particularly important in equivalence trials. To verify the equivalence of a new experimental treatment to the standard one, it must be shown that the difference between the treatment effects Ex and Es remains below a prespecified limit T defining the range of equivalence. Here, one can take either absolute or relative difference [i.e., equivalence can be defined either as −T < Ex − Es < T (absolute difference) or as 1 − T < Ex /Es < 1 + T (relative difference)]. For example, using relative difference with a range of equivalence of ±10% can be formally written as 0.90 < Ex /Es < 1.10. Then, a common method of analysis is to construct a CI for the difference Ex − Es or Ex /Es based

−T

0

3

T

(a) −T

0 (b)

T

−T

0 (c)

T

0

T

−T

(d) Figure 5. Possible locations of a confidence interval for the difference of treatment effects in an equivalence trial (if the true difference lies between the limits −T and T, treatments are regarded as equivalent)

on the observed sample, and to accept treatment equivalence if this CI is fully contained in the interval (−T, T) or (1 − T, 1 + T). The rationale behind this approach is that one would like to know whether the unknown true difference (rather than the difference between the particular observed samples) lies in the range of equivalence. According to this method, the interpretation of the CIs in Fig. 5 is as follows. a. A good evidence for equivalence. b. The new treatment is less effective than the standard one. c. The new treatment is noninferior (i.e., it may be either more effective or equivalent). d. The trial failed to have enough power to allow any definite conclusion concerning the relation between treatment effects. A larger trial is needed. Confidence intervals are also helpful in the interpretation of the results of superiority trials. Let D denote the smallest difference between the experimental and the standard treatment that is still clinically relevant. Then, the CI for the difference between treatment effects (Ex − Es ) (absolute difference) allows drawing the following conclusions (Fig. 6).

4

CONFIDENCE INTERVAL

many cases (e.g., because of the discreteness of the distribution, see Fig. 7), global measures were introduced to evaluate the performance of the various methods. The tra(b) ditional confidence coefficient, also called minimum coverage, is defined as the minimum (or infimum), whereas the mean coverage (c) (8) is the average of the coverage probability over the whole parameter range (e.g., for the binomial parameter, over the [0,1] (d) range). An important difference exists in the interpretation of these measures. The confidence coefficient can be interpreted as a lower (e) bound of the coverage rate, guaranteed by the procedure. Mean coverage, however, being 0 D based on averaging over the whole parameter (f) set, does not have any meaningful interpretation in relation with a single problem, in Figure 6. Possible locations of a confidence interwhich the parameter is assumed to be a cerval for the difference of treatment effects in a tain fixed value. (Note that, in the Bayesian superiority trial (a difference greater than D is model, averaging could be naturally made regarded as clinically relevant) with respect to the prior distribution of the parameter.) A CI constructing method is said to be cona. Experimental treatment is definitely worse. servative if its minimum coverage is never b. Experimental treatment is either worse less than the nominal and anticonservaor the difference is clinically irrelevant. tive otherwise. Although conservatism is a c. No clear conclusion: A larger trial is desirable property as compared with antineeded. conservatism, it is unsatisfactory if a method d. Experimental treatment is better but is too conservative— say, if at the nomthe difference is clinically irrelevant. inal level of 95%, the method produces a CI with actual coverage of 98%—because it e. Experimental treatment is better: The results in unnecessarily wide intervals. A difference may or may not be clinically way to control conservatism is to determine relevant. the actual coverage and to adjust it if necf. Experimental treatment is definitely betessary. Reiczigel (9) describes a computer ter: The difference is clinically relevant. intensive level-adjustment procedure varying the nominal level iteratively until the confidence coefficient gets close enough to the 2 PROPERTIES AND EVALUATION OF desired level. The procedure can be combined PERFORMANCE with any reasonable CI construction method, and it can also be applied for adjusting the The coverage probability of a CI for a parammean coverage. eter is defined as the probability that the The notion ‘‘exactness’’ of a CI is associinterval contains the parameter. However, ated with various meanings in the literature. this probability may depend on the true value In one sense, exactness means that the CI of the parameter as well as on some nuiis based on the exact probability distribusance parameters. A good method produces a tion of the given test statistic. In another CI with coverage probabilities possibly close sense, exactness means the same as conserto the desired (nominal) level irrespective of vatism above (i.e., that the coverage probabilthe values of the parameters: Ideally, the ity reaches at least the nominal level for any coverage probability should always be equal parameter value). The third sense is that the to the nominal. Because it is impossible in coverage is strictly equal to the nominal level (a)

CONFIDENCE INTERVAL

5

Coverage 1.00 0.95 0.90 0.85 Figure 7. Actual coverage of the exact’’ 95% Clopper-Pearson CI for the binomial parameter P. Although minimum coverage is 95%, for most parameter values, the coverage probability is high above 95%

0.80

for all parameter values. Computer simulations by several investigators show that exact CIs meant in the first two senses may perform rather poorly (8, 10), which is because a method may be too conservative even if its minimum coverage is equal to the nominal one, because for some parameter values the coverage probability may well exceed its minimum (as it is the case in Fig. 7). Therefore, it is a little misleading to designate the property that the minimum coverage is equal to the nominal as exactness. In fact, a further necessary condition exists for a method to perform well: Its coverage probability should not show much fluctuation. The so-called asymptotic or ‘‘large-sample’’ CIs are based on an approximate probability distribution of the test statistic, and therefore only have approximately the stated confidence level and only for large samples. Whereas an exact CI (meant in the third sense above) is valid for any sample size, an asymptotic CI is valid only for a sufficiently large sample size—and it depends on the procedure as well as on the true value of the parameter to determine what ‘‘sufficiently large’’ means. Such approximations are most often made by the normal distribution, based on the Central Limit Theorem (see below).

0

0.2

0.4

0.6

0.8

1 true p

when estimation is made by the maximum likelihood method), an asymptotic CI can be obtained as the point estimate plus and minus the asymptotic standard error of the estimate multiplied by a critical value from the normal distribution. This is demonstrated by two examples: 1. Let x and s denote the mean and standard deviation of a variable, estimated from a sample of n. Then, a 95% asymptotic CI for the true √ mean can be obtained as x ± 1.96 s/ n. 2. Let ln(OR) denote the logarithm of the odds ratio estimated from a two-bytwo table, that is, ln(OR) = ln(ad) − ln(bc) where a, b, c, d denote the table cell counts. It can be proven that the asymptotic standard error of ln(OR) is
s = 1/a + 1/b + 1/c + 1/d. From this equation, a 95% asymptotic CI for the true ln(OR) is obtained as ln(OR) ± 1.96 s. The actual coverage of such a CI approaches the nominal if the sample size tends to infinity. Recommendations exist (for each case separately!) for what sample size is the approximation acceptable. 3.2 Test Inversion

3 METHODS FOR CONSTRUCTING CONFIDENCE INTERVALS 3.1 Normal Approximation In case of consistency and asymptotic normality of a point estimator (in most cases,

Test inversion can be applied if a hypothesis test exists for the parameter θ that one wants to estimate. Suppose one wants to invert a test of H0 : θ = θ0 based on a test statistic t, to get a 95% CI for θ . The basic idea is that a 95% confidence set should consist of

6

CONFIDENCE INTERVAL

test statistic t

t2 tobs t1

L

Q0

U

true parameter Q

all such θ 0 values for which H0 : θ = θ0 is not rejected at the 95% level. That is, 95% acceptance regions (t1 , t2 ) of H0 : θ = θ0 should be determined for all possible θ 0 , and the smallest and largest θ 0 for which the acceptance region contains the observed tobs should be taken (Fig. 8). Note that, for some tests, the endpoints of the acceptance regions do not increase monotonically with θ . In such cases, the set of those θ 0 values for which H0 : θ = θ0 is not rejected at a given level may not form a proper interval, but it may contain ‘‘holes.’’ Of course, taking the smallest and largest θ 0 always results in a proper interval. Properties of the CI are easy to derive from the properties of the acceptance regions. For example, an exact test (i.e., one having exact acceptance regions for all θ 0 ) results in an exact CI; if the acceptance regions of one test are contained in those of another test for all θ 0 , then even the CIs are contained in the intervals obtained from the other test, and so on. CIs obtained by inverting a likelihood ratio test, as well as those based on the socalled empirical likelihood, are reported to have good properties (11). 3.3 Bootstrap Resampling Bootstrap CIs are applied either in the nonparametric case (i.e., if one has no parametric model for the population or process that generated the observed sample) or if the sampling distribution of an estimator is difficult or impossible to determine analytically (e.g., for the correlation coefficient). When bootstrapping from the sample, the observed sample itself is used as an estimate

Figure 8. 95% acceptance region (t1 ,t2 ) of the test H0 : θ = θ0 (thick vertical line) and confidence interval (L,U) belonging to the observed value tobs of the test statistic (thick horizontal line). L and U are obtained as the smallest and largest value of θ, for which the 95% acceptance region contains tobs

of the underlying population. A large number of samples are taken from the original (observed) sample by sampling with replacement, and these bootstrap samples are used to obtain an approximation of the sampling distribution of the estimate—to determine bias, standard error, tail probabilities, and so on—or to construct bootstrap CIs (12). Although sometimes this determination can be made analytically, typically it is carried out by computer simulation. A variety of methods exist for constructing bootstrap CIs, and many improvements to these methods also exist, so one must consult the latest books and articles before implementing a method for a specific problem. REFERENCES 1. M. J. Gardner and D. G. Altman, Statistics with Confidence: Confidence Intervals and Statistical Guidelines. London: BMJ Books, 1989. 2. B. C. Arnold and M. Shavelle, Joint confidence sets for the mean and variance of a normal distribution. Amer. Stat. 1998; 52: 133–140. 3. Uniform Requirements for Manuscripts Submitted to Biomedical Journals. JAMA 1993; 269: 2282–2286. 4. Uniform Requirements for Manuscripts Submitted to Biomedical Journals. (2003). (online). Available: http://www.icmje.org. 5. M. J. Gardner and D. G. Altman, Confidence intervals rather than p-values: estimation rather than hypothesis testing. BMJ 1986; 292: 746–750. 6. L. R´ozsa, J. Reiczigel, and G. Majoros, Quantifying parasites in samples of hosts. J. Parasitol. 2000; 86: 228–232.

CONFIDENCE INTERVAL 7. J. A. C. Sterne and G. D. Smith, Sifting the evidence: what’s wrong with significance tests? BMJ 2001; 322: 226–231. 8. R. G. Newcombe, Two-sided confidence intervals for the single proportion: comparison of seven methods. Stat. Med. 1998; 17: 857–872. 9. J. Reiczigel, Confidence intervals for the binomial parameter: some new considerations. Stat. Med. 2003; 22: 611–621. 10. A. Agresti and B. A. Coull, Approximate is better than ‘‘exact’’ for interval estimation of binomial proportions. Amer. Stat. 1998; 52: 119–126. 11. A. B. Owen, Empirical Likelihood. London: Chapman and Hall/CRC, 2001. 12. B. Efron and R. Tibshirani, An Introduction to the Bootstrap. New York: Chapman & Hall, 1993.

7

CONFIDENCE INTERVALS AND REGIONS

idea is not entirely misleading. An example should help to explain the technical meaning.

Confidence intervals are used for interval estimation. Whether an interval estimate is required depends upon the reason for the statistical analysis. Consider the analysis of measurements of the compressive strength of test cylinders made from a batch of concrete. If we were concerned with whether the mean strength of the batch exceeds some particular value, our problem would be one of hypothesis testing∗ . Our conclusion might be to accept or to reject the hypothesis, perhaps with an associated degree of confidence. If a simple indication of the strength likely to be achieved under the particular conditions of test is required, the observed mean strength might be quoted as an estimate of the true mean strength. This is called point estimation∗ . Interval estimation is the quoting of bounds between which it is likely (in some sense) that the real mean strength lies. This is appropriate when it is desired to give some indication of the accuracy with which the parameter is estimated. A large number of statistical problems may be included in the classes of hypothesis testing, point estimation, or interval estimation. It must be pointed out that there are several schools of thought concerning statistical inference. To quote confidence intervals is the interval estimation method advocated by the most widely accepted of these schools, variously referred to as the Neyman-Pearson∗ , Neyman-Pearson-Wald, frequentist, or classical school. There are other ways of obtaining interval estimates and we will refer to them later. (See also BAYESIAN INFERENCE, FIDUCIAL INFERENCE, LIKELIHOOD, STRUCTURAL INFERENCE.)

Example 1. Suppose that some quantity is measured using a standard testing procedure. Suppose that the quantity has a welldefined true value µ, but that the measurement is subject to a normally distributed error that has known variance σ 2 . Let X denote the random variable that is the result of a single measurement and let x be a particular value for X. Now X is normally distributed with mean µ and variance σ 2 . Using the properties of the normal distribution we can make probability statements about X; e.g., Pr[µ − 1.96σ
X
µ + 1.96σ ] = 0.95. (1) We could rewrite this as Pr[X − 1.96σ
µ
X + 1.96σ ] = 0.95 (2) or Pr[µ ∈ (X − 1.96σ , X + 1.96σ )] = 0.95. (3) Although µ may appear to be the subject of statements (2) and (3), the probability distribution referred to is that of X, as was more obvious in statement (1). If X is observed to be x, we say that we have 95% confidence that x − 1.96σ
µ
x + 1.96σ or say that (x − 1.96σ , x + 1.96σ ) is a 95% confidence interval for µ. No probability statement is made about the proposition x − 1.96σ
µ
x + 1.96σ

(4)

involving the observed value, x, since neither x nor µ has a probability distribution. The proposition (4) will be either true or false, but we do not know which. If confidence intervals with confidence coefficient p were computed on a large number of occasions, then, in the long run, the fraction p of these confidence intervals would contain the true parameter value. (This is provided that the occasions are independent and that there is no selection of cases.)

BASIC IDEA OF A CONFIDENCE INTERVAL The term ‘‘confidence interval’’ has an intuitive meaning as well as a technical meaning. It is natural to expect it to mean ‘‘an interval in which one may be confident that a parameter lies.’’ Its precise technical meaning differs substantially from this (see Jones [13], Cox [7], and Dempster [9]) but the intuitive

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

CONFIDENCE INTERVALS AND REGIONS

CONFIDENCE INTERVALS BASED ON A SINGLE STATISTIC Many confidence intervals can be discussed in terms of a one-dimensional parameter θ and a one-dimensional statistic T(X), which depends upon a vector of observations X. A more general formulation will be given below under the heading ‘‘Confidence Regions.’’ Provided that T(X) is a continuous random variable, given probabilities α1 and α2 , it is possible to find T1 (θ ) and T2 (θ ) such that Pr[T(X)
T1 (θ )|θ ] = α1

This confidence interval may be described as a central confidence interval because α1 = α2 (= 0.025). Noncentral confidence intervals are seldom quoted except when we are primarily concerned about large values for the parameter or about small values. In such cases it is common to quote only a single confidence limit. In Example 2 Pr[T/θ < 2.73] = 0.05 or, equivalently, Pr[θ > 0.366T] = 0.05.

(5)

Thus (0, 0.366T) is a confidence interval for θ = σ 2 at confidence level 0.95.

and Pr[T(X)  T2 (θ )|θ ] = α2 .

(6)

DISCRETE DISTRIBUTIONS

In other words, T1 (θ ) and T2 (θ ) are as shown in Fig. 1. Another diagram that can be used to illustrate the functions T1 (θ ) and T2 (θ ) is Fig. 2. For every particular value of θ the probability that T lies between T1 (θ ) and T2 (θ ) is 1 − α1 − α2 . The region between the curves T = T1 (θ ) and T = T2 (θ ) is referred to as a confidence belt. In terms of Fig. 2, the basic idea of confidence intervals is to express confidence 1 − α1 − α2 that the point (θ , T) lies in the confidence belt after T has been observed. If T1 and T2 are well-behaved functions, they will have inverse functions θ2 and θ1 , as shown in the figure, and the three propositions ‘‘T1 (θ )
T(X)
T2 (θ ),’’ ‘‘θ lies in the confidence belt,’’ and ‘‘θ1 (T)
θ
θ2 (T)’’ will be equivalent. Thus (θ1 (T), θ2 (T)) is a (1 − α1 − α2 ) confidence interval for θ .

When the statistic T is a discrete∗ random variable it is generally not possible to find functions T1 and T2 such that (5) and (6) hold precisely. Instead, we ask that Pr[T(X)
T1 (θ )|θ ] be as large as possible but not greater than α1 and that Pr[T(X)  T2 (θ )|θ ] be as large as possible but not greater than α2 . The functions T1 and T2 define a confidence belt which generally has a staircase-shaped perimeter. Keeping [14, p. 98] and Kendall and Stuart [15, p. 105] give examples.

Example 2. Consider eight observations from a normal distribution with known mean µ and unknown variance σ 2 . Take θ = σ 2 , X = (X1 , . . . , X8 ), and T(X) = 8 i=1 (Xi − µ)2 .

= (1 − p)12 + 12p(1 − p)11

From the fact that T/θ has a χ 2 distribution with 8 degrees of freedom, we know that Pr[T/θ
2.18] = 0.025 and that Pr[T/θ  17.53] = 0.025. Thus we take T1 (θ ) = 2.18θ , T2 (θ ) = 17.53θ and calculate θ1 (T) = 0.057T, θ2 (T) = 0.46T. The interval (0.057T, 0.46T) may be quoted as a 95% confidence interval for σ 2 .

Example 3. Consider the problem of finding a 90% confidence interval for the probability, p, of success on each trial in a sequence of independent trials, if two successes are observed in 12 trials. Some calculation yields that Pr[number of successes
2|p]

+ 66p2 (1 − p)10 = 0.05 if p = 0.438 < 0.05 if p > 0.438 and Pr[number of successes  2|p] = 1 − (1 − p)12 − 12p(1 − p)11 = 0.05 if p = 0.03046 < 0.05 if p < 0.03046.

CONFIDENCE INTERVALS AND REGIONS

3

Figure 1. Illustration of the meanings of T1 and T2 for fixed θ.

Figure 2. Confidence limits for θ based on the statistic T.

Thus the required 90% confidence interval is (0.03046, 0.0348). (Although this method of construction does not make the probability of including the true value of p to be equal to 90%, it does ensure that this probability is not less than 90%.) NUISANCE PARAMETERS∗ AND SIMILAR REGIONS∗ Under some circumstances it is easy to find confidence intervals despite the presence of a nuisance parameter. Consider the following example. Example 4. Suppose that X1 , X2 , . . . , Xn are normally distributed with mean µ and variance σ 2 , both unknown. Let X and s denote the sample mean and √ sample standard deviation. Now (X − µ) n/s has a t-distribution with n − 1 degrees of freedom no matter what the value of σ 2 . Therefore, letting t denote the 1 − 12 α quantile∗ of that t-distribution. √ √ Pr[µ − ts/ n
X
µ + ts/ n] = 1 − α;

or, equivalently, √ √ Pr[X − ts/ n
µ
X + ts/ n] = 1 − α. √ √ The interval (X − ts/ n, X + ts/ n) is a confidence level for µ at confidence level 1 − α. The parameter σ 2 is described as a nuisance parameter because we are not interested in estimating it, but it does affect the probability distribution of the observations. The regions of the sample space of the form (X − as, X + as) are described as similar regions because the probability of each of them is independent of the parameters. Confidence regions are generally based on similar regions when they exist. However, they often do not exist. CONFIDENCE REGIONS Confidence regions are a generalization of confidence intervals in which the confidence set is not necessarily an interval. Let θ be a (possibly multidimensional) parameter and let  denote the set of possible values for

4

CONFIDENCE INTERVALS AND REGIONS

θ . Let X denote a random variable, generally vector-valued. A function I that gives a subset of  for a value x of X is said to be a confidence set estimator or a confidence region for θ with confidence coefficient p if Pr[θ ∈ I(X)] = p.

(7)

For any such confidence region, to reject the hypothesis θ = θ0 whenever θ0 is not in I(X) is a Neyman—Pearson hypothesis test which has probability 1 − p of wrongly rejecting the hypothesis θ = θ0 . Choosing between Possible Confidence Regions There may be many functions I such that Pr[θ ∈ I(X)|θ ] = p for every θ . How should we choose which to use? Within the formulation where confidence intervals are based on a single statistic T, the problem is essentially that of choosing a statistic on which to base the confidence intervals. Perhaps confidence intervals based on the sample median∗ would be better in some ways than confidence intervals based on the sample mean. A number of criteria have been advanced to help decide between alternative confidence regions. We discuss some of them briefly. Standard texts on theoretical statistics may be consulted for further details. Confidence intervals should be based on sufficient statistics (see SUFFICIENT STATISTICS) and should be found conditional on the value of ancillary statistics (see ANCILLARY STATISTICS —I). A confidence region I is said to be unbiased if Pr[θ1 ∈ I(X)|θ2 ]
p for all θ1 , θ2 ∈ . This means that wrong values for the parameter are not more likely to be included in the region I(X) than the correct values. The region I is said to be shorter, more accurate, or more selective than the region J if Pr[θ1 ∈ I(X)|θ2 ]
Pr[θ1 ∈ J(x)|θ2 ] for all θ1 , θ2 ∈ . Intuitively, this means that incorrect values for θ are more likely to be

in J than in I. More selective regions correspond to more powerful tests of hypotheses and unbiased regions correspond to unbiased tests when parametric hypotheses are rejected whenever the parameter does not lie in the confidence region. The term ‘‘more selective’’ is preferred to ‘‘shorter’’ (which stems from Neyman [19]) to avoid confusion with the expected length of confidence intervals. For complex problems it may be difficult or impossible to apply some of these and other criteria. Sometimes it may only be possible to show that a particular confidence region is optimal in some sense within a particular class of regions, such as those invariant in some way. Different criteria sometimes suggest different regions. There is no completely general way of deciding which confidence interval to use. CRITICISMS OF THE THEORY OF CONFIDENCE INTERVALS There have been many arguments about the foundations of statistical inference, and there will probably be many more. Three (not independent) criticisms of the theory of confidence intervals are mentioned below. Note that they are criticisms of the frequentist school of thought, not merely of confidence intervals, which are the interval estimation technique used by that school. Likelihood Principle∗ The likelihood principle states that the ‘‘force’’ of an experiment should depend only upon the likelihood function, which is the probability density for the results obtained as a function of the unknown parameters. Many people find this principle compelling. Pratt [20] presents a persuasive defense of it in an entertaining way. Confidence interval theory violates the likelihood principle essentially because confidence intervals are concerned with the entire sample space. Coherence∗ It has been shown in several ways (e.g., Savage [23]), using various simple coherence conditions, that inference must be Bayesian if it is to be coherent. This means that every

CONFIDENCE INTERVALS AND REGIONS

Neyman confidence interval procedure that is not equivalent to a Bayesian procedure violates at least one of each set of coherence properties. Conditional Properties For a confidence region I such that Pr[θ ∈ I(X)] = α

for all θ ,

if there is a subset C of the sample space and a positive number  such that either Pr[θ ∈ I(X)|X ∈ C]
α − 

Conditional properties of confidence intervals for practical problems are seldom as poor as for this example and those of Robinson [22]. Note particularly that the complements of relevant subsets are not necessarily relevant. However, such examples do illustrate the point made by Dempster [9], Hacking [11], and others that confidence coefficients are a good measure of uncertainty before the data have been seen, but may not be afterward. LINKS WITH BAYESIAN INFERENCE∗

for all θ

or Pr[θ ∈ I(X)|X ∈ C]  α + 

5

for all θ ,

then the set C is a relevant subset. The idea stems from Fisher’s use of the term ‘‘recognizable subset’’ [10] and was formalized by Buehler [6]. Some people argue that the existence of a relevant subset implies that the confidence coefficient α is not an appropriate measure of confidence that θ ∈ I(x) when it happens that x belongs to the relevant subset. Consider the following quite artificial example, in which there are only two possible parameter values and four values for a random variable that is observed only once. Example 5. Suppose that when θ = θ1 , Pr[X = 1] = 0.9, Pr[X = 2] = 0.01, Pr[X = 3] = 0.05, and Pr[X = 4] = 0.04, whereas when θ = θ2 , Pr[X = 1] = 0.02, Pr[X = 2] = 0.9, Pr[X = 3] = 0.03, and Pr[X = 4] = 0.05. The region
{θ1 } if X = 1 or X = 3 I(X) = . {θ2 } if X = 2 or X = 4 is a confidence region for θ with confidence coefficient 0.95. However, Pr[θ ∈ I(X)|X ∈ {1, 2}, θ ]  90/92 for both θ values

(8)

and Pr[θ ∈ I(X)|X ∈ {3, 4}, θ ]
5/8 for both θ values.

(9)

Thus both {1, 2} and {3, 4} are relevant subsets.

Bayesian confidence regions are derived by taking a prior distribution, usually considered to represent subjective belief about unknown parameters, modifying it using observed data and Bayes’ theorem∗ to obtain a posterior distribution, and quoting a region of the parameter space which has the required probability according to the posterior distribution∗ (see BAYESIAN INFERENCE). Bayesian procedures satisfy most coherence principles, satisfy the likelihood principle, and have good conditional properties. However, their conclusions depend upon the arbitrarily or subjectively chosen prior distribution, not merely upon the data, and this is widely considered to be undesirable. A clear distinction must be made between proper and improper Bayesian procedures. Proper Bayesian procedures are those based on prior distributions which are proper (i.e., are probability distributions) and which use bounded loss and utility functions should loss or utility functions be required. Other Bayesian procedures are called improper and sometimes lack some of the desirable properties of proper Bayesian procedures. However, they are often used because they are more tractable mathematically. The bases of the frequentist and Bayesian schools of thought are quite different. However, many statistical procedures that are widely used in practice are both confidence interval procedures and improper Bayesian procedures. (see Bartholomew [1], Jeffreys [12], de Groot [8], and Lindley [17]). Of direct interest to people using the confidence intervals that may also be derived as improper Bayesian interval estimates is that the alternative derivation is often sufficient to ensure that these confidence intervals have

6

CONFIDENCE INTERVALS AND REGIONS

most of the desirable properties of proper Bayesian procedures. An exception is that there are relevant subsets for the usual confidence intervals based on the t-distribution∗ for the unknown mean of a normal distribution when the variance is also unknown (see Brown [4]. RELATIONSHIP TO FIDUCIAL INFERENCE∗ Fiducial inference generally proceeds by finding pivotal variables∗ , functions of both random variables and parameters which have a distribution that is independent of all parameters, and assuming that those pivotal variables have the same distribution after the random variables have been observed. Given the observed values of the random variables, the distribution of the pivotal variables implies a distribution for the parameters, called a fiducial distribution. To the extent that fiducial inference and confidence intervals both involve asserting faith, after seeing the data, in statements for which probabilities could be quoted before seeing the data they are similar theories. Bartlett [2] has argued that resolving the difference between these two theories is less important than resolving the difference between the pair of them and Bayesian methods. The clearest point of disagreement between them is that they support different solutions to the Behrens—Fisher problem∗ .

Example 6. Suppose that we wish to discriminate between two simple hypotheses, H0 : X has a standard normal distribution and H1 : X is distributed normally with mean 3 and unit variance, on the basis of a single observation. A standard Neyman—Pearson procedure would be to accept H0 (or fail to reject H1 ) is X
1.5 and to accept H1 if X > 1.5, and note that the probability of being correct is 0.933 as the measure of conclusiveness. That the same conclusiveness is expressed when X = 1.6 and when X = 3.6 seems unsatisfactory to Kiefer. Kiefer’s idea is to partition the sample space and to evaluate the conclusiveness of a statistical procedure conditionally for each subset of the partition. Here the sample space might be partitioned into three sets: (−∞, 0] ∪ (3, ∞), (0, 1] ∪ (2, 3], and (1, 2]. Conditionally on X being in the various sets, the probabilities of the decision to accept H0 or H1 being correct are 0.9973, 0.951, and 0.676. These could be considered to indicate ‘‘quite conclusive,’’ ‘‘reasonably conclusive,’’ and ‘‘slight’’ evidence, respectively. The article and discussion of Kiefer [16] refers to most other work relevant to conditional confidence regions. Most research has addressed the problem of which partitions of the sample space to use. Until the theory is further developed, it is difficult to see whether it will escape from the known weaknesses of Neyman—Pearson inference.

CONDITIONAL CONFIDENCE REGIONS

CONFIDENCE INTERVALS IN PRACTICAL STATISTICS

Brownie and Kiefer [5] consider that one of the weaknesses of Neyman—Pearson∗ methodology is that classical procedures generally do not give a measure of conclusiveness, which depends upon the data observed. Most other schools of thought do vary their measure of conclusiveness with the data. Kiefer [16] has developed a theory of conditional confidence which extends Neyman—Pearson methodology to allow both a data-dependent measure of conclusiveness and a frequency interpretation of this measure. The basic idea is most easily explained by an example of testing between two hypotheses. (See also CONDITIONAL INFERENCE.)

Confidence intervals are widely used in practice, although not as widely supported by people interested in the foundations of statistics. One reason for this dominance is that the most readily available statistical computer programs are based on the methods of the Neyman—Pearson school. Another reason is that many common confidence intervals (those based on normal, t, and binomial distributions) may also be derived as improper Bayesian procedures and do not suffer from most of the possible weaknesses of confidence intervals. These common procedures have some robustness∗ with respect to the vagaries of inference theory and may therefore be used without worrying very much

CONFIDENCE INTERVALS AND REGIONS

about the theory behind a particular derivation of them. Furthermore, it is fairly safe to use the intuitive notion of confidence rather than the restricted technical notion in such cases. When interpreting confidence intervals for several comparable parameters it should be noted that for two confidence intervals to overlap does not imply that the confidence interval for the difference between the two parameters would include the point zero. Also note that comparing more than two parameters at a time requires special theory (see MULTIPLE COMPARISONS —I). REFERENCES 1. Bartholomew, D. J. (1965). Biometrika, 52, 19–35. 2. Bartlett, M. S. (1965). J. Amer. Statist. Ass., 60, 395–409. 3. Birnbaum, A. (1962). J. Amer. Statist. Ass., 57, 269–326. (Very difficult to read.) 4. Brown, L. D. (1967). Ann. Math. Statist., 38, 1068–1071. 5. Brownie, C. and Kiefer, J. (1977). Commun. Statist. A—Theory and Methods, 6, 691–751. 6. Buehler, R. J. (1959). Ann. Math. Statist., 30, 845–863. (Fundamental reference on conditional properties of statistical procedures.) 7. Cox, D. R. (1958). Ann. Math. Statist., 29, 357–372. 8. de Groot, M. H. (1973). J. Amer. Statist. Ass., 68, 966–969. 9. Dempster, A. P. (1964). J. Amer. Statist. Ass., 59, 56–66. 10. Fisher, R. A. (1956). Statistical Methods and Scientific Inference, Oliver and Boyd, Edinburgh. (See p. 32.) 11. Hacking, I. (1965). Logic of Statistical Inference. Cambridge University Press, Cambridge. 12. Jeffreys, H. (1940). Annals of Eugenics, 10, 48–51. 13. Jones, H. L. (1958). J. Amer. Statist. Ass., 53, 482–490. 14. Keeping, E. S. (1962). Introduction to Statistical Inference. Van D. Nostrand, Princeton, N. J. 15. Kendall, M. G. and Stuart, A. (1961). The Advanced Theory of Statistics, Vol. 2: Inference and Relationship. Charles Griffin, London.

7

16. Kiefer, J. (1977). J. Amer. Statist. Ass., 72, 789–827. 17. Lindley, D. V. (1965). Introduction to Probability and Statistics from a Bayesian Viewpoint, Part 2, Inference. Cambridge University Press, Cambridge, England. 18. Neyman, J. (1934). J. R. Statist. Soc. A, 97, 558–606. (Especially Note I, p. 589, and discussion by Fisher R. A. p. 614. Mainly of historical interest.) 19. Neyman, J. (1937). Philos. Trans. R. Soc. Lond. A, 236, 333–380. (Fundamental reference on confidence intervals. These papers by Neyman are reproduced in Neyman, J. A Selection of Early Statistical Papers of J. Neyman, Cambridge University Press, Cambridge, 1967.) 20. Pratt, J. W. (1962). J. Amer. Statist. Ass., 57, 314–316. 21. Rao, C. R. (1965). Linear Statistical Inference and Its Applications. Wiley, New York. 22. Robinson, G. K. (1975). Biometrika, 62, 155–161. (Contrived, but reasonably simple examples.) 23. Savage, L. J. (1954). The Foundations of Statistics. Wiley, New York. (Argues that Bayesian inference is the only coherent theory.) 24. Savage, L. J. (1962). The foundations of Statistical Inference (a Discussion). Methuen, London. See also BAYESIAN INFERENCE; CONDITIONAL INFERENCE; FIDUCIAL INFERENCE; and INFERENCE, STATISTICAL.

G. K. ROBINSON

CONFIRMATORY TRIALS

when such trials are run, strict adherence to protocols and standard operating procedures is mandatory. Reporting of confirmatory trials requires discussion of any unavoidable changes and their impact on the study (1). Confirmatory trials typically address only a limited number of questions, and fundamental components of the analysis and justification for the design of confirmatory trials are set out in the protocol (1).

RAFE JOSEPH MICHAEL DONAHUE Vanderbilt University Medical Center, Nashville, Tennessee

1 DEFINITION AND DESCRIPTION 1.1 What They Are Confirmatory clinical trials, as defined in ICH guideline E9 (1), are ‘‘adequately controlled trials in which the hypotheses are stated in advance and evaluated.’’ ICH guideline E8 (2) partitions clinical studies according to objectives; ‘‘Therapeutic Confirmatory’’ studies are one class, along with ‘‘Human Pharmacology,’’ ‘‘Therapeutic Exploratory,’’ and ‘‘Therapeutic Use.’’ These classes correspond to a partition of the drug development process into what are typically viewed as four phases: Phase I (human pharamcology), Phase II (therapeutic exploratory), Phase III (therapeutic confirmatory), and Phase IV (therapeutic use). As such, confirmatory clinical trials are most often considered Phase III trials. These Phase III therapeutic confirmatory trials have objectives that are typically demonstration or confirmation of efficacy, establishment of a safety profile, construction of an adequate basis for assessing the benefit/risk relationship in order to support licensing, or establishment of a dose-response relationship. ICH E9 continues: ‘‘As a rule, confirmatory trials are necessary to provide firm evidence of efficacy or safety. In such trials the key hypothesis of interest follows directly from the trial’s primary objective, is always predefined, and is the hypothesis that is subsequently tested when the trial is complete. In a confirmatory trial it is equally important to estimate with due precision the size of the effects attributable to the treatment of interest and to relate these effects to their clinical significance.’’ This prespecification of the hypothesis that will be used to examine the primary objective is the key component of confirmatory trials. Since confirmatory trials are intended to provide firm evidence in support of claims,

1.2 What They Are Not Confirmatory trials are not exploratory trials. Although exploratory trials have clear and precise objectives like all clinical trials, these objectives may not lead to simple tests of predefined hypotheses. Such trials cannot, by themselves, provide sufficient evidence of efficacy; yet they may contribute to the full body of evidence (1). 2

CONSEQUENCES

In January 2005, the innovative pharmaceutical industry, represented worldwide by the European Federation of Pharmaceutical Industries and Associations (EFPIA), the International Federation of Pharmaceutical Manufacturers and Associations (IFPMA), the Japanese Pharmaceutical Manufacturers Association (JPMA), and the Pharmaceutical Research and Manufacturers of America (PhRMA), committed itself to increasing the transparency of their members’ clinical trials (3). One outcome of this commitment was the establishment of a registry of the results from confirmatory clinical trials. An industrywide registry, www.clinicalstudyresults.org, houses the results from ‘‘all clinical trials, other than exploratory trials’’ (3). Another registry of clinical trials, www .clinicaltrials.gov, contains lists of ongoing confirmatory and exploratory clinical trials. 3

ISSUES AND CONTROVERSIES

D’Agostino and Massaro (4) address issues involved in the design, conduct, and analysis of confirmatory clinical trials. Among the

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

CONFIRMATORY TRIALS

issues discussed in detail are study objectives, target populations, sample population, efficacy variables, control groups, study design, comparisons, sample size, trial monitoring, data analysis sets, unit of analysis, missing data, safety, subsets, and clinical significance. All of these issues need to be addressed to develop successfully a confirmatory clinical trial. Parmar et al. (5) address issues around whether a specific, unique, confirmatory trial is necessary. They present a Bayesian statistical framework in which differences are assessed. Through this, they provide a method to assess the need for a confirmatory trial. Their examples revolve around non-smallcell lung cancer. Their argument centers on the belief that a major reason in performing confirmatory randomized clinical trials is a prior skepticism over whether the new treatment is likely to be clinically worthwhile. They argue that it might be wiser to accept the treatment into practice than to wait perhaps years to accrue patients and carry out a confirmatory trial. An editorial by Berry (6) discusses the pros and cons of the Parmer et al. approach. He concludes that confirmatory trials are certainly important from a scientific perspective, but whether they are ethical is a different matter. He points out the fact that earlier trials may have been carried out in different settings and that a confirmatory trial might likely show a different treatment effect—usually a smaller one—than was observed in the exploratory trials, due to regression to the mean. This regression to the mean bias makes it difficult to assess the magnitude of the treatment benefit as observed in a single confirmatory trial. REFERENCES 1. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH). ICH Guideline E9: General considerations for clinical trials. ICH web site. Available: www.ich.org, Accessed 2006.12.31. 2. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH). ICH

Guideline E8: Statistical Principles for Clinical Trials. ICH web site. Available: www.ich.org, Accessed 2006.12.31. 3. Joint Position on the Disclosure of Clinical Trial Information via Clinical Trial Registries and Databases. European Federation of Pharmaceutical Industries and Associations (EFPIA) web site. Available: www.efpia.org, Accessed 2006.12.31. 4. R. B. D’Agostino Sr, and J. M. Massaro, New devlopments in medical clinical trials. J Dent Res. 2004; 83 Spec No C: C18-24. 5. M. K. B. Parmar, R. S. Ungerleider, R. Simon, Assessing whether to perform a confirmatory randomized clinical trial. J. Natl. Cancer Inst. 1996; 88: 1645-1651. 6. D. A. Berry, When is a confirmatory randomized clinical trial needed? J. Natl. Cancer Inst. 1996; 88, No. 22: 1606-1607.

CROSS-REFERENCES Phase III Trials Pivotal Trial

CONFOUNDING

the agents whose properties we wish to study (emphasis added).

SANDER GREENLAND University of California, Los Angeles, CA, USA

It should be noted that, in Mill’s time, the word ‘‘experiment’’ referred to an observation in which some circumstances were under the control of the observer, as it still is used in ordinary English, rather than to the notion of a comparative trial. Nonetheless, Mill’s requirement suggests that a comparison is to be made between the outcome of his experiment (which is, essentially, an uncontrolled trial) and what we would expect the outcome to be if the agents we wish to study had been absent. If the outcomes is not as one would expect in the absence of the study agents, then his requirement ensures that the unexpected outcome was not brought about by extraneous circumstances. If, however, those circumstances do bring about the unexpected outcome, and that outcome is mistakenly attributed to effects of the study agents, then the mistake is one of confounding (or confusion) of the extraneous effects with the agent effects. Much of the modern literature follows the same informal conceptualization given by Mill. Terminology is now more specific, with ‘‘treatment’’ used to refer to an agent administered by the investigator and ‘‘exposure’’ often used to denote an unmanipulated agent. The chief development beyond Mill is that the expectation for the outcome in the absence of the study exposure is now almost always explicitly derived from observation of a control group that is untreated or unexposed. For example, Clayton & Hills (2) state of observational studies,

The word confounding has been used to refer to at least three distinct concepts. In the oldest usage, confounding is a bias in estimating causal effects (see Causation). This bias is sometimes informally described as a mixing of effects of extraneous factors (called confounders) with the effect of interest. This usage predominates in nonexperimental research, especially in epidemiology and sociology. In a second and more recent usage, confounding is a synonym for noncollapsibility, although this usage is often limited to situations in which the parameter of interest is a causal effect. In a third usage, originating in the experimental-design literature, confounding refers to inseparability of main effects and interactions under a particular design. The term aliasing is also sometimes used to refer to the latter concept; this usage is common in the analysis of variance literature. The three concepts are closely related and are not always distinguished from one another. In particular, the concepts of confounding as a bias in effect estimation and as noncollapsibility are often treated as identical, although there are many examples in which the two concepts diverge (8,9,14); one is given below. 1 CONFOUNDING AS A BIAS IN EFFECT ESTIMATION 1.1 Confounding

. . . there is always the possibility that an important influence on the outcome . . . differs systematically between the comparison [exposed and unexposed] groups. It is then possible [that] part of the apparent effect of exposure is due to these differences, [in which case] the comparison of the exposure groups is said to be confounded (emphasis in the original).

A classic discussion of confounding in which explicit reference is made to ‘‘confounded effects’’ is Mill [15, Chapter X] (although in Chapter III Mill lays out the primary issues and acknowledges Francis Bacon as a forerunner in dealing with them). There, he lists a requirement for an experiment intended to determine causal relations:

In fact, confounding is also possible in randomized experiments, owing to systematic improprieties in treatment allocation, administration, and compliance. A further

. . . none of the circumstances [of the experiment] that we do know shall have effects susceptible of being confounded with those of

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

CONFOUNDING

and somewhat controversial point is that confounding (as per Mill’s original definition) can also occur in perfect randomized trials due to random differences between comparison groups (6,8). Various mathematical formalizations of confounding have been proposed. Perhaps the one closest to Mill’s concept is based on a formal counterfactual model for causal effects. Suppose our objective is to determine the effect of applying a treatment or exposure x1 on a parameter µ of population A, relative to applying treatment or exposure x0 . For example, A could be a cohort of breast-cancer patients, treatment x1 could be a new hormone therapy, x0 could be a placebo therapy, and the parameter µ could be the 5-year survival probability. The population A is sometimes called the target population or index population; the treatment x1 is sometimes called the index treatment; and the treatment x0 is sometimes called the control or reference treatment (which is often a standard or placebo treatment). The counterfactual model assumes that µ will equal µA1 if x1 is applied, µA0 if x0 is applied; the causal effect of x1 relative to x0 is defined as the change from µA0 to µA1 , which might be measured as µA1 − µA0 or µA1 /µA0 . If A is observed under treatment x1 , then µ will equal µA1 , which is observable or estimable, but µA0 will be unobservable. Suppose, however, we expect µA0 to equal µB0 , where µB0 is the value of the outcome µ observed or estimated for a population B that was administered treatment x0 . The latter population is sometimes called the control or reference population. Confounding is said to be present if in fact µA0
= µB0 , for then there must be some difference between populations A and B (other than treatment) that is affecting µ. If confounding is present, a naive (crude) association measure obtained by substituting µB0 for µA0 in an effect measure will not equal the effect measure, and the association measure is said to be confounded. For example, if µB0
= µA0 , then µA1 − µB0 , which measures the association of treatments with outcomes across the populations, is confounded for µA1 − µA0 , which measures the effect of treatment x1 on population A. Thus, saying a measure of association such as

µA1 − µB0 is confounded for a measure of effect such as µA1 − µA0 is synonymous with saying the two measures are not equal. The preceding formalization of confounding gradually emerged through attempts to separate effect measures into a component due to the effect of interest and a component due to extraneous effects (1,4,10,12,13). These decompositions will be discussed below. One noteworthy aspect of the above formalization is that confounding depends on the outcome parameter. For example, suppose populations A and B have a different 5-year survival probability µ under placebo treatment x0 ; that is, suppose µB0
= µA0 , so that µA1 − µB0 is confounded for the actual effect µA1 − µA0 of treatment on 5-year survival. It is then still possible that 10-year survival, ν, under the placebo would be identical in both populations; that is, ν A0 could still equal ν B0 , so that ν A1 − ν B0 is not confounded for the actual effect of treatment on 10-year survival. (We should generally expect no confounding for 200-year survival, since no treatment is likely to raise the 200-year survival probability of human patients above zero.) A second noteworthy point is that confounding depends on the target population of inference. The preceding example, with A as the target, had different 5-year survivals µA0 and µB0 for A and B under placebo therapy, and hence µA1 − µB0 was confounded for the effect µA1 − µA0 of treatment on population A. A lawyer or ethicist may also be interested in what effect the treatment x1 would have had on population B. Writing µB1 for the (unobserved) outcome of B under treatment x1 , this effect on B may be measured by µB1 − µB0 . Substituting µA1 for the unobserved µB1 yields µA1 − µB0 . This measure of association is confounded for µB1 − µB0 (the effect of treatment x1 on 5-year survival in population B) if and only if µA1
= µB1 . Thus, the same measure of association, µA1 − µB0 , may be confounded for the effect of treatment on neither, one, or both of populations A and B. 1.2 Confounders A third noteworthy aspect of the counterfactual formalization of confounding is that it invokes no explicit differences (imbalances)

CONFOUNDING

between populations A and B with respect to circumstances or covariates that might influence µ (8). Clearly, if µA0 and µB0 differ, then A and B must differ with respect to factors that influence µ. This observation has led some authors to define confounding as the presence of such covariate differences between the compared populations. Nonetheless, confounding is only a consequence of these covariate differences. In fact, A and B may differ profoundly with respect to covariates that influence µ, and yet confounding may be absent. In other words, a covariate difference between A and B is a necessary but not sufficient condition for confounding. This point will be illustrated below. Suppose now that populations A and B differ with respect to certain covariates, and that these differences have led to confounding of an association measure for the effect measure of interest. The responsible covariates are then termed confounders of the association measure. In the above example, with µA1 − µB0 confounded for the effect µA1 − µA0 , the factors responsible for the confounding (i.e. the factors that led to µA0
= µB0 ) are the confounders. It can be deduced that a variable cannot be a confounder unless it can affect the outcome parameter µ within treatment groups and it is distributed differently among the compared populations (e.g. see Yule (23), who however uses terms such as ‘‘fictitious association’’ rather than confounding). These two necessary conditions are sometimes offered together as a definition of a confounder. Nonetheless, counterexamples show that the two conditions are not sufficient for a variable with more than two levels to be a confounder as defined above; one such counterexample is given in the next section. 1.3 Prevention of Confounding Perhaps the most obvious way to avoid confounding in estimating µA1 − µA0 is to obtain a reference population B for which µB0 is known to equal µA0 . Among epidemiologists, such a population is sometimes said to be comparable to or exchangeable with A with respect to the outcome under the reference treatment. In practice, such a population may be difficult or impossible to find. Thus, an

3

investigator may attempt to construct such a population, or to construct exchangeable index and reference populations. These constructions may be viewed as design-based methods for the control of confounding. Perhaps no approach is more effective for preventing confounding by a known factor than restriction. For example, gender imbalances cannot confound a study restricted to women. However, there are several drawbacks: restriction on enough factors can reduce the number of available subjects to unacceptably low levels, and may greatly reduce the generalizability of results as well. Matching the treatment populations on confounders overcomes these drawbacks and, if successful, can be as effective as restriction. For example, gender imbalances cannot confound a study in which the compared groups have identical proportions of women. Unfortunately, differential losses to observation may undo the initial covariate balances produced by matching. Neither restriction nor matching prevents (although it may diminish) imbalances on unrestricted, unmatched, or unmeasured covariates. In contrast, randomization offers a means of dealing with confounding by covariates not accounted for by the design. It must be emphasized, however, that this solution is only probabilistic and subject to severe constraints in practice. Randomization is not always feasible, and (as mentioned earlier) many practical problems, such as differential loss and noncompliance, can lead to confounding in comparisons of the groups actually receiving treatments x1 and x0 . One somewhat controversial solution to noncompliance problems is intention-totreat analysis, which defines the comparison groups A and B by treatment assigned rather than treatment received. Confounding may, however, affect even intentionto-treat analyses. For example, the assignments may not always be random, as when blinding is insufficient to prevent the treatment providers from protocol violations. And, purely by bad luck, randomization may itself produce allocations with severe covariate imbalances between the groups (and consequent confounding), especially if the study size is small (6,8,19). Block randomization can help ensure that random imbalances on

4

CONFOUNDING

the blocking factors will not occur, but it does not guarantee balance of unblocked factors. 1.4 Adjustment for Confounding Design-based methods are often infeasible or insufficient to prevent confounding. Thus there has been an enormous amount of work devoted to analytic adjustments for confounding. With a few exceptions, these methods are based on observed covariate distributions in the compared populations. Such methods can successfully control confounding only to the extent that enough confounders are adequately measured. Then, too, many methods employ parametric models at some stage, and their success may thus depend on the faithfulness of the model to reality. These issues cannot be covered in depth here, but a few basic points are worth noting. The simplest and most widely trusted methods of adjustment begin with stratification on confounders. A covariate cannot be responsible for confounding within internally homogeneous strata of the covariate. For example, gender imbalances cannot confound observations within a stratum composed solely of women. More generally, comparisons within strata cannot be confounded by a covariate that is constant (homogeneous) within strata. This is so regardless of whether the covariate was used to define the strata. Generalizing this observation to a regression context, we find that any covariate with a residual variance of zero conditional on the regressors cannot confound regression estimates of effect (assuming that the regression model is correct). A broader and more useful observation is that any covariate that is unassociated with treatment conditional on the regressors cannot confound the effect estimates; this insight leads directly to adjustments using a propensity score. Some controversy has existed about adjustment for covariates in randomized trials. Although Fisher asserted that randomized comparisons were unbiased, he also pointed out that they could be confounded in the sense used here (e.g. see Fisher [6, p. 49]). Fisher’s use of the word ‘‘unbiased’’ was unconditional on allocation, and therefore of little guidance for analysis of a given trial. The ancillarity

of the allocation naturally leads to conditioning on the observed distribution of any pretreatment covariate that can influence the outcome parameter. Conditional on this distribution, the unadjusted treatment–effect estimate will be biased if the covariate is associated with treatment; this conditional bias can be removed by adjustment for the confounders (8,18). Note that the adjusted estimate is also unconditionally unbiased, and thus is a reasonable alternative to the unadjusted estimate even without conditioning. 1.5 Measures of Confounding The parameter estimated by a direct unadjusted comparison of cohorts A and B is µA1 − µA0 . A number of authors have measured the bias (confounding) of the unadjusted comparison by (10,12) (µA1 − µB0 ) − (µA1 − µA0 ) = µA0 − µB0 . When the outcome parameters, µ, are risks (probabilities), epidemiologists use instead the analogous ratio µA0 µA1 /µB0 = µA1 /µA0 µB0 as a measure of bias (1,4,14); µA0 /µB0 is sometimes called the confounding risk ratio. The latter term is somewhat confusing because it is sometimes misunderstood to refer to the effect of a particular confounder on risk. This is not so, although the ratio does reflect the net effect of the differences in the confounder distributions of A and B. 1.6 Residual Confounding Suppose now that adjustment for confounding is done by subdividing the total study population (A + B) into K strata indexed by k. Let µA1k be the parameter of interest in stratum k of populations A and B under treatment x0 . The effect of treatment x1 relative to x0 in stratum k may be defined as µA1k − µA0k or µA1k /µA0k . The confounding that remains in stratum K is called the residual confounding in the stratum, and is measured by µA0k − µB0k or µA1k /µB0k .

CONFOUNDING

Like effects, stratum-specific residual confounding may be summarized across the strata in a number of ways, for example by standardization methods or by other weighted-averaging methods. As an illustration, suppose we are given a standard distribution p1 , . . . , pK for the stratum index k. In ratio terms, the standardized effect of x1 vs. x0 on A under this distribution is
pk µA1k k

RAA =


, pk µA0k

5

pose population k is given treatment xk , even though it could have been given some other treatment. The absolute effect of x1 vs. x2 on µ in population 1 is µ1 (x1 ) − µ1 (x2 ) = (x1 − x2 )β. Substitution of µ2 (x2 ), the value of µ in population 2 under treatment x2 , for µ1 (x2 ) yields µ1 (x1 ) − µ2 (x2 ) = α1 − α2 + (x1 − x2 )β, which is biased by the amount

k

whereas the standardized ratio comparing A with B is
pk µA1k k

RAB =


. pk µB0k

k

The overall residual confounding in RAB is thus
pk µA0k RAB k =
, RAA pk µB0k k

which may be recognized as the standardized ratio comparing A and B when both are given treatment x0 , using p1 , . . . , pK as the standard distribution. 1.7 Regression Formulations For simplicity, the above presentation has focused on comparing two populations and two treatments. The basic concepts extend immediately to the consideration of multiple populations and treatments. Paired comparisons may be represented using the above formalization without modification. Parametric models for these comparisons then provide a connection to more familiar regression models. As an illustration, suppose population differences and treatment effects follow the model µk (x) = αk + xβ, where the treatment level x may range over a continuum, and k indexes populations. Sup-

µ1 (x2 ) − µ2 (x2 ) = α1 − α2 . Thus, under this model no confounding will occur if the intercepts α k equal a constant α across populations, so that µk (x) = α + βx. When constant intercepts cannot be assumed and nothing else is known about the intercept magnitudes, it may be possible to represent our uncertainty about α k via the following mixed-effects model: µk (x) = α + xβ + k . Here, α k has been decomposed into α +  k , where  k has mean zero, and the confounding in µ1 (x1 ) − µ2 (x2 ) has become an unobserved random variable,  1 −  2 . Correlation of population membership k with xk leads to a correlation of  k with xk , which in turn leads to bias in estimating β. This bias may be attributed to or interpreted as confounding for β in the regression analysis. Confounders are now covariates that causally ‘‘explain’’ the correlation between  k and xk . In particular, confounders normally reduce the correlation of xk and  k when entered in the model. The converse is false, however: a variable that reduces the correlation of xk and  k when entered need not be a confounder; it may, for example, be a variable affected by both the treatment and the exposure. 2 CONFOUNDING AND NONCOLLAPSIBILITY Much of the statistics literature does not distinguish between the concept of confounding as described above and the concept

6

CONFOUNDING

of noncollapsibility. Nonetheless, the two concepts are distinct: for certain outcome parameters, confounding may occur with or without noncollapsibility and noncollapsibility may occur with or without confounding (8,9,14,17,20,22). Mathematically identical conclusions have been reached by other authors, albeit with different terminology in which noncollapsibility corresponds to ‘‘bias’’ and confounding corresponds to covariate imbalance (7,11). As an example of no collapsibility with no confounding, consider the response distributions under treatments x1 and x0 given in Table 1 for a hypothetical index population A, and the response distribution under treatment x0 given in Table 2 for a hypothetical reference population B. If we take the odds of response as the outcome parameter µ, we get 1460 = 2.70 µA1 = 540 and µA0 = µB0 =

1000 = 1.00. 1000

Table 1. Distribution of Responses for Population A, within Strata of Z and Ignoring Z, under Treatments x1 and x0 Number of Responses Under Subpopulation

x1

x0

Subpopulation Size

Z=1 Z=2 Z=3 Totals

200 900 360 1460

100 600 300 1000

400 1200 400 2000

Table 2. Distribution of Responses for Population B, within Strata of Z and Ignoring Z, under Treatment x0

Subpopulation

Number of Responses Under x0

Subpopulation Size

Z=1 Z=2 Z=3 Totals

200 200 600 1000

800 400 800 2000

There is thus no confounding of the odds ratio: µA1 /µA0 = µA1 /µB0 = 2.70/1.00 = 2.70. Nonetheless, the covariate Z is associated with response and is distributed differently in A and B. Furthermore, the odds ratio is not collapsible: within levels of Z, the odds ratios comparing A under treatment x1 with either A or B under x0 are (200/200)/(200/600) = (900/300)/(200/200) = (360/40)/(600/200) = 3.00, a bit higher than the odds ratio of 2.70 obtained when Z is ignored. The preceding example illustrates a peculiar property of the odds ratio as an effect measure: treatment x1 (relative to x0 ) elevates the odds of response by 170% in population A, yet within each stratum of Z it raises the odds by 200%. When Z is associated with response conditional on treatment but unconditionally unassociated with treatment, the stratum-specific effects on odds ratios will be further from the null than the overall effect if the latter is not null (7). This phenomenon is often interpreted as a ‘‘bias’’ in the overall odds ratio, but in fact there is no bias if one does not interpret the overall effect as an estimate of the stratum-specific effects. The example also shows that, when µ is the odds, the ‘‘confounding odds ratio’’ (µA1 /µB0 )/(µA1 /µA0 ) = µA0 /µB0 may be 1 even when the odds ratio is not collapsible over the confounders. Conversely, we may have µA0 /µB0
= 1 even when the odds ratio is collapsible. More generally, the ratio of crude and stratum-specific odds ratios does not equal µA0 /µB0 except in some special cases. When the odds are low, however, the odds will be close to the corresponding risks, and so the two ratios will approximate one another. The phenomenon illustrated in the example corresponds to the differences between cluster-specific and population-averaged (marginal) effects in nonlinear mixedeffects regression (16). Specifically, the clusters of correlated outcomes correspond to the strata, the cluster effects correspond to covariate effects, the cluster-specific treatment effects correspond to the stratumspecific log odds ratios, and the populationaveraged treatment effect corresponds to the crude log odds ratio. Results of Gail (7) imply that if the effect measure is the difference or ratio of response proportions, then the above phenomenon–

CONFOUNDING

noncollapsibility over Z without confounding by Z–cannot occur, nor can confounding by Z occur without noncollapsibility over Z. More generally, when the effect measure is an expectation over population units, confounding by Z and noncollapsibility over Z are algebraically equivalent. This equivalence may explain why the two concepts are often not distinguished. 3 CONFOUNDING IN EXPERIMENTAL DESIGN Like the bias definition, the third usage of confounding stems from the notion of mixing of effects. However, the effects that are mixed are main (block) effects and interactions (or different interactions) in a linear model, rather than effects in the nonparametric sense of a counterfactual model. This definition of confounding differs even more markedly from other definitions in that it refers to an intentional design feature of certain experimental studies, rather than a bias. The topic of confounded designs is extensive; some classic references are Fisher (6), Cochran & Cox (3), Cox (5), and Scheff´e (21). Confounding can serve to improve efficiency in estimation of certain contrasts and can reduce the number of treatment groups that must be considered. The price paid for these benefits is a loss of identifiability of certain parameters, as reflected by aliasing of those parameters. As a simple example, consider a situation in which we wish to estimate three effects in a single experiment: that of treatments x1 vs. x0 , y1 vs. y0 , and z1 vs. z0 . For example, in a smoking cessation trial these treatments may represent active and placebo versions of the nicotine patch, nicotine gum, and buspirone. With no restrictions on number or size of groups, a fully crossed design would be reasonable. By allocating subjects to each of the 23 = 8 possible treatment combinations, one could estimate all three main effects, all three two-way interactions, and the threeway interaction of the treatments. Suppose, however, that we were restricted to use of only four treatment groups (e.g. because of cost or complexity considerations).

7

A naive approach would be to use groups of equal size, assigning one group to placebos only (x0 , y0 , z0 ) and the remaining three groups to one active treatment each: (x1 , y0 , z0 ), (x0 , y1 , z0 ), and (x0 , y0 , z1 ). Unfortunately, with a fixed number N of subjects available, this design would provide only N/4 subjects under each active treatment. As an alternative, consider the design with four groups of equal size with treatments (x0 , y0 , z0 ), (x1 , y1 , z0 ), (x1 , y0 , z1 ), and (x0 , y1 , z1 ). This fractional factorial design would provide N/2 subjects under each active treatment, at the cost of confounding main effects and interactions. For example, no linear combination of group means containing the main effect of x1 vs. x0 would be free of interactions. If one could assume that all interactions were negligible, however, this design could provide considerably more precise estimates of the main effects than the naive four-group design. To see these points, consider the following linear model: µXYZ = α + β1 X + β2 Y + β3 Z + γ1 XY + γ2 XZ + γ3 YZ + δXYZ, where X, Y, and Z equal 1 for x1 , y1 , and z1 , and 0 for x0 , y0 , and z0 , respectively. The group means, in the fractional factorial design are then µ000 = α, µ110 = α + β1 + β2 + γ1 , µ101 = α + β1 + β3 + γ2 , µ011 = α + β2 + β3 + γ3 . Treating the means as observed and the coefficients as unknown, the above system is underidentified. In particular, there is no solution for any main effect β j in terms of the means µijk . Nonetheless, assuming all γ j = 0 yields immediate solutions for all the β j . Additionally assuming a variance of σ 2 for each estimated group mean yields that the main-effect estimates under this design would have variances of σ 2 , as opposed to 2σ 2 for the main-effect estimates from the naive four-group design of the same size. For

8

CONFOUNDING

example, under the confounded fractional factorial design (assuming no interactions) βˆ1 =

µˆ 110 + µˆ 101 − µˆ 000 − µˆ 011 , 2

so var(βˆ1 ) = 4σ 2 /4 = σ 2 , whereas under the naive design, βˆ = µˆ 100 − µˆ 000 so var(βˆ1 ) = 2σ 2 . Of course, the precision advantage of the confounded design is purchased by the assumption of no interaction, which is not needed by the naive design. REFERENCES 1. Bross, I. D. J. (1967). Pertinency of an extraneous variable, Journal of Chronic Diseases 20, 487–495. 2. Clayton, D. & Hills, M. (1993). Statistical Models in Epidemiology. Oxford University Press, New York. 3. Cochran, W. G. & Cox, G. M. (1957). Experimental Designs, 2nd Ed. Wiley, New York. 4. Cornfield, J., Haenszel, W., Hammond, W. C., Lilienfeld, A. M., Shimkin, M. B. & Wynder, E. L. (1959). Smoking and lung cancer: recent evidence and a discussion of some questions, Journal of the National Cancer Institute 22, 173–203. 5. Cox, D. R. (1958). The Planning of Experiments. Wiley, New York.

10. Groves, E. R. & Ogburn, W. F. (1928). American Marriage and Family Relationships. Henry Holt & Company, New York, pp. 160–164. 11. Hauck, W. W., Neuhaus, J. M., Kalbfleisch, J. D. & Anderson, S. (1991). A consequence of omitted covariates when estimating odds ratios, Journal of Clinical Epidemiology 44, 77–81. 12. Kitagawa, E. M. (1955). Components of a difference between two rates, Journal of the American Statistical Association 50, 1168–1194. 13. Miettinen, O. S. (1972). Components of the crude risk ratio, American Journal of Epidemiology 96, 168–172. 14. Miettinen, O. S. & Cook, E. F. (1981). Confounding: essence and detection, American Journal of Epidemiology 114, 593–603. 15. Mill, J. S. (1843). A System of Logic, Ratiocinative and Inductive. Reprinted by Longmans, Green & Company, London, 1956. 16. Neuhaus, J. M., Kalbfleisch, J. D. & Hauck, W. W. (1991). A comparison of cluster-specific and population-averaged approaches for analyzing correlated binary data, International Statistical Review 59, 25–35. 17. Pearl, J. (2000). Causality. Cambridge University Press, New York, Ch. 6. 18. Robins, J. M. & Morgenstern, H. (1987). The mathematical foundations of confounding in epidemiology, Computers and Mathematics with Applications 14, 869–916. 19. Rothman, K. J. (1977). Epidemiologic methods in clinical trials, Cancer 39, 1771–1775.

6. Fisher, R. A. (1935). The Design of Experiments. Oliver & Boyd, Edinburgh. 7. Gail, M. H. (1986). Adjusting for covariates that have the same distribution in exposed and unexposed cohorts, in Modern Statistical Methods in Chronic Disease Epidemiology, S. H. Moolgavkar & R. L. Prentice, eds. Wiley, New York. 8. Greenland, S. & Robins, J. M. (1986). Identifiability, exchangeability, and epidemiological confounding, International Journal of Epidemiology 15, 413–419.

20. Rothman, K. J. & Greenland, S. (1998). Modern Epidemiology, 2nd ed. Lippincott, Philadelphia, Ch. 4. 21. Scheff´e, H. A. (1959). The Analysis of Variance. Wiley, New York. 22. Wickramaratne, P. & Holford, T. (1987). Confounding in epidemiologic studies: the adequacy of the control groups as a measure of confounding, Biometrics 43, 751–765.

9. Greenland, S., Robins, J. M. & Pearl, J. (1999). Confounding and collapsibility in causal inference. Statistical Science 14, 29–46.

23. Yule, G. U. (1903). Notes on the theory of association of attributes in statistics, Biometrika 2, 121–134.

CONSORT A report of a randomized controlled trial (RCT) should convey to the reader, in a transparent manner, why the study was undertaken, and how it was conducted and analyzed. To assess the strengths and limitations of an RCT, the reader needs and deserves to know the quality of its methodology. Despite several decades of educational efforts, RCTs still are not being reported adequately [2, 5, 10]. The Consolidated Standards of Reporting Trials (CONSORT) statement, published in the Journal of the American Medical Association in 1996 [1], was developed to try to help rectify this problem. The CONSORT statement was developed by an international group of clinical trialists,

statisticians, epidemiologists and biomedical editors. The CONSORT statement is one result of previous efforts made by two independent groups, the Standards of Reporting Trials (SORT) group [9] and the Asilomar Working Group on Recommendations for Reporting of Clinical Trials in the Biomedical Literature [11]. The CONSORT statement consists of two components, a 21-item checklist (Table 1) and a flow diagram (Figure 1). The checklist has six major headings that pertain to the contents of the report of a trial, namely Title, Abstract, Introduction, Methods, Results and Discussion. Within these major headings there are subheadings that pertain to specific items that should be included in any clinical trial manuscript. These items constitute the key pieces of information necessary for authors to address when reporting

Registered or Eligible Patients (n =...)

Not Randomized (n =...) Reasons (n =...)

Randomization

Received Standard Intervention as Allocated (n =...)

Received Standard Intervention as Allocated (n =...)

Did not Receive Standard Intervention as Allocated (n =...)

Did not Receive Standard Intervention as Allocated (n =...)

Followed UP (n =...)

Followed UP (n =...)

Timing of Primary and Secondary Outcomes

Timing of Primary and Secondary Outcomes

Withdrawn (n =...) Intervention ineffective (n =...) Lost to Follow-up (n =...) Other (n =...)

Completed Trial (n =...)

Withdrawn (n =...) Intervention ineffective (n =...) Lost to Follow-up (n =...) Other (n =...)

Completed Trial (n =...)

Figure 1 CONSORT flowchart. Reproduced with permission from the Journal of the American Medical Association, 1996, Volume 276, 637–665. Copyrighted (1996), American Medical Association Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

2

CONSORT

Table 1

CONSORT checklist

Heading

Subheading

Title Abstract Introduction

Methods

Protocol

Assignment

Masking (Blinding)

Results

Comment

Participant Flow and Follow-up Analysis

Descriptor Identify the study as a randomized trial. Use a structured format. Identify the study as a randomized trial. Use a structured format. State prospectively defined hypothesis, clinical objectives, and planned subgroup or covariate analyses Describe 1. Planned study population, together with inclusion/exclusion criteria. 2. Planned interventions and their timing. 3. Primary and secondary outcome measure(s) and the minimum important difference(s), and indicate how the target sample size was projected. 4. Rationale and methods for statistical analyses, detailing main comparative analyses and whether they were completed on an intention-to-treat basis. 5. Prospectively defined stopping rules (if warranted) Describe 1. Unit of randomization (e.g. individual, cluster, geographic). 2. Method used to generate the allocation schedule. 3. Method of allocation concealment and timing of assignment. 4. Method to separate the generator from the executor of assignment. Describe mechanism (e.g. capsules, tablets); similarity of treatment characteristics (e.g. appearance, taste); allocation schedule control (location of code during trial and when broken); and evidence for successful blinding among participants, person doing intervention, outcome assessors, and data analysts. Provide a trial profile (Figure 1) summarizing participant flow, numbers and timing of randomization assignment, interventions, and measurements for each randomized group. State estimated effect of intervention on primary and secondary outcome measures, including a point estimate and measure of precision (confidence interval). State results in absolute numbers when feasible (e.g. 10/20, not 50%). Present summary data and appropriate descriptive and inferential statistics in sufficient detail to permit alternative analyses and replication. Describe prognostic variables by treatment group and any attempt to adjust for them. Describe protocol deviations from the study as planned, together with the reasons. State specific interpretation of study findings, including sources of bias and imprecision (internal validity) and discussion of external validity, including appropriate quantitative measures when possible. State general interpretation of the data in light of the totality of the available evidence.

Was it reported? Page no.?

CONSORT the results of an RCT. Their inclusion is based on evidence, whenever possible. For example, authors are asked to report on the methods they used to achieve allocation concealment, possible in every randomized trial. There is growing evidence that inadequately concealed trials, compared with adequately concealed ones, exaggerate the estimates of intervention benefit by 30%–40%, on average [7, 8]. Additional benefits of the checklist (and flow diagram) include facilitating editors, peer reviewers and journal readers to evaluate the internal and external validity of a clinical trial report. The flow diagram pertains to the process of winnowing down the number of participants from those eligible or screened for a trial to those who ultimately completed the trial and were included in the analysis. The flow diagram pertains particularly to a two-group, parallel design, as stated in the CONSORT statement. Other checklists and flow diagrams have been developed for reporting cluster randomized trials [4] and other designs (see http://www.consortstatement.org). The flow diagram, in particular, requests relevant information regarding participants in each of the intervention and control groups who did not receive the regimen for the group to which they were randomized, those who during the course of the trial were discontinued, withdrew, became lost to follow-up, and those who have incomplete information for some other reason. There is emerging evidence to suggest that the quality of reporting of RCTs, based on the use of the CONSORT statement, compared with not using it, is higher on several dimensions, such as reduced reporting of unclear allocation concealment [6]. Similarly, use of the flow diagram was associated with better overall reporting of RCTs [3]. The CONSORT statement (checklist and flow diagram) is available on the CONSORT website (www.consort-statement.org). This site includes information on the growing number of health care journals and biomedical editorial groups, such as the International Council of Medical Journal Editors (ICMJE), who support the use of the CONSORT statement for reporting RCTs. At this writing the CONSORT statement is undergoing revision. Present plans call for the revised Statement to appear in Spring 2001 along with an

3

extensive explanatory and elaboration document to overcome some of the shortcomings of the original statement, both of which will be available on the above website.

References [1]

Begg, C., Cho, M., Eastwood, S., Horton, R., Moher, D., Olkin, I., Pitkin, R., Rennie, D., Schulz, K.F., Simel, D. & Stroup, D.F. (1996). Improving the quality of reporting of randomized controlled trials: the CONSORT Statement, Journal of the American Medical Association 276, 637–639. [2] Dickinson, K., Bunn, F., Wentz, R., Edwards, P. & Roberts, I. (2000). Size and quality of randomized controlled trials in head injury: review of published studies, British Medical Journal 320, 1308–1311. [3] Egger, M., J¨uni, P., Bartlett, C. for the CONSORT Group (2001). The value of CONSORT flow charts in reports of randomized controlled trials: bibliographic study, Journal of the American Medical Association, in press. [4] Elbourne, D.R. & Campbell, M.K. (2001). Extending the CONSORT statement to cluster randomized trials: for discussion, Statistics in Medicine 20, 489–496. [5] Hotopf, M., Lewis, G. & Normand, C. (1997). Putting trials on trial – the costs and consequences of small trials in depression: a systematic review of methodology, Journal of Epidemiology and Community Health 51, 354–358. [6] Moher, D., Jones, A., Lepage, L. for the CONSORT Group (2001). Does the CONSORT statement improve the quality of reports of randomized trials: a comparative before and after evaluation?, Journal of the American Medical Association, in press. [7] Moher, D., Pham, B., Jones, A., Cook, D.J., Jadad, A.R., Moher, M. & Tugwell, P. (1998). Does the quality of reports of randomized trials affect estimates of intervention efficacy reported in meta-analyses?, Lancet 352, 609–613. [8] Schulz, K.F., Chalmers, I., Hayes, R.J. & Altman, D.G. (1995). Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials, Journal of the American Medical Association 273, 408–412. [9] The Standards of Reporting Trials Group (1994). A proposal for structured reporting of randomized controlled trials, Journal of the American Medical Association 272, 1926–1931. Correction: Journal of the American Medical Association 273, 776. [10] Thornley, B. & Adams, C.E. (1998). Content and quality of 2000 controlled trials in schizophrenia over 50 years, British Medical Journal 317, 1181–1184. [11] Working Group on Recommendations for Reporting of Clinical Trials in the Biomedical Literature (1994). Call for comments on a proposal to improve reporting

4

CONSORT of clinical trials in the biomedical literature: a position paper, Annals of Internal Medicine 121, 894–895.

(See also QUORUM) DAVID MOHER

CONTRACT RESEARCH ORGANIZATION (CRO) A Contract Research Organization (CRO) is a person or an organization (commercial, academic, or other) contracted by the sponsor to perform one or more of the trial-related duties and functions of the sponsor. A sponsor may transfer any or all of the trial-related duties and functions of the sponsor to a CRO, but the ultimate responsibility for the quality and integrity of the trial data always resides with the sponsor. The CRO should implement quality assurance and quality control. Any trial-related duty and function that is transferred to and assumed by a CRO should be specified in writing. Any trial-related duties and functions not specifically transferred to and assumed by a CRO are retained by the sponsor. All references to a sponsor in this guideline also apply to a CRO to the extent that a CRO has assumed the trial-related duties and functions of a sponsor.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

CONTROL GROUPS

Informal observational methods have been used to identify treatments throughout the history of medicine. Control groups were, in essence, previously treated patients assessed according to general clinical impression. The humoralistic philosophy of medicine— treatment to restore the balance of blood, phlegm, black bile, and yellow bile in the sick—espoused by Hippocrates and codified by Galen (130 AD) remained largely unchallenged and untested until the Renaissance (4). Unfortunately, the replacement theories were not much better, nor were the treatments or experimental methods. Lack of adequately controlled trials might not have been so important except that so many treatments were actively harmful. Numerical methods were introduced in the early 1800s, leading to important epidemiologic observations such as Snow’s discovery that cholera is a waterborne infectious disease (5). However, the methods were not as useful in assessing treatment effectiveness because if they were applied at all, they were applied using unreliable control groups. For example, a diphtheria antiserum was introduced in Europe in 1894 to 1895, and death rates due to diphtheria declined—but the decline had started before 1894 and rose again to pre-1894 rates by 1924 (6). The mix of responsible bacteria changed over time, making the contribution of treatment to the initial decline uncertain. An early exception to use of control groups was the seawater-controlled 6 arm, 12 patient scurvy study undertaken by James Lind in 1753, which has been noted to be ‘‘the first deliberately planned controlled experiment ever undertaken on human subjects’’ (7). This success lacked a completely happy ending, however, in that it took decades before the results were reliably applied and accepted (8, 9). Although principles of controlled medical experiments were expressed as early as 1866 (2), the modern randomized controlled trial is a relatively recent development. The first properly randomized controlled treatment trial was a study of streptomycin in tuberculosis. In 2 years, the trial convincingly demonstrated that streptomycin plus

STEPHANIE GREEN Clinical Biostatistics Pfizer, Inc. New London, Connecticut

A control group for a clinical trial is a group of uniformly treated patients selected to be compared with a group receiving a test (new or different) treatment. For the comparison to be a valid assessment of differences in outcome due to test treatment, the two groups must be as similar as possible with the exception of treatment. Various types of control groups are employed, and the choice is critical to the interpretation of a trial (1). The type chosen for a particular trial will depend on many factors such as availability of resources, subjectivity of the primary outcome, availability of effective treatments, current extent of knowledge concerning treatment, severity of disease, and ethical considerations. Randomized trials comparing a control group with an experimental treatment group are the most convincing and reliable method for minimizing bias and demonstrating effectiveness of new treatments. Depending on standard of care, control groups for randomized trials may be patients randomized to no treatment, a placebo (i.e., an inactive agent or a procedure with the appearance of the new treatment), or standard active therapy with or without a placebo. Because of practical and ethical constraints, not all clinical questions can be addressed with a randomized trial. Nonrandomized controlled trials can have specific comparator groups or be controlled in the sense that carefully chosen historical information from the literature is used for reference. Proper use of control groups has been a significant advance in medicine, with controlled clinical trials now the mainstay of clinical research without which ‘‘the doctor walks at random and becomes the sport of illusion’’ (2, 3). 1

HISTORY

It can be argued that the history of clinical research is the history of control groups.

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

CONTROL GROUPS

bed rest was superior to the control treatment of bed rest alone, in stark contrast to centuries of unanswered questions on previous tuberculosis treatments (10). Sir Austin Bradford Hill, the statistical champion for this study, is credited as being the most instrumental individual in fostering use of the randomized controlled clinical trial (11). Since then, use of controlled trials has resulted in numerous treatment breakthroughs, such as the beautifully executed Salk vaccine trial for prevention of polio (12). Perhaps of equal or greater importance, controlled trials have also proven the lack of effectiveness of purported breakthroughs, such as the Cardiac Arrhythmia Suppression Trial (CAST) demonstrating that encainide and flecainide increased the death rate in patients with recent myocardial infarction instead of decreasing it, despite the well-documented suppression of ventricular arrhythmia by these agents (13). Use of proper control groups continues to be a critical principle in clinical research methods. 2

ETHICS

Ethical considerations with respect to experiments using control groups and experimental treatments center on the tension between care of individual patients and the need to study treatment effectiveness. The welfare of the individual patient should not be compromised by inclusion in a clinical research trial. On the other hand, the welfare of patients in general depends on identification of new effective treatments and discarding ineffective treatments. We often do not know whether the welfare of the individual is being compromised or enhanced, thus treatment without knowledge also raises ethical issues—consider the CAST study example of unknowingly doing harm instead of good. International guidance for ethics in medical research such as the Declaration of Helsinki (14) emphasizes protection of health and rights of patients. The Helsinki declaration acknowledges the physicians’ responsibilities to individual subjects: ‘‘considerations related to the well-being of the human subject should take precedence over the interests of science and society’’ and ‘‘it is the duty

of the physician in medical research to protect the life, health, privacy, and dignity of the human subject.’’ It also acknowledges responsibilities to patients as a whole: ‘‘it is the duty of the physician to safeguard the health of the people,’’ ‘‘medical progress is based on research which ultimately must rest in part on experimentation involving human subjects,’’ and ‘‘the benefits, risks, burdens and effectiveness of a new method should be tested against those of the best current prophylactic, diagnostic, and therapeutic methods.’’ The U.S. Belmont report (15) identified three major ethical principles: respect for persons (requiring informed and voluntary consent), beneficence (requiring that benefits outweigh risks plus careful study monitoring), and justice (requiring equitable selection of subjects and protection of vulnerable subjects). The main principle related to use of a control group is beneficence. Items from the Declaration of Helsinki that help inform decisions related to use of a control group include a requirement for careful assessment of risk–benefit, agreement that risks have been assessed and can be adequately managed, agreement that the importance of the objective outweighs the inherent risks to individuals, and a reasonable likelihood that populations in which the research is carried out stand to benefit from the results. Key considerations in the type of control group to be used are severity of disease and availability of effective treatments. If the disease is severe and there is an effective treatment, for instance, a no-treatment or placebo-alone arm on a randomized trial will not be an appropriate control as the risks to the individual are too high. The Declaration of Helsinki explicitly states that use of placebos or no-treatment are not precluded although ‘‘extreme care’’ must be taken in the decision to employ these. The current wording on this point is controversial, and the implications are perhaps too restrictive (16). Another key to ethical use of a control group is equipoise, the acknowledgment that usefulness of the new treatment compared with the control treatment is not yet known. Individual investigators may hold strong opinions and decline to participate in a trial,

CONTROL GROUPS

but the general sense in the scientific community should be one of uncertainty. In the CAST study, for example, there was considerable controversy in doing a randomized controlled trial as many investigators incorrectly believed that the striking arrhythmia evidence made a control arm unethical. Fortunately, the level of uncertainty in the scientific community allowed the trial to proceed, thereby sparing future patients a harmful treatment. An interesting illustration of the issues in using placebo-treated control groups was a double-blind transplantation trial in patients with Parkinson’s disease that used the standard treatment plus sham surgery as the control arm (17). The sham surgery included general anesthesia, a scalp incision, a partial burr hole, antibiotics and cyclosporine, and positron emission tomography (PET) studies, all of which entailed some risk to the patient. Because outcomes in Parkinson’s disease are subjective, use of a placebo-treated control group was the best way to assess the effectiveness of new therapy. The question was important, there was no adequate alternative therapy, the new treatment held promise but was uncertain, and future patients would benefit. Thus, the suitability of placebo use centered on whether the risk–benefit ratio for patients treated with placebo was acceptable. The potential benefits were the contribution to science, the no-cost standard medical treatment provided as part of the study, and later transplant at no cost if the treatment was found to be beneficial. As well, these patients were spared the additional risks of the transplant if it was found not to be beneficial. The risks included the possibility of injury or death due to the sham procedure and the inconvenience and discomfort of an extensive procedure with no possible benefit. As presented in a New England Journal of Medicine sounding board (18, 19), the case illustrates how assessment of levels of risk and benefit is not always clear cut, and how reasonable people may disagree on whether the ratio, and therefore the trial, is acceptable.

3

3 TYPES OF CONTROL GROUPS: HISTORICAL CONTROLS A comparison of control versus experimental groups will indicate whether outcomes are different in the experimental group, but without randomization causality cannot be assumed. Any difference in the groups (or lack of difference) may be due to factors other than treatment. Any particular choice of control group, no matter how carefully chosen, will be systematically different from the experimental group in many (often unknown or unmeasurable) ways due to the systematic reasons patients are chosen for treatment. Many factors contribute to a choice of treatment, and many of these factors are related to outcome; to the extent that the experimental group has better or worse prognosis than the control group, the comparison will be biased. Biases can make the test treatment appear either more or less effective it actually is. Because of potential toxicities, for instance, investigators may be inclined to include only patients who are relatively healthy; in other circumstances, they may choose only patients who are too compromised for other options. For the first case, an ineffective treatment might appear to be an improvement over historical treatment. For the second, an effective treatment might appear ineffective. If a historically controlled trial is to be done, the patient population for the control group should be as similar as possible to the population for the experimental group. This includes similar general health status, similar general medical care, and use of the same diagnostic and screening procedures. Also, the primary outcome should be defined and assessed the same way in both groups and be objective so that results from the experimental group have the same interpretation as for results from the historical control group. Results of such trials must always be interpreted cautiously because of the potential for bias. 3.1 Historical Control from the Medical Literature Historically controlled trials are usually single-arm trials without specific control groups. Rather, these trials are controlled

4

CONTROL GROUPS

in the sense that statistical hypotheses are based on estimates from the literature: instead of being compared with a specific set of patients, patients on the new treatment are assessed against a fixed value. For instance, if the success rate of the standard treatment in the literature is consistently 20%, a test may be done to ascertain whether the percentage of patients with success on the new treatment is significantly greater than 20%. Or if a particular time to event distribution is hypothesized, a 1-sample logrank test (20) might be used to test superiority. Such trials are conducted for reasons such as a severely limited patient population or a need for preliminary assessment before planning a large definitive trial. This approach works best if historical estimates are well characterized and are stable over time with low variability. This may be the case for uniformly nonresponsive disease with no effective treatment, resulting in low variability in patient outcomes. There should also be no recent treatment improvements, changes in staging definitions, improvements in diagnostic procedures, or changes in the way primary outcome is assessed so that results will not be confounded by temporal changes. Of course, definitions, treatments, and methods typically do not remain stable over time, and uniformity is not common, so historical estimates for the same treatment in ostensibly the same patient population may vary substantially. It is often difficult to ascertain which estimates may be appropriate for assessing a new treatment. Considering the high potential for bias of unknown magnitude or direction, in most circumstances a single-arm trial will not provide a definitive answer to the question of whether the new treatment is an improvement over the standard. In the past, the probability of success for various types of cancer was uniformly dismal, so this setting provides an example of feasible use of this approach. With recent treatment and diagnostic advances, old assumptions no longer hold, so the approach is becoming less informative. Other approaches to controlling early studies of new regimens in

cancer are becoming more common as variability in treatment outcome increases and as previously stable historical rates improve. 3.2 Specific Historical Control Groups Trials with control groups consisting of a specific set of past patients are sometimes designed. For this type of trial, individual patient data from the control group are used for comparison with patients on the test treatment. Such trials may be justified when the probability of success and expected benefit of therapy are both considered to be so high that ethical considerations dictate a current control arm would not be in the best interests of patients. A limited patient population may also be a reason to design a historically controlled trial. A carefully chosen specific control group potentially will be better than using an estimate from the literature but will still be subject to selection biases. Known prognostic factors should be available for use in the analysis of this type of trial, and only large differences after suitable adjustment for prognostic factors should be accepted as evidence of improvement due to experimental therapy. An example illustrating how difficult it is to select a suitable specific historical control is provided by a sequence of four Southwest Oncology Group studies in multiple myeloma. The trials had the same eligibility criteria, and largely the same institutions participating on the trials. The overall survival estimates by study were remarkably similar across all four trials. There were no significant improvements demonstrated for any of the test treatments. This stability over time would suggest that use of a prior control arm for assessing the next treatment might be reasonable. However, when the identically treated control arms for each these four trials were compared, the results appeared different, and the test of differences was suggestive (P = 0.07) of heterogeneity among the arms (21). Comparison of a new regimen with the worst of these control arms could have resulted in a spurious positive result. If comparability cannot be counted on in ideal cases such as this, then it is unlikely to apply in less ideal cases. One should assume that there will be systematic differences between the

CONTROL GROUPS

chosen control group and the test treatment no matter how carefully they are chosen. The myeloma trials also provide an example of how poorly chosen historical control groups can mislead. A specific comparison of a high-dose therapy regimen with one of the trial arms indicated a striking benefit due to the experimental regimen. It is tempting to conclude in such cases that not all of the difference could be due to bias. However, patients must be in better health to receive a high-dose regimen than to receive the standard treatment. When the comparison was restricted to younger patients with good renal function, a much smaller difference remained (21). A randomized trial was deemed necessary, and it did not demonstrate a benefit due to high-dose therapy, despite the initial promising comparison (22). It should be noted that design considerations are different when a retrospective control group is used instead of a prospective control. In this case, sample size considerations are a function of the results of the control group, and they may also account for covariate adjustment (23–25). 4 TYPES OF CONTROL GROUPS: RANDOMIZED CONTROLS A control versus experimental comparison allows an assessment of whether outcomes are different in the experimental group. Randomization allows the possibility of concluding that the difference is actually caused by the test treatment, due to the elimination of biases in treatment assignment. Although randomization is necessary, it may not be sufficient for attributing differences to the test treatment. A poorly conducted randomized trial may still result in substantial bias. Some potential sources of bias in the comparison of groups are structural and can be avoided relatively easily. For instance, outcome assessment schedules should be the same for both the control group and the test group. If, for instance, time to failure is assessed less frequently in one group, then results will be biased in favor of this group. Methods of assessment should also be the same. If the method of assessment for failure in one group is more sensitive, then the

5

results will be biased against this group. Criteria for inclusion in the analysis should also be the same. If information is available in one group but not the other, then this information should not be used to exclude patients from analysis. For example, disease information in a surgical test group might identify patients unlikely to benefit from treatment, but these cannot be excluded from analysis because similarly unsuitable patients from the control group will not be excluded. Other sources of bias are not so easily eliminated. To the extent these differ according to treatment group, the treatment comparison is compromised. For instance, patients in the control group may be less compliant with trial requirements than those in the test group or may drop out of the trial all together, potentially resulting in worse than expected outcome. Or if the test treatment includes over-the-counter agents, patients in the control group may treat themselves with the agents, potentially resulting in a better than expected outcome. Subjective outcomes are particularly problematic because assessments are easily influenced by knowledge of the treatment assignment. Investigators may overestimate improvement in the test group in anticipation or hope of benefit from a promising new treatment, or adverse events may be more likely to be attributed to treatment in the experimental group compared with the control group. Patients are also subject to the wellknown placebo effect. A proportion of patients will report improvement in subjective disease symptoms or experience of treatment side effects whether or not the treatment is active (26). For example, trials of venlafaxine in sexual dysfunction (27), hot flashes (28), panic disorder (29), generalized anxiety disorder (30), and migraine pain (31) all noted improvements with placebo, sometimes less than the effect of venlafaxine, sometimes not. Or consider an antiemetic trial of placebo versus prochlorperazine versus tetrahydrocannabinol (THC, the active marijuana component). In this trial, sedation side effects of treatment were reported in 46% of placebo patients, and ‘‘highs’’ were reported in 12% of prochlorperazine patients (32), presumably due in part to anticipation of being on

6

CONTROL GROUPS

THC. In a study for which the placebo effect occurs mainly in the experimental group, an improvement in the experimental group may be observed even when the test treatment is inactive. 4.1 Untreated Control Group, Randomized Trial The category of untreated controls includes control groups for which patients receive the same palliative care or routine monitoring as patients in the experimental group, but the control group receives no additional treatment while the experimental group receives the test treatment. A trial with this type of control group has potential for all of the biases previously discussed. Problems may be particularly acute in this type of trial because of the no-treatment control. For instance, patients may have little motivation to return to the clinic for routine visits, so outcomes may not be carefully collected. As well, the placebo effect will occur only in the experimental group. 4.2 Standard Treatment Control Group, Add-On Randomized Trial When not treating is considered unethical or unfeasible, one option may be to treat both the control group and the experimental group with standard treatment and to add the test treatment in the experimental group. Bias issues are similar to those for an untreated control group. Use of this type of control group allows for assessment of improvement over standard treatment due to the test treatment. However, conclusions cannot be made concerning the usefulness of the test treatment alone. Because of potential treatment synergy or inhibition, improvement over the standard treatment does not prove singleagent activity, and lack of improvement over the standard treatment does not disprove single-agent activity. 4.3 Placebo-Treated Control Group, Randomized Trial The placebo-treated control category includes control groups for which patients receive the same palliative treatment or routine monitoring as the experimental group plus a placebo

while the experimental group receives the test treatment. Patients are given identical appearing treatments in each treatment arm to mask knowledge of the assigned treatment. Blinded placebo-controlled trials are done to reduce the potential for biases related to knowledge of the treatment assignment. For patients, compliance is enhanced, and supplemental treatments, while not necessarily eliminated, should at least be balanced across the groups. For investigators, the objectivity of outcome assessments is improved. In addition, use of a placebo will control for the placebo effect. Both groups will experience the placebo effect, so differences between the groups will be due to the active treatment. Sometimes only patients are blinded (single blind), but preferably both patients and investigators will be blinded (double blind) to avoid bias from both sources. Although it would seem best always to have a placebo-treated control group, it may not always be practical or ethical. Blinded placebo-controlled trials are resource intensive. Significant time and money are needed for manufacturing of the placebo; for labeling, shipping, and tracking the coded placebos and active agents; for setting up mechanisms for distribution with pharmacies; and for arranging for emergency unblinding. If the outcomes are objective and other problems associated with knowledge of the treatment assignment are anticipated to be minor, it may be concluded unnecessary to blind. In other cases, it may be impossible to blind, such as when a treatment has a distinctive side effect that cannot be reproduced in a placebo. In yet other cases, the placebo treatment may entail too much risk to the patient. Sham surgeries are particularly controversial, as already noted. 4.4 Placebo-Treated Control Group, Add-On Randomized Trial For an add-on randomized trial, patients in the control group can receive standard treatment and a placebo that appears identical to the test treatment while patients in the experimental group receive the standard plus the test treatment. The issues are the same as for a placebo-treated control group. Again, use of this type of control group allows for

CONTROL GROUPS

assessment of improvement over the standard treatment due to the test treatment, but it would not address the usefulness of the test treatment alone. 4.5 Active Control Group When no-treatment or placebo-only are not appropriate and when adding the test treatment to the standard is not of interest, then an active control trial may be done. The control group in this type of trial receives standard treatment while the experimental group receives the test treatment. The test treatment in this case may be a single agent, a combination of therapies, different ways of administering treatment, new schedules, or other variations that cannot be described simply as standard plus new. Double placebos are sometimes used to mask treatment assignment for these trials. Risks and benefits of forgoing proven useful treatment must be assessed carefully in this setting. Such trials are appropriate when standard treatment is useful but not fully effective and a delay in receiving standard treatment is acceptable. The aim of an active control trial may be to show superiority of the test treatment or to show equivalence. A superiority trial will allow for assessment of the difference between control and the test treatment. However, unlike an add-on trial, if the test treatment is a combination of therapies, the cause of any difference will likely not be attributable to a particular component of the combination. Equivalence trials aim to show that the test treatment is as good as the standard. ‘‘As good as’’ is difficult to establish. Equivalence trials need both active control groups and historical information because results of the standard versus test comparison must not only demonstrate that the test treatment is similar to control, but also that it would have been superior to placebo had the trial been placebo controlled. Design considerations for such trials are complex, and they are the subject of ongoing research and discussion (33–35). Lack of accurate estimates of the benefit of active control compared with placebo is a common challenge in this setting. As for other historical control settings, the assumption of no change over time is particularly troublesome.

7

4.6 Multiple Control Groups In some studies, more than one control group may be used. When appropriate, using both a placebo-only control group and an active control group can be useful for addressing both the absolute effect of a new treatment and the effect relative to the standard treatment. It may also provide information on why a trial fails to show usefulness of a new treatment. If the standard group also fails to show improvement compared with the placebo, the failure may be due to the trial rather than to lack of efficacy of the new treatment. 5

CONCLUSION

Choice of the control group in a clinical trial is critical to the success of the trial. A poor choice may result in biases that severely compromise interpretation of results. A blinded randomized trial with a placebo-treated control group will provide the most definitive evidence concerning the usefulness of a new treatment. If it is not possible to conduct this type of trial, the best control group under the circumstances should be used. The trial should be randomized if feasible. For any choice, every effort should be made to ensure the control group is as similar as possible to the experimental group. Reliability of conclusions about new treatments depends on it! REFERENCES 1. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH Harmonised Tripartite Guideline: E10 Choice of Control Group and Related Issues in Clinical Trials. Current Step 4 version, July 20, 2000. Available at: http://www.ich.org/LOB/media/ MEDIA486.pdf 2. J. P. Boissel, Impact of randomized clinical trial on medical practices. Control Clin Trials. 1989; 10: 120S–134S. 3. C. L. Bernard, Introduction a` l’Etude de la M´edecine Exp´erimentale. First published in 1866. London: Garnier-Flammarion, 1966. 4. D. DeMoulin, A Short History of Breast Cancer. Dorderct, Germany: Klewer, 1989.

8

CONTROL GROUPS 5. D. Freedman, From association to causation: some remarks on the history of statistics. Stat Sci. 1999; 14: 243–258.

21. S. Green, J. Benedetti, and J. Crowley, Clinical Trials in Oncology. Boca Raton, FL: Chapman and Hall/CRC Press, 2003.

6. H.O. Lancaster, Quantitative Methods in Biological and Medical Sciences. New York: Springer-Verlag, 1994.

22. B. Barlogie, R. Kyle, K. Anderson, P. Greipp, H. Lazarus, et al., Standard chemotherapy compared with high-dose chemoradiotherapy for multiple myeloma: final results of phase III US Intergroup Trial S9321. J Clin Oncol. 2006; 24: 929–936. 23. R. Makuch and R. Simon, Sample size consideration for non-randomized comparative studies. J Chronic Dis. 1980; 33: 175–181.

7. C. Stuart and D. Guthrie, eds., Lind’s Treatise on Scurvy. Edinburgh: University Press, 1953. 8. G. Cook, Scurvy in the British Mercantile Marine in the 19th century, and the contribution of the Seamen’s Hospital Society. Postgrad Med J. 2004; 80: 224–229. 9. D. Thomas, Sailors, scurvy and science. J Roy Soc Med. 1997; 90: 50–54. 10. W. Silverman, Doctoring: From art to engineering. Control Clin Trials. 1992; 13: 97–99. 11. W. Silverman and I. Chalmers, Sir Austin Bradford Hill: an appreciation. Control Clin Trials. 1991; 10: 1–10. 12. J. Smith, Patenting the Sun: Polio and the Salk Vaccine Trial. 1990. New York: William and Morrow, 1990. 13. D. Echt, P. Liebson, L. Mitchell, R. Peters, D. Obias-Manno, et al., and the CAST investigators. Mortality and morbidity in patients receiving encainide, flecainide or placebo: the Cardiac Arrhythmia Suppression Trial. N Engl J Med. 1991; 324: 781–788. 14. World Medical Association General Assembly. World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. J Int Bioethique. 2004; 15: 124–129. 15. Protection of human subjects; Belmont Report: notice of report for public comment. Fed Regist. 1979; 44: 23191–23197. 16. S. Piantadosi, Clinical Trials: A Methodologic Approach, 2nd ed. New York: Wiley, 2005. 17. C. W. Olanow, C. Goetz, J. Kordower, A. J. Stoessl, V. Sossi, et al., A double-blind controlled trial of bilateral fetal nigral transplantation in Parkinson’s disease. Ann Neurol. 2003; 54: 403–414. 18. T. Freeman, D. Vawtner, P. Leaverton, J. Godbold, R. Hauser, et al., Use of placebo surgery in controlled trials of a cellular based therapy for Parkinson’s disease. N Engl J Med. 1999; 341: 988–992. 19. R. Macklin, The ethical problems with sham surgery in clinical research. N Engl J Med. 1999; 341: 992–996. 20. R. Woolson, Rank tests and a one-sample logrank test for comparing observed survival data to a standard population. Biometrics. 1981; 37: 687–696.

24. D. Dixon and R. Simon, Sample size consideration for studies comparing survival curves using historical controls. J Clin Epidemiol. 1988; 41: 1209–1213. 25. J. O’Malley, S. L. Normand, and R. Kuntz, Sample size calculation for a historically controlled clinical trial with adjustment for covariates. J Biopharm Stat. 2002; 12: 227–247. 26. A. Shapiro and K. Shapiro, The Powerful Placebo: From Ancient Priest to Modern Physician. Baltimore: Johns Hopkins University Press, 1997. 27. S. Kilic, H. Ergin, and Y. Baydinc, Venlafaxine extended release for the treatment of patients with premature ejaculation: a pilot, single-blind, placebo-controlled, fixeddose crossover study on short-term administration of an antidepressant drug. Int J Androl. 2005; 28: 47–52. 28. M. Evans, E. Pritts, E. Vittinghoff, K. McClish, K. Morgan, and R. Jaffe, Management of postmenopausal hot flushes with venlafaxine hydrochloride: a randomized, controlled trial. Obstet Gynecol. 2005; 105: 161–166. 29. M. Pollack, U. Lepola, H. Koponen, N. Simon, J. Worthington, et al., A double-blind study of the efficacy of venlafaxine extended-release, paroxetine, and placebo in the treatment of panic disorder. Depress Anxiety. 2006; 2007; 24: 1–14. 30. A. Gelenberg, R. B. Lydiard, R. Rudolph, L. Aguiar, J. T. Haskins, and E. Salinas, Efficacy of venlafaxine extended-release capsules in nondepresssed outpatients with generalized anxiety disorder: a 6-month randomized controlled trial. JAMA. 2000; 283: 3082–3088. 31. S. Ozyalcin, G. K. Talu, E. Kiziltan, B. Yucel, M. Ertas, and R. Disci, The efficacy and safety of venlafaxine in the prophylaxis of migraine. Headache. 2005; 45: 144–152. 32. S. Frytak, C. Moertel, J. O’Fallon, J. Rubin, E. Cregan, et al., Delta-9-Tetrahydrocannabinol

CONTROL GROUPS as an antiemetic for patients receiving cancer chemotherapy. Ann Int Med. 1979; 91: 825–830. 33. R. D’Augustino, J. Massaro, and L. Sullivan, Non-inferiority trials: design concepts and issues—the encounters of academic consultants in statistics. Stat Med. 2003; 22: 169–186. 34. S. Durrleman and P. Chaikin, The use of putative placebo in active control trials: two applications in a regulatory setting. Stat Med. 2003; 22: 941–952. 35. Y. Tsong, S. J. Wang, H. M. Hung, and L. Cui, Statistical issues on objective, design and analysis of non-inferiority activecontrolled clinical trial. J Biopharm Stat. 2003; 13: 29–41.

CROSS-REFERENCES Adequate and well-controlled trial Bellmont Report Historical control Placebo-controlled trial Active-controlled trial

9

COOPERATIVE NORTH SCANDINAVIAN ENALAPRIL SURVIVAL STUDY

2

CONSENSUS was a randomized, doubleblind, placebo-controlled, parallel-group trial (2). To be eligible, patients had to have the most severe form of heart failure, New York Heart Association (NYHA) class IV, which means that the symptoms of heart failure were present at rest. Patients were receiving optimal treatment for heart failure at the start of the trial, including digitalis and diuretics, and continued to receive these treatments during the trial. In addition, patients were randomized to receive either enalapril or placebo. The starting dose was 5 mg twice a day, and could be titrated up to 20 mg twice a day, depending on clinical response. Note that early in the trial the occurrence of symptomatic hypotension in some patients led the investigators to reduce the starting dose to 2.5 mg daily for some high-risk patients. The primary endpoint of the trial was death by any cause within 6 months of randomization. Secondary endpoints included 12-month mortality and mortality during the entire trial period. The sample size was calculated to be 400 patients, 200 per treatment group, based on the assumption that the 6month mortality rate would be 40% in the placebo group, and would be reduced to 24% by enalapril. This sample size provides 90% power at a two-sided significance level of 5%. Differences in mortality between treatment groups were to be analyzed using life-table methods, and the analysis was to be by intention-to-treat; that is, the survival information for each patient from the date of randomization to the date of death or study termination was to be included in the analysis. Although the trial was to be monitored by an ethical review committee, there was no formal rule governing the committee’s decisions. As its name suggests, CONSENSUS took place in three countries in the north of Scandinavia: Finland, Norway, and Sweden (see also Multinational [Global] Trial). The CONSENSUS Trial Study group consisted of the set of investigators in these three countries, a steering committee chaired by John Kjekshus, an ethical review committee (some-

STEVEN M. SNAPINN Biostatistics, Amgen, Inc. Thousand Oaks, California

Heart failure, also known as congestive heart failure, is a condition that occurs when the heart is unable to pump sufficient blood to meet the body’s oxygen demand. Signs and symptoms include dyspnea, or shortness of breath, particularly when lying down; edema, or the buildup of fluid; and cough. It is a very serious condition and, in its most severe form, is associated with a high mortality rate. Heart failure can be caused by a number of factors such as hypertension, ischemic heart disease, and cardiomyopathy. As of the mid-1980s, the standard treatment for heart failure usually included digitalis and diuretics. There was also growing evidence that direct-acting vasodilator therapy would be beneficial. However, a metaanalysis by Furberg and Yusuf (1) found little evidence for improved survival with this therapy, and instead suggested that angiotensinconverting enzyme (ACE) inhibition (see also the article on ACE inhibitors) held the most promise. This relatively new class of drugs, including enalapril, captopril, and lisinopril, was known to be effective in the treatment of hypertension and had been associated with symptomatic improvement in patients with heart failure. However, the effects of these agents on survival was unknown. 1

STUDY DESIGN

OBJECTIVES

The primary objective of the Cooperative North Scandinavian Enalapril Survival Study, also known as CONSENSUS, was to study the effect on mortality of enalapril compared with placebo, when added to conventional therapy, in patients with severe heart failure (2). Other objectives included evaluating the safety of enalapril, its effect on symptoms of heart failure, and its effects on neurohormones known to be associated with mortality in these patients.

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

COOPERATIVE NORTH SCANDINAVIAN ENALAPRIL SURVIVAL STUDY

times referred to in other trials as a data and safety monitoring board or a data monitoring committee) chaired by Jacobus Lubsen, and an administrative office run by the study’s sponsor, Merck Research Laboratories. 3

RESULTS

3.1 Interim Analysis Results CONSENSUS terminated ahead of schedule based on a recommendation from the ethical review committee (2). Their opinion was that the results favoring enalapril were so strong that continuation of the trial would be of limited scientific interest and could not be justified from an ethics perspective. An editorial in the Lancet (3) commented that, while correct, the decision to stop CONSENSUS was highly unusual, and that it is desirable for trials to run their entire course. This is especially true given that CONSENSUS had no formal stopping rule. Lubsen (4) provided a detailed review of the ethical review committee’s deliberations. Table (1) contains the interim mortality results reviewed by the committee. The members were informed by telephone of the June 27, 1986, results and agreed to a meeting to discuss them. At the meeting, held on September 14, 1986, the sponsor provided a more current update: the 6-month mortality rates were now 24 and 49% (P = 0.0002). Despite this large difference, the committee felt that they needed additional information from the sponsor on patient characteristics. This required a large data entry effort, and the information was not available until December 7, 1986. At this meeting, the 6-month mortality rates were 27 and 48%

(P = 0.001), and a review of the baseline characteristics showed balance between groups and a consistent treatment effect among important subgroups. Therefore, the committee decided to recommend termination of the trial, and they authorized the committee chair to inform the steering committee. This took place on December 13, 1986; the steering committee accepted this recommendation, set December 14, 1986, as the study termination date, and informed the clinical centers. 3.2 Final Analysis Results The final results of CONSENSUS were reported in 1987 (2) and 1988 (5). A total of 253 patients were randomized, 127 to enalapril and 128 to placebo. The mean age was approximately 70 years, approximately 70% of the patients were men, and most patients had an etiology of ischemic heart disease. The median duration of heart failure before enrollment was approximately 48 months. The final mortality results are summarized in Table (2) and displayed in Figure (1), which shows the Kaplan-Meier curves for overall mortality (see also the article on Kaplan-Meier plots). Overall, 68 placebo patients (54%) had died, compared with 50 enalapril patients (39%, P = 0.003 using lifetable methods). The benefit of enalapril was restricted to progression of heart failure: 44 placebo patients and 22 enalapril patients died of this cause, and 24 placebo patients and 28 enalapril patients died of other causes (primarily sudden cardiac death). Patients treated with enalapril also experienced symptomatic benefit. In the enalapril group, 54 surviving patients had an endof-study NYHA class of between I and III,

Table 1. Interim Mortality Results Enalapril Status date January 1, 1986 April 1, 1986 May 1, 1986 June 27, 1986 September 14, 1986 December 7, 1986

Placebo

No. of deaths

No. randomized

No. of deaths

No. randomized

8 16 20 26 28 44

52 77 84 100 101 124

15 24 31 44 52 66

51 72 78 93 99 120

COOPERATIVE NORTH SCANDINAVIAN ENALAPRIL SURVIVAL STUDY

3

Table 2. Final Mortality Results

Mortality within 6 months Mortality within 1 year Total mortality

Placebo (n = 126) No. %

Enalapril (n = 127) No. %

55 66 68

33 46 50

44 52 54

Reduction in relative risk risk (%)

P-value (life-table analysis)

40 31 27

0.002 0.001 0.003

26 36 39

Cumulative Probability of Death

0.8 0.7 0.6 0.5 0.4 0.3 0.2 Placebo Enalapril

0.1

0

1

2

3

4

5

6 7 Months

8

9

10

11

12

Placebo N = 126 102

78

63

59

53

47

42

34

30

24

18

17

Enalapril N = 127 111

98

88

82

79

73

64

59

49

42

31

26

Figure 1. Kaplan-Meier curves comparing total mortality for enalapril and placebo.

including 16 patients with class I or II; in the placebo group, there were only 27 surviving patients with an end-of-study NYHA class of between I and III and only two patients in class I or II. Enalapril was very welltolerated; the major side effect associated with enalapril was hypotension. 3.3 Neurohormone Analysis CONSENSUS provided a wealth of information on neurohormones. Blood samples were obtained from most randomized patients at the time of randomization and after 6 weeks of treatment. The first set of publications focused on the following neurohormones: angiotensin-converting enzyme, angiotensin II, aldosterone, noradrenaline, adrenaline, dopamine, and atrial natriuretic factor (6, 7). When looking at the placebo group, there were strong and statistically significant

associations between several of these neurohormones and mortality: angiotensin II (P < 0.05), aldosterone (P = 0.003), noradrenaline (P < 0.001), adrenaline (P = 0.001), and atrial natriuretic factor (P = 0.003). However, similar associations were not seen in the enalapril group. In addition, the mortality benefit of enalapril appeared stronger among patients with values of these neurohormones above the median. These results suggested that the effect of enalapril on mortality is related to hormonal activation, and to activation of the renin-angiotensin system in particular. Subsequent publications focused on the novel neurohormones N-terminal proatrial natriuretic factor ANF(1-98) and atrial natriuretic peptides ANP(1-98) and ANP(99-126) (8, 9). It was concluded that the magnitude of changes in these neurohormones provide important information on prognosis and therapeutic effects of enalapril.

4

COOPERATIVE NORTH SCANDINAVIAN ENALAPRIL SURVIVAL STUDY

3.4 Long-Term Follow-Up When CONSENSUS terminated in December of 1986, all surviving patients were removed from blinded study medication but were informed of the reason for terminating the trial and given the option of taking openlabel enalapril. In addition, patients continued to be followed for their survival status. The follow-up mortality data were analyzed in two ways. First, the follow-up data were analyzed as if the surviving patients were randomized (using the original randomization schedule) into a new study beginning on the date the blinded portion of the trial terminated, December 15, 1986. Second, the data from the blinded portion of the trial were combined with the follow-up data, as if the trial had never stopped. The first report of additional follow-up came 8.5 months after the end of blinded therapy (10). There were 58 surviving patients from the placebo group of whom 18 (31%) died during the follow-up period, and 77 surviving patients from the enalapril group of whom 16 (21%) died during the follow-up period. The next report included 2-year follow-up information (11). By the end of this period, 26 patients from the original placebo group were still alive, compared with 38 patients from the original enalapril group. The final follow-up publication was based on a 10-year follow-up period (12). By the end of the 10-year follow-up period, only five patients were still alive, all of whom had been in the enalapril group during the blinded portion of the trial. This analysis showed that, despite the fact that enalapril was made available to all surviving patients at the termination of the trial, the benefit accrued to patients randomized to enalapril persisted for at least 3.5 years after termination. 4

failure and the adoption of ACE inhibitors as a standard treatment for these patients. CONSENSUS was the first heart failure study to show dramatic benefits from ACE inhibition (14, 15). However, CONSENSUS was conducted in a population with severe heart failure and did not answer the question of whether this treatment was beneficial in patients with milder forms of the disease. Subsequent to CONSENSUS, two major trials evaluated the effects of enalapril in patients with NYHA class II–III heart failure. The second Veterans Administration Cooperative Vasodilator-Heart Failure Study (VHeFT-II) compared enalapril with a vasodilator regimen in 804 men. Over a 5years of follow-up period, mortality was consistently lower in the enalapril group, but not significantly so (P = 0.08). The Studies of Left Ventricular Dysfunction (SOLVD) Treatment trial compared enalapril and placebo with respect to total mortality in 2569 patients. Cumulative 4-year all-cause mortality among patients randomized to enalapril was 16% lower than in the placebo group (P = 0.004). As a result of these trials, ACE inhibition is now standard treatment for NYHA class II–IV heart failure. Note that there was a later trial with a similar name: Cooperative New Scandinavian Enalapril Survival Study (CONSENSUS II). Although this trial did involve the same experimental treatment, enalapril, it was conducted in patients with an acute myocardial infarction rather than in patients with heart failure. In addition, the letter ‘‘N’’ in the acronym refers to ‘‘New’’ rather than ‘‘North,’’ a reference to the fact that CONSENSUS II included the three countries that were part of CONSENSUS plus Denmark. Therefore, to avoid confusion with CONSENSUS II, CONSENSUS is now sometimes referred to as CONSENSUS I.

CONCLUSIONS

CONSENSUS was a milestone for treatment of patients with heart failure (13) and a landmark trial in the history of cardiovascular clinical research (see also the article on disease trials for cardiovascular diseases). The highly significant results and the magnitude of the clinical benefit led to the approval of enalapril for the treatment of severe heart

REFERENCES 1. C. D. Furberg and S. Yusuf, Effect of drug therapy on survival in chronic congestive heart failure. Am J Cardiol. 1988; 62: 41A–45A. 2. CONSENSUS Trial Study Group. Effects of enalapril on mortality in severe congestive heart failure: results of the Cooperative North Scandinavian Enalapril Survival Study

COOPERATIVE NORTH SCANDINAVIAN ENALAPRIL SURVIVAL STUDY (CONSENSUS). N Engl J Med. 1987; 316: 1429–1435. 3. Lancet [editorial]. Consensus on heart failure management? Lancet. 1987; 330: 311–312. 4. J. Lubsen, for the CONSENSUS Ethical Review Committee. Appendix: monitoring methods, considerations, and statement of the Cooperative North Scandinavian Enalapril Survival Study (CONSENSUS) Ethical Review Committee. Am J Cardiol. 1988; 62: 73A–74A. 5. K. Swedberg and J. Kjekshus, Effects of enalapril on mortality in severe congestive heart failure: results of the Cooperative North Scandinavian Enalapril Survival Study (CONSENSUS). Am J Cardiol. 1988; 62: 60A–66A. 6. K. Swedberg, P. Eneroth, J. Kjekshus, and S. Snapinn, for the CONSENSUS Trial Study Group. Effects of enalapril and neuroendocrine activation on prognosis in Severe Congestive Heart Failure (follow-up of the CONSENSUS Trial). Am J Cardiol. 1990; 66: 40D–45D. 7. K. Swedberg, P. Eneroth, J. Kjekshus, L. Wilhelmsen, for the CONSENSUS Trial Study Group. Hormones regulating cardiovascular function in patients with severe congestive heart failure and their relation to mortality. Circulation. 1990; 82: 1730–1736. 8. C. Hall, J. Kjekshus, P. Eneroth, and S. Snapinn, The plasma concentration of Nterminal proatrial natriuretic factor ANF(198) is related to prognosis in severe heart failure. Clin Cardiol. 1994; 17: 191–195. 9. S. V. Eriksson, K. Caidahl, C. Hall, P. Eneroth, J. Kjekshus, et al. Atrial natriuretic petptide ANP(1-98) and ANP(99-126) in patients with severe chronic congestive heart failure: relation to echocardiographic measurements. A subgroup analysis from the Cooperative North Scandinavian Enalapril Survival Study (CONSENSUS). J Cardiac Failure. 1995; 1: 109–116.

5

10. K. Swedberg and J. Kjekshus. Effect of enalapril on mortality in congestive heart failure: follow-up survival data from the CONSENSUS Trial. Drugs. 1990; 39(Suppl 4): 49–52. 11. J. Kjekshus, K. Swedberg, and S. Snapinn, for the CONSENSUS Trial Group. Effects of enalapril on long-term mortality in severe congestive heart failure. Am J Cardiol. 1992; 69: 103–107. 12. K. Swedberg, J. Kjekshus, and S. Snapinn, for the CONSENSUS Investigators. Long-term survival in severe heart failure in patients treated with enalapril: ten year follow-up of CONSENSUS I. Eur Heart J 1999; 20: 136–139. 13. G. A. J. Riegger, Lessons from recent randomized controlled trials for the management of congestive heart failure. Am J Cardiol. 1993; 71: 38E–40E. 14. W. B. Hood, Role of converting enzyme inhibitors in the treatment of heart failure. J Am Coll Cardiol. 1993; 22(Suppl A): 154A–157A. 15. J. B. Young, Angiotensin-converting enzyme inhibitors in heart failure: new strategies justified by recent clinical trials. Int J Cardiol. 1994; 43: 151–163. 16. P. Sleight, Angiotensin II and trials of cardiovascular outcomes. Am J Cardiol. 2002; 89(Suppl): 11A–17A.

CROSS-REFERENCES Multinational (global) trial Kaplan-Meier plot Data and safety monitoring board Angiotensin-converting enzyme (ACE) inhibitors Disease trials for cardiovascular diseases

Cooperative Studies Program, US Department of Veterans Affairs The Department of Veterans Affairs (VA) is in a unique position in the US, and perhaps the world, in conducting multicenter clinical trials. This is due to several factors: (1) its network of 172 medical centers geographically dispersed throughout the country, under one administrative system; (2) a dedicated group of talented physicians and other health professionals serving at these medical centers; (3) a loyal and compliant patient population of nearly four million veterans; (4) a system of experienced coordinating centers that provide biostatistical, data processing, pharmacy and administrative support; and (5) a research service that recognizes the uniqueness and importance of the program and strongly supports its mission. The VA has conducted multicenter clinical trials for more than half a century, beginning with its first trial, which was organized in 1945 to evaluate the safety and efficacy of chemotherapeutic agents for tuberculosis. This article describes the history of the program, its organization and operating procedures, some of its noteworthy trials, and current challenges and opportunities.

History of the Cooperative Studies Program (CSP) The first cooperative clinical trial conducted by the VA was a joint study with the US Armed Forces to evaluate the safety and efficacy of chemotherapeutic agents for tuberculosis. Drs John B. Barnwell and Arthur M. Walker initiated a clinical trial to evaluate various drugs in the treatment of tuberculosis, including the antibiotic streptomycin [3, 48]. The challenge of caring for 10 000 veterans suffering from the disease following World War II was the impetus for the study. Not only did the results revolutionize the treatment of tuberculosis, they also led to the development of an innovative method for testing the effectiveness of new therapies – the cooperative clinical trial. A VA Program for conducting cooperative studies in psychiatry was started in 1955 and supported by a newly developed Central Neuropsychiatric

Research Laboratory at the Perry Point, Maryland VA Medical Center (VAMC). This Program emphasized the design and conduct of randomized trials for the treatment of chronic schizophrenia. Trials were completed evaluating the efficacy of prefrontal lobotomy [2], chlorpromazine and promazine [8], phenothiazine derivatives [10], other psychotropic drugs [9, 31], the reduction or discontinuation of medication [6], the combination of medication and group psychotherapy [20], brief hospitalization and aftercare [7], the need for long-term use of antiparkinsonian drugs [30], and intermittent pharmacotherapy [43]. Noteworthy VA cooperative clinical trials in other disease areas were started in the late 1950s and 1960s. A VA cooperative study group on hypertension was started in the 1950s (and still exists today). This group was the first to show that antihypertensive drug therapy reduces the long-term morbidity and mortality in patients with severe [54] and moderate [55] elevations of blood pressure. Other areas researched by the early VA cooperative studies included: use of long-term anticoagulants after myocardial infarction; lipid lowering drugs to prevent myocardial and cerebral infarction; treatment of gastric ulcer disease; efficacy of gamma globulin in posttransfusion hepatitis; analgesics to reduce postoperative pain; surgical treatment of coronary artery disease; the effect of portal caval shunt in esophageal varices; and the effects of radical prostatectomy, estrogens, and orchiectomy in the treatment of prostate cancer. In 1962, the VA developed a concept, novel in Federal Government medical research programs at that time, of providing its investigators access to techniques and specialized help and information essential to their research. Four regional research support centers were established: the Eastern Research Support Center at the West Haven, CT VAMC; the Midwest Research Support Center at the Hines, IL VAMC; the Southern Research Support Center at the Little Rock, AR VAMC; and the Western Research Support Center at the Sepulveda, CA VAMC (see Data Management and Coordination). Individual investigators were assisted in such areas as research design, statistical methods, data management, computer programming, and biomedical engineering. The early VA cooperative studies were coordinated by VA Central Office staff in Washington, DC, by these regional research support

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

2

Cooperative Studies Program, US Department of Veterans Affairs

centers, and by contracts with university coordinating centers. The program was led by Mr Lawrence Shaw. Beginning in 1972, a special emphasis was placed on the CSP in the VA’s Medical Research Service and its budget was quadrupled over the next decade. Under the leadership of James Hagans, MD, PhD, the program’s current organization and structure were developed and codified in the Cooperative Studies Program Guidelines [1]. This included the establishment of four statistical/data processing/management coordinating centers and a research pharmacy coordinating center solely dedicated to conducting cooperative studies; central human rights committees attached to each of the statistical coordinating centers; a standing central evaluation committee for the review of all new proposals for VA cooperative studies and all ongoing studies every three years; and clearly defined procedures for the planning, implementation, conduct, and closeout of all VA cooperative studies. The Central Neuropsychiatric Research Laboratory at the Perry Point, MD VAMC; the Eastern Research Support Center at the West Haven, CT VAMC; the Midwest Research Support Center at the Hines, IL VAMC; and a new center at the Palo Alto, CA VAMC were established as the four new VA Cooperative Studies Program Coordinating Center (CSPCCs). The Cooperative Studies Program Clinical Research Pharmacy Coordinating Center (CSPCRPCC) was established at the Washington, DC VAMC, but later relocated to the Albuquerque, NM VAMC in 1977. Daniel Deykin, MD was the first person to head simultaneously the VA research programs both in Health Services Research and Development and the CSP, from 1985 to 1996. He took advantage of this opportunity to promote the development of a series of multicenter clinical trials in the organization and delivery of health services. These trials represented unique challenges in design and conduct. Some of these trials have recently been completed and are in the process of being published [25, 44, 57]. In 1996, John Feussner, MD, MPH was appointed as the VA’s Chief Research & Development Officer, and simultaneously assumed leadership of the CSP. Up until the time Dr Feussner was appointed, the VA Research Service was composed of three major research programs – Medical Research (of which the CSP was a part), Rehabilitation Research & Development, and Health Services Research & Development.

Dr Feussner moved the CSP out of the VA Medical Research Service and elevated the Program to an equal level with the three other major VA research programs. New emphases brought to the Program by Dr Feussner include: initiation of a strategic planning process; more integration and interdependence of the coordinating centers; institution of good clinical practices and standard operating procedures at the coordinating centers; pharmaceutical manufacturing; experimentation to improve the process of informed consent [32]; educational programs in clinical research for VA investigators; partnering with industry, National Institutes of Health (NIH), and international clinical trials groups; the development of three new Epidemiology Research and Information Centers at the VAMCs in Seattle, WA, Boston, MA, and Durham, NC [5]; and Intranet and Internet communications. The strategic planning process initiated in 1997 defined the vision, mission, and specific goals for the Program (Table 1).

Organization and Functioning of the CSP This section describes how a VA cooperative study evolves and the support provided by the VACSP. Table 1

Vision/mission/goals of the VACSP

Vision • The CSP is a premier research program conducting multicenter studies with world-wide impact on health care Mission • To advance the health and care of veterans through education, training, and collaborative research studies that produce innovative and effective solutions to national healthcare problems Goals • To enhance the proficiency of CSP staff and CSP partners (chairpersons, participating investigators) in the conduct of multicenter trials • To enhance the consistency of management support for the CSP • To increase the flow of new research ideas for cooperative studies • To increase the application of research products into clinical practice • To enhance the interdependence of the CSP coordinating centers • To improve the capabilities of dissemination of research findings

Cooperative Studies Program, US Department of Veterans Affairs

3

Investigator Submits Idea to CSP Headquarters Office

Chief, CSP Obtains Reviews from 4-5 Experts on study Idea and Makes Decision

Disapprove

About 70% of Study Ideas are Disapproved

Approve Chief, CSP Assigns Study to One of the 4 CSPCCs and to CSPCRPCC if Drug / Device Study

First Planning Meeting is Held Chief, CSP Must Decide if Satisfactory Progress is Made

No

Study Terminated

Yes Major Changes Changes Made and Resubmitted

Draft Protocol Developed and Second Planning Meeting Held

Changes Requested

HRC Reviews Protocol at Second Meeting

Disapprove

Study Terminated

Approved Minor Changes

Final Protocol Prepared and Submitted to CSEC

Disapproved but Resubmittal Encouraged

CSEC Review

Disapprove

Study Terminated About 40% of protocols are Disapproved

Approve

Figure 1

Development of a VA cooperative study

These aspects of the Program have been reported previously [24, 27, 28].

Planning Request Initiation of a planning request through the evaluation phase is outlined in Figure 1. The VA Research Program, including the CSP, involves strictly intramural research. To receive VA research funding, the investigator must be at least five-eighths time VA. One of the strengths of the CSP is that most of its studies are investigator-initiated. The research questions come from investigators throughout the VA health care system who are on the front lines in providing health care for veterans.

To start the process, the investigator submits to VA Headquarters a 5–10 page planning request outlining the background of the problem, the hypothesis, a brief overview of the design, and anticipated size of the study. The planning request is given a CSP number to aid in tracking the study through its evolutionary phases. The planning request is sent to four or five independent experts in the field who initially judge the importance and feasibility of the study. If this review is sufficiently positive, the study is put into planning and assigned to one of the CSPCCs (and the CSPCRPCC if it involves drugs or devices) for development of the full proposal. This process has evolved to satisfy two important needs. First, the CSP recognizes that the ability to

4

Cooperative Studies Program, US Department of Veterans Affairs

Once the study is approved for planning, the resources of the CSPCCs are applied to the development of the full proposal. Within the coordinating centers, the study is assigned to a specific biostatistician and clinical research pharmacist. These individuals work with the principal proponent in nominating a planning committee which is reviewed and approved by the CSPCC and CSP Directors. The planning committee generally consists of the principal proponent, study biostatistician, study clinical research pharmacist, CSPCC Director, two or three potential participating site investigators, and outside consultants, as needed. The planning committee is funded for two planning meetings. At the first meeting, the basic design features of the study are agreed upon (hypothesis, patient population, treatment groups, primary and secondary endpoints, pharmacologic and drug handling issues, baseline and follow-up data collection and frequency, treatment effect size, sample size, number of sites, duration of study, publication plan, and budget). The full proposal is then written, and at the second planning meeting a draft of the protocol is fine-tuned. Development of a full proposal generally requires six to nine months.

CSPCC. This committee is comprised of scientists and laypeople from the community who review proposals for all new VA cooperative studies and all ongoing studies annually. The committee serves as a central Institutional Review Board (IRB) for studies assigned to the CSPCC and considers such aspects of the proposal as risks versus benefits to the patients, patient management, burdens placed on the patients from participation in the study, community equipoise with regard to the treatments being compared, and the informed consent procedures. This committee has absolute authority over approval or disapproval of the study. Only the HRC has the authority to change its own decisions. The composition of the committee follows VA regulations and is consistent with Food and Drug Administration (FDA) guidelines. Minimum membership of the HRC includes a VA chairperson, a practicing physician from the community, a nonphysician scientist, a veteran representative, a member of a recognized minority group, a clergyman or ethicist, and an attorney. If the HRC approves the study, then the proposal is submitted to VA Headquarters for scientific review and a funding decision. The proposal is sent to four or five experts in the field and a biostatistician for written reviews. All cooperative study proposals are reviewed by a standing scientific review committee, called the Cooperative Studies Evaluation Committee (CSEC). This committee is composed of senior physician scientists and biostatisticians who have had extensive experience in cooperative studies and clinical trials. The CSEC meets in the spring and fall of each year. The principal proponent and study biostatistician present the study to the CSEC in person (reverse site visit), defend their proposal and answer questions from the CSEC members. The CSEC then goes into executive session, and decides to recommend approval or disapproval of the study and, for approvals, gives a scientific priority score, ranging from 10 to 50 with 10 being the best score. The final funding decision is made by the CSP Director. The advantages of this review process are that the study investigators have the opportunity to interact personally with the review body to answer their criticisms and concerns, and the final decision is known immediately following the review and executive session.

Evaluation Phase

Implementation of the Trial

The completed proposal is first reviewed by the Human Rights Committee (HRC) attached to the

Implementation and conduct of a VA cooperative trial are outlined in Figure 2. Once the CSP Director

come up with a good idea needing rigorous test does not necessarily carry with it the ability to pull together all the expertise necessary to plan a clinical trial. This was especially true in the early days, when “trialists” were few and far between, and clinical researchers seldom had training in modern statistical trials design. So it is important to provide access to this expertise early in the planning process. However, such aid is expensive and scarce, so it is important not to waste it on ideas that do not show promise. Thus, the second need is for an initial concept review. This has proved to be a very efficient allocation method; about 70% of all initial proposals are not approved to go on to planning, and of the surviving 30%, about two-thirds complete the planning process. Of those that are successfully planned, about three-quarters are approved and funded. Thus, the method helps to avoid the problem of insufficiently developed protocols, while conserving the scarce resources of planning.

Planning Phase

Cooperative Studies Program, US Department of Veterans Affairs

5

CSEC Review Approved Scientific Priority Score In Fundable Range

N0

Study Terminated

Yes Study is Funded

Organizational Phase: Forms Approval and Printed; Executive Committee and DMB Meet; Organization Meeting, etc.

Patient Recruitment & Follow-up

Study Group & Executive Committee Meet Yearly

DMB & HRC Meet Yearly; Both Committees Must Approve Continuation

Disapprove

Study Terminated

Approve Disapprove CSEC Mid-Year Or Every 3-Year Review

Study Terminated

Approve No Study Completed Yes Closeout Procedures Performed

Manuscripts Prepared

Figure 2

Conduct of a VA cooperative study

approves funding, the implementation phase of the cooperative study begins. All activities in this process are closely coordinated by the CSPCC, CSPCRPCC and the Study Chairperson’s Office. The necessity for carefully controlled medical treatment and data collection procedures for the successful conduct of multicenter clinical trials is well recognized. Because of its administrative structure, the VA provides an environment that is uniquely suited to this type of research. Each participating

facility is funded by one control point for the entire period of the study, and the VAMC system provides a structure in which a relatively high degree of medical, scientific, and administrative control can be exercised. This same degree of control is often more difficult to obtain in studies that involve participating sites from different administrative systems [27]. The CSPCC recommends funding levels and monitors the performance of the individual medical centers. This information is reviewed regularly by the

6

Cooperative Studies Program, US Department of Veterans Affairs

study biostatistician, center director, the Executive Committee, and at least annually by the Data and Safety Monitoring Board. This integrated monitoring of scientific, biostatistical, and administrative aspects by the CSPCC provides a comprehensive approach to the management of multicenter clinical trials, in contrast to other clinical trials biostatistics groups that are responsible only for the analytical and data processing aspects and exercise no administrative control [27]. The Research and Development Committee and the IRB of each participating medical center must review and approve a cooperative study before it can be implemented at that facility. They are able to make modifications to the prototype consent form approved by the CSPCC HRC, but all local modifications must be reviewed and approved by the CSPCC. Included in the implementation component of a cooperative study is the establishment of the Executive Committee and Data and Safety Monitoring Board (DSMB) who share responsibility for the conduct and monitoring of the cooperative study in the ongoing phase. The Executive Committee, which often includes several members of the original Planning Committee, consists of the study chairperson who heads the committee, the study biostatistician, the clinical research pharmacist, two or three participating investigators, and one or two consultants. This committee is responsible for the detailed operational aspects of the study during its ongoing phase and ensures adherence to the study protocol, including aspects relating to patient recruitment, treatment, laboratories, data collection and submission, biostatistical analysis, data processing, subprotocols and reporting. The Executive Committee sometimes recommends probation or termination of sites whose performance is poor. The DSMB consists of five to eight individuals who have not been involved in the planning and development of the proposal, and includes one or two biostatisticians and two or more subject-matter experts in the field(s) of the cooperative study. This committee is charged with the responsibility of monitoring and determining the course of the ongoing study and considers such aspects as patient accrual; performance of the participating sites, CSPCC, and chairperson’s office; and safety and efficacy data. Perhaps a unique feature of the CSP is that the CSPCC HRC also reviews each ongoing study annually and receives the same data reports as presented

to the DSMB. The study chairperson, participating site investigators, and other members of the Executive Committee are masked to the outcome data during the course of the study. Only the DSMB, HRC, CSPCC and CSPCRPCC see the outcome data during the conduct of the study. The fourth body involved in the conduct of the cooperative study is the Study Group, which consists of all participating investigators, the study chairperson (co-chairpersons), biostatistician, clinical research pharmacist, and consultants. This body meets once annually to consider the progress of the study and to resolve problems that may arise at the participating centers. Within the CSPCC, the biostatistician heads a team of administrative, programming and data management personnel that provides regular monitoring of the study. This team develops an operations manual (see Clinical Trials Protocols), in conjunction with the chairperson’s office, to train study personnel in the day-to-day conduct of the trial. They also develop a computer data management system to edit, clean, and manage the study data. Automated query reports are generated by the computer system and sent to the participating sites for data checking and cleaning. Statistical progress reports are published by the CSPCC and distributed to the Study Group, Executive Committee, and DSMB at scheduled meetings. An initial kickoff meeting is held before the study starts to train site personnel in the conduct of the study. Annual meetings are held thereafter to refresh training and discuss issues in the conduct of the study. Frequent conference calls of the study committees are also used to facilitate communication and training. Another unique aspect of the CSP is the CRPCC, which operationalizes the pharmaceutical aspects of the clinical trials (Table 2). In the planning stages of the study the clinical research pharmacist designs the drug or device treatment and handling protocol and works with the study chairperson and pharmaceutical and device industries to purchase or obtain donations of clinical supplies for the study. The CRPCC coordinates the development of appropriate drug dosage formulations and the manufacture of study drugs or devices. In the event that drug donations are not possible, the CRPCC has the expertise and capability to provide the in-house manufacture of active drugs and matching placebo. Drugs for all cooperative studies

Cooperative Studies Program, US Department of Veterans Affairs Table 2 Unique functions and roles of the CSP pharmacy coordinating center • • • • • • • • • • •

Design of a drug or device handling protocol for each study involving drugs or devices Preparation and submission of INDAs or IDEs Obtaining donations or purchase of clinical supplies for study Coordination of appropriate drug dosage formulations and manufacture of study drugs or devices Quality control testing of drugs Development of blinding methods Storage, packaging and shipment of clinical supplies to pharmacies at the participating sites Computerized drug inventory and information system to track and replenish supplies at site pharmacies Monitoring adverse medical events and reporting to appropriate authorities Monitoring, auditing and education services to ensure sites are in compliance with GCP Preparation of final drug/device accountability reports

must pass the testing of the CRPCC’s quality control testing laboratory. The CRPCC also assesses study product blinding methods. At the CRPCC, study medications are stored in an electronically controlled and secured environment. The CRPCC customizes labels and packages all study medications, which are centrally distributed to the pharmacies at the participating sites. In doing so, the CRPCC provides a computerized drug inventory and information system for complete accountability of clinical supplies. This includes automated study supply tracking and replenishment systems for maintaining adequate study supplies at participating sites as well as automated telephone randomization and drug assignment systems. The clinical research pharmacist is then able to direct and monitor the study prescribing and dispensing activities as well as to monitor the compliance with the study protocol treatments at the participating sites. At the end of the study the CRPCC directs the retrieval and disposition of unused clinical supplies and prepares a final drug/device accountability report. The clinical research pharmacist also works closely with the study chairperson and the manufacturers to prepare, submit, and maintain Investigational New Drug Application (INDAs) or Investigational Device Exemption (IDEs), which

7

includes preparing and submitting annual and special reports to the FDA. Along with this responsibility, the clinical research pharmacist coordinates the monitoring and reporting of all adverse medical events to study management, FDA and associated manufacturers. Recently the CRPCC established a central Good Clinical Practices (GCP) Assurance Program. The Program provides monitoring, auditing, and educational services for all VA cooperative studies to ensure that the participating sites are in GCP compliance. If needed, the Program is capable of providing full GCP monitoring for studies under regulatory (FDA) scrutiny.

Final Analysis and Publication Phase Upon completion of patient intake and follow-up, the study enters the final analysis and publication phase. If the Executive Committee, the CSPCC, and the study biostatistician have performed their tasks well, this phase should be quite straightforward. It requires an updating of all study files and the processing and analysis of the complete data set. The interim statistical analyses that were run during the ongoing phase of the study are now executed on the complete data. In addition, some analyses may point to additional questions that would be of interest; however, it is anticipated that the majority of final analyses and interpretation of results will occur within 6 to 12 months after study termination. All publications emanating from the cooperative study must be approved by the Executive Committee. Although the responsibility of the DSMB terminates at the end of data collection, the Board is at times requested to review manuscripts and give advice prior to submission for publication [27]. Usually each trial generates a number of manuscripts. The Executive Committee establishes priorities for statistical analyses and manuscript development and appoints writing committees composed of members of the Executive Committee and Study Group for each paper. Authorship of the main paper(s) usually consists of the chairperson, study biostatistician, study clinical research pharmacist, members of the Executive Committee, and, in some cases, the participating site investigators. Secondary papers are often written by other members of the Executive Committee and site investigators.

8

Cooperative Studies Program, US Department of Veterans Affairs

The CSPCC serves as the final data repository for the study. The study database, protocol, operations manuals, forms, study correspondence and interim and final statistical progress reports are archived at the CSPCC.

Role of the Biostatistician and Pharmacist in the CSP One of the unique features of the CSP is that the biostatistician at the CSPCC plays a major organizational, fiscal, and administrative role, in addition to the usual technical role. In recent times, as the administration of studies has become more complex, the biostatistician may be assisted by a study coordinator but, as in the past, the greater part of the burden of management falls on the biostatistician. In contrast to the pharmaceutical industry and to many Contract Research Organization (CROs), the biostatistician is responsible for monitoring site adherence to protocol, recruitment, and many other aspects of the study conduct. In addition, the study pharmacist plays a key role in monitoring adverse effects, maintaining supplies of the study drug, regulatory reporting, and the like. In a sense, the study team is deployed to support the investigators, but has independent authority and responsibility as well. One of the strengths of this approach to study management is that it is possible to guarantee some degree of uniformity in the conduct of the studies, independent of the varying managerial skills and style of the study chairs. The biostatistician and pharmacist, together with the coordinating centers of which they are a part, provide institutional memory and continuity. Their central position on the study teams reinforces the key idea that the studies mounted by the VACSP are the joint responsibility of the program and the investigators. Such an intramural program can only succeed on a limited budget if issues of cost and complexity are kept to the forefront during the planning process. A consequence that is easily observed is that the typical CSP trial is a lean, focused attack on a single important clinical question, rather than a broad-based research project with many interwoven strands of investigation. In contrast to the much larger NIH clinical trials efforts, which are organized along disease lines, the CSP biostatisticians and CSPCCs are generalists, doing studies in all areas of medicine with

relevance to the VA. Along the way, some centers have developed some special experience in certain areas, but there has never been a “heart” center or a “cancer” center. Because the CSP has such a broad medical purview, but a relatively low volume of studies, it has not made economic sense to specialize. The scarce resource of statistical and data management expertise has needed to be allocated efficiently to support the proposals that were emerging from the field. Since VA resources have followed the strength of the proposals rather than disease areas, the CSPCCs have not specialized to any large degree. While there are undoubted advantages to specialization, as shown by the contributions made by the National Cancer Institute (NCI) (see Cooperative Cancer Trials) and the National Heart, Lung, and Blood Institute (NHLBI) (see Cooperative Heart Disease Trials) statisticians to the statistical science of their disease areas, there are some advantages to generalizing. In particular, it has been possible to transplant methods and lessons learned from well-studied areas such as cancer and heart disease, to other areas such as psychopharmacology, device research, health services research, and trials of surgical procedures. The absence of disease-area “stovepiping” has facilitated a high general level of sophistication in the conduct of trials, with techniques travelling readily across borders. This cross-pollination has also been facilitated by the structure of the VACSP scientific peer review. The standing committee that reviews and recommends studies for funding mixes disciplines with common expertise in multisite studies. Ad hoc reviewers provide the crucial discipline-specific input to the committee, but the same committee may review a heart failure trial in the morning and a psychopharmacology trial in the afternoon. The result is a high degree of uniformity in the standards for the research across disease areas, and this has been an enduring strength of the program.

Ongoing and Completed Cooperative Studies (1972–2000) One hundred and fifty-one VA cooperative studies were completed or are currently ongoing in the period 1972–2000. Table 3 presents the health care areas

Cooperative Studies Program, US Department of Veterans Affairs Table 3 Health care areas of ongoing and completed VA cooperative studies (1972–2000) Health care area Cardiology/cardiac surgery Hypertension Gastrointestinal Substance abuse Mental health Infectious diseases Cancer Dental General surgery/anesthesia Cerebrovascular Peripheral vascular Military service effects Ambulatory care Epilepsy Genitourinary Diabetes Renal Sleep Pulmonary Hematology Hearing One each in seven areasa Total

Number of studies

Percent of studies

24 15 14 11 10 9 8 6 6 5 5 4 4 4 4 3 3 3 2 2 2 7 151

15.9 10.0 9.3 7.3 6.6 6.0 5.3 4.0 4.0 3.3 3.3 2.6 2.6 2.6 2.6 2.0 2.0 2.0 1.3 1.3 1.3 4.7 100.0

a Analgesics, arthritis, geriatrics, hospital-based home care, laboratory quality control, computerized neuropsychological testing, ophthalmology

of these studies. These areas are generally reflective of the major health problems of the US veteran population, consisting mainly of middle-aged and senior adult males. Studies in cardiology and cardiac surgery represent 15.9% of the 151 studies, followed by hypertension (10.0%), gastrointestinal diseases (9.3%), substance abuse (7.3%), mental health (6.6%), infectious diseases (6.0%), and cancer (5.3%). There are a few notable disease areas that are prevalent in the VA population and yet might be considered underrepresented in the CSP. These include diabetes (2.0%), renal diseases (2.0%), pulmonary diseases (1.3%), hearing diseases (1.3%), arthritis (0.7%), and ophthalmologic diseases (0.7%). Because the CSP mainly relies on investigatorinitiated studies, the conclusion might be drawn that these subspecialties have underutilized the Program. Although studies on effects of military service represent only 2.6% of the 151 studies, studies listed in other categories have investigated treatments for

9

service-connected illnesses (e.g. posttraumatic stress disorder studies are categorized under mental health, and the substance abuse studies could be considered consequences of military service). Table 4 briefly summarizes some of the noteworthy VA cooperative clinical trials that were completed in the 1980s and 1990s. Many of these trials resulted in advances in clinical medicine that could immediately be applied to improve the health care of US veterans and the US population in general.

Current Challenges and Opportunities Although the VACSP has had numerous past successes, it faces many challenges and opportunities in the future. These include: (1) changes in the VA health care system and their effects on research; (2) nationwide concerns about violations of patients’ rights in research; (3) increasing the efficiency and interdependence among the coordinating centers and standardizing procedures; (4) ensuring the adequacy of flow of research ideas and training of investigators; and (5) partnering with industry, other federal agencies, nonprofit organizations, and international clinical trial groups to enhance the capacity of the Program.

Changes in the VA Health Care System The VA health care system has been undergoing substantial changes that could adversely affect research. In 1996, the VA reorganized into 22 geographically defined Veterans Integrated Service Networks (VISNs). Much of the central authority, decisionmaking, and budgeting once performed in VA Headquarters in Washington, DC, has been delegated to the 22 VISN offices. Within the VISNs, administrative and health care services and in some cases entire VAMCs are being consolidated. The largest component of the VA patient population, the World War II veterans, is rapidly declining. Health care personnel in some VISNs are experiencing reductions in force, with the result that the remaining personnel have limited time to devote to research. These factors may already be adversely affecting the Program’s ability to meet recruitment goals in ongoing trials [23].

Concerns About Patients’ Rights in Research The nature of the veteran population treated at VA hospitals raises some special issues in human rights

10

Cooperative Studies Program, US Department of Veterans Affairs Table 4 • • • • • • • • • • • • • • • • • • • • • • • • • • •

Noteworthy VA cooperative studies

80% of strokes in patients with atrial fibrillation can be prevented with low-dose warfarin [15] Carotid endarterectomy is effective in preventing strokes in symptomatic and transient ischemic attacks in asymptomatic patients [26, 39] Aggressive treatment of moderate hypertension works well in elderly patients [19, 37] Age and racial groupings can be used to optimize selection of first line drugs in hypertension [38] Coronary artery bypass surgery prevents mortality in patients with left main disease and in high-risk patients without left main disease [42, 49] Low dose aspirin reduces heart attacks and death in 50% of patients with unstable angina [34] Vasodilators and angiotensin converting enzyme inhibitors prevent deaths in patients with congestive heart failure [12, 13] Low dose aspirin started 6 hours after coronary artery bypass surgery and continued for one year prevents the occlusion of the bypass grafts [17, 18] Mechanical artificial aortic heart valves prolong survival more than bioprosthetic aortic heart valves [22] A conservative, ischemia-guided strategy is safe and effective for management of patients with non-Q-wave myocardial infarction [4] Digoxin does not reduce mortality but does reduce hospitalizations in patients with congestive heart failure [50] The rate of coronary events (myocardial infarction or death) in men with coronary heart disease can be reduced by 22% with Gemfibrozil therapy, which increases high density lipoprotein cholesterol and lowers triglyceride levels [45] Progression of human immunodeficiency virus (HIV) infection to full blown acquired immune deficiency syndrome (AIDS) can be delayed with the drug zidovudine [21] Steroid therapy does not improve survival of patients with septic shock [52] Patients with advanced laryngeal cancer can be treated with larynx-sparing chemotherapy and radiation compared with standard surgical removal of the larynx and have equivalent long-term survival [14] The drug Terazosin is more effective than Finasteride in relieving the symptoms of benign prostatic hyperplasia [33]. Transurethral resection of the prostate is an effective operation, but Watchful Waiting can be effective in many patients [56] An implantable insulin pump is more effective than multiple daily insulin injections in reducing hypoglycemic side-effects, and enhancing quality of life in adult-onset Type II diabetes mellitus [46] Multi-channel are superior to single-channel cochlear implants in restoring hearing to patients with profound hearing loss [11] Sclerotherapy is an effective treatment for esophageal varices in patients who have had prior bleeds but not in patients without prior bleeds [51] Antireflux surgery is more effective than medical therapy in patients with complicated gastroesophageal reflux disease [47] Severely malnourished patients benefit from pre-operative total parenteral nutrition but mildly malnourished patients do not [53] Clozapine is a cost-effective treatment for patients with refractory schizophrenia who have high hospital use [44] Erythropoietin administered subcutaneously compared with intravenously can significantly reduce the costs of hemodialysis [29] Use of intrapleural tetracycline reduces recurrence rate by 39% in patients with spontaneous pneumothorax [35] Rapid access to high quality primary care for patients with severe chronic illnesses greatly improves patient satisfaction with care but may lead to an increase in hospital readmissions [57] Levomethadyl acetate (LAAM) is a safe and efficacious drug to use for heroin addiction. Studies were used to gain FDA approval for LAAM as treatment for heroin addiction [16, 36] Systemic corticosteroids improve clinical outcomes up to three months in patients with chronic obstructive pulmonary disease [40]

Cooperative Studies Program, US Department of Veterans Affairs protections (see Medical Ethics and Statistics). The VA treats about four million veterans, who tend to be less well off than the average veteran (or the average citizen). They are often more severely ill than non-VA patients with the same age and diagnosis, and often have multiple co-morbidities. They are on average more dependent on the VA for their health care than the typical non-VA patient is on his or her usual health care provider. Against this background we note the extraordinary willingness of the veteran patient to engage in research, trusting the clinical researcher to an astonishing degree. Such trust demands an extraordinary level of protection in response. The CSP has instituted a unique framework of human subjects’ protections, going beyond the usual procedures that other federal sponsors and drug companies require. This begins in the planning stage, when each proposal must undergo a rigorous review by the HRC attached to the coordinating center. It typically meets for several hours over a single protocol, reviewing it in fine detail. The protocol cannot go forward without their independent approval. The CSP also requires the usual individual site IRB approval, and other reviews that are mandated at the local site, before a study can start at a site. The ongoing IRB reviews at the local sites (annually, or more often, as stipulated in the initial review) are monitored by the CSP staff. As has become standard in multisite trials, each CSP study has its own independent DSMB that meets at least annually to review the progress of the study. The unique CSP innovation to this process is the joint review by the HRC and DSMB. Thus, after every DSMB meeting, the two groups meet to review and recommend, with the same basis of information on study progress. The CSP has found that the HRC is able to hear the recommendation of the DSMB, which is typically heavily weighted with subjectmatter expertise, and interpret it in the light of the other perspectives they bring. The CSP believes that this has been a successful experiment in resolving the knotty issue of how to obtain full and informed ongoing review of studies where investigators are kept blind, and site-level information must be far less informative than the big picture presented to the DSMB. We believe that such joint reviews add considerably to the level of protection of human subjects.

11

In addition, members of the central HRCs conduct three site visits per year during which patients are interviewed about their participation in the trials. Thus, the Program as a whole conducts 12 such visits per year. The Albuquerque auditing group periodically site visits VAMCs participating in cooperative studies and performs audits to ensure that the sites are complying with GCP guidelines. The CSPCCs also receive copies of consent forms from all patients in all of the trials as further evidence of proper consent procedures. The VA recently established its own office to oversee the protection of patients’ rights in VA research, performing functions similar to those of the Office of Protection from Research Risks (OPRR) of the Department of Health and Human Services. IRBs at VAMCs currently are required to be accredited by an external, non-VA entity. In addition to these standard procedures, followed in all studies, the CSP has recognized two other areas of human subjects’ protection in which it can make a contribution. The Enhancing the Quality of Informed Consent (EQUIC) program [32] is designed to institutionalize the process of testing innovations in methods for obtaining informed consent. It piggybacks tests of new methods on ongoing CSP studies, and provides a centralized assessment of the quality of informed consent encounters (by remote telephone interview of patients). In the spirit of EQUIC, a substudy is being conducted in one ongoing VA cooperative study to evaluate the utility of an informed consent document developed by a focus group of subjects eligible for the trial [41]. The second topic that the CSP has engaged are the ethical, legal, and social implications of genetics research, specifically of deoxyribonucleic acid (DNA) banking with linked clinical (phenotype) data. The CSP has begun a project to provide uniform methods for obtaining and banking such samples. Steps to ensure human subjects’ protection in VA cooperative studies are listed in Table 5.

Efficiency and Interdependence of the CSPCCs The VACSP recently contracted with an outside vendor to help develop standard operating procedure (SOPs) for the CSPCCs. Twenty-two SOPs were developed in the areas of administration, planning and implementing clinical trials, data management,

12

Cooperative Studies Program, US Department of Veterans Affairs

Table 5 Steps to ensure human subjects’ protection in the VACSP • • • • • • • • • • • • • •

Investigator’s integrity Development of proposal through collaboration between investigators and CSPCCs HRC review of proposal initially Site Monitoring and Review Team (SMART) audit of consent form contents CSEC review of proposal Initial review of proposal by participating site R&D and IRB Annual central reviews of trial by DSMB and HRC Annual reviews of study by local R&D committee and IRB SMART audit of participating sites HRC site visits and interviews of study patients Receipt of copies of patient consent forms by CSPCC, local research offices, and local pharmacies Implementation of SOPs and good clinical practices Compliance with all FDA and VA regulations Innovative studies on improving informed consent

study closeout, and study oversight (Table 6). By standardizing among and within the coordinating centers certain procedures that are performed in every study, we will achieve an even higher level of support to all studies more efficiently than previously done. The SOPs will also enable the CSPCCs to be in better compliance with GCP principles and International Conference on Harmonization (ICH) guidelines. Since 1996, the Directors of the Program and centers have been meeting as a group semiannually to identify current and future challenges and opportunities, and to develop annual strategic plans to respond to these challenges and opportunities. This has enhanced the development of mutual projects which the centers can work on together to further the goals of the organization as a whole, such as the development of a Clinical Research Methods Course, a one-year sabbatical program for clinical investigators to enhance their training and skills, and SOPs for the central HRCs.

Ensuring the Adequacy of Flow of Ideas and Training of Investigators In recent years, the CSP has developed several educational opportunities to help train VA investigators

Table 6

Recently adopted SOPs for the VACSP

Administration • Preparing, issuing and updating SOPs • Training and training documentation Planning and implementation of clinical trials • Developing, approving and amending protocols • Study/training meetings • Preparing and approving operations manuals • Study initiation • Developing and approving case report forms • Creating and validating data entry screens and programs • Preparing, documenting and validating data checking programs • Preparing, documenting and validating statistical programs • Developing and conducting statistical analyses Handling data from clinical trials • Randomization, blinding and unblinding • Central monitoring • Case report form flow and tracking • Data entry and verification • Data cleaning • Reporting adverse events Study closeout • Archiving study documentation • Study closeout Study oversight • Assuring site R&D and IRB approvals • DSMB • HRC

in clinical research and to encourage utilization of the Program to answer important clinical questions. These include a five-day course in clinical trials and sabbatical and career development programs focused on clinical research methodology. The five-day course is taught once each year and involves 10 faculty members (two from each of the five coordinating centers) and 60 VA investigators selected from applications from throughout the country. The course consists of 15 lecture/discussion sessions on various aspects of designing a clinical trial, interspersed with breakout sessions during which the students are divided into five planning committees to design a clinical trial. On the last day of the course, the student groups take turns in presenting their clinical trials and receiving critiques from the audience. The course has been taught twice and has received excellent feedback from the students.

Cooperative Studies Program, US Department of Veterans Affairs

13

The CSP Career Development Program provides protected time to clinician–investigators for a period of concentrated clinical research activity. The objective is to build capacity in a wide geographic distribution for the Department of Veterans Affairs to conduct clinical research in acute-care hospitals, long-term care facilities, or outpatient settings. The Program is designed to foster the research careers of clinician–scientists who are not yet fully independent but who seek to become independent clinical researchers. The award provides three years of salary and some supplemental research support, and the awardees are expected to work at least part of the time at one of the five CSPCCs or three Epidemiology Research and Information Center (ERICs). In 1999 CSP announced a sabbatical program for established clinician–scientists to train at one of the CSPCCs or ERICs for up to one year. The purpose of the program is to support clinician–investigators who wish to secure training time to learn about the conduct of cooperative studies and epidemiologic research.

Concluding Remarks

Partnering with Outside Organizations

In summary, we believe that there are considerable strengths to conducting multicenter clinical trials in

The VACSP has partnered with NIH and industry for many years in conducting multicenter clinical trials. In recent years, a special emphasis has been placed on partnering with outside agencies to enhance the effect of the limited VA research funding, and these efforts have been fruitful. Recent examples of this partnering include: the Digitalis in Heart Failure Trial, sponsored by the VA, NHLBI, and Burroughs–Wellcome Company and conducted in 302 VA and non-VA sites in the US and Canada; a series of trials sponsored by the VA and the National Institute of Drug Abuse (NIDA) to evaluate new treatments for drug abuse; the Prostate Cancer Intervention Versus Observation Trial (PIVOT), sponsored by the VA, Agency for Healthcare Quality and Research (AHQR) and NCI; the Beta-Blocker Evaluation of Survival Trial (BEST), funded by the VA, NHLBI, and industry; the Clinical Outcomes Utilizing Revascularization and Aggressive Drug Evaluation (COURAGE) trial, supported by the VA and 10 pharmaceutical companies; the VA/National Institute of Deafness and Other Communication Disorders (NIDCD) Hearing Aid Clinical Trial; and the Shingles Prevention Study sponsored by the VA, NIH, and a pharmaceutical company.

The VACSP has been working with the American College of Surgeons to promote clinical trials evaluating new surgical operations and technologies. This collaboration has resulted in a VA trial comparing the outcomes of laparoscopic vs. open tensionfree inguinal hernia repair, a trial comparing open tension-free hernia repair vs. watchful waiting funded by AHQR, and a trial comparing pallidotomy vs. deep brain stimulation in Parkinson’s Disease. The VACSP has also issued a program announcement for the development of multinational clinical trials between the VA and the Medical Research Councils of Canada and the UK. As the field of clinical trials matures, it is likely that achievable treatment effect sizes will decrease, necessitating larger and larger trials, or “mega” trials. These types of collaborations will be important in the future, as the larger trials will exceed the capacity of any single clinical trials program.

Table 7

Strengths of the VACSP

Related to VA health care system • Large veteran population willing to participate in research, well-represented by minority groups • Largest integrated healthcare system in US with 172 medical centers under single administrative system • High-quality physician–investigators • National administrative databases that allow for tracking of patients • Supportive management in VA Headquarters • System of local research offices and IRBs at participating sites that facilitate multicenter research Related to the CSP • Quality and experience of the coordinating centers • Well-established mechanisms for conducting multi-site trials • Planning process usually produces tightly focused, cost-effective protocols • Rigorous review process by HRC and CSEC • Guidelines and SOPs for conducting trials • Multiple levels of protection of research subjects • Ability to conduct trials with high power and generalizability, so the impact on changing health care practices is maximized compared with other research programs

14

Cooperative Studies Program, US Department of Veterans Affairs

Table 8

Limitations of the VACSP

[3]

Related to VA health care system • • • •

Primarily male population, limiting generalizability of results Large studies in female and childhood diseases are not possible Changes in the health care system, including aging and declining of veteran population, decentralization and consolidation of facilities Reduction in dedicated research time for physician–investigators

Related to the CSP • • •

Long duration from submission of planning request to publication of main results raises the risk of study becoming outdated Limitation of funding Limited capacity to conduct mega trials within VA system

[4]

[5]

[6]

[7]

[8]

the VA health care system, as enumerated in Table 7. There are also some acknowledged limitations of the Program, some of which can be addressed in the future (Table 8). This article has described the history, organization and productivity of a clinical trials program designed as an integral part of a large health care system. The biostatistical and pharmacy positions in the Program are ideal from the standpoint that these people are integrally involved in the research from beginning to end and play a major role in the conduct of the trials. The Program is an example of how clinician–investigators and methodologists can work together successfully to design and conduct largescale clinical research.

[9]

[10]

[11]

Acknowledgments We are extremely indebted to the foresight and support of the US Congress, Executive Branch, and VA management, and to the dedication of the health care providers and veteran patients in the VA system to enable us to carry out this important research.

[12]

[13]

References [1]

[2]

Anonymous (1997). Cooperative Studies Program Guidelines for the Planning and Conduct of Cooperative Studies. Office of Research and Development, Department of Veterans Affairs, Washington, DC. Ball, J., Klett, C.J. & Gresock, C.J. (1959). The Veterans Administration study of prefrontal lobotomy, Journal of Clinical and Experimental Psychopathology 20, 205–217.

[14]

[15]

Barnwell, J.B., Bunn, P.A. & Walker, A.M. (1947). The effect of streptomycin upon pulmonary tuberculosis, American Review of Tuberculosis 56, 485–507. Boden, W.E., O’Rourke, R.A., Crawford, M.H., et al. (1998). Outcomes in patients with acute non-Q-wave myocardial infarction randomly assigned to an invasive as compared with a conservative management strategy, New England Journal of Medicine 338, 1785–1792. Boyko, E.J., Koepsell, T.D., Gaziano, J.M., et al. (2000). U.S. Department of Veterans Affairs medical system as a resource to epidemiologists, American Journal of Epidemiology 151, 307–314. Caffey, Jr, E.M., Diamond, L.S., Frank, T.V., et al. (1964). Discontinuation or reduction of chemotherapy in chronic schizophrenics, Journal of Chronic Diseases 17, 347–358. Caffey, Jr, E.M., Galbrecht, C.R. & Klett, C.J. (1971). Brief hospitalization and aftercare in the treatment of schizophrenia, Archives of General Psychiatry 24, 81–85. Casey, J.F., Bennett, I.F., Lindley, C.J., et al. (1961). Drug therapy in schizophrenia: a controlled study of the relative effectiveness of chlorpromazine, promazine, phenobarbital, and placebo, Archives of General Psychiatry 4, 381–389. Casey, J.F., Hollister, L.E., Klett, C.J., et al. (1961). Combined drug therapy of chronic schizophrenics: a controlled evaluation of placebo, dextroamphetamine, imipramine, isocarboxazid, and trifluoperazine added to maintenance doses of chlorpromazine, American Journal of Psychiatry 117, 997–1003. Casey, J.F., Lasky, J.J., Klett, C.J. & Hollister, L.E. (1960). Treatment of schizophrenic reactions with phenothiazine derivatives: A comparative study of chlorpromazine, triflupromazine, mepazine, prochlorperazine, perphenazine and phenobarbital, American Journal of Psychiatry 117, 97–105. Cohen, N.L., Waltzman, S.B., Fisher, S.G., et al. (1993). A prospective, randomized study of cochlear implants, New England Journal of Medicine 328, 233–237. Cohn, J.N., Archibald, D.G., et al. (1986). Effect of vasodilator therapy on mortality in chronic congestive heart failure. Results of a Veterans Administration Cooperative Study, New England Journal of Medicine 314, 1547–1552. Cohn, J.N., Johnson, G., Zeische, S., et al. (1991). A comparison of enalapril with hydralazine isorbide dinitrate in the treatment of chronic congestive heart failure, New England Journal of Medicine 325, 303–310. Department of Veterans Affairs Laryngeal Cancer Study Group (1991). Induction chemotherapy plus radiation compared with surgery plus radiation in patients with advanced laryngeal cancer. New England Journal of Medicine 324, 1685–1690. Ezekowitz, M.D., Bridges, S.L., James, K.E., et al. (1992). Warfarin in the prevention of stroke associated

Cooperative Studies Program, US Department of Veterans Affairs

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]

with nonrheumatic atrial fibrillation, New England Journal of Medicine 327, 1406–1412. Fudala, P.J., Vocci, F., Montgomery, A. & Trachlenberg, A.I. (1997). Levomethyl acetate (LAAM) for the treatment of opioid dependence: a multisite, open-label study of LAAM safety and an evaluation of the product labeling and treatment regulations, Journal of Maintenance in the Addictions 1, 9–39. Goldman, S., Copeland, J., Moritz, T., et al. (1989). Saphenous vein graft patency 1 year after coronary artery bypass surgery and effects of antiplatelet therapy. Results of a Veterans Administration Cooperative Study, Circulation 80, 1190–1197. Goldman, S., Copeland, J., Moritz, T., et al. (1991). Starting aspirin therapy after operation. Effects on early graft patency. Circulation 84, 520–526. Goldstein, G., Materson, B.J., Cushman, W.C., et al. (1990). Treatment of hypertension in the elderly: II. Cognitive and behavioral function. Results of a Department of Veterans Affairs Cooperative Study, Hypertension 15, 361–369. Gorham, D.R., Pokorny, A.D. & Moseley, E.C. (1964). Effects of a phenothiazine and/or group psychotherapy with schizophrenics, Diseases of the Nervous System 25, 77–86. Hamilton, J.D., Hartigan, P.M., Simberkoff, M.S., et al. (1992). Early versus later zidovudine therapy of patients with symptomatic human immunodeficiency virus infection: results of a randomized, double-blind VA Cooperative Study, New England Journal of Medicine 326, 437–443. Hammermeister, K., Sethi, G.K., Henderson, W.G., et al. (1999). Outcomes 15 years after valve replacement with a mechanical versus a bioprosthetic valve: final report of the VA randomized trial, Journal of the American College of Cardiology 36, 1152–1158. Henderson, W.G. (2000). Is it becoming more difficult to attain target sample sizes in clinical trials? Presentation at the 21st Annual Meeting of the Society for Clinical Trials. Toronto, Canada, April 16–19. Henderson, W.G. (1980). Some operational aspects of the Veterans Administration Cooperative Studies Program from 1972–1979, Controlled Clinical Trials 1, 209–226. Henderson, W.G., Demakis, J., Fihn, S.D., et al. (1998). Cooperative studies in health services research in the Department of Veterans Affairs, Controlled Clinical Trials 19, 134–148. Hobson, R.W., Weiss, D.G., Fields, W.S., et al. (1993). Efficacy of carotid endarterectomy for asymptomatic carotid stenosis, New England Journal of Medicine 328, 221–227. James, K.E. (1980). A model for the development, conduct, and monitoring of multicenter clinical trials in the Veterans Administration, Controlled Clinical Trials 1, 193–207. Kathe, B.A., Chan, Y.-K., Buehler, D.A., et al. (1981). Protection of patient rights and welfare in the VA

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

[37]

[38]

[39]

[40]

[41]

15

Cooperative Studies Program, Controlled Clinical Trials 2, 267–274. Kaufman, J.S., Reda, D.J., Fye, C.L., et al. (1998). Subcutaneous compared with intravenous epoetin in patients receiving hemodialysis, New England Journal of Medicine 339, 578–583. Klett, C.J. & Caffey, Jr, E.M. (1972). Evaluating the long-term need for antiparkinson drugs by chronic schizophrenics, Archives of General Psychiatry 26, 374–379. Lasky, J.J., Klett, C.J., Caffey, Jr, E.M., et al. (1962). Drug treatment of schizophrenic patients: a comparative evaluation of chlorpromazine, chlorprothixene, fluphenazine, reserpine, thioridazine and triflupromazine, Diseases of the Nervous System 23, 698–706. Lavori, P.W., Sugarman, J., Hays, M.T. & Feussner, J.R. (1999). Improving informed consent in clinical trials: a duty to experiment, Controlled Clinical Trials 20, 187–193. Lepor, H., Williford, W.O., Barry, M.J., et al. (1996). The efficacy of terazosin, finasteride, or both in benign prostatic hyperplasia, New England Journal of Medicine 335, 533–539. Lewis, H.D., Davis, J.W., Archibald, D.G., et al. (1983). Protective effects of aspirin against acute myocardial infarction and death in men with unstable angina: results of a Veterans Administration Cooperative Study, New England Journal of Medicine 309, 396–403. Light, R.W., O’Hara, V.S., Moritz, T.E., et al. (1990). Intrapleural tetracycline for the prevention of recurrent spontaneous pneumothorax. Results of a Department of Veterans Affairs Cooperative Study, Journal of the American Medical Association 264, 2224–2230. Ling, W., Charuvastra, C.V., Kaim, S.C. & Klett, C.J. (1976). Methyl acetate and methadone as maintenance treatment for heroin addicts: a Veterans Administration cooperative study, Archives of General Psychiatry 33, 709–720. Materson, B.J., Cushman, W.C., Goldstein, G., et al. (1990). Treatment of hypertension in the elderly: I. Blood pressure and clinical changes. Results of a Department of Veterans Affairs Cooperative Study, Hypertension 15, 348–360. Materson, B.J., Reda, D.J., Cushman, W.C., et al. (1993). Single-drug therapy for hypertension in men. A comparison of six antihypertensive agents with placebo, New England Journal of Medicine, 328, 914–921. Mayberg, M.R., Wilson, S.E., Yatsu, F., et al. (1991). Carotid endarterectomy and prevention of cerebral ischemia in symptomatic carotid stenosis, Journal of the American Medical Association 266, 3289–3294. Niewoehner, D., Erbland, M.L., Deupree, R.H., et al. (1993). Effect of systemic glucocorticoids on exacerbations of chronic obstructive pulmonary disease, New England Journal of Medicine 340, 1941–1947. Peduzzi, P., Guarino, P., Donta, S., et al. (2000). Design of an informed consent study to evaluate the utility of a

16

[42]

[43]

[44]

[45]

[46]

[47]

[48]

[49]

Cooperative Studies Program, US Department of Veterans Affairs focus group consent document in the VA cooperative study. A randomized multicenter controlled trial of multi-modal therapy in veterans with Gulf War illness (CSP 470). Presentation at the 21st Annual Meeting of the Society for Clinical Trials. Toronto, Canada. April 16–19. Peduzzi, P., Kamina, A. & Detre, K. (1998). Twenty-two year follow-up in the VA cooperative study of coronary artery bypass surgery for stable angina, American Journal of Cardiology 81, 1393–1399. Prien, R.F., Gillis, R.D. & Caffey, Jr, E.M. (1973). Intermittent pharmacotherapy in chronic schizophrenia, Hospital and Community Psychiatry 24, 317–322. Rosenheck, R., Cramer, J., Xu, W., et al. (1997). A comparison of clozapine and haloperidol in hospitalized patients with refractory schizophrenia, New England Journal of Medicine 337, 809–815. Rubins, H.B., Robins, S.J., Collins, D., et al. (1999). Gemfibrozil for the secondary prevention of coronary heart disease in men with low levels of highdensity lipoprotein cholesterol, New England Journal of Medicine 341, 410–418. Saudek, C.D., Duckworth, W.C., Giobbie-Hurder, A., et al. (1996). Implantable insulin pump vs. multipledose insulin for non-insulin-dependent diabetes mellitus. A randomized clinical trial, Journal of the American Medical Association 276, 1322–1327. Spechler, S.J. & the Department of Veterans Affairs Gastroesophageal Reflux Disease Study Group (1992). Comparison of medical and surgical therapy for complicated gastroesophageal reflux disease in veterans, New England Journal of Medicine 326, 786–792. Streptomycin Committee (1947). The effects of streptomycin on tuberculosis in man, Journal of the American Medical Association 135, 634–641. Takaro, T., Hultgren, H.N., Lipton, M.J., et al. (1976). The VA cooperative randomized study of surgery for coronary arterial occlusive disease. II. Subgroup with significant left main lesions, Circulation 54, (Suppl. III), 107–117.

[50]

[51]

[52]

[53]

[54]

[55]

[56]

[57]

The Digitalis Investigation Group (1997). The effect of digoxin on mortality and morbidity in patients with heart failure, New England Journal of Medicine 336, 525–533. The VA Cooperative Variceal Sclerotherapy Group (1991). Prophylactic sclerotherapy for esophageal varices in male alcoholics with cirrhosis: a randomized single blind multi-center clinical trial, New England Journal of Medicine, 324, 1779–1784. The Veterans Administration Systemic Sepsis Cooperative Study Group (1987). Effect of high-dose glucocorticoid therapy on mortality in patients with clinical signs of systemic sepsis, New England Journal of Medicine 317, 659–665. The Veterans Affairs Total Parenteral Nutrition Cooperative Study (1991). Perioperative total parenteral nutrition in surgical patients, New England Journal of Medicine 325, 525–532. VA Cooperative Study Group on Antihypertensive Agents (1967). Effects of treatment on morbidity in hypertension. Results in patients with diastolic blood pressures averaging 115 through 129 mm Hg, Journal of the American Medical Association 202, 1023–1034. VA Cooperative Study Group on Antihypertensive Agents (1970). Effects of treatment on morbidity in hypertension. II. Results in patients with diastolic blood pressure averaging 90 through 114 mm Hg, Journal of the American Medical Association 213, 1143–1152. Wasson, J.H., Reda, D.J., Bruskewitz, R.C., et al. (1995). A comparison of transurethral surgery with watchful waiting for moderate symptoms of benign prostatic hyperplasia, New England Journal of Medicine 332, 75–79. Weinberger, M., Oddone, E.Z., Henderson, W.G., et al. (1996). Does increased access to primary care reduce hospital readmissions?, New England Journal of Medicine 334, 1441–1447.

WILLIAM G. HENDERSON, PHILIP W. LAVORI, PETER PEDUZZI, JOSEPH F. COLLINS, MIKE R. SATHER & JOHN R. FEUSSNER

COORDINATING COMMITTEE A Coordinating Committee is a committee that a sponsor may organize to coordinate the conduct of a multicenter trial. If a coordinating committee and/or coordinating investigator(s) are used in multicenter trials, their organization and/or selection is the responsibility of the sponsor.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

COORDINATING INVESTIGATOR The Coordinating Investigator is an investigator who is assigned the responsibility for the coordination of the investigators at different centers that participate in a multicenter trial. If a coordinating committee and/or coordinating investigator(s) are used in multicenter trials, their organization and/or selection is the responsibility of the sponsor.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

CORONARY DRUG PROJECT

2.2 Outcome Variables The primary outcome variable was all-cause mortality for the entire follow-up period, with secondary outcomes of CHD and CV death, recurrent definite nonfatal MI, the combination of CHD death or definite nonfatal MI, cerebral stroke, and others (2).

PAUL L. CANNER Maryland Medical Research Institute Baltimore, Maryland

By 1960, evidence had accrued that linked elevated blood lipid levels with increased incidence of coronary heart disease (CHD). At the same time, the pharmaceutical industry was developing drugs that were effective in reducing blood cholesterol in persons with hyperlipidemia. The time had come to assess whether reduction of lipid levels would be effective in the treatment and possible prevention of CHD. In November 1960, proceedings were started that culminated in a line item in the Federal budget for funding in 1966 of the Coronary Drug Project (CDP) (1). 1

2.3 Sample Size and Duration of Follow-up From March 1966 to October 1969, 53 Clinical Centers (in the United States and Puerto Rico) recruited a total of 8341 patients— about 1100 in each of the five drug groups and 2789 in the placebo group (2). The 2.5 to 1 ratio of patients in the placebo group relative to each drug group was designed to minimize the total sample size while achieving a specified power relative to each of the five drug–placebo comparisons (3,4). Patients were followed until July 1974 with clinic visits and examinations every 4 months for a minimum of 5 years, a maximum of 8.5 years, and a mean of 6.2 years per patient on their assigned treatment regimen (5).

OBJECTIVES

The CDP was a randomized, double-blind, placebo-controlled clinical trial of the efficacy and safety of five lipid-modifying agents in men with previous myocardial infarction (MI) (2). The patients were followed for a minimum of 5 years to determine whether pharmaceutical modification of blood lipids would lead to improved survival and a reduction in cardiovascular (CV) mortality and morbidity. A secondary objective was to identify baseline prognostic factors for CV mortality and morbidity in the placebo group of patients.

2.4 Eligibility Criteria A prospective participant in the CDP had to be a male aged 30 to 64 years with electrocardiogram-documented evidence of an MI that occurred not less than 3 months previously. Insulin-dependent diabetics and persons already on lipid-modifying medications at time of entry were excluded (2). 2.5 The CDP Aspirin Study

2 STUDY DESIGN AND METHODS 2.1 Treatments

Three CDP treatment regimens were discontinued early because of adverse effects. The patients in these three groups who were eligible and willing (1529 patients altogether) were randomized into a short (10 to 28 months, mean 22 months) double-blind trial of 972 mg/day aspirin and placebo. As with the CDP, the primary outcome variable for the CDP Aspirin Study was all-cause mortality (6).

CDP patients were randomly allocated to six treatment arms: mixed conjugated equine estrogens at two dosage levels (2.5 and 5.0 mg/day), clofibrate (1.8 g/day), dextrothyroxine (6.0 mg/day), nicotinic acid (or niacin, 3.0 g/day), and a lactose placebo (3.8 g/day) (2). These treatments were dispensed in identical-appearing capsules (9 per day at full dosage). Both the patients and the clinical staff were blinded as to each patient’s treatment group except that side effects of the estrogen and niacin treatments tended to unblind these treatment groups.

2.6 Administrative Structure In addition to 53 clinical centers, the CDP had a Coordinating Center (at the present usually

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

CORONARY DRUG PROJECT

called Data Coordinating Center; University of Maryland, Baltimore), a Central Laboratory (Centers for Disease Control, Atlanta), an ECG Reading Center (University of Minnesota, Minneapolis), and a Drug Procurement and Distribution Center (Public Health Service Supply Service Center, Perry Point, Maryland). Linkage of these centers with the National Heart, Lung, and Blood Institute (NHLBI) was provided through a Medical Liaison Officer, members of the Biometrics Research Branch, and a Health Scientist Administrator for medical, biostatistical, and budgetary matters, respectively. A Steering Committee composed of representatives of the Clinical Centers, Coordinating Center, Central Laboratory, ECG Reading Center, and NHLBI provided scientific direction for the study at the operational level (7). 2.7 Data and Safety Monitoring A Data and Safety Monitoring Committee (DSMC) composed of representatives from the Coordinating Center, Central Laboratory, ECG Reading Center, and NHLBI, the Chairman and Vice-Chairman of the Steering Committee, one outside statistician, and three clinicians who were not participants in the Clinical Centers met at least twice a year to review the treatment group data and make recommendations with regard to continuation or discontinuation of the study treatment groups. A Policy Board, which was composed of a clinical pharmacologist, a biostatistician, and three clinicians/cardiologists who had no direct involvement in the CDP, acted in a senior advisory capacity on policy matters throughout the duration of the study. The Policy Board reviewed and either ratified or disapproved recommendations from the DSMC (7,8). This model of both a DSMC and an independent Policy Board has evolved into the present-day Data and Safety Monitoring Board (DSMB) made up of scientists who are independent of the study. The members of the CDP DSMC and Policy Board wished not to be blinded as to identification of the study treatments in the data monitoring reports. They recognized that assimilating hundreds of pages of tables and graphs on a great variety of safety and efficacy outcomes in a short period was difficult

enough without their being blinded to treatment group identification as well. With treatment group blinding, significant patterns in the data with respect to treatment efficacy or safety might be missed easily. Furthermore, they realized that decisions concerning treatment efficacy are not symmetrical with those concerning treatment safety, with more evidence required for early stopping for efficacy than for safety. 3 RESULTS Three CDP treatment groups were terminated early, that is, before the scheduled end of the study; these groups were both estrogen groups and the dextrothyroxine group (9–11). The two remaining drug groups—clofibrate and niacin—and the placebo group were followed until the scheduled conclusion of the trial (5). 3.1 High Dose Estrogen In May 1970, a decision was reached to discontinue the 5.0 mg/day estrogen (ESG2) group because of an increased incidence of cardiovascular events. One major consideration in the deliberations over the ESG2 findings had to do with possible bias in diagnosing definite nonfatal MI and nonfatal thromboembolic events because of the treatment being unblinded in a large percentage of the patients because of feminizing side effects of the medication. Several special analyses were carried out to assess the possibility and extent of such bias. These analyses pertained to (1) incidence and duration of hospitalization for cardiac problems; (2) incidence of subsequent cardiovascular death in patients with definite nonfatal MI since entry; (3) incidence of several nonfatal cardiovascular events ranked in order of severity, counting only the single most serious event for a given patient; and (4) comparison of the centrally read electrocardiographic findings taken in connection with new MI events for the ESG2 and placebo groups. These analyses did not yield any evidence in support of the hypothesis of overdiagnosis of nonfatal MI in the ESG2 group (9,12). At its meeting of May 13, 1970, the DSMC reviewed the subgroup analyses shown in

CORONARY DRUG PROJECT

3

Table 1. Mortality and Morbidity in the High Dose Estrogen (ESG2) and Placebo Groups, Coronary Drug Project

Event

ESG2

Placebo

Risk groupa

n

%

n

%

z-value

All 1 2 All 1 2

1118 738 380 1022 684 338

8.1 5.1 13.9 6.2 6.7 5.0

2789 1831 958 2581 1689 892

6.9 6.1 8.5 3.2 2.9 3.7

1.33 −0.95 3.02 4.11 4.30 1.05

Total mortality Definite nonfatal MI

a Risk 1 = men with one MI without complications prior to entry into the study; Risk 2 = men with more than one previous MI or one MI with complications prior to entry. Source: Coronary Drug Project Research Group (12); reprinted from Controlled Clinical Trials © 1981, with permission from Elsevier.

Table 1. For this analysis, Risk 1 comprised men with one previous MI without complications prior to entry into the study and Risk 2 included men with either more than one previous MI or one MI with complications prior to entry. For the total group the z-value for the ESG2-placebo difference in total mortality was 1.33 and the corresponding z-value for definite nonfatal MI was 4.11. [A z-value is defined here as a drug-placebo difference in proportions of a given event, divided by the standard error of the difference; z-values of ±1.96 correspond to a conventional P-value of 0.05. However, given the multiple treatment groups, multiple endpoints (here, total mortality and definite nonfatal MI), and multiple reviews of the data during the course of the study, it was judged necessary to require much larger z-values than these to establish statistical significance (13,14)]. The DSMC agreed to discontinue the ESG2 treatment in Risk 2 patients only, but the Policy Board voted to discontinue the entire ESG2 treatment group (9,12). 3.2 Dextrothyroxine In October 1971, a decision was reached to discontinue the dextrothyroxine (DT4) treatment group in the CDP, based primarily on a higher mortality in the DT4 group compared with placebo. Although the z-value (1.88) for the difference did not achieve the conventional P < 0.05 level of statistical significance, and even less so when taking into account the five treatment versus placebo group comparisons, the DSMC members did not think that

conventional significance levels necessarily applied to adverse treatment effects. The deliberations that led to the decision for discontinuation focused largely on the question of whether the excess mortality was present consistently throughout the total group of DT4-treated patients or whether it was concentrated in certain subgroups (10). Table 2 gives the observed DT4 and placebo group findings for total mortality in subgroups defined by baseline risk categorization, history of angina pectoris, and ECG heart rate. Within each higher risk subgroup, there was a substantially higher mortality in the DT4 group than in the placebo group. Conversely, DT4 showed a somewhat lower mortality than placebo in the three lower risk subgroups. These subgroups that showed adverse effects of DT4 were identified following a survey of 48 baseline variables. Because no a priori hypotheses concerning DT4 effects in defined subgroups had been specified, the observed effects were treated as a posteriori findings in the evaluation of their statistical significance. The statistical analysis of these subgroup findings lay primarily in two directions (10). First, because the observed subgroup findings emerged from an analysis that involved 48 different baseline variables, it was desirable to determine whether the observed differences were any greater than might be expected by chance alone from evaluation of 48 variables. Computer simulations showed that the observed interactions with treatment for baseline heart rate and for history of

4

CORONARY DRUG PROJECT

Table 2. Percentage of Deaths in Selected Subgroups, Dextrothyroxine (DT4) and Placebo Groups, Coronary Drug Project Baseline Characteristic Risk Group,a 8/1/71 Risk 1 Risk 2 History of angina pectoris, 8/1/71 Negative Suspect/definite ECG heart rate, 8/1/71 ICC1 =

s2X s2X

+ s2e

.

95% Confidence Interval:
95% CI(ICC) = F=

 F/FL − 1 F/FU − 1 ; , with n + F/FU − 1 n + F/FL − 1

MSX , MSE MSX = mean square of X, MSE = mean square of e,

FU and FL = upper and lower points of the F-distribution,

method 1 - method 2

a−1 P(FL ≤ Fa(n−1) ≤ FU ) = 1 − α.

+ 2*SD

Mean

− 2*SD

Average of method 1 and method 2 Figure 3. Example for a Bland-Altman plot

3.2.2 McNemar’s Test. For the analysis of reliability in terms of hypothesis testing, McNemar’s test may be used. For dichotomous outcomes, McNemar’s test determines whether the probability of a positive test result is the same for two methods. McNemar’s test is based on the determination of the symmetry in the discordant pairs of observations (i.e., where the two methods provide different outcomes). The hypotheses and test statistic are shown in Table 10. In case

of clustered data (i.e., where several observations are obtained from the same subject like the detection of focal liver lesions in the same patient), the cluster-adjusted McNemar test (4, 5) should be used. For illustration, McNemar’s test is calculated for the ultrasound example (Table 3). Let X be reader 1 and Y reader 2, then PX = 34/169 = 20%, PY = 51/169 = 30%, and the SE(PX −PY ) = 0.0423. The test statistic then is χ 2 = (34 − 17 − 1)2 /51 = 5.02 and leads to

DIAGNOSTIC STUDIES

9

Table 8. Formulas for ICC2 and Associated Parameters (10) ICC2 by Mean Squares: Let BMS =

n 1  (xi − mi · p)2 be the mean square between methods · n mi i=1

and WMS =

n  xi · (mi − xi )2 1 · be the mean square within the methods n(m − 1) mi i=1

with mi = number of ratings on the ith patient with mean m =

n 1  mi , · n i=1

n 

m0 = m −

p=

(mi − m)2

i=1

n · (n − 1) · m

, xi = number of positive ratings on the ith patient,

n 1  · xi = mean proportion of positive ratings and nm i=1

n = number of patients, = > ICC2 =

BMS − WMS . BMS + (m0 − 1) · WMS

statistical significance as compared with the critical value of χ 1,0.05 2 = 3.841. 4

VALIDITY

The validity of a diagnostic procedure is substantial for its performance. It is defined as the accuracy or the grade of truthfulness of the test result, but can only be evaluated when the true disease status is known. The main question regarding the validity of a procedure is, therefore, whether it measures what it is intended to measure. Often, the observations for estimating the validity are on binary scale (i.e., with two outcomes) or dichotomized (i.e., reduced to two outcomes) to estimate validity parameters like sensitivity and specificity. When the observations are on ordinal scale (i.e., with more than two ordered outcomes like rating scores on 5-scale) or when the data are continuous (e.g., laboratory values of a PSA test as marker for prostate cancer), the area under the Receiver Operator Characteristic curve (ROC) and likelihood ratios are commonly used to analyze the validity of diagnostic

procedures. Most often, the primary efficacy objective in diagnostic studies is based on the validity (i.e., the diagnostic accuracy and the ability of the diagnostic procedure to discriminate the disease status from the nondisease status). The advantage of the new diagnostic procedure (i.e., the add-on value) is often based on an increased validity compared with some control. 4.1 Validity Parameters Basis for the definition of validity parameters is the 2 × 2-table defining true positive, false positive, false negative, and true negative outcomes (please refer to section ‘‘True Positives, True Negatives, False Positives, False Negatives’’). Then the prevalence of a disease is defined as the number of patients with a disease divided by all patients in the target population at a given and fixed point in time. The prevalence is called the prior or pretest probability for the proportion of patients with the disease. Validity parameters are sensitivity, specificity, and accuracy. The sensitivity is defined as the number of diseased patients

10

DIAGNOSTIC STUDIES Table 9. Formulas for Lin’s Concordance Coefficient and Associated Parameters (15) Lin’s Concordance Coefficient: rLin =

2sM1,M2 , s2M1 + s2M2 + (µM1 − µM2 )2 M1 = method 1, M2 = method 2, S2M1 of method M1, S2M2 of method M2, sM1,M2 = covariance of M1 and M2, µM1 = mean for method M1, µM2 = mean for method M2

Variance: Var(rLin ) =

1 n−2



 4 · r3Lin · (1 − rLin ) · u2 (1 − r2 ) · rLin · (1 − r2Lin ) 2 · r4Lin · u4 + , − r2 r r2

µM1 − µM2 r = Pearson’s correlation coefficient, u = √ sM1 · sM2

Table 10. Formulas for McNemar’s Test and Associated Parameters (10) McNemar’s Test: Hypotheses: Let M1 = method 1 and M2 = method 2, then H 0 : PM1 = PM2 vs. H 1 : PM1 = PM2 ⇔ H 0 : PM1 − PM2 = 0 vs. H 1 : PM1 − PM2 = 0, PM1 =

a+b , n

PM2 =

a+c , a, b, c, d and n according to Table 1. n

b−c = difference in proportions n √ b+c . with standard error SE(PM1 − PM2 ) = n

= > PM1 − PM2 =

Test Statistic:
χ12 =

|PM1 − PM2 | − 1/n SE(PM1 − PM2 )

2 =

(|b − c| − 1)2 with 1 degree of freedom. b+c

with positive test result divided by the overall number of patients with the disease. In statistical terms, the sensitivity can also be expressed as the probability for a positive test result under the condition that the disease is present. Therefore, the sensitivity is exclusively based on patients with the disease.

The specificity, on the other hand, is based on nondiseased subjects only. It is defined as the number of nondiseased patients with a negative test result divided by the overall number of patients without the disease. In statistical terms, the specificity is the probability of a negative test result under the

DIAGNOSTIC STUDIES

condition that the disease is absent. A combination of the sensitivity and specificity is the accuracy, where the information of both diseased patients and nondiseased patients is used. The accuracy is defined as the number of patients with either true positive or true negative test result divided by the overall number of patients under investigation. The disadvantage of the accuracy is its dependence on the prevalence of the disease: When the prevalence is low, the accuracy is mainly driven by the specificity (i.e., the number of true negative test results). When the prevalence is high, the accuracy is mainly influenced by the sensitivity (i.e., the number of true positive test results). As a consequence, a low specificity at a high prevalence and a low sensitivity at a low prevalence may be masked and missed. Other possibilities of combining sensitivity and specificity are the prevalence-independent receiver operating characteristic curve and the likelihood ratio, which are described later. Other validity parameters are the positive and negative predictive value. The positive predictive value (PPV) is the number of true positive test results divided by the overall number of positive test results, both true results and false results. It expresses the probability that a positive test result is correct, which is equivalent to the post-test probability for the disease when a positive test result is found. The negative predictive value (NPV) is the number of correct negative test results divided by the number of negative test results overall, which also is true only in cases where the prevalence in the sample is equal to the prevalence in the target population. In this case, the NPV is the probability for a negative test result being correct. The NPV is equivalent to the post-test probability that a negative test result is correct. Both PPV and NPV are measurements of how well the diagnostic procedure performs and can also be expressed in terms of sensitivity and specificity. In Zhou et al. (18), further descriptions can be found for situations in which the prevalences of the sample and the target population differ. In Table 11 (19), the formulas of all validation parameters and the associated parameters like the prevalence are given for a 2 × 2 table.

11

To demonstrate the calculation of the validation parameters, assume a study with 100 patients with suspected prostate cancer to test the performance of a new laboratory test to discriminate diseased and nondiseased patients. The test measures the PSA level in blood. A cut-off value is chosen to discriminate the diseased and nondiseased patients. Biopsy is used as standard of reference to obtain information on the true disease status. In Table 12, hypothetical results of such a study are in terms of a 2 × 2 table. The validation parameters and associated parameters can be calculated to investigate the performance of the new laboratory test on PSA levels. The results are: Se = 55/70 = 78.6%, Sp = 20/30 = 66.7%, Acc = 75/100 = 75%, PPV = 55/65 = 84.6%, and NPV = 20/35 = 57.1%. For further reading, see Panzer (20) as an introductory text; Zhou et al. (18) for advanced readers regarding validation parameters, their variances, and confidence intervals; and Agresti (21) for categorical data in general. 4.2 Analysis of Validity 4.2.1 Likelihood Ratio. As a measurement of the discrimination of a diagnostic procedure, the likelihood ratio combines information on the sensitivity and specificity simultaneously. For dichotomous data, the likelihood ratio for diseased patients (LR+) is the probability for a positive test result in diseased patients (i.e., the sensitivity) compared with nondiseased patients (i.e., 1-specificity). Analogously, the likelihood ratio for nondiseased patients is the probability for a negative test result in diseased patients compared with the patients without the disease. In Table 13, the likelihood ratios are given for dichotomous but also ordinal data with more than two categories along with the 95% confidence interval. With a likelihood ratio of 1, the diagnostic procedure adds no information, with increasing LR+ and decreasing LR−, the ability of the diagnostic procedure to discriminate between diseased and nondiseased patients increases. With the data from the PSA example (Table 12), the likelihood ratios for the PSA test would lead to LR+ = 0.786/(1 − 0.667) = 2.360 with 95% CI = [1.402; 3.975] and

12

DIAGNOSTIC STUDIES

LR− = (1 − 0.786)/0.667 = 0.321 with 95% CI = [0.192; 0.538]. In Knottnerus (3), more introductory explanations can be found.

4.2.2 ROC. A standard approach to evaluate the sensitivity and specificity of diagnostic procedures simultaneously is the Receiver Operating Characteristic (ROC) curve. For ordinal and continuous outcomes, a curve

is fitted to describe the inherent tradeoff between sensitivity and specificity of a diagnostic procedure. In difference to the accuracy, the ROC is independent from the prevalence of the disease. Each point on the ROC curve is associated with a specific diagnostic criterion (i.e., a specific cut-off value to classify the patient with regard to the presence/absence of the disease). The ROC was

Table 11. Formulas for the Validation Parameters and Associated Parameters for a 2 × 2 Table (19) True positives, true negatives, false positives, false negatives: see section ‘‘True Positives, False Positives, True Negatives, False Negatives’’. Prevalence: Pr =

TP + FN N

Sensitivity: Se = P(T + | D+) =  Standard Error: SE(Se) =

TP , where T + = test positive, D + = disease present. TP + FN Se · (1 − Se) (TP + FN)

95% Confidence Interval: 95% CI(Se) = Se ± 1.96 · SE(Se) Specificity: Sp = P(T − | D−) =  Standard Error: SE(Sp) =

TN , where T- = test negative, D- = disease absent. FP + TN Sp · (1 − Sp) (TN + FP)

95% Confidence Interval: 95% CI(Sp) = Sp ± 1.96 · SE(Sp) Accuracy: Acc =

TP + TN TP + FP + FN + TN

Positive Predictive Value: PPV = P(D + | T+) =

TP Pr ·Se = TP + FP Pr ·Se + (1 − Pr) · (1 − Sp)

Negative Predictive Value: NPV = P(D − | T−) =

TN (1 − Pr) · Sp = TN + FN (1 − Pr) · Sp + Pr ·(1 − Se)

Table 12. An 2 × 2 Table for an Hypothetical Clinical Study in Patients with Suspected Prostate Cancer

Test result T + (test positive) T − (test negative)

Standard of reference D+ D− (disease present) (disease absent) 55 15 70

10 20 30

65 35 100

DIAGNOSTIC STUDIES

13

Table 13. Formulas for the Likelihood Ratio and Associated Parameters (3) Likelihood Ratio for data with X categories (X > 2): likelihood ratio for outcome X: LRX =

P(X| D+) , P(X| D−)

P(X— D+) = probability for outcome X in diseased patients, P(X— D−) = probability for outcome X in non-diseased patients. Likelihood Ratios for dichotomous data: likelihood ratio for diseased patients: LR+ =

P(T + | D+) Se = , P(T + | D−) 1 − Sp

likelihood ratio for non-diseased patients: LR− =

1 − Se P(T − | D+) = , P(T − | D−) Sp

P(T + — D +) = probability for a positive test result in diseased patients, P(T + — D −) = probability for a positive test result in non-diseased patients, P(T − — D +) = probability for a negative test result in diseased patients, P(T − — D −) = probability for a negative test result in non-diseased patients, Se = sensitivity, Sp = specificity. 95% Confidence Interval for likelihood ratio:  95%CI(LRX ) = exp ln

 p1 ± 1.96 · p0

 1 − p1 1 − p0 + , p1 · n1 p0 · n0

p1 = P(X—D +) based on sample size n1 of diseased patients, p0 = P(X—D −) based on sample size n0 of non-diseased patients.

developed in the signal detection theory during World War II for the analysis of radar images and was adopted to medical statistics in the 1960s and 1970s for the analysis and interpretation of diagnostic procedures where signals on images or screens were to be evaluated. For the estimation of a ROC curve, first the sample is divided into the groups truly diseased and truly not diseased patients. These groups have different distributions of the outcomes of the diagnostic procedure (see Fig. 4). Assuming that higher test results indicate the disease, the test procedure is set to ‘‘positive’’ when the outcome is equal or

higher than the cut-off value and to ‘‘negative’’ when the result is lower than the cut-off value. Each possible cut-off value c represents an inherent tradeoff between sensitivity and specificity and results in a 2 × 2 table. For the ROC analysis, the outcome of a diagnostic procedure has to be at least ordinal (see Table 14). In the example, the image quality is measured on a score from 1 to 10, the cut-off value dichotomizes the outcome so that, for example, scores 1 to 5 would lead to the outcome ‘‘poor image’’ and scores 6 to 10 would lead to the outcome ‘‘good image.’’ The discriminatory accuracy of a diagnostic procedure can be represented graphically

14

DIAGNOSTIC STUDIES

Cut-off value c Test negative

Test positive

TN

TP

not diseased patients

diseased patients FN

FP

score of diagnostic procedure

Figure 4. Distribution of patients with and without disease with cut-off value Table 14. Outcome Categories and True Disease Status in a 2 × C Table Truth Test result categories

D + (D = 1): disease present

D − (D = 0): disease absent

n11 n12 ... n1K n1

n01 n02 ... n0K n0

1 2 ... K

Sensitivity

1.0

0 0

1.0 1-Specificity

Figure 5. Example of a ROC curve

by a ROC curve (see Fig. 5), where the sensitivity (i.e., the rate of true positive test results) is displayed on the vertical axis and 1-specificity (i.e., the rate of false positive results) on the horizontal axis. If the ROC hits the upper-left corner, an optimal differentiation between diseased and not diseased patients (i.e., the true positive rate = 1 and

n.1 n.2 ... n.K N

the false negative rate = 0) is achieved. At the bisector of the graph, the discrimination of the diagnostic procedure is equal to chance. For c = + ∞ (plus infinity), all patients are set as ‘‘not diseased’’; for c = − ∞ (minus infinity), all patients are set as ‘‘diseased.’’ The advantages of the ROC analysis are that the curve is invariant with regard to the scale of the diagnostic procedure itself and is invariant to monotonic transformations. These advantages enable scientists to compare diagnostic procedures with different scales based on ROC curves. ROC curves are, in addition, independent from the prevalence of the disease. The likelihood ratio can be extracted from the ROC curve as the slope of the tangent line at a given cut-off value. As a measurement of the validity, the area under the ROC curve (AUC) summarizes the sensitivity and specificity over the whole range of cut-off values. An AUC of 0.5 indicates that the procedure is equal to chance alone, whereas a value of 1 indicates a perfect discrimination of ‘‘diseased’’ and ‘‘not diseased’’ patients. The AUC is limited to the interval [0, 1]. In practice, the AUC should be at least 0.5. Values below 0.5 lead to a diagnostic procedure that would be worse than chance

DIAGNOSTIC STUDIES

1.0

15

Sensitivity

Sensitivity

1.0

AUC = 100%

AUC = 50%

0

0 0

1.0

1.0

0

1-Specificity

1-Specificity

Figure 6. Examples of AUCs

alone. In general, the AUC can be interpreted as the average sensitivity across all values of the specificity and vice versa. For the assessment of a diagnostic procedure, in addition to the AUC, the ROC curve should also be provided and taken into consideration to check whether the curves of the test and control procedure cross or whether one procedure is consistently superior to the other (see Fig. 6). For further reading, see Dorfman (22), Hanley and McNeil (23), and especially Zweig and Campbell (24) as an introductory text and Zhou et al. (18) for advanced readers. REFERENCES 1. CPMP, CPMP points to consider: points to consider on the evaluation of diagnostic agents, Committee for Proprietary Medicinal Products (CPMP) of the European Agency for the Evaluation of Medicinal Products (EMEA), CPMP/EWP/1119/98, 2001. 2. FDA Draft Guideline, Developing Medical Imaging Drugs and Biological Products. Washington, DC: FDA, 2000 3. A. J. Knottnerus, The Evidence Base of Clinical Diagnosis. London: BMJ Books, 2002. 4. M. Eliasziw and A. Donner, Application of the McNemar test to non-independent matched pair data. Stat. Med. 1991; 10: 1981–1991. 5. M. M. Shoukri and V. Edge, Statistical Methods for the Health Sciences. Boca Raton, FL: CRC Press, 1996. 6. A. Donner, Statistical Methods in Ophthalmology: An Adjusted Chi-Square Approach. Biometrics, 1989; 45(2): 605–611. 7. Fleiss1981.

8. J. M. Bland and D. G. Altman, Measuring agreement in method comparison studies. Stat. Meth. Med. Res. 1999; 8: 135–160. 9. M. M. Shoukri, Measures of Interobserver Agreement. Boca Raton, FL: CRC Press, 2003. 10. Fleiss1999. 11. J. Cohen, A coefficient of agreement for nominal scales. Educ. Psycholog. Measur. 1960; 20: 37–46. 12. J. Landis and G. Koch, The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159–174. 13. Searle1992. 14. L. G. Portney and M. P. Watkins, Foundations of Clinical Research: Applications to Practice. Norwalk, CT: Appleton & Lange, 1993. 15. L. I. Lin, A concordance correlation coefficient to evaluate reproducibility. Biometrics 1989; 45: 255–268. 16. J. M. Bland and D. G. Altman, Statistical method for assessing agreement between two methods of clinical measurement. Lancet 1986; i: 307–310. 17. J. J. Bartko, General methodology II. Measures of agreement: a single procedure. Stat. Med. 1994; 13: 737–745. 18. X-H. Zhou, N. Obuchowski, and D. K. McClish, Statistical Methods in Diagnostic Medicine. New York: John Wiley and Sons, 2002. 19. L. Edler and C. Ittrich, Biostatistical methods for the validation of alternative methods for in vitro toxicity testing. ATLA 2003; 31(Suppl 1): S5–S41. 20. R. J. Panzer, E. R. Black, and P. F. Griner, Diagnostic Strategies for Common Medical Problems. Philadelphia, PA: American College of Physicians, 1991.

16

DIAGNOSTIC STUDIES

21. A. Agresti, An Introduction to Categorical Data Analysis. New York: John Wiley & Sons, 1996. 22. D. D. Dorfman, Maximum-likelihood estimation of parameters of signal-detection theory and determination of confidence intervals rating method data. J. Math. Psychol. 1969; 6: 487–496. 23. J. A. Hanley and B. J. McNeil, The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982; 143: 29–36. 24. M. H. Zweig and G. Campbell, Receiveroperating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clin. Chem. 1993; 39/4: 561–577.

FURTHER READING P. Armitage and G. Berry, Statistical Methods in Medical Research. 3rd ed. Oxford: Blackwell Science, 1994.

DISCRIMINANT ANALYSIS

Another important task in clinical medicine is the prediction of a diagnosis for a certain patient given a set of p variables. In medical practice, situations often exist where the definitive diagnosis can only be made after a large number of clinical investigations, not seldom only after autopsy. What is needed here is a statistically-supported decision, taking into consideration available features x = (x1 , x2 , . . . , xp ) such as clinical or laboratory-type observations that can be accessed without too much inconvenience to the patient and without high cost. In a first (training) step of constructing a rule of prediction, one knows the class membership of patients (even from autopsy) and tries to ‘‘repeat’’ the diagnostic process by means of the trained prediction rule using only the available features x mentioned. If successful, further patients with unknown class membership can then be allocated (diagnosed) on the basis of the rule obtained using only the p available features x1 , . . . , xp (prediction step).

KLAUS-D. WERNECKE The Humboldt University of Berlin Charit´e —University Medicine Berlin Berlin, Germany

1

INTRODUCTION

Classification problems occur in a wide range of medical applications in which n objects (e.g., patients) are to be assigned to K certain classes (e.g., diagnoses, tumor classes, lesion types) on the basis of p observations x = (x1 , x2 , . . . , xp ) (e.g., symptoms, complaints, intraocular pressure, blood pressure, X-ray, histology, grading) on these patients. As a result of a set of p ≥ 1 observations (features or variables) for every patient, the corresponding statistical methods are designated as multivariate ones. Given a set of data from n patients in p variables, one task is the identification of groups, classes, or clusters of patients that are ‘‘similar’’ (mostly in terms of a distance) with respect to the variables x1 , . . . , xp within the classes and different between the classes. The question here is to give a description of (diagnostic) patient groups on the basis of the observed variables x1 , . . . , xp .

Example 2: Diagnosis of neuroborreliosis burgdorferi The correct diagnosis of neuroborreliosis burgdorferi is known to be extremely difficult in children. In the university hospital of Graz, Austria, various clinical outcomes were examined for their qualification to predict this diagnosis. Two patient groups with a known diagnosis served as a starting point of the investigations:

Example 1: Clustering with parameters from blood tests The features x1 : pH, x2 : bicarbonate, x3 : pO2 , x4 : pCO2 , x5 : lactate, x6 : glucose, x7 : sodium, x8 : potassium, x9 : chloride, x10 : carbamide, x11 : creatinine, x12 : content of serum iron, and x13 : content of alkaline phosphatase should be used for a description of diagnostic classes such as hepatitis, liver cirrhosis, obstructive jaundice, or healthy persons (1). Figure 1 shows a scatter-plot with 20 (numbered) patients using the two variables, x12 and x13 , respectively. Groups of patients, ‘‘similar’’ in x12 , x13 , and corresponding to the diagnoses in question, are bordered (H: hepatitis, LC: liver cirrhosis, OI: obstructive jaundice, HP: healthy persons).

• Group 1 (LNB): 20 patients suffering

from liquor-manifest neuroborreliosis (positive B. burgdorferi—CSF-titer— definitive neuroborreliosis) • Group 2 (NNB): 41 patients without neuroborreliosis (control group) (CSF: cerebro spinal fluid). The following features were examined: • Cephalea, Stiff Neck, Paresis, CSF cells, CSFigg, CSFalb, CSFprot, SBigg (SB: serum binding, igg: immune globulin g, alb: albumin, prot: protein).

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

DISCRIMINANT ANALYSIS

4.0 17

3.5

8

3.0

Content of serum iron [mg/l]

2.5

LC

5 18

2.0 1.5

H

16 7 10 14

3 HP

20

6 1.0 .5

OI

2

4 9

11

19 1 15

13

12

0.0 0 1 2 3 Content of alkaline phosphatase [U/I]

It was unknown, to which extend certain features and combinations of features are important for the diagnosis and which variables may be omitted for further diagnostic purposes (2). Following the remarks above, two principal tasks exist in classification (3): clustering (also known as unsupervised learning, classification, cluster analysis, class discovery, and unsupervised pattern recognition) and discrimination (also designated as supervised learning, classification, discriminant analysis, class prediction, and supervised pattern recognition). In the first case, the classes are unknown a priori and need to be discovered from the given observations. Then, the procedure consists in the estimation of the number K of classes (or clusters) and the determination of the classes in connection with the assignment of the given patients to these classes. This type of problem will not be considered in the following. However, in the second case, the existence of K distinct populations, groups, or classes is known a priori and a random sample has been drawn from each of these classes (for reasons of simplification, it is restricted to K = 2 classes). The given data are used to construct a classification rule (or classifier), which allows an allocation or prediction to

4

5

Figure 1. Visualization of the patients in a scatter-plot using blood tests

the predefined classes on the basis of the given observations x1j = (x1j1 , x1j2 , . . . , x1jp ), j = 1, . . . , n1 from class C1 , and x2j = (x2j1 , x2j2 , . . . , x2jp ), j = 1, . . . , n2 from class C2 , respectively. In this procedure, the given observations (with known class membership)— the socalled training sample {x} = {x11 , . . . , x2n2 }— serve for the construction of the classifier (training step) that forms the basis for a later allocation of patients with unknown class membership (prediction step). A close relationship exists between the two classification problems. On the one hand, the classes discovered from clustering can be used as a starting point for discrimination. On the other hand, in discriminant analysis, an evaluation of the homogeneity of the predefined classes can be achieved by a clustering of the data. 1.1 Exploratory Data Analysis Before applying a statistical procedure like discriminant analysis, it is always important to check the data with respect to the distributions of the given variables x1 , . . . , xp or to look for outliers (particularly when using procedures with serious preconditions on the data, for example, see Section 4).

DISCRIMINANT ANALYSIS

3

3000 NNB LNB

2000

1000

Figure 2. Data on SBigg and CSFprot for patients from LNB and NNB, respectively

Sbigg [mg/l]

0

−1000

0 20 40 CSFprot [mg/l]

Example 2 (continuation): Diagnosis of neuroborreliosis burgdorferi The next step should be a visualization of the data in order to judge whether a discrimination is worthwhile, which can be done by means of two-dimensional scatter-plots using some appropriately selected variables from the given set of features. Figure 2 shows the scatter-plot of the observations in the two classes LNB and NNB with respect to SBigg and CSFprot. In case of many given variables, a scatter-plot matrix can be helpful, but it may be principally difficult to select features of possible importance for discrimination (the general problem of feature selection will be considered in Section 4.2). One possibility of visualizing the data as a test of their suitability for discrimination is a transformation [under the criterion of minimal loss of information such as principal component analysis; see, among others, Ripley, (4)] of the given observation vectors x = (x1 , x2 , . . . , xp ) into vectors with lower dimension and their visualization in a two- or three-dimensional space. Figure 3 shows a scatter-plot with the first two principal components of the neuroborreliosis data. Third, one might test for a difference between the two groups of patients by a multivariate analysis of variance (MANOVA)

60

80

100

120

140

160

using the Lawley-Hotelling test statistics [e.g., Morrison (5)]. Whereas discriminant analysis allows an individual allocation for every single patient to one of the given classes, the MANOVA proves the hypothesis of differences in the middle between the classes. The analysis results in a significant difference between the classes LNB and NNB with
ni respect to the mean vectors xi = ( n1 j=1 xij1 , i
ni . . . , n1 x ); i = 1, 2. Although this outj=1 ijp i come may be interesting regarding the problem of the neuroborreliosis diagnostics from a scientific point of view, an affected patient is certainly not much interested in a statement for a group of patients than in the definitive diagnosis for himself or herself. In other words, one is looking for a decision rule that allows for an individual allocation of a given patient to exactly one of the classes (diagnoses) in question.

1.2 Fisher’s Approach R. A. Fisher (6) proposed a procedure for finding a linear combination of the variables x1 , x2 , . . . , xp that gives a separation ‘‘as good as possible’’ between the two classes C1 , C2 . Moreover, Fisher (6) suggested that such a

4

DISCRIMINANT ANALYSIS

3 2

Second Principal Component

1 0 −1 −2

NNB LNB

−3 −4 −2

−1

0

1

2

3

First Principal Component

linear combination can be obtained by a multiple regression approach zj = λ0 +

p 

λl xjl + j ,

j = 1, . . . , n,

l=1

n = n1 + n2 where  j is the jth error term that denotes the difference between the actual observation zj and the average value of the response z. λ0 , and λ1 , . . . , λp are unknown parameters that have to be estimated. Fisher (6) took zj = (−1)i ni /n if xj ∈ Ci , i = 1, 2 as indicators for the group membership. With the least-squares estimations of λ0 , and λ1 , . . . , λp from the equation above, one can calculate a corresponding predicted value zˆ j for every observation vecz1 = tor x
j = (xj1 , . . . , xjp ), which means that
1/n1 j zˆ j for all xj ∈C1 and z2 = 1/n2 j zˆ j for all xj ∈C2 , respectively. A plausible rule for an (individual) prediction of a given observation xj to one of the two given classes is then the allocation to that class Cm with the minimal distance to the corresponding mean: zˆ j − zm = mini [ˆzj − zi ](i = 1, 2).

4

Figure 3. Dimension reduction with principal component analysis. The first two principal components for patients from the LNB and NNB group are shown

Table 1. Classification Table for the Neuroborreliosis Data Using Multiple Regression Resubsitution Observed (true) class LNB NNB Total

Predicted class LNB NNB 14 0 14

6 41 47

Total 20 41 61

Example 2 (continuation): Diagnosis of neuroborreliosis burgdorferi Using this approach, the following results are obtained for the neuroborreliosis data, arranged in a so-called classification table (Table 1), in which for every observation the prediction has been compared with the (observed) true class of origin (that means a resubstitution of every observation). Thus, altogether 6 wrong predictions were made, which stands for an error rate for this decision rule of 9.84%.

2 ALLOCATION RULES AND THEIR ERROR RATE As already mentioned, the example is restricted to K = 2 classes to simplify matters.

DISCRIMINANT ANALYSIS

Nevertheless, all the methods to be described can be generalized to K > 2 easily. In the neuroborreliosis example, a decision rule is sought that allows for an (individual) prediction or allocation of a given patient to one of the two diagnoses in question. Generally, R(x) is defined as a prediction rule or an allocation rule on the basis of p observations x = (x1 , x2 , . . . , xp ) of a patient, where R(x) = i implies that the patient is to be assigned to the ith class Ci (i = 1, 2). The rule R(x) divides the features space into two disjunctive regions R1 and R2 , and, if x falls in the region Ri , the patient is allocated to class Ci . The probability of a wrong allocation from class C1 to class C2 for any patient (from class C1 ) is given by P12 (R(x)) = P(R(x) = 2|x ∈ C1 ) (which is the probability of an allocation to class C2 provided the patient belongs to class C1 ). The probability of a wrong allocation from class C2 to class C1 is defined analogously by P21 (R(x)) = P(R(x) = 1|x ∈ C2 ). Supposing patients in class Ci have feature vectors x = (x1 , . . . , xp ) with a class-specific p-dimensional distribution F i (x) and a corresponding density function f i (x)(i = 1, 2), then P12 (R(x)) and P21 (R(x)) can be expressed by  P12 (R(x)) = f1 (x)dx and P21 (R(x)) R2

 =

f2 (x)dx

respectively.

R1

As Fig. 4 illustrates for p = 1 and normal distributions F i (x) with densities f i (x),  integral R f1 (x)dx, for example, defines the 2 probability of patients from class 1 to have observed values x∈R2 (i.e., within the region R2 ), and it is nothing more than the area under the curve f 1 (x) over R2 (hatched areas in Fig. 4). The overall probability of misallocations, also known as (overall) misallocation rate, or (overall) error rate (7), can then be calculated to E(R(x)) = P12 (R(x)) + P21 (R(x))   = f1 (x)dx + f2 (x)dx R2

R1

5

Figure 4 shows the overall probability of misallocations as the sum of the two hatched areas in case of two normal distributions f i (x)(i = 1, 2) and p = 1. 2.1 Estimations of Error Rates The practical use of a discriminator depends on the quality by which newly appearing patients of unknown class membership can be allocated on the basis of the allocation rule R(x). Suppose a training sample {x} such as introduced in Section 1, estimate the overall probability of misallocations or the overall error rate connected with R(x). Among a lot of proposals for estimating that error rate [for a review, see Ambroise and McLachlan (8)] the leaving-one-out method (9) and the so-called π -error estimation (10), are frequently used (Fig. 5). In the leaving-one-out procedure, the available data are repeatedly reduced by one patient. The allocation rule is formed from the remaining subset and then assessed to the replaced patient. As a result of its known class membership, one can decide for a wrong or correct allocation. The procedure estimates the so-called true error rate, namely, the error rate with an allocation rule constructed on the basis of a training sample and referring to an individual patient not belonging to the training set (in the later application of the classifier, referring to patients with unknown class membership). The training sample {x} = {x11 , . . . , x2n2 }, which is customarily ordered by classes, has to be randomly mixed before starting the algorithm in order to secure a representative sample {x}rem . The leave-one-out error estimation and the wellknown resubstitution method (discriminator is trained and tested by the whole training sample, such as in Section 1.2 for the neuroborreliosis data) will be almost identical if the sample size n is large enough (relative to the number of features). Generally, the resubstitution method proves to be too small, thus it is overly optimistic (11). Another rather often used method of estimating the error rate is the bootstrap method (12). Again, one generates new samples (bootstrap samples) by drawing randomly one (or more) observation(s) from the training sample {x} and repeating this sampling several times (drawing with replacement). The error

6

DISCRIMINANT ANALYSIS

class 1 [f1(x )]

class 2 [f2(x )]

x R2

R1

wrongly all. (from 1 into 2) R2 f1(x ) dx

wrongly all. (from 2 into 1) R1 f2(x ) dx

Figure 4. Misallocation in case of two normal distributions f 1 (x) and f2 (x)(p = 1)

rate is then calculated on the basis of the bootstrap samples. 2.2 Error Rates in Diagnostic Tests The results of the error estimation will be summarized in a classification table. Table 2 shows the corresponding quantities for diagnostic tests, where C1 denotes the population of diseased, say D+, and C2 the population of non-diseased, say D−. Prediction to class C1 means a positive test result, say T+, and prediction to class C2 a negative test result, say T−. h11 , h12 , h21 , and h22 are the obtained frequencies of allocation, tp [tn] denotes the frequency of true positive [negative] decisions, and fp [fn] the frequency of false positive [negative] decisions. The probability P(T + |D+) of true positive decisions (prediction to D+ for patients from D+) is known as sensitivity; the probability P(T − |D−) of true negative (prediction to D− for patients from D−) is known

as specificity. For clinical practice, it is even more important to know what the probability of a true positive decision is, referring to all positive decisions (i.e., the probability of membership to D+ provided that the patient has been allocated to D+). That is called positive predictive value (PPV) with probability P(D + |T+). Analogously, the probability of membership to D−, provided that the patient has been allocated to D−, is denoted by P(D − |T−) [negative predictive value (NPV)]. The calculation of the above-mentioned statistical quantities in diagnostic tests are summarized in Table 3. Another calculation for the predictive values goes back to a very famous mathematical theorem, already established in 1763 by Thomas Bayes (13)—the so-called Bayes theorem. The Bayes theorem combines prior assessments (pre-test probabilities) with the eventual test results to obtain a posterior

Table 2. Classification Table in Diagnostic Tests True class (disease) D+

D− Total

Prediction (test result) T+ T− h11 (tp) sensitivity pos. pred. value h21 (fp) 1-spec. 1-PPV h11 + h21

h12 (fn) 1-sensit. 1-NPV h22 (tn) specificity neg. pred. value h12 + h22

Total h11 + h12

h21 + h22 n = h11 + h12 + h21 + h22

DISCRIMINANT ANALYSIS

7

k=1 Remove a subset {x}k = {xk1, ..., xkm} from {x} {x}rem = {x} − {x}k k=k+1

Train the classifier on {x}rem Test the classifier with {x}k pk =

m

j =1

pkj with pkj =

n

0, if xkj correctly classified 1, if xkj wrongly classified

k < n/m ?

π− error−estimation: E(p) =

y

1 n

n/m k =1

Table 3. Statistical Quantities in Diagnostic Tests Probability

Description

P(T + |D+)

Sensitivity

P(T − |D−)

Specificity

P(D + |T+)

Positive predictive value PPV

tp tp + fn tn tn + fp tp tp + fp

P(D − |T−)

Negative predictive value NPV

tn tn + fn

Estimation

assessment of the diagnosis (post-test probability). Taking the pre-test probabilities P(D + ) as prevalence of the disease, and P(D − ) as probability for no disease, positive predictive value P(D + |T+) can be calculated according to Bayes theorem to: P(D + |T+) = =

Figure 5. π -error-estimation (m = 1: leaving-one-out method)

pk

Example 2 (continuation): Diagnosis of neuroborreliosis burgdorferi For reasons of simplification, only the resubsitution method is applied for estimating the error rate in Section 1.2. To complete the calculations, the results are now presented using Fisher’s approach as allocation rule and the leaving-one-out method for error estimation, supplemented with the statistical quantities mentioned above (Table 4). The overall error rate of 16.39% is connected with a sensitivity of only 60.00% but a high specificity of 95.12% (i.e., the prediction of patients from class NNB is better than those from LNB). Positive and negative predictive values are sufficiently high (85.71% and 82.98%, respectively), referring to the prevalence from the given sample (this prevalence is not necessarily representative for the common population).

P(T + |D+)P(D+) P(T+) P(T + |D+)P(D+) P(T + |D+)P(D+)+ P(T + |D−)P(D−)

(P(T + |D+): sensitivity, P(T + |D−): 1-specificity), which shows the dependence of P(D + |T+) on the prevalence. Analogously, one calculates NPV.

3

BAYES ALLOCATION RULES

The Bayes theorem, mentioned in the previous section, based the (posterior) probability of belonging to a certain class on a prior estimate of that probability. The prior probability that a patient x belongs to class C1 may be denoted by π 1 = P(x∈C1 ) (e.g., the patient

8

DISCRIMINANT ANALYSIS

Table 4. Classification Table for the Neuroborreliosis Data Row [%]: sensitivity and specificity; Col. [%]: positive predictive value and negative predictive value leave-one-out Row [%] Col. [%] True class LNB

NNB Total

Fisher’s approach

3.1 Prediction A Bayes rule RI (x) is defined by RI (x) = m,

if

πm fm (x) ≥ πi fi (x) or

P(Cm |x) = max P(Ci |x); Predicted to class LNB NNB 12 60.00% 85.71% 2 4.88% 14.29% 14

8 40.00% 17.02% 39 95.12% 82.98% 47

i

Total 20

41

61

i = 1, 2

after which a patient will be assigned to that class Cm with the maximum posterior probability P(Cm | x). This rule is also called prediction or identification, respectively. Figure 6 illustrates the identification rule in case of two normal distributions and equal priors π 1 = π 2. 3.2 Action-Oriented Discrimination

has the disease, known as prior probability of the disease or prevalence). Analogously, membership to class C2 is assigned to π 2 = P(x∈C2 ) (e.g., prior probability of no disease), supposing π 1 + π 2 = 1. Furthermore, it is known a priori the distribution function F i (x) of feature vectors (x1 , . . . , xp ) from class Ci and their corresponding densities f i (x)(i = 1, 2). The posterior probability that a patient with unknown class membership and given observation vector x belongs to class Ci is given by

The allocation rule RI (x) assesses every misallocation with the same probability 1 − P(Ci | x); i = 1, 2 (i.e., equally for all classes), which may be disadvantageous in medical applications, where certain diagnosis have to be predicted with particular safety. Therefore, losses or costs lij of allocation are introduced when an patient from Ci is allocated to Cj , with lij = 0 for i = j (i.e., zero costs for a correct prediction). A Bayes’ rule RA (x), given a patient x, is then defined as RA (x) = m,

= min[CP1 (R(x)), CP2 (R(x))] with

P(patient ∈ Ci |x) = P(Ci |x) = πi fi (x)/[π1 f1 (x) + π2 f2 (x)];

i = 1, 2

according to Bayes’ theorem (13). In other words, given a patient with vector of observations x and unknown class membership, are asks for the (posterior) probability that the patient belongs to class Ci , given the (prior) probabilities of class membership π i , and the class-specific probability functions P( x| x ∈ Ci ) = fi ( x) of class Ci (i = 1, 2). In decision theory terms, any specific allocation rule that minimizes the overall probability of misallocations or the so-called optimal error rate (14) for known π i and f i (x) is said to be a Bayes rule.

if CPm (R(x))

CP1 (R(x)) = π2 l21 f2 (x) CP2 (R(x)) = π1 l12 f1 (x)

after which a patient will be assigned to that class Cm with the minimum misallocation loss CPm (R(x)), conditional on x. This rule is also called the action-oriented classification rule (15). It can easily be seen that the rule RI (x) follows immediately from RA (x) for equal losses l12 = l21 . As a result of the rather arbitrary nature of assigning losses of misallocation in practice, they are often taken as equal. According to McLachlan (11) this assignment is not as arbitrary as it may appear at first sight. For Bayes’ rule, only the ratio of l12 and l21 is relevant; therefore, the losses can be scaled by π 1 l12 + π 2 l21 . For an example, given two classes where C1 are patients suffering from

DISCRIMINANT ANALYSIS

class 2 [f2(x )]

Figure 6. Allocation according to identification for equal priors π1 = π2

9

class 1 [f1(x )]

x x ⇒ class 2

x ⇒ class 1

[f2(x ) ≥ f1(x )]

[f1(x ) ≥ f2(x )]

a rare disease and C2 are not, then, although π 1 and π 2 are substantially different, the cost of misallocating a patient from C1 may be much greater than the cost of misclassifying a healthy individual. Consequently, π 1 l12 and π 2 l21 may be comparable and the assumption of equal priors with unit costs of misallocation is not very far from real situations. 3.3 Maximum-Likelihood Discrimination Rules In the real world, class priors π i and class conditional densities f i (x) are unknown. Then, the frequentist analogue of Bayes’ rule is the maximum likelihood discriminant rule, in which the unknown quantities have to be estimated appropriately. For known class conditional densities f i (x), the maximum likelihood (ML) discriminator allocates an observation with vector x to the class with the greatest likelihood: RML (x) = m, if P(Cm |x) = maxi f i (x). In the case of equal class priors π i , it means maximizing the posterior probabilities [i.e., the optimal or Bayes rule RI (x)]. Otherwise, the rule RML (x) is not optimal (i.e., it does not minimize the overall error rate). Approaches of estimating the class conditional densities f i (x) have been developed by various authors, among others so-called plug-in rules (see Section 4.1), or nonparametric methods like kernel- or nearest neighbor-methods (see Sections 5 and 7, respectively).

4 DISCRIMINATION USING NORMAL MODELS The precondition of known priors is usually not fulfilled in practice. In medical diagnostics, the prevalence rates can be taken as estimations of the priors if available. If the training sample {x} = {x11 , . . . , x2n2 } has been obtained by a mixture C of the supposed classes C1 and C2 , then the prior π i can be estimated by its maximum likelihood estimator πˆ i = ni /n (i = 1, 2). Even more problematic is the estimation of the usually unknown class conditional densities f i (x) from the given training sample {x}. With the precondition of p-dimensional normally distributed observation vectors (x1 , . . . , xp ) [i.e., x ∼ N(µi , i )], with corresponding class densities f i (x) as normal (the parameters µ1 , µ2 denote the (expected) class means and  1 ,  2 the corresponding covariance matrices in the classes), two special decision rules from Bayes’ formula are obtained: linear discriminant analysis (LDA) and quadratic discriminant analysis (QDA), respectively. These two decision rules are widely used classification tools and implemented in most of the known software packages for statistics. Another approach to discrimination in case of normally distributed observations x is the already mentioned Fisher’s discriminant analysis (6) that uses a multiple linear regression approach with the class indicator variable as response and the observations x = (x1 , . . . , xp ) as regressors (16).

10

DISCRIMINANT ANALYSIS

4.1 Linear Discriminant Analysis Supposing equal covariance matrices in all classes (i.e.,  1 =  2 ), the optimal or Bayes’ decision rule (i = 1, 2) is obtained RO (x) = m, (x − µm )

if

−1

(x − µm )T − ln πm

= min[(x − µi ) −1 (x − µi )T − ln πi ] i

diagnostic process often consists in the investigation of the quantitative scaled data. For that reason, two groups of patients: were investigated 58 (n1 ) patients (class 1: glaucoma) and 41 (n2 ) patients (class 2: ocular hypertension) using the following features: • daily intraocular pressure (IOP), bra-

chialis blood pressure (BP), • ophthalmica blood pressure (OP), perfu-

sions pressure (difference BP − OP),

Supposing a training sample {x}, the corresponding maximum likelihood estimations from {x} are used for the unknown parameters and a so-called plug-in rule will be obtained LDA(x) = m, (x − xm )S

−1

if

(x − xm )T − ln πˆ m

= min[(x − xi )S−1 (x − xi )T − ln πˆ i ] i

ˆ where S = n/(n − 2) is the so-called biascorrected matrix of the estimated covariance ˆ Therefore, a patient is allocated matrix . with observation vector x0 to class Cm for which the Mahalanobis distance Mi ( x0 )(i = 1, 2) Mm (x0 ) = min Mi (x0 ) i

= min(x0 − xi )S−1 (x0 − xi )T i

will be minimal (except for − ln πˆ i ). Figure 7 shows the geometrical visualization [for  = I (I identity matrix), Mahalanobis distance equals to Euclidian distance]. In Fig. 7, the patient with observation vector x0 will be allocated to class C1 . Example 3: Differential diagnosis of glaucoma and ocular hypertension A difficult problem in ophthalmological diagnostics is the differentiation between glaucoma and ocular hypertension. In order to achieve the diagnosis, various clinical investigations (such as measurements of intraocular pressure, fields of vision, different anamnestic findings, clinical history, etc.) are accomplished, resulting in both quantitative and categorical scaled data. The first step of the

• treatment—test (IOP after treatment),

and others (2). Table 5 shows the results obtained using linear discriminant analysis [i.e., rule LDA(x)] supposing equal priors with unit losses. The true error rates have been estimated according to the leave-one-out method, including the first four features (IOP, BP, OP, BP-OP), which resulted in altogether 16 wrong allocations, which means an overall error estimation of 16.16%. As to be expected, the rule LDA(x) has a very good quality for patients suffering from ocular hypertension (only one patient wrongly allocated: 2.44%), whereas 15 patients from 58 (25.86%) were wrongly diagnosed in the glaucoma class. On the other hand, the positive predictive value of glaucoma is rather large with 97.73% (i.e., the probability of a glaucoma is high when predicted). 4.2 Feature Selection Particularly in medical applications, the number p of features may be very large relative to the sample size. Then, consideration

Table 5. Classification Table for the Glaucoma Data leave-one-out LDA Row [%] Col. [%] Allocated to class True class glaucoma hypertension glaucoma

hyper -tension Total

43 74.14% 97.73% 1 2.44% 2.27% 44

15 25.86% 27.27% 40 97.56% 72.73% 55

Total 58

41

99

DISCRIMINANT ANALYSIS

11

x2

M2(x 0)

x02 x12

M1(x 0) x22

x11

x01

x21

x1

Figure 7. Geometrical illustration of linear discriminant analysis (patients from class C1 with mean x1 are marked by crosses, class C2 with mean x2 marked by circles)

might be given to methods for reducing the number of features because too many features may produce problems with the performance of the sample discriminant function. On the other hand, each feature selection is an intervention into the given data and always causes a loss of information (17). Nevertheless, for practical applications in medicine, it is often interesting to separate important variables from redundant ones. Principally, if one wants to know which set of variables is effective, one must examine all possible subsets of x1 , x2 , . . . , xp . As many subsets may exist, a common strategy is to accomplish the selection in a stepwise procedure, forward (including, step-by-step, the most important variable) or backward (excluding redundant variables). As a measure for the ‘‘importance’’ of a variable, the amount of the overall error rate as selection criterion is preferred, which means a measure of class separation over one or a few linear combinations of the features (4). Furthermore, a cross-validated feature selection is always performed. Thereby, the given sample is reduced repeatedly by a certain number of observations and, with the remaining, the feature selection process is carried out. In every step, the removed patients are allocated using the decision rule obtained and the wrong allocations are counted (crossvalidated error rate). In the end, those features that appear most frequently in the

validation steps are selected for a discrimination rule (Fig. 8). If certain features show up again and again in the validation steps, they can be considered as particularly stable. These most frequent features were recommended to the medical doctors, and for those features, a new error estimation has been accomplished (recommended error rate). Example 3 (continuation): Diagnosis of glaucoma and ocular hypertension The discrimination result of an overall error rate of 16.16% was reached for the four features IOP, BP, OP, BP-OP. In addition, it is always interesting to estimate the true error rate for certain features univariately in order to get an overview of the performance of the features separately. On the other hand, it may be of interest to judge the relationships between features and their impact on the classification results when combined. Table 6 gives an overview about the error estimations (leaving-one-out estimation) separately and in building-up steps of feature selection (without cross-validation). This procedure selects in each step those features that have the best error estimation, accordingly. As a result, IOP is the best-predicting feature among the four given characteristics.

12

DISCRIMINANT ANALYSIS

Table 6. Error Rates of Separate Features and Consecutive Combinations Feature

Error rate (separately)

Error rate (in combination)

IOP BP OP BP-OP

16.16% 38.38% 19.19% 34.34%

16.16% 15.15% 18.18% 16.16%

vectors (x1 , x2 , . . . , xp ), another decision rule, the so-called quadratic discriminant analysis - QDA, is obtained written in the form of a plug-in rule, as QDA(x) = m,

if Qm (x) = min Qi (x) and

Qi (x) = 1/2(x −

i

xi )S−1 i (x

− xi )T

+ 1/2 ln |Si | − ln πˆ i ; i = 1, 2 Example 2 (continuation): Diagnosis of neuroborreliosis burgdorferi For the diagnosis mentioned, the doctors want to separate important from redundant variables. When selecting features, the obtained set of variables will certainly depend on the arbitrariness of the given training sample. Therefore, a serious feature selection must always include a cross-validation in order to avoid that randomness (crossvalidated feature selection). Table 7 gives the results of the crossvalidation process for the neuroborreliosis data. A set of three features was mostly chosen (five times), and the variables 5 (SBigg), 8 (CSF-prot), and 3 (Paresis) were selected with the frequencies 11, 11, 10, respectively. Thus, the feature set with 3 variables including 5 (SBigg), 8 (CSF-prot), and 3 (Paresis) was recommended to the medical doctors. The recommended error estimation (leave-one-out) for this feature set resulted in 11.48%. For reasons of brevity, the corresponding classification table is not presented here. 4.3 Quadratic Discriminant Analysis In case of unequal covariance matrices,  1   2 , but with normally distributed feature = Table 7. Results of a Cross-Validation in 11 Steps Results of cross-validation selected numbers of features corresponding frequencies selected features corresponding frequencies

3

4

5

5

4

2

5 11

8 11

3 10

1 6

2 3

where the estimations of the covariance ˆ i are replaced by the bias-corrected matrices  ˆ i /(ni − 1). This rule is estimations Si = ni  also implemented in most of the commercial software packages, although it is rather seldom worth applying, because of non essential improvements of the results in the majority of applications. It could be shown in numerous simulation experiments that quadratic discriminant analysis should be applied only in case of ‘‘huge’’ differences between the covariance matrices (18). 5 NONPARAMETRIC DISCRIMINATION The methods of discrimination described so far suppose some model [for the class conditional densities f i (x)] on the basis of which certain parameters have to be estimated. As an example, refer to Section 4 for normally distributed densities f i (x). Such approaches are called parametric. On the other side, procedures that have been developed without any postulation of a certain model are called nonparametric. The focus here is on kernelmethods that dominate the nonparametric density estimation literature. Given a training sample {x} = {x11 , . . . , x2n2 }, in the nonparametric kernel estimation of the class conditional density f i (x), a ‘‘kernel’’ is laid around each observation xij of the training sample. The kernel function K (i) (x, xij ) may have any shape, but has to fulfil the conditions of a probability density function. A convenient kernel that is very often used is a multivariate normal density (i = 1, 2) ˆ i |−1/2 K (i) (x, xij ) = (2π )−p/2 | ˆ −1 (x − xij )T } × exp{− 12 (x − xij ) i

DISCRIMINANT ANALYSIS

13

k=1 Remove a subset {x}k = {xk1,...,xkm} from {x} {x}rem = {x} − {x}k Train the classifier on {x}rem and select a feature set [Mk]

k=k+1

Test the classifier with {x}k 0, if xkj correctly classified 1, if xkj wrongly classified

pkj =

m ·k < n ?

n

y

Selection of the feature set [Mko ], which was most frequent in the validations and estimation of the corresponding error on the basis of the whole sample Cross-validated error: F (cv ) =

1 n

m

Sk

k =1 j =1

pkj

Recommended error: F (rec) =

1 n

n j =1

pko j

Figure 8. Cross-validated feature selection (compare also Fig. 5)

ˆ i = ςi i with the (diagonal) covariance ( matrix i = diag{s2i1 , . . . , s2ip } of the sample standard deviations s2il = 1/(ni − 1)
ni 2 (l = 1, . . . , p) and elements j=1 (xijl − xil ) ςi —so-called smoothing parameters—which have to be estimated from the training sample). The kernel estimator of f i (x) in class Ci is then defined as the arithmetic mean of all K (i) (x,xij ) in the ith class (i = 1, 2) ni 1  K (i) (x, xij ) fˆi (x) = ni j=1

A graphical representation of a kernel estimation in class Ci gives Fig. 9 with 7 observations and p = 1 (15). With the estimated class conditional densities fˆi (x), decision rule RI (x) or RA (x), can be applied for discrimination, respectively.

Example 4: The IIBM (International Investigation of Breast MRI) study Accuracy data about contrast-enhanced MRI in the literature vary, and concern exists that a widespread use of contrast-enhanced MRI combined with insufficient standards might cause unsatisfying results. Therefore, an international multicenter study was established (19) in order to find out whether— based on MRI enhancement characteristics— a statistically founded distinction between benign and malignant lesions is possible. The study resulted in findings of more than 500 patients. The analyses were performed with the MRI signal intensities SIGs to six consecutive sample points (s = 1, . . . , 6). The differentiation between ‘‘malignant’’ and ‘‘benign’’ was made on the basis of histological classes, resulting in 132 malignant and 63 benign lesions (one per patient) with complete datasets. For each lesion, a total of 12 parameters were calculated from the original MRI signal intensities. From the training sample, it was concluded that the

14

DISCRIMINANT ANALYSIS

fˆi (x)

x

Figure 9. Kernel estimation fˆi (x) of the class conditional density f i (x)

Table 8. Classification with Kernel Discriminator and Two Different Variants of Priors leave-o-out Row [%] Col. [%] True class malignant

benign Total

πˆ 1 = πˆ 2 = 0.5 ζˆ1 = 0.31, ζˆ2 = 0.24 Allocated to class malignant benign 120 90.91% 86.33% 19 30.16% 13.67% 139

12 9.09% 21.43% 44 69.84% 78.57% 56

Total 132

108 81.82% 87.80% 15 23.81% 12.20% 123

malignant 63 benign 195

assumption of normally distributed densities f i (x) was not fulfilled. Thus, a nonparametric kernel discriminator was applied and achieved after a (cross-validated) feature selection and with equal priors πˆ 1 = πˆ 2 = 0.5 a best π -error-estimation of 15.90%. Table 8 (left side) presents the results obtained, showing both high sensitivity (90.91%) and high positive predictive value (86.33%). But the study aimed at a discrimination with a possible high specificity. For this reason, the original priors πˆ 1 were replaced by πˆ 12 = πˆ 1 l12 , and πˆ 2 by πˆ 21 = πˆ 1 l21 , respectively, in order to realize the influence of different losses lij (compare Section 3.2). With the priors πˆ 12 = 0.3 and πˆ 21 = 0.7, the specificity has been increased to 76.19% with a slightly improved positive predictive value of 87.80%, but at the expense of a higher overall error rate of 20.00% (Table 8, right side). 6

πˆ 12 = 0.3, πˆ 21 = 0.7 ζˆ1 = 0.31, ζˆ2 = 0.24 Allocated to class malignant benign

leave-o-out Row [%] Col. [%] True class

LOGISTIC DISCRIMINANT ANALYSIS

According to J. A. Anderson (20), logistic discrimination is broadly applicable in a wide variety of distributions, including multivariate normal distributions with equal

Total

24 18.18% 33.33% 48 76.19% 66.67% 72

Total 132

63

195

covariance matrices, multivariate discrete distributions following the log-linear model with equal interaction terms, joint distributions of continuous and discrete random variables, and therefore, particularly advantageous in applications with mixed data. The logistic discriminant analysis is called semi-parametric because the class conditional densities f i (x) are not modeled for themselves but for the ratio between them. With given priors π 1 and π 2 , the model log

π1 f1 (x) = β0 + βxT π2 f2 (x)

is postulated where β 0 and β = (β 1 , . . . ,β p ) are unknown parameters that have to be estimated. Remembering P(Ci | x) = πi fi ( x) (without the denominator π1 f1 ( x) + π2 f2 ( x)) from rule RI (x)) leads to P(C1 | x)/[1 − P(C1 | x)] = exp(β0 + β xT ), from which one obtains

P(C1 |x) =

exp(β0 + βxT ) 1 + exp(β0 + βxT )

DISCRIMINANT ANALYSIS

and

P(C2 |x) =

1 1 + exp(β0 + βxT )

analogously as posterior probabilities for the corresponding allocations. After estimating the unknown parameters β 0 ,β 1 , . . . β p according to the maximum likelihood principle, it is again decided for the class with the greatest posterior probability [i.e., on the basis of decision rule RLOG (x) = m, if P(Cm |x) = maxi P(Ci |x) (as maximum likelihood discrimination rule)]. The software packages mostly contain an equivalent decision: allocation of the patient with observation vector x to the class C1 if P(C1 |x) > 0.5, otherwise to C2 . Example 3 (continuation): Diagnosis of glaucoma and ocular hypertension Beside the already mentioned metrically scaled measurements, categorical observations from various clinical investigations have been raised: • fields of vision according to Goldmann,

Tuebinger, Octopus • different anamnestic findings. The whole set of metrical variables in combination with two categorical features (fields of vision according to Tuebinger and Octopus) has been included in a logistic regression analysis in the sense of logistic discrimination as described above. With the estimated parameters βˆ0 , βˆ1 , . . . βˆp , one is able to determine the corresponding posterior probabilities and to decide for the corresponding allocations (predicted allocation). Together with the observed allocations (true classes), one can calculate a corresponding classification table (Table 9), which resulted in an overall error rate of 10.10%. As error estimations according to the π -method are not offered in commercial software for logistic discrimination, the numbers are shown with the resubstitution method, unfortunately. A comparison with previous results is therefore only possible by using the same error estimation. The LDA resulted in an overall error of 11.11%, only using the four features IOP, BP, OP, and BP— OP and resubsitution

15

Table 9. Classification Table for the Glaucoma Mixed Data Resubstitution Row [%] Col. [%] Observed glaucoma

hypertension Total

Logistic discrimination Predicted glaucoma hypertension 52 89.66% 92.86% 4 9.76% 7.14% 56

6 10.34% 13.95% 37 90.24% 86.05% 43

Total 58

41

99

as error estimation (2). Generally, it can be expected that the inclusion of clinically relevant features will improve the performance of a prediction rule. The problem of discriminant analysis for mixed data has been particularly treated by Wernecke (21), who gives a powerful algorithm for coupling the results of various discriminators, each adequate for the different scaling of data. Practical examples from medicine demonstrate the improvement of discrimination by using the full dataset of both quantitative and categorical features. 7 FURTHER METHODS OF DISCRIMINATION Beside the methods of discrimination mentioned, a number of further procedures for discrimination exist. A short overview is provided of some methods of particular interest for medical applications. In many medical applications, the data are categorically scaled. For those data, a special discrimination with categorical features can be applied. Instead of the original realizations of the given features x1 , . . . , xp , one most regard here the frequencies h1 (x1 , . . . , xp ), . . . , hK (x1 , . . . , xp ) by which the feature combinations appear in the classes C1 , . . . , CK (where for every (x1 , . . . , xp ) all combinations of category levels have to be inserted). Such frequencies are usually arranged in contingency tables. As an example, consider a study with patients suffering from migraine. Table 10 shows a corresponding contingency table in

16

DISCRIMINANT ANALYSIS

Table 10. Two-Dimensional Contingency Table for Class C1 with p = 2 Variables variable x1 (nausea)

cat. 1 (low)

cat. 1 (low) cat. 2 (strong)

h1 (x11 , x21 ) = 16 h1 (x12 , x21 ) = 10

six cells with two features (p = 2) in three respective two categories, each, for class C1 . Accordingly, there are 16 patients with low headache and nausea complaints, 28 patients with moderate headache and low nausea complaints, and 22 patients with strong headache and low nausea complaints, respectively overall, 66 patients exist with low nausea, 38 patients exist with strong headache complaints, and so on, and altogether, n1 = 116 patients in the class C1 . After estimating the unknown cell probabilities pi (x1 , . . . , xp ) with the help of the corresponding frequencies hi (x1 , . . . , xp ) (e.g., by
using pˆ i (x1 , . . . , xp ) = hi (x1 , . . . , xp )/ni , ni = hi (x1 , . . . , xp )—so-called actuarial model) (2), a patient with observation vector x will be assigned to the class Cm for which RC (x) = m,

if πm pˆ m (x)

= max πi pˆ i (x); i

i = 1, 2, . . . , K

[Bayes optimal for known π i , pi (x)—Linhart (22)]. Friedman (23) proposed a regularized discriminant analysis as a compromise between normal-based linear and quadratic discriminant analysis. In this approach, two parameters are introduced that control shrinkage of ˆ i (i = 1, . . . , the heteroscedastic estimations  ˆ K) toward a common estimate  and shrinkage toward a series of prespecified covariance matrices. Nearest neighbor classifiers are based on a pairwise distance function dj (x) = d(x,xj ) [known from cluster analysis; see, for example, Everitt (24)] between an observation x with unknown class membership and some vector of observations xj from the given training sample. Denoting the sorted distances by d(1) ( x) ≤ d(2) ( x) ≤ · · · ≤ d(n) ( x), class posterior probability estimates are obtained for class Ci as the fraction of class Ci observations among the k nearest neighbors to x (25)

variable x2 (headache) cat. 2 (moderate) h1 (x11 , x22 ) = 28 h1 (x12 , x22 ) = 24

cat. 3 (strong) h1 (x11 , x23 ) = 22 h1 (x12 , x23 ) = 16

n  ˆ i |x) = 1 [dj (x) ≤ d(k) (x)] P(C k j=1

for all xj ∈ Ci ˆ m |x) = and allocate x to the class Cm with P(C ˆ maxi P(Ci |x); i = 1, . . . , K. The number k of nearest neighbors is often taken as k = 1. Another approach is a cross-validated selection, where for given k = 1, 3, 5, . . . a leaveone-out error estimation (see Section 2.1) is applied to the training sample and that k with the minimal error rate will be chosen. A discriminant analysis for longitudinal data (repeated measurements) has been introduced by Tomasko et al. (26) and extended by Wernecke et al. (27). The procedure is a modification of the linear discriminant analysis using the mixed-model MANOVA for the estimation of fixed effects and for a determination of various structures of covariance matrices, including unstructured, compound symmetry, and autoregressive of order 1. Among the nonparametric statistical classification methods, Classification and Regression Trees (CART) play an important role, especially in medical applications (28). Given a set of risk factors x1 , . . . , xp that influence a response variable y, the data is split step-bystep into subgroups, which should be internally as homogeneous and externally as heterogeneous as possible, measured on a function F(y|xl ); l = 1, . . . , p (supposing dichotomous variables, the function F(y|xl ) will be the maximal measure of dependence between y and x1 from a χ 2 -statistics). Every split corresponds to a so-called node and every observation will be localized at exactly one terminal node in the end. The obtained tree defines a classification rule by assigning every terminal node to a class Ci by comparing the risks at the beginning and at the terminal nodes (29).

DISCRIMINANT ANALYSIS

Support vector machines were introduced by Vapnik (30) and refer to K = 2 classes. The discrimination is defined by the ‘‘best’’ hyperplane separating the two classes, where ‘‘best’’ means that hyperplane that minimizes the sum of distances from the hyperplane to the closest correctly classified observations while penalizing for the number of misclassifications (25).

8 JUDGING THE RELIABILITY OF THE ESTIMATED POSTERIOR PROBABILITY In the previous sections, the author dealt with discriminant rules on the basis of posterior probabilities P(patient ∈ Ci | x) =
P(Ci | x) = πi fi ( x)/[ k πk fk ( x)](i = 1, . . . , K), which could be estimated from a training sample
{ x} = { x1 , . . . , xn } in different ways (n = i ni , xj = (xj1 , . . . , xjp ), j = 1, . . . , n). The overall performance of the discrimination rules described can generally be judged by an appropriate estimation of the error rate (See Section 2.1). Nevertheless, one problem is left and should be mentioned at least: How can one judge the reliability of an allocation on the basis of the estimated posteriors, conditional on the observed patient x? As emphasized by McLachlan (11), even in case of low error rates, there may still be patients about them where great uncertainty as to their class of origin. Conversely, assuming high error rates, it may still be possible to allocate some patients with a great certainty. The error rate (as an average criterion) addresses the global performance of the decision rule and judges the quality of the allocation of some chosen patient with unknown class membership. Mainly in clinical medicine, it is often more appropriate to proceed conditionally on the observed individual patient x with its particular realizations of the variables x1 , . . . , xp . One way to assess the mentioned reliability is the calculation of standard errors and also interval estimates (95% confidence ˆ i |x), intervals) for the point estimations P(C conditional on x. For reasons of brevity, corresponding results are not presented here, but the reader is referred to the literature [see, among others, McLachlan (11)].

17

9 SOFTWARE PACKAGES FOR DISCRIMINATION

Software packages for discrimination are numerous. Disregarding the permanent changes of program packages and the development of new software, some remarks may be worthwhile, concentrating on the most important statistical packages. R. Goldstein (31) gives an excellent overview over commercial software in biostatistics, generally. Judgements for classification software (discrimination, clustering, and neural networks) were given in some detail by D. Michie et al. (3). In this critical overview, different program packages have been described and recommendations for their use given. For applications in clinical trials, not so many discrimination methods are worth considering. The applications of discriminant analysis in medical studies might be rather often confined to the methods described in Sections 4 and 6, respectively, including error estimations (Section 2.1) and feature selection (Section 4.2) (unfortunately usually not validated). Therefore, it seems to be sufficient to refer to the well-known commercial software packages SAS (SAS Institute, Inc., Cary, NC, USA), and SPSS (SPSS, Inc., Chicago, Il, USA). SPSS offers both linear and quadratic discrimination as well as logistic discriminant analysis with the mentioned π -error estimations (not for logistic regression) and stepwise feature selection (not validated). Moreover, various diagnostics, such as tests for equality of the covariance matrices  i , can be established. SAS provides a broader variety of discrimination methods including linear and quadratic discriminant analysis, logistic discrimination, nonparametric kernel discrimination, and nearest neighbor methods. Error estimations, (partly) according to Section 2.1, and stepwise feature selection (not validated) are also implemented. Users with ambitions to develop certain routines for themselves may be referred to S-PLUS 2000 (Data Analysis Products Division, MathSoft, Seattle, WA, USA). The

18

DISCRIMINANT ANALYSIS

above-mentioned examples have been calculated using mostly SAS and SPSS, respectively, and special routines (such as crossvalidated feature selection or discrimination for categorical features) developed in S-PLUS (available in the public domain on request for everybody).

of Statistical Science. New York: John Wiley and Sons, 1982, pp. 389–397. 15. J. Hermans, J. D. F. Habbema, T. K. D. Kasanmoentalib, and J. W. Raatgever, Manual for the alloc80 discriminant analysis program. The Netherlands: Department of Medical Statistics, University of Leiden, 1982. 16. B. Flury, A First Course in Multivariate Statistics. New York: Springer, 1997.

REFERENCES 1. H-J. Deichsel and G. Trampisch, Clusteranalyse und Diskriminanzanalyse. Stuttgart: Gustav Fischer Verlag, 1985. 2. K-D. Wernecke, Angewandte Statistik fur ¨ die Praxis. Bonn, Germany: Addison-Wesley, 1996. 3. D. Michie, D. J. Spiegelhalter, and C. C. Taylor, Machine Learning, Neural and Statistical Classification. Chichester: Ellis Horwood, 1994. 4. B. D. Ripley, Pattern Recognition and Neural Networks. Cambridge: Cambridge Universitiy Press, 2004. 5. D. F. Morrison, Multivariate analysis of variance. In: P. Armitage and T. Colton (eds.), Encyclopedia of Biostatistics. New York: John Wiley and Sons, 1998, pp. 2820–2825. 6. R. A. Fisher, The use of multiple measurements in taxonomic problems. Ann. Eugen. 1936; 7: 179–188. 7. P. A. Lachenbruch, Discriminant Analysis. New York: Hafner Press, 1975. 8. C. Ambroise and G. J. McLachlan, Selection bias in gene extraction on the basis of microarray gene-expression data. Proc. Natl. Acad. Sci. 2002; 99(10): 6562–6566. 9. P. A. Lachenbruch and M. R. Mickey, Estimation of error rates in discriminant analysis. Technometrics 1968; 10: 1–11. 10. G. T. Toussaint, Bibliography on estimation of misclassification. IEEE Trans. Inform. Theory 1974; 20: 472. 11. G. J. McLachlan, Discriminant Analysis and Statistical Pattern Recognition. New York: John Wiley and Sons, 1992. 12. B. Efron, The Jackknife, the Bootstrap and Other Resampling Plans. Philadelphia: SIAM, 1982. 13. T. Bayes, An essay towards solving a problem in the doctrine of chance. Communicated by Mr. Price, in a letter to John Canton, AM., F.R.S., December 1763. 14. P. A. Lachenbruch, Discriminant analysis. In: S. Kotz and N. L. Johnson (eds.), Encyclopedia

¨ 17. J. Lauter, Stabile Multivariate Verfahren (Diskriminanzanalyse, Regressionsanalyse, Faktoranalyse). Berlin: Akademie Verlag, 1992. 18. S. Marks and O. J. Dunn, Discriminant function when covariance matrices are unequal. JASA 1974; 69: 555–559. 19. S. H. Heywang-K¨obrunner, U. Bick, W. G. Bradley, B. Bon´e, J. Casselmanand, A. ¨ Coulthard, U. Fischer, M. Muller-Schimpfle, H. Oellinger R. Patt, J. Teubner, M. Friedrich, G. Newstead, R. Holland, A. Schauer, E. A. Sickles, L. Tabar, J. Waisman, and KD. Wernecke, International investigation of breast MRI: results of a multicentre study (11 sites) concerning diagnostic parameters for contrast-enhanced MRI based on 519 histopathologically correlated lesions. Eur. Radiol. 2001; 11: 531–546. 20. J. A. Anderson, Logistic discrimination. In: P. R. Krishnaiah and L. Kanal (eds.), Handbook of Statistics, 2: Amsterdam: North Holland, 1982, pp. 169–191. 21. K-D. Wernecke, A coupling procedure for the discrimination of mixed data. Biometrics 1992; 48(2): 497–506. 22. H. Linhart, Techniques for discriminant analysis with discrete variables. Metrica 1959; 2: 138–149. 23. J. H. Friedman, Regularized discriminant analysis. J. Amer. Statist. Assoc 1989; 84: 165–175. 24. B. S. Everitt, Cluster Analysis. London: Halstead Press, 1980. 25. S. Dudoit and J. Fridlyand, Classification in microarray experiments. In: T. Speed (ed.), Statistical Analysis of Gene Expression Microarray Data. Boca Raton, FL: Chapman and Hall/CRC, 2003. 26. L. Tomasko, R. W. Helms, and S. M. Snapinn, A discriminant analysis extension to mixed models. Stat. Med. 1999; 18: 1249–1260. 27. K-D. Wernecke, G. Kalb, T. Schink, and B. Wegner, A mixed model approach to discriminant analysis with longitudinal data. Biom. J. 2004; 46(2): 246–254.

DISCRIMINANT ANALYSIS 28. L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone, Classification and Regression Trees. Monterey, CA: Wadsworth, 1984. 29. K-D. Wernecke, K. Possinger, G. Kalb, and J. Stein, Validating classification trees. Biom. J. 1998; 40(8): 993–1005. 30. V. Vapnik, Statistical Learning Theory. New York: Wiley and Sons, 1998. 31. R. Goldstein, Software, biostatistical. In: P. Armitage and T. Colton (eds.), Encyclopedia of Biostatistics. New York: John Wiley and Sons, 1998, pp. 2820–2825.

19

DISEASE TRIALS FOR DENTAL DRUG PRODUCTS

filled teeth or % sites bleeding on probing). It has become accepted to average ordinal scores from gingival and plaque indices. Statistical properties of such measures have been examined by Sullivan and D’Agostino (3). These can also be examined using the percent of sites, teeth, or surfaces exceeding a specified index level. Summaries over subsets of teeth or sites of particular interest, such as pockets harboring specific bacterial species at baseline, or tooth surfaces diagnosed with ‘‘white spots’’ can provide more efficient and focused comparisons. In clinical trials, summary measures for each subject for primary outcomes of interest should be the basic unit of analysis for comparisons across treatment groups, not individual sites or teeth (4). This having been said, often great interest develops in understanding the dynamics of disease and treatment effects at specific sites or tooth surfaces. In such situations, it is critical that analyses of site-specific associations recognize and take into account that individual teeth and sites within subjects are not statistically independent units. Statistical methods for analyzing sitespecific association in the presence of intrasubject dependency are based on two general approaches. The first approach considers each subject as a stratum, computes a site-specific measure of association for each subject, evaluates the homogeneity of the individual measures of association across subjects, and, if appropriate, generates a summary measure of association. Examples are the use of Mantel–Haenszel procedures for categorical outcomes (5) or the incorporation of subject effect as a fixed effect indicator variable in a regression analysis of site-specific covariables (6). The second approach uses statistical methods explicitly incorporating intrasubject correlations among sites into the statistical model. As an example, generalized estimating equation procedures, described with applications to multiple site data in Reference 7, can accommodate logistic and standard multiple regression models, subject and site level covariates, unequal numbers of sites per subject, explicitly specified or empirically estimated correlation structures, and model-based or robust

MAX GOODSON RALPH KENT The Forsyth Institute—Clinical Research Boston, Massachusetts

Disease trials for dental drug products do not greatly differ from disease trials for medical drug products and should adhere to standard principles for the design, conduct, and evaluation of clinical trials (1). Oral research is, however, associated with certain important simplifications and some complications. One simplifying factor is that dentistry is principally concerned with treatment of two diseases: periodontal diseases and dental caries. In addition, however, a surprising number of dental drug products are designed for prevention, improved esthetics, and reduced pain. A principal complication is the multiplicity created by teeth. A unique attribute of statistical development in oral research is the 30-year presence of a statistical ‘‘think tank,’’ the Task Force on Design and Analysis (2). This organization has been the source of many American Dental Association Guidelines, professional input to the FDA, and has sponsored meetings and conferences to discuss methodological issues in oral research. Throughout this era, it has provided a forum for debate on substantial statistical issues connected with evaluation of drugs used in treatment of oral diseases. 1 COMMON FEATURES OF CLINICAL TRIALS FOR TESTING DENTAL DRUG PRODUCTS 1.1 Data Summary In oral disease research, evaluations are made for numerous teeth, tooth surfaces, or sites for each subject. Most commonly, observations are quantitative, dichotomous, or, as in the case of widely used indices for plaque and gingivitis, ordinal. Summary values for each subject are obtained in various ways, such as means (e.g., mean pocket depth) or percent or counts of sites or teeth that exhibit an attribute (e.g., # of decayed, missing or

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

DISEASE TRIALS FOR DENTAL DRUG PRODUCTS

estimates of model parameters and standard errors. 1.2 Training and Calibration The subjectivity and variability of many assessments makes it critical to use examiner training and calibration (8,9), assignment of specific examiners to follow each subject throughout the study, blinding of examiners, and balanced assignment of examiners across experimental groups. 1.3 Adjustment for Baseline Severity Baseline levels of outcome measures for a subject are frequently highly correlated with subsequent posttreatment observations. Stratification or matching by baseline level, evaluation of change from baseline, and analysis of covariance are commonly used to reduce the effects of baseline variation across subjects for primary outcome measures. For example, the variability of an assay is largely reduced by selecting periodontal sites in a relatively narrow baseline pocket depth range (10). 2 TESTING DRUGS IN ORAL HYGIENE STUDIES Dental plaque is the soft, cheesy white material that accumulates on teeth in the absence of oral hygiene. It is composed of bacterial cells in a polysaccharide polymer created by the bacteria. It is considered by most individuals to be the precursor for most oral diseases. Among the more common clinical trials conducted to test dental products are those associated with toothpastes, mouth rinses, and toothbrushes used for suppression of dental plaque formation or removal. From the FDA viewpoint, dental plaque is not considered to be a disease but a cosmetic issue, so that these products are largely unregulated in this area. Studies of dental plaque removal or suppression depend largely on clinical indices designed to measure the extent or magnitude of plaque accumulation. The Turesky modification of the Quigley–Hein Index for plaque (11) is the most commonly used plaque index. By this method, stained plaque tooth coverage is

evaluated by assigned a value between 0 (none) and 5 (> = 2/3 coverage) for each tooth surface evaluated. When plaque becomes calcified, the hard, adherent mass is called dental calculus or tartar. Suppression of formation or removal of calculus is also most often measured by indices. In this case, the Volpe–Manhold index (12) is the index most commonly used. By this method, calculus accretion is evaluated by measuring the extent of tooth coverage at three locations on the tooth using a periodontal probe.

3 TESTING DRUGS IN GINGIVITIS STUDIES The most common periodontal disease is gingivitis, a mild inflammation of the periodontal tissues with no evidence of connective tissue destruction (13). This condition exists at some time in virtually everyone’s mouth (14). Many outcome variables have been defined for measurement of drugs used to treat gingivitis (15). All outcome variables depend on evaluating either gingival redness and/or gingival bleeding. Drugs most commonly tested for treatment of gingivitis are antibacterial mouth rinses and toothpaste. It is generally difficult if not impossible to demonstrate product efficacy in treatment of gingivitis by starting with patients having natural accumulations of plaque or tartar (calcified plaque, calculus) (16). For this reason, most studies are initiated with a thorough tooth cleaning and efficacy is demonstrated by the ability of a test product to reduce the rate of reaccumulation of plaque or tartar (17). The standard trial for treatment of gingivitis uses the protocol recommended by the American Dental Association Acceptance program (18) and evaluates gingival index (19) as the primary outcome variable. There are issues concerning use of qualitative ordinal indices as outcome variables. Difficulty in interpreting mean index values and the underlying assumptions of combining observations such as redness and bleeding into a single index have lead some researchers to consider statistical analysis of dichotomized index values (20).

DISEASE TRIALS FOR DENTAL DRUG PRODUCTS

4 TESTING DRUGS IN PERIODONTITIS STUDIES Several periodontal diseases currently are differentiated by their clinical characteristics. Periodontitis is a complicated multifactorial chronic inflammatory disease condition that involves specific bacterial pathogens, host responses, and biochemical mediators that create inflammation in the tissues that support the teeth and may result in their ultimate destruction (21). The critical difference between periodontitis and gingivitis is that loss of supporting bone and periodontal ligament occurs in the former but not the latter. This loss of alveolar bone may be seen using radiographs, but because of the biohazard of gamma radiation Institutional Review Boards will seldom approve radiography as a clinical research measurement tool. For this reason, periodontal disease is most often evaluated indirectly by periodontal probing. Periodontal probing is used as a clinical measure of periodontal disease progression. Pocket depth (Fig. 1) is the distance from the free margin of the gingiva to the depth of the pocket formed by the disease process. In normal healthy individuals, a periodontal sulcus measures 2–3 mm in depth. In the early stages of periodontitis, a periodontal pocket forms adjacent to the tooth and can progressively deepen up to the length of the tooth root (12–15 mm). Periodontal pocket depth (PD), a continuous measure, is the most common outcome Periodontal Probe Measurements

variable used to evaluate periodontal disease. The periodontal probe is a cylindrical, thin (usually 0.5 mm diameter), hand-held measurement tool calibrated in millimeters and designed to be placed in the periodontal pocket. Skilled clinicians can measure pocket depth paired differences with a standard deviation of 0.6 mm. Several variants of this measurement are also used, notably, attachment level (AL or, more precisely, clinical attachment level), which estimates the distance from a fixed anatomical tooth landmark (the cementoenamel junction) to the base of the periodontal pocket (Fig. 1). This measure is generally considered to be the best indirect estimate of bone loss (22) because it corrects for gingival swelling (hypertrophy) or shrinkage (recession). Because AL is the difference between the probe measurements of PD and recession (B) (23), reproducibility is less than with PD (skilled clinicians can measure AL paired differences with a standard deviation of 0.8 mm). As one would expect, AL is highly correlated with PD. Drugs most commonly tested to treat periodontitis are antibacterial agents or antibiotics, occasionally anti-inflammatory agents. Subject selection should include cognizance of possible confounding with age, smoking (24), obesity, and diabetes (25). Proposed guidelines for periodontal therapy trials have been published (26). Anatomical Landmarks Tooth Enamel

Pocket Depth (PD)

“B” measure Attachment Level (AL)

AL = PD-B

3

Cemento-enamel Junction (CEJ) Gingiva Periodontal Pocket Tooth root (cementum-covered) Bone

Figure 1. Periodontal probe measurements and anatomical landmarks. Pocket depth (PD) is a single measurement most commonly recorded in clinical practice. The anatomically adjusted measurement of attachment level (AL) requires two measurements, PD and recession (B), and is associated with increased variability. The value of B carries a positive sign for sites with gingival hypertrophy and a negative sign for sites with gingival recession.

4

DISEASE TRIALS FOR DENTAL DRUG PRODUCTS

4.1 Statistical Issues in Periodontitis Studies The appropriate unit of statistical analysis has been the subject of considerable controversy. The concept of periodontal sites was introduced into the literature as an analytical feature defining 4 to 6 locations around the tooth that were identified by anatomical landmarks and evaluated longitudinally to preserve information concerning the local site-specific changes in PD, AL, and bacteria potentially related to disease (27). Subsequent microbiological evaluations indicated that individual sites within the same mouth can have a completely different microbial ecology (28). Because bacterial diseases are generally defined by the infecting agent(s), the hypothesis was advanced that different periodontal diseases may coexist in the same mouth and that it would be inappropriate to pool periodontal sites simply because they had the same clinical signs of periodontitis. Some investigators interpreted these fundamental observations related to defining the pathogenic process as an effort to justify summarizing statistical data on periodontal sites and thereby inflating statistical significance (each subject evaluated at 6 sites/tooth may contribute as many as 192 sites). The choice of the unit of statistical analysis depends on the questions being asked. Most often, the subject is the appropriate unit of statistical summary. If sites, tooth surfaces, teeth or quadrants within subjects are to be used as the unit of analysis, however, then it is important to use procedures, such as described in Section 1.1, appropriate to the evaluation of correlated observations. The measurement of numerous bacterial species, host factors, clinical parameters, and genetic markers has created serious issues of multiple testing. For limited numbers of variables, standard multiple comparisons and multiple testing procedures can be used to control the Type I or ‘‘false positive’’ error rate. Loss of statistical power, however, can be severe for large numbers of variables. The application of groupings derived from data mining procedures such as cluster analysis is an alternate approach that has been useful in microbiological studies (29). For example, a clinical trial using ‘‘red complex’’ bacteria derived from cluster analysis as a primary

outcome variable successfully evaluated the antibacterial response of a locally applied antibiotic in the presence of conventional therapy (30). Because a high level of symmetry is found in the mouth, many investigators have suggested that test and control therapies in some instances can be conducted in a single subject with an assumed reduction in response variability. Potential problems introduced by these designs are considerable (31,32). Because all teeth are nourished by the same vascular system, one can never be certain that treatment effects are truly isolated. Evidence suggests that even toothcleaning on one side of the mouth may affect the other side (33). Carry-across effects can reduce differences between treatment and control responses. For example, statistically significant differences observed in single-site local antibiotic therapy (34) could not be repeated when multiple diseased sites were treated, presumably because antibiotic from multiple locally treated sites in one quadrant affected the response of control sites in other quadrants (35). Although it is true that the most likely error in split mouth designs is an underestimation of the therapeutic effectiveness of the test product, the magnitude of the consequences may not be acceptable. Without question, parallel designs are more dependable in evaluating therapeutic responses in periodontal disease therapy. 5 TESTING DRUGS FOR TREATMENT OF DENTAL CARIES Dental caries or dental decay occurs in more than 90% of our population (36). Although much may be said about dental caries as a disease process, from the clinical testing point of view, it can be viewed as the result of bacteria that partially ferment carbohydrates and produce organic acids that dissolve the calcium hydroxyapatite from teeth to form cavities. The disease process starts with a ‘‘white spot’’ that represents decalcification of the tooth enamel and proceeds to a cavity that if not treated can result in tooth loss. Outcome variables for the measurement of dental caries are discrete measurement variables that represent the number of holes in

DISEASE TRIALS FOR DENTAL DRUG PRODUCTS

teeth, white spots in teeth, missing teeth, filled teeth, and so forth. The classic measure of decay in permanent teeth is DMFT (37): the number of decayed, missing, or filled permanent teeth (or, DMFS for surfaces). Criteria for evaluating dental caries using visual criteria or detection with a sharp probe vary greatly between investigators. A review of criteria that have been used to score dental decay found 29 unique systems (38). The NIDCR criteria (39) have been used for largescale epidemiologic surveys. The widespread acceptance of fluoride toothpaste as an effective decay prevention product often demands that trials be conducted using noninferiority designs (40). Guidelines for the conduct of clinical trials for agents used to treat dental caries have been published (41) and are widely accepted as a standard. Characteristically, clinical trials for prevention of dental caries measure net conversion from sound to cavitated tooth surfaces or teeth over an appropriate time interval (incidence). Reliance on classification rather than measurement results in inefficient clinical trials that require many subjects. In addition, these methods are not easily adapted to measuring treatments designed to reverse demineralization of tooth surfaces. Newer methods have been developed to address this problem (42), analytic methods have been described that can be adapted to this measurement system (43), and clinical trials using these methods have been conducted (36). At this time, it is fair to say that these methods will require more research before being generally accepted as dependable. 6 TESTING DRUGS FOR LOCAL ANESTHETICS AND ANALGESICS By far, the most commonly used drugs in dental therapy are local anesthetics and analgesics. The standard for analgesic efficacy, the third molar extraction model, has become a standard used in medicine as well. Third molar extraction as a standard pain stimulus is generally evaluated using a visual analog scale (VAS). This scale is a 10-cm line on which the subject marks a perceived pain level between the two extremes of no pain

5

and worst pain. Mean centimeter differences between treatments and controls are then evaluated (44). The onset and duration of local anesthesia is often evaluated by electrical pulp testing (45). Guidelines for local anesthetic testing have been published (46). 7 TESTING DRUGS FOR TOOTH WHITENING Tooth whitening has become an extremely popular product area. Research in this area is particularly demanding because of the intensely subjective nature of the response. The most common clinical measure is visual comparison by a professional examiner with a tooth shade guide. Shade guides are not linear and contain different levels of several pigments. Results using this approach are somewhat arbitrary and highly subjective. One may also solicit a response rating from the subject but this rating can be associated with a high placebo response. Objective measurement by reflectance spectrophotometers (chromameters) or photographic equivalents calibrated in the LAB color domain have also been used. These devices produce results that are somewhat insensitive and often difficult to interpret. Published guidelines for testing tooth whitening products (47) suggest that studies be conducted using both clinical and quantitative measures. Studies that find concordance between objective and clinical evaluations tend to provide the most convincing evidence from clinical testing (48). REFERENCES 1. R. B. D’Agostino, and J. M. Massaro, New developments in medical clinical trials. J. Dent. Res. 2004; 83 (Spec No C): C18–24. 2. A. Kingman, P. B. Imrey, B. L. Pihlstrom, and S. O. Zimmerman, Chilton, Fertig, Fleiss, and the Task Force on Design and Analysis in Dental and Oral Research. J. Dent. Res. 1997; 76: 1239–1243. 3. L. M. Sullivan, and R. B. D’Agostino, Robustness of the t test applied to data distorted from normality by floor effects. J. Dent. Res. 1992; 71: 1938–1943. 4. J. L. Fleiss, S. Wallenstein, N. W. Chilton, and J. M. Goodson, A re-examination of withinmouth correlations of attachment level and of

6

DISEASE TRIALS FOR DENTAL DRUG PRODUCTS change in attachment level. J. Clin. Periodontol. 1988; 15: 411–414.

and gingivitis. J. Am. Dent. Assoc. 1986; 112: 529–532.

5. P. P. Hujoel, W. J. Loesche, and T. A. DeRouen, Assessment of relationships between site-specific variables. J. Periodontol. 1990; 61: 368–372.

19. H. Loe and J Silness, Periodontal disease in pregnancy. I. Prevalence and severity. Acta Odontol. Scand. 1963; 21: 533–551.

6. T. A. DeRouen, Statistical Models for Assessing Risk of Periodontal Disease. In: J. D. Bader (ed.), Risk Assessment in Dentistry. Chapel Hill, NC: University of North Carolina Dental Ecology, 1990. 7. T. A. DeRouen, L. Mancl, and P. Hujoel, Measurement of associations in periodontal diseases using statistical methods for dependent data. J. Periodontal. Res. 1991; 26: 218–229.

20. T. M. Marthaler, Discussion: Current status of indices of plaque. J. Clin. Periodontol. 1986; 13: 379–380. 21. M. A. Listgarten, Pathogenesis of periodontitis. J. Clin. Periodontol. 1986; 13: 418–430. 22. G. P. Kelly, R. J. Cain, J. W. Knowles, R. R. Nissle, F. G. Burgett, R. A. Shick, et al., Radiographs in clinical periodontal trials. J. Periodontol. 1975; 46: 381–386.

8. A. M. Polson, The research team, calibration, and quality assurance in clinical trials in periodontics. Ann. Periodontol. 1997; 2: 75–82.

23. S. P. Ramfjord, J. W. Knowles, R. R. Nissle, F. G. Burgett, and R. A. Shick, Results following three modalities of periodontal therapy. J. Periodontol. 1975; 46: 522–526.

9. E. G. Hill, E. H. Slate, R. E. Wiegand, S. G. Grossi, and C. F. Salinas, Study design for calibration of clinical examiners measuring periodontal parameters. J. Periodontol. 2006; 77: 1129–1141.

24. J. Haber, J. Wattles, M. Crowley, R. Mandell, K. Joshipura, and R. L. Kent, Evidence for cigarette smoking as a major risk factor for periodontitis. J. Periodontol. 1993; 64: 16–23.

10. J. M. Goodson, M. A. Cugini, R. L. Kent, G. C. Armitage, C. M. Cobb, D. Fine, et al., Multicenter evaluation of tetracycline fiber therapy: I. Experimental design, methods, and baseline data. J. Periodontal. Res. 1991; 26: 361–370. 11. S. L. Fischman, Current status of indices of plaque. J. Clin. Periodontol. 1986; 13: 371–380. 12. A. R. Volpe, J. H. Manhold, and S. P. Hazen, In Vivo Calculus Assessment. I. A method and its examiner reproducibility. J. Periodontol. 1965; 36: 292–298. 13. R. C. Page, Gingivitis. J. Clin. Periodontol. 1986; 13: 345–359. 14. J. W. Stamm, Epidemiology of gingivitis. J. Clin. Periodontol. 1986; 13: 360–370. 15. S. G. Ciancio, Current status of indices of gingivitis. J. Clin. Periodontol. 1986; 13: 375–378. 16. E. F. Corbet, J. O. Tam, K. Y. Zee, M. C. Wong, E. C. Lo, A. W. Mombelli, et al., Therapeutic effects of supervised chlorhexidine mouthrinses on untreated gingivitis. Oral Dis. 1997; 3: 9–18. 17. J. C. Gunsolley, A meta-analysis of sixmonth studies of antiplaque and antigingivitis agents. J. Am. Dent. Assoc. 2006; 137: 1649–1657. 18. Council on Dental Therapeutics, Guidelines for acceptance of chemotherapeutic products for the control of supragingival dental plaque

25. R. J. Genco, S. G. Grossi, A. Ho, F. Nishimura, and Y. Murayama, A proposed model linking inflammation to obesity, diabetes, and periodontal infections. J. Periodontol. 2005; 76: 2075–2084. 26. P. B. Imrey, N. W. Chilton, B. L. Pihlstrom, H. M. Proskin, A. Kingman, M. A. Listgarten, et al., Proposed guidelines for American Dental Association acceptance of products for professional, non-surgical treatment of adult periodontitis. Task Force on Design and Analysis in Dental and Oral Research. J. Periodontal. Res. 1994; 29: 348–360. 27. J. M. Goodson, A. C. Tanner, A. D. Haffajee, G. C. Sornberger, and S. S. Socransky, Patterns of progression and regression of advanced destructive periodontal disease. J. Clin. Periodontol. 1982; 9: 472–481. 28. S. S. Socransky, A. C. Tanner, J. M. Goodson, A. D. Haffajee, C. B. Walker, J. L. Ebersole, et al., An approach to the definition of periodontal disease syndromes by cluster analysis. J. Clin. Periodontol. 1982; 9: 460–471. 29. S. S. Socransky, A. D. Haffajee, M. A. Cugini, C. Smith, and R. L. Kent, Jr., Microbial complexes in subgingival plaque. J. Clin. Periodontol. 1998; 25: 134–144. 30. J. M. Goodson, J. C. Gunsolley, S. G. Grossi, P. S. Bland, J. Otomo-Corgel, F. Doherty, et al., Minocycline HCl microspheres reduce red-complex bacteria in periodontal disease therapy. J. Periodontol. 2007; 78: 1568–1579.

DISEASE TRIALS FOR DENTAL DRUG PRODUCTS 31. P. B. Imrey, Considerations in the statistical analysis of clinical trials in periodontitis. J. Clin. Periodontol. 1986; 13: 517–532. 32. P. P. Hujoel and T. A. DeRouen, Validity issues in split-mouth trials. J. Clin. Periodontol. 1992; 19: 625–627. 33. A. P. Pawlowski, A. Chen, B. M. Hacker, L. A. Mancl, R. C. Page, and F. A. Roberts, Clinical effects of scaling and root planing on untreated teeth. J. Clin. Periodontol. 2005; 32: 21–28. 34. J. M. Goodson, M. A. Cugini, R L. Kent, G. C. Armitage, C. M. Cobb, D. Fine, et al., Multicenter evaluation of tetracycline fiber therapy: II. Clinical response. J. Periodontal. Res. 1991; 26: 371–379. 35. C. L. Drisko, C. M. Cobb, W. J. Killoy, B. S. Michalowicz, B. L. Pihlstrom, R. A. Lowenguth, et al., Evaluation of periodontal treatments using controlled-release tetracycline fibers: clinical response. J. Periodontol. 1995; 66: 692–699. 36. J. D. Bader, D. A. Shugars, and A. J. Bonito, Systematic reviews of selected dental caries diagnostic and management methods. J. Dent. Educ. 2001; 53: 960–968. 37. H. Klein, C. E. Palmer, and J. W. Knutson, Studies on Dental Caries. Public Health Reports 1938; 53: 751–765. 38. A. I. Ismail, Visual and visuo-tactile detection of dental caries. J. Dent. Res. 2004; 83 Spec No C:C56–C66. 39. 39 Oral Health Surveys of the National Institute of Dental Research. Diagnostic Criteria and Procedures. NIH publication No. 91-2870 1991; 1–99. 40. R. B. D’Agostino, Sr., J. M. Massaro, and L. M. Sullivan, Non-inferiority trials: design concepts and issues—the encounters of academic consultants in statistics. Stat. Med. 2003; 22: 169–186. 41. 41 Guidelines for the acceptance of fluoridecontaining dentifrices. Council on Dental Therapeutics. J. Am. Dent. Assoc. 1985; 110: 545–547. 42. A. Hall and J. M. Girkin, A review of potential new diagnostic modalities for caries lesions. J. Dent. Res. 2004; 83 Spec No C:C89–C94. 43. P. B. Imrey and A. Kingman. Analysis of clinical trials involving non-cavitated caries lesions. J. Dent. Res. 2004; 83 Spec No C:C103–C108. 44. K. M. Hargreaves and K. Keiser, Development of new pain management strategies. J. Dent. Educ. 2002; 66: 113–121.

7

45. P. A. Moore, S. G. Boynes, E. V. Hersh, S. S. DeRossi, T. P. Sollecito, J. M. Goodson, et al., The anesthetic efficacy of 4 percent articaine 1:200,000 epinephrine: two controlled clinical trials. J. Am. Dent. Assoc. 2006; 137: 1572–1581. 46. 46 Guideline on appropriate use of local anesthesia for pediatric dental patients. Pediatr. Dent. 2005; 27 (7 Suppl):101–106. 47. 47 Guidelines for the acceptance of peroxidecontaining oral hygiene products. J. Am. Dent. Assoc. 1994; 125: 1140–1142. 48. M. Tavares, J. Stultz, M. Newman, V. Smith, R. Kent, E. Carpino, et al., Light augments tooth whitening with peroxide. J. Am. Dent. Assoc. 2003; 134: 167–175.

FURTHER READING J. L. Fleiss, The Design and Analysis of Clinical Experiments. New York: John Wiley & Sons, 1986.

CROSS-REFERENCES Surrogate Endpoints Data Mining Generalized Estimating Equations Analysis of Covariance (ANCOVA) Multiple Comparisons Data Mining Paired T-Test Two-sample T-Test Nonparametrics

DISEASE TRIALS ON REPRODUCTIVE DISEASES

creases the trust of the public in reproductive medicine and the science on which it relies. Fortunately, several moves have been employed to improve research in the area of reproductive health. The action by distinguished researchers has identified research standards and guides researchers in the development of translational and clinical research studies. Multiple issues have been identified that impair reproductive health science: a welldefined objective; diagnostic accuracy of the disease; appropriate study population; appropriate sample size; appropriate sampling and data collection; specifically developed materials and methods; appropriate randomization of treatment groups; avoidance of selection bias, test reliability and reproducibility; and appropriate index test and/or reference standards. In addition, for infertility and treatment, potential bias can exist because of previous failed or successful treatment and the use of crossover trials. Even when the ‘‘gold standard’’ of clinical trials (the prospective, RCT) is selected, serious limitations have been observed when examining treatment of infertility (4). To estimate the limitations of reproductive medicine studies, researchers tested the accuracy of studies published in two distinguished journals: Human Reproduction and Fertility and Sterility. Using the Standards for Reporting of Diagnostic Accuracy checklist (STARD), which is a checklist to test for 25 measures of diagnostic accuracy, Coppus et al. (5) assessed publications in 1999 and 2004. They found that less than one half of the studies reported at least 50% of the items demonstrating lack of accuracy with no improvement between 1999 and 2004. Unfortunately, the reporting of individual items varied widely between studies; no consistent problem with diagnostic accuracy was observed in the articles reviewed. Clearly, researchers need to consider these issues when developing their studies, and journals need to ask that all items of diagnostic accuracy be addressed in the publication. Using the Consolidated Standards for Reporting Trials (CONSORT), some improvement has occurred in the past 12 years in the

JULIA V. JOHNSON University of Vermont College of Medicine Burlington, Vermont

1

INTRODUCTION

Reproductive health issues are common for men and women, but clinical research in this area has been limited in study number and quality. The importance of these health issues is clear. To use infertility as an example of reproductive health issues, this disease affects 15% of women aged 15–44 years (1). Indeed, infertility is one of the most common medical disorders that affect reproductive aged men and women. Although highquality prospective, randomized controlled trials (RCTs) are required to determine the ideal methods for diagnosis and treatment of this common disease, the quality of research is historically poor (2). The limitation of federal funding for reproductive health issues has led to programs in the NICHD in the past 10 years (3). In the interim, most reproductive research has been funded by pharmaceutical companies and academic centers without set standards for these clinical trials. The limited research funding impairs scientific analysis of the available data, and it adversely affects provider’s ability to care for patients. This article will review the limitation of the current research in reproductive medicine and recommend the standards for researchers and clinicians to consider when reviewing the literature. Also, the current NICHD programs to optimize research in reproductive medicine will be discussed. 2

LIMITATIONS OF CURRENT STUDIES

As in all medical fields, evidence-based medicine is critical to allow effective decision making for providers. The quality of studies in the reproductive sciences has not been consistent, which limits providers’ ability to make decisions with their patients and de-

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

DISEASE TRIALS ON REPRODUCTIVE DISEASES Table 1. Use of CONSORT in Reproductive Medicine RTCs 1990 (% of Studies)

1996

2002

16% 11% 67%

40% 20% 77%

48% 37% 95%

Randomized Trial Blinded Allocation Parallel Design

reporting of RCTs published in 1990, 1996, and 2002 (4). Collecting studies from the Cochrane Menstrual Disorder and Subfertility Group (MDSG), the researcher found that ideal methods of treatment allocation (randomization), concealment of allocation (blinding), and study design (parallel design) improved (see Table 1). However, many studies did not reach the characteristics suggested by CONSORT. In addition, this study demonstrated that 40% of studies had errors in patient flow (number of subjects randomized vs. number analyzed). Unit-of-analysis errors decreased over the 12 years but continued to occur. As is commonly the case with reproductive research, the data on pregnancy outcome was still lacking. Although miscarriages were reported in 62% of studies in 2002, only 28% reported ectopic pregnancies and 44% reported multiple gestations. An excellent example of a serious study design flaw is the use of crossover trials to assess infertility treatment. As described by Khan et al. (6), crossover trials overestimate the effectiveness of treatment, compared to parallel trials, by an odds ratio of 74%. Although it is tempting to use crossover trials to the lower sample size, the outcome (pregnancy) prevents participation in the second arm of the trial. Fortunately, the crossover study design was recognized as flawed; this information presumably led to the increase in parallel trials from 1990 (67%) to 2002 (95%) (4). 3

IDEAL REPRODUCTIVE STUDY DESIGN

Excellent reviews have been written to summarize the ideal clinical study in reproductive medicine (7,8). The first consideration, as with any research project, is to identify the objective clearly and determine whether the study will be effective in testing the hypothesis. In reproductive medicine, the sample size

is critical to consider. Because the sample size significantly increases if the difference with treatment is small, multicenter trials may be required for adequate subject recruitment. The target population must also be considered carefully. In fertility trials, for example, successful treatment is based on multiple factors, which include women’s age, sperm quality, previous pregnancy or failed therapy, gynecologic disorders, and the multiple causes of infertility. The inclusion and exclusion criteria need to be established early in the design of the study to assure that the appropriate population can be recruited. Once the patient population is established, a randomized controlled trial requires an appropriate study design. The CONSORT statement should be reviewed prior to beginning the study, not just for publication of the trial. Studies demonstrate that the use of CONSORT improves the quality of studies (9). Appropriate randomization, performed immediately before treatment to minimize postrandomization withdrawals, must be blinded to the researcher and subject to prevent selection and ascertainment biases. Subjects are followed closely to assure no unintended influences, or cointervention, that may alter results. Once subjects are randomized, those who discontinue treatment or are lost to follow-up must be included in the intent-to-treat analysis of the data. Also, as noted in the example described above, parallel trials are necessary for RCTs examining the treatment of infertility. The ideal study has a fully considered objective with an appropriate sample size to test the hypothesis as well as a lack of bias through randomization of a parallel trial and an analysis of all subjects. Maximal possible blinding is advised, as possible, for subjects, investigators, and health care providers, as well as laboratory personnel.

DISEASE TRIALS ON REPRODUCTIVE DISEASES

4 IMPROVING REPRODUCTIVE MEDICINE RESEARCH The importance of reproductive health issues and the demands of high-quality clinical research are recognized. The large sample size required for these studies has led to networks, such as the Reproductive Medicine Network (3). This network allows multicenter trials that involve eight centers and a datacoordinating center. Recently, the Reproductive Medicine Network completed a landmark study that demonstrated the most effective ovulation induction agent for women with polycystic ovarian syndrome (10). The unexpected results of this well-designed study altered medical care for women with this common disorder. Additional programs include the Specialized Cooperative Center Program in Reproduction Research (SCCPRR), which was developed to increase translational research in reproductive sciences. This program encourages collaboration with other centers within the institution and with outside entities. Four SCCPRR focus groups include endometrial function, ovarian physiology, male reproduction, and neuroendocrine function. The National Cooperative Program for Infertility Research (NCPIR) has two sites that involve patient-oriented research. Currently, the NCPIR is emphasizing the genetic basis of polycystic ovarian syndrome. The addition of the Cochrane MDSG has improved the interpretation of reproductive medicine trials (11). With more than 2000 randomized controlled trials in fertility, this group allows assessment of the quality and content of the published trials. 5

CONCLUSIONS

Providers rely on high-quality clinical research to guide their practice; researchers rely on standards that optimize study design and ensure appropriate analysis of results. Most importantly, patients rely on research to determine the most effective diagnostic tests and reliable treatment options. Classically, reproductive medicine had limited funding, which resulted in suboptimal clinical studies. The factors that complicate these

3

clinical trials are now recognized, and appropriate patient selection and study design has been clarified. The use of standards such as CONSORT and STARD will improve the research development and publication significantly. In addition, the efforts from the NICHD to increase multicenter and collaborative trials will set the standard for highquality research in reproductive medicine. REFERENCES 1. J. C. Abama, A. Chandra, W. D. Mosher, et al., Fertility, family planning, and women’s health: new data from the 1995 National Survey of Family Growth. Vital Health Stat. 1997; 19: 1–114. 2. S. Daya, Methodological issues in infertility research. Best Prac. Res. Clin. Obstet. Gynecol. 2006; 20: 779–797. 3. L. V. DePaolo and P. C. Leppert, Providing research training infrastructures for clinical research in the reproductive sciences. Am. J. Obstet. Gynecol. 2002; 187: 1087–1090. 4. A. Dias, R. McNamee, and A. Vail, Evidence of improving quality of reporting of randomized controlled trials in subfertility. Hum. Reprod. 2006; 21: 2617–2627. 5. S. F. P. J. Coppus, F. van der Venn, P. M. M. Bossuyt, and B. W. J. Mol, Quality of reporting of test accuracy studies in reproductive medicine: impact of the Standards for Reporting of Diagnostic Accuracy (STARD). Fertil. Steril. 2006; 86: 1321–1329. 6. K. S. Khan, S. Daya, JA. Collins, and S. D. Walter, Empirical evidence of bias in infertility research: overestimation of treatment effect in crossover trials using pregnancy as the outcome measure. Fertil. Steril. 1996; 65: 939–945. 7. J-C. Acre, A. Nyboe Anderson, and J. Collins, Resolving methodological and clinical issues in the design of efficacy trials in assisted reproductive technologies: a mini-review. Hum. Reprod. 2005; 20: 1751–1771. 8. S. Daya, Pitfalls in the design and analysis of efficacy trials in subfertility. Hum. Reprod. 2003: 18; 1005–1009. 9. D. Moher, K. F. Schulz, and D. Altman for the CONSORT Group, The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. JAMA 2001; 285: 197–191. 10. R. S. Legro, H. X. Barnhart, W. D. Sclaff, B. R. Carr, et al., for the Cooperative Multicenter

4

DISEASE TRIALS ON REPRODUCTIVE DISEASES

Reproductive Medicine Network, Clomiphene, metformin, or both for infertility in the polycystic ovarian syndrome. N. Engl. J. Med. 2007; 356: 551–566. 11. C. M. Farquhar, A. Prentice, D. H. Barlow, J. L. H. Evers, P. Vandekerckhove, and A. Vail, Effective treatment of subfertility: introducing the Cochrane menstrual disorders and subfertility group. Hum. Reprod. 1999; 14: 1678–1683.

FURTHER READING E. G. Hughes, Randomized clinical trials: The meeting place of medical practice and clinical research. Semin. Reprod. Med. 2003; 21: 55–64. G. Piaggio and A. P. Y. Pinol, Use of the equivalence approach in reproductive health clinical trials. Stats. Med. 2001; 20: 3571–3587. K. F. Schultz, I. Chalmers, R. J. Hayes, et al., Empirical evidence of bias: dimensions of methodological quality associated with treatment effects in controlled trials. JAMA 1995; 273: 408–412. L. Roenbert, Physician-scientists: endangered and essential. Science 1999; 288: 331–332. A. Vail and E. Gardener, Common statistical error in the design and analysis of subfertility trials. Hum. Reprod. 2003; 18: 1000–1004.

DISEASE TRIALS ON PEDIATRIC PATIENTS

Researchers face unique obstacles when conducing RCTs in children, including the limited number of children with major medical diseases, issues related to ethics, and finally, measurable objective outcomes. Unlike adults, most children in the United States are healthy. Of the estimated 1,400,000 patients newly diagnosed each year with cancer, only about 10,000 are in children 0–14 years of age (2). The most common chronic disease in childhood is asthma, which affects approximately 10% of the 75,000,000 children 0 to 18 years of age. However, the vast majority of even these children have mild to moderate disease. Few seek care in emergency rooms, and even fewer are hospitalized. These examples are two examples of important childhood diseases, but they affect such limited numbers of children that conducting RCTs is difficult. Investigators in child health, therefore, frequently have to resort to multicenter trials. A good example is the Children’s Oncology Group (COG), which enrolls children around the United States in clinical trials and has been conducting multisite trials for decades (3). Virtually every child in the United States with cancer can enroll in a clinical trial. More recent examples include the Pediatric Research in Office Settings group (PROS)(4), a pediatric emergency network in the United States (5), and a clinical trials unit in the United Kingdom (6). PROS is a practice-based network, established in 1986, which includes over 700 practices in the United States, Puerto Rico, and Canada. It was established by the American Academy of Pediatrics (AAP). A research staff at the AAP helps coordinate projects and assists with identifying funding and data analysis. Research ideas are vetted through numerous committees, which include biostatisticians, epidemiologists, and clinicians. The Pediatric Emergency Care Applied Research Network (PECARN) is the first federally funded collaborative network for research in pediatric emergency medicine. The group recently published the results of a large RCT that assessed the role of dexamethasone for infants diagnosed with bronchiolitis in the emergency

HOWARD BAUCHNER Boston University School of Medicine Boston Medical Center Boston, Massachusetts

Conducting clinical trials in which children are the subjects is important. Appropriately powered randomized clinical trials (RCTs) remain the most influential study design. Some clinical trials change practice on their own; others contribute through metaanalyses. Both RCTs and meta-analyses are considered the cornerstone for the Cochrane Collaboration and the primary basis for recommendations of the United States Preventive Services Task Force. In a recent analysis, we found a large gap between high-quality study designs in children and adults. We assessed all research articles published in the New England Journal of Medicine, Journal of the American Medical Association, Annals of Internal Medicine, Pediatrics, Archives of Internal Medicine and Archives of Adolescent and Pediatric Medicine during the first 3 months of 2005 (1). 405 original research reports were reviewed, of which 189 included only adults as subjects and 181 included only children as subjects (total N = 370). Both RCTs and systematic reviews were significantly more common in studies that involved adults compared with those that involved children. Of the 370 trials, 32.6% were RCTs, 23.8% involving adults and 8.8% involving children. Of the 12.3% of studies that were systematic reviews, 10.6% involved adults and 1.7% involved children. Cross-sectional studies (considered less robust designs) were twice as common in the pediatric literature (38.1% vs. 17.7%). Unlike large RCTs, in which both known and unknown biases are usually equally distributed between groups, in cross-sectional designs, confounding is of major concern. This survey highlights the concern about lack of high-quality study designs—RCTs and systematic reviews—in child health research but begs the question why this gap exists.

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

DISEASE TRIALS ON PEDIATRIC PATIENTS

room setting (5). Twenty emergency rooms were involved in this study. The United Kingdom network, Medicines for Children Research Network (MCRN), was created in 2005 because of the concern about lack of high-quality clinical trials in children. The unit is based in Liverpool, but six local research networks are established, covering most of England, and nine clinical studies groups exist, too, including anaesthesia, diabetes, neonatal, neurosciences, and respiratory. The goal is to develop and conduct RCTs that will enroll children from around the United Kingdom. Research consortia like COG, PROS, PECARN, and MCRN are critical to the future health of children. However, unlike single-site trials, multisite trials are far more difficult to conduct because of issues related to expense, reliability of data, and ethics committee approval. Ethical issues are an important consideration when conducting research with children (7). According to federal regulation, children are considered a vulnerable population and, as such, are afforded additional protections by Institutional Review Boards. For example, children can participate in research with greater than minimal risk only as long as they are likely to benefit from research. The risk/benefit ratio must be perceived at least as favorable as available alternatives. The issue of when children, rather than their parents, should give consent is also complicated. When considering whether children are capable of consenting, IRBs consider age, maturity, and psychological state of the children. The adolescent years pose another potential problem for researchers. Determining when adolescents become ‘‘adults’’ and can consent to participate in research without their parents being present or contacted is another complicated issue that can vary from state to state. A final consideration in conducting trials in children is appropriate health outcomes. Often trials with adults use as objective outcomes, death, hospitalization, or major medical problems, such as myocardial infarction or stroke. These outcomes are quite rare in children, so the outcomes used in many pediatric trials relate to functional measures of performance, including cognitive outcomes and physical activity. These outcomes are

affected by many variables, so ensuring adequate sample sizes in RCTs that involve children is critical. In addition, many other variables can modify the results (effect modifiers). In addition, many outcomes that are important to child health researchers occur in the adolescent and adult years. For example, researchers in childhood cancer are particularly interested in rates of secondary cancer. Many of these occur 20–30 years after initial treatment, hence the need for longterm follow-up studies. The same is true for Type 1 diabetes and cystic fibrosis, cases in which investigators are interested in complications and/or other outcomes that occur in the adult years. Many problems in child health necessitate long-term follow-up, which is both expensive and logistically difficult. Clinical trials improve health care. However, the number of clinical trials that involve children is limited, with researchers facing a unique set of obstacles. However, I remain optimistic that the development of research networks and the growing recognition that we need to conduct more clinical trials that involve children will lead to a growing number of RCTs. REFERENCES 1. C. Martinez-Castaldi, M. Silverstein, and H. Bauchner, Child versus adult research: The gap in high quality study design. Abstract, Pediatric Academic Societies Meeting, May 2006. Available: http://www.abstracts2view. com/pasall/search.php?query=Bauchner&where []=authors&intMaxHits=10&search=do. 2. http://www.cancer.org/downloads/STT/CAFF 2007PWSecured.pdf. 3. http://www.curesearch.org/. 4. http://www.aap.org/pros/abtpros.htm. 5. H. M. Corneli et al., A multicentre, randomized, controlled trial of dexamethasone for bronchiolitis. NEJM. 2007; 357: 331–339. Available: http://ctuprod.liv.ac.uk/mcrn/library/docs/ MCRNCC%202005-06.doc. 6. http://www.mcrn/org.uk. 7. http://www.bu.edu/crtimes/.

DNA BANK

research applications. Because DNA collected at a particular time can survive for a long time, the scope of use of the specimen may be far broader than specified at the time of collection.

THERU A SIVAKUMARAN Department of Epidemiology and Biostatistics, Case Western Reserve University, Cleveland, Ohio

1.1 Common or Complex Diseases Modest-sized genetic studies have been successful in the identification of single genes that, when mutated, cause rare, highly heritable mendelian disorders. However, studies that are aimed at identifying genes responsible for predisposition to common diseases, such as heart disease, stroke, diabetes, agerelated macular degeneration, cancer, and so on, are challenging. Common diseases are thought to be caused by many modest-sized, often additive effects, which represent the outcome of genetic predisposition, as well as lifestyle factors such as diet and smoking habits, and the environment in which we live and work. Locating these complex interactions between genes and the environment, and their role in disease depends on the collection of well-documented epidemiological clinical information, as well as biological specimens. The concept of obtaining and maintaining prospective cohorts to understand health and disease has a precedent in epidemiological research, with the investment in studies such as the Framingham Heart Study (3), but this paradigm has only recently been broadened to genetic epidemiology investigations. In response to the large sample sizes needed to obtain statistical power to detect the predisposing genes, as well as the recent technological developments in identifying the genetic variants, DNA biobanks are growing from small, local biological repositories to large population-based collections.

SUDHA K IYENGAR Departments of Epidemiology and Biostatistics, Genetics, and Ophthalmology, Case Western Reserve University, Cleveland, Ohio

1 DEFINITION AND OBJECTIVES OF DNA BIOBANKS DNA bank, also known as DNA biobank or Biobank, is an organized collection of DNA or biological samples, such as blood plasma, and so on, that are used to isolate DNA. These warehouses also contain information about the donor of the material, such as demographic characteristics, the type of disease associated with the sample, the outcome of the disease, treatment, and so on. According the American Society of Human Genetics (ASHG) policy statement (1), a DNA bank is defined as a facility to store DNA for future analysis. The American National Bioethics Advisory commission defined a DNA bank as a facility that stores extracted DNA, transformed cell lines, frozen blood or other tissue or biological materials for future DNA analysis (2). Although many types of DNA banks are available (described below), this article focuses mainly on the DNA banks that are meant for human genome research (i.e., academic laboratory DNA banks, populationbased public DNA biobanks, and commercial biobanks). Population-based public or national DNA biobanks represent a new paradigm for biomedical research and differ from traditional academic laboratory DNA banks in several aspects. The public or commercial DNA biobanks that obtain samples may not be engaged in research but may be only intermediary brokers who supply specimens to other researchers. They act as repositories that can be used for many

1.2 Pharmacogenomics Medications prescribed by a physician can have associated side effects called adverse drug reactions in some individuals. Approximately 100,000 people die each year from adverse reactions to drugs (4), and millions of people must bear uncomfortable or even dangerous side effects. Currently, no simple

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

DNA BANK

method is available to determine whether people will respond well, badly, or not at all to a particular drug. Therefore, all pharmaceutical companies follow a ‘‘one size fits all’’ system, which allows for development of drugs to which the ‘‘average’’ patient will respond. Once a drug is administered, it is absorbed and distributed to its site of action, where it interacts with numerous proteins, such as carrier proteins, transporters, metabolizing enzymes, and multiple types of receptors. These proteins determine drug absorption, distribution, excretion, target site of action, and pharmacological response. Moreover, drugs can also trigger downstream secondary events that may vary among patients, for example, seldane (terfenadine), which is an antihistamine used to treat allergies, hives (urticaria), and other allergic inflammatory conditions, was withdrawn because it caused increased risk for long QT syndrome and other heart rhythm abnormalities. Similarly, Vioxx (Merck & Co., Whitehouse Station, NJ), which is a drug prescribed to relieve signs and symptoms of arthritis and acute pain in adults, as well as painful menstrual cycles, has also been removed from the market because of an increased risk for heart attack and stroke. Thus, the way a person responds to a drug is a complex trait that is influenced by many different genes that code for these proteins. Without knowing all of the genes involved in drug response, scientists have found it difficult to develop genetic tests that could predict a person’s response to a particular drug. Pharmacogenomics is a science that examines the inherited variations in genes that dictate drug response and explores the ways these variations can be used to predict whether a patient will have a good response to a drug, a bad response to a drug, or no response at all. The concept of pharmacogenetics, which is a discipline that assesses the genetic basis of drug response and toxicity, originated from the clinical observation that many patients had very high or very low plasma or urinary drug concentrations, followed by the realization that the biochemical traits that lead to this variation were inherited. The studies conducted in early 1950s examine the drug metabolizing enzyme variants in

the cytochrome P450 family. The cytochrome P450 monooxygenase system is responsible for a major portion of drug metabolism in humans. This large family of genes has been intensely studied, and among the numerous subtypes, CYP2D6, 3A4/3A5, 1A2, 2E1, 2C9, and 2C19 play particularly critical roles in genetically determined responses to a broad spectrum of drugs (5). Because drugs that act on the central nervous system penetrate the blood-brain barrier, renal excretion is minimal for these compounds, and cytochrome P450 metabolism, particularly CYP2D6 and CYP2C19, provides the only means of effective drug elimination. The activity of the CYP2D6 enzyme is extremely variable because of more than 50 genetic variants, and patients who are homozygous for the CYP2D6 null alleles have impaired degradation and excretion of many drugs, which include debrisoquine, metoprolol, nortriptyline, and propafone (6). These patients are termed ‘‘poor metabolizers’’ for CYP2D6 selective drugs, and they are more likely to exhibit adverse drug reactions. Similarly, patients who are homozygous for the ‘‘null’’ allele of the P450 isoform CYP2C19 are highly sensitive to omeprazole, diazepam, propranolol, mephenytoin, amitriptyline, hexobarbital, and other drugs. Today, clinical trial researchers use genetic tests for variations in cytochrome P450 genes to screen and monitor patients prior to drug administration. 1.3 Finding-Causative Genetic Factors The process of identifying genes that are responsible for common disease as well as drug response begins with scanning the genetic variations in the entire DNA sequence of many individuals with the disease, or those who respond to a particular drug, and contrasting this information with that from individuals without the disease, or who do not respond to the drug. The most commonly found human genetic variations between two individuals are variants that involve a single nucleotide, named single nucleotide polymorphisms (SNPs). In comparing two haploid genomes, a SNP was observed to occur on average every 1331 bases. When more than two haploid genomes are surveyed, a SNP

DNA BANK

is expected to occur on average every 240 bases (7). In the general population, the collective human genome is expected to contain about 10 million SNPs, and testing all of them would be very expensive. Systematic studies that identify these common genetic variants are facilitated by the fact that it is not necessary to interrogate every possible variant. The International HapMap Project (8) showed that a particular SNP allele at one site often carries information about specific alleles at other nearby variant sites; this association is known as linkage disequilibrium (LD). Thus, genetic variants that are located close to each other tend to be inherited together; these variants are often labeled as a haplotype or LD block. Because little or no recombination is observed within this haplotype block, a small subset of SNPs, called tag SNPs, are sufficient to distinguish each pair of patterns in the block and this reduces the necessity of querying all 10 million SNPs. Approximately 1 million common SNPs carry most information on common variants. Examination of these SNPs by traditional methods, such as sequencing, is very slow and expensive, but DNA microarray technology has made it possible to genotype large numbers of SNPs in a single array/chip. In the past few years, the number of SNPs available on a single microarray chip has grown from 10,000 to 1,000,000. Once a particular area of the genome is associated with disease is identified through scanning thousands of tag SNPs, it should be possible to zero in on specific genes involved in the disease process much more quickly. This process occurs because when a new causal variant develops through mutation, it is initially tethered to a unique chromosomal region that is marked by a haplotype block. If a disease is associated with a tag SNP within that particular haplotype block, then variants that contribute to the disease might be located somewhere within or near that haplotype block. In summary, it is feasible to use surrogate markers to locate disease genes or to find genes that affect the action of pharmaceutical compounds. Biobanks play a significant role in this discovery process.

2

3

TYPES OF DNA BIOBANKS

Several kinds of DNA banks are based on usage and location. 2.1 Academic DNA Banks Repositories are housed in the laboratories of researchers who study one or more genetic disorders. This collection typically contains DNA samples obtained from families at risk for the disease of interest or from cases with certain common disease, such as heart disease, diabetes, cancer, and so on, as well as samples from healthy controls. One of the first academic biobanks was the Centre d’etude du Polymorphism Humaine (CEPH) (9), which facilitated human gene mapping and was the nexus for the Human Genome Project and the International HapMap Project. Unlike most other collections, the CEPH collection was initially assembled with no information on disease. 2.2 Population-Based DNA Biobanks These centers are large private/public repositories of donated human DNA, with health and other relevant information collected from volunteers with and without the disease. The aim of public initiatives, which are supported by local governments, is to provide specimens to the scientific community for research geared toward identifying genes that contribute to common disease as well as to drug response and toxicity. The volunteers for these biobanks are approached mainly through their health care providers. Most pharmaceutical companies have their own large collections of DNA samples acquired through research collaborations or from subjects enrolled in clinical trials (10). 2.3 Commercial Biobanks Several commercial biobanks have been established in the last few years with the goal of storing personal genetic information and providing this information to researchers, as well as to pharmaceutical companies, for-a-fee. Some commercial biobanks in the United States are Genomics Collaborative, Inc., Ardais Corporation, and DNA Sciences Inc. (11). Genomics Collaborative Inc. claims

4

DNA BANK

to have a repository that contains about 500,000 biological samples and also health data from 120,000 people from all over the world. Ardais Corporation, in agreement with several physicians, recruits patients to donate samples, and the repository contains more than 220,000 tissue samples collected from over 15,000 donors. DNA Sciences Inc. has a collection of over 18,000 DNA samples in its repository (11). 2.4 State Forensic DNA Data Banks DNA profiling and databases have become common in criminal investigation and prosecution. Forensic DNA data banks serve as repositories for long-term storage of DNA collected from certain defined criminal offenders and the profiles derived from analysis of the DNA. Most states in the United States permit DNA profile databasing of offenders, of missing persons and their relatives, and of DNA profiles from criminal-case evidence in which the depositor is unknown. The national DNA database of the United Kingdom also stores DNA samples and other specimens collected from crime scenes as well as samples taken from individuals in police custody. 2.5 Military DNA Data Banks Military DNA banks assist in the identification of human remains. The U.S. military uses DNA profiles in place of traditional means of identification such as ‘‘dog tags,’’ and new recruits routinely supply blood and saliva samples used to identify them in case they are killed in action. 2.6 Nonhuman DNA Banks DNA banking has become an important resource in worldwide efforts to address the biodiversity crisis, manage the world’s genetic resources, and maximize their potential. Several plant and animal DNA banks are located around the world. A Japan-based National Institute of Agrobiological sciences’ DNA Bank was established in 1994 as a section of Ministry of Agriculture, Forestry, and Fisheries. It maintains the DNA materials and genomic information that were collected as part of rice, animal, and silkworm genome projects (12). Royal Botanic

Gardens (RBG) at Kew established a DNA bank database with the aim of extracting DNA from all plant species that are grown at RBG. Currently, it has genomic DNA from over 22,000 plants in its facility, which are stored at −80◦ C, and these samples are sent to collaborators all over the world for research purposes (13). The Australian plant DNA bank has a comprehensive collection of DNA from both Australian native and important crop plant species in its repository. It provides DNA samples and the associated data to the research community (14). Recently, Cornell University’s College of veterinary medicine has established a DNA bank called Cornell Medical Genetic Archive to understand the genetic basis of disease in many species such as dogs, cats, horses, cows, and exotic animals. The blood samples taken from these species, with the owner’s written consent, will be used for DNA extraction, and these samples will be used by the researchers at Cornell (15). In summary, DNA banks can be speciesspecific, and their use is tailored to particular needs. In general, human biobanks have been established to (1) study a specific genetic disease under collaborative guidelines; (2) to accrue information from populations of entire countries (see below) for current surveillance or for future use; (3) in pharmacogenetic studies, as part of clinical trials; and (4) in hospital based settings to be used for current and future research.

3 TYPES OF SAMPLES STORED

A variety of biological materials is now banked for isolating genomic DNA for genetic research. All nucleated cells, which include cells from the hair follicle, buccal swab, saliva, and urine, are suitable for isolating DNA. Many large biobanks obtain whole blood, as it provides amounts of DNA necessary for current applications. Whole blood is generally stored in three different forms in the biobank (i.e., genomic DNA, dried blood spots, and Epstein-Barr virus (EBV)transformed cell lines).

DNA BANK

3.1 Genomic DNA Blood is most often collected using ethylenediaminetetraacetic acid (EDTA) or anticoagulants that include heparin and acid citrate dextrose and genomic DNA is isolated using homebrewn methods or commercially available kits. The purified genomic DNA is stored at 4◦ C or −20◦ C. Most samples stored in the biobanks are isolated genomic DNA (16). Often, an EDTA biospecimen can also serve as a source of material for other bioassays, as this type of specimen is suitable for a variety of biochemical, hematologic, and other types of tests (17).

5

EBV-transformed lymphoid cells are stored at even lower temperatures. Some factors that generally affect the quality or integrity of DNA samples are the following: 1. Improper storage conditions, such as temperature, evaporation, frequent freeze, and thaw, may lead to the degradation of samples. 2. Possibility of cross contamination of neighboring samples either by sample carry over during handling or by sample switches during processing. 3. Improper handling of DNA samples (i.e., improper mixing and pipetting).

3.2 Dried Blood Spots Dried blood spots, which are also known as Guthrie cards, are generally used in newborn screening programs to quantify the analytes that include metabolic products, enzymes, and hormones (18). Blood spots from military personnel are also stored to serve as biologic ‘‘dog tags’’ for identification purposes. Blood spots may yield enough DNA to genotype several variants (19,20), although with current genotyping technology it is feasible to genotype thousands of markers. The United Kingdom’s national biobank is planning to spot whole blood onto filter paper in a 384well configuration for storage (10). 3.3 EBV Transformed Cell Lines EBV transformed lymphocytes provide an unlimited source of DNA, to be used for whole genome scans. These cells are also used for functional studies. (See Cell Line for additional details). 4 QUALITY ASSURANCE AND QUALITY CONTROL IN DNA BIOBANKS The quality of the DNA and other biological specimens in biobanks is of primary importance, as they are the resource for the downstream processes such as genotyping, sequencing, and other bioassays. The storage of these specimens varies according the type of samples, that is, DNA samples are generally stored at 4◦ C and −20◦ C; blood and other tissue samples used for DNA extraction are stored in −80◦ C; and viable cells such as

The ASHG policy statement on DNA banking (1) proposed minimal standards for quality assurance, which include: (1) A DNA bank should occupy space equipped with an alarm system that is separate from other functions; (2) the bank should maintain a manual of procedures and train personnel in meticulous technique; (3) samples should be coded and very few individuals have access to the identity; (4) records should be maintained for the receipt, disposition, and storage of each sample; (5) samples should be divided and stored in more than once place; and (6) control samples should analyzed before deposit and at periodic intervals to demonstrate that DNA profiles are unaffected by storage. To ensure the quality and integrity of biological specimens stored over a long term, biobanks follow standard procedures at every step, such as collection, transport, storage, and handling. Some procedures that are followed to ensure the quality of samples include: • Proper labeling of samples and the asso-

ciated data at the time of collection and during downstream processes. • All samples are barcoded and scanned at every step during procedures. Storage units are highly organized and compartmentalized, in which every sample occupies a unique and computer-tracked location. • Measures are taken to ensure the safety of the samples as well as to prevent damage caused by disasters, such as

6

DNA BANK

•

•

•

•

availability of backup power and backup cooling systems in case of power failure. Conducting DNA profiling of each sample and generating an extensive database of signature profiles prior to storage for later identification. Storing samples in conditions that protect them from degradation and following procedures to maintain their integrity caused by frequent freeze and thaw. Performing periodic maintenance checks to ensure sample traceability and integrity. This process is usually done by DNA profiling samples and comparing the results with the original profiles stored in a database. Finally, the clinical information that accompanies the samples and other relevant information is stored in computer systems that are run on complex database management and analysis software.

With proliferation in the number of samples stored in biobanks and the increased need to access more samples quickly and accurately, researchers started using automated systems for sample storage and retrieval (10). These automated sample management systems dramatically increase the throughput and eliminate manual errors. Some automated biological management systems are the DNA archive and reformatting (DART) system that is used by AstraZeneca, the Polar system developed by The Automated Partnership, and the RTS Life Science’s automated sample management systems. The sample storage and use of DART system at AztraZeneca is controlled by a Laboratory Information Management System. The DART system can store over 400,000 tubes and can handle over 650 tubes per hour. A complete description of the automation of sample management can be observed elsewhere (10,21). 5

ETHICAL ISSUES

Biobanks cause concerns about several technical, ethical, and legal challenges. In the

realm of ethics, these concerns revolve primarily around how donors of biological material or data will be assured that their privacy and interests will be protected, given the increasing number of large for-profit companies and entirely private DNA biobanks. 5.1 Informed Consent Informed consent is a key ethical standard required for clinical research, and its underlying principle is respect for the autonomy of individuals. The codes of ethical practice, such as the Declaration of Helsinki (22), state that a research participant must be provided the opportunity for voluntary informed consent before a study begins. In the United States, the rights and welfare of the human subjects in genetic research is protected by institutional review boards (IRB). The IRB is an independent ethics committee that is governed by principles established by the Office of Human Research Protections in the Department of Health and Human Services (23), and it is located at the institution where the research is conducted. IRBs are also run by commercial enterprises to assist governmental and industrial bodies in regulating research. In general, informed consent is requested using a special form approved by the IRB. All aspects relative to the handling of the preserved material, as well as the data, must be written clearly in the consent form. Elements of a traditional model of informed consent include the following: • The document should explain the pro-

posed research, its purpose, as well as the duration, procedures of the study, and description of potential risks. • The document should state that the sample will be used exclusively for diagnostic or research purpose and never for direct profit. If profit is the goal, then the beneficiaries should be clearly listed. • The potential benefits derived from the use of these samples to the individual and/or the entire community should be explained in lay terms. • The document should clarify the procedure for handling the data to ensure anonymity as well as the protection of

DNA BANK

confidentiality of the sample and its related investigations. • The informed consent document should state whom to contact for questions about the subjects’ rights or about studyspecific questions. • Finally, information on how to withdraw consent should be outlined, with provisions for immediate withdrawal of samples and participation. Although the principle of informed consent is recognized, its translation into the biobanks that store samples and data for long-term use, and provide this data to any entity with appropriate permissions (such as an IRB) on a fee-for-service basis, leads to many practical difficulties (24–26). Some of these difficulties include: (1) potential participants can only be given information about the sort of research that will be performed with the present knowledge, but it would be difficult to describe all the types of research that would be feasible at a later date; (2) at the time of recruitment, it is not possible to describe the information that will be subsequently collected from the volunteers because they would not know which disease they are going to develop (27); (3) it is not possible to give information about the research or research teams who will be using the samples; and (4) it is not possible to give information back to the participant on their risk profile if the research is being conducted by a third party. Because of these difficulties, no clear international guidelines on informed consent for biobanks are available. Several models of informed consent are found in the literature (27–29), and the common recommendation proposed by the United Nations Educational, Scientific and Cultural Organization (UNESCO) International Bioethics Committee is called ‘‘Blanket consent.’’ Herein, research participants can consent once about the future use their biological samples (30). The U.K. Biobank and the Estonian Biobank use the blanket approach (29,27), whereas the Icelandic Biobank uses two different consent forms: Form ‘‘A’’ authorizes the use of samples for specific research, and form ‘‘B’’ authorizes the use of samples for specific research and for

7

additional similar research if approved by National Bioethics Committee (31) without further contact with the participant. 5.2 Confidentiality The term confidentiality is defined as the safety of identifiable information that an individual discloses with the expectation that it will not be provided to third parties (23). The possibility of tracing the person from whom sample and data were derived varies according to how samples are linked to their donor identity. In general, the labeling of a sample can range from irreversible anonymity to complete identification based on the choices expressed in the written informed consent form. 5.3 Anonymous Samples are identified only by a code from the start, and all identifying links to the donors are destroyed at the time of collection. Therefore, it is not possible to link the sample to a given person, and this mechanism offers the most protection of confidentiality. 5.4 Anonymized The patient’s personal data are removed after a code is assigned, after which it is no longer possible to connect the two. 5.5 Double-Coding A anonymous number is assigned to the participants as well as to the samples, and a code that links the anonymous number and the personal information is placed is secure location, which is accessible to the person in charge of the biobank and his/her immediate colleagues. 5.6 Identifiable The identity of individuals is directly linked to the samples or data. This option is only possible on explicit request by the interested party and in any case only for exclusive personal and family use.

8

6

DNA BANK

CURRENT BIOBANK INITIATIVES

Centralized population-based biobanks have been established in at least eight countries, which include Iceland, the United Kingdom, Canada, Japan, Sweden, Latvia, Singapore, and Estonia. Population homogeneity and a strong interest in genealogy made for the establishment of the world’s first populationbased biobank in Iceland. deCODE Genetics, which is a private company, successfully partnered with the Icelandic parliament in 1998 to create and operate a centralized database of nonidentifiable health data (32). They aimed to enroll most of the country’s 270,000 citizens and currently have genetic samples from more than 100,000 volunteers (10,33). deCODE links data from the Iceland Health Database as well as public genealogical records. In Canada, three population-based biobank initiatives are in various stages of development. One project is CARTaGENE, which is a publicly funded biobank in Quebec. In the first phase, it aims to recruit a random sample of about 20,000 adults between 45 to 69 years, which represents about 1% of its citizens of this age group in the selected regions of Quebec. This resource will be available to researchers who are interested in conducting population-based genomic research (34). Two other population-based genetic biobank projects planned in Canada are the national birth control cohort and a longitudinal study on aging. The U.K. Biobank was started in August 2006 with the goal to study the separate and combined effects of genetic and nongenetic risk factors in the development of multifactorial diseases of adult life. This program plans to involve 0.5 million people 45–69 years of age and intends to collect environmental and lifestyle data, as well as information from medical records, along with biological samples. The center will also coordinate activities of the six scientific regional collaborating centers; each is a consortium of academic and research institutions responsible locally for recruitment and collection of data and samples. This biobank will serve as a major resource that can support a diverse range of research intended to improve the prevention, diagnosis, and treatment of illness and the promotion of health throughout society (35).

A genome project set up by the tiny European country of Estonia, founded in 2001, is accelerating plans to take blood samples from three quarters (i.e., 1 million of its 1.4 million) of its population and promises to be the biggest such initiative. This project aims not only to enable research on the genetic and nongenetic components of common disease, but also to create a biological description of a large and representative sample of the Estonian population (36). The Latvian Genome Project, which was launched in 2002, is a unified national network of genetic information and data processing to collect representative amounts of genetic material for genotyping of the Latvian population and to compare genomic data with clinical and pedigree information. This project is planned for a period of 10 years, with an expected sample size of 60,000 in its pilot phase (37). The Swedish National Biobank is estimated to house about 50–100 million human samples, which increases at the rate of 3–4 million samples per year (38). GenomEUtwin is a project that aims to analyze twin and general cohorts to determine the influence of genetic and nongenetic factors that cause predisposition to obesity, stature, coronary heart disease, stroke, and longevity, and to create synergies in genetic epidemiology. The implementation is coordinated by the Finland National Public Health Institute and University of Helsinki and it builds on existing twin cohorts from a few other European countries (39). The Biobank Japan project, which was established in 2003 with the support of the Japanese government, plans to collect DNA, sera, and clinical information from 300,000 patients in participating hospitals (40). Genetic Repositories Australia is a central national facility for establishing, distributing, and maintaining the long-term secure storage of human genetic samples. They provide Australian medical researchers with DNA and cell lines, as well as associated clinical information, collected from patients and controls (41). China has also launched its first biobank program, called ‘‘The Guangzhou Biobank Cohort Study,’’ with the aim of creating profiles on about 11,000 Guangzhou people aged above 50 years in the first phase (42).

DNA BANK

Several biobanks are located in the U.S., and one of the largest populationbased biobanks is the personalized medicine research project of the Marshfield Medical Clinic in Wisconsin. The investigators of this project are planning to enroll at least 100,000 people who live in northern and central Wisconsin and make the samples, as well as data, available to other researchers. Currently, this repository contains information from 40,000 participants (43,44). The Center for Genetic Medicine of Northwestern University also initiated the first hospital DNA biobank, and it plans to collect DNA samples with associated clinical healthcare information from 100,000 volunteers who receive their healthcare at Northwesternaffiliated hospitals and clinics (45). The DNA bank and tissue repository at the Center for Human Genetics at Duke University is one of the oldest academic DNA banks in the United States, and it contains samples of more than 127,500 individuals (46). Howard University is also planning to establish a large DNA and health database on individuals of African American decent. They aim to enroll about 25,000 volunteers over 5 years and use the data to study the genetic and lifestyle factors that contribute to common diseases (47). Children’s Hospital of Philadelphia has launched a biobank of Children’s DNA with the aim to collect DNA from 100,000 children and use it to study common childhood diseases such as asthma, diabetes, obesity, and so on. They plan to create a database of children’s genetic profiles, which hospital researchers can use to develop diagnostic tests and drugs (48). The U.S. Department of Veterans Affairs has also proposed a national gene bank that would link DNA donated by up to 7 million veterans and their family members with anonymous medical records (49). Other similar initiatives are being implemented in many parts of the United States, but no centralized federally mandated repository is active. 7

CONCLUSIONS

Biobanking initiatives have been globally embraced because the value of these enterprises in solving the problem of emerging chronic diseases has been recognized.

9

However, translating the basic knowledge obtained from specific gene variants to the clinic remains the current challenge. The current pipeline to pharmaceutical drug discovery is very inefficient, and in general, the yield has been disappointingly low. Although, it was initially hoped that the Human Genome Project and the International HapMap Project would accelerate drug discovery, this phenomenon did not follow the hypothesized pace because discovery of variants in genes and drug targets do not follow parallel paths. The first has been the domain of academic scientists interested in the biological mechanisms of disease, whereas the latter, drug discovery and chemical screening, has been the province of the pharmaceutical industry, each with its respective governance and pipeline. In the past decade, there has been a proliferation in the size of biobanks and the clinical data collected by academic and industry researchers alike, all with the hope that the superior sample sizes, and revolutions in technology will enable them to develop better models for disease and health. REFERENCES 1. ASHG Ad Hoc Committee on DNA technology, DNA banking and DNA analysis—points to consider. Am. J. Hum. Genet. 1988; 42: 781–783. 2. National Bioethics Advisory Commission, Research Involving Human Biological Materials: Ethical Issues and Policy Guidance. 1999. National Bioethics Advisory Commission, Rockville, MD. 3. Framingham Heart Study. Available at: http://www.framinghamheartstudy.org/. 4. J. Lazarou, B. H. Pomeranz, and P. N. Corey, Incidence of adverse drug reactions in hospitalized patients—a meta-analysis of prospective studies. JAMA. 1998; 279: 1200–1205. 5. W. W. Weber, Pharmacogenetics. New York: Oxford University Press, 1997. 6. F. Broly, A. Gaedigk, M. Heim, M. Eichelbaum, K. Morike, and U. A. Meyer, Debrisoquine sparteine hydroxylation genotype and phenotype— analysis of common mutations and alleles of Cyp2D6 in a european population. DNA Cell. Biol. 1991; 10: 545–558. 7. The International SNP Map Working Group. Nature 2001; 409: 928–933.

10

DNA BANK

8. The International HapMap Consortium. The International HapMap Project. Nature 2003; 426: 789–796. 9. Centre d’etude du Polymorphism Humaine (CEPH). Available at http://www.cephb.fr/. 10. Thornton, M., Gladwin, A., Payne, R., Moore, R., Cresswell, C., McKechnie, D., Kelly, S., and March, R. Automation and validation of DNA-banking systems. Drug Discov. Today 2005; 10: 1369–1375. 11. J. Kaiser, Biobanks—Private biobanks spark ethical concerns. Science 2002; 298: 1160. 12. NIAS DNA Bank. Available at: http://www. dna.affrc.go.jp/about/. 13. Royal Botanic Gardens, Kew: Plant DNA Bank database. Available at: http://www. kew.org/data/dnaBank/homepage.html. 14. The Australian Plant DNA Bank. Available at: https://www.dnabank.com.au/. 15. The Cornell Medical Genetic Archive. Available at; http://www.vet.cornell.edu/research/ DNABank/intro.htm. 16. I. Hirtzlin, C. Dubreuil, N. Preaubert, J. Duchier, B. Jansen, J. Simon, P. L. de Faria, A. Perez-Lezaun, B. Visser, G. D. Williams, and A. Cambon-Thomsen, An empirical survey on biobanking of human genetic material and data in six EU countries. Eur. J. Hum. Genet. 2003; 11: 475–488. 17. S. Clark, L. D. Youngman, A. Palmer, S. Parish, R. Peto, and R. Collins, Stability of plasma analytes after delayed separation of whole blood: implications for epidemiological studies. Int. J. Epidemiol. 2003; 32: 125–130. 18. E. R. B. Mccabe, S. Z. Huang, W. K. Seltzer, and M. L. Law, DNA microextraction from dried blood spots on filter-paper blotters—potential applications to newborn screening. Hum. Genet. 1987; 75: 213–216. 19. J. V. Mei, J. R. Alexander, B. W. Adam, and W. H. Hannon, Use of filter paper for the collection and analysis of human whole blood specimens. J. Nutr. 2001; 131: 1631S–1636S. 20. K. Steinberg, J. Beck, D. Nickerson, M. Garcia-Closas, M. Gallagher, M. Caggana, Y. Reid, M. Cosentino, J. Ji, D. Johnson, R. B. Hayes, M. Earley, F. Lorey, H. Hannon, M. J. Khoury, and E. Sampson, DNA banking for epidemiologic studies: A review of current practices. Epidemiology 2002; 13: 246–254. 21. S. Mahan, K. G. Ardlie, K. F. Krenitsky, G. Walsh, and G. Clough, Collaborative design for automated DNA storage that allows for rapid, accurate, large-scale studies. Assay Drug Devel. Technol. 2004; 2: 683–689.

22. World Medical Association, Declaration of Helsinki: Ethical principles for medical research involving human subjects. 2000. http://www.wma.net/e/policy/pdf/17c.pdf. 23. IRB Guidebook, Office of Human Research Protections, Department of Health and Human Services. Available at: http://www. hhs.gov/ohrp/irb/irb guidebook.htm. 24. A. Cambon-Thomsen, Science and society—The social and ethical issues of post-genomic human biobanks. Nat. Rev. Genet. 2004; 5: 866–873. 25. M. Deschenes, G. Cardinal, B. M. Knoppers, and K. C. Glass, Human genetic research, DNA banking and consent: a question of ‘form’? Clin. Genet. 2001; 59: 221–239. 26. B. Godard, J. Schmidtke, J. J. Cassiman, and S. Ayme, Data storage and DNA banking for biomedical research: informed consent, confidentiality, quality issues, ownership, return of benefits. A professional perspective. Eur. J. Hum. Genet. 2003; 11: S88–S122. 27. D. Shickle, The consent problem within DNA biobanks. Stud. Hist. Phil. Biol & Biomed. Sci. 2006; 37: 503–519. 28. M. G. Hansson, J. Dillner, C. R. Bartrarn, J. A. Carlson, and G. Helgesson, Should donors be allowed to give broad consent to future biobank research? Lancet Oncol. 2006; 7: 266–269. 29. K. J. Maschke, Navigating an ethical patchwork—human gene banks. Nat. Biotechnol. 2005; 23: 539–545. 30. UNESCO. Human genetic data: Preliminary study by the IBC on its collection, processing, storage and use. 2002. http://portal.unesco. org/shs/en/files/2138/10563744931Rapfinal gendata en.pdf/Rapfinal gendata en.pdf. 31. V. Arnason, Coding and consent: moral challenges of the database project in Iceland. Bioethics 2004; 18: 27–49. 32. M.A. Austin, S. Harding, and C. McElroy, Genebanks: a comparison of eight proposed international genetic databases. Community Genet. 2007; 6: 500–502. 33. deCODE Genetics. Available at: http://www. decode.com/biorepository/index.php. 34. CARTaGENE project. Available at http:// www.cartagene.qc.ca/accueil/index.asp. 35. The UK Biobank. Available at http://www. ukbiobank.ac.uk/about/what.php. 36. The Estonian Genome Project. Available at http://www.geenivaramu.ee/index.php?lang= eng&sub=58.

DNA BANK 37. The Latvian Genome Project. Available at http://bmc.biomed.lu.lv/gene/. 38. Swedish National Biobank Program. Available at http://www.biobanks.se/. 39. GenomeEUtwin. Available at http://www. genomeutwin.org/index.htm. 40. Nakamura, Y. Biobank Japan Project: Towards personalised medicine. 2007. http:// hgm2007.hugo-international.org/Abstracts/ Publish/Plenaries/Plenary01/hgm02.html. 41. Genetic Repositories Australia. Available at http://www.powmri.edu.au/GRA.htm. 42. C. Q. Jiang, G. N. Thomas, T. H. Lam, C. M. Schooling, W. S. Zhang, X. Q. Lao, R. Adab, B. Liu, G. M. Leung, and K. K. Cheng, Cohort profile: the Guangzhou Biobank Cohort Study, a Guangzhou-Hong Kong-Birmingham collaboration. Int. J. Epidemiol. 2006; 35: 844–852. 43. C. A. McCarty, R. A. Wilke, P. F. Giampietro, S. D. Wesbrook, and M. D. Caldwell, Marshfield Clinic Personalized Medicine Research Project (PMRP): design, methods and recruitment for a large population-based biobank. Personalized Med. 2005; 2: 49–79. 44. H. Swede, C. L. Stone, and A. R. Norwood, National population-based biobanks for genetic research. Genet. Med. 2007; 9: 141–149. 45. NUgene Project. Available at: http://www. nugene.org/. 46. DNA Bank and Tissue Repository, Center for Human Genetics, Duke University. Available at http://www.chg.duke.edu/ research/dnabank.html. 47. J. Kaiser, Genomic medicine—AfricanAmerican population biobank proposed. Science 2003; 300: 1485–1485. 48. J. Kaiser, Genetics—US hospital launches large biobank of children’s DNA. Science 2006; 312: 1584–1585. 49. J. Couzin, Veterans Affairs—Gene bank proposal draws support—and a competitor. Science 2005; 309: 684–685.

CROSS-REFERENCES Genetic Association Analysis Pharmacogenomics Microarray Repository Two-Stage Genetic Association Studies

11

ESCALATION AND UP-AND-DOWN DESIGNS

at a time to target a wide range of quantiles
. Storer (10) and Korn et al. (11) used decision rules of group designs to suggest several designs for dose finding. Among the designs studied in Korn et al. (11) and Shih and Lin (12) were versions of the traditional or 3 + 3 design widely used in dose-finding trails in oncology. Lin and Shih (13) generalized the 3 + 3 by introducing A + B designs.

ANASTASIA IVANOVA University of North Carolina at Chapel Hill Chapel Hill, North Carolina

Up-and-down designs are widely used in both preclinical (animal) and clinical dose-finding trials. These designs operate on prespecified set of doses of the investigational product. The dose for the next subject (or a cohort of subjects) is repeated, increased, or decreased according to the outcome of the subject (or a cohort of subjects) having received the immediate prior dose. The decision rules used in up-and-down designs are very simple and intuitive. The goal is to find the maximally tolerated dose (MTD). The MTD is sometimes defined as the dose just below the lowest dose level with unacceptable toxicity rate; or the MTD can be defined as the dose with the probability of toxicity closest to a prespecified rate
. The underlying model assumption is that the probability of toxicity is a nondecreasing function of dose. Upand-down designs do not require any other assumptions on the dose–toxicity relationship.

2

BIASED COIN DESIGNS

In biased coin designs, subjects are assigned to a dose level of a drug one at a time. The biased coin design from Durham and Flournoy (4), developed for the case of
≤ 0.5, uses a biased coin with the probability of heads equal to b =
/(1−
), 0 ≤ b ≤ 0.5. If the outcome of the most recent subject is toxicity, the dose is decreased. If the outcome of the most recent subject is no toxicity, the dose is increased if the biased coin’s toss results in heads, and repeated if the toss results in tails. This process is continued until prespecified number of subjects is assigned. To determine the next assignment, biased coin designs use the outcome of a single (most recent) subject only and hence are not efficient when data from more than one subject at a dose are available.

1 HISTORY OF UP-AND-DOWN DESIGNS FOR DOSE FINDING

3 Von B´ek´esy (1) and Dixon and Mood (2) described an up-and-down design where the dose level increased after a nontoxic response and decreased if toxicity is observed. This approach clusters the treatment distribution around the dose for which the probability of toxicity is equal to
= 0.5. To target any quantile
, Derman (3) modified the decision rule of the design using a biased coin with the probability of heads computed as a function of the target quantile
. Durham and Flournoy 4, 5 considered two biased coin designs in the spirit of Derman. Giovagnoli and Pintacuda (6) later obtained interesting theoretical results on biased coin designs. Wetherill (7) and Tsutakawa (8, 9) proposed assigning subjects in groups rather than one

GROUP UP-AND-DOWN DESIGNS

Group up-and-down designs (7–9) are the building blocks for many dose-finding designs (14). A group up-and-down design induces a Markov chain on the set of doses. Hence, some statistical properties of a group design can be obtained using the Markov chain theory. Gezmu and Flournoy (15) studied small sample properties of group designs. Ivanova et al. (16) studied large sample properties. Let d1 < . . . < dK be dose levels selected for the study, and p1 < . . . < pK be corresponding probabilities of toxicity at these doses. In a group up-and-down design, subjects are assigned to treatment in cohorts of size s starting with the lowest dose. Let X be the number of toxicities in the most recent

Wiley Encyclopedia of Clinical Trials, Copyright © 2007, John Wiley & Sons, Inc.

1

2

ESCALATION AND UP-AND-DOWN DESIGNS

cohort assigned to dose dj , X ∼ Bin(s,pj ), where Bin(s,p) denotes the binomial random variable with parameters s and p. Let cL and cU be two integers such that 0 ≤ cL < cU ≤ s. Then, 1. if X ≤ cL , the next cohort of s subjects is assigned to dose dj+1 . 2. if cL < X < cU , the dose is repeated for the next cohort of s subjects. 3. if X ≥ cU , the next cohort of s subjects is assigned to dose dj−1 . Appropriate adjustments are made at the lowest and highest dose so that the assignments stay within d1 , . . . , dK . The process is continued until a prespecified number of patients are assigned. Assignments in group up-and-down design for large sample sizes are clustered around the dose with toxicity rate
s , where
s is the solution of Pr{Bin(s,
s ) ≤ cL } = Pr{Bin(s,
s ) ≥ cU }. That is, if there is a dose dk such that
s = pk , the assignments are clustered around dk . If pk – 1 <
s < pk , the assignments are clustered around dose k − 1 or k (16). To find
s for given parameters s, cL , and cU , one needs to write the equation above using formulas for binomial probabilities. For example, for the group design with group size s, cL = 0 and cU = 1, the equation has the form (1−
s )s = 1 − (1−
s )s with the solution
s = 1 − (0.5)1/s . For most of the group up-and-down designs, closed form solutions of this equation do not exist, but the equation can be easily solved numerically. For most practical applications of group designs, approximation
s ≈ (cU /s − cL /s)/2 can be used. 4

ESCALATION DESIGNS

Escalation designs are widely used in firsttime-in-human phase I studies in a variety of therapeutic areas. Such trials usually investigate three to eight dose levels, and patients are typically assigned to cohorts with a size of six to eight patients, some among whom are receiving placebo (instead of the investigational agent). All patients receiving investigational product in a cohort receive the same

dose. Doses are increased by one level for each subsequent cohort. The trial is stopped when an unacceptable number of adverse events or an unacceptable type of adverse event is observed, when the highest dose level is reached, or for other reasons as well. The ‘‘target dose,’’ the dose recommended for future trials, is usually determined on the basis of the rates or types of adverse events (or both) at the dose levels studied, in addition often to pharmacokinetic parameters and considerations. If only the rate of adverse events is taken into account with the goal of finding the target dose, the dose with adverse events rate of
, the escalation design can be more formally defined as follows. Patients are assigned to treatment in cohorts of size m starting with the lowest dose. Let design parameter CU be an integer such that 0 ≤ CU < m. Assume that the most recent cohort of patients was assigned to dose level dj , j = 1, . . . , K − 1. Let X be the number of adverse events in a cohort assigned to dj . Then if X ≤ CU , the next cohort of m patients is assigned to dose dj+1 ; otherwise, the trial is stopped. The dose one level below the dose where > CU adverse events were observed is the estimated target dose. If the escalation was not stopped at any of the doses, the highest dose dK is recommended. The obvious choice of CU is CU /m ≤
< (CU + 1)/m. The frequency of stopping escalation at a certain dose level depends on the adverse event rate at this dose as well as rates at all lower dose levels. Ivanova (14) studied how cohort size and the choice of dose levels affect the precision of the escalation design. Clearly, designs with large cohort size have better statistical precision, hence values m < 6 are not recommended. From a safety standpoint, especially in early phase studies, it might be risky for all m patients in a cohort to receive the investigational product at the same time. In such trials, the cohort of m can be split, for example, into two subcohorts. Patients are assigned in the first subcohort and then to the second, if treatment in the first cohort was well tolerated. A similar strategy is used in the A + B designs described in the next section.

ESCALATION AND UP-AND-DOWN DESIGNS

5

A + B DESIGNS

The A + B designs (13) include a stopping rule as in escalation designs but save resources at lower doses. In general terms, A + B design without de-escalation is defined as follows. Let A and B be positive integers. Let cL , cU , and CU be integers such that 0 ≤ cL < cU ≤ A, cU − cL ≥ 2, and cL ≤ CU < A + B. Patients are assigned to doses in cohorts of size A starting with the lowest dose. Assume that the most recent cohort was a cohort of A patients assigned to receive dose dj , j = 1, . . . , K − 1. Let X A be the number of toxicities in a cohort of size A assigned to dose dj , and let X A+B be the number of toxicities in a cohort of size A + B. Then,

Table 1. Examples of A + B designs.

1. 0 ≤ cL < cU ≤ A, cU − cL ≥ 2, and cL ≤ CU < A + B. 2. (cU /A − cL /A)/2 ≈
or slightly exceeds
.

A = B = 5, cL A = B = 3, cL A = B = 4, cL A = B = 4, cL A = B = 3, cL

0.1 0.2 0.3 0.4 0.5

= 0, cU = 0, cU = 0, cU = 1, cU = 0, cU

= 2, CU = 2, CU = 3, CU = 3, CU = 3, CU

=1 =1 =2 =3 =3

3. CU /(A + B) <
< (CU + 0.5) (A + B). The choice of cohort sizes A and B so that A ≤ B yields more effective designs on average. Several A + B designs that satisfy the rules above are presented in Table (1). The values of
in Table (1) were computed from constraint [3] as the midinterval rounded to the nearest decimal. Designs described here are A + B designs without dose de-escalation. The description of A + B designs with dose de-escalation and of several modifications of the A + B designs can be found in Shih and Lin (12, 13).

Then, if in the combined cohort assigned to dj , X A+B ≤ CU , the next cohort of size A receives dose dj+1 ; otherwise, the trial is stopped:

The dose one level below the dose where an unacceptable number of toxicities were observed ( ≥ cU toxicities in a cohort of size A or > CU toxicities in a cohort of size A + B) is the estimated MTD. If the escalation was not stopped at any of the doses, the highest dose dK is recommended. The frequency of stopping escalation at a certain dose level depends on the toxicity rate at this dose as well as the rates at all lower dose levels. Hence, it is impossible to identify a quantile targeted by a certain A + B design. However, some practical guidelines on how to choose the design parameters can be formulated (14). If
is the target quantile, parameters A, B, cL , cU , and CU in the A + B design can be selected according to the following constraints:

Design parameters




1. if X A ≤ cL , the next cohort of A patients is assigned to dose dj+1 . 2. if cL < X A < cU , the cohort of B patients is assigned to dose dj .

1. if X A ≥ cU , the trial is stopped.

3

6

Traditional or 3 + 3 Design

The traditional or 3 + 3 design frequently used in dose finding in oncology (see the article on phase I oncology trials for a description of the 3 + 3 design) is a special case of an A + B design without de-escalation (13), as was described here with A = B = 3, cL = 0, cU = 2, and CU = 1. REFERENCES 1. G. von B´ek´esy, A new audiometer. Acta Otolaryngologica. 1947; 35: 411–422. 2. W. J. Dixon and A. M. Mood, A method for obtaining and analyzing sensitivity data. J Am Stat Assoc. 1954; 43: 109–126. 3. C. Derman, Nonparametric up and down experimentation. Ann Math Stat. 1957; 28: 795–798. 4. S. D. Durham and N. Flournoy, Random walks for quantile estimation. In: S. S. Gupta and J. O. Berger (eds.), Statistical Decision Theory and Related Topics V. New York: SpringerVerlag, 1994, pp. 467–476.

4

ESCALATION AND UP-AND-DOWN DESIGNS

5. S. D. Durham and N. Flournoy, Up-and-down designs I. Stationary treatment distributions. In: N. Flournoy and W. F. Rosenberger (eds.), Adaptive Designs. Hayward, CA: Institute of Mathematical Statistics, 1995, pp. 139–157. 6. A. Giovagnoli and N. Pintacuda, Properties of frequency distributions induced by general ‘‘up-and-down’’ methods for estimating quantiles. J Stat Plan Inference. 1998; 74: 51–63. 7. G. B. Wetherill, Sequential estimation of quantal response curves. J R Stat Soc Ser B Methodol. 1963; 25: 1–48. 8. R. K. Tsutakawa, Random walk design in bioassay. J Am Stat Assoc. 1967; 62: 842–856. 9. R. K. Tsutakawa, Asymptotic properties of the block up-and-down method in bio-assay. Ann Math Stat. 1967; 38: 1822–1828. 10. B. E. Storer, Design and analysis of phase I clinical trials. Biometrics. 1989; 45: 925–937. 11. L. Korn, D. Midthune, T. T. Chen, L. V. Rubinstein, M. C. Christian, and R. M. Simon, A comparison of two phase I trial designs. Stat Med. 1994; 13: 1799–1806. 12. W. J. Shih and Y. Lin, Traditional and modified algorithm-based designs for phase I cancer clinical trials. In: S. Chevret (ed.), Statistical Methods for Dose Finding. New York: Wiley, 2006, pp. 61–90.

13. Y. Lin and W. J. Shih, Statistical properties of the traditional algorithm-based designs for phase I cancer clinical trials. Biostatistics. 2001; 2: 203–215. 14. A. Ivanova, Escalation, group and A + B designs for dose-finding trials. Stat Med. 2006; 25: 3668–3678. 15. M. Gezmu and N. Flournoy, Group up-anddown designs for dose-finding. J Stat Plan Inference. 2006; 136: 1749–1764. 16. A. Ivanova, N. Flournoy, and Y. Chung, Cumulative cohort design for dose-finding. J Stat Plan Inference. 2007; 137: 2316–2327.

FURTHER READING S. Chevret, ed., Statistical Methods for Dose Finding. New York: Wiley, 2006. J. Crowley, ed., Handbook of Statistics in Clinical Oncology. New York/Basel: Marcel Dekker, 2006. N. Ting, ed., Dose Finding in Drug Development. New York: Springer-Verlag, 2006.

CROSS-REFERENCES Phase I trials in oncology

DOSE ESCALATION GUIDED BY GRADED TOXICITIES

in which level 7 is the correct level. Patients enter the study sequentially. The working dose-toxicity curve, which is taken from the CRM class (described below), is refitted after each inclusion. The curve is then inverted to identify which available level has an associated estimated probability as close as we can get to the targeted acceptable toxicity level. The next patient is then treated at this level. The cycle is continued until a fixed number of subjects has been treated or until we apply some stopping rule (1,2). The di , which is often multidimensional, describes the actual doses or combinations of doses being used. We assume monotonicity, and we take monotonicity to mean that the dose levels are equally well identified by their integer subscripts i (i = 1, . . . , k), which are ordered whereby the probability of toxicity at level i is greater than that at level i whenever i is greater than i . The monotonicity requirement or the assumption that we can so order our available dose levels is important. The dose for the jth entered patient, X j can be viewed as random taking values xj , most often discrete in which case xj
{d1 , . . . , dk } but possibly continuous where X j = x; x
R+ . In light of the remarks of the previous two paragraphs we can, if desired, entirely suppress the notion of dose and retain only information that pertains to dose level. This information is all we need, and we may prefer to write xj
{1, . . . , k}. Let Y j be a binary random variable (0, 1) where 1 denotes severe toxic response for the jth entered patient (j = 1, . . . , n). We model R(xj ), which is the true probability of toxic response at Xj = xj ; xj
{d1 , . . . , dk } or xj
{1, . . . , k} via

JOHN O’QUIGLEY NOLAN WAGES University of Virginia Charlottesville, Virginia

1

BACKGROUND

In phase 1 and phase 2 dose-finding studies, the endpoints of interest are typically the presence or absence of toxicity and/or the presence or absence of some indication of therapeutic effect. In numerical terms, these outcomes are represented as simple binary variables. Most protocols, however, will stipulate that intermediary degrees, or grades, of toxicity be recorded. Our purpose here is to consider how such intermediary information may be used to obtain a more accurate estimation of the maximum tolerated dose, both at the end of the study and for those patients being treated during the course of the study. For now, we limit our attention to phase 1 studies alone, in which toxicity is the focus of our interest. Working with two-stage continual reassessment method (CRM) designs, which are briefly described in the following section, we can observe that considerable use can be made of graded information both during the initial escalation period as well as during the second stage of a two-stage design. Here, we appeal to simple working models. 2

TWO-STAGE CRM DESIGNS

R(xj ) = Pr(Yj = 1|Xj = xj )

The purpose of the design is to identify a level, from among the k dose levels available d1 , . . . , dk , such that the probability of toxicity at that level is as close as possible to some value
. The value
is chosen by the investigator such that he or she considers probabilities of toxicity higher than
to be unacceptably high, whereas those lower than
are unacceptably low in that they indicate, indirectly, the likelihood of too weak an antitumor effect. Figure 1 illustrates typical behavior of a CRM design with a fixed sample

= E(Yj |xj ) = (xj , a) for some one-parameter working model (xj , a). For given fixed x, we require that (x, a) be strictly monotonic in a. For fixed a, we require that (x, a) be monotonic increasing in x or, in the usual case of discrete dose levels di , i = 1, . . . , k, that (di , a) > (dm , a) whenever i > m. The true probability of toxicity at x (i.e., whatever treatment

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

DOSE ESCALATION GUIDED BY GRADED TOXICITIES Trial History CRM toxicity

9 8

Dose Level

7 6 5 4 3 2

Figure 1. A typical trial history for a two-stage CRM design using accelerated early escalation based on grade. MTD corresponds to a toxicity rate of 25% and is found at level 7.

1 1

7

10

13

16

19

22

25

28

Subject No

combination has been coded by x) is given by R(x), and we require that, for the specific doses under study (d1 , . . . , dk ) values of a, say a1 , . . . , ak exist such that (di , ai ) = R(di ), (i = 1, . . . , k). In other words, our oneparameter working model has to be rich enough to model the true probability of toxicity at any given level. We call it a working model because we do not anticipate a single value of a to work precisely at every level, that is, we do not anticipate a1 = a2 = · · · = ak = a. Many choices are possible. Excellent results have been obtained with the simple choice: (di , a) = αia , (i = 1, . . . , k)

4

(1)

where 0 < α1 < · · · < αk < 1 and 0 < a < ∞. It can be sometimes advantageous to make use of the reparameterized model (di , a) = exp(a) so that no constraints are placed on αi the parameter a. Of course, likelihood estimates of are unchanged. Once a model has been chosen and we have data in the form of the set j = {y1 , x1 , . . . , yj , xj }, the outcomes of ˆ i ), the first j experiments obtain estimates R(d (i = 1, . . . , k) of the true unknown probabilities R(di ), (i = 1, . . . , k) at the k dose levels. The target dose level is that level having associated with it a probability of toxicity as close as we can get to
. The dose or dose level xj assigned to the jth included patient is

such that ˆ j ) −
| < |R(d ˆ i ) −
|, (i = 1, . . . , k; xj = dj ) |R(x Thus xj is the closest level to the target level in the above precise sense. Other choices of closeness could be made by incorporating cost or other considerations. We could also weight ˆ j )− the distance, for example multiply |R(x
| by some constant greater than 1 when ˆ j ) >
. This method would favor conserR(x vatism; such a design tends to experiment more often below the target than a design without weights. Similar ideas have been pursued by Babb et al. (3). After the inclusion of the first j patients, the log-likelihood can be written as: Lj (a) =

j


y log (x , a)

=1

+

j
(1 − y ) log(1 − (x , a)) (2) =1

and is maximized at a = aˆ j . Maximization of Lj (a) can easily be achieved with a Newton Raphson algorithm or by visual inspection using some software package such as Microsoft Excel (Microsoft Corporation, Redmond, WA). Once we have calculated aˆ j , we can next obtain an estimate of the probability of toxicity at each dose level di via: ˆ i ) = (di , aˆ j ) (i = 1, . . . , k) R(d

DOSE ESCALATION GUIDED BY GRADED TOXICITIES

We would not anticipate these estimates to be consistent at all dose levels, which would usually require a richer model than what we work with. However, under broad conditions, we will (4) obtain consistency at the recommended maximum tolerated dose (MTD). Based on this formula, the dose to be given to the (j + 1)th patient, xj+1 is determined. The experiment is considered as not being fully underway until we have some heterogeneity in the responses. These examples could develop in a variety of different ways, which include use of the standard Up and Down approach, use of an initial Bayesian CRM as outlined below, or use of a design believed to be more appropriate by the investigator. Once we have achieved heterogeneity, the model kicks in and we continue as prescribed above iterating between estimation and dose allocation. The design is then split into two stages: an initial exploratory escalation followed by a more refined homing in on the target. Storer (5) was the first to propose twostage designs in the context of the classic Up and Down schemes. His idea was to enable more rapid escalation in the early part of the trial where we may be far from a level at which treatment activity could be anticipated. Moller (6) was the first to use the idea in the context of CRM designs. Her idea was to allow the first stage to be based on some variant of the usual Up and Down procedures. In the context of sequential likelihood estimation, the necessity of an initial stage was pointed out by O’Quigley and Shen (7), because the likelihood equation fails to have a solution on the interior of the parameter space unless some heterogeneity in the responses has been observed. Their suggestion was to work with any initial scheme, such as Bayesian CRM or Up and Down. For any reasonable scheme, the operating characteristics seem relatively insensitive to this choice. However, something very natural and desirable is observed in two stage designs, and currently they could be taken as the designs of choice. The reason is the following: Early behavior of the method, in the absence of heterogeneity (i.e., lack of toxic response), seems to be rather arbitrary. A decision to escalate after inclusion of three patients who tolerated some level, or after a

3

single patient tolerated a level or according to some Bayesian prior, however constructed, is translating directly (although less directly for the Bayesian prescription) the simple desire to try a higher dose because we’ve encountered no toxicity thus far. We can make use of information on toxicity grade in either one of these two stages. In the first stage, no model is being used, and we use graded toxicities simply to escalate more rapidly when it seems we are far below any level likely to result in doselimiting toxicities. Once we begin to observe some intermediary toxicity, then we slow the escalation down. The ideas are straightforward and appeal mostly to common sense arguments. Nonetheless, it can be observed that use of graded toxicity information in the first stage alone can make an important contribution to increased efficiency. Use of graded toxicity information in the second stage requires an additional model to that already used to demonstrate the rate of toxicities. We consider these two different situations in the following two sections. 3 USING GRADED INFORMATION IN THE FIRST STAGE Consider the following example of a twostage design that has been used in practice. Many dose levels were used, and the first included patient was treated at a low level. As long as we observe very low-grade toxicities, then we escalate quickly, which includes only a single patient at each level. As soon as we encounter more serious toxicities, escalation is slowed down. Ultimately, we encounter dose-limiting toxicities at which time the second stage, based on fitting a CRM model, comes fully into play. This method is done by integrating this information and that obtained on all the earlier non-dose-limiting toxicities to estimate the most appropriate dose level. We can use information on low-grade toxicities in the first stage of a two-stage design to allow rapid initial escalation, because it may be the case that we be far below the target level. Specifically, we define a grade severity variable S(i) to be the average toxicity severity observed at dose level i (i.e., the sum of the

4

DOSE ESCALATION GUIDED BY GRADED TOXICITIES

severities at that level divided by the number of patients treated at that level). The rule is to escalate providing S(i) is less than 2. Furthermore, once we have included three patients at some level, escalation to higher levels only occurs if each cohort of three patients does not experience dose-limiting toxicity. This scheme means that, in practice, as long as we observe only toxicities of severities coded 0 or 1, we escalate. Only a single patient is necessary (for whom little or no evidence of any side effects is observed) to decide to escalate. The first severity coded 2 necessitates another inclusion at this same level and, anything other than a 0 severity for this inclusion would require yet another inclusion and a non-dose-limiting toxicity before being able to escalate. This design also has the advantage that, should we be slowed down by a severe (severity 3), albeit non-doselimiting toxicity, we retain the capability of picking up speed (in escalation) should subsequent toxicities be of low degree (0 or 1). This method can be helpful in avoiding being handicapped by an outlier or an unanticipated and possibly not drug-related toxicity. Many variants on this particular escalation scheme and use of graded severity are available. It is for the investigator to decide which scheme is suitable for the given circumstance and which scheme seems to provide the best balance between rapid escalation and caution in not moving so quickly as to overshoot the region where we will begin to encounter dose-limiting toxicites. Once a dose-limiting toxicity has been encountered, this phase of the study (the initial escalation scheme) ends, and we proceed to the second stage based on a CRM modelbased recommendation. Although the initial phase is closed, the information obtained on both dose-limiting and non-dose-limiting toxicities is used in the second stage.

a statistician is to consider that the response variable, which is toxicity, has been simplified when going from five levels to two and that it may help to employ models accommodating multilevel responses. In fact, we do not believe that progress is to be made using these methods. The issue is not that of modeling a response (toxicity) at 5 levels but of controlling for dose-limiting toxicity, mostly grade 4 but possibly also certain kinds of grade 3. Lower grades are helpful in that their occurrence indicates that we are approaching a zone in which the probability of encountering a dose-limiting toxicity is becoming large enough to be of concern. This idea is used implicitly in the two-stage designs described in the section entitled, ‘‘Using graded information in the first stage.’’ Hopefully, if we proceed more formally and extract yet more information from the observations, then we need models to relate the occurrence of dose-limiting toxicities to the occurrence of lower-grade toxicities. By modeling the ratio of the probabilities of the different types of toxicity, we can make striking gains in efficiency because the more frequently observed lower grade toxicities carry a great deal of information on the potential occurrence of dose-limiting toxicities. Such a situation would also allow gains in safety because, at least hypothetically, it would be possible to predict at some level the rate of occurrence of dose-limiting toxicities without necessarily having observed very many, the prediction leaning largely on the model. At the opposite end of the model/hypothesis spectrum, we might decide we know nothing about the relative rates of occurrence of the different toxicity types and simply allow the accumulating observations to provide the necessary estimates. In this case, it turns out that we neither lose nor

4 USE OF GRADED TOXICITIES IN THE SECOND STAGE

Table 1. Toxicity ‘‘Grades’’ (Severities) for Trial

Although we refer to dose-limiting toxicities as a binary (0,1) variable, most studies record information on the degree of toxicity, from 0, complete absence of side effects, to 4, lifethreatening toxicity. The natural reaction for

0 1 2 3 4

No toxicity 1 Mild toxicity (non-dose-limiting) Non-mild toxicity (non-dose-limiting) Severe toxicity (non-dose-limiting) Dose-limiting toxicity

DOSE ESCALATION GUIDED BY GRADED TOXICITIES

5

Table 2. Compared Frequency of Final Recommendations of a Standard CRM and a Design Using Known Information on Graded Toxicities

Rk Standard Using grades

1

2

3

4

5

6

0.05 0.04 0.00

0.11 0.22 0.09

0.22 0.54 0.60

0.35 0.14 0.29

0.45 0.06 0.02

0.60 0.00 0.00

gain efficiency, and the method behaves identically to one in which the only information we obtain is whether the toxicity is dose limiting. These two situations suggest a middle road might exist, using a Bayesian prescription, in which very careful modeling can lead to efficiency improvements, if only moderate, without making strong assumptions. To make this model more precise, let us consider the case of three toxicity levels, the highest being dose limiting. Let Y j denote the toxic response for subject j, (j = 1, . . . , n). The variable Y j can assume three levels: 1, 2, and 3. The goal of the trial is to identify a level of dose whose probability of severe toxicity is closest to a given percentile of the dosetoxicity curve. Supposing, for patient j, that xj = di , then a working model for the CRM could be: exp(a)

Pr(Yj = 3) = 1 (xj , a) = αi Pr(Yj = 2 or Yj = 3) = (xj , a, b) exp(a+b)

= αi

from which Pr(Yj = 1) = 1 − 2 (xj , a, b) and exp(a+b) exp(a) Pr(Yj = 2) = αi − αi . The contributions to the likelihood are: 1 − 2 (xj , a, b) when Y j = 1, (xj , a) when Y j = 3 and 2 (xj , a, b) − 1 (xj , a) when Y j = 2. With no prior information, and being able to maximize the likelihood, we obtain almost indistinguishable results to those obtained with the more usual one-parameter CRM, which is caused by near-parameter orthogonality. Therefore, no efficiency gain occurs, although there is the advantage of learning about the relationship between the different toxicity types. However, based on previous studies, we often have a very precise idea concerning the relative rates between certain toxicity grades. We can imagine that this relationship can be estimated with good precision. Suppose that the parameter b is known precisely.

The model need not be correctly specified, although b should maintain interpretation outside the model, for instance some simple function of the ratio of grade 3 to grade 2 toxicities. Efficiency gains can then be substantial. Table 2 provides a simple illustration of the order of magnitude of the gains we might anticipate when we are targeting a value of
around 0.25. The rate of lower grade toxicities is known to be twice this rate. A Bayesian framework would allow us to make weaker assumptions on the parameter b so that any errors in assumptions can then be overwritten by the data. More work is needed on this subject, but the early results are very promising. REFERENCES 1. J. O’Quigley and E. Reiner, A stopping rule for the continual reassessment method. Biometrika 1998; 85: 741–748. 2. J. O’Quigley, Estimating the probability of toxicity at the recommended dose following a Phase I clinical trial in cancer. Biometrics 1992; 48: 853–862. 3. Baab, J., Rogatko, A., and Zacks, S., (1998). Cancer phase I clinical trials: Efficient close escalation with overdose control. Statistics in Medicine 17. 1103–1120. 4. L. Z. Shen and J. O’Quigley, Consistency of continual reassessment method in dose finding studies. Biometrika 1996; 83: 395–406. 5. B. E. Storer, Phase I clinical trials. In: Encylopedia of Biostatistics. 1998. Wiley, New York. 6. S. Moller, An extension of the continual reassessment method using a preliminary up and down design in a dose finding study in cancer patients in order to investigate a greater number of dose levels. Stats. Med. 1995; 14: 911–922. 7. J. O’Quigley and L. Z. Shen, Continual reassessment method: a likelihood approach. Biometrics 1996; 52: 163–174.

6

DOSE ESCALATION GUIDED BY GRADED TOXICITIES

FURTHER READING C. Ahn, An evaluation of phase I cancer clinical trial designs. Stats. Med. 1998; 17: 1537–1549. S. Chevret, The continual reassessment method in cancer phase I clinical trials: a simulation study. Stats. Med. 1993; 12: 1093–1108. D. Faries, Practical modifications of the continual reassessment method for phase I cancer clinical trials. J. Biopharm. Stat. 1994; 4: 147–164. C. Gatsonis and J. B. Greenhouse, Bayesian methods for phase I clinical trials. Stats. Med. 1992; 11: 1377–1389. S. Goodman, M. L. Zahurak, and S. Piantadosi, Some practical improvements in the continual reassessment method for phase I studies. Stats. Med. 14: 1149–1161. J. O’Quigley, M. Pepe, and L. Fisher, Continual reassessment method: a practical design for Phase I clinical trials in cancer. Biometrics 1990; 46: 33–48. J. O’Quigley and S. Chevret, Methods for dose finding studies in cancer clinical trials: a review and results of a Monte Carlo study. Stats. Med. 1991; 10: 1647–1664. J. O’Quigley, L. Shen, and A. Gamst, Two sample continual reassessment method. J. Biopharm. Statist. 1999; 9: 17–44. S. Piantadosi and G. Liu, Improved designs for dose escalation studies using pharmacokinetic measurements. Stats. Med. 1996; 15: 1605–1618. J. Whitehead and D. Williamson, Bayesian decision procedures based on logistic regression models for dose-finding studies. J. Biopharm. Statist. 1998; 8: 445–467.

DOSE-FINDING STUDIES

both efficacy and safety, as one of the key drivers of the high attrition rates currently plaguing late phase clinical trials across the pharmaceutical industry (7, 8). PhRMA has constituted a working group to evaluate and propose recommendations to address this specific issue (8). In light of these ongoing discussions and activities, this article reviews some of the key methodologies used in the dose-finding trials typically encountered in the late stage of drug development, that is, in phase II and/or phase III clinical trials. Out of scope for this overview are dose-finding studies in early development, which often take place under different constraints and use different methodologies, such as the traditional 3 + 3 designs, up-and-down designs, or continual reassessment methods (4, 5).

FRANK BRETZ Novartis Parma AG Basel, Switzerland

JOSE´ C. PINHEIRO Novartis Pharmaceuticals East Hanover, New Jersey

Understanding and adequately representing the dose–response profile of a compound, with respect to both efficacy and safety, is a fundamental objective of clinical drug development. An indication of its importance is the early publication of the International Conference on Harmonization E4 guideline on dose–response studies (1). The dose–response profile describes how the expected response—for example, a clinical endpoint of interest—varies in relation to the dose levels being administered. Proper understanding of this relationship is crucial for two critical decisions required during drug development: (1) whether there is an overall dose–response effect (proof of concept), and (2) if so, which dose level(s) should be selected for further development (dose finding). Selecting too low a dose decreases the chance of showing efficacy in later studies, whereas selecting too high a dose may result in tolerability or safety problems. Indeed, it may occur that only after having marketed a new drug at a specified dose does it become apparent that the level was set too high. This phenomenon has been documented by the U.S. Food and Drug Administration (FDA), who reported that approximately 10% of drugs approved between 1980 and 1989 have undergone dose changes—mostly decreases—of greater than 33% (2, 3). Over the past several years, an increase in interest and research activities in this area has taken place. Illustrating this trend, three books solely dedicated to dose finding in clinical drug development, from different perspectives, have recently been published (4–6). In addition, both the FDA and the Pharmaceutical Research and Manufacturers of America (PhRMA) have identified poor dose selection resulting from incorrect or incomplete knowledge of the dose–response relationship, for

1

MULTIPLE COMPARISON PROCEDURES

The analysis of dose-finding studies can be classified into two major strategies: modeling techniques (9, 10) and multiple comparison procedures (MCP) (11, 12). Modeling techniques assume a functional relationship between the dose (taken as a quantitative factor) and the response variable, according to a prespecified parametric model (defined in the study protocol). In this section we consider MCP, in which the dose is regarded as a qualitative factor and very few, if any, assumptions are made about the underlying dose–response model. MCP can be used either for detecting an overall dose-related signal by means of trend tests or for the estimation of target doses by stepwise testing strategies, while preserving the overall type I error rate at a prespecified level α. Such procedures are relatively robust to the underlying dose–response shape, but they are not designed for extrapolation of information beyond the observed dose levels. Inference is thus confined to the selection of the target dose among the dose levels under investigation. A classic method proposed by Dunnett (13) compares several treatments with a control. Because it relies on pairwise comparisons based on t-tests using the pooled variance estimate, structural information from the

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

DOSE-FINDING STUDIES

logical ordering of the dose levels is not incorporated. Trend tests exist that borrow strength from neighboring dose levels to increase the likelihood of successfully detecting a dose–response signal at the end of the study. The likelihood ratio test (14) is an example of a powerful test for detecting such a dose–response trend. However, because its critical values are difficult to compute, its application is reduced to balanced one-way layouts and other simple designs (15). Single contrast tests provide a popular alternative (16, 17), but these tests are potentially less powerful than competing methods if the true dose–response shape deviates substantially from the prespecified vector of contrast coefficients. Multiple contrast tests have been proposed instead, which take the maximum test statistic over several single contrast tests, properly adjusting it for multiplicity (18, 19). The advantage of such an approach is its greater robustness with regard to the uncertainty about the unknown true dose–response model, resulting from testing simultaneously different shapes. An additional appealing feature of multiple contrast tests is that they can be easily extended to general linear models incorporating covariates, factorial treatment structures, and random effects. Many standard trend tests are in fact special cases of (or at least closely related to) multiple contrast tests (20). Bretz et al. (21) provide more technical details on multiple contrast tests. As mentioned, the second major goal of dose-finding studies is to estimate target doses of interest, such as the minimum effective dose (MinED), which is the smallest dose showing a statistically significant and clinically relevant effect (1, 3); the maximum safe dose, which is the largest dose that is still safe; or the maximum effective dose, which is the smallest dose showing a maximum effect (1). If MCP are used, the dose levels are typically tested in a fixed order. Different possibilities exist for choosing the sequence of hypotheses to be tested and the appropriate test statistics (e.g., pairwise comparisons with the control group or any of the previously mentioned trend tests). Tamhane et al. (17, 22) and Strassburger et al. (23) provide more details.

2 MODELING TECHNIQUES Modeling approaches are commonly used to describe the functional dose–response relationship, and many different models have been proposed (24). One classification of such models is according to whether they are linear in their parameters (standard linear regression model, linear in log-dose model, etc.) or not (Emax , logistic, etc.). Pinheiro et al. (9) described several linear and nonlinear regression dose–response models commonly used in practice, including the clinical interpretations for the associated model parameters. Once a dose–response model is fitted (10), it can be used to test whether a doserelated effect is present. For example, one could test whether the slope parameter is different from 0 in a linear regression model. If a dose–response signal has been detected, the fitted dose response could then be used to estimate a target dose achieving a desired response. In contrast to MCP, dose estimation under modeling is not confined to the set of dose levels under investigation. Although such a modeling approach provides flexibility in investigating the effect of doses not used in the actual study, the validity of its conclusions depends highly on the correct choice of the dose–response model, which is typically unknown. We will describe a more detailed description of model-based approaches and potential solutions to overcome their disadvantages. A major pitfall when conducting statistical modeling is related to the inherent model uncertainty. The intrinsic problem is the introduction of a new source of variability by selecting a particular model M (for example) at any stage before the final analysis. Standard statistical analysis neglects this fact and reports the final outcomes without accounting for this extra variability. Typically, one
is interested in computing the variance var θˆ of a parameter estimate θˆ . In practice, how
 ever, the conditional variance var θˆ |M for a given model
M  is computed and stated as if it were var θˆ , ignoring the model uncertainty. In addition, substantial bias in estimating the parameters of interest can be introduced from the model selection process. Whereas it is admittedly a more difficult task

DOSE-FINDING STUDIES Set of candidate models

to compute unbiased estimates conditional on the selected model, ignoring completely the model uncertainty can lead to very undesirable effects (25, 26). A common approach to addressing the model selection problem is to use information criteria based on a reasonable discrepancy measure assessing the lack of fit. Many model selection criteria are available, and the discussion of which gives the best method is still ongoing (27, 28). Examples of such model selection criteria include the Akaike information criterion (AIC), the Bayesian information criterion (BIC), and the order restricted information criterion (29). It should be kept in mind, however, that the application of any of the model selection criteria will always lead to the selection of a model, irrespective of its goodness of fit to the observed data. Different approaches have thus been proposed to overcome the problem of conditional inference on a selected model. Such proposals include (1) weighting methods, which incorporate, rather than ignore, model uncertainty by computing estimates for quantities of interest which are defined for all models (such as the MinED) using a weighted average across the models (30–32); (2) computer-intensive simulationbased inferences, such as cross-validation techniques, which select the model with the best predictive ability across the replications (26, 33); and (3) considering model selection as a multiple hypotheses testing problem, where the selection of a specific model is done while controlling the overall type I error rate (34). 3 HYBRID APPROACHES COMBINING MCP AND MODELING Hybrid dose-finding methods combine principles of MCP with modeling techniques; see Tukey et al. (35) for an early reference. Bretz et al. (36) proposed a methodology for dose-finding studies that they called MCP-Mod, which provides the flexibility of modeling for dose estimation while preserving the robustness to model misspecification associated with MCP. The general flow of the MCP-Mod methodology, including its key steps, is depicted in Figure 1. Practical considerations regarding the implementation of

3

Optimum contrast coefficients Selection of significant models while controlling the overall type I error

Selection of a single model using max-T, AIC, ..., possibly combined with external data Dose estimation and selection (minimum effective dose, ...)

Figure 1. Combining multiple comparisons and modeling techniques in dose-finding studies

this methodology were discussed by Pinheiro et al. (37). Extensions to Bayesian methods estimating or selecting the dose–response curve from a sparse dose design have also been investigated (38, 39). The central idea of the MCP-Mod methodology is to use a set of candidate dose–response models to cover the possible shapes anticipated for the dose–response relationship. Multiple comparison procedures are applied to a set of test statistics, determined by optimal contrasts representing the models in the candidate set, to decide which shapes give statistically significant signals. If no candidate model is statistically significant, the procedure stops and declares that no dose–response relationship can be established from the observed data (i.e., no proof of concept). Otherwise, the maximum contrast test and possibly further contrast tests are statistically significant. Out of the statistically significant models in the candidate set, a best model is selected for dose estimation in the last stage of the procedure. The selection of the dose-estimation model can be based on the minimum P-value (of the model contrast tests) or some other relevant model selection criteria like the AIC or the BIC. The selected dose–response model is then employed to estimate target doses using inverse regression techniques and possibly incorporating information on clinically relevant effects. The precision of the estimated doses can be assessed using, for example, bootstrap methods. In contrast to a direct application of modelbased dose estimation, the MCP step accounts

4

DOSE-FINDING STUDIES

for possible model misspecification and includes the associated statistical uncertainty in a hypothesis-testing context. Note that different model selection criteria may lead to different dose estimates because of different sources of information and/or decision goals. Information criteria, such as the AIC or BIC, are statistical decision rules taking into account only the data from the study under consideration. Bayesian decision rules, on the other hand, may additionally include information external to the study, though still based on statistical reasoning. Finally, nonstatistical selection rules based on updated clinical knowledge, economic reasons, and so forth may also be used in the dose-selection process. Simulation results suggest that hybrid methods are as powerful as standard trend tests while allowing more precise estimation of target doses than MCP due to their modeling component (9, 36). It is worth pointing out that these methods can be seen as seamless designs that combine proof of concept (phase Ib/IIa) with dose finding (phase IIb) in one single study. 4

ADAPTIVE DOSE-FINDING DESIGNS

A fast emerging field of research is the area of adaptive dose-finding designs, which can be employed to extend the previously described methods. These designs offer efficient ways to learn about the dose response through repeated looks at the data being accrued during the conduct of a clinical trial. This interim information can be used to guide decision making on which dose to select for further development or whether to discontinue a program. It is both feasible and advantageous to design a proof-of-concept study as an adaptive dose-finding trial. The continuation of a dose-finding trial into a confirmatory stage through a seamless design is a further opportunity to increase information earlier in development on the correct dose, and thus reduce the total duration of the clinical development program. Accordingly, we briefly review (1) adaptive dose ranging studies as investigated by the related PhRMA working group (8), (2) flexible designs that strongly control the overall type I error rate, and (3) Bayesian adaptive designs.

The PhRMA working group on adaptive dose ranging studies has evaluated several existing dose-finding methods and provided recommendations on their use in clinical drug development (8). The methods considered comprise a representative cross-section of currently available dose-finding procedures, ranging from more traditional methods based on analysis of variance up to adaptive designs based on an advanced Bayesian model. Through an extensive simulation study based on a common set of scenarios (sample sizes, number of doses, etc.) for all procedures, the strengths and weaknesses of each method were investigated, in particular with respect to the ability of the procedures to learn from the data and adapt to emerging information. Flexible designs that strongly control the overall type I error rate of incorrectly rejecting any null hypothesis of no dose level effect are available for adaptive dose selection in multistage clinical trials. In the context of adaptive dose-finding trials, multiplicity concerns typically arise due to (1) the comparison of several doses with a control and (2) multiple interim looks at the data for decision making. Performing each hypothesis test at the nominal level α intended for the whole trial would inflate the overall type I error rate. Therefore, the significance levels of the individual tests have to be adjusted appropriately. Classic group sequential designs are a type of flexible design in which the planned sample size or, more generally, the planned amount of information may be updated as a result of the trial. In these trials, test statistics produced at interim analyses are compared with prespecified upper or lower stopping boundaries that ensure the overall type I error rate control (40–42). Stallard and Todd (43) extended classic group sequential procedures to multiarm clinical trials incorporating treatment selection at the interim analyses. Flexible designs, which may be regarded as an extension of classic group sequential designs, offer more flexibility for adaptation within the multistage framework. These methods offer a high level of flexibility for decision making during the trial, such as increasing the sample size based on the observed effect, modifying the target patient population, or selecting good treatments

DOSE-FINDING STUDIES

(44, 45). They require little prespecification of decision rules before the beginning of a trial; therefore, the total information available at each interim time point can be used in designing or adjusting the next stage. Bayesian adaptive dose-finding designs are an important alternative to flexible designs. Information can either be updated continuously as data are accrued in the trial, or in cohorts of patients. This makes this class of designs very appealing to sequential decision making and experimentation, including clinical studies. Bayesian approaches enable the calculation of predictive probabilities of future results for any particular design, which allows comparison of designs on the basis of probabilities of their consequences. Although control of the type I error rate is not an intrinsic property of a Bayesian design, simulations can be used to tailor a Bayesian adaptive trial such that it maintains this and other desirable frequentist operational characteristics. A potential downside to the Bayesian approach is the computational complexity coupled with the absence of commercial software packages to assist with study design and analysis. Berry et al. (46) and Krams et al. (47) have more methodological details and an example of a clinical study employing Bayesian dose-finding methods in practice. 5

CONCLUSION

Dose-finding studies play a key role in any drug development program and are often the gatekeeper for the large confirmatory studies in phase III. Many approaches exist for the proper design and analysis of these trials. The ultimate choice of the method to be applied depends on the particular settings and goals. Dose-finding studies should thus be tailored to best fit the needs of the particular drug development program under consideration. Methods are available, for example, to allow the conduct of seamless proof-ofconcept and dose-finding studies. Alternatively, if it is desired to extend dose-finding trials straight into a confirmatory phase III study, adaptive designs offer efficient possibilities to control the overall type I error rate at a prespecified level. We encourage the consideration and implementation of advanced

5

dose-finding methods, which efficiently make use of accumulating information during the drug development process. REFERENCES 1. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH Harmonised Tripartite Guideline: E4 Dose-Response Information to Support Drug Registration. Step 4 version, March 1994. Available at: http://www.ich.org/LOB/media/ MEDIA480.pdf. 2. FDC Reports from May 6, 1991. The Pink Sheet. 1991: 53(18): 14–15. 3. S. J. Ruberg, Dose response studies I. Some design considerations. J Biopharm Stat. 1995; 5: 1–14. 4. N. Ting (ed.), Dose Finding in Drug Development. New York: Springer, 2006. 5. S. Chevret (ed.), Statistical Methods for Dose Finding Experiments. New York: Wiley, 2006. 6. R. Krishna (ed.), Dose Optimization in Drug Development. New York: Informa Healthcare, 2006. 7. U.S. Food and Drug Administration, Department of Health and Human Services. Innovation/Stagnation: Challenge and Opportunity on the Critical Path to New Medical Products. March 2004. Available at: http://www.fda.gov/ oc/initiatives/criticalpath/whitepaper.html 8. B. Bornkamp, F. Bretz, A. Dmitrienko, G. Enas, B. Gaydos, et al., Innovative Approaches for Designing and Analyzing Adaptive DoseRanging Trials. White Paper of the PhRMA Working Group on Adaptive Dose-Ranging Studies. J Biopharm Stat. 2007, in press. Available at: http://www.biopharmnet.com/ doc/phrmaadrs white paper.pdf 9. J. Pinheiro, F. Bretz, M. Branson, Analysis of dose response studies: modeling approaches In: N. Ting, (ed.), Dose Finding in Drug Development. New York: Springer, 2006, pp. 146–171. 10. D. M. Bates and D. G. Watts, Nonlinear Regression Analysis and Its Applications. New York: Wiley, 1988. 11. Y. Hochberg, and A. C. Tamhane, Multiple Comparisons Procedures. New York: Wiley, 1987. 12. J. C. Hsu, Multiple Comparisons. New York: Chapman and Hall, 1996. 13. C. W. Dunnett, A multiple comparison procedure for comparing several treatments

6

14.

15.

16.

17.

18.

19.

20.

DOSE-FINDING STUDIES with a control. J Am Stat Assoc. 1955; 50: 1096–1121. D. J. Bartholomew, Ordered tests in the analysis of variance. Biometrika. 1961; 48: 325–332. T. Robertson, F. T. Wright, and R. L. Dykstra, Order Restricted Statistical Inference. New York: Wiley, 1988. S. J. Ruberg, Contrast for identifying the minimum effective dose. J Am Stat Assoc. 1989; 84: 816–822. A. C. Tamhane, C. W. Dunnett, and Y. Hochberg, Multiple test procedures for dose finding. Biometrics. 1996; 52: 21–37. ¨ L. A. Hothorn, M. Neuhauser, H. F. Koch, Analysis of randomized dose-finding studies: closure test modifications based on multiple contrast tests. Biom J. 1997; 39: 467–479. W. H. Stewart and S. J. Ruberg, Detecting dose response with contrasts. Stat Med. 2000; 19: 913–921. F. Bretz, An extension of the Williams trend test to general unbalanced linear models. Comput Stat Data Anal. 2006; 50: 1735–1748.

21. F. Bretz, J. Pinheiro, and A. C. Tamhane, Multiple testing and modeling in dose response problems In: A. Dmitrienko, A. C. Tamhane, and F. Bretz (eds.), Multiple Testing Problems in Pharmaceutical Statistics. New York: Taylor & Francis, 2009 (in press). 22. A. C. Tamhane, C. W. Dunnett, J. W. Green, and J. D. Wetherington, Multiple test procedures for identifying the maximum safe dose. J Am Stat Assoc. 2001; 96: 835–843. 23. K. Strassburger, F. Bretz, and H. Finner, Ordered multiple comparisons with the best and their applications to dose-response studies. Biometrics. 2007; May 8 (e-pub). 24. D. A. Ratkowsky, Handbook of Nonlinear Regression Models. New York: Marcel Dekker, 1989. 25. D. Draper, Assessment and propagation of model uncertainty. J R Stat Soc Ser B Methodol. 1995; 57: 45–97. 26. J. S. Hjorth, Computer Intensive Statistical Methods—Validation, Model Selection and Bootstrap. London: Chapman & Hall, 1994. 27. W. Zucchini, An introduction to model selection. J Math Psychol. 2000; 44: 41–61. 28. J. B. Kadane and N. A. Lazar, Methods and criteria for model selection. J Am Stat Assoc. 2004; 99: 279–290. 29. K. Anraku, An information criterion for parameters under a simple order restriction. Biometrika. 1999; 86: 141–152.

30. S. T. Buckland, K. P. Burnham, and N. H. Augustin, Model selection: an integral part of inference. Biometrics. 1997; 53: 603–618. 31. J. Hoeting, D. Madigan, A. E. Raftery, and C. T. Volinsky, Bayesian model averaging. Stat Sci. 1999; 14: 382–417. 32. K. H. Morales, J. G. Ibrahi, C. J. Chen, and L. M. Ryan, Bayesian model averaging with applications to benchmark dose estimation for arsenic in drinking water. J Am Stat Assoc. 2006; 101: 9–17. 33. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning. New York: Springer, 2001. 34. H. Shimodaira, An application of multiple comparison techniques to model selection. Ann Inst Stat Math. 1998; 50: 1–13. 35. J. W. Tukey, J. L. Ciminera, and J. F. Heyse, Testing the statistical certainty of a response to increasing doses of a drug. Biometrics. 1985; 41: 295–301. 36. F. Bretz, J. Pinheiro, and M. Branson, Combining multiple comparisons and modeling techniques in dose response studies. Biometrics. 2005; 61: 738–748. 37. J. Pinheiro, B. Bornkamp, and F. Bretz, Design and analysis of dose finding studies combining multiple comparisons and modeling procedures. J Biopharm Stat. 2006; 16: 639–656. 38. T. Neal, Hypothesis testing and Bayesian estimation using a sigmoid Emax model applied to sparse dose response designs. J Biopharm Stat. 2006; 16: 657–677. 39. A. Wakana, I. Yoshimura, and C. Hamada, A method for therapeutic dose selection in a phase II clinical trial using contrast statistics. Stat Med. 2007; 26: 498–511. 40. S. J. Pocock, Group sequential methods in the design and analysis of clinical trials. Biometrika. 1977; 64: 191–199. 41. P. C. O’Brien and T. R. Fleming, A multiple testing procedure for clinical trials. Biometrics. 1979; 35: 549–556. 42. C. Jennison, and B. W. Turnbull, Group Sequential Methods with Applications to Clinical Trials. London: Chapman and Hall, 2000. 43. N. Stallard and S. Todd, Sequential designs for phase III clinical trials incorporating treatment selection. Stat Med. 2003; 22: 689–703. 44. G. Hommel, Adaptive modifications of hypotheses after an interim analysis. Biom J. 2001; 43: 581–589. 45. F. Bretz, H. Schmidli, F. K¨onig, A. Racine, and W. Maurer, Confirmatory seamless phase

DOSE-FINDING STUDIES II/III clinical trials with hypotheses selection at interim: general concepts (with discussion). Biom J. 2006; 48: 623–634. ¨ 46. D. A. Berry, P. Muller, A. P. Grieve, M. Smith, T. Parke, et al., Adaptive Bayesian designs for dose-ranging drug trials. In: C. Gatsonis, B. Carlin, and A. Carriquiry (eds.), Case Studies in Bayesian Statistics V. New York: Springer, New York, 2001, pp. 99–181. 47. M. Krams, K. R. Lees, W. Hacke, A. P. Grieve, J. M. Orgogozo, G. A. Ford, Acute stroke therapy by inhibition of neutrophils (ASTIN): an adaptive dose response study of UK-279,276 in acute ischemic stroke. Stroke. 2003; 34: 2543–2548.

CROSS-REFERENCES Hypothesis testing Minimum effective dose (MinED) Multiple comparisons

7

DOSE RANGING CROSS-OVER DESIGNS

doses. Too much anticoagulation and undesired bleeding may result, which threatens a patient’s well-being; too little anticoagulation could mean that thromboembolic events might not be treated or prevented (9). The choice of what series of doses of warfarin to use depends on many factors—age, nutritional status, gender, what other drugs the patient is taking, and so on. These factors make warfarin challenging to use in practice, and frequent laboratory monitoring of blood coagulation level is used to protect patients (8), although this seems not to be uniformly successful (9). Not all drugs, however, require such a high level of monitoring. Other drugs may be more benign in terms of maintaining safety over a range of doses. Early drug development focuses on exploration of the dose–response relationship (for both undesired and desired effects). The focus here (in what is commonly referred to as Phase I and II) is on ‘‘learning’’ (10) about the compound’s properties. Once a safe and effective range of doses is identified, confirmatory clinical trials are performed to provide regulators with sufficient evidence of benefit relative to risk to support access of a product at certain prescribed doses to patients in the marketplace. This article focuses on the use of crossover designs (11–13) in clinical drug development, which are used for dose-ranging to evaluate desired and undesired response. Those interested in the general topic of clinical doseresponse may find recent works in References 14–16 helpful in their research. These works tend to concentrate on application of dose-ranging in parallel group designs (most frequently, those used in oncology), and we cross-reference the findings of these works when appropriate to their application in crossover designs. Those interested in the application of techniques in nonclinical, toxicology drug research will find Hothorn and Hauschke’s work in Chapters 4 and 6 of Reference 17 of interest. After brief discussions of current international guidance and of clinical dose-ranging study design, we discuss two categories of dose ranging crossover designs. In the next

SCOTT D. PATTERSON Wyeth Research & Development, Collegeville, Pennsylvania

BYRON JONES Pfizer Pharmaceuticals, Sandwich, UK ´ NEVINE ZARIFFA

GlaxoSmithKline, Philadelphia, Pennsylvania

1

INTRODUCTION

Why do we care about dose? Many people seemingly are of the opinion that ‘‘if a little is good, more is probably better.’’ However, this opinion is not necessarily the case in drug treatment. It is more complicated than that. As illustrated by Sheiner and Steimer (1) and described in more detail in References 2 and 3, administration of a dose of drug results in exposure to drug at the site of action once drug enters the systemic circulation. At the site of action, drug molecules bind to receptors in the relevant organ, tissue, and so on, and hopefully cause a desired response, like lowering blood pressure, which improves how a patient feels, how long a patient lives, or some other desired clinical outcome. Of course, once in the circulatory system, the drug may go to other sites than that desired and cause an undesired response. For example, consider the fatal example of terfenadine (4). Doses of this drug resulted in cardiac arrythmia and death when given with a metabolic inhibitor. Experiences such as this have confirmed that a thorough understanding of how dose relates to response is critically important for a proper understanding of a drug’s usefulness (or lack thereof) (5–7). An example in which response to dose can be monitored is in the use of a drug called warfarin. Such a drug is dangerous to use, but its use is said to prevent ‘‘twenty strokes for every episode of bleeding’’ (8). The drug is titrated to the desired level of effective and safe anticoagulation in a series of daily

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

DOSE RANGING CROSS-OVER DESIGNS

section, titration designs and extension studies are discussed. Then, fully randomized crossover studies are considered. Each section includes references to examples of data, the statistical model, and methods of inference. A comprehensive exposition of dose-response modeling in crossover designs is beyond the scope of this article, and we concentrate on models which in our experience are most frequently applied, providing references for those interested in more details. 1.1 Objectives of Dose-Ranging and Summary of Current International Regulations The International Conference on Harmonization guidance ICH E4 (18) denotes choice of dose as an ‘‘integral part of drug development.’’ However, the guidance is complex and calls for determination of several factors: for example, the starting dose, the maximum tolerated dose (MTD), the minimum effective dose, the maximum effective dose, and a titration algorithm for physicians to use to determine which dose to use. Thus, a doseranging trial has several objectives that can be covered by statements as nebulous as, for example, ‘‘characterizing dose response’’ to as precise as confirming that ‘‘X mg is the MTD in a certain population.’’ In addition to choice of objective, the choice of endpoint is critical to the selection of design. Drug research contains examples of both acute, reversible responses (e.g., blood pressure), and chronic responses that change (generally for the worse) with time (e.g., survival time), which depend on the therapy area being investigated. Studies used to assess dose-response should be ‘‘well-controlled, using randomization and blinding’’ and ‘‘should be of adequate size’’ (18). In early phase development, ‘‘placebo controlled individual subject titration designs (18)’’ support later, larger trials in patients. ICH E4 recommends the use of parallel-group dose response studies; however, this method is not in keeping with regulatory trends toward the individualization of therapy (5) and it requires greater consideration of different regimens within the same patient. ICH E4 lists several potential study designs

1. Parallel dose-response designs— Subjects are randomized to fixed-dose groups and followed while on that dose to evaluate response. This type of dose ranging design is the most common in drug development but does not generally permit the evaluation of individual response (18). These designs are discussed in References 15 and 16 and are not discussed here in favor of discussion of more powerful (11) designs as follows. 2. Titration designs—Subjects (forced titration) or subjects not yet achieving a response (optional titration) receive predetermined increases in dose until the desired effect is observed, or until dosing is stopped because patients are not tolerating the regimen. Although this approach confounds time-related effects with dose and with cumulative dose response effects, it does permit the evaluation of individual dose-response under certain assumptions. Titration designs are discussed in the section entitled, ‘‘Titration designs and extension studies’’. 3. Crossover designs—Subjects are randomized to sequences of different doses with each period of treatment with a given dose separated by a wash-out period to enable the body’s function to return to basal levels (11, 12). This type of design is discussed in the section entitled, ‘‘Randomized designs.’’ FDA’s implementation of ICH E4 (18) may be found in Reference 19. This guideline enhances several aspects of ICH E4 by calling, in part, for (19): 1. Prospectively defined hypotheses/objectives, 2. Use of an appropriate control group, 3. Use of randomization to ensure comparability of treatment groups and to minimize bias. Both guidelines (18, 19) suggest that results of dose-response studies can serve as supporting evidence for confirmatory studies (conducted subsequently in drug development). This evidence may mitigate requirements for multiple confirmatory trials in

DOSE RANGING CROSS-OVER DESIGNS

some settings. Especially critical to this role as supporting evidence is the choice of endpoint. If the dose-response study or studies consider a clinically relevant endpoint related directly to dose with little to no time lag to response (e.g., blood pressure for a hypertensive agent), then use of many alternative designs (parallel, titration, or crossover) can be informative, and the outcome of the study may support registration directly. Where significant hysteresis (see Reference 3 for a definition) is present, more care in choice of design is warranted, and the relationship of dose ranging to registration is defined less well, see Fig. 1. The lists of designs in References 18 and 19 are by no means exhaustive, and other alternatives may be used. For example, another regulatory perspective may be found in Hemmings’ Chapter 2 of Reference 15, which discusses several alternative crossover study designs: 1. Randomized titration designs in nonresponders—Here, dose is titrated to effect in subjects randomly assigned to a dose or a control group. Those subjects who respond to the dose continue on that dose whereas nonresponders are randomized to continue on the dose (to evaluate whether prolonged exposure

Figure 1. Selected points to consider in different doseresponse study designs from FDA guidance (19).

3

will result in an effect) or to a higher dose. 2. Randomized withdrawal designs— Here, dose is titrated to effect in subjects randomly assigned to a dose of drug or placebo. Subjects randomly assigned to drug who respond to treatment are randomly assigned to switch to placebo or to continue on drug (to enable consideration of duration of effect once drug is discontinued). The latter type of study is known as Discontinuation designs (15), and it is a class of incomplete crossover designs (11). This design is sometimes called ‘‘Retreatment’’ designs (20) or ‘‘Enrichment’’ designs (21–24). We do not examine such studies further here as: 1. They are not generally analyzed (25, 26) in a manner consistent with crossover designs (11), and 2. Published examples examined for this article (23, 27, 28) were not dose-ranging [despite suggestions that such designs have use in this area (24)]. Thus, it can be observed that the objective of dose-response studies may be tactical {i.e., exploratory in the sense of deciding how

4

DOSE RANGING CROSS-OVER DESIGNS

to dose in subsequent studies (10)], strategic (i.e., decide how to dose in confirmatory trials for regulatory submission), and/or regulatory (i.e., provide data directly supporting submissions in place of a confirmatory trial). For the latter, ICH E5 (29) for example calls for collection of dose-response as a prerequisite for bridging findings from one ethnicity to another for registration. However, practical implementation of such an approach is region-specific (30–32). As a practical matter for those reviewing such trial designs (or their results), one can generally deduce what the true purpose of the design is by looking at the choice of endpoint and how many assumptions are made in design. If the endpoint is a biomarker (c.f., 33) and the design is titration (making the assumption that time related effects are small relative to the effect of dose), then the study is probably designed for tactical, exploratory purposes. If, however, the endpoint is a surrogate marker (33) or clinical endpoint (34), and the study employs a fully randomized control and sequence of doses [(e.g., for a crossover a William’s square (11)], then it is most likely for strategic or for regulatory purposes. Generally, the number of subjects in the study is not a good way to deduce the purpose of a study, as regulatory considerations for extent of exposure recommended for registration are more complex than this, see Reference 35. 1.2 Statistical Aspects of the Design of Dose Ranging Studies As in all protocols, the choice of population is of critical importance as are the controlled aspects of study procedures, but we will neglect those subjects here to consider statistical analysis for dose-ranging trials. Statistical consideration of such dose-ranging studies has been geared mainly toward provision of confirmatory hypothesis tests (generally used in the later stages of drug development referred to as Phase IIb-III). Modeling is the procedure that should most often be used in exploratory trials (in the early stages of drug development, referred to as Phase I and IIa). The analysis method chosen depends directly on choice of objective and endpoint.

The choice of endpoint generally determines whether one can consider a crossover (see the section entitled ‘‘Randomized designs’’) trial design in this setting (11, 13). The advantage of such a design is that one can compare the response at different doses of drug ‘‘within-subject,’’ which presumably leads to a more accurate and precise measurement (11). A potential disadvantage is that responses from the previous administration may carryover to the next treatment period, which complicates (and potentially confounds) a clear inference. If one is considering a response that is fairly ‘‘permanent’’ (i.e., survival time), then one is likely constrained by circumstance to use a parallel group or titration design (see the section entitled ‘‘Titration designs and extension studies’’). This latter design is useful here if one can assume that time related effects are small relative to the effects of dose. If one is looking at an endpoint that is reversible (i.e., generally goes back to where it started when one stops treatment), then crossover designs are recommended. The key question many statisticians face in design of dose ranging trials is, ‘‘how many subjects do we need?’’ This question is very complex to answer if one adopts a traditional approach to powering. Closed form solutions amenable to use in confirmatory trials can become very complicated as multiple doses, multiple looks, adaptive or group-sequential adjustment are involved (e.g., see Reference 36). We recommend the simpler procedure of simulation be employed. Simulated crossover data are generated very easily using a variety of commercial software packages (13), and we will not dwell on technical details here. For an example, see Chapter 6 of Reference 13. Simulation of data permits one to consider a variety of modeling and testing techniques at the design stage, as appropriate to the objective and endpoint of the trial, and we now consider two such approaches: modeling and hypothesis tests. Modeling of data from dose response trials follows the general principle 9 of regression and has been described by many authors, for example, References 1, 10, 14, and 37–42. These approaches examine response as a mathematical function of dose. These may

DOSE RANGING CROSS-OVER DESIGNS

be linear, nonlinear, or some step-function of dose as appropriate to the response involved. A well-known model used in this setting is the Power model (40) as follows: yik = (α + ξk ) + πi + β(ld) + γi−1 + εik

9 of Reference 16) is: E=

Emax(CN ) + E0 , EC50N + CN

where E is the effect being modeled, E0 is the effect observed without any drug present, C is the concentration of drug in the effect compartment, EC50 is the concentration needed to cause a 50% response, N is the slope factor (estimating ‘‘sensitivity of the response to the dose range’’), and Emax is the maximum effect that can occur with drug treatment. This example shows a nonlinear (in concentration) additive model. If drug concentration in the effect compartment is not related to effect, then Emax and EC50 would be zero. Note that instead of concentration C, one may use the dose or the log-dose in the expression if no effect compartment is being used. As a practical matter when modeling, we recommend that those who devleop such follow the principle of parsimony—Section 4.10.1 of Reference 44; the simplest model that accurately describes the data is best. The parameters estimated from such a model may be used for picking doses in subsequent studies as described in References 13, 45–47— see additional examples below. Assessment of model fit is done typically by inspection of residual values (i.e., predicted less observed), although more formal methods may sometimes be applied (48). Such

(1)

5

6

logAUC 7 8

9

10

where α is the overall mean pharmacokinetic response at a unit dose (logDose, ld = 0) known in statistics as the population intercept, ξk is the random intercept that accounts for each subject (k) as their own control, πi is the time-related effect of period i on the response, β is the slope parameter of interest regressed on logDose (parameter ld), γi−1 is any effect of the regimen from the previous period [known as a carry-over effect, (11)], and εij denotes within-subject error for each log-transformed response (yik , e.g., AUC or Cmax, see Reference 13). See for example Fig. 2. The authors have found this model particularly useful in the setting of pharmacokinetics (43). Modern computing technology has permitted the application of even more complex models in this setting (1). In brief, nonlinear mixed effect models for pharmacokinetics are described in an ‘‘effect compartment’’ (1) (which is a hypothesized part of the body where pharmacodynamic effect is thought to be induced by drug treatment); a function of this model is related to response using a statistical model (41). An Emax model (Chapter

Figure 2. Estimated logDose versus logAUC curve with individual data points [reproduced from Patterson and Jones (13), Example 7.2.1, with the permission of CRC Press)].

5

2

3

4 logDose

5

6

DOSE RANGING CROSS-OVER DESIGNS

model-based approaches may be extended to situations in which more than one endpoint is of importance (49). These approaches are also amenable to statistical optimization (50, 51) although such techniques are rarely applied in drug development. Bayesian approaches are not rare in oncology (52–54), and extensions to consider more than one endpoint are known (55). Although Bayesian extensions to the models described above may also be applied in exploratory studies in other therapy areas (43, 56–59), such use is not yet widespread (60). Results of models such as those applied later in this article may be used for confirmatory purposes in the testing of hypotheses. These testing procedures are geared toward provision of statistical confirmation of effect and are designed more to make a claim. As such, results must prove responses as desired beyond a reasonable doubt under a traditional hypothesis testing framework for consideration by regulators. ICH E4 (18) does not require this proof, but it is still often done for strategic and confirmatory purposes. One might test for example: H0 : µDi − µP ≤ 0

(2)

for all i where Di denotes dose i = 1, 2, 3, 4, , , relative to Placebo P. This hypothesis is to be tested versus the alternative hypothesis: H1 : µDi − µP > 0

(3)

for at least one Di in the study. Modifications of the traditional statistical hypothesis tests can be used for this purpose to adjust for the multiplicity inherent to such analyses in later phase (IIb–III) development. Many alternatives exist: the Fisher’s LSD test, the Bonferroni-Dunn test, the Tukey’s HSD test, the Newman-Keuls test, the Scheff´e test, the Dunnett test to name a few—see Chapter 24 of Reference 61, as well as References 62 and 63 for details. These alternative procedures are many and varied, and they are geared toward protection against a type 1 error (which is declaring a dose as providing a given response, incorrectly, relative to a null hypothesis that is true). The focus in this article is on human clinical trials, but the confirmatory

techniques discussed can be applied to nonhuman studies of the same purpose (and vice-versa). For readers interested in more on this topic, we recommend Hothorn and Hauschke’s work in Chapters 4 and 6 of Reference 17. Recent summaries of such testing procedures may be found in References 64 and 65. Use of such procedures can have an impact on study design, and it is recommended that those who employ such approaches see References 51 and 66 for discussion of relevant aspects of study design. Note that in such confirmatory trials, control of type 1 error can become complex because group-sequential or adaptive looks are commonly employed (67–71). In theory, one could also use such an approach to inference in a titration (see the next section) or discontinuation design, but given the assumptions necessary to perform such an analysis (i.e., the confounding of time with dose), it is rarely done in practice, and emphasis is placed on model-based interpretation. We now turn to such data. 2 TITRATION DESIGNS AND EXTENSION STUDIES Titration designs are used when one wishes to observe the safety and the efficacy results at one dose before choosing subsequent doses in a given experimental unit. These types of approaches are commonly used in real-world clinical practice (37) to titrate to effect under a forced (predetermined) or, most often, an optional escalation structure. Warfarin is an historical example of such an approach. Eligible patients are titrated at weekly doses of warfarin between 27.5 to 42.5 mg to achieve an international normalized ratio (INR, a measure of the blood’s ability to coagulate—the greater the value, the lesser the coagulation) of 2 to 3 (8). Weekly INR measurements are taken to determine how to adjust the dose upward or downward at the physician’s discretion. Warfarin is by no means the only drug used in this manner—see Reference 72 and Chapter 18 of Reference 3 for other examples. Clinical trials also use such approaches when the endpoint involved is reversible and/or concerns develop with tolerability at

DOSE RANGING CROSS-OVER DESIGNS

higher doses. In such cases, a titration design may be used. For example in first-time-inhuman trials, doses are generally chosen for inclusion in the study based on allometric scaling of findings from experiments in animals or in vitro studies (73, 74). Once this range of potentially safe doses is determined, testing in humans commences with small, well-controlled, data-intensive trials. See Chapter 7 of Reference 13 for more details. Exposure data from such a trial is shown in Table 1. In this case, it is determined that dose-escalation in this three-period, randomized (to placebo), single blind, crossover design would be halted when mean exposure neared 20 ng.h/mL. Dosing in this study began with the first subject at 100 mg and proceeded in cohorts of subjects until exposure levels neared the desired level. Once this was achieved, it was desired to determine what exposure levels were at lower levels (i.e., 50 mg), and several subjects were dosed at that level. Such data are easily analyzed using the power model of expression [Equation (1)] that accounts for each subject as their own control

7

as described in the section entitled, ‘‘Statistical aspects of the design of dose ranging studies’’ (see Chapter 7 of Reference 13 for more details). This procedure may be performed after the study (as illustrated here) or while the study is ongoing to aid in the choice of doses. Note that in this study, each study period was separated by a washout of at least seven days to ensure no effect of the previous period carried over to the next period. Study procedures, which include the assay, were standardized and systematic in all periods to minimize the potential for any time related effects. Therefore, we can assume that the effect of the previous dose does not carry over (i.e., in some manner explain or confound) the results at the next dose (i.e., that γi−1 = 0) and that any time-related effects are negligible (i.e., that πi = 0) relative to the effect of dose. Note also that placebo data are not included in this model, as their null values are challenging to account for in such a small sample size relative to nondetectable levels of the exposure assay. Different approaches to analysis may be taken if this is not the case (75).

Table 1. Example of Exposures (ng.h/mL) in an Optional Titration Cross-over Design Subject

Sequence

50 mg

100 mg

250 mg

500 mg

1000 mg

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 18 19

APA AAP PAA PAA AAP APA APA AAP PAA PAA PPA APA PAA APA PPA AAP APA PAA

· · · · · · · · · · · 1.857 ND 0.963 1.939 1.387 · 1.812

2.688 4.258 3.21 · 4.379 · · · · · · · · · · · 2.802 4.146

· 8.651 · 5.373 8.111 2.624 6.568 · · · · · · · · · 5.256 ·

13.255 · 19.508 5.436 5.436 7.931 · 7.84 8.069 · · · · · · · · ·

· · · · · · 9.257 · 8.664 7.07 14.196 · 15.054 · · · · ·

P: Placebo, A: Active Dose of Drug ND: Not Detectable by Assay

1500 mg · · · · · · · · · 8.595 20.917 17.418 · 11.496 36.783 13.516 · ·

8

DOSE RANGING CROSS-OVER DESIGNS

Thus, the model is a reasonable choice given the design, it looks to describe the data well, and the resulting estimates can be useful for decision making. However, we have not accounted for uncertainty in the choice of subsequent doses conditioned on the observed data, and implicitly, the physician and the medical staff will use such data to help determine their next choice of dose under protocol. If one wishes to take this sort of approach statistically, then methods are available to accommodate such models. This example shows an inductive reasoning approach of Bayesian analysis (13), but we will not discuss this approach here as application is extremely limited to date in drug development (60). In this case, after the study, we see that average exposure levels overlapped the desired maximum at 1500 mg as described in Table 2. Predicted mean responses may be compared using pairwise statistical comparisons as described in the first section, although this comparison is not often done in practice. When one uses the conservative Tukey’s approach (63), for example, one finds that exposure is significantly greater across the increasing doses used in this study until one reaches the 500–1500-mg doses. The average exposure does not seem to increase significantly with dose (adjusted P > 0.10).

Table 2. Predicted Mean (90% Confidence Interval, CI) Exposures (ng.h/mL) in an optional Titration Cross-over Design Dose (mg) 50 100 250 500 1000 1500

relative to a control. For example, a study might be designed in which n eligible subjects randomly receive one of two active doses or placebo in period 1, and treatment is double blind. At the end of this session of therapy, the data is assessed, and if the drug was effective, then every subject receives the higher of the two doses in period 2, see Table 3. If this method is planned from the start of the study [often done for the purposes of collecting additional safety data under (35)], and if the endpoint is not biased by the open-label extension nature of period 2 (i.e., one is assessing a hard endpoint like blood level where knowledge of treatment dose not influence outcome or a ‘‘softer’’ endpoint is assessed by a blinded adjudication committee), then one can regard this design as an incomplete Balaam’s design (11). Subjects are randomized de facto to sequence at study start, and data may be used for the purposes of treatment effect assessment (which one would expect would be more precise than the between-group assessment of period 1 data alone). One can assess continuity of effect with treatment by looking at data from the HH sequence and can confirm that the onset of treatment effects observed between groups in period 1 is similar to that observed withinsubjects in the PH sequence. One can also assess using the LH sequence data whether those ‘‘under-treated’’ initially benefit from later treatment with the higher dose. The models applied are similar to those described

Predicted Mean (90% CI) 1.7 (1.3-2.2) 2.8 (2.2-3.4) 5.2 (4.4-6.3) 8.5 (7.1-10.2) 13.7 (11.2-16.8) 18.2 (14.4-22.8)

We now turn to a related topic in drug development known as extension studies. Extension studies are nonrandomized crossover studies in which everyone receives a certain dose after the close of the double blind randomized portion of the study

Table 3. Example of a Random Allocation of Sequences to Subject in an Extension Design Subject

Sequence

Period 1

1 2 3 · · · n

LH HH PH · · · HH

L H P · · · H

L: Low Dose H: High Dose P: Placebo

Period 2 H H H · · · H

DOSE RANGING CROSS-OVER DESIGNS

in a titration design above or in nonrandomized crossover studies (see Chapter 7 of Reference 13) and so are omitted here. To illustrate, consider the following summary of such a simulated data set (n = 1000) based on a dose-response curve for lipid data found in Reference 76. See Table 4. As one would anticipate from the data of Table 4, the data of period 1 confirms that a dose response is present (Tukey adjusted P < 0.0001); however, when one takes into account each subject as their own control by inclusion of the period 2 data (accounting for sequence and dose in the model), standard errors for the comparisons of interest narrow by 20%, which is consistent with the increased degree of precision expected in accounting for within-subject controls. This analysis is additionally informative in confirming (P < 0.0001) that subjects differ in their response to dose depending on sequence assignment, which is consistent with the differential response observed by dose in period 1. Analysis by sequence confirms that the effect of treatment with 500 mg after the placebo treatment of period 1 is similar in magnitude to the effect between groups in period 1 [i.e., 28.0 (SE = 0.05) versus 27.7 (SE = 0.15)], which suggests that delay in active treatment is not detrimental to a subject’s later response. However, comparison of the data from the HH sequence suggests that continuation of treatment may result in a decreased average response over time of approximately 0.9 (SE = 0.1, P < 0.0001). Thus, we might expect to see treatment effects diminishing over time in long studies.

Table 4. Unadjusted Mean (SE) of Response Data in an Extension Design Sequence

n

Period 1

Period 2

LH HH PH

333 334 333

4.7 (0.1) 28.0 (0.1) 0.1 (0.1)

27.7 (0.1) 26.9 (0.1) 27.9 (0.1)

L: 100 mg H: 500 mg P: Placebo

9

Inclusion of the period 2 data in this setting seems to be informative. Of note, however, if such models are applied, then one must take care to assess what assumptions are being made to determine whether it is appropriate to properly evaluate the results that the length of treatment in period 2 is sufficient to preclude the need for a wash-out from period 1 (i.e., no carryover), and that disease state in the subjects does not change significantly, all else being equal, across the length of the trial (i.e., period effects are null). If either of these assumptions are of concern, perhaps the body acquires resistance to the drug for example or the body is sufficiently improved in period 1 to allow that disease state is modified by period 2, then one may wish to alter the design to better accommodate unbiased assessment of dose-response over time (i.e., randomize to more sequences including LL, etc.), which leads into the next topic: fully randomized, dose-ranging crossover designs. 3

RANDOMIZED DESIGNS

Situations often develop in which a range of safe doses are available, and it is necessary to determine either the dose-response relationship or to decide which doses are superior to placebo. For some medical conditions, it is possible to give subjects a sequence of doses and in this situation crossover designs offer large reductions in sample size compared with traditional parallel group designs. The simplest crossover design compares either a single dose against placebo or one active dose against another. For example, Jayaram (77) compared 10 mg of montelukast against placebo to determine the eosinophils-lowering effect of the dose in asthma patients. The plan of such a design is given in Table 5. In this table, it will be observed that subjects are randomized to receive one of two treatment sequences: Dose-Placebo or PlaceboDose, over two periods of time. In Jayaram (77), for example, the periods were 4 weeks long. Ideally, an equal number of subjects should be allocated to each sequence group. Often, a wash-out period is inserted between the two periods to ensure that the entire drug has been eliminated before the second period

10

DOSE RANGING CROSS-OVER DESIGNS

Table 5. Plan of Two-treatment, Two-period Cross-over Trial

Table 6. Latin Square Design for Four Treatments

Period Group

1

1 2

Dose Placebo

Period 2 Placebo Dose

begins. In situations in which no wash-out period occurs, or in which the wash-out period is of inadequate length, it is possible that the effect of the drug given in the first period carries over into the second period. If carryover effects cannot be removed by a wash-out period, then the simple two-period design is unsuitable and a design with either more than two periods or treatment sequences should be considered. See Reference 11 for a comprehensive description and discussion of crossover designs for two treatments. When more than one dose is to be compared with placebo or a dose-response relationship over three or more doses is to be determined, then crossover designs with three or more periods may be used. As with the two-treatment designs, the medical condition studied must be such that crossover designs are appropriate. With three or more doses and periods, a range of designs exist to choose from. It is important to make the correct choice as some designs require many more subjects than others. One advantage of multiperiod crossover designs is that carryover effects can be allowed for in the design and the analysis and so it is not essential to have wash-out periods between the treatment periods. However, where possible it is good practice to include wash-out periods and/or to ensure the length of the treatment periods is adequate to remove carryover effects. As an example of a design with four periods, we refer to Reference 78, who compared three doses of magnesium and placebo using 12 subjects in three Latin squares. In a single Latin square, as many different treatment sequences exist treatments. An example of such a design for four treatments is given in Table 6. Every subject gets each treatment over the four periods. To ensure the

Sequence

1

2

3

4

1 2 3 4

Placebo Dose1 Dose2 Dose3

Dose1 Dose2 Dose3 Placebo

Dose2 Dose3 Placebo Dose1

Dose3 Placebo Dose1 Dose2

design has adequate power to detect differences of given sizes between the treatments, it is usually necessary to assign more than one subject to each sequence (or to use multiple Latin squares, as done in Reference 78). We note that each treatment occurs an equal number of times in the design, and it is balanced in the sense that all estimated pairwise comparisons among the treatments have the same variance. The ordering of the treatments is of critical importance because a bad choice can lead to a very inefficient design if carryover effects are present. A useful measure of design efficiency is defined in Reference 11. This design compares the variance of an estimated treatment comparison to a theoretical lower bound for the variance. If the lower bound is achieved, then the design has an efficiency of 100%. The design in Table 6 has an efficiency of 18.18% if differential carry-over effects are present, so it is very inefficient. On the other hand, the design in Table 7 has an efficiency of 90.91%. Another way of expressing efficiency is to say, ‘‘the design in Table 6 will require about 90.91/18.18 = 5 times more subjects to achieve the same power as the design in Table 7.’’ In the absence of differential carryover effects, both designs have an efficiency of 100%. Therefore, it is important at the planning stage to decide whether differential carryover effects are likely. The design in Table 7 is an example of a Williams design for four treatments. Such designs exist for all numbers of treatments, but for an odd number of treatments, the number of sequences is twice the number of treatments. Some exceptions to this rule exist. For example, a balanced design for 9 treatments only requires 9 sequences, not 18. An example for three treatments is given in

DOSE RANGING CROSS-OVER DESIGNS Table 7. Balanced Latin Square Design for Four Treatments

11

Table 9. Incomplete Design for Three Treatments

Period

Period

Subject

1

2

3

1 2 3 4

Placebo Dose1 Dose2 Dose3

Dose3 Placebo Dose1 Dose2

Dose1 Dose2 Dose3 Placebo

4 Dose2 Dose3 Placebo Dose1

Table 8. Williams Design for Three Treatments Period Sequence

1

2

1 2 3 4 5 6

Placebo Dose1 Dose2 Placebo Dose1 Dose2

Dose1 Dose2 Placebo Dose2 Placebo Dose1

Sequence

1

2

1 2 3 4 5 6 7 8 9 10 11 12

Dose1 Dose2 Dose3 Dose4 Dose1 Dose2 Dose3 Dose4 Dose1 Dose2 Dose3 Dose4

Dose2 Dose1 Dose4 Dose3 Dose3 Dose4 Dose1 Dose2 Dose4 Dose3 Dose2 Dose1

3 Dose3 Dose4 Dose1 Dose2 Dose4 Dose3 Dose2 Dose1 Dose2 Dose1 Dose4 Dose3

3 Dose2 Placebo Dose1 Dose1 Dose2 Placebo

Table 8. The efficiency of this design is 80% in the presence of differential carryover effects. When many doses exist in a study, for example, six plus a placebo, it is unlikely that a design with as many periods as treatments will be practical or desirable. Fortunately, many designs are available to deal with this situation. Tables of designs for up to nine treatments are given in Reference 11 and cover all practical situations. Some designs have the same number of periods as treatments, other designs have fewer periods than treatments, and still others have more periods than treatments. An example of a design with more treatments than periods is given in Reference 79, in which the authors a crossover trial to assess the dose proportionality of rosuvastatin in healthy volunteers. Four doses were compared over three periods. Although several designs could be suggested, the design in Table 9 would be a suitable choice. It has an efficiency of 71.96% in the presence of differential carry-over effects and an efficiency of 88.89% if differential carryover effects are absent.

An important and more recent use of crossover designs is in safety trials to determine whether certain doses of a drug prolong the QTc interval, see Reference 80. The QTc interval is determined from an electrocardiogram of the heart. If the QTc interval is prolonged sufficiently in humans, then potentially fatal cardiac arrhythmias can result. New drugs, and potentially existing drugs, that seek new indications must study and rule out the potential for QTc interval. An example of such data may be found in Chapter 8 of Reference 13. Regimen E was a dose of an agent known to be a mild prolonger of the QTc interval and was included as a positive control. Regimen C was a therapeutic dose, and D was a supra-therapeutic (greater than that normally administered) dose of a moderate QTc prolonging agent. Forty-one normal, healthy volunteers are in the example data set, and QTc was measured in triplicate at baseline (time 0) and over the course of the day at set times after dosing. Triplicate measurements were averaged at each time of ECG sampling (i.e., 0, 0.5, 1, 1.5, 2.4, 4, etc.) for inclusion in the analysis, and only samples out to four hours post dose are included here. In Figure 3, mild (E) and moderate degrees of prolongation (C) relative to regimen F (placebo) are observed with slightly greater prolongation observed at the supratherapeutic dose of the drug being studied (D). Both mild and moderate prolongation

12

DOSE RANGING CROSS-OVER DESIGNS Table 10. Mean Changes (90% CI) between following a single Dose in (n = 41), reproduced from Patterson and Jones [13] Example 8.1 with the permission of CRC Press Comparison

Time

Difference

C-F

0.5 1 1.5 2.5 4 0.5 1 1.5 2.5 4 0.5 1 1.5 2.5 4

4.4923 8.1830 6.0120 3.7444 5.2944 6.6868 10.4591 7.4421 6.2212 5.7591 2.0069 7.5171 6.2216 6.9994 8.4446

D-F

E-F

90% CI (2.1997, 6.7848) (5.8904, 10.4755) (3.7195, 8.3045) (1.4518, 6.0369) (3.0018, 7.5869) (4.4035, 8.9701) (8.1758, 12.7425) (5.1588, 9.7255) (3.9379, 8.5046) (3.4757, 8.0424) (−0.2778, 4.2915) (5.2324, 9.8017) (3.9369, 8.5062) (4.7147, 9.2840) (6.1599, 10.7292)

C = Therapeutic Dose D = Supra-therapeutic Dose E = Dose of Positive Control F = Placebo

refer to effect sizes greater than zero but less than the ICH E14 (80) level of probable concern for causing arrhythmia of 20 msec (6). In the example, it is observed that moderate and statistically significant (note lower 90% confidence bounds exceed zero) QTc prolongation is observed in C and D within a half-hour of dosing, and it remains prolonged out to four hours post dosing. Significant prolongation for E is not observed until a 1/2 hour after dosing and returns to parity with F immediately after 4 hours post dose (data not shown). If a randomized crossover design can be employed under circumstances in which parallel group or titration designs are considered, then it probably should be used. In addition to the substantial gains in statistical efficiency and power (34) by the use of each subject as their own control, the use of these designs allows for evaluation of time-related and carryover effects and consideration of their impact on the study results. Within-subject inclusion of a positive control seemingly is very valuable to Regulator’s seeking to confirm the validity of a trial (80).

4 DISCUSSION AND CONCLUSION Dose finding studies are of critical importance in drug development. In fact, it is generally accepted that it is one of the ‘‘opportunities’’ (7) waiting to be realized in the development of new medicines. As such, it is vitally important to design, execute, and interpret dose ranging studies early in the development of a new medicine. Broadly speaking, three types of dose ranging studies exist. The first type is the very early studies conducted in Phase I to help ascertain the likely range of doses for additional study of a new molecule. These studies are primarily pharmacokinetic studies that aim to translate the animal exposure to the exposure observed in humans. These trials are often exploratory in terms of the statistical framework. An example of this type of early dose ranging trial was highlighted in the section entitled, ‘‘Titration designs and extension studies.’’ The results of these early trials often set the dosage range for studies conducted in early Phase IIa, with a biomarker or a surrogate endpoint. Although this second type of dose ranging trials can be

420

DOSE RANGING CROSS-OVER DESIGNS

13

Figure 3. Mild and moderate QTc prolongation (n = 41) [reproduced from Patterson and Jones (13), Example 8.1, with the permission of CRC Press].

360

Adjusted Mean QTo 380 400

C D E Placebo

performed in the target patient population and with the registration endpoint of interest [e.g., blood pressure or lipid endpoints in cardiovascular research], it is much more common that these studies are moderately sized trials in low-risk patient populations that aim to refine the understanding of mechanistic effects of the drug under study, see the section entitled, ‘‘Titration designs and extension studies.’’ The last and perhaps most important of the dose ranging trials are the Phase IIb trials conducted in the patient population itself (or a very closely related patient population) with the registration endpoint or a well-accepted clinical surrogate (e.g., QTc, see the section entitled, ‘‘Randomized designs’’). The duration of dosing in these trials is often lengthy, and the use of a crossover design implies a very meaningful lengthening of the trial duration, which makes these potentially impractical. The last type of dose ranging trials is often viewed as time consuming and has historically been an area of opportunity for enhancement in drug development. A key issue in this last category of trials includes the choice of a control arm. Not discussed at length in this article is the option to include an active agent for the purpose of calibrating the trial results with something more convincing than historic data from older agents. This decision is often difficult, but it can add a tremendous amount of value when interpreting a trial’s findings.

In summary, beyond the statistical intricacies of the design and analysis of dose finding studies that are reviewed in this article, the fact remains these trials are often the lynch pin of drug development and deserve the full attention of statisticians.

1

2

3

4

Time (h) following a Single Dose

REFERENCES 1. L. B. Sheiner and J-L. Steimer, Pharmacokinetic-pharmacodynamic modeling in drug development. Annu. Rev. Pharmacol. Toxicol. 2000; 40: 67–95. 2. M. Rowland and T.N. Tozer, Clinical Pharmacokinetics: Concepts and Applications. Philadelphia, PA: Lea and Febidger, 1980. 3. A. Atkinson, C. Daniels, R. Dedrick, C. Grudzinskas, and S. Markey, eds., Principles of Clinical Pharmacology. San Diego, CA: Academic Press, 2001. 4. C. Pratt, S. Ruberg, J. Morganroth, B. McNutt, J. Woodward, S. Harris, J. Ruskin, and L. Moye, Dose-response relation between terfenadine (Seldane) and the QTc interval on the scalar electrocardiogram: distinguishing drug effect from spontaneous variability. Am. Heart J. 1996; 131: 472–480. 5. R. Temple, Policy developments in regulatory approval. Stats. Med. 2002; 21: 2939–2948. 6. R. Temple, Overview of the concept paper, history of the QT/TdP concern; Regulatory implications of QT prolongation. Presentations at Drug Information Agency/FDA Workshop, 2003. Available: www.diahome.org.

14

DOSE RANGING CROSS-OVER DESIGNS

7. FDA Position Paper, Challenge and Opportunity on the Critical Path to New Medical Products. 2004. 8. J. Horton and B. Bushwick, Warfarin therapy: evolving strategies in anticoagulation. Am. Fam. Physician 1999; 59: 635–648.

24. 25.

9. M. Reynolds, K. Fahrbach, O. Hauch, G. Wygant, R. Estok, C. Cella, and L. Nalysnyk, Warfarin anticoagulation and outcomes in patients with atrial fibrillation. Chest 2004; 126: 1938–1945.

26.

10. L.B. Sheiner, Learning versus confirming in clinical drug development. Clin. Pharmacol. Therapeut. 1997; 61, 275–291.

27.

11. B. Jones and M. G. Kenward, Design and Analysis of Cross-over Trials, 2nd ed. London: Chapman and Hall, CRC Press, 2003. 12. S. Senn, Cross-over Trials in Clinical Research, 2nd ed. New York: John Wiley and Sons, 2002.

28.

13. S. Patterson and B. Jones, Bioequivalence and Statistics in Clinical Pharmacology. London: Chapman and Hall, CRC Press, 2006. 14. R. Tallarida, Drug Synergism and Dose-Effect Data Analysis. London: Chapman and Hall, CRC Press, 2000. 15. S. Chevret, ed., Statistical Methods for Dose Finding Experiments. West Sussex, UK: Wiley, 2006.

29.

30.

16. N. Ting, ed., Dose Finding in Drug Development. New York: Springer, 2006. 17. S. C. Chow and J. Liu, eds., Design and Analysis of Animal Studies in Pharmaceutical Development. New York: Marcel Dekker, 1998.

31.

18. International Conference on Harmonization, E4: Dose Response Information to Support Drug Registration. 1994. Available: http://www.fda.gov/cder/guidance/.

32.

19. FDA Guidance, Exposure-Response Relationships—Study Design, Data Analysis, and Regulatory Applications. 2003. Available: http://www.fda.gov/cder/guidance/. 20. C. Dunger-Baldouf, A. Racine, G. Koch, Retreatment studies: design and analysis. Drug Informat. J. 2006; 40: 209–217. 21. R. Temple, Enrichment designs: efficiency in development of cancer treatments. J. Clin. Oncol. 2005; 23: 4838–4839. 22. R. Temple, FDA perspective on trials with interim efficacy evaluations. Stats. Med. 2006; 25: 3245–3249. 23. R. Temple, Special study designs: early

33.

34.

35.

escape, enrichment, studies in nonresponders. Communicat. Stat. Theory Methods 1994; 23: 499–531. R. Temple, Government viewpoint of clinical trials. Drug Informat. J. 1982; 1: 10–17. B. Freidlin and R. Simon, Evaluation of randomized discontinuation design. J. Clin. Oncol. 2005; 23: 5094–5098. J. Kopec, M. Abrahamowicz, and J. Esdaile, Randomized discontinuation trials: utility and efficiency. J. Clin. Epidemiol. 1993; 46: 959–971. W. Stadler, G. Rosner, E. Small, D. Hollis, B. Rini, S. Zaentz, J. Mahoney, and M. Ratain, Successful implementation of the randomized discontinuation trial design: an application to the study of the putative antiangiogenic agent carboxyaminoimidazole in renal cell carcinoma -CALGB69901. J. Clin. Oncol. 2005; 23: 3726–3732. G. Rosner, W. Stadler, and M. Ratain, Randomized discontinuation design: application to cytostatic antineoplastic agents. J. Clin. Oncol. 2002; 20: 4478–4484. International Conference on Harmonization, E5: Guidance on Ethnic Factors in the Acceptability of Foreign Clinical Data. 1998. Available: http://www.fda.gov/cder/guidance/. R. Nagata, H. Fukase, and J. Rafizadeh-Kabe, East-West development: Understanding the usability and acceptance of foreign data in Japan. Internat. J. Clin. Pharmacol. Therapeut. 2000; 38: 87–92. S. Ono, C. Yoshioka, O. Asaka, K. Tamura, T. Shibata, and K. Saito, New drug approval times and clinical evidence in Japan. Contemp. Clin. Trials 2005; 26: 660–672. Y. Uyama, T. Shibata, N. Nagai, H. Hanaoka, S. Toyoshima, and K. Mori, Successful bridging strategy based on ICH E5 guideline for drugs approved in Japan. Clin. Pharmacol. Therapeut. 2005; 78: 102–113. Biomarker Definition Working Group, Biomarkers and surrogate endpoints: preferred definitions and conceptual framework. Clin. Pharmacol. Therapeut. 2001; 69: 89–95. L. Lesko and A. Atkinson, Use of biomarkers and surrogate markers in drug development. Annu. Rev. Pharmacol. Toxicol. 2001; 41: 347–366. International Conference on Harmonization, E1: The Extent of Population Exposure to Assess Clinical Safety for Drugs Intended for Long-term Treatment of NonLife-Threatening Conditions. 1995. Available: http://www.fda.gov/cder/guidance/.

DOSE RANGING CROSS-OVER DESIGNS 36. A. Tamhane, K. Shi, and K. Strassburger, Power and sample size determination for a step-wise test procedure for determining the maximum safe dose. J. Stat. Planning Infer. 2006; 36: 2163–2181. 37. L. B. Sheiner, S. L. Beal, and N. C. Sambol, Study designs for dose-ranging. Clin. Pharmacol. Therapeut. 1989; 46: 63–77. 38. L. B. Sheiner, Y. Hashimoto, and S. Beal, A simulation study comparing designs for dose ranging. Stats. Med. 1991; 10: 303–321. 39. L. B. Sheiner, Bioequivalence revisited. Stats. Med. 1992; 11: 1777–1788. 40. R. E. Walpole, R. H. Myers, and S. L. Myers, Probability and Statistics for Engineers and Scientists, 6th ed. Englewood Cliffs, NJ: Prentice Hall, 1998. 41. S. Machado, R. Miller, and C. Hu, A regulatory perspective on pharmacokinetic and pharmacodynamic modelling. Stats. Methods Med. Res. 1999; 8: 217–245. 42. W. Slob, Dose-response modeling of continuous endpoints. Toxicol. Sci. 2002; 66: 298–312. 43. S. Patterson, S. Francis, M. Ireson, D. Webber, and J. Whitehead, A novel Bayesian decision procedure for early-phase dose-finding studies. J. Biopharmaceut. Stats. 1999; 9: 583–598. 44. F. Harrell, Regression Modelling Strategies. New York: Springer, 2001. 45. T. G. Filloon, Estimating the minimum therapeutically effective dose of a compound via regression modeling and percentile estimation. Stats Med. 1995; 14: 925–932. 46. T. N. Johnson, Modelling approaches to dose estimation in children. Br. J. Clin. Pharmacol. 2005; 59: 663–669. 47. T. Johnson, J. Taylor, R. Haken, and A. Eisbruch, A Bayesian mixture model relating dose to critical organs and functional complication in 3D conformal radiation therapy. Biostatistics 2005; 6: 615–632. 48. P. Lupinacci, and D. Raghavarao, Designs for testing lack of fit for a nonlinear dose-response curve model. J. Biopharmaceut. Stats. 2000; 10: 43–53. 49. J. Whitehead, Y. Zhou, J. Stevens, G. Blakey, J. Price, and J. Leadbetter, Bayesian decision procedures for dose-escalation based on evidence of undesirable events and therapeutic benefit. Stats. Med. 2006a; 25: 37–53. 50. V. Fedorov and P. Hackl, Model-Oriented Design of Experiments. New York: Springer, 1997.

15

51. S. Biedermann, H. Dette, and W. Zhu, Optimal designs for dose-response models with restricted design spaces. J. Am. Stats. Assoc. 2006; 101: 747–759. 52. L. Desfrere, S. Zohar, P. Morville, A. Brunhes, S. Chevret, G. Pons, G. Moriette, E. Reyes, and J. M. Treluyers, Dose-finding study of ibuprofen in patent ductus arteriosus using the continual reassessment method. J. Clin. Pharm. Therapeut. 2005; 30: 121–132. 53. M. Gonen, A Bayesian evaluation of enrolling additional patients at the maximum tolerated dose in phase I trials. Contemp. Clin. Trials 2005; 26: 131–140. 54. Y. Loke, S. Tan, Y. Cai, and D. Machin, A Bayesian dose finding design for dual endpoint phase I trials. Stats. Med. 2006; 25: 3–22. 55. P. Thall and J. Cook, Dose-finding based on efficacy-toxicity trade-offs. Biometrics 2004; 60: 684–693. 56. J. Whitehead, Y. Zhou, S. Patterson, D. Webber, and S. Francis, Easy-to implement Bayesian methods for dose-escalation studies in healthy volunteers. Biostatistics 2001a; 2: 47–61. 57. J. Whitehead, Y. Zhou, N. Stallard, S. Todd, and A. Whitehead, Learning from previous responses in phase 1 dose escalation studies. Br. J. Clin. Pharmacol. 2001b; 52: 1–7. 58. J. Whitehead, Y. Zhou, A. Mander, S. Ritchie, A. Sabin, and A. Wright, An evaluation of Bayesian designs for dose-escalation studies in healthy volunteers. Stats. Med. 2006b; 25: 433–445. 59. Y. Zhou, J. Whitehead, E. Bonvini, and J. Stevens, Bayesian decision procedures for binary and continuous bivariate doseescalation studies. Pharmaceut. Stats. 2006; 5: 125–133. 60. C. Buoen, O. Bjerrum, and M. Thomsen, How first-time-in-humans studies are being performed: a survey of phase I dose-escalation trials in healthy volunteers published between 1995 and 2004. J. Clin. Pharmacol. 2005; 45: 1123–1136. 61. D. Sheshkin, Handbook of Parametric and Nonparametric Statistical Procedures. London: Chapman & Hall, CRC Press, 2000. 62. J. Hsu, Multiple Comparisons: Theory and Methods. London: Chapman & Hall, CRC Press, 1996. 63. J. W. Tukey, J. L. Ciminera, and J. F. Heyse, Testing the statistical certainty of a response to increasing doses of a drug. Biometrics 1985; 41: 295–301.

16

DOSE RANGING CROSS-OVER DESIGNS

64. P. Bauer, J. Rohmel, W. Maurer, and L. Hothorn, Testing strategies in multi-dose experiments including active control. Stats. Med. 1998; 17: 2133–2146. 65. A. Dmitrienko, W. Offen, O. Wang, and D. Xiao, Gate-keeping procedures in doseresponse clinical trials based on the Dunnett test. Pharmaceut. Stats. 2006; 5: 19–28. 66. Y. Cheung, Coherence principles in dosefinding studies. Biometrika 2005; 92: 863–873. 67. J. A. Robinson, Sequential choice of an optimal dose: a prediction intervals approach. Biometrika 1978; 65: 75–78. 68. C. Jennison and B.W. Turnbull, Group Sequential Methods with Applications to Clinical Trials. New York: Chapman and Hall, 2000.

77. L. Jayaram, M. Duong, M. M. M. Pizzichini, E. Pizzichini, D. Kamada, A. Efthimiadis, and F. E. Hargreave, Failure of montelukast to reduce sputum eosinophilia in high-dose corticosteroid-dependent asthma. Euro. Respir. J. 2005; 25: 41–46. 78. M. M. Huycke, M. T. Naguib, M. M. Stroemmel, K. Blick, K. Monti, S. Martin-Munley, and C. Kaufman, A double-blind placebocontrolled crossover trial of intravenous magnesium sulfate for foscarnet-induced ionized hypocalcemia and hypomagnesemia in patients with AIDS and cytomegalovirus infection. Antimicrob. Agents Chemother. 2000; 44: 2143–2148. 79. P. D. Martin, M. J. Warwick, A. L. Dane, and M. V. Cantarini, A double-blind, randomized, incomplete crossover trial to assess the dose proportionality of rosuvastatin in healthy volunteers. Clin. Therapeut. 2003; 25: 2215–2224.

69. M. Krams, K. Lees, W. Hacke, A. Grieve, J.-M. Orgogozo, and G. Ford, for the ASTIN Study Investigators, Acute stroke therapy by inhibition of neutrophils: an adaptive dose-response study of UK-279276 in acute ischemic stroke. Stroke 2003; 34: 2543–2548.

80. International Conference on Harmonization, E14: The Clinical Evaluation of QT/QTc Interval Prolongation and Proarrythmic Potential for Non-Antiarrythmic Drugs. 2005. http://www.fda.gov/cder/guidance/.

70. V. Dragalin and V. Fedorov, Adaptive ModelBased Designs for Dose-Finding Studies. GSK BDS Technical Report 2004–02, 2004.

FURTHER READING

71. L. Kong, G. Koch, T. Liu, and H. Wang, Performance of some multiple testing procedures to compare three doses of a test drug and placebo. Pharmaceut. Stats. 2004; 4: 25–35. 72. H. Moldofsky, F. Lue, C. Mously, B. RothSchechter, and W. Reynolds, The effect of zolpidem in patients with fibromyalgia: a dose ranging, double blind, placebocontrolled, modified cross-over study. J. Rheumatol. 1996; 23: 529–533. 73. FDA Draft Guidance, Estimating the Safe Starting Dose in Clinical Trials for Therapeutics in Adult Healthy Volunteers. 2006. Available: http://www.fda.gov/cder/guidance/. 74. B. Reigner and K. Blesch, Estimating the starting dose for entry into humans: principles and practice. Euro. J. Clin. Pharmacol. 2001; 57: 835–845. 75. C. Chuang-Stein and W. Shih, A note on the analysis of titration studies. Stats. Med. 1991; 10: 323–328. 76. J. Mandema, D. Hermann, W. Wang, T. Sheiner, M. Milad, R. Bakker-Arkema, and D. Hartman, Model-based development of gemcabene, a new lipid-altering agent. AAPS J. 2005; 7: E513–E522.

International Conference on Harmonization, E9: Statistical Principles for Clinical Trials. 1998. Available: http://www.fda.gov/cder/guidance/.

DOUBLE-DUMMY

3

KENNETH F. SCHULZ

Perhaps most crucially, blinding aids in reducing differential assessment of outcomes (commonly termed information or ascertainment bias) prompted by knowledge of the group assignment of individuals being observed (3–5). For example, if unblinded outcome assessors believe a new intervention is better, they could record more ‘‘charitable’’ responses to that intervention. Indeed, in a placebo-controlled multiple sclerosis trial, the unblinded, but not the blinded, neurologists’ assessments showed an apparent benefit of the intervention (10). Subjective outcomes (e.g., pain scores) present greater opportunities for bias (9). Even some outcomes considered as objective, such as myocardial infarction, can be fraught with subjectivity. In general, blinding becomes less important to reduce information bias as the outcomes become less subjective. Objective (hard) outcomes, such as death, leave little opportunity for bias. Less understood, blinding also operationally improves compliance and retention of trial participants and reduces biased supplemental care or treatment (sometimes called co-intervention) (3–5). Many potential advantages emanate from participants, investigators, and outcome assessors not knowing the intervention group to which the participants have been assigned (Table 1) (11).

Family Health International Research Triangle Park, North Carolina

1

INTRODUCTION

Blinding in research began over two centuries ago (1). Most researchers and readers grasp the concept. Unfortunately, beyond that general comprehension lies confusion. Terms such as ‘‘single-blind,’’ ‘‘double-blind,’’ and ‘‘triple-blind’’ mean different things to different people (2). Although clinical trial textbooks (3–5), clinical trial dictionaries (6, 7), and a new edition of Last’s epidemiology dictionary (8) address blinding, they do not entirely clear the lexicographical fog (9). Investigators, textbooks, and published articles all vary greatly in their interpretations (2). In other words, terminological inconsistencies surround blinding, and doubledummy terms add a whole additional level of confusion. I will discuss blinding in general and then describe the relevance of doubledummy blinding 2

POTENTIAL IMPACTS OF BLINDING

‘‘DOUBLE-BLINDING’’ DEFINED

The terminology ‘‘double-blind’’ (or doublemask) usually means that trial participants, investigators (usually health care providers), and assessors (those collecting outcome data) all remain oblivious to the intervention assignments throughout the trial (9) so that they will not be influenced by that knowledge. Given that three groups are involved, ‘‘double-blind’’ appears misleading. In medical research, however, an investigator also frequently assesses, so, in that instance, the terminology accurately refers to two individuals. When I use ‘‘double-blind’’ or its derivatives in this article, I mean that steps have been taken to blind participants, investigators, and assessors. In reporting randomized control trials (RCTs), authors should explicitly state what steps were taken to keep whom blinded, as clearly stated in the CONSORT guidelines.

4

PLACEBOS AND BLINDING

Interventions (treatments) at times have no effect on the outcomes being studied (9). When an ineffective intervention is administered to participants in the context of a well-designed RCT, however, beneficial effects sometimes occur on participants attitudes, which in turn influences outcomes (3). Researchers refer to these phenomena as the ‘‘placebo effect.’’ A placebo refers to a pharmacologically inactive agent that investigators administer to participants in the control group in a trial (9). The use of a placebo control

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

DOUBLE-DUMMY Table 1. Potential Advantages of Successfully Blinding Participants, Investigators, and Assessors (11) If blinded?

Potential advantages

Participants

• • • •

Less likely to have biased responses to the interventions More likely to comply with trial treatments Less likely to seek supplementary interventions More likely to continue in the trial, providing outcome

Trial investigators

• • • • •

Less likely to transfer their preferences or viewpoints to participants Less likely to differentially dispense co-interventions Less likely to differentially alter dosage Less likely to differentially withdraw participants Less likely to differentially dissuade participants to continue in the trial

Assessors

• Less likely to have biases influence their outcome assessments

group balances the placebo effect in the treatment group allowing for the independent assessment of the treatment effect. Although placebos may have effects mediated through psychological mechanisms, they are administered to participants in a trial because they are otherwise ‘‘inactive.’’ Although the effect of placebos is contentious (12), when assessing the effects of a proposed new treatment for a condition for which no effective treatment already exists, the widespread view remains that placebos should be administered, whenever possible, to participants in the control group (3, 4). Placebos are generally used in trials of drugs, vaccines, and other medicinal interventions, but can sometimes also be used in trials of procedures, such as ultrasound, acupuncture, and occasionally, surgery. 5

DOUBLE-DUMMY BLINDING

As most double-blinding involves drugs, I confine my comments to such instances. When an effective standard treatment exists, it is usually used in the control group for comparison against a new treatment (9). Thus, trialists compare two active treatments. In that circumstance, they usually have three options to double-blind their trial. First, they could obtain the drugs in raw form and package the two treatments identically, such as in capsules or pills. Preferably, taste would also be equilibrated. Participants

would simply take one normal-sized capsule or pill. Frequently, however, this approach presents production problems. Pharmaceutical manufacturers seldom provide their drugs in any form other than the standard formulations. Moreover, even if they did, generating different formulations might raise objections with government regulatory bodies, such as the U.S. Food and Drug Administration (FDA), as to the equivalent bioavailability of the created formulations, even with drugs with approved formulations (13). Erasing those objections probably involves more research, more expense, and more time. This identical packaging approach frequently presents formidable production obstacles. Second, the standard formulation drugs could be encapsulated in a larger capsule, which alleviates some production concerns, but, again, may trigger equivalent bioavailability questions (13), which generates additional research expense and delays. Moreover, although the participants would only take one capsule, it may be so large as to discourage ingestion. Furthermore, some encapsulation systems allow the participant to open the capsules thereby breaking the blind (13). This encapsulation approach still involves production difficulties and also presents compliance impediments and unblinding opportunities. Third, they could conduct a double-dummy (double-placebo) design where participants receive the assigned active drug and the

DOUBLE-DUMMY

placebo matched to the comparison drug. The trial involves two active drugs and two matching placebos. For example, in comparing two agents, one in a blue capsule and the other in a red capsule, the investigators would acquire blue placebo capsules and red placebo capsules. Then both treatment groups would receive a blue and a red capsule, one active and one inactive. This option protects the double-blinding better than the second option and as well as the first. Unlike the other two options, it does not raise equivalent bioavailability questions nor does it involve production problems, delays, and costs. The only downside tends to be that participants take more pills, which could hurt enrollment or compliance. However, when examined, investigators found minimal impact on enrollment (13). Pragmatically, investigators should find fewer procedural problems with double-dummy blinding. No wonder that this approach appears the most frequently used of the three options. For simplicity of presentation, most of the discussions focused on drugs in capsule, pill, or tablet formulations. The concepts, however, extend easily to other formulations, such as intravenous fluids and ampoules administered through injections or intravenous drips (Panel 1). Where double-dummy blinding becomes difficult to implement are those situations in which blinding itself is difficult under any circumstances, such as in surgical trials. Unfortunately, some authors have disparaged placebos. Many of those efforts have been misguided. Placebos have served as the scapegoat for the problem of having an inactive treatment control group when ethically an active treatment control group is indicated because an effective treatment exists. Inappropriate control groups are the problem, not placebos. Thus, placebos emerge as critical to double-blinding in randomized trials. Inescapably, placebos are usually scientifically necessary for double-blinding, regardless of whether the control group is active or inactive. In trials with a comparison group receiving no active treatment, placebos have obvious importance. That manifestation fits the common paradigm of placebo

3

usage. However, as displayed above, placebos also frequently emerge as necessary if a trial compares two or more active treatments. If those treatments differ,\emdash for example, in shape, size, weight, taste, or color, the double-dummy technique (using two placebos) nearly always indispensably serves methodological and production concerns. 5.1 Panel 1: Examples of Descriptions of ‘‘Double-Dummy’’ Blinding ‘‘The patients were allocated to doubledummy treatment with dalteparin 100 IU/kg subcutaneously twice a day (Fragmin, Pharmacia and Upjohn, Stockholm, Sweden) and placebo tablets every day, or aspirin tablets 160 mg every day (Albyl-E, Nycomed Pharma, Oslo, Norway) and placebo ampoules subcutaneously twice a day’’ (14). ‘‘To maintain masking, each patient received two simultaneous infusions, one active and one placebo. The placebo infusion regimen was identical to its respective active counterpart’’ (15). REFERENCES 1. T. J. Kaptchuk, Intentional ignorance: a history of blind assessment and placebo controls in medicine. Bull. Hist. Med. 1998; 72: 389–433. 2. P. J. Devereaux, B. J. Manns, W. A. Ghali, H. Quan, C. Lacchetti, V. M. Montori et al., Physician interpretations and textbook definitions of blinding terminology in randomized controlled trials. JAMA 2001; 285: 2000–2003. 3. S. J. Pocock, Clinical Trials: A Practical Approach. Chichester: Wiley, 1983. 4. C. L. Meinert, Clinical Trials: Design, Conduct, and Analysis. New York: Oxford University Press, 1986. 5. L. Friedman, C. Furberg, and D. DeMets, Fundamentals of Clinical Trials. St. Louis, MO: Mosby, 1996. 6. C. L. Meinert, Clinical Trials Dictionary. Baltimore, MD: The Johns Hopkins Center for Clinical Trials, 1996. 7. S. Day, Dictionary for Clinical Trials. Chichester: Wiley, 1999. 8. J. M. Last (ed.), A Dictionary of Epidemiology. Oxford: Oxford University Press, 2001.

4

DOUBLE-DUMMY

9. K. F. Schulz, I. Chalmers, and D. G. Altman, The landscape and lexicon of blinding in randomized trials. Ann. Intern. Med. 2002; 136: 254–259. 10. J. H. Noseworthy, G. C. Ebers, M. K. Vandervoort, R. E. Farquhar, E. Yetisir, and R. Roberts, The impact of blinding on the results of a randomized, placebo- controlled multiple sclerosis clinical trial. Neurology 1994; 44: 16–20. 11. K. F. Schulz and D. A. Grimes, Blinding in randomised trials: hiding who got what. Lancet 2002; 359: 696–700. 12. A. Hrobjartsson and P. C. Gøtzsche, Is the placebo powerless? An analysis of clinical trials comparing placebo with no treatment. N. Engl. J. Med. 2001; 344: 1594–1602. 13. B. K. Martin, C. L. Meinert, and J. C. Breitner, Double placebo design in a prevention trial for Alzheimer’s disease. Control Clin. Trials 2002; 23: 93–99. 14. E. Berge, M. Abdelnoor, P. H. Nakstad, and P. M. Sandset, Low molecular-weight heparin versus aspirin in patients with acute ischaemic stroke and atrial fibrillation: a double-blind randomised study. HAEST Study Group. Heparin in acute embolic stroke trial. Lancet 2000; 355: 1205–1210. 15. F. Follath, J. G. Cleland, H. Just, J. G. Papp, H. Scholz, K. Peuhkurinen et al., Efficacy and safety of intravenous levosimendan compared with dobutamine in severe low-output heart failure (the LIDO study): a randomised double-blind trial. Lancet 2002; 360: 196–202.

DRIFT (FOR INTERIM ANALYSES)

The solution can be found by numerical integration, and a variety of software for doing so is available (10–13). Slud and Wei (14) developed an equation similar to Equation (2). Lan and DeMets related the α k to time through an alpha spending function α*(t). In the spending function approach, an increasing function α*(t) is defined such that α*(0) = 0, α*(1) = α, the overall type 1 error rate, and α k = α* (tk ) – α*(tk −1). In this way, a certain proportion of the overall type 1 error is ‘‘spent’’ at each interim analysis. To relate the Brownian motion process to accumulating evidence in a clinical trial directly, we use an analogy with partial sums (5,15,16). Consider N independent normal variables X1 , X2 , . . . , XN with unknown mean δ and variance equal to one. The sum of the first nk , S(nk ), is distributed normally with mean nk δ and variance nk , and the Brownian motion process at tk is distributed normally with mean tk θ and variance tk . By equating the proportion of observations in the partial sum and the Brownian motion time scale, √ tk = nk /N, the relation √ of means is δ = θ / N so that B(tk ) and S(nk )/ N have the same distri√ bution. √ Also, the joint distribution of S1 / N, . . . , SK / N is multivariate normal with the same covariance described by Equation (1). The Z statistics used for testing at each interim analysis are also related to the Brownian motion and partial sums. If Z(tk ) is the standard normal summary statistic√at interim analysis k, then B(tk ) and Z(tk ) tk have the same distribution and covariance structure. Note that this covariance structure holds for a wide variety of trials with different study outcomes, including any for which the summary is a maximum likelihood estimate (17,18). For factorial or crossover designs or studies with multiple primary endpoints, the situation is more complex, but the analogy holds for any trial that can be summarized by a single Z statistic. Thus, there is a correspondence among a Brownian motion process B(t), interim test statistics, and partial sums representing accumulating data in the study. A technical point associated with this description is that the time t must be understood as ‘‘information time’’ (16,19,20). That is, although actual analyses occur in calen-

DAVID M. REBOUSSIN Wake Forest University School of Medicine, Division of Public Health Sciences, Winston-Salem, North Carolina

1

INTRODUCTION

An early step in the development of group sequential testing as it is commonly applied to phase III clinical trials was the investigation of repeated significance tests (1). This led to consideration of sequentially recruited groups of equal sized (2,3) and then into procedures that allow some flexibility in the number and timing of interim analyses as a trial progresses using the spending function approach (4–6). We will focus on the spending function approach, but similar development can be applied to other approaches to group sequential testing (7,8). Brownian motion is a stochastic process that starts at zero and has independent, normally distributed increments (9). Group sequential testing is implemented using computed probabilities associated with a standard Brownian motion process, B(t), with mean zero observed at times 0 < t1 , . . . , distribution of B(t) at t = tK ≤ 1. The joint
(t1 , ... tk ) is N(θ t, ), where  kl

= min(tk , tl )

(1)

The parameter θ denotes the drift or mean of the Brownian motion process over time. Figure 1 illustrates an example of Brownian motion over time with a positive drift. A set of (two-sided symmetric) bounds ±b1 , . . . , ±bK associated with the values B(t1 ), . . . , B(tK ) can be determined iteratively. B(t1 ) has a normal distribution with mean zero and variance t1 so that calculation of b1 is straightforward; however, determination of subsequent bounds must account for not having stopped earlier. For k > 1, bk solves the equation Pr|{B(tj )| ≤ bj , j = 1, . . . , k − 1; |B(tk )| > bk } = αk (2)

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

DRIFT (FOR INTERIM ANALYSES)

dar time—a certain number of days since the start of randomization—the time scale that best describes the unfolding evidence about a treatment effect is the proportion of the expected statistical information that has accumulated when the analysis is conducted. This idea can be extended to various designs (19), and in many situations, statistical information accrues more or less uniformly in calendar time, so that the distinction is not critical. However, being mindful of the difference between calendar time and information time can prevent confusion in certain situations, for example, in survival studies when the event rate is lower than anticipated and calendar time runs out before the expected number of events has occurred. When determining bounds, the computation is done assuming B(t) has drift zero as it does under the null hypothesis and that there is no treatment effect. However, the same equation can be used to determine probabilities associated with nonzero drift means given a fixed set of bounds: The drift corresponding to given cumulative exit probability can be determined. This is the basis for determining the effect of a specific monitoring plan on sample size requirements, estimation, and computing of the ‘‘conditional power,’’ all of which are detailed below. 2 SAMPLE SIZE DETERMINATION FOR TRIALS INVOLVING GROUP SEQUENTIAL TESTS With the background above, sample size determinations that take into account the group sequential design can be done by relating the test statistic summarizing the treatment effect to a Brownian motion (5,13,15). Consider the required nonsequential sample size per group for a comparison of means in two normal populations, which is Nfixed = (z1−α/2 + z1 −β )2 (2σ 2 /(µT − µC )2 ) When there is no sequential monitoring, the drift corresponding to power 1−β is θ fixed = z1−α/2 + z1 −β . For two-sided tests with alpha = 0.05 and 80%, 85%, and 90% power, θ fixed is approximately 2.8, 3, and 3.25, respectively. For a given set of interim analysis times

and bounds, the corresponding drift can be computed as described above and inserted in place of z1−α/2 + z1 −β so that 2 (2σ 2 /(µT − µC )2 ) Nseq = θseq

For example, the drift corresponding to five equally spaced, two-sided O’Brien-Flemingtype bounds for alpha = 0.05 and 90% power is 3.28, so if σ 2 = 1 and µT − µC = 0.5, Nseq = 3.282 × (2/0.25) = 86. Note that the sequential design requires a slightly larger sample size, and the increase is the square of the ratio of the sequential drift to the fixed drift. In fact the details of the relationship between design parameters and the sequential drift are not essential in order to adjust a fixed sample size to account for sequential monitoring. The drift associated with the same design monitored sequentially with given bounds and times can be computed using the equation above and available software (13). Then the required proportional increase in sample size is just the ratio of the drift under the planned sequential analyses to the drift for the fixed design. In the example above, the increase is (3.28/3.25)2 or about 2%. This small increase reflects the early conservatism of the O’Brien-Fleming strategy, which is often desirable. Minor deviations in the number and timing of interim analyses has little effect on the drift (5), so it is reasonable to design a study based on an anticipated monitoring plan and expect that the actual monitoring will have negligible effect on power. For sample size determination, then, understanding that the drift of an underlying Brownian motion is related to the expectation of interim Z statistics provides a straightforward way to adjust the sample size for an anticipated sequential monitoring plan. 3 ESTIMATION AFTER GROUP SEQUENTIAL TESTING There are a variety of approaches to estimation and bias reduction in the context of a group sequential test, and some refer to an underlying Brownian motion with nonzero drift (21–27). As an example that is closely related to the discussion of sample size in the

DRIFT (FOR INTERIM ANALYSES)

3

previous section, we can consider the median unbiased estimator (23). This is related to a confidence interval procedure that defines the 1 − γ lower confidence limit as the smallest value of the drift for which an outcome as least as extreme as the observed has probability of at least γ , and likewise for the upper limit (22). Equation (2) in this context is used to determine the value of the drift. The last bound bk is replaced with the final observed Z statistic, and α k is replaced with γ or 1 − γ . The median unbiased estimator sets γ = α k = 0.5, and for a 95% confidence interval, the upper limit sets α k = 0.025 and the lower α k = 0.975. Thus, some approaches to estimation make use of the drift of an underlying Brownian motion to form confidence intervals and point estimates.

assumptions for θ . Typically the assumptions of particular interest are θ = 0, θ = B(t)/t (the current best estimate of the drift), and θ = θ des (the value assumed for the study design). For example, consider a study for which t = 0.6 and Z(0.6) = 1.28. Then B(0.6) = 0.99 and U(t) has a mean of θ × 0.4 and a variance of 0.4. The current estimate of drift is B(0.6)/0.6 = 1.65, so under this assumption, U(t) has a mean of 0.66. Using the current estimate for drift, the conditional probability that the final Z statistic, Z(1), will exceed 1.96 given that √ Z(0.6) = 1.28 is Pr(Z > (1.96 – 1.28 – 0.66)/ 0.4) = Z0.49 = 0.31 or 31%. In this way, use of the underlying Brownian motion drift provides a simple way of assessing partway through the study the likelihood of a statistically significant final result.

4

5

STOPPING FOR FUTILITY

The drift of a Brownian motion process is also a useful model for assessing probabilities associated with the decision to terminate a trial when interim analyses show that the null hypothesis is unlikely to be rejected. Lachin (28) provides a detailed review of methods based on ‘‘conditional power’’ or the probability that the final study result will be statistically significant given the accumulated data and an assumption about future trends. This approach has also been described as stochastic curtailment (29–31). Lan and Wittes (32) describe the B-value, which is related to a Brownian motion process and provides a straightforward way to compute conditional power. The B-value is a translation of the interim Z statistic to a scale on which it can be assessed with respect to the drift of the Brownian motion underlying the study’s accumulating √ data. At time t, the B-value is B(t) = Z(t) t and the time remaining before the planned end of the study is 1 − t. The entire study can be thought of as decomposed into two pieces before and after t: B(t) and U(t) = B(1) – B(t). The distribution of U(t), representing the as-yet unobserved data, is normal and independent of B(t) with mean θ × (1 − t) and variance 1 − t. B(t) + U(t) equals B(1) and Z(1), the Z value at the end of the study. Given the observed B(t), the probability that B(t) + U(t) exceeds a critical value and can be computed under various

CONCLUSION

Calculation of bounds for a group sequential procedure and the associated testing can be done without reference to Brownian motion or drift. However, the connection between interim test statistics and Brownian motion drift is tractable and facilitates understanding of such useful elaborations as sample size determination, estimation, and stopping for futility. REFERENCES 1. P. Armitage, C. K. McPherson, and B. C. Rowe, Repeated significance tests on accumulating data. J. R. Stat. Soc. Series A (General) 1969; 132(2):235–244. 2. P. C. O’Brien and T. R. Fleming, A multiple testing procedure for clinical trials. Biometrics 1979 Sep; 35(3):549–556. 3. S. J. Pocock, Group sequential methods in design and analysis of clinical-trials. Biometrika 1977; 64(2):191–200. 4. K. K. G. Lan and D. L. DeMets, Discrete sequential boundaries for clinical-trials. Biometrika 1983; 70(3):659–663. 5. K. Kim and D. L. DeMets, Design and analysis of group sequential tests based on the type i error spending rate function. Biometrika 1987 Mar; 74(1):149–154. 6. D. L. DeMets and K. K. Lan, Interim analysis: the alpha spending function approach. Stat. Med. 1994 Jul 15; 13(13-14):1341–1352.

4

DRIFT (FOR INTERIM ANALYSES) 7. S. S. Emerson and T. R. Fleming, Symmetric group sequential test designs. Biometrics 1989 Sep; 45(3):905–923. 8. J. Whitehead and I. Stratton, Group sequential clinical trials with triangular continuation regions. Biometrics 1983 Mar; 39(1):227–236. 9. P. Billingsley, Probability and Measure, 3rd ed. New York: Wiley, 1995.

10. Cytel Inc., East v. 5: Advanced Clinical Trial Design, Simulation and Monitoring System. Cambridge, MA: Cytel Inc., 2007. 11. MSP Research Unit, PEST 4: Operating Manual. Reading, UK: The University of Reading, 2000. 12. Insightful Corp., S + SeqTrial. Seattle, WA: Insightful Corp., 2006. 13. D. M. Reboussin, D. L. DeMets, K. Kim, and K. K. G. Lan, Computations for group sequential boundaries using the Lan-DeMets spending function method. Controlled Clin. Trials 2000; 21(3):190–207. 14. E. Slud and L. J. Wei, Two-sample repeated significance tests based on the modified Wilcoxon statistic. J. Am. Stat. Assoc. 1982 Dec; 77(380):862–868. 15. K. Kim and D. L. DeMets, Sample size determination for group sequential clinical trials with immediate response. Stat. Med. 1992 Jul; 11(10):1391–1399. 16. K. K. Lan and D. M. Zucker, Sequential monitoring of clinical trials: the role of information and Brownian motion. Stat. Med. 1993 Apr 30; 12(8):753–765. 17. C. Jennison and B. W. Turnbull, Groupsequential analysis incorporating covariate information. J. Am. Stat. Assoc. 1997 Dec; 92(440):1330–1341. 18. D. O. Scharfstein, A. A. Tsiatis, and J. M. Robins, Semiparametric efficiency and its implication on the design and analysis of group-sequential studies. J. Am. Stat. Assoc. 1997 Dec; 92(440):1342–1350. 19. K. K. G. Lan, D. M. Reboussin, and D. L. DeMets, Information and information fractions for design and sequential monitoring of clinical-trials. Commun. Stat. Theory Methods 1994; 23(2):403–420. 20. K. Lan and D. L. DeMets, Group sequential procedures: Calendar versus information time. Stat. Med. 1989 Oct; 8(10):1191–1198. 21. C. Jennison and B. W. Turnbull, Interim analyses: The repeated confidence interval approach. J. R. Stat. Soc. Series B (Methodological) 1989; 51(3):305–361.

22. K. Kim and D. L. DeMets, Confidence intervals following group sequential tests in clinical trials. Biometrics 1987 Dec; 43(4):857–864. 23. K. Kim, Point estimation following group sequential tests. Biometrics 1989 Jun; 45(2):613–617. 24. M. LeBlanc and J. Crowley, Using the bootstrap for estimation in group sequential designs: An application to a clinical trial for nasopharyngeal cancer. Stat. Med. 1999 Oct 15; 18(19):2635–2644. 25. Z. Q. Li ZQ and D. L. DeMets, On the bias of estimation of a Brownian motion drift following group sequential tests. Statistica Sinica 1999 Oct; 9(4):923–937. 26. A. Y. Liu and W. J. Hall, Unbiased estimation following a group sequential test. Biometrika 1999 Mar; 86(1):71–78. 27. J. C. Pinheiro and D. L. DeMets, Estimating and reducing bias in group sequential designs with Gaussian independent increment structure. Biometrika 1997 Dec; 84(4):831–845. 28. J. M. Lachin, A review of methods for futility stopping based on conditional power. Stat. Med. 2005 Sep 30; 24(18):2747–2764. 29. M. Halperin, K. K. Lan, E. C. Wright, and M. A. Foulkes, Stochastic curtailing for comparison of slopes in longitudinal studies. Control Clin. Trials 1987 Dec; 8(4):315–326. 30. K. K. G. Lan, D. L. DeMets, and M. Halperin, more flexible sequential and nonsequential designs in long-term clinicaltrials. Commun. Stat. Theory Methods 1984; 13(19):2339–2353. 31. M. Halperin, K. K. Lan, J. H. Ware, N. J. Johnson, and D. L. DeMets, An aid to data monitoring in long-term clinical trials. Control Clin Trials 1982 Dec; 3(4):311–323. 32. K. K. Lan and J. Wittes, The B-value: a tool for monitoring data. Biometrics 1988 Jun; 44(2):579–585.

CROSS-REFERENCES Group Sequential Designs Interim Analyses Stopping Boundaries Alpha-spending function Conditional power

DRUG DEVELOPMENT

Pre-clinical research

Clinical studies E

Synthesis and purification

NDA review

Phase 1 E

Phase 2 Phase 3

Accelerated development/review Animal testing

E

Short -term

Treatment IND Parallel track

Long -term Institutional review boards Industry time FDA time IND submitted

NDA submitted Review decision Sponsor/FDA meetings encouraged Early access: Sponsor answers E subpart E Advisory committees any questions from review

Under the Food and Drug Administration (FDA) requirements, a sponsor first must submit data that shows that the drug is reasonably safe for use in initial, small-scale clinical studies. Depending on whether the compound has been studied or marketed previously, the sponsor may have several options for fulfilling this requirement: (1) compiling existing nonclinical data from past in vitro laboratory or animal studies on the compound, (2) compiling data from previous clinical testing or marketing of the drug in the United States or another country whose

population is relevant to the U.S. population, or (3) undertaking new preclinical studies designed to provide the evidence necessary to support the safety of administering the compound to humans. During preclinical drug development, a sponsor evaluates the toxic and pharmacologic effects of the drug through in vitro and in vivo laboratory animal testing. Genotoxicity screening is performed, as well as investigations on drug absorption and metabolism, the toxicity of the metabolites of the drug, and the speed with which the drug and its metabolites are excreted from the body. At the preclinical stage, the FDA generally will ask, at a minimum, that sponsors: (1) develop

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/handbook/develop.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

DRUG DEVELOPMENT

a pharmacologic profile of the drug, (2) determine the acute toxicity of the drug in at least two species of animals, and (3) conduct shortterm toxicity studies ranging from 2 weeks to 3 months, depending on the proposed duration of use of the substance in the proposed clinical studies. The research process is complicated, time consuming, and costly, and the end result is never guaranteed. Literally hundreds and sometimes thousands of chemical compounds must be made and tested in an effort to find one that can achieve a desirable result. The FDA estimates that it takes approximately 8.5 years to study and test a new drug before it can be approved for the general public. This estimate includes early laboratory and animal testing and later clinical trials that use human subjects. No standard route exists through which drugs are developed. A pharmaceutical company may decide to develop a new drug aimed at a specific disease or medical condition. Sometimes, scientists choose to pursue an interesting or promising line of research. In other cases, new findings from university, government, or other laboratories may point the way for drug companies to follow with their own research. New drug research starts with an understanding of how the body functions, both normally and abnormally, at its most basic levels. The questions raised by this research help determine a concept of how a drug might be used to prevent, cure, or treat a disease or medical condition. This concept provides the researcher with a target. Sometimes scientists find the right compound quickly, but usually hundreds or thousands must be screened. In a series of test tube experiments called assays, compounds are added one at a time to enzymes, cell cultures, or cellular substances grown in a laboratory. The goal is to find which additions show some effect. This process may require testing hundreds of compounds because some may not work but will indicate ways of changing the chemical structure of the compound to improve its performance. Computers can be used to simulate a chemical compound and design chemical structures that might work against the compound. Enzymes attach to the correct site on the

membrane of a cell, which causes the disease. A computer can show scientists what the receptor site looks like and how one might tailor a compound to block an enzyme from attaching there. Although computers give chemists clues as to which compounds to make, a substance still must be tested within a living being. Another approach involves testing compounds made naturally by microscopic organisms. Candidates include fungi, viruses, and molds, such as those that led to penicillin and other antibiotics. Scientists grow the microorganisms in what is known as a ‘‘fermentation broth,’’ with one type of organism per broth. Sometimes, 100,000 or more broths are tested to see whether any compound made by a microorganism has a desirable effect. In animal testing, drug companies make every effort to use as few animals as possible and to ensure their humane and proper care. Generally, two or more species (one rodent, one nonrodent) are tested because a drug may affect one species differently than another. Animal testing is used to measure how much of a drug is absorbed into the blood, how it is broken down chemically in the body, the toxicity of the drug and its breakdown products (metabolites), and how quickly the drug and its metabolites are excreted from the body. Short-term testing in animals ranges in duration from 2 weeks to 3 months, depending on the proposed use of the substance. Long-term testing in animals ranges in duration from a few weeks to several years. Some animal testing continues after human tests begin to learn whether long-term use of a drug may cause cancer or birth defects. Much of this information is submitted to FDA when a sponsor requests to proceed with human clinical trials. The FDA reviews the preclinical research data and then makes a decision as to whether to allow the clinical trials to proceed. The new drug application (NDA) is the vehicle through which drug sponsors formally propose that the FDA approve a new pharmaceutical for sale in the United States. To obtain this authorization, a drug manufacturer submits in an NDA nonclinical (animal) and clinical (human) test data and analyses,

DRUG DEVELOPMENT

drug information, and descriptions of manufacturing procedures. An NDA must provide sufficient information, data, and analyses to permit FDA reviewers to reach several key decisions, including • Whether the drug is safe and effective

for its proposed use(s), and whether the benefits of the drug outweigh its risks; • Whether the proposed labeling of the drug is appropriate, and, if not, what the label of the drug should contain; and • Whether the methods used in manufacturing the drug and the controls used to maintain the quality of the drug are adequate to preserve the identity, strength, quality, and purity of the drug. The purpose of preclinical work—animal pharmacology/toxicology testing— is to develop adequate data to undergird a decision that it is reasonably safe to proceed with human trials of the drug. Clinical trials represent the ultimate premarket testing ground for unapproved drugs. During these trials, an investigational compound is administered to humans and is evaluated for its safety and effectiveness in treating, preventing, or diagnosing a specific disease or condition. The results of this testing will comprise the single most important factor in the approval or disapproval of a new drug. Although the goal of clinical trials is to obtain safety and effectiveness data, the overriding consideration in these studies is the safety of the people in the trials. CDER monitors the study design and conduct of clinical trials to ensure that people in the trials are not exposed to unnecessary risks.

3

DRUG REGISTRATION AND LISTING SYSTEM (DRLS)

Biological Products) whether they enter interstate commerce. All domestic distributors and foreign firms that import drug products into the United States must obtain a labeler code and must list all of their products.

The Food and Drug Administration (FDA) attempted a comprehensive drug inventory for drug listings by establishing two voluntary programs. However, these two voluntary programs were unsuccessful. To make these efforts mandatory, the FDA instituted the Drug Listing Act of 1972; this regulatory policy is in the 21 Code of Federal Regulations (CFR) Part 207. The 21 CFR Part 207 addresses definitions, drug registration requirements, and drug listing requirements by FDA. This Act amended Section 510 of the Federal Food, Drug, and Cosmetic Act and defines the applicable following terms:

2

All firms, unless exempted by the Act, are requested to list their commercially marketed drug products with FDA within 5 days after the beginning of operation. They are required to list/update their drug products listing twice a year (June and December). The initial listing and updates of a product is completed on a form FDA 2657. Manufacturers are allowed to list the products for distributors on form FDA 2658. To assist the firms with the mandatory update in June, the Product Information Management Branch mails a Compliance Verification Report (CVR) to the firms. The CVR goes to all firms which have at least one prescription product listed with FDA. The firm is required to update the CVR and mail it back within 30 days.

• The term Firm refers to a company

engaged in the manufacture, preparation, propagation, compounding, or processing of a drug product. • The term Drug Products refers to human drugs, veterinary drugs, and medicated animal feed premixes that include biological products, but do not include blood and blood components. • The term Manufacturing and Processing refers to repackaging or otherwise changing the container, wrapper, or labeling of any drug product package in the distribution process from the original ‘‘maker’’ to the ultimate consumer. 1

LISTING REQUIREMENTS

3

REGISTRATION EXEMPTIONS

The following is a list of those parties exempt from registration: pharmacies, hospitals, and clinics that dispense drug products at retail; licensed physicians who use drug products solely for purposes related to their professional practice; and/or persons who use drug products solely for their professional needs and are not for sale.

REGISTRATION REQUIREMENTS

A firm must register all drug products (Domestic Manufacturers, Domestic Repackers, Domestic Labelers, and submissions for New Human Drug Application, New Animal Drug Application, Medicated Feed Application, Antibiotic Drug Application, and Establishment License Application to Manufacture

4

REGISTRATION PROCESS

Firms can register by obtaining a Registration of Drug Establishment Form, FDA 2656 within 5 days after the beginning of operation or submission of an application. Firms are required to re-register annually by returning an Annual Registration of Drug Establishment Form, FDA 2656E, within 30 days after receiving it from the Product Information Management Branch.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/handbook/druglist.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

DRUG PACKAGING The sponsor should ensure that the investigational product(s) (including active comparator(s) and placebo, if applicable) is characterized as appropriate to the stage of development of the product(s), is manufactured in accordance with any applicable Good Manufacturing Practice (GMP), and is coded and labeled in a manner that protects the blinding, if applicable. In addition, the labeling should comply with applicable regulatory requirement(s). The sponsor should determine, for the investigational product(s), acceptable storage temperatures, storage conditions (e.g., protection from light), storage times, reconstitution fluids and procedures, and devices for product infusion, if any. The sponsor should inform all involved parties (e.g., monitors, investigators, pharmacists, storage managers) of these determinations. The investigational product(s) should be packaged to prevent contamination and unacceptable deterioration during transport and storage. In blinded trials, the coding system for the investigational product(s) should include a mechanism that permits rapid identification of the product(s) in case of a medical emergency, but it does not permit undetectable breaks of the blinding. If significant formulation changes are made in the investigational or comparator product(s) during the course of clinical development, the results of any additional studies of the formulated product(s) (e.g., stability, dissolution rate, bioavailability) are needed to assess whether these changes would significantly alter the pharmacokinetic profile of the product should the product be available prior to the use of the new formulation in clinical trials.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

DRUG SUPPLY

The sponsor should: • Take steps to ensure that the investi-

gational product(s) are stable over the period of use. • Maintain sufficient quantities of the investigational product(s) used in the trials to reconfirm specifications, should it become necessary, and maintain records of batch sample analyses and characteristics. To the extent stability permits, samples should be retained either until the analyses of the trial data are complete or as required by the applicable regulatory requirement(s), whichever represents the longer retention period.

The sponsor is responsible for supplying the investigator(s)/institution(s) with the investigational product(s). The sponsor should not supply an investigator/institution with the investigational product(s) until the sponsor obtains all required documentation [e.g., approval/favorable opinion from IRB (Institutional Review Board)/IEC (Independent Ethics Committee) and regulatory authority(ies)]. The sponsor should ensure that written procedures include instructions that the investigator/institution should follow for the handling and storage of investigational product(s) for the trial and documentation thereof. The procedures should address adequate and safe receipt, handling, storage, dispensing, retrieval of unused product from subjects, and return of unused investigational product(s) to the sponsor [or alternative disposition if authorized by the sponsor and in compliance with the applicable regulatory requirement(s)]. The sponsor should: • Ensure timely delivery of investigational

product(s) to the investigator(s). • Maintain records that document ship-

ment, receipt, disposition, return, and destruction of the investigational product(s). • Maintain a system for retrieving investigational products and documenting this retrieval (e.g., for deficient product recall, reclaim after trial completion, expired product reclaim). • Maintain a system for the disposition of unused investigational product(s) and for the documentation of this disposition.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

EASTERN COOPERATIVE ONCOLOGY GROUP (ECOG)

review of tumor measurement data to determine response and progression. This article also gives an early version of the well-known five-point ECOG performance status scale for classifying overall patient status. It is notable that these early studies did not include predefined criteria for the amount of increase and decrease in tumor measurements needed to establish response or progression but instead relied on a vote of the investigators based on blinded review of the data. Standardized criteria were later developed, and the ECOG criteria used for many years are given in Reference 2. During the 1970s and 1980s the ECOG Statistical Center was led by Dr. Marvin Zelen, first at the State University of New York at Buffalo and later at the Dana-Farber Cancer Institute. During this period, the Statistical Center included many leading biostatisticians who made numerous important contributions to the statistical methodology for clinical trials, including basic work on methods for analyzing time-to-event data, for sample size and power calculations, on randomization algorithms, and on methods for monitoring studies.

ROBERT J. GRAY Dana-Farber Cancer Institute Boston, Massachusetts

The Eastern Cooperative Oncology Group (ECOG) is one of ten NCI-funded Cooperative Groups. ECOG has been continuously in operation since the Cooperative Group program was established in 1955. During the period 2004–2007, an average of 75 ECOGled protocols were open to accrual each year, of which 65 were therapeutic (17 Phase III), the others were variously prevention, symptom management, health practices, and laboratory studies. At any time, 50–60 studies are also closed to accrual but still in active follow-up before the primary analysis. Accrual to the ECOG-led therapeutic trials varied during this period from 2393 in 2005 to 5092 in 2007, and ECOG members also enrolled an average of 1580 therapeutic cases per year on studies coordinated by other groups. ECOG manages this large portfolio of studies with quite-limited grant funding. The major operational challenge for ECOG and the other cooperative groups has been conducting high-quality clinical trials as efficiently as possible. In this article, the history, structure, policies and procedures, and major accomplishments of ECOG are briefly described. 1

2

ORGANIZATION AND STRUCTURE

ECOG has a hierarchical membership structure with main institutions (usually academic centers) and affiliate institutions (usually community based). Also, separate community networks are funded through the Community Clinical Oncology Program (CCOP) of the NCI’s Division of Cancer Prevention. The principal investigators of the main institutions and CCOPs form the governing body of the Group. They elect the Group chair and approve the Group’s policies and procedures. The scientific program of ECOG is organized into disease-site committees (Breast, Gastrointestinal, Head and Neck, Lymphoma, Thoracic, Melanoma, Genitourinary, Leukemia, Myeloma, and a working group in Brain Tumors) and modality committees (such as surgery, radiation therapy, developmental therapeutics, and laboratory science).

HISTORY

ECOG was founded in 1955 (the original name was the Eastern Cooperative Group in Solid Tumor Chemotherapy) under the leadership of Dr. C. G. Zubrod. At the time, development of standardized methodology for conducting multicenter clinical trials in oncology and for measuring effects of chemotherapy was needed. Methodology developed for early ECOG trials is described in Reference 1, including use of sealed envelopes for treatment assignment; collection of detailed data on ‘‘flow sheets,’’ tumor measurement forms, and summary forms; and blinded central

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

EASTERN COOPERATIVE ONCOLOGY GROUP (ECOG)

Also, a separately funded program in prevention and cancer control research has been developed. The Group chair appoints the committee chairs, who then organize the scientific research program within each area and bring proposals for studies forward to the Group. An executive review committee reviews all study concepts and decides whether they are suitable for development in ECOG. All concepts and protocols must also be reviewed and approved by NCI through either the Cancer Therapy Evaluation Program or the Division of Cancer Prevention. A Group-wide meeting of the membership is held twice each year, at which each of the committees meets to discuss concepts for new studies and to review the progress of ongoing studies. ECOG has a Group Chair’s Office (in Philadelphia, PA), a combined Operations Office and Data Management Center (at Frontier Science and Technology Research Foundation in Brookline, MA) and a Statistical Office (at the Dana-Farber Cancer Institute). The head of the Statistical Center (the Group statistician) is appointed by the Group chair with the approval of the principal investigators. Approximately nine full-time-equivalent statisticians provide all statistical support for ECOG activities. These statisticians are assigned to work with the disease and modality committees. They collaborate actively with the investigators during the concept and protocol development process, and are responsible for developing the statistical design of the study and for reviewing all drafts of protocols. The statisticians also prepare the registration/randomization materials and collaborate with the data management staff on the development of the data collection instruments. After activation, statisticians monitor the progress of studies and adverse events and perform interim efficacy analyses when appropriate under the design. Reports are generated on all ongoing studies twice each year. Final reports are written by the statistician for each completed ECOG trial. Statisticians also work extensively on lab correlative studies conducted on the clinical trials and on investigator-initiated grants to support various aspects of ECOG activities (especially to provide support for lab studies).

3 PROCEDURES The Cancer Cooperative Groups have developed highly efficient systems for conducting clinical trials, but these systems also involve some compromises relative to standards in the pharmaceutical industry. A key difference is the ongoing relationship between ECOG and its member institutions; ECOG runs many studies simultaneously in the same network of centers, and ECOG studies are generally open to all member institutions. 3.1 Data Quality Control The focus in institutional monitoring in ECOG is on ensuring that the institution is following appropriate procedures and meeting acceptable data quality standards rather than on direct verification of all individual data items. Instead of relying on extensive (and expensive) on-site monitoring for each study to ensure complete accuracy of data, ECOG uses an audit system to verify that participating centers are performing well overall. All centers are audited at least once every 3 years, and an audit generally involves full review of at least 12% of the cases enrolled at a site as well as review of regulatory compliance and pharmacy procedures. Centers with deficiencies are required to submit written corrective action plans and may be reaudited more frequently or, in extreme cases, may be suspended. With only a limited proportion of cases audited at the institutions, ECOG also relies heavily on central review of data for internal consistency. Submitted data are checked for completeness and for invalid values, and extensive checks are made among submitted data items to ensure consistency of all data. Copies of limited source documentation, such as surgical pathology reports, are also collected and reviewed. For assessing response and progression, actual tumor measurements and lab results are required to provide documentation that protocol criteria have been met. A final review of the data coding is performed by the study chair. For the most part, ECOG study chairs are volunteers who are not paid for their time spent on ECOG studies. Available funding does not permit using paid independent clinical reviewers. Generally a blinded central radiology review is not

EASTERN COOPERATIVE ONCOLOGY GROUP (ECOG)

performed because of the time and expense required for collection and review of scans and other imaging studies. The extensive central review of reported measurements, together with the limited on-site auditing, has demonstrated acceptable results without central radiology review. 3.2 Adverse-Event Monitoring Timely monitoring of adverse events is a crucial part of any treatment trial. In ECOG, real-time monitoring occurs primarily through the expedited reporting of severe adverse events. Expedited reporting of certain classes of severe events is required by the NCI and FDA. Expedited reports are now submitted through an online system developed and maintained by NCI. The ECOG data management staff (and study chairs and statisticians) then can access the reports through the same system. A cumulative listing of expedited adverse-event reports is sent to the study chair, the study statistician, and the disease site toxicity monitor monthly if significant new events have been reported. They are each required to review the list and to determine whether any problems might require more review or other action. Adverse events are also routinely submitted on case report forms. Consistency between expedited reports and routine submission is checked as part of the data review process. Statisticians also prepare summary reports on the routine adverse-event data twice each year. These reports are made available to the study chairs, to the group membership, and, for Phase III studies, to the Data Monitoring Committee (DMC). For some studies (especially double-blind studies), more extensive information is also provided to the DMC. 3.3 Blinding of Treatment Until recent years, double-blind studies in oncology were rare. This rarity was partly because of the high level and sometimes characteristic nature of the toxicities and other side effects of many chemotherapy drugs. Also, many Phase III studies have had survival as their primary endpoint, and this endpoint is less subject to potential bias than other endpoints. With the advent of less-toxic

3

targeted therapies and increasing emphasis on endpoints such as progression-free survival, double-blind placebo-controlled studies have become more common. All registrations in ECOG now occur through a web-based registration system. On blinded studies, treatment assignments are prematched to drug identification numbers. At the time of randomization, the treatment assignment is determined using the randomization algorithm for the study, and then the assignment is matched to the drug identification. The treating center and the drug distribution center are only notified of the drug identification number. The links between the treatment codes and the drug identification numbers are kept in a database table that can only be accessed by senior database administrators and a few senior staff who need access for emergency unblinding and other reasons. All data review is performed blinded to treatment assignment. Statisticians obtain the treatment codes from the senior DBA when interim analyses are performed. Only pooled information on adverse events is made available to the group membership. Analyses by treatment arm are only presented to the DMC, until the DMC decides to release the information. Requests from institutions for unblinding treatment assignments are reviewed centrally by a committee of senior staff. Unblinding requires approval of a senior statistician and a senior clinician (usually the group statistician and the group executive officer, except in emergencies, if they are not available). Some studies permit routine unblinding of the treating physician at progression, if the information is generally needed for determining the next treatment (especially if the study involves crossover at progression). 3.4 Standardizing Terminology, Data Elements, and Criteria Standardization of data elements and terminology is important for efficient operations. As early as the mid-1980s, ECOG reached agreement with the Southwest Oncology Group (another NCI-funded Cooperative Group) on partial standardization of the data collected on breast cancer studies. Beginning in 1998, efforts at data standardization among

4

EASTERN COOPERATIVE ONCOLOGY GROUP (ECOG)

the Cooperative Groups were integrated into the NCI’s Common Data Elements (CDE) project. Efficient conduct of large studies requires that no unnecessary data items be collected; however, given the variety of diseases, treatments, modalities, endpoints, and ancillary scientific objectives, considerable variation in the data items is needed to address the objectives of a study. Consequently, the focus of the CDE project has been on improving efficiency through standardization of definitions and terminology. The CDE project has now become part of the NCI’s caBIG initiative, in the Vocabularies and Common Data Elements workspace (see https://cabig.nci.nih.gov). Standardization of evaluation criteria is another important issue for facilitating participation in studies led by different groups, for interpreting results across studies, and for combining data from multiple studies (e.g., in meta-analyses). Cooperative Groups have had a long history of standardizing criteria within their own studies (see, e.g., Reference 2), but they have not always considered standardization across groups. Recently, the NCI has taken a major role, coleading the development of the RECIST (3) solid tumor response criteria and developing the Common Terminology Criteria for Adverse Events (http://ctep.cancer.gov/reporting/ctc.html). Standardization of endpoints across studies and use of common terminology and definitions for different endpoints is another important issue for reporting of results. An important step in this direction for adjuvant breast cancer studies was recently taken by the Cooperative Group breast cancer committees (4). Although recognizing that different endpoints might be needed in different types of studies, they have given a common set of definitions and terminology for various possible endpoints. Whereas some issues are specific to breast cancer, such as the role of new ductal carcinoma in situ cancers in defining the primary endpoint for an adjuvant breast cancer study, some terminology could be applied much more broadly in studies of adjuvant therapy in other diseases. 3.5 Data Monitoring Committee ECOG typically has 20–30 Phase III studies being monitored at any time. Resource

limitations prevent setting up separate independent data monitoring committees for each study. Instead, ECOG has a single data monitoring committee (DMC) that monitors the entire portfolio of Phase III studies. This committee has nine voting members who are selected to have expertise in diverse areas of oncology and hematology. A majority of the voting members must have no affiliation with ECOG, and they must include an outside statistician and a consumer representative. The ECOG DMC meets regularly twice each year. Interim analyses are scheduled to occur in conjunction with the DMC meetings, using database cutoffs 8 weeks before the meeting date. Based on experience with Cooperative Group DMCs, it has been recommended that interim analyses of efficacy on Cooperative Group studies should be scheduled for every DMC meeting until the study reaches full information (5). Recent ECOG Phase III studies have often followed this recommendation, but it also requires considerable extra effort to have study databases cleaned for interim analyses twice each year. Another aspect of ECOG DMC operations is that the study statistician, rather than an independent outside statistician, performs the interim analysis. This aspect is contrary to the recommendations of the U.S. FDA (‘‘Establishment and Operation of Clinical Trial Data Monitoring Committees,’’ FDA Guidance Document, 2006, http://www.fda. gov/cber/gdlns/clintrialdmc.pdf). With the resource limitations of the Cooperative Groups, it is generally not feasible to have a second statistician funded to be sufficiently familiar with the study to perform meaningful analyses. The multiple levels of review required for protocol design changes within the disease committees, the Group’s executive committee, and the NCI, make manipulation of the design of the study on the basis of knowledge of interim results highly unlikely. However, the risk of errors in analysis and/or interpretation that could occur if the study statistician is not involved in the monitoring, because of lack of detailed understanding the protocol or of the study data collection and processing procedures, seems like the greater risk. This approach does require that the study statistician be very careful not to give any kind of hint of interim study results

EASTERN COOPERATIVE ONCOLOGY GROUP (ECOG)

to the study investigators, but in ECOG’s experience, this has not been a problem. 4

MAJOR ACCOMPLISHMENTS

ECOG has made many contributions advancing the care of cancer patients. A few key recent results are establishing the unfavorable risk–benefit profile of autologous bonemarrow transplant therapy in breast cancer (6,7); establishing the benefit of the antiCD20 monoclonal antibody rituximab in initial treatment of diffuse, aggressive nonHodgkin’s lymphoma (8); establishing the benefit of adding the anti-VEGF monoclonal antibody bevacizumab to chemotherapy in advanced lung cancer (9) and breast cancer (10) and in combination with oxaliplatinbased chemotherapy in advanced colorectal cancer (11); and establishing a new standard for initial treatment in multiple myeloma (12). Major studies currently accruing include Phase III studies of sorafenib in metastatic melanoma (E2603), of sorafenib or sunitinib for adjuvant renal cancer (E2805), of bevacizumab in combination with chemotherapy for adjuvant treatment of non-small cell lung cancer (E1505), of bevacizumab in combination with chemotherapy for adjuvant treatment of breast cancer (E5103), and a study of using the Oncotype DX genomic assay (13) to select patients for adjuvant chemotherapy in breast cancer (TAILORx). The NHL study E4494 of rituximab therapy (8) is an interesting case study of the complexity that can occur in monitoring and analyzing studies in oncology. While this study was underway, the GELA group in Europe released results of a similar study (14) with significantly better PFS and overall survival with rituximab. At that time, the ECOG DMC reviewed interim results from E4494 and the results from the GELA trial and decided that E4494 should continue, but the ECOG DMC also decided that they needed to review updated interim results every 6 months to reevaluate whether this recommendation continued to be appropriate, although only two interim analyses had been specified in the design. E4494 also involved an induction randomization between standard CHOP chemotherapy and

5

CHOP+rituximab and a maintenance therapy randomization of induction responders to rituximab versus observation, whereas the GELA trial only involved the induction randomization. The potential confounding of the effect of the maintenance treatment on the induction comparison was a significant problem for interpreting E4494. This problem led to the use of novel weighted analysis methods (see, e.g., Reference 15) for estimating the effect of induction rituximab in the absence of maintenance rituximab, although such analyses had not been specified prospectively. These analyses ultimately became part of the submission to the FDA for approval of rituximab for this indication. This study illustrates that when Phase III studies take an extended period to complete, plans developed at initiation can require significant modification. In such circumstances, it is important to have the flexibility to modify the original plans and to do so in accordance with sound statistical principles. The Cooperative Group Program is the premiere public system in the United States for conducting randomized comparative studies of cancer therapy. In this article, we have provided an overview of ECOG’s contributions to this program and its methods for achieving these results. REFERENCES 1. C. G. Zubrod, M. Schneiderman, E. Frei, C. Brindley, G. L. Gold, B. Shnider, R. Oviedo, J. Gorman J, R. Jones, Jr., U. Jonsson, J. Colsky, T. Chalmers, B. Ferguson, M. Dederick, J. Holland, O. Selawry, W. Regelson, L. Lasagna, A. H. Owens, Jr. Appraisal of methods for the study of chemotherapy of cancer in man: comparative therapeutic trial of nitrogen mustard and triethylene thiophosphoramide, J. Chron. Dis. 1960; 11:7–33 2. M. M. Oken, R. H. Creech, D. C. Tormey, J. Horton, T. E. Davis, E. T. McFadden, P. P. Carbone, Toxicity and response criteria of the Eastern Cooperative Oncology Group, Am. J. Clin. Oncol. 1982; 5:649–655. 3. P. Therasse, S. G. Arbuck, E. A. Eisenhauer, J. Wanders, R. S. Kaplan, L. Rubinstein, J. Verweij, M. V. Glabbeke, A. T. van Oosterom, M. C. Christian, and S. G. Gwyther. New guidelines to evaluate the response to treatment in solid tumors, J. Natl. Cancer Inst. 2000; 92:205–216.

6

EASTERN COOPERATIVE ONCOLOGY GROUP (ECOG)

4. C. A. Hudis, W. E. Barlow, J. P. Costantino, R. J. Gray, K. L. Pritchard, J. A. W. Chapman, J. A. Sparano, S. Hunsberger, R. A. Enos, R. D. Gelber, and J. Zujewski, Proposal for standardized definitions for efficacy end points in adjuvant breast cancer trials: the STEEP system, J. Clin. Oncol. 2007; 25:2127–2132. 5. B. Freidlin, E. L. Korn, and S. L. George, Data monitoring committees and interim monitoring guidelines, Controlled Clinical Trials. 1999; 20:395–40. 6. E. A. Stadtmauer, A. O’Neill, L. J. Goldstein, P. A. Crilley, K. F. Mangan, J. N. Ingle, I. Brodsky, S. Martino, H. M. Lazarus, J. K. Erban, C. Sickles, and J. H. Glick, Conventional-dose chemotherapy compared with high-dose chemotherapy plus autologous hematopoietic stem cell transplantation for metastatic breast cancer, N. Engl. J. Med. 2000; 342(15):1069–1076. 7. M. S. Tallman, R. J. Gray, N. J. Robert, C. F. LeMaistre, C. Osborne, W. P. Vaughan, W. J. Gradishar, T. M. Pisansky, J. H. Fetting, E. M. Paietta, H. M. Lazarus, Conventional adjuvant chemotherapy with or without high-dose chemotherapy and autologous stem cell transplantation in high-risk breast cancer, N. Engl. J. Med. 2003; 349(1):17–26. 8. T. M. Habermann, E. Weller, V. A. Morrison, R. Gascoyne, P. A. Cassileth, J. B. Cohn, S. R. Dakhil, B. Woda, R. I. Fisher, B. A. Peterson, and S. J. Horning, RituximabCHOP versus CHOP alone or with maintenance rituximab in older patients with diffuse large B-cell lymphoma, J. Clin. Oncol. 2006; 24(19)3121–3127. 9. A. B. Sandler, R. J. Gray, M. C. Perry, J. R. Brahmer, J H. Schiller, A. Dowlati, R. Lilenbaum, and D. H. Johnson, Paclitaxel plus Carboplatin with or without Bevacizumab in advanced non-squamous nonsmall cell lung cancer: a randomized study of the Eastern Coopertive Oncology Group, N. Engl. J. Med. 2006; 355(24):542–2550. 10. K. D. Miller, M. Wang, J. Gralow, M. Dickler, M. A. Cobleigh, E. A. Perez, T. Shenkier, D. F. Cella, and N. E. Davidson, Paclitaxel plus bevacizumab versus paclitaxel alone for metastatic breast cancer, N. Engl. J. Med. 2007; 357:2666–2676. 11. B. J. Giantonio, P. J. Catalano, N. J. Meropol, P. J. O’Dwyer, E. P. Mitchell, S. R. Alberts, M. A. Schwartz, and A. B. Benson III, Bevacizumab in combination with Oxaliplatin, Fluorouracil, and Leucovorin (FOLFOX4) for previously treated metastatic colorectal cancer: results from the Eastern Cooperative

Oncology Group Study E3200, J. Clin. Oncol. 2007; 25:1539–1544. 12. S. V. Rajkumar, S. J. Jacobus, N. S. Callander, R. Fonseca, D. H. Vesole, M. E. Williams, R. Abonour, D. S. Siegel, and P. R. Greipp, Phase III trial of lenalidomide plus highdose dexamethasone versus lenalidomide plus low-dose dexamethasone in newly diagnosed multiple myeloma (E4A03): a trial coordinated by the Eastern Cooperative Oncology Group [abstract]. Proc. ASCO 2007. Abstract LBA8025. 13. S. Paik, S. Shak, G. Tang, C. Kim, J. Baker, M. Cronin, F. L. Baehner, M. G. Walker, D. Watson, F. Park, W. Hiller, E. R. Fisher, D. L. Wickerham, J. Bryant, and N. Wolmark, A multigene assay to predict recurrence of tamoxifen-treated, node-negative breast cancer, N. Engl. J. Med. 2004; 351:2817–2826. 14. B. Coiffier, E. Lepage, J. Briere, R. Herbrecht, H. Tilly, R. Bouabdallah,. P. Morel, E. van den Neste, G. Salles, P. Gaulard, F. Reyes, and C. Gisselbrecht,. CHOP chemotherapy plus rituximab compared with CHOP alone in elderly patients with diffuse large-B-cell lymphoma, N. Engl. J. Med. 2002; 346:235–242. 15. J. K. Lunceford, M. Davidian, and A. A. Tsiatis, Estimation of survival distributions of treatment policies in two-stage randomization designs in clinical trials, Biometrics. 2002; 58:48–57.

CROSS-REFERENCES Adverse Event Clinical Data Management Data Monitoring Committee National Cancer Institute Southwest Oncology Group

ELIGIBILITY AND EXCLUSION CRITERIA

the eligibility criteria should be chosen to allow valid inferences to that population. For example, the Hypertension Prevention Trial (HPT) was aimed at normotensive individuals 25–49 years old with diastolic blood pressure between 78 mm Hg and 90 mm Hg, and these were the main eligibility criteria (3). Choosing the narrow eligibility criteria often appropriate for an explanatory trial can make it difficult to apply the results to a broader population (11). Yusuf (23), moreover, argues that a truly homogeneous cohort cannot be constituted because even apparently similar individuals can have very different outcomes. The consensus is that most Phase III randomized trials should be regarded as pragmatic.

MARY A. FOULKES Food & Drug Administration, Rockville, MD, USA

The choice of eligibility criteria in a clinical trial can increase or decrease the magnitude of between-patient variation, which will in turn decrease or increase the statistical power of the trial for a given sample size. Theoretically, the more homogeneous the trial population, the greater is the power of the trial, but the more limited is the ability to generalize the results to a broad population. Thus, the choice of eligibility criteria can profoundly influence both the results and the interpretation of the trial. Besides controlling variation, the Institute of Medicine (IOM) Committee on the Ethical and Legal Issues Relating to the Inclusion of Women in Clinical Studies (16) discusses four other issues related to the choice of trial population; namely, disease stage, clinical contraindications, regulatory or ethical restrictions, and compliance considerations. We will discuss these and the related issues of explanatory vs. pragmatic trials, screening and recruitment processes, and the impact of eligibility criteria on the generalizability of trial results. Other factors influencing the selection of patients, such as factors in the selection of institutions in multicenter studies and physician preferences are discussed elsewhere (2,22). 1

1.1 The Uncertainty Principle Byar et al. (4) describe the simplest possible form of eligibility criteria for a trial, in which patients are eligible provided the treating physician and the patient have ‘‘substantial uncertainty’’ as to which of the treatment options is better. This definition, known as the uncertainty principle, incorporates all factors that contraindicate one or more of the treatment options including stage of disease, co-existing disease, and patients’ preferences. However, it also largely devolves definition of eligibility to the individual physicians participating in the trial. The consequent lack of control and strict definition of the cohort of patients entering the trial has been unattractive to some investigators. 2 CONTROL OF VARIATION VS. EASE OF RECRUITMENT

EXPLANATORY VS. PRAGMATIC TRIALS

The objectives of a trial affect the appropriate eligibility criteria (20). If the trial is designed to estimate the biological effect of a treatment (explanatory trial), then the eligibility criteria should be chosen to minimize the impact of extraneous variation, as in early investigations of protease inhibitors against human immunodeficiency virus (HIV) infection (18). If, however, the trial is designed to estimate the effectiveness of a treatment in a target population (pragmatic trial), then

The debate over the uncertainty principle highlights the tension between two different ways of improving the precision of the estimated effect of treatment in a randomized trial. By using very strict eligibility criteria we seek to reduce between-patient variation in clinical outcomes, leading to improved precision of the treatment difference estimate. By using very flexible eligibility criteria (as with the uncertainty principle), we seek to allow a wider entry to the trial, thereby

1

2

ELIGIBILITY AND EXCLUSION CRITERIA

increasing the number of eligible patients and usually the precision of the treatment difference estimate. The question is, therefore, do we try to control variation and accept the smaller size of the sample, or do we try to increase the sample size and accept a wider between-patient variation? While this debate continues, the general consensus among clinical trial statisticians is that it is generally difficult to control between-patient variation successfully because often we do not know the important determinants of prognosis. Therefore, attempts to use very strict eligibility criteria are less successful than attempts to gain precision by entering very large numbers of patients into trials (23). However, if there are categories of patients who are considered very unlikely to benefit from the treatment, it is clearly conceivable to exclude them from the trial (see later sections on Stage of Disease and Clinical Contraindications). Begg (2) criticizes the common practice of introducing a long list of eligibility criteria in clinical trials, particularly in the treatment of cancer. Such an approach greatly increases the difficulty of recruiting patients in large numbers. In examining such lists it is often found that many of the criteria are of questionable importance and do not relate directly to the safety of the patient or to the lack of benefit to be derived from the treatment. 2.1 Issues in the Screening and Recruitment Process Establishing eligibility often involves a screening process. Examples include choosing individuals for a heart disease trial with ejection fraction between 0.35 and 0.8 and a specific number of ectopic beats, or choosing HIV-infected individuals with slowly rather than rapidly progressing disease (15). Some eligibility criteria may be implicit in this process. For example, the recruitment method may require the patients to be accessible by telephone contact or to be able to read and write in English, such as trials in which the initial contact is via a prepaid postal response card. Multiple ‘‘baseline’’ visits that are sometimes used in the screening process can provide multiple opportunities for exclusion, e.g. the Coronary Primary Prevention

Trial used five baseline visits and the HPT used three baseline visits. Thus, those ultimately enrolled may affect the recruitment and screening mechanisms and resources for multiple participant contacts as much as the protocol-specific eligibility criteria. The impact of eligibility criteria and of recruitment procedures on the overall cost of the trial has rarely been investigated. Borhani (3) indicated that the ordering of the application of eligibility criteria can substantially affect costs. These costs are also sensitive to the cutoffs applied to continuous responses, e.g. diastolic blood pressure, high density lipoprotein cholesterol, coronary ejection fraction, and T-cell lymphocyte counts. As mentioned earlier, eligibility criteria can have a strong impact on the ease of recruitment. For example, the need to enroll newly diagnosed or previously untreated patients can severely restrict the ability to recruit. The need to enroll rapidly after a stroke, myocardial infarction, head trauma, or exposure to infectious agent can lead to difficulties. If the condition renders the patient unconscious for some period of time, or the patient lives far from the treatment center, or is unaware that an infection, stroke, infarction, or other event has occurred, it is less likely that they will be available for enrollment. Similarly, Carew (5) suggests that recruitment be enhanced by broad eligibility criteria, allowing potentially more sites and more individuals to participate. 2.2 Stage of Disease Often the stage of disease strongly affects the outcome of treatment, and is a primary source of variation. Eligibility is often restricted to the stages of disease most appropriately managed by the treatment. For many diseases, classification or staging systems have been developed to aid clinical management. Eligibility criteria involving stage of disease are best defined using an established classification system that is in wide use. Examples of such classification systems include the coronary functional class (7), Dukes’ colon cancer staging system, and the World Health Organization staging system for HIV infection (26).

ELIGIBILITY AND EXCLUSION CRITERIA

3

2.3 Clinical Contraindications

2.6 Implementing the Eligibility Criteria

Exclusions arising because one of the treatments is clearly contraindicated are common (14). For example, 18% of those screened for the Beta Blocker Heart Attack Trial were excluded due to contraindications to the administration of propranolol (12). Since these prior conditions would preclude use of some of the treatments in a trial, the trial results could not apply to individuals with those conditions. Some argue that contraindications should be clearly delineated in the protocol to avoid investigator or regional differences in their use.

The characterization of the target population and baseline homogeneity can be subverted by deviations during the conduct of the trial from the protocol specified eligibility criteria. If extensive, these can adversely affect the assumptions underlying analyses and the interpretation of the results. Thus, monitoring the determination of eligibility criteria during the conduct of the trial is an important component of the implementation of the trial. Often, the office that conducts the randomized treatment assignment checks the eligibility criteria before enrolling the patient. Finkelstein & Green (9) discuss the exclusion from analysis of individuals found to be ineligible after enrollment in the trial.

2.4 Compliance Considerations A run-in (or qualification) period is sometimes built into the trial design so as to identify potential noncompliers and exclude them from enrollment. This reduces the dilution of treatment differences that noncompliance introduces. In some studies, this period can also be used to eliminate placebo responders. In these cases, the determination of noncompliance becomes one of the outcome measures of the trial. 2.5 Regulatory or Ethical Considerations Various demographically or otherwise defined populations have been excluded from clinical trials in the past. For example, in trials of heart disease prevention, women have been excluded as their incidence of heart disease is lower than in men and their inclusion would have required a larger sample size. Similarly, minority groups have sometimes had little or no representation because no special efforts had been made to include them. Recent changes in US regulations have required special justification for the exclusion of women, minorities, or the elderly from National Institutes of Health sponsored trials. The scientific argument for including these groups is that it provides a more solid basis for extrapolating the results of the trial to the general population (6,8,10,13,17,19,24,25). There will usually be inadequate statistical power for detecting different effects in subpopulations, but sometimes meta-analysis of several studies may be able to detect such differences.

2.7 Generalization of Results to Broader Populations Treatment trials (or prevention trials) are usually conducted on samples of convenience, enrolling participants who present at specific hospitals or clinical sites. Therefore, the population to whom the trial results apply is generally not well defined. External validity—the ability to generalize from the trial to some broader population—is the ultimate goal of any trial. Adequately randomized trials can be assumed to produce valid results for the specific group of individuals enrolled, i.e. internal validity; the difficulties arise in extending the inference beyond that limited cohort. Since complete enumeration of the target population is rarely possible, inferences from studies are based on substantive judgment. A strong argument that is often used is that treatment differences in outcome are generally less variable among different patient populations than the outcomes themselves (23). Following publication, critics questioned the generalizability of the results of the International Cooperative Trial of Extracranial– Intracranial (EC/IC) Arterial Anastomosis to evaluate the effect of the EC/IC procedure on the risk of ischemic stroke. The results showed a lack of benefit that surprised many in the surgical profession. It became clear that many of the eligible patients at the participating clinical sites did

4

ELIGIBILITY AND EXCLUSION CRITERIA

not enter the trial, while those enrolled in the trial were considered to have poorer risk and some argued that they were less likely to benefit from surgery (1,21). The ensuing controversy slowed acceptance of the trial results by the surgical community, although eventually they had a profound effect on the frequency with which EC/IC was performed.

3

CONCLUSIONS

The goals and objectives of the trial, the intended target population, and the anticipated inferences from the trial results should all be carefully specified from the outset. If that is done, then the appropriate choice of eligibility criteria usually becomes clearer. Experience has shown that simplifying eligibility criteria generally enhances recruitment, allows a wider participation, and gives greater justification for generalizing the results to a broader population.

REFERENCES 1. Barnett, H. J. M., Sackett, D., Taylor, D. W., Haynes, B., Peerless, S. J., Meissner, I., Hachinski, V. & Fox, A. (1987). Are the results of the extracranial–intracranial bypass trial generalizable?, New England Journal of Medicine 316, 820–824. 2. Begg, C. B. (1988). Selection of patients for clinical trials, Seminars in Oncology 15, 434–440. 3. Borhani, N. O., Tonascia, J., Schlundt, D. G., Prineas, R. J. & Jefferys, J. L. (1989). Recruitment in the Hypertension Prevention Trial, Controlled Clinical Trials 10, 30S–39S. 4. Byar, D. P., Schoenfeld, D. A. & Green, S. B. (1990). Design considerations for AIDS trials, New England Journal of Medicine 323, 1343–1348. 5. Carew, B. D., Ahn, S. A., Boichot, H. D., Diesenfeldt, B. J., Dolan, N. A., Edens, T. R., Weiner, D. H. & Probstfield, J. L. (1992). Recruitment strategies in the Studies of Left Ventricular Dysfunction (SOLVD), Controlled Clinical Trials 13, 325–338. 6. Cotton P. (1990). Is there still too much extrapolation from data on middle-aged white men?, Journal of the American Medical Association 263, 1049–1050.

7. Criteria Committee of NYHA (1964). Diseases of the Heart and Blood Vessels: Nomenclature and Criteria for Diagnosis, 6th Ed. Little, Brown & Company, Boston. 8. El-Sadr, W. & Capps, L. (1992). Special communication: the challenge of minority recruitment in clinical trials for AIDS, Journal of the American Medical Association 267, 954–957. 9. Finkelstein, D. M. & Green, S. B. (1995). Issues in analysis of AIDS clinical trials, in AIDS Clinical Trials, D. M. Finkelstein & D. A. Schonfeld, eds. Wiley–Liss, New York, pp. 243–256. 10. Freedman L. S., Simon, R., Foulkes, M. A., Friedman, L., Geller, N. L., Gordon, D. J. & Mowery, R. (1995). Inclusion of women and minorities in clinical trials and the NIH Revitalization Act of 1993—the perspective of NIH clinical trialists, Controlled Clinical Trials 16, 277–285. 11. Gail, M. H. (1985). Eligibility exclusions, losses to follow-up, removal of randomized patients, and uncounted events in cancer clinical trials, Cancer Treatment Reports 69, 1107–1112. 12. Goldstein, S., Byington, R. & the BHAT Research Group (1987). The Beta Blocker Heart Attack Trial: recruitment experience, Controlled Clinical Trials 8, 79 S–85 S. 13. Gurwitz, J. H., Col, N. F. & Avorn, J. (1992). The exclusion of the elderly and women from clinical trials in acute myocardial infarction, Journal of the American Medical Association 268, 1417–1422. 14. Harrison K., Veahov, D., Jones, K., Charron, K. & Clements, M. L. (1995). Medical eligibility, comprehension of the consent process, and retention of injection drug users recruited for an HIV vaccine trial, Journal of Acquired Immune Deficiency Syndrome 10, 386–390. 15. Haynes, B. F., Panteleo, G. & Fauci, A. S. (1996). Toward an understanding of the correlates of protective immunity to HIV infection, Science 271, 324–328. 16. IOM Committee on the Ethical and Legal Issues Relating to the Inclusion of Women in Clinical Studies, (1996). Women and Health Research: Ethical and Legal Issues of Including Women in Clinical Studies, A. C. Mastroianni, R. Faden & D. Federman, eds. National Academy Press, Washington. 17. Lagakos, S., Fischl, M. A., Stein, D. S., Lim, L. & Vollerding, P. (1991). Effects of zidovudine therapy in minority and other subpopulations with early HIV infection, Journal of the American Medical Association 266, 2709–2712.

ELIGIBILITY AND EXCLUSION CRITERIA 18. Markowitz, M., Mo, H., Kempf, D. J., Norbeck, D. W., Bhat, T. N., Erickson, J. W. & Ho, D. D. (1996). Triple therapy with AZT, 3TC, and ritonavir in 12 subjects newly infected with HIV-1, Eleventh International Conference on AIDS, Abstract Th.B. 933. 19. Patterson, W. B. & Emanuel, E. J. (1995). The eligibility of women for clinical research trials, Journal of Clinical Oncology 13, 293–299. 20. Schwartz, D., Flamant, R. & Lellouch, J. (1980). Clinical Trials. Academic Press, New York. 21. Sundt, T. M. (1987). Was the international randomized trial of extracranial–intracranial arterial bypass representative of the population at risk?, New England Journal of Medicine 316, 814–816. 22. Taylor, K. M., Margolese, R. G. & Soskolne, C. L. (1984). Physicians’ reasons for not entering eligible patients in a randomized clinical trial of surgery for breast cancer, New England Journal of Medicine 310, 1363–1367. 23. Yusuf, S., Held, P., Teo, K. K. & Toretsky, E. R. (1990). Selection of patients for randomized controlled trials: implications of wide or narrow eligibility criteria, Statistics in Medicine 9, 73–86. 24. Yusuf, S. & Furberg, C. D. (1991). Are we biased in our approach to treating elderly patients with heart disease?, American Journal of Cardiology 68, 954–956. 25. Wenger, N. K. (1992). Exclusion of the elderly and women from coronary trials: is their quality of care compromised?, Journal of the American Medical Association 268, 1460–1461. 26. World Health Organization (1990). Acquired immune deficiency syndrome (AIDS): interim proposal for a WHO staging system for HIV infection and disease, Weekly Epidemiology Record 65, 221–228.

CROSS-REFERENCES Intention to Treat Analysis

5

EMERGENCY USE INVESTIGATIONAL NEW DRUG (IND) The need for an investigational drug may develop in an emergency situation that does not allow time for submission of an Investigational New Drug (IND) Application in accordance with 21 CFR (Code of Federal Regulations) 312.23 or 21 CFR 312.34. In such a case, the Food and Drug Administration (FDA) may authorize shipment of the drug for a specified use in advance of submission of an IND. A request for such authorization may be transmitted to FDA by telephone or by other rapid communication means. For investigational biological drugs regulated by the Center for Biologics Evaluation and Research (CBER), the request should be directed to the Office of Communication, Training and Manufacturers Assistance (HFM–40), Center for Biologics Evaluation and Research. For all other investigational drugs, the request for authorization should be directed to the Division of Drug Information (HFD–240), Center for Drug Evaluation and Research. After normal working hours, Eastern Standard Time, the request should be directed to the FDA Office of Emergency Operations (HFA–615). Except in extraordinary circumstances, such authorization will be conditioned on the sponsor making an appropriate IND submission as soon as possible after receiving the authorization.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/regulatory/applications/ ind page 1.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

END-OF-PHASE II MEETING

more fully in the Food and Drug Administration (FDA) Staff Manual Guide 4850.7 that is publicly available under the FDA public information regulations in part 20. Arrangements for an End-of-Phase II meeting are to be made with the division in the FDA Center for Drug Evaluation and Research or the Center for Biologics Evaluation and Research that is responsible for review of the IND. The meeting will be scheduled by FDA at a time convenient to both FDA and the sponsor. Both the sponsor and FDA may bring consultants to the meeting. The meeting should be directed primarily at establishing agreement between FDA and the sponsor of the overall plan for Phase III and the objectives and design of particular studies. The adequacy of the technical information to support Phase III studies and/or a marketing application may also be discussed. FDA will also provide its best judgment, at that time, of the pediatric studies that will be required for the drug product and whether their submission will be deferred until after approval. Agreements reached at the meeting on these matters will be recorded in minutes of the conference that will be taken by FDA in accordance with Sec. 10.65 and provided to the sponsor. The minutes along with any other written material provided to the sponsor will serve as a permanent record of any agreements reached. Barring a significant scientific development that requires otherwise, studies conducted in accordance with the agreement shall be presumed to be sufficient in objective and design for the purpose of obtaining marketing approval for the drug.

The purpose of an End-of-Phase II Meeting is to determine the safety of proceeding to Phase III, to evaluate the Phase III plan and protocols and the adequacy of current studies and plans to assess pediatric safety and effectiveness, and to identify any additional information necessary to support a marketing application for the uses under investigation. Although the End-of-Phase II meeting is designed primarily for Investigational New Drugs (IND) that involve new molecular entities or major new uses of marketed drugs, a sponsor of any IND may request and obtain an End-of-Phase II meeting. To be most useful to the sponsor, Endof-Phase II meetings should be held before major commitments of effort and resources to specific Phase III tests are made. The scheduling of an End-of-Phase II meeting is not intended, however, to delay the transition of an investigation from Phase II to Phase III. 1

ADVANCE INFORMATION

At least 1 month in advance of an End-ofPhase II meeting, the sponsor should submit background information on the sponsor’s plan for Phase III, including summaries of the Phase I and II investigations, the specific protocols for Phase III clinical studies, plans for any additional nonclinical studies, plans for pediatric studies (including a time line for protocol finalization, enrollment, completion, and data analysis or information to support any planned request for waiver or deferral of pediatric studies), and, if available, tentative labeling for the drug. The recommended contents of such a submission are described This article was modified from the website of the United States Food and Drug Administration (http://frwebgate.access.gpo.gov/cgi-bin/getcfr.cgi?TITLE=21&PART=312&SECTION=47& YEAR=1999&TYPE=TEXT) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

END-OF-PHASE I MEETING When data from Phase I clinical testing are available, the sponsor again may request a meeting with Food and Drug Administration (FDA) reviewing officials. The primary purpose of this meeting is to review and reach agreement on the design of Phase II controlled clinical trials, with the goal that such testing will be adequate to provide sufficient data on the safety and effectiveness of the drug to support a decision on its approvability for marketing, and to discuss the need for, as well as the design and timing of, studies of the drug in pediatric patients. For drugs for life-threatening diseases, FDA will provide its best judgment, at that time, whether pediatric studies will be required and whether their submission will be deferred until after approval. The procedures outlined in Title 21 312.47(b)(1) with respect to End-of-Phase II conferences, including documentation of agreements reached, would also be used for End-of-Phase I meetings.

This article was modified from the website of the United States Food and Drug Administration (http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/ cfCFR/CFRSearch.cfm?CFRPart=312&showFR=1 &subpartNode=21:5.0.1.1.3.5) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

• Target

ENRICHMENT DESIGN

responder selection: Choosing the subpopulation that is more likely to respond or to experience an event than the general population, such as people who responded to a treatment at the initial stage, or those with a recurrent history of a certain event (10, 11).

VALERII V. FEDOROV Biomedical Data Sciences GlaxoSmithKline Pharmaceuticals Collegeville, Pennsylvania

TAO LIU These selection maneuvers are not mutually exclusive. An enrichment process can often achieve more than one of the above objectives. An effective enrichment can greatly increase the power of detecting the target treatment effect (if only for the selected subpopulation). However, a very strict, multistage enrichment process may lead to a small subpopulation size, and consequently to lower statistical precision or to a prolonged recruitment period. An ideal enrichment design should be based on a careful trade-off between these two aspects. There are many variants of enrichment designs. Figures 1, 2, and 3 show some relatively simple schemes. The randomized discontinuation trial (RDT) (Figure 1) was first proposed by Amery and Dony (4) as an alternative to the classic placebo (or comparator)-controlled, randomized clinical trial (RCT) to reduce the trial’s duration and the degree of the patients’ exposure to inert placebo. In this design, after all of the eligible population have provided informed consent for randomization, they are assigned to an experimental treatment at the first stage. This stage is called the open stage (4); at the end of the open stage, the individuals’ responses (often surrogate endpoints) are collected and evaluated by the study clinician. The individuals who had no response or showed serious adverse effects are excluded from the study. The rest (openstage responders) are then randomized to a placebo or to the experimental treatment (or a comparator) in a double-blind fashion. The first stage serves as a filter for removing those who are unlikely to respond to treatment. The rationale is that the nonresponders would contribute little information about the population for whom the treatment can be useful. The second stage serves

Department of Biostatistics & Epidemiology University of Pennsylvania School of Medicine Philadelphia, Pennsylvania

Enrichment designs for evaluating certain treatments or drugs had been used for decades before Hallstrom and Friedman (1), Temple (2), and Pablos-Mendez et al (3) gave them formal discussion and definition in 1990s. In such designs, a subpopulation is selected or screened out from the general population for an experimental study. The procedure for selection of such a subpopulation is called enrichment (2). The goal of the design is to enhance the signal of an external intervention in the enriched subpopulation and separate it from the interference of many other undesired factors. The discussion and employment of such designs can be traced back to the 1970s (4). Recent years have seen these designs gain great popularity in many different disciplines, particularly in the field of clinical oncology. The enrichment intentions can be roughly classified into the following categories (2). • Variance suppressing selection: Selec-

ting the most homogeneous subpopulation, such as those patients with the greatest tendency to adhere to the study protocol, those whose cholesterol level falls within a certain range, or those with a similar tumor size and health condition (5). • Response

enhanced selection: Identifying the subpopulation with the strongest potential magnitude of response, such as improvement in mental condition, extension of survival, or reduction in tumor growth rate (6–9).

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

ENRICHMENT DESIGN

Eligible population for a study

Experimental Treatment

Non-responders

Treatment responders Randomization

Stop follow-up Experimental Treatment

Placebo Figure 1. The diagram of a randomized discontinuation trial.

Statistical analysis

Eligible population

Placebo run-in: Complied?

Compliers

Non-compliers

Randomization

Stop follow-up

Experimental Treatment

Placebo

Statistical analysis Figure 2. The diagram of a placebo run-in trial.

to distinguish whether the treatment adds anything over the placebo effect. A commonly accepted assumption with an RDT is that ‘‘the treatment will not cure the condition during the open stage’’ (4). For this reason, an RDT is generally applied

under conditions that require sustained use of a therapy (6, 12, 13), such as stabilizing tumor growth or treating some chronic disease. Another often accepted assumption with this design is that the treatment effect at

ENRICHMENT DESIGN

3

Eligible subjects who agree to be randomized Active treatment (100-2a)% of sample

Responders leave the study

Nonresponders continue DB study and receive placebo

Placebo treatment a % of sample

Responders leave the study

Nonresponders continue DB study and receive active treatment

Placebo treatment a% of sample

Responders leave the study

Nonresponders continue DB study and receive placebo

Figure 3. Study design for a major depressive disorder trial. DB, double blind. (From Fava et al [11].)

the open stage will not carry over to the second stage. In oncology, this might mean that the tumor growth rate is uniquely defined by the treatment and is changed as the treatment is changed (6). Traditionally, the statistical analysis of an RDT uses only the outcomes from the second stage, treating it as an RCT rendered on the enriched subpopulation. Capra (14) compared the power of an RDT with that of an RCT when the primary endpoints are individuals’ survival times. Kopec et al (7) evaluated the utility and efficiency of the RDT when the endpoints are binary; they compared the relative sample size required for the desired power of the RDT versus the RCT under various scenarios and parameter settings. Fedorov and Liu (10) considered maximum likelihood estimation of the treatment effect for binary endpoints. With some moderate assumptions, they incorporated the information from the open stage into their inference. The placebo run-in trial (PRIT) is another often used enrichment design (15, 16). It is very similar to the RDT in its setup structure, except that the block ‘‘experimental treatment’’ is replaced by ‘‘placebo treatment’’ and ‘‘responders’’ by ‘‘compliers’’ (Figure 2). An assumption with the PRIT design is that the participants behave coherently throughout the trial. If a patient’s adherence to the protocol is poor during the placebo runin, then his adherence will be poor during the second stage, and vice versa. This design can be more efficient than a conventional RCT

when the compliance of the general population is known or expected to be poor (5) or ‘‘when poor adherence is associated with a substantial reduction of therapy’’ (17). Davis et al (18) examined the efficiency of the PRIT design through empirical evaluations, in the setting of evaluating a cholesterol-lowering drug for elderly patients. The analyses were carried out using the outcomes from the second stage only, as if it were a conventional RCT. Both RDT and PRIT designs are fairly simple schemes. In reality, researchers often employed these designs with certain modifications to meet each study’s requirements. Sometimes RDT and PRIT are even used in combination. For example, Fava et al (11) proposed a study design they named the ‘Sequential Parallel Comparison Design’ for their psychiatric disorder study (see Figure 3 for the design diagram). The first stage of the design consists of three double-blinded (DB) arms: two placebo arms and one treatment arm with unequal randomization (usually more patients are on the placebo arms). Only the nonresponders of the first stage continue to the second stage, and they are assigned in a double-blinded way to the active treatment or placebo, depending on the arm they were on at the first stage. The rationale behind the design is that ‘‘since patients on the second stage have already ‘failed placebo’, their placebo response will be reduced.’’ The data analysis of this design is similar to RDT and PRIT. Fava et al (11) have discussed the statistical model for the design and the design optimization.

4

1

ENRICHMENT DESIGN

parameters of interest can be µk∗ j , µk∗ j and πk∗ , where j and j denote two comparative treatments, and the other parameters can be viewed as nuisance. Often the parameter estimation is complemented or replaced by hypotheses testing. The population model (1) should be complemented by models that describe the evolution of the response to treatment, and the observation processes. For instance, the enrichment process of an RDT is often achieved using only surrogate endpoints at the end of the first stage, which are less accurate measures than the primary endpoints, and can lead to misclassification of treatment responders and nonresponders (Figure 1). Fedorov and Liu (10) proposed to model such an imperfect enrichment process through the introduction of false-positive and falsenegative detection rates. This model builds the connection between the outcomes at the first and the second stages, and hence makes it possible to use the observed information

MODELS

In most publications, it is (implicitly and explicitly) assumed that the general population consists of K subpopulations: ϕ(x, θ ) =

K


πk ϕ(x, θkj ),

j = 1, 2, . . . , J, (1)

k=1

false positive = 0.10, false negative = 0.10 0.5

0.25

0.30

where x is the endpoint of interest (which can be a collection of several variables), π k is a fraction of the k-th subpopulation, ϕ(x, θ kj ) is the distribution density of x in the k-th subpopulation under the j-th treatment, θ kj are unknown parameters defining the distribution of x in each subpopulation, ϕ(x, θ ) is the marginal distribution of x, and the vector θ comprises all π k and θ kj . The goal can be the estimation of all components of θ , or typically a subset of θ , such as the fraction of responders πk∗ and the parameters θk∗ j . In popular settings for continuous x, ϕ(x, θ kj ) is a normal density with θ 1kj = µkj = E(x | k) and θ2kj = σkj 2 = Var(x|k). In this case, the

0.15

0.2

0.10

0.20

0.3

0.15

0.0

0.05

0.1

0.0

0.2

0.4

0.6

0.8

Figure 4. The area below each curve corresponds to the rate for which a randomized discontinuation trial (RDT) is superior to a randomized controlled trail (RCT) given the fraction of patients randomized to placebo.

ENRICHMENT DESIGN

from both stages by constructing the complete data likelihood. In other settings (6), the outcome at the first stage is x(t1 ) (e.g., tumor size) at moment t1 , while at the final stage it is x(t1 + t2 ). Thus, a model describing the relationship between x(t1 ) and x(t1 + t2 ) is needed. In oncology, x(t) can be a tumor growth model. With this model in place, the optimal selection of t1 given t1 + t2 can be considered (6, 8). 2

DESIGN AND EFFICIENCY

In terms of design, the choice of randomization scheme, rates of randomization, and selection of the length of the first stage can be diversified in many ways. Let us consider a simple RDT design (Figure 1) with binary outcomes as an example. Figure 4 shows the efficiency comparison between an RDT design and an RCT with two equal arms (10). Suppose that, at the end of the open phase, 10% of the responders to an active treatment are misclassified as nonresponders, and that 10% of the nonresponders are misclassified as responders. The x-axis represents the population response rate to placebo, and the y-axis the increase in the response rate due to the treatment, the estimation of which is the primary interest. Numbers next to each curve indicate the fraction of patients randomized to the placebo arm in the second stage of an RDT (usually the smaller value is more ethical). In the region under each curve, the RDT dominates the RCT in terms of efficiency. This figure demonstrates that an RDT has better performance under some scenarios (particularly when the treatment effect is small). However this efficiency gain is not universal, even in this simple, idealized setting. For more realistic cases, careful selection of models, combined with ethical and logistical considerations, is essential. In some publications (6, 8, 9, 14), investigators use the Monte Carlo simulations to compare different designs. 3

APPLICATIONS

Majority of applications of enrichment designs are in the field of clinical oncology. Typical examples include a study with

5

enrichment screening based on preliminary data on erlotinib (19), a study of a cytostatic antineoplastic agent (9), and a study of the putative antiangiogenic agent carboxyaminoimoidazole in renal cell carcinoma (20). Temple (21) provided a review and discussion of enrichment designs in cancer treatments. For the early development of molecularly targeted anticancer agents, RDTs were employed to select sensitive subpopulations when an effect assay for such separation was not available (6, 8). Other applications of enrichment designs can be found in children’s health research (22, 23), clinical research in psychiatry (11, 24, 25), cardiac arrhythmia suppression study in cardiology (26), and a few other therapeutic areas (27–30). Similar enrichment strategies can also be found in some two-stage surveys (31). Freidlin and Simon (8) evaluated cytostatic drugs using a design in which they expected only certain patients to be sensitive to their target treatment. 4

DISCUSSION

Because the goal of enrichment is to separate certain subpopulations from the general population and randomize only the selected subpopulation in the trial, the enrichment design is usually only capable of detecting the efficacy rather than effectiveness. This type of efficacy distinction is often of main interest in oncology, which is the reason that RDT is frequently used for screening for a treatment activity. As certain assumptions are satisfied, the efficacy detection can be greatly enhanced. The work by Kopec et al (7) illustrated that, when compared with an RCT, the sample size required for an RDT can be reduced by more than 50%. Fedorov and Liu (10) showed that the increase in efficiency of efficacy detection can be even higher if additional assumptions are made and the information from the open stage can be seamlessly included. However, enrichment designs are not always superior to other designs, even when all of the ethical and logistical conditions are acceptable. For example, studies that have compared the relative efficiency between

6

ENRICHMENT DESIGN

RDTs and the classic RCT (7, 8, 10, 32) have found that the RCT can be more efficient under certain conditions, even when the separation of the subpopulation at the first stage is perfect (i.e., no misclassifications). Other limitations of enrichment designs include: 1. The benefits of using enrichment designs come at the expense of the applicability of the study results to the general population. The Coronary Drug Project Research Group study (22) illustrated the effect of compliance on the conclusions for the enriched subpopulations and the general population. 2. The recruitment process for an enrichment design could last much longer than for a conventional RCT. 3. The use of surrogate endpoints of RDT at the first stage can affect the performance of enrichment process. Fedorov and Liu (10) showed the consequences of misclassification on design efficiency in the first stage of an RDT. 4. The screening phases (the run-in phase for PRIT and the open phase for RDT) are not free; they cost the sponsors time and money and come with errors (33, 34). In general, an enrichment design should undergo a cost-benefit analysis (35) in which efficiency is not the only contributor to the utility function. The enrichment (selection) process is not universally applicable to all scenarios, and may prove ethically controversial (36–38), even though its rationale is well supported and it has been natural to apply to the most commonly reported applications. Researchers must scrutinize the validity of the assumptions and consider the possible ethical issues associated with enrichment before the design is carried out.

REFERENCES 1. A. P. Hallstrom and L. Friedman, Randomizing responders. Control Clin Trials. 1991; 12: 486–503.

2. R. J. Temple, Special study designs: early escape, enrichment, studies in nonresponders. Commun Stat Theory Methods. 1994; 23: 499–531. 3. A. Pablos-Mendez, A. G. Barr, and S. Shea, Run-in periods in randomized trials. JAMA. 1998; 279: 222–225. 4. W. Amery and J. Dony, Clinical trial design avoiding undue placebo treatment. J Clin Pharmacol. 1975; 15: 674–679. 5. E. Brittian and J. Wittes, The run-in period in clinical trials: the effect of misclassification on efficacy. Control Clin Trials. 1990; 11: 327–338. 6. G. L. Rosner, W. Stadler, and M. J. Ratain, Discontinuation design: application to cytostatic antineoplastic agents. J Clin Oncol. 2002; 20: 4478–4484. 7. J. Kopec, M. Abrahamowicz, and J. Esdaile, Randomize discontinuation trials: utility and efficiency. J Clin Epidemiol. 1993; 46: 959–971. 8. B. Freidlin and R. Simon, Evaluation of randomized discontinuation design. J Clin Oncol. 2005; 23: 5094–5098. 9. L. V. Rubinstein, E. L. Korn, B. Freidlin, S. Hunsberger, S. P. Ivy, and M. A. Smith, Design issues of randomized phase II trials and proposal for phase II screening trials. J Clin Oncol. 2005; 23: 7199–7206. 10. V. V. Fedorov and T. Liu, Randomized discontinuation trials: design and efficiency. GlaxoSmithKline Biomedical Data Science Technical Report, 2005-3. 11. M. Fava, A. E. Evins, D. J. Dorer, and D. A. Schoenfeld, The problem of the placebo response in clinical trials for psychiatric disorders: culprits, possible remedies, and a novel study design approach. Psychother Psychosom. 2003; 72: 115–127. 12. C. Chiron, O. Dulac, and L. Gram, Vigabatrin withdrawal randomized study in children. Epilepsy Res. 1996; 25: 209–215. 13. E. L. Korn, S. G. Arbuck, J. M. Pulda, R. Simon, R. S. Kaplan, and M. C. Christian, Clinical trial designs for cytostatic agents: are new approaches needed? J Clin Oncol. 2001; 19: 265–272. 14. W. B. Capra, Comparing the power of the discontinuation design to that of the classic randomized design on time-to-event endpoints. Control Clin Trials. 2004; 25: 168–177. 15. The SOLVD Investigators. Effect of enalapril on survival in patients with reduced left ventricular ejection fractions and congestive heart failure. N Engl J Med. 1991; 325: 293–302.

ENRICHMENT DESIGN 16. J. E. Buring and C. E. Hennekens, Cost and efficiency in clinical trials: the US Physicians’ Health Study. Stat Med. 1990; 9: 29–33. 17. K. B. Schechtman and M. E. Gordon, A comprehensive algorithm for determining whether a run-in strategy will be a costeffective design modification in a randomized clinical trial. Stat Med. 1993; 12: 111–128. 18. C. E. Davis, W. B. Applegate, D. J. Gordon, R. C. Curtis, and M. McCormick, An empirical evaluation of the placebo run-in. Control Clin Trials. 1995; 16: 41–50. 19. Tarceva (erlotinib). Tablet package Insert. Melville, NY: OSI Pharmaceuticals, December 2004. 20. W. M. Stadler, G. Rosner, E. Small, D. Hollis, B. Rini, S. D. Zaentz, and J. Mahoney, Successful implementation of the randomized discontinuation trial design: an application to the study of the putative antiangiogenic agent carboxyaminoimidazole in renal cell carcinoma—CALGB 69901. J Clin Oncol. 2005; 23: 3726–3732. 21. R. J. Temple, Enrichment designs: efficiency in development of cancer treatments. J Clin Oncol. 2005; 23: 4838–4839. 22. The Coronary Drug Project Research Group. Influence of adherence to treatment and response of cholesterol on mortality in the Coronary Drug Project. N Engl J Med. 1980; 303: 1038–1041. 23. P. Casaer, J. Aicardi, P. Curatolo, K. Dias, M. Maia, et al, Flunazirizine in alternating hemiplegia in childhood. An international study in 12 children. Neuropediatrics. 1987; 18: 191–195. 24. F. M. Quitkin and J. G. Rabkin, Methodological problems in studies of depressive disorder: utilities of the discontinuation design. J Clin Psychopharmacol. 1981; 1: 283–288. 25. D. D. Robinson, S. C. Lerfald, B. Bennett, D. Laux, E. Devereaux, et al, Continuation and maintenance treatment of major depression with the monoamine oxidase inhibitor phenelzine: a double blind placebo-controlled discontinuation study. Psychopharmacol Bull. 1991; 27: 31–39.

7

blocker with intrinsic sympathomimetic activity. Arch Intern Med. 1988; 148: 1725–1728. 28. D. S. Echt, P. R. Liebson, L. B. Mitchell, R. W. Peters, D. Obias-Manno, et al, and the CAST Investigators. Mortality and morbidity in patients receiving encainide, flecainide, or placebo: the Cardiac Arrhythmia Suppression Trial. N Engl J Med. 1991; 324: 781–788. 29. J. R. Evans, K. Pacht, P. Huss, D. V. Unverferth, T. M. Bashore, and C. V. Leier, Chronic oral amrinone therapy in congestive heart failure: a double-blind placebo-controlled withdrawal study. Int J Clin Pharmacol Res. 1984; 4: 9–18. 30. G. H. Guyatt, M. Townsend, S. Nogradi, S. O. Pugsley, J. L. Keller, and M. T. Newhouse, Acute response to bronchodilator. An imperfect guide for bronchodilator therapy in chronic airflow limitation. Arch Intern Med. 1988; 148: 1949–1952. 31. J. L. Vazquez-Barquero, J. F. Diez-Manrique, C. Pena, R. G. Quintanal, and M. Labrador Lopez, Two stage design in a community survey. Br J Psychiatry. 1986; 149: 88–897. 32. C. Mallinckrodt, C. Chuang-Stein, P. McSorley, J. Schwartz, D. G. Archibald, et al, A case study comparing a randomized withdrawal trial and a double-blind long term trial for assessing the long-term efficacy of an antidepressant. Pharm Stat. 2007; 6: 9–22. 33. E. Brittain and J. Wittes, The run-in period in clinical trials: the effect of misclassification on efficiency. Control Clin. Trials. 1990; 11: 327–338. 34. R. J. Glynn, J. E. Buring, C. H. Hennekens, Riley, D, T. J. Kaptchuk, et al, Concerns about run-in periods in randomized trials. JAMA. 1998; 279: 1526–1527. 35. K. B. Schechtman and M. E. Gordon, A comprehensive algorithm for determining whether a run-in strategy will be a costeffective design modification in a randomized clinical trial. Stat Med. 1993; 12: 111–128. 36. S. J. Senn, A personal view of some controversies in allocating treatment to patients in clinical trials. Stat Med. 1995; 14: 2661–2674.

26. D. S. Echt, P. R. Liebson, and L. B. Mitchell, Morality and morbidity in patients receiving encainide, flecainide, or placebo. N Engl J Med. 1991; 324: 781–788.

37. S. J. Senn, Ethical considerations concerning treatment allocation in drug development trials. Stat Methods Med Res. 2002; 11: 403–411.

27. T. D. Giles, G. E. Sander, L. Roffidal, M. G. Thomas, D. P. Mersch, et al, Remission of mild to moderate hypertension after treatment with carteolol, a beta-adrenoceptor

38. P. D. Leber and C. S. Davis, Threats to validity of clinical trials employing enrichment strategies for sample selection. Control Clin Trials. 1998; 19: 178–187.

8

ENRICHMENT DESIGN

CROSS-REFERENCES Clinical trial/study Adaptive design Randomization Sample size estimation Run-in period Estimation Inference Hypothesis testing Efficacy Effectiveness

ENVIRONMENTAL ASSESSMENTS (EAS) Under the National Environmental Policy Act of 1969 (NEPA), all Federal agencies are required to assess the environmental impact of their actions and to ensure that the interested and affected public is informed of the environmental analyses. The Center for Drug Evaluation and Research’s (CDER) Environmental Assessment of Human Drug and Biologics Applications (Issued 7/1998, Posted 7/24/98) provides detailed information on a variety of topics related to preparing and filing EAs. In CDER, adherence to NEPA is demonstrated by the EA portion of the drug application. This section focuses on the environmental implications of consumer use and disposal from use of the candidate drug. However, because approval of many drugs are unlikely to have significant environmental effects, CDER has provisions for submission of abbreviated EAs rather than full EAs under certain circumstances or has categorically excluded certain classes of actions. FDA has reevaluated its NEPA regulations found in 21 CFR (Code of Federal Regulations) Part 25 and has proposed to improve its efficiency in the implementation of NEPA and reduce the number of EAs by increasing the number of applicable categorical exclusions. The notice of proposed rule making was posted in the Federal Register on April 3, 1996.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/handbook/environ.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

EQUIVALENCE TRIALS AND EQUIVALENCE LIMITS1

noninferiority hypothesis, and δ is the noninferiority margin. On the other hand, in some applications, noninferiority testing may be intended to study some other objectives, particularly when a placebo is absent from the trial. In such cases, the noninferiority margin δ may have a special meaning; for instance, the noninferiority testing may be to infer that the test treatment would have beaten the placebo had a placebo been in the trial or that the test treatment would have retained a certain fraction of the control’s effect.

H. M. JAMES HUNG Division of Biometrics I, Office of Biostatistics Office of Translational Sciences, Center for Drug Evaluation and Research U.S. Food and Drug Administration Silver Spring, Maryland

In clinical trials, when a test treatment is compared with an active or positive control that is being used in medical practices, one possible study objective may be to show that the test treatment is ‘‘equivalent’’ to the active control with respect to therapeutic benefits or adverse effects. Traditionally, equivalence means that the difference between the test treatment and the control treatment is clinically insignificant. The threshold of clinical insignificance would need to be defined, and this threshold is the so-called equivalence limit or equivalence margin.

2

EQUIVALENCE LIMIT OR MARGIN

In general, there is only one equivalence margin that defines the acceptable degree of clinical difference between the test treatment and the control, particularly when two effective treatments are compared to show that either treatment is not inferior to the other. As already mentioned, for clinical endpoints, the unacceptable margin of the inferiority of the test treatment to the control may have a special definition; for example, it may be an explicit function of the postulated control’s effect. When such a noninferiority margin is determined to show that the test treatment is better than placebo or that the test treatment retains a certain fraction of the control’s effect, it may be too large to conclude that the test treatment is not unacceptably inferior to the control. This margin is certainly irrelevant for defining the degree of superiority of the test treatment over the control; thus, if equivalence testing is pursued in these applications, the equivalence hypothesis might have two limits, one for inferiority and one for superiority.

1 EQUIVALENCE VERSUS NONINFERIORITY In pharmaceutical applications, the concept of equivalence is mostly applied to assessment of a test formulation of a medical product relative to an established reference in so-called bioequivalence studies on a pharmacokinetic variable. In contrast, clinical trials are rarely aimed at showing equivalence; instead, the concept of noninferiority may be more applicable (1–18). The distinction between equivalence and noninferiority can be subtle. On one hand, a noninferiority hypothesis can be viewed as one side of an equivalence hypothesis. That is, if δ is the equivalence margin, then the equivalence hypothesis is that the test treatment and the control treatment differ by an extent smaller than δ. If the hypothesis at stake is that the test treatment is inferior to the control at most by a degree smaller than δ, it is a

3 DESIGN, ANALYSIS, AND INTERPRETATION OF EQUIVALENCE TRIALS The necessary considerations for design, analysis, and interpretation of a noninferiority trial as stipulated in the references are, in principle, applicable to equivalence testing. To accept the equivalence hypothesis defined by the equivalence limits, the confidence

1

The views presented in this paper are not necessarily those of the U.S. Food and Drug Administration.

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

EQUIVALENCE TRIALS AND EQUIVALENCE LIMITS

interval derived from the equivalence trial for the difference between the test treatment and the active control must completely fall within the equivalence range determined by the equivalence margins. This is in contrast to the noninferiority testing that requires that the confidence interval exclude only the relevant noninferiority margin. REFERENCES 1. P. Bauer and M. Kieser, A unifying approach for confidence intervals and testing of equivalence and difference. Biometrika. 1996; 83: 934–937. 2. Committee for Medicinal Products for Human Use (CHMP), European Medicines Agency. Guideline on the Choice of the Non-Inferiority Margin. London, UK, July 27, 2005. Available at: http://www.emea.europa.eu/pdfs/human/ ewp/215899en.pdf. 3. A. F. Ebbutt and L. Frith, Practical issues in equivalence trials. Stat Med. 1998; 17: 1691–1701. 4. S. S. Ellenberg and R. Temple, Placebocontrolled trials and active-control trials in the evaluation of new treatments. Part 2: Practical issues and specific cases. Ann Intern Med. 2000; 133: 464–470. 5. T. R. Fleming, Treatment evaluation in active control studies. Cancer Treat Reports. 1987; 71: 1061–1064. 6. T. R. Fleming, Design and interpretation of equivalence trials. Am Heart J. 2000; 139: S171–S176. 7. A. L. Gould, Another view of active-controlled trials. Control Clin Trials. 1991; 12: 474–485. 8. D. Hauschke, Choice of delta: a special case. Drug Inf J. 2001; 35: 875–879. 9. D. Hauschke and L. A. Hothorn, Letter to the Editor. Stat Med. 2007; 26: 230–233. 10. H. M. J. Hung, S. J. Wang, and R. O’Neill, Noninferiority trial. In: R. D’Agostino, L. Sullivan,

and J. Massaro (eds.), Encyclopedia of Clinical Trials. New York, Wiley, 2007. 11. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH Harmonised Tripartite Guideline: E9 Statistical Principles for Clinical Trials. Step 4 version, February 5, 1998. Available at: http://www.ich.org/LOB/media/ MEDIA485.pdf. 12. B. Jones, P. Jarvis, J. A. Lewis, and A. F. Ebbutt, Trials to assess equivalence: the importance of rigorous methods. BMJ. 1996; 313: 36–39. 13. T. H. Ng, Choice of delta in equivalence testing. Drug Inf J. 2001; 35: 1517–1527. 14. G. Pledger and D. B. Hall, Active control equivalence studies: do they address the efficacy issue? In: K. E. Peace (ed.), Statistical Issues in Drug Research and Development. New York: Marcel Dekker, 1990, pp. 226–238. 15. J. Rohmel, Therapeutic equivalence investigations: statistical considerations. Stat Med. 1998; 17: 1703–1714. 16. R. Temple, Problems in interpreting active control equivalence trials. Account Res. 1996; 4: 267–275. 17. R. Temple and S. S. Ellenberg, Placebocontrolled trials and active-control trials in the evaluation of new treatments. Part 1: Ethical and scientific issues. Ann Intern Med. 2000; 133: 455–463. 18. B. Wiens, Choosing an equivalence limit for non-inferiority or equivalence studies. Control Clin Trials. 2002; 23: 2–14.

CROSS-REFERENCES Non-inferiority Trial Non-inferiority Margin Bioequivalence

ESSENTIAL DOCUMENTS

Any or all documents addressed in this guideline may be subject to, and should be available for, audit by the sponsor’s auditor and inspection by the regulatory authority(ies).

Essential Documents are those documents that permit evaluation individually and collectively of the conduct of a trial and the quality of the data produced. These documents serve to demonstrate the compliance of the investigator, sponsor, and monitor with the standards of Good Clinical Practice (GCP) and with all applicable regulatory requirements. Essential documents also serve several other important purposes. Filing essential documents at the investigator/institution and sponsor sites in a timely manner can assist greatly in the successful management of a trial by the investigator, sponsor, and monitor. Also, these documents usually are audited by the sponsor’s independent audit function and are inspected by the regulatory authority(ies) as part of the process to confirm the validity of the trial conduct and the integrity of data collected. The minimum list of essential documents has been developed. The various documents are grouped in three sections according to the stage of the trial during which they will normally be generated: (1) Before the clinical phase of the trial commences, (2) during the clinical conduct of the trial, and (3) after completion or termination of the trial. A description is given of the purpose of each document, and whether it should be filed in either the investigator/institution or the sponsor files, or both. It is acceptable to combine some documents provided the individual elements are readily identifiable. Trial master files should be established at the beginning of the trial, both at the investigator/institution’s site and at the sponsor’s office. A final close-out of a trial can be performed only when the monitor has reviewed both investigator/institution and sponsor files and has confirmed that all necessary documents are in the appropriate files. This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

ETHICAL CHALLENGES POSED BY CLUSTER RANDOMIZATION

are discussed in Section 4. Section 5 concludes the article largely as a means of promoting debate, to provide recommendations for the reporting of ethical issues in cluster randomization trials. Readers interested in a more detailed discussion might wish to consult Donner and Klar (9), from which much of this article was abstracted.

NEIL KLAR Cancer Care Ontario Division of Preventive Oncology Toronto, Ontario, Canada

ALLAN DONNER The University of Western Ontario Department of Epidemiology and Biostatistics London, Ontario, Canada

1

2

EXAMPLES 1. A group of public health researchers in Montreal (10) conducted a household randomized trial to evaluate the risk of gastrointestinal disease due to consumption of drinking water. Participating households were randomly assigned to receive an in-home water filtration unit or were assigned to a control group that used tap water. Overall, 299 families (1206 individuals) were assigned to receive water filters and 308 families (1201 individuals) were in the control group. 2. The National Cancer Institute of the United States funded the Community Intervention Trial for Smoking Cessation (COMMIT), which investigated whether a community-level, 4-year intervention would increase quit rates of cigarette smokers (11) Communities were selected as the natural experimental unit because investigators assumed that interventions offered at this level would reach the greatest number of smokers and possibly change the overall environment, thus making smoking less socially acceptable. Overall 11 matched-pairs of communities were enrolled in this study with one community in each pair randomly assigned to the experimental intervention with the remaining community serving as a control. 3. Antenatal care in the developing world has attempted to mirror care that is offered in developed countries even though not all antenatal care interventions are known to be effective.

INTRODUCTION

Ethical guidelines for medical experimentation were first put forward in 1947 with publication of the Nuremberg Code (1, 2). Numerous national and international ethical codes have since been developed, written almost exclusively in the context of clinical trials in which a physician’s patients are individually assigned for the purpose of evaluating the effect of therapeutic interventions. In spite of some limited early examples (3), far less attention has been given to the distinct ethical challenges of cluster randomization (4–7), reflecting, perhaps, the recent growth of interest in this design. Experience has shown that cluster randomization is primarily adopted to evaluate nontherapeutic interventions, including lifestyle modification, educational programs, and innovations in the provision of health care. The limited attention given to the ethical aspects of this design may be more broadly related to the fact that trials of disease prevention have tended to be largely exempt from ethical constraints, possibly because of a perception that participation in such trials carries only minimal risk (8). In Section 2, the key ethical issues raised by cluster randomization trials will be examined by discussing them in the context of several recently published studies. The relative potential for harm and benefit to subjects participating in either individually randomized or cluster randomized trials is described in Section 3, whereas difficulties in obtaining informed consent before random assignment

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

ETHICAL CHALLENGES POSED BY CLUSTER RANDOMIZATION

The World Health Organization (WHO) antenatal care randomized trial (12) compared a new model of antenatal care that emphasized health-care interventions known to be effective with the standard model of antenatal care. The primary hypothesis in this equivalence trial was that the new model of antenatal health care would not increase the risk of having a low birthweight () representing the impact of one variable on another, and a double-headed arrow () representing covariance between pairs of variables. In building a CFA model, researchers use these symbols within the framework of three basic configurations, each of which represents an important component in the analytic process. We turn now to the CFA model presented in Figure 1, which represents the postulated four-factor structure of nonacademic self-concept (SC) as tapped by items comprising the Self Description QuestionnaireI (SDQ-I; [15]). As defined by the SDQ-I, nonacademic SC embraces the constructs of physical and social SCs. On the basis of the geometric configurations noted above, decomposition of this CFA model conveys the following information: (a) there are four factors, as indicated by the four ellipses labeled Physical SC (Appearance; F1), Physical SC (Ability; F2), Social SC (Peers; F3), and Social SC (Parents; F4); (b) the four factors are intercorrelated, as indicated by the six two-headed arrows; (c) there are 32 observed variables, as indicated by the 32 rectangles (SDQ1SDQ66); each represents one item from the SDQ-I; (d) the observed variables measure the factors in the following pattern: Items 1, 8, 15, 22, 38, 46, 54, and 62 measure Factor 1, Items 3, 10, 24, 32, 40, 48, 56, and 64 measure Factor 2, Items 7, 14, 28, 36, 44, 52, 60, and 69 measure Factor 3, and Items 5, 19, 26, 34, 42, 50, 58, and 66 measure Factor 4; (e) each observed variable measures one and only one factor; and (f) errors of measurement associated with each observed variable (E1-E66) are uncorrelated (i.e., there are no double-headed arrows connecting any two error terms. Although the error variables, technically speaking, are unobserved variables, and should have ellipses around them, common convention in such diagrams omits them in the interest of clarity. In summary, a more formal description of the CFA model in Figure 1 argues that: (a) responses to the SDQ-I are explained by four factors; (b) each item has a nonzero loading on the nonacademic SC factor it was designed to measure (termed target loadings), and zero loadings on all other factors (termed nontarget loadings); (c) the four factors are correlated; and (d) measurement error terms are uncorrelated.

3

Structural Equation Specification of the Model From a review of Figure 1, you will note that each observed variable is linked to its related factor by a single-headed arrow pointing from the factor to the observed variable. These arrows represent regression paths and, as such, imply the influence of each factor in predicting its set of observed variables. Take, for example, the arrow pointing from Physical SC (Ability) to SDQ1. This symbol conveys the notion that responses to Item 1 of the SDQ-I assessment measure are ‘caused’ by the underlying construct of physical SC, as it reflects one’s perception of his or her physical ability. In CFA, these symbolized regression paths represent factor loadings and, as with all factor analyses, their strength is of primary interest. Thus, specification of a hypothesized model focuses on the formulation of equations that represent these structural regression paths. Of secondary importance are any covariances between the factors and/or between the measurement errors. The building of these equations, in SEM, embraces two important notions: (a) that any variable in the model having an arrow pointing at it represents a dependent variable, and (b) dependent variables are always explained (i.e., accounted for) by other variables in the model. One relatively simple approach to formulating these structural equations, then, is first to note each dependent variable in the model and then to summarize all influences on these variables. Turning again to Figure 1, we see that there are 32 variables with arrows pointing toward them; all represent observed variables (SDQ1-SDQ66). Accordingly, these regression paths can be summarized in terms of 32 separate equations as follows: SDQ1 = F1 + E1 SDQ8 = F1 + E8 SDQ15.= F1 + E15 .. SDQ62 = F1 + E62 SDQ3 = F2 + E3 SDQ10.= F2 + E10 .. SDQ64 = F2 + E64 SDQ7 = F3 + E7 SDQ14.= F3 + E14 .. SDQ69 = F3 + E69

4

Factor Analysis: Confirmatory SDQ5 = F4 + E5 SDQ19.= F4 + E19 .. SDQ66 = F4 + E66 (1)

Although, in principle, there is a one-to-one correspondence between the schematic presentation of a model and its translation into a set of structural equations, it is important to note that neither one of these representations tells the whole story. Some parameters, critical to the estimation of the model, are not explicitly shown and thus may not be obvious to the novice CFA analyst. For example, in both the schematic model (see Figure 1) and the linear structural equations cited above, there is no indication that either the factor variances or the error variances are parameters in the model. However, such parameters are essential to all structural equation models and therefore must be included in the model specification. Typically, this specification is made via a separate program command statement, although some programs may incorporate default values. Likewise, it is equally important to draw your attention to the specified nonexistence of certain parameters in a model. For example, in Figure 1, we detect no curved arrow between E1 and E8, which would suggest the lack of covariance between the error terms associated with the observed variables SDQ1 and SDQ8. (Error covariances can reflect overlapping item content and, as such, represent the same question being asked, but with a slightly different wording.)

Testing a Hypothesized CFA Model Testing for the validity of a hypothesized CFA model requires the satisfaction of certain statistical assumptions and entails a series of analytic steps. Although a detailed review of this testing process is beyond the scope of the present chapter, a brief outline is now presented in an attempt to provide readers with at least a flavor of the steps involved. (For a nonmathematical and paradigmatic introduction to SEM based on three different programmatic approaches to the specification and testing of a variety of basic CFA models, readers are referred to [6–9]; for a more detailed and analytic approach to SEM, readers are referred to [3], [14], [16] and [17].)

Statistical Assumptions As with other multivariate methodologies, SEM assumes that certain statistical conditions have been met. Of primary importance is the assumption that the data are multivariate normal (see Catalogue of Probability Density Functions). In essence, the concept of multivariate normality embraces three requirements: (a) that the univariate distributions are normal; (b) that the joint distributions of all variable combinations are normal; and (c) that all bivariate scatterplots are linear and homoscedastic [14]. Violations of multivariate normality can lead to the distortion of goodness-of-fit indices related to the model as a whole (see e.g., [12]; [10]; and (see Goodness of Fit) to positively biased tests of significance related to the individual parameter estimates [14]).

Estimating the Model Once the researcher determines that the statistical assumptions have been met, the hypothesized model can then be tested statistically in a simultaneous analysis of the entire system of variables. As such, some parameters are freely estimated while others remain fixed to zero or some other nonzero value. (Nonzero values such as the 1’s specified in Figure 1 are typically assigned to certain parameters for purposes of model identification and latent factor scaling.) For example, as shown in Figure 1, and in the structural equation above, the factor loading of SDQ8 on Factor 1 is freely estimated, as indicated by the single-headed arrow leading from Factor 1 to SDQ8. By contrast, the factor loading of SDQ10 on Factor 1 is not estimated (i.e., there is no single-headed arrow leading from Factor 1 to SDQ10); this factor loading is automatically fixed to zero by the program. Although there are four main methods for estimating parameters in CFA models, maximum likelihood estimation remains the one most commonly used and is the default method for all SEM programs.

Evaluating Model Fit Once the CFA model has been estimated, the next task is to determine the extent to which its specifications are consistent with the data. This evaluative process focuses on two aspects: (a) goodness-of-fit of the model as a whole, and (b) goodness-of-fit of individual parameter estimates. Global assessment of fit

Factor Analysis: Confirmatory is determined through the examination of various fit indices and other important criteria. In the event that goodness-of-fit is adequate, the model argues for the plausibility of postulated relations among variables; if it is inadequate, the tenability of such relations is rejected. Although there is now a wide array of fit indices from which to choose, typically only one or two need be reported, along with other fit-related indicators. A typical combination of these evaluative criteria might include the Comparative Fit Index (CFI; Bentler, [1]), the standardized root mean square residual (SRMR), and the Root Mean Square Error of Approximation (RMSEA; [18]), along with its 90% confidence interval. Indicators of a well-fitting model would be evidenced from a CFI value equal to or greater than .93 [11], an SRMR value of less than .08 [11], and an RMSEA value of less than .05 [4]. Goodness-of-fit related to individual parameters of the model focuses on both the appropriateness (i.e., no negative variances, no correlations >1.00) and statistical significance (i.e., estimate divided by standard error >1.96) of their estimates. For parameters to remain specified in a model, their estimates must be statistically significant.

Post Hoc Model-fitting Presented with evidence of a poorly fitting model, the hypothesized CFA model would be rejected. Analyses then proceed in an exploratory fashion as the researcher seeks to determine which parameters in the model are misspecified. Such information is gleaned from program output that focuses on modification indices (MIs), estimates that derive from testing for the meaningfulness of all constrained (or fixed) parameters in the model. For example, the constraint that the loading of SDQ10 on Factor 1 is zero, as per Figure 1 would be tested. If the MI related to this fixed parameter is large, compared to all other MIs, then this finding would argue for its specification as a freely estimated parameter. In this case, the new parameter would represent a loading of SDQ10 on both Factor 1 and Factor 2. Of critical importance in post hoc model-fitting, however, is the requirement that only substantively meaningful parameters be added to the original model specification.

5

Interpreting Estimates Shown in Figure 2 are standardized parameter estimates resulting from the testing of the hypothesized CFA model portrayed in Figure 1. Standardization transforms the solution so that all variables have a variance of 1; factor loadings will still be related in the same proportions as in the original solution, but parameters that were originally fixed will no longer have the same values. In a standardized solution, factor loadings should generally be less than 1.0 [14]. Turning first to the factor loadings and their associated errors of measurement, we see that, for example, the regression of Item SDQ15 on Factor 1 (Physical SC; Appearance) is .82. Because SDQ15 loads only on Factor 1, we can interpret this estimate as indicating that Factor 1 accounts for approximately 67% (100 × .822 ) of the variance in this item. The measurement error coefficient associated with SDQ15 is .58, thereby indicating that some 34% (as a result of decimal rounding) of the variance associated with this item remains unexplained by Factor 1. (It is important to note that, unlike the LISREL program [13], which does not standardize errors in variables, the EQS program [2] used here does provide these standardized estimated values; see Structural Equation Modeling: Software.) Finally, values associated with the double-headed arrows represent latent factor correlations. Thus, for example, the value of .41 represents the correlation between Factor 1 (Physical SC; Appearance) and Factor 2 (Physical SC; Ability). These factor correlations should be consistent with the theory within which the CFA model is grounded. In conclusion, it is important to emphasize that only issues related to the specification of first-order CFA models, and only a cursory overview of the steps involved in testing these models has been included here. Indeed, sound application of SEM procedures in testing CFA models requires that researchers have a comprehensive understanding of the analytic process. Of particular importance are issues related to the assessment of multivariate normality, appropriateness of sample size, use of incomplete data, correction for nonnormality, model specification, identification, and estimation, evaluation of model fit, and post hoc model-fitting. Some of these topics are covered in other entries, as well as the books and journal articles cited herein.

6

Factor Analysis: Confirmatory

0.82 0.70* Physical SC (Appearance) F1

0.82* 0.85* 0.58* 0.72* 0.69* 0.71*

SDQ1

0.58

E1*

SDQ8

0.71

E8*

SDQ15

0.58

E15*

SDQ22

0.53

E22*

SDQ38

0.82

E38*

SDQ46

0.69

E46*

SDQ54

0.73

E54*

SDQ62

0.71

E62*

SDQ3

0.66

E3*

SDQ10

0.85

E10*

SDQ24

0.84

E24*

SDQ32

0.85

E32*

SDQ40

0.61

E40*

SDQ48

0.69

E48*

SDQ56

0.48

E56*

SDQ64

0.82

E64*

SDQ7

0.79

E7*

SDQ14

0.74

E14*

SDQ28

0.68

E28*

SDQ36

0.79

E36*

SDQ44

0.77

E44*

SDQ52

0.78

E52*

SDQ60

0.71

E60*

SDQ69

0.61

E69*

SDQ5

0.78

E5*

0.63 0.56*

SDQ19

0.83

E19*

0.60* 0.55* 0.66* 0.73* 0.83* 0.69*

SDQ26

0.80

E26*

SDQ34

0.83

E34*

SDQ42

0.76

E42*

SDQ50

0.68

E50*

SDQ58

0.55

E58*

SDQ66

0.73

E66*

0.41*

0.55*

Physical SC (Ability) F2

0.75 0.53* 0.54* 0.52* 0.80* 0.73* 0.88* 0.57*

0.52* 0.29* 0.61 0.67* Social SC (Peers) F3 0.21*

0.74* 0.61* 0.64* 0.62* 0.71* 0.79*

0.42*

Social SC (Parents) F4

Figure 2

Standardized estimates for hypothesized CFA model

Factor Analysis: Confirmatory

References [1] [2] [3] [4]

[5]

[6]

[7]

[8]

[9]

[10]

Bentler, P.M. (1990). Comparative fit indexes in structural models, Psychological Bulletin 107, 238–246. Bentler, P.M. (2004). EQS 6.1: Structural Equations Program Manual, Multivariate Software Inc, Encino. Bollen, K. (1989). Structural Equations with Latent Variables, Wiley, New York. Browne, M.W. & Cudeck, R. (1993). Alternative ways of assessing model fit, in Testing Structural Equation Models, K.A. Bollen & J.S. Long eds, Sage, Newbury Park, pp. 136–162. Bryant, F.B. & Yarnold, P.R. (1995). Principal-components analysis and exploratory and confirmatory factor analysis, in Reading and Understanding Multivariate Statistics, L.G. Grimm & P.R. Yarnold eds, American Psychological Association, Washington. Byrne, B.M. (1994). Structural Equation Modeling with EQS and EQS/Windows: Basic Concepts, Applications, and Programming, Sage, Thousand Oaks. Byrne, B.M. (1998). Structural Equation Modeling with LISREL, PRELIS, and SIMPLIS: Basic Concepts, Applications, and Programming, Erlbaum, Mahwah. Byrne, B.M. (2001a). Structural Equation Modeling with AMOS: Basic Concepts, Applications, and Programming, Erlbaum, Mahwah. Byrne, B.M. (2001b). Structural equation modeling with AMOS, EQS, and LISREL: comparative approaches to testing for the factorial validity of a measuring instrument, International Journal of Testing 1, 55–86. Curran, P.J., West, S.G. & Finch, J.F. (1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis, Psychological Methods 1, 16–29.

7

[11]

Hu, L.-T. & Bentler, P.M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives, Structural Equation Modeling 6, 1–55. [12] Hu, L.-T., Bentler, P.M. & Kano, Y. (1992). Can test statistics in covariance structure analysis be trusted? Psychological Bulletin 112, 351–362. [13] J¨oreskog, K.G. & S¨orbom, D. (1996). LISREL 8: User’s Reference Guide, Scientific Software International, Chicago. [14] Kline, R.B. (1998). Principles and Practice of Structural Equation Modeling, Guildwood Press, New York. [15] Marsh, H.W. (1992). Self Description Questionnaire (SDQ) I: A Theoretical and Empirical Basis for the Measurement of Multiple Dimensions of Preadolescent Self-concept: A Test Manual and Research Monograph, Faculty of Education, University of Western Sydney, Macarthur, New South Wales. [16] Maruyama, G.M. (1998). Basics of Structural Equation Modeling, Sage, Thousand Oaks. [17] Raykov, T. & Marcoulides, G.A. (2000). A First Course Oin Structural Equation Modeling, Erlbaum, Mahwah. [18] Steiger, J.H. (1990). Structural model evaluation and modification: an interval estimation approach, Multivariate Behavioral Research 25, 173–180.

(See also History of Path Analysis; Linear Statistical Models for Causation: A Critical Review; Residuals in Structural Equation, Factor Analysis, and Path Analysis Models; Structural Equation Modeling: Checking Substantive Plausibility) BARBARA M. BYRNE

FACTORIAL DESIGNS IN CLINICAL TRIALS

a recent chapter discussing factorial designs in medical studies given by Piantadosi (24).

Steven Piantadosi

1 BASIC FEATURES OF FACTORIAL DESIGNS

Johns Hopkins University Baltimore, MD, USA

The simplest factorial design has two treatments (A and B) and four treatment groups (Table 1). There might be n patients entered into each of the four treatment groups for a total sample size of 4n and a balanced design. One group receives neither A nor B, a second receives both A and B, and the other two groups receive one of A or B. This is called a 2 × 2 (two by two) factorial design. The design generates enough information to test the effects of A alone, B alone, and A plus B. The 2 × 2 design generalizes to higher order factorials. For example, a design studying three treatments, A, B, and C, is the 2 × 2 × 2. Possible treatment groups for this design are shown in Table 2. The total sample size is 8n if all treatment groups have n subjects. These examples highlight some of the prerequisites necessary for, and restrictions on, using a factorial trial. First, the treatments must be amenable to being administered in combination without changing dosage in the presence of each other. For example, in Table 1, we would not want to reduce the dose of A in the lower right cell where B is present. This requirement implies that the side effects of the treatments cannot be cumulative to the point where the combination is impossible to administer. Secondly, it must be ethically acceptable to withhold the individual treatments, or administer them at lower doses as the case may be. In some situations, this means having a no-treatment or placebo group in the trial. In other cases A and B may be administered in addition to a ‘‘standard’’ so that all groups receive some treatment. Thirdly, we must be genuinely interested in learning about treatment combinations; otherwise, some of the treatment groups might be unnecessary. Alternately, to use the design to achieve greater efficiency in

Factorial experiments test the effect of more than one treatment (factor) using a design that permits an assessment of interactions between the treatments. A treatment could be either a single therapy or a combination of interventions. The essential feature of factorial designs is that treatments are varied systematically (i.e. some groups receive more than one treatment), and the experimental groups are arranged in a way that permits testing whether or not the treatments interact with one another. The technique of varying more than one treatment in a single study has been used widely in agriculture and industry based on work by Fisher 10,11 and Yates (33). Influential discussions of factorial experiments were given by Cox (8) and Snedecor & Cochran (28). Factorial designs have been used relatively infrequently in medical trials, except recently in disease prevention studies. The discussion here will be restricted to randomized factorial clinical trials. Factorial designs offer certain advantages over conventional comparative designs, even those employing more than two treatment groups. The factorial structure permits certain comparisons to be made that cannot be achieved by any other design. In some circumstances, two treatments can be tested in a factorial trial using the same number of subjects ordinarily used to test one treatment. However, the limitations of factorial designs must be understood before deciding whether or not they are appropriate for a particular therapeutic question. Additional discussions of factorial designs in clinical trials can be found in Byar & Piantadosi (6) and Byar et al. (7). For a discussion of such designs related to cardiology trials, particularly in the context of the ISIS-4 trial (17), see Lubsen & Pocock (20). This article is based on

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

FACTORIAL DESIGNS IN CLINICAL TRIALS

Table 1. Treatment Groups and Sample Sizes in a 2 × 2 Balanced Factorial Design No Yes Total

n n 2n

n n 2n

2n 2n 4n

Table 2. Treatment Groups in a Balanced 2 × 2 × 2 Factorial Design 1 2 3 4 5 6 7 8

No Yes No No Yes No Yes Yes

No No Yes No Yes Yes No Yes

No No No Yes No Yes Yes Yes

n n n n n n n n

are group averages of some normally distributed response denoted by Y. The subscripts on Y indicate which treatment group it represents. Half the patients receive one of the treatments (this is also true in higher order designs). For a moment, further assume that the effect of A is not influenced by the presence of B. There are two estimates of the effect of treatment A compared to placebo in the design, Y A − Y 0 and Y AB − Y B . If B does not modify the effect of A, the two estimates can be combined (averaged) to estimate the overall effect of A(β A ), βA =

(Y A − Y 0 ) + (Y AB − Y B ) . 2

(1)

(Y B − Y 0 ) + (Y AB − Y A ) . 2

(2)

Similarly, studying two or more treatments, we must know that some interactions do not exist. Fourthly, the therapeutic questions must be chosen appropriately. We would not use a factorial design to test treatments that have exactly the same mechanisms of action (e.g. two ACE inhibitors for high blood pressure) because either would answer the question. Treatments acting through different mechanisms would be more appropriate for a factorial design (e.g. radiotherapy and chemotherapy for tumors). In some prevention factorial trials, the treatments tested also target different diseases. 1.1 Efficiency Factorial designs offer certain very important efficiencies or advantages when they are applicable. Consider the 2 × 2 design and the estimates of treatment effects that would result using an additive model for analysis (Table 3). Assume that the responses

Table 3. Treatment Effects from a 2 × 2 Factorial Design A No Yes No Yes

Y0 YA

YB Y AB

βB =

Thus, in the absence of interactions (i.e. the effect of A is the same with or without B, and vice versa), the design permits the full sample size to be used to estimate two treatment effects. Now suppose that each patient’s response has a variance σ 2 that is the same in all treatment groups. We can calculate the variance of β A to be var (βA ) =

1 4σ 2 σ2 × = . 4 n n

This is the same variance that would result if A were tested against placebo in a single twoarmed comparative trial with 2n patients in each treatment group. Similarly, var (βB ) =

σ2 . n

However, if we tested A and B in separate trials, we would require 4n subjects in each trial or a total of 8n patients to have the same precision. Thus, in the absence of interactions, factorial designs estimate main effects efficiently. In fact, tests of both A and B can be conducted in a single factorial trial with the same precision as two single-factor trials using twice the sample size.

FACTORIAL DESIGNS IN CLINICAL TRIALS

2

INTERACTIONS

for treatment A,

The effect of A might be influenced by the presence of B (or vice versa). In other words, there might be a treatment interaction. Some of the efficiencies just discussed will be lost. However, factorial designs are even more relevant when interactions are possible. Factorial designs are the only type of trial design that permits study of treatment interactions. This is because the design has treatment groups with all possible combinations of treatments, allowing the responses to be compared directly. Consider again the two estimates of A in the 2 × 2 design, one in the presence of B and the other in the absence of B. The definition of an interaction is that the effect of A in the absence of B is different from the effect of A in the presence of B. This can be estimated by comparing βAB = (Y A − Y 0 ) − (Y AB − Y B )

(3)

to zero. If β AB is near zero, we would conclude that no interaction is present. It is straightforward to verify that β AB = β BA . When there is an AB interaction present, we must modify our interpretation of the main effects. For example, the estimates of the main effects of A and B [(1) and (2)] assumed no interaction was present. We may choose to think of an overall effect of A, but recognize that the magnitude (and possibly the direction) of the effect depends on B. In the absence of the other treatment, we could estimate the main effects using βA
= (Y A − Y 0 )

(4)

βB
= (Y B − Y 0 ).

(5)

and

In the 2 × 2 × 2 design, there are three main effects and four interactions possible, all of which can be tested by the design. Following the notation above, the effects are βA = 14 [(Y A − Y 0 ) + (Y AB − Y B ) + (Y AC − Y C ) + (Y ABC − Y BC )],

3

(6)

βAB = 12 {[(Y A − Y 0 ) − (Y AB − Y B )] +[(Y AC − Y C ) − (Y ABC − Y BC )]}, (7) for the AB interaction, and βABC = [(Y A − Y 0 ) − (Y AB − Y B ) − (Y AC −Y C ) − (Y ABC − Y BC )]

(8)

for the ABC interaction. When certain interactions are present, we may require an alternative estimator for β A or β BA (or for other effects). Suppose that there is evidence of an ABC interaction. Then, instead of β A , one possible estimator of the main effect of A is βA
= 12 [(Y A − Y 0 ) + (Y AB − Y B )], which does not use β ABC . Other estimators of the main effect of A are possible. Similarly, the AB interaction could be tested by
= (Y A − Y 0 ) − (Y AB − Y B ), βAB

for the same reason. Thus, when treatment interactions are present, we must modify our estimates of main effects and lower order interactions, losing some efficiency. 2.1 Scale of Measurement In the examples just given, the treatment effects and interactions have been assumed to exist on an additive scale. This is reflected in the use of sums and differences in the formulas for estimation. Other scales of measurement may be useful. As an example, consider the response data in Table 4, where the effect of Treatment A is to increase the baseline response by 10 units. The same is

Table 4. Response Data from a Factorial Trial Showing no Interaction on an Additive Scale A No

Yes

No Yes

5 15

15 25

4

FACTORIAL DESIGNS IN CLINICAL TRIALS

true of B and there is no interaction between the treatments on this scale because the joint effect of A and B is to increase the response by 20 units. In contrast, in Table 5 are shown data in which the effects of both treatments are to multiply the baseline response by 3.0. Hence, the combined effect of A and B is a nine fold increase which is greater than the joint treatment effect for the additive case. If the analysis model were multiplicative, then Table 4 would show an interaction, whereas if the analysis model were additive, then Table 5 would show an interaction. Thus, to discuss interactions, we must establish the scale of measurement. 2.2 Main Effects and Interactions In the presence of an interaction in the 2 × 2 design, one cannot speak simply about an overall, or main, effect of either treatment. This is because the effect of A is different depending on the presence or absence of B. In the presence of a small interaction, where all patients benefit from A regardless of the use of B, we might observe that the magnitude of the ‘‘overall’’ effect of A is of some size and that therapeutic decisions are unaffected by the presence of an interaction. This is called ‘‘quantitative’’ interaction, so-named because it does not affect the direction of the treatment effect. For large quantitative interactions, it may not be sensible to talk about overall effects. In contrast, if the presence of B reverses the effect of A, then the interaction is ‘‘qualitative’’, and treatment decisions may need to be modified. Here, we cannot talk about an overall effect of A, because it could be positive in the presence of B, negative in the absence of B, and could yield an average effect near zero.

Yes

No Yes

5 15

Motivation for the estimators given above can be obtained using general linear models. There has been little theoretic work on analyses using other models. One exception is the work by Slud (27) describing approaches to factorial trials with survival outcomes. Suppose we have conducted a 2 × 2 factorial experiment with group sizes given by Table 1. We can estimate the AB interaction effect using the linear model E{Y} = β0 + βA XA + βB XB + βAB XA XB , (9) where the Xs are indicator variables for the treatment groups and β AB is the interaction effect. The design matrix has dimension 4n × 4 and is 

1 0
X = 0 0

... ... ... ...

1 1 0 0

... ... ... ...

15 45

1 0 1 0

... ... ... ...

1 1 1 1

 ... . . . , . . . ...

where there are four blocks of n identical rows representing each treatment group and the columns represent effects for the intercept, treatment A, treatment B, and both treatments, respectively. The vector of responses has dimension 4n × 1 and is Y
= {Y01 , . . . , YA1 , . . . , YB1 , . . . , YAB1 , . . .}. The ordinary least squares solution for the model (9) is βˆ = (X
X)−1 X
Y. The covariance matrix is (X’X)−1 σ 2 , where the variance of each observation is σ 2 . We have  4 2
 XX = n× 2 1 

Table 5. Response Data From a Factorial Trial Showing no Interaction on a Multiplicative Scale A No

2.3 Analysis

(X
X)−1

2 2 1 1

2 1 2 1

 1 1 , 1 1

 1 −1 −1 1 1 −1 2 1 −2 , = × n −1 1 2 −2 1 −2 −2 4

FACTORIAL DESIGNS IN CLINICAL TRIALS

5

and

and 



 3 1 1 2 4 −2 −2 σ
∗ } = cov{β × − 12 1 0 . n 1 −2 0 1

Y 0 + Y A + Y B + Y AB   Y A + Y AB
, XY =n×   Y B + Y AB Y AB where Y i denotes the average response in the ith group. Then,  Y0   −Y 0 + Y A , βˆ =    −Y 0 + Y B Y 0 − Y A − Y B + Y AB 

(10)

which corresponds to the estimators given above in (3)–(5). However, if we assume no interaction, then the β AB effect is removed from the model, and we obtain the estimator  3  1 1 1 4 Y 0 + 4 Y A + 4 Y B − 4 Y AB  1  1 1 1  βˆ ∗ =  − 2 Y 0 + 2 Y A − 2 Y B + 2 Y AB  . 1 1 1 1 − 2 Y 0 − 2 Y A + 2 Y B + 2 Y AB The main effects for A and B are as given above in (1) and (2). The covariance matrices for these estimators are   1 −1 −1 1 2 −1 2 1 −2 σ 
= × cov{β} −1 1 2 −2 n 1 −2 −2 4

In the absence of an interaction, the main effects of A and B are estimated independently and with higher precision than when an interaction is present. The interaction effect is relatively imprecisely estimated, indicating that larger sample sizes are required to have a high power to detect such effects. 3

EXAMPLES

Several clinical trials conducted in recent years have used factorial designs. A sample of such studies is shown in Table 6. One important study using a 2 × 2 factorial design is the Physicians’ Health Study 16,30. This trial has been conducted, in 22 000 physicians in the US and was designed to test the effects of (i) aspirin on reducing cardiovascular mortality and (ii) β-carotene on reducing cancer incidence. The trial is noteworthy in several ways, including its test of two interventions in unrelated diseases, use of physicians as subjects to report outcomes reliably, relatively low cost, and an all-male (high risk)

Table 6. Some Recent Randomized Clinical Trials using Factorial Designs Trial Design Physicians’ Health Study ATBC Prevention Trial Desipramine ACAPS Linxian Nutrition Trial Retinitis pigmentosa Linxian Cataract Trial Tocopherol/deprenyl Womens’ Health Initiative Polyp Prevention Trial Cancer/eye disease Cilazapril/hydrochlorothiazide Nebivolol Endophthalmitis vitrectomy study Bicalutamide/flutamide ISIS-4 Source: adapted from Piantadosi

Reference 2×2 2×2 2×2 2×2 24 2×2 2×2 23 2×2 2×2 4×3 4×3 2×2 2×2 23

Hennekens & Eberlein Heinonen et al. Max et al. ACAPS Group Li et al. Berson et al. Sperduto et al. Parkinson Study Group Assaf & Carleton Greenberg et al. Green et al. Pordy Lacourciere et al. Endophthalmitis Vitrectomy Study Group Schellhammer et al. ISIS-4 Collaborative Group

6

FACTORIAL DESIGNS IN CLINICAL TRIALS

study population. This last characteristic has led to some unwarranted criticism. In January 1988 the aspirin component of the Physicians’ Health Study was discontinued, because evidence demonstrated convincingly that it was associated with lower rates of myocardial infarction (20). The question concerning the effect of β-carotene on cancer remains open and will be addressed by continuation of the trial. In the likely absence of an interaction between aspirin and β-carotene, the second major question of the trial will be unaffected by the closure of the aspirin component. Another noteworthy example of a 2 × 2 factorial design is the α-tocopherol β-carotene Lung Cancer Prevention Trial conducted in 29 133 male smokers in Finland between 1987 and 1994 3,15. In this study, lung cancer incidence is the sole outcome. It was thought possible that lung cancer incidence could be reduced by either or both interventions. When the intervention was completed in 1994, there were 876 new cases of lung cancer in the study population during the trial. Alpha-tocopherol was not associated with a reduction in the risk of cancer. Surprisingly, β-carotene was associated with a statistically significantly increased incidence of lung cancer (4). There was no evidence of a treatment interaction. The unexpected findings of this study have been supported by the recent results of another large trial of carotene and retinol (32). The Fourth International Study of Infarct Survival (ISIS-4) was a 2 × 2 × 2 factorial trial assessing the efficacy of oral captopril, oral mononitrate, and intravenous magnesium sulfate in 58 050 patients with suspected myocardial infarction 12,17. No significant interactions among the treatments were observed and each main effect comparison was based on approximately 29 000 treated vs. 29 000 control patients. Captopril was associated with a small but statistically significant reduction in five-week mortality. The difference in mortality was 7.19% vs. 7.69% (143 events out of 4319), illustrating the ability of large studies to detect potentially important treatment effects even when they are small in relative magnitude. Mononitrate and magnesium therapy did not significantly reduce five-week mortality.

4 SIMILAR DESIGNS 4.1 Fractional and Partial Factorial Designs Fractional factorial designs are those which omit certain treatment groups by design. A careful analysis of the objectives of an experiment, its efficiency, and the effects that it can estimate may justify not using some groups. Because many cells contribute to the estimate of any effect, a design may achieve its intended purpose without some of the cells. In the 2 × 2 design, all treatment groups must be present to permit estimating the interaction between A and B. However, for higher order designs, if some interactions are thought biologically not to exist, omitting certain treatment combinations from the design will still permit estimates of other effects of interest. For example, in the 2 × 2 × 2 design, if the interaction between A, B, and C is thought not to exist, omitting that treatment cell from the design will still permit estimation of all the main effects. The efficiency will be somewhat reduced, however. Similarly, the two-way interactions can still be estimated without Y ABC . This can be verified from the formulas above. More generally, fractional high-order designs will produce a situation termed ‘‘aliasing’’, in which the estimates of certain effects are algebraically identical to completely different effects. If both effects are biologically possible, the design will not be able to reveal which effect is being estimated. Naturally, this is undesirable unless additional information is available to the investigator to indicate that some aliased effects are zero. This can be used to advantage in improving efficiency and one must be careful in deciding which cells to exclude. See Cox (8) or Mason & Gunst (21) for a discussion of this topic. The Women’s Health Initiative clinical trial is a 2 × 2 × 2 partial factorial design studying the effects of hormone replacement, dietary fat reduction, and calcium and vitamin D on coronary disease, breast cancer, and osteoporosis (2). All eight combinations of treatments are given, but participants may opt to join one, two, or all three of the randomized components. The study is expected to accrue over 64 000 patients and is projected to finish in the year 2007. The dietary

FACTORIAL DESIGNS IN CLINICAL TRIALS

component of the study will randomize 48 000 women using a 3:2 allocation ratio in favor of the control arm and nine years of followup. Such a large and complex trial presents logistical difficulties, questions about adherence, and sensitivity of the intended power to assumptions that can only roughly be validated. 4.2 Incomplete Factorial Designs When treatment groups are dropped out of factorial designs without yielding a fractional replication, the resulting trials have been termed ‘‘incomplete factorial designs’’ (7). In incomplete designs, cells are not missing by design intent, but because some treatment combinations may be infeasible. For example, in a 2 × 2 design, it may not be ethically possible to use a placebo group. In this case, one would not be able to estimate the AB interaction. In other circumstances, unwanted aliasing may occur, or the efficiency of the design to estimate main effects may be greatly reduced. In some cases, estimators of treatment and interaction effects are biased, but there may be reasons to use a design that retains as much of the factorial structure as possible. For example, they may be the only way in which to estimate certain interactions.

REFERENCES 1. ACAPS Group (1992). Rationale and design for the Asymptomatic Carotid Artery Plaque Study (ACAPS), Controlled Clinical Trials 13, 293–314. 2. Assaf, A. R. & Carleton, R. A. (1994). The Women’s Health Initiative clinical trial and observational study: history and overview, Rhode Island Medicine 77, 424–427. 3. ATBC Cancer Prevention Study Group (1994). The alpha-tocopherol beta-carotene lung cancer prevention study: design, methods, participant characteristics, and compliance, Annals of Epidemiology 4, 1–9. 4. ATBC Cancer Prevention Study Group (1994). The effect of vitamin E and beta carotene on the incidence of lung cancer and other cancers in male smokers, New England Journal of Medicine 330, 1029–1034.

7

5. Berson, E. L., Rosner, B., Sandberg, M. A., Hayes, K. C., Nicholson, B. W., WeigelDiFranco, C. & Willett, W. (1993). A randomized trial of vitamin A and vitamin E supplementation for retinitis pigmentosa, Archives of Ophthalmology 111, 761–772. 6. Byar, D. P. & Piantadosi, S. (1985). Factorial designs for randomized clinical trials, Cancer Treatment Reports 69, 1055–1063. 7. Byar, D. P., Herzberg, A. M. & Tan, W.-Y. (1993). Incomplete factorial designs for randomized clinical trials, Statistics in Medicine 12, 1629–1641. 8. Cox, D. R. (1958). Planning of Experiments. Wiley, New York. 9. Endophthalmitis Vitrectomy Study Group. Results of the Endophthalmitis Vitrectomy Study (1995). A randomized trial of immediate vitrectomy and of intravenous antibiotics for the treatment of postoperative bacterial endophthalmitis, Archives of Ophthalmology 113, 1479–1496. 10. Fisher, R. A. (1935). The Design of Experiments. Collier Macmillan, London. 11. Fisher, R. A. (1960). The Design of Experiments, 8th Ed. Hafner, New York. 12. Flather, M., Pipilis, A., Collins, R. et al. (1994). Randomized controlled trial of oral captopril, of oral isosorbide mononitrate and of intravenous magnesium sulphate started early in acute myocardial infarction: safety and haemodynamic effects, European Heart Journal 15, 608–619. 13. Green, A., Battistutta, D., Hart, V., Leslie, D., Marks, G., Williams, G., Gaffney, P., Parsons, P., Hirst, L., Frost, C. et al. (1994). The Nambour Skin Cancer and Actinic Eye Disease Prevention Trial: design and baseline characteristics of participants, Controlled Clinical Trials 15, 512–522. 14. Greenberg, E. R., Baron, J. A., Tosteson, T. D., Freeman, D. H., Jr, Beck, G. J., Bond, J. H., Colacchio, T. A., Coller, J. A., Frankl, H. D., Haile, R. W., Mandel, R. W., Nierenberg, J. S., Rothstein, D. W., Richard, S., Dale, C., Stevens, M. M., Summers, R. W. & vanStolk, R. U. (1994). A clinical trial of antioxidant vitamins to prevent colorectal adenoma. Polyp Prevention Study Group, New England Journal of Medicine 331, 141–147. 15. Heinonen, O. P., Virtamo, J., Albanes, D. et al. (1987). Beta carotene, alpha-tocopherol lung cancer intervention trial in Finland, in Proceedings of the XI Scientific Meeting of the International Epidemiologic Association, Helsinki, August, 1987. Pharmy, Helsinki.

8

FACTORIAL DESIGNS IN CLINICAL TRIALS

16. Hennekens, C. H. & Eberlein, K. (1985). A randomized trial of aspirin and beta-carotene among U.S. physicians, Preventive Medicine 14, 165–168.

cancer. Casodex Combination Study Group, Urology 45, 745–752. 27. Slud, E. V. (1994). Analysis of factorial survival experiments, Biometrics 50, 25–38.

17. ISIS-4 Collaborative Group (1995). ISIS-4: a randomized factorial trial assessing early captopril, oral mononitrate, and intravenous magnesium-sulphate in 58 050 patients with suspected acute myocardial infarction, Lancet 345, 669–685.

28. Snedecor, G. W. & Cochran, W. G. (1980). Statistical Methods, 7th Ed. The Iowa State University Press, Ames. 29. Sperduto, R. D., Hu, T. S., Milton, R. C., Zhao, J. L., Everett, D. F., Cheng, Q. F., Blot, W. J., Bing, L., Taylor, P. R., Li, J. Y. et al. (1993). The Linxian cataract studies. Two nutrition intervention trials, Archives of Ophthalmology 111, 1246–1253. 30. Stampfer, M. J., Buring, J. E., Willett, W. et al. (1985). The 2 × 2 factorial design: its application to a randomized trial of aspirin and carotene in U.S. physicians, Statistics in Medicine 4, 111–116. 31. Steering Committee of the Physicians’ Health Study Research Group (1989). Final report on the aspirin component of the ongoing physicians’ health study. New England Journal of Medicine 321, 129–135. 32. Thornquist, M. D., Owenn, G. S., Goodman, G. E. et al. (1993). Statistical design and monitoring of the carotene and retinol efficacy trial (CARET), Controlled Clinical Trials 14, 308–324.

18. Lacourciere, Y., Lefebvre, J., Poirier, L., Archambault, F. & Arnott, W. (1994). Treatment of ambulatory hypertensives with nebivolol or hydrochlorothiazide alone and in combination. A randomized double-blind, placebo-controlled, factorial-design trial, American Journal of Hypertension 7, 137–145. 19. Li, B., Taylor, P. R., Li, J. Y., Dawsey, S. M., Wang, W., Tangrea, J. A., Liu, B. Q., Ershow, A. G., Zheng, S. F., Fraumeni, J. F., Jr et al. (1993). Linxian nutrition intervention trials. Design, methods, participant characteristics, and compliance, Annals of Epidemiology 3, 577–585. 20. Lubsen, J. & Pocock, S. J. (1994). Factorial trials in cardiology (editorial), European Heart Journal 15, 585–588. 21. Mason, R. L. & Gunst, R. L. (1989). Statistical Design and Analysis of Experiments. Wiley, New York. 22. Max, M. B., Zeigler, D., Shoaf, S. E., Craig, E., Benjamin, J., Li, S. H., Buzzanell, C., Perez, M. & Ghosh, B. C. (1992). Effects of a single oral dose of desipramine on postoperative morphine analgesia, Journal of Pain & Symptom Management 7, 454–462. 23. Parkinson Study Group (1993). Effects of tocopherol and deprenyl on the progression of disability in early Parkinson’s disease, New England Journal of Medicine 328, 176–183. 24. Piantadosi, S. (1997). Factorial designs, in Clinical Trials: a Methodologic Perspective. Wiley, New York. See Chapter 15. 25. Pordy, R. C. (1994). Cilazapril plus hydrochlorothiazide: improved efficacy without reduced safety in mild to moderate hypertension. A double-blind placebo-controlled multicenter study of factorial design, Cardiology 85, 311–322. 26. Schellhammer, P., Shariff, R., Block, N., Soloway, M., Venner, P., Patterson, A. L., Sarosdy, M., Vogelzang, N., Jones, J. & Kiovenbag, G. (1995). A controlled trial of bicalutamide versus flutamide, each in combination with lutenizing hormone-releasing hormone analogue therapy, in patients with advanced prostate

33. Yates, F. (1935). Complex experiments (with discussion), Journal of the Royal Statistical Society, Series B 2, 181–247.

FAST TRACK ‘‘Fast Track’’ is a formal mechanism to interact with the U.S. Food and Drug Administration (FDA) using approaches that are available to all applicants for marketing claims. The Fast Track mechanism is described in the Food and Drug Administration Modernization Act of 1997 (FDAMA). The benefits of Fast Track include scheduled meetings to seek FDA input into development plans, the option of submitting a New Drug Application (NDA) in sections rather than all components simultaneously, and the option of requesting evaluation of studies using surrogate endpoints. The Fast Track designation is intended for the combination of a product and a claim that addresses an unmet medical need but is independent of the Priority Review and Accelerated Approval programs. An applicant may use any or all of the components of Fast Track without the formal designation. Fast Track designation does not necessarily lead to Priority Review or Accelerated Approval.

This article was modified from the website of the United States Food and Drug Administration (http://www.accessdata.fda.gov/scripts/cder/onctools /Accel.cfm#FastTrack) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

FDA DIVISION OF PHARMACOVIGILANCE AND EPIDEMIOLOGY (DPE)

Center with access to several large recordlinked databases. The reports produced by the Data Encryption Standard (DES) are integral to the ongoing risk assessment and the risk management performed by CDER review divisions of a product’s risk versus benefit profile. In addition, DPE epidemiologists are called on to meet with industry over important safety issues or to present their work before FDA advisory committees.

The Center for Drug Evaluation and Research’s (CDER) Division of Pharmacovigilance and Epidemiology (DPE) also carries out an epidemiologic function in the monitoring of drug safety. This function is performed by a multidisciplinary professional staff of physicians and Ph.D. epidemiologists, pharmacists, and program/project managers. The primary work is directed toward the evaluation and the risk assessment of drugs in the postmarketing environment using the tools of epidemiology. Epidemiologists integrate the medical/ clinical details of the underlying disease being treated with the influence of patient factors, concomitant diseases, and medications, as well as the clinical pharmacology of the specific product under study. DPE’s Epidemiology staff work closely with the Post-Marketing Safety Reviewers to provide clinical and epidemiologic case-series reviews of spontaneous adverse event reports submitted to the Food and Drug Administration (FDA). These data are used in a variety of ways to develop, refine, and investigate signals of clinical importance related to drug safety. As a complement, drug-use data are used frequently to estimate the size and to characterize the demographic composition of the population exposed to a given prescription product. Additionally, epidemiologists are involved in the design and the critique of Phase IV protocols for safety studies performed by industry and in the review of study findings. They also design, execute, and help to analyze data from epidemiologic studies performed through the mechanism of the DPE’s cooperative agreement program that provides the This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/handbook/epidemio.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

FDA MODERNIZATION ACT (FDAMA) OF 1997

drug and biological manufacturing changes, and to reduce the need for environmental assessment as part of a product application. The act also codifies FDA’s regulations and practice to increase patient access to experimental drugs and medical devices and to accelerate review of important new medications. In addition, the law provides for an expanded database on clinical trials that will be accessible by patients. With the sponsor’s consent, the results of such clinical trials will be included in the database. Under a separate provision, patients will receive advance notice when a manufacturer plans to discontinue a drug on which they depend for life support or sustenance, or for a treatment of a serious or debilitating disease or condition.

The Food and Drug Administration (FDA) Modernization Act (FDAMA), which was enacted November 21, 1997, amended the Federal Food, Drug, and Cosmetic Act relating to the regulation of food, drugs, devices, and biological products. With the passage of FDAMA, Congress enhanced FDA’s mission to recognize that the Agency would be operating in a twenty-first century characterized by increasing technological, trade, and public health complexities. 1

PRESCRIPTION DRUG USER FEES

The act reauthorizes, for five more years, the Prescription Drug User Fee Act of 1992 (PDUFA). In the past five years, the program has enabled the agency to reduce to 15 months the 30-month average time that used to be required for a drug review before PDUFA. This accomplishment was made possible by FDA managerial reforms and the addition of 696 employees to the agency’s drugs and biologics program, which was financed by $329 million in user fees from the pharmaceutical industry. 2

3 INFORMATION ON OFF-LABEL USE AND DRUG ECONOMICS The law abolishes the long-standing prohibition on dissemination by manufacturers of information about unapproved uses of drugs and medical devices. The act allows a firm to disseminate peer-reviewed journal articles about an off-label indication of its product, provided the company commits itself to file, within a specified time frame, a supplemental application based on appropriate research to establish the safety and effectiveness of the unapproved use. The act also allows drug companies to provide economic information about their products to formulary committees, managed care organizations, and similar large-scale buyers of health-care products. The provision is intended to provide such entities with dependable facts about the economic consequences of their procurement decisions. The law, however, does not permit the dissemination of economic information that could affect prescribing choices to individual medical practitioners.

FDA INITIATIVES AND PROGRAMS

The law enacts many FDA initiatives undertaken in recent years under Vice President Al Gore’s Reinventing Government program. The codified initiatives include measures to modernize the regulation of biological products by bringing them in harmony with the regulations for drugs and by eliminating the need for establishment license application, to eliminate the batch certification and monograph requirements for insulin and antibiotics, to streamline the approval processes for This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/opacom/backgrounders/mo dact.htm), (http://www.fda.gov/oc/fdama/default. htm) by Ralph D’Agostino and Sarah Karl.

4

PHARMACY COMPOUNDING

The act creates a special exemption to ensure continued availability of compounded drug

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

FDA MODERNIZATION ACT (FDAMA) OF 1997

products prepared by pharmacists to provide patients with individualized therapies not available commercially. The law, however, seeks to prevent manufacturing under the guise of compounding by establishing parameters within which the practice is appropriate and lawful. 5 RISK-BASED REGULATION OF MEDICAL DEVICES The act complements and builds on the FDA’s recent measures to focus its resources on medical devices that present the greatest risk to patients. For example, the law exempts from premarket notification class I devices that are not intended for a use that is of substantial importance to prevent impairment of human health, or that do not present a potential unreasonable risk of illness or injury. The law also directs FDA to focus its postmarket surveillance on high-risk devices, and it allows the agency to implement a reporting system that concentrates on a representative sample of user facilities—such as hospitals and nursing homes—that experience deaths and serious illnesses or injuries linked with the use of devices. Finally, the law expands an ongoing pilot program under which FDA accredits outside— so-called ‘‘third party’’— experts to conduct the initial review of all class I and low-tointermediate risk class II devices. The act, however, specifies that an accredited person may not review devices that are permanently implantable, life-supporting, life-sustaining, or for which clinical data are required. 6

FOOD SAFETY AND LABELING

The act eliminates the requirement of the FDA’s premarket approval for most packaging and other substances that come in contact with food and may migrate into it. Instead, the law establishes a process whereby the manufacturer can notify the agency about its intent to use certain food contact substances and, unless FDA objects within 120 days, may proceed with the marketing of the new product. Implementation of the notification process is contingent on additional appropriations to cover its cost to the agency. The

act also expands procedures under which the FDA can authorize health claims and nutrient content claims without reducing the statutory standard. 7 STANDARDS FOR MEDICAL PRODUCTS Although the act reduces or simplifies many regulatory obligations of manufacturers, it does not lower the standards by which medical products are introduced into the market place. In the area of drugs, the law codifies the agency’s current practice of allowing in certain circumstances one clinical investigation as the basis for product approval. The act, however, does preserve the presumption that, as a general rule, two adequate and well-controlled studies are needed to prove the product’s safety and effectiveness. In the area of medical devices, the act specifies that the FDA may keep out of the market products whose manufacturing processes are so deficient that they could present a serious health hazard. The law also gives the agency authority to take appropriate action if the technology of a device suggests that it is likely to be used for a potentially harmful unlabeled use.

FEDERAL FOOD, DRUG, AND COSMETIC ACT

shortcomings of the 1906 act and brought a new consciousness of consumer needs. After several unpopular attempts to revise the Pure Food and Drug Act during the administration of Franklin D. Roosevelt, public outcry over the ‘‘Elixir Sulfanilamide’’ disaster in 1937—a mass poisoning incident in which a popularly marketed drug killed over 100 people—led to the Food, Drug, and Cosmetic Act of June 25, 1938.

The 1906 Pure Food and Drug Act (the ‘‘Wiley Act’’) prohibited the manufacture, sale, and interstate shipment of ‘‘adulterated’’ and ‘‘misbranded’’ foods and drugs. Product labels were required to make a truthful disclosure of contents but were not required to state weights or measures. By 1913, food manufacturers had grown alarmed by growing variety of state-level weight and measure laws and sought uniformity at the federal level through the Gould Amendment, which required net contents to be declared, with tolerances for reasonable variations. Under the Wiley Act, the U.S. federal government’s Bureau of Chemistry—which in 1927 became the Food, Drug, and Insecticide Administration, then in 1931, the Food and Drug Administration—could challenge illegal products in court but lacked the affirmative requirements to guide compliance. Food adulteration continued to flourish because judges could find no specific authority in the law for the standards of purity and content that the Food and Drug Administration (FDA) had set up. Such products as ‘‘fruit’’ jams made with water, glucose, grass seed, and artificial color were undercutting the market for honest products. False therapeutic claims for patent medicines also escaped control in 1912 when Congress enacted an amendment that outlawed such claims but required the government to prove them fraudulent; to escape prosecution, defendants had only to show that they personally believed in the fake remedy, a major weakness in the law for 26 years. The 1906 law became obsolete because technological changes were revolutionizing the production and marketing of foods, drugs, and related products. In addition, economic hardships of the 1930s magnified the many

1

THE PREVENTIVE AMENDMENTS

The 1938 Food, Drug, and Cosmetic Act in conjunction with World War II greatly expanded the FDA’s workload. Wartime demands had stimulated the development of new ‘‘wonder drugs,’’ especially the antibiotics, which were made subject to FDA testing, beginning with penicillin in 1945. Although there was now a law requiring premarket clearance of new drugs, consumers continued to be guinea pigs for a host of new chemical compounds of unknown safety. The law prohibited poisonous substances but did not require proof that food ingredients were safe. It also provided exemptions and safe tolerances for unavoidable or necessary poisons such as pesticides. When the FDA attempted to set a pesticide tolerance, an adverse court decision showed that the lengthy procedure required by law was unworkable. The FDA could stop the use of known poisons and did so in numerous cases, but the vast research efforts needed to ensure that all food chemicals were safe were clearly beyond government resources. Thus, three amendments fundamentally changed the character of the U.S. food and drug law: the Pesticide Amendment (1954), the Food Additives Amendment (1958), and the Color Additive Amendments (1960). These laws provide that no substance can legally be introduced into the U.S. food supply unless there has been a prior determination that it is safe, and the manufacturers themselves are required to prove a product’s safety. Also very significant was the proviso in the food and color additive laws that no additive could be deemed safe (or

This article was modified from the website of the United States Food and Drug Administration (http://www.cfsan.fda.gov/∼lrd/histor1a.html) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

FEDERAL FOOD, DRUG, AND COSMETIC ACT

given FDA approval) if it was found to cause cancer in humans or experimental animals. Known as the ‘‘Delaney Clause,’’ this section was initially opposed by the FDA and by scientists, who agreed that an additive used at very low levels need not necessarily be banned only because it might cause cancer at high levels. However, its proponents justified the clause on the basis that cancer experts have not been able to determine a safe level for any carcinogen. This was the underlying basis for the 1959 nationwide FDA recall of cranberries contaminated by the weed killer aminotriazole, which was beneficial in convincing farmers that pesticides must be used with care. Preventing violations through premarket clearance has given consumers immeasurably better protection than merely prosecuting the few violations that could be proved by investigating after injuries were reported.

FEDERAL REGISTER

of the nature of the comments received. Each time a proposal is substantively revised or amended, a notice is published in the Federal Register.

The Federal Register is one of the most important sources for information on the activities of the U.S. Food and Drug Administration (FDA) and other government agencies. Published daily, Monday through Friday, the Federal Register carries all proposed and finalized regulations and many significant legal notices issued by the various agencies as well as presidential proclamations and executive orders. Subscriptions to the Federal Register can be purchased from the federal government’s Superintendent of Documents. As an alternative, copies can usually be found in local libraries, county courthouses, federal buildings, or on the Internet. 1

3

Ultimately, a ‘‘Final Rule’’ is published, which specifies the date when the new regulatory requirements or regulations become effective. 4

REGULATORY AGENDA

Twice a year (April and October), the entire Department of Health and Human Services, including the FDA, publishes an agenda in the Federal Register that summarizes policysignificant regulations, regulations that are likely to have a significant economic impact on small entities, and other actions under development. Each item listed includes the name, address, and telephone number of the official to contact for more information.

ADVANCE NOTICE

Often, the FDA publishes ‘‘Notices of Intent’’ in the Federal Register to give interested parties the earliest possible opportunity to participate in its decisions. These are notices that the FDA is considering an issue and that outside views are welcome before a formal proposal is made. 2

FINAL REGULATIONS

5

MEETINGS AND HEARINGS

Notices are published in the Federal Register announcing all meetings of the FDA’s advisory committees and all public meetings that provide an information exchange between FDA and industry, health professionals, consumers, and the scientific and medical communities. The notice contains the date, time, and place of the meeting as well as its agenda. The Federal Register also announces administrative hearings before the FDA and public hearings to gain citizen input into Agency activities (see ‘‘Citizen Petition’’).

PROPOSED REGULATIONS

When a formal proposal is developed, the FDA publishes a ‘‘Notice of Proposed Rulemaking’’ in the Federal Register. The notice provides the timeframe in which written comments about the proposed action can be submitted. A written request also can be submitted that FDA officials extend the comment period. If FDA extends the period, a notice of the extension is published in the Federal Register. Occasionally, a second or third proposal is published in the Federal Register because This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/ora/fed state/Small business /sb guide/fedreg.html) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

FILEABLE NEW DRUG APPLICATION (NDA) After a New Drug Application (NDA) is received by the U.S. Food and Drug Administration’s Center for Drug Evaluation and Research (CDER), it undergoes a technical screening, generally referred to as a completeness review. This evaluation ensures that sufficient data and information have been submitted in each area to justify ‘‘filing’’ the application—that is, to justify initiating the formal review of the NDA.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/handbook/fileable.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

FINANCIAL DISCLOSURE

The financial arrangements that must be disclosed include the following:

The U.S. Food and Drug Administration (FDA) reviews data generated in clinical studies to determine whether medical device applications are approvable. Financial interest of a clinical investigator is one potential source of bias in the outcome of a clinical study. To ensure the reliability of the data, the financial interests and arrangements of clinical investigators must be disclosed to the FDA. This requirement applies to any clinical study submitted in a marketing application that the applicant or the FDA relies on to establish that the product is effective or is used to show equivalence to an effective product, and to any study in which a single investigator makes a significant contribution to the demonstration of safety. The requirement does not apply to studies conducted under the emergency use, compassionate use, or treatment use provisions. Financial compensation or interests information is used in conjunction with information about the design and purpose of the study as well as information obtained through on-site inspections in the agency’s assessment of the reliability of the data. As of February 1999, anyone who submits a Premarket Approval (PMA) or Premarket Notification 510(k) that contains a covered clinical study must submit certain information concerning the compensation to and financial interests of any clinical investigator conducting clinical studies covered in the application. Applicants must certify the absence of certain financial interests of clinical investigators on Financial Interest Form (Certification: Financial Interests and Arrangements of Clinical Investigations, FDA Form 3454) or disclose those financial interests on Financial Interest Form (Disclosure: Financial Interests and Arrangements of Clinical Investigators, FDA Form 3455).

• Compensation made to the investigator

in which the value of the compensation could be affected by the study outcome. • Significant payments to the investigator or institution with a monetary value of $25,000 or more (e.g., grants, equipment, retainers for ongoing consultation, or honoraria) over the cost of conducting the trial. Any such payments to the investigator or institution during the time the investigator is conducting the study and for 1 year after study completion must be reported. • Proprietary interest in the device, such as a patent, trademark, copyright, or licensing agreement. • Significant equity interest in the sponsor such as ownership, interest, or stock options. All such interests whose value cannot be readily determined through reference to public prices must be reported. If the sponsor is a publicly traded company, any equity interest whose value is greater than $50,000 must be reported. Any such interests held by the investigator while the investigator was conducting the study and for 1 year after study completion must be reported. This requirement applies to investigators and subinvestigators, including their spouses and dependent children, but does not apply to full-time or part-time employees of the sponsor or to hospital or office staff. (For studies completed before February 1999, the requirements are reduced. That is, the sponsor does not need to report equity interest in a publicly held company or significant payments of other sorts. Other reporting still applies.) Sponsors are responsible for collecting financial information from investigators, and clinical investigators are responsible for providing financial disclosure information to the sponsor. The investigator’s agreement with the sponsor should require the investigator to provide the sponsor with accurate financial disclosure information. Certification or

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cdrh/devadvice/ide/financial. shtml) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

FINANCIAL DISCLOSURE

disclosure information should not be included in the Investigational Device Exemption (IDE) application. If the FDA determines that the financial interests of any clinical investigator raise a serious question about the integrity of the data, the FDA will take any action it deems necessary to ensure the reliability of the data, including: • Initiating agency audits of the data de-

rived from the clinical investigator in question. • Requesting that the applicant submit additional analyses of data (e.g., to evaluate the effect of the clinical investigator’s data on the overall study outcome). • Requesting that the applicant conduct additional independent studies to confirm the results of the questioned study. • Refusing to use the data from the covered clinical study as the primary basis for an agency action, such as PMA approval or 510(k) clearance.

FISHER’S EXACT TEST

obtained using the standard χ 2 test. But we cannot trust the accuracy of this approximation when it is based on observations on so few patients. Fisher’s exact test provides a way around this difficulty. The reasoning behind the test is as follows. Suppose that the treatment was totally ineffectual, and that each patient’s recovery over the subsequent five days was unaffected by whether the treatment were applied or not. Precisely three patients recovered. If the treatment was ineffectual, then these three, and only these three, individuals would have recovered regardless of whether they were assigned to the treatment or control group. The fact that all three did indeed appear in the treatment group would then have been just a coincidence whose probability could be calculated as follows. When four out of the eight subjects were randomly chosen for the treatment group, the chance that all three of those destined to recover should end up in the treatment group is given by the hypergeometric distribution as (3 C3 )(5 C1 ) = 0.071. 8 C4

RICK ROUTLEDGE Simon Fraser University Vancouver, British Columbia, Canada

Fisher’s exact test can be used to assess the significance of a difference between the proportions in two groups. The test was first described in independently written articles by Irwin (14) and Yates (25). Yates used the test primarily to assess the accuracy of his correction factor to the χ 2 test, and attributed the key distributional result underlying the exact test to R.A. Fisher. Fisher successfully promoted the test in 1935, presenting two applications, one to an observational study on criminal behavior patterns (8), and another to an artificial example of a controlled experiment on taste discrimination (9). Typical recent applications are to the results of simple experiments comparing a treatment with a control. The design must be completely randomized, and each experimental unit must yield one of two possible outcomes (like success or failure). Consider, for example, the study reported by Hall et al. (13). This was a randomized, double-blind, placebo-controlled study on the effect of ribavirin aerosol therapy on a viral infection (RSV) of the lower respiratory tract of infants. After five days of treatment, each infant was examined for the continued presence of viral shedding in nasal secretions. There were 26 patients in the randomized trial. For illustrative purposes, the following discussion focuses on hypothetical results from a smaller set of only eight patients. Also, a patient showing no signs of viral shedding in nasal secretions will be said to have recovered. Consider, then, the ‘‘results’’ displayed in Table 1. All three recoveries were in the treatment group. For a frequency table based on only four treatments and four control subjects, the evidence could hardly be more convincing, but is it statistically significant? Had the experiment included more patients, an approximate P value could have been

This is the standard P value for Fisher’s exact test of the null hypothesis of no treatment effect against the one-sided alternative that the treatment has a positive benefit. Consider the more general setting, as portrayed in Table 2. The P value for testing the null hypothesis that the treatment has no impact vs. the onesided alternative that it has a positive value is min(n,S)
(S Cy )(F Cn−y ) . (1) P= N Cn y=a For a two-sided alternative there is no universally accepted definition. The two most common approaches are (i) to double the onesided P value, or (ii) to extend the above sum over the other tail of the distribution, including all those terms which are less than or equal to the probability for the observed

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

FISHER’S EXACT TEST

Table 1. Results From a Small, Comparative Experiment

Treatment Control Totals

Recovered

Not recovered

Totals

3 0 3

1 4 5

4 4 8

Table 2. Notation for a 2 × 2 Frequency Table of Outcomes From a Comparative Experiment

Treatment Control Totals

Recovered

Not recovered

Totals

a c S

b d F

n m N

table. The latter strategy is deployed by the major statistical packages, BMDP (Two-Way Tables in (5)), JMP (Contingency Table Analysis in Fit Y by X in (21)), SAS (FREQ procedure in (20)), S-PLUS (function, fisher.test in (17)), SPSS (Crosstabs in (23)), StatXact (6), and Systat (Tables in (24)). Gibbons & Pratt (12) discuss possible alternatives. The test can be extended to an r × c contingency table, as proposed by Freeman & Halton (11). It is also used on r × 2 tables for multiple comparisons, with the usual controversy over adjustments for simultaneous inferences on a single data set (see (22) and references therein). 1

APPLICABILITY AND POWER

A major advantage of Fisher’s exact test is that it can be justified solely through the randomization in the experiment. The user need not assume that all patients in each group have the same recovery probability, nor that patient recoveries occur independently. The patients could, for example, go to one of four clinics, with two patients per clinic. Two patients attending the same clinic might well have experienced delayed recovery from having contacted the same subsidiary infection in their clinic, but the above argument would still be valid as long as individuals were randomly selected without restriction

from the group of eight for assignment to the treatment vs. control groups. If, however, the randomization was applied at the clinic level, with two of the four clinics selected for assignment to the treatment group, then the test would be invalid. Compared with the hypergeometric distribution, the data would most likely be overdispersed. Similarly, if the randomization was restricted by blocking with respect to clinic, the pair of individuals from each clinic being randomly split between the treatment and control groups, then the test would again be invalid. These alternative designs certainly have their place, particularly in larger experiments with more subjects, but the results would have to be analyzed with another test. The example also illustrates a major weakness of Fisher’s exact test. The evidence for a table based on only four subjects in each of two groups could hardly have been more favorable to the alternative. Yet the P value still exceeds 5%, and most observers would rate the evidence as not statistically significant. It is in general difficult to obtain a statistically significant P value with Fisher’s exact test, and the test therefore has low power. The most effective way to increase the power may well be to take quantitative measurements. Suppose, for instance, that all four patients who received the treatment showed reduced nasal shedding of the virus. By quantifying this evidence, and subjecting the quantitative measurements to a test of significance, the experimenter could, in many instances, generate a more powerful test. One could also, of course, consider running the study on a larger group of patients. 2 COMPETING BINOMIAL-MODEL TEST It is also possible to obtain greater power by analyzing the above table with another statistical model. The most commonly used competitor involves assuming that the numbers of recovered patients in each group are independently binomially distributed. The test was mentioned by Irwin (14), and promoted by Barnard (2). Although he soon withdrew his support (3), it has since become a popular alternative. Its increased power has been

FISHER’S EXACT TEST

amply demonstrated by D’Agostino et al. (7) and others. For the above table, the P value is 0.035 vs. the 0.071 for Fisher’s exact test. The P value based on this binomial model is typically smaller than the one generated by Fisher’s exact test. The main reason for the difference is that the standard definition of the P value contains the probability of the observed table, and this probability is higher for Fisher’s exact test than for the binomial model 1,10,18. Thus the null hypothesis is more frequently rejected, and the binomialmodel test is more powerful. This test is available in StatXact (6). However, the increased power comes at a cost. To justify the binomial model, one must either assume that all patients within each group have the same recovery probability, or envisage that the patients were randomly sampled from some larger group. The trial must also have been conducted so as to ensure that patient deaths occur independently. They cannot, for example, attend four clinics, with two patients per clinic. There is another, more subtle problem with the binomial model. Simple calculations show that had fewer than three or more than five patients recovered, then neither P value could possibly have been significant. This puts the researcher in an awkward quandary. For example, had only two patients recovered after 5 days, the researcher would have had an incentive either to present the results after more than five days of treatment when at least one more patient had recovered, or to incorporate more patients into the experiment. One does not win accolades for announcing the results of experiments that are not only statistically insignificant, but also apparently barely capable of ever producing significant results. These are important complications when it comes to interpreting the results of these sorts of small experiments. Suppose, for example, that in the above experiment the researcher was to have adjusted the five-day reporting time, if necessary, so as to guarantee between three and five recoveries. Then the binomial P value would be invalid. The probability of obtaining a table at least as favorable to the treatment as the above one can be shown to be 0.056, not 0.035, as generated by the standard binomial model.

3

3

THE MID-P VALUE

The P value of 0.071 generated by Fisher’s exact test is still large compared with the 0.056 figure produced by this modified binomial model. There is yet another alternative with important theoretical and practical advantages (see, for example, 16,4,1, and (19)). This is the mid-P value, first introduced in 1949 by Lancaster (15). In place of the standard definition, P value = Pr (evidence at least as favorable to Ha as observed |H0 ), they propose the alternative, mid-P value = Pr(evidence more favorable to Ha as observed |H0 ) + 12 Pr(evidence equally favorable to Ha as observed |H0 ). Table 3 summarizes the possible P values for the above example. This table illustrates that the mid-P has the potential to provide a smaller, more significant-looking P value, and to reduce the discrepancy between P values generated by competing models. However, by using a smaller P value, one may reject a valid null hypothesis too frequently. Fortunately, amongst other desirable attributes of the mid-P, its routine use does indeed control a quantity closely related to the type I error rate (see (19), and references therein). The computer package, StatXact (6) facilitates the calculation of the mid-P by providing the probability of the observed table along with the standard P value.

Table 3. Comparison of P Values for the Data in Table (1) Modified Fisher’s Binomial binomial exact test model model Standard P value Mid-P value

7.1% 3.6%

3.5% 2.0%

5.6% 3.0%

4

4

FISHER’S EXACT TEST

CONCLUSION

Fisher’s exact test provides a widely applicable way to assess the results of simple randomized experiments leading to 2 × 2 contingency tables. But it has low power, especially when the standard P value is used. The power can be increased considerably through (i) using the mid-P value, or (ii) carefully constructing a test based at least in part on a binomial model. Further power increases can be generated through (iii) taking quantitative measurements on each subject, or (iv) running the trial with a larger number of patients. REFERENCES 1. Agresti, A. (1990). Categorical Data Analysis. Wiley–Interscience, New York. 2. Barnard, G. A. (1945). A new test for 2 × 2 tables, Nature 156, 177. 3. Barnard, G. A. (1949). Statistical inference, Journal of the Royal Statistical Society, Series B 11, 115–139. 4. Barnard, G. A. (1989). On alleged gains in power from lower p-values, Statistics in Medicine 8, 1469–1477. 5. BMDP Statistical Software, Inc. (1990). BMDP Statistical Software Manual: To Accompany the 1990 Software Release. University of California Press, Berkeley. 6. Cytel Software Corporation (1995). StatXact-3 for Windows. Cytel Software Corporation, Cambridge, Mass. 7. D’Agostino, R. B., Chase, W. & Belanger, A. (1988). The appropriateness of some common procedures for testing the equality of two independent binomial populations, American Statistician 42, 198–202. 8. Fisher, R. A. (1935). The logic of inductive inference, Journal of the Royal Statistical Society, Series A 98, 39–84. 9. Fisher, R. A. (1935). The Design of Experiments. Oliver & Boyd, Edinburgh. 10. Franck, W. E. (1986). P-values for discrete test statistics, Biometrical Journal 4, 403–406. 11. Freeman, G. H. & Halton, J. H. (1951). Note on an exact treatment of contingency, goodness of fit and other problems of significance, Biometrika 38, 141–149. 12. Gibbons, J. D. & Pratt, J. W. (1975). Pvalues: interpretation and methodology, American Statistician 29, 20–25.

13. Hall, C. B., McBride, J. T., Gala, C. L., Hildreth, S. W. & Schnabel, K. C. (1985). Ribavirin treatment of respiratory syncytial viral infection in infants with underlying cardiopulmonary disease, Journal of the American Medical Association 254, 3047–3051. 14. Irwin, J. O. (1935). Tests of significance for differences between percentages based on small numbers, Metron 12, 83–94. 15. Lancaster, H. O. (1949). The combination of probabilities arising from data in discrete distributions, Biometrika 36, 370–382. 16. Lancaster, H. O. (1961). Significance tests in discrete distributions, Journal of the American Statistical Association 56, 223–234. 17. MathSoft, Inc. (1993). S-PLUS Reference Manual, Version 3.2. MathSoft, Inc., Seattle. 18. Routledge, R. D. (1992). Resolving the conflict over Fisher’s exact test, Canadian Journal of Statistics 20, 201–209. 19. Routledge, R. D. (1994). Practicing safe statistics with the mid-p, Canadian Journal of Statistics 22, 103–110. 20. SAS Institute, Inc. (1989). SAS/STAT User’s Guide, Version 6, 4th Ed., Vol. 1. SAS Institute Inc., Cary. 21. SAS Institute, Inc. (1995). JMP Statistics and Graphics Guide, Version 3.1. SAS Institute Inc., Cary. 22. Savitz, D. A. & Olshan, A. F. (1995). Multiple comparisons and related issues in the interpretation of epidemiological data, American Journal of Epidemiology 142, 904–908. 23. SPSS, Inc. (1991). SPSS Statistical Algorithms, 2nd Ed. SPSS Inc., Chicago. 24. SYSTAT, Inc. (1992). SYSTAT for Windows: Statistics, Version 5. SYSTAT, Inc., Evanston. 25. Yates, F. (1934). Contingency tables involving small numbers and the χ 2 test, Journal of the Royal Statistical Society, Supplement 1, 217–235.

FLEXIBLE DESIGNS

the adaptation rules need not be specified in advance. Different ways have been used to define flexible multi-stage designs with adaptive interim analyses (2–5). In order to control the overall type I error probability, they all adhere to a common invariance principle: Separate standardized test statistics are calculated from the samples at the different stages and aggregated in a predefined way to test statistics that are used for the test decisions. Under the null hypothesis, the distributions of these separate test statistics are known, for example, stage-wise Pvalues follow stochastically independent uniform distributions on [0, 1], or stage-wise Z-scores follow independent standard normal distributions. Assume that, given no design modifications are permitted, the test procedure applied to the aggregated test statistics controls the level α. Then, every design modification that preserves the distributional properties of the separate stage-wise test statistics does not inflate the level α of the test procedure (6). The method of Proschan and Hunsberger (7) based on the definition of the conditional error function can also be defined in terms of a test combining standardized stage-wise test statistics according to a prefixed rule (8–10). The self-designing method of Fisher (11, 12) for two stages also fits into this concept (8). However, it does not allow for an early rejection of the null hypothesis in the interim analysis. The method allows for multiple stages with design modifications (but no test decisions in the interim analyses), and it is up to the experimenter to decide if the trial is completed with a test decision after the next stage. Some additional flexibility exists because the experimenter can choose how future stages will be aggregated to the final test statistics. The weight of the last stage, however, is determined by the weights of the previous stages. A similar approach, that also allows for rejection of the null hypothesis in the interim analyses, has been proposed by Hartung and Knapp (13) and is based on the sum of χ 2 distributed test statistics. ¨ ¨ Muller and Schafer (14) use the notion of the conditional error function to extend the flexibility to the adaptive choice of the number

MARTIN POSCH PETER BAUER WERNER BRANNATH Medical University of Vienna Vienna, Austria

1

INTRODUCTION

Classical frequentist statistical inference is based on the assumption that the inferential procedure is completely specified in advance. Consequently, the consensus guideline International Conference on Harmonization (ICH) E9 on Statistical Principles for Clinical Trials (1) requires for confirmatory trials that hypotheses and the statistical analysis plan is laid down in advance. For the planning of optimal trial designs knowledge on quantities as the expected efficacy of a new treatment, the safety properties, the appropriate doses or application forms of a treatment, the success rate in the control group, and the variability of outcome parameters are essential. Typically, in the planning phase of a clinical trial, many of these quantities are unknown. However, relevant information may aggregate in the course of the trial. Based on this information, changes in the trial design can become desirable or even inevitable. The necessity to allow for cautious adaptations is also recognized in the ICH E9 document: If it becomes necessary to make changes to the trial, any consequent changes to the statistical procedures should be specified in an amendment to the protocol at the earliest opportunity, especially discussing the impact on any analysis and inferences that such changes may cause. The procedure selected should always ensure that the overall probability of type I error is controlled (1).

Statistical inference based on adaptive designs allows implementation of design adaptations without inflating the type I error. The crucial point is that the adaptations may be based on the unblinded data collected so far as well as external information and

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

FLEXIBLE DESIGNS

of interim analyses. The invariance principle behind their approach can be described in simple terms: After every stage, the remainder of the design can be replaced by a design that, given what has been observed up to now, would not result in a higher conditional type I error probability than the preplanned design. Or, in other words, design modifications at any time that preserve the conditional error probability of the original design do not compromise the overall type I error probability. This principle can be defined concisely also in terms of the recursive application of simple two-stage combination tests. This allows the construction of an overall P-value and confidence intervals (15). By this generalization in the interim analysis, an experimenter may decide to insert a further interim analysis for saving time (if he sees good chances to get an early rejection or if information from outside asks for quick decisions). Adaptive designs can also be applied to time-to-event data (16) when information from the first-stage sample is used also in the second-stage test statistics (e.g., exploiting the independent increment structure of the log-rank statistics). However, restrictions exist on the type of information from the first stage, which may be utilized for adaptations (17). Up to now, sample size reassessment (4, 5, 7, 10, 18–23) has been an issue of major interest. However, various other design modifications like changing doses (24), dropping treatment arms (25–27), redesigning multiple endpoints (28, 29), changing the test statistics (30–32), and selecting goals between non-inferiority and superiority (33, 34) have been considered.

is based on a suitably defined combination function C(p1 , p2 ), which is assumed to be left continuous, monotonically increasing in both arguments, and strictly increasing in at least one: If C(p1 , p2 ) ≤ c, one rejects the hypothesis, otherwise one accepts. Note that for α 0 = 1, no stopping for futility is applied. If, in addition, α 1 = 0, no early test decision will be taken and the interim analysis is only performed for adaptation purposes. The adaptive testing procedure is summarized in Fig. 1. If, under H 0 , the P-values are independently and uniformly distributed on [0, 1], then the level condition to determine c, α 1 , and α 0 can be written as
α1 +

α0 α1




1

1[C(x,y)≤c] dy dx = α

Here, the indicator function 1[·] equals 1 if C(x, y) ≤ c and 0 otherwise. One of the combination functions considered in the literature plays a special role because of its relationship to group sequential tests (see Group Sequential Designs). Assume that the one-sided null hypothesis H0 :µA = µB versus H 1 :µA > µB is tested for comparing the means of two normal distributions with known variance (w.l.o.g. σ 2 = 1). The

First Stage p1 a1

0

a0

Reject H01

2

(1)

0

1 Accept H01

THE GENERAL FRAMEWORK Adaptation

To start fairly general, let us assume that a one-sided null hypothesis H0 is planned to be tested in a two-stage design. The test decisions are performed by using the P-values p1 and p2 calculated from the samples at the separate stages. Early decision boundaries are defined for p1 : If p1 ≤ α 1 (where α 1 < α), one stops after the interim analysis with an early rejection; if p1 > α 0 (where α 0 > α), one stops with an acceptance of H0 (stopping for futility). In case of proceeding to the second stage, the decision in the final analysis

Second Stage C (p1,p2) 0

1

c

Reject H01

Accept H01

Figure 1. The adaptive testing procedure.

FLEXIBLE DESIGNS

weighted inverse normal combination function (4, 5) can be defined as C(p1 , p2 ) = 1 − [w1 zp1 + w2 zp2 ], 0 < wi < 1, w21 + w22 = 1

(2)

where zγ is the (1 − γ ) quantile of the standard normal distribution. For the group sequential test of the normal mean with samples balanced over treatments n1 = n1A = n1B and n 2 = n2A = n2B at the  two stages, and w1 = n1 /(n1 + n2 ), w2 = n2 /(n1 + n2 ). The term in the squared brackets of Equation (2) is simply the standardized difference of the treatment means calculated  n /(n from the total sample, Z = Z 1 1 1 + n2 ) +  Z2 n2 /(n1 + n2 ). Here, Z1 and Z2 are the standardized mean treatment differences calculated from the separate stages. Note that C(p1 , p2 ) as defined in Equation (2) is just the P-value for the test based on the total sample. Hence, if no adaptation is performed, the test decision is the same as in the classical group sequential test with an early rejection boundary zα1 . Given zα1 and an early stopping for futility boundary zα0 , the critical boundary zα2 for the test statistic Z in the final analysis can be derived from Equation (1), which then is equivalent to the common level condition Prob(Z1 ≥ zα1 ) + Prob(zα0 ≤ Z1 < zα1 , Z ≥ zα2 ) = α (3) for the corresponding group sequential test with obligatory stopping for futility. It is obvious how this analogy works for more than two stages. The conditional error function A(p1 ) = Prob(reject H 0 |p1 ) in the group sequential context leads to the so-called linear conditional error function (7) ⎧ 0 ⎪ ⎪ ⎪ ⎨1 − [(z √n + n α2 1 2 A(z1 ) = √ √ ⎪ −z n )/ n ] 1 1 2 ⎪ ⎪ ⎩ 1

if z1 < zα0 if zα0 ≤ z1 ≤ zα1 if z1 ≤ zα1 (4)

In the following, n˜ 2 denotes the adapted second-stage sample size, which may be

3

different to n2 planned a priori. Let further Z˜ 2 denote the standardized mean of the actual second-stage sample and  Z˜ = Z1 n1 /(n1 + n2 ) + Z˜ 2 n2 /(n1 + n2 ) the adaptive test statistics based on the preassigned weights tocombine the two stages. Now, setting wi = ni /(n1 + n2 ), i = 1, 2 then Z˜ ≥ zα2 is equivalent to C(p1 , p2 ) < α 2 , where C(p1 , p2 ) is defined in Equation (2) and p1 , p2 are the P-values of the first and the possibly adapted second stage. Note that Z˜ ≥ zα2 is also equivalent to Z˜ 2 ≥ zA(z1 ) , so that formally the test in the second-stage sample is performed at the level A(z1 ). Hence, when proceeding to the second stage, the choice of n˜ 2 can simply be based on sample size formulas for the fixed sample size case using the adjusted level A(z1 ). Some comments have to be made here. (1) The crucial property of these flexible designs is that the adaptation rule needs not to be specified a priori. (2) An alternative approach is to start with an a priori specified sample size reassessment rule n˜ 2 (p1 ) and to use the classical test statistics for the final test (19, 20). To control the type I error, either an adjusted critical boundary or constraints for the sample sizes have to be applied. As this procedure weights the stage-wise test statistics always according to the actual sample sizes, it can be expected to be more efficient than when fixed weights are used. Unless extreme sample sizes are used, this difference, however, is likely to be small (compare Reference 4). Note that with a predefined sample size reassessment rule, one can also define a combination test that uses the classical test statistics as long as one does not deviate from this prespecified rule. The corresponding combination function is identical to Equation (2), but in the weights wi the preplanned second-stage sample size n2 is replaced by n˜ 2 (p1 ). Using this combination function, one can also here deviate from the prespecified sample size reassessment rule. However, a deviation from this rule implies that the classical test statistics is no longer used. Hence, designs with prespecified mandatory adaptation rules can be looked at as a special case of a flexible design. (3) Clearly, combination functions can also be used for tests in a distributional environment completely different to the normal.

4

FLEXIBLE DESIGNS

As long as the stage-wise P-values are independent and uniformly distributed under the global null hypothesis, the methodology will also apply. By transforming the resulting Pvalues via the inverse normal combination method, one formally arrives at independent standard normal increments so that all the results known for group sequential trials under the normal distribution can also be utilized for completely different testing problems (4). (4) The assumptions can even be relaxed, only requiring that the stage-wise P-values follow a distribution that, given the results of the previous stages, is stochastically larger or equal to the uniform distribution (15). A very general formulation of adaptive designs is given in Reference 35. (5) Classical group sequential tests (see Group Sequential Designs) are a special case of the more general concept of adaptive combination tests, because they result from a special combination function for the aggregation of the stage-wise test statistics. (6) Hence, classical group sequential trials can be planned in the context of adaptive multistage designs. Moreover, if the trial has in fact been performed according to the preplanned schedule and no adaptations have been performed, no price at all has to be paid for the option to deviate from the preplanned design: then the classical test statistic and rejection boundaries can be used. However, if adaptations are performed because of the fixed weights, the classical test statistics is no longer used in the final analysis. (7) Estimation faces the problems of sequential designs and the lack of a complete specification of the future sample space because of the flexibility. Several proposals for the construction of point estimates and confidence intervals have been made (4, 15, 34, 36–40). The crucial question is how to use the wide field of flexibility opened by this general concept in practice. Note that, in principle, at every interim analysis, a ‘‘new’’ trial at a significance level equal to the conditional error probability can be planned. This conditional error probability accounts for the past and assures that the overall type I error probability for the future is always controlled. Furthermore, by adopting the concept of (formally) performing interim looks without early test decisions after every sample unit,

this concept of the conditional error function can also be applied for mid-trial design modifications in trials without any preplanned interim analyses (41). Clearly, because of the large number and diversity of possible adaptations, their merits, for example, for the practice of clinical trials, are difficult to establish. In the following, some types of adaptations are discussed. 3 CONDITIONAL POWER AND SAMPLE SIZE REASSESSMENT The conventional measure to assess the performance of tests in a particular design is the overall power: In the long run, the experimenter performing equally powered studies gets rejections with probability 1 − β under the alternative. In case of a sequential design with early decision, however, unblinded information on the observed effect is available. So when proceeding to the next stage, one is tempted to have a look at the chances to get rejection in the end, given the results up to now. The overall power takes expectation over all possible outcomes in the interim analysis. Now, when being halfway through the trial and having preliminary estimates, should the experimenter in a concrete situation average his perspectives over outcomes for which he definitely knows that they have not been observed in the current trial? Or should he argue based on what he already knows? The conditional power (given Z1 ) for the comparison of two normal means by the weighted inverse normal combination function in Equation (2) is given by (σ 2 = 1)  √ √ CP(z1 ) = 1 −  (zα2 n1 + n2 − z1 n1 )/ √  √  n˜ 2 n2 − √ 2

(5)

where  = µA − µB is the relevant difference on which the power is targeted (7, 10, 18, 21–23, 42). In the interim analysis, the second-stage sample size n˜ 2 to achieve a rejection probability of 1 − β in the final analysis is determined by solving the equation CP(z1 ) = 1 − β for n˜ 2 . The question of which value should be plugged in for the targeted effect  develops.

FLEXIBLE DESIGNS

3.1 Using the Estimated Effect It has been proposed to calculate the conditional power by replacing  by its firstˆ (12). Figure 2 shows this stage estimate  ‘‘predictive’’ conditional power for two group sequential tests (here, for comparability, α = 0.025, one-sided, 1 − β = 0.8 is chosen) with equal sample sizes at the two stages balanced over treatments as a function of z1 . Clearly, for small observed effects, the predictive conditional power does not promise a good chance to reject at the end; for large effects only slightly smaller than the rejection boundary zα1 , however, the predictive conditional power exceeds 0.8, particularly in the O’Brien-Fleming (43) design. The predictive conditional power is a random variable and Fig. 3 shows its distribution function given that the trial proceeds to the second stage. Under the alternative, the conditional probability to end up with a predictive conditional power below 0.5 (given the event that the trial continues) is 0.54 for the Pocock design (44) and 0.35 for the O’Brien-Fleming design. Under the null hypothesis, the predictive conditional power will remain below 0.2 in more than 80% of the cases for both designs, which explains the findings that sample size reassessment based on the ‘‘predictive’’ conditional power by using the estimate of the effect size will, in general, lead to large expected sample sizes. Jennison and Turnbull (10) have shown this when applying a sample size reassessment rule used by Cui za1

1

et al. (5) allowing the reassessed sample size to become as large as 25 times the preplanned one. They apply a very large zα1 (>4) and the trial may go on even when a negative effect has been observed. They suggest to instead use group sequential trials that are overpowered for the (optimistic) targeted treatment differences but still have reasonable power for a (pessimistic) smaller margin. The expected sample size for the larger treatment difference (although not being optimal for usual power values) then is still sufficiently small. Other arguments also exist that have been brought up against the use of the mid-trial estimate of the effect size for sample size recalculation. In such procedures, the power does not sufficiently increase with the effect size resulting in flat power curves (10). Moreover, when used in comparisons with placebo, they may aim at small and irrelevant effect sizes. The relevant effect size should follow from medical considerations accounting also for risks and costs, which is true in theory, but not enough knowledge of all issues concerned often exists in advance. Some of them may become clearer from the cumulating observations (e.g., when aggregating sufficient data on safety). In comparisons between active treatments, it is even more difficult to specify the relevant difference because any improvement of the standard therapy may be of interest (e.g., if safer therapies evolve). The current discussion around the choice of equivalence margins in large equivalence trials is a

0.8

0.6

0.6

0.4

0.4

0.2

0.2 0

1 z1

2

Pocock Boundaries

za1

1

0.8

−1

5

−1

0

1 z1

2

O’Brien & Fleming Boundaries

Figure 2. The conditional power (bold line) and the predictive conditional power (thin line) as a function of z1 for a group sequential design with balanced sample sizes over treatments and stages, overall power 80%, and α = 0.025. The dotted line denotes the conditional error function.

6

FLEXIBLE DESIGNS

1

1

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2 0.2

0.4 0.6 0.8 Conditional Power

1

Pocock Boundaries

0.2

0.4 0.6 0.8 Conditional Power

1

O’Brien & Fleming Boundaries

Figure 3. The conditional distribution functions (conditional on proceeding to the second stage) of the conditional power (bold lines) and the predictive conditional power (thin lines) for the null (dashed lines) and the alternative (continuous lines) hypothesis.

good example for this changed environment. More emphasis is put on precise confidence intervals of the treatment differences in order to be able to position the treatment within the set of comparators, which, quite naturally, leads to the consideration of families of null hypotheses with diverging equivalence margins (45), which complicates the choice of an a priori sample size (33, 34). 3.2 Staying with the a Priori Defined Effect Size One alternative is to base conditional power on the a priori fixed relevant treatment differences. Figure 2 gives the conditional power for this strategy depending on z1 for two different group sequential tests. When proceeding to the second stage under the alternative in Pocock’s design, only for very large z1 will the conditional power be above 0.8. In the Fleming-O’Brien design, the conditional power is higher because the final critical boundary zα2 is smaller. Figure 3 gives the distribution function of the conditional power given that one proceeds to the second stage. Under the alternative, this conditional power will stay below 0.8 in 74.6% of the cases for Pocock’s design and in only 46.5% for the O’Brien-Fleming design. On the other hand, under the null hypothesis conditional power values above 0.5 would be found in 14.8% and 20.8% of the cases respectively. Denne (21) looked at properties of the resulting test in terms of power and expected sample size when sample size reassessment is

based on conditional power. He shows that, compared with the group sequential design, the power is slightly higher when the effect is smaller than expected, however, for the price of serious inflations of the expected sample size. Posch et al. (46) considered designs with sample size reassessment according to a restricted conditional power rule, where an upper limit on the second-stage sample size is applied. Such a design leads to considerable savings in expected sample size, compared with a two-stage group sequential design, in an important region of the alternative hypothesis. These savings come for a small price to be paid in terms of expected sample size close to the null hypothesis and in terms of maximal sample size. Brannath and Bauer (42) derived the optimal combination test in terms of average sample size when the conditional power rule is applied. Admittedly, for multistage designs, arguments will become much more complicated so that it will be difficult to quantify the impact of sample size reassessment rules. 3.3 Overpowered Group Sequential Versus Adaptive Trials As mentioned above, cautious sample size reassessment rules based on conditional power have good properties in terms of expected sample size. Additionally, sample size reassessment in adaptive designs does not need to be performed according to strict predefined data-based rules. Information may develop from other sources (e.g.,

FLEXIBLE DESIGNS

safety considerations not having been incorporated in a fixed rule, or information from outside the trial) that could strongly favor a change of the preplanned sample size. For example, the demand for a larger sample size may come up to achieve an appropriate judgement of the risk-benefit relationship, or the treatment allocation rule may be changed to get more information on patients under a particular treatment. To adhere to the design and start another trial may then not be considered to be a reasonable strategy by ethical and economical reasons. Clearly, if the cautious approach to overpowered group sequential designs will be more extensively used in practice, then the stopping for futility option will have to be considered thoroughly and carefully in order to avoid radically large sample sizes under the null hypothesis. The very large maximal sample sizes to be laid down in the planning phase may be a further obstacle in the implementation of such designs. In practice, experimenters may rather tend to give up with a negative outcome if large increments in sample size are required to reach the next decision (and the chance for getting a positive decision is small). It is questionable if overall power rather than conditional power arguments will prevent such unscheduled stopping for futility decisions based on ongoing results. But then the perspectives of the overpowered group sequential designs may not apply in real-life scenarios. 3.4 Inconsistency of Rejection Regions At this point, the question about the price to be paid for the option of adaptations has to be answered. As mentioned in Section (2), no price has to be paid if the design is performed according to the preplanned group sequential design (if the appropriate inverse normal combination function is applied), which seems to be surprising and seems like a free lunch. However, the price to be paid may be the potential of being misled by the observed data and to modify a design, which, in fact, may be optimal for the true state of nature. Additionally, in case of an adaptation in terms of sample size reassessment, the decisions are based on statistics deviating from the minimal sufficient statistics.

7

Note that the way how to combine the data from the different stages has to be laid down before the stages are in fact performed (e.g., the weights when using the normal inverse normal method), either from the very beginning or recursively during the trial (14, 15). Denne (21) discussed the case where the adaptive test rejects but the group sequential two-stage test with weights corresponding to the actual sample sizes would not reject. He suggests to avoid such a type of inconsistency by rejecting in the adaptive test only if this group sequential test is rejected too. He shows that, for sample size reassessment based on conditional power, this additional condition has practically no impact on the performance of the adaptive test procedure. A more fundamental inconsistency occurs if the adaptive test rejects but the fixed sample size test would fail to reject (i.e., Z˜ ≥ zα2 ,   ˜ 2 n˜ 2 /(n1 + n˜ 2 ) < zα , but Z1 n1 /(n1 + n˜ 2 ) + Z where zα2 ≥ zα ). To get a complete picture, consider all possible constellations for which such inconsistencies occur with positive probability (i.e., there exist z˜ 2 values that lead to an inconsistency). It turns out that these constellations can be characterized in terms of the sample ratio r = n˜ 2 /(n1 + n˜ 2 ). Thus, if balanced stages were preplanned n1 = n2 ) r = 1/2 corresponds to the case of no sample size reassessment (n˜ 2 = n2 = n1 ), for n˜ 2 → ∞, one has r → 1, and if n˜ 2 = 1, then r = 1/(n1 + 1) (which approaches 0 for increasing n1 ). Figure 4 gives the regions where such inconsistencies occur with positive probability depending on Z1 when applying Pocock or O’Brien-Fleming boundaries with equal sample sizes per stage. The bad news is that such inconsistencies are possible for all values of Z1 . One sees that the area for such inconsistencies is smaller with constant rejection boundaries (Pocock), because the adaptive test has to exceed the larger final decision boundary. Furthermore, if the sample size is increased in case of small and decreased in case of large observed Z1 -values, which is a reasonable behavior in practice, no such inconsistencies may ever occur, which is good news. The lines in the figures denote the conditional power rule when performing sample size reassessment after halfway through the group sequential designs (α = 0.025, one-sided) with overall power 0.8. It can be

FLEXIBLE DESIGNS

1

1

0.75

0.75

0.5

0.5

r

r

8

0.25

0.25

0

0 0

za1 z1 Pocock Boundaries

0

za1 z1 O’Brien & Fleming Boundaries

Figure 4. For sample size reassessment [expressed in terms of the ratio r = n˜ 2 /(n1 + n˜ 2 )] in the shaded regions with a positive (but possibly very small) probability, the adaptive test may lead to a rejection, whereas the Z-test statistics of the pooled sample falls short of zα . For sample size reassessments in the white region, such inconsistencies never occur. The line corresponds to sample size reassessment according to the conditional power rule.

seen that, following this rule, inconsistencies never occur. A general way to deal with inconsistencies without imposing restrictions on the sample size reassessment rule is to reject in the adaptive design only if the fixed sample size test in the end also rejects at the level α. 4 EXTENDING THE FLEXIBILITY TO THE CHOICE OF THE NUMBER OF STAGES Interim analyses are not only costly and time consuming but unblinding may also have an unfavorable impact on the course of the remainder of a trial. Hence, interim analysis should only be performed if either relevant information for the adaptation of the trial design can be expected or if a good chance exists to arrive at an early decision. A priori, it is often difficult to assess the right number of interim analyses. Given the results of, for example, the first interim analysis, one might want to cancel the second if the conditional probability to get an early decision is small. If, on the other hand, a high chance exists to stop the trial early, one might want to add further interim analyses. Also, for external reasons, a further interim analysis might be favorable, for example, if a competitor enters the market such that an early decision would be a competitive advantage.

Assume that in the first interim analysis of a group sequential test with at least three stages, no early stopping condition applies. Then, by a generalization of Equation (4), the conditional error function A(z1 ) gives the probability (under the null hypothesis) that, given Z1 = z1 , the original design rejects at a later stage. Hence, it is the type I error probability that can be spent for later decisions. Thus, one can either spend this type I error probability by performing a single second stage with a final test at level A(z1 ) or, alternatively, proceed with the group sequential test. The decision of which way to go can be based on all information collected so far (14, 15). Posch et al. (46) investigated the expected sample size of a three-stage design with early rejection of the null hypothesis, where the second interim analysis is dropped when the chance for an early rejection is low. It is shown that such a design has nearly the expected average sample size of a three-stage group sequential test. At the same time, it has a lower maximal sample size and saves a considerable number on interim analysis, especially under the null hypothesis. Instead of dropping an interim analysis, as in the above example, one can also add further interim analyses. For example, one can start out with a two-stage design, then

FLEXIBLE DESIGNS

compute the conditional error A(z1 ) in the interim analysis and plan a further two-stage design with level A(z1 ). This procedure can be applied recursively (14, 15). 5

SELECTION OF THE TEST STATISTICS

Additionally, adaptive interim analysis give the opportunity to adaptively choose the test statistics used in the second stage, which allows one to select scores or contrasts based on the interim data (30, 31, 47, 48). If it turns out that a covariable can explain a substantial part of the variability of the primary variable, for the second-stage test an analysis accounting for this covariable can be specified. 6 MORE GENERAL ADAPTATIONS AND MULTIPLE HYPOTHESES TESTING 6.1 The Closed Testing Principle and Adaptive Designs A more general type of adaptation occurs if, in the interim analysis, the null hypothesis to be tested is changed, which is the case, for example, if doses are selected, endpoints are re-weighted, or the study population is adapted. If in the first stage a hypothesis, H 0,1 and, in the second stage a modified hypothesis, H0,2 are tested, then the combination test tests only the intersection hypothesis H0,1 ∩ H0,2 . The rejection of this intersection null hypothesis implies that H0,1 or H 0,2 is false. If, in a dose response setting, the proof of principle for efficacy of at least one dose is intended, it maybe sufficient. Also, when multiple endpoints are considered, the proof of principle for efficacy of at least one of the endpoints may suffice. To make inference on the individual hypotheses, a multiple testing procedure to control the multiple level (i.e., the probability to erroneously reject one or more null hypotheses) has to be applied. A general principle that guarantees control of the multiple level is the closed testing principle (49)! 6.1.1 Closed Testing Procedure. Assume a set I of null hypotheses is to be tested at multiple level α. To reject an individual

9

hypothesis j ∈ I, for all subsets J ⊂ I that contain j, the intersection hypothesis H0, J = ∩i∈J H0,i (stating that all hypotheses in J are true) has to be rejected at local level α. For example, the Bonferroni and Bonferroni Holm procedure can be formulated as special cases of this principle. The closed testing procedure can easily be integrated in adaptive designs by defining adaptive tests for all intersection hypotheses (26, 28, 29, 50), which opens a new dimension of flexibility: The hypotheses to be tested can also be adapted; some hypotheses can be dropped, and new hypotheses can be included in the interim analysis. 6.2 Selection and Addition of Hypotheses The general formulation of the adaptive multiple testing procedure is quite technical (29) such that one demonstrates the methodology with a simple example. Assume that in the first stage two hypotheses H0,A , H 0,B are tested (e.g., corresponding to two treatment groups that are compared with placebo). Then, according to the closed testing principle, level α tests for all intersection hypotheses H0,J , J ∈ J = {{A}, {B}, {A, B}} have to be defined. Thus, in the planning phase of the adaptive test for all hypotheses H0,J , J ∈ J, a level α combination test C(·,·) with decision boundaries α 0 , α 1 , cα has to be specified. In the following, assume for simplicity that all hypotheses are tested with the same combination test and set α 0 = 1, α 1 = 0 such that no early stopping is possible. Now, firststage tests and P-values p1, J , J ∈ J for all (intersection) hypotheses have to be defined. 6.2.1 Selecting a Single Hypothesis. Assume that, in the interim analysis, it is decided that only hypothesis H0,A is selected for the second stage. Then, only a second-stage test for H0,A with P-value p2,A is specified. H 0,A is rejected in the final analyses at multiple level α if both, C(p1,{A,B} , p2A ) ≤ cα (which is a test for the intersection hypothesis H 0,{A,B} ) and C(p1,A , p2,A ) ≤ cα . Note that, in this case, the second-stage test for H 0,A is also used as a test for the intersection hypothesis H0,{A,B} .

10

FLEXIBLE DESIGNS

6.2.2 Selecting Both Hypotheses. If, in the interim analysis, it has been decided to continue with both hypotheses, A and B, the second-stage test for the intersection hypothesis can be based on data for both hypotheses leading to a P-value p2,{A,B} . Then, the individual hypothesis i∈{A, B} can be rejected at multiple level α if C(p1,{A,B} , p2,{A,B} ) ≤ cα and C(p1,i , p2,i ) ≤ cα . 6.2.3 Adding a Hypothesis. Finally, assume that it is decided in the interim analysis to add a new hypothesis H0,C . For simplicity, it is additionally assumed that the hypotheses H0,A , H 0,B have been dropped in the interim analysis. Then H0,C can be rejected at multiple level α if all tests for the intersection hypotheses can be rejected:

Hypothesis

Test

H0,C ∩ H0,A ∩ H0,B H0,C ∩ H0,A H0,C ∩ H0,B H 0,C

C(p2,{A,B} , p2,C ) ≤ cα C(p2,A , p2,C ) ≤ cα C(p2,B , p2,C ) ≤ cα p2,C ≤ α

A simple way to construct P-values for the intersection hypotheses are Bonferroni adjusted P-values. In the example, one can set p1,{A,B} = min[2 min(p1,A , p1,B ),1]. More general tests for intersection hypotheses allow to give different weights to the individual hypothesis or to specify a hierarchical ordering among the individual hypotheses. If several doses are tested, the intersection hypothesis could be tested by testing for a positive trend. Note that the second-stage tests for all (intersection) hypothesis can be chosen adaptively based on the data from the first stage. Clearly, sample size reassessment can be performed additionally to the adaptive choice of hypotheses carried on to the second stage. The allocation ratio to different treatments could be changed, for example, investigating a larger sample for a test treatment. Also, more efficient tests could be planned for the second stage relying on the interim information.

6.3 Adaptation of Hypotheses in Clinical Trials 6.3.1 Treatment Selection. The selection of treatments or doses allows the integration of several phases in the drug development process into a single trial. Assume that in a first stage several dose groups are tested against placebo. In the interim analysis, one or more doses can be selected for the second stage. The selection process will typically be based on safety as well as efficacy information collected in the first stage as well as possible external information coming, for example, from other trials or experiments. 6.3.2 Adapting Multiple Endpoints. If multiple endpoints are considered, the adaptive interim analysis allows one to select or even add new endpoints in the second stage. Consequently, endpoints that appear to be highly variable or for which the interim data shows no efficacy at all can be dropped in the interim analysis. If a composite endpoint is used that summarizes multiple individual variables, the weights of these individual variables in the composite endpoint can be adapted. 6.3.3 Adapting the Population. Another option is to adapt the study population in the interim analysis, which may be desirable if, for example, the interim data show a strong treatment effect in a subpopulation that was not specified beforehand or safety problems in a subpopulation occur. 7 AN EXAMPLE The methodology has been exploited in a two-stage design for an international, multicenter, five-armed clinical phase II dose finding study (51, 52). The objectives for the first stage (433 patients recruited) was to obtain some initial evidence of the primary efficacy variable (infarct size measured by the cumulative release of alpha-HBDH from time 0 to 72 h), select a subset of doses to carry through to stage two and determine the sample size to be applied at stage two. No strict adaptation rules were laid down in the protocol because, in this early phase, the decisions were planned to be taken by utilizing all the

FLEXIBLE DESIGNS

information collected up to the interim analysis from in and outside the trial. A global proof of principle for the existence of a dose response relationship was intended by using the product p1 p2 of the stage-wise P-values pi as the predefined combination function. The predefined first-stage test to give p1 was a linear trend test among the increasing doses including placebo. Multiply controlled inference on the comparisons of the doses with a control was intended by applying the closed testing principle. The second and third highest doses were used in stage two in a balanced comparison with placebo. Based on a hierarchical testing strategy, the comparison of the highest dose applied in the second stage to placebo was laid down to create the secondstage P-value p2 . The second-stage sample size was fixed to 316 per group. The decisions were taken in a two-day meeting by a group of persons: an independent interim analysis group, the principal investigator, safety experts, and others. Finally, the overall product combination test (C(p1 , p2 ) = p1 p2 ) after the recruitment of 959 patients at the second stage failed to proof a dose response relationship because the promising first-stage result on the second highest dose could not be reproduced at the second stage. Still, it was argued a posteriori that the adaptive design saved time and resources to arrive at the decision as compared with other designs. 8

CONCLUSION

The crucial point in adaptive designs considered here is that the adaptation rule does not need to be fully specified in advance. Hence, information from all sources can be incorporated into the adaptation and a full modeling of the decision process is not required. Tsiatis (53) showed that, given a fixed adaptation rule, any adaptive design can be outperformed in terms of average sample size by a likelihood ratio-based sequential design with the same type I error spending function, which implies that in such a design, an interim look has to be performed at any sample size where the adaptive design has a positive probability to reject the null hypothesis. Hence, in case of sample size reassessment, one essentially ends up with continuous monitoring. But group sequential designs have

11

been introduced just to avoid this type of monitoring, which is usually not practical and too costly in clinical trials. Additionally, every group sequential design specifies a combination function and is, thus, a special case of an adaptive design. However, the adaptive design gives the opportunity of extra flexibility. If extensive adaptations are performed in the interim analysis as, for example, a reweighting of endpoints or a change of the study population, the transparency of the testing procedure may get lost and the trial can lose persuasiveness. Although many adaptations are possible in the sense that the type I error is controlled, not all of them are feasible as the interpretability of results may suffer. Another point to consider is how to keep the integrity of the trial by avoiding any negative impact by a leakage of interim results to investigators or other persons involved in the trial. Even the decisions taken in the interim analysis may allow to draw conclusions on the interim results: If, for example, the second-stage sample size is increased, it may indicate that a poor interim treatment effect has been observed. Although the availability of such information, in principle, does not harm the validity of the flexible design (the type I error is still controlled), it may give cause for problems concerning the motivation of investigators or recruitment. When performing adaptations, one has to keep in mind that findings from small first-stage samples (‘‘internal pilot studies’’) in general will be highly variable. So, by performing adaptations, the experimenter may quite frequently be detracted from a good design carefully laid down in the planning phase by looking at interim results. Adaptive designs open a wide field of flexibility with regard to mid-trial design modifications. The authors believe that adaptive treatment selection may be considered as the main advantage of adaptive designs that hardly can be achieved by other methods. However, sample size reassessment has attracted most of the attention up to now. Clearly, the acceptance of the methodology will be higher if the scope of adaptations to be performed is anticipated in the protocol, which, as in the example above, does not

12

FLEXIBLE DESIGNS

mean that the adaptation rule needs to be prespecified in any detail.

¨ ¨ 16. H. Schafer and H-H. Muller, Modification of the sample size and the schedule of interim analyses in survival trials based on data inspections. Stat. Med. 2001; 20: 3741–3751.

REFERENCES

17. P. Bauer and M. Posch, A letter to the editor. Stat. Med., 2004; 23: 1333–1334.

1. European Agency for the Evaluation of Medical Products, ICH Topic E9: Notes for Guidance on Statistical Principles for Clinical Trials, 1998. 2. P. Bauer, Sequential tests of hypotheses in consecutive trials. Biometr. J. 1989; 31: 663–676. 3. P. Bauer and K. K¨ohne, Evaluation of experiments with adaptive interim analyses. Biometrics 1994; 50: 1029–1041. 4. W. Lehmacher and G. Wassmer, Adaptive sample size calculations in group sequential trials. Biometrics 1999; 55: 1286–1290. 5. L. Cui, H. M. J. Hung, and S. Wang, Modification of sample size in group sequential clinical trials. Biometrics 1999; 55: 321–324. 6. P. Bauer, W. Brannath, and M. Posch, Flexible two stage designs: an overwiew. Meth. Inform. Med. 2001; 40: 117–121. 7. M. A. Proschan and S. A. Hunsberger, Designed extension of studies based on conditional power. Biometrics 1995; 51: 1315–1324. 8. M. Posch and P. Bauer, Adaptive two stage designs and the conditional error function. Biometr. J. 1999; 41: 689–696. 9. G. Wassmer, Statistische Testverfahren fur ¨ gruppensequentielle und adaptive Plane ¨ in klinischen Studien. M¨onch, Germany: Verlag Alexander, 1999. 10. C. Jennison and B. Turnbull, Mid-course sample size modification in clinical trials based on the observed treatment effect. Stat. Med. 2003; 22: 971–993. 11. L. D. Fisher, Self-designing clinical trials. Stat. Med. 1998; 17: 1551–1562. 12. Y. Shen and L. Fisher, Statistical inference for self-designing clinical trials with a one-sided hypothesis. Biometrics 1999; 55: 190–197. 13. J. Hartung and G. Knapp, A new class of completely self-designing clinical trials. Biometr. J. 2003; 45: 3–19. ¨ ¨ 14. H-H. Muller and H. Schafer, Adaptive group sequential designs for clinical trials: combining the advantages of adaptive and of classical group sequential approaches. Biometrics 2001; 57: 886–891. 15. W. Brannath, M. Posch, and P. Bauer, Recursive combination tests. J. Amer. Stat. Assoc. 2002; 97: 236–244.

18. M. Posch and P. Bauer, Interim analysis and sample size reassessment. Biometrics 2000; 56: 1170–1176. 19. Z. Shun, Sample size reestimation in clinical trials. Drug Inform. J. 2001; 35: 1409–1422. 20. Z. Shun, W. Yuan, W. E. Brady, and H. Hsu, Type I error in sample size re-estimations based on observed treatment difference. Stat. Med. 2001; 20: 497–513. 21. J. S. Denne, Sample size recalculation using conditional power. Stat. Med. 2001; 20: 2645–2660. 22. A. L. Gould, Sample size re-estimation: recent developments and practical considerations. Stat. Med. 2001; 20: 2625–2643. 23. T. Friede and M. Kieser, A comparison of methods for adaptive sample size adjustment. Stat. Med. 2001; 20: 3861–3874. 24. P. Bauer and J. R¨ohmel, An adaptive method for establishing a dose response relationship. Stat. Med. 1995; 14: 1595–1607. 25. M. Bauer, P. Bauer, and M. Budde, A simulation program for adaptive two stage designs. Comput. Stat. Data Anal. 1998; 26: 351–371. 26. P. Bauer and M. Kieser, Combining different phases in the development of medical treatments within a single trial. Stat. Med. 1999; 18: 1833–1848. 27. T. Friede, F. Miller, W. Bischoff, and M. Kieser, A note on change point estimation in dose-response trials. Comput. Stat. Data Anal. 2001; 37: 219–232. 28. M. Kieser, P. Bauer, and W. Lehmacher, Inference on multiple endpoints in clinical trials with adaptive interim analyses. Biometr. J. 1999; 41: 261–277. 29. G. Hommel and S. Kropf, Clinical trials with an adaptive choice of hypotheses. Drug Inform. J. 2001; 35: 1423–1429. 30. T. Lang, A. Auterith, and P. Bauer, Trendtests with adaptive scoring. Biometr. J. 2000; 42: 1007–1020. ¨ 31. M. Neuhausen, An adaptive location-scale test. Biometr. J. 2001; 43: 809–819. 32. M. Kieser, B. Schneider, and T. Friede, A bootstrap procedure for adaptive selection of the test statistic in flexible two-stage designs. Biometr. J. 2002; 44: 641–652.

FLEXIBLE DESIGNS 33. S-J. Wang, H. M. J. Hung, Y. Tsong, and L. Cui, Group sequential test strategies for superiority and non-inferiority hypotheses in active controlled clinical trials. Stat. Med. 2001; 20: 1903–1912. 34. W. Brannath, P. Bauer, W. Maurer, and M. Posch, Sequential tests for non-inferiority and superiority. Biometrics 2003; 59: 106–114. 35. Q. Liu, M. A. Proschan, and G. W. Pledger, A unified theory of two-stage adaptive designs. J. Amer. Statist. Assoc. 2002; 97: 1034–1041. 36. J. Lawrence and H. M. Hung, Estimation and confidence intervals after adjusting the maximum information. Biom. J. 2003; 45: 143–152. 37. W. Brannath, F. K¨onig, and P. Bauer, Improved repeated confidence bounds in trials with a maximal goal. Biom. J. 2003; 45: 311–324. 38. H. Frick, On confidence bounds for the BauerK¨ohne two-stage test. Biom. J. 2002; 44: 241–249. 39. S. Coburger and G. Wassmer, Conditional point estimation in adaptive group sequential test designs. Biometr. J. 2001; 43: 821–833. 40. S. Coburger and G. Wassmer, Sample size reassessment in adaptive clinical trials using a bias corrected estimate. Biometr. J. 2003; 45: 812–825. ¨ ¨ 41. H. H. Muller and H. Schafer, A general statistical principle for changing a design anytime during the course of a trial. Stat. Med. 2004; 23: 2497–2508. 42. W. Brannath and P. Bauer, Optimal conditional error functions for the control of conditional power. Biometrics 2004; 60: 715–723. 43. P. C. O’Brien and T. R. Fleming, A multiple testing procedure for clinical trials. Biometrics 1979; 5: 549–556. 44. S. J. Pocock, Group sequential methods in the design and analysis of clinical trials. Biometrika 1977; 64: 191–199. 45. P. Bauer and M. Kieser, A unifying approach for confidence intervals and testing of equivalence and difference. Biometrika 1996; 83: 934–937. 46. M. Posch, P. Bauer, and W. Brannath, Issues in designing flexible trials. Stat. Med. 2003; 30: 953–969. 47. J. Lawrence, Strategies for changing the test statistic during a clinical trial. J. Biopharm. Stat. 2002; 12: 193–205. 48. M. Kieser and T. Friede, Simple procedures for blind sample size adjustment that do not

13

affect the type i error rate. Heidelberg, Germany: Medical Biometry Unit, University of Heidelberg, Germany, 2002. 49. R. Marcus, E. Peritz, and K. R. Gabriel, On closed testing procedures with special reference to ordered analysis of variance. Biometrika 1976; 63: 655–660. 50. G. Hommel, Adaptive modifications of hypotheses after an interim analysis. Biometr. J. 2001; 43(5): 581–589. 51. U. Zeymer, H. Suryapranata, J. P. Monassier, G. Opolski, J. Davies, G. Rasmanis, G. Linssen, U. Tebbe, R. Schroder, R. Tiemann, T. Machnig, and K. L. Neuhaus, The Na+ /H + exchange inhibitor Eniporide as an adjunct to early reperfusion therapy for acute myocardial infarction. J. Amer. College Cardiol. 2001; 38: 1644–1651. 52. U. Zeymer, H. Suryapranata, and J. P. Monassier et al., Evaluation of the safety and cardioprotective effects of eniporide, a specific Sodium/Hydrogen exchange inhibitor, given as adjunctive therapy to reperfusion in patients with acute myocardial infarction. Heart Drug 2001; 1: 71–76. 53. A. A. Tsiatis and C. Mehta, On the inefficiency of the adaptive design for monitoring clinical trials. Biometrika 2003; 90: 367–378.

FOOD AND DRUG ADMINISTRATION (FDA)

responsible for advancing the public health by helping to speed innovations that make medicines and foods more effective, safer, and more affordable and by helping the public get the accurate, science-based information they need to use medicines and foods to improve their health. FDA is the federal agency responsible for ensuring that foods are safe, wholesome, and sanitary; human and veterinary drugs, biological products, and medical devices are safe and effective; cosmetics are safe; and electronic products that emit radiation are safe. FDA also ensures that these products are represented honestly, accurately and informatively to the public. Some of the specific responsibilities of the agency include the following:

The U.S. Food and Drug Administration is a scientific, regulatory, and public health agency that oversees items accounting for 25 cents of every dollar spent by consumers. Its jurisdiction encompasses most food products (other than meat and poultry); human and animal drugs; therapeutic agents of biological origin; medical devices; radiation-emitting products for consumer, medical, and occupational use; cosmetics; and animal feed. The agency grew from a single chemist in the U.S. Department of Agriculture in 1862 to a staff of approximately 9,100 employees and a budget of $1.294 billion in 2001, comprising chemists, pharmacologists, physicians, microbiologists, veterinarians, pharmacists, lawyers, and many others. About one third of the agency’s employees are stationed outside of the Washington, D. C. area, staffing over 150 field offices and laboratories, including 5 regional offices and 20 district offices. Agency scientists evaluate applications for new human drugs and biologics, complex medical devices, food and color additives, infant formulas, and animal drugs. Also, the FDA monitors the manufacture, import, transport, storage, and sale of about $1 trillion worth of products annually at a cost to taxpayers of about $3 per person. Investigators and inspectors visit more than 16,000 facilities a year and arrange with state governments to help increase the number of facilities checked. The FDA is responsible for protecting the public health by assuring the safety, efficacy, and security of human and veterinary drugs, biological products, medical devices, our national food supply, cosmetics, and products that emit radiation. The FDA is also

Biologics • Product and manufacturing establish-

ment licensing • Safety of the national blood supply • Research to establish product standards

and develop improved testing methods Cosmetics • Safety • Labeling

Drugs • Product approvals • OTC and prescription drug labeling • Drug manufacturing standards

Foods • Labeling • Safety of all food products (except meat

and poultry) • Bottled water

Medical Devices This article was modified from the website of the United States Food and Drug Administration (http: //www.fda.gov/oc/history/historyoffda/default.htm), (http://www.fda.gov/opacom/morechoices/mission. html), (http://www.fda.gov/comments/regs.html) by Ralph D’Agostino and Sarah Karl.

• Premarket approval of new devices • Manufacturing and performance stan-

dards • Tracking reports of device malfunction-

ing and serious adverse reactions

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

FOOD AND DRUG ADMINISTRATION (FDA)

Radiation-Emitting Electronic Products • Radiation safety performance standards

for microwave ovens, television receivers, and diagnostic equipment • X-ray equipment, cabinet x-ray systems (such as baggage x-rays at airports), and laser products • Ultrasonic therapy equipment, mercury vapor lamps, and sunlamps • Accrediting and inspecting mammography facilities Veterinary Products • Livestock feeds • Pet foods • Veterinary drugs and devices

FRAILTY MODELS

One can distinguish two broad classes of frailty models:

ANDREAS WIENKE

1. models with an univariate survival time as endpoint and 2. models that describe multivariate survival endpoints (e.g., competing risks, recurrence of events in the same individual, occurrence of a disease in relatives).

University Halle-Wittenberg Institute of Medical Epidemiology Biostatistics and Informatics Germany

The notion of frailty provides a convenient way to introduce random effects, association, and unobserved heterogeneity into models for survival data. In its simplest form, a frailty is an unobserved random proportionality factor that modifies the hazard function of an individual or related individuals. In essence, the frailty concept goes back to the work of Greenwood and Yule (1) on ‘‘accident proneness.’’ The term frailty itself was introduced by Vaupel et al. (2) in univariate survival models, and the model was substantially promoted by its application to multivariate survival data in a seminal paper by Clayton (3) (without using the notion ‘‘frailty’’) on chronic disease incidence in families. Frailty models are extensions of the proportional hazards model, which is best known as the Cox model (4), the most popular model in survival analysis. Normally, in most clinical applications, survival analysis implicitly assumes a homogenous population to be studied, which means that all individuals sampled into that study are subject in principle under the same risk (e.g., risk of death, risk of disease recurrence). In many applications, the study population cannot be assumed to be homogeneous, but must be considered as a heterogeneous sample (i.e., a mixture of individuals with different hazards). For example, in many cases, it is impossible to measure all relevant covariates related to the disease of interest, sometimes because of economical reasons, although the importance of some covariates is still sometimes unknown. The frailty approach is a statistical modeling concept that aims to account for heterogeneity, caused by unmeasured covariates. In statistical terms, a frailty model is a random effect model for time-to-event data, where the random effect (the frailty) has a multiplicative effect on the baseline hazard function.

In the first case, an univariate (independent) lifetime is used to describe the influence of unobserved covariates in a proportional hazards model (heterogeneity). The variability of survival data is split into a part that depends on risk factors, and is therefore theoretically predictable, and a part that is initially unpredictable, even when all relevant information is known. A separation of these two sources of variability has the advantage that heterogeneity can explain some unexpected results or give an alternative interpretation of some results, for example, crossing-over effects or convergence of hazard functions of two different treatment arms [see Manton and Stallard (5)] or levelingoff effects, which means the decline in the increase of mortality rates, which could result in a hazard function at old ages parallel to the x-axis [see Aalen and Tretli (6)]. More interesting, however, is the second case when multivariate survival times are considered in which one aims to account for the dependence in clustered event times, for example, in the lifetimes of patients in study centers in a multi-center clinical trial, caused by center-specific conditions [see Andersen et al. (7)]. A natural way to model dependence of clustered event times is through the introduction of a cluster-specific random effect—the frailty. This random effect explains the dependence in the sense that, had the frailty been known, the events would be independent. In other words, the lifetimes are conditional independent, given the frailty. This approach can be used for survival times of related individuals like family members or recurrent observations on the same person. Different extensions of univariate frailty models to multivariate models are possible and will be considered below.

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

FRAILTY MODELS

The author wants to explain the key ideas of univariate frailty models by an illustrative example from Aalen and Tretli (6). The authors analyzed the incidence of testis cancer by means of a frailty model based on data from the Norwegian Cancer Registry collected from 1953 to 1993. The incidence of testicular cancer is greatest among younger men, and then declines from a certain age. The frailty is considered to be established by birth, and caused by a mixture of genetic and environmental effects. The idea of the frailty model is that a subgroup of men is particularly susceptible to testicular cancer, which would explain why testis cancer is primarily a disease of young men. As time goes by, the members of the frail group acquire the disease, and at some age this group is more or less exhausted. Then the incidence, computed on the basis of all men at a certain age, will necessarily decline. 1

UNIVARIATE FRAILTY MODELS

The standard situation of the application of survival methods in clinical research projects assumes that a homogeneous population is investigated when subject under different conditions (e.g., experimental treatment and standard treatment). The appropriate survival model then assumes that the survival data of the different patients are independent form each other and that each patient’s individual survival time distribution is the same (independent and identically distributed failure times). This basic presumption implies a homogeneous population. However, in the field of clinical trials, one observes in many most practical situations that patients differ substantially. The effect of a drug, a treatment, or the influence of various explanatory variables may differ greatly between subgroups of patients. To account for such unobserved heterogeneity in the study population, Vaupel et al. (2) introduced univariate frailty models into survival analysis. The key idea is that individuals possess different frailties, and that those patients who are most frail will die earlier than the others. Consequently, systematic selection of robust individuals (that means patients with low frailty) takes place.

When mortality rates are estimated, one may be interested in how these rates change over time or age. Quite often it is observed that the hazard function (or mortality rate) rises at the beginning, reaches a maximum, and then declines (unimodal intensity) or levels off at a constant value. The longer the patient lives after manifestation of the disease, the more improved are his or her chances of survival. It is likely that unimodal intensities are often a result of a selection process acting in a heterogeneous population and do not reflect individual mortality. The population intensity may start to decline simply because the high-risk individuals have already died out. The hazard rate of a given individual might well continue to increase. If protective factors or risk factors are known, those could be included in the analysis by using the proportional hazards model, which is of the form µ(t, X) = µ0 (t) exp(β T X) where µ0 (t) denotes the baseline hazard function, assumed to be unique for all individuals in the study population. X is the vector of observed covariates and β the respective vector of regression parameters to be estimated. The mathematical convenience of this model is based on the separation of the effects of aging in the baseline hazard µ0 (t) from the effects of covariates in the parametric term exp(β T X). Two main reasons exist why it is often impossible to include all important factors on the individual level into the analysis. Sometimes too many covariates exist to be considered in the model, in other cases, the researcher does not know or is not able to measure all the relevant covariates. In both cases, two sources of variability exist in survival data: variability accounted for by measurable risk factors, which is thus theoretically predictable, and heterogeneity caused by unknown covariates, which is thus theoretically unpredictable, even if knowing all the relevant information. Advantages to separating these two sources of variability exist because heterogeneity in contrast to variability can explain some ‘‘unexpected’’ results or can provide an alternative explanation

FRAILTY MODELS

of some results. Consider, for example, nonproportional hazards or decreasing hazards when unexpected extra variability prevails. In a proportional hazards model, neglect of a subset of the important covariates leads to biased estimates of both regression coefficients and the hazard rate. The reason for such bias lies in the fact that the timedependent hazard rate results in changes in the composition of the study population over time with respect to the covariates. If two groups of patients exist in a clinical trial where some individuals experience a higher risk of failure, then the remaining individuals at risk tend to form a more or less selected group with a lower risk. An estimate of the individual hazard rate, without taking into account the unobserved frailty, would therefore underestimate the true hazard function and the extent of underestimation would increase as time progresses. The univariate frailty model extents the Cox model such that the hazard of an individual depends in addition on an unobservable random variable Z, which acts multiplicatively on the hazard function: µ(t, Z, X) = Zµ0 (t) exp(β T X)

(1)

Again, µ0 (t) is the baseline hazard function, β the vector of regression coefficients, X is the vector of observed covariates, and Z now is the frailty variable. The frailty Z is a random variable varying over the population, which lowers (Z < 1) or increases (Z > 1) the individual risk. Frailty corresponds to the notions liability or susceptibility in different settings (8). The most important point here is that frailty is unobservable. The respective survival function S, describing the fraction of surviving individuals in the study population, is given by
t S(t|Z, X) = exp(−Z

µ0 (s)ds exp(β T X)) (2) 0

S(t|Z, X) may be interpreted as the fraction of individuals surviving the time t after beginning of the follow-up given the vector of observable covariates X and frailty Z. Note, that Equation (1) and Equation (2) describe the same model using different notions. Up

3

to now, the model has been described at the level of individuals. However, this individual model is not observable. Consequently, it is necessary to consider the model at the population level. The survival function of the total population is the mean of the individual survival functions (Equation 2). It can be viewed as the survival function of a randomly drawn individual, and corresponds to that which is actually observed. It is important to note that the observed hazard function will not be similar to the individual hazard rate. What may be observed in the population is the net result for a number of individuals with different Z. The population hazard rate may have a completely different shape than the individual hazard rate as shown in Fig. 1. One important problem in the area of frailty models is the choice of the frailty distribution. The frailty distributions most often applied are the gamma distribution (2, 3), the positive stable distribution (9), a three-parameter distribution (PVF) (10), the compound Poisson distribution (11, 12) , and the log-normal distribution (13). Univariate frailty models are widely applied. A few examples that can be consulted for more details are listed here. Aalen and Tretli (6) applied the compound Poisson distribution to testicular cancer data already introduced above. The idea of the model is that a subgroup of men is particularly susceptible to testicular cancer, which results in selection over time. Another example is the malignant melanoma dataset including records of patients who had radical surgery for malignant melanoma (skin cancer) at the University Hospital of Odense in Denmark. Hougaard (14) compared the traditional Cox regression model with a gamma frailty and PVF frailty model, respectively, to these data. The third example deals with the time from insertion of a catheter into dialysis patients until it has to be removed because of infection. A subset of the complete data, including the first two infection times of 38 patients, was published by McGilchrist and Aisbett (13). To account for heterogeneity within the data, Hougaard (14) used a univariate gamma frailty model. Henderson and Oman (15) tried to quantify the bias that may occur in estimated

4

FRAILTY MODELS

Conditional and unconditional hazard rates Z=1

Z=2 hazard rates

Z = 0.5

unconditional

0

10

20

30

40

50 60 age

70

80

90 100 110

covariate effects, and fitted marginal distributions when frailty effects are present in survival data, but the latter are ignored in a misspecified proportional hazards analysis. Congdon (16) investigated the influence of different frailty distributions (gamma, inverse Gaussian, stable, binary) on total and cause-specific mortality from the London area (1988–1990).

Figure 1. Conditional and unconditional hazard rates in a simulated data set of human mortality. The red lines denote the conditional (individual) hazard rates for individuals with frailty 0.5, 1, and 2, respectively. The blue line denotes the unconditional (population) hazard rate

Averaging over an assumed distribution for the latent variables (e.g., using a gamma, log-normal, stable distribution) then induces a multivariate model for the observed data. In the case of paired observations, the twodimensional survival function is of the form
∞ S(t1 , t2 ) =

S(t1 |z, X 1 )S(t2 |z, X 2 )g(z)dz 0

2

MULTIVARIATE FRAILTY MODELS

A second important application of frailty models is in the field of multivariate survival data. Such kind of data occurs, for example, if lifetimes (or times of onset of a disease) of relatives (twins, parent-child) or recurrent events like infections in the same individual are considered. In such cases, independence between the clustered survival times cannot be assumed. Multivariate models are able to account for the presence of dependence between these event times. A commonly used and very general approach is to specify independence among observed data items conditional on a set of unobserved or latent variables (14). The dependence structure in the multivariate model develops from a latent variable in the conditional models for multiple observed survival times, for example, let S(t1 |Z, X 1 ) and S(t2 |Z, X 2 ) be the conditional survival functions of two related individuals with different vectors of observed covariates X 1 and X 2 , respectively [see Equation (2)].

where g denotes the density of the frailty Z. In the case of twins, S(t1 ,t2 ) denotes the fraction of twins pairs with twin 1 surviving t1 and twin 2 surviving t2 . Frailty models for multivariate survival data are derived under conditional independence assumption by specifying latent variables that act multiplicatively on the baseline hazard. 2.1 The Shared Frailty Model The shared frailty model is relevant to event times of related individuals, similar organs, and repeated measurements. Individuals in a cluster are assumed to share the same frailty Z, which is why this model is called shared frailty model. It was introduced by Clayton (3) and extensively studied in Hougaard (14). The survival times are assumed to be conditional independent with respect to the shared (common) frailty. For ease of presentation, the case of groups with pairs of individuals will be considered (bivariate failure times,

FRAILTY MODELS

for example, event times of twins or parentchild). Extensions to multivariate data are straightforward. Conditional on the frailty Z, the hazard function of an individual in a pair is of the form Zµ0 (t)exp(β T X), where the value of Z is common to both individuals in the pair, and thus is the cause for dependence between survival times within pairs. Independence of the survival times within a pair corresponds to a degenerate frailty distribution (Z = 1, V(Z) = σ 2 = 0). In all other cases with σ 2 > 0, the dependence is positive by construction of the model. Conditional on Z, the bivariate survival function is given as S(t1 , t2 |Z) = S1 (t1 )Z S2 (t2 )Z In most applications, it is assumed that the frailty distribution (i.e., the distribution of the random variable Z) is a gamma distribution with mean 1 and variance σ 2 . Averaging the conditional survival function produces under this assumption survival functions of the form 2

2

S(t1 , t2 ) = (S1 (t1 )−σ + S2 (t2 )−σ − 1)1/σ

2

Shared frailty explains correlations between subjects within clusters. However, it does have some limitations. First, it forces the unobserved factors to be the same within the cluster, which may not always reflect reality. For example, at times it may be inappropriate to assume that all partners in a cluster share all their unobserved risk factors. Second, the dependence between survival times within the cluster is based on marginal distributions of survival times. However, when covariates are present in a proportional hazards model with gamma distributed frailty, the dependence parameter and the population heterogeneity are confounded, which implies that the joint distribution can be identified from the marginal distributions (10). Third, in most cases, a one-dimensional frailty can only induce positive association within the cluster. However, some situations exist in which the survival times for subjects within the same cluster are negatively associated. For example, in the Stanford Heart Transplantation Study, generally the longer an individual must wait for an available heart, the shorter he or she is likely to survive after

5

the transplantation. Therefore, the waiting time and the survival time afterwards may be negatively associated. To avoid the above-mentioned limitations of shared frailty models, correlated frailty models were developed. 2.2 The Correlated Frailty Model Originally, correlated frailty models were developed for the analysis of bivariate failure time data, in which two associated random variables are used to characterize the frailty effect for each pair. For example, one random variable is assigned for partner 1 and one for partner 2 so that they would no longer be constrained to have a common frailty. These two variables are associated and have a joint distribution. Knowing one of them does not necessarily imply knowing the other. A restriction no longer exists on the type of correlation. These two variables can also be negatively associated, which would induce a negative association between survival times. Assuming gamma distributed frailties, Yashin and Iachine (17) used the correlated gamma frailty model resulting in a bivariate survival distribution of the form S(t1 , t2 ) =

S1 (t1 )1−ρ S2 (t2 )1−ρ (S1 (t1 )−σ 2 + S2 (t2 )−σ 2 − 1)ρ/σ 2

Examples of the use of multivariate frailty models are various and emphasize the importance of this family of statistical models for survival data. – a shared log-normal frailty model for the catheter infection data mentioned above used by McGilchrist and Aisbett (13); – a shared frailty model with gamma and log-normal distributed frailty, applied to the recurrence of breast cancer by dos Santos et al. (18); – a shared positive stable frailty model, applied by Manatunga and Oakes (19) to the data from the Diabetic Retinopathy Study, which examined the effectiveness of laser photo-coagulation in delaying the onset of blindness in patients with diabetic retinopathy. The positive stable frailty allows for proportional hazards both in the marginal and the conditional model;

6

FRAILTY MODELS

– a study of Andersen et al. (7), who tested for center effects in multi-center survival studies by means of a frailty model with unspecified frailty distribution; – a correlated gamma-frailty model, applied by Pickles et al. (20) to age of onset of puberty and antisocial behavior in British twins; – a correlated gamma-frailty model by Yashin and Iachine (17) and Yashin et al. (21) to analyze mortality in Danish twins; – a correlated gamma-frailty model by Wienke et al. (22) and Zdravkovic et al. (23) to analyze genetic factors involved in mortality caused by coronary heart disease in Danish and Swedish twins, respectively; – an extension of the correlated gammafrailty model by Wienke et al. (25) used to model death due to coronary heart disease in Danish twins; – different versions of the correlated gamma-frailty model applied by Zahl (26) to causespecific cancer mortality in Norway to model the excess hazard.

3

SOFTWARE

Stata 7 (procedure st streg) allows one to explore univariate models with gamma and inverse Gaussian distributed frailty. aML 2 supports log-normal frailty models in univariate analysis. WinBugs is designed for analysis of shared frailty models with different frailty distribution, using Markov Chain Monte Carlo methods. On the Internet, several SAS, GAUSS, S-Plus, and R routines are available dealing with different frailty models. REFERENCES 1. M. Greenwood and G. U. Yule, An inquiry into the nature of frequency distributions representative of multiple happenings with particular reference to the occurrence of multiple attacks of disease or of repeated accidents. J. Royal Stat. Soc. 1920; 83: 255–279.

2. J. W. Vaupel, K. G. Manton, and E. Stallard, The impact of heterogeneity in individual frailty on the dynamics of mortality. Demography 1979; 16: 439–454. 3. D. G. Clayton, A model for association in bivariate life tables and its application in epidemiological studies of familial tendency in chronic disease incidence. Biometrika 1978; 65: 141–151. 4. D. R. Cox, Regression models and life-tables. J. Royal Stat. Soc. B 1972; 34: 187–220. 5. S. Manton, Methods for evaluating the heterogeneity of aging processes in human populations using vital statistics data: explaining the black/white mortality crossover by a model of mortality selection. Human Biol. 1981; 53: 47–67. 6. O. O. Aalen and S. Tretli, Analysing incidence of testis cancer by means of a frailty model. Cancer Causes Control 1999; 10: 285–292. 7. P. K. Andersen, J. P. Klein, and M-J. Zhang, Testing for centre effects in multi-centre survival studies: a Monte Carlo comparison of fixed and random effects tests. Stat. Med. 1999; 18: 1489–1500. 8. D. S. Falconer, The inheritance of liability to diseases with variable age of onset, with particular reference to diabetes mellitus. Ann. Human Genet. 1967; 31: 1–20. 9. P. Hougaard, A class of multivariate failure time distributions. Biometrika 1986; 73: 671–678. 10. P. Hougaard, Survival models for heterogeneous populations derived from stable distributions. Biometrika 1986; 73: 671–678. 11. O. O. Aalen, Heterogeneity in survival analysis. Stat. Med. 1988; 7: 1121–1137. 12. O. O. Aalen, Modeling heterogeneity in survival analysis by the compound Poisson distribution. Ann. Appl. Probabil. 1992; 4(2): 951–972. 13. C. A. McGilchrist and C. W. Aisbett, Regression with frailty in survival analysis. Biometrics 1991; 47: 461–466. 14. P. Hougaard, Analysis of Multivariate Survival Data. New York: Springer, 2000. 15. R. Henderson and P. Oman, Effect of frailty on marginal regression estimates in survival analysis. J. Royal Stat. Soc. B 1999; 61: 367–379. 16. P. Congdon, Modeling frailty in area mortality. Stat. Med. 1995; 14: 1859–1874. 17. A. I. Yashin and A. I. Iachine, Genetic analysis of durations: correlated frailty model applied

FRAILTY MODELS

18.

19.

20.

21.

22.

23.

to survival of danish twins. Genet. Epidemiol. 1995; 12: 529–538. D. M. dos Santos, R. B. Davies, and B. Francis, Nonparametric hazard versus nonparametric frailty distribution in modeling recurrence of breast cancer. J. Stat. Plan. Infer. 1995; 47: 111–127. A. K. Manatunga and D. Oakes, Parametric analysis of matched pair survival data. Lifetime Data Anal. 1999; 5: 371–387. A. Pickles et al., Survival models for developmental genetic data: age of onset of puberty and antisocial behavior in twins. Genet. Epidemiol. 1994; 11: 155–170. A. I. Yashin, J. W. Vaupel, and I. A. Iachine, Correlated individual frailty: an advantageous approach to survival analysis of bivariate data. Math. Pop. Studies 1995; 5: 145–159. A. Wienke, N. Holm, A. Skytthe, and A. I. Yashin, The heritability of mortality due to heart diseases: a correlated frailty model applied to Danish twins. Twin Res. 2001; 4: 266–274. S. Zdravkovic et al., Heritability of death from coronary heart disease: a 36-year follow-up of 20,966 Swedish twins. J. Intern. Med. 2002; 252: 247–254.

7

24. S. Zdravkovic et al., Genetic influences on CHD-death and the impact of known risk factors: Comparison of two frailty models. Behavior Genetics 2004; 34: 585–591. 25. A. Wienke, K. Christensen, A. Skytthe, and A. I. Yashin, Genetic analysis of cause of death in a mixture model with bivariate lifetime data. Stat. Model. 2002; 2: 89–102. 26. P. H. Zahl, Frailty modelling for the excess hazard. Stat. Med. 1997; 16: 1573–1585.

FURTHER READING P. Hougaard, Modeling heterogeneity in survival data. J. Appl. Probabil. 1991; 28: 695–701. A. Wienke, P. Lichtenstein, K. Czene, and AA. I. Yashin, The role of correlated frailty models in studies of human health, ageing and longevity. In: Applications to Cancer and AIDS Studies, Genome Sequence Analysis, and Survival Analysis. N. Balakrishnan, J.-L. Auget, M. Mes¨ bah, G. Molenberghs (eds.), Birkhauser, 2006, pp. 151–166.

FUTILITY ANALYSIS

with a one-sided hypothesis. Key contributions to development of the futility monitoring concept include introduction of stochastic curtailment by Lan et al. (6) and adaptation of this methodology to the futility index by Ware et al. (1). First, we present the common statistical approaches to futility monitoring. Two examples of studies stopped for futility are then provided in detail. Finally, practical issues of futility monitoring are discussed.

BORIS FREIDLIN Biometric Research Branch National Cancer Institute Bethesda, Maryland

Interim monitoring of outcome data has become a well-accepted component of randomized clinical trials (RCT); a trial can be stopped early for efficacy if a treatment arm appears definitively better than another arm. Most clinical trials are designed to demonstrate benefit of experimental vs. standard treatment and are thus implicitly addressing a one-sided hypothesis. The one-sided nature of the question provides rationale for both efficacy and futility monitoring. In technical terms, futility monitoring refers to a statistical procedure for stopping the trial early if it appears that the experimental arm is unlikely to be shown definitively better than the control arm if the trial is continued to the final analysis. In the context of evidence-based clinical research, the primary goal of a phase III trial is to provide data on the benefit-to-risk profile of the intervention that is sufficiently compelling to change medical practice. From this perspective, a futility boundary should be interpreted as the point at which convincing evidence exists to resolve the original question—that is, to convince the clinical community that the new treatment is not beneficial. The advantages of early futility stopping are obvious in terms of minimizing patient exposure to ineffective potentially toxic experimental treatments as well as in terms of optimizing the use of resources (1). However, these potential advantages should be weighed against the risk of stopping short of obtaining sufficiently compelling evidence that the new treatment is not beneficial (2–4). Such premature stopping wastes time and resources that went into the design and conduct of the trial and may adversely affect similar ongoing and future trials. There is a rich literature on futility monitoring. DeMets and Ware (5) were among the first to point out the need for a more aggressive monitoring for futility in studies

1 COMMON STATISTICAL APPROACHES TO FUTILITY MONITORING 1.1 Statistical Background For simplicity, the following presentation assumes that the study outcome has a normal distribution (or has been transformed to a normal outcome). The results are readily adapted to other common clinical outcome types (e.g., time-to-event or binary endpoints) where the normal approximation is routinely used in the monitoring of accumulating data (7, 8). Consider a RCT that compares an experimental arm (arm A) to the control arm (arm B). Let θ denote the primary measure of treatment effect (e.g., difference in cholesterol levels or log hazard ratio for survival) with θ = 0 corresponding to no difference between the arms and positive values of θ corresponding to an advantage of the experimental treatment. The trial is designed to test H 0 :θ = 0 versus HA :θ = θ A , where θ A > 0 is the minimally clinically meaningful treatment effect. Relevant design calculations are θˆ often based on the test statistic Z = SE( θˆ ) ˆ is an estimate of the standard where SE(θ) ˆ The quantity SE(θ) ˆ −2 is called error of (θ). the information (I) about the parameter θ . The design rejects H 0 if at the final analysis Z > cα (cα denotes the 1 − α quantile of the standard normal distribution). Type I error and type II error of the design are defined as PH0 {Z > cα } and PH1 {Z ≤ cα }, respectively. The amount of information needed to detect θ A with type I error α and type II error β is:
I=

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

cα + cβ θA

2 .

2

FUTILITY ANALYSIS

CPk (θ ) = P(Zk > cα |Zk = zk , θ )

curtailment (6). The most common implementation of stochastic curtailment calculates conditional power at the minimally clinically meaningful treatment effect θ A and stops at the kth analysis if CPk (θ A ) < γ for some prespecified γ . This formulation of futility rule is easily understood by nonstatistical consumers of clinical trials such as clinicians and patients (1). Use of formal rules based on conditional power at the design alternative CPk (θ A ) is sometime criticized because it conditions on current data and a value of θ that may be inconsistent with the current data. This criticism is somewhat misplaced because CPk (θ A ) is intended to quantify the degree of consistency between the observed data and the trial goal as represented by parameters (θ A ,α,β). In any case, the real life applications of stochastic curtailment method to futility monitoring typically involve evaluation of conditional power over a range of potential values of parameter θ that includes both biologically plausible values consistent with the accumulated data and the treatment effect used in the design of the study (11). In another variation of stochastic curtailment, the stopping rule is based on the conditional power calculated under the current estimate of treatment effect θˆk , that is, stop at kth analysis if CPk (θˆk ) < γ . This so-called ‘‘current trend’’ approach is problematic as it may lead to premature stopping, especially early into the trial when the estimate θˆk may be highly unreliable (see the antenatal corticosteroids example). As an alternative, Pepe and Anderson (12) proposed a more conservative approach using CPk (θˆk + SE(θˆk )). Futility boundaries based on conditional power can be derived as follows. At the kth interim analysis, the conditional distribution is √ ZK , given √ Zk = zk is normal with mean zk tk + θ IK (1 − tk ) and variance 1 − tk . The conditional power is then   √ √ cα − zk tk − θ IK (1 − tk ) CPk (θ ) = 1 −  √ 1 − tk

The conditional power is an appealing way to summarize the accumulated data to quantify the feasibility of continuing the study to the full accrual. If CPk (θ ) is low for θ in the desirable range, the study may be stopped for futility. This approach is known as stochastic

where ( ) is the cumulative standard normal distribution function. Hence the condition CPk (θ ) < γ corresponds to stopping for futility at kth analysis if √  (1 − tk ) 1 − tk cα − θ IK √ Zk < √ − cγ √ tk tk tk

For a specific outcome type, information I is translated into the required sample size. For ethical and economic reasons, it is important to stop the trial as soon as sufficiently compelling evidence has accumulated for either rejection of H0 in favor of H A (efficacy monitoring) or if there is no longer a reasonable chance that H0 can be rejected (futility monitoring). Most RCTs have prospectively specified interim monitoring guidelines that integrate the interim and the final analysis into a single framework. A typical interim analysis plan specifies K analyses, with the first K − 1 analyses designated as interim and the Kth as the final analysis. The test statistic and information at the kth analysis (k = 1, . . . , K) are denoted by Zk and Ik , respectively. The quantity tk = Ik /IK (0 < tk ≤ tK = 1) represents the proportion of the total information at the kth analysis and is often referred to as the ‘‘information time’’ (9). Information time provides a convenient universal notation and will be used in the following presentation. Formal derivation of futility boundaries is based on the following property that holds for normal outcomes with independent observations: statistics (Z1 , . . . , ZK ) are multivariate with √ normal √ √ K-dimensional mean vector √ θ IK ( t1 , . . . , tK ) and cov(Zl , Zk ) = tl /tk for l < k. It can also be shown that under some mild conditions the result holds asymptotically for most non-normally distributed outcomes that have been transferred to normal outcomes (10). 1.2 Conditional Power and Stochastic Curtailment Conditional power at the kth analysis is defined as the probability of rejecting the null hypothesis at the final analysis given the currently observed data and a hypothesized true treatment effect θ :

FUTILITY ANALYSIS

Because futility monitoring does not involve rejecting H 0 , the type I error of the study is not inflated. Futility monitoring does inflate the type II error. Lan et al. (6) have shown that continuous application of the stochastic curtailment futility rule CPk (θ A ) < γ on a fixed sample size study with type II error β has type II error bounded by β/(1 − γ ). Because in most implementations interim monitoring is not continuous, this bound is very conservative. A more accurate bound can be found in Davis and Hardy (13). See Lachin (14) and Moser and George (15) for a discussion of integrating the stochastic curtailment futility boundary with an efficacy boundary for simultaneous control of type I and type II error rates. An alternative to calculating conditional power at a fixed value of parameter θ is the use of predictive power (16, 17). It is derived by averaging the conditional power with respect to the posterior distribution for parameter θ :  PPk = CPk (θ )π (θ |Dk )dθ where π (θ |Dk ) denotes the posterior distribution and Dk is the observed data at the kth analysis. 1.3 Group Sequential Formulation of Futility Boundary Another common approach to futility monitoring is the one-sided group sequential rule of Pampallona and Tsiatis (18) (herein referred to as PT) that is based on the power family group sequential tests (19). The rule combines both efficacy and futility boundaries to control the overall type I and type II error rates. Boundaries depend on a parameter  and are defined by pairs of critical values (ak ,bk ) for k = 1, . . . , K, subject to ak ≥ bk and aK = bK . At the kth analysis time, the decision rule is: if Zk > ak , stop for efficacy; if Zk < bk , stop for futility; otherwise, continue. The critical values ak and bk are of the form ak = C1 t−0.5 k and , bk = θA Ik0.5 − C2 t−0.5 k

3

where constants C1 and C2 are selected to satisfy the specified type I and type II error rates for fixed . Here, we focus on the futility boundary {bk }. This futility procedure corresponds to a one-sided group sequential boundary for testing the hypothesis HA against H 0 . The boundary uses a spending function to control the overall error rate at level β. The shape of the spending function is determined by parameter . For example,  = 0 corresponds to an O’Brien–Fleming type boundary (20) whereas  = 0.5 corresponds to a Pocock type boundary (21). In practice, the most commonly used value of  is 0. It can be shown that a PT futility boundary with  = 0 is similar to the stochastic curtailment with γ = 0.5 (22). More generally, Moser and George (15) show that a PT boundary can be expressed in terms of the generalized stochastic curtailment procedure and vice versa. A number of published group sequential approaches to futility are based on repeated tests of the alternative hypothesis. For example, the following rule is suggested (23, 22): at an interim analysis, terminate the trial for futility if a one-sided test of consistency of the data with HA is statistically significant at some fixed nominal significance level (e.g., 0.005 or 0.001). This procedure has negligible effect on power (22). The repeated test rules are statistically equivalent to the approaches based on repeated confidence intervals (4, 24, 25). 1.4 Other Statistical Approaches to Constructing Futility Boundaries Another simple futility rule was proposed by Wieand et al. (26) (see also Ellenberg and Eisenberg [27]). A single futility analysis is performed when half of the total trial information is available. If the estimate of the treatment effect θˆ < 0 (e.g., the log hazard ratio of the control arm to the experimental arm is less than 0), early termination of the trial is recommended. This simple rule results in negligible deflation of the type I error and less than 2% loss of the fixed-sample-size design power. This approach was used for futility stopping in randomized studies of the somatostatin analogue octreotide for advanced colon cancer

4

FUTILITY ANALYSIS

(28) and for advanced pancreatic cancer (29). Although the simplicity of this rule is appealing, having long intervals between scheduled analyses may not be appropriate in some settings (30). A Bayesian framework can offer a useful tool for futility monitoring deliberations. Bayesian analysis provides a formal way for combining the totality of the prior and/or external information with the data accumulated in the trial. A futility decision is typically based on considering an ‘‘enthusiastic’’ prior—that is, a prior reflecting the opinion of the proponents of the new treatment. For example, in a normal setting, such a prior may be defined by centering on the alternative hypothesis θ A and fixing the probability of no benefit (θ < 0) at some low value (e.g., 5%). The posterior distribution of the treatment effect θ given the current data can then be used to model how the accumulated data affect the beliefs of the proponents of the intervention and whether the observed negative trend is sufficiently compelling to stop the trial for futility. For further discussion, see Spiegelhalter et al. (31). 2

EXAMPLES

Examples of futility stopping from two randomized clinical trials are now presented in detail. 2.1 Optimal Duration of Tamoxifen in Breast Cancer National Surgical Breast and Bowel Project (NSABP) B-14 trial investigated the use of

tamoxifen in early-stage breast cancer. The relevant portion of the study for the discussion here was designed to address the optimal duration of tamoxifen administration. Breast cancer patients who had completed 5 years of tamoxifen therapy free of disease recurrence were randomized to either tamoxifen or placebo for another 5 years (second randomization). The study was designed to have 85% power (α = 0.05 one-sided) to detect a 40% reduction in disease-free survival (DFS) hazard rate after observing 115 events (an event was defined as recurrence of breast cancer, occurrence of new primary cancer, or death from any cause). The study was closed after the third interim analysis with the conclusion that a meaningful benefit of continuing tamoxifen would not be demonstrated. This result went against the prevailing clinical opinion at that time. The study team provided a detailed account of the various futility monitoring approaches that had lead to the decision (32). The summary of DFS and the secondary endpoint overall survival (OS) from the three interim analyses is given in Table 1. At the time of the first interim analysis, a developing negative trend was observed but the data were considered immature. At the time of the second interim analysis, conditional power (at 115 events) under the design alternative of 40% reduction in hazard was less than 5%. In fact, even under a threefold reduction (67%) the conditional power was less than 5%. However, an argument was made that [1] the design alternative (40% reduction in hazard) that was roughly equivalent to the size of benefit for the patients

Table 1. Interim Analysis of Results from the National Surgical Breast and Bowel Project (NSABP) B-14 trial Analysis number Information time (observed/planned number of events) Placebo arm: DFS events (deaths) Tamoxifen arm: DFS events (deaths) Hazard ratio (tamoxifen/placebo) DFS P-value (two-sided) Conditional power under the design alternative

1

2

3

40% (46/115)

58% (67/115)

76% (88/115)

18 (6) 28 (9) NA NA NA

24 (10) 43 (19) 1.8 0.028 0) was 13%, and the posterior probability of the treatment effect exceeding θ A was 0.00003.

2.2 A Randomized Study of Antenatal Corticosteroids Pregnant women at risk for preterm delivery are often given weekly courses of antenatal corticosteroids. To assess the efficacy of weekly administration, a randomized doubleblind, placebo-controlled, phase III trial of weekly administration versus a single course was conducted Guinn et al. (33). The trial was designed to enroll 1,000 women to give 90% power to detect a 33% reduction in composite neonatal morbidity from 25% to 16.5% (0.66 relative risk) at the 0.05 two-sided significance level (note that while a two-sided significance level was used for sample size calculation the study is addressing a one-sided question). The study results are summarized in Table 2 (34). At a planned interim analysis after 308 women had been randomized and evaluated, observed composite morbidity was 24% on the weekly arm and 27% on single-course groups, corresponding to only a 11% relative reduction. The investigators reported that under the ‘‘observed trend’’ the conditional power at 1,000 patients was less than 2%. (Guinn et al. do not detail the derivation of the 2% result, but based on the reported data the conditional power under ‘‘trend observed’’ in the first 308 patients was approximately 14%.) The reported conditional power (along with some external safety reports) was used as the rationale to stop the trial after accrual of 502 patients (194 additional patients). Had the investigators calculated conditional power under the design alternative, they would have obtained conditional power of 74%. After the study was finished, observed morbidity rates in the last

Table 2. Interim Analysis Results from Antenatal Corticosteroids Trial (as reported in [34]) First 308 patients Weekly # of patients # of morbidities (%) RR 95% CI Conditional power ‘‘current trend’’ Conditional power design alternative

Single

Last 194 patients Weekly

Single

161 147 39 (24.2) 40 (27.2) .89 (.61, 1.30) 14%

95 99 17 (17.8) 26 (26.2) .68 (.40, 1.74) 75%

74%

77%

Source: Jenkins et al. JAMA. 2002; 287: 187–188.

5

Total 502 patients Weekly

Single

256 246 56 (21.9) 66 (26.8) .82 (.6, 1.11) 40% 77%

6

FUTILITY ANALYSIS

194 patients were 17.9% in the weekly arm and 26.2% in the single-course arm, corresponding to a 32% reduction (very close to the 34% target effect) (34). At the time of the trial closure with 502 patients, conditional power (under the design alternative) was approximately 77%, and conditional power under the ‘‘current trend’’ was 40%. These data also illustrate the unstable nature of the ‘‘current trend’’ approach early in the study: conditional power ranged from 14% in the first 308 patients to 75% in the next 194. 3

DISCUSSION

The decision to stop a trial for futility is inherently complex. In most cases, it involves considerations beyond whether the primary outcome crossed a formal boundary. Such a fundamental decision requires a thorough evaluation of all relevant data (external and internal). Issues requiring careful consideration include consistency of results across primary and secondary outcomes, treatment group similarity, and complete and unbiased evaluation of outcome (35). The degree of evidence required to address the study objectives as well as justify early futility stopping depends on the nature of the intervention, the context of the disease, and the current standard of care. Randomized clinical trials are often conducted in the presence of efficacy evidence from randomized trials in similar clinical settings, epidemiologic studies, and surrogate endpoints. In some situations, experimental intervention is already in limited use in the community, and the trial is designed to provide definitive evidence to support widespread use. In such cases, stopping the trial early requires a clear refutation of meaningful benefit (4). Hence, futility decisions are typically made only after a definitive negative trend is established. This point of view is often taken in large trials in cardiovascular, cancer-prevention, and adjuvant cancer settings (32, 36). In contrast, in settings such as advanced cancer, the requirements for preliminary evidence of efficacy are relatively low and a large proportion of agents tested in phase III studies turn out to be inactive. In such settings, there is a need for an efficient approach for testing new

agents and weaker evidence (or rather a lack of evidence of activity) is sufficient to affect medical practice (28, 29). These more aggressive futility standards allow one to redirect patients to other trials. A discrepancy between the futility monitoring rule and the degree of evidence required in the particular setting may result in the failure to answer an important clinical question as well as waste time and resources. A somewhat extreme example is provided by the use of the ‘‘current trend’’ conditional power in the antenatal corticosteroids trial (33). The investigators used conditional power under the ‘‘observed trend’’ with less than one-third of the total information available to justify stopping the trial. At that time, conditional power under the design alternative was 74%. Moreover, the observed data at that time did not rule out the target relative risk of 0.66. Consequently, the use of ‘‘current trend’’ conditional power early in this study was inappropriate. One can argue that the study failed to address its goal (37). Some of the rules described previously suggest stopping for futility with the experimental arm doing nontrivially better than the control arm. For example, consider the commonly used PT futility approach with  = 0. For a trial designed to detect a hazard ratio of 1.5 with four interim analyses (one-sided α = 0.025 and β = 0.1), the rule calls for stopping for futility at the fourth analysis (with 80% of total information) if the observed hazard ratio is (just less than) 1.22 with 90% confidence interval (0.98, 1.52; P = 0.068). For a trial that is designed to show a 50% improvement in median survival, a 22% increase in median survival time is a meaningful clinical effect in many cases. Furthermore, the upper confidence limit includes the design target of 1.5 while the lower confidence limit excludes all but a negligible inferiority of the experimental arm. Individuals reading the results of the study may infer that the experimental treatment is worth pursuing even though it was not as effective as the study investigators may have hoped. This leads to questions as to whether stopping the trial early was a correct decision. This example reiterates the point that investigators designing a trial should carefully examine the monitoring boundaries they are considering to ensure that they

FUTILITY ANALYSIS

would be comfortable stopping the trial for futility for certain observed positive differences (22). In addition to the potential harm to the integrity of the ongoing study, a premature stopping may jeopardize completion of other ongoing trials addressing this or similar questions (38). The decision to stop (or not to stop) a large study that is addressing a major public health question is often challenged by a wider audience. Even after thorough deliberation, some may question the wisdom and implications of the ultimate decision. For example, the early stopping of NSABP B14 was criticized (39). Some interim monitoring approaches integrate efficacy and futility boundaries under a single framework (15, 18). This allows simultaneous control of the type I and type II error rates. Others take a more informal approach to futility and implicitly argue that because futility monitoring does not inflate type I error no formal adjustment is needed. In general, due to the complexity of the monitoring process, interim monitoring boundaries are considered to be guidelines rather than strict rules. This is especially relevant to the futility stopping that in practice is often based on a synthesis of efficacy, safety, and feasibility considerations both internal and external to the ongoing trial (by contrast, common efficacy monitoring scenarios are driven by strong internal evidence of benefit with respect to the primary clinical outcome). On a purely mathematical level, the integration of the efficacy and futility boundaries typically involves adjusting the upper boundary downward to compensate for the deflation of the type I error due to the possibility of futility stopping. If the futility boundary is not strictly adhered to, the overall type I error of the design is not maintained. Therefore, due to the different nature of the efficacy and futility stopping and the priority to control type I error in many settings, the downward adjustment of the efficacy boundary may not be justified. This can potentially become an issue when the study is intended to support approval by a regulatory agency (40). Thus far, an implicit assumption in the discussion has been that the design treatment effect θ A represents a minimally clinically meaningful benefit and that it is within

7

the range of biological plausibility. However, some RCTs are sized using a hypothesized treatment effect that is a compromise between a realistic target supported by clinical/biological evidence and the feasibility considerations (achievable sample size and timely completion of the study). As a result, some studies, especially those in rare diseases, are often sized using an unrealistic target treatment effect. When an interim futility analysis conditioned on the observed data suggests that the study is unlikely to be positive if continued to the planned sample size, this may to a large degree reflect the unrealistic target effect. In such situations, more conservative futility boundaries should be used. The exact cut-off value γ used in stochastic curtailment may vary depending on the context of the trial. However, the use of γ = 0.5 or higher (i.e., to stop a trial for futility with a conditional power of 0.5 or higher) seems to be inconsistent with the original design. Recall that this conditional power calculation is done under the alternative hypothesis such that at the beginning of the trial the conditional power is the unconditional power (0.9, or possibly 0.8). This high unconditional power reflects the supporting preliminary data and rationale that are used to justify the expending of effort and resources to set up and conduct a trial. Therefore, once the trial is started, the drop in power from 0.9 (or 0.8) to 0.5 might not provide sufficient evidence to stop the trial early and risk wasting the resources that went into setting up and conducting the trial. It is instructive to draw an analogy with the widely accepted O’Brien–Fleming efficacy boundary. As mentioned previously, the boundary corresponds to stopping with conditional power 0.5 under the null (CPk (0) = 0.5); that is, the ‘‘conditional type I error’’ is 0.5. Thus, for a trial designed with ‘‘unconditional’’ 0.05 (0.025) type I error, this rule requires a 10- (20)-fold increase in conditional power to justify stopping the trial for efficacy. Although the exact choice of γ depends on circumstances, values in the 0.1 to 0.3 range are generally more consistent with common trial designs.

8

FUTILITY ANALYSIS

In many cases, it may be difficult to identify a simple expression for a stopping boundary that has the desired properties throughout the trial. Many commonly used futility rules may appear to be too conservative or too liberal at different information/logistic stages of a study. A futility boundary based on one statistical approach can usually be defined through other formulations (e.g., under a general definition, any futility stopping rule can be expressed in terms of the general stochastic curtailment approach or a repeated testing of the alternative hypothesis approach [15]). This can be useful in developing futility rules with desirable operational characteristics, in elucidating these rules to the data monitoring committee members, and in presetting the study results to the clinical community (41). In addition, increased flexibility can be achieved by selecting different values for the conditional power cut-off γ (or, equivalently, selecting different significance levels for the alternative hypothesis testing formulation), depending on the proportion of available information or depending on where the trial is in terms of accrual and follow-up. For example, different values of conditional power can be used during [1] the accrual period and [2] the follow-up period when all of the patients are off treatment. This strategy adjusts for the different impact of stopping for the study patients and the results. The first scenario implies terminating accrual and possible treatment change for the patients who are still on the study medication; some data are lost, and the total design information is not reached. The second scenario means only releasing the study data early; complete follow-up data are generally available after an additional follow-up period. In general, we recommend [1] scheduling futility analyses at the same time as efficacy analyses (starting at 25% to 30% of the total information) (30), and [2] requiring a stronger level of evidence for stopping early in the trial. The responsibility of monitoring of RCTs is best handled by a data monitoring committee that is independent from the principal investigator and the sponsor and thus is free from potential conflicts of interest. A failure to provide transparent and independent futility monitoring may lead to controversy. For example, Henke et al. (42) reported results

from a randomized, placebo-controlled trial of erythropoietin for head and neck cancer. The paper was unclear as to who was doing the interim monitoring and stated that the drug company sponsoring the study had decided to omit the second of the two planned interim analyses. Two years after the scheduled date of the omitted interim analysis, the final analysis was carried out. It revealed a significant impairment in cancer control and in survival for the erythropoietin arm relative to the placebo arm: the locoregional progression-free survival (the primary endpoint) hazard ratio was 1.62 (P = 0.0008), and the overall survival hazard ratio was 1.39 (P = 0.02). Although the study outcome at the time of the omitted analysis is not available, it is hypothetically possible that if the omitted interim analysis were carried out the study would have closed for futility and the medical community would have had the benefit of such information 2 years earlier than they did. The fact that the decision to cancel the second interim analysis was made by the sponsor rather than by an independent data monitoring committee created an apparent potential conflict of interest (43). In addition to the futility considerations based on low conditional power (disappointing observed treatment effect), futility may also be affected by such factors as lower than expected event rates or slower than expected accrual (44). If the observed control arm event rate is lower than that used at the design stage (to calculate the sample-size/duration needed for the desired power), the study may not be able to accumulate enough information (within a reasonable time period) to achieve adequate power and reliably address its goals. For example, one of the considerations in the early termination of the aspirin component of the Physicians Health Study (45) was that, due to a fivefold lower than expected cardiovascular death rate, the trial would have to be extended another 10 years to achieve the prespecified power (46). Note that such post hoc unconditional power analyses are usually used in conjunction with a conditional power analysis (11). In summary, a clinical trial to which substantial time and resources have been expended should provide an unequivocal resolution of the primary question. Premature

FUTILITY ANALYSIS

stopping may have a major impact on the future of the intervention, clinical care, and research directions. Therefore, futility stopping rules should be consistent with the study objectives in requiring a degree of evidence that is sufficiently convincing to the relevant medical community. Futility procedures should be carefully outlined in the protocol. If a trial is stopped early, then a clear explanation of the procedure and justification should be reported. REFERENCES 1. J. H. Ware, J. E. Muller, and E. Braunwald, The futility index. Am J Med. 1985; 78: 635–643. 2. S. J. Pocock, When to stop a clinical trial. BMJ. 1992; 305: 235–240. 3. P. W. Armstrong and C. D. Furberg, Clinical trial data and safety monitoring boards. Circulation. 1995; 91: 901–904. 4. D. L. DeMets, S. J. Pocock, and D. G. Julian, The agonizing negative trend in monitoring of clinical trials. Lancet. 1999; 354: 1983–1988. 5. D. L. DeMets and J. H. Ware, Group sequential methods for clinical trials with one-sided hypothesis. Biometrika. 1980; 67: 651–660. 6. K. K. G. Lan, R. Simon, and M. Halperin, Stochastically curtailed tests in long-term clinical trials. Commun Stat Theory Methods. 1982; 1: 207–219. 7. J. Whitehead, The Design and Analysis of Sequential Clinical Trials, 2nd ed. Chichester, UK: Wiley, 1997. 8. C. Jennison and B. W. Turnbull, Group Sequential Methods with Applications to Clinical Trials. Boca Raton, FL: Chapman & Hall/CRC, 2000. 9. K. K. G. Lan and J. Wittes, The B-value: a tool for monitoring data. Biometrics. 1988; 44: 579–585. 10. D. O. Scharfstein, A. A. Tsiatis, and J. M. Robins, Semiparametric efficiency and its implications on the design and analysis of the group-sequential trials. J Am Stat Assoc. 1997; 92: 1342–1350. 11. Lachin JM and Lan SH, for the Lupus Nephritis Collaborative Study Group. Termination of a clinical trial with no treatment group difference: the lupus nephritis collaborative study. Control Clin Trials. 1992; 13: 62–79. 12. M. S. Pepe and G. L. Anderson, Two-stage experimental designs: early stopping with a negative result. Appl Stat. 1992; 41: 181–190.

9

13. B. R. Davis and H. J. Hardy, Upper bounds on type I and type II error rates in conditional power calculations. Commun Stat Theory Methods. 1990: 19: 3572–3584. 14. J. M. Lachin, A review of methods for futility stopping based on conditional power. Stat Med. 2005; 24: 2747–2764. 15. B. K. Moser and S. L. George, A general formulation for a one-sided group sequential design. Clin Trials. 2005; 2; 519–528. 16. S. C. Choi, P. J. Smith, and D. P. Becker, Early decision in clinical trials when treatment differences are small. Control Clin Trials. 1985; 6: 280–288. 17. D. J. Spiegelhalter, L. S. Freedman, and P. R. Blackburn, Monitoring clinical trials: conditional or predictive power. Control Clin Trials. 1986; 7: 8–17. 18. S. Pampallona and A. A. Tsiatis, Group sequential designs for one-sided and two-sided hypothesis testing with provision for early stopping in favor of the null hypothesis. J Stat Plan Inference. 1994; 42: 19–35. 19. S. K. Wang and A. A. Tsiatis, Approximately optimal one-parameter boundaries for group sequential trials. Biometrics. 1987; 43: 193–200. 20. P. C. O’Brien and T. R. Fleming, A multiple testing procedure for clinical trials. Biometrics. 1979; 35: 549–556. 21. S. J. Pocock, Interim analyses for randomized clinical trials: the group sequential approach. Biometrics. 1982; 38: 153–162. 22. B. Freidlin and E. L. Korn, A comment on futility monitoring. Control Clin Trials. 2002; 23: 355–366. 23. T. R. Fleming, D. P. Harrington, and P. C. O’Brien, Designs for group sequential tests. Control Clin Trials. 1984; 5: 348–361. 24. T. R. Fleming and L. F. Watelet, Approaches to monitoring clinical trials. J Natl Cancer Inst. 1989; 81: 188–193. 25. C. Jennison and B. W. Turnbull, Interim analyses: the repeated confidence interval approach. J R Stat Soc Ser B Methodol. 1989: 51: 305–361. 26. S. Wieand, G. Schroeder, and J. R. O’Fallon, Stopping when the experimental regimen does not appear to help. Stat Med. 1994; 13: 1453–1458. 27. S. S. Ellenberg and M. A. Eisenberger, An efficient design for phase III studies of combination chemotherapies. Cancer Treat Rep. 1985; 69: 1147–1152.

10

FUTILITY ANALYSIS

28. R. M. Goldberg, C. G. Moertel, H.S. Wieand, J. E. Krook, A. J. Schutt, et al., A phase III evaluation of a somatostatin analogue (octreotide) in the treatment of patients with asymptomatic advanced colon carcinoma. Cancer. 1995; 76: 961–966. 29. P. A. Burch, M. Block, G. Schroeder, J. W. Kugler, D. J. Sargent, et al., Phase III evaluation of octreotide versus chemotherapy with 5-fluorouracil or 5-fluorouracil plus leucovorin in advanced exocrine pancreatic cancer. Clin Cancer Res. 2000; 6: 3486–3492. 30. B. Freidlin, E. L. Korn, and S. L. George, Data monitoring committees and interim monitoring guidelines. Control Clin Trials. 1999; 20: 395–407. 31. D. J. Spiegelhalter, L. S. Freedman, and M. K. B. Parmar, Bayesian approaches to randomized trials. J R Stat Soc Ser A Stat Soc. 1994; 157: 357–416. 32. J. J. Dignam, J. Bryant, H. S. Wieand, B. Fisher, and N. Wolmark, Early stopping of a clinical trial when there is evidence of no treatment benefit: protocol B-14 of the National Surgical Adjuvant Breast and Bowel Project. Control Clin Trials. 1998; 19: 575–588. 33. D. A. Guinn, M. W. Atkinson, L. Sullivan, M. Lee, S. MacGregor, et al., Single vs weekly courses of antenatal corticosteroids for women at risk of preterm delivery: A randomized controlled trial. JAMA. 2001; 286: 1581–1587. 34. T. M. Jenkins, R. J. Wapner, E. A. Thom, A. F. Das, and C. Y. Spong, Are weekly courses of antenatal steroids beneficial or dangerous? JAMA. 2002; 287: 187–188. 35. P. J. Schwartz, How reliable are clinical trials? The importance of the criteria for early termination. Eur Heart J. 1995; 16(Suppl G): 37–45. 36. G. S. Omenn, G. E. Goodman, M. D. Thornquist, J. Balmes, M. R. Cullen, et al., Effects of a combination of beta carotene and vitamin A on lung cancer and cardiovascular disease. N Engl J Med. 1996; 334: 1150–1155. 37. K. E. Murphy, M. Hannah, and P. Brocklehurst, Are weekly courses of antenatal steroids beneficial or dangerous? [letter]. JAMA. 2002; 287: 188. 38. S. Green, J. Benedetti, and J. Crowley, Clinical Trials in Oncology. London: Chapman & Hall, 2002.

41.

42.

43.

44.

45.

46.

47.

48.

49.

50.

51.

(CHMP). Reflection Paper on Methodological Issues in Confirmatory Clinical Trials with Flexible Design and Analysis Plan. Draft. March 23, 2006. Available at: http://www.emea.eu.int/pdfs/human/ewp/ 245902en.pdf S. S. Emerson, J. M. Kittelson, and D. L. Gillen, On the Use of Stochastic Curtailment in Group Sequential Clinical Trials. University of Washington Biostatistics Working Paper Series, no. 243. Berkeley, CA: Berkeley Electronic Press, March 9, 2005. Available at: http://www.bepress.com/uwbiostat/paper243/ ¨ M. Henke, R. Laszig, and C. Rube, Erythropoietin to treat head and neck cancer patients with anemia undergoing radiotherapy: randomized, double-blind, placebo controlled trial, Lancet. 2003; 362: 1255–1260. B. Freidlin and E. L. Korn, Erythropoietin to treat anemia in patients with head and neck cancer [letter]. Lancet. 2004; 363: 81. E. L. Korn and R. Simon, Data monitoring committees and problems of lower-thanexpected accrual or events rates. Control Clin Trials. 1996; 17: 526–535. Steering Committee of the Physicians’ Health Study Research Group. Final report on the aspirin component of the ongoing Physicians’ Health Study. N Engl J Med. 1989; 321: 129–135. L. Friedman, C. D. Furberg, and D. L. DeMets, Fundamentals of Clinical Trials, 3rd ed. New York: Springer, 1999. S. S. Ellenberg, T. R. Fleming, and D. L. DeMets. Data Monitoring Committees in Clinical Trials. Chichester, UK: Wiley, 2002. A. Dmitrienko and M. D. Wang, Bayesian predictive approach to interim monitoring in clinical trials. Stat Med. 2006; 25: 2178–2195. K. K. Lan and D. M. Zucker, Sequential monitoring of clinical trials: the role of information and Brownian motion. Stat Med. 1993; 12: 753–765. P. K. Andersen, Conditional power calculations as an aid in the decision whether to continue a clinical trial. Control Clin Trials. 1987; 8: 67–74. O. M. Bautista, R. P. Bain, and J. M. Lachin, A flexible stochastic curtailing procedure for the log-rank test. Control Clin Trials. 2000; 21: 428–439.

39. R. Peto, Five years of tamoxifen-or more? J Natl Cancer Inst.. 1996; 88: 1791–1793.

FURTHER READING

40. European Medicines Agency, Committee for Medicinal Products for Human Use

Excellent overviews of the futility monitoring are provided in Friedman, Furberg, and DeMets

FUTILITY ANALYSIS 1996 (46), and Ellenberg, Fleming, and DeMets 2002 (47). A review of recent developments in Bayesian approach to futility monitoring is provided in Dmitrienko and Wang (48). A comprehensive review of the statistical methodology can be found in Whitehead (7), Jennison and Turnbull (8), and Emerson, Kittelson and Gillen (41). A detailed discussion of information quantification in various outcome settings is provided in Lan and Zucker (49). Practical applications of stochastic curtailment for the time to event endpoint are presented in Andersen (50) and Bautista et al. (51).

CROSS-REFERENCES Conditional power Interim analysis Group sequential designs Stopping boundaries Trial monitoring

11

GENERALIZED ESTIMATING EQUATIONS

the entire class of models addressed in their proposed framework. The theoretical justification of and the practical application of GLMs have since been described in many articles and books; McCullagh and Nelder (2) is the classic reference. GLMs address a wide range of commonly used models, which include linear regression for continuous outcomes, logistic regression for binary outcomes, and Poisson regression for count data outcomes. A particular GLM requires specification of a link function to characterize the relationship of the mean response to a vector of covariates, and specification of a function to characterize the variance of the outcomes in terms of the mean. The derivation of the iteratively reweighted least squares algorithm appropriate for fitting GLMs begins with the likelihood specification for the single parameter exponential family of distributions. Within the usual iterative Newton-Raphson algorithm, an updated estimate of the coefficient vector can be computed via weighted ordinary least squares. This estimation is then iterated to convergence, for example, until the change in the estimated coefficient vector is smaller than some specified tolerance. The GLM response, with or without the conditioning of predictors, is a member of the single parameter exponential family of distributions described by

JAMES W. HARDIN University of South Carolina, Columbia South Carolina

JOSEPH M. HILBE Arizona State University, Tempe Arizona

Parametric model construction specifies the systematic and random components of a model. Inference from maximum likelihood (ML)-based models relies on the validity of these specified components, and model construction proceeds from the specification of a likelihood based on some distribution to the implied estimating equation. In the case of ML models, an estimating equation is defined as the derivative of the log-likelihood function (with respect to one of the parameters of interest) set to zero where there is one estimating equation for each unknown parameter. Solution of the (vector-valued) estimating equation then provides point estimates for each unknown parameter. In fact, the estimating equation is so called because its solution leads to such point estimates. Two obvious approaches to the analysis of correlated data include fixed-effects and random-effects models. Fixed-effects models incorporate a fixed increment to the model for each group (allowing group-specific intercepts), whereas random-effects models assume that the incremental effects from the groups are perturbations from a common random distribution; in such a model, the parameter (variance components) of the assumed random-effects distribution is estimated rather than the much larger collection of individual effects. 1

f(y) = exp{[y θ − b(θ )]/φ + c(y, φ)} where θ is the canonical parameter, φ is a proportionality constant or scale, y is the response, and b(θ ) is the cumulant, the moments of which describe the mean and the variance of the random response variable. After introducing covariates and associated regression parameters, this density function leads to an estimated p × 1 regression coefficient vector β estimated by solving the estimating equation

GENERALIZED LINEAR MODELS

The theory and an associated computational method for obtaining estimates in which the response variable follows a distribution from the single parameter exponential family was introduced in Nelder and Wedderburn (1). The authors introduced the term generalized linear models (GLMs), which refers to

(β) =

n


i =

i=1

n


Xi T (yi − µi )/[φV(µi )]

i=1

[∂µi /∂ηi ] = 0(p×1)

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

(1)

2

GENERALIZED ESTIMATING EQUATIONS

In the estimating equation, Xi is the ith row of an n × p matrix of covariates X, µi = g(xi β), which represents the mean or expected outcome E(y) = b
(θ ). µi is a transformation of the linear predictor ηi = xi β via a monotonic (invertible) link function g(), and the variance V(µi ) is a function of the expected value proportional to the variance of the outcome Var(yi ) = φ V(µi ). If the link-variance pair of functions coincide with those functions implied by a specific member of the exponential family of distributions, then the resulting coefficient estimates are equivalent to maximum likelihood estimates. However, data analysts are not limited to only those pairs of link and variance functions. When selection of variance and link functions do not coincide to the canonical form of a particular exponential family member distribution, the estimating equation is said to imply the existence of a quasilikelihood, and the resulting estimates are referred to as maximum-quasilikelihood estimates. 2 THE INDEPENDENCE MODEL FOR CORRELATED DATA When observations are clustered because of repeated measurements on the sampling unit or because the observations are grouped by identification of a cluster identifier variable, the model is written in terms of the observations yit for the clusters I = 1, . . . , n and the within-cluster (repeated) observations t = 1, . . . , ni . The total number of observations is then N =  i ni . In this presentation, the clusters i are independent, but the withinclusters observations are assumed to be correlated. The individual-level model, which is otherwise known as the independence model, assumes no within-cluster correlation and is written by considering the n vector-valued observations yi as if they defined N independent observations. The independence model is a special case of correlated data models (such as those specified through GEEs). Although the independence model assumes that the repeated measures are independent, the model still provides consistent estimators in the presence of correlated data. This consistency is

paid for through inefficiency, but Glonek and McCullagh (3) show that loss of efficiency is not always large. As such, this simple model remains an attractive alternative because of its computational simplicity as well as its straightforward interpretation. In addition, the independence model serves as a reference model in the derivation of diagnostics for more sophisticated models for clustered and longitudinal data (such as GEE models). 3 ESTIMATING VARIANCE The validity of the (naive) model-based variance estimators depends on the correct specification of the variance; in turn, this value depends on the correct specification of the working correlation model. A formal justification for an alternative estimator known as the sandwich variance estimator is given in Huber (4) and discussed at length in Hardin (5) and Hardin and Hilbe (6). Analysts can use the independence model to obtain point estimates along with standard errors based on the modified sandwich variance estimator to ensure that inference is robust to any type of within-cluster correlation. Although the inference regarding marginal effects is valid (assuming that the model for the mean is correctly specified), the estimator from the independence model is not efficient when the data are correlated. It should be noted that, assuming independence is not always conservative, the modelbased (naive) variance estimates based on the observed or expected Hessian matrix are not always smaller than those of the modified sandwich variance estimator. Because the sandwich variance estimator is sometimes called the robust variance estimator, this result may seem counterintuitive. However, this result is easily understood by assuming negative within-cluster correlation that leads to clusters with both positive and negative residuals. The cluster-wise sums of those residuals will be small, and the resulting modified sandwich variance estimator will yield smaller standard errors based on squaring the group-wise sum of residuals than the model-based Hessian variance estimators based on squaring and summing each residual.

GENERALIZED ESTIMATING EQUATIONS

4 SUBJECT SPECIFIC VERSUS POPULATION-AVERAGED MODELS Two main approaches are used to deal with correlation in repeated or longitudinal data. The population-averaged approach focuses on the marginal effects averaged across the individuals. The subject-specific approach focuses on the effects for given values of the random effects by fitting parameters of the assumed random-effects distribution. The population-averaged approach models the average response for observations that share the same covariates (across all of the clusters or subjects), whereas the subjectspecific approach explicitly models the source of heterogeneity so that the fitted regression coefficients have an interpretation in terms of the individuals. The most commonly described GEE model was introduced in Liang and Zeger (7). This method is a population-averaged approach. Although it is possible to derive subjectspecific GEE models, such models are not commonly supported in commercial software packages and so do not appear nearly as often in the literature. The basic idea behind the populationaveraged approach is illustrated as follows. We initially consider the estimating equation for GLMs. The estimating equation, in matrix form, for the exponential family of distributions can be expressed as (β) =

n


i =

i=1

=

Xi T D[∂µi /∂ηi ]

i=1

V n


n


−1

(µi ) (yi − µi )/φ

Xi T D[∂µi /∂ηi ] V −1/2 (µi )I(n×n)

i=1

V −1/2 (µi ) (yi − µi )/φ = 0(p×1) (2) This equation corresponds to the independence model we have previously discussed. However, the specification of the identity matrix between the factors of the variance matrix signals the point at which secondorder variance (within-cluster correlation) can be introduced. Formally, Liang and Zeger (7) introduce a second estimating equation for the structural parameters of the working correlation

3

matrix. The authors then establish the properties of the estimators that result from the solution of these estimating equations. The GEE moniker was applied because the model is derived through a generalization of the GLM estimating equation; the second-order variance components are introduced directly into the estimating equation rather than appearing in consideration of a multivariate likelihood. Many major statistical software packages support estimation of these models, including R, SAS, S-PLUS, STATA, LIMDEP, SPSS, GENSTAT, and SUDAAN. R and S-PLUS users can easily find userwritten software tools for fitting GEE models, whereas such support is included in the other packages. 5 ESTIMATING THE WORKING CORRELATION MATRIX One should carefully consider the parameterization of the working correlation matrix because including the correct parameterization leads to more efficient estimates. We carefully consider this choice even if we employ the modified sandwich variance estimator for calculation of the standard errors of the regression parameters estimates. Although the use of the modified sandwich variance estimator assures robustness in the case of misspecification of the working correlation matrix, the advantage of more efficient point estimates is still worth the effort of trying to identify the correct structural constraints to place on the correlation matrix. No controversy surrounds the fact that the GEE estimates are consistent, but there is some controversy with regard to their efficiency. Typically, a careful analyst chooses some small number of candidate parameterizations. Pan (8) discusses the quasilikelihood information criterion measures for choosing between candidate parameterizations of the correlation matrix. This criterion measure is similar to the well-known Akaike information criterion. The most common choices for the working correlation R matrix are given by structural constraints that parameterize the elements of the matrix as provided in Table 1.

4

GENERALIZED ESTIMATING EQUATIONS Table 1. Common correlation structures Ruv = 0 Ruv = α Ruv = α|u−v | Ruv = α|u−v | if |u-v| ≤ k 0 otherwise Ruv = α (u,v) if |u-v| ≤ k 0 otherwise Ruv = α (u,v)

Independent Exchangeable Autocorrelated − AR1 Stationary (k) Nonstationary (k) Unstructured Values are given for u = v; Ruu

=

1.

The independence model admits no extra parameters, and the resulting model is equivalent to a generalized linear model specification. The exchangeable correlation parameterization admits one extra parameter. The most general approach is to consider the unstructured (only imposing symmetry) working correlation parameterization that admits M(M − 1)/2 − M extra parameters where M = maxi {ni }. The exchangeable correlation specification, which is the most commonly used correlation structure for GEEs, is also known as equal correlation, common correlation, and compound symmetry. The elements of the working correlation matrix are estimated using Pearson residuals, which are calculated following each iteration of model fit. Estimation alternates between estimating the regression parameters β, assuming the that the initial estimates of α are true, and then obtaining residuals to update the estimate of α, and then using new estimates of α to calculate updated parameter estimates, and so forth until convergence. GEE algorithms are typically estimated using the same routines used for estimation of GLMs with an additional subroutine called to update values of α. Estimation of GEE models using other correlation structures use a similar methodology; only the properties of each correlation structure differs. A schematic for representing how the foremost correlation structures appear is found below. Discussion on how the elements in each matrix are to be interpreted in terms of model fit can be found in Twisk (9) and Hardin and Hilbe (10).

Independent Exchangeable 1 0 0 0

1 0 0

1 0

1 p p p

1

1 p p

1 p

1

Stationary or M-Dependent (2-DEP) Autoregressive (AR-1) 1 p1 p2 0

1 p1 p2

1 p1

1

1 p p2 p3

1 p p2

1 p

1

1 p6

1

Nonstationary (2-DEP) Unstructured 1 p1 p2 0

1 p3 p4

1 p5

1

1 p1 p2 p3

1 p4 p5

6 EXAMPLE The progabide data are commonly used as an example to demonstrate the various GEE correlation structures. The data are available in Thall and Vail (11). The data are from a panel study in which four 2-week counts of seizures were recorded for each epileptic patient. The response variable, or dependent variable of interest, is seizure, which is a count that ranges from 0 to 151. Explanatory predictors include time (1 = followup; 0 = baseline), progabide (1 = treatment; 0 = placebo), and time × prog (an interaction term). The natural log of the time period, which ranges from 2–8, is entered into the model as an offset.

GENERALIZED ESTIMATING EQUATIONS

5

GENERALIZED LINEAR MODEL seizures

time progabide timeXprog cons lnPeriod

Coef.

.111836 .0275345 −.1047258 1.347609 (offset)

Robust Std. Err.

z

.1169256 .2236916 .2152769 .1587079

0.96 0.12 −0.49 8.49

P > |z|

0.339 0.902 0.627 0.000

[95% Conf. Interval]

−.1173339 −.410893 −.5266608 1.036547

.3410059 .465962 .3172092 1.658671

GEE MODEL ASSUMING INDEPENDENCE seizures

time progabide timeXprog cons lnPeriod

Coef.

.111836 .0275345 −.1047258 1.347609 (offset)

Robust Std. Err.

z

.1169256 .2236916 .2152769 .1587079

0.96 0.12 −0.49 8.49

P > |z|

0.339 0.902 0.627 0.000

[95% Conf. Interval]

−.1173339 −.410893 −.5266608 1.036547

.3410059 .465962 .3172092 1.658671

Estimated within-id correlation matrix R: r1 r2 r3 r4 r5

c1 1.0000 0.8102 0.6565 0.5319 0.4309

c2

c3

c4

c5

1.0000 0.8102 0.6565 0.5319

1.0000 0.8102 0.6565

1.0000 0.8102

1.0000

In all, 59 patients participated in the study. Patients are identified with the variable identification (id). A robust cluster or modified sandwich variance estimator, clustered on id, is applied to the standard errors of the parameter estimates. Because the response is a count, we use a Poisson regression to model the data. We first model the data using a generalized linear model, which assumes that the observations are independent. Because five observations were obtained per patient, the data are likely correlated (on id); therefore, we use a robust variance estimator clustered on id to adjust the standard errors for the extra correlation. We subsequently model the data using GEE with various correlation structures. Each type of structure attempts to capture the correlation in a specific manner. Only the table of parameter estimates and associated statistics is shown for each model. GEE models have an accompanying within-id correlation matrix displayed to show how the model is adjusted.

The independence model is identical to the generalized linear model; that is, the correlation in the data caused by the clustering effect of id is not adjusted by an external correlation structure. It is only adjusted by means of a robust variance estimator. The exchangeable correlation matrix is nearly always used with clustered, nonlongitudinal data. A single correlation parameter is associated with this structure, which means that each cluster is assumed to be internally correlated in a similar manner. Subjects are assumed to be independent. The autoregressive correlation structure assumes that within time intervals, the correlation coefficients decrease in value based on the respective power increase in measurement. The values are the same for each respective off diagonal. Theoretically, the second-level diagonal values are the squares of the first-level diagonals. The third-level diagonals are the cube of the first, with a likewise increase in power for each larger diagonal. A large working correlation matrix,

6

GENERALIZED ESTIMATING EQUATIONS

GEE MODEL ASSUMING EXCHANGEABLE CORRELATION seizures time progabide timeXprog cons lnPeriod

Coef.

Semi-robust Std. Err.

.111836 .0275345 −.1047258 1.347609 (offset)

P > |z|

z

.1169256 .2236916 .2152769 .1587079

0.96 0.12 −0.49 8.49

0.339 0.902 0.627 0.000

[95% Conf. Interval]

−.1173339 −.410893 −.5266608 1.036547

.3410059 .465962 .3172092 1.658671

Estimated within-id correlation matrix R: r1 r2 r3 r4 r5

c1 1.0000 0.7767 0.7767 0.7767 0.7767

c2

c3

c4

c5

1.0000 0.7767 0.7767 0.7767

1.0000 0.7767 0.7767

1.0000 0.7767

1.0000

GEE MODEL ASSUMING AUTOREGRESSIVE (AR-1) CORRELATION seizures time progabide timeXprog cons lnPeriod

Coef.

Semi-robust Std. Err.

.1522808 .019865 −.1292328 1.3128 (offset)

P > |z|

z

.1124191 .2135299 .2620191 .1631003

1.35 0.09 −0.49 8.05

0.176 0.926 0.622 0.000

[95% Conf. Interval]

−.0680567 −.3986458 −.6427809 .9931291

.3726183 .4383758 .3843153 1.632471

Estimated within-id correlation matrix R: r1 r2 r3 r4 r5

c1 1.0000 0.8102 0.6565 0.5319 0.4309

c2

c3

c4

c5

1.0000 0.8102 0.6565 0.5319

1.0000 0.8102 0.6565

1.0000 0.8102

1.0000

which represents more within-group observations, will have increasingly small values in the extreme off diagonals. If the actual correlation structure varies considerably from the theoretical, then one should investigate using another structure. The nonstationary correlation structure is like the stationary except that the offdiagonal values are not the same. We would want to use these structures if we want to account for measurement error at each time period or measurement level. We observe above the same stopping point as in the stationary structure.

The unstructured correlation structure assumes that all correlations are different. This structure usually fits the model the best, but it does so by losing its interpretability—especially for models that have more than three predictors. Note that the number of unstructured correlation coefficients is based on the number of predictors p. Therefore the number of coefficients equals p(p − 1)/2.

GENERALIZED ESTIMATING EQUATIONS

7

GEE MODEL ASSUMING STATIONARY(2) or M-DEPENDENT(2) CORRELATION seizures time progabide timeXprog cons lnPeriod

Coef.

Semi-robust Std. Err.

.0866246 .0275345 −.1486518 1.347609 (offset)

P > |z|

z

.1739279 .2236916 .2506858 .1587079

0.50 0.12 −0.59 8.49

0.618 0.902 0.553 0.000

[95% Conf. Interval]

−.2542677 −.410893 −.639987 1.036547

.4275169 .465962 .3426833 1.658671

Estimated within-id correlation matrix R: r1 r2 r3 r4 r5

c1 1.0000 0.8152 0.7494 0.0000 0.0000

c2

c3

c4

c5

1.0000 0.8152 0.7494 0.0000

1.0000 0.8152 0.7494

1.0000 0.8152

1.0000

GEE MODEL ASSUMING NONSTATIONARY(2) CORRELATION seizures time progabide timeXprog cons lnPeriod

Coef.

Semi-robust Std. Err.

.0866246 .0275345 −.1486518 1.347609 (offset)

P > |z|

z

.1739279 .2236916 .2506858 .1587079

0.50 0.12 −0.59 8.49

0.618 0.902 0.553 0.000

[95% Conf. Interval]

−.2542677 −.410893 −.639987 1.036547

.4275169 .465962 .3426833 1.658671

Estimated within-id correlation matrix R: r1 r2 r3 r4 r5

c1 1.0000 0.9892 0.7077 0.0000 0.0000

c2

c3

c4

c5

1.0000 0.8394 0.9865 0.0000

1.0000 0.7291 0.5538

1.0000 0.7031

1.0000

It is possible to use other correlation structures with GEE models, but the ones shown above are the most common. More details on these structures can be found in Hardin and Hilbe (10). 7

CONCLUSION

GEE models are an extension of generalized linear models. GLMs are based on likelihoods (or quasilikelihoods), which assume the independence of observations in the model. When this assumption is violated because of clustering or longitudinal effects, then an appropriate adjustment needs to be made to the

model to accommodate the violation. GEE is one such method; fixed and random effects models are alternatives to GEE. Likewise, hierarchical and mixed models have been used to deal with clustering and longitudinal effects, which bring extra correlation into the model. The GEE approach is generally known as a population-averaging approach in which the average response is modeled for observations

8

GENERALIZED ESTIMATING EQUATIONS

GEE MODEL ASSUMING UNSTRUCTURED CORRELATION seizures time progabide timeXprog cons lnPeriod

Coef.

Semi-robust Std. Err.

.0826525 .0266499 −.1002765 1.335305 (offset)

P > |z|

z

.1386302 .224251 .2137986 .1623308

0.60 0.12 −0.47 8.23

0.551 0.905 0.639 0.000

[95% Conf. Interval]

−.1890576 −.4128741 −.5193139 1.017142

.3543626 .4661738 .318761 1.653467

Estimated within-id correlation matrix R: r1 r2 r3 r4 r5

c1 1.0000 0.9980 0.7149 0.8034 0.6836

c2

c3

c4

c5

1.0000 0.8290 0.9748 0.7987

1.0000 0.7230 0.5483

1.0000 0.6983

1.0000

across clusters and longitudinal subjects. This method varies from the subject-specific approach, which provides information concerning individual observations rather than averages of observations. The method has been used extensively in clinical research and trials, and it is widely available in the leading software packages. REFERENCES 1. J. A. Nelder and R. W. M. Wedderburn, Generalized linear models. J. Royal Stat. Soc. Series A 1972; 135: 370–384. 2. P. McCullagh and J. A. Nelder, Generalized Linear Models, 2nd ed. London: Chapman & Hall, 1989. 3. G. F. V. Glonek and R. McCullagh, Multivariate logistic models. J. Royal Stat. Soc. Series B 1995; 57: 533–546. 4. P. J. Huber, The behavior of maximum likelihood estimates under nonstandard conditions. Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Vol. 1. Berkeley, CA: University of California Press, 1967, pp. 221–223. 5. J. W. Hardin, The sandwich estimate of variance. In: T. B. Fomby and R. C. Hill (eds.), Advances in Econometrics, Vol. 17. 2006, pp. 45–73. 6. J. W. Hardin and J. M. Hilbe, Generalized Linear Models and Extensions, 2nd ed. College Station, TX: Stata Press, 2007.

7. K.-Y. Liang and S. L. Zeger, Longitudinal data analysis using generalized linear models. Biometrika 1986; 73: 13–22. 8. W. Pan, Akaike’s information criterion in generalized estimating equations. Biometrics 2001; 57: 120–125. 9. J. W. R. Twisk, Applied Longitudinal Data Analysis for Epidemiology: A Practical Guide. Cambridge, UK: Cambridge University Press, 2003. 10. J. W. Hardin and J. M. Hilbe, Generalized Estimating Equations. Boca Raton, FL: Chapman & Hall/CRC Press, 2002. 11. P. F. Thall and S. C. Vail, Some covariance models for longitudinal count data with overdispersion. Biometrics 1990; 46: 657–671.

CROSS-REFERENCES Generalized linear models mixed-effects models sandwich variance estimator

GENERALIZED LINEAR MODELS

functional relationship between y and x. As η is linear in the parameters, it is also called the linear predictor. Beside the structural part, the model contains the residual ε capturing all variation that is not included in η, which is the stochastic variability of y. Conventionally, one assumes normality for ε with zero mean and variance σ 2 . With this in mind, the structural part of model (1) can be written as

¨ GORAN KAUERMANN

University Bielefeld Postfach 300131, Bielefeld, Germany

JOHN NORRIE University of Aberdeen Health Services Research Unit Polwarth Building, Foresterhill, Aberdeen, Scotland, United Kingdom

1

µ = E(y|x) = η

That is, the mean of the response y given the explanatory covariate x is related in a linear fashion to x through the linear predictor η. Even though Equation (1) is a cornerstone in statistics, it is not an appropriate model if normality of the residuals does not hold. This applies, for instance, if y is a binary response variable taking values 0 and 1. To accommodate such a response, data model (1) needs to be generalized, and the decomposition of the structural and the stochastic part plays the key role in the GLM. For the stochastic part, one assumes that y for a given linear predictor η follows an exponential family distribution. This step provides an elegant mathematical framework and embraces several commonly used distributions, including the normal (or Gaussian), binomial, Poisson, exponential, gamma, and inverse normal distributions. Generalization of the structural part results as follows. Note that for normal response data, both sides of Equation (2) take values in the real numbers. This is not the case for other distributions. If, for instance, y is Binomially distributed, then E(y|x) takes values in [0, 1], so that the set of possible values for the left- and right-hand sides of Equation (2) differ. To overcome this problem, one introduces a transformation function g(·) called the link function and generalizes (2) to

INTRODUCTION

1.1 Motivation Generalized linear models (GLMs) provide a flexible and commonly used tool for modeling data in a regression context. The unifying approach traces back to Nelder and Wedderburn (1), although its general breakthrough was about a decade later, initiated by the book of McCullagh and Nelder (2). The availability of software to fit the models increased the acceptance of GLMs (see Reference 3). The approach has had a major impact on statistical modeling technology with research on a wide range of extensions continuing today. Recent references include Myers, et al. (4) and Dobson (5), which are of an introductory level, and Fahrmeir and Tutz (6), which concentrates on extensions toward multivariate models. For a concise overview of GLMs, see also Firth (7). For clarity of presentation, the generalization of regression models starts with the classic linear model, which can be written as y = β0 + xβx + ε

(2)

(1)

with y as response, x as some explanatory variable(s), and ε as the residual having mean zero. Model (1) plays a central role in statistics and mathematics, developed in the early nineteenth century by Gauß and Legendre (see Reference 8). A more recent reference is Stuart and Ord (9). This statistical model comprises two components: a structural and a random part. The structural part is given by η = β0 + xβx and models the

g(µ) = g{E(y|x)} = η

(3)

The link function g(·) guarantees that the possible values of both sides of Equation (3) remain the real numbers. For a Binomially distributed response, a convenient choice for g(·) is the logit link g(µ) = logit(µ) = log(µ) − log(1 − µ), which uniquely transforms the set

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

GENERALIZED LINEAR MODELS

[0, 1] into the real numbers. Alternatively, the probit link g(µ) = −1 (µ) can be used, with (·) as standard normal distribution function. If the response follows a Poisson distribution, as is often the case for count data, E(y|x) is a positive number. In this case, a suitable link is taking g(·) as the log function. Both examples will be revisited later. Note that classic linear regression is included by taking g(·) as the identity function.

where x = 1 indicates the CBT + TAU group and x = 0 is the TAU group as reference category. The parameter estimates (with standard errors in brackets) are βˆ0 = −0.631(0.247) and βˆx = −0.881(0.393). For binary response, the odds ratio is a nicely interpretable quantity that results via exp(βx ) = exp{logit(E(y = 1|x = 1)) − logit(E(y = 1|x = 0))}

1.2 Example: Logistic Regression In a randomized controlled trial, the relapse in 144 schizophrenics was observed. Let y = 1 indicate the occurrence of a relapse measured by the positive and negative symptom score (PANSS) psychopathology scale, and let y = 0 otherwise. In this trial, there were two randomized treatments, cognitive behavioral therapy (CBT) on top of treatment as usual (TAU) against treatment as usual. The 144 schizophrenics (who fulfilled the American Psychiatric Association DSM-IV [1994] criteria for schizophrenia) were all receiving antipsychotic medication and considered relapse prone (relapse within last 2 years or living in stressful environment or alone or problems with taking antipsychotic medication). The PANSS is a 30-item observer-rated scale, scored as 1 (no psychopathology) to 7 (severe). The first seven items comprise the positive score, and a relapse was defined as admission to hospital or a 50% increase in positive score sustained for 1 week (for those with a positive item at baseline ≥3) or reaching a positive item ≥3 (if no positive item ≥3 at baseline). Further details of the methods and results are reported by Gumley et al. (10). Note that in that report, the relapse outcome was modeled using Cox regression, incorporating the time to relapse. The exposition here uses logistic regression, ignoring the time to relapse and concentrating on the occurrence of the event within the first 12 months only. During this 12-month followup, 13 out of 72 (18.1%) of those randomized to CBT + TAU relapsed, compared with 25 out of 72 (34.7%) in the TAU group. A logistic regression model was fitted with logit{E(y|x)} = β0 + xβx

Here, the odds ratio of relapse on CBT + TAU against TAU is estimated by 0.414 with confidence interval (0.191, 0.896). This example is revisited later. 2 GENERALIZED LINEAR MODELS 2.1 Modeling 2.1.1 Stochastic Component. For the stochastic part, one assumes that response y given predictor η = g(µ), with g(·) the link function and µ the mean of y given the explanatory covariates, follows the exponential family distribution. The likelihood can be written in the form 
yθ − b(θ ) + h(y, φ) (4) L(µ, φ) = exp wφ where b(·) and h(·) are known functions, φ and w are dispersion parameters, and θ = θ (µ) is called the natural or canonical parameter. The dispersion is thereby decomposed into two parts: a known weight w, which might vary between observations, and a dispersion parameter φ (sometimes referred to as a nuisance parameter), which is the same for all observations. The latter depends on the underlying distribution and is either known or unknown. The natural parameter θ and the mean µ are related via the first-order partial derivative of b(·) with respect to θ ; that is, ∂b(θ ) = µ(θ ) ∂θ

(5)

Thus, the natural parameter θ determines the mean value µ and vice versa. As the function is invertible, θ can also be written in dependence of µ as θ (µ).

GENERALIZED LINEAR MODELS

Important and commonly used statistical distributions belong to the exponential family [Equation (4)]. Taking, for instance, y ∈ {0, 1} as binary response, one obtains the likelihood function L(µ, ϕ) = µy (1 − µ)(1−y) = exp{yθ − b(θ )} (6) with θ = log{µ/(1 − µ)} and b(θ ) = log[1/{1 + exp(θ )}]. The joint likelihood for m independent identically distributed binary responses yi (a Binomial distribution) is written as
  yθ − b(θ ) m L(µ, ϕ) = exp + log , (7) y (1/m) with y = i yi /m and w = 1/m as the weight. For a Poisson distributed response variable y with mean µ, the likelihood can be written as L(µ, ϕ) =

µy exp{−µ} y!

= exp{y log(µ) − µ − log(y!)}

(8)

so that θ = log(µ) and b(θ ) = exp(θ ). Finally, for Normally distributed response y, one gets   L(µ, ϕ) = exp (2yµ − µ2 )/(2σ 2 ) + h(y, σ 2 ) with h(·) collecting terms that are not dependent on the mean, and θ = 2µ and b(θ ) = µ2 . Table 1 gives an overview of commonly used distributions belonging to Equation (4). 2.1.2 Variance Function. The exponential family implies a dispersion structure for response y. By standard results from exponential families, one finds by differentiation ∂ 2 b(θ ) 1 =: v(µ) Var(y|x) = wφ ∂θ 2

(9)

with v(µ) also called the variance function, which captures the dependence of the variance of y on the mean value. For binomial response, the variance function is v(µ) = µ (1 − µ), whereas for Poisson data, it is v(µ) = µ. For normal responses v(µ) = 1, reflecting that the variance does not depend on the mean. The role of v(µ) is then to allow variance heterogeneity to be automatically incorporated into the estimation.

3

2.1.3 Structural Component. For the structural part of the model, the mean value of y is linked to the linear predictor η via the link function g(·). This function must be appropriately chosen so that both sides of Equation (2) have the same range of possible values. The choice of g(·) can therefore in principle be made in isolation from the stochastic part with the only requirement being that g(·) maps the space of mean values for y to the real numbers and vice versa. However, natural candidates for the link function are suggested by the stochastic part of the model. Such link functions are called natural or canonical links. They are obtained by setting θ = η in Equation (4); that is, the linear predictor is set to equal the natural parameter of the exponential family. This result gives µ(θ ) = µ(η) from Equation (5), which is mathematically as well as numerically a convenient choice. Natural links enjoy widespread use in practice, with the logit link µ(θ ) = µ(η) = exp(η)/(1 + exp(η)) the natural choice for the Binomial response, whereas for Poisson response, the natural link is µ(θ ) = exp(θ ). For a normally distributed response y, the natural link is the identity function, so that the classic regression model (1) results as a special case. 2.2 Estimation 2.2.1 Maximum Likelihood Estimation. Taking the logarithm of Equation (4) yields the log likelihood l(µ, φ) = {yθ − b(θ )}/(wφ) + h(y, φ).

(10)

It is notationally convenient to parameterize the likelihood by β instead of µ. Assuming independent observations yi , i = 1, . . . , n, with corresponding covariates xi , one obtains from the individual log likelihood contributions [Equation (10)] the log likelihood function l(β, φ) =

n 

li (β, φ)

i=1

=

n  [{yi θi − b(θi )}/(wi φ) i=1

+ {h(yi , φ)}]

(11)

4

GENERALIZED LINEAR MODELS

Table 1. Examples of Exponential Family Distributions Distribution N(µ, σ 2 )

Normal

b(θ)

µ(θ) = b (θ)

v(µ) = b (θ)

φ

θ 2 /2

1

σ2

µ(1 − µ)

1

exp(θ) − log(−θ)

θ exp(θ) 1 + exp(θ) exp(θ) −1/θ

exp(θ) µ2

1 ν −1

−(−2θ)1/2

(−2θ)−1/2

µ3

σ2

θ(µ)

Binomial

B(1, µ)

Poisson Gamma Inverse Normal

P(µ) G(µ, ν)

 µ

µ log (1 − µ) log(µ) −1/µ

IN(µ, σ 2 )



log(1 + exp(θ))

1/µ2

with θi = θ (µi ) = θ {g−1 (ηi )} and linear predictor ηi = xi β. The maximum likelihood estimate for β is now obtained by maximizing Equation (11), which is achieved by setting the first derivative of l(β, φ) to zero. Simple differentiation using the chain rule leads to the score equation 0=

n  ∂ ηˆ i ∂ µˆ i ∂ θˆi (yi − µˆ i ) ∂β ∂η ∂µ

(12)

i=1

with ηˆ i = βˆ0 + xi βˆx , µˆ i = g−1 (ηˆ i ), and θˆi = θ (µˆ i ). If g(·) is chosen as natural link, one has θ = η so that (12) becomes 0=

n  ∂ ηˆ i i=1

∂β

2.3 Example: Multiple Logistic Regression (yi − µˆ i ).

(13)

Note that η = (1, x)β so that ∂η/∂β = (1, x)T . Hence, for natural links, Equation (13) has a simple structure. Moreover, with normal response y and natural link, Equation (13) reads as X T (Y − Xβ), with X T = ((1, x1 )T , . . . , (1, xn )T ) and Y = (y1 , . . . , yn ), which is easily solved analytically by βˆ = (X T X)−1 X T Y. In general, however, µi = µ(θi ) is a nonlinear function so that even for natural links, no analytic solution of Equation (13) will be available. Iterative fitting, for instance, based on Fisher scoring, is therefore required. 2.2.2 Fisher Scoring. A solution of Equation (12) can be found following a Newton Raphson strategy. Taking β (0) as starting value, then obtain an update β (t+1) from β (t) , t = 0, 1, 2, . . ., from the iterative procedure  β

(t+1)

=β

(t)

−

In practice, for ease of computation, it is convenient to replace the second-order derivative in Equation (14) by its expectation, otherwise known as the Fisher matrix. The resulting estimation routine is therefore called Fisher scoring. The procedure is sometimes also called iterative reweighted least squares, because the second-order derivative as well as the first-order derivative contain dispersion weights v(µi ), which depend on the parameter β and have to be updated iteratively using weighted least squares in each step.

∂ 2 l(β (t) , φ) ∂β∂β T

−1

∂l(β (t) , φ) . ∂β (14)

The subsequent example is taken from the textbook Brown (11). For 53 patients, nodal involvement of tumor was recorded as response variable y, indicating whether the tumor had spread to neighboring lymph nodes (y = 1) or not (y = 0). Explanatory variables are all binary, xage indicating whether the patient is 60 years or older (=1), xstage equal to 1 if the tumor has been classified as serious (=0 otherwise), xgrade describing the pathology of the tumor (=1 if assessed as serious, =0 otherwise), xxray if X-ray of the tumor led to a serious classification, and finally xacid indicating whether the serum acid phosphatase level exceeds the value 0.6. Parameter estimates of the multivariable model are shown in Table 2, with positive estimates indicating a tendency for an increased likelihood of tumor spread. The t-value in Table 2 is calculated by dividing the estimate by its standard deviation. The significance of the covariate can be assessed by comparing the t-value to standard normal distribution quantiles. By so doing, one should keep in mind the

GENERALIZED LINEAR MODELS Table 2. Parameter Estimates and Standard Deviation for Logistic Regression Example Covariate

Estimate

Standard Deviation

t-Value

Intercept age stage grade xray acid

−3.08 −0.29 1.37 0.87 1.80 1.68

0.98 0.75 0.78 0.81 0.81 0.79

−3.13 −0.39 1.75 1.07 2.23 2.13

5

Table 3. Parameter Estimates and Standard Deviation for Poisson Example

β0 βt β tt

model with linear time

model with quadratic time

5.178 (0.046) −0.0628 (.0057) –

5.011 (0.076) 0.0081 (.0238) −0.0043 (.0015)

quadratic fit is plotted in Fig. 1. The example is picked up again later. effects shown are conditional effects, meaning given the values of the other covariates. For instance, age exhibits a negative effect given the explanatory information contained in stage, grade, xray, and acid. It seems that some covariates show a significant effect, with their t-values taking values larger than 2 in absolute term, whereas, for instance, age does not show a significant effect, again conditional on the other variates in the model (stage, grade, xray, acid). A variable selection to classify covariate effects as significant and nonsignificant will be demonstrated in a subsequent section.

3

INFERENCE

As observed in the above three examples, it is necessary to have inferential tools at hand to select an appropriate model. This model contains only those covariates that need to be in the model (1) either because they are prognostically valuable or they are necessary for external reasons, or (2) they show significant effect. The following chapter discusses alternatives for the assessment of the latter point (2).

2.4 Poisson Regression Poisson regression can be useful for responses that are counts or frequencies. Consider, for example, the number of deaths from injury in childhood reported to the Registrar General in Scotland between 1981 and 1995. Morrison et al. (12) give full details of this data and explore socioeconomic differentials. Here the data are considered as overall number of deaths by year. Figure 1 shows the number of deaths per year, indicating a clear decline over the 15 years of observation, from a high of 173 in the late 1980s to a low of just over 50 in 1995. The number of deaths in any year are modeled by a Poisson distribution with mean µ as a function of calender year T, starting with 1981 (T = 1) to 1995 (T = 15). The simplest model would relate time T to the mean in a linear fashion: µ = exp(β0 + tβt ). In addition, a more complicated function, such as a quadratic function, can be used to accommodate nonlinearity: µ = exp(α + βt t + t2 βtt ). Table 3 provides the corresponding parameter estimates. The resulting log linear and log

3.1 Variance of Estimates 3.1.1 Fisher Matrix. Standard likelihood theory can be applied to obtain variances for the maximum likelihood estimates βˆ = (βˆ0 , βˆx ), which requires the calculation of the Fisher matrix by second-order differentiation of Equation (11). As ∂η/∂β = (1, x)T does not depend on β, one finds for the second-order derivative of l(β) an expression consisting of two components:  ∂ηi ∂θi v(µi ) ∂θi ∂ηi ∂ 2 l(β) =− T ∂β ∂η wi φ ∂η ∂β T ∂β∂β n

(15)

i=1

+

n  ∂ηi ∂ 2 θi ∂ηi yi − µi ∂β ∂η2 ∂β T wi φ

(16)

i=1

The first component in Equation (15) is a deterministic term, whereas Equation (16) is a stochastic one. Keeping in mind that

6

GENERALIZED LINEAR MODELS

170 150

(log)quadratic

Deaths

130 110

(log)linear

90 70 Figure 1. Accidental deaths in children in Scotland 1981 (T = 1) to 1995 (T = 15).

50 T Deaths

1

2

3

4

5

6

7

8

9

10 11 12 13 14 15

135 148 143 173 148 123 124 105 88 82 120 70 88 67 56

E(yi |xi ) = µi , the stochastic term has mean zero so that the Fisher matrix equals   ∂ 2 l(β) F(β) = E − ∂β∂β T =

n  ∂ηi ∂θi v(µi ) ∂θi ∂ηi ∂β ∂η wi φ ∂η ∂β T

(17)

i=1

For natural link functions, the structure nicely simplifies because with θ (η) = η, one has ∂θi /∂η = 1 and ∂ 2 θi /∂η2 = 0. Hence, Equation (16) vanishes and Equation (15) has the simple structure

F(β) =

n  ∂ηi v(µi ) ∂ηi ∂β wi φ ∂β T i=1

In particular, in this case, the expected and the observed Fisher matrices coincide. From standard likelihood theory, we know that the variance of the maximum likelihood estimate βˆ is asymptotically equal to the Fisher a ˆ = F(β)−1 , and the cenmatrix, that is, Var(β) tral limit theorem provides asymptotic normality: a βˆ ∼ N(β, F(β)−1 )

Therefore, standard statistical methods can be used for the parameter estimation. 3.1.2 Dispersion Parameter. The variance formulas above contain the dispersion parameter φ. Dependent on the stochastic model

being used, φ is either known or unknown. For instance, for binomial as well as for Poisson responses, one has φ = 1, which follows directly from Equation (6) and Equation (8), respectively. Hence, estimation of φ is not necessary. In contrast, for normally distributed response, component φ is the residual variance that is typically unknown and has to be estimated from the data. In principle, this estimation can be done by maximum likelihood theory based on the likelihood function (11). For the general case, however, maximum likelihood estimation of φ is not recommended (see Reference 2, p. 295) and instead a moment-based approach should be preferred. Based on variance formula (9), an estimate is found by φˆ =

n  (yi − µˆ i )2 /n. wi v(µˆ i )

(18)

i=1

For normal response models, the momentbased estimate is identical to the maximum likelihood estimate. This case is, however, special, and it does not hold in general. Moreover, replacing the denominator in Equation (18) by n − p, with p as number of parameters, reduces the bias occurring because of the use of fitted residuals (see Reference 2 for details). 3.2 Variable Selection It can be desirable within a statistical model with many explanatory covariates to only include covariates that make a substantive contribution to the understanding of the

GENERALIZED LINEAR MODELS

relationship between the covariates and the response. This question is one of parsimony, that is, to check whether all covariates are needed or whether a model with less covariates fits the data similarly well. A large amount of literature on variable selection in regression is available, with a particularly readable overview given by Harrel (13). Also, the issue of model inconsistency should be considered, particularly in the nonlinear model (see Reference 14) as one adds and subtracts covariates from the model. The problem can be tackled by testing the significance of subsets of covariate effects, and there are three different possibilities in common use. All three methods coincide for normal response y, and they are asymptotically equivalent for the general case. 3.2.1 Wald Statistics. Let the set of explanatory variables contained in x be split into xa and xb , that is, x = (xa , xb ). Model (3) is rewritten as g(µ) = xa βa + xb βb

(19)

For notational convenience, the intercept is included in xa . The focus of interest is to test the hypothesis βb = βb0 , with βb0 some given vector. The primary focus is usually on testing βb0 = 0. Taking advantage of standard likelihood theory, one can test this hypothesis using a Wald Statistic. Let the Fisher matrix F(β) be decomposed to  F(β) =

Faa (β) Fab (β) Fba (β) Fbb (β)



with submatrices matching to the dimensions of βa and βb . The Wald Statistic is then defined as wb = (βˆb − βb0 )T (F bb (β))−1 (βˆb − βb0 )

(20)

with F bb (β) as the bottom right matrix of F(β)−1 . For βb = βb0 , the test statistic follows asymptotically a chi-squared distribua tion with p degrees of freedom; i.e., wb ∼ χp , where p is the dimension of βb . 3.2.2 Score Statistics. A second test statistic is available from the score contributions, that is, the first derivatives of the

7

likelihood function. Let β˜ = (β˜a , βb0 ) be the maximum likelihood estimate in the hypothetical model with βb = βb0 . The Score Statistic is then defined by

sb =

˜ φ) ∂l(β, ∂βbT

˜ F bb (β)

˜ φ) ∂l(β, ∂βb

(21)

The idea behind the Score Statistic is that if βb = βb0 , then the score contribution at β˜ will be close to zero, which indicates that the maximum likelihood estimate βˆb in the complete model is close to βb0 . Asymptotic a likelihood theory shows sb ∼ χp . 3.2.3 Likelihood Ratio Statistic. Two models are called nested if the smaller one results in setting some parameters in the larger model to zero (or to some specific value). In this respect, the model with βb = βb0 is nested in the larger model with no constraints on βb . Testing βb = βb0 can then be pursued in a general framework using the Likelihood Ratio Statistics: ˜ φ) − l(β, ˆ φ)} λb = −2{l(β,

(22)

Again, standard likelihood theory indicates a the asymptotic distribution λb ∼ χp , assuming the smaller model to holds. 3.2.4 Remarks for Practical Application. It should be noted that the Wald Statistic is ˆ the estimate in the altercalculated from β, native model, whereas the Score Statistics ˜ the estimate in the is calculated from β, hypothetical model. For the Likelihood Ratio Statistics, both estimates are required for calculation. Standard software usually calculates the Wald Statistic or the likelihood ratio statistic only, whereas the score statistic is typically not provided. In terms of asymptotic behavior, all three test statistics follow standard likelihood theory (see Reference 15) based on the central limit theorem. The fundamental assumption for the asymptotics to hold even in small samples is that Fisher matrices are well conditioned and grow asymptotically at the same rate as the sample size. ‘‘Well conditioned’’ means here that both the design matrix of covariates x as well as the coefficients β

8

GENERALIZED LINEAR MODELS

are well behaved. The first is a common feature in any linear regression model, whereas the second is a special feature occurring in GLMs. For example, consider a binary covariate x taking values 0 and 1. If the binary response y = 1 for x = 1 and y = 0 for x = 0, then regardless of the design of x, the Fisher matrix will not have full rank because estimates tend to infinity in absolute terms. A thorough investigation of the effect of design on the asymptotic behavior is found in Fahrmeir and Kaufmann (16, 17). General guidelines on when the asymptotic approximations above are good enough to be reliable are difficult to derive, because the behavior depends strongly on the underlying stochastic model. Some discussion is found in Agresti (18, p. 246) or Fahrmeir and Tutz (6, p. 50). In the case of binomial and Poisson responses, Cressie and Read (19) introduced a power divergence statistics with improved small sample behavior. Further discussion of this point is also found in Santner and Duffy (20). As rule of thumb, one can take that asymptotic results can be satisfactorily applied in finite samples if (1) the covariate design is well conditioned and (2) the mean value µ is not close to to the boundary of its parameter space (for example, for a binary response y, we want µ to be not close to 0 or 1). If one of the two conditions is violated, the small sample behavior can be doubtful. 3.3 Example: Logistic Regression (Revisited) The different test statistics are exemplified with the logistic regression example on schizophrenic relapse from the first section by testing the treatment effect and are given in Table 4. All three quantities are of similar size and show a significant effect of treatment based on an asymptotic χ 2 distribution with one degree of freedom. The corresponding P-values are also shown in Table 4. 3.4 Deviance and Goodness of Fit The deviance is strongly linked to the likelihood ratio statistics. It results by comparing the model fit with the saturated model, that is, the model that fully describes the data via E(yi ) = ηi for i = 1, . . . , n. Let η˜ = (η˜ 1 , . . . , η˜ n ) be the maximizer in the saturated model,

Table 4. Test Statistics for the Logistic Regression Example Statistics

Estimate

P-Value

5.006 5.148 5.217

0.0253 0.0233 0.0224

Wald Score Likelihood ratio

and denote with li (ηi ) the likelihood contributions in Equation (11) evaluated at the linear predictor. The deviance is then defined as D(Y, µ) = 2φ



wi {li (η˜ i ) − li (ηˆ i )}

i

where Y = (y1 , . . . , yn ) and µ = (µ1 , . . . , µn ), The deviance is a helpful tool for model selection. Let, therefore, Ma denote the model with g(µ) = xa βa . Accordingly, Mab denotes the model built from g(µ) = xβ, with x = (xa , xb ) and β = (βa , βb ). The models are nested, and one can test Ma against Mab by making use of the asymptotic result that under the hypothetical model Ma a

D(Y, µMa ) − D(Y, µMab ) ∼ χ 2 (dfa −dfab )

(23)

where D(M) is the deviance calculated under the corresponding model and df a and df ab are the degrees of freedom for the two considered models, that is, n minus the number of estimated parameters. The deviance can also be used to validate the overall goodness of fit of a model. Assume that x is a metrically scaled covariate, which is included in the model in a linear form; that is, η = β0 + xβx . A general and important question is whether the model is in fact linear or whether the functional relationship is different. Note that theoretically the model could be checked by comparing its deviance with the deviance of the saturated model, that is, using D(Y, µ) − D(Y, Y). Based on Equation (23), this would follow a χ 2 distribution with dfa − 0 degrees of freedom. As n → ∞, one has dfa → ∞ and it results that the convergence rate is useless for practical proposes. This in fact forbids the deviance to be used in this way for checking the overall goodness of fit (see

GENERALIZED LINEAR MODELS

Reference 7 for more details). Instead, one can extend the model and test it against a more complex functional form. To check, for instance, the linear model η = β0 + xβx , one can test it against an extended polynomial model; e.g., η = β0 + xβx + x2 βxx + . . . . In this case, the difference of the deviance for both models follows a well-behaved asymptotic pattern and with Equation (23) the linear model can be checked. Over the last decade, such parametric model checks have increasingly been replaced by nonparametric methods where the alternative model is specified through g(µ) = m(x), with m(·) as unknown but smooth function to be estimated from the data. The idea can be found in Firth et al. (21); for further references and examples, one should consult Bowman and Azzalini (22). 3.5 Example: Multiple Logistic Regression (Revisited) For the nodal involvement example from above, a variable selection is desirable to include only those covariates in the model that are significant. Two methods are illustrated here. The first is a Wald test using Equation (20); the second is a likelihood ratio test stated in Equation (22). The latter is equivalent to a deviance comparison making use of Equation (23). Table 5 shows the deviance for several models. The models are nested, in that in each new row, a parameter of the precedent row is set to zero. Using Equation (23), the difference of the deviance in two consecutive rows is chisquared distributed with 1 degree of freedom. The corresponding P-value is also shown in Table 5. In the same form, the P-value for the Wald Statistic is provided, testing the significance of the effect, which is omitted from the preceding row. It would seem that the model including the stage of the tumor, the X-ray assessment, and the serum acid phosphatase level seems appropriate for describing the data.

9

4 MODEL DIAGNOSTICS AND OVERDISPERSION 4.1 Residuals It is a wise principle to check the model assumptions carefully before relying on inferential conclusions drawn from it. A first and helpful step in this direction is the numerical and graphical investigation of the fitted residuals yi − µˆ i . However, even though this is a manifest approach in standard regression with normally distributed residuals, in the generalized case, one is faced with two additional features. First, residuals are not homoscedastic, that is, they have different variances, and second, residuals can take clustered values so that visual investigation is cumbersome. Both of these problems have been adequately addressed. Pierce and Schafer (23) provide an early discussion of this point; a very readable account of regression diagnostics is provided by Atkinson (24). For a detailed discussion of residual based model checking, one may consult Cook and Weisberg (25). Heterogeneity of the residuals can be taken into account by standardization that leads to the Pearson residuals defined by yi − µˆ i εˆ Pi =

v(µˆ i )/wi

(24)

The sum of the squared Pearson residuals is also known as the Pearson statistics: X 2 = (yi − µˆ i )2 /{wi v(µˆ i )}. Beside the Pearson residuals, the deviance residual is in frequent use. This residual is defined as εˆ Di =

√ wi sign(η˜ i − ηˆ i )[2{li (η˜ i ) − li (ηˆ i )}]1/2 (25)

with sign(·) as the sign function and ηˆ i = βˆ0 + xi βˆx . Finally, a third type of residual has been suggested, given by transforming yi by the dµ function T(·) defined through T(·) = , v1/3 (µ) which leads to the so-called Anscombe residuals εˆ Ai =

ˆ T(yi ) − E{T(y i )}

Var{T(yi )}

10

GENERALIZED LINEAR MODELS

Standard software packages give both Pearson and deviance residuals. In contrast, Anscombe residuals are not well established in software packages, and therefore, they are less frequently used, principally because of the additional numerical burden resulting from estimating both the expectation and the variance of T(yi ). All three residuals correct for heterogeneity. For normal responses, they all reduce to the classic fitted residuals εˆ i = yi − µˆ i . For discrete data, however, the residuals exhibit an additional problem because they take clustered values. For instance, if yi is binary in a logistic regression on x, the residuals will mirror the binary outcomes of yi , which makes simple residual exploration by eye complicated or even impossible. To overcome this point, one can plot the residuals against x and apply some smoothing methods to explore whether there is any structure left that would indicate a lack of fit. Such approaches have been suggested by, among others, le Cessie and van Houwelingen (26), and they have become increasingly popular over the last couple of years. Bowman and Azzalini (22) can serve as a general reference.

4.2 Example: Multiple Logistic Regression (Revisited) For the nodal involvement data, the Pearson is Plotted against deviance residuals for the model, including xstage , xxray , and xacid as covariates. For better visibility, some small random noise has been added to avoid overlaying points in the plot. There is one observation with a large Pearson residual and a somewhat inflated deviance residual. This observation would need some extra consideration (Fig. 2).

4.3 Overdispersion For several distributions, the variance of y is completely determined by the mean. For such distributions, the dispersion parameter φ is a known constant and the variance of yi depends exclusively on v(µi ) and some known weights wi . Examples are the binomial distribution and the Poisson distribution. When using such distributions to model data, the analyst sometimes encounters the phenomena of overdispersion, in which the empirical residual variance is larger than the variance determined by the model. Overdispersion can arise from several sources. A possible explanation is that relevant covariates have been omitted in the model, which could, for instance, happen if covariates have not been recorded or are unmeasurable (or latent). Two commonly used modeling approaches correct for overdispersion. First, instead of assuming φ to be fixed and determined from the stochastic model, one allows φ to be unknown. One assumes Var(yi |ηi ) = v(µi )wi φ with wi as known weights, v(·) as a known variance function, and φ as an unknown dispersion parameter. Estimation can then be carried out with Equation (18). The resulting stochastic model with φ unknown is not generally in a tractable form. There are, however, some exceptions. For instance, if y is Poisson distributed with mean m, say, and m is assumed to be random following a gamma distribution with mean µ and variance τ , one obtains a negative binomial distribution for y. Integrating out the random mean m yields the overdispersed model with E(y) = µ and Var(y) = µφ, where φ = (1 + τ )/τ . More details are found in McCullagh and Nelder (2). Alternatively it has been suggested to accommodate overdispersion by fitting a mixed model with latent random effect. This random effect mimics the latent covariates that are assumed to be responsible for

Table 5. Deviance and Wald Statistics for Logistic Regression Example Model xray + acid + stage + grade + age xray + acid + stage + grade xray + acid + stage xray + acid

Deviance

P-Value for Difference of Deviance

P-Value for Wald Statistic

47.61 47.76 49.18 54.79

0.699 0.233 0.024

0.698 0.235 0.018

GENERALIZED LINEAR MODELS

11

Deviance Residuals

4

• 2

••• ••••

0

••••••••

•••• ••••• •••••• •• •• • •••

−2

•

−2

Figure 2. Pearson versus deviance residuals for nodal involvement data.

the observed overdispersion. Estimation is then carried out by numerical or approximative integration over the random effect (see Reference 27 for more details). The approach is attractive because it remains in the distributional framework. It requires, however, additional numerical effort. Tests on overdispersion can be constructed in two ways. First, one can use a Likelihood Ratio Statistic comparing the fitted models with and without overdispersion. For instance, in the Poisson/gamma model sketched above, one can test whether a random mean m has variance 0; i.e., τ = 0 (without overdispersion). The resulting test based on a log likelihood ratio is, however, nonstandard, because the hypothesis τ = 0 lies on the boundary of the parameter space τ > 0. Second, one can simply fit a model with a variance specified by Var(yi |xi ) = µi + αh(µi ), where h(·) is a prespecified, e.g., quadratic function. Overdispersion is then tested by α = 0. Details on tests of overdispersion are found in Cameron and Trivedi (28). Joint modeling of mean and dispersion is treated in more depth in Jørgensen (29). 4.4 Example: Poisson Regression (Revisited) In Table 6, the different goodness of fit measures are given for the Poisson data example from above. As can be observed, the difference between deviance and Pearson statistics

0 2 Pearson Residuals

4

Table 6. Goodness of Fit for Poisson Data Example

X2 D df

Model with Linear Time

Model with Quadratic Time

39.59 38.56 13

31.52 30.67 12

is negligible. Based on the difference of the goodness of fit statistics in the linear and quadratic model, the quadratic time effect seems significant with P-values 0.0045 and 0.0049 for the Pearson and deviance statistics, respectively. There is no free dispersion parameter for the Poisson distribution, and so for a wellfitting model, the ratio of the Pearson χ 2 to its degrees of freedom should tend toward unity. A simple correction for overdispersion, proposed by McCullagh and Nelder (2), is to inflate the standard errors of the parameter estimates by an estimated dispersion parameter, and so give some protection against over-interpreting the data because of inappropriately small confidence intervals around the parameter estimates. Based on Equation (18), this means

that confidence intervals are inflated by φˆ = X 2 /df . In the linear time model, this is 9.59/13 or 1.745, and

12

GENERALIZED LINEAR MODELS

in the quadratic model, it is 31.52/12 or 1.621. After this correction, the estimated standard error for the quadratic time parameter increases to 0.0015 × 1.621 = 0.0024. In terms of estimating the change in accidents over, say, a 10-year period, the first model, linear in time but with scale parameter fixed at 1, would estimate this as a 47% reduction (95% confidence interval 40% to 52%), whereas the linear model with scale parameter estimated would return the same point estimate, i.e., 47%, but now the 95% confidence interval would stretch from 35% to 56%. For the quadratic model with scale parameter estimated, the change for the first 10 years 1981 to 1991 would be estimated as a 39% reduction with a 95% confidence interval of 23% to 52%. 5

5.2 Quasi-Likelihood

EXTENSIONS

5.1 Multicategorial and Ordinal Models In the standard form of a GLM, one assumes that the response y is univariate. It can be readily extended to accommodate multivariate response variables, for example, if y is a categorical response with possible outcomes y ∈ {1, 2, . . . , k}. Recoding y by the dummy variables
1 if y = r y˜ r = 0 otherwise for r = 1, . . . , k − 1, the distribution of the resulting vector y˜ = (˜y1 , . . . , y˜ k−1 )T can be written in exponential family form exp{˜yT θ˜ − b(θ˜ )} with multivariate parameter vector θ˜ . Accordingly, the structural part of the model (3) is written in multivariate form

g(µ) = g{E(˜y|x)} = η

different influence on the separate cells of the discrete distribution. Multicategorical responses are often measured on an ordinal scale. In this case, it is plausible to model cumulated probabilities P(y ≤ r) = rl=1 E(˜yl ) for r = 1, . . . , k − 1. Taking a logit-based link leads to the Cumulative Logistic Model or Proportional Odds Model, respectively, defined by the link funk g(·) with components gr (µ) = log{P(y ≤ r)/P(y > r)}. The linear predictor should now be specified as ηr = β0r + xβx , mirroring a joint influence of covariate x on all categories of x but with separate category specific intercepts, with β0r ≤ β0r+1 for r = 1, . . . , k − 2. The model can also be interpreted via a latent univariate response variable as motivated in depth in McCullagh and Nelder (2); see also in Fahrmeir and Tutz (6).

(26)

with g(·) = {g1 (·), . . . , gk−1 (·)}T and η˜ = (η˜ 1 , . . . , η˜ k−1 )T as a multivariate linear predictor. The natural link is then gr (η) = log{P(y = r)/P(y = k)} with P(y = r) = E(˜yr ) and P(y = k) = 1 − k−1 yl ). For notational simplicl=1 E(˜ ity, the dependence on x is thereby omitted. For the linear predictor, one may set ηr = β0r + xβxr to allow covariate x to have a

The cornerstone of so-called quasi-likelihood estimation (see Reference 30) is that the variance function v(µ) is inherited from the assumed distribution, but otherwise the distributional assumption is given up. As a consequence, no proper likelihood function is available for estimation, and instead, estimating equations are used. The structural part is extended to accommodate both mean and variance. The latter also allows multivariate response models. The mean specification can be written in matrix form as E(Y|X) = µ(β), where Y = (y1 , . . . yn ) is the response vector and X is the design matrix comprising the rows (1, xi ). Note that the explicit dependence on parameters β is not now of primary interest, but it is retained for notational convenience. The distributional assumption is now expressed as Var(Y|X) = V(µ)Wφ, with W = diag(wi ) the matrix of known weights and V(µ) as the prespecified variance matrix. In particular, correlated observations can be incorporated by appropriate specification of nondiagonal entries in V(µ). In the case of independent errors, V(µ) simplifies to diag{v(µi )}. Based on estimating equation theory (see Reference 31), the best estimating equation for β becomes ˆ Y − µˆ −1 ∂ µ ˆ := {V(µ)W} s(β) ˆ ∂β T φ

(27)

GENERALIZED LINEAR MODELS

which can be solved by Newton Raphson or Fisher scoring in the same way as in a likelihood framework. An attractive feature of the approach is that score and information identities are inherited; that is, if the prespecified variance is the true variance, one gets E{s(β)} = 0,

E(−

∂s(β) ) = Var(s(β)) ∂β

This approach became fashionable in the early 1990s with the introduction of generalized estimating equations (GEEs) (see Reference 32). 6

SOFTWARE

The software package GLIM (see Francis et al. (1993)) was to some extent a precursor on the computational side of GLMs. Nowadays, however, most standard software packages are well equipped with fitting routines for GLMs. SPSS (www.spss.com SPSS Inc, 233 S. Wacker Dr, Chicago, IL 60606, USA) allows one to fit most parametric GLMs including multinomial and ordinal models (see GLM procedure). The same holds for SAS (www.sas.com SAS Inst Inc, 100 SAS Campus Dr Cary, NC 27513, USA; see, for instance, PROC, GENMOD, and MIXED). The widest range of fitting possibilities, in particular using nonparametric approaches, is found in Splus (www.insightful.com Insightful, ¨ Kagensti 17 4153 Reinach, Switzerland) or its open source clone R (www.r-project.org; see function glm). Beside these mainstream programs, several smaller and specialized tools provide estimation routines for GLMs. A comprehensive overview is found in Fahrmeir and Tutz (6). REFERENCES 1. J. A. Nelder and R. W. M. Wedderburn, Generalized linear models. Journal of the Royal Statistical Society, Series B 1972; B 34: 370– 384. 2. P. McCullagh and J. A. Nelder, Generalized Linear Models. 2nd ed. New York: Chapman and Hall, 1989. 3. M. Aitkin, D. Anderson, B. Francis, and J. Hinde, Statistical Modelling in GLIM. Oxford, U.K.: Oxford University Press, 1989.

13

4. R. H. Myers, D. C. Montgomery, and G. G. Vining, Generalized Linear Models: With Applications in Engineering and the Sciences. New York: Wiley, 2001. 5. A. J. Dobson, Introduction to Generalized Linear Models. Boca Raton, FL: Chapman & Hall/CRC, 2001. 6. L. Fahrmeir and G. Tutz, Multivariate Statistical Modelling Based on Generalized Linear Models. 2nd ed. New York: Springer Verlag, 2001. 7. D. Firth, Generalized linear models. In: D. V. Hinkley, N. Reid, and E. J. Snell (eds.), Statistical Theory and Modelling. London: Chapman and Hall, 1991. 8. S. M. Stigler, The History of Statistics: The Measurement of Uncertainty Before 1900. Cambridge, MA: Harvard University Press, 1986. 9. A. Stuart and J. K. Ord, Kendall’s Advanced Theory of Statistics, Vol. 2A: Classical Inference & the Linear Model. New York: Oxford University Press, 1999. 10. A. Gumley, M. O’Grady, L. McNay, J. Reilly, K. Power, and J. Norrie, Early intervention for relapse in schizophrenia: Results of a 12 month randomised controlled trial of cognitive behavioural therapy. Psychological Medicine 2003; 419–431. 11. B. Brown, Prediction analysis for binary data. In: R. Miller, B. Efron, B. Brown, and L. Moses (eds.), Biostatistics Casebook. New York: Wiley, 1980, pp. 3–18. 12. A. Morrison, D. H. Stone, A. Redpath, H. Campbell, and J. Norrie, Trend analysis of socio-economic differentials in deaths from injury in childhood in scotland, 1981–95. British Medical Journal 1999; 318: 567– 568. 13. F. E. J. Harrel, Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis. New York: Springer Series in Statistics, 2001. 14. J. Norrie and I. Ford, The role of covariates in estimating treatment effects and risk in long-term clinical trials. Statistics in Medicine 2002; 21(19): 2899–2908. 15. T. A. Severini, Likelihood Methods in Statistics. Oxford, U.K.: Oxford University Press, 2000. 16. L. Fahrmeir and H. Kaufmann, Consistency and asymptotic normality of the maximum likelihood estimator in generalized linear models. The Annals of Statistics 1985; 13: 342–368.

14

GENERALIZED LINEAR MODELS

17. L. Fahrmeir and H. Kaufmann, Asymptotic inference in discrete response models. Statistical Papers 1986; 27: 179–205. 18. A. Agresti, Categorical Data Analysis. New York: Wiley, 1990. 19. N. Cressie and T. R. C. Read, Multinomial goodness-of-fit tests. Journal of the Royal Statistical Society, Series B 1984; 46: 440– 464. 20. T. J. Santner and D. E. Duffy, The Statistical Analysis of Discrete Data. New York: Springer Verlag, 1990. 21. D. Firth, J. Glosup, and D. V. Hinkley, Model checking with nonparametric curves. Biometrika 1991; 78: 245–252. 22. A. W. Bowman and A. Azzalini, Applied Smoothing Techniques for Data Analysis: The Kernel Approach with S-Plus Illustrations. Oxford, U.K.: Oxford University Press, 1997. 23. D. A. Pierce and D. W. Schafer, Residulas in Generalized Linear Models. Journal of the American Statistical Association 1986; 81: 977–986. 24. A. Atkinson, Plots, Transformations and Regression. An Introduction to Graphical Methods of Diagnostic Regression Analysis. Oxford Statistical Science Series. Oxford, U.K.: Clarendon Press, 1985. 25. R. D. Cook and S. Weisberg, Applied Regeression including Computing and Graphics. New York: Wiley, 1999.

26. S. le Cessie and J. van Houwelingen, A goodness-of-fit test for binary regression models, based on smoothing methods. Biometrics 1991; 47: 1267–1282. 27. M. Aitkin, A general maximum likelihood analysis of variance components in generalized linear models. Biometrics 1999; 55: 218–234. 28. A. Cameron and P. Trivedi, Regression Analysis of Count Data. Cambridge, U.K.: Cambridge University Press, 1998. 29. B. Jørgensen, The Theory of Dispersion Models. Boca Raton, FL: Chapman & Hall, 1997. 30. R. W. M. Wedderburn, Quasi-likelihood functions, generalized linear models, and the gauss-newton method. Biomtrika 1974; 61: 439–447. 31. V. Godambe and C. Heyde, Quasi-likelihood and optimal estimation. International Statistical Review 1987; 55: 231–244. 32. Diggle, P. J., K.-Y. Liang, and S. L. Zeger, Analysis of Longitudinal Data. Oxford, U.K.: Oxford University Press, 1994. 33. B. Francis, M. Green, C. Payne. The GLIM System: Release 4 Claredon Press,

GENERIC DRUG REVIEW PROCESS

the application contains all the necessary components, then an ‘‘acknowledgment letter’’ is sent to the applicant to indicate its acceptability for review and to confirm its filing date. Once the application has been determined to be acceptable for filing, the Bioequivalence, Chemistry/Microbiology, and Labeling reviews may begin. If the application is missing one or more essential components, a ‘‘Refuse to File’’ letter is sent to the applicant. The letter documents the missing component(s) and informs the applicant that the application will not be filed until it is complete. No additional review of the application occurs until the applicant provides the requested data and the application is acceptable and complete. The Bioequivalence Review process establishes that the proposed generic drug is bioequivalent to the reference listed drug, which is based on a demonstration that both the rate and the extent of absorption of the active ingredient of the generic drug fall within established parameters when compared with that of the reference listed drug. The FDA requires an applicant to provide detailed information to establish bioequivalency. Applicants may request a waiver from performing in vivo (testing done in humans) bioequivalence studies for certain drug products where bioavailability (the rate and extent to which the active ingredient or active moiety is absorbed from the drug product and becomes available at the site of action) may be demonstrated by submitting data such as (1) a formulation comparison for products whose bioavailability is self evident, for example, oral solutions, injectables, or ophthalmic solutions where the formulations are identical or (2) comparative dissolution. Alternatively, in vivo bioequivalence testing that compares the rate and the extent of absorption of the generic versus the reference product is required for most tablet and capsule dosage forms. For certain products, a head-to-head evaluation of comparative efficacy based on clinical endpoints may be required. On filing an ANDA, an establishment evaluation request is forwarded to the Office of

An Applicant is any person (usually a pharmaceutical firm) who submits an abbreviated new drug application (ANDA), or an amendment or supplement to them, to obtain Food and Drug Administration (FDA) approval to market a generic drug product, and/or is any person (or firm) who owns an approved application or abbreviated application. A generic drug product is one that is comparable to an innovator drug product [also known as the reference listed drug (RLD) product as identified in the FDA’s list of Approved Drug Products with Therapeutic Equivalence Evaluations] in dosage form, strength, route of administration, quality, performance characteristics, and intended use. The ANDA contains data that, when submitted to the FDA’s Center for Drug Evaluation and Research, Office of Generic Drugs, provides for the review and ultimate approval of a generic drug product. Once approved, an applicant may manufacture and market the generic drug product provided all issues related to patent protection and exclusivity associated with the RLD have been resolved. Generic drug applications are termed ‘‘abbreviated’’ in that generally they are not required to include preclinical (animal) and clinical (human) data to establish safety and effectiveness. These parameters were established on the approval of the innovator drug product, which is the first version of the drug product approved by the FDA. An application must contain sufficient information to allow a review to be conducted in an efficient and timely manner. On receipt of the application, a pre-filing assessment of its completeness and its acceptability is performed by a project manager within the Regulatory Support Branch, Office of Generic Drugs. If this initial review documents that This article was modified from the website of the United States Food and Drug Administration http://www.fda.gov/cder/handbook/ by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

GENERIC DRUG REVIEW PROCESS

Compliance to determine whether the product manufacturer, the bulk drug substance manufacturer, and any outside testing or packaging facilities are operating in compliance with current Good Manufacturing Practice (cGMP) regulations as outlined in 21 CFR 211. Each facility listed on the evaluation request is evaluated individually and an overall evaluation for the entire application is made by the Office of Compliance. Furthermore, a preapproval product specific inspection may be performed on to assure data integrity of the application. The Chemistry/Microbiology review process provides assurance that the generic drug will be manufactured in a reproducible manner under controlled conditions. Areas such as the applicant’s manufacturing procedures, raw material specifications and controls, sterilization process, container and closure systems, and accelerated and room temperature stability data are reviewed to assure that the drug will perform in an acceptable manner. The Labeling review process ensures that the proposed generic drug labeling (package insert, container, package label, and patient information) is identical to that of the reference listed drug except for differences caused by changes in the manufacturer, distributor, pending exclusivity issues, or other characteristics inherent to the generic drug product (tablet size, shape or color, etc.). Furthermore, the labeling review serves to identify and to resolve issues that may contribute to medication errors such as similar sounding or appearing drug names, and the legibility or prominence of the drug name or strength. If at the conclusion of the Bioequivalence Review, it is determined that deficiencies exist in the bioequivalence portion of the application, a Bioequivalence Deficiency Letter is issued by the Division of Bioequivalence to the applicant. This deficiency letter details the deficiencies and requests information and data to resolve them. Alternatively, if the review determines that the applicant has satisfactorily addressed the bioequivalence requirements, the Division of Bioequivalence will issue a preliminary informational letter to indicate that no additional questions exist at this time. If deficiencies are involved in the Chemistry/Manufacturing/Controls, Microbiology,

or Labeling portions of the application, these deficiencies are communicated to the applicant in a facsimile. The facsimile instructs the applicant to provide information and data to address the deficiencies and provides regulatory direction on how to amend the application. Once the above sections are found to be acceptable, as well as the preapproval inspection and bioequivalence portion of the application, then the application moves toward approval. If after undergoing a final Office-level administrative approval review by all review disciplines no additional deficiencies are noted, then the application can be approved. A satisfactory recommendation from the Office of Compliance based on an acceptable preapproval inspection is required prior to approval. The preapproval inspection determines compliance with cGMPs and a product specific evaluation that concerns the manufacturing process of the application involved. If an unsatisfactory recommendation is received, a ‘‘not approvable’’ letter may be issued. In such a case, approval of the generic drug product will be deferred pending a satisfactory reinspection and an acceptable recommendation. After all components of the application are found to be acceptable, an ‘‘Approval’’ or a ‘‘Tentative Approval’’ letter is issued to the applicant. The letter details the conditions of the approval and allows the applicant to market the generic drug product with the concurrence of the local FDA district office. If the approval occurs prior to the expiration of any patents or exclusivities accorded to the reference listed drug product, a tentative approval letter is issued to the applicant that details the circumstances associated with the tentative approval of the generic drug product and delays final approval until all patent/exclusivity issues have expired. A tentative approval does not allow the applicant to market the generic drug product.

GENE THERAPY

Table 1. Gene therapy clinical trial indications (from www.wiley.co.uk/genemed/clinical)

SAMANTHA L GINN and IAN E ALEXANDER

Indication

Gene Therapy Research Unit Children’s Medical Research Institute and The Children’s Hospital at Westmead NSW, Australia

Cancer diseases Cardiovascular diseases Monogenic diseases Infectious diseases Neurological diseases Ocular diseases Other diseases Gene marking Healthy volunteers Total

Gene therapy was conceived originally as an approach to the treatment of inherited monogenic diseases; it is defined by the United Kingdom’s Gene Therapy Advisory Committee as ‘‘the deliberate introduction of genetic material into human somatic cells for therapeutic, prophylactic or diagnostic purposes (1).’’ Ongoing improvements in gene-transfer technologies and biosafety have been accompanied by a growing appreciation of the broader scope of gene therapy. Pathophysiological processes such as wound healing (2), chronic pain (3) and inflammation (4), cancer (5), and acquired infections such as HIV-1 (6) are now becoming realistic targets for this promising therapeutic modality. The first authorized gene transfer study took place at the National Institutes of Health (NIH) in 1989. In this marker study, tumor-infiltrating lymphocytes were harvested, genetically tagged using a retroviral vector, and reinfused with the intention of examining the tumor-homing capacity of these cells. This landmark study provided the first direct evidence that human cells could be genetically modified and returned to a patient without harm (7). Since then, over 1300 trials have been approved, initiated, or completed worldwide, which are performed predominantly in the United States (8, 9). Most studies have focused on cancer, with cardiovascular and monogenic diseases the next most frequent indications (Table 1). These predominantly early-phase trials have provided invaluable proof-of-concept for gene therapy by confirming that desired changes to the target cell phenotype can be achieved successfully. In most trials, however, an insufficient number of cells have been genetically modified to achieve therapeutic benefit.

Number of protocols 871 (66.5%) 119 (9.1%) 109 (8.3%) 85 (6.5%) 20 (1.5%) 12 (0.9%) 21 (1.6%) 50 (3.8%) 22 (1.7%) 1309

Notable exceptions to the lack of therapeutic efficacy have been the successful treatment of several primary immunodeficiencies that affect the hematopoietic compartment (10–16). 1 REQUIREMENTS FOR SUCCESSFUL THERAPEUTIC INTERVENTION The prerequisites for successful gene therapy are complex and disease-specific, but invariably they include the availability of an efficient gene delivery technology and an understanding of its properties, capacity and limitations, a detailed knowledge of the biology of the target cell population, and a precise understanding of the molecular basis of the target disease (Fig. 1). Although significant progress is being made in each of these areas, the ability to achieve efficient gene delivery has been described as ‘‘the Achilles heel of gene therapy (17).’’ 1.1 Gene Delivery Technology Gene delivery systems can be classified into two broad categories: nonviral physicochemical approaches and recombinant viral systems. The comparative strengths of nonviral approaches include ease of chemical characterization, simplicity and reproducibility of production, larger packaging capacity, and reduced biosafety risks (18, 19). Gene delivery, however, is relatively inefficient and

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

GENE THERAPY

Gene Delievery Technology

Gene Therapy Target Cell Biology

Understanding of the Target Disease

Figure 1. Venn diagram of requirements for successful therapeutic intervention.

the effects are often transient. Examples of nonviral systems include microinjection of individual cells, DNA-calcium phosphate coprecipitation, and the formulation of DNA into artificial lipid vesicles (20–24). In contrast, viral systems, which are commonly modified to render them replicationincompetent, are markedly more efficient and exploit favorable aspects of virus biology (17, 25–27). Viral vectors can be divided into two main categories: nonintegrating and integrating, based on the intracellular fate of their genetic material. These properties are important when considering the required duration of the treatment. Nonintegrating vectors are maintained extrachromosomally in the nucleus, whereas the genome of integrating vectors becomes incorporated into the host cell genome that provides the potential for stable long-term gene expression. Limitations include increased biosafety risks caused by contamination by replication-competent virus, the presence of potentially toxic viral gene products, and insertional mutagenesis when integrating vectors are used (28). Irrespective of vector type, another important limitation is the induction of unwanted immune responses directed against components of the delivery system and/or the encoded transgene product. These responses may be either cell-mediated or humoral and depend on several variables that include

the nature of the transgene, the vector and promoter used, the route and site of administration, the vector dose, and the host factors. Ultimately, host-vector immune responses have the potential to influence clinical outcomes negatively (29–32). Accordingly, the optimization of gene delivery systems and strategies to evade deleterious immune responses remains a fundamentally important challenge (33–35). Difficulty in producing high-titre vector preparations and constraints on packaging capacity are also drawbacks. Hybrid vectors that combine the advantageous features of more than one virus have also been developed, although their application has been largely in in vitro models (36–40). Despite these limitations, the relative efficiency of gene transfer has resulted in the predominant use of viral vectors in pre-clinical and clinical gene therapy studies up to the present time. The uses of gene-correction rather than gene-addition strategies, which include targeted recombination (41–43), antisense oligonucleotide-induced exon skipping (44), and RNA interference (45), are also being investigated. Such strategies will be particularly important in the context of dominant disease processes in which the simple addition of a functionally normal gene is insufficient. Currently, these approaches lack the efficiency required for human gene therapy applications. Finally, although recombinant viral vectors have the most immediate potential for clinical use, it is envisaged that these systems will be supplanted by hybrid and derivative systems that combine the simplicity and safety of nonviral gene delivery with favorable aspects of viral biology. Virosomes are a prototypic example of such a system and among these, the Hemagglutinating Virus of Japan liposome has been the most extensively investigated (46, 47). 1.2 Target Cell Biology Each viral vector system possesses a unique set of biological properties (Table 2), and their use is governed largely by the biology of the target cell. For example, integration provides the molecular basis for stable long-term gene expression as would be required for the

GENE THERAPY

3

Table 2. Properties of widely used viral vector systems Vector

Adenovirus

Retrovirus∗

Lentivirus†

AAV

Genome Insert capacity Location in cell nucleus Cell-cycle dependent gene transfer Duration of transgene expression Functional titre (per mL) Immunogenicity

dsDNA 7 to 30 kb extrachromosomal

ssRNA 7 kb integrated

ssRNA 10 kb integrated

no

yes

no

ssDNA 4.8 kb integrated/ extrachromosomal no

transient

long-term

long-term

long-term

≥ 1012

≥ 107

≥ 109

≥ 109

high

low

low

low

AAV, adeno-associated virus. ∗ Most commonly derived from Moloney-Murine Leukemia virus (MoMLV). † Most commonly derived from HIV-1.

treatment of genetic disease in replicating cell populations. Integration also provides the potential for gene-modified cells to be expanded from a modest number of progenitors. This feature is of paramount importance when the genetic modification of cells capable of enormous proliferative expansion is required. This feature has been powerfully illustrated in the first successful treatment of a genetic disease by gene therapy (10, 11). In this study, a selective growth and survival advantage was conferred on hematopoietic stem cells following transduction with a retroviral vector based on the Moloney murine leukemia virus. Despite the advantages of viral integration, nonintegrating vectors are potentially effective if the target cell is nondividing. Efforts are also being made to develop integration-deficient forms of integrating vectors in an attempt to increase biosafety (48). Additional constraints, such as the replication state of the target cell or whether the target cell is amenable to ex vivo manipulation define vector choice and gene transfer protocols even more. For example, vectors based on lentiviruses and adeno-associated viruses can modify postmitotic and nondividing cells (49–52), which makes them of particular interest for targets such as muscle and the nervous system.

1.3 Disease Pathophysiology Before a gene-therapy approach can be considered feasible for a disease or physiological process, a precise understanding of the underlying molecular basis is required. The requirement for transient or persistent transgene expression must also be considered. For example, the extrachromosomal nature of adenoviral vectors has the potential to limit the duration of gene expression by dilutional loss during cell division (53). In some contexts, in which only transient gene expression is required, such as anticancer gene therapy, this may be a positive attribute. The pathophysiology of the target disease also defines the number of cells that must be successfully gene-modified to achieve the desired therapeutic effect. For example, it is anticipated that for the treatment of hemophilia B, levels of Factor IX as low as 1% of normal will be therapeutic (54). For some more demanding disease phenotypes, expansion of gene-modified cells through in vivo selection is one strategy by which the fundamental challenge of gene-modifying sufficient cells to achieve therapeutic benefit may be overcome. Ultimately, however, the development of more efficient gene delivery technologies is required to allow the effective treatment of the many human diseases and pathophysiological processes that are potentially amenable to gene therapy.

4

2

GENE THERAPY

PRECLINICAL RESEARCH

Before a clinical gene therapy protocol can be considered for human application, extensive preclinical testing is required. The data generated is vital in establishing whether the target cell can be safely gene-modified to produce the phenotypic changes required for disease correction. This involves years of preclinical experimental testing progressing from tissue culture models to small animal models (most commonly mice), and finally to large animal models when feasible. 2.1 In Vitro Studies An important first step in establishing proofof-concept data for a clinical gene therapy protocol is provided by in vitro studies. The manipulation of mammalian cells in culture can help define several important experimental parameters. Taking into account the biology of the target cell, several gene transfer approaches could potentially be available. For example, if neuron-targeted gene transfer is required, several recombinant viral vector systems are available, such as those based on adeno-associated virus, herpes simplex virus, and lentiviruses (55–60). Using cells in culture, it is relatively easy to select the vector system that is most effective in genetically modifying the cell type of interest. Important parameters that can be determined in vitro include the tropism of the virus for the relevant target cell population, the minimum vector dose required to achieve the desired phenotypic changes, the level and the duration of transgene expression, and vector toxicity. In vitro systems also allow aspects of expression cassette design to be examined such as the use of tissue-specific promoters or regulated gene expression. Immortalized cell lines are commonly used for such studies, but frequently they are transduced more readily than primary cells and do not consistently model the challenge of transducing specific target cell populations in vivo. Culture of primary cells, and tissue explants where possible. Therefore, is preferable to offer a more realistic representation of the target cell population in its native state before proceeding to animal models.

2.2 Animal Models Another prerequisite for successful gene therapy is the availability of an animal model that approximates the human disease for preclinical testing. Indeed, successful phenotype correction in mouse models of human disease is now frequently reported (61–66). Up to the present time, these successes have rarely been replicated in large animal models or human clinical trials. The explanation for this is primarily quantitative. Success in larger subjects demands that proportionally greater numbers of target cells be gene-modified to reach a therapeutically meaningful threshold. The average child, for example, is approximately 1000-fold bigger than a mouse, and, therefore, presents a correspondingly greater gene transfer challenge. In addition to the challenge of size, other factors, such as species-specific differences, exert potent effects in some contexts. Animal models also provide valuable safety data required by regulatory bodies before approving early-phase human trials, but these models do not always accurately predict adverse effects subsequently observed in human subjects. For example, in a therapeutically successful gene therapy trial for the X-linked form of severe combined immunodeficiency (SCID-X1) 4 of 11 patients developed a leukemia-like illness as a direct consequence of retroviral insertion into chromosomal DNA (28). The risk of insertional mutagenesis when using integrating vectors had long been recognized, but formerly considered low because retroviral vectors had previously been employed extensively without incident in animal models and in almost 300 documented human clinical protocols (9). Interestingly, concurrent with the above report of vector-mediated insertional mutagenesis in humans, the first report of essentially the same phenomenon in mice was published (67). Collectively, these events illustrate the inherent challenge in predictive safety testing, whether in animal models or in early phase human clinical trials. Preferably such tests must be configured with specific adverse events in mind, and where possible, in a manner that accommodates the possible contribution of the particular disease for which gene therapy is being contemplated.

GENE THERAPY

3 CONDUCTING A HUMAN CLINICAL GENE THERAPY TRIAL In comparison with drug-based clinical trials, several additional factors must be considered when undertaking a human gene therapy application. These include additional layers of regulatory oversight, ethical considerations related to the genetic modification of the subject’s cells and availability of appropriate skills, infrastructure, and reagents. 3.1 Regulatory Oversight Clinical research with genetic material poses safety and methodological challenges not shared by other forms of human investigation. As a result, several regulatory requirements must be satisfied before human studies that involve gene transfer can be initiated. In most countries, compared with requirements for pharmaceutical products, these requirements are achieved through an additional layer, or layers, of regulation. Depending on the host country, regulatory oversight can be complicated even more by the fact that existing regulatory frameworks have evolved for more conventional therapeutic products. As for all human clinical research gene therapy, trials must also be conducted according to a set of standards referred to as Good Clinical Practice that are based on the Declaration of Helsinki (68). In the United States, it is a federal requirement that clinical protocols that involve the administration of recombinant DNA products be reviewed and approved by filing an investigational new drug application with the Food and Drug Administration (FDA). In addition, applications must be approved by local institutional human ethics and biosafety committees. The key regulatory issues for U.S.-based clinical gene therapy trials have been reviewed by Cornetta and Smith (69). In the United Kingdom, gene therapy applications are similarly regulated by the Medicines and Healthcare Products Regulatory Agency, the Gene Therapy Advisory Committee and local institutional committees. This regulatory complexity is particularly burdensome in the context of multinational trials and is a major driver behind efforts for the global harmonization.

5

Such efforts will not only facilitate international studies but also improve data quality and participant safety (70). 3.2 Special Ethical Considerations In contrast to drug-based clinical trials, several special ethical issues must be considered for human gene transfer studies. These issues include the possibility of inadvertent germ-line transmission and, depending on the type of delivery vehicle used, the ability to introduce lifelong modifications to the subject’s chromosomal DNA, the latter resulting in the need for long-term clinical follow-up. Currently, only somatic cell gene therapy protocols have been initiated. The use of germline manipulation, in which the genomes of germ cells are deliberately modified with the likely effect of passing on changes to subsequent generations, is opposed at this time (71). Although genetic manipulation of the human germ-line is illegal in many countries, this consensus is not unanimous and its use remains the subject of vigorous debate (72, 73). For any research team who attempt to develop a new medical treatment, patient safety is of paramount importance, and the decision to proceed with a gene therapy approach requires a careful balancing of the associated risks and benefits. For example, bone marrow transplantation from an HLAmatched sibling donor is the treatment of choice for diseases such as SCID-X1. Unfortunately, however, this option is available for only one in five affected infants. The alternative is transplantation from a haploidentical or matched unrelated donor, and it caries a substantial risk of severe immunologic complications and of early mortality (74). For these children, a gene therapy approach may carry a lower risk even when possible adverse events associated with gene therapy, such as leukemia induction through insertional mutagenesis, are taken into account (75). Another ethical concern for gene therapy trials is the enrollment of infants and children given their inability to provide informed consent. Although it may be preferable to undertake early phase clinical trials in adults, many severe disease phenotypes are restricted to the pediatric age group, or

6

GENE THERAPY

where meaningful intervention would only be possible early in the course of the target disease. Examples include SCID-X1 (10, 76) and cystic fibrosis (77). Accordingly, depending on the disease context, equally potent counterbalancing ethical arguments are in favor of early phase trials in the pediatric population. Another important consideration is whether a need exists for long-term monitoring of the subject after the gene transfer protocol. Parameters that include the ability of the delivery vehicle to integrate into the genome, the site of integration, vector persistence, biology of the target cell, and transgene-specific effects all influence the risk associated with the treatment. If no vector persistence exists, the risk is analogous to that of any new drug, and long-term follow-up may not necessarily be required (78). 3.3 Skills, Infrastructure, and Reagents To undertake a gene therapy clinical trial, a research team requires access to specialized facilities as well as appropriately trained staff to perform procedures in accordance with required standards. In most countries, the rigor of these requirements increases in late-phase trials to the level of good manufacturing practice. For each gene therapy protocol, the set of skills required are governed largely by the biology of the target cell. For example, in trials that involve gene transfer to hematopoietic stem cells, an ex vivo approach is the method of choice. This approach requires a medical team to harvest the subject’s bone marrow and personnel who can maintain the target cells in sterile culture for up to five days after harvest (10). This approach also requires the availability of an on-site clean-room to perform the cellular manipulations. Alternatively, it might be necessary to deliver the vector directly to the target cell in vivo. Examples of this approach include gene transfer to organs such as the eye or brain or in gene therapy protocols that deliver oncolytic agents to a solid tumor. Experimental products used for gene transfer studies are often complex and difficult to characterize completely in comparison with conventional pharmaceutical agents, which is true particularly for virus-based gene delivery systems that are also challenging to produce on a large-scale and cannot

be sterilized by autoclaving or radiation. Biological variability may also result from the packaging of empty virions, titre differences between different production runs, and loss of titre during storage. The propensity of vectors to undergo inadvertent sequence alteration during production through mechanisms such as recombination or via transcription by error prone polymerases (79) must also be monitored. 4 CLINICAL TRIALS Although originally conceived as a treatment for monogenic diseases, the major indication addressed by gene therapy trials to date has been cancer (66.5%, Table 1). This finding is predominantly caused by the poor prognosis of many cancer phenotypes that make the risk/benefit ratio more favorable for experimental intervention. Although initial trials have been largely unsuccessful, some positive outcomes have occurred. For example, in 2006, Morgan et al. (5) have observed cancer regression in patients who received autologous peripheral blood lymphocytes when modified by a retroviral vector to express a T cell receptor. Although regression was observed in only 2 of 15 patients (13%), which is considerably lower than the 50% response rate achieved when patients received tumorinfiltrating lymphocytes (TILs) in a similar trial (80), this method may prove useful in instances in which TILs are not available. Cardiovascular and monogenic diseases are the next most frequently addressed indications with 119 (9.1%) and 109 (8.3%) trials, respectively, approved worldwide (Table 1). Shortly after the first authorized gene transfer trial was undertaken in 1989 (7), the first therapeutic trial that involved two children who suffered from a severe combined immunodeficiency caused by adenosine deaminase deficiency (ADA-SCID) was approved. This trial was unsuccessful for several reasons that include maintaining the patients on PEG-ADA therapy and using patient T lymphocytes as the target cell population (81). Removal of PEG-ADA coupled with a myeloablative conditioning regime and the targeting of hematopoietic stem cells with an improved transduction protocol resulted

GENE THERAPY 2007 2006 2005 2004 2003 2002 2001 2000 1999 1998 1997 1996 1995 1994 1993 1992 1991 1990 1989 Unknown

7

33 97 98 95 81 89 108 95 116 68 82 51 67 38 37 14 8 2 1 129

Figure 2. Number of gene therapy clinical trials approved worldwide from 1989 to 2007 (reprinted with permission from http://www.wiley.co.uk/genmed/clinical/).

in the successful treatment of patients in subsequent trials for ADA-SCID (13, 14), which highlighted the need to impart a positive growth or survival advantage to the transduced cells. After initiation of the first therapeutic trial, a progressive increase occurred in the number of gene therapy trials approved in the following years (Fig. 2). This increase slowed briefly in the mid-1990s after an NIH review committee co-chaired by Stuart Orkin and Arno Motulsky concluded that ‘‘Significant problems remain in all basic aspects of gene therapy. Major difficulties at the basic level include shortcomings in all current gene transfer vectors and an inadequate understanding of the biological interaction of these vectors with the host (82).’’ This trend toward increasing numbers of gene therapy trials leveled off after 1999 coincident with report of the first severe adverse event as a direct consequence of gene therapy (83). In a Phase I dose escalation study that investigated the safety of an adenoviral vector for the treatment of ornithine transcarbamylase deficiency, a young adult trial participant died as a result of a severe systemic inflammatory reaction to the injected adenoviral vector. This incident resulted in the suspension of all trials at the host institution by the

FDA, and a senate subcommittee investigation that also looked more broadly at clinical trial safety and reporting across the United States (84). In light of the information that emerged, a renewed emphasis exists on critical evaluation of risk/benefit ratios for trial participants and comprehensive reporting of trial outcomes and adverse events. To date, gene therapy trials have been performed in 28 countries (Fig. 3) spanning five continents. These trials have been most comprehensively reviewed by Edelstein et al. (8). Most trials have been conducted in the United States with 864 trials (66%) followed by the United Kingdom (150 trials, 11.5%). Viral vectors are the most frequently used gene delivery system (Table 3) because of their superior gene transfer efficiency over nonviral methods. Of the viral gene delivery systems available, adenoviral and retroviral vectors (derived from murine retroviruses) have been most commonly used accounting for 24.9% and 23.1% of trials, respectively (Table 3). Lentiviral vectors derived from Human Immunodeficiency virus Type 1 (HIV1) are now being used clinically (85) after safety considerations relating to the inherent properties of the parent virus were addressed. Because of differences in their integration site patterns, lentiviral vectors may offer a

8

GENE THERAPY

Figure 3. Geographical distribution of gene therapy clinical trials (reprinted with permission from http://www.wiley.co.uk/genmed/clinical/). Table 3. Gene delivery systems in clinical trial use (from www.wiley.co.uk/genemed/clinical) Vector

Number of protocols

Adenovirus Retrovirus Naked/plasmid DNA Lipofection Vaccinia virus Poxvirus Adeno-associated virus Herpes simplex virus Poxvirus and vaccinia virus RNA transfer Lentivirus Flavivirus Gene Gun Others Unknown Total

326 (24.9%) 302 (23.1%) 239 (18.3%) 102 (7.8%) 65 (5.0%) 60 (4.6%) 48 (3.7%) 43 (3.3%) 26 (2.0%) 17 (1.3%) 9 (0.7%) 5 (0.4%) 5 (0.4%) 22 (1.7%) 40 (3.1%) 1309

safer alternative to retroviral vectors derived from murine retroviruses (86–88). 5

LESSONS LEARNED

It is now more than 15 years since the first authorized gene transfer trial was undertaken in 1989. Since then, over 1300 clinical trials have been initiated worldwide (Table 1) with several notable successes since 2000. Although results from these predominantly early-phase trials have been largely

unsuccessful in providing clinical benefit to human subjects, they have (1) provided clear proof-of-concept for gene therapy, (2) demonstrated that gene therapy is relatively safe, and (3) highlighted several important issues that must be considered to advance the field. The field has also recently experienced the first commercial release of a gene therapy treatment in China by Shenzhen SiBiono GenTech’s Gendicine for head-andneck squamous cell carcinoma (89, 90).

GENE THERAPY

5.1 The Power of In Vivo Selection The major reason gene therapy has been unsuccessful in providing clinical benefit in many disease contexts is low gene transfer efficiencies. Expansion of gene-modified cells, through in vivo selection, is one strategy by which the fundamental challenge of genemodifying sufficient cells to achieve therapeutic benefit can be overcome (91). The power of in vivo selection has been impressively illustrated in the SCID-X1 trial, which is the first successful treatment of a genetic disease by gene therapy (10). For most diseases, however, the gene corrected cells will not have a selective growth or survival advantage. Therefore, efforts to develop strategies for providing modified cells with an exogenous selective advantage are being made. One such strategy exploits mutant forms of the DNA repair enzyme methylguanine methyltransferase and targeting expression to hematopoietic progenitor cells using integrating vector systems. This strategy imparts genetic chemoprotection to the gene-modified cells and has been successfully employed in large animal models (92, 93). 5.2 Insertional Mutagenesis Insertional mutagenesis is now recognized as a clinically established risk associated with the use of integrating vector systems. Random integration events have the potential to drive a cell toward neoplastic transformation through inappropriate activation or inactivation of genes involved in the regulation of cell growth and differentiation. The risk of mutagenesis of cellular sequences promoting a malignant phenotype has been estimated to be about 10−7 per insertion (94). Although avoidance of integrating vector systems is not currently a viable option for gene therapy targeting the hematopoietic compartment, two broad strategies by which the risk of insertional mutagenesis can be significantly reduced include (1) reduction in the absolute number of integration sites to which patients are exposed and (2) reduction of the risk associated with individual integration events. Achievement of the former will depend on more sharply defining both the desired target cell population and the minimum effective dose of gene-corrected cells

9

to produce the desired phenotypic effect as well as optimization of transduction conditions. Reduction of the risk associated with individual integration events is theoretically achievable by careful choice of integrating vector system and optimized expression cassette design that lacks strong viral promoter/ enhancer elements. Whether leukemia represents a generic risk associated with the use of integrating vectors to target hematopoietic progenitor cells, or is linked more directly to specific features of the SCID-X1 gene therapy trial has yet to be established. Whatever the answer, future gene therapy studies that employ integrating vectors must be assessed against the, as yet, poorly quantified risk of the development of neoplasia as a consequence of insertional mutagenesis. 6

THE WAY FORWARD

The capacity of gene therapy to cure human disease is now an established reality, but for now, most disease phenotypes and pathophysiological processes potentially amenable to this exciting therapeutic approach lie beyond the reach of existing technology. The major challenge for the future, therefore, is to address the inherent shortcomings in the efficacy and safety of available gene delivery systems. Developments in this area will be complemented by an improved understanding of target cell biology, in particular the capacity to identify and manipulate stem cell populations. Finally, unwanted host vector interactions, such as immune responses directed against the vector and encoded transgene product, must be better understood and avoided. Such progress is fundamentally dependent on sound basic and pre-clinical research coupled with iterative human clinical trials. REFERENCES 1. Gene Therapy Advisory Committee (GTAC). United Kingdom Department of Health. Available: http://www.advisorybodies.doh.gov.uk/ genetics/gtac. 2. L. K. Branski, C. T. Pereira, D. N. Herndon, and M. G. Jeschke, Gene therapy in wound healing: present status and future directions. Gene Ther. 2007; 14: 1–10.

10

GENE THERAPY

3. E. D. Milligan, E. M. Sloane, S. J. Langer, T. S. Hughes, B. M. Jekich, M. G. Frank, J. H. Mahoney, L. H. Levkoff, S. F. Maier, P. E. Cruz, T. R. Flotte, K. W. Johnson, M. M. Mahoney, R. A. Chavez, L. A. Leinwand, and L. R. Watkins, Repeated intrathecal injections of plasmid DNA encoding interleukin-10 produce prolonged reversal of neuropathic pain. Pain 2006; 126: 294–308. 4. C. H. Evans, P. D. Robbins, S. C. Ghivizzani, M. C. Wasko, M. M. Tomaino, R. Kang, T. A. Muzzonigro, M. Vogt, E. M. Elder, T. L. Whiteside, S. C. Watkins, and J. H. Herndon, Gene transfer to human joints: progress toward a gene therapy of arthritis. Proc. Natl. Acad. Sci. U.S.A. 2005; 102: 8698–8703. 5. R. A. Morgan, M. E. Dudley, J. R. Wunderlich, M. S. Hughes, J. C. Yang, R. M. Sherry, R. E. Royal, S. L. Topalian, U. S. Kammula, N. P. Restifo, Z. Zheng, A. Nahvi, C. R. de Vries, L. J. Rogers-Freezer, S. A. Mavroukakis, and S. A. Rosenberg, Cancer regression in patients after transfer of genetically engineered lymphocytes. Science 2006; 314: 126–129. 6. O. ter Brake, P. Konstantinova, M. Ceylan, and B. Berkhout, Silencing of HIV-1 with RNA interference: a multiple shRNA approach. Mol. Ther. 2006; 14: 883–892. 7. S. A. Rosenberg, P. Aebersold, K. Cornetta, A. Kasid, R. A. Morgan, R. Moen, E. M. Karson, M. T. Lotze, J. C. Yang, and S. L. Topalian, Gene transfer into humans-immunotherapy of patients with advanced melanoma, using tumor-infiltrating lymphocytes modified by retroviral gene transduction. N. Engl. J. Med. 1990; 323: 570–578. 8. M. L. Edelstein, M. R. Abedi, J. Wixon, and R. M. Edelstein, Gene therapy clinical trials worldwide 1989–2004-an overview. J. Gene Med. 2004; 6: 597–602. 9. Gene Therapy Clinical Trials Worldwide. The Journal of Gene Medicine. Available: http:// www.wiley.co.uk/genmed/clinical/2007. 10. M. Cavazzana-Calvo, S. Hacein-Bey, G. de Saint Basile, F. Gross, E. Yvon, P. Nusbaum, F. Selz, C. Hue, S. Certain, J. L. Casanova, P. Bousso, F. L. Deist, and A. Fischer, Gene therapy of human severe combined immunodeficiency (SCID)-X1 disease. Science 2000; 288: 669–672. 11. S. Hacein-Bey-Abina, F. Le Deist, F. Carlier, C. Bouneaud, C. Hue, J. P. de Villartay, A. J. Thrasher, N. Wulffraat, R. Sorensen, S. Dupuis-Girod, A. Fischer, E. G. Davies, W. Kuis, L. Leiva, and M. Cavazzana-Calvo, Sustained correction of X-linked severe combined

immunodeficiency by ex vivo gene therapy. N. Engl. J. Med. 2002; 346: 1185–1193. 12. M. G. Ott, M. Schmidt, K. Schwarzwaelder, S. Stein, U. Siler, U. Koehl, H. Glimm, K. Kuhlcke, A. Schilz, H. Kunkel, S. Naundorf, A. Brinkmann, A. Deichmann, M. Fischer, C. Ball, I. Pilz, C. Dunbar, Y. Du, N. A. Jenkins, N. G. Copeland, U. Luthi, M. Hassan, A. J. Thrasher, D. Hoelzer, C. von Kalle, R. Seger, and M. Grez, Correction of X-linked chronic granulomatous disease by gene therapy, augmented by insertional activation of MDS1-EVI1, PRDM16 or SETBP1. Nat. Med. 2006; 12: 401–409. 13. A. Aiuti, S. Vai, A. Mortellaro, G. Casorati, F. Ficara, G. Andolfi, G. Ferrari, A. Tabucchi, F. Carlucci, H. D. Ochs, L. D. Notarangelo, M. G. Roncarolo, and C. Bordignon, Immune reconstitution in ADA-SCID after PBL gene therapy and discontinuation of enzyme replacement. Nat. Med. 2002; 8: 423–425. 14. A. Aiuti, S. Slavin, M. Aker, F. Ficara, S. Deola, A. Mortellaro, S. Morecki, G. Andolfi, A. Tabucchi, F. Carlucci, E. Marinello, F. Cattaneo, S. Vai, P. Servida, R. Miniero, M. G. Roncarolo, and C. Bordignon, Correction of ADA-SCID by stem cell gene therapy combined with nonmyeloablative conditioning. Science 2002; 296: 2410–2413. 15. H. B. Gaspar, K. L. Parsley, S. Howe, D. King, K. C. Gilmour, J. Sinclair, G. Brouns, M. Schmidt, C. von Kalle, T. Barington, M. A. Jakobsen, H. O. Christensen, A. Al Ghonaium, H. N. White, J. L. Smith, R. J. Levinsky, R. R. Ali, C. Kinnon, A. J. Thrasher, Gene therapy of X-linked severe combined immunodeficiency by use of a pseudotyped gammaretroviral vector. Lancet 2004; 364: 2181–2187. 16. H. B. Gaspar, E. Bjorkegren, K. Parsley, K. C. Gilmour, D. King, J. Sinclair, F. Zhang, A. Giannakopoulos, S. Adams, L. D. Fairbanks, J. Gaspar, L. Henderson, J. H. Xu-Bayford, E. G. Davies, P. A. Veys, C. Kinnon, and A. J. Thrasher, Successful reconstitution of immunity in ADA-SCID by stem cell gene therapy following cessation of PEG-ADA and use of mild preconditioning. Mol. Ther. 2006; 14: 505–513. 17. N. Somia and I. M. Verma, Gene therapy: trials and tribulations. Nat. Rev. Genet. 2000; 1: 91–99. 18. S. D. Li and L. Huang, Gene therapy progress and prospects: non-viral gene therapy by systemic delivery. Gene Ther. 2006; 13: 1313–1319.

GENE THERAPY

11

19. D. J. Glover, H. J. Lipps, D. A. Jans, Towards safe, non-viral therapeutic gene expression in humans. Nat. Rev Genet. 2005; 6: 299–310.

32. A. K. Zaiss and D. A. Muruve, Immune responses to adeno-associated virus vectors. Curr. Gene Ther. 2005; 5: 323–331.

20. C. C. Conwell and L. Huang,. Recent advances in non-viral gene delivery. Adv. Genet. 2005; 53: 3–18.

33. D. B. Schowalter, L. Meuse, C. B. Wilson, P. S. Linsley, and M. A. Kay, Constitutive expression of murine CTLA4Ig from a recombinant adenovirus vector results in prolonged transgene expression. Gene Ther. 1997; 4: 853–860.

21. S. Mehier-Humbert and R. H. Guy, Physical methods for gene transfer: improving the kinetics of gene delivery into cells. Adv. Drug Deliv. Rev. 2005; 57: 733–753. 22. S. Simoes, A. Filipe, H. Faneca, M. Mano, N. Penacho, N. Duzgunes, and M. P. de Lima, Cationic liposomes for gene delivery. Expert. Opin. Drug Deliv. 2005; 2: 237–254. 23. C. Louise, Nonviral vectors. Methods Mol. Biol. 2006; 333: 201–226. 24. M. D. Lavigne and D. C. Gorecki, Emerging vectors and targeting methods for nonviral gene therapy. Expert. Opin. Emerg. Drugs 2006; 11: 541–557. 25. W. Walther and U. Stein, Viral vectors for gene transfer: a review of their use in the treatment of human diseases. Drugs 2000; 60: 249–271. 26. M. A. Kay, J. C. Glorioso, and L. Naldini, Viral vectors for gene therapy: the art of turning infectious agents into vehicles of therapeutics. Nat. Med. 2001; 7: 33–40. 27. I. M. Verma and M. D. Weitzman, Gene therapy: twenty-first century medicine. Annu. Rev Biochem. 2005; 74: 711–738. 28. S. Hacein-Bey-Abina, C. von Kalle, M. Schmidt, F. Le Deist, N. Wulffraat, E. McIntyre, I. Radford, J. L. Villeval, C. C. Fraser, M. Cavazzana-Calvo, and A. Fischer, A serious adverse event after successful gene therapy for X-linked severe combined immunodeficiency. N. Engl. J. Med. 2003; 348: 255–256. 29. N. Chirmule, K. Propert, S. Magosin, Y. Qian, R. Qian, and J. Wilson, Immune responses to adenovirus and adeno-associated virus in humans. Gene Ther. 1999; 6: 1574–1583.

34. E. Dobrzynski, J. C. Fitzgerald, O. Cao, F. Mingozzi, L. Wang, and R. W. Herzog, Prevention of cytotoxic T lymphocyte responses to factor IX-expressing hepatocytes by gene transfer-induced regulatory T cells. Proc. Natl. Acad. Sci. U.S.A. 2006; 103: 4592–4597. 35. B. D. Brown, M. A. Venneri, A. Zingale, S. L. Sergi, and SL, Naldini, L. Endogenous microRNA regulation suppresses transgene expression in hematopoietic lineages and enables stable gene transfer. Nat. Med. 2006; 12: 585–591. 36. F. G. Falkner and G. W. Holzer, Vaccinia viral/retroviral chimeric vectors. Curr. Gene Ther. 2004; 4: 417–426. 37. A. L. Epstein and R. Manservigi, Herpesvirus/retrovirus chimeric vectors. Curr. Gene Ther. 2004; 4: 409–416. 38. A. Oehmig, C. Fraefel, X. O. Breakefield, and M. Ackermann, Herpes simplex virus type 1 amplicons and their hybrid virus partners, EBV, AAV, and retrovirus. Curr. Gene Ther. 2004; 4: 385–408. 39. A. Recchia, L. Perani, D. Sartori, C. Olgiati, and F. Mavilio, Site-specific integration of functional transgenes into the human genome by adeno/AAV hybrid vectors. Mol. Ther. 2004; 10: 660–670. 40. H. Wang and A. Lieber, A helper-dependent capsid-modified adenovirus vector expressing adeno-associated virus rep78 mediates sitespecific integration of a 27-kilobase transgene cassette. J. Virol. 2006; 80: 11699–11709. 41. D. W. Russell and R. K. Hirata, Human gene targeting by viral vectors. Nat. Genet. 1998; 18: 325–330.

30. C. S. Manno, V. R. Arruda, G. F. Pierce, B. Glader, M. Ragni, J. Rasko, M. C. Ozelo, K. Hoots, P. Blatt, B. Konkle, M. Dake, R. Kaye, M. Razavi, A. Zajko, J. Zehnder, H. Nakai, A. Chew, D. Leonard, et al., Successful transduction of liver in hemophilia by AAV-Factor IX and limitations imposed by the host immune response. Nat. Med. 2006; 12: 342–347.

42. R. M. Kapsa, A. F. Quigley, J. Vadolas, K. Steeper, P. A. Ioannou, E. Byrne, and A. J. Kornberg, Targeted gene correction in the mdx mouse using short DNA fragments: towards application with bone marrow-derived cells for autologous remodeling of dystrophic muscle. Gene Ther. 2002; 9: 695–699.

31. E. K. Broberg and V. Hukkanen, Immune response to herpes simplex virus and gamma134.5 deleted HSV vectors. Curr. Gene Ther. 2005; 5: 523–530.

43. D. de Semir and J. M. Aran, Targeted gene repair: the ups and downs of a promising gene therapy approach. Curr. Gene Ther. 2006; 6: 481–504.

12

GENE THERAPY

44. G. McClorey, H. M. Moulton, P. L. Iversen, S. Fletcher, and S. D. Wilton, Antisense oligonucleotide-induced exon skipping restores dystrophin expression in vitro in a canine model of DMD. Gene Ther. 2006; 13: 1373–1381. 45. M. Schlee, V. Hornung, and G. Hartmann, siRNA and isRNA: two edges of one sword. Mol. Ther. 2006; 14: 463–470. 46. Y. Kaneda, Y. Saeki, and R. Morishita, Gene therapy using HVJ-liposomes: the best of both worlds? Mol. Med. Today 1999; 5: 298–303. 47. Y. Kaneda, Virosomes: evolution of the liposome as a targeted drug delivery system. Adv. Drug Deliv. Rev. 2000; 43: 197–205. 48. R. J. Yanez-Munoz, K. S. Balaggan, A. Macneil, S. J. Howe, M. Schmidt, A. J. Smith, P. Buch, R. E. Maclaren, P. N. Anderson, S. E. Barker, Y. Duran, C. Bartholomae, C. von Kalle, J. R. Heckenlively, C. Kinnon, R. R. Ali, and A. J. Thrasher, Effective gene therapy with nonintegrating lentiviral vectors. Nat. Med. 2006; 12: 348–353. 49. L. Naldini, U. Blomer, P. Gallay, D. Ory, R. Mulligan, F. H. Gage, I. M. Verma, and D. Trono, In vivo gene delivery and stable transduction of nondividing cells by a lentiviral vector. Science 1996; 272: 263–267. 50. L. Naldini, U. Blomer, F. H. Gage, D. Trono, and I. M. Verma, Efficient transfer, integration, and sustained long-term expression of the transgene in adult rat brains injected with a lentiviral vector. Proc. Natl. Acad. Sci. U.S.A. 1996; 93: 11382–11388. 51. M. A. Adam, N. Ramesh, A. D. Miller, and W. R. Osborne, Internal initiation of translation in retroviral vectors carrying picornavirus 5’ nontranslated regions. J. Virol. 1991; 65: 4985–4990. 52. P. E. Monahan and R. J. Samulski, AAV vectors: is clinical success on the horizon? Gene Ther. 2000; 7: 24–30. 53. M. Ali, N. R. Lemoine, and C. J. Ring, The use of DNA viruses as vectors for gene therapy. Gene Ther. 1994; 1: 367–384. 54. A. C. Nathwani and E. G. Tuddenham, Epidemiology of coagulation disorders. Baillieres Clin. Haematol. 1992; 5: 383–439. 55. D. J. Fink, L. R. Sternberg, P. C. Weber, M. Mata, W. F. Goins, and J. C. Glorioso, In vivo expression of beta-galactosidase in hippocampal neurons by HSV-mediated gene transfer. Hum. Gene Ther. 1992; 3: 11–19. 56. X. Xiao, J. Li, T. J. McCown, and R. J. Samulski, Gene transfer by adeno-associated virus

vectors into the central nervous system. Exp. Neurol. 1997; 144: 113–124. 57. U. Blomer, L. Naldini, T. Kafri, D. Trono, I. M. Verma, and F. H. Gage, Highly efficient and sustained gene transfer in adult neurons with a lentivirus vector. J. Virol. 1997; 71: 6641–6649. 58. J. C. Glorioso, N. A. DeLuca, and D. J. Fink, Development and application of herpes simplex virus vectors for human gene therapy. Annu. Rev. Microbiol. 1995; 49: 675–710. 59. W. T. Hermens and J. Verhaagen, Viral vectors, tools for gene transfer in the nervous system. Prog. Neurobiol. 1998; 55: 399–432. 60. J. Fleming, S. L. Ginn, R. P. Weinberger, T. N. Trahair, J. A. Smythe, and I. E. Alexander, Adeno-associated virus and lentivirus vectors mediate efficient and sustained transduction of cultured mouse and human dorsal root ganglia sensory neurons. Hum. Gene Ther. 2001; 12: 77–86. 61. R. O. Snyder, C. Miao, L. Meuse, J. Tubb, B. A. Donahue, H. F. Lin, D. W. Stafford, S. Patel, A. R. Thompson, T. Nichols, M. S. Read, D. A. Bellinger, K. M. Brinkhous, and M. A. Kay, Correction of hemophilia B in canine and murine models using recombinant adenoassociated viral vectors. Nat. Med. 1999; 5: 64–70. 62. G. M. Acland, G. D. Aguirre, J. Ray, Q. Zhang, T. S. Aleman, A. V. Cideciyan, S. E. PearceKelling, V. Anand, Y. Zeng, A. M. Maguire, S. G. Jacobson, W. W. Hauswirth, and J. Bennett, Gene therapy restores vision in a canine model of childhood blindness. Nat. Genet. 2001; 28: 92–95. 63. T. M. Daly, K. K. Ohlemiller, M. S. Roberts, C. A. Vogler, and M. S. Sands, Prevention of systemic clinical disease in MPS VII mice following AAV-mediated neonatal gene transfer. Gene Ther. 2001; 8: 1291–1298. 64. A. Bosch, E. Perret, N. Desmaris, D. Trono, and J. M. Heard, Reversal of pathology in the entire brain of mucopolysaccharidosis type VII mice after lentivirus-mediated gene transfer. Hum. Gene Ther. 2000; 11: 1139–1150. 65. R. Pawliuk, K. A. Westerman, M. E. Fabry, E. Payen, R. Tighe, E. E. Bouhassira, S. A. Acharya, J. Ellis, I. M. London, C. J. Eaves, R. K. Humphries, Y. Beuzard, R. L. Nagel, and P. Leboulch, Correction of sickle cell disease in transgenic mouse models by gene therapy. Science 2001; 294: 2368–2371. 66. T. H. Nguyen, M. Bellodi-Privato, D. Aubert, V. Pichard, A. Myara, D. Trono, and N. Ferry, Therapeutic lentivirus-mediated neonatal in

GENE THERAPY vivo gene therapy in hyperbilirubinemic Gunn rats. Mol. Ther. 2005; 12: 852–859. 67. Z. Li, J. Dullmann, B. Schiedlmeier, M. Schmidt, C. von Kalle, J. Meyer, M. Forster, C. Stocking, A. Wahlers, O. Frank, W. Ostertag, K. Kuhlcke, H. G. Eckert, B. Fehse, and C. Baum, Murine leukemia induced by retroviral gene marking. Science 2002; 296: 497. 68. Declaration of Helsinki. The World Medical Association. Available: http://www.wma. net/e/policy/b3.htm. 69. K. Cornetta and F. O. Smith, Regulatory issues for clinical gene therapy trials. Hum. Gene Ther. 2002; 13: 1143–1149. 70. S. M. Dainesi, Seeking harmonization and quality in clinical trials. Clinics 2006; 61: 3–8. 71. J. Spink and D. Geddes, Gene therapy progress and prospects: bringing gene therapy into medical practice: the evolution of international ethics and the regulatory environment. Gene Ther. 2004; 11: 1611–1616. 72. D. B. Resnik and P. J. Langer, Human germline gene therapy reconsidered. Hum. Gene Ther. 2001; 12: 1449–1458. 73. M. Fuchs, Gene therapy. An ethical profile of a new medical territory. J. Gene Med. 2006; 8: 1358–1362. 74. C. Antoine, S. Muller, A. Cant, M. CavazzanaCalvo, P. Veys, J. Vossen, A. Fasth, C. Heilmann, N. Wulffraat, R. Seger, S. Blanche, W. Friedrich, M. Abinun, G. Davies, R. Bredius, A. Schulz, P. Landais, and A. Fischer, Long-term survival and transplantation of haemopoietic stem cells for immunodeficiencies: report of the European experience 1968–99. Lancet 2003; 361: 553–560. 75. M. Cavazzana-Calvo, A. Thrasher, and F. Mavilio, The future of gene therapy. Nature 2004; 427: 779–781. 76. A. J. Thrasher, S. Hacein-Bey-Abina, H. B. Gaspar, S. Blanche, E. G. Davies, K. Parsley, K. Gilmour, D. King, S. Howe, J. Sinclair, C. Hue, F. Carlier, C. von Kalle, B. G. de Saint, F. Le Deist, A. Fischer, and M. CavazzanaCalvo, Failure of SCID-X1 gene therapy in older patients. Blood 2005; 105: 4255–4257.

13

79. Structural characterization of gene transfer vectors. Federal Drug Authority. Available: http://www.fda.gov/OHRMS/DOCKETS/ ac/00/backgrd/3664b1a.doc. 80. M. E. Dudley, J. R. Wunderlich, P. F. Robbins, J. C. Yang, P. Hwu, D. J. Schwartzentruber, S. L. Topalian, R. Sherry, N.P. Restifo, A. M. Hubicki, M. R. Robinson, M. Raffeld, P. Duray, C. A. Seipp, L. Rogers-Freezer, K. E. Morton, S. A. Mavroukakis, D. E. White, and S. A. Rosenberg, Cancer regression and autoimmunity in patients after clonal repopulation with antitumor lymphocytes. Science 2002; 298: 850–854. 81. R. M. Blaese, K. W. Culver, A. D. Miller, C. S. Carter, T. Fleisher, M. Clerici, G. Shearer, L. Chang, Y. Chiang, P. Tolstoshev, et al. T lymphocyte-directed gene therapy for ADASCID: initial trial results after 4 years. Science 1995; 270: 475–480. 82. S. H. Orkin and A. G. Motulsky, Report and recommendations of the panel to assess the NIH investment in research on gene therapy. National Institutes of Health. Available: http://www.nih.gov/news/panelrep.html. 83. S. E. Raper, N. Chirmule, F. S. Lee, N. A. Wivel, A. Bagg, G. P. Gao, J. M. Wilson, and M. L. Batshaw, Fatal systemic inflammatory response syndrome in a ornithine transcarbamylase deficient patient following adenoviral gene transfer. Mol. Genet. Metab. 2003; 80: 148–158. 84. J. Savulescu, Harm, ethics committees and the gene therapy death. J. Med. Ethics 2001; 27: 148–150. 85. B. L. Levine, L. M. Humeau, J. Boyer, R. R. Macgregor, T. Rebello, X. Lu, G. K. Binder, V. Slepushkin, F. Lemiale, J. R. Mascola, F. D. Bushman, B. Dropulic, and C. H. June, Gene transfer in humans using a conditionally replicating lentiviral vector. Proc. Natl. Acad. Sci. U.S.A. 2006; 103: 17372–17377. 86. X. Wu, Y. Li, B. Crise, and S. M. Burgess, Transcription start regions in the human genome are favored targets for MLV integration. Science 2003; 300: 1749–1751.

77. A/ Jaffe, S. A. Prasad, V. Larcher, and S. Hart, Gene therapy for children with cystic fibrosis-who has the right to choose? J. Med. Ethics 2006; 32: 361–364.

87. R. S. Mitchell, B. F. Beitzel, A. R. Schroder, P. Shinn, H. Chen, C. C. Berry, J. R. Ecker, and F. D. Bushman, Retroviral DNA integration: ASLV, HIV, and MLV show distinct target site preferences. PLoS. Biol. 2004; 2: E234.

78. K. Nyberg, B. J. Carter, T. Chen, C. Dunbar, T. R. Flotte, S/ Rose, D. Rosenblum, S. L. Simek, and C. Wilson, Workshop on long-term followup of participants in human gene transfer research. Mol. Ther. 2004; 10: 976–980.

88. P. Hematti, B. K. Hong, C. Ferguson, R. Adler, H. Hanawa, S. Sellers, I. E. Holt, C. E. Eckfeldt, Y. Sharma, M. Schmidt, C. von Kalle, D. A. Persons, E. M. Billings, C. M. Verfaillie, A. W. Nienhuis, T. G. Wolfsberg, C. E. Dunbar,

14

89.

90.

91.

92.

93.

GENE THERAPY and B. Calmels, Distinct genomic integration of MLV and SIV vectors in primate hematopoietic stem and progenitor cells. PLoS. Biol. 2004; 2:E423. Z. Peng, Current status of gendicine in China: recombinant human Ad-p53 agent for treatment of cancers. Hum. Gene Ther. 2005; 16: 1016–1027. H. Jia, Controversial Chinese gene-therapy drug entering unfamiliar territory. Nat. Rev Drug Discov. 2006; 5: 269–270. T. Neff, B. C. Beard, and H. P. Kiem, Survival of the fittest: in vivo selection and stem cell gene therapy. Blood 2006; 107: 1751–1760. T. Neff, B. C. Beard, L. J. Peterson, P. Anandakumar, J. Thompson, and H. P. Kiem, Polyclonal chemoprotection against temozolomide in a large-animal model of drug resistance gene therapy. Blood 2005; 105: 997–1002. T. Neff, P. A. Horn, L. J. Peterson, B. M. Thomasson, J. Thompson, D. A. Williams, M. Schmidt, G. E. Georges, C. von Kalle, and H. P. Kiem, Methylguanine methyltransferasemediated in vivo selection and chemoprotection of allogeneic stem cells in a large-animal model. J. Clin. Invest. 2003; 112: 1581–1588.

94. Stocking, C, Bergholz, U, Friel, J, Klingler, K, Wagener, T, Starke, C, Kitamura, T, Miyajima, A, Ostertag, W. Distinct classes of factor-independent mutants can be isolated after retroviral mutagenesis of a human myeloid stem cell line. Growth Factors 1993; 8: 197–209.

FURTHER READING J. A. Wolff and J. Lederberg, An early history of gene transfer and therapy. Hum. Gene Ther. 1994; 5: 469–480.

CROSS-REFERENCES Risk-benefit analysis, safety, translation

GENETIC ASSOCIATION ANALYSIS

can also perform Fisher’s exact test, which is more accurate but more time-consuming. In many situations, the effects of SNP alleles are roughly additive; that is, the probability of response for the heterozygote (i.e., genotype aA) is intermediate between those of the two homozygotes (i.e., genotypes aa and AA). Then it is desirable to use the Armitage (1) trend test statistic:

D. Y. LIN Department of Biostatistics, University of North Carolina, Chapel Hill, North Carolina

In genetic association analysis, we assess the association between a genetic marker and a response variable. The association can originate in two ways: (1) The genetic marker is causally related to the response variable; and (2) the genetic marker is correlated with a causal variant. The latter might suffice prediction/classification purposes, whereas the former would be necessary for identifying potential drug targets. The most commonly used genetic markers are single nucleotide polymorphisms or SNPs. Virtually all SNPs are biallelic (i.e., having only two possible nucleotides or alleles). Thus, three possible genotypes exist at each SNP site: homozygous with allele A, homozygous with allele a, or heterozygous with one allele A and one allele a, where A and a denote the two possible alleles. In the case of a binary response variable with a total of n individuals, the data are represented in a 3 × 2 table:

nn.1 n.2 (n21 /2n.1 + n31 /n.1 − n22 /2n.2 − n32 /n.2 )2 n(n2. /2 + n3. ) − (n2. /2 + n3. )2 (2)

which has one degree of freedom. This test will be more powerful than Equation (1) if the genetic effects are indeed additive. We can also tailor our analysis to other genetic models. Under the dominant and recessive modes of inheritance, the test statistics become

Yes

No

Total

aa aA AA Total

n11 n21 n31 n.1

n12 n22 n32 n.2

n1. n2. n3. n

i=1 j=1

(nij − eij )2 , eij

n{n32 (n11 + n21 ) − n31 (n12 + n22 )}2 (n1. + n2. )n3. n.1 n.2

(4)

respectively. Again, Fisher’s exact tests can also be used. We measure the strength of association by the odds ratio or the difference of response rates. The choice of the test statistic should ideally be driven by the model of inheritance. Unfortunately, this knowledge is rarely available. The most common practice is to use the Armitage trend test, which should perform well unless the effects are far from additive. Test statistics (2)-(4) can be obtained as the score statistics under the logistic regression model:

To test the independence between the genotype and the response, we can use the well-known Pearson’s chi-squared statistic 3
2


(3)

and

Response Genotype

n{n11 (n22 + n32 ) − n12 (n21 + n31 )}2 n1. (n2. + n3. )n.1 n.2

logit{Pr(Y = 1|X)} = α + βX,

(5)

where Y is the response variable, X is the genotype score, α is the intercept, and β is the log odds ratio. Under the additive mode of inheritance, X denotes the number of A alleles; under the dominant model, X is the indicator for at least one A allele; and under the recessive model, X is the indicator for

(1)

where eij = ni. n.j /n. The null distribution of this test statistic is asymptotically chisquared with two degrees of freedom. We

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

GENETIC ASSOCIATION ANALYSIS

genotype AA. Inference on the odds ratio (i.e., eβ ) is based on the maximum likelihood theory. Test statistic (1) can also be generated under model (5). An important advantage of the logistic modeling is that it can readily accommodate multiple SNPs, environmental factors (e.g., treatment assignments), and gene-environment interactions. We adopt the general form of the logistic regression model: logit{Pr(Y = 1|X)} = α + β
X where X consists of genotype scores of (possibly) multiple SNPs, environmental factors, and products of genotype scores and environmental factors, and β is a vector of log odds ratios. In particular, we can assess the interactions between treatment assignments and genetic markers under this model. If the response variable is continuous, then we replace the logistic regression model with the familiar linear regression model. Indeed, we can use any generalized linear model (2). The only difference from the traditional regression analysis lies in the incorporation of appropriate genotype scores. If the response variable is survival time or event time, then we employ the proportional hazards model (3). All the analysis can be carried out in standard statistical software. It can be problematic to include many SNPs, some of which are highly correlated, in the regression analysis. An alternative approach is to consider haplotypes. A haplotype contains alleles from one parent. Association analysis based on haplotypes can reduce the degrees of freedom and increase the power to capture the combined effects of multiple causal variants, as compared with SNPsbased analysis. The current genotyping technologies do not separate the two homologous chromosomes of an individual, so haplotypes are not directly observable. Maximum likelihood methods have been developed to perform haplotype analysis based on genotype data (4). Haplotypes can also be used to infer the alleles of an untyped SNP (i.e., an SNP that is not on the genotyping chip) and thus allows analysis of association between an untyped SNP and a response variable (5). A potential drawback of genetic association analysis is that spurious associations

may result from unknown population structure or stratification. This problem originates when a genetic subgroup has a higher response rate than another, so that any SNP with allele proportions that differ among the subgroups will appear to be associated with the response variable. Several methods have been proposed to deal with population stratification. The most popular current approach is to infer axes of genetic variation from genomic markers through principal components analysis and then include those axes as covariates in the association analysis (6). Multiple testing is a serious issue in genetic association analysis, especially if many markers are examined. The common practice is to use the Bonferroni correction, which essentially divides the overall significance level by the number of tests performed. This strategy is conservative, especially when the markers are highly correlated. Accurate control of the type I error can be achieved by incorporating the correlations of the test statistics into the adjustments of multiple testing (7). The description thus far has been focused on the frequentist paradigm. There is an increasing interest in using Bayesian methods in genetic association analysis (8). REFERENCES 1. P. Armitage, Test for linear trend in proportions and frequencies. Biometrics 1955; 11:375–386. 2. P. McCullagh and J. A. Nelder, Generalized Linear Models, 2nd ed. New York: Chapman & Hall, 1989. 3. D. R. Cox. Regression models and life-tables (with discussion). J. R. Stat. Soc. Ser. B 1972; 34:187–220. 4. D. Y. Lin and D. Zeng. Likelihood-based inference on haplotype effects in genetic association studies (with discussion). J. Am. Stat. Assoc. 2006; 101:89–118. 5. D. Y. Lin, Y. Hu, and B. E. Huang. Simple and efficient analysis of SNP-disease association with missing genotype data. Am J. Hum. Genet. In press. 6. A. L. Price, N. J. Patterson, R. M. Plenge, M. E. Weinblatt, N. A. Shadick, and D. Reich. Principal components analysis corrects for stratification in genome-wide association studies. Nat. Genet. 2006; 38:904–909.

GENETIC ASSOCIATION ANALYSIS 7. D. Y. Lin. An efficient Monte Carlo approach to assessing statistical significance in genomic studies. Bioinformatics 2005; 21:781–787. 8. J. Marchini, B. Howie, S. Myers, G. McVean, and P. Donnelly. A new multipoint method for genome-wide association studies by imputation of genotypes. Nat. Genet. 2007; 39:906–913.

3

GLOBAL ASSESSMENT VARIABLES

treatment benefit (or harm)?’’ with all outcomes expected to have an effect in the same direction. To answer the first question, separate tests for each single outcome can be performed. Methods for these comparisons are discussed later in this section in comparison to global assessment. To address the directional hypothesis requires either a composite outcome or a global assessment of treatment benefit. Composite outcomes (see Composite Outcomes) refer to the situation where multiple outcomes are combined using a scoring system, which is defined a priori and considered to be clinically meaningful. For the situation where there is no such scoring system, O’Brien (2) as well as Pocock et al. (3) introduced the novel global statistical test (GST) methodology to combine information across outcomes and examine whether a treatment has a global benefit. GSTs have optimal statistical power when treatment effect is of similar magnitude across all outcomes (common dose effect). Because these methods can test whether a treatment has a global benefit without the need of multiple tests, the GST approach has been widely applied in medical studies (4–8), and the extensions of the GST method have appeared in the literature. Below we review different versions of the GST and describe where each GST can be appropriately used.

BARBARA TILLEY Medical University of South Carolina, Charleston, South Carolina

PENG HUANG John Hopkins University

PETER C. O’BRIEN Mayo Clinic

Although most clinical trials are designed using a single primary outcome, in some trials it is difficult to find the single most appropriate outcome for the main objective of the study. In the NINDS Stroke Trial, a consistent and persuasive poststroke improvement on multiple outcomes was required to define treatment efficacy (1). In Parkinson’s disease clinical trials, a treatment would not be considered to slow progression if the treatment improved only motor score and other measures of outcome deteriorated. In studying quality of life, we may be interested in treatment effects on all subscales being measured. In all cases, no single measure is sufficient to describe the treatment effect of interest, and no validated summary score exists. In a quality-of-life example, we could measure an overall quality-of-life score by summing across subscales, but we may under or overestimate the effect of treatment depending on how the treatment affects the individual subscales.

2 GENERAL COMMENTS ON THE GST 2.1 Definition of a GTE

1 SCIENTIFIC QUESTIONS FOR MULTIPLE OUTCOMES

Many GSTs can be described through two quantities. The first we call the global treatment effect (GTE), and it measures a treatment’s overall benefit across multiple outcomes. The concept of GTE was first introduced by Huang et al. (9); they defined it as a scale-free quantity. However, treatment’s overall benefit for many GSTs is defined through some scale-dependent quantity. We continue to call such a quantity a GTE with the understanding that it may not be exactly the same as that defined by Huang et al. (9) The other quantity is the statistic that tests the significance of this GTE. For those GSTs without GTE defined, the interpretation of

The choice of statistical method used for assessing treatment effect on multiple outcomes depends on the scientific question under investigation. Two types of questions lead to two different classes of tests. The first type is a directionless question: ‘‘Is there any difference, helpful or harmful between the two treatments in these outcomes?’’ A treatment difference could be detected if both strong positive and negative effects were observed. The other approach addressed a directional question: ‘‘Is there any global

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

GLOBAL ASSESSMENT VARIABLES

GST is more difficult as is the qualification of treatment benefit. The GTE from Huang et al. (9) is defined as the difference of two probabilities: the probability that a control group patient will do better if the patient is switched to the treatment, and the probability that a treatment patient will do better if the patient is switched to the control. Larger positive GTE value corresponds to stronger evidence of treatment benefit. Huang et al. (10) gave an unbiased estimate of the GTE for a ranksum GST. Detailed discussion of GTE was given in Huang et al. (9). A major advantage of using a GTE is that it can easily combine a treatment’s effects across different outcomes, no matter which scales are used and whether the outcomes have skewed distributions. Such advantage is not achieved by GTEs from parametric GSTs. A good discussion of advantages of using such a scale-free measure is given by Acion et al. (11). Based on the rank-sum GST, suppose the GTE = 0.4 or 40%. This value implies that a patient in the control group would have a [(70%−P/2) = (100% + 40%−P)/2] probability of a better overall outcome if he/she had been assigned to the treatment group, where P is the probability that the two groups being compared, have tied observations. If no ties are observed, then the probability would be equal to 70%. As the types of GSTs are described, the GTEs will be indicated or we will indicate that the GTE cannot be computed. 3

RECODING OUTCOME MEASURES

Because a global assessment of outcome variables is sensitive to the direction of the treatment effect on each of the variables, it is important to reverse code any variables in which the treatment effect would be measured in the opposite direction. For example, one can multiply variables by (− 1) if smaller observations indicate better outcomes for some but not all variables. In other words, all variables are coded in the way such that larger observations are preferred for all variables before applying a GST. Hereafter, we assume that all variables are coded in the way that larger value is preferred, although

the procedure could easily be applied when the all variables are coded such that the smaller value is preferred. 3.1 Assumptions Most GSTs require a common dose assumption, which implies that the treatment has a similar effect on all outcomes. The parametric GSTs must also meet the distributional assumptions of the test.

4 TYPES OF GLOBAL STATISTICAL TESTS (GSTS) 4.1 Rank-Sum-Type GST O’Brien (2) proposed a simple nonparametric GST for multiple outcomes using ranks. First, for each outcome, all patients are ranked according to their values on this outcome. Second, for each patient, the patient’s ranks are summed to obtain a rank score. Finally, a t-test (or a one-way ANOVA if more than 2 groups are involved) is applied to compare the rank scores between the treatment and control group. More generally, any univariate statistical procedure for comparing two samples may be applied to the rank sum scores. The GTE of this test is a rescaled probability that a control group patient will do better if the patient switches to the treatment, as defined by Huang et al. (9). Because a probability takes value between 0 and 1, they linearly rescaled the probability by minus 0.5 and then multiply by 2 to a value between −1 and +1 so that GTE = 0 implies no overall treatment benefit, a positive GTE value implies treatment benefit, and a negative GTE value implies treatment detrimental. The rank-sum-type GST is very simple to carry out. It is flexible in its ability to analyze multiple outcomes without the need of parametric assumptions regarding the distributions of the outcomes and the correlation among outcomes. The test is invariant to any monotone transformation of the data and is robust to outliers. Thus, rank-sum-type GST is applicable to broad medical research settings. Huang et al. (9) provided sample size computation formulas and corresponding Splus program code when rank-sum-type GST is used as the primary analysis.

GLOBAL ASSESSMENT VARIABLES

4.2 Adjusted Rank-Sum-Type GST Huang et al. (9) extended O’Brien’s ranksum-type GST to the case where variances in two treatment groups are different. The adjusted GST is computed similar to O’Brien’s unadjusted rank-sum GST, but it is divided by an adjusting factor. As for O’Brien’s rank sum test, the GTE is defined as the difference of two probabilities: the probability that a control group patient will do better if the patient is switched to the treatment, and the probability that a treatment patient will do better if the patient is switched to control. If no tied observations are recorded, then this GTE measures the probability that a patient will do better if he/she switched to the new treatment. Because this adjusted GST does not require the two treatment groups to have the same distribution under both the null and the alternative hypotheses, it can be applied to the Behrens-Fisher’s problem. The sample size formulas given by Huang et al. (9) can be applied to this adjusted rank-sum-type test. 4.3 Ordinary Least Square (OLS)-Based GST and Generalized Least Square (GLS)-Based GST These two tests were designed to have optimal power when the common-dose assumption is met (2, 3). The test statistics for OLS-based GST uses the standardized arithmetic mean (with equal weights) of all outcomes, and it is constructed based on the ordinary least square estimate of the GTE. The common standardized mean difference is the GTE. The test statistic for GLS-based GST uses a weighted average of all outcomes with weights depending on the correlation among the outcomes, and it is constructed based on the generalized least square estimate of GTE. These two GSTs are most appropriate for cases in which the treatment effect is measured by the change in outcome means. When the common-dose assumption holds, the GLSbased GST has higher power than other tests with the same type I error rate. Its limitation is that when some weights are negative, the test will be difficult to interpret. It will be difficult to determine whether a larger or a smaller test statistic is preferred or indicates treatment benefit. The negative weights can develop when the outcomes are diverse and

3

do not all have the same correlation with each other. Because of this problem with negative weights, several authors recommend the use of unweighted ordinary least squares GST (3, 11, 12). 4.4 Likelihood Ratio (LR)-Based GST for Normal Outcomes Recognizing that the common-dose assumption may not hold in practice and the GLS may lose power in this case, Tang et al. (12) studied the GLS for normally distributed outcomes assuming that treatment effects on all outcomes are in the same direction (i.e., all improved or all worsened), but the magnitudes of changes are a multiplier of (or equivalent, proportional to) a set of prespecified and possibly different values (13). The GTE of the LR-based GST is this common multiplier. The test statistic for LR-based GST is constructed in a similar manner as the GLS-based GST, but the outcome weights are determined by the prespecified magnitudes of change and the correlations among the outcomes. This GST is also designed for normally distributed outcomes. Tang et al. (14) illustrated how to apply the LR based GST in interim analysis and clinical trial design. Like GLS-based GST, the LR-based GST has a good power when the prespecified magnitudes of changes are true for the data, but in practice, these weights are not always known. 4.5 Approximate Likelihood Ratio (ALR)-Based GST for Normal Outcomes Although the LR-based GST has good power when true magnitudes of treatment effects on all outcomes are proportional to some prespecified values, its power decreases rapidly as the true treatment effects deviate from the prespecified values. Because it would be difficult for investigators to know the true treatment effects in advance, Tang et al. (14) proposed an ALR-based GST that maintains a good power in all possible magnitudes of treatment effects (all positive values). This GST is also designed for normally distributed outcomes. Its GTE is not easy to define. The test statistic is more complicated to compute than the previous GSTs. Simulations showed that ALR test is more powerful than

4

GLOBAL ASSESSMENT VARIABLES

O’Brien’s GLS-based GST when treatment effects are larger on some outcomes than on others. However, the authors noticed that the ALR test ignores differences going in the wrong direction, so caution was needed when using one-sided multivariate testing. 4.6 GST Using Scaled Linear Mixed Model for Continuous Outcomes When all outcomes are normally distributed and are measuring the same underlying event, Lin et al. (15) proposed a scaled linear mixed model to estimate treatment’s common dose effect. The method is flexible in studying multiple outcomes measured in different scales. Another advantage of this method is that it can adjust for confounding covariates and can be implemented by repeatedly fitting standard mixed models (See Mixed Models). 4.7 Follmann’s GST for Normal Outcomes Follmann proposed a simple test that has a good power when the sum of treatment effects on all outcomes is positive (17). Follmann’s GST uses Hotelling’s T2 test statistic to make a decision. It rejects the null hypothesis when T2 exceeds its 2α critical value and the sum of treatment differences on all outcomes is positive. The test controls the type I error α whether the covariance matrix of the outcomes is known. Under the common dose assumption, Follmann’s test is less powerful than O’Brien’s GLS-based GST. However, when the common dose assumption is violated, Follmann’s test could have a higher power than O’Brien’s GST and should avoid problems in interpretation if the correlation structure leads to negative weights for the GST. 4.8 GST for Binary Outcomes Lefkopoulou et al. (18) applied the quasilikelihood methods to model multiple binary outcomes. Later they extended these methods to incorporate analyses of clusters of individuals and they derived a scores test for multiple binary outcomes (19). They showed the scores test to be a special case of Pocock et al. (3). The common relative risk is the GTE measure of the GST, between different groups across all outcomes and represents

the assumed common-dose effect. The relative risk can be derived from the common odds ratio using the methods of Lu and Tilley (20). Binary GSTs without the common dose assumption were also compared using GEE by Lefkopoulou et al. (18) and Legler et al. (21). As the number of unaffected outcomes included in the GST increases, the global treatment effect is diluted, and the statistical power of the GST decreases quickly. 4.9 GST for Binary and Survival Outcomes Pocock et al. (3) gave a computational formula for GST when outcomes consist of binary outcomes and a survival outcome. First, a test statistic is constructed for each single outcome. Then, a multivariate test statistic is constructed based on these univariate test statistics and covariances among them. Rejection of the null hypothesis implies that treatment difference is observed on at least one outcome. Because this test does not assess whether a treatment has a global benefit, no GTE can be defined for the test. Also, because a global benefit is not computed, it could be considered a composite test (See Composite Endpoints). Simpler methods for accommodating binary time to event endpoints might also be considered. Specifically, one could use the actual binary values and the rank sum scores associated with each type of endpoint, respectively. 4.10 GST Using Bootstrap Bloch et al. (22) defined the treatment benefit as an improvement on at least one outcome and non-inferior on all outcomes. For example, if we have two outcomes, 1 is the mean treatment improvement on the first outcome, and 2 is the mean treatment improvement on the second outcome. They proposed to prespecify two positive numbers δ 1 and δ 2 such that treatment benefit is defined by the satisfaction of the following three conditions simultaneously: max {1 , 2 } > 0 and 1 > − δ1 and 2 > − δ2 This method is very similar to the approach proposed earlier by Jennison and Turnbull

GLOBAL ASSESSMENT VARIABLES

(23) who used an L-shaped subset in parameter space to define treatment benefit. The test statistic T is similar to the Hotelling’s T2 test statistic, but T is positive only when estimated 1 and 2 values are greater than some critical values u1 and u2 , respectively, and T = 0 otherwise. The null hypothesis of no treatment benefit is rejected when T > c for some critical value c. The critical values u1 , u2 , and c need to be determined to control the type I error rate. Because it is difficult to derive the null distribution of T analytically, the authors use bootstrap methods to estimate u1 , u2 , and c. Their simulations show that proposed method controls type I error rate and has good power for both normal and non-normal data, such as a mixture of normal distributions and a mixture of exponential distributions. Again, GTE as a global treatment effect is not computed and again, this method could be considered more like a composite test (See Composite Endpoints). 4.11 Extension of GST in Group Sequential Trial Design When all observations are accrued independently, all GSTs discussed above can be extended to group sequential design settings. Tang et al. (14) discussed how to apply likelihood ratio based GST for normal outcomes in group-sequential trials. A more general theory is given by Jennison and Turnbull (24) when the joint distribution of multiple outcomes is known except for a few parameters. The key assumption in their methods is that the additional data used in the next interim analysis must be independent to the data used in the previous interim analyses. For example, if the first interim analysis uses data from n1 subjects, the second interim analysis uses data from original n1 subjects plus n2 additional subjects. Observations from these n2 subjects are the additional data in the second interim analysis, and they must be independent to the data used in the first interim analysis. If data are not accrued independently, such as the case when patients are measured repeatedly and the data from the same patients are used in different interim analyses, then the computation of the critical values for the

5

stopping rule will be much more complicated. When data are not normally distributed but the normal theory is used to determine the critical values for the test statistics in different interim analyses, both the sample size at each interim analysis and the sample size difference between two consecutive interim analyses need to be large to make the Central Limit Theory applicable. 5 OTHER CONSIDERATIONS 5.1 Power of a GST A GST generally provides a univariate test statistic to describe the overall benefit (GTE). Because data from multiple outcomes are used to assess the GTE, it generally has a higher power than a univariate test (3,14). More importantly, a GST helps us to make a decision regarding whether a treatment is preferred using single test, rather using than multiple tests on different outcomes with the associated penalty for multiple testing. Where some outcomes strongly suggest benefit and others strongly suggest harm, power would be reduced, and we would be unlikely to reject the null hypothesis using a GST because we are testing the benefit of treatment. In the latter case, Hotellings T2 would be likely to reject the null hypothesis (See Multiple Comparisons). Because the power of the GST is generally greater than or equal to the power of the corresponding univariate test, using any single outcome one can use the smallest sample size calculated from the corresponding univariate tests as the sample size for the GST. However, this result may lead to a larger sample size than needed. Rochon (25) applied generalized estimating equation (GEE) approach to develop a sample size computation method for univariate outcome measured longitudinally. His method can be used for a cross-sectional study with multiple outcomes. Rochon’s approach is established under assumptions that variance–covariance matrix of outcomes is a function of outcome means and observations from all subjects in the same subgroup have exact the same covariates, mean values, and variance–covariance matrix. A sample size computational algorithm is provided along with a table of minimum required sample sizes for binary outcomes.

6

GLOBAL ASSESSMENT VARIABLES

5.2 Interpreting the GST The interpretation of GST depends on the variables included in the GST and the type of GST used. When it can be computed, the GTE measures a treatment’s overall benefit across multiple outcomes. For those GSTs without GTE defined, the interpretation of GST is more difficult. Estimation of GTE and its confidence interval may often provide more information about treatment effect than the binary decision regarding to whether the null hypothesis is rejected. Framing the hypothesis in terms of the GTE is helpful in sample size estimation and the design of clinical trials. For normal outcomes, if a null hypothesis is rejected by the OLS-based GST or by the GLS-based GST, then it is implied that, for all outcomes included in the GST, the common mean difference between the two groups, which is its GTE measure, is claimed to be significantly different from zero. For LR-based GST, a rejection of the null hypothesis implies that treatment improves variable means by the same factor, which is measured by the GTE. Because a GTE is not defined for ALR test, it is not easy to quantify treatment effect when the null hypothesis is rejected. What we can say about ALR test is that treatment improves all outcomes, but the amount of improvement for each outcome is not provided. For binary outcomes, because the GTE is defined as the common odds ratio of the success probabilities between the two groups, a rejection of the null hypothesis by the 1-degree-of-freedom GST implies that treatment improves success probabilities by the same factor whose value equals to the exponential of the GTE. For both rank-sum-type GST and adjusted rank-sum-type GST, the interpretation of treatment effect is similar. Rejection of a null hypothesis by these tests implies that treatment group patients have a greater chance of better overall outcome than the control group patients do. It is important to note that GST is designed to test treatment’s global effect on multiple outcomes. It does not provide separated assessment of the treatment effect on single outcomes. If an investigator is interested to

test the treatment effect for each of the single outcomes after the global null hypothesis is rejected, several sequential procedures are available to do so 26–29. These procedures listed do not allow repeated tests on the same outcome such as we do in the interim analysis. Kosorok et al. (30) proposed a group sequential design in which each single outcome is tested repeatedly in the interim analyses by partitioning the alternative region into mutually exclusive rectangular subregions, each corresponding to a combination of decisions based on all of the single outcomes. The decisions on each outcome can be ‘‘the treatment is better,’’ ‘‘the treatment is worse,’’ or ‘‘undetermined’’. Thus, a total of 3K − 1 subregions exist. The global type I error is defined as the probability that the test statistic incorrectly rejects the global null hypothesis of no treatment effect on any of the outcomes. The type II error is defined as the probability that the test statistic incorrectly leads to a conclusion that is inconsistent with the true distribution. For each outcome, stopping boundaries are constructed by spending its type I and type II errors separately, then combined to form multivariate stopping boundaries. To preserve the overall type I and type II errors, the boundaries are multiplied by a constant c that is determined through simulation. The advantage of using this method in clinical trials is that when a trial is stopped in an interim analysis, investigators can determine which outcomes are improved, which are worsened, and which are unchanged without inflation of the type I error. The limitation is its intensive computation. The authors provide software for this computation under the link http://www.bios.unc.edu/∼kosorok/clinicaltrial/main.html. 5.3 Choosing Outcome Variables to Include in the GST All variables included in the GST must be related to the scientific question to be addressed and should all be expected to respond to treatment. If the common dose assumption is required, then the treatment is expected to have a similar benefit on all outcomes. If several correlated variables are considered,

GLOBAL ASSESSMENT VARIABLES

whether one of them, some combinations of them, or all of them should be included into a GST depends on how we want to express treatment’s effect on these variables. Adding redundant variables into GST can bias the conclusion of GST. Simulations of Khedouri have shown that inclusion of highly correlated redundant variables artificially affects the power of different GSTs, which includes the OLS/GLS based GSTs, the GEE score equation derived GST, and rank-sum-type GST (31). 6

OTHER METHODS

The main rationale for using a global outcome in a clinical trial is that, by providing a more comprehensive and thus more informative assessment of the patient, it also provides a more clinically meaningful primary endpoint than use of any single variable by itself. Two other approaches for using multiple endpoints in the primary analysis of a clinical trial are the T2 statistic and use of multiple comparison procedures (See Multiple Comparisons). 6.1 T2 Statistic The T2 statistic computes the mean difference between groups for each of the endpoints, and then computes the distance between these two sets of means relative to a multivariate measure of the variability in the data that accounts for the correlation structure among the individual variables. Importantly, it does not take into account whether individual group differences were in a positive or negative direction. Thus, by itself the statistic is not meaningful. Furthermore, the failure to use this information about the derivation of the differences results in very low power for identifying efficacy. For these reasons, it is rarely used as a primary analysis in clinical trials. 6.2 Bonferroni Multiple comparison procedures are commonly used, especially Bonferroni adjustment of individual P-values. Specifically, the two treatment groups are compared in the usual way for each variable, and then the

7

several P-values are adjusted by multiplying them by the number of tests carried out. Statistical significance is claimed only if the smallest of these adjusted P-values is less than the prespecified threshold (typically .05). For example, if five primary endpoints are observed, then the smallest unadjusted Pvalue must be less than .01 to achieve overall significance at the 0.05 level. An argument sometimes made in favor of the Bonferroni approach is that interpretation is more straightforward and less ambiguous than using a global outcome, because when overall statistical significance is achieved, the endpoint that produced the smallest individual P-value can be reliably identified as an endpoint that is improved by the experimental treatment. Although this decreased ambiguity is an advantage, closer inspection of the Bonferroni approach reveals that interpretation may actually be more problematic than with a global outcome. To illustrate the problem with the Bonferroni method, imagine two primary endpoints, and suppose that the evidence for a treatment effect was identical for each variable with univariate P-values of .01, except that the direction of the treatment effect favored the experimental treatment for one endpoint and the placebo treatment for the other. Using the Bonferroni approach, the experimental treatment is considered efficacious, P = .02. However, using the rank sum method for defining a global outcome, the mean difference between groups in rank sum scores would be 0, which appropriately reflects that no overall benefit was associated with either treatment. The problem with the Bonferroni approach is that it fails to rely on a comprehensive assessment of the patient. This failure to use important information is reflected in lower power (See Power) for the Bonferroni approach if the experimental treatment has a beneficial effect across all primary endpoints. The ability of the various tests to control the overall type I error rate (falsely declaring efficacy when none exists) at the specified level of .05 and the power of the procedures to identify treatment effects is shown in Tables 1 and 2, respectively [adapted from a simulation study (2)]. In these tables, n indicates the sample size in

8

GLOBAL ASSESSMENT VARIABLES

Table 1. Observed Type I Error Rates for Various Tests Type I Error Rate∗ n

K

Rank-sum

GLS

Least Squares

Equal

5 20 20 50 5

2 2 5 5 50

.045 .050 .051 .048 .041

.041 .050 .059 .058 —

.062 .053 .045 .053 .061

Normal

Unequal

20 50 5

5 5 50

.047 .053 .050

.047 .051 —

.060 .049 .053

Skewed

Equal

5 20 20 50 5

2 2 5 5 50

.043 .062 .056 .035 .052

.041 .063 .064 .038 —

.045 .064 .063 .036 .054

Outliers

Equal

5 20 20 50 5

2 2 5 5 50

.047 .063 .051 .052 .053

.021 .032 .027 .036 —

.024 .032 .028 .037 .028

Distribution

Correlation

Normal

∗α =

.05. Data from O’Brien (2)

Table 2. Comparison of Power for Various Tests Power∗ Distribution

Treatment Effect

Correlation

n

K

Ranksum

GLS

Least squares

Bonferroni

T2

Equal

Normal

Equal

20 50 5

5 5 50

.64 .91 .28

.67 .92 —

.67 .92 .31

.52 .85 .17

.26 .66 —

Staggered

Normal

Equal

20 50 5

5 5 50

.32 .62 .14

.35 .64 —

.32 .64 .17

.34 .69 .10

.20 .58 —

Equal

Normal

Unequal

20 50 5

5 5 50

.62 .92 .18

.74 .97 —

.64 .93 .18

.51 .84 .06

.33 .78 —

Equal

Skewed

Equal

20 50 5

5 5 50

.77 .99 .38

.66 .93 —

.66 .93 .37

.57 .86 .16

.29 .67 —

Equal

Outliers

Equal

20 50 5

5 5 50

.23 .42 .11

.07 .08 —

.08 .08 .06

.02 .02 .01

.05 .05 —

∗α =

.05. Data from O’Brien (2)

GLOBAL ASSESSMENT VARIABLES

each treatment group, and K represents the number of univariate primary endpoints. As expected, all procedures control the overall type I error rate when sampling from a normal distribution, but only the rank sum test does so reliably for non-normal distributions (Table 1). Simulations to compare power (Table 2) assumed that the treatment improved all endpoints equally (the case for which the rank sum, least squares, and GLS tests should perform well) except for a ‘‘staggered effect size,’’ in which the effects ranged from 1/K, 2/K, . . . , K/K for endpoints 1, 2, . . . , K, respectively. As expected, the Bonferroni method had lower power for the equal effects situation, but can provide greater power when effects vary among the endpoints, depending on the variability, the number of endpoints, and the sample size. The T2 test has uniformly low power. 6.3 Composite Endpoints Lefkopoulou and Ryan (19) compared two different methods to assess treatment’s global effect on multiple binary outcomes. One is the test of composite outcomes (referred to as a combined test) that collapses multiple outcomes into a single binary outcome: A success is observed if success on any one of multiple binary outcomes is observed (See Composite Endpoints). The other is the 1-degree-offreedom GST that assumes a common odds ratio. The GST can be constructed using GEE (See Generalized Estimating Equations). When the response proportions are small (> 50%), both the composite test and the GST are approximately equally efficient. However, as the response proportions increase, the GST becomes much more efficient than the composite (combined) test (19). When a combined composite endpoint is used (any one of a set of outcomes, or any two, etc.), information from all outcomes is not taken into account. Someone with a stroke and myocardial infarction (MI) would be rated the same as someone having only an MI, and an increased risk of stroke, for example, could be missed. See Composite Outcomes (8) for a discussion of the limitation of the composite outcome and the ACR criteria used in the study of rheumatoid arthritis. However, computation of composite outcomes generally does not

9

require a common dose assumption. Also, for outcomes such as the ACR criteria, individual subjects are classified as successes or failures that are not generally the case for GSTs. 7

EXAMPLES OF THE APPLICATION OF GST

The NINDS t-PA Stroke Group performed two randomized, double-blind, placebo-controlled trials for patients with acute ischemic stroke (1). In both trials, the treatment success was defined as a ‘‘consistent and persuasive difference’’ in the proportion of patients who achieve favorable outcomes on the Barthel Index, Modified Rankin Scale, Glasgow Outcome Scale, and National Institutes of Health Stroke Scale. Four primary outcomes were used because a positive result from any single outcome was not believed to provide a sufficient evidence of treatment efficacy. Trial Coordinating Center proposed the use of a GST. All four outcomes are dichotomized to represent an outcome of normal or mild disability versus more severe disability or death. Trial investigators believed the assumption of a common dose effect to be valid for all four binary outcome measures. Table 3 gives the odds ratios for individual tests of each outcome and the 95% confidence intervals computed using a Mantal-Haenszel approach. The GST was computed using methods of Lefkopoulou and Ryan for binary outcomes (19). The GTE (odds ratio) for the GST is 1.73. No single outcome would be considered significant based on the Bonferroni adjustment. However, because the GST indicated an overall favorable outcome (P = 0.008), it was (a priori) considered to provide weak protection of alpha, and a test of the individual outcomes were conducted at the 0.05 level. Using these criteria, all individual outcomes indicated a benefit of treatment. See Table 3. Tilley et al. (8) used GSTs to analyze clinical trials in rheumatoid arthritis. O’Brien used GST in analysis of a randomized trial comparing two therapies, experimental and conventional, for the treatment of diabetes. The objective of the study was to determine whether the experimental therapy resulted in better nerve function, as measured by 34 electromyographic (EMG) variables, than the

10

GLOBAL ASSESSMENT VARIABLES

Table 3. NINDS t-PA Stroke Trial, Part II Data∗ Proportion With Favorable Outcome Outcome Barthel Modified Rankin Glasgow NIH Stroke Scale GST∗

t-PA (n = 168) 50 39 44 31

Placebo (n = 165) 38 26 32 20

OR 1.63 1.68 1.64 1.72 1.73

95% CI 1.06–2.49 1.09–2.59 1.06–2.53 1.05–2.84 1.16–2.60

P .026 .019 .025 .033 .008

∗ Global statistical test for binary outcomes developed by Lefkopoulou and Ryan (19). Data from the NINDS t-PA Stroke Trial (1).

standard therapy. Six subjects were randomized to standard therapy, and five subjects were randomized to experimental therapy. Despite the small sample size, the objective of the study was viewed primarily as confirmatory rather than exploratory. The medical question at issue was controversial, which required an overall quantitative and objective probability statement. There were six EMG variables for which the difference between groups was statistically significant (P > .05). The two smallest P-values were P = .002 and P = .015. The treated group did better in 28 of 34 variables, with P > .50 as a criterion; this result indicated the type of main effect for which the authors had hoped. Although these results seem to support the hypothesis of a beneficial effect associated with the experimental group, an overall test was needed. A T2 test was not appropriate. Application of a multiple-comparisontype per-experiment error rate is also meaningless here because of the small sample size relative to the large number of variables. When applied to all the data, the GST yielded a P-value of .033 (2).

8

CONCLUSIONS

Global statistical tests and associated GTEs summarize a treatment’s global benefit without the need of performing multiple univariate tests for individual outcomes. In contrast to composite tests, some GSTs take the correlation among outcomes into account. When a treatment shows benefit on each of the single outcomes, the GST often has a higher power than univariate tests because of increased information included in the GST. When a

treatment shows strong benefit on some outcomes and harm on other outcomes, the GST generally loses power compared with Bonferroni adjustment. This loss of power is also an advantage of the GST because investigators will be reluctant to accept a treatment if it shows strong harmful effect on equally important outcomes. The GST approach of analyzing multiple primary outcomes is a new emerging area in both statistical research and clinical applications. A variety of GSTs is available to address diverse situations. We believe that GSTs will play a significant role in medical research and future scientific findings. REFERENCES 1. The National Institute of Neurological Disorders and Stroke rt-PA Stroke Study Group, Tissue plasminogen activator for acute ischemic stroke. N. Engl. J. Med. 1996; 333: 1581–1587. 2. P. C. O’Brien, Procedures for comparing samples with multiple endpoints. Biometrics 1984; 40: 1079–1087. 3. S. J. Pocock, N. L. Geller, and Tsiatis AA. The analysis of multiple endpoints in clinical trials. Biometrics 1987; 43: 487–498. 4. L. Hothorn, Multiple comparisons in longterm toxicity studies. Environmental Health Perspectives. 1994; 102(suppl 1): 33–38. 5. K. D. Kaufman, E. A. Olsen, D. Whiting, R. Savin, R. DeVillez, W. Bergfeld, et al., Finasteride in the treatment of men with androgenetic alopecia. Finasteride Male Pattern Hair Loss Study Group. J. Am. Acad. Dermatol. 1998; 39: 578–589. 6. D. K. Li, G. J. Zhao, and D. W. Paty, Randomized controlled trial of interferon-beta-1a in secondary progressive MS: MRI results. Neurology 2001; 56: 1505–1513.

GLOBAL ASSESSMENT VARIABLES 7. R. S. Shames, D. C. Heilbron, S. L. Janson, J. L. Kishiyama, D. S. Au, and D. C. Adelman, Clinical differences among women with and without self-reported perimenstrual asthma. Ann. Allergy Asthma Immunol. 1998; 81: 65–72. 8. B. C. Tilley, S. R. Pillemer, S. P. Heyse, S. Li, D. O. Clegg, and G. S. Alarcon, Global statistical tests for comparing multiple outcomes in rheumatoid arthritis trials. MIRA Trial Group. Arthritis Rheum. 1999; 42: 1879–1888. 9. P. Huang, R. F. Woolson, and P.C. O’Brien, A rank-based sample size method for multiple outcomes in clinical trials. Stat. Med. In press. 10. P. Huang, B. C. Tilley, R. F. Woolson, and S. Lipsitz, Adjusting O’Brien’s test to control type I error for the generalized nonparametric Behrens-Fisher problem. Biometrics 2005; 61: 532–539. 11. L. Acion, J. J. Peterson, S. Temple, and S. Arndt, Probabilistic index: an intuitive nonparametric approach to measuring the size of treatment effects. Stats. Med. 2006; 25: 591–602. 12. D. I. Tang, N. L. Geller, and S. J. Pocock, On the design and analysis of randomized clinical trials with multiple endpoints. Biometrics 1993; 49: 23–30. 13. D. Follmann, Multivariate tests for multiple endpoints in clinical trials Stats. Med. 1995; 14: 1163–1175. 14. D. Tang, C. Gnecco, and N. Geller, Design of group sequential clinical trials with multiple endpoints. J. Am. Stat. Assoc. 1989; 84: 776–779. 15. D. Tang, C. Gnecco, and N. Geller An approximate likelihood ratio test for a normal mean vector with nonnegative components with application to clinical trials. Biometrika 1989; 76: 577–583. 16. X. Lin, L. Ryan, M. Sammel, D. Zhang, C. Padungtod, and X. Xu, A scaled linear mixed model for multiple outcomes. Biometrics 2000; 56: 593–601. 17. D. Follmann, A simple multivariate test for one-sided alternatives. J. Am. Stat. Assoc. 1996; 91: 854–861. 18. M. Lefkopoulou, D. Moore, and L. Ryan, The analysis of multiple correlated binary outcomes: application to rodent teratology experiments. J. Am. Stat. Assoc. 1989; 84: 810–815. 19. M. Lefkopoulou and L. Ryan Global tests for multiple binary outcomes. Biometrics 1993; 49: 975–988. 20. M. Lu and B. C. Tilley, Use of odds ratio or relative risk to measure a treatment effect in

11

clinical trials with multiple correlated binary outcomes: data from the NINDS t-PA Stroke Trial. Stats. Med. 2001; 20: 1891–1901. 21. J. Legler, M. Lefkopoulou, and L. Ryan, Efficiency and power of tests for multiple binary outcomes. J. Am. Stat. Assoc. 1995; 90: 680–693. 22. D. A. Bloch, T. L. Lai, and P. Tubert-Bitter, One-sided tests in clinical trials with multiple endpoints. Biometrics 2001; 57: 1039–1047. 23. C. Jennison and B. Turnbull, Group Sequential tests for bivariate response: interim analysis of clinical trials with both efficacy and safety endpoints. Biometrics 1993; 49: 741–752. 24. C. Jennison and B. Turnbull, Group Sequential Methods with Applications to Clinical Trials. 2000. CRC Press Inc., Boca Raton, FL. 25. J. Rochon, Application of GEE procedures for sample size calculations in repeated measures experiments. Stats. Med. 1998; 17: 1643–1658. 26. R. Falk, Hommel’s Bonferroni-type inequality for unequally spaced levels. Biometrika 1989; 76: 189–191. 27. S. Holm, A simple sequentially rejective multiple test procedure. Scand. J. Stats. 1979; 6: 65–70. 28. W. Lehmacher, G. Wassmer, and P. Reitmeir, Procedures for two-sample comparisons with multiple endpoints controlling the experimentwise error rate. Biometrics 1991; 47: 511–521. 29. R. Marcus, E. Peritz, and K. R. Gabriel On closed testing procedures with special references to ordered analysis of variance. Biometrika 1976; 63: 655–660. 30. M. R. Kosorok, S. Yuanjun, and D. L. DeMets, Design and analysis of group sequential clinical trials with multiple primary endpoints. Biometrics 2004; 60: 134–145. 31. C. Khedouri, Correcting Distortion in Global Statistical Tests With Application to Psychiatric Rehabilitation. Charleston, SC: Medical University of South Carolina, 2004. 32. T. Karrison and P. O’Brien, A rank-sum-type test for paired data with multiple endpoints. J. Appl. Stat. 2004; 31: 229–238. 33. N. Freemantle, M. Calvert, J. Wood, J. Eastaugh, and C. Griffin, Composite outcomes in randomized trials: greater precision but with greater uncertainty? JAMA 2003; 289: 2554–2559.

12

GLOBAL ASSESSMENT VARIABLES

CROSS-REFERENCES Type I Error Mixed Effect Models Multiple Comparisons Composite Variables (Endpoints or Indices) Generalized Estimating Equations Composite Outcomes

GOLD STANDARD

2

Many diagnostic tests have a binary response. An error-free, binary diagnostic test differentiates perfectly between patients who have a disease and those who do not, and it is referred to as a ‘‘perfect’’ gold standard test. Perfect gold standard tests are usually not available or not feasible because of technical, economic, or implementation constraints. Instead, investigators use tests or procedures with acceptably low error rates. An example is the use of clinical signs and symptoms, enzyme assays, and electrocardiograms instead of direct examination of the tissue of the heart to determine whether a patient has had a myocardial infarction. If a test is the best available, even though it may be imperfect and subject to error, it is called a gold standard test and is used as the reference with which other tests are compared. When a diagnostic test is not perfect, other characteristics of the test must be considered before declaring it a gold standard. Sensitivity is the conditional probability that the test will be positive given that the subject has the disease. Specificity is the conditional probability that the test will be negative given that the subject does not have the disease (3, p. 30). Very high values for both sensitivity and specificity are desirable if the test is to be considered useful. A test that has high sensitivity will be good at detecting a disease if it is present, and a test with high specificity will be good at ruling out a disease if it is not. For example, Jobst et al. (4) examined various tests used to diagnose Alzheimer’s disease. The study population consisted of normal elderly subjects as well as of subjects with non-Alzheimer’s disease dementias or Alzheimer’s disease that had been histopathologically confirmed (the gold standard test). A clinical diagnosis of probable Alzheimer’s disease according to the National Institute of Neurological and Communicative Disorders and Stroke–Alzheimer’s Disease and Related Disorders Association (NINCDSADRDA) had only 49% sensitivity in the Alzheimer’s subjects but 100% specificity in the subjects who did not have Alzheimer’s disease.

MARILYN A. AGIN Pfizer Global Research and Development, Ann Arbor, Michigan

In clinical trials, the concept of a gold standard is most often applied to diagnostic tests or methods of measurement that are considered to be the best in some general sense among all other tests or methods available. The gold standard test or measurement may be error-free or have an acceptably small error rate. New tests or measurements are compared with the gold standard. Closely related are the agreement of two or more types of tests or measurements when neither is a gold standard, and the reproducibility of a given test result or measurement. This article provides an overview of gold standard tests and measurements as used in clinical trials. 1

DIAGNOSTIC TESTS

THE GOLD STANDARD

In economics, the gold standard refers to a monetary standard under which the currency of a country is equal in value to and redeemable by a certain quantity of gold (1). Currencies of different countries can thus be compared with each other by assessing their relative value in gold. The United States was on the gold standard from time to time until 1971 (2). Today, informal usage of the term ‘‘gold standard’’ has expanded beyond economics to denote a practice or principle that is a model of excellence with which things of the same class can be compared. In clinical trials, the concept of a gold standard is most often applied to diagnostic tests or methods of measurement that are considered to be the best in some general sense among all other tests or methods available. New tests or measurements are compared with the gold standard. Closely related are the agreement of two or more types of tests or measurements when neither is a gold standard, and the reproducibility of a given test result or measurement. This article provides an overview of gold standard tests and measurements as used in clinical trials.

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

GOLD STANDARD

What a clinician usually wants to know, however, is the probability that a patient has a disease given that a test is positive or the probability that a patient does not have a disease given that a test is negative. If the proportion of patients in the population who have the disease is known, then Bayes’ theorem can be used to estimate these conditional probabilities retrospectively (3, p. 12). Most often, this proportion is not known and must be approximated. Other important aspects of a diagnostic test are its error rates. A false-positive error means that the test result was positive but the subject does not have the disease. A falsenegative error means that the test result was negative but the subject does have the disease. The error rate of the reference test is sometimes overlooked in evaluating the new test, and this can lead to bias in estimating the errors in the new test. When declaring an imperfect test to be a gold standard, the purpose for which the test is administered should be considered along with the sensitivity, specificity, and error rates. Sometimes two tests are available, and one or neither is considered a gold standard. When comparing a new test with a gold standard, the objective is to be able to declare the new test almost as good as (or no worse than) the gold standard. The null hypothesis should state that the two tests are different. The alternative hypothesis should state that the new test is as good as (equivalence) or no worse than (noninferiority) the gold standard since this is what one wishes to conclude. If the two statistical hypotheses had been reversed, then no claim could have been made about the new test being as good as or no worse than the gold standard. Not being able to reject the null hypothesis is not evidence that it is true, i.e., an ‘‘absence of evidence is not evidence of absence’’ (5). If neither test is a gold standard, then the alternative hypothesis should state that the two tests are equivalent or similar. A practical difficulty encountered when testing for equivalence or noninferiority is quantifying how similar the tests must be in order to claim they are not different. Chen et al. (6) propose a method to test the hypothesis of noninferiority with respect to sensitivity and specificity when

the response is binary. The new test is compared with a reference test and with a perfect gold standard test under retrospective and prospective sampling plans. A latent continuous or categorical variable is one that cannot be observed directly (7). In clinical trials, when a perfect gold standard test does not exist, then the true disease state can be thought of as a latent variable. For example, postmortem examination of tissue may be the only way to detect a certain type of pneumonia or a recurrence of cancer so the true disease state is latent as long as the patient is alive. If two imperfect diagnostic tests are used, a common statistical assumption is that the results of the tests are conditionally independent given the latent variable. 3 MEASUREMENT METHODS As with diagnostic tests, clinical measurements often cannot be made directly so indirect methods are used. For example, the duration of ventricular depolarization and repolarization of the heart is evaluated indirectly by measuring the QT interval on an electrocardiogram (ECG). Two laboratories using different methods of measurement on the same ECG (8), a technician reading the same ECG twice (9), or two different computer algorithms applied to the same ECG (10) could produce two different values for the QT interval. Since the true value of the QT interval is unknown, no method provides an unequivocally correct measurement (perfect gold standard) and the degree to which the measurements agree must be assessed. Since the act of measuring almost always involves some inherent measurement error or subjective judgment, a gold standard method of measurement is usually not perfect but is widely accepted and the reference to which other methods are compared. For example, an established assay for determining the plasma concentration of a drug is usually the reference for a new assay. The manner in which two or more sets of measurements agree or disagree is important since tests of agreement are usually specific to certain types of disagreement. For example, Pearson’s correlation coefficient

GOLD STANDARD

indicates only the linear relationship between the two sets of measurements. The correlation could be very strong, but one measurement could be consistently longer than another, or the measurements could follow a nonlinear relationship very closely but with little correlation. A paired t-test will detect only if the means of the two measurement groups are different and not provide any information about the marginal distributions of each measurement. Regression analysis, Lin’s (8) concordance correlation coefficient, and the intraclass correlation coefficient (9) address other types of agreement and reproducibility. Altman and Bland (10) and Bland and Altman (11,12) provide a clear explanation of the problem of comparing two methods of measurement and statistical principles of a graphical method for assessing agreement. The differences between measurements on the same observational unit are plotted against their averages. The within- and between-unit variability and the dependence of the difference between measurements on the mean are examined. Bartko (13) expand on Altman and Bland’s analyses by suggesting a bivariate confidence ellipse to amplify dispersion. St. Laurent (14) and Harris et al. (15) propose models and estimators for assessing agreement of one or more approximate methods of measurement with an accepted gold standard. Agreement is assessed by a ratio of variances equal to the square of the correlation between the approximate and the gold standard measurements. This correlation depends on the variability in the gold standard measurements and is identical in form to the intraclass correlation coefficient used when comparing two approximate methods. An acceptable degree of agreement depends on the error in the approximate method and can be defined for the specific clinical application under investigation. 4

CONCLUSION

When assessing diagnostic tests or methods of measurement, the first step should be to plot the data from each test or method against the others. Understanding how the tests or

3

sets of measurements agree or differ and the potential for incorrect conclusions can help to determine how to improve a test or measurement, assess the results statistically, and provide evidence for replacing a former imperfect or accepted gold standard test or method of measurement with a new one. REFERENCES 1. The American Heritage College Dictionary, 3rd ed. New York: Houghton Mifflin Company, 1997, p. 585. 2. World Book Encyclopedia Online. 2004. World Book, Inc. 3. A. Agresti, Categorical Data Analysis. New York: Wiley, 1990. 4. K. A. Jobst, L. P. D. Barnetson, and B. J. Shepstone, Accurate Prediction of Histologically Confirmed Alzheimer’s Disease and the Differential Diagnosis of Dementia: The Use of NINCDS-ADRDA and DSM-III-R Criteria, SPECT, X-Ray CT, and Apo E4 in Medial Temporal Lobe Dementias. International Psychogeriatrics 1998; 10: 271-302. Published online by Cambridge University Press 10Jan2005. 5. D. G. Altman and J. M. Bland, Absence of evidence is not evidence of absence. BMJ 1995; 311: 485. 6. J. J. Chen, H. Hsueh, and J. Li, Simultaneous non-inferiority test of sensitivity and specificity for two diagnostic procedures in the presence of a gold standard. Biomet. J. 2003; 45: 47-60. 7. P. F. Lazarsfeld, The logical and mathematical foundation of latent structure analysis. In: S. Stouffer (ed.), Measurement and Prediction. Princeton, NJ: Princeton University Press, 1950. 8. S. Patterson, M. A. Agin, R. Anziano, T. Burgess, C. Chuang-Stein, A. Dmitrienko, G. Ferber, M. Geraldes, K. Ghosh, R. Menton, J. Natarajan, W. Offen, J. Saoud, B. Smith, R. Suresh, and N. Zariffa, Investigating druginduced QT and QTc prolongation in the clinic: A review of statistical design and analysis considerations: Report from the Pharmaceutical Research and Manufacturers of America QT Statistics Expert Team. Drug Inf J. 2005; 39: 243-264. 9. N. Sarapa, J. Morganroth, J. P. Couderc, S. F. Francom, B. Darpo, J. C. Fleishaker, J. D. McEnroe, W. T. Chen, W. Zareba, A. J. Moss, Electrocardiographic identification of

4

GOLD STANDARD

drug-induced QT prolongation: Assessment by different recording and measurement methods. Ann Noninvas. Electrocardiol. 2004; 9(1): 48-57. 10. J. L. Willems, P. Arnaud, J. H. van Bemmel, P. J. Bourdillon, C. Brohet, S. Dalla Volta, J. D. Andersen, R. Degani, B. Denis, and M. Demeester, Assessment of the performance of electrocardiographic computer programs with the use of a reference data base. Circulation 1985; 71(3): 523-534. 11. L. I. Lin, A concordance correlation coefficient to evaluate reproducibility. Biometrics 1989; 45(1): 255-268. 12. P. E. Shrout and J. L. Fleiss, Intraclass correlations: Uses in assessing rater reliability. Psychol. Bull. 1979; 2: 420-428. 13. D. G. Altman and J. M. Bland, Measurement in medicine: The analysis of method comparison studies. Statistician 1983; 32: 307-317. 14. J. M. Bland and D. G. Altman, Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;(February): 307-310. 15. J. M. Bland and D. G. Altman, Measuring agreement in method comparison studies. Stat. Methods Med. Res. 1999; 8:135-160. 16. J. J. Bartko, Measures of agreement: a single procedure. Stat. Med. 1994; 13:737-745. 17. R. T. St. Laurent, Evaluating agreement with a gold standard in method comparison studies. Biometrics 1998; 54: 537-545. 18. I. R. Harris, B. D. Burch, and R. T. St. Laurent, A blended estimator for a measure of agreement with a gold standard. J. Agricult. Biol. Environ. Stat. 2001; 6: 326-339. 19. S. D. Walter and L. M. Irwig, Estimation of test error rates, disease prevalence and relative risk from misclassified data: A review. Clin. Epidemiol. 1988; 41: 923-937. 20. S. L. Hui and X. H. Zhou, Evaluation of diagnostic tests without gold standards. Stat. Methods Med. Res. 1998; 7: 354-370. 21. L. Joseph, Reply to comment on ‘‘Bayesian estimation of disease prevalence and the parameters of diagnostic tests in the absence of a gold standard’’. Am. J. Epidemiol. 1997; 145: 291-291.

22. N. Dendukuri and L. Joseph, Bayesian approaches to modeling the conditional dependence between multiple diagnostic tests. Biometrics 2001; 57: 208-217. 23. N. Dendukuri, E. Rahme, P. B´elisle, and L. Joseph, Bayesian sample size determination for prevalence and diagnostic test studies in the absence of a gold standard test. Biometrics 2004; 60: 388-397. 24. J. L. Fleiss, 1999. The Design and Analysis of Clinical Experiments. New York: Wiley, 1999.

CAMBRIDGE JOURNALS ONLINE Further reading Walter and Irwig (19) review publications on error rates of diagnostic tests. Hui and Zhou (20) review statistical methods for estimating sensitivity and specificity of diagnostic tests when one or more new diagnostic tests are evaluated and no perfect gold standard test exists, with or without an imperfect gold standard test as a reference. They also discuss models that relax the assumption of conditional independence between tests given the true disease status. Joseph (21), Dendukuri and Joseph (22), and Dendukuri et al. (23) use a Bayesian approach to account for the potential dependence between the tests instead of assuming conditional independence. Dallal’s website http://www.tufts.edu/ gdallal/compare.html provides an excellent introduction to the problem of comparing two measurement devices. Fleiss (24, Chapter 1) provides a comprehensive overview of the reliability of quantitative measurements.

CROSS-REFERENCES False Positive – False Negative Inter-rater reliability Intra-rater reliability Intraclass Correlation Coefficient Sensitivity Specificity Type I Error Type II

GOOD CLINICAL PRACTICE (GCP)

based on adequately performed laboratory and animal experimentation as well as on a thorough knowledge of the scientific literature. The right of the research subject to safeguard his/her integrity must always be respected. Physicians should abstain from engaging in research projects that involve human subjects unless they are satisfied that the hazards involved are predictable and should cease any investigations if the hazards are found to outweigh the potential benefits. Clinical tests in humans are performed under many regulatory controls. Currently, these controls include the regulations promulgated under the Food Drug and Cosmetic Act, European Union rules, and guidelines issued by the International Conference on Harmonization (ICH) (1). These regulations and guidances are collectively termed ‘‘Good Clinical Practices.’’ The testing of medical products, which include pharmaceuticals, biologicals, and medical devices on humans, is regulated by Good Clinical Practice standards (i.e., regulations and guidances). These standards are applied to the conduct of these tests or clinical investigations. International regulatory bodies require that these standards be met for the medical products to be approved for use. The experimental protocol must be reviewed by an independent body [Independent Ethics Committee (IEC) or Institutional Review Board (IRB)] constituted of medical professionals and nonmedical members, whose responsibility is to ensure the protection of the rights, safety and well-being of human subjects involved in a clinical investigation. This independent body provides continuing review of the trial protocol and amendments and of all materials to be used in obtaining and documenting the informed consent of the trial subjects.

DAVID B. BARR Kendle International, Cincinnati, Ohio

The ultimate goal of most clinical investigations is to obtain marketing approval of the product. In the case of a drug product, the Food and Drug Administration (FDA) requires the submission of a New Drug Application (NDA) that consists of a compilation of all clinical and other test data from the clinical studies plus extensive information on the development, manufacture, and testing of the drug product. This document is often massive, running in excess of 100 volumes. The FDA reviews this information. The review often includes meetings with the sponsor and may include a request for additional testing by the sponsor. Drug approval is for a specific drug, as well as the chemical entity, formulation, and the patient population on which it is approved for use, and the indications or disease state for which it is approved for use. For a sponsor to add other indications or patient populations to the drug labeling usually requires additional clinical studies to support these additional uses. 1

HUMAN RIGHTS AND PROTECTIONS

The need to protect human subjects during clinical studies is paramount, and the need for these protections is, in large part, based on abuses that had occurred in many countries including the medical experiments conducted by the Nazis and certain studies conducted in other countries without the full knowledge of the subjects. The rights of the subjects in a clinical trial are based on the principles stated in Declaration of Helsinki in 1964. Good clinical investigations must be conducted in accordance with these stated ethical principles. The rights, safety, and well-being of the trial subjects are paramount. Clinical research on human subjects must conform to generally accepted scientific principles and should be

2

INFORMED CONSENT

Subjects must volunteer to enter a clinical investigation and must be fully informed of their rights and a description of the study that includes risks and benefits. The investigator is required to provide each subject with

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

GOOD CLINICAL PRACTICE (GCP)

a written consent form approved by the IRB, which includes the following basic elements: a statement that the study involves research, the expected duration of the subject’s participation, a description of the procedures to be followed, identification of any procedures that are experimental, a description of any reasonably foreseeable risks or discomforts, a disclosure of any alternative procedures or courses of treatment that might be advantageous to the subject, an explanation of who to contact, and a statement that participation is voluntary. This consent form must be given to the subject, explained, and signed by the subject or his/her representative [21 CFR 50.25 -50.27 (2)]. Informed consent must be signed prior to initiation of the subject into the study. 3

INVESTIGATIONAL PROTOCOL

The clinical investigation must be described in a written protocol, which includes: a summary of information from both nonclinical studies that potentially have clinical significance and from clinical trials that have been conducted; a summary of the known and potential risks and benefits; information about the investigational product(s), route of administration, dosage, dosage regimen, and treatment periods; the population to be studied; references to literature and data that are relevant and provide background material for the trial; a description of the trial design, a statement of primary endpoints and any secondary endpoints, measures to be taken to minimize bias, discontinuation criteria, investigational product accountability procedures, identification of data to be recorded directly on the Case Report Form (CRF); criteria for the selection and withdrawal of subjects; description of subject treatment including the name(s) of all products, dosing schedule, route/mode of administration, treatment period(s) including follow-up period(s); permitted and prohibited medications, procedures for monitoring subject compliance; methods of assessments of efficacy and safety; statistical methods; quality control and quality assurance procedures; data handling and record-keeping requirements; and any supplements to the protocol.

4 INVESTIGATOR’S BROCHURE The Investigator’s Brochure (IB) is a compilation of the relevant clinical and nonclinical data on the investigational product(s). The IB must be provided to IRBs and IECs for their evaluation into whether the state of knowledge of the drug justifies the risk/benefit balance for the clinical trial. The IB is given to each investigator to provide to the clinical team with an understanding of the study details and facilitate their compliance with the protocol. 5 INVESTIGATIONAL NEW DRUG APPLICATION FDA requires the submission of an Investigational New Drug Application (IND) if a sponsor intends to conduct most clinical investigations (some exemptions apply see Reference 2). The IND is required to contain information on the sponsor and the investigational drug. The IND also contains brief descriptions of the drug, any relevant previous human experience with the drug, the overall investigational plan, nonclinical studies and safety information, protocol(s), and the IB. IND filed for Phase 1 studies are relatively brief at about one or two volumes, whereas those intended for Phase 2 and 3 typically contain detailed protocols that describe all aspects of the study. The IND must be approved before any human trial is initiated. Many other countries require similar applications. 6 PRODUCTION OF THE INVESTIGATIONAL DRUG Investigational products should be manufactured, handled, and stored in accordance with applicable current Good Manufacturing Practices (cGMPs ) and used in accord with the protocol. It is important that all product formulation, manufacturing process, specification, and test method development activities for pharmaceutical ingredients and products that support submissions of the IND and marketing applications be properly conducted and documented. Such information

GOOD CLINICAL PRACTICE (GCP)

may be requested for examination by FDA during preapproval inspections. All development activities should be performed applying the principles of Good Scientific Practices (GSPs) to ensure accuracy and reproducibility of data and other information. Examples of GSPs include accurate and complete documentation of all research and development activities; use of properly maintained and calibrated equipment; use of reagents, standards, and solutions that meet compendia standards or internal specifications; use of standard operating procedures, protocols, or guidelines; and performance of activities by appropriately qualified and trained personnel. At each stage of development, critical parameters should be defined for manufacturing processes (e.g., temperature, time) and product specifications (e.g., pH, viscosity). Test results and other data collected on product quality characteristics should be obtained using credible analytical methods. Critical parameters from production processes for product stability should be performed according to written procedures or prospective protocols typically with approval by the company’s quality unit. Changes in the product, process, specifications, and test methods should be controlled and documented. All data/information, whether favorable or unfavorable, should be recorded and included in development history documents. 7

CLINICAL TESTING

Clinical testing is performed according to the written protocols and procedures. A welldesigned and well-executed clinical investigation consists of many individual tests, stages, and subjects. The trial must be in compliance with the approved protocol and all pertinent regulations. Trials must be scientifically sound and described in a clear, detailed experimental protocol. Biomedical research that involves human subjects should be conducted only by scientifically qualified persons and under the supervision of a clinically competent medical person. In 1962, the Kefauver-Harris amendments to the Food Drug and Cosmetic Act (FD&C Act) increased FDA’s regulatory authority

3

over the clinical testing of new drugs. These amendments required that new drugs be approved by the FDA and that the application include ‘‘full reports of investigations which have been made to show whether or not such drug is safe for use and whether such drug is effective in use . . . ’’ [FD&C Act section 505 (3)]. Once compounds have been identified as potential drugs, preclinical testing begins. Clinical testing of new drugs and biologicals is conducted only after extensive preclinical animal testing to determine the safety of the potential drug. Currently, typical clinical testing consists of several phases, tests, and human subjects. Many tests may be required for these experimental products. For instance, in pharmaceuticals several tests are needed to permit the development of the estimates of the safe starting dose for use in clinical investigations. Tests are generally conducted for: single and repeated dose toxicity studies, reproductive toxicity, carcinogenic potential, genotoxicity, local tolerance, and pharmacokinetics. The ICH and several regulatory bodies have published guidelines on the conduct of these studies. Additionally, regulatory requirements control of many of these studies, for example, FDA’s Good Laboratory Practice regulations [GLPs at 21CFR 58 (4)]. Once animal safety studies are conducted and show relative safety, the experimental medical product may then begin testing in humans. An experimental drug requires a sponsor. The FDA defines a sponsor in the code of federal regulations [21CFR 312.3 (5)] as ‘‘a person who takes responsibility for and initiates a clinical investigation. The sponsor may be an individual or a pharmaceuticalcompany, governmental agency, academic institution, private organization, or other organization.’’ Before testing may be conducted on humans an Investigational New Drug (IND) or Investigational Device Exemption (IDE) application needs to be approved by regulatory bodies such as the FDA. Preapproval clinical testing is generally performed in three phases; each successive phase involves more subjects. These phases are conducted in accord with regulations [21 CFR 312.21 (5)], not by requirements of law.

4

GOOD CLINICAL PRACTICE (GCP)

Some clinical testing may also be performed post-approval. Phase One studies are closely controlled and monitored studies performed primarily to assess the safety of the drug in a few patients (typically 20–80) or healthy subjects. These studies are designed to determine the actions of the drug in the human body: absorption, metabolism, and pharmacological actions including degradation and excretion of the drug, side effects associated with increasing doses, and, if possible, to gain early evidence on effectiveness. These studies may also investigate the best method of administration and the safest dosage as well as side effects that occur as dosage levels are increased. Phase Two studies are usually the initial, well monitored, and well-controlled studies conducted to evaluate the effectiveness of the drug for a particular indication or condition. These studies generally involve a limited number of subjects, up to several hundred, who have the disease or condition under study. These studies are typically conducted to: evaluate the efficacy of the drug, determine the dose range(s), determine drug interactions that may occur, to confirm the safety of the drug, and to compare it with similar approved drugs. These studies may or may not be blinded. Phase Three studies are typically the last studies performed prior to filing the NDA with the regulatory bodies (e.g., FDA or European Agency for the Evaluation of Medicinal Products). These studies are expanded controlled and uncontrolled trials. They are performed after preliminary evidence that suggests effectiveness of the drug has been obtained. These studies are intended to gather the additional information about effectiveness and safety that is needed to evaluate the overall risk/benefit relationship of the drug and to provide and adequate basis for physician labeling. These trials are often multicenter studies with several thousands of subjects. These studies: provide the statistically significant data for the efficacy of the drug, assess the safety data, and determine the final dosage(s) and dosage forms. These studies are typically conducted as doubleblinded studies against similar approved drugs and/or placebos.

Phase Four studies are postmarketing studies that may be conducted by the sponsor and often required by FDA concurrent with marketing approval to delineate additional information about the risks, benefits, and optimal use of the drug. These studies include those studying different doses or schedules of administration, the use of the drug in other patient populations or stages of disease, or usage over a longer time period (see Fig. 1). Good Clinical Practice standards include most processes and organizations involved in the conduct of a pharmaceutical clinical study, such as the following: • Sponsors • Monitors • Contract Research Organizations

(CROs) • Institutional Research Boards (IRBs) or

Independent Ethics Committees (IECs) • Investigators

FDA regulations that govern the clinical investigations are found in Title 21 of the Code of Federal Regulations and in Guidelines promulgated by FDA and other regulatory bodies as well as those developed and issued by ICH. The ICH is an organization whose membership includes the regulators and Industry members from the EU, Japan, and the United States. As such, the ICH guidelines are in use in all of the EU countries, Japan and the United States and are widely used as guidelines worldwide. 8 SPONSORS The sponsor of an experimental drug initiates the studies and assumes the ultimate responsibility for compliance with all legal and regulatory requirements. The sponsor is responsible for the selection of all the parties utilized in the conduct of the clinical investigation including the monitors and investigators. According to FDA ‘‘The sponsor may be an individual or a pharmaceutical company, governmental agency, academic institution, private organization, or other organization.’’ The 21 Code of Federal Regulations, part

GOOD CLINICAL PRACTICE (GCP)

5

Compound Success Rates by Stage: Discovery: (2 to 10 years)

5,000 to 10,000 Screened

Preclinical Testing: Laboratory and animal testing

250 enter Preclinical testing

Phase I: 20 to 80 volunteers used to determine safety and dosage

5 enter Clinical testing:

Phase II: 100 to 300 volunteers; monitor for efficacy and side effects Phase III: 1,000 to 5,000 volunteers; monitor for adverse reactions to long-term use 1 FDA Approval

FDA Review/Approval Postmarket Testing 0

2

4

6

8 10 Years

12

14

16

Figure 1.

312 (5) (drugs) and part 812 (6) ( biologicals) contain the regulations that include the sponsors’ obligations. The obligations of sponsors include that they perform the following duties:

9. Ensure that the clinical investigation is conducted in accordance with the general investigational plan and protocols contained in the IND. 10. Maintain a current IND with respect to the investigations. 11. Evaluate and report adverse experiences. 12. Ensure that FDA and all participating investigators are promptly informed of significant new adverse effects (SADEs) or risks with respect to the drug.

1. Ensure that all clinical studies are conducted according to the regulations and an approved study protocol. (Both the FDA and an IRB must approve the study protocol.) 2. Obtain regulatory approval, where necessary, before studies begin. 3. Manufacture and label investigational products appropriately. 4. Initiate, withhold, or discontinue clinical investigations as required. 5. Refrain from commercialization investigational products.

of

6. Select qualified investigators to conduct the studies. 7. Provide the investigators with the information they need to conduct an investigation properly (training, written guidance including the investigators brochure). 8. Ensure proper monitoring of the clinical investigation.

Sponsors are responsible for reviewing data from the studies and from all other sources as soon as the materials are received to ensure that the safety of the subjects is not compromised. In any case, where the compliance of the investigator (or investigator site) has deviated from the approved protocol or regulations, it is the responsibility of the sponsor to ensure the site is either brought into compliance or, if compliance cannot be attained, to stop the investigation and assure the return of all clinical supplies. 9

CONTRACT RESEARCH ORGANIZATION

Contract Research Organization (CRO) is defined by FDA [(21 CFR 312.3 (b) (5)] as

6

GOOD CLINICAL PRACTICE (GCP)

‘‘a person that assumes, as an independent contractor with the sponsor, one or more of the obligations of a sponsor, e.g. design of a protocol, selection or monitoring of investigations, evaluation of reports, and preparation of materials to be submitted to the Food and Drug Administration.’’ The transfers of obligations to a CRO must be described in writing. If all obligations are not transferred, then the sponsor is required to describe each of the obligations being assumed by the CRO. If all obligations are transferred, then a general statement that all obligations have been transferred is acceptable [21 CFR 312.52 (5)] 10

MONITORS

Monitors are persons who oversee the progress of a clinical investigation and ensure that it is conducted, recorded, and reported accurately. The purposes of monitoring are to verify that the rights and well-being of the subjects are protected; the reported trial data are accurate, complete, and verifiable from source documents; and the conduct of the trial complies with the currently approved protocol amendments, GCP, and all applicable regulatory requirements. It is the responsibility of the sponsor to ensure that the clinical investigations are monitored adequately. Monitors are selected by the sponsor and must be adequately trained; have the scientific and/or clinical knowledge needed to monitor the trial adequately; and be thoroughly familiar with the investigational product, protocol, informed consent form, and any other materials presented to the subjects, the sponsors Standard Operating Procedures (SOPs), GCP, and all other applicable regulatory requirements. The extent and nature of monitoring should be set by the sponsor and must be based on multiple considerations such as the objective, purpose, design, complexity, blinding, size, and endpoints of the trial. Typically, on-site monitoring occurs before, during, and after the trial. 11

INVESTIGATORS

Investigators are the persons who actually conduct the clinical investigation (i.e., under

whose immediate direction the drug is administered or dispensed to a subject). In the event a team of individuals conducts an investigation, the investigator is the responsible leader of the team. ‘‘Subinvestigator’’ includes any other individual member of that team 21 CFR 312.3(b) (5)]. Investigators have several responsibilities, which include the following: 1. To ensure the investigation is conducted according to all applicable regulations and the investigational plan 2. To ensure the protection of the subjects’ rights, safety, and welfare 3. To ensure control of the drugs under investigation

12 DOCUMENTATION Acceptable documentation, from the sponsor, monitor, investigator, IRB, and so on, is critical for ensuring the success of clinical investigations. Sponsors and monitors must document their procedures and policies. SOPs are commonly used and must cover all aspects of their operations including Quality Assurance and monitoring. Investigators also require written procedures and CRFs. It is essential for investigators to assure the CRFs are up to date and complete and to have source documents (laboratory reports, hospital records, patient charts, etc.) that support the CRFs. All documentation in support of an NDA must be retained. FDA regulations require that for investigational drugs, all records and reports required during the studies must be retained for at least 2 years after the marketing application is approved. If the application is not approved, all applicable records should be retained for 2 years after the shipment of the study drug is discontinued and the FDA notified. The quality of the documentation is always critical. Records, including the CRFs must be legible; corrections should be cross outs with dates and initials and never obliterate the original data. The records should be controlled and retrievable for the monitors, FDA inspection, and other quality assurance

GOOD CLINICAL PRACTICE (GCP)

needs. Records must be credible and complete. They need to convince third parties (monitors, auditors, FDA Investigators, and reviewers) of their merit. The presence of accurate and reliable data and information in an application submitted to the FDA for scientific review and approval is essential. If a submission is misleading because of the inclusion of incorrect or misleading data, the FDA may impose the Application Integrity Policy (AIP). The AIP deals with applications that contain unreliable or inaccurate data. This policy enables the agency to suspend review of an applicant’s pending submitted applications that are in the review and approval process at FDA when a pattern of data integrity problems have been found in one or more of those applications and those data integrity problems are determined to be material to the review. Where data is collected electronically, such as in the use of electronic patient diaries, the FDA regulations on Electronic Records and Electronic Signatures [21 CFR Part 11 (7)] must be followed. These rules require many controls to ensure the authenticity, integrity, and confidentiality of electronic records. The required controls include (among many others) the validation of the system(s) assuring accuracy, reliability, and the ability to discern invalid or altered records. 13

CLINICAL HOLDS

A clinical hold is an order issued by the FDA to the sponsor to delay a proposed clinical investigation or to suspend an ongoing investigation. Clinical holds may apply to one or more of the investigations covered by an IND. When a proposed study is placed on clinical hold, subjects may not be given the investigational drug. When an ongoing study is placed on clinical hold, no new subjects may be recruited to the study and placed on the investigational drug; in the interest of patient safety, patients already in the study should be taken off therapy that involves the investigational drug unless specifically permitted by the FDA (2). Clinical holds are imposed when the FDA finds issues that may negatively affect patient safety, which include the following:

7

1. Human subjects are or would be exposed to an unreasonable and significant risk of illness or injury. 2. The investigator(s) are not qualified. 3. The investigators brochure is misleading, erroneous, or materially incomplete. 4. The IND does not contain sufficient information to assess the risks to the subjects. For Phase 2 and 3 studies, clinical holds may also be imposed if the plan or protocol is clearly deficient in design to meet its stated objectives. 14

INSPECTIONS/AUDITS

Audits and regulatory inspections are conducted to ensure that the clinical investigations are conducted correctly, data is accurately recorded and reported, and all procedures and applicable rules and regulations are followed. During the course of the study, the sponsor’s Clinical Quality Assurance staff generally conducts audits. These audits cover the Investigator(s) sites, data collection, and other sites that are pertinent to the clinical investigation. Any problems or issues revealed during the course of these audits typically result in corrections that enable the study to proceed without any major incidents or delays. Regulatory inspections, such as those conducted by FDA, are typically conducted after the sponsor files for marketing approval such as an NDA (the regulatory bodies may conduct inspections during the course of the study although it is generally only done when an issue develops). Regulatory bodies typically inspect a sampling of clinical sites from any given marketing application. Inspections of other facilities, such as sponsors, monitors, Institutional Review Boards, and so on, are typically performed from a sampling of all such facilities in their establishment inventory. The FDA has established a Bioresearch Monitoring Program that is used to provide oversight of the conduct of clinical studies. FDA has several Compliance Programs (CPs)

8

GOOD CLINICAL PRACTICE (GCP)

that provide guidance and specific instruction for inspections of: Investigators (CP7348.811) Sponsors/Monitors (CP7348.810) Institutional Review Boards (CP7348.809) Nonclinical Laboratories (CP7348.808) These CPs are available at www.fda.gov/ ora/cpgm. REFERENCES 1. International Conference on Harmonization, (ICH-E6), Good Clinical Practice: Consolidated Guidance, 1996. Available: www.ich.org. 2. 21 Code of Federal Regulations, Part 50, Protection of Human Subjects. 3. FD&C Act, Section 505. 4. 21 Code of Federal Regulations, Part 58. 5. 21 Code of Federal Regulations, Part 312, Investigational New Drug Application. 6. 21 Code of Federal Regulations, Part 812. 7. 21 Code of Federal Regulations, Part 11, Electronic Records and Signatures.

FURTHER READING Guideline for Industry Clinical Safety Data Management: Definition and Standards for Expedited Reporting (ICH-E2A), Federal Register, March 1, 1995 (60 FR 11284). W. K. Sietsema, Preparing the New Drug Application: Managing Submissions Amid Changing Global Requirements. FDAnews. Washington, DC, 2006. W. K. Sietsema, Strategic Clinical Development Planning: Designing Programs for Winning Products. FDAnews. Washington, DC, 2005. Food and Drug Administration. Available: http://www.fda.gov. Food and Drug Administration; Center for Drug Evaluation and Research. Available: http:// www.fda.gov/cder. Department of Health and Human Services. Available: http://www.os.dhhs.gov. European Union. Available: http://www.europa. eu.int. International Conference on Harmonization. Available: http://www.ich.org.

GOOD LABORATORY PRACTICE (GLP)

The regulations establish standards for the conduct and the reporting of nonclinical laboratory studies and are intended to assure the quality and the integrity of safety data submitted to the FDA. The FDA relies on documented adherence to GLP requirements by nonclinical laboratories in judging the acceptability of safety data submitted in support of research and/or marketing permits. The FDA has implemented this program of regular inspections and data audits to monitor laboratory compliance with the GLP requirements.

The Federal Food, Drug, and Cosmetic Act and Public Health Service Act require that sponsors of Food and Drug Administration(FDA)-regulated products submit evidence of their product’s safety in research and/or marketing applications. These products include food and color additives, animal drugs, human drugs and biological products, human medical devices, diagnostic products, and electronic products. The FDA uses the data to answer questions regarding: • The toxicity profile of the test article. • The

observed no-adverse-effect dose level in the test system. • The risks associated with clinical studies that involve humans or animals. • The potential teratogenic, carcinogenic, or other adverse effects of the test article. • The level of use that can be approved. The importance of nonclinical laboratory studies to the FDA’s public health decisions demands that they be conducted according to scientifically sound protocols and with meticulous attention to quality. In the 1970s, FDA inspections of nonclinical laboratories revealed that some studies submitted in support of the safety of regulated products had not been conducted in accord with acceptable practice; accordingly, data from such studies were not always of the quality and of the integrity to assure product safety. As a result of these findings, the FDA promulgated the Good Laboratory Practice (GLP) Regulations, 21 CFR (Code of Federal Regulations) Part 58, on December 22, 1978 (43 FR (Federal Register) 59986). The regulations became effective June 1979. This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/ora/compliance ref/bimo/7348 808/part I.html) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

GOODNESS OF FIT

continuous or a step function, later discussion will deal with continuous and discrete X i separately. There has been a recent resurgence of interest in the theory of goodness-of-fit tests. Technical advances have been made with some of the older tests, while new tests have been proposed and their power properties examined. This progress can be attributed in part to the availability of mathematical development in the theory of probability* and stochastic processes*. However, it is also in large measure due to the advent of the highspeed computer, the associated numerical technology, and the increased demand for statistical services. This article can only summarize some of the available results and refer the reader to special sources for further detail. Many statistical texts have introductory chapters on goodness-of-fit testing. For example, Kendall and Stuart [18, Vol. 2] and Lindgren (21) contain pertinent material lucidly presented. Pearson and Hartley (25) also contains accounts of specific tests illustrated by numerical examples. The following general notation will be adopted, additional special symbols being introduced as required:

G. M. TALLIS In general, the term ‘‘goodness of fit’’ is associated with the statistical testing of hypothetical models with data. Examples of such tests abound and are to be found in most discussions on inference*, least-squares* theory, and multivariate analysis.* This article concentrates on those tests that examine certain features of a random sample to determine if it was generated by a particular member of a class of cumulative distribution functions* (CDFs). Such exercises fall under the broad heading of hypothesis testing*. However, the feature that tends to characterize these ‘‘goodness-of-fit tests’’ is their preoccupation with the sample CDF, the population CDF, and estimates of it. More specifically, let X 1 , X 2 , . . . , X n be a random sample generated by CDF GX (x). It is required to test H0 : GX (x) = FX (x, θ ),

θ ∈
,

(1)

where θ is a q-dimensional vector of parameters belonging to the parameter space
. If θ is fixed at some value θ 0 , say, then F X (x, θ 0 ) = F X (x) is fully specified and H 0 is simple. Otherwise, the hypothesis states that GX (x) is some unspecified member of a family of CDFs and is composite*. As an example, consider the normal* family N X (x;θ ), θ  = (θ 1 , θ 2 ), where θ 1 is the mean and θ 2 the variance of N X . In this case
= (−∞, ∞) × (0, ∞) and it might be required to test whether or not a sample was generated by N X (x;θ ) for some unknown θ ∈
. Intuitively, and in fact, this is an intrinsically more difficult problem than testing whether the sample was generated by a particular normal CDF with known mean and variance. The latter case can always be reduced to the standard situation of testing GX (x) = N(x;θ 0 ), θ  0 = (0, 1). Most useful tests are parameter-free; i.e., the distribution of the test statistics does not depend on θ . Among such tests are found both parametric and nonparametric tests which are either distribution specific or distributionfree. Since tests may require F(x, θ ) to be

Probability density function* (PDF) corresponding to F X (x, θ ) and GX (x) (when they exist): f X (x, θ ), gX (x). Order statistics*: X i ≤ X  2 ≤ . . . ≤ X  n . Expected values of order statistics: E[X  i ] = ηi . Sample CDF: Gn (x) = [no. of X i ≤ x]/n. A chi-square random variable with d degrees of freedom: χ 2 (d). 100(1 − α) percentile of the chi-square distribution* with d degrees of freedom: 2 (d) . χ1−α The uniform density on [0, 1]: U [0, 1]. If X n is a sequence of random variables, L

then Xn → χ 2 (d) will indicate convergence in law to a chi-square distribution with d degrees of freedom; if X n is a sequence of ranL

dom vectors, then Xn → N(µ, ) will indicate

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

GOODNESS OF FIT

convergence* in law to a normal distribution with mean vector µ and covariance matrix*. 1

DISCRETE RANDOM VARIABLES

1.1 Simple H0 Suppose that X is a discrete random variable, Pr X = x = f X (x, θ ) for θ ∈
and x ∈ X, where X is a finite or countable set of real numbers. By suitable definition, categorical data* with no evident numerical structure can be brought within this framework. The simplest case is where X = 1, 2, . . . , k, f X (j, θ 0 ) = f j is fully specified, and it is required to test H 0 : gX (j) = f j . Let N j be the number of X i in the sample such that X i = j; then the probability under H 0 of obtaining the particular
outcome vector, n = n1 , n2 , . . . , nk , where j nj = n, is n!  nj fj . P(n) =  j nj !

(2)

j

An exact test of H 0 can, in principle, be constructed as follows: 1. Calculate P(m) for all possible outcome vectors, m. 2. Order the P(m). 3. Sum all P(m) which are less than or equal to P(n). 4. Reject H 0 at level α if the cumulative probability of 3 is less than or equal to α. This is known as the multinomial test of goodness of fit, and the necessary calculations must be carried out on a computer. Even then, this is only practicable if n and k are small. Fortunately, for large n, there are ways around the distributional problem. Likelihood ratio theory can be invoked; the likelihood ratio test* of H 0 : gX (j) = f j against the alternative H 1 : gX (j) = f j is formed from the  ratio  = kj=1 (nfj /Nj )Nj . It is known that −2 ln  = −2

k 

  Nj ln nfj − ln Nj

j=1 L

→ χ 2 (k − 1).

The null hypothesis is rejected at level α if the calculated value of −2ln exceeds χ 1−α (k − 1). A very old test dating back to the beginning of this century is based on X2 =

k 

(Nj − nfj )2 /(nfj ).

(3)

j=1

This is known as Pearson’s chi-square and has the same limiting distribution as (2). Since N j is the observed number of X i in the sample with X i = j, (Oj ), and E[N j ] = nf j = (Ej ), (3) is sometimes written in the form of a mnemonic, X2 =

k  (Oj − Ej )2 /Ej .

(4)

j=1

Not only do (2) and (3) share the same limiting central distribution, but they are also asmptotically equivalent in probability. However, since X 2 is a direct measure of agreement between observation and expectation under H 0 , it has some intuitive appeal not shared by (2). Both (2) and (3) give asymptotic tests which tend to break down if the nf j are too small. A common rule is that all nf j should be greater than 1 and that 80% of them should be greater than or equal to 5. These conditions are sometimes hard to meet in practice. For a general discussion and further references, see Horn (16). Radlow and Alf (29) point out that a direct comparison of X 2 with the multinomial test may be unjustified. The latter test orders experimental outcomes, m, in terms of P(m) instead of ordering them in terms of discrepancies from H 0 . It is suggested that X 2 should be compared with the following exact procedure: 1. Calculate P(m) for all possible outcomes m. 2. Calculate X 2 for each m based on H 0 , X 2 (m). 3. Sum P(m) for which X 2 (m) ≥ X 2 (n), the observed X 2 value. 4. Reject H 0 at level α if this sum exceeds α.

GOODNESS OF FIT

Numerical comparisons of this exact test with X 2 showed that the two agreed remarkably well, even for small n. The agreement of the exact multinomial test with X 2 , on the other hand, was poor. Thus care must be exercised in the assessment of the performances of large-sample tests using small n. Appropriate baseline exact tests must be used for comparisons. Another procedure is the discrete Kolmogorov–Smirnov* goodness-of-fit test (5). Let D− = maxx (FX (x) − Gn (x)),

The philosophy adopted in the preceding subsection for rejecting H 0 is used here, the 2 (k − q − 1). critical level being χ1−α Although (4) and (8) are the standard tests recommended in textbooks and have received the most attention by practitioners and theoreticians, there are others. For example, a general class of goodness-of-fit tests can be based on quadratic form* theory for multinormally distributed random variables. Under sufficient regularity conditions the following results can be established by routine methods of probability calculus. Put

D+ = maxx (Gn (x) − FX (x)), −

N = (N1 , N2 , . . . , Nk ),

+

D = max(D , D ); then D− , D+ , and D test, respectively, H 1 : GX (x) ≤ F X (x), H 1 : GX (x) ≥ F X (x), and H 1 : GX (x) = F X (x). A discussion of the application of these statistics is given in Horn (16), where their efficiency relative to X 2 is discussed and a numerical example given. For more recent asymptotic results, see Wood and Altavela (39). 1.2 Composite H0 When H 0 : GX (x) = F X (x, θ ), θ ∈
, is to be tested, θ must be estimated and the theory becomes more elaborate. However, provided that asymptotically efficient estimators θˆ n are used, tests (2) and (3) extend in a natural way and continue to be equivalent in probability. More specifically, since X i is assumed discrete, put H 0 : gX (j) = f j (θ ) and H 1 : gX (j) = f j (θ ), θ ∈
. Let θˆ n be as above; e.g., θˆ n could be the maximum likelihood estimator* (MLE) for θ under H 0 . Then under H 0 , −2 ln (θˆ n ) = −2

k  j=1

→ χ 2 (k − q − 1), X 2 (θˆ n ) =

k 

(Nj − nfj (θˆ n ))2 /(nfj (θˆ n ))

j=1 L

Ni = Ni /n, (f(θ)) = (f1 (θ), f2 (θ), . . . , fk (θ )); then

√

L

n(N − f (θ)) → N(0, V), where V = [υij ], υii = fi (θ )(1 − fi (θ )), υij = −fi (θ)fj (θ ), i = j,

and rank (V) = k − 1. Now suppose that θ ∗n is any estimator for θ which can be expressed in the locally, suitably regular functional form θ ∗n = g(N). Let D = [dij ], dij = ∂gi (N)/∂Nj , i = 1, 2, . . . , q; j = 1, 2, . . . , k, Q = [qrs ], qrs = ∂fr (θ )/∂θs , r = 1, 2, . . . , k; s = 1, 2, . . . , q. Then √ P √ n(N − f (θ ∗n )) → n(I − QD)(N − f (θ )) L

→ N(0, ),

Nj [ln nfj (θˆ n ) − ln Nj ]

L

→ χ 2 (k − q − 1).

3

where  = (I − QD)V(I − QD) . If  g is any generalized inverse* of  (i.e.,  g  = ), it then follows that, under H 0 , Qn (θ ∗n ) = n(N − f (θ ∗n ))  g (N − f (θ ∗n )) L

→ χ 2 (k − q − 1).

4

GOODNESS OF FIT

The power of tests such as (5), which include (8) as a special case, can also be examined using sequences of local alternatives, similar arguments, and the noncentral chi-square* distribution. However, such studies are of limited use, as tests of fit should detect broad alternatives, a performance feature that can be checked only by computer simulations. Note that  g may be a function of θ , which can be replaced by θ ∗n or any other consistent estimator for θ without affecting the asymptotic distribution of Qn (θ ∗n ). For an early, rigorous account of the theory of chi-square tests*, see Cram´er (6). A modern and comprehensive treatment of this theory with many ramifications is given by Moore and Spruill (24). Their paper is technical and covers cases where the Xi are random vectors and θ is estimated by a variety of methods. An easy-to-read over-view of this work is given by Moore (23). 2

CONTINUOUS RANDOM VARIABLES

The testing of goodness-of-fit hypotheses when f X (x, θ ) is a continuous function of x introduces features not exhibited by the discrete tests discussed in the section ‘‘Discrete Random Variables.’’ However, by suitable constructions some of the latter tests remain useful. These points will be expanded on below. 2.1 Simple H0 In order to apply results of the section ‘‘Discrete Random Variables,’’ partition the real line into k ≥ 2 sets: I1 = (−∞, a1 ],

Ij is used in the tests. The k classes are usually chosen to keep the npj acceptably high. In order to achieve some standardization, it seems reasonable to use pj = k−1 and to determine the ai by the equations F X (a1 ) = k−1 , F X (a2 ) − F X (a1 ) = k−1 , etc. (see Kendall and Stuart (18)). Nevertheless, there remains an essential nonuniqueness aspect to the tests. Given the same set of data, different statisticians can reach different conclusions using the same general procedures. In fact, these tests condense the data and examine whether or not gX (x) is a member of the particular class of density functions with given content pj for Ij . Despite these drawbacks, the approach outlined above has enjoyed wide support and is most commonly used in practice. The method of condensation of data presented above when X is a continuous random variable may also have to be practiced when X is discrete. In this case subsets of X are used in place of individual elements to achieve cell expectations sufficiently large to render the asymptotic distribution theory valid. A useful way of visually checking the adequacy of H 0 is to examine the order statistics X  1 , X  2 , . . . , X  n . Since f X (x) is fully specified, E[X  i ] = ηi can be calculated and plotted against X  i . If H 0 holds, this plot should be roughly linear. There are analytical counterparts to the simple order statistics plots. Let 0 < λ1 < λ2 < · · · < λk < 1, ni = [nλi ] + 1, where [x] is the greatest integer less than or equal to x, and consider Xn i , i = 1, 2, . . . , k. Under suitable regularity conditions on f X (x),

I2 = (a1 , a2 ], . . . ,

Ik = (ak−1 , ∞). To test H 0 : gX (x) = f X (x), let N j be the number of X i ∈ Ij in the sample and put pj = Ij f X (x)dx. Then, under H 0 , the N j have a multinomial distribution* with parameters
pj , n = kj=1 Nj and any of the tests from the discussion of simple H 0 in the section ‘‘Discrete Random Variables’’ can be applied. Clearly, making a situation discrete which is essentially continuous leads to a loss of precision. The actual values of the X i are suppressed and only their relationship with the

Y2 = n

k  {[FX (Xn i ) − FX (Xn i−1 )] − pi }2 p−1 i i=1

L

→ χ 2 (k − 1),

where pi = λi − λi−1 (4). This bypasses the problem of constructing intervals Ij and uses part of the natural ordering of the sample. A number of tests make specific use of the sample CDF, Gn (x), for testing H 0 : GX (x) = F X (x). Some of these are now listed.

GOODNESS OF FIT

2.1.1 Kolmogorov–Smirnov* Dn − , Dn + , and Dn . Define

Statistics,

D+ n

Wn2 = n

= supx [Gn (x) − FX (x)],

Dn = max(Dn − , Dn + ). Then Dn − and Dn + can be used to test H 0 against the one-sided alternatives H 1 : GX (x) ≤ F X (x) and H 1 : GX (x) ≥ F X (x), respectively, while Dn tests H 1 : GX (x) = F X (x). The CDFs of the three statistics are known exactly and are independent of F X (x) (8). To see this, let U = F X (X); then Dn + = sup0≤u≤1 Gn (u) − u, etc. The most useful set of tables is given by Pearson and Hartley (25), who also include some numerical examples. A derivation of the asymptotic distributions of Dn + and Dn can be based on the stochastic process yn (t) =

√ n(Gn (t) − t),

0 ≤ t ≤ 1,

(5)

which has zero mean and C[y(s), y(t)] = min(s, t) − st,

0 ≤ s, t ≤ 1. (6) The central limit theorem* ensures that [yn (t1 ), yn (t2 ), . . . , yn (tk )] is asymptotically multinormal with null mean vector and the above covariance structure. Thus the finitedimensional distributions of Y n (t) converge to those of y(t), tied-down Brownian motion*. Intuitively, the distributions of supt yn (t) and supt —yn (t)— will tend to those of supt y(t) and supt —y(t)—. This can be verified using the theory of weak convergence. The two crossing problems thus generated can be solved to yield the desired limiting CDFs (8). For a different approach, see Feller (13). It is interesting that these investigations show 4n(Dn + )2 to be asymptotically distributed as χ 2 (2). 2.1.2 Cram´er–von Mises Test*. Let  ∞ [Gn (X) − FX (x)]2 dx; Wn2 = n

By means of the probability transformation U = F X (X), (8) can be written 

D− n = supx [FX (x) − Gn (x)],

(7)

−∞

then Wn2 is a measure of the agreement between Gn (x) and F X (x) for all x and is known as the Cram´er–von Mises statistic.

5

1

[Gn (u) − u]2 du,

(8)

0

emphasizing that this test is also distribution-free. The CDF of Wn2 is not known for all n but has been approximated; the asymptotic distribution is derived in Durbin (8). For easy-to-use tables, see Pearson and Hartley (25). 2.1.3 Tests Related to the Cram´er–von Mises Test. Various modifications of Wn2 are used for specific purposes. For instance, a weight  1 function ψ(t) can be introduced to give 0 [Gn (t) − t]2 ψ(t) dt as a test statistic. When ψ(t) = [t(1 − t)]−1 , the resulting statistic is called the Anderson–Darling statistic, A2n , and leads to the Anderson–Darling test*. Since E[n[Gn (t) − t]2 ] = [t(1 − t)], this weights discrepancies by the reciprocal of their standard deviations and puts more weight in the tails of the distribution, a feature that may be important. The same remarks made for Wn2 apply to A2n . A number of scientific investigations yield data in the form of directions and it may be required to test the hypothesis that these are orientated at random. Since each direction is represented by an angle measured from a fixed position P, such data can be represented as points on a unit circle. The test then concerns the randomness of the distribution of the points on this circle. Watson (38) introduced the statistic  Un2 = n

1

[Gn (t) − t − Gn (t) − t]2 dt,

(9)

0

1 where 0 [Gn (t) − t]dt = Gn (t) − t. It can be shown that Un2 is independent of the choice of P. The asymptotic distribution of Un2 is known (8) and appropriate tables may be found in Pearson and Hartley (25). Under H 0 : GX (x) = F X (x), the variables U i = F X (X  1i ) are distributed as U [0, 1]. Hence Gn = 0 Gn (u) du = 1 − U, has expectation 12 , variance (12n)−1 and tends rapidly to normality. This provides a direct large-sample test

6

GOODNESS OF FIT

of H 0 , although exact significance points are available (34). Tests related to Dn − , Dn + , Dn , and Wn2 have been proposed by Riedwyl (31). He defines the ith discrepancy as di = F(X  i )  ) and examines tests based on − F n (X i
n
n 2
n d , etc. 1 i 1 |di |, 1 di , maxi di , maxi |di |, Some pertinent exact and asymptotic results are given. Hegazy and Green (15) considered tests
based on the forms T1 = n−1 n1 |Xi − νi | and
T2 = n−1 n1 (Xi − νi )2 , where ν i = ηi and ν i = ξ i , the mode of X  i . Tests of the hypothesis H 0 : GX (x) = F X (x) can be reduced as shown above to testing whether or not U i = F X (X i ) is distributed U [0, 1]. Thus ηi = i/(n + 1) and ξ i = (i − 1)/(n − 1). The powers of these T tests were examined against normal*, Laplace*, exponential*, and Cauchy* alternatives and compared with the powers of other tests. The conclusion was that T 1 and T 2 have similar performances and that it is slightly better to use ξ i than ηi . These T statistics generally compare favorably with the tests just described, or minor modifications of them. Hegazy and Green (15) provide an extensive bibliography of other studies of power of goodness-of-fit tests. 2.2 Composite H0 The most common hypothesis that requires testing is H 0 : GX (x) = F X (x, θ ) for some θ ∈
. The introduction of nuisance parameters* creates new technical difficulties which can only be touched on briefly here. In general, however, the same form of tests as those just presented are used, with modifications. In order to make use of the results in the discussion of composite H 0 in the section ‘‘Discrete Random Variables,’’ k intervals are introduced as in the preceding subsection. The interval contents are functions of θ ,  pj (θ ) = I f (x, θ ) dx and if N j is the number j of X i in Ij , a multinomial system is generated, the parameters being functions of the unknown θ . The whole problem may now be treated by the methods of the section on discrete variables, and the same comment concerning loss of information and nonuniqueness due to grouping applies.

A number of special points need emphasis. The estimation of θ must be made from the data in the grouped state if the distribution theory of the section on discrete variables is to hold. For instance, θ estimated from the X i and f (x, θ ) should not be used in the X 2 (θ ) statistic. Doing so results in a limiting CDF which depends on θ and a conservative test if 2 (k − q − 1) significance level is used. the χ1−α Since θ is not known, there is some difficulty defining the intervals Ij . In general, the boundaries of the intervals are functions of θ ; Moore and Spruill (24) have shown that, provided that consistent estimators of the boundary values are used, the asymptotic results (4), (8), and (5) remain valid if the random intervals are used as if they were the true ones. For example, reconsider the problem of testing H 0 : GX (x) = N X (x, θ ). Consistent
estimators of θ 1 = µ and θ 2 = σ 2 are X = n1 Xi /n
and S2 = n1 (Xi − X)2 /(n − 1) and it is appropriate that the Ij be constructed with X and S in place of µ and σ to ensure approximate contents of k−1 . Using these estimated intervals, the procedure requires that µ and σ 2 be estimated efficiently, by maximum likelihood, for instance, and the tests applied in the usual way. A test developed by Moore (22) and suggested by Rao and Robson (30) has interesting flexibility and power potential. Let Vn (θ ) be a k-vector with ith component (Ni − nfi (θ))/ nfi (θ), B(θ ) a k × q matrix with elements pi (θ )−1/2 ∂pi (θ )/∂θ j , and J(θ ) the usual information matrix for F X (x, θ ). Define the statistic Tn (θˆ n ) = Vn (θˆ n )[I − B(θˆ n )J−1 (θˆ n ){B(θˆ n )} ]−1 ×Vn (θˆ n ), where θˆn is the ungrouped MLE for θ ; then L Tn (θˆ n ) → χ 2 (k − 1). The problem of estimating intervals can be bypassed by the use of quantile* statistics. Define λi and pi as for (6), and the statistic Yn2 (θ ) = n

k  {[FX (Xn i , θ ) 1

−FX (Xn i−1 , θ)] − pi }2 p−1 i .

GOODNESS OF FIT L If θ = θ n minimizes Yn2 (θ), then Y 2 (θ˜ n ) → χ 2 (k − q − 1). Alternatively, the following test is available. Put Xni /n = N∗i and let N* be the (k × 1) vector of the N∗i ; then it is well known that √ L n(N∗ − ν) → N(0, V), where ν i is defined by F X (ν i ) = λi and υ ij = λi (1 − λj )[f X (ν i )f X (ν j )]−1 , i ≤ j. In general, both ν and V are functions of unknown θ , so define

An (θ ) = n(N∗ − ν(θ)) V−1 (θ ) ×(N∗ − ν(θ)) and choose θ = θ ∗n to minimize An (θ ). Then L

An (θ ∗n ) → χ 2 (k − q), k > q. If q = 2, θ 1 and θ 2 are location and scale parameters, respectively, and an explicit expression exists for θn∗ . The matrix V for the standardized variable (X − θ 1 )/θ 2 can be used in (12) and a single matrix inversion is needed to complete the test (37). The tests described in the discussion of simple H 0 do not extend readily to composite hypotheses. In general, for the cases considered and reported in the literature to date, the resulting tests are not distribution-free but depend on F(x, θ ) and on the method used ˆ This is because the CDF to estimate θ , θ. has a different limiting distribution when the parameters are estimated to that which results when the null hypothesis is simple (9). Hence tables of critical values constructed for simple hypothesis cases cannot be used for testing composite hypotheses. In fact, different critical values are needed for each hypothesis tested; the tests are carried out ˆ in the expressions replacing F X (x) by F(x, θ) of the preceding section. 2.2.1 Kolmogorov–Smirnov Statistics Dn − , A technique for obtaining exact critical values was developed by Durbin (10), who applied it to obtain values for testing the composite hypothesis of exponentiality

Dn + , Dn .

H0 : f (x, θ ) = θ −1 exp(−x/θ ), 0 < x, θ ∈ (0, ∞). The technique is complicated, however, and has not been applied to other cases.

7

By a variety of techniques, including Monte Carlo methods*, Stephens (35) has given procedures for finding accurate critical values for testing composite hypotheses involving the normal and the exponential distributions. These procedures are also described by Pearson and Hartley (25). For a treatment of this problem using sufficient statistics*, see Kumar and Pathak (20). 2.2.2 Cram´er–von Mises Statistic∗ Wn2 . No technique is yet available for obtaining exact critical values of Wn2 for testing composite hypotheses. The first accurate calculations of asymptotic significance points for testing exponentiality and normality were made by Durbin et al. (12). Further extensions and related results were given by Stephens (36). Again, methods of obtaining good approximations to finite-sample critical values for tests of exponentiality and normality are given by Stephens (35). 2.2.3 Tests Related to the Cram´er–von Mises Tests. Similar treatments to those of Wn2 are given to A2n and Un2 by Stephens (35,36) and Durbin et al. (12) for testing exponentiality and normality. In summary, then, the development of tests of fit for composite hypotheses using the sample CDF has centered largely on the exponential and the normal distributions. The most useful reference for the practitioner is Stephens (35), where tables cater for most of the common tests when the hypothesis is simple, and for the composite hypothesis cases mentioned above. If the data are censored*, see Pettitt and Stephens (27) and Dufour and Maag (7). 3

FURTHER TESTS AND CONSIDERATIONS

In this final section a few other special tests and some additional ideas impinging on goodness of fit will be mentioned. 3.1 Two Sample Tests Let X 1 , X 2 , . . . , X m and Y 1 , Y 2 , . . . , Y n , m ≤ n, be random samples from two different populations with continuous CDFs F X (x) and F Y (y) and sample CDFs FXm (t) and FYn (t).

8

GOODNESS OF FIT

In analogy with the discussion on the Kolmogorov–Smirnov statistics Dn − , Dn + , and Dn pertaining to simple H 0 in the section ‘‘Continuous Random Variables,’’ the hypothesis H 0 : F X (t) = F Y (t) can be tested against the alternatives H 1 : F X (t) ≤ F Y (t), H 1 : F X (t) ≥ F Y (t), and H 1 : F X (t) = F Y (t) by the respective statistics

the power properties. Nevertheless, a general pattern emerged; singly, the two statistics are useful for detecting departures from specific types of alternatives, and in combination they are reasonably robust against a large variety of alternatives.

n m D− mn = supt [FY (t) − FX (t)],

There are fruitful extensions to the technique of plotting order statistics* against expectation as introduced in the discussion of simple H 0 in the section ‘‘Continuous Random Variables.’’ Let the hypothesis be

m n D+ mn = supt [FX (t) − FY (t)], + Dmn = max[D− mn , Dmn ].

The exact distributions of these statistics are known; for finite sample critical points of Dmn , see Pearson and Hartley (25). For further references to tabulations, see Steck (33). If the statistics above are multiplied by [mn(m + n)−1 ]1/2 limiting distributions √ exist n Dn − , which are the same as those for √ √ + n Dn , and n Dn . Similar modifications can be made to the Cram´er–von Mises statistic to cater for two sample tests; again, see Durbin (8). 3.2 Tests of Departure from Normality* In view of the central role of the normal distribution in statistical theory and practice, a great deal of effort has been spent in developing tests of normality (see the first section of this article). Some of these tests have been dealt with in previous sections; only special tests tailored for the normal distribution will be covered here.
Let mr = n−1 n1 (Xi −√X)r and S2 = nm2 (n − 1)−1 ; then statistics b1 = m3 /S3 and b2 = m4 /S4 measure skewness and kurtosis in the sample. If the population √ from which the sample is drawn is normal, b1 and b2 should be near 0 and 3, respectively, and departure from these values is evidence to the contrary. √ Both b1 and b2 have been examined separately, jointly, and as a linear combination (see Pearson and Hartley (25) and Pearson et al. (26)). Both were compared with other tests for power; a variety of skewed and leptokurtic distributions were used as alternatives. The picture is somewhat confused, due in part to the wide spectrum of alternative distributions used and to the use of small numbers of Monte Carlo trials to establish

3.3 Wilks–Francia Test*

H0 : G(x, θ ) = F((x − θ1 ) θ2 ),

(10)

i.e., G is determined up to a location and scale parameter. Then a plot of X  i against the expectation of the standardized order statistics, ηi , should lie near the line θ 1 + θ 2 ηi under H 0 . Now, the unweighted esti
least-squares
mator for θ 2 is θ 2 = n1 Xi ηi / n1 ηi2 and the residual sum of squares is R2n =

n 

(Xi − X − θ 2 ηi )2

1

=

n 

 (Xi

− X) − 2

1

n 

2 bi Xi

,

1


where bi = ηi /( ni ηi2 )1/2 . Dividing both sides
n  2 by 1 (Xi − X) to remove the scale effect yields  n 

−1 (Xi

1

=1−

− X) 

2

n  1

R2n 2

bi Xi

n



(Xi − X)2

1

= 1 − Wn . Then W  n is the Wilks–Francia test statistic and measures the departure of the order statistics from their expectations; it has been used to test normality specifically, but it clearly enjoys a wider application. To carry out the test, tables of ηi are required as well as critical points; reference is made to the original paper by Shapiro and

GOODNESS OF FIT

Francia (32). Note that small values of W  n are significant and that the test has been shown to be consistent. An asymptotic distribution for the test has been established (see Durbin (11), where further tests of fit using order statistics are discussed). For completeness, it is pointed out that there exist goodness-of-fit procedures using the differences, or spacings*, between successive order statistics. Some of these tests are reviewed by Pyke (28), who developed the limiting distributions of functions of spacings and certain general limit theorems. More recent work in this area is reported by Kale (17) and Kirmani and Alam (19). 4

FINAL REMARKS

A few general considerations are worth raising. The single sample procedures outlined in previous sections deal with the problem of testing H 0 : GX (x) = F X (x, θ ), where θ is either fixed or is specified only up to a set
. If θ is not fixed, an estimator θˆ is substituted for it in F and the concordance of this estimated model with the data assessed. It is important in carrying out tests of fit not to lose sight of the fundamental purpose of the exercise. For example, tests of normality are often required as an intermediate step to further analyses. Alternatively, the performance of specific statistical processes, such as a random number generator, may need to be checked against specification. In these instances, the philosophy of using F, or a good estimate of it, to test against available data seems entirely reasonable. A different situation is generated if predictions* are required. In this case an estimate of F is to be used to predict future outcomes of the random variable X. It is possible that FX (x, θˆ ) may allow satisfactory predictions to be made, especially if the model was appropriate and θˆ based on a large sample. But there may be other candidates which would do a better job of ˆ such as to set up prediction than FX (x, θ), a measure of divergence of one PDF from another (see Ali and Silvey (3)) and then to try to find that PDF, based on the data, which comes closest to the estimated PDF. This treatment may need Bayesian arguments to construct predictive densities (2).

9

More specifically, let f X (x, θ ) be the density which is to be estimated and introduce the weight function p(θ —z) on
based on data z. Put  p(θ |z)fX (x, θ ) dθ ; (11) qX (x|z) =


then qX is called a predictive density for f X . On the other hand, for any estimator for θ based on z, θˆ (z), fX (x, θˆ (z)) is called an estimative density. Using the Kullback–Leibler directed measure of divergence, Aitchison (2) showed that qX (x—z) is optimal in the sense that it is closer to f X than any other competing density, in particular fX (x, θˆ (z)). Although this result may depend on the divergence measure used, it shows that fX (x, θˆ (z)) may not always be the appropriate estimator for f X (x, θ ). A chi-squared type of goodness-of-fit test for the predictive density has been developed by Guteman (14). REFERENCES 1. References are classified as follows: (A), applied; (E), expository; (R), review; (T), theoretical. 2. Aitchison, J. (1975). Biometrika, 62, 547–554. (T) 3. Ali, S. M. and Silvey, S. D. (1966). J. R. Statist. Soc. B, 28, 131–142. (T) 4. Bofinger, E. (1973). J. R. Statist. Soc. B, 35, 277–284. (T) 5. Conover, W. J. (1972). J. Amer. Statist. Ass., 67, 591–596. (T) 6. Cram´er, H. (1945). Mathematical Methods of Statistics. Princeton University Press, Princeton, N.J. (E) 7. Dufour, R. and Maag, U. R. (1978). Technometrics, 20, 29–32. (A) 8. Durbin, J. (1973). Distribution Theory for Tests Based on the Sample Distribution Function. Reg. Conf. Ser. Appl. Math. SIAM, Philadelphia. (T, R) 9. Durbin, J. (1973). Ann. Statist., 1, 279–290. (T) 10. Durbin, J. (1975). Biometrika, 62, 5–22. (T) 11. Durbin, J. (1977). Goodness-of-fit tests based on the order statistics. Trans. 7th Prague Conf. Inf. Theory, Statist. Decision Functions, Random Processes / 1974 Eur. Meet. Statist., Prague, 1974, Vol. A, 109–118. (R)

10

GOODNESS OF FIT

12. Durbin, J., Knott, M., and Taylor, C. C. (1975). J. R. Statist. Soc. B, 37, 216–237. (T) 13. Feller, W. (1948). Ann. Math. Statist., 19, 177. (T) 14. Guteman, I. (1967). J. R. Statist. Soc. B, 29, 83–100. (T) 15. Hegazy, Y. A. S. and Green, J. R. (1975). Appl. Statist., 24, 299–308. (A) 16. Horn, S. D. (1977). Biometrics, 33, 237–248. (A, R) 17. Kale, B. K. (1969). Sankhya¯ A, 31, 43–48. (T) 18. Kendall, M. G. and Stuart, A. (1973). The Advanced Theory of Statistics, Vol. 2: Inference and Relationship, 3rd ed. Hafner Press, New York. (E) 19. Kirmani, S. N. U. A. and Alam, S. N. (1974). Sankhya¯ A, 36, 197–203. (T) 20. Kumar, A. and Pathak, P. K. (1977). Scand. Statist. J., 4, 39–43. (T) 21. Lindgren, B. W. (1976). Statistical Theory, 3rd ed. Macmillan, New York. (E) 22. Moore, D. S. (1977). J. Amer. Statist. Ass., 72, 131–137. (T) 23. Moore, D. S. (1979). In Studies in Statistics, R. V. Hogg, ed. Mathematical Association of America, Washington, D.C., pp. 66–106. (T, E) 24. Moore, D. S. and Spruill, M. C. (1975). Ann. Statist., 3, 599–616. (T) 25. Pearson, E. S. and Hartley, H. O. (1972). Biometrika Tables for Statisticians, Vol. 2. Cambridge University Press, Cambridge. (E) 26. Pearson, E. S., D’Agostino, R. B., and Bowman, K. O. (1977). Biometrika, 64, 231–246. (A) 27. Pettitt, A. N. and Stephens, M. A. (1976). Biometrika, 63, 291–298. (T) 28. Pyke, R. (1965). J. R. Statist. Soc. B, 27, 395–436. (T) 29. Radlow, R. and Alf, E. F. (1975). J. Amer. Statist. Ass., 70, 811–813. (A) 30. Rao, K. C. and Robson, D. S. (1974). Commun. Statist., 3, 1139–1153. (T) 31. Riedwyl, H. (1967). J. Amer. Statist. Ass., 62, 390–398. (A) 32. Shapiro, S. S. and Francia, R. S. (1972). J. Amer. Statist. Ass., 67, 215–216. (A) 33. Steck, G. P. (1969). Ann. Math. Statist., 40, 1449–1466. (T) 34. Stephens, M. A. (1966). Biometrika, 53, 235–240. (T) 35. Stephens, M. A. (1974). J. Amer. Statist. Soc., 69, 730–743. (A, E)

36. Stephens, M. A. (1976). Ann. Statist., 4, 357–369. (T) 37. Tallis, G. M. and Chesson, P. (1976). Austr. J. Statist., 18, 53–61. (T) 38. Watson, G. S. (1961). Biometrika, 48, 109–114. (T) 39. Wood, C. L. and Altavela, M. M. (1978). Biometrika, 65, 235–239. (T)

GROUP-RANDOMIZED TRIALS

variance of any group-level statistic beyond what would be expected with random assignment of members to conditions. Moreover, with a limited number of groups, the degrees of freedom (df) available to estimate grouplevel statistics are limited. Any test that ignores either the extra variation or the limited df will have a Type I error rate that is inflated (3). This problem will only worsen as the ICC increases (4–6). As a result of these problems, RCTs are preferred over GRTs whenever randomization of individual participants is possible. However, individual randomization is not always possible, especially for many public health interventions that operate at a group level, manipulate the physical or social environment, or cannot be delivered to individuals. Just as the RCT is the gold standard in public health and medicine when allocation of individual participants is possible, the GRT is the gold standard when allocation of identifiable groups is necessary. The purpose of this article is to put GRTs in context in terms of other kinds of designs and in terms of the terminology used in other fields, to summarize their development in public health, to characterize the range of public health research areas that now employ GRTs, to characterize the state of practice with regard to the design and analysis of GRTs, to consider their future in public health research, and to review the steps required to plan a new GRT.

DAVID M. MURRAY The Ohio State University Division of Epidemiology School of Public Health Columbus, Ohio

1

INTRODUCTION

Group-randomized trials (GRTs) are comparative studies used to evaluate interventions that operate at a group level, manipulate the physical or social environment, or cannot be delivered to individuals (1). Examples include school-, worksite-, and communitybased studies designed to improve the health of students, employees, or residents. Four characteristics distinguish the GRT from the more familiar randomized clinical trial (RCT) (1). First, the unit of assignment is an identifiable group; such groups are not formed at random, but rather through some physical, social, geographic, or other connection among their members. Second, different groups are assigned to each condition, creating a nested or hierarchical structure for the design and the data. Third, the units of observation are members of those groups so that they are nested within both their condition and their group. Fourth, only a limited number of groups assigned to each condition usually exists. Together, these characteristics create several problems for the design and analysis of GRTs. The major design problem is that a limited number of often heterogeneous groups makes it difficult for randomization to distribute potential sources of confounding evenly in any single realization of the experiment, which increases the need to employ design strategies that will limit confounding and analytic strategies to deal with confounding where it is detected. The major analytic problem is that an expectation exists for positive intraclass correlation (ICC) among observations on members of the same group (2). That ICC reflects an extra component of variance attributable to the group above and beyond the variance attributable to its members. This extra variation will increase the

2 GROUP-RANDOMIZED TRIALS IN CONTEXT GRTs represent a subset of a larger class of designs often labeled nested, hierarchical, multilevel, or clustered designs. Units of observation are nested within identifiable groups or clusters, which are in turn nested within study conditions. This description defines a hierarchy of at least three levels in the design: units of observation, units of assignment, and study conditions. More complex designs may have even more levels. For example, in cohort or repeated measures designs, repeat observations are further nested within units of observation.

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

GROUP-RANDOMIZED TRIALS

As used here, the label group-randomized trial refers to a design in which identifiable groups are assigned to study conditions for the express purpose of assessing the impact of one or more interventions on one or more endpoints. The terms nested, hierarchical, multilevel, and clustered designs can be used more broadly to refer to any dataset that has a hierarchical structure, and these more general terms are often used to characterize observational studies as well as comparative studies. Many examples of observational and comparative hierarchical designs can be found in education, where students are nested within classrooms, which are nested within schools, which are nested within school districts, which are nested within communities, and so on. Investigators in education often refer to such designs as hierarchical or multilevel designs (7–10). Other examples can be found in survey sampling, and in disciplines that employ surveys, such as epidemiology, sociology, and demography. In these disciplines, cluster sampling is a commonly used technique (2). Cluster-sampling designs can be a good way to limit cost when the investigator lacks a complete enumeration of the population of interest and does not want to expend the resources required to generate such an enumeration. As simple random sampling is impossible without a complete enumeration, clusters such as blocks or neighborhoods or other identifiable groups are enumerated and sampled in a first stage, followed by enumeration and sampling of individuals within the selected clusters in a second stage. Applied properly, cluster-sampling methods can yield unbiased estimates of population rates or means at a lower cost than would have been the case with simple random sampling. Unfortunately, cluster sampling invariably leads to increased variation and often to limited degrees of freedom. These problems are well known to survey-sampling statisticians (2, 11–13). Biostatisticians often use the term clusterrandomization study to refer to a grouprandomized trial (14, 15). This terminology is based on the fact that an identifiable group is a cluster. It borrows from the terminology of survey sampling. With the

broad definition given to ‘‘group’’ in this text, the phrases cluster-randomization study and group-randomized trial are equivalent. Epidemiologists have often used the terms community trial and community-intervention trial (16–18). These terms emerged from the community-based heart disease prevention studies of the late 1970s and the 1980s (19–23). None of those studies were randomized trials, but all involved whole communities as the unit of assignment with collection of data from individuals within those communities. Community trial is an attractive label, because it includes both randomized designs and nonrandomized designs. However, it is often thought to refer only to studies that involve whole communities (e.g., Reference 24), and so creates confusion when applied to studies involving other identifiable groups. 3 THE DEVELOPMENT OF GROUP-RANDOMIZED TRIALS IN PUBLIC HEALTH GRTs gained attention in public health in the late 1970s with the publication of a symposium on coronary heart disease prevention trials in the American Journal of Epidemiology (3, 21, 25–27). Cornfield’s paper in particular has become quite famous among methodologists working in this area, as it identified the two issues that have vexed investigators who employ GRTs from the outset: extra variation and limited degrees of freedom. The last 25 years have witnessed dramatic growth in the number of GRTs in public health and dramatic improvements in the quality of the design and analysis of those trials. Responding directly to Cornfield’s warning, Donner and colleagues at the University of Western Ontario published a steady stream of papers on the issues of analysis facing group-randomized trials beginning in the early 1980s. Murray and colleagues from the University of Minnesota began their examination of the issues of design and analysis in group-randomized trials in the mid-1980s. Other investigators from the National Institutes of Health, the University of Washington, the New England Research Institute, and elsewhere added to

GROUP-RANDOMIZED TRIALS

this growing literature in public health, especially in the 1990s. By the late 1980s and early 1990s, many group-randomized trials were under way that were of very high quality in terms of their methods of design and analysis. Examples include the Community Intervention Trial for Smoking Cessation (COMMIT) (28), the Working Well Trial (29), and the Child and Adolescent Trial for Cardiovascular Health (CATCH) (30). These improvements occurred as investigators and reviewers alike gradually came to understand the special issues of design and analysis that face grouprandomized trials and the methods required to address them. Unfortunately, the improvements were not represented in all group-randomized trials. Even in the 1990s, grants were funded and papers were published based on poor designs and poor analyses. Simpson et al. (31) reviewed GRTs that were published between 1990 and 1993 in the American Journal of Public Health (AJPH) and in Preventive Medicine (Prev Med). They reported that fewer than 20% dealt with the design and analytic issues adequately in their sample size or power analysis and that only 57% dealt with them adequately in their analysis. In 1998, the first textbook on the design and analysis of GRTs appeared (1). It detailed the design considerations for the development of GRTs, described the major approaches to their analysis both for Gaussian data and for binary data, and presented methods for power analysis applicable to most GRTs. The second textbook on the design and analysis of GRTs appeared in 2000 (15). It provided a good history on GRTs, examined the role of informed consent and other ethical issues, focused on extensions of classical methods, and included material on regression models for Gaussian, binary, count, and time-to-event data. Other textbooks on analysis methods germane to GRTs appeared during the same period (10, 32, 33). Murray et al. recently reviewed a large number of articles on new methods relevant to the design and analysis of GRTs published between 1998 and 2003 (34). Of particular importance for investigators planning GRTs has been the increased availability of estimates of ICC for a variety of

3

endpoints in a variety of groups. Donner and Klar reported a number of ICCs in their text (15). Murray and Blitstein (35) identified more than 20 papers published before 2002 that reported ICCs, whereas Murray et al. identified a similar number of papers published between 1998 and 2003 that reported ICCs (34). Murray and Blitstein also reported on a pooled analysis of ICCs from worksite, school, and community studies. They confirmed earlier reports that the adverse impact of positive ICC can be reduced either by regression adjustment for covariates or by taking advantage of over-time correlation in a repeated-measures analysis. More recently, Janega et al. provided additional documentation that standard errors for intervention effects from end-of-study analyses that reflect these strategies are often different from corresponding standard errors estimated from baseline analyses that do not (36, 37). As the ICC of concern in any analysis of an intervention effect is the ICC as it operates in the primary analysis (1), these findings reinforce the need for investigators to use estimates in their power analysis that closely reflect the endpoints, the target population, and the primary analysis planned for the trial. 4

THE RANGE OF GRTS IN PUBLIC HEALTH

Varnell et al. examined GRTs published between 1998 and 2002 in the AJPH and in Prev Med (38). They found 58 GRTs published in just those two journals during that 5-year period. That rate of 11.6 articles per year was double the publication rate observed for GRTs in the same journals between 1990 and 1993 in the earlier review by Simpson et al. (31). Those trials were conducted for a variety of primary outcomes. The largest fraction (25.9%) targeted smoking prevention or cessation, with trials focused on dietary representing the next largest fraction (20.7%). Studies designed to evaluate screening programs, particularly for cancer, represented the next largest fraction (12.1%). Other studies focused on alcohol or drug use, or on a combination of tobacco, alcohol, and drug use (8.6%); on sun protection (5.2%); on physical or sexual abuse (3.4%); on physician preventive practices (3.4%); on work place health

4

GROUP-RANDOMIZED TRIALS

and safety measures (3.4%); or on multiple health outcomes (8.6%). Those trials were also conducted in a variety of settings. The largest fraction (29.3%) were conducted in schools or colleges. Worksites accounted for 19.0% of the trials. Medical practices accounted for 15.5%, whereas communities accounted for 12.1%. Trials were also conducted in housing projects (5.2%) or churches (5.2%), and 13.8% were conducted in other settings. The size of the group used in these GRTs varied considerably. Almost 30% were conducted in groups having more than 100 members, 24.1% in groups having 51 to 100 members, 32.8% in groups having 10 to 50 members, and 13.8% in groups having fewer than 10 members. Most of these trials employed a pretest-post-test design (55.2%), whereas 13.8% relied on a post-test-only design; 29.3% employed designs having three time points; and 12% involved more than three time points. Most of the trials employed cohort designs (63.8%), whereas 20.7% were crosssectional and 15.5% used a combination of cohort and cross-sectional designs. As this pattern suggests, GRTs are applicable to a wide variety of problems within public health and may be used in a wide variety of settings. It remains the best design available whenever the intervention operates at a group level, manipulates the physical or social environment, or cannot be delivered to individuals. 5 CURRENT DESIGN AND ANALYTIC PRACTICES IN GRTS IN PUBLIC HEALTH As noted above, two textbooks now exist that provide guidance on the design and analysis of GRTs (1, 15), as well as more recent summary papers focused on these issues (39–43). These sources identify several analytic approaches that can provide a valid analysis for GRTs. In most, the intervention effect is defined as a function of a condition-level statistic (e.g., difference in means, rates, or slopes) and assessed against the variation in the corresponding grouplevel statistic. These approaches included mixed-model ANOVA/ANCOVA for designs

having only one or two time intervals, random coefficient models for designs having three or more time intervals, and randomization tests as an alternative to the modelbased methods. Other approaches are generally regarded as invalid for GRTs because they ignore or misrepresent a source of random variation, which include analyses that assess condition variation against individual variation and ignore the group, analyses that assess condition variation against individual variation and include the group as a fixed effect, analyses that assess the condition variation against subgroup variation, and analyses that assess condition variation against the wrong type of group variation. Still other strategies may have limited application for GRTs. Application of fixed-effect models with post hoc correction for extra variation and limited df assumes that the correction is based on an appropriate ICC estimate. Application of survey-based methods or generalized estimating equations (GEE) and the sandwich method for standard errors requires a total of 40 or more groups in the study, or a correction for the downward bias in the sandwich estimator for standard errors when fewer than 40 groups exist in the study (34). Varnell et al. recently reported on the state of current practice in GRTs with regard to design and analytic issues for the 58 GRTs published between 1998 and 2002 in the AJPH and Prev Med (38). They reported that only 15.5% provided evidence that they dealt with the design and analytic issues adequately in their sample size or power analysis, either in the published paper or in an earlier background paper. To qualify as providing evidence, the paper or background paper had to report the ICC estimate expected to apply to the primary analysis in the study, the variance components used to calculate that ICC, or a variance inflation factor calculated from that ICC. The vast majority of GRTs published in these two journals did not provide such evidence. More surprising, the proportion reporting such evidence was actually somewhat lower than reported by Simpson et al. when they reviewed the same journals for the period between 1990 and 1993 (31). Varnell et al. noted that it was possible that many of

GROUP-RANDOMIZED TRIALS

these studies performed such power calculations but did not report them. However, 27 (46%) of the reviewed studies had fewer than 10 groups per condition, and of that number, only one reported evidence of estimating sample size using methods appropriate for GRTs. Varnell et al. concluded that it was likely that many investigators reporting small GRTs planned their study without considering issues of sample size adequately. Varnell et al. reported that only 54.4% of the GRTs published between 1998 and 2002 in the AJPH and Prev Med dealt with design and analytic issues adequately in their analysis (38). Most of the studies that reported using only appropriate methods used mixedmodel regression methods (61.2%), whereas others used two-stage methods (35.4%) or GEE with more than 40 groups (6.4%). Of the remaining studies, 26.3% reported a combination of appropriate and inappropriate methods, whereas 19.3% reported using only inappropriate methods. The most widely used inappropriate methods were analysis at an individual level, ignoring group-level ICC; analysis at a subgroup level, ignoring grouplevel ICC; and use of GEE or another asymptotically robust method with fewer than 40 groups and no correction for the downward bias identified in those methods under those conditions. Varnell et al. reported appreciable differences between the AJPH and Prev Med with regard to these patterns (38). For example, among the 27 GRTs published in AJPH, 66.7% reported only analyses taking ICC into account properly, compared with 43.3% in Prev Med. At the same time, only 14.5% of the GRTs published in AJPH reported only inappropriate analyses, compared with 23.3% in Prev Med. The range of practices across other journals is likely to be even wider. Particularly surprising in this review was that 33.3% of the studies reviewed reported analyses considered inappropriate well before the studies were published, which suggests that some investigators, reviewers, and journal editors have not yet taken to heart the long-standing warnings against analysis at an individual or subgroup level that ignores the group-level ICC, or an analysis that includes group as a fixed effect.

5

Indeed, investigators continued to put forward false arguments to justify the use of inappropriate methods, including claims that appropriate methods could obscure important clinical changes, or that the observed ICCs were small and therefore ignorable. Such arguments ignore the methodological realities of the GRT and represent exactly the kind of self-deception that Cornfield warned against more than 25 years ago (3). 6 THE FUTURE OF GROUP-RANDOMIZED TRIALS Publication of disappointing results for several large GRTs in the mid-1990s, such as the Stanford Five City Project (44), the Minnesota Heart Health Program (45), the Pawtucket Heart Health Program (20), and COMMIT (46, 47), led some to question the value of GRTs in general and of community trials in particular. As noted at the time, it is both shortsighted and impractical to question GRTs in general based on disappointing results even from a number of large and expensive trials (48). Whenever the investigator wants to evaluate an intervention that operates at a group level, manipulates the social or physical environment, or cannot be delivered to individuals, the GRT is the best comparative design available. However, many challenges exist to the design and analysis of GRTs, and the investigator must take care to understand those challenges and the strategies that are available to meet them: The question is not whether to conduct GRTs, but rather how to do them well (1). Certainly, no decline in the number of GRTs has occurred since the mid-1990s. As reported by Varnell et al., the annual rate of publication of GRTs in AJPH and Prev Med during 1998 to 2002 was double that for the period 1990 to 1993. The number of GRTs proposed to NIH has also increased considerably over the past 10 to 15 years, and a study section now exists at NIH that counts GRT applications as one of its most common design types (Community Level Health Promotion, formerly known as SNEM-1). Even so, many challenges facing GRTs remain. For example, no question exists that

6

GROUP-RANDOMIZED TRIALS

it is harder to change the health behavior and risk profile of a whole community than it is to make similar changes in smaller identifiable groups such as those at worksites, physician practices, schools, and churches. And although no quantitative analysis has been published, it seems that the magnitude of the intervention effects reported for GRTs has been greater for trials that involved smaller groups than for trials involving such large aggregates as whole communities. With smaller groups, it is possible to include more groups in the design, thereby improving the validity of the design and the power of the trial. With smaller groups, it is easier to focus intervention activities on the target population. With smaller groups, the cost and difficulty of the implementation of the study generally are reduced. For these and similar reasons, future group-randomized trials may do well to focus on more and smaller identifiable groups rather than on whole cities or larger aggregates. Indeed, that pattern is evidenced in a number of recent trials (49–54). At the same time, positive effects have been reported for some studies involving very large aggregates. An example is the ASSIST project from the National Cancer Institute (55). Although this trial was not randomized, 17 states participated in the ASSIST intervention to reduce smoking, and the remaining 33 states plus the District of Columbia served as control sites. As such, the units of assignment in the ASSIST trial were even larger aggregates than are used in most GRTs. The encouraging results from that trial confirm that it is possible to successfully deliver an intervention even in very large aggregates. Another challenge is simply the difficulty in developing interventions strong enough to change the health behaviors of the target populations. This point is not new, but it is one that has been well known to investigators working in GRTs for some time (56–58). The methods for the design and analysis of GRTs have evolved considerably from the 1970s and 1980s; however, interventions continue to be employed that often prove ineffective. One of the problems for some time has been that interventions are proposed that lack even preliminary evidence of efficacy (42).

Efficacy trials in health promotion and disease prevention often are begun without the benefit of prototype studies, and often even without the benefit of adequate pilot studies, which has happened in large part because the funding agencies have been reluctant to support pilot and prototype studies, preferring instead to fund efficacy and effectiveness trials. Unfortunately, the interventions that lead to GRTs tend to be more complicated than those in other areas or those that lead to clinical trials. As such, it is even more important to subject them to adequate testing in pilot and prototype studies. These earlier phases of research can uncover important weaknesses in the intervention content or implementation methods. Moving too quickly to efficacy trials risks wasting substantial time and resources on interventions that could have been substantially improved through the experience gained in those pilot and prototype studies. Hopefully, the funding agencies will recognize this point and begin to provide better support for pilot and prototype studies. The R21 mechanism at NIH is wellsuited for that kind of pilot and feasibility work. Prototype studies will typically be based on only one or two groups per condition, and so are particularly problematic if the investigator wants to make causal inferences relating the intervention as delivered to the outcomes as observed. Studies based on only one group per condition cannot estimate variation because of the group independent of variation due to condition (59). Studies based on only a few groups per condition cannot estimate that component of variance with much accuracy. Some of the methods described earlier as having limited application to GRTs can be used, but all would require strong assumptions. For example, application of a post hoc correction would require the strong assumption that the external estimate of ICC is valid for the data at hand. Application of the subgroup or batch analysis would require the strong assumption that the subgroup captures the group variance. Does this mean that prototype studies should not be conducted? The answer is clearly no, and, more to the point, the answer must be that such studies simply should not be used to make causal inferences relating

GROUP-RANDOMIZED TRIALS

the intervention and the outcome. Studies involving only one or two groups per condition are prototype studies, not efficacy trials, and must be analyzed with that in mind. With only one or two groups per condition, the investigator can estimate the magnitude of the intervention effect, but it will not be possible to estimate a standard error for that effect with any degree of accuracy. As a result, it will not be possible to put a narrow confidence bound around the intervention effect or to draw any conclusions about the statistical significance of that effect. Even so, if the effect is much smaller than expected, that finding should push the investigators to rework their intervention, as it is not likely that a reasonably sized efficacy trial would show such a small intervention effect to be significant. If the effect is as large as expected or larger, that finding should provide good support to the investigator in seeking support for the efficacy trial required to establish the causal link between the intervention and the outcome. Methodological challenges remain as well. For example, a number of recent studies have documented the downward bias in the sandwich estimator used in GEE when fewer than 40 groups exist in the study (4, 60, 61). Some of these studies, and others, have proposed corrections for that estimator (61–66). Unfortunately, none of these corrections appear in the standard software packages, so they are relatively unavailable to investigators who analyze GRTs. Absent an effective correction, the sandwich estimator will have an inflated Type I error rate in GRTs having less than 40 groups, and investigators who employ this approach continue to risk overstating the significance of their findings. As another example, a number of recent studies have proposed methods for survival analysis that could be applied to data from GRTs (15, 67–71). Some of these methods involved use of the sandwich estimator, and so would be subject to the same concern as noted above for GEE. None of the methods appear in the standard software packages, so they also are relatively unavailable to investigators who analyze GRTs. As a third example, permutation tests have been advocated over model-based methods because they require fewer assumptions.

7

At the same time, they tend to have lower power. To overcome this problem, Feng et al. developed an optimal randomization test that had nominal size and better power than alternative randomization tests or GEE, although it was still not as powerful as the model-based analysis when the model was specified correctly (72). Additional research is needed to compare Braun and Feng’s optimal randomization test and model-based methods under model misspecification. Every reason exists to expect that continuing methodological improvements will lead to better trials. Evidence also exists that better trials tend to have more satisfactory results. For example, Rooney and Murray presented the results of a meta-analysis of group-randomized trials in the smokingprevention field (73). One of the findings was that stronger intervention effects were associated with greater methodological rigor. Stronger intervention effects were reported for studies that planned from the beginning to employ the unit of assignment as the unit of analysis, that randomized a sufficient number of assignment units to each condition, that adjusted for baseline differences in important confounding variables, that had extended follow-up, and that had limited attrition. One hopes that such findings will encourage use of good design and analytic methods. A well-designed and properly executed GRT remains the method of choice in public health and medicine when the purpose of the study is to establish the efficacy or effectiveness of an intervention that operates at a group level, manipulates the social or physical environment, or cannot be delivered to individuals. However, efficacy and effectiveness trials should occur in proper sequence. They should occur only after pilot studies have established the feasibility and acceptability of the materials and protocols for the intervention and evaluation. They should occur only after prototype studies have shown that the magnitude of the intervention effect is large enough to warrant the larger trials. Efficacy and effectiveness trials should be large enough to ensure sufficient power, with groups assigned at random from within wellmatched or stratified sets to protect against bias. Investigators should measure exposure

8

GROUP-RANDOMIZED TRIALS

and other process variables in all conditions. They should select a model for the analysis that reflects the design of the study and the nature of the endpoints. Importantly, investigators should be cautious of strategies that appear to easily solve or avoid the design and analytic problems that are inherent in group-randomized trials, for those methods are likely to prove to be inappropriate. 7 PLANNING A NEW GROUP-RANDOMIZED TRIAL The driving force behind any GRT must be the research question. The question will be based on the problem of interest and will identify the target population, the setting, the endpoints, and the intervention. In turn, those factors will shape the design and analytic plan. Given the importance of the research question, the investigators must take care to articulate it clearly. Unfortunately, that does not always happen. Investigators may have ideas about the theoretical or conceptual basis for the intervention, and often even clearer ideas about the conceptual basis for the endpoints. They may even have ideas about intermediate processes. However, without very clear thinking about each of these issues, the investigators may find themselves at the end of the trial unable to answer the question of interest. To put themselves in a position to articulate their research question clearly, the investigators should first document thoroughly the nature and extent of the underlying problem and the strategies and results of previous efforts to remedy that problem. A literature review and correspondence with others working in the field are ingredients essential to that process, as the investigators should know as much as possible about the problem before they plan their trial. Having become experts in the field, the investigators should choose the single question that will drive their GRT. The primary criteria for choosing that question should be: (1) Is it important enough to do?, and (2) Is this the right time to do it? Reviewers will ask both questions, and the investigators must be able to provide well-documented answers.

Most GRTs seek to prevent a health problem, so that the importance of the question is linked to the seriousness of that problem. The investigators should document the extent of the problem and the potential benefit from a reduction in that problem. The question of timing is also important. The investigators should document that the question has not been answered and that the intervention has a good chance to improve the primary endpoint in the target population, which is most easily done when the investigators are thoroughly familiar with previous research in the area; when the etiology of the problem is well known; when a theoretical basis exists for the proposed intervention; when preliminary evidence exists on the feasibility and efficacy of the intervention; when the measures for the dependent and mediating variables are well-developed; when the sources of variation and correlation as well as the trends in the endpoints are well understood; and when the investigators have created the research team to carry out the study. If that is not the state of affairs, then the investigators must either invest the time and energy to reach that state or choose another question. Once the question is selected, it is very important to put it down on paper. The research question is easily lost in the dayto-day details of the planning and execution of the study, and because much time can be wasted in pursuit of issues that are not really central to the research question, the investigators should take care to keep that question in mind. 7.1 The Research Team Having defined the question, the investigators should determine whether they have sufficient expertise to deal with all the challenges that are likely to develop as they plan and execute the trial. They should identify the skills that they do not have and expand the research team to ensure that those skills are available. All GRTs will need expertise in research design, data collection, data processing and analysis, intervention development, intervention implementation, and project administration. As the team usually will need to convince a funding agency that they are appropriate

GROUP-RANDOMIZED TRIALS

for the trial, it is important to include experienced and senior investigators in key roles. No substitute exists for experience with similar interventions, in similar populations and settings, using similar measures, and similar methods of data collection and analysis. As those skills are rarely found in a single investigator, most trials will require a team, with responsibilities shared among its members. Most teams will remember the familiar academic issues (e.g., statistics, data management, intervention theory), but some may forget the very important practical side of trials involving identifiable groups. However, to forget the practical side is a sure way to get into trouble. For example, a schoolbased trial that does not include on its team someone who is very familiar with school operations is almost certain to get into trouble with the schools. A hospital-based trial that does not include on its team someone who is very familiar with hospital operations is almost certain to get into trouble with the hospitals. And the same can be said for every other type of identifiable group, population, or setting that might be used. 7.2 The Research Design The fundamentals of research design apply to GRTs as well as to other comparative designs. As they are discussed in many familiar textbooks (11, 74–77), they will be reviewed only briefly here. Additional information may be found in two recent textbooks on the design and analysis of GRTs (1, 15). The goal in the design of any comparative trial is to provide the basis for valid inference that the intervention as implemented caused the result(s) as observed. Overall, three elements are required: (1) control observations must exist, (2) a minimum of bias must exist in the estimate of the intervention effect, and (3) sufficient precision for that estimate must exist. The nature of the control observations and the way in which the groups are allocated to treatment conditions will determine in large measure the level of bias in the estimate of the intervention effect. Bias exists whenever the estimate of the intervention effect is different from its true value. If that bias is substantial, the investigators will be misled

9

about the effect of their intervention, as will the other scientists and policy makers who use their work. Even if adequate control observations are available so that the estimate of the intervention effect is unbiased, the investigator should know whether the effect is greater than would be expected by chance, given the level of variation in the data. Statistical tests can provide such evidence, but their power to do so will depend heavily on the precision of the intervention effect estimate. As the precision improves, it will be easier to distinguish true effects from the underlying variation in the data.

7.3 Potential Design Problems and Methods to Avoid them For GRTs, the four sources of bias that are particularly problematic and should be considered during the planning phase are selection, differential history, differential maturation, and contamination. Selection bias refers to baseline differences among the study conditions that might explain the results of the trial. Bias because of differential history refers to some external influence that operates differentially among the conditions. Bias because of differential maturation reflects uneven secular trends among the groups in the trial favoring one condition or another. These first three sources of bias can either mask or mimic an intervention effect, and all three are more likely given either nonrandom assignment of groups or random assignment of a limited number of groups to each condition. The first three sources of bias are best avoided by randomization of a sufficient number of groups to each study condition, which will increase the likelihood that potential sources of bias are distributed evenly among the conditions. Careful matching or stratification can increase the effectiveness of randomization, especially when the number of groups is small. As a result, all GRTs planned with fewer than 20 groups per condition would be well served to include careful matching or stratification before randomization.

10

GROUP-RANDOMIZED TRIALS

The fourth source of bias is caused by contamination, which occurs when interventionlike activities find their way into the comparison groups; it can bias the estimate of the intervention effect toward the null hypothesis. Randomization will not protect against contamination; although investigators can control access to their intervention materials, they can often do little to prevent the outside world from introducing similar activities into their control groups. As a result, monitoring exposure to activities that could affect the trial’s endpoints in both the intervention and comparison groups is especially important in GRTs, which will allow the investigators to detect and respond to contamination if it occurs. Objective measures and evaluation personnel who have no connection to the intervention are also important strategies to limit bias. Finally, analytic strategies, such as regression adjustment for confounders, can be very helpful in dealing with any observed bias. 7.4 Potential Analytic Problems and Methods to Avoid them The two major threats to the validity of the analysis of a GRT that should be considered during the planning phase are misspecification of the analytic model and low power. Misspecification of the analytic model will occur if the investigator ignores or misrepresents a measurable source of random variation, or misrepresents the pattern of any over-time correlation in the data. To avoid model misspecification, the investigator should plan the analysis concurrent with the design, plan the analysis around the primary endpoints, anticipate all sources of random variation, anticipate the error distribution for the primary endpoint, anticipate patterns of over-time correlation, consider alternate structures for the covariance matrix, consider alternate models for time, and assess potential confounding and effect modification. Low power will occur if the investigator employs a weak intervention, has insufficient replication, has high variance or ICC in the endpoints, or has poor reliability of intervention implementation. To avoid low power,

investigators should plan a large enough study to ensure sufficient replication, choose endpoints with low variance and ICC, employ matching or stratification before randomization, employ more and smaller groups instead of a few large groups, employ more and smaller surveys or continuous surveillance instead of a few large surveys, employ repeat observations on the same groups or on the same groups and members, employ strong interventions with good reach, and maintain the reliability of intervention implementation. In the analysis, investigators should employ regression adjustment for covariates, model time if possible, and consider post hoc stratification. 7.5 Variables of Interest and their Measures The research question will identify the primary and secondary endpoints of the trial. The question may also identify potential effect modifiers. It will then be up to the investigators to anticipate potential confounders and nuisance variables. All these variables must be measured if they are to be used in the analysis of the trial. In a clinical trial, the primary endpoint is a clinical event, chosen because it is easy to measure with limited error and is clinically relevant (75). In a GRT, the primary endpoint need not be a clinical event, but it should be easy to measure with limited error and be relevant to public health. In both RCTs and GRTs, the primary endpoint, together with its method of measurement, must be defined in writing before the start of the trial. The endpoint and its method of measurement cannot be changed after the start of the trial without risking the validity of the trial and the credibility of the research team. Secondary endpoints should have similar characteristics and also should be identified before the start of the trial. In a GRT, an effect modifier is a variable whose level influences the effect of the intervention. For example, if the effect of a schoolbased drug-use prevention program depends on the baseline risk level of the student, then baseline risk is an effect modifier. Effect modification can be seen intuitively by looking at separate intervention effect estimates for the levels of the effect modifier. If they differ to

GROUP-RANDOMIZED TRIALS

a meaningful degree, then the investigator has evidence of possible effect modification. A more formal assessment is provided by a statistical test for effect modification, which is accomplished by including an interaction term between the effect modifier and condition in the analysis and testing the statistical significance of that term. If the interaction is significant, then the investigator should present the results separately for the levels of the effect modifier. If not, the interaction term is deleted and the investigator can continue with the analysis. Proper identification of potential effect modifiers comes through a careful review of the literature and from an examination of the theory of the intervention. Potential effect modifiers must be measured as part of the data-collection process so that their role can later be assessed. A confounder is related to the endpoint, not on the causal pathway, and unevenly distributed among the conditions; it serves to bias the estimate of the intervention effect. No statistical test for confounding exists; instead, it is assessed by comparing the unadjusted estimate of the intervention effect to the adjusted estimate of that effect. If, in the investigator’s opinion, a meaningful difference exists between the adjusted and unadjusted estimates, then the investigator has an obligation to report the adjusted value. It may also be appropriate to report the unadjusted value to allow the reader to assess the degree of confounding. The adjusted analysis will not be possible unless the potential confounders are measured. Proper identification of potential confounders also comes through a careful review of the literature and from an understanding of the endpoints and the study population. The investigators must take care in the selection of potential confounders to select only confounders and not mediating variables. A confounder is related to the endpoint and unevenly distributed in the conditions, but is not on the causal pathway between the intervention and the outcome. A mediating variable has all the characteristics of a confounder, but is on the causal pathway. Adjustment for a mediating variable, in the false belief that it is a confounder, will bias the estimate of the intervention effect toward the null hypothesis.

11

Similarly, the investigator must take care to avoid selecting as potential confounders variables that may be affected by the intervention. Such variables will be proxies for the intervention itself, and adjustment for them will also bias the estimate of the intervention effect toward the null hypothesis. An effective strategy to avoid these problems is to restrict confounders to variables measured at baseline. Such factors cannot be on the causal pathway, nor can their values be influenced by an intervention that has not been delivered. Investigators may also want to include variables measured after the intervention has begun, but will need to take care to avoid the problems described above. Nuisance variables are related to the endpoint, not on the causal pathway, but evenly distributed among the conditions. They cannot bias the estimate of the intervention effect, but they can be used to improve precision in the analysis. A common method is to make regression adjustment for these factors during the analysis so as to reduce the standard error of the estimate of the intervention effect, thereby improving the precision of the analysis. Such adjustment will not be possible unless the nuisance variables are measured. Proper identification of potential nuisance variables also comes from a careful review of the literature and from an understanding of the endpoint. The cautions described above for the selection of potential confounding variables apply equally well to the selection of potential nuisance variables. 7.6 The Intervention No matter how well designed and evaluated a GRT may be, strengths in design and analysis cannot overcome a weak intervention. Although the designs and analyses employed in GRTs were fair targets for criticism during the 1970s and 1980s, the designs and analyses employed more recently have improved, with many examples of very well-designed and carefully analyzed trials. Where intervention effects are modest or short-lived, even in the presence of good design and analytic strategies, investigators must take a hard look at the intervention and question whether it was strong enough.

12

GROUP-RANDOMIZED TRIALS

One of the first suggestions for developing the research question was that the investigators become experts on the problem that they seek to remedy. If the primary endpoint is cigarette smoking among ninth graders, then the team should seek to learn as much as possible about the etiology of smoking among young adolescents. If the primary endpoint is obesity among Native American children, then the team should seek to learn as much as possible about the etiology of obesity among those young children. If the primary endpoint is delay time in reporting heart attack symptoms, then the team should seek to learn as much as possible about the factors that influence delay time. And the same can be said for any other endpoint. One of the goals of developing expertise in the etiology of the problem is to identify points in that etiology that are amenable to intervention. Critical developmental stages may exist, or critical events or influences that trigger the next step in the progression, or it may be possible to identify critical players in the form of parents, friends, coworkers, or others who can influence the development of that problem. Without careful study of the etiology, the team will largely be guessing and hoping that their intervention is designed properly. Unfortunately, guessing and hoping rarely lead to effective interventions. Powerful interventions are guided by good theory on the process for change, combined with a good understanding of the etiology of the problem of interest. Poor theory will produce poor interventions and poor results, which was one of the primary messages from the community-based heart disease prevention studies, where the intervention effects were modest, generally of limited duration, and often within chance levels. Fortmann et al. noted that one of the major lessons learned was how much was not known about how to intervene in whole communities (56). Theory that describes the process of change in individuals may not apply to the process of change in identifiable groups. If it does, it may not apply in exactly the same way. Good intervention for a GRT will likely need to combine theory about individual change with theory about group processes and group change.

A good theoretical exposition will also help identify channels for the intervention program. For example, strong evidence exists that recent immigrants often look to longterm immigrants of the same cultural group for information on health issues. This fact has led investigators to try to use those longterm immigrants as change agents for the more recent immigrants. A good theoretical exposition will often indicate that the phenomenon is the product of multiple influences and so suggest that the intervention operate at several different levels. For example, obesity among schoolchildren appears to be influenced most proximally by their physical activity levels and by their dietary intake. In turn, their dietary intake is influenced by what is served at home and at school and their physical activity is influenced by the nature of their physical activity and recess programs at school and at home. The models provided by teachers and parents are important both for diet and physical activity. This multilevel etiology suggests that interventions be directed at the school foodservice, physical education, and recess programs; at parents; and possibly at the larger community. As noted earlier, GRTs would benefit by following the example of clinical trials, where some evidence of feasibility and efficacy of the intervention is usually required before launching the trial. When a study takes several years to complete and costs hundreds of thousands of dollars or more, that expectation seems very fair. Even shorter and less expensive GRTs would do well to follow that advice. 7.7 Power A detailed exposition on power for GRTs is beyond the scope of this article. Excellent treatments exist, and the interested reader is referred to those sources for additional information. Chapter 9 in the Murray text provides perhaps the most comprehensive treatment of detectable difference, sample size, and power for GRTs (1). Even so, a few points bear repeating here. First, the increase in between-group variance because of the ICC in the simplest analysis is calculated as 1 + (m − 1)ICC,

GROUP-RANDOMIZED TRIALS

where m is the number of members per group; as such, ignoring even a small ICC can underestimate standard errors if m is large. Second, although the magnitude of the ICC is inversely related to the level of aggregation, it is independent of the number of group members who provide data. For both of these reasons, more power is available given more groups per condition with fewer members measured per group than given just a few groups per condition with many members measured per group, no matter the size of the ICC. Third, the two factors that largely determine power in any GRT are the ICC and the number of groups per condition. For these reasons, no substitute exists for a good estimate of the ICC for the primary endpoint, the target population, and the primary analysis planned for the trial, and it is unusual for a GRT to have adequate power with fewer than 8 to 10 groups per condition. Finally, the formula for the standard error for the intervention effect depends on the primary analysis planned for the trial, and investigators should take care to calculate that standard error, and power, based on that analysis.

8

ACKNOWLEDGEMENT

The material presented here draws heavily on work published previously by David M. Murray and his colleagues (1, 34, 38, 43). Readers are referred to those sources for additional information.

REFERENCES 1. D. M. Murray, Design and Analysis of GroupRandomized Trials. New York: Oxford University Press, 1998. 2. L. Kish, Survey Sampling. New York: John Wiley & Sons, 1965.

13

5. D. M. Murray and R. D. Wolfinger, Analysis issues in the evaluation of community trials: progress toward solutions in SAS/STAT MIXED. J. Community Psychol. CSAP Special Issue 1994: 140–154. 6. D. M. Zucker, An analysis of variance pitfall: the fixed effects analysis in a nested design. Educ. Psycholog. Measur. 1990; 50: 731–738. 7. A. S. Bryk and S. W. Raudenbush, Hierarchical Linear Models: Applications and Data Analysis Methods. Newbury Park, CA: Sage Publications, 1992. 8. H. Goldstein, Multilevel Models in Educational and Social Research. New York: Oxford University Press, 1987. 9. I. Kreft and J. De Leeuw, Introducing Multilevel Modeling. London: Sage Publications, 1998. 10. S. W. Raudenbush and A. S. Bryk, Hierarchical Linear Models, 2nd ed. Thousand Oaks, CA: Sage Publications, 2002. 11. L. Kish, Statistical Design for Research. New York: John Wiley & Sons, 1987. 12. E. L. Korn and B. I. Graubard, Analysis of Health Surveys. New York: John Wiley & Sons, 1999. 13. C. J. Skinner, D. Holt, and T. M. F. Smith, Analysis of Complex Surveys. New York: John Wiley & Sons, 1989. 14. A. Donner, N. Birkett, and C. Buck, Randomization by cluster: sample size requirements and analysis. Amer. J. Epidemiol. 1981; 114(6): 906–914. 15. A. Donner and N. Klar, Design and Analysis of Cluster Randomization Trials in Health Research. London: Arnold, 2000. 16. COMMIT Research Group, Community Intervention Trial for Smoking Ccessation (COMMIT): summary of design and intervention. J. Natl. Cancer Inst. 1991; 83(22): 1620–1628. 17. A. Donner, Symposium on community intervention trials. Amer. J. Epidemiol. 1995; 142(6): 567–568. 18. R. V. Luepker, Community trials. Prevent. Med. 1994; 23: 602–605.

3. J. Cornfield, Randomization by group: a formal analysis. Amer. J. Epidemiol. 1978; 108(2): 100–102.

19. H. Blackburn, Research and demonstration projects in community cardiovascular disease prevention. J. Public Health Policy 1983; 4(4): 398–420.

4. D. M. Murray, P. J. Hannan, and W. L. Baker, A Monte Carlo study of alternative responses to intraclass correlation in community trials: is it ever possible to avoid Cornfield’s penalties? Eval. Rev. 1996; 20(3): 313–337.

20. R. A. Carleton et al., The Pawtucket Heart Health Program: community changes in cardiovascular risk factors and projected disease risk. Amer. J. Public Health 1995; 85(6): 777–785.

14

GROUP-RANDOMIZED TRIALS

21. J. W. Farquhar, The community-based model of life style intervention trials. Amer. J. Epidemiol. 1978; 108(2): 103–111.

36.

22. J. W. Farquhar et al., The Stanford five-city project: design and methods. Amer. J. Epidemiol. 1985; 122(2): 323–324. 23. D. R. Jacobs et al., Community-wide prevention strategies: evaluation design of the Minnesota Heart Health Program. J. Chronic Dis. 1986; 39(10): 775–788. 24. D. E. Lilienfeld and P. D. Stolley, Foundations of Epidemiology, 3rd ed. New York: Oxford University Press, 1994. 25. S. B. Hully, Symposium on CHD prevention trials: design issues in testing life style intervention. Amer. J. Epidemiol. 1978; 108(2): 85–86. 26. R. Sherwin, Controlled trials of the diet-heart hypothesis: some comments on the experimental unit. Amer. J. Epidemiol. 1978; 108(2): 92–99. 27. S. L. Syme, Life style intervention in clinicbased trials. Amer. J. Epidemiol. 1978; 108(2): 87–91.

37.

38.

39.

40.

41.

42.

28. M. H. Gail et al., Aspects of statistical design for the Community Intervention Trial for Smoking Cessation (COMMIT). Controlled Clin. Trials 1992; 13: 6–21. 29. D. B. Abrams et al., Cancer Control at the Workplace: The Working Well Trial. Prevent. Med. 1994; 23: 15–27. 30. D. M. Zucker et al., Statistical Design of the Child and Adolescent Trial for Cardiovascular Health (CATCH): implication of cluster randomization. Controlled Clin. Trials 1995; 16: 96–118. 31. J. M. Simpson, N. Klar, and A. Donner, Accounting for cluster randomization: a review of Primary Prevention Trials, 1990 through 1993. Amer. J. Public Health 1995; 85(10): 1378–1383. 32. H. Brown and R. Prescott, Applied Mixed Models in Medicine. Chichester, UK: John Wiley & Sons, Inc., 1999. 33. C. E. McCulloch and S. R. Searle, Generalized, Linear and Mixed Models. New York: John Wiley & Sons, 2001. 34. D. M. Murray, S. P. Varnell, and J. L. Blitstein, Design and analysis of grouprandomized trials: a review of recent methodological developments. Amer. J. Public Health 2004; 94(3): 423–432. 35. D. M. Murray and J. L. Blitstein, Methods to reduce the impact of intraclass correlation

43.

44.

45.

46.

47.

48.

in group-randomized trials. Eval. Rev. 2003; 27(1): 79–103. J. B. Janega et al., Assessing intervention effects in a school-based nutrition intervention trial: which analytic model is most powerful? Health Educ. Behav. 2004; 31(6): 756–774. J. B. Janega et al., Assessing the most powerful analysis method for school-based intervention studies with alcohol, tobacco and other drug outcomes. Addict. Behav. 2004; 29(3): 595–606. S. Varnell et al., Design and analysis of grouprandomized trials: a review of recent practices. Amer. J. Public Health 2004; 94(3): 393–399. Z. Feng et al., Selected statistical issues in group randomized trials. Annu. Rev. Public Health 2001; 22: 167–187. Z. Feng and B. Thompson, Some design issues in a community intervention trial. Controlled Clin. Trials 2002; 23: 431–449. N. Klar and A. Donner, Current and future challenges in the design and analysis of cluster randomization trials. Stat. Med. 2001; 20: 3729–3740. D. M. Murray, Efficacy and effectiveness trials in health promotion and disease prevention: design and analysis of group-randomized trials. In: N. Schneiderman et al., (eds.), Integrating Behavioral and Social Sciences with Public Health. Washington, DC: American Psychological Association, 2000, pp. 305–320. D. M. Murray, Statistical models appropriate for designs often used in group-randomized trials. Stat. Med. 2001; 20: 1373–1385. J. W. Farquhar et al., Effects of communitywide education on cardiovascular disease risk factors: The Stanford Five-City Project. JAMA 1990; 264(3): 359–365. R. V. Luepker et al., Community education for cardiovascular disease prevention: risk factor changes in the Minnesota Heart Health Program. Amer. J. Public Health 1994; 84(9): 1383–1393. COMMIT Research Group, Community Intervention Trial for Smoking Cessation (COMMIT): I. Cohort results from a four-year community intervention. Amer. J. Public Health 1995; 85(2): 183–192. COMMIT Research Group, Community Intervention Trial for Smoking Cessation (COMMIT): II. Changes in adult cigarette smoking prevalence. Amer. J. Public Health 1995; 85(2): 193–200. M. Susser, Editorial: the tribulations of trials—intervention in communities. Amer. J. Public Health 1995; 85(2): 156–158.

GROUP-RANDOMIZED TRIALS 49. M. Ausems et al., Short-term effects of a randomized computer-based out-of-school smoking prevention trial aimed at elementary schoolchildren. Prevent. Med. 2002; 34: 581–589. 50. D. Lazovich et al., Effectiveness of a worksite intervention to reduce an occupational exposure: the Minnesota Wood Dust Study. Amer. J. Public Health 2002; 92(9): 1498–1505. 51. J. A. Mayer et al., Promoting skin cancer prevention counseling by pharmacists. Amer. J. Public Health 1998; 88(7): 1096–1099. 52. M. J. Rotheram-Borus, M. B. Lee, M. Gwadz, and B. Draimin, An intervention for parents with Aids and their adolescent children. Amer. J. Public Health 2001; 91(8): 1294–1302. 53. J. Segura et al., A randomized controlled trial comparing three invitation strategies in a breast cancer screening program. Prevent. Med. 2001; 33: 325–332. 54. L. I. Solberg, T. E. Kottke, and M. L. Brekke, Will primary care clinics organize themselves to improve the delivery of preventive services? A randomized controlled trial. Prevent. Med. 1998; 27: 623–631. 55. F. A. Stillman et al., Evaluation of the American Stop Smoking Intervention Study (ASSIST): a report of outcomes. J. Amer. Cancer Inst. 2003; 95(22): 1681–1691. 56. S. P. Fortmann et al., Community intervention trials: reflections on the Stanford FiveCity Project experience. Amer. J. Epidemiol. 1995; 142(6): 576–586. 57. T. D. Koepsell et al., Invited commentary: symposium on community intervention trials. Amer. J. Epidemiol. 1995; 142(6): 594–599. 58. D. M. Murray, Design and analysis of community trials: lessons from the Minnesota Heart Health Program. Amer. J. Epidemiol. 1995; 142(6): 569–575. 59. S. Varnell, D. M. Murray, and W. L. Baker, An evaluation of analysis options for the one group per condition design: can any of the alternatives overcome the problems inherent in this design? Eval. Rev. 2001; 25(4): 440–453. 60. Z. Feng, D. McLerran, and J. Grizzle, A comparison of statistical methods for clustered data analysis with Gaussian error. Stat. Med. 1996; 15: 1793–1806. 61. M. D. Thornquist and G. L. Anderson, Small sample properties of generalized estimating equations in group-randomized designs with gaussian response. Presented at the 120th Annual APHA Meeting, Washington, DC, 1992.

15

62. R. M. Bell and D. F. McCaffrey, Bias reduction in standard errors for linear regression with multi-stage samples. Survey Methodol. 2002; 28(2): 169–181. 63. M. Fay and P. Graubard, Small-sample adjustments for Wald-type tests using sandwich estimators. Biometrics 2001; 57: 1198–1206. 64. G. Kauermann and R. J. Carroll, A note on the efficiency of sandwich covariance matrix estimation. J. Amer. Stat. Assoc. 2001; 96(456): 1387–1396. 65. L. A. Mancl and T. A. DeRouen, A covariance estimator for GEE with improved small-sample properties. Biometrics 2001; 57: 126–134. 66. W. Pan and M. M. Wall, Small-sample adjustments in using the sandwich variance estimator in generalized estimating equations. Stat. Med. 2002; 21: 1429–1441. 67. T. Cai, S. C. Cheng, and L. J. Wei, Semiparametric mixed-effects models for clustered failure time data. J. Amer. Stat. Assoc. 2002; 95(458): 514–522. 68. R. J. Gray and Y. Li, Optimal weight functions for marginal proportional hazards analysis of clustered failure time data. Lifetime Data Anal. 2002; 8: 5–19. 69. D. Hedeker, O. Siddiqui, and F. B. Hu, Random-effects regression analysis of correlated grouped-time survival data. Stat. Meth. Med. Res. 2000; 9: 161–179. 70. M. R. Segal, J. M. Neuhaus, and I. R. James, Dependence estimation for marginal models of multivariate survival data. Lifetime Data Anal. 1997; 3: 251–268. 71. K. K. Yau, Multilevel models for survival analysis with random effects. Biometrics 2001; 57: 96–102. 72. T. Braun and Z. Feng, Optimal permutation tests for the analysis of group randomized trials. J. Amer. Stat. Assoc. 2001; 96: 1424–1432. 73. B. L. Rooney and D. M. Murray, A Metaanalysis of smoking prevention programs after adjustments for errors in the unit of analysis. Health Educ. Quart. 1993; 23(1): 48–64. 74. R. E. Kirk, Experimental Design: Procedures for the Behavioral Sciences, 2nd ed. Belmont, CA: Brooks/Cole Publishing Company, 1982. 75. C. L. Meinert, Clinical Trials. New York: Oxford University Press, 1986. 76. W. R. Shadish, T. D. Cook, and D. T. Campbell, Experimental and Quasi-Experimental Designs for Generalized Causal Inference.

16

GROUP-RANDOMIZED TRIALS

Boston, MA: Houghton Mifflin Company, 2002. 77. B. J. Winer, D. R. Brown, and K. Michels, Statistical Principles in Experimental Design. New York: McGraw-Hill, 1991.

GROUP SEQUENTIAL DESIGNS

interim analysis is performed in which a decision is made to stop or to continue the trial. Particularly, a trial can be stopped with the early rejection of a hypothesis H0 . A trial can also be stopped for futility if the data indicate that H 0 probably cannot be rejected at the planned end of the trial. As a result of the repeated testing at the interim analyses, the Type I error rate α will be inflated. Group sequential designs offer the possibility to account for this kind of multiplicity and provide test procedures that control the Type I error rate. In the simplest case, the maximum number of stages, K, to be performed and the number of observations per stage n1 , . . ., nK are fixed in advance. For example, in a two-stage design (i.e., K = 2), a prespecified number n1 of subjects are observed in the first stage. If the P-value for testing H0 falls below a specified boundary α1
, H 0 can be rejected and no further subjects need to be recruited. If the P-value is too large (e.g., greater than 0.50), it is unlikely to obtain a statistically significant result after the observation of the second-stage data. Hence, further recruitment of patients will be stopped and, of course, H0 cannot be rejected. If it is decided to continue the trial, in the second stage a prespecified number of subjects n2 are observed. If the P-value calculated from the total dataset (i.e., n1 + n2 observations) falls below a boundary α2
, H 0 is rejected, otherwise H 0 is not rejected. As a result of the interim look, a false rejection of H0 can occur in the interim as well as in the final analysis and two tests of significance are repeatedly applied to the accumulating data. Hence, if the P-values obtained from the first and from both stages, respectively, are compared with the unadjusted level α, the overall Type I error rate exceeds α. If α1
= α2
= α/2, the procedure controls α because of the Bonferroni correction. The choice of more sophisticated boundaries is one important and fundamental topic in the theory of group sequential test designs. Peter Armitage was the first who adopted the sequential testing approach to clinical research. The first edition of his book Sequential Medical Trials was published in

GERNOT WASSMER Institute for Medical Statistics, Informatics, and Epidemiology University of Cologne Cologne, Germany

1

INTRODUCTION

In clinical research there is a great interest in interim analyses for ethical, economical, and organizational or administrative reasons. Interim analyses are performed within sequential designs that offer the possibility to stop a trial early with a statistically significant test result. This trial will possibly need less patients than the trial with a fixed sample size where a test decision can be made only at the end of the trial. A therapy which was shown to be superior can be applied earlier and the inferior therapy can be replaced by the better one. Furthermore, in interim analyses the quality of the performance of the trial can be assessed and possibly be improved when necessary. Interim analyses are also carried out for the assessment of the safety of the treatments under consideration, and the observation of serious adverse events can lead to an early stopping of the trial. The appointment of an Independent Data and Safety Monitoring Board (DSMB) in a clinical trial is a recommended constitution to perform these issues according to generally accepted standards of Good Clinical Practice (GCP) (1–3). Another issue is the redesign of the trial (e.g., sample size reassessment), but it was not intended originally nor is it generally possible with ‘‘classical’’ group sequential designs (see Flexible Design, Interim Analysis, Stopping Boundaries). Sequential designs were originally developed for the analysis of observations as soon as they become available. In group sequential designs, statistical inference is performed after the observation of groups of subjects, which is appealing, especially in medical research, because such a trial does not need to be continuously monitored, but can be organized in several stages. After each stage, an

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

GROUP SEQUENTIAL DESIGNS

1960 (4). The concept of repeated significance tests and its application in clinical trials was initiated by him. Armitage et al. (5) introduced the recursive integration formula, which enables the implementation of the repeated significance testing approach. Pocock (6) and O’Brien and Fleming (7), on the other hand, gave the major impetus for the development of group sequential test procedures that are widely used today, especially in clinical research. For a short historical review of the early work about group sequential procedures, see Jennison and Turnbull (References 8 and 9, Section 1.3) Ghosh (10) provides a history of sequential analysis in general, taking into account the developments beginning in the seventeenth century. The development of group sequential designs was, for the most part, within the repeated significance testing approach. This approach is conceptually different from the ‘‘purely’’ sequential approach. Applying the latter, both types of error probabilities are under control and used to determine the stopping boundaries of the sequential test procedure. The comprehensive theoretical development of these procedures owes much to the optimality of the sequential probability ratio test (SPRT), and to the derivation of analytic expressions for the decision regions and certain test characteristics. The SPRT was initiated by Abraham Wald (11). It is an optimum test in the sense that the expected sample size is minimum under both the null hypothesis and the alternative hypothesis. Several textbooks contain the theoretical developments in sequential designs (e.g., References 12–15). Theoretical research on repeated significance tests was also conducted. A series of papers were concerned with finding a bound for the critical value and with an approximation for the power and the expected sample size. Much of this work is a result of the research group of David Siegmund, in which many of the theoretical results were obtained from renewal theory. Essential developments and the mathematical background are provided in the book by Siegmund (13). An important part in the development of sequential designs was concerned with ‘‘triangular plans,’’ in which the stopping region for the sum of the observations is defined

by two straight lines that cross at the end of the trial. These are ‘‘closed’’ or ‘‘truncated’’ plans in which a decision to stop the trial is fixed at some maximum amount of information. Much of the theoretical development is concerned with the overshooting problem that occurs when the test statistic is not on the boundary. Groups of observations as well as continuous monitoring can be taken into account. Mathematical sophistication has led to the ‘‘Christmas tree correction,’’ which turns out to be quite accurate in many cases of practical importance (16). Whitehead (15) provides a comprehensive overview of sequential methods with special reference to the triangular plans in clinical trials. In a review article, Whitehead (17) argues against the distinction of group sequential methods from the wider family of sequential methods. This is certainly true, but the development and investigation of group sequential designs was in some sense separated from the rigorous mathematical derivation of results within the sequential theory, which is also a result of the rapid development of computer power that made the computations of the recursive numerical integral introduced by Armitage et al. (5) possible. Easy-to-use computer programs are available today to investigate the characteristics of the procedures numerically. The group sequential designs introduced by Pocock and O’Brien and Fleming stand for the ‘‘classical group sequential designs.’’ The basic notions of these designs and some of their generalizations are discussed in the following section. 2 CLASSICAL DESIGNS Pocock (6) proposed two-sided tests for normal responses with known variance assuming that the number of observations is equal between the K stages (i.e., n1 = . . .= nK = n, where nk denotes the sample size per stage of the trial). This situation applies to testing the difference of means in a parallel group design as well as to testing a single mean in a one-sample design (or a design with paired observations). In a group sequential test design, the hypothesis can be rejected

GROUP SEQUENTIAL DESIGNS

at the kth stage and the trial is terminated after that stage if the P-value is smaller or equal than a value αk
. The adjusted nominal significance levels αk
, k = 1, . . . , K, are determined such that the overall probability of a false rejection of H0 in the sequential scheme does not exceed α, which can also be defined in terms of the adjusted critical bounds uk , k = 1, . . . , K, for the standardized test statistic ∗ Zk of the accumulating data. Any sequence of critical bounds u1 , . . . , uK that fulfills ∗

∗

PH0 (|Z1 | ≥ u1 or . . . or |ZK | ≥ uK ) = α

(1)

defines a valid level-α test procedure for the two-sided test design. The statistics ∗ ∗ ∗ Z1 , . . . , ZK are correlated as each Zk incorporates all observations from the first k stages, k = 1, . . . , K. Therefore, the probability on the left-hand side in Equation (1) must in principal be computed with the multivariate normal integral, and the critical values u1 , . . . , uK are found by ∗ a numerical search. Each statistic Zk (k > 1), however, can be written as a sum of independent statistics. As a result of this independent increments structure, one can use the recursive integration formula of Armitage et al. (5) to facilitate the computation of the multiple integral and the determination of the critical values. Pocock’s design is characterized by assuming constant critical values u1 = . . .= uk = u or, equivalently, constant adjusted nominal significance levels α1
= . . . = αk
= α
over the stages of the trial. O’Brien and Fleming’s design, on the other hand, is defined through monotonically decreasing critical values uk √ given by uk = c/ k, where c is a suitable constant such that Equation (1) holds. Therefore, this approach postulates more conservative nominal significance levels for early stages, which yields an increasing sequence of adjusted significance levels. In Table 1, the nominal adjusted levels αk
along with the corresponding critical values uk for the standardized test statistic are presented for both designs for α = 0.05 and K = 2, 3, 4, 5. These choices of K are in common use and the figures in the table can be immediately used for application. Figure 1 graphically illustrates the decision regions of a four-stage

3

O’Brien and Fleming’s and Pocock’s design in terms of the critical values uk . As K increases, the necessary adjustment becomes stronger. For Pocock’s design, the improvement of using an adjusted bound satisfying Equation (1) in place of the simple Bonferroni correction is obvious: Instead of using α/K at each stage of the test procedure, one can use the less stringent adjusted significance levels αk
of Table 1. In O’Brien and Fleming’s design, strong evidence of an effect is required to stop the trial at early stages, but it is easier to reject H0 later on. As a consequence, the last-stage critical value is near to the critical value of the two-sided fixed sample size design (i.e., no interim analysis). For α = 0.05, the latter is given by zα/2 = 1.96, where zα/2 denotes the upper α/2 quantile of the standard normal distribution. In other words, the price to pay in terms of having to use a more conservative level of the Type I error rate is low for interim looks when using O’Brien and Flemings test design. Even stricter criteria for interim looks were earlier independently proposed by Haybittle (18) and Peto et al. (19). They suggested using u1 = . . .= uK−1 = 3 and uK = zα/2 . The actual Type I error rate of this procedure exceeds the nominal level α, but the excess is small, and one can simply adopt the approach to adjust the critical value uK such that the Type I error rate of the procedure is maintained. For example, for K = 5 and α = 0.05, the adjusted critical level for the kth stage is given by 1.990, which only slightly exceeds 1.960 and is considerably smaller than the values 2.413 and 2.040 required by the Pocock and the O’Brien and Fleming design (Table 1). Different group sequential test designs can be compared with respect to the necessary maximum sample size and the average sample size or average sample number (ASN). Usually, one fixes an effect size ϑ* and looks for the sample size such that the probability to reject H 0 is equal to some desired power 1 − β. Given ϑ*, the probability to reject H 0 increases as the sample sizes n per stage increase. Hence, in a two-sided design with K stages, the sample size per stage, n, to achieve power 1 − β for a group sequential design at a prespecified effect size ϑ* is found

4

GROUP SEQUENTIAL DESIGNS Table 1. Adjusted Nominal Significance Levels α
k and Critical Values uk , k = 1, . . ., K for Pocock’s and O’Brien and Fleming’s Design (Two-Sided α = 0.05)

Z*k

Pocock

O’Brien and Fleming αk
uk

K

k

αk


2

1 2

0.02939 0.02939

2.178 2.178

0.00517 0.04799

2.797 1.977

3

1 2 3

0.02206 0.02206 0.02206

2.289 2.289 2.289

0.00052 0.01411 0.04507

3.471 2.454 2.004

4

1 2 3 4

0.01821 0.01821 0.01821 0.01821

2.361 2.361 2.361 2.361

0.00005 0.00420 0.01942 0.04294

4.049 2.863 2.337 2.024

5

1 2 3 4 5

0.01581 0.01581 0.01581 0.01581 0.01581

2.413 2.413 2.413 2.413 2.413

0.00001 0.00126 0.00845 0.02256 0.04134

4.562 3.226 2.634 2.281 2.040

uk

4 OBF

reject H0

3 P 2

continue trial

1 0 −1

continue trial

−2

P

−3

reject H0

OBF

−4

1

3

2 stage

4

Figure 1. Decision regions for a two-sided four-stage group sequential design according to O’Brien and Fleming (OBF) and Pocock (P), respectively (α = 0.05).

numerically by solving ASNϑ ∗ = n + nPϑ ∗ (|Z1 | < u1 ) ∗

∗

Pϑ ∗ (|Z1 | ≥ u1 or . . . or |ZK | ≥ uK ) = 1 − β (2)

for n. The maximum sample size is N = Kn. The ASN given ϑ* becomes

+ nPϑ ∗ (|Z1 | < u1 , |Z2 | < u2 ) + . . . + nPϑ ∗ (|Z1 | < u1 , . . . , |ZK−1 | < uK−1 ) (3) The ASN can also be calculated under some other effect size. If the parameter specified in the alternative is true, ASNϑ ∗ specifies

GROUP SEQUENTIAL DESIGNS

the expected sample size if the assumption made in the sample size calculation is true. In case of normally distributed observations, it can be shown that N and the corresponding ASNϑ ∗ are inversely proportional to ϑ ∗2 in analogy to the fixed sample size design. Particularly, it suffices to provide tables that specify the maximum and average sample size as compared with the fixed sample size design. In Table 2, the values of the relative change are provided for two-sided α = 0.05 and different values of K and 1 − β. The factor N/nf is called the inflation factor of a particular design because it specifies how much the maximum sample size in the group sequential design must be inflated relative to the fixed sample size design. Comprehensive tables for the inflation factor in different designs can be found in Reference 9. The inflation factor can be used to perform sample size calculations for a group sequential test design in very different situations. For example, let the necessary sample size in a fixed sample design at given α = 0.05 and 1 − β = 0.80 be calculated as, say, nf = 54. If one wants to use a five-stage Pocock design, the maximum sample size necessary to achieve power 80% is the next integer larger than 1.228 × 54 = 66.3. That is, 67 observations need to be suitably distributed among the stages of the trial. Clearly, the assumption of equal sample sizes between the stages cannot exactly be fulfilled in most cases. It can be shown, however, that a slight departure from this assumption has virtually no effect on the test characteristics. That is, it might be reasonable in this case to start with n1 = 14 observations, and the subsequent sample sizes are n2 = 14, n3 = n4 = n5 = 13. The average sample size of this design is 0.799 × 54 = 43.1, which means that an essential reduction in sample size as compared with the fixed sample design is expected in this trial. A further important test characteristic is the probability of rejecting the null hypothesis at a specific stage of the trial. In this example, if the alternative is true, the probabilities to reject H0 in the first, second, third, fourth, and fifth stage are given by 0.153, 0.205, 0.187, 0.148, and 0.107, respectively. That is, some chance exists to stop the trial with the rejection of H0 at very early stages, saving time and money. On the other

5

hand, if one wants to use an O’Brien and Fleming design, the maximum and average sample size are 1.028 × 54 = 55.5 and 0.818 × 54 = 44.2, respectively. The corresponding probabilities to reject H 0 in one of the five stages of the trial are 0.001, 0.076, 0.259, 0.276, and 0.188, respectively. That is, it is very unlikely with this design to reject H0 at early stages, but the maximum sample size is only slightly above the sample size in a fixed sample design. One must balance the reasons for choosing a design also including other aspects than reducing the sample size. It is an essential characteristic of the classical design, however, that a particular design chosen must be thoroughly discussed before the start of the trial. In the ongoing trial, this design cannot be changed (see Flexible Designs). For given power, the maximum sample size in O’Brien and Fleming’s design is only slightly larger than the sample size necessary in a fixed sample size design. On the other hand, the ASN under the specified alternative, ASNϑ ∗ , in Pocock’s design is smaller. For increasing number of stages, K, Table 2 shows that ASNϑ ∗ decreases and the maximum number of observations increases. Although generally not true (in fact, the ASNϑ ∗ in Pocock’s design starts slightly increasing again for very large K), the most relevant reduction in ASNϑ ∗ is reached for K = 5. This result was first shown in References 20 and 21 and provides additional reasoning for not performing too many stages. From an organizational perspective, even the choice of less than four, one or two, say, interim analyses might be preferred. Pocock’s and O’Brien and Fleming’s designs form the milestone in the development of many other types of designs that were developed during the last two decades. In a subsequent article, Pocock (21) found optimum critical values that, given K, α, and 1 − β, minimize the expected sample size under the alternative. These critical values were found by a K-dimensional grid search. For example, if K = 5, α = 0.05, and 1 − β = 0.95, the optimum critical values are given

6

GROUP SEQUENTIAL DESIGNS Table 2. Average Sample Size ASNϑ ∗ and Maximum Sample Size N Relative to the Sample Size nf in a Fixed Sample Size Design for Pocock’s and O’Brien and Fleming’s Design (Two-Sided α = 0.05) Pocock N/nf ASNϑ ∗ /nf

O’Brien and Fleming ASNϑ ∗ /nf N/nf

K

1−β

2

0.80 0.90 0.95

0.853 0.776 0.718

1.110 1.100 1.093

0.902 0.851 0.803

1.008 1.007 1.007

3

0.80 0.90 0.95

0.818 0.721 0.649

1.166 1.151 1.140

0.856 0.799 0.751

1.017 1.016 1.015

4

0.80 0.90 0.95

0.805 0.697 0.619

1.202 1.183 1.170

0.831 0.767 0.716

1.024 1.022 1.021

5

0.80 0.90 0.95

0.799 0.685 0.602

1.228 1.206 1.191

0.818 0.750 0.696

1.028 1.026 1.025

by (u1 , u2 , u3 , u4 , u5 ) = (2.446, 2.404, 2.404, 2.404, 2.396) For K = 5, α = 0.05, and 1 − β = 0.50, these values are given by (u1 , u2 , u3 , u4 , u5 ) = (3.671, 2.884, 2.573, 2.375, 2.037) That is, for high power values, a design with constant critical levels over the stages is (nearly) optimum with respect to the average sample size. For low power values, on the other hand, a design with decreasing critical values, like O’Brien and Fleming’s design, should be used. Wang and Tsiatis (22) suggested that the optimum critical values can approximately be found within the -class of critical values uk given by uk = c(K, α, )k−0.5 , k = 1, . . . , K

(4)

where c(K, α, ) is a suitably chosen constant. They showed that the optimized ASNϑ ∗ is only negligibly larger when searching within the one-parameter family of critical values defined through Equation (4) as compared with the K-dimensional search. For  = 0

and  = 0.5, one obtains O’Brien and Fleming’s and Pocock’s design, respectively. The Wang and Tsiatis -class family offers a wide class of boundaries with different shapes. The critical values and the properties of the tests with intermediate values of  were extensively tabulated (9, 23) or can be found using a suitable software package (24, 25). One-sided test designs were also considered in the literature. However, the issues involved in the question of whether a test should be two-sided or one-sided are getting more complex when considering multistage designs (26). Nevertheless, it is conceptually straightforward and occasionally reasonable to define one-sided tests in the context of group sequential designs. A set of critical values u1 , . . . , uK satisfying ∗

∗

PH0 (Z1 ≥ u1 or . . . or ZK ≥ uK ) = α

(5)

defines a one-sided group sequential test design. The power and the ASN are defined analogously. It is interesting that the critical values defined through Equation (5) are virtually (but not exactly) identical to the two-sided critical values defined through Equation (1) at significance level 2α, which is analogous to the fixed sample design. As the difference of the exact and the approximate critical values is less than 0.000001 for all

GROUP SEQUENTIAL DESIGNS

practically relevant situations, it suffices to consider only one case. DeMets and Ware (27, 28) considered the issue of stopping the trial for futility, which becomes evident in the one-sided setting. If, for example, the one-sided P-value at some stage k is greater than 0.50, the effect is directed opposed to the alternative. Hence, no reasonable chance to obtain a significant result at the end of the trial may exist. In the planning phase of the trial, it can be decided that in this case the trial will be stopped. Taking into account the stopping for futility option, the critical values are different as compared with the original design defined through Equation (5). Indeed, the critical values are somewhat smaller and, most importantly, if the null hypothesis is true, the ASN reduces considerably. DeMets and Ware considered various choices of stopping for futility boundaries, including the constant boundary (i.e. stopping the trial if the P-value exceeds a constant α 0 > α). Taking into account the possibility of stopping the trial in favor of H0 was also considered in the context of two-sided tests. Emerson and Fleming (29) proposed one-sided and two-sided group sequential designs in which both error probabilities are under control. These designs were termed ‘‘symmetric designs’’ and were generalized by Pampallona and Tsiatis (30). Within the -class of boundaries, they found designs that are optimal with respect to the average sample size under the alternative hypothesis. The application of the group sequential test design discussed so far is also possible for other designs. In simulation studies (6), it was shown that the use of the critical values derived for normally distributed observations with a known variance provide sufficient control of the nominal level if the variance is unknown and the t-test situation applies. Furthermore, in designs with binary or exponential response or many other types of response variables, the boundaries derived for the normal case can be applied. Generally, if the sample size is not too small, the Type I error rate is approximately equal to α, if the increments are independent and approximately normal, which is also (approximately) true for studies with a survival endpoint, certain longitudinal designs, and

7

for nonparametric testing situations, which offers a wide application of group sequential designs in clinical trials (e.g., References 31–34). Some work was also done on the exact calculation for group sequential trials in other designs such as trials comparing two treatments with dichotomous response (35); exact calculations for the t-test, χ 2 , and Ftest situation (36); or exact group sequential permutational tests (37). 3

THE α-SPENDING FUNCTION APPROACH

The test procedures discussed in the last section were originally designed for a fixed maximum number of stages and equally sized groups of observations per stage. In practice, however, the latter assumption is rarely exactly fulfilled. The schedule of interim analyses can at best ensure that the stage sizes are roughly the same. In practice of clinical trials, the strict assumption of equal sample sizes may prove impractical. The size and power requirement are not greatly affected, however, if this assumption is not grossly violated. Nevertheless, situations exist with a substantial effect on size and power (38). Hence, the importance of more flexible procedures is obvious. One can obtain valid tests relaxing the strict assumption of equal sample sizes between the stages. First of all, one may prespecify sample sizes n1 , . . ., nK , which are not necessarily equal. For example, consider a four-stage design in which the first interim analysis should take place after observation of 10% of all planned sampling units, the second interim after 40%, and the third interim analysis after 70% of observations. With these so-called information rates
tk = ki=1 ni /N specified in advance, it is possible to derive the maximum sample size, N, such that the procedure has desired power 1 − β, given K, α, and the desired boundary shape. The critical values and the test characteristics differ from those derived with equally spaced stages. One can even optimize a group sequential plan with regard to, say, the ASN under the alternative considering varying information rates and varying boundary shapes (39, 40). The α-spending function or use function approach, proposed by Lan and DeMets (41)

8

GROUP SEQUENTIAL DESIGNS

and extended by Kim and DeMets (42), is conceptually different. The idea is to specify the amount of significance level spent up to an interim analysis rather than the shape of the adjusted critical levels. One uses a function α*(tk ) that specifies the cumulative Type I error rate spent at a time point tk of the kth analysis. α*(tk ) is a non-decreasing function with α*(0) = 0 and α*(1) = α that must be fixed before the start of the trial. As above, tk represents the information rate that develops from the specific course of the study (43). The information rates need not be prespecified before the actual course of the trial but must be chosen independently of the observed effect sizes. Consequently, the number of observations at the kth analysis and the maximum number K of analyses are flexible. In the two-sided case, the critical value for the first analysis is given by u1 = −1 (1 − α*(t1 )/2), where t1 = n1 /N and −1 (·) refers to the inverse normal cumulative distribution function. The critical values for the remaining stages are then computed successively. That is, once uk−1 is calculated, the critical value at stage k is calculated through ∗

∗

πk = PH0 (|Z1 | < u1 , . . . , |Zk−1 | < uk−1 , ∗

|Zk | ≥ uk ) ∗

∗

= α (tk ) − α (tk−1 )

(6)

π k denotes the probability of a Type I error
at stage k and, therefore, K k=1 πk = α . The one-sided case is treated analogously. In this way, the sequence of critical values u1 , . . . , uK that defines a group sequential test is not specified in advance but results from the information rates observed during the course of the trial. The use function approach is particularly attractive if the interim analyses are planned at specific time points rather than after a specific number of evaluable observations has been obtained. Particularly, the sample size per stage may be an unknown quantity that will be observed when the interim analysis is performed. With this approach, even the number of interim analyses must not be fixed in advance. Instead, a maximum amount of information must be specified. In the simplest case, it is the maximum sample size N of

the trial. Through the use of a specified use function, the significance level spent up to this information is fixed in advance, which enables the calculation of the adjusted levels. The overall significance level α is maintained if the study stops whenever tk = tK = 1. The actually observed number of observations, however, may fall short of or exceed N. To account for the latter case ∗ (i.e., tK >1), the α-spending function α˜ (tk ) = ∗ min(α, α (tk )) might be used to account for random overrunning. If the study stops with a smaller maximum sample size than antic∗ ipated (i.e., tK < 1), then setting α˜ (tK ) = α forces the procedure to fully exhaust the level α up to the last stage (44). An important application of group sequential designs is in trials where the endpoint is the time to an event (e.g., survival data). It was shown by Tsiatis (34) that the usual logrank test can be embedded into the group sequential design. The information is the observed number of events (i.e., deaths), and the use function approach turns out to be a very useful and flexible instrument for analyzing such trials in a sequential design (32, 45). A number of proposals were designed in the literature for the form of the function α*(tk ). The α-spending functions ∗

(7) α1 (tk ) = α ln(1 + (e − 1) · tk ) and  √ 2(1 − (zα/2 / tk )) (one-sided case) ∗ α2 (tk ) = √ 4(1 − (zα/4 / tk )) (two-sided case) (8) approximate Pocock’s and O’Brien and Fleming’s group sequential boundaries, respectively. Kim and DeMets (42) proposed a one parameter family of α-spending functions ∗

ρ

α3 (ρ, tk ) = αtk

(9)

The spending functions in Equations (7)–(9) are illustrated in Fig. 2. Notice that the use of constant adjusted significance levels (i.e., Pocock’s design) does not correspond to a linear α-spending function. Instead, assuming a linear α-spending function (i.e., ∗ using α3 (ρ) with ρ = 1) and equally spaced

GROUP SEQUENTIAL DESIGNS a*(tk ) 0.05 0.04 a*3 (2)

0.03

a*3 (1.5) a*3 (1) a*1

0.02

a*2

0.01 0.00 0.0

0.2

0.4

0.6

0.8

1.0

tk

Figure 2. Examples of α-spending functions. α1∗ and α2∗ approximate Pocock’s and O’Brien and Fleming’s group sequential boundaries, respectively. Kim and DeMets α-spending function class α3∗ (ρ) is shown for ρ = 1, 1.5, 2.

stage sizes yields somewhat decreasing critical values. Hwang et al. (46) introduced a one parameter family of α-spending functions and showed that the use of this family yields approximately optimal plans similar to the class of Wang and Tsiatis (22). Optimum tests adapting the use function approach were also found by Jennison (47). Similar approaches were proposed by Fleming et al. (48) and Slud and Wei (33). They are defined in terms of the Type I error rates π k in Equation (6) by specifying π 1 , . . ., π K summing to α, which is different from the spending function approach that specifies π k at some information rate tk . Nevertheless, it requires the same technique for computing the decision regions. It is tempting to use the results of an interim analysis to modify the schedule of interim looks, which is particularly true for the α-spending function approach because the maximum number of analyses must not be prespecified. For example, if the test result is very near to showing significance, it could be decided to conduct the next interim analysis earlier than originally planned. However, a data-driven analysis strategy is not allowed for the α-spending function approach. Cases exist in which the Type I error rate is seriously inflated, as was shown by several

9

authors (8, 38). In this case, therefore, one should use adaptive or flexible designs that are designed specifically for a data-driven analysis strategy and offer an even larger degree of flexibility (see Flexible Designs). The α-spending function approach can be generalized in several ways. First, it is easy to implement an asymmetric procedure for the determination of an upper and a lower bound in a two-sided test situation. Then, two αspending functions must be given specifying the Type I error rate for the lower and the upper bound, respectively. Second, for planning purposes, it is quite natural to consider a function describing how the power of the procedure should be spent during the stages of the study, known as the power spending function. Approaches that use the power spending function are described in References 49–51 . 4 POINT ESTIMATES AND CONFIDENCE INTERVALS An important field of research in group sequential designs was concerned with the parameter estimation. Through the use of a stopping rule (i.e., the possibility of stopping a trial early with the rejection (or acceptance) of the null hypothesis), point estimates that are derived for the fixed sample size case (e.g., maximum likelihood estimates) are biased. In the long run, hence, one is faced with the over-estimation or underestimation of the true parameter. Point estimates were proposed that correct for the (overall) estimation bias through numerical methods (e.g., References 52–60). Two conceptually different methods for the calculation of confidence intervals were considered in the literature. The first method enables the calculation after the trial has stopped and a final decision of rejection or acceptance of the null hypothesis was reached (e.g., References 59, 61–66). This approach requires the strict adherence to the stopping rule and depends on the ordering in the sample space, which means that it has to be decided if, for example, an observed effect leading to the rejection of the null hypothesis in the first interim is ‘‘more extreme’’ than an effect that is larger but observed in the second interim analysis. If so, the resulting ordering rates effects obtained in earlier

10

GROUP SEQUENTIAL DESIGNS

stages of the trial are stronger than effects obtained later on. However, other orderings exist that are reasonable choices too, and no general agreement exists over which ordering is the best. Generally, it is possible to derive confidence intervals based on such orderings with confidence level exactly equal to 1 − α. Bias adjusted point estimates can be derived as the 50% confidence limit yielding median unbiased estimates. Furthermore, dependent on the ordering of the sample space, it is also possible to obtain overall P-values after completion of the trial (67). The second method merely takes into account the multiplicity that develops from the repeated looks at the data. The resulting intervals are called Repeated Confidence Intervals (RCIs). They are defined as a sequence of intervals Ik for which the coverage probability is simultaneously fulfilled, i.e., Pϑ (ϑ ∈ Ik , k = 1, . . . , K) = 1 − α where ϑ is the unknown parameter. This concept was introduced by Jennison and Turnbull (68, 69) [see also Lai (70)]. The calculation of RCIs is straightforward. When considering the family of group sequential tests for testing H 0 : ϑ = ϑ 0 , one simply has to find, at given stage k, the values ϑ 0 that do not lead to a rejection of H0 . Typically, these values are easy to find, and closed form analytical expressions exist in many cases. In more complicated situations, the RCI can be found iteratively. For example, when considering the mean of normally distributed observations, the sequence of RCIs is given by the sequence of intervals Ik = [xk − uk SEM; xk + uk SEM], k = 1, . . . , K where xk is the observed mean of observations up to stage k, SEM denotes the standard error of the mean, and u1 , . . ., uK are the adjusted critical values corresponding to a particular group sequential test design. The RCIs are wider than the confidence interval in a fixed sample design because they take into account the repeated looks at the accumulating data. For a design with larger critical values at early stages (e.g., O’Brien and Fleming’s design), these intervals are

fairly wide in the first stage, but become narrower as the study progresses. The RCIs can be calculated at each interim analysis. Based on the RCI, it might be decided to terminate or to continue the trial. For example, consider testing the hypothesis H 0 : ϑ = 0. H 0 can be rejected at stage k if the RCI at stage k does not contain ϑ = 0. But also if, in this case, it is decided to continue the trial, the calculation of RCIs in subsequent stages is possible and might be used for the decision process. The RCIs are independent of the adherence to the stopping rule and can therefore be calculated also if the study is going on. As a consequence, these intervals are conservative if not all stages of the trial were actually performed. 5 SUPPLEMENTS Several important issues are not addressed in this article and are referred to further study of the literature. Incorporating covariate information is considered in References 71–73 . In trials with multiple endpoints or in multi-armed trials, such as dose-response trials, the multiplicity occurs in two dimensions. First, the study is performed sequentially and one has to account for the multiple looks effect. Second, at each stage of the procedure, the need exists to perform multiple tests on several hypotheses concerning single endpoints or one wants to perform multiple comparisons between the treatment groups. Furthermore, it should be possible to drop inferior treatments in a multi-armed trial. In multi-armed trials, group sequential methods can be applied, too, and some current work exists in this area (e.g., References 74–80). Parametric and nonparametric group sequential tests for multiple endpoints were proposed in, for example, References 81–86. Last, group sequential designs in equivalence trials were among other designs discussed in References 40, 87–90. Group sequential designs require the need of specialized software. Software packages are available that are specifically designed for planning and analyzing a sequentially planned trial (24, 25, 91), which helps for the correct implementation of a group sequential design and ensures the accurate analysis.

GROUP SEQUENTIAL DESIGNS

For advanced users, it is also possible to perform the relevant calculations using standard statistical software packages. SAS/IML (92) as well as S-PLUS (93) provide modules for the calculation and the assessment of group sequential test designs.

REFERENCES 1. P. Armitage, Interim analysis in clinical trials. Stat. Med. 1991; 10: 925–937. 2. K. McPherson, Sequential stopping rules in clinical trials. Stat. Med. 1990; 9: 595–600. 3. A. J. Sankoh, Interim analysis: an update of an FDA reviewer’s experience and perspective. Drug Inf. J. 1999; 33: 165–176. 4. P. Armitage, Sequential Medical Trials. 2nd ed. Oxford: Blackwell, 1975. 5. P. Armitage, C. K. McPherson, and B. C. Rowe, Repeated significance tests on accumulating data. J. Roy. Stat. Soc. A 1969; 132: 235–244. 6. S. J. Pocock, Group sequential methods in the design and analysis of clinical trials. Biometrika 1977; 64: 191–199. 7. P. C. O’Brien and T. R. Fleming, A multiple testing procedure for clinical trials. Biometrics 1979; 35: 549–556. 8. C. Jennison and B. W. Turnbull, Group sequential tests and repeated confidence intervals. In: B. K. Ghosh and P. K. Sen (eds.), Handbook of Sequential Analysis. New York: Marcel Dekker, 1991, pp. 283–311. 9. C. Jennison and B. W. Turnbull, Group Sequential Methods with Applications to Clinical Trials. Boca Raton, FL: Chapman & Hall, 2000.

11

16. J. Whitehead and I. Stratton, Group sequential clinical trials with triangular rejection regions. Biometrics 1983; 39: 227–236. 17. J. Whitehead, Sequential methods. In: C. K. Redmond and T. Colton (eds.), Biostatistics in Clinical Trials. Chichester: Wiley, 2001, pp. 414–422. 18. J. L. Haybittle, Repeated assessments of results in clinical trials of cancer treatment. Brit. J. Radiol. 1971; 44: 793–797. 19. R. Peto, M. C. Pike, P. Armitage, N. E. Breslow, D. R. Cox, S. V. Howard, N. Mantel, K. McPherson, J. Peto, and P. G. Smith, Design and analysis of randomized clinical trials requiring prolonged observation of each patient. I. Introduction and design. Brit. J. Cancer 1976; 34: 585–612. 20. K. McPherson, On choosing the number of interim analyses in clinical trials. Stat. Med. 1982; 1: 25–36. 21. S. J. Pocock, Interim analyses for randomized clinical trials: the group sequential approach. Biometrics 1982; 38: 153–162. 22. S. K. Wang and A. A. Tsiatis, Approximately optimal one-parameter boundaries for group sequential trials. Biometrics 1987; 43: 193–199. 23. G. Wassmer, Statistische Testverfahren fur ¨ gruppensequentielle und adaptive Plane ¨ in klinischen Studien. Theoretische Konzepte und deren praktische Umsetzung mit SAS. K¨oln: Verlag Alexander M¨onch, 1999. 24. Cytel Software Corporation, EaSt 2000: Software for the Design and Interim Monitoring of Group Sequential Clinical Trials. Cambrigde, MA: Cytel, 2000. 25. G. Wassmer and R. Eisebitt, ADDPLAN: Adaptive Designs - Plans and Analyses. Cologne: ADDPLAN GmbH, 2002.

10. B. K. Ghosh, A brief history of sequential analysis. In: B. K. Ghosh and P. K. Sen (eds.), Handbook of Sequential Analysis. New York: Marcel Dekker, 1991, pp. 1–19.

26. P. C. O’Brien, Data and safety monitoring. In: P. Armitage and T. Colton (eds.), Encyclopedia of Biostatistics. Chichester: Wiley, 1998, pp. 1058–1066.

11. A. Wald, Sequential Analysis. New York: Wiley, 1947.

27. D. L. DeMets and J. H. Ware, Group sequential methods for clinical trials with a one-sided hypothesis. Biometrika 1980; 67: 651–660.

12. B. K. Ghosh, Sequential Tests of Statistical Hypotheses. Reading, MA: Addison-Wesley, 1970. 13. D. Siegmund, Sequential Analysis. New York: Springer, 1985.

28. D. L. DeMets and J. H. Ware, Asymmetric group sequential boundaries for monitoring clinical trials. Biometrika 1982; 69: 661–663.

14. G. B. Wetherill, Sequential Methods in Statistics. London: Chapman and Hall, 1975.

29. S. S. Emerson and T. R. Fleming, Symmetric group sequential test designs. Biometrics 1989; 45: 905–923.

15. J. Whitehead, The Design and Analysis of Sequential Clinical Trials, rev. 2nd ed. Chichester: Wiley, 1997.

30. S. Pampallona and A. A. Tsiatis, Group sequential designs for one-sided and two-sided hypothesis testing with provision for early

12

GROUP SEQUENTIAL DESIGNS stopping in favour of the null hypothesis. J. Stat. Plan. Inf. 1994; 42: 19–35.

31. D. L. DeMets and M. H. Gail, Use of logrank tests and group sequential methods at fixed calender times. Biometrics 1985; 41: 1039–1044. 32. K. K. G. Lan and J. M. Lachin, Implementation of group sequential logrank tests in a maximum duration trial. Biometrics 1990; 46: 759–770. 33. E. Slud and L. J. Wei, Two-sample repeated significance tests based on the modified Wilcoxon statistic. J. Amer. Stat. Ass. 1982; 77: 862–868. 34. A. A. Tsiatis, Repeated significance testing for a general class of statistics used in censored survival analysis. J. Amer. Stat. Ass. 1982; 77: 855–861. 35. D. Y. Lin, L. J. Wei, and D. L. DeMets, Exact statistical inference for group sequential trials. Biometrics 1991; 47: 1399–1408. 36. C. Jennison and B. W. Turnbull, Exact calculations for sequential t, chi-square and F tests. Biometrika 1991; 78: 133–141. 37. C. R. Mehta, N. Patel, P. Senchaudhuri, and A. A. Tsiatis, Exact permutational tests for group sequential clinical trials. Biometrics 1994; 50: 1042–1053. 38. M. A. Proschan, D. A. Follmann, and M. A. Waclawiw, Effects on assumption violations on type I error rate in group sequential monitoring. Biometrics 1992; 48: 1131–1143. 39. E. H. Brittain and K. R. Bailey, Optimization of multistage testing times and critical values in clinical trials. Biometrics 1993; 49: 763–772.

45. K. Kim and A. A. Tsiatis, Study duration for clinical trials with survival response and early stopping rule. Biometrics 1990; 46: 81–92. 46. I. K. Hwang, W. J. Shih, and J. S. DeCani, Group sequential designs using a family of Type I error probability spending functions. Stat. Med. 1990; 9: 1439–1445. 47. C. Jennison, Efficient group sequential tests with unpredictable group sizes. Biometrika 1987; 74: 155–165. 48. T. R. Fleming, D. P. Harrington, and P. C. O’Brien, Designs for group sequential trials. Contr. Clin. Trials 1984; 5: 348–361. 49. P. Bauer, The choice of sequential boundaries based on the concept of power spending. Biom. Inform. Med. Biol. 1992; 23: 3–15. 50. M. N. Chang, I. K. Hwang, and W. J. Shih, Group sequential designs using both type I and type II error probability spending functions. Comm. Stat. Theory Meth. 1998; 27: 1323–1339. 51. S. Pampallona, A. A. Tsiatis, and K. Kim, Interim monitoring of group sequential trials using spending functions for the type I and type II error probabilities. Drug Inf. J. 2001; 35: 1113–1121. 52. S. S. Emerson, Computation of the uniform minimum variance unbiased estimator of the normal mean following a group sequential trial. Comput. Biomed. Res. 1993; 26: 68–73. 53. S. S. Emerson and T. R. Fleming, Parameter estimation following group sequential hypothesis testing. Biometrika 1990; 77: 875–892. 54. S. S. Emerson and J. M. Kittelson, A computationally simpler algorithm for the UMVUE of a normal mean following a sequential trial. Biometrics 1997; 53: 365–369.

¨ ¨ 40. H-H. Muller and H. Schafer, Optimization of testing times and critical values in sequential equivalence testing. Stat. Med. 1999; 18: 1769–1788.

55. K. Kim, Point estimation following group sequential tests. Biometrics 1989; 45: 613–617.

41. K. K. G. Lan and D. L. DeMets, Discrete sequential boundaries for clinical trials. Biometrika 1983; 70: 659–663.

56. A. Liu and W. J. Hall, Unbiased estimation following a group sequential test. Biometrika 1991; 86: 71–78.

42. K. Kim and D. L. DeMets, Design and analysis of group sequential tests based on the Type I error spending rate function. Biometrika 1987; 74: 149–154.

57. J. C. Pinheiro and D. L. DeMets, Estimating and reducing bias in group sequential designs with Gaussian independent increment structure. Biometrika 1997; 84: 831–845.

43. K. K. G. Lan and D. L. DeMets, Group sequential procedures: calender versus information time. Stat. Med. 1989; 8: 1191–1198.

58. E. Skovlund and L. Walløe, Estimation of treatment difference following a sequential clinical trial. J. Amer. Stat Ass. 1989; 84: 823–828.

44. K. Kim, H. Boucher, and A. A. Tsiatis, Design and analysis of group sequential logrank tests in maximum duration versus information trials. Biometrics 1995; 51: 988–1000.

59. S. Todd, J. Whitehead, and K. M. Facey, Point and interval estimation following a sequential clinical trial. Biometrika 1996; 83: 453–461.

GROUP SEQUENTIAL DESIGNS 60. J. Whitehead, On the bias of maximum likelihood estimation following a sequential test. Biometrika 1986; 73: 573–581. 61. M. N. Chang and P. C. O’Brien, Confidence intervals following group sequential tests. Contr. Clin. Trials 1986; 7: 18–26. 62. D. S. Coad and M. B. Woodroofe, Corrected confidence intervals after sequential testing with applications to survival analysis. Biometrika 1996; 83: 763–777. 63. D. E. Duffy and T. J. Santner, Confidence intervals for a binomial parameter based on multistage tests. Biometrics 1987; 43: 81–93. 64. K. Kim and D. L. DeMets, Confidence intervals following group sequential tests in clinical trials. Biometrics 1987; 43: 857–864. 65. G. L. Rosner and A. A. Tsiatis, Exact confidence intervals following a group sequential trial: a comparison of methods. Biometrika 1988; 75: 723–729. 66. A. A. Tsiatis, G. L. Rosner, and C. R. Mehta, Exact confidence intervals following a group sequential test. Biometrics 1984; 40: 797–803. 67. K. Fairbanks and R. Madsen, P values for tests using a repeated significance test design. Biometrika 1982; 69: 69–74. 68. C. Jennison and B. W. Turnbull, Repeated confidence intervals for group sequential clinical trials. Contr. Clin. Trials 1984; 5: 33–45. 69. C. Jennison and B. W. Turnbull, Interim analysis: the repeated confidence interval approach. J. R. Statist. Soc. B 1989; 51: 305–361. 70. T. L. Lai, Incorporating scientific, ethical and economic considerations into the design of clinical trials in the pharmaceutical industry: a sequential approach. Comm. Stat. Theory Meth. 1984; 13: 2355–2368. 71. C. Jennison and B. W. Turnbull, Group sequential analysis incorporating covariate information. J. Amer. Stat. Ass. 1997; 92: 1330–1341. 72. D. O. Scharfstein, A. A. Tsiatis, and J. M. Robins, Semiparametric efficiency and its implication on the design and analysis of group-sequential studies. J. Amer. Stat. Ass. 1997; 92: 1342–1350. 73. A. A. Tsiatis, G. L. Rosner, and D. L. Tritchler, Group sequential tests with censored survival data adjusting for covariates. Biometrika 1985; 365–373. 74. D. A. Follmann, M. A. Proschan, and N. L. Geller, Monitoring pairwise comparisons in multi-armed clinical trials. Biometrics 1994; 50: 325–336.

13

75. N. L., Geller M. A. Proschan, and D. A. Follmann, Group sequential monitoring of multiarmed clinical trials. Drug Inf. J. 1995; 29: 705–713. 76. M. Hellmich, Monitoring clinical trials with multiple arms. Biometrics 2001; 57: 892–898. 77. M. Hughes, Stopping guidelines for clinical trials with multiple treatments. Stat. Med. 1993; 12: 901–915. 78. W. Liu, A group sequential procedure for allpairwise comparisons of k treatments based on range statistics. Biometrics 1995; 51: 946–955. 79. K. J. Lui, A simple generalization of the O’Brien and Fleming group sequential test procedure to more than two treatment groups. Biometrics 1993; 49: 1216–1219. 80. M. A. Proschan, D. A. Follmann, and N. L. Geller, Monitoring multi-armed trials. Stat. Med. 1994; 13: 1441–1452. 81. C. Jennison and B. W. Turnbull, Group sequential tests for bivariate response: interim analyses of clinical trials with both efficacy and safety endpoints. Biometrics 1993; 49: 741–752. 82. J. M. Lachin, Group sequential monitoring of distribution-free analyses of repeated measures. Stat. Med. 1997; 16: 653–668. 83. S. J. Lee, K. Kim, and A. A. Tsiatis, Repeated significance testing in longitudinal clinical trials. Biometrika 1996; 83: 779–789. 84. J. Q. Su and J. M. Lachin, Group sequential distribution-free methods for the analysis of multivariate observations. Biometrics 1992; 48: 1033–1042. 85. D-I. Tang, C. Gnecco, and N. L. Geller, Design of group sequential clinical trials with multiple endpoints. J. Amer. Stat. Ass. 1989; 84: 776–779. 86. S. Todd, Sequential designs for monitoring two endpoints in a clinical trial. Drug Inf. J. 1999; 33: 417–426. 87. S. Durrlemann and R. Simon, Planning and monitoring of equivalence studies. Biometrics 1990; 46: 329–336. 88. C. Jennison and B. W. Turnbull, Sequential equivalence testing and repeated confidence intervals, with application to normal and binary responses. Biometrics 1993; 49: 31–43. 89. J. M. Kittelson and S. S. Emerson, A unifying family of group sequential test designs. Biometrics 1999; 55: 874–882. 90. J. Whitehead, Sequential designs for equivalence studies. Stat. Med. 1996; 15: 2703–2715.

14

GROUP SEQUENTIAL DESIGNS

91. MPS Research Unit, PEST 4: Operating Manual. Reading, UK: University of Reading, 2000. 92. SAS Institute, Inc., SAS/IML Software: Changes and Enhancements through Release 6. 11. Cary, NC: SAS Institute, Inc., 1995. 93. Mathsoft Inc., S-Plus 2000. Seattle, WA: MathSoft, 2000.

HAZARD RATE MITCHELL H. GAIL National Cancer Institute Bethesda, MD, USA

The hazard rate at time t of an event is the limit λ(t) = limit↓0 −1 Pr (t  T≺ t + |t  T), where T is the exact time to the event. Special cases and synonyms of hazard rate, depending on the event in question, include force of mortality (where the event is death), instantaneous incidence rate, incidence rate, and incidence density (where the event is disease occurrence). For events that can only occur once, such as death or first occurrence of an illness, the probability that the event occurs
t in the interval [0, t) is given by 1 − exp (− 0 λ(u)du). The
t quantity 0 λ(u)du is known as the cumulative hazard. Often, the theoretical hazard rate λ(u) is estimated by dividing the number of events that arise in a population in a short time interval by the corresponding person-years at risk. The various terms, hazard rate, force of mortality, incidence density, person–years incidence rate, and incidence rate are often used to denote estimates of the corresponding theoretical hazard rate.

1

HAZARD RATIO CLAUDIA SCHMOOR ERIKA GRAF

risk ratio is illustrated by comparing two theoretical populations. Then, using data from a hypothetical clinical trial, the calculation of various estimators is demonstrated for the hazard ratio and comments are made on their mathematical properties. Some references for further reading conclude the article.

and

University Hospital Freiburg Center of Clinical Trials Freiburg, Germany

1

2

INTRODUCTION

DEFINITIONS

Mathematically, the hazard rate of a population is defined as

Hazard ratios are useful measures to compare groups of patients with respect to a survival outcome. In the context of clinical trials ‘‘survival outcome’’ or ‘‘survival time’’ are used generically to denote the time ‘‘survived’’ from a specified time origin, like diagnosis of cancer, start of treatment, or first infarction, to a specific event of interest, like death, recurrence of cancer, or reinfarction (see survival analysis). Broadly speaking, the hazard rate quantifies the risk of experiencing an event at a given point in time. The hazard ratio at that time is the ratio of this risk in one population divided by the risk in a second population chosen as reference group. The hazard ratio between two populations can vary over time. For a given point in time, hazard ratios below 1.0 indicate that the risk of the event is lower in the first group than the risk in the reference group, whereas a hazard ratio of 1.0 indicates equal risks, and values above 1.0 imply a higher risk in the first group compared with the reference population. Often, it is plausible to assume proportional hazards (see Cox’s proportional hazards model) (i.e., to assume that although the respective hazard rates in each group may change in time, their ratio is constant over time). In this case, the constant hazard ratio adequately summarizes the relative survival experience in the two groups over the whole time axis. It describes the size of the difference between groups, similarly as a risk ratio or relative risk, which is used to compare proportions. However, the interpretation is different, as will be explained later. In the sequel, a formal definition of the hazard ratio is first provided, and the interpretation of hazard rate, hazard ratio, and

λ(t) = −

d d 1 · S(t) = − (log S(t)) S(t) dt dt

(1)

where S(t) denotes the survival function of the population (i.e., the proportion surviving beyond time t). Thus, λ(t) is the slope or instantaneous rate at which the population diminishes in the time point t, rescaled by the factor 1/S(t) to the surviving proportion S(t) of the population. Therefore, λ(t) is also called ‘‘instantaneous failure rate.’’ ‘‘Failure rate,’’ ‘‘force of mortality,’’ ‘‘hazard (rate) function,’’ or ‘‘hazard’’ have also been used to denote λ(t). λ(t) is approximately equal to the proportion of individuals that will die within one time unit from t, out of the subgroup of those that are still alive in t. Either S(t) or λ(t) may be used to characterize the survival distribution of a population, so that if one is known, the other is automatically determined. If two populations A and B with hazard rates λA (t) and λB (t) are now considered, their hazards may be compared by forming the so-called relative hazard function θ (t) =

λB (t) λA (t)

(2)

θ (t) is the hazard ratio (or relative hazard) of population B compared with population A at time t. In cases where the relative hazard function θ (t) of the two populations does not depend on t, the constant ratio θ (t) = θ is called the hazard ratio (or relative hazard) of population B compared to population A. If the hazard ratio θ (t) = θ is constant, a relationship can be established between

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

HAZARD RATIO

the hazard ratio and the two survival functions: Using Equation (1), it can be shown by mathematical transformations that for all t θ=

log[SB (t)] log[SA (t)]

(3)

Conversely, SB (t) can then be expressed as a function of SA (t) and the constant hazard ratio θ : SB (t) = SA (t)θ

for all

t,

(4)

following Equation (3). In contrast, the risk ratio (or relative risk) of population B versus population A at time t, RR(t) =

1 − SB (t) , 1 − SA (t)

(5)

compares cumulative risks of dying up to time t. 3 ILLUSTRATION OF HAZARD RATE, HAZARD RATIO, AND RISK RATIO To illustrate the interpretation of survival functions, hazard rates, hazard ratios, risk ratios (or relative risks), and the relationship between them, two hypothetical populations A and B are considered. Assume that the hypothetical control population A has a survival rate that can be described by the function SA (t) = exp( − (0.3 t)1.3 ). The corresponding survival rates of this population at yearly intervals are displayed in Table 1: 81.1% survive the first year from diagnosis, 59.8% survive the first two years, and so on, until after 10 years nearly the entire population has died. Now assume a new treatment is introduced, to the effect of reducing, at all time points t, the instantaneous death rate by 50% in the corresponding population B, which implies a constant hazard ratio of θ = 0.5 for population B versus population A as in Equation (3), so that SB (t) = SA (t)0.5 , following Equation (4). As a result, survival improves to 81.1%0.5 = 90.1% after 1 year, to 59.8%0.5 = 77.3% after 2 years, and so on. The hazard rates are also shown in Table 1. Note that although they increase over time, their ratio for population B versus population

A is indeed constant over time and equal to θ = 0.5. In contrast, the risk ratio displayed in the last column of Table 1 strongly depends on t. Here, the absolute risks of dying up to time t are being compared for population B versus population A. For example, after 4 years from diagnosis, 100% − 28.2% = 71.8% have died in the control population A, compared with 100% − 53.1% = 46.9% in population B, yielding a risk ratio of 46.9/71.8 = 0.653 for treatment versus control at t = 4. In other words, treatment decreases the risk of dying within 4 years from diagnosis by a factor of 0.653. This risk ratio is higher than the hazard ratio of 0.5, and it has a different interpretation. Recall that at all times t the hazard rate can roughly be approximated by the proportion of patients dying within 1 year among the patients still alive at the beginning of the year. S(t) − S(t + 1) is calculated to see how many patients die within 1 year from t, and the result is divided by S(t) to rescale this rate to the patients still alive at time t. For example, to approximate the hazard rate at t = 4 in the control population A, note that out of the 28.2% who survive to the beginning of year 4, SA (4)-SA (5) = 9.8% die in the following year, so that λA (4) ≈ 9.8/28.2 = 0.347, as a crude approximation (the true value is 0.412, see Table 1). Obviously, this method is too inaccurate to be of any practical relevance. More precise approximations could be obtained by using smaller time units  like weeks, days, or seconds instead of years, so that as  → 0, the approximation would yield the true hazard rate in t. However, the rough approximation approach illustrates that at all times t the hazard ratio compares the two populations with respect to the instantaneous risks of death in t only within the individuals surviving up to t. At each t, the current death rate is calculated with respect to a new reference population, consisting of the individuals still at risk of death at time t. In contrast, risk ratios compare cumulative risks of dying up to time t. Although the hazard ratio function of two populations can either vary in time or remain constant over time, the risk ratio necessarily has to vary over time. From Equation (5), one can see that a constant risk ratio (RR(t) =

HAZARD RATIO

3

Table 1. Survival and Hazard Rates in Two Hypothetical Populations A and B: Comparing Survival Experience Based on Hazard Ratio and Risk Ratio Years from diagnosis t 0 1 2 3 4 5 6 7 8 9 10

Proportion surviving t SB (t) = SA (t)0.5 SA (t) = exp(−(0.3t)1.3 ) 100.0% 81.1% 59.8% 41.8% 28.2% 18.4% 11.7% 7.3% 4.4% 2.6% 1.5%

100.0% 90.1% 77.3% 64.7% 53.1% 42.9% 34.2% 26.9% 21.0% 16.2% 12.4%

Hazard rate in t λA (t) λB (t) 0 0.272 0.335 0.378 0.412 0.440 0.465 0.487 0.507 0.525 0.542

Hazard ratio in t θ(t)

Risk ratio in t RR(t)

0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5

0.526 0.564 0.607 0.653 0.700 0.745 0.788 0.826 0.860 0.889

0 0.136 0.167 0.189 0.206 0.220 0.233 0.244 0.254 0.263 0.271

Table 2. Comparison of Hazard Ratio and Risk Ratio Topic

Hazard ratio

Risk ratio

Interpretation of the quantities compared in the ratio?

instantaneous risk of death in t among those surviving up to time t yes or no

cumulative risk of death up to time t no

yes, if roughly constant in time t

no

no

yes

Ratio constant function of time t? Useful summary measure across time axis? Useful to compare survival rates at a specific time t?

RR) would imply that the proportions surviving in the two populations are multiples of each other over the whole time axis: SB (t) = RR SA (t), which is virtually impossible to occur in real life examples unless the survival curves are identical (RR = 1), because it would mean that one of the two populations cannot start with 100% of the individuals at risk in t = 0. Therefore, in survival analysis, estimation of the hazard ratio will provide a suitable summary measure of the relative survival experiences in two populations over the whole time axis whenever the assumption of a roughly constant relative hazard function seems plausible. Estimation of a risk ratio only makes sense if, for some reason, the populations should be compared with each other at one particular point in time. Table 2 summarizes the properties of hazard ratio and risk ratio.

4 EXAMPLE ON THE USE AND USEFULNESS OF HAZARD RATIOS The estimation of the hazard ratio is illustrated by means of the example of a clinical trial (hypothetical data) introduced in a seminal paper by Peto et al. (1). In this hypothetical example, 25 patients were randomized between two treatment arms A and B, and the effect of treatment on the survival time of the patients shall be analyzed. Additionally, as a covariate, the renal function of the patients at the time of randomization is known, which may also have an influence on survival. Not all patients were observed until death and therefore their survival time was censored at the end of follow-up (censoring). The survival time is calculated as number of days from randomization to death if the patient died during follow-up, or as number of days from randomization to the end

4

HAZARD RATIO

Table 3. Data of Patients of Hypothetical Study Patient number 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 1 2 3 4

Treatment1

Renal function2

Survival time3

Survival time censored?4

A B B A A B A A B B B A B B A B B A A B A A B A B

I N N N I N N I N N N N I N N I N N N N I N N N I

8 180 632 852 52 2240 220 63 195 76 70 8 13 1990 1976 18 700 1296 1460 210 63 1328 1296 365 23

N N N Y N Y N N N N N N N Y Y N N Y Y N N Y N Y N

Treatment group (A or B). Renal function (N: normal or I: impaired). Time between randomization and death or end of follow-up (survival time) in days. Indicator whether the patient was observed until death (survival time censored = N) or not (survival time censored = Y).

of follow-up. Additionally, an indicator is necessary giving the information whether the calculated survival time is censored or uncensored. Table 3 shows the data for the 25 study patients. 5 AD HOC ESTIMATOR OF THE HAZARD RATIO The effect of treatment on the survival time of the patients can be characterized quantitatively through an estimate of the hazard ratio between the treatment groups A and B. In doing so, it is assumed implicitly that the hazard ratio is constant in time. At first, the recorded survival times in both treatment groups are reviewed (Table 4). For a calculation of the hazard ratio estimator, the survival times have to be arranged in the following way: For each time point,

when one or more deaths occurred (death times = uncensored survival times), the number of patients who died at that time and the number of patients alive just before this time point are needed, separately for both treatment groups. This information is displayed in Table 5. In total, the observed number of deaths in treatment group i is then given by (i = A, B)

Oi =

J


dij

j=1

Under the hypothesis that the hazard or instantaneous risk of dying in treatment group A is identical to the instantaneous risk of dying in treatment group B at all times, the expected number of deaths in treatment

HAZARD RATIO

group i is given by (i = A, B) Ei =

J


dj

j=1

which may be interpreted as the ratio of the relative death rates in the treatment groups (1, 2). In this example, this estimator gives a 6/8.338 = 0.5666 (i.e., hazard ratio of HR1 = 11/8.662 the hazard rate or instantaneous risk of dying in treatment group A is estimated as about 57% of that in treatment group B).

nij nj

The reasoning behind this calculation is as follows: If the hazards are identical in both groups and a total of dj deaths are observed at time point tj , then one expects dj nij /nj out of these deaths to come from group i, because nij /nj is the proportion of patients of group i among the patients at risk of death at time point tj . The total number in treatment group i is obtained by summation over all time points. Table 6 shows the required arrangement of the data and the calculations for the example, which results in OA = 6, OB = 11, EA = 8.338, and EB = 8.662. Note that the sum of observed deaths and the sum of expected deaths is the same: OA + OB = EA + EB . This relationship provides a simple check for the correctness of the calculations. An ad hoc estimator of the hazard ratio between treatment group A and treatment group B is then given by the ratio of the two relative proportions of observed to expected deaths HR1 =

5

6 CONFIDENCE INTERVAL OF THE AD HOC ESTIMATOR If no difference existed in the risk of dying between the treatment groups, the hazard ratio would be equal to 1. In order to give information about the precision of the estimator, and in order to judge whether the observed difference between treatment groups is a result of chance, a confidence interval for the hazard ratio should be calculated. With this confidence interval, also the hypothesis θ = 1 of no treatment effect (i.e., no difference of the hazard functions of treatment groups A and B) may be tested. This test may be performed at significance level α by calculating a 100(1 − α)% confidence interval. If this interval excludes 1, it may be concluded that at level α a significant difference between treatment groups exists.

OA /EA OB /EB

Table 4. Survival Times by Treatment Group (+: Censored Survival Time) Treatment Group

Survival time (days)

A B

8, 8, 52, 63, 63, 220, 365+, 852+, 1296+, 1328+, 1460+, 1976+ 13, 18, 23, 70, 76, 180, 195, 210, 632, 700, 1296, 1990+, 2240+

Table 5. Arrangement of Data for Hazard Ratio Estimation Death time t1 t2 t3 ... tJ total

Group A

Number of deaths Group B

total

Group A

dA1 dA2 dA3 ... dAJ OA

dB1 dB2 dB3 ... dBJ OB

d1 d2 d3 ... dJ OA + OB

nA1 nA2 nA3 ... nAJ

Number at risk Group B nB1 nB2 nB3 ... nBJ

tj = ordered death times (uncensored survival times) in ascending order; j = 1, . . . , J dij = observed number of deaths at time tj in treatment group i; i = A, B; j = 1, . . . , J dj = observed total number of deaths at time tj ; j = 1, . . . , J nij = number of patients being alive in treatment group i just before time tj (called ‘number at risk’); i = A, B; j = 1, nj = total number of patients being alive just before time tj ; j = 1, . . . , J J= number of death times

total n1 n2 n3 ... nJ

...,

J

6

HAZARD RATIO

A confidence interval for the hazard ratio may be calculated by considering the logarithm of the hazard ratio, because this is approximately normally distributed. A 100(1 − α)% confidence interval for the logarithm of the hazard ratio is given by

is equal to 0.4852, and the 95% confidence interval of the hazard ratio estimator HR1 is equal to [exp(−0.5681 − 1.96 × 0.4852), exp(−0.5681 + 1.96 × 0.4852)] = [0.219, 1.467]

[log(HR1 ) − z1−α/2 SE(log(HR1 )), log(HR1 ) + z1−α/2 SE(log(HR1 ))] 7 AD HOC ESTIMATOR STRATIFIED FOR THE COVARIATE RENAL FUNCTION

where z1−α/2 is the (1 − α/2)-quantile of the standard normal distribution. For a 95% confidence interval, this quantity is equal to 1.96 (3). SE(log (HR1 )) is an estimator for the standard error of the logarithm of the estimated hazard ratio and given by SE(log(HR1 )) =



In the previous estimation of the hazard ratio, a simple two-group comparison of treatment A and B was performed without taking into account covariate information of the patients (i.e. in this example the renal function). Although the patients were assigned randomly to the treatment groups, the proportion of patients with impaired renal function is not exactly identical in treatment groups A and B (group A: 4 out of 12 (33%) patients with impaired renal function; group B: 3 out of 13 (23%) patients with impaired renal function). Thus, if the covariate renal function had an influence on survival, it should be taken into account in the analysis of the effects of treatment on survival.

1/EA + 1/EB

A 100(1 − α)% confidence interval for the hazard ratio is then given by [exp(log(HR1 ) − z1−α/2 SE(log(HR1 ))), exp(log(HR1 ) + z1−α/2 SE(log(HR1 )))] In this example, the logarithm of the hazard ratio is equal to −0.5681, the standard error of the logarithm of the estimated hazard ratio

Table 6. Arrangement of Data of the Hypothetical Study for Hazard Ratio Estimation

Death time

Number of deaths Group A Group B total

Number, at risk’ Group A Group B total

Expected number of deaths Group A Group B nAj

dAj

dBj

dj

nAj

nBj

nj

dj

8 13 18 23 52 63 70 76 180 195 210 220 632 700 1296

2 0 0 0 1 2 0 0 0 0 0 1 0 0 0

0 1 1 1 0 0 1 1 1 1 1 0 1 1 1

2 1 1 1 1 2 1 1 1 1 1 1 1 1 1

12 10 10 10 10 9 7 7 7 7 7 7 5 5 4

13 13 12 11 10 10 10 9 8 7 6 5 5 4 3

25 23 22 21 20 19 17 16 15 14 13 12 10 9 7

0.960 0.435 0.455 0.476 0.500 0.947 0.412 0.438 0.467 0.500 0.538 0.583 0.500 0.556 0.571

1.040 0.565 0.545 0.524 0.500 1.053 0.588 0.562 0.533 0.500 0.462 0.417 0.500 0.444 0.429

Total

6

11

17

8.338

8.662

nj

dj

nBj

tj

nj

HAZARD RATIO

7

Table 7. Arrangement of Data of the Hypothetical Study for Hazard Ratio Estimation Stratified for Renal Function Death time Renal function normal

impaired

tj

Number of deaths Group A Group B total

Number, at risk’ Group A Group B total

Expected number of deaths Group A Group B nAj

dBj

dj

nAj

nBj

nj

dj

8

1

0

1

8

10

18

0.444

0.556

70 76 180 195 210 220 632 700 1296

0 0 0 0 0 1 0 0 0

1 1 1 1 1 0 1 1 1

1 1 1 1 1 1 1 1 1

7 7 7 7 7 7 5 5 4

10 9 8 7 6 5 5 4 3

17 16 15 14 13 12 10 9 7

0.412 0.438 0.467 0.500 0.538 0.583 0.500 0.556 0.571

0.588 0.562 0.533 0.500 0.462 0.417 0.500 0.444 0.429

Total

2

8

10

5.009

4.991

8

1

0

1

4

3

7

0.571

0.429

13 18 23 52 63

0 0 0 1 2

1 1 1 0 0

1 1 1 1 2

3 3 3 3 2

3 2 1 0 0

6 5 4 3 2

0.500 0.600 0.750 1.000 2.000

0.500 0.400 0.250 1.000 0.000

Total

4

3

7

5.421

1.579

For an analysis of the effect of the renal function on survival, one may use the same methods as described above. A simple twogroup comparison of patients is performed with impaired and with normal renal function by estimating the hazard ratio with corresponding 95% confidence interval. The calculations are not outlined in detail, but the result is simply shown. Using the notation Oi and Ei , i = I,N, for the observed and expected number of deaths in patients with impaired and normal renal function, the results are as follows: OI = 7, ON = 10, EI = 1.6, and EN = 15.4, which results in an estimated hazard ratio of patients with impaired renal function versus patients with normal renal function of 6.7375 with a 95% confidence interval of [1.3227, 34.319]. So, a strong negative influence of an impaired renal function exists on survival, because the confidence interval is far from 1.

nj

dj

nBj

dAj

nj

This result indicates that the analysis of the effect of treatment on survival should be corrected for the renal function of the patients. For this purpose, a so-called stratified analysis will be performed, which means the calculations of observed and expected numbers of deaths in treatment groups A and B described above are done separately in patients with impaired and normal renal function and combined thereafter to get a hazard ratio estimator of treatment group A versus B. The calculations are shown in Table 7. The stratified calculation leads to the following numbers for observed and expected numbers in treatment groups A and B: OA = 2 + 4 = 6, OB = 8 + 3 = 11, EA = 5.009 + 5.421 = 10.43, and EB = 4.991 + 1.579 = 6.57. Inserting these numbers in the same formulas as used for the unstratified estimator (see above), the hazard ratio estimator of treatment group A versus treatment group

8

HAZARD RATIO

B stratified for renal function is then HR2 = 6/10.43 11/6.57 = 0.3436 with a 95% confidence interval of [0.129, 0.912]. With the stratified analysis, one may now conclude that treatment A is superior to treatment B with respect to survival, because the confidence interval does not include 1. As mentioned above, a strong effect of the renal function exists on survival, and the proportion of patients with impaired renal function is slightly higher in treatment group A than in treatment group B. The stratified analysis adjusts for this imbalance and, therefore, is able to show more clearly the superiority of treatment A as compared with treatment B with respect to survival. 8

PROPERTIES OF THE AD HOC ESTIMATOR

Although not having optimal statistical properties, the ad hoc estimator of the hazard ratio above has gained popularity because of its simplicity. One should remark that this estimate is not ‘‘consistent’’ for a true hazard ratio different from 1. With increasing sample size, this estimate does not necessarily converge to the true hazard ratio, which is an important property usually required for a good statistical estimator. Thus, this ad hoc estimate should only be used for getting a rough impression of the size of the difference between treatment groups and better estimators with better mathematical properties should be applied, generally. One such class of improved estimators are the so-called generalized rank estimators. 9 CLASS OF GENERALIZED RANK ESTIMATORS OF THE HAZARD RATIO The class of so-called generalized rank estimators of the hazard ratio is defined by J 

HRGR =

j=1 J  j=1

d

wj nAj Aj

d

wj nBj Bj

which is the ratio of the sum of weighted death rates between treatment groups, weighted by some known factor wj . The construction of a statistical test based on the

class of generalized rank estimators leads to the class of so-called generalized rank tests. For different choices of the weight factor wj , different well-known two-sample tests for survival data result. Different weights can be used to weight deaths differently according to whether they occur early or late in time. Therefore, the resulting estimators have different statistical properties depending on the form of the true hazard functions. In particular, the weight factor

wj =

nAj nBj nj

results in the well-known logrank test. Under the proportional hazard assumption (i.e., an underlying constant hazard ratio), the logrank test is the most efficient test in the class of generalized rank tests, which means the logrank test is the most powerful test in this class. The generalized rank estimator with this so-called logrank weight factor will then be denoted by HR3 and can be expressed as J 

HR3 =

j=1 J  j=1

nAj nBj dAj nj nAj nAj nBj dBj nj nBj

J 

=

j=1 J  j=1

nBj nj dAj nAj nj dBj

For this example, the necessary calculations are displayed in Table 8. The generalized rank estimator with logrank weights results in HR3 =

3.009 = 0.5627 5.347

The calculation of a confidence interval for the hazard ratio based on this estimator is not as easy as for the ad hoc estimator given above. The formula is rather complicated, so it is dropped here, and only the result is presented [0.214, 1.477]. It is also possible, in accordance to the stratified ad hoc estimator, to calculate a generalized rank estimator with logrank weights that is stratified for renal function. This calculation is not presented here but another possibility for stratified estimation is presented later in this article.

HAZARD RATIO

As mentioned above, under the proportional hazard assumption, the logrank test is the most efficient test in the class of generalized rank tests. Accordingly, the corresponding hazard ratio estimator with logrank weights is the most efficient estimator in the class of generalized rank estimators, which means it has smaller variance than any other generalized rank estimator. Another estimator of the hazard ratio that has even better mathematical properties than the logrank estimator is the estimator from the Cox’s proportional hazard model.

10 ESTIMATION OF THE HAZARD RATIO WITH COX’S PROPORTIONAL HAZARD MODEL Two reasons exist to prefer Cox’s proportional hazard model (Cox model) for estimation of the hazard ratio over the methods presented so far. The most important reason is the fact that a regression model like the Cox model allows one to analyze the effect of different factors on survival simultaneously. The Cox model is the most prominent multiple regression model in survival analysis.

9

The use of multiple regression models is necessary whenever the effects of other patient characteristics besides treatment, so-called prognostic factors or covariates, are of additional interest. Reasons for this interest may either stem from the necessity of adjusting the effect of treatment for other factors, as in the example above, or from the desire to study the effect of these factors on survival per se. Another reason is that the estimator of the hazard ratio derived from the Cox model has the best mathematical properties if the proportional hazards assumption is true. It is the most efficient estimator (i.e., it has the smallest variance). In the Cox model, treatment and other factors are assumed to affect the hazard function in a multiplicative way, which implies the proportional hazards assumption (i.e., a constant hazard ratio over time). For an illustration of the Cox model with the example of the hypothetical study, an indicator X 1 for the randomized treatment (X 1 = 1, if treatment A is allocated, X 1 = 0, if treatment B is allocated) and an indicator X 2 for the factor renal function (X 2 = 1, if renal

Table 8. Calculation of the Generalized Rank Estimator with Logrank Weights HR3 for the Hazard Ratio Between Treatment Groups Death time tj 8 13 18 23 52 63 70 76 180 195 210 220 632 700 1296 total

Weight

Death rate Group A Group B

Nominator

Denominator

nBj

nAj nBj dAj nj nAj

nAj nBj dBj nj nBj

0 0.077 0.083 0.091 0 0 0.100 0.111 0.125 0.143 0.167 0 0.200 0.250 0.333

1.042 0 0 0 0.500 1.053 0 0 0 0 0 0.417 0 0 0

0 0.435 0.455 0.476 0 0 0.412 0.438 0.467 0.500 0.539 0 0.500 0.556 0.571

3.009

5.347

nAj nBj

dAj

dBj

nj

nAj

6.240 5.652 5.455 5.238 5.000 4.737 4.118 3.938 3.733 3.500 3.231 2.917 2.500 2.222 1.714

0.167 0 0 0 0.100 0.222 0 0 0 0 0 0.143 0 0 0

10

HAZARD RATIO

function is impaired, and X 2 = 0, if renal function is normal) are introduced. In the simplest situation of a Cox model, the hazard function depends on only one factor. If it is assumed that the hazard function only depends on the randomized treatment, the Cox model is formulated as

approximately normally distributed. A 100(1 − α)% confidence interval for the hazard ratio is then, as in the above situation of the ad hoc estimation, given by

λ(t|X1 ) = λ0 (t) · exp(β1 X1 )

where z1−α/2 is again the (1 − α/2)-quantile of the standard normal distribution (for a 95% confidence interval equal to 1.96). SE(log (HR4 )) is an estimator for the standard error of the logarithm of the estimated hazard ratio. In the Cox model, no explicit formula can be given for this quantity, and it has to be calculated with appropriate computer software, too. In the example, the estimated standard error is SE(log (HR4 )) = 0.5096 resulting in a 95% confidence interval of the hazard ratio estimator HR4 of [0.208, 1.531]. If one takes the renal function into account in the analysis of the treatment effect, similarly to the calculation of the ad hoc estimator stratified for renal function shown above, it can be done very easily with the Cox model. Depending on the randomized treatment and on the renal function, using the indicators X 1 and X 2 , the hazard function is now formulated as

The so-called regression coefficient β 1 represents the effect of X 1 on the hazard function. With the notation used in the introduction, this can easily be seen by calculating the ratio of the hazard functions θ (t) of patients with X 1 = 1 and X 1 = 0: θ (t) =

λA (t) λ(t|X1 = 1) λ0 (t) · exp(β1 ) = = λB (t) λ(t|X1 = 0) λ0 (t)

= exp(β1 ) which shows that the hazard ratio of treatment group A versus treatment group B is equal to exp(β 1 ) in the formulation of the Cox model, and reflects the fact that the hazard ratio in the Cox model is constant over time t. Thus, a Cox model including only one factor yields—as an alternative to the ad hoc estimator HR1 and the generalized rank estimator with logrank weights HR3 —another possibility for estimation of the hazard ratio between treatment groups. One drawback of using the Cox model for estimation of the hazard ratio is that the calculations cannot be performed by hand. The regression coefficient β 1 is estimated from empirical patient data by the so-called maximum partial likelihood procedure. This method requires complicated mathematical iteration procedures, which can only be done by appropriate computer software like the procedure PHREG of the Statistical Analysis System (SAS). In the example, the calculation results in a value of −0.5728 for β 1 , and consequently the hazard ratio of treatment group A versus treatment group B from the Cox model is estimated as HR4 = exp(−0.5728) = 0.5639. A confidence interval for the hazard ratio based on the Cox model can again be calculated by considering the logarithm of the hazard ratio log(HR4 ) = β 1 , because it is

[exp(log(HR4 ) − z1−α/2 SE(log(HR4 ))), exp(log(HR4 ) + z1−α/2 SE(log(HR4 )))]

λ(t|X1 , X2 ) = λ0 (t) · exp(β1 X1 + β2 X2 ) The unknown regression coefficient β 1 again represents the effect of X 1 on the hazard function, as can be seen easily by calculating the ratio of the hazard functions θ (t) of patients with X 1 = 1 and X 1 = 0: θ (t) = =

λA (t) λ(t|X1 = 1, X2 ) = λB (t) λ(t|X1 = 0, X2 ) λ0 (t) · exp(β1 + β2 X2 ) = exp(β1 ) λ0 (t) · exp(β2 X2 )

Again, one can see that the hazard ratio of treatment group A versus treatment group B is equal to exp(β 1 ). Similarly, the ratio of the hazard functions of patients with an impaired renal function versus patients with a normal renal function is given by λ(t|X1 , X2 = 1) λ0 (t) · exp(β1 X1 + β2 ) = λ(t|X1 , X2 = 0) λ0 (t) · exp(β1 X1 ) = exp(β2 )

HAZARD RATIO

11

Table 9. Result of Simultaneous Analysis of Effects of Treatment and Renal Function on Survival with the Cox Model Factor

log(HR)

SE(log(HR))

Hazard Ratio

95% confidence interval

Treatment (A vs. B) Renal function (impaired vs. normal)

−1.2431 4.1055

0.5993 1.1645

0.2885 60.673

[0.089, 0.934] [6.191,594.62]

Table 10. Overview of the Results of the Different Estimators for the Hazard Ratio of Treatment Group A vs. Treatment Group B Estimator Ad hoc estimator Ad hoc estimator stratified for renal function Generalized rank estimator with logrank weights Cox model Cox model adjusted for renal function

Accounts for renal function

Notation

Hazard Ratio

95% confidence interval

no yes

HR1 HR2

0.5666 0.3436

[0.219, 1.467] [0.129, 0.912]

no

HR3

0.5627

[0.214, 1.477]

no yes

HR4 HR5

0.5639 0.2885

[0.208, 1.531] [0.089, 0.934]

showing that the hazard ratio of patients with impaired renal function versus patients with normal renal function is equal to exp(β 2 ). The big difference to the model above, where X 2 was not included, is that the regression coefficients β 1 and β 2 will now be estimated simultaneously from the data. In the Cox model, this analysis of the treatment effect is called adjusted for the renal function. Besides the stratified ad hoc estimator HR2 and the stratified generalized rank estimator, this Cox model estimator of the treatment effect represents another possibility for taking renal function of the patients into account when estimating the hazard ratio between treatment groups. While in the stratified approaches renal function is only included in order to enhance the estimator of the treatment effect, in the Cox model, one has the additional benefit that its effect on survival can be studied simultaneously with the effect of treatment. Table 9 shows the result of this analysis. The estimator of the hazard ratio between treatment groups from this analysis is then HR5 = exp(− 1.2431) = 0.2885 with a 95% confidence interval of [0.089, 0.934]. Table 10 gives an overview of the results of the different estimators for the hazard ratio of treatment group A versus treatment group B presented in this article.

The estimators taking the renal function of the patients into account, HR2 and HR5 , come to the same conclusion that treatment A is superior to treatment B with respect to survival, because the confidence intervals do not include 1. 11

FURTHER READING

Definition and estimation of hazard ratios belong to the broader subject of survival analysis. In a series of 4 papers, Clark et al. (4–7) give an excellent, comprehensive introduction with great emphasis on clinical applications. Girling et al. (2) give a more basic introduction in the general context of clinical trials in their chapter 9.3.4, and a simple formula for sample size calculation in chapter 5.4.3. For those that aim to understand the subject in depth, including the more technical aspects even with limited statistical experience, the monograph by Marubini and Valsecchi (8) is recommended. REFERENCES 1. R. Peto et al., Design and analysis of randomized clinical trials requiring prolonged observation of each patient. II. Analysis and examples. Brit. J. Cancer 1977; 35: 1–39.

12

HAZARD RATIO

2. D. J. Girling et al., Clinical Trials in Cancer: Principles and Practice. Oxford: Oxford University Press, 2003. 3. D. G. Altman, Practical Statistics for Medical Research. London: Chapman & Hall, 1991. 4. T. G. Clark, M. J. Bradburn, S. B. Love, and D. G. Altman, Survival analysis part I: basic concepts and first analyses. Brit. J. Cancer 2003; 89: 232–238. 5. M. J. Bradburn, T. G. Clark, S. B. Love, and D. G. Altman, Survival analysis part II: multivariate data analysis–an introduction to concepts and methods. Brit. J. Cancer 2003; 89: 431–436. 6. M. J. Bradburn, T. G. Clark, S. B. Love, and D. G. Altman, Survival analysis part III: multivariate data analysis—choosing a model and assessing its adequacy and fit. Brit. J. Cancer 2003; 89: 605–611. 7. T. G. Clark, M. J. Bradburn, S. B. Love, and D. G. Altman, Survival analysis part IV: further concepts and methods in survival analysis. Brit. J. Cancer 2003; 89: 781–786. 8. E. Marubini and M. G. Valsecchi, Analysing Survival Data from Clinical Trials and Observational Studies. Chichester: John Wiley & Sons, 1995.

Heritability Before discussing what genetic heritability is, it is important to be clear about what it is not. For a binary trait, such as whether or not an individual has a disease, heritability is not the proportion of disease in the population attributable to, or caused by, genetic factors. For a continuous trait, genetic heritability is not a measure of the proportion of an individual’s score attributable to genetic factors. Heritability is not about cause per se, but about the causes of variation in a trait across a particular population.

Definitions Genetic heritability is defined for a quantitative trait. In general terms it is the proportion of variation attributable to genetic factors. Following a genetic and environmental variance components approach, let Y have a mean µ and variance σ 2 , which can be partitioned into genetic and environmental components of variance, such as additive genetic variance σa2 , dominance genetic variance σd2 , common environmental variance σc2 , individual specific environmental variance σe2 , and so on. Genetic heritability in the narrow sense is defined as σa2 , (1) σ2 while genetic heritability in the broad sense is defined as σg2 , (2) σ2 where σg2 includes all genetic components of variance, including perhaps components due to epistasis (gene–gene interactions; see Genotype) [3]. In addition to these random genetic effects, the total genetic variation could also include that variation explained when the effects of measured genetic markers are modeled as a fixed effect on the trait mean. The concept of genetic heritability, which is really only defined in terms of variation in a quantitative trait, has been extended to cover categorical traits by reference to a genetic liability model. It is assumed that there is an underlying, unmeasured continuous “liability” scale divided into categories by “thresholds”. Under the additional assumption that

the liability follows a normal distribution, genetic and environmental components of variance are estimated from the pattern of associations in categorical traits measured in relatives. The genetic heritability of the categorical trait is then often defined as the genetic heritability of the presumed liability (latent variable), according to (1) and (2).

Comments There is no unique value of the genetic heritability of a characteristic. Heritability varies according to which factors are taken into account in specifying both the mean and the total variance of the population under consideration. That is to say, it is dependent upon modeling of the mean, and of the genetic and environmental variances and covariances (see Genetic Correlations and Covariances). Moreover, the total variance and the variance components themselves may not be constants, even in a given population. For example, even if the genetic variance actually increased with age, the genetic heritability would decrease with age if the variation in nongenetic factors increased with age more rapidly. That is to say, genetic heritability and genetic variance can give conflicting impressions of the “strength of genetic factors”. Genetic heritability will also vary from population to population. For example, even if the heritability of a characteristic in one population is high, it may be quite different in another population in which there is a different distribution of environmental influences. Measurement error in a trait poses an upper limit on its genetic heritability. Therefore traits measured with large measurement error cannot have substantial genetic heritabilities, even if variation about the mean is completely independent of environmental factors. By the definitions above, one can increase the genetic heritability of a trait by measuring it more precisely, for example by taking repeat measurements and averaging, although strictly speaking the definition of the trait has been changed also. A trait that is measured poorly (in the sense of having low reliability) will inevitably have a low heritability because much of the total variance will be due to measurement error (σe2 ). However, a trait with relatively little measurement error will have a high heritability if all the nongenetic factors are known and taken into account in the modeling of the mean.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

2

Heritability

Fisher [1] recognized these problems and noted that whereas . . . the numerator has a simple genetic meaning, the denominator is the total variance due to errors of measurement [including] those due to uncontrolled, but potentially controllable environmental variation. It also, of course contains the genetic variance . . . Obviously, the information contained in [the genetic variance] is largely jettisoned when its actual value is forgotten, and it is only reported as a ratio to this hotch-potch of a denominator.

and Vandenburg’s F = 1/[1 − σa2 /σ 2 )] [6]. Furthermore, the statistical properties of these estimators do not appear to have been studied.

References [1] [2]

[3] [4]

Historically, other quantities have also been termed heritabilities, but it is not clear what parameter is being estimated, e.g. Holzinger’s H = (rMZ − rDZ ) (the correlation between monozygotic twins minus the correlation between dizygotic twins) (see Twin Analysis) [2], Nichol’s H R = 2(rMZ − rDZ )/rMZ [5], the E of Neel & Schull [4] based on twin data alone,

[5]

[6]

Fisher, R.A. (1951). Limits to intensive production in animals, British Agricultural Bulletin 4, 217–218. Holzinger, K.J. (1929). The relative effect of nature and nurture influences on twin differences, Journal of Educational Psychology 20, 245–248. Lush, J.L. (1948). Heritability of quantitative characters in farm animals, Suppl. Hereditas 1948, 256–375. Neel, J.V. & Schull, W.J. (1954). Human Heredity. University of Chicago Press, Chicago. Nichols, R.C. (1965). The National Merit twin study, in Methods and Goals in Human Behaviour Genetics, S.G. Vandenburg, ed. Academic Press, New York. Vandenberg, S.G. (1966). Contributions of twin research to psychology, Psychological Bulletin 66, 327–352.

JOHN L. HOPPER

HISTORICAL CONTROL

Numerous articles focus on HCT designs, especially in the area of oncology. Some authors avoid explicit discussion of historical controls in their studies by comparing their results with an (assumed) standard response rate without acknowledgment of how the reference rate was established. Articles with references to the methodological literature have previously appeared in other encyclopedia articles by Gehan (3) as well as by Beach and Baron (4). Other common references to the comparative observational study literature are Cook and Campbell (5), Rosenbaum (6), and Rothman and Greenland (7).

NEAL THOMAS Pfizer Inc., Global Research and Development, New London, Connecticut

1

HISTORICAL CONTROLS AND BIAS

The use of historical data from previously completed clinical trials, epidemiological surveys, and administrative databases to estimate the effect of placebo or standard-of-care treatment is one approach for assessing the benefit of a new therapy. Study designs without collection of concurrent control data have been largely replaced by randomized clinical trials (RCTs) with concurrently collected control data in many clinical settings. Studies comparing a new treatment with historical experience are frequently criticized and regarded by many researchers as unreliable for any purpose other than hypothesis generation because the historical controlled trial (HCT) may be subject to unknown sources of bias such as follows:

2 METHODS TO IMPROVE THE QUALITY OF COMPARISONS WITH HISTORICAL DATA There has been more acceptance of HCTs in therapeutic areas in which it is difficult to enroll patients in RCTs. HCTs are common in early development trials in oncology. The Simon two-stage design is an example of a commonly used HCT design with tumor response as the primary endpoint (8). Statisticians and other authors involved in these therapeutic areas are more positive regarding the value of HCTs, provided that they are well designed (3). Pocock (9) notes several features of a welldesigned HCT:

1. Changes in the patients over time due to changing treatment patterns. Improvements in diagnostic tests may also change the staging of patients receiving treatment.

1. Control group has received the precisely defined treatment in a recent previous study. 2. Criteria for eligibility and evaluation must be the same. 3. Prognostic factors should be known and be the same for both treatment groups. 4. No unexplained indications lead one to expect different results.

2. Changes in the measurement of the disease. These changes could be due to evolving laboratory tests or shifts in clinical rating assessments. 3. Changes in the use of concomitant medications and treatments. 4. The lack of blinding when collecting new clinical data for comparison with a known target (control) performance is another source of potential bias. Because a new treatment is typically thought to be superior to merit evaluation, the lack of blinding may be partially responsible for the perception that HCTs are often biased in favor of new treatments.

Covariate adjustment methods [Snedecor and Cochran (10)] can be applied in HCTs provided appropriately detailed patient-level data are available from earlier trials. If a large pool of patient-level data is available, matching methods (6,7) may also be used. These adjustments are examples of direct standardization in the observational study

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

HISTORICAL CONTROL

literature. When patient-level data are not available, it may still be possible to assess the potential for bias using summaries of baseline covariate information. Specialized methods for covariate adjustments and differences in exposure and follow-up time have been developed for time-to-event endpoints (11). 3 BAYESIAN METHODS TO COMBINE CONCURRENT AND HISTORICAL CONTROLS Bayesian statistical inference provides a method to combine data from historical and concurrently randomized controls (9,12). Within the Bayesian framework, it is possible to make explicit judgments about the relevance of past data accounting for potential bias. For example, a 1000-patient historical study may be regarded as equivalent to 100 concurrently randomized patients due to potentially large biases. This approach is ethically appealing because it can reduce the number of patients receiving placebo, while retaining the features of a randomized, blinded clinical trial. There has been extensive methodological development and applications of this approach in preclinical studies using animal laboratory data 13–15. Because the animals are bred and maintained in relatively homogeneous conditions for several generations, it is possible to both limit and estimate the variability between cohorts over time. The variability between cohorts provides a more objective basis for weighting data from past cohorts with concurrently randomized controls. 4 EFFECTS BASED ON TIMES-SERIES ESTIMATES Times series of event rates in large populations, often based on large administrative or federal epidemiological data collections, are another source of historical control information. Because detailed data about treatments and baseline conditions are most likely missing, time series are the least reliable source of historical control information. Using time series statistical modeling (5,6), the response to prevailing treatments during a past time period can be projected to more recent time

periods presuming no change in treatments has occurred. The projections are then compared with recent response rates after major changes in common treatments or policies. This methodological approach is common in marketing research. There are also applications for assessing the effects of medical treatments and environmental exposures. One famous example is the increasing rates of lung cancer that were associated with increased smoking rates in the early part of the twentieth century, which was cited as one source of evidence for the effect of cigarette smoking on lung cancer (16). REFERENCES 1. W. Cochran and D. Rubin, Controlling bias in observational studies: A review, Sankya, Series A 1973; 35: 417–446. 2. R. Rosenbaum and D. Rubin, The central role of the propensity score in observational studies for causal effects. Biometrika, 1983; 70: 41–55. 3. E. Gehan, Historical controls. In: Encyclopedia of Statistical Sciences, vol. 3. New York: Wiley, 1983. 4. M. Beach and J. Baron, Bias from historical controls. In: Encyclopedia of Biostatistics, vol. 1. New York: Wiley, 1998. 5. T. Cook and D. Campbell, QuasiExperimentation. Boston, MA: HoughtonMifflin, 1979. 6. P. Rosenbaum, Observational Studies. New York: Springer-Verlag, 1995. 7. K. Rothman and S. Greenland, Modern Epidemiology, 2nd ed. Philadelphia, PA: Lippincott-Raven, 1998. 8. R. Simon, Optimal two-stage designs for phase II clinical Trials. Controll. Clin. Trials 1989; 10: 1–10. 9. S. Pocock, The combination of randomized and historical controls in clinical trials. J. Chron. Dis. 1976; 29: 175–188. 10. G. Snedecor and W. Cochran, Statistical Methods, 8th ed. Ames, IA: Iowa State University Press, 1989. 11. N. Keiding, Historical controls in survival analysis. In: Encyclopedia of Biostatistics, vol. 3, New York: Wiley, 1998. 12. D. Berry and D. Strangl, Bayesian Biostatistics, New York: Marcel Dekker, 1996. 13. A. Dempster, M. Selwyn, and W. Weeks, Combining historical and randomized controls

HISTORICAL CONTROL assessing trends in proportions. J. Am. Stat. Assoc. 1983; 78: 221–227. 14. R. Tamura and S. Young, The incorporation of historical control information in tests of proportions: simulation study of Tarone’s procedure. Biometrics 1988; 42: 221–227.

3

15. R. Tarone, The use of historical control information in testing for a trend in proportions. Biometrics 1982; 38: 215–220. 16. R. Doll and B. Hill, Smoking and carcinoma of the lung. Brit. Med. J. 1950: 739–747.

HYPOTHESIS

to the discovery of Pluto in 1864, which supported Newton’s theory of gravitation. The theory, however, failed to explain irregularities of the orbit of Mercury, which could finally be understood using Einstein’s theory of relativity. According to Popper (2) and Hempel (3), the process of falsifying hypotheses has to be deductive rather than inductive. Only a deductive approach enables science to sort out incorrect hypotheses and to keep only those that passed multiple attempts to falsify them. The falsification process works in principle as follows: Let H denote a hypothesis and Ii a series of implications such that

GERD ROSENKRANZ Biostatistics and Statistical Reporting Novartis Pharma AG Basel Switzerland,

1

SCIENTIFIC HYPOTHESES

A hypothesis is ‘‘an idea or a suggestion put forward as a starting point for reasoning and explanation’’ (1). Hypotheses cannot be generated in a mechanistic fashion from empirical data: their invention is a creative process. In the course of this process, one tries to discover regularities or structures in a set of observations in an attempt to explain or organize the findings. This process is therefore inductive in nature (i.e., an attempt to derive general rules from one or more individual cases). The verification of hypotheses, however, needs a different approach. Popper (2) claimed that a necessary criterion for hypotheses to be scientific is that they are testable. Only those hypotheses that have stood up against relevant tests can be assigned a certain degree of validity and credibility. In fact, he emphasized that, strictly speaking, scientific hypotheses can only be falsified, but never become truly verified: the possibility always exists that future observations contradict a hypothesis. An example of a scientific hypothesis that passed several tests before it was eventually falsified is Newton’s theory of gravitation. It is based on the hypothesis that the gravitational forces between two bodies are proportional to the product of their masses divided by the square of their distance. One implication of this theory is that planets follow elliptical orbits on their way around the sun. When irregularities of the orbit of Neptune were observed in the nineteenth century, it first looked as if this phenomenon could not be explained by Newton’s theory. However, if one postulates the existence of another (at that point in time) unknown planet that disturbs the movements of Neptune, Newton’s theory would apply again. This reasoning led

H −→ I1 −→ . . . −→ Ik where ‘‘−→’’ stands for ‘‘implies.’’ If any of the implications turn out to be incorrect, then H is incorrect as well on purely logical grounds. In the contrary case, if Ik is true, then it cannot be concluded that H is true as well. It is easily seen that true implications can be derived from a wrong hypothesis, but not the other way a round. In this sense, deductive reasoning provides firm criteria for the verification of scientific hypotheses. The deductive method also helps to identify potential traps in hypothesis testing. Two examples are considered. First, assume that I is an implication of hypothesis H if a further assumption A holds; for example, H ∧ A −→ I. The symbol ‘‘∧’’ stands for the logical ‘‘and.’’ If I can be proven to be incorrect, one can only conclude that H or A are not both correct. In particular, the falsification of H is only possible if assumption A is known to be true. Often, two or more competing hypotheses exist. In such a situation, it would be desirable to decide which one is correct. Unfortunately, this decision is not possible in general: Either one can falsify one or both, but the falsification of one does not imply the correctness of the other with the exception of the entirely trivial case in which one claims exactly the contrary of the other.

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

2

HYPOTHESIS

STATISTICAL HYPOTHESES

Pr[k|H, n] =


 n k p (1 − p0 )n−k k 0

(1)

2.1 An Introductory Example The deductive framework has been successful in testing (i.e., falsifying) hypotheses that claim to be correct under all circumstances (like the theory of gravitation). Such hypotheses are called general or universal. However, hypotheses exist that are not universal but probabilistic in nature. These hypotheses will be called statistical hypotheses. As an example, consider the hypothesis that a vaccine is effective. Although an effective vaccine increases the likelihood of being protected against a specific infection, a vaccine can never be completely protective. As a consequence, if some vaccinees become infected despite vaccination, it is generally not possible to claim that the vaccine is not protective at all: It can, nevertheless, protect the majority of vaccinees. This example demonstrates that, unlike with universal hypothesis, probabilistic hypotheses cannot be falsified by some counter examples. A way to approach testability of statistical hypotheses is to make use of the law of large numbers: If the proportion of infected individuals among a large number of vaccinees is small, the hypothesis that the vaccine is protective is supported. If, on the contrary, the proportion of infected individuals is large, the hypothesis might be regarded as not acceptable. It needs to be emphasized, however, that regardless of how many vaccinated persons have been monitored to detect an infection, no complete certainty exists that the results look the same in the non-observed individuals. To look at the matter more closely, let E be an event of interest, like the breakthrough of an infection in a vaccinated individual, and let p = Pr[E], the probability that E occurs. Consider the hypothesis that this probability equals some number 0 < p0 < 1, or formally, H : p = p0 . To test this hypotheses, consider performing n experiments or randomly sampling n objects and to count the number of events. If H is true and the experiments are independent (i.e., the occurrence of an event in one experiment is not predictive of an experiment in another), the probability to observe k events is given by

In the context of the vaccination example, one could determine a critical number 0 ≤ c(H, n) ≤ n such that if k ≥ c(H, n), one would reject the hypothesis because too many vaccinees would become infected. Otherwise, the hypothesis would have passed the test. The number c(H, n) can be reasonably defined such that the probability of the number of events to exceed c(H, n) is small (e.g., less than some 0 < α < 1) if H is true. Hence, c(H, n) is defined such that

Pr[reject H|H ] =


 n  n j p0 (1 − p0 )n−j ≤ α j

j=c(H,n)

(2) Although a test was found to reject H, the criterion derived above has been developed under additional assumptions: One assumption is that the experiments are independent. Another requirement is that the experiments should be identically distributed (i.e., come from a single, common distribution), otherwise it is not clear whether Pr[E] = p0 holds for all experiments. Hence, strictly speaking, if k ≥ c(H, n) holds, one has only found that Pr[E] = p0 , the repeatability and the independence of the experiments cannot be true at the same time. The design and conduct of the experiments have to ensure that the assumptions of identically distributed and independent experiments are met in order to draw conclusions about the hypothesis of interest. The basic idea of statistical hypothesis tests is to identify events that are unlikely under a hypothesis H and to reject H when such events occur. It remains to be shown whether this approach delivers optimal tests or whether tests exist that enable H to be rejected with a higher probability given the same information. The key for answering this question is to compare the hypothesis H with another hypothesis K, which cannot be true at the same time; for example, K : p = p1 for some p1 > p0 . The probability to reject H if K

HYPOTHESIS

is true is then given by Pr[reject H|K ] =


 n  n j p1 (1 − p1 )n−j j

j=c(H,n)

(3) Although this probability depends on p1 , the test based on k ≥ c(H, n) can be shown to be optimal for all p1 > p0 . 2.2 The Structure of a Statistical Test In the previous section, the idea of a statistical hypothesis and a test to reject it was described for a specific situation. It is worthwhile to discuss some general ideas of hypothesis testing before moving on to more complex hypotheses. The first step in constructing a statistical test is to summarize the information contained in data x = (x1 , . . . , xn ) in an adequate way by some function T(x). In the previous section, the data were 0 or 1 depending on whether an event occurred and the statistic n was the number of events (i.e., T(x) = i=1 xi ). Next, one has to identify a rejection region S such that the hypothesis H will be rejected if T ∈ S. In the example above, S = {k; k ≥ c(H, n)}. T and S have to fulfill the following requirements. To reduce the risk to reject H in case it is correct, one requests Pr[T ∈ S|H ] ≤ α

(4)

for some small 0 < α < 1, which is called the level of the test. Second, under no alternative K should the probability of rejection be less than α, otherwise alternatives would exist under which the rejection of H is less likely than under H itself: Pr[T ∈ S|K ] ≥ α

(5)

Such tests are called unbiased. Ideally, one would also like to control the risk to fail to reject the hypothesis if it is incorrect, which is generally not possible. Instead one aims at finding an optimal test in the sense that for an alternative K, Pr[T ∈ S|K ] = max

(6)

3

Tests that fulfill Equation (6) for all alternatives K are called uniformly most powerful (UMP). As shown above, the test based on k ≥ c(H, n) is UMP for testing H : p = p0 against K : p = p1 for all p1 > p0 . UMP tests are always unbiased and exist for a wide variety of hypotheses (see Reference 4). Just as a test for a universal hypothesis aims to falsify the hypothesis, a test of a statistical hypothesis intends to reject it. However, some differences exist. A falsified universal hypothesis is falsified once for all, whereas some uncertainty about the correctness of the decision to reject a statistical hypothesis always exists. Second, a universal hypothesis can, in principle, be falsified by a single counterexample. To reject a statistical hypothesis, sufficient evidence is needed that contradicts the hypothesis, the degree of which is determined by the level of a statistical test. The choice of the level is often a matter of debate. When tables of critical values were produced in the beginning of the last century, α = 0.05 or α = 0.01 were used to limit the number of tables. Although these selections are somewhat arbitrary, α = 0.05 has been widely accepted. If the statistical test is in one direction, as in the example above, a level of α = 0.025 is often required. Applied to clinical studies, it would allow one out of 40 inefficacious drugs to pass clinical testing successfully. As this result does not look acceptable to health authorities, they tend to require two successful clinical studies (at least for certain indications) before a drug can become approved. For a level of 2.5% for an individual study, this result would correspond to α = 0.000625 if one big study is conducted instead of two separate trials (see Reference 5). Discussions can also develop concerning the size of the power of a statistical test. It was said above that, if possible, one would perform an optimal test (i.e., one that maximizes the probability to reject H if a specific alternative K is true). Apart from further assumptions, the power of a test depends on K and on the sample size (i.e., the number of independent data points). Theoretically, the probability to reject H can be made close to one for a specific K by increasing the sample size, as can be concluded from Equations

4

HYPOTHESIS

(2) and (3) for the vaccination example. This is often not feasible from a resource point of view, nor is it desirable from a scientific point of view to avoid that alternatives that differ from H in an irrelevant manner lead to rejection of H. 3 SPECIFIC HYPOTHESES IN CLINICAL STUDIES As explained above, the intention behind testing of statistical hypotheses is to reject a specific hypothesis under certain criteria that limit the uncertainty that the decision taken is correct. Hypothesis testing is therefore primarily applied in phase III studies. The role of these studies in the clinical development process is to confirm the results of previous findings by rejecting testable hypotheses. Dose finding studies or studies that intend to verify the biological concept behind a potential effect need different statistical methods. In the following, some aspects of hypotheses testing in confirmatory clinical studies will be addressed. 3.1 Superiority, Equivalence, and Non-Inferiority The formulation of an appropriate statistical hypothesis depends on the objective of the clinical study. If, for example, a clinical study is conducted to demonstrate the superiority of a new treatment over a comparator, it is most reasonable to consider testing the hypothesis that the new treatment is worse or at best equally efficacious. This approach enables one to control the risk to reject the hypothesis if it is true by selecting the level of the test. In the following, it is assumed that the difference in effect between two treatments can be described in terms of a parameter θ , which is zero in case of equal effects and positive in case of superiority. For example, one can think of θ to represent the difference in the reduction of blood pressure or the log-odds ratio of two success rates. Demonstrating superiority then means to be able to reject H : θ ≤ 0 in favor of K : θ > 0

(7)

Circumstances exist where the intention is not to demonstrate superiority but to provide evidence for similarity. If a new formulation of a drug becomes available, it can become necessary to figure out whether the bioavailability of the new formulation is comparable with that of the existing one. If so, doses and dose regimen of the existing formulation would also apply for the new formulation. In this situation, it is preferable to try to reject the hypothesis that the bioavailability of the two formulations is different. Ideally, one would like to have a level α test of H : θ = 0 versus K : θ = 0 that would only reject with a probability >α if θ is exactly zero. Unfortunately, it is not possible to construct such a test. A solution is to select an equivalence margin δ > 0 and to aim to reject H : θ ∈ (−δ, δ) in favor of K : θ ∈ (−δ, δ) (8) Another case of interest is the development of a new drug that is not necessarily more efficacious than an existing one but has other advantages, for example, a better safety profile. In this case, one would like to demonstrate that the efficacy of the new drug is comparable with the one that exists without excluding the possibility that it can be even better. At first glance, one would like to reject H : θ < 0 if K : θ ≥ 0 is true. The test would reject for values of θ being 0 or greater. Although this test looks very desirable, it is generally not possible to construct a test with a level α for all θ < 0 and >α for θ ≥ 0. As before, it is necessary to select a δ > 0, the non-inferiority margin, and to set out to reject H : θ ≤ −δ in favor of K : θ ≥ −δ (9)

Although for both hypotheses, Equation (8) and Equation (9) UMP unbiased tests exist in many relevant situations, some new issues coming up for bioequivalence and non-inferiority hypotheses exist. First, the equivalence or non-inferiority margin has to defined. For bioequivalence studies, where the ratio of pharmacokinetic parameters like area under the concentration curve or maximum concentration is of concern, general agreement exists to define δ such that ratios

HYPOTHESIS

between 0.80 and 1.25 (or the logarithms of the ratio between −0.22314 and 0.22314) belong to K. The selection of a non-inferiority margin for clinical studies is more difficult for several reasons. A study that compares a new drug with an active drug with the intention to demonstrate non-inferiority of the new one should also demonstrate that the new drug is superior to placebo, which is not an issue when placebo, the new compound, and the active comparator are tested in the same study in which both superiority over placebo and non-inferiority compared with an active compound can be demonstrated. However, in disease areas where placebo is not an option, only indirect comparisons with placebo are possible. Under these circumstances, the effect of the active comparator over placebo has to be determined from historical data through a meta-analysis. As a consequence, doubts might develop about whether the active comparator would still be superior to placebo in the setting of the present trial. If these doubts are justified, the study runs the risk to demonstrate noninferiority with respect to placebo. To some extent, this concern can be met by designing the non-inferiority study such that it is similar in design to previous studies of the active comparator in regard to patient population, co-medication, doses and regimen, and so on. Although a straightforward proposal is to set δ to a fraction of the difference between an active comparator and placebo, a non-inferiority margin needs to be justified on a case-by-case basis. An intrinsic issue of equivalence or noninferiority studies is that their credibility cannot be protected by methods like randomization and blinding (6). For example, a transcription error in the randomization code would make results under treatment and control look more similar. If this happens too often in a superiority study, the objective of the trial would be compromised, whereas these errors would favor the desired outcome in an non-inferiorty study. The impact of these intrinsic problems is somewhat lowered by the fact that clinical studies have usually more than one objective. Non-inferiority studies need to demonstrate advantages of a new drug in other areas like safety or tolerability,

5

otherwise little reason would exist to prefer the new medication over an existing one. To achieve these additional objectives, correct experimentation is required to maximize the chance of demonstrating an improvement. 3.2 Multiple Hypotheses Clinical studies often have more than one objective so that more than one hypothesis is to be tested. In the context of non-inferiority, it was already mentioned that basically two hypotheses are of interest: superiority over placebo and non-inferiority in regard to an active comparator. In other cases, more than one dose of a new drug is to be compared with a control. In many studies, a series of secondary objectives follow the primary objective or more than one primary objective is of concern. A general issue with multiple questions is that if more than one hypothesis is to be tested, more than one can be falsely rejected. As a consequence, the level set for a test of an individual hypothesis needs to be smaller to keep the overall error rate for a set of hypotheses. If the combination of hypotheses is what really matters, the definition of the level of a hypothesis test and, consequently, its power need some reconsideration. Instead of going through a general discussion of the multiplicity problem, a series of examples that frequently occur in clinical trials is presented. Interested readers are referred to Refences 7 and 8. First, consider a study that is to compare two doses of a new drug against placebo. Let θi denote the differences over placebo for dose i = 1, 2, Hi : θi ≤ 0 and Ki : θi > 0 and Ti the corresponding level α tests with critical regions Si . The objective is to demonstrate that at least one dose is superior over placebo to claim efficacy of the new drug. In statistical terms, one would like to reject the hypothesis that neither dose is superior over placebo or to reject H = H1 ∧ H2 in favor of K = K1 ∨ K2 (10) Note that the symbol ‘‘∨’’ stands for ‘‘one or the other or both.’’ For the level λ = Pr[T1 ∈ S1 ∨ T2 ∈ S2 |H1 ∧ H2 ] of the corresponding

6

HYPOTHESIS

test, α ≤ λ ≤ Pr[T1 ∈ S1 |H1 ] + Pr[T2 ∈ S2 |H2 ] = 2α (11) Hence, the level of the overall test can be somewhere between α and 2α. The simplest way to remedy the situation is to make T i level α/2 tests. Alternatively, one can apply some more powerful multiple comparison methods (see References 7–8). Combinations of several drugs are often required for the treatment of chronic diseases like hypertension and diabetes. To demonstrate the usefulness of combinations, one has to demonstrate that the efficacy of a combination is superior to the efficacy of each of its components. The hypothesis to be rejected is that the combination is not better than the best of the individual components against the alternative that it is better than each of them. Using notation from above, one intends to reject H = H1 ∨ H2 in favor of K = K1 ∧ K2 (12) As a win only exists if both hypotheses can be rejected, no adjustment of the level of the individual tests is required for Pr[T1 ∈ S1 ∧ T2 ∈ S2 |H ] ≤ Pr[T1 ∈ S1 |H1 ] = α Unfortunately, it is not possible to exhaust the significance level any better without further assumptions. One plausible assumption could be that the combination cannot be much better than one component but worse than the other. However, as pointed out earlier, if H is rejected under the additional assumption only, one could only conclude that H and the assumption cannot be correct simultaneously. Sometimes, a hierarchy exists between two objectives of a clinical study such that the achievement of one is only worthwhile if the other one can be achieved as well. An example is a non-inferiority study that intends to claim a superior safety profile for the new drug. In this setting, the better safety profile would be irrelevant if efficacy of the new compound would be inferior to that of an active comparator. If H1 stands for the noninferiority hypothesis and H2 for the safety

hypothesis, one tries first to reject H1 . Only if H 1 has been rejected, one attempts to reject H 2 , which implies λ = Pr[T1 ∈ S1 |H1 ] + Pr[T2 ∈ S2 ∧ T1 ∈ S1 |H ] = Pr[T1 ∈ S1 |H1 ] = α

(13)

Hence, an overall level α test can be obtained if the tests of the individual hypotheses have level α. In fact, Equation (13) shows that the only situation under which the test T1 ∈ S1 ∨ T2 ∈ S2 has level α is that T2 can only reject if T1 can. (Of course, the roles of T1 and T2 can be reversed.) The same reasoning can be applied if H2 is the hypothesis of no difference in efficacy. Thus, a test for noninferiority can be followed by a level α test for superiority after non-inferiority has been demonstrated without affecting the overall level. The downside of the hierarchical approach is that if the first hypothesis cannot be rejected, no way to reject the second one exists. In the absence of a clear hierarchy among hypotheses, like in a study that compares two doses against a control, hierarchical testing should be avoided. A final remark is on the power of tests of multiple hypotheses. For the two doses against a control study [Equation (10)], the power is given by Pr[T1 ∈ S1 ∨ T2 ∈ S2 |K ], which is usually greater than the power of the individual tests. Hence, even if one halves the level of the individual tests to achieve an acceptable overall level, a loss in power does not necessarily occur. Only if one is also interested in achieving a certain power for each of the individual hypotheses, the sample size of the study has to be increased. For the combination study, the power of T1 ∈ S1 ∧ T2 ∈ S2 is clearly smaller than the power of each individual test. However, the power of a combination test is slightly higher than the product of the power of the individual tests because they are always correlated since the data from the combination are used in both. For the hierarchical procedure, the power of the first test is unaffected, but some loss in power for the second test exist because the second hypothesis will only be considered if

HYPOTHESIS

the first could be rejected: Pr[T2 ∈ S2 |K ] = Pr[T2 ∈ S2 |T1 ∈ S1 , K ]Pr[T1 ∈ S1 |K ] ≤ Pr[T1 ∈ S1 |K ] The worst case obtains when both tests are independent. REFERENCES 1. A. S. Hornby, E. V. Gatenby, H. Wakefield, The Advanced Learner’s Dictionary of Current English. London: Oxford University Press, 1968. 2. K. Popper, The Logic of Scientific Discovery. London: Routledge, 2002. 3. C. G. Hempel, Philosophy of Natural Sciences. Englewood Cliffs, NJ: Prentice Hall, 1969. 4. E. L. Lehmann, Testing Statistical Hypotheses, 2nd ed. New York: Springer, 1986. 5. L. Fisher, One large, well-designed, multicenter study as an alternative to the usual FDA paradigm. Drug Inform. J. 1999; 33: 265–271. 6. S. Senn, Inherent difficulties with active controlled equivalence trials. Stat. Med. 1993; 12: 2367–2375. 7. Y. Hochberg and A. C. Tamhane, Multiple Comparison Procedures. New York: Wiley, 1987. 8. P. H. Westfall and S. S. Young, ResamplingBased Multiple Testing. New York: Wiley, 1993.

7

HYPOTHESIS TESTING

hypothesis testing is the process of evaluating the sample data to determine whether the null hypothesis is false. If based on data from the clinical trial the null hypothesis is shown to be improbable, then one concludes that the evidence supports the research hypothesis. The null hypothesis is denoted by H0 , whereas the alternative hypothesis is denoted by Ha or H1 (1, 2). Hypothesis tests can be used to test hypotheses about a single population parameter, or they can be used to compare parameters from two or more populations. Hypothesis tests can also be one-sided or two-sided depending on the nature of the alternative hypothesis. For example, a one-sided test regarding a single population mean would be specified as

NICOLE M. LAVALLEE PROMETRIKA, LLC, Cambridge, MA

MIGANUSH N. STEPANIANS PROMETRIKA, LLC, Cambridge, MA

Hypothesis testing is one of the two main areas of statistical inference, the other being parameter estimation. Although the objective of parameter estimation is to obtain point estimates or interval estimates (i.e., confidence intervals) for population parameters, hypothesis testing is used when the objective is to choose between two competing alternative theories or hypotheses regarding population parameters (e.g., population means, population proportions, population standard deviations, etc.). For example, the makers of a new antihypertensive drug believe that their medication will decrease systolic blood pressure, on average, by at least 20 mmHg in patients with moderate hypertension. The possible outcomes to this question can be specified as dichotomous, either it does or it does not. To test this claim, a statistical hypothesis test can be performed. To establish firm evidence of safety or efficacy of a new drug, clinical development plans ordinarily include at least one clinical trial that is designed to test prespecified hypotheses about population parameters that reflect the safety and/or efficacy of that drug. 1

H0 : µ ≤ µ0 versus Ha : µ > µ0 or H0 : µ ≥ µ0 versus Ha : µ < µ0 Whereas a two-sided test would be specified as H0 : µ = µ0 versus Ha : µ = µ0 Similarly, a one-sided test comparing two population means would be specified as H0 : µ1 ≤ µ2 versus Ha : µ1 > µ2 or H0 : µ1 ≥ µ2 versus Ha : µ1 < µ2 Whereas a two-sided test would be specified as

SPECIFICATION OF THE HYPOTHESES

H0 : µ1 = µ2 versus Ha : µ1 = µ2 .

The two competing theories being compared in a hypothesis test are referred to as the null hypothesis and the alternative hypothesis. The alternative hypothesis, which is also called the research hypothesis, is the statement that the researcher is trying to prove, for example, that the new drug provides greater efficacy than a placebo. The null hypothesis is the antithesis of the research hypothesis (e.g., the new drug provides no greater efficacy than placebo). Statistical

The choice between a one-sided and twosided alternative hypothesis depends on what the researcher is trying to demonstrate. For example, if the objective of a given trial is to show that there is a decrease in systolic blood pressure from baseline to the end of the trial, then a one-sided test regarding the mean change from baseline in systolic blood pressure will be performed. The one-sided

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

HYPOTHESIS TESTING

alternative hypothesis can be stated as Ha : µ < 0, where µ is the population mean change from baseline in systolic blood pressure. Similarly, if the objective is to show that mean systolic blood pressure for patients receiving a given medication (µ1 ) is lower than the mean for those receiving placebo (µ2 ), then you would choose a one-sided alternative hypothesis (Ha : µ1 < µ2 ). If the objective is simply to show a difference between the two treatments without specifying which is better, then you would choose a two-sided alternative (Ha : µ1 = µ2 ). It should be noted that although a onesided alternative hypothesis appears appropriate when the objective is to show efficacy relative to placebo, regulatory guidelines and industry conventions are that the test of superiority of an active drug over placebo should be specified as a two-sided test. This is done to make the test consistent with the twosided confidence intervals, which are appropriate for assessing the difference between two treatments (3). 2

ERRORS IN HYPOTHESIS TESTING

To reject the null hypothesis, we want to have a reasonable level of certainty that the alternative hypothesis is true. Since we can never know with complete certainty whether the alternative hypothesis is true, two types of errors can be made in hypothesis testing. One mistake would be to reject H0 when it is true, which is referred to as a Type I error (or false positive). A different mistake would be to fail to reject H0 when the alternative is true, which is referred to as a Type II error (or false negative). The possible results of a hypothesis test are summarized in Table 1. Consider the following example. The efficacy of a new drug is being assessed in a placebo-controlled trial. The null hypothesis is that there is no difference between the new drug and the placebo, on average, with respect to a particular outcome measure. The alternative hypothesis is that the new drug is different from the placebo with respect to the particular outcome measure. A Type I error would occur if it was concluded that there was a difference between the new drug and the placebo, when in fact there was not. A Type

II error would occur if it was concluded that there was no difference between the new drug and the placebo, when there actually was. A Type I error is generally considered more serious than a Type II error. This is particularly true in clinical trials, because it is important to guard against making incorrect claims regarding the potential benefit of a new treatment. Therefore, hypothesis tests are constructed to ensure a low probability of making a Type I error. The probability of a Type I error is called the significance level of the test and is denoted by α. Most often in clinical trials the significance level is set at α = 0.05, so that there is only a 5% chance of making a Type I error. The probability of a Type II error is denoted by β. Ideally, one would want to limit both the probability of a Type I error (α) and the probability of a Type II error (β). However, for any sample of a given size, the Type I error and the Type II error are inversely related; as the risk of one decreases, the risk of the other increases. The probability of a Type II error, β, can be reduced while maintaining the probability of a Type I error at a fixed level of α by increasing the sample size of the experiment. Increasing the sample size reduces the sampling variability, and therefore, it increases the probability that the test will reject the null hypothesis when the alternative hypothesis is true. This probability is referred to as the power of the test and is equal to 1 −β. The goal of hypothesis testing is to determine whether there is sufficient evidence to reject the null hypothesis in favor of the alternative hypothesis at a specific significance level (i.e., with a specified probability of committing a Type I error). If there is insufficient evidence, then we fail to reject the null hypothesis. Failing to reject the null hypothesis does not mean that the null hypothesis is accepted, however, only that there was not sufficient evidence from the sample to disprove it. For a given hypothesis test, α is always known, since it is preset at a certain level; however, the determination of β is complex, and one cannot readily specify the probability of committing a Type II error. Hence, it is recommended to reserve judgment and to avoid declaring that the null

HYPOTHESIS TESTING

3

Table 1. Possible Hypothesis Test Outcomes Truth

Conclusion from Hypothesis Test

Do Not Reject H0 Reject H0

hypothesis is accepted unless the probability of a Type II error can be provided. 3 TEST STATISTICS AND DETERMINATION OF STATISTICAL SIGNIFICANCE The decision of whether to reject the null hypothesis is made based on the evidence provided from the sample data. A test statistic is a value calculated from the sample data that summarizes the information available about the population parameter(s), and it is used to decide whether to reject the null hypothesis. The choice of the test statistic depends on the hypotheses of interest, the sample size, and the assumed distribution of the underlying population. For example, if we are performing a test regarding the mean of a normally distributed population with known variance, the appropriate test statistic is a function of the sample mean X as well as of the sample size n and the population standard deviation σ . Consider a hypothesis test of the form H0 : µ ≤ µ0 versus Ha : µ > µ0. It can be shown that the test statistic Z=

X − µo √ σ/ n

has a standard normal distribution under the null hypothesis. When the population variance is unknown but the sample size is large (a common threshold used is n > 30), test statistics of the above form can be calculated by substituting the sample standard deviation s for σ and can be assumed to have a standard normal distribution even if the underlying population is not normally distributed. The basic premise of a hypothesis test is that we will assume H0 is true unless the data provide sufficient evidence to the contrary.

H0 is true

H0 is false

Correct Decision Type I Error

Type II Error Correct Decision

However, we must specify what constitutes sufficient evidence before performing the test. Intuitively, for the above hypotheses, a sample mean, X, that is much larger than µ0 would provide evidence in favor of Ha . Under the assumption that µ = µ0 , the test statistic Z would have a positive value and fall in the right tail of the standard normal distribution. The probability of obtaining a test statistic as extreme or more extreme than the observed value calculated under the assumption that H0 is true is referred to as the P-value. Intuitively, if the P-value is very small, then it can be concluded that the data support Ha . Based on knowledge of the distribution of the test statistic, we can obtain the P-value. The smaller the P-value, the stronger the evidence that H0 is unlikely to be true. In hypothesis testing, we reject the null hypothesis if the P-value of the test is less than the prespecified significance level α. In clinical trials, when the null hypothesis is rejected in favor of the research hypothesis of interest, then it is common to state that statistical significance has been reached. The set of values for the test statistic that would cause the null hypothesis to be rejected is called the critical region. The boundary of the critical region is called the critical value. The test statistic must be more extreme than this value (i.e., further in the tail of the distribution of the test statistic) in order to reject the null hypothesis. The critical value(s) depends on the significance level for the test and whether the test is one-sided or two-sided. For one-sided hypothesis tests, there will be one critical value in the lower or upper tail of the distribution. For two-sided tests, there will be two critical values, marking the rejection regions in both tails of the distribution (provided that the test statistic has a symmetric distribution under the null hypothesis).

4

4

HYPOTHESIS TESTING

EXAMPLES

We illustrate the concepts and techniques of hypothesis testing by two examples. The first example demonstrates how to perform a test on a single population mean, and the second example compares the means from two independent samples. 4.1 One-Sample Test A new drug has been developed to treat hypertension. A Phase II trial is conducted to test whether treatment with the new drug will lower systolic blood pressure (SBP) by more than 20 mmHg, on average, in patients with moderate hypertension who are treated for two months. A random sample of 50 patients from the population the drug is intended to treat is enrolled in the trial. For each patient, the change in SBP from baseline to the end of the treatment period is determined. This is the outcome measure on which the test will be conducted. The null and alternative hypotheses can be stated as H0 : µ ≤ 20

4.2 Two-Sample Test Now suppose that instead of testing the within-patient change in SBP, the primary objective of another study is to test whether patients treated with the new drug experience a significantly greater decrease in SBP than patients treated with placebo. The null and alternative hypotheses can be stated as H0 : µ1 = µ2 or equivalently, µ1 − µ2 = 0

Ha : µ > 20

Ha : µ1 = µ2 or equivalently, µ1 − µ2 = 0

The significance level of the test is specified as α = 0.05. The sample mean and standard deviation are determined to be 26 and 13, respectively. The test statistic is equal to Z=

test, corresponding to 5% of the area under the standard normal curve. H0 will be rejected for all test statistics that are greater than 1.645. Since Z > 1.645, we reject H0 at a 0.05 level of significance and conclude that the new drug will lower SBP by more than 20 mmHg, on average, in moderate hypertensive patients. The probability of obtaining a test statistic of this magnitude or larger, under the assumption that H0 is true, is P = 0.00055. The P-value can be calculated using a statistical software package and is equal to the area under the standard normal probability density function corresponding to Z ≥ 3.264.

X − µo 26 − 20 √ = √ = 3.264 S/ n 13/ 50

where µ1 represents the mean change for the population of patients treated with the new drug and µ2 represents the mean change for the population of patients treated with placebo. For a test of two independent means, it can be shown that the test statistic

Since under the null hypothesis, this test statistic follows a standard normal distribution, the critical value is z0.05 = 1.645. Figure 1 depicts the critical region for this

X1 − X2 Z =
  S22 S21 n1 + n2

0.05

0

1.645

Figure 1. Standard normal curve with one-sided rejection region.

HYPOTHESIS TESTING

where X1 and X2 are the sample means corresponding to µ1 and µ2 , respectively, and S1 and S2 are the corresponding sample standard deviations. The test statistic Z has a standard normal distribution under the null hypothesis given that both n1 and n2 are large. For this trial, 80 patients are enrolled and randomized in equal numbers to the active drug and placebo. At the end of the trial, the sample means and standard deviations are calculated for each group. The test statistic is calculated as 22 − 16 X1 − X2 Z =
  =
 2  = 2.145 2 2 13 122 S2 S1 + + n1 n2 40 40 has a standard normal distribution. The critical value for this test is z0.025 = 1.96. Figure 2 depicts the critical region for this test, corresponding to 2.5% of the area under the standard normal curve in each tail. H0 will be rejected for all test statistics that are less than −1.96 or greater than 1.96. Since Z > 1.96, we reject H0 at a 0.05 level of significance and conclude that the new drug lowers SBP significantly more than the placebo after two months of treatment. This result is significant with a P-value of 0.0320. The P-value is equal to the area under the standard normal probability density function corresponding to the combined areas for Z > 2.145 and Z < −2.145. For the above examples of one and two sample tests of means, each had sufficiently large sample sizes, so that a test statistic could be constructed that followed a standard normal distribution. Note that in the case of

small sample sizes ( P(di ) (for all i where i = l). We call such a dose the most successful dose (MSD), and our purpose is, rather than find the MTD, to find the MSD. The relationship between toxicity and dose (xj ) and the relationship between response given no toxicity and dose are modeled through the use of two one-parameter models (1). The reasoning behind the use of underparameterized models is given in Shen and O’Quigley (8). Roughly the idea is that, at the MTD itself, only a single parameter is required to specify the probability of toxicity. Since sequential designs using updating will tend to settle on a single level, it is usually enough to work with a model only rich enough to characterize correctly the probabilities at a single level. Underparameterization, on the

MOTIVATION

O’Quigley et al. (1) proposed designs that take account of efficacy of treatment and toxic side effects simultaneously. In HIV studies, the toxicity is often the ability to tolerate the treatment, whereas in the cancer setting, this is more likely to be the presence or not of serious side effects. For HIV, efficacy will be seen readily and is most often a measure of the impact of treatment on the viral load. For cancer, it may take longer to see any effects of efficacy, although it is becoming increasingly common to make use of biological markers and other pharmacological parameters that can be indicative of a positive effect of treatment. Non-biological measurements may also be used, and in the light of developments in assessing the extent of tumor regression, for example, it is sometimes possible to use measures that previously were considered to be less reliable. Several articles (2–4) take a Bayesian approach to the question and base their findings on an efficacy–toxicity tradeoff. The approach of O’Quigley et al. can also take advantage of any prior information that is available. To do this, we would need to put prior distributions on both the toxicity and efficacy parameters and then use the Bayes formula in much the same way as used by Whitehead et al. (5). Ivanova (6) considered the algorithms of O’Quigley et al. (1), together with some further modification. The problem

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

378

IDENTIFYING THE MOST SUCCESSFUL DOSE (MSD) IN DOSE-FINDING STUDIES

other hand, means that extrapolation can be poor. However, since extrapolation is mostly only local and only used to provide rough guidance, this is a lesser concern to that of accurate local estimation. The same ideas carry through when modeling the most successful dose. Note that R(di ) and Q(di ) refer to exact, usually unknown, probabilities. We use the model based equivalents of these as ψ and φ, respectively, with these quantities coinciding when the working model is correct. Generally though we are interested in the case where the working model is only a first approximation to a more complex reality so that we do not expect R(di ) and ψ to coincide at more than one of the available levels. The same applies to Q(di ) and φ Equality will hold though for, at least, one level. Understanding the equality sign in this sense, i.e., in the sense of a working model, we can then write R(di ) = ψ(di , a) = αia and Q(di ) = φ(di , b) = β bi where 0 < α 1 < · · · < α k < 1, 0 < a < ∞, 0 < β 1 < · · · < β k < 1 and 0 < b < ∞. After the inclusion of j patients, R(di ), Q(di ), and P(di ) are estimated using maximum likelihood (11). The methods are robust to the choice of the α i (and, by extension, the choice of the β i so that these arbitrary choices have a very weak impact on operating characteristics. For large samples, under very broad conditions (9), the choice will have no impact at all. Our practical suggestion would be to divide roughly the interval (0,1) into comparable segments. For four dose levels, for example, we could choose both the α i and the β i to be 0.2, 0.4, 0.6, and 0.8. Note that, in all events, the designs remain invariant to power transformations on these values so that the operating characteristics for small samples based on the above model are exactly identical to, say, the square of the model, i.e., taking α i and β i to be equal to 0.04, 0.16, 0.36, and 0.64. The (j + 1)th patient will be included at the dose level xj , which maximizes the probˆ i ) with ˆ j+1 ) > P(d ability of success, so that P(x i = 1, . . . , k; xj+1 = di (1). To maximize the log likelihood function, we must have heterogeneity in patient observation in terms of toxicity and response. Some initial escalating scheme is then required, and here, we base

379

this on a simple up-and-down algorithm that includes patients in groups of 3 at a time. If all three experience no toxicity, then the dose level is escalated. If a single toxicity is observed, then we remain at the same dose level. If two or more toxicities are observed, then the dose level is lowered. We continue in this way until the first response is encountered. As soon as we have both a response and a toxicity, we are in a position to fit the parameters of the underparameterized model. The estimated parameters enable us to reconstruct the working dose toxicity and dose response relationships. From these we can target, for each subsequent inclusion or group of inclusions, the dose believed to be the most successful (MSD). Usually, from this point, patients are then allocated one at a time although, of course, it is possible to continue with grouped inclusion, based on the model, if desired. In estimating the MSD, all available information is taken into account via the use of our two-parameter model. Apart from unusual situations, this model will be underparameterized in practice. This approach, for an operational standpoint, represents the most obvious extension to the usual CRM strategy for dealing with toxicity alone. It works well as judged by the simulations of Zohar and O’Quigley (11) across a broad range of situations. Instead of the above prescription, O’Quigley et al. (1) suggested a compromise solution in which a CRM (10) model is used to target the rate of toxicity alone. This rate is no longer fixed, however, and can be changed in the course of the study. The solution is likely to be of greatest utility in those cases where we anticipate little or negligible toxicity. This is often the case for HIV studies and is becoming more the case for cancer studies where we work with non-cytotoxic agents. Initially we target a low rate of toxicity, keeping track of the overall success rate. Should the overall success rate be below some aimed for minimum amount, then we will allow the targeted rate of toxicity to be increased. We continue in this way until we either find some level with an overall success rate that we can take to be acceptable or until all levels have been eliminated. Specifically, a likelihood CRM (10,12) approach is used to find the dose level associated with the target probability of toxicity

380

IDENTIFYING THE MOST SUCCESSFUL DOSE (MSD) IN DOSE-FINDING STUDIES

θ . After the inclusion of j − 1 patients, the jth patient will be treated at the dose level xj , which is associated with the estimated toxicity probability closest to the target, such that |ψ(xj , aˆ j ) − θ | < |ψ(di , aˆ j ) − θ | with xj
{d1 , . . . , dk } and ψ(di , a)= α ai . Again, we use a maximum likelihood approach, and so, in order to maximize the log likelihood function, we must have heterogeneity among the toxicity observations, i.e., at least one toxicity together with at least one nontoxicity. An upand-down dose allocation procedure is used until this heterogeneity is observed. Unlike the approach based on an underparameterized model in which, before the model is fitted, we need both heterogeneity in the toxicities and in the responses, here we only need heterogeneity in the toxicities. We do not need to consider any de-escalation, and in the particular simulation here, we use inclusion of three patients at a time until the first toxicity is encountered. Following this patients are allocated typically one at a time although, once again, if desired, grouped inclusions require no real additional work. This design uses the reestimated dose-toxicity relationship for the dose allocation scheme, but as the trial progresses and our inferences are updated, then more information about the rate of success becomes available. At each dose level, the
probability of success is
estimated by Pˆ i = l≤j vl I (xl = di , yl = 0)/ l≤j I(xj =di ) As inclusions increase, we can base a decision on whether to continue, stop and recommend the current level or to reject all levels on the value of Pˆ i . . 2.1 Criteria for Stopping As the trial proceeds and information is gathered about success at the dose level di , a decision is taken based on a sequential probability ratio test (13). The hypotheses to be compared are naturally composite, but for operational simplicity, we compare two point hypotheses-modifying the parameters to these in order to obtain desirable operating characteristics. The first composite hypothesis is that the success rate is greater than p1 at di . The second composite hypothesis is that the success rate is lower than p0 at this dose level. A conclusion in favor of

a success rate greater than p1 leads the trial to end with di recommended. By contrast, a conclusion in favor of a success rate lower than p0 at di leads us to remove that dose level and lower ones, and at the same time, the target toxicity probability is increased from θ to θ + θ (until some upper maximum target toxicity rate is reached). The trial then continues using those dose levels that have not been eliminated. The sequential probability ratio test (SPRT) when treating these as point hypotheses can be expressed as   p < (ni − wi ) log p1 0     1−pi 1−
2 + wi log 1−p < log


log


2 1−
1



0

1

where ni is the number of patients included at dose level di wi is the number of successes (response with no toxicity) at di . The sequential Type I and Type II error rates are
1 and
2 . For fixed sample size, the dose level recommended at the end of the trial for all approaches is the dose level that would have been given to the (n + 1)th patient. 3 GENERAL OBSERVATIONS Identification of the MSD, as opposed to the MTD, provides a step for the statistical methodology toward current clinical practice. Here, even if not always well used, it is now common to record information of toxicity at the same time as that on toxicity. Many potential new treatments can fail because either they showed themselves to be too toxic or, at an acceptable level of toxicity, they failed to show adequate response. The classic format for a dose finding study in cancer has been to ignore information relating to response or efficacy. It has been argued that, in some sense, toxicity itself is serving a role as a surrogate for efficacy— the more toxic the treatment, then the more effect we anticipate seeing. However, this is clearly limited. We might see quite a lot of toxicity and yet little effect. We might also be able to achieve a satisfactory effect at lower rates of toxicity than are commonly used in this setting. Some of the newer biological formulations

IDENTIFYING THE MOST SUCCESSFUL DOSE (MSD) IN DOSE-FINDING STUDIES

may not even be well described by the simple paradigm-more toxicity equates to more effect. Before seeing any toxicity, it might be required to increase the given levels of dose by orders of magnitude so that it becomes increasingly likely that the methodology of dose finding will need to include more and more information relating to the desired outcome and not just the toxic side effects. To give the best chance to any promising new therapy, it becomes vital for the investigators to be able to identify, during the early development, the level of drug, the MSD, which the evidence shows provides the best chance of overall success. The work presented here is broad enough for many situations that can originate in the cancer context. Nonetheless it would be desirable, before undertaking any study using these methods, to carry out simulations of operating characteristics at the kind of sample sizes that can be realistically envisioned. This is not difficult and can also provide an informative background to the necessary discussions that will take place among the clinicians, the pharmacologists, the nursing staff, and the statisticians involved. In the context studied here, that of jointly finding a dose based on both toxicity and response, it is clear that the concept of the maximum tolerated dose (MTD) is not fully adequate. Attention instead focuses on the MSD, and our belief is that the MSD is likely to become of increasing interest in the early development of new therapeutics. In situations where we expect to see very little or almost no toxicity or in situations where we expect to see very little or response rates no greater than, say, 5%, then current designs focusing on only one endpoint are likely to perform quite adequately. The greatest gains with the suggested methods are likely to be found when there are non-negligible effects of both toxicity and efficacy and, in particular, where we weight them in a similar way; i.e., we are most interested in whether a treatment can be considered a failure-either due to toxicity or due to lack of efficacy. Then it makes sense to look for the dose that maximizes overall success rate (MSD). The proposed methods can be considered in conjunction with several others, currently available. The choice of the most appropriate method depends, as always, on the precise

381

context under investigation. Although, for the situations described in the above paragraph, existing designs would seem to require no modification, it could still be argued that by including a toxicity criterion in an efficacy dose finding design, we add a safeguard at little extra cost. One difficulty, not addressed here, is that there may be different precisions in our measurement of efficacy when compared with our measurement of toxicity, the latter tending to be typically more objective. REFERENCES 1. J. O’Quigley, M. D. Hughes, and T. Fenton, Dose-finding designs for HIV studies. Biometrics 2001; 57(4): 1018–1029. 2. P. F. Thall and K. E. Russell, A strategy for dose-finding and safety monitoring based on efficacy and adverse outcomes in phase I/II clinical trials. Biometrics 1998; 54(1): 251–264. 3. P. F. Thall, E. H. Estey, and H. G. Sung, A new statistical method for dose-finding based on efficacy and toxicity in early phase clinical trials. Invest New Drugs 1999; 17(2): 155–167. 4. P. F. Thall and J. D. Cook, Dose-finding based on efficacy-toxicity trade-offs. Biometrics 2004; 60(3): 684–693. 5. J. Whitehead, S. Patterson, D. Webber, S. Francis, and Y. Zhou, Easy-to-implement Bayesian methods for dose-escalation studies in healthy volunteers. Biostatistics 2001; 2(1): 47–61. 6. A. Ivanova, A new dose-finding design for bivariate outcomes. Biometrics 2003; 59(4): 1001–1007. 7. T. M. Braun, The bivariate continual reassessment method. Extending theCRM to phase I trials of two competing outcomes. Control Clin. Trials 2002; 23(3): 240–256. 8. L. Z. Shen and J. O’Quigley, Using a oneparameter model to sequentially estimate the root of a regression function. Computat. Stat. Data Anal. 2000; 34(3): 357–369. 9. L. Z. Shen and J. O’Quigley, Consistency of continual reassessment method under model misspecification. Biometrika 1996; 83: 395–405. 10. J. O’Quigley, M. Pepe, and L. Fisher, Continual reassessment method: A practical design for phase 1 clinical trials in cancer. Biometrics 1990; 46(1): 33–48.

382

IDENTIFYING THE MOST SUCCESSFUL DOSE (MSD) IN DOSE-FINDING STUDIES

11. S. Zohar and J. O’Quigley, Identifying the most successful dose (MSD) in dose-finding studies in cancer. Pharm. Stat. 2006; 5(3): 187–199. 12. J. O’Quigley and L. Z. Shen, Continual reassessment method: A likelihood approach. Biometrics 1996; 52(2): 673–684. 13. G. B. Wetherill and K. D. Glazebrook, Sequential methods in Statistics, 3th ed. London, U.K.: Chapman and Hall, 1986.

IMAGING SCIENCE IN MEDICINE

at all levels from the atomic through molecular, cellular, and tissue to the whole body and the social influences on disease patterns. At present, a massive research effort is focused on acquiring knowledge about genetic coding (the Human Genome Project) and the role of genetic coding in human health and disease. This effort is progressing at an astounding rate and gives rise to the belief among many medical scientists that genetics and bioinformatics (mathematical modeling of biological information, including genetic information) are the major research frontiers of medical science for the next decade or longer. The human body is an incredibly complex system. Acquiring data about its static and dynamic properties yields massive amounts of information. One of the major challenges to researchers and clinicians is the question of how to acquire, process, and display vast quantities of information about the body, so that the information can be assimilated, interpreted, and used to yield more useful diagnostic methods and therapeutic procedures. In many cases, the presentation of information as images is the most efficient approach to this challenge. As humans, we understand this efficiency; from our earliest years we rely more heavily on sight than on any other perceptual skill in relating to the world around us. Physicians also increasingly rely on images to understand the human body and intervene in the processes of human illness and injury. The use of images to manage and interpret information about biological and medical processes is certain to continue to expand in clinical medicine and also in the biomedical research enterprise that supports it. Images of a complex object such as the human body reveal characteristics of the object such as its transmissivity, opacity, emissivity, reflectivity, conductivity, and magnetizability, and changes in these characteristics with time. Images that delineate one or more of these characteristics can be analyzed to yield information about underlying properties of the object, as depicted in Table 1. For example, images (shadowgraphs) created by X rays transmitted through a region of the body

WILLIAM R. HENDEE Medical College of Wisconsin Milwaukee, WI

1

INTRODUCTION

Natural science is the search for ‘‘truth’’ about the natural world. In this definition, truth is defined by principles and laws that have evolved from observations and measurements about the natural world that are reproducible through procedures that follow universal rules of scientific experimentation. These observations reveal properties of objects and processes in the natural world that are assumed to exist independently of the measurement technique and of our sensory perceptions of the natural world. The purpose of science is to use these observations to characterize the static and dynamic properties of objects, preferably in quantitative terms, and to integrate these properties into principles and, ultimately, laws and theories that provide a logical framework for understanding the world and our place in it. As a part of natural science, human medicine is the quest for understanding one particular object, the human body, and its structure and function under all conditions of health, illness, and injury. This quest has yielded models of human health and illness that are immensely useful in preventing disease and disability, detecting and diagnosing conditions of illness and injury, and designing therapies to alleviate pain and suffering and restore the body to a state of wellness or, at least, structural and functional capacity. The success of these efforts depends on our depth of understanding of the human body and on the delineation of effective ways to intervene successfully in the progression of disease and the effects of injuries. Progress in understanding the body and intervening successfully in human disease and injury has been so remarkable that the average life span of humans in developed countries is almost twice that expected a century ago. Greater understanding has occurred

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

IMAGING SCIENCE IN MEDICINE

reveal intrinsic properties of the region such as its effective atomic number Z, physical density (grams/cm3 ) and electron density (electrons/cm3 ). Nuclear medicine images, including emission computed tomography (ECT) where pharmaceuticals release positrons [positron emission tomography (PET)] and single photons [single photon emission computed tomography (SPECT)], reveal the spatial and temporal distribution of target-specific pharmaceuticals in the human body. Depending on the application, these data can be interpreted to yield information about physiological processes such as glucose metabolism, blood volume, flow and perfusion, tissue and organ uptake, receptor binding, and oxygen utilization. In ultrasonography, images are produced by capturing energy reflected from interfaces in the body that separate tissues that have different acoustic impedances, where the acoustic impedance is the product of the physical density and the velocity of ultrasound in the tissue. Magnetic resonance imaging (MRI) of relaxation characteristics following magnetization of tissues can be translated into information about the concentration, mobility, and chemical bonding of hydrogen and, less frequently, other elements present in biological tissues. Maps of the electrical field (electroencephalography) and the magnetic field (magnetoencephalography) at the surface of the skull can be analyzed to identify areas of intense electrical activity in the brain. These and other techniques that use

the energy sources listed in Table 1 provide an array of imaging methods useful for displaying structural and functional information about the body that is essential to improving human health by detecting and diagnosing illness and injury. The intrinsic properties of biological tissues that are accessible by acquiring and interpreting images vary spatially and temporally in response to structural and functional changes in the body. Analysis of these variations yields information about static and dynamic processes in the human body. These processes may be changed by disease and disability, and identification of the changes through imaging often permits detecting and delineating the disease or disability. Medical images are pictures of tissue characteristics that influence the way energy is emitted, transmitted, reflected, etc. by the human body. These characteristics are related to, but not the same as, the actual structure (anatomy), composition (biology and chemistry), and function (physiology and metabolism) of the body. Part of the art of interpreting medical images is to bridge the gap between imaging characteristics and clinically relevant properties that aid in diagnosing and treating disease and disability. 2 ADVANCES IN MEDICAL IMAGING Advances in medical imaging have been driven historically by the ‘‘technology push’’ principle. Especially influential have been

Table 1. Energy Sources and Tissue Properties Employed in Medical Imaging Image Sources

Image Influences

Image Properties

• X rays • γ rays • Visible light • Ultraviolet light • Annihilation radiation • Electric fields • Magnetic fields • Infrared • Ultrasound • Applied voltage

• Mass Density • Electron density • Proton density • Atomic number • Velocity • Pharmaceutical location • Current flow • Relaxation • Blood volume/flow • Oxygenation level of blood • Temperature • Chemical state

• Transmissivity • Opacity • Emissivity • Reflectivity • Conductivity • Magnetizability • Resonance absorption

IMAGING SCIENCE IN MEDICINE

imaging developments in other areas, particularly in the defense and military sectors, that have been imported into medicine because of their potential applications in detecting and diagnosing human illness and injury. Examples include ultrasound developed initially for submarine detection (sonar), scintillation detectors and reactorproduced isotopes (including 131 I and 60 Co) that emerged from the Manhattan Project (the United States World War II effort to develop the atomic bomb), rare-earth fluorescent compounds synthesized initially in defense and space research laboratories, electrical conductivity detectors for detecting rapid blood loss on the battlefield, and the evolution of microelectronics and computer industries from research funded initially for security and surveillance, defense, and military purposes. Basic research laboratories have also provided several imaging technologies that have migrated successfully into clinical medicine. Examples include reconstruction mathematics for computed tomographic imaging and nuclear magnetic resonance techniques that evolved into magnetic resonance imaging and spectroscopy. The migration of technologies from other arenas into medicine has not always been successful. For example, infrared detection devices developed for night vision in military operations have so far not proven useful in medicine, despite early enthusiasm for infrared thermography as an imaging method for early detection of breast cancer. Today the emphasis in medical imaging is shifting from a ‘‘technology push’’ approach toward the concept of ‘‘biological/clinical pull.’’ This shift in emphasis reflects a deeper understanding of the biology underlying human health and disease and a growing demand for accountability and proven usefulness of technologies before they are introduced into clinical medicine. Increasingly, unresolved biological questions important in diagnosing and treating human disease and disability are used as an incentive for developing new imaging methods. For example, the function of the human brain and the causes and mechanisms of various mental disorders such as dementia, depression, and schizophrenia are among the greatest biological enigmas that confront biomedical

3

scientists and clinicians. A particularly fruitful method for penetrating this conundrum is the technique of functional imaging that employs tools such as ECT and MRI. Functional magnetic resonance imaging (fMRI) is especially promising as an approach to unraveling some of the mysteries of human brain function in health and in various conditions of disease and disability. Another example is the use of X-ray computed tomography and magnetic resonance imaging as feedback mechanisms to shape and guide the optimized deployment of radiation beams for cancer treatment. The growing use of imaging techniques in radiation oncology reveals an interesting and rather recent development. Until about three decades ago, the diagnostic and therapeutic applications of ionizing radiation were practiced by a single medical specialty. In the late 1960s, these applications began to separate into distinct medical specialties, diagnostic radiology and radiation oncology, that have separate training programs and clinical practices. Today, imaging is used extensively in radiation oncology to characterize the cancers to be treated, design the plans of treatment, guide the delivery of radiation, monitor the response of patients to treatment, and follow patients over the long term to assess the success of therapy, the occurrence of complications, and the frequency of recurrence. The process of accommodating this development in the training and practice of radiation oncology is encouraging a closer working relationship between radiation oncologists and diagnostic radiologists. 3 EVOLUTIONARY DEVELOPMENTS IN IMAGING Six major developments are converging today to raise imaging to a more prominent role in biological and medical research and in the clinical practice of medicine (1): • The ever-increasing sophistication of the

biological questions that can be addressed as knowledge expands and understanding grows about the complexity of the human body and its static and dynamic properties.

4

IMAGING SCIENCE IN MEDICINE • The ongoing evolution of imaging tech-

•

•

•

•

nologies and the increasing breadth and depth of the questions that these technologies can address at ever more fundamental levels. The accelerating advances in computer technology and information networking that support imaging advances such as three- and four-dimensional representations, superposition of images from different devices, creation of virtual reality environments, and transportation of images to remote sites in real time. The growth of massive amounts of information about patients that can best be compressed and expressed by using images. The entry into research and clinical medicine of young persons who are amazingly facile with computer technologies and comfortable with images as the principal pathway to acquiring and displaying information. The growing importance of images as an effective means to convey information in visually oriented developed cultures.

A major challenge confronting medical imaging today is the need to exploit this convergence of evolutionary developments efficiently to accelerate biological and medical imaging toward the realization of its true potential. Images are our principal sensory pathway to knowledge about the natural world. To convey this knowledge to others, we rely on verbal communications that follow accepted rules of human language, of which there are thousands of varieties and dialects. In the distant past, the acts of knowing through images and communicating through languages were separate and distinct processes. Every technological advance that brought images and words closer, even to the point of convergence in a single medium, has had a major cultural and educational impact. Examples of such advances include the printing press, photography, motion pictures, television, video games, computers, and information networking. Each of these technologies has enhanced the shift from using words to communicate information toward a more efficient synthesis of images to provide insights and words

to explain and enrich insights (2). Today, this synthesis is evolving at a faster rate than ever before, as evidenced, for example, by the popularity of television news and documentaries and the growing use of multimedia approaches to education and training. A two-way interchange of information is required to inform and educate individuals. In addition, flexible means are needed for mixing images and words and their rate and sequence of presentation to capture and retain the attention, interest, and motivation of persons engaged in the educational process. Computers and information networks provide this capability. In medicine, their use in association with imaging technologies greatly enhances the potential contribution of medical imaging to resolving patient problems in the clinical setting. At the beginning of the twenty-first century, the six evolutionary developments listed before provide the framework for major advances in medical imaging and its contributions to improvements in the health and well-being of people worldwide. 3.1 Molecular Medicine Medical imaging has traditionally focused on acquiring structural (anatomic) and, to a lesser degree, functional (physiological) information about patients at the organ and tissue levels. This focus has nurtured the correlation of imaging findings with pathological conditions and led to enhanced detection and diagnosis of human disease and injury. At times, however, detection and diagnosis occur at a stage in the disease or injury where radical intervention is required and the effectiveness of treatment is compromised. In many cases, detection and diagnosis at an earlier stage in the progression of disease and injury are required to improve the effectiveness of treatment and enhance the well-being of patients. This objective demands that medical imaging refocus its efforts from the organ and tissue levels to the cellular and molecular levels of human disease and injury. Many scientists believe that medical imaging is well positioned today to experience this refocusing as a benefit of knowledge gained at the research frontiers of molecular biology and genetics. This benefit is

IMAGING SCIENCE IN MEDICINE

often characterized as the entry of medical imaging into the era of molecular medicine. Examples include the use of magnetic resonance to characterize the chemical composition of cancers, emission computed tomography to display the perfusion of blood in the myocardium, and microfocal X-ray computed tomography to reveal the microvasculature of the lung. Contrast agents are widely employed in X ray, ultrasound, and magnetic resonance imaging techniques to enhance the visualization of properties correlated with patient anatomy and physiology. Agents in wide use today localize in tissues either by administration into specific anatomic compartments such as the gastrointestinal or vascular systems or by reliance on nonspecific changes in tissues such as increased capillary permeability or alterations in the extracellular fluid space. These localization mechanisms frequently do not yield a sufficient concentration differential of the agent to reveal subtle tissue differences associated with the presence of an abnormal condition. New contrast agents are needed that exploit growing knowledge about biochemical receptor systems, metabolic pathways, and ‘‘antisense’’ (variant DNA) molecular technologies to yield concentration differentials sufficient to reveal subtle variations among various tissues that may reflect the presence of pathological conditions. Another important imaging application of molecular medicine is using imaging methods to study cellular, molecular, and genetic processes. For example, cells may be genetically altered to attract metal ions that (1) alter the magnetic susceptibility, thereby permitting their identification by magnetic resonance imaging techniques; or (2) are radioactive and therefore can be visualized by nuclear imaging methods. Another possibility is to transect cells with genetic material that causes expression of cell surface receptors that can bind radioactive compounds (3). Conceivably this technique could be used to tag affected cells and monitor the progress of gene therapy. Advances in molecular biology and genetics are yielding new knowledge at an astonishing rate about the molecular and genetic infrastructure that underlie the static and

5

dynamic processes of human anatomy and physiology. This new knowledge is likely to yield increasingly specific approaches to using imaging methods to visualize normal and abnormal tissue structure and function at increasingly microscopic levels. These methods will in all likelihood lead to further advances in molecular medicine. 3.2 Human Vision Images are the product of the interaction of the human visual system with its environment. Any analysis of images, including medical images, must include at least a cursory review of the process of human vision. This process is outlined here; a more detailed treatment of the characteristics of the ‘‘end user’’ of images is provided in later sections of this Encyclopedia. 3.2.1 Anatomy and Physiology of the Eye. The human eye, diagrammed in Fig. 1, is an approximate sphere that contains four principal features: the cornea, iris, lens, and retina. The retina contains photoreceptors that translate light energy into electrical signals that serve as nerve impulses to the brain. The other three components serve as focusing and filtering mechanisms to transmit a sharp, well-defined light image to the retina. 3.2.1.1 Tunics. The wall of the eye consists of three layers (tunics) that are discontinuous in the posterior portion where the optic nerve enters the eye. The outermost tunic is a fibrous layer of dense connective tissue that includes the cornea and the sclera. The cornea comprises the front curved surface of the eye, contains an array of collagen fibers and no blood vessels, and is transparent to visible light. The cornea serves as a coarse focusing element to project light onto the observer’s retina. The sclera, or white of the eye, is an opaque and resilient sheath to which the eye muscles are attached. The second layer of the wall is a vascular tunic termed the uvea. It contains the choroid, ciliary body, and iris. The choroid contains a dense array of capillaries that supply blood to all of the tunics. Pigments in the choroid reduce internal light reflection that would otherwise blur the images. The ciliary body

6

IMAGING SCIENCE IN MEDICINE

its thickness. Accommodation is accompanied by constriction of the pupil, which increases the depth of field of the eye. The lens loses its flexibility from aging and is unable to accommodate, so that near objects can be focused onto the retina. This is the condition of presbyopia in which reading glasses are needed to supplement the focusing ability of the lens. Clouding of the lens by aging results in diminution of the amount of light that reaches the retina. This condition is known as a lens cataract; when severe enough it makes the individual a candidate for surgical replacement of the lens, often with an artificial lens. 3.2.1.3 Retina. The innermost layer of the eye is the retina, which is composed of two components, an outer monolayer of pigmented cells and an inner neural layer of photoreceptors. Because considerable processing of visual information occurs in the retina, it often is thought of more as a remote part of the brain rather than as simply another component of the eye. There are no photoreceptors where the optic nerve enters the eye, creating a blind spot. Near the blind spot is the mucula lutae, an area of about 3 mm2 over which the retina is especially thin. Within the macula lutae is the fovea centralis, a slight depression about 0.4 mm in diameter. The fovea is on the optical axis of the eye and is the area where the visual cones are concentrated to yield the greatest visual acuity.

FP

O

contains the muscles that support and focus the lens. It also contains capillaries that secrete fluid into the anterior segment of the eyeball. The iris is the colored part of the eye that has a central aperture termed the pupil. The diameter of the aperture can be altered by the action of muscles in the iris to control the amount of light that enters the posterior cavity of the eye. The aperture can vary from about 1.5–8 mm. 3.2.1.2 Chambers and Lens. The anterior and posterior chambers of the eye are filled with fluid. The anterior chamber contains aqueous humor, a clear plasma-like fluid that is continually drained and replaced. The posterior humor is filled with vitreous humor, a clear viscous fluid that is not replenished. The cornea, aqueous and vitreous humors, and the lens serve collectively as the refractive media of the eye. The lens of the eye provides the fine focusing of incident light onto the retina. It is a convex lens whose thickness can be changed by action of the ciliary muscles. The index of refraction of the lens is close to that of the surrounding fluids in which it is suspended, so it serves as a fine-focusing adjustment to the coarse focusing function of the cornea. The process of accommodation by which near objects are brought into focus is achieved by contraction of the ciliary muscles. This contraction causes the elastic lens to bow forward into the aqueous humor, thereby increasing

Figure 1. Horizontal section through the human eye (from Ref. 4, with permission).

IMAGING SCIENCE IN MEDICINE

The retina contains two types of photoreceptors, termed rods and cones. Rods are distributed over the entire retina, except in the blind spot and the fovea centralis. The retina contains about 125 million rods, or about 105 /mm2 . Active elements in the rods (and in the cones as well) are replenished throughout an individual’s lifetime. Rods have a low but variable threshold to light and respond to very low intensities of incident light. Vision under low illumination levels (e.g., night vision) is attributable almost entirely to rods. Rods contain the light-sensitive pigment rhodopsin (visual purple) which undergoes chemical reactions (the rhodopsin cycle) when exposed to visible light. Rhodopsin consists of a lipoprotein called opsin and a chromophore (light-absorbing chemical compound called 11-cis-retinal) (5). The chemical reaction begins with the breakdown of rhodopsin and ends with the recombination of the breakdown products into rhodopsin. The recovery process takes 20–30 minutes, which is the time required to accommodate to low levels of illumination (dark adaptation). The process of viewing with rods is known as ‘‘scotopic’’ vision. The rods are maximally sensitive to light of about 510 nm in the blue–green region of the visible spectrum. Rods have no mechanisms to discriminate different wavelengths of light, and vision under low illumination conditions is essentially ‘‘colorblind.’’ More than 100 rods are connected to each ganglion cell, and the brain has no way of discriminating among these photoreceptors to identify the origin of an action potential transmitted along the ganglion. Hence, rod vision is associated with relatively low visual acuity in combination with high sensitivity to low levels of ambient light. The retina contains about 7 million cones that are packed tightly in the fovea and diminish rapidly across the macula lutae. The density of cones in the fovea is about 140,000/mm2 . Cones are maximally sensitive to light of about 550 nm in the yellow–green portion of the visible spectrum. Cones are much (1/104 ) less sensitive to light than rods, but in the fovea there is a 1:1 correspondence between cones and ganglions, so that

7

visual acuity is very high. Cones are responsible for color vision through mechanisms that are imperfectly understood at present. One popular theory of color vision proposes that three types of cones exist; each has a different photosensitive pigment that responds maximally to a different wavelength (450 nm for ‘‘blue’’ cones, 525 nm for ‘‘green’’ cones, and 555 nm for ‘‘red’’ cones). The three cone pigments share the same chromophore as the rods; their different spectral sensitivities result from differences in the opsin component. 3.2.2 Properties of Vision. For two objects to be distinguished on the retina, light rays from the objects must define at least a minimum angle as they pass through the optical center of the eye. The minimum angle is defined as the visual angle. The visual angle, expressed in units of minutes of arc, determines the visual acuity of the eye. A rather crude measure of visual acuity is provided by the Snellen chart that consists of rows of letters that diminish in size from top to bottom. When viewed from a distance of 20 feet, a person who has normal vision can just distinguish the letters in the eighth row. This person is said to have 20:20 vision (i.e., the person can see from 20 feet what a normal person can see at the same distance). At this distance, the letters on the eighth row form a visual angle of 1 minute of arc. An individual who has excellent vision and who is functioning in ideal viewing conditions can achieve a visual angle of about 0.5 minutes of arc, which is close to the theoretical minimum defined by the packing density of cones on the retina (6). A person who has 20:100 vision can see at 20 feet what a normal person can see at 100 feet. This individual is considered to have impaired visual acuity. Other more exact tests administered under conditions of uniform illumination are used for actual clinical diagnosis of visual defects. If the lettering of a Snellen chart is reversed (i.e., white letters on a black chart, rather than black letters on a white chart), the ability of observers to recognize the letters from a distance is greatly impaired. The eye is extremely sensitive to small amounts of light. Although the cones do not

8

IMAGING SCIENCE IN MEDICINE

respond at illumination levels below a threshold of about 0.001 cd/m2 , rods are much more sensitive and respond to just a few photons. For example, as few as 10 photons can generate a visual stimulus in an area of the retina where rods are at high concentration (7). Differences in signal intensity that can just be detected by the human observer are known as just noticeable differences (JND). This concept applies to any type of signal, including light, that can be sensed by the observer. The smallest difference in signal that can be detected depends on the magnitude of the signal. For example, we may be able to discern the brightness difference between one and two candles, but we probably cannot distinguish the difference between 100 and 101 candles. This observation was quantified by the work of Weber who demonstrated that the JND is directly proportional to the intensity of the signal. This finding was quantified by Fechner as
dS = k

dI I

 ,

(1)

where I is the intensity of stimulus, dS is an increment of perception (termed a limen), and k is a scaling factor. The integral form of this expression is known as the Weber–Fechner law: S = k (log I) + C, (2) or, by setting C = −k(logI0 ),
S=k

log I I0

 .

(3)

This expression states that the perceived signal S varies with the logarithm of the relative intensity. The Weber–Fechner law is similar to the equation for expressing the intensity of sound in decibels and provides a connection between the objective measurement of sound intensity and the subjective impression of loudness. A modification of the Weber–Fechner law is known as the power law (6). In this expression, the relationship between a stimulus and the perceived signal can be stated as
 dI dS , =n S I

(4)

which, when integrated, yields log S = n(log I) + K

(5)

and, when K is written as −n[logI0 ],
S=

I I0

n ,

(6)

where I0 is a reference intensity. The last expression, known as the power law, states that the perceived signal S varies with the relative intensity raised to the power n. The value of the exponent n has been determined by Stevens for a variety of sensations, as shown in Table 2. 3.3 Image Quality The production of medical images relies on intercepting some form of radiation that is transmitted, scattered, or emitted by the body. The device responsible for intercepting the radiation is termed an image receptor (or radiation detector). The purpose of the image receptor is to generate a measurable signal as a result of energy deposited in it by the intercepted radiation. The signal is often, but not always, an electrical signal that can be measured as an electrical current or voltage pulse. Various image receptors and their uses in specific imaging applications are described in the following sections. In describing the properties of a medical image, it is useful to define certain image characteristics. These characteristics and their definitions change slightly from one type of imaging process to another, so a model is needed to present them conceptually. X-ray projection imaging is the preferred model because this process accounts for more imaging procedures than any other imaging method used in medicine. In X-ray imaging, photons transmitted through the body are intercepted by an image receptor on the side of the body opposite from the X-ray source. The probability of interaction of a photon of energy E in the detector is termed the quantum detection efficiency η (8). This parameter is defined as η = 1 − e−µ(E)t ,

(7)

IMAGING SCIENCE IN MEDICINE

9

Table 2. Exponents in the Power Law for a Variety of Psychophysical Responses∗ Perceived Quantity Loudness Brightness Smell Taste Temperature Vibration Duration Pressure Heaviness Electric shock ∗ From

Exponent

Stimulus

0.6 0.5 0.55 1.3 1.0 0.95 1.1 1.1 1.45 3.5

Binaural Point source Coffee Salt Cold on arm 60 Hz on finger White noise stimulus Static force on palm Lifted weight 60 Hz through fingers

Ref. 7.

where µ(E) is the linear attenuation coefficient of the detector material that intercepts X rays incident on the image receptor and t is the thickness of the material. The quantum detection efficiency can be increased by making the detector thicker or by using materials that absorb X rays more readily (i.e., have a greater attenuation coefficient µ(E) because they have a higher mass density or atomic number. In general, η is greater at lower Xray energies and decreases gradually with increasing energy. If the absorbing material has an absorption edge in the energy range of the incident X rays, however, the value of η increases dramatically for X-ray energies slightly above the absorption edge. Absorption edges are depicted in Fig. 2 for three detectors (Gd2 O2 S, YTaO4 , and CaW04 ) used in X-ray imaging. 3.3.1 Image Noise. Noise may be defined generically as uncertainty in a signal due to random fluctuations in the signal. Noise is present in all images. It is a result primarily of forming the image from a limited amount of radiation (photons). This contribution to image noise, referred to as quantum mottle, can be reduced by using more radiation to form the image. However, this approach also increases the exposure of the patient to radiation. Other influences on image noise include the intrinsically variable properties of the tissues represented in the image, the type of receptor chosen to acquire the image, the image receptor, processing and display electronics, and the amount of scattered radiation that contributes to the image. In most

instances, quantum mottle is the dominant influence on image noise. In an image receptor exposed to N 0 photons, the image is formed with ηN 0 photons, and the photon image noise σ can be estimated as σ =(ηN 0 )1/2 . The signal-to-noise ratio (SNR) is ηN0 (ηN0 )1/2

(8)

= (ηN0 )1/2

(9)

SNR =

A reduction in either the quantum detection efficiency η of the receptor or the number of photons N 0 used to form the image yields a lower signal-to-noise ratio and produces a noisier image. This effect is illustrated in Fig. 3. A complete analysis of signal and noise propagation in an imaging system must include consideration of the spatialfrequency dependence of both the signal and the noise. The propagation of the signal is characterized by the modulation transfer function (MTF), and the propagation of noise is described by the Wiener (noise power) spectrum W(f ). A useful quantity for characterizing the overall performance of an imaging system is its detective quantum efficiency DQE(f ), where (f ) reveals that DQE depends on the frequency of the signal. This quantity describes the efficiency with which an imaging system transfers the signal-to-noise ratio of the radiative pattern that emerges from the patient into an image to be viewed by an observer. An ideal imaging system has

IMAGING SCIENCE IN MEDICINE

FP

O

10

Figure 2. Attenuation curves for three materials used in X-ray intensifying screens (from Ref. 9, with permission).

a DQE(f ) = η at all spatial frequencies. In actuality, DQE(f ) is invariably less than η, and the difference between DQE(f ) and η becomes greater at higher spatial frequencies. If DQE = 0.1η at a particular frequency, then the imaging system performs at that spatial frequency as if the number of photons were reduced to 1/10. Hence, the noise would increase by 101/2 at that particular frequency. 3.3.2 Spatial Resolution. The spatial resolution of an image is a measure of the smallest visible interval in an object that can be seen in an image of the object. Greater spatial resolution means that smaller intervals can be visualized in the image, that is, greater spatial resolution yields an image that is sharper. Spatial resolution can be measured and expressed in two ways: (1) by a test object

that contains structures separated by various intervals, and (2) by a more formal procedure that employs the modulation transfer function (MTF). A simple but often impractical way to describe the spatial resolution of an imaging system is by measuring its point-spread function (PSF; Fig. 4). The PSF(x,y) is the acquired image of an object that consists of an infinitesimal point located at the origin, that is, for an object defined by the coordinates (0,0). The PSF is the function that operates on what would otherwise be a perfect image to yield an unsharp (blurred) image. If the extent of unsharpness is the same at all locations in the image, then the PSF has the property of being ‘‘spatially invariant,’’ and the relationship of the image to the object (or

11

FP

O

IMAGING SCIENCE IN MEDICINE

FP O

Figure 3. Illustration of quantum mottle. As the illumination of the image increases, quantum mottle decreases and the clarity of the image improves, as depicted in these classic photographs (from Ref. 10).

Figure 4. The point-spread function PSF (x,y).

perfect image) is Image(x, y) = PSF(x, y) ⊗ object(x, y),

(10)

where the ‘‘⊗’’ indicates a mathematical operation referred to as ‘‘convolution’’ between the two functions. This operation can be stated as Image(x, y) = PSF(x − u, y − v) object(u, v) du dv. (11)

The convolution effectively smears each value of the object by the PSF to yield the image. The convolution (blurring) operation can be expressed by a functional operator, S[ . . . ], such that PSF(x, y) = S[point(x, y)]

(12)

where S[ . . . ] represents the blurring operator, referred to as the linear transform of the system.

12

IMAGING SCIENCE IN MEDICINE

The modulation transfer function MTF (m,n) is obtained from the PSF(x,y) by using the two-dimensional Fourier transform F MTF(m, n) = F[PSF(x, y)],

(or line source) is defined mathematically as +∞ point(x, y) dy. line(x) =

(13)

where (m,n) are the conjugate spatial frequency variables for the spatial coordinates (x,y). This expression of MTF is not exactly correct in a technical sense. The Fourier transform of the PSF is actually termed the system transfer function, and the MTF is the normalized absolute value of the magnitude of this function. When the PSF is real and symmetrical about the x and y axes, the absolute value of the Fourier transform of the PSF yields the MTF directly (11). MTFs for representative X-ray imaging systems (intensifying screen and film) are shown in Fig. 5. The PSF and the MTF, which is in effect the representation of the PSF in frequency space, are important descriptors of spatial resolution in a theoretical sense. From a practical point of view, however, the PSF is not very helpful because it can be generated and analyzed only approximately. The difficulty with the PSF is that the source must be essentially a singular point (e.g., a tiny aperture in a lead plate exposed to X rays or a minute source of radioactivity positioned at some distance from a receptor). This condition allows only a few photons (i.e., a very small amount of radiation) to strike the receptor, and very long exposure times are required to acquire an image without excessive noise. In addition, measuring and characterizing the PSF present difficult challenges. One approach to overcoming the limitations of the PSF is to measure the line-spread function (LSF). In this approach, the source is represented as a long line of infinitesimal width (e.g., a slit in an otherwise opaque plate or a line source of radioactivity). The LSF can be measured by a microdensitometer that is scanned across the slit in a direction perpendicular to its length. As for the point-spread function, the width of the line must be so narrow that it does not contribute to the width of the image. If this condition is met, the width of the image is due entirely to unsharpness contributed by the imaging system. The slit

(14)

−∞

The line-spread function LSF(x) results from the blurring operator for the imaging system operating on a line source (Fig. 6), that is,  +∞   LSF(x) = S[line(x)] = S  point(x, y) dy −∞

+∞ = S[point(x, y)] dy

(15)

−∞

LSF(x) =

+∞ PSF(x, y) dy,

(16)

−∞

that is, the line-spread function LSF is the point-spread function PSF integrated over the y dimension. The MTF of an imaging system can be obtained from the Fourier transform of the LSF. F[LSF (x)] +∞ LSF(x) exp(−2π imx) dx =

(17) (18)

−∞

  +∞ +∞  PSF(x, y) exp(−2π imx) dy dx (19) = −∞

−∞

 +∞ +∞   = PSF(x, y) exp[−2π i(mx  −∞

−∞

 +my)] dy dy n = 0

= F[PSF(x, y)]n=0 = MTF(m, 0),

(20) (21)

that is, the Fourier transform of the linespread function is the MTF evaluated in one dimension. If the MTF is circularly symmetrical, then this expression describes the MTF completely in the two-dimensional frequency plane. One final method of characterizing the spatial resolution of an imaging system is by using the edge-response function STEP (x,y).

FP

O

IMAGING SCIENCE IN MEDICINE

Figure 6. The line-spread function is the image of an ideal line object, where S represents the linear transform of the imaging system (from Ref. 11, with permission).

FP

O

Figure 5. Point-spread (top) and modulation-transfer (bottom) functions for fast and medium-speed CaWO4 intensifying screens (from Ref. 9, with permission).

13

14

IMAGING SCIENCE IN MEDICINE

In this approach, the imaging system is presented with a source that transmits radiation on one side of an edge and attenuates it completely on the other side. The transmission is defined as: STEP(x, y) = 1 if x0, and 0 if x < 0

(22)

This function can also be written as x STEP(x, y) =

line(x) dx

(23)

−∞

The edge-spread function ESF (x) can be computed as ESF(x) = S[STEP(x, y)]  x   = S  line(x) dx

(24)

−∞

x =

S[line(x)] dx

(25)

LSF(x) dx

(26)

−∞

x = −∞

This relationship, illustrated in Fig. 7, shows that the LSF (x) is the derivative of the edgeresponse function ESF (x). LSF(x) =

d [ESF(x)] dx

(27)

3.3.3 Contrast. Image contrast refers conceptually to the difference in brightness or darkness between a structure of interest and the surrounding background in an image. Usually, information in a medical image is presented in shades of gray (levels of ‘‘grayness’’). Differences in gray shades are used to distinguish various types of tissue, analyze structural relationships, and sometimes quantify physiological function. Contrast in an image is a product of both the physical characteristics of the object being studied and the properties of the image receptor used to form the image. In some cases, contrast can be altered by the exposure conditions chosen for the examination [for example, selection of the photon energy (kVp ) and use of a contrast agent in X-ray imaging]. Image contrast is also influenced by perturbing factors such as scattered radiation and the presence of extraneous light in the detection and viewing systems. An example of the same image at different levels of contrast is shown in Fig. 8. In most medical images, contrast is a consequence of the types of tissue represented in the image. In X-ray imaging, for example, image contrast reveals differences in the attenuation of X rays among various regions of the body, modified to some degree by other factors such as the properties of the image receptor, exposure technique, and the presence of extraneous (scattered) radiation. A simplified model of the human

Figure 7. The edge-spread function is derived from the image of an ideal step function, where S represents the linear transform of the imaging system (from Ref. 11, with permission).

FP

O

The relationship between the ESF and the LSF is useful because one can obtain a microdensitometric scan of the edge to yield an

edge-spread function. The derivative of the ESF yields the LSF, and the Fourier transform of the LSF provides the MTF in one dimension.

15

FP

O

IMAGING SCIENCE IN MEDICINE

Figure 8. Different levels of contrast in an image (from Ref. 12, with permission)

body consists of three different body tissues: fat, muscle, and bone. Air is also present in the lungs, sinuses, and gastrointestinal tract, and a contrast agent may have been used to accentuate the attenuation of X rays in a particular region. The chemical composition of the three body tissues, together with their percentage mass composition, are shown in Table 3. Selected physical properties of the tissues are included in Table 4, and the mass attenuation coefficients for different tissues as a function of photon energy are shown in Fig. 9. In Table 4, the data for muscle are also approximately correct for other soft tissues such as collagen, internal organs (e.g., liver

and kidney), ligaments, blood, and cerebrospinal fluid. These data are very close to the data for water, because soft tissues, including muscle, are approximately 75% water, and body fluids are 85% to nearly 100% water. The similarity of these tissues suggests that conventional X-ray imaging yields poor discrimination among them, unless a contrast agent is used to accentuate the differences in X-ray attenuation. Because of the presence of low atomic number (low Z) elements, especially hydrogen, fat has a lower density and effective atomic number compared with muscle and other soft tissues. At less than 35 keV, X

Table 3. Elemental Composition of Tissue Constituents∗ % Composition (by Mass)

Adipose Tissue

Hydrogen Carbon Nitrogen Oxygen Sodium Magnesium Phosphorus Sulfur Potassium Calcium ∗ From

11.2 57.3 1.1 30.3

0.06

Muscle (Striated) 10.2 12.3 3.5 72.9 0.08 0.02 0.2 0.5 0.3 0.007

water 11.2

88.8

Bone (Femur) 8.4 27.6 2.7 41.0 7.0 7.0 0.2 14.7

Ref. 11, with permission.

Table 4. Properties of Tissue Constituents of the Human Body∗ Material Air Water Muscle Fat Bone ∗ From

Effective Atomic Number

Density (kg/m3 )

7.6 7.4 7.4 5.9–6.3 11.6–13.8

1.29 1.00 1.00 0.91 1.65–1.85

Ref. 11, with permission.

Electron Density (electrons/kg) 3.01 × 1026 3.34 × 1026 3.36 × 1026 3.34–3.48 × 1026 3.00–3.10 × 1026

IMAGING SCIENCE IN MEDICINE

FP O

16

rays interact in fat and soft tissues predominantly by photoelectric interactions that vary with Z3 of the tissue. This dependence provides higher image contrast among tissues of slightly different composition (e.g., fat and muscle) when low energy X rays are used, compared with that obtained from higher energy X rays that interact primarily by Compton interactions that do not depend on atomic number. Low energy X rays are used to accentuate subtle differences in soft tissues (e.g., fat and other soft tissues) in applications such as breast imaging (mammography), where the structures within the object (the breast) provide little intrinsic contrast. When images are desired of structures of high intrinsic contrast (e.g., the chest where bone, soft tissue, and air are present), higher energy X rays are used to suppress X-ray attenuation in bone which otherwise would create shadows in the image that could hide underlying soft-tissue pathology. In some accessible regions of the body, contrast agents can be used to accentuate tissue contrast. For example, iodine-containing contrast agents are often injected into the circulatory system during angiographic imaging of blood vessels. The iodinated agent is watersoluble and mixes with the blood to increase its attenuation compared with surrounding soft tissues. In this manner, blood vessels can be seen that are invisible in X-ray images without a contrast agent. Barium is another element that is used to enhance contrast, usually in studies of the gastrointestinal (GI) tract. A thick solution of a barium-containing

Figure 9. Mass attenuation coefficient of tissues.

compound is introduced into the GI tract by swallowing or enema, and the solution outlines the borders of the GI tract to permit visualization of ulcers, polyps, ruptures, and other abnormalities. Contrast agents have also been developed for use in ultrasound (solutions that contain microscopic gas bubbles that reflect sound energy) and magnetic resonance imaging (solutions that contain gadolinium that affects the relaxation constants of tissues). 3.3.4 Integration of Image Noise, Resolution and Contrast—The Rose Model. The interpretation of images requires analyzing all of the image’s features, including noise, spatial resolution, and contrast. In trying to understand the interpretive process, the analysis must also include the characteristics of the human observer. Collectively, the interpretive process is referred to as ‘‘visual perception.’’ The study of visual perception has captured the attention of physicists, psychologists, and physicians for more than a century—and of philosophers for several centuries. A seminal investigation of visual perception, performed by Albert Rose in the 1940s and 1950s, yielded the Rose model of human visual perception (13). This model is fundamentally a probabilistic analysis of detection thresholds in low-contrast images. Rose’s theory states that an observer can distinguish two regions of an image, called ‘‘target’’ and ‘‘background,’’ only if there is enough information in the image to permit making the distinction. If the signal is

IMAGING SCIENCE IN MEDICINE

assumed to be the difference in the number of photons used to define each region and the noise is the statistical uncertainty associated with the number of photons in each region, then the observer needs a certain signal-tonoise ratio to distinguish the regions. Rose suggested that this ratio is between 5 and 7. The Rose model can be quantified by a simple example (11) that assumes that the numbers of photons used to image the target and background are Poisson distributed and that the target and background yield a lowcontrast image in which N = number of photons that define the target ∼ number of photons that define the background N = signal = difference in the number of photons that define target and background A = area of the target = area of background region C = contrast of the signal compared with background

of photons that define the target and the background: N (28) C= N Signal = N = CN

SNR =

CN signal = noise (N)1/2

= C(N)1/2 = C(A)1/2

(30)

where  is the photon fluence (number of photons detected per unit area) and A is the area of the target or background. Using the experimental data of his predecessor Blackwell, Rose found that the SNR has a threshold in the range of 5–7 for differentiating a target from its background. The Rose model is depicted in Fig. 3. A second approach to integrating resolution, contrast, and noise in image perception involves using contrast-detail curves. This method reveals the threshold contrast needed to perceive an object as a function of its diameter. Contrast-detail curves are shown in Fig. 10 for two sets of images; one was acquired at a relatively high signal-to-noise ratio (SNR), and the other at a lower SNR. The curves illustrate the intuitively obvious

FP

Figure 10. Results of tests using the contrastdetail phantom of Fig. 2–14 for high-noise and low-noise cases. Each dashed line indicates combinations of size and contrast of objects that are just barely visible above the background noise (from Ref. 9, with permission).

(29)

For Poisson-distributed events, noise=(N)1/2 , and the signal-to-noise ratio (SNR) is

O

The contrast between target and background is related to the number of detected photons N and the difference N between the number

17

18

IMAGING SCIENCE IN MEDICINE

conclusion that images of large objects can be seen at relatively low contrast, whereas smaller objects require greater contrast to be visualized. The threshold contrast curves begin in the upper left corner of the graph (large objects [coarse detail], low contrast) and end in the lower right corner (small objects [fine detail], high contrast). Contrast-detail curves can be used to compare the performance of two imaging systems or the same system under different operating conditions. When the performance data are plotted, as shown in Fig. 10, the superior imaging system is one that encompasses the most visible targets or the greatest area under the curve. 3.4 Image Display/Processing Conceptually, an image is a two-(or sometimes three-) dimensional continuous distribution of a variable such as intensity or brightness. Each point in the image is an intensity value; for a color image, it may be a vector of three values that represent the primary colors red, green, and blue. An image includes a maximum intensity and a minimum intensity and hence is bounded by finite intensity limits as well as by specific spatial limits. For many decades, medical images were captured on photographic film, which provided a virtually continuous image limited only by the discreteness of image noise and film granularity. Today, however, many if not most medical images are generated by computer-based methods that yield digital images composed of a finite number of numerical values of intensity. A two-dimensional medical image may be composed of J rows of K elements, where each element is referred to as a picture element or pixel, and a threedimensional image may consist of L slices, where each slice contains J rows of K elements. The three-dimensional image is made up of volume elements or voxels; each voxel has an area of one pixel and a depth equal to the slice thickness. The size of pixels is usually chosen to preserve the desired level of spatial resolution in the image. Pixels are almost invariably square, whereas the depth of a voxel may not correspond to its width and length. Interpolation is frequently used

to adjust the depth of a voxel to its width and length. Often pixels and voxels are referred to collectively as image elements or elements. Digital images are usually stored in a computer so that each element is assigned to a unique location in computer memory. The elements of the image are usually stored sequentially, starting with elements in a row, then elements in the next row, etc., until all of the elements in a slice are stored; then the process is repeated for elements in the next slice. There is a number for each element that represents the intensity or brightness at the corresponding point in the image. Usually, this number is constrained to lie within a specific range starting at 0 and increasing to 255 [8 bit (1 byte) number], 65,535 [16 bit (2 byte) number], or even 4,294,967,295 [32 bit (4 byte) number]. To conserve computer memory, many medical images employ an 8-bit intensity number. This decision may require scaling the intensity values, so that they are mapped over the available 256 (0–255) numbers within the 8-bit range. Scaling is achieved by multiplying the intensity value Ii in each pixel by 255/Imzx to yield an adjusted intensity value to be stored at the pixel location. As computer capacity has grown and memory costs have decreased, the need for scaling to decrease memory storage has become less important and is significant today only when large numbers of high-resolution images (e.g., X-ray planar images) must be stored. To display a stored image, the intensity value for each pixel in computer memory is converted to a voltage that is used to control the brightness at the corresponding location on the display screen. The intensity may be linearly related to the voltage. However, the relationship between voltage and screen brightness is a function of the display system and usually is not linear. Further, it may be desirable to alter the voltage: brightness relationship to accentuate or suppress certain features in the image. This can be done by using a lookup table to adjust voltage values for the shades of brightness desired in the image, that is, voltages that correspond to intensity values in computer memory are adjusted by using a lookup table to other voltage values that yield the desired distribution of image brightness. If a number of lookup

IMAGING SCIENCE IN MEDICINE

Figure 11. (a) A linear display mapping; (b) a nonlinear display to increase contrast (from Ref. 6, with permission).

O

FP

O

3.4.1 Image Processing. Often pixel data are mapped onto a display system in the manner described before. Sometimes, however, it is desirable to distort the mapping to accentuate certain features of the image. This process, termed image processing, can be used to smooth images by reducing their

noisiness, accentuate detail by sharpening edges in the image, and enhance contrast in selected regions to reveal features of interest. A few techniques are discussed have as examples of image processing. In many images, the large majority of pixels have intensity values that are clumped closely together, as illustrated in Fig. 12a. Mapping these values onto the display, either directly or in modified form as described earlier is inefficient because there are few bright or dark pixels to be displayed. The process of histogram equalization improves the efficiency by expanding the contrast range within which most pixels fall, and by compressing the range at the bright and dark ends where few pixels have intensity values. This method can make subtle differences in intensity values among pixels more visible. It is useful when the pixels at the upper and lower ends of the intensity range are not important. The process of histogram equalization is illustrated in Fig. 12b. Histogram

FP

tables are available, the user can select the table desired to illustrate specific features in the image. Examples of brightness: voltage curves obtained from different lookup tables are shown in Fig. 11. The human eye can distinguish brightness differences of about 2%. Consequently, a greater absolute difference in brightness is required to distinguish bright areas in an image compared with dimmer areas. To compensate for this limitation, the voltage applied to the display screen may be modified by the factor ekV to yield an adjusted brightness that increases with the voltage V. The constant k can be chosen to provide the desired contrast in the displayed images.

19

Figure 12. (a) Representative image histogram; (b) intensity- equalized histogram.

IMAGING SCIENCE IN MEDICINE

FP O

20

equalization is also applicable when the pixels are clumped at the high or low end of intensity values. All images contain noise as an intrinsic product of the imaging process. Features of an image can be obscured by noise, and reducing the noise is sometimes desired to make such features visible. Image noise can be reduced by image smoothing (summing or averaging intensity values) across adjacent pixels. The selection and weighting of pixels for averaging can be varied among several patterns; representative techniques are included there as examples of image smoothing. In the portrayal of a pixel and its neighbors shown in Fig. 13, the intensity value of the pixel (j,k) can be replaced by the average intensity of the pixel and its nearest neighbors (6). This method is then repeated for each pixel in the image. The nearest neighbor approach is a ‘‘filter’’ to reduce noise and yield a smoothed image. The nearest neighbor approach is not confined to a set number of pixels to arrive at an average pixel value; for example, the array of pixels shown in Fig. 13 could be reduced from 9 to 5 pixels or increased from 9 to 25 pixels, in arriving at an average intensity value for the central pixel. An averaging of pixel values, in which all of the pixels are averaged by using the same weighting to yield a filtered image, is a simple approach to image smoothing. The averaging process can be modified so that the intensity values of some pixels are weighted more

Figure 13. The nearest neighbor approach to image smoothing.

heavily than others. Weighted filters usually emphasize the intensity value of the central pixel (the one whose value will be replaced by the averaged value) and give reduced weight to the surrounding pixels in arriving at a weighted average. An almost universal rule of image smoothing is that when smoothing is employed, noise decreases, but unsharpness increases (i.e., edges are blurred). In general, greater smoothing of an image to reduce noise leads to greater blurring of edges as a result of increased unsharpness. Work on image processing often is directed at achieving an optimum balance between increased image smoothing and increased image unsharpness to reveal features of interest in specific types of medical images. The image filtering techniques described before are examples of linear filtering. Many other image-smoothing routines are available, including those that employ ‘‘nonlinear’’ methods to reduce noise. One example of a nonlinear filter is replacement of a pixel intensity by the median value, rather than the average intensity, in a surrounding array of pixels. This filter removes isolated noise spikes and speckle from an image and can help maintain sharp boundaries. Images can also be smoothed in frequency space rather than in real space, often at greater speed. 3.4.2 Image Restoration. Image restoration is a term that refers to techniques to remove or reduce image blurring, so that the image is ‘‘restored’’ to a sharper condition that is more representative of the object. This

IMAGING SCIENCE IN MEDICINE

technique is performed in frequency space by using Fourier transforms for both the image and the point-spread function of the imaging system. The technique is expressed as O(j, k) =

I(j, k) P(j, k)

(31)

where I (j,k) is the Fourier transform of the image, P (j,k) is the Fourier transform of the point-spread function, and O (j,k) is the Fourier transform of the object (in threedimensional space, a third spatial dimension l would be involved). This method implies that the unsharpness (blurring) characteristics of the imaging device can be removed by image processing after the image has been formed. Although many investigators have pursued image restoration with considerable enthusiasm, interest has waned in recent years because two significant limitations of the method have surfaced (6). The first is that the Fourier transform P (j,k) can be zero for certain values of (j,k), leading to an undetermined value for O (j,k). The second is that image noise is amplified greatly by the restoration process and often so overwhelms the imaging data that the restored image is useless. Although methods have been developed to reduce these limitations, the conclusion of most attempts to restore medical images is that it is preferable to collect medical images at high resolution, even if sensitivity is compromised, than to collect the images at higher sensitivity and lower resolution, and then try to use image-restoration techniques to recover image resolution. 3.4.3 Image Enhancement. The human eye and brain act to interpret a visual image principally in terms of boundaries that are presented as steep gradients in image brightness between two adjacent regions. If an image is processed to enhance these boundary (edge) gradients, then image detail may be more visible to the observer. Edgeenhancement algorithms function by disproportionately increasing the high-frequency components of the image. This approach also tends to enhance image noise, so that edgeenhancement algorithms are often used together with an image-smoothing filter to suppress noise.

4

21

CONCLUSION

The use of images to detect, diagnose, and treat human illness and injury has been a collaboration among physicists, engineers, and physicians since the discovery of X rays in 1895 and the first applications of X-ray images to medicine before the turn of the twentieth century. The dramatic expansion of medical imaging during the past century and the ubiquitous character of imaging in all of medicine today have strenghthened the linkage connecting physics, engineering, and medical imaging. This bond is sure to grow even stronger as imaging develops as a tool for probing the cellular, molecular, and genetic nature of disease and disability during the first few years of the twenty-first century. Medical imaging offers innumerable challenges and opportunities to young physicists and engineers interested in applying their knowledge and insight to improving the human condition. REFERENCES 1. W. R. Hendee, Rev. Mod. Phys. 71(2), Centenary, S444–S450 (1999). 2. R. N. Beck, in W. Hendee and J. Trueblood, eds., Digital Imaging, Medical Physics, Madison, WI, 1993, pp. 643–665. 3. J. H. Thrall, Diagn. Imaging (Dec.), 23–27 (1997). 4. W. R. Hendee, in W. Hendee and J. Trueblood, eds., Digital Imaging, Medical Imaging, Madison, WI, 1993, pp. 195–212. 5. P. F. Sharp and R. Philips, in W. R. Hendee and P. N. T. Wells, ed., The Perception of Visual Information, Springer, NY, 1997, pp. 1–32. 6. B. H. Brown et al., Medical Physics and Biomedical Engineering, Institute of Physics Publishing, Philadelphia, 1999. 7. S. S. Stevens, in W. A. Rosenblith, ed., Sensory Communication, MIT Press, Cambridge, MA, 1961, pp. 1–33. 8. J. A. Rowlands, in W.R. Hendee, ed., Biomedical Uses of Radiation, Wiley-VCH, Weinheim, Germany, 1999, pp. 97–173. 9. A. B. Wolbarst, Physics of Radiology, Appleton and Lange, Norwalk, CT, 1993. 10. A. Rose, Vision: Human and Electronic, Plenum Publishing, NY, 1973.

22

IMAGING SCIENCE IN MEDICINE

11. B. H. Hasegawa, The Physics of Medical X-Ray Imaging, 2nd ed., Medical Physics, Madison, WI, 1991, p. 127. 12. W. R. Hendee, in C. E. Putman and C. E. Ravin, eds., Textbook of Diagnostic Imaging, 2nd ed., W.B. Saunders, Philadelphia, 1994, pp. 1–97. 13. A. Rose, in Proc. Inst. Radioengineers 30, 293–300 (1942).

IMPUTATION

analysis to only those subjects with no missing data) is generally acceptable. Missing data are missing at random (MAR) if missingness depends only on observed values of variables and not on any missing values; for example, if the value of blood pressure at time two is more likely to be missing when the observed value of blood pressure at time one was normal, regardless of the value of blood pressure at time two. If missingness depends on the values that are missing, even after conditioning on all observed quantities, the missing data mechanism is not missing at random (NMAR). Missingness must then be modeled jointly with the data—the missingness mechanism is ‘‘nonignorable.’’ The specific imputation procedures described here are most appropriate when the missing data are MAR and ignorable (see References 2 and 3 for details). Imputation (multiple) can still be validly used with nonignorable missing data, but it is more challenging to do it well.

SAMANTHA R. COOK and DONALD B. RUBIN Harvard University

1

INTRODUCTION

Missing data are a common problem with large databases in general and with clinical and health-care databases in particular. Subjects in clinical trials may fail to provide data at one or more time points or may drop out of a trial altogether, for reasons including lack of interest, untoward side effects, and change of geographical location. Data may also be ‘‘missing’’ due to death, although the methods described here are generally not appropriate for such situations. An intuitive way to handle missing data is to fill in (i.e., impute) plausible values for the missing values, thereby creating completed datasets that can be analyzed using standard complete-data methods. The past 25 years have seen tremendous improvements in the statistical methodology for handling incomplete datasets. After briefly discussing missing data mechanisms, the authors present some common imputation methods, focusing on multiple imputation (1). They then discuss computational issues, present some examples, and conclude with a short summary.

2

3

SINGLE IMPUTATION

Single imputation refers to imputing one value for each missing datum. Singly imputed datasets are straightforward to analyze using complete-data methods, which makes single imputation an apparently attractive option with incomplete data. Little and Rubin (3, p. 72) offer the following guidelines for creating imputations. They should be (1) conditional on observed variables; (2) multivariate, to reflect associations among missing variables; and (3) drawn from predictive distributions rather than set equal to means, to ensure that variability is reflected. Unconditional mean imputation, which replaces each missing value with the mean of the observed values of that variable, meets none of the three guidelines listed above. Conditional mean imputation can satisfy the first guideline by filling in missing values with means calculated within cells defined by variables such as gender and/or treatment arm, but it does not meet the second or third guidelines. Regression imputation can satisfy the first two guidelines by replacing

MISSING DATA MECHANISMS

A missing data mechanism is a probabilistic rule that governs which data will be observed and which will be missing. Rubin (2) and Little and Rubin (3) describe three types of missing data mechanisms. Missing data are missing completely at random (MCAR) if missingness is independent of both observed and missing values of all variables. MCAR is the only missing data mechanism for which ‘‘complete case’’ analysis (i.e., restricting the

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

IMPUTATION

the missing values for each variable with the values predicted from a regression (e.g., least squares) of that variable on other variables. Stochastic regression imputation adds random noise to the value predicted by the regression model, and when done properly can meet all three guidelines. Hot deck imputation replaces each missing value with a random draw from a donor pool of observed values of that variable; donor pools are selected, for example, by choosing individuals with complete data who have ‘‘similar’’ observed values to the subject with missing data, e.g., by exact matching or using a distance measure on observed variables to define similar. Hot deck imputation, when done properly, can also satisfy all three of the guidelines listed above. Even though analyzing a singly imputed dataset with standard techniques is easy, such an analysis will nearly always result in estimated standard errors that are too small, confidence intervals that are too narrow, and P-values that are too significant, regardless of how the imputations were created. Thus, single imputation is almost always statistically invalid. However, the multiple version of a single imputation method will be valid if the imputation method is ‘‘proper.’’ Proper imputations satisfy the three criteria of Little and Rubin.

4 PROPERLY DRAWN SINGLE IMPUTATIONS Let Y represent the complete data; i.e., all data that would be observed in the absence of missing data, and let Y = {Y obs ,Y mis }, the observed and missing values, respectively. For notational simplicity, assume ignorability of the missing data mechanism. Also let θ represent the (generally vector-valued) parameter associated with an appropriate imputation model, which consists of both a sampling distribution on Y governed by θ , p(Y|θ ), and a prior distribution on θ , p(θ ). A proper imputation is often most easily obtained as a random draw from the posterior predictive distribution of the missing data given the observed data, which formally

can be written as
p(Ymis |Yobs ) = p(Ymis , θ |Yobs ) dθ
=

p(Ymis |Yobs , θ )p(θ |Yobs ) dθ.

If the missing data follow a monotone pattern (defined below), this distribution is straight-forward to sample from. When missing data are not monotone, iterative computational methods are generally necessary, as described shortly. 5 PROPERLY DRAWN SINGLE IMPUTATIONS—THEORY WITH MONOTONE MISSINGNESS A missing data pattern is monotone if the rows and columns of the data matrix can be sorted in such a way that an irregular staircase separates Y obs and Y mis . Figures 1 and 2 illustrate monotone missing data patterns. Let Y 0 represent fully observed variables, Y 1 the incompletely observed variable with the fewest missing values, Y 2 the variable with the second fewest missing values, and so on. Proper imputation with a monotone missing data pattern begins by fitting an appropriate model to predict Y 1 from Y 0 and then using this model to impute the missing values in Y 1 . For example, fit a regression of Y 1 on Y 0 using the units with Y 1 observed, draw the regression parameters from their posterior distribution, and then draw the missing values of Y 1 given these parameters and Y 0 . Next impute the missing values for Y 2 using Y 0 and the observed and imputed values of

X

Y(1)

X

Y(0)

? ? Treated

Control

Figure 1. Pattern of missing data for intergel trial.

IMPUTATION X

Y(1)

X

Y(0)

? Treated

Control

Figure 2. Pattern of missing data for genzyme trial.

Y 1 . Continue in this manner until all missing values have been imputed. The collection of imputed values is a proper imputation of the missing data, Y mis , and the collection of univariate prediction models is the implied full imputation model. When missing data are not monotone, this method of imputation as described cannot be used. 6 PROPERLY DRAWN SINGLE IMPUTATIONS—THEORY WITH NONMONOTONE MISSINGNESS Creating imputations when the missing data pattern is nonmonotone generally involves iteration because the distribution p(Ymis |Yobs ) is often difficult to draw from directly. However, the data augmentation algorithm [DA; Tanner and Wong (4)] is often straightforward to implement. Briefly, DA involves iterating between sampling missing data given a current draw of the model parameters and sampling model parameters given a current draw of the missing data. The draws of Y mis form a Markov Chain whose stationary distribution is p(Ymis |Yobs ). Thus, once the Markov Chain has reached approximate convergence, a draw of Y mis from DA is effectively a proper single imputation of the missing data from the target distribution p(Ymis |Yobs ). Many programs discussed in Section (9) use DA to impute missing values. Other algorithms that use Markov Chain Monte Carlo methods for imputing missing values include Gibbs sampling and Metropolis–Hastings. See Gelman et al. (5, Chapter 11) for more details. As discussed, analyzing a singly imputed dataset using complete-data methods usually leads to anticonservative results because

3

imputed values are treated as if they were known, thereby underestimating uncertainty. Multiple imputation corrects this problem, while retaining the advantages of single imputation. 7

MULTIPLE IMPUTATION

Described in detail in Rubin (6), multiple imputation (MI) is a Monte Carlo technique that replaces the missing values Y mis with m > 1 plausible values, {Y mis,1 , . . . ,Y mis,m }, and therefore reveals uncertainty in the imputed values. Each set of imputations creates a completed dataset, thereby creating m completed datasets: Y (1) = {Y obs , Y mis,1 }, . . ,. Y (l) = {Y obs , Y mis,l }, . . ,.Y (m) = {Y obs , Y mis,m }. Typically m is fairly small: m = 5 is a standard number of imputations to use. Each m completed dataset is then analyzed as if there were no missing data and the results combined using simple rules described shortly. Obtaining proper MIs is no more difficult than obtaining a single proper imputation—the process for obtaining a proper single imputation is simply repeated independently m times. 8 COMBINING RULES FOR PROPER MULTIPLE IMPUTATIONS As in Rubin (6) and Schafer (7), let Q represent the estimated of interest (e.g., the mean of a variable, a relative risk, the intentionto-treat effect, etc.), let Qest represent the complete-data estimator of Q (i.e., the quantity calculated treating all imputed values of Y mis as known observed data), and let U represent the estimated variance of Qest − Q. Let Qest,l be the estimate of Q based on the lth imputation of Y mis with associated variance U l ; that is, the estimate of Q and associated variance are based on the complete-data analysis of the lth completed dataset, Y l = {Y obs , Y mis,l }. The MI estimate of Q is simply  the average of the m estimates: QMIest = m l=1 Qest,l /m . The estimated variance of QMIest − Q is found by combining between and within imputation variance, as with the analysis of variance: T = U ave + (1 + m−1 )B, where Uave =  m l=1 Ul /m is the within imputation variance

4

IMPUTATION

 2 and B = m l=1 (Qest,l − QMIest ) /(m − 1) is the between imputation variance. The quantity T −1/2 (Q − QMIest ) follows an approximate tν distribution with degrees of freedom ν = (m − 1)(1 + U ave /((1 + m−1 )B))2 . See Rubin and Schenker (8) for additional methods for combining vector-valued estimates, significance levels, and likelihood ratio statistics and Barnard and Rubin (9) for an improved expression for ν with small complete data sets. 9 COMPUTATION FOR MULTIPLE IMPUTATION Many standard statistical software packages now have built-in or add-on functions for MI. The S-plus libraries NORM, CAT, MIX, and PAN, for analyzing continuous, categorical, mixed, and panel data, respectively, are freely available [(7), http://www.stat.psu.edu /∼jls/], as is MICE [(10), http://web.inter.nl. net/users/S.van.Buuren/mi/html/mice.htm], which uses regression models to impute all types of data. SAS now has procedures PROC MI and PROC MIANALYZE; in addition IVEwear (11) is freely available and can be called using SAS (http://support.sas.com/rnd/ app/da/new/dami.html). New software packages have also been developed specifically for multiple imputation, for example, the commercially available SOLAS (http://www.statsol.ie/solas/solas.htm), which has been available for years and is most appropriate for datasets with a monotone or nearly monotone pattern of missing data, and the freely available NORM, a standalone Windows version of the S-plus function NORM (http://www.stat. psu.edu/∼jls/). For more information, see www.multiple-imputation.com or Horton and Lipsitz (12). 10

EXAMPLE: LIFECORE

Intergel is a solution developed by Lifecore Corporation to prevent surgical gynecological adhesions. A double-blind, multicenter randomized trial was designed for the U.S. Food and Drug Administration (FDA) to determine whether Intergel significantly reduces the formation of adhesions after a first surgery. The data collection procedure for this study

was fairly intrusive: Patients had to undergo a minor abdominal surgery weeks after the first surgery in order for doctors to count the number of gynecological adhesions. The trial, therefore, suffered from missing data because not all women were willing to have another surgery, despite having initially agreed to do so. The original proposal from FDA for imputing the missing values was to fill in the worst possible value (defined to be 32 adhesions—most patients with observed data had 10 or fewer adhesions) for each missing datum, which should lead to ‘‘conservative’’ results because there were more missing data in the treatment arm than in the placebo arm. This method ignores observed data when creating imputations; for example, one woman in the treatment group refused the second look surgery because she was pregnant, which would have been impossible with more than a few gynecological adhesions. Furthermore, because the imputed values are so much larger than the observed values, the standard errors based on these worst-possible value imputations were inflated, making it unlikely to be able to get significant results even when the two treatments were significantly different. Figure 1 displays the general pattern of monotone missing data in this case, with X representing covariates, Y(0) outcomes under placebo, and Y(1) outcomes under Intergel. The question marks represent missing values. Colton et al. (13) instead used an MI hot deck procedure to impute missing values. Donor pools were created within cells defined by treatment group, treatment center, and baseline seriousness of adhesions, which were observed for all patients: For each patient whose outcome was missing, the donor pool consisted of the two patients in the same treatment group and treatment center who had the closest baseline adhesion scores. Each value in the donor pool was used as an imputation. Formally this method is improper, but the limited donor pools should still make the method conservative because the matches are not as close as they would be with bigger sample sizes.

IMPUTATION

11

EXAMPLE: GENZYME

Fabrazyme is a drug developed by Genzyme Corporation to treat Fabry’s disease, a rare and serious genetic disease caused by an inability to metabolize creatinine. Preliminary results from a Phase 3 FDA trial of Fabrazyme versus placebo showed that the drug appeared to work well in patients in their 30s, who were not yet severely ill, in the sense that it lowered their serum creatinine substantially. A similar Phase 4 trial involved older patients who were more seriously ill. As there is no other fully competitive drug, it was desired to make Fabrazyme commercially available early, which would allow patients randomized to placebo to begin taking Fabrazyme, but would create missing outcome data among placebo patients after they began taking Fabrazyme. The study had staggered enrollment, so that the number of monthly observations of serum creatinine for each placebo patient depended on his time of entry into the study. Figure 2 illustrates the general pattern of monotone missing data with the same length follow-up for each patient. Again, X represents baseline covariates, Y(0) represents repeated measures of serum creatinine for placebo patients, and Y(1) represents repeated measures of serum creatinine for Fabrazyme patients. To impute the missing outcomes under placebo, a complex hierarchical Bayesian model was developed for the progression of serum creatinine in untreated Fabry patients. In this model, inverse serum creatinine varies linearly and quadratically in time, and the prior distribution for the quadratic trend in placebo patients is obtained from the posterior distribution of the quadratic trend in an analogous model fit to a historical database of untreated Fabry patients. Thus, the historical patients’ data only influence the imputations of the placebo patients’ data subtly—via the prior distribution on the quadratic trend parameters. Although the model fitting algorithm is complex, it is straightforward to use the algorithm to obtain draws from p(θ |Yobs ) for the placebo patients, and then draw Y mis conditional on the drawn value of θ , where θ represents all model parameters. Drawing

5

the missing values in this way creates a sample from p(Y mis |Y obs ) and thus an imputation for the missing values in the placebo group. The primary analysis will consider the time to an event, defined as either a clinical event (e.g., kidney dialysis, stroke) or a substantial increase in serum creatinine relative to baseline. The analysis will be conducted on each imputed dataset and the results will be combined (as outlined earlier) to form a single inference. 12

EXAMPLE: NMES

The National Medical Expenditure Survey (NMES) collects data on medical costs from a random sample of the U.S. population. The data include medical expenditures, background information, and demographic information. Multiple imputation for NMES was more complicated than in the previous two examples because the missing data pattern was not monotone. Figure 3 shows a simplification of the missing data pattern for NMES, where if Y 1 was fully observed, the missing data pattern would be monotone. Rubin (14) imputed the missing data in NMES by capitalizing on the simplicity of imputation for monotone missing data by first imputing the missing values that destroyed the monotone pattern (the ‘‘nonmonotone missing values’’) and then proceeding as if the missing data pattern was in fact monotone, and then iterating this process. More specifically, after choosing starting values for the missing data, iterate between the following two steps: (1) Regress each variable with any nonmonotone missing values (i.e., Y 1 ), on all other variables (i.e., Y 0 ,Y 2 ,Y 3 ), treating the current imputations as true values, but use this regression to impute only the

Y0 Y1 Y2 Y3 ?

?

? Figure 3. Illustrative display for type of pattern of missing data in NMES.

6

IMPUTATION

nonmonotone missing values. (2) Impute the remaining missing values in the monotone pattern; first impute the variable with the fewest missing values (Y 2 in Fig. 3), then the variable with the second fewest missing values (Y 3 in Fig. 3), and so on, treating the nonmonotone missing values filled in step 1 as known. This process was repeated five times to create five sets of imputations in the NMES example. 13

SUMMARY

Multiple imputation is a flexible tool for handling incomplete datasets. MIs are often straightforward to create using computational procedures such as DA or using special MI software now widely available. Moreover, the results from singly imputed datasets are easy to combine into a single MI inference. Although MI is Bayesianly motivated, many MI procedures have been shown to have excellent frequentist properties (15). In small samples, the impact of the prior distribution on conclusions can be assessed by creating MIs using several different prior specifications. Furthermore, although only MAR procedures have been considered here, missing data arising from an NMAR mechanism may be multiply imputed by jointly modeling the data and the missingness mechanism; in some cases, results are insensitive to reasonable missingness models and the missing data can then be effectively treated as being MAR (6). Rubin (6), Schafer (7), and Little and Rubin (3) are excellent sources for more detail on the ideas presented here, the last two being less technical and more accessible than the first. REFERENCES 1. D. B. Rubin, Multiple imputations in sample surveys: A phenomenological Bayesian approach to nonresponse (with discussion). ASA Proc. Section on Survey Research Methods, 1978: 20–34. 2. D. B. Rubin, Inference and missing data. Biometrika 1976; 63: 581–590. 3. R. J. A. Little, and D. B. Rubin, Statistical Analysis with Missing Data. New York: Wiley, 2002.

4. M. A. Tanner, and W. H. Wong, The calculation of posterior distributions by data augmentation (with discussion). J. Amer. Stat. Assoc. 1987; 82: 528–550. 5. A. Gelman, J. B. Carlin, H. S. Stern, and D. B. Rubin, Bayesian Data Analysis. London: Chapman & Hall, 1995. 6. D. B. Rubin, Multiple Imputation for Nonresponse in Surveys. New York: Wiley, 1987. 7. J. L. Schafer, Analysis of Incomplete Multivariate Data. London: Chapman & Hall, 1997. 8. D. B. Rubin, and N. Schenker, Multiple imputation in health-care databases: An overview and some applications. Stat. Med. 1991; 10: 585–598. 9. J. Barnard, and D. B. Rubin, Small-sample degrees of freedom with multiple imputation. Biometrika 1999; 86: 948–955. 10. S. van Buuren, H. C. Boshuizen, and D. L. Knook, Multiple imputation of missing blood pressure covariates in survival analysis. Stat. Med. 1999; 18: 681–694. 11. T. E. Raghunathan, J. M. Lepkowski, J. van Hoewyk, and P. Solenberger, A multivariate technique for multiply imputing missing values using a sequence of regression models. Survey Methodol. 2001; 27: 85–95. 12. N. J. Horton, and S. R. Lipsitz, Multiple imputation in practice: Comparison of software packages for regression models with missing variables. Amer. Statistician 2001; 55: 244–254. 13. T. Colton, S. Piantadosi, and D. B. Rubin, Multiple imputation for second-look variables based on Intergel pivotal trial data. Report submitted to FDA. 2001. 14. D. B. Rubin, Nested multiple imputation of NMES via partially incompatible MCMC. Statistica Neerlandica 2003; 57: 3–18. 15. D. B. Rubin, Multiple imputation after 18 + years (with discussion). J. Amer. Stat. Assoc. 1996; 91: 473–520.

INCOMPETENT SUBJECTS AND PROXY CONSENT

make a will), but this person is entirely competent to make a medical treatment decision because she understands the risks, benefits, and alternatives of the proposed medical procedure. Children are presumed to be incompetent, and important decisions, such as medical treatment decisions, are generally made on their behalf by their parents.

LEONARD H. GLANTZ Boston University School of Public Health Boston, Massachusetts

Research with subjects who are incompetent presents the most serious ethical challenges than any other category of research. By definition, incompetent research subjects cannot consent to their participation in research, and consent is an essential element of the protection of the rights of human subjects. The Nuremberg Code proclaims that the informed and voluntary consent of the human subject is ‘‘absolutely essential.’’ However, requiring consent of every human subject would preclude research on certain classes of people. 1

2 CONSIDERATIONS IN ENROLLING INCOMPETENT SUBJECTS The use of incompetent subjects in research is inherently suspect because they cannot consent voluntarily to participation, which is generally regarded as an essential ethical and legal condition for research participation. To avoid this problem, incompetent subjects should not be included in a research project unless it is impossible to conduct the research with competent subjects. For example, there is no reason to conduct research on a new antibiotic that will be used in the general population on incompetent subjects. However, research to test the efficacy of a treatment for the alleviation of serious dementia or developing a treatment for premature newborns could not use competent subjects. Even where good scientific reasons are provided for using incompetent subjects, serious ethical challenges remain. One can think of research as existing on a spectrum from pure research, in which the sole goal of the research is the creation of knowledge, to the last stages of clinical research, in which substantial data support the conclusion that the use of the test material holds out a realistic prospect of direct benefit to the subject. For example, substantial ethical differences exist between exposing an incompetent subject to the early toxicity testing of drug in which subjects are used to determine the negative effects of different doses, and administering that drug at the end of the testing process when it is designed to determine whether subjects will benefit from the use of the drug. Although research and treatment are never

DEFINITION OF INCOMPETENCE

Both legal and ethical principles presume that every adult is competent to make important decisions about themselves. This determination is essential to maximize the autonomy of individuals and enable them to act on what they believe is in their own best interests. This presumption is based on the belief that adults can weigh the benefits, risks, and alternatives to proposed actions and decisions. However, once it is determined that adults cannot weigh these considerations, they are deemed to be incompetent and are deprived of their decision-making capacity. Adults may be incompetent as a result of serious dementia, serous developmental disabilities, certain types of psychosis (not all psychotic people are incompetent), and the influence of mind-altering substances (once the influence of such substances disappear, the person will again be competent). Incompetence is determined by establishing the inability to understand the risks, benefits, and alternatives of particular activities. For example, a person may be incompetent to make a will because a mental illness makes them incapable of recognizing that they have children (a basic requirement to competently

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

INCOMPETENT SUBJECTS AND PROXY CONSENT

the same thing, the closer the research resembles treatment the more permissible the use of incompetent subjects becomes. Although it is important for researchers to consider their own ethical obligations not to exploit the decisional incapacity of subjects, important considerations must be reviewed for surrogate decision makers who permit the use of their wards in research programs. Both parents and legal guardians are supposed to act in ways that protect their children or wards from unnecessary harm. Thus, some ethicists believe surrogate decision makers may never permit their wards to be subjects in research that presents risk of harm with no corresponding benefits. Other ethicists believe that surrogates may give such permission if the risks are trivial and benefits to knowledge are potentially important. For these ethicists, the issue is often how to define a trivial risk. Noninvasive interactions that have no risk for research subjects, such as taking hair clippings or weighing children, are not problematic for either group of ethicists. But drawing blood, conducting skin biopsies, or taking spinal fluid, although all perfectly routine in the clinical setting, are more problematic in the research setting in which the subject can receive no benefit from the physical intrusions. It is also important to recognize that investigators and surrogates are obligated to protect incompetent subjects from fear, distress, and discomforts. For example, whereas an MRI procedure may present no physical risks, many people find the noise and claustrophobia that are inherent in the process to be frightening. Taking a demented person from their home and subjecting them to an MRI can cause substantial fear and distress that may not be ethically justified. 3

SUBJECT ASSENT

Even in the instance where a person is incompetent to give consent, it is important to try to obtain an incompetent subjects assent to a procedure. Even where subjects cannot understand the reasons for a procedure, they may understand the procedure itself. For example, a person might not understand why a doctor wants to take blood from them, but

they can understand that it involves putting a needle in their arm. Although no benefit is provided to the person for blood withdrawal, it is widely accepted that the person’s permission for the intrusion should be obtained. The Federal research regulations require this permission for research in children when the Institutional Review Board (IRB) determines that the children in the study can give assent, and the research holds out no prospect of direct benefit to the child. 4 SUBJECT DISSENT For informed consent to be valid, it must be given by a competent person. However, an incompetent person can express dissent. Using the MRI procedure discussed above as an example, if the demented person seems to object to being placed in the device, either verbally or through her actions, it is widely accepted that such dissent should be respected. As a matter of ethics and fairness, there is a significant difference between performing a procedure on someone who does not consent but is compliant, and a subject who objects who is forcibly intruded on. 5 RESEARCH ADVANCED DIRECTIVES Because of the ethical issues presented by research with incompetent subjects, it has been suggested that potentially incompetent research subjects give advanced consent to being a research subject for specified classes of research. This consent is similar to the use of ‘‘living wills’’ or advanced directives that people execute to determine their medical care should they become incompetent to make medical decisions in the future. Research advance directives might prove to be useful where the condition being studied is progressive and will likely lead to incompetence in the future. For example, a person with the genetic markers for Huntington Disease might be willing to enroll in longitudinal research on the condition and may execute an advance directive that permits researchers to conduct physically intrusive procedures, such as spinal taps, when she becomes incompetent. As a result, the research that is conducted on this person when he/she becomes

INCOMPETENT SUBJECTS AND PROXY CONSENT

incompetent will be based on his/her consent, not the consent of a surrogate. Researchers have little experience with such documents, so one cannot know their effectiveness in addressing the difficult ethical questions presented by research on incompetent subjects. For example, if a person who has executed a research advanced directive dissents to participation in a procedure when he/she is incompetent, should the previous executed document or the current dissent govern? REFERENCES FURTHER READING G. J. Annas and L. H. Glantz, Rules for research in nursing homes. New Engl. J. Med. 1986; 315: 1157–1158. R. Dresser, Research involving persons with mental disabilities: a review of policy issues. In National Bioethics Advisory Commission, Research Involving Persons with Mental Disorders that May Affect Decisionmaking Capacity, Vol. II 1999, pp. 5–28. Available: http://bioethics.georgetown.edu/nbac/ pubs.html. M. J. Field and R. E. Behrman, eds. Ethical Conduct of Clinical Research Involving Children. Washington, DC: National Academy Press, 2004. L. H. Glantz, Nontherapeutic research with children; Grimes v. Kennedy Krieger Institute. Am. J. Public Health 2002; 92: 4–7. M. A. Grodin and L. H. Glantz, eds. Children as Research Subjects: Science, Ethics and Law. New York: Oxford U. Press, 1994. National Bioethics Advisory Commission, Research Involving Persons with Mental Disorders that May Affect Decisionmaking Capacity, Vol. 1, 1999. The Department of Health and Human Services, Protection of Human Subjects, Title 45 Code of Federal Regulations part 46 Subpart D, 1991.

CROSS-REFERENCES Informed Consent Process, Forms, and Assent

3

INDEPENDENT ETHICS COMMITTEE (IEC) An Independent Ethics Committee (IEC) is an independent body (a review board or a committee, institutional, regional, national, or supranational) that is composed of medical/scientific professionals and nonmedical/ nonscientific members. Their responsibility is to ensure the protection of the rights, safety, and well-being of human subjects involved in a trial and to provide public assurance of that protection, by, among other things, reviewing and approving/providing favorable opinion on the trial protocol, the suitability of the investigator(s), the facilities, and the methods and material to be used in obtaining and documenting informed consent of the trial subjects. The legal status, composition, function, operations, and regulatory requirements that pertain to Independent Ethics Committees may differ among countries, but they should allow the Independent Ethics Committee to act in agreement with Good Clinical Practice (GCP) as described in this guideline.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) by Ralph D’Agostino and Sarah Karl.

1

inferences∗ and superpopulation inferences∗ to the extent that the validity of the claimed generality is model dependent∗ via its sensitivity to model misspecifications∗ . Also, it is possible to regard Bayesian inferences and superpopulation inferences as providing some unity to the role of design-based and model-based considerations. Thus a focus of distinction here between design-based and model-based inference is the population to which results are generalized rather than the nature of statistical methods. Models can be useful conceptually in either context; also they can shed light on the robustness∗ of inferences to their underlying assumptions. The related issue of external validity includes substantive justification for the area of application and statistical evaluation of the plausibility of model assumptions. For other pertinent discussion, see Deming [10, Chap. 7], Fisher [12], Godambe and Sprott [15], Johnson and Smith [18], Kempthorne and Folks [23, Chap. 17], Namboodiri [29], and INFERENCE, STATISTICAL. The distinctions between design-based inference and model-based inference can be expressed in clearest terms for comparative experimental studies (e.g., multicenter clinical trials∗ ). Typically, these involve a set of blocks (or sites) which are selected on a judgmental basis. Similarly, the experimental units may be included according to convenience or availability. Thus, these subjects constitute a fixed set of finite local study populations. When they are randomly assigned to two or more treatment groups, corresponding samples are obtained for the potential responses of all subjects under study for each of the respective treatments. By virtue of the research design, randomization model methods (e.g., Kruskal-Wallis tests∗ ) in CHI-SQUARE TEST —I can be used to obtain design-based inferences concerning treatment comparisons without any external assumptions. Illustrative examples are given in CHI-SQUARE TEST —I, and LOG-RANK SCORES. A limitation of design-based inferences for experimental studies is that formal conclusions are restricted to the finite population of subjects that actually received treatment.

INFERENCE, DESIGN-BASED VS. MODEL-BASED Design-based inference and model-based inference are alternative conceptual frameworks for addressing statistical questions from many types of investigations. These include: 1. Experimental studies of randomly allocated subjects 2. Historical (observational) and followup∗ studies of all subjects in a fortuitous, judgmental, or natural population 3. Sample surveys∗ of randomly selected subjects For these situations and others, there is interest in the extent of generality to which conclusions are expressed and the rationale by which they are justified. Some of the underlying inference issues can be clarified by directing attention at the sampling processes for data collection and the assumptions necessary for the data plausibly to represent a defined target population. A statistical analysis whose only assumptions are random selection of observational units or random allocation of units to experimental conditions may be said to generate design-based inferences; i.e., design-based inferences are equivalent to randomization∗ based inferences as discussed by Kempthorne [20–22], Kish [24, Chap. 14], Lehmann [27, pp. 55–57], and others. Also, such inferences are often said to have internal validity∗ (see Campbell and Stanley [3]) when the design is adequate to eliminate alternative explanations for the observed effects other than the one of interest. In this sense, internal validity requires only that the sampled population∗ and the target population∗ be the same. Alternatively, if assumptions external to the study design are required to extend inferences to the target population, then statistical analyses based on postulated probability distributional forms (e.g., binomial, normal, Poisson, Weibull, etc.) or other stochastic processes yield model-based inferences. These can be viewed as encompassing Bayesian

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

INFERENCE, DESIGN-BASED VS. MODEL-BASED

For agricultural crop studies and laboratory animal studies undertaken at local facilities, such issues merit recognition in a strict sense. However, for medical clinical trials∗ undertaken by multiple investigators at geographically diverse locations, it often may be plausible to view the randomized patients as conceptually representative of those with similar characteristics in some large target population of potential patients. In this regard, if sites and subjects had been selected at random from larger eligible sets, then models with random effects provide one possible way of addressing both internal and external validity considerations. However, such an approach may be questionable if investigators and/or patients were not chosen by a probability sampling mechanism. In this more common situation, one important consideration for confirming external validity is that sample coverage include all relevant subpopulations; another is that treatment differences be homogeneous across subpopulations. More formally, probability statements are usually obtained via assumptions that the data are equivalent to a stratified simple random sample from the partition of this population into homogeneous groups according to an appropriate set of explanatory variables. This stratification is necessary because the patients included in a study may overrepresent certain types and underrepresent others, even though those of each of the respective types might be representative of the corresponding target subpopulations. For categorical (or discrete) response measures, the conceptual sampling process described here implies the product multinomial distribution. As a result, model-based inferences concerning treatment comparisons and their interactions with the explanatory variable stratification can be obtained by using maximum likelihood or related methods as discussed in CHI-SQUARE TEST —I and LOGLINEAR MODELS IN CONTINGENCY TABLES. Illustrative examples are given in LOGISTIC REGRESSION. In a similar spirit, least-squares methods can be used for model-based inferences when continuous response variables have approximately normal distributions with common variance within the respective strata; and analogous procedures are applicable to other distributional structures (e.g.,

see Cox [9], McCullagh [28], and Nelder and Wedderburn [30]. The principal advantages of model-based inferences for such situations are their more general scope and the comprehensive information they provide concerning relationships of response measures to treatment and stratification variables. Contrarily, their principal limitation is that subjects in a study may not represent any meaningful population beyond themselves. See Fisher [13], Kempthorne [22], Neyman et al. [31], and Simon [37] for further discussion. For historical (observational) studies, model-based inferences are usually emphasized because the target population is more extensive than the fortuitous, judgmental, or naturally defined group of subjects included. Also, their designs do not involve either random allocation or random selection, as illustrated by the following examples: 1. A study of driver injury relative to vehicle size, vehicle age, and vehicle model year for all police-reported automobile accidents in North Carolina during 1966 or 1968–1972 (see Koch et al. [26]) 2. A nonrandomized prospective study to compare the experience of patients receiving a new treatment with that of a historical control population (see Koch et al. [26]) 3. A nonrandomized study to compare nine treatments for mastitis in dairy cows relative to their pretreatment status (see CHI-SQUARED TESTS —II). 4. Market research studies involving quota sampling∗ as opposed to random selection (see Kalton [19]) The assumptions by which the subjects are considered representative of the target population and the methods used for analysis are similar to those previously described for experimental studies. Otherwise, designbased inferences are feasible for historical studies through tests of randomization as a hypothesis in its own right, but their use should be undertaken cautiously; specific illustrations are given in CHI-SQUARE TEST —I, and Koch et al. [26]. More extensive discussion of various aspects of inference for observational studies appears in Anderson

INFERENCE, DESIGN-BASED VS. MODEL-BASED

et al. [1], Breslow and Day [2], Cochran [8], Fairley and Mosteller [11], and Kleinbaum et al. [25]. Design-based inferences are often emphasized for sample surveys because the target population is usually the same as that from which subjects have been randomly selected. They are obtained by the analysis of estimates for population averages or ratios and their estimated covariance matrix which are constructed by means of finite population sampling methodology. An illustrative example is given in CHI-SQUARE TEST —I. For sample surveys, the probabilistic interpretation of design-based inferences such as confidence intervals is in reference to repeated selection from the finite population via the given design. In constrast, model-based inferences are obtained from a framework for which the target population is a superpopulation with assumptions characterizing the actual finite population as one realization; and so their probabilistic interpretation is in reference to repetitions of the nature of this postulated sampling process. The latter approach can be useful for situations where the subjects in a sample survey are not necessarily from the target population of interest. For example, Clarke et al. [6] discuss the evaluation of several pretrial release programs for a stratified random sample of 861 defendants in a population of 2,578 corresponding to January-March 1973 in Charlotte, North Carolina. Since the entire population here is a historical sample, any sample of it is also necessarily a historical sample. Thus issues of model-based inference as described for historical studies would be applicable. Another type of example involves prediction to a date later than that at which the survey was undertaken; e.g., Cassel et al. [5] studied prediction of the future use of a bridge to be constructed in terms of number of vehicles. Otherwise, it can be noted that statistical methods for designbased inferences often are motivated by a linear model; e.g., a rationale for ratio estimates involves regression through the origin. A more general formulation for which a linear model underlies the estimator and its esti¨ mated variance is given in Sarndal [35,36]. Additional discussion concerning aspects of design-based or model-based approaches to

3

sample survey data or their combination is given in Cassel et al. [4], Cochran [7], Fuller [14], Hansen et al. [16], Hartley and Sielken [17], Royall [32], Royall and Cum¨ berland [33], Sarndal [34], Smith [38], and LABELS. The distinction between design-based inference and model-based inference may not be as clear cut as the previous discussion might have suggested. For example, some type of assumption is usually necessary in order to deal with missing data; and stratification undertaken purely for convenient study management purposes (rather than statistical efficiency) is sometimes ignored. Also, a model-based approach may be advantageous for estimation for subgroups with small sample sizes (i.e., small domain estimation; see Kalton [19]). For these and other related situations, the issue of concern is the robustness∗ of inferences to assumptions. In summary, design-based inferences involve substantially weaker assumptions than do model-based inferences. For this reason, they can provide an appropriate framework for policy-oriented purposes in an adversarial setting (e.g., legal evidence). A limitation of design-based inferences is that their scope might not be general enough to encompass questions of public or scientific interest for reasons of economy or feasibility. Of course, this should be recognized as inherent to the design itself (or the quality of its implementation) rather than the rationale for inference. In such cases, model-based inferences may provide relevant information given that the necessary assumptions can be justified. It follows that design-based inference and model-based inference need not be seen as competing conceptual frameworks; either they can be interpreted as directed at different target populations and thereby at different statistical questions (e.g., experimental studies), or their synthesis is important to dealing effectively with the target population of interest (e.g., sample surveys).

4

INFERENCE, DESIGN-BASED VS. MODEL-BASED

Acknowledgments The authors would like to thank Wayne Fuller, Peter Imrey, Graham Kalton, Oscar Kempthorne, ¨ Jim Lepkowski, Carl Sarndal, and Richard Simon for helpful comments relative to the preparation of this entry. It should be noted that they may not share the views expressed here. This research was partially supported by the U.S. Bureau of the Census through Joint Statistical Agreement JSA80-19; but this does not imply any endorsement.

REFERENCES 1. Anderson, S., Auquier, A., Hauck, W. W., Oakes, D., Vandaele, W., and Weisberg, H. I. (1980). Statistical Methods for Comparative Studies. Wiley, New York. 2. Breslow, N. E. and Day, N. E. (1980). Statistical Methods in Cancer Research, 1: The Analysis of Case Control Studies. International Agency for Research on Cancer, Lyon. 3. Campbell, D. T. and Stanley, J. C. (1963). Handbook on Research on Teaching, Rand McNally, Chicago, pp. 171–246. (Experimental and quasi-experimental designs for research on teaching.) ¨ 4. Cassel, C. M., Sarndal, C. E., and Wretman, J. H. (1977). Foundations of Inference in Survey Sampling. Wiley, New York. ¨ 5. Cassel, C. M., Sarndal, C. E., and Wretman, J. H. (1979). Scand. J. Statist., 6, 97–106. (Prediction theory for finite populations when model-based and design-based principles are combined.) 6. Clarke, S. H., Freeman, J. L., and Koch, G. G. (1976). J. Legal Stud., 5(2), 341–385. (Bail risk: a multivariate analysis.) 7. Cochran, W. G. (1946). Ann. Math. Statist., 17, 164–177. (Relative accuracy of systematic and stratified random samples for a certain class of populations.) 8. Cochran, W. G. (1972). Statistical Papers in Honor of George W. Snedecor, T. A. Bancroft, ed. Iowa State University Press, Ames, Iowa, pp. 77–90. 9. Cox, D. R. (1972). J. R. Statist. Soc. B, 34, 187–220. [Regression models and life tables (with discussion).] 10. Deming, W. E. (1950). Some Theory of Sampling. Wiley, New York. 11. Fairley, W. B. and Mosteller, F. (1977). Statistics and Public Policy. Addison-Wesley, Reading, Mass.

12. Fisher, R. A. (1925). Proc. Camb. Philos. Soc., 22, 700–725. (Theory of statistical estimation.) 13. Fisher, R. A. (1935). The Design of Experiments. Oliver & Boyd, Edinburgh (rev. ed., 1960). 14. Fuller, W. A. (1975). Sankhya C, 37, 117–132. (Regression analysis for sample survey.) 15. Godambe, V. P. and Sprott, D. A. (1971). Foundations of Statistical Inference. Holt, Rinehart and Winston, Toronto. 16. Hansen, M. H., Madow, W. G., and Tepping, B. J. (1978). Proc. Survey Res. Meth. Sec., Amer. Statist. Ass., pp. 82–107. [On inference and estimation from sample surveys (with discussion).] 17. Hartley, H. O. and Sielken, R. L. (1975). Biometrics, 31, 411–422. (A ‘‘super-population viewpoint’’ for finite population sampling.) 18. Johnson, N. L. and Smith, H., eds. (1969). New Developments in Survey Sampling. Wiley, New York. 19. Kalton, G. (1983). Bull. Int. Statist. Inst. (Models in the practice of survey sampling.) 20. Kempthorne, O. (1952). The Design and Analysis of Experiments. Wiley, New York. 21. Kempthorne, O. (1955). J. Amer. Statist. Ass. 50, 946–967. (The randomization theory of experimental inference.) 22. Kempthorne, O. (1979). Sankhya B, 40, 115–145. (Sampling inference, experimental inference, and observation inference.) 23. Kempthorne, O. and Folks, L. (1971). Probability, sity Press, Ames, Iowa. 24. Kish, L. (1965). Survey Sampling. Wiley, New York. 25. Kleinbaum, D. G., Kupper, L. L., and Morgenstern, H. (1982). Epidemiologic Research: Principles and Quantitative Methods. Lifetime Learning Publication, Belmont, Calif. 26. Koch, G. G., Gillings, D. B., and Stokes, M. E. (1980). Annu. Rev. Public Health, 1, 163–225. (Biostatistical implications of design, sampling, and measurement to health science data analysis.) 27. Lehmann, E. L. (1975). Nonparametrics: Statistical Methods Based on Ranks. Holden-Day, San Francisco. 28. McCullagh, P. (1980). J. R. Statist. Soc. B, 42, 109–142. (Regression models for ordinal data.) 29. Namboodiri, N. K. (1978). Survey Sampling and Measurement. Academic Press, New York. 30. Nelder, J. A. and Wedderburn, R. W. M. (1972). J. R. Statist. Soc. A, 135, 370–384.

INFERENCE, DESIGN-BASED VS. MODEL-BASED (Generalized linear models.) 31. Neyman, J., Iwaskiewicz, K., and Kolodziejczyk, S. (1935). J. R. Statist. Soc. (Suppl. 1), 2, 107–154. (Statistical problems in agricultural experimentation.) 32. Royall, R. M. (1976). Amer. J. Epidemiol., 104, 463–473. (Current advances in sampling theory: implications for human observational studies.) 33. Royall, R. M. and Cumberland, W. G. (1981). J. Amer. Statist. Ass., 76, 66–77. (An empirical study of the ratio estimator and estimators of its variance.) ¨ 34. Sarndal, C. E. (1978). Scand. J. Statist., 5, 27–52. (Design-based and model-based inference in survey sampling.) ¨ 35. Sarndal, C. E. (1980). Biometrika, 67, 639–650. (On π -inverse weighting vs. best linear unbiased weighting in probability sampling.) ¨ 36. Sarndal, C. E. (1982). J. Statist. Plann. Infer., 7, 155–170. 37. Simon, R. (1979). Biometrics, 35, 503–512. (Restricted randomization designs in clinical trials.) 38. Smith, T. M. F. (1976). J. R. Statist. Soc. A, 139, 183–204. [The foundations of survey sampling: a review (with discussion).]

GARY G. KOCH DENNIS B. GILLINGS

5

INFORMED CONSENT PROCESS, FORMS, AND ASSENT

1

THE PURPOSE FOR INFORMED CONSENT

The term informed consent would seem to be redundant. One may well ask how one can give consent to an undertaking about which they are not informed. The term is a response to the reality that before the introduction of the doctrine of informed consent patients, often gave their consent to surgeries or other medical interventions without being provided with information about the benefits, risks, and alternatives of an intervention a physician proposed to perform. In one of the earliest decisions that mandated physicians to obtain the informed consent of patients before performing surgery, the court explained that the purpose of the doctrine is ‘‘to enable the patient’s chart this course knowledgeably . . . ’’ (1) and, it found that to do so, the patient needed to have reasonable familiarity with the therapeutic alternatives to the proposed procedure and the inherent risks of the procedure and the alternatives. The court also explained that because the physician is an expert he has knowledge of the risks inherent in the proposed procedure and the probability of success. But that the weighing of the risks is not an expert skill, but a ‘‘nonmedical judgment reserved to the patient alone.’’ Similarly, the decision whether to become a research subject lies entirely in the hands of the potential subject, and without adequate information, which the potential subject can obtain only from the researcher, the subject’s consent would be uninformed. It is this reality that forms the foundation for both the ethical and legal obligation of the researcher to obtain a potential subject’s informed consent.

LEONARD H. GLANTZ Boston University School of Public Health, Boston, Massachusetts

Obtaining the informed consent of human subjects is an integral part of the overall obligations and duties inherent in the researcher role. Researchers are obliged to protect both the rights and the welfare of research subjects. The welfare of research subjects is protected by using the safest and least invasive methods possible to obtain the necessary information from subjects. For example, if the research involves obtaining blood from a subject, then the researchers should use the fewest needle sticks possible. Protecting the rights of research subjects requires empowering potential subjects to decide freely and intelligently whether they wish to become research subjects and to decide whether they wish to continue to be subjects once they have been enrolled. In this regard, the doctrine of informed consent was created to protect and enhance the potential subjects’ autonomy to decide whether to participate in a researcher’s project. Obtaining the truly informed consent of the potential subjects may well conflict with a researcher’s goals. Obviously, the researcher hopes to recruit a sufficient number of subjects in a timely manner, but fully informing potential subjects of the risks and discomforts, particularly if no direct benefit occurs for the subjects, may result in a potential subject’s refusal to participate. Indeed, if a researcher obtains a 100% participation rate, it may be the result of subjects not fully understanding the risks and discomforts that are an inherent part of the research. Because of this conflict, the potential subject’s informed consent is required to be obtained by law and regulation in addition to professional ethics. However, in obtaining the subject of informed consent, it is useful for the ethical researcher to be aware of this potential conflict.

2 THE NUREMBERG CODE AND INFORMED CONSENT In the research context, the clearest statement of the obligation to obtain consent is found in the Nuremberg Code. The Nuremberg Code was created as part of the judgment in the post-World War II criminal trials of Nazi physicians who conducted atrocious

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

INFORMED CONSENT PROCESS, FORMS, AND ASSENT

experiments on concentration camp inmates. The first provision of the code states (2): The voluntary consent of the human subject is absolutely essential. This means that the person involved should have legal capacity to give consent; should be situated as to be able to exercise free power of choice, without the intervention of any element of force, fraud, deceit, duress, over-reaching, or other ulterior form of constraint or coercion, and should have sufficient knowledge and comprehension of the elements of the subject matter involved as to enable him to make an understanding and enlightened decision. This latter element requires that before the acceptance of an affirmative decision by the experimental subject there should be made known to him the nature, duration, and purpose of the experiment; the method and means by which it is to be conducted; all inconveniences and hazards reasonably to be expected; and the effects upon his health or person which may possibly come from his participation in the experiment.

This section of the code sets forth the goals and purposes of informed consent in the research context. It requires that a subject’s consent be both informed and voluntary, requires that both inconveniences and hazards that reasonably are to be expected be disclosed, and clearly states that the goal of the process is to enable the potential subjects to make ‘‘an understanding and enlightened decision.’’ The Nuremberg Code says nothing about consent forms and does not require them. Rather, it requires that the informed consent process be designed to enlighten potential subjects so they are in a position to decide voluntarily to enroll as a subject or to reject the investigator’s offer. It is the process of informed consent, not a consent form, that is essential to a well-informed decision. Indeed, the fact that the informed consent process and obtaining written informed consent are two separate matters is stated clearly in the federal research regulations. To approve research, the regulations require that the institutional review board (IRB) determine that ‘‘informed consent will be sought from each perspective subject or the subjects legally authorized representative’’ and separately requires the IRB to determine

that ‘‘informed consent will be appropriately documented . . . ’’ (3). 3 FEDERAL REGULATIONS AND INFORMED CONSENT Similar to the Nuremberg Code, the federal regulations require that that consent be obtained in circumstances that provide the potential subject ‘‘sufficient opportunity to consider whether or not to participate and that minimize the possibility of coercion or undue influence. Furthermore, the regulations require that the language used by the investigator when attempting to obtain a potential subject’s informed consent must be ‘understandable to the subject.’’’ The regulations, similar to the Nuremberg Code, require that the ‘‘basic elements of informed consent’’ include the following: 1. a statement that the study involves research, an explanation of the purposes of the research and the expected duration of the subject’s participation, a description of the procedures to be followed, and identification of any procedures that are experimental, 2. a description of any reasonably foreseeable risks or discomforts to the subject, 3. a description of any benefits to the subject or to others which reasonably may be expected from the research, 4. a disclosure of appropriate alternative procedures or courses of treatment, if any, that might be advantageous to the subject, 5. a statement describing the extent, if any, to which confidentiality of records identifying the subject will be maintained, 6. for research involving more than minimal risk, an explanation as to whether any compensation and an explanation as to whether any medical treatments are available if injury occurs, and, if so, what they consist of or where more information may be obtained, and ... 8. a statement that participation is voluntary, refusal to participate will involve

INFORMED CONSENT PROCESS, FORMS, AND ASSENT

no penalty or loss of benefits to which the subject is otherwise entitled, and the subject may discontinue participation at any time without penalty or loss of benefits to which the subject is otherwise entitled. Additional elements of informed consent are required in certain circumstances when appropriate. The federal rules also authorize the IRB to waive some or all elements of informed consent in limited circumstances when the investigator makes such a request. Such a waiver is authorized when ‘‘(1) the research involves no more than minimal risk to the subjects; (2) the waiver or alteration will not adversely affect the rights and welfare of the subjects; (3) the research could not practicably be carried out without the waiver or alteration; and (C) whenever appropriate, the subjects will be provided with additional pertinent information after participation.’’ 4 DOCUMENTATION OF INFORMED CONSENT A separate section of the regulations requires that informed consent be documented, with certain limited exceptions, and requires that the consent form, which must be signed by the subject or the subject’s legally authorized representative, contain the elements of the informed consent process. It is often not recognized or understood that obtaining a signature of a potential subject on an informed consent form does not mean that the person actually has been adequately informed and actually has given a knowledgeable and voluntary consent. The form is meant solely to memorialize the consent process, not substitute for that process. Unfortunately, over time, both investigators and institutional review boards have focused overly on the forms and have not emphasized the importance of the process itself. Several reasons for this are found. The first reason is that the only ‘‘contact’’ IRBs have with subjects is through the consent forms, and therefore they attempt to fulfill their obligation to protect the rights of subjects by focusing on the form. The second reason is

3

that a signed consent form is what investigators use to demonstrate that they have obtained the informed consent of subjects. Without somebody sitting in the room with the investigator and a potential subject, no other way is available for the investigator to demonstrate that she has met her obligation to obtain informed consent. Finally, when the federal regulatory authorities, the Office for Human Research Protection and the Food and Drug Administration, audit institutions and their IRBs for compliance with federal regulations, they focus almost entirely on the paperwork that is involved in approving research, including informed consent forms. Because of the concern that federal auditors might determine after the fact that a problem did occur with the informed consent form, the forms have become longer and more complicated so that they seem to be more complete. Furthermore, institutions have come to view these forms as ways of fending off potential litigation by subjects, even in the absence of any evidence that this fear is realistic. One major difficulty in creating a useful form is presenting complicated information in easyto-understand language. Indeed, other than consent forms, investigators never have to write in language a layperson can understand, and therefore they have little or no experience in doing so. It is not surprising that investigators tend to use the jargon and abbreviations of their professions that are so readily understood by them, their assistants, and their colleagues in the study. The length and complexity of the forms and the use of jargon raise questions as to the capacity of the investigator (and IRBs) to adequately explain the nature of the research to potential subjects in the context of the informed consent process. 5 IMPROVING THE PROCESS OF INFORMED CONSENT One challenge of the informed consent process is that it is neither medical nor scientific in nature. Rather, the informed consent process should be an educational undertaking; it requires careful thinking about what the ‘‘student’’ (subject) needs to know and how the ‘‘teacher’’ (investigator) can impart that

4

INFORMED CONSENT PROCESS, FORMS, AND ASSENT

knowledge in the best way. Teachers use just not reading material but also visual aids and assignments. For example, for certain types of research, the informed consent process can involve watching a video or even assigning potential subjects materials that they will discuss with the investigator when they come back at a later time. It has also been recommended that investigators test knowledge of potential subjects through the use of oral or written quizzes after the investigator informs the potential subject of the risks, benefits, and alternatives. If it is determined that the potential research subject is not aware of certain material matters, then the investigator can go back and explain those matters more until the potential subject is properly informed. If the informed consent form is going to be used as part of the teaching materials that will be provided to the potential subject, then it is essential that it be understandable to a layperson. Investigators will need to use common terms that may seem to them to be less precise but in fact will be more informative from the perspective of a layperson. For example, ‘‘catheters’’ are actually ‘‘thin tubes,’’ ‘‘extremities’’ are actually ‘‘arms and legs,’’ and ‘‘chemotherapy’’ is actually ‘‘medicines to try to treat your cancer.’’ In some studies, subjects may be given several drugs, and one often sees an exhaustive list of potential side effects for each drug. But, if the subject is receiving three drugs and all of them may cause nausea, then no need exists to say this three separate times—which drug causes the nausea in a particular subject is not of interest to the subject—it is the potential for nausea that matters. In sum, better consent processes will come about only if investigators seriously consider how to educate potential subjects fully about the risks, discomforts, potential benefits, and alternatives that might be derived from becoming a research subject. Instead of thinking about this as a technical matter, investigators might find it useful to think about how they would go about informing a close family member who is considering entering a research project. If investigators could accomplish this, then they would be in compliance with legal and ethical requirements.

6 ASSENT Assent is a term that is used in a situation where one cannot obtain true informed consent from the potential research subject because of their immaturity or incompetence. In such circumstances, the researcher needs to obtain the acquiescence of the subject to participate in research. The goal is to try to ensure that research is not conducted forcibly on objecting subjects. However, assent is more than the lack of objection. The federal regulations define assent as the ‘‘affirmative agreement to participate in research. Mere failure to object should not, absent affirmative agreement, be construed as assent.’’ In the context of the federal regulations, the term ‘‘assent’’ is used in the context of children who cannot give legally binding consent to become research subjects. Rather, the regulations state that if parents give ‘‘permission’’ and children ‘‘assent’’ to become a research subject, then the research may be conducted. Although no specific regulations have been developed on this topic, this process should also apply to subjects who are incompetent to provide consent. The incompetent person’s legally authorized representative would provide permission, and the incompetent person would give assent. When participation in research is likely to provide a direct benefit to an incompetent person, their lack of assent is not considered a barrier to their enrollment in the research. REFERENCES 1. Cobbs v. Grant, 8Cal. 3d 229 (California Supreme Court, 1972). 2. Trials of War Criminals before the Nuremberg Military Tribunals under Control Council Law No. 10, vol. 2. Nuremberg October 1946–April 1949. Washington, D.C.: U.S. Government Printing Office (n.d.), pp. 181–182. 3. The Department of Health and Human Services, Protection of Human subjects, Title 45 Code of Federal Regulations, part 46 (1991).

FURTHER READING G. J. Annas and M. A. Grodin (eds.), The Nazi Doctors and the Nuremberg Code. New York: Oxford University Press, 1992.

INFORMED CONSENT PROCESS, FORMS, AND ASSENT J. W. Berg, P. S. Appelbaum, L. S. Parker, and C. W. Lidz, Informed Consent: Legal Theory and Clinical Practice. New York: Oxford University Press, 2001. Office for Human Research Protections, Policy Guidances. Available: http://www.hhs.gov /ohrp/policy/index.html. The informed consent process. In D. D. Federman, K. E. Hanna, and L. L. Rodriguez (eds.), Responsible Research: A Systems Approach to Protecting Research Participants. Washington, D.C.: National Academy Press, 2002, pp. 119–127.

5

INSTITUTION

positions of leadership in multicenter settings. sponsor: 1. A person or agency that is responsible for funding a designated function or activity; sponsoring agency. 2. A person or agency that plans and carries out a specified project or activity. 3. The person or agency named in an Investigational New Drug Application or New Drug Application; usually a drug company or person at such a company, but not always (as with an INDA submitted by a representative of a research group proposing to carry out a phase III or phase IV drug trial not sponsored by a drug company). 4. A firm or business establishment marketing a product or service (5). study center: [trials] 1. data collection site; study clinic 2. Data collection or data generation site. 3. The center from which activities are directed; coordinating center; project office. 4. An operational unit in the structure of a study, especially a multicenter structure, separate and distinct from other such units in the structure, responsible for performing specified functions in one or more stages of the study (e.g., a clinical center or resource center) (5).

CURTIS L. MEINERT The Johns Hopkins University Bloomberg School of Public Health Center for Clinical Trials Baltimore, Maryland

institution: An established organization or corporation (as a college or university) especially of a public character (1). institution: Any public or private entity or agency (including federal, state, and other agencies) (2). institution: Any public or private entity or agency or medical or dental facility where clinical trials are conducted (3). institution: A place where a study is undertaken; usually a hospital or similar establishment (4). institution: An established organization, corporation, or agency, especially one that has a public character (5). institute: An organization for promotion of a cause; an educational institution and especially one devoted to technical fields (1). agency: an administrative unit through which power or influence is exerted to achieve some stated end or to perform some specified function (5). principal investigator (PI): [research] 1. The person having responsibility for conduct of the research proposed in a grant application submitted to the National Institutes of Health; such a person in any funding application submitted to the NIH, whether for grant or contract funding; such a person named on any funding proposal, regardless of funding source. 2. The person in charge of a research project; the lead scientist on a research project. 3. The head of a center in a multicenter study. 4. The chair of a multicenter study. 5. The head of a clinical center in a multicenter trial Usage note: Avoid in the sense of defn 2 in settings having multiple ‘‘principal investigators’’; use center director and chair of study to designate

In the parlance of clinical trials, most commonly used in reference to the corporate entity housing one or more study centers involved in carrying out a trial (e.g., a university housing such a study center), it is typically the fiscal officer of the institution who has legal authority for receiving and use of funds awarded to the institution for a study, consistent with the needs and dictates of the scientist responsible for the study center within the institution— usually referred to as principal investigator (see usage note for principal investigator). The most common usage in adjective form is in institutional review board (IRB). Most academic institutions engaged in research on human beings have one or more standing IRBs comprised of people from within and outside the institution. Investigators in the institution are obliged to submit all proposal

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

INSTITUTION

involving human beings to their IRB of record and may not initiate such research until or unless approved by that IRB and must thereafter maintain approval by submission of renewal requests not less frequently than annually, as dictated by the IRB. Typically, the term refers to boards within an investigators institution, but it is also used to refer to boards located outside the institution, including commercial IRBs. Virtually all multicenter studies are multiinstitutional in that they involve two institutions, such as recruiting and treating sites and a data center or coordinating center. REFERENCES 1. Merriam-Webster’s Collegiate Dictionary, 10th ed. Springfield, MA: Merriam-Webster, Inc., 2001. 2. Office for Human Research Protection, Code of Federal Regulations, Title 45: Public Welfare, Part 46: Protection of Human Subjects. Bethesda, MD: Department of Health and Human Services, National Institutes of Health, (revised) June 18, 1991. 3. International Conference on Harmonisation, E6 Good Clinical Practice. Washington, DC: U.S. Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research (CDER), and the Center for Biologics Evaluation and Research (CBER), April 1996. 4. S. Day, Dictionary for Clinical Trials. Chichester: Wiley, 1999. 5. C. L. Meinert, Clinical Trials Dictionary: Terminology and Usage Recommendations. Baltimore, MD: The Johns Hopkins Center for Clinical Trials, 1996.

INSTITUTIONAL AND INDEPENDENT REVIEW BOARDS

1.1 Federalwide Assurance (FWA) Although federal regulations guide IRB structure and function, these regulations are intended as a minimum standard rather than a maximum. Any IRB can have policies that exceed the minimum standard set by regulation. In fact, many IRBs have policies that apply to all human subjects research at its institution, whether the research is federally funded or FDA regulated. An FWA is an agreement between an institution and U.S. Department of Health and Human Services’ Office of Human Research Protection that specifies how the institution complies with applicable law and regulations. In this FWA, an institution may state that it applies federal regulations only to federally funded research, or it may state that it applies the regulations and its policies to all research on human subjects at its institution.

SUSAN S. FISH Boston University Schools of Public Health and Medicine Boston, Massachusetts

1

PURPOSE OF IRBs

The purpose of an IRB is to protect the rights and welfare of human research subjects (45 CFR 46, 21 CFR 56). An IRB provides external and independent review of research that involves human subjects, with a focus on the ethical and regulatory issues. The IRB reviews protocols prior to their implementation and during the course of the research. In the United States, independent review of research was first required in the mid1960s for research that was supported by the U.S. Public Health Service. However, the regulations that now guide clinical research review were initially promulgated in 1981 and have been amended periodically since then. Many institutions in the United States required independent review of research prior to requirements from various federal agencies, and some IRBs have been in existence since 1966. Currently IRB review is required for all studies that are either conducted or funded by various federal agencies as well as for all studies performed to support a marketing application to the Food and Drug Administration (FDA) for a drug, device or biologic agent. The regulations initially promulgated by the Department of Health and Human Services (45 CFR 46) were eventually adopted by 17 federal agencies, and have thus become known as the ‘‘Common Rule.’’ Both the Common Rule and the FDA regulations (21 CFR 50 and 56) regulate IRBs for the research for which their agencies are responsible. Although these two sets of regulations are different and occasionally in conflict with each other, the structure and function of IRBs described in both sets of regulations are, for the most part, identical.

2 STRUCTURE OF AN IRB 2.1 Membership of an IRB The structure of IRBs is guided by federal regulations, which describe the minimum requirements of membership. IRBs, by regulation, consist of at least five members. However, many IRBs have many more than five members; boards of 15–25 members are not uncommon. These members must have varying backgrounds so that they have the expertise to review the types of research submitted to that IRB. The membership must be diverse in terms of race, gender, cultural background, and profession. In addition, the membership must be familiar with and sensitive to community attitudes, communities in which the research is taking place, and from which research subjects are recruited, as well as the research community that is submitting the research for review. These sensitivities are intended to ‘‘promote respect for (the IRB’s) advice and counsel in safeguarding the rights and welfare of human subjects.’’ The membership should also include members with the knowledge and understanding of institutional commitments, applicable state and federal laws and regulations, and professional standards in the disciplines of research

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

INSTITUTIONAL AND INDEPENDENT REVIEW BOARDS

that is overseen by that IRB. The regulations go on to explain that if an IRB regularly reviews research that involves a certain category of subjects who might be vulnerable to coercion or undue influence concerning research participation, then the IRB membership should include at least one member who has knowledge about and experience in working with this group of people. An example of such a vulnerable group is children and adolescents. If an IRB regularly reviews biomedical studies in the pediatric population, then a pediatrician or pediatric nurse might be an IRB member who can meet this requirement and have the expertise with this population of subjects. If an IRB reviews educational studies that take place in the school system, then a teacher might meet this requirement. On any IRB, there must be at least one member whose ‘‘primary concerns’’ are in nonscientific areas and at least one member who focuses on scientific areas. In reality, most IRBs have many scientific members and a few nonscientific members. The nonscientific member(s) can provide a balancing perspective for the enthusiasm of the scientist members and can provide the perspective of the research subjects. Most IRB members are drawn from the institution whose research the IRB oversees. In its wisdom, regulations require that at least one member be unaffiliated with the institution, as a check and balance on potential institutional conflict of interest, who can bring to the table some awareness of broader community standards. When an IRB is faced with reviewing a study that requires expertise that is not possessed by its membership, the IRB may and should obtain consultation with one or more individuals who can supply that expertise. It may be that the consultation is in a scientific area (e.g., xenotransplantation), a methodologic area (e.g., qualitative research methodology), or a cultural setting in which the research may take place (e.g., orphans in rural Zambia). Although consultants may not vote as members of the IRB, their input can be invaluable to the understanding of the research by the IRB members who do vote. Consultation should be sought liberally by the IRB.

Membership on an IRB may be a volunteer activity or a paid activity. The duration of membership varies among IRBs. The IRB Chair facilitates meetings, conducts business, and has the authority and responsibility to act for the IRB in certain situations. What is consistent across IRBs is that IRB members are intelligent and dedicated people who take their role in protecting human research subjects quite seriously.

2.2 Conflict of Interest in the IRB The diversity of the IRB membership guards against the influence of potential conflicts of interest. Although IRB decisions are made by a majority vote of members present, most decisions are the result of near-consensus. It is unlikely that one person with an undisclosed conflict could single-handedly affect the outcome of a decision. However, an IRB must have a policy that defines a conflict of interest, and federal regulations prohibit an IRB member with a conflicting interest from participating in the review of a protocol. One conflict of interest that is almost universally recognized is that of being a researcher on a protocol under review. When an IRB member’s own protocol is being reviewed by the board, that member/researcher may not participate in the review except to provide information. In this case, the member/researcher must recuse him/herself from the discussion and vote and leave the meeting room during that time.

2.3 Authority of the IRB The IRB’s authority is assigned by the institution. Once that authority is assigned, it cannot be overridden by any part of the institution. IRB approval is necessary, but not sufficient, for conduct of a research study using human subjects. In other words, if an IRB approves a study, another body in the institution may prevent the project from being conducted. However, if an IRB does not approve a study, then it cannot be conducted. No body within an institution can overturn disapproval by an IRB.

INSTITUTIONAL AND INDEPENDENT REVIEW BOARDS

3 ACCREDITATION OF HUMAN RESEARCH PROTECTION PROGRAMS An IRB is part of a larger human research protection program (HRPP), which also includes the institution, the researchers, the sponsors, and others involved in the clinical research enterprise. Voluntary accreditation of HRPPs has developed in the last decade as a means of identifying those programs that not only meet the minimal federal regulations, but also strive toward best practices in implementing human research ethics. Some IRBs are accredited whereas many others are not. 4

FUNCTION

IRB function is described in its policies and procedures, which are based on ethical principles and regulatory requirements. Each IRB will develop its own policies and procedures, and thus functioning differs among IRBs. However, some functions are common across IRBs. Research protocols that are not reviewed by expedited procedures (45 CFR 46.110) must be reviewed at a regular meeting of the members of the IRB. For official action to be taken at an IRB meeting, a quorum of members must be present, which includes at least one nonscientific member. According to regulations, the written procedures must contain, at a minimum, policies for the following: • Conducting initial review of a protocol • Conducting continuing review of a pro• • • • •

tocol Reporting IRB findings and actions to the investigator Reporting IRB findings and actions to the institution Determining which projects must be reviewed more often than annually Determining which projects should be audited Ensuring that proposed changes to the research be submitted and approved prior to their implementation, except as needed to protect a subject from immediate harm

3

• Ensuring prompt reporting to IRB, to

the Institution, and to funding agencies of unanticipated problems involving risks to subjects or others • Ensuring prompt reporting to IRB, to the Institution, and to funding agencies of serious or continuing noncompliance with federal regulations or policies of the IRB • Ensuring prompt reporting to the Institution, and to funding agencies of suspensions of terminations of IRB approval of a protocol The approval of a protocol by an IRB is required before a researcher may begin to conduct a study. The approval period for protocols may vary depending on the risks of the research and on other factors, but the maximum duration of approval is one year. Prior to the expiration of approval, the IRB must conduct continuing review of the ongoing protocol. The study cannot be continued beyond its approval expiration date unless the IRB has reapproved the study at its continuing review and issued a new approval expiration date. 4.1 Criteria for IRB Approval For an IRB to approve a research study, it must assure that certain criteria are met. Unless all criteria are met, the IRB cannot issue an approval. These criteria are derived directly from the three Belmont principles: respect for persons, beneficence, and justice (see ‘‘Belmont Report’’) Both initial review and continuing review require that these same criteria have been satisfied. As with all federal regulations, these are minimal criteria; any IRB may include additional criteria to be met. These minimal criteria are listed in the federal regulations (45 CFR 46.111; 21 CFR 56.111). 1. Risks to subjects are minimized: (1) By using procedures that are consistent with sound research design and that do not unnecessarily expose subjects to risk and (2) whenever appropriate, by using procedures already being performed on the subjects for diagnostic or treatment purposes.

4

INSTITUTIONAL AND INDEPENDENT REVIEW BOARDS

2. Risks to subjects are reasonable in relation to anticipated benefits, if any, to subjects, and the importance of the knowledge that may reasonably be expected to result (see ‘‘Benefit-Risk Assessment’’ and ‘‘Risk-Benefit Analysis’’). 3. Selection of subjects is equitable. 4. Informed consent will be sought from each prospective subject or the subject’s legally authorized representative (see ‘‘Informed Consent,’’ ‘‘informed consent process,’’ ‘‘legally authorized representative,’’ and ‘‘assent’’). 5. Informed consent will be appropriately documented. (see ‘‘Informed Consent Form’’). 6. When appropriate, the research plan makes adequate provision for monitoring the data collected to ensure the safety of subjects (see ‘‘Data Safety Monitoring Board, DSMB’’ and ‘‘Data Monitoring Committee’’). 7. When appropriate, adequate provisions are implemented to protect the privacy of subjects and to maintain the confidentiality of data. 8. When some or all subjects are likely to be vulnerable to coercion or undue influence, additional safeguards have been included in the study to protect the rights and welfare of these subjects (see ‘‘Vulnerable Subjects’’). For an IRB to conduct a thorough review of the research once it has been implemented, the IRB has the authority to audit or observe the research, which includes the informed consent process (see ‘‘Informed Consent Process’’). Various IRBs perform these observations in different ways, but the authority for this activity exists for all IRBs. Researchers must be informed in writing as to the decision of the IRB concerning their protocol. If any changes to the protocol are required by the IRB so that it can approve the protocol, then those required changes must be specified in writing to the researcher. If a protocol is not approved, then the reasons for that decision must be communicated to the researcher in writing.

4.2 IRB Staff Most IRBs have staff support, although it can take a variety of forms. At one extreme is the IRB that has a part-time secretary/administrator who may be responsible for answering phones, typing letters to investigators, and supporting the IRB meeting by assuring that all members have the documents needed to review protocols. At the other extreme is an IRB office staffed with many certified IRB professionals who may review protocols for science, ethics, and regulatory compliance, as well as educate researchers about these same topics. 5 DIFFERENCES BETWEEN INSTITUTIONAL AND INDEPENDENT REVIEW BOARDS Federal regulations proscribe much of the structure and function of IRBs. When these regulations were written in 1981, most research was conducted in a single academic medical center. In the subsequent years, many changes have occurred in the research environment, which include clinical research being conducted in free-standing clinics and private doctors’ offices that are not affiliated with academic medical centers. The need for review of studies conducted in these other sites generated independent review boards that are not affiliated with a particular institution. In addition, the complexity of multiple IRB reviews for a multicenter study, in which each IRB makes different demands of the protocol but for scientific validity a protocol must be conducted identically at each site, has led sponsors to attempt to undergo a single IRB review using an independent IRB whose approval applies to all sites. Other terms for these unaffiliated boards are commercial IRBs, central IRBs, for-profit IRBs, and noninstitutional IRBs. 5.1 Institutional Review Boards Institutional IRBs report to a high-ranking administrative official within the institution, which is usually a dean of a school or president of a hospital. Membership on the IRB consists mostly of members of the institution, with the exception of the nonaffiliated member(s) and possibly the nonscientist member(s). The members are usually volunteers,

INSTITUTIONAL AND INDEPENDENT REVIEW BOARDS

5

and some may be minimally compensated for time or may be relieved from other responsibilities at the institution. The IRB may interact with other committees that review the research proposals, which include the scientific review committee, radiation safety committee, institutional biostafety committee, resource utilization committee, conflict of interest committee, grants administration, and others. Institutional IRBs have oversight for the research conducted at its institution(s) and by its investigators. Board members and IRB staff frequently know the researchers and work with them. Thus, the IRB must confront the issue of individual nonfinancial conflict of interest. Members and staff have relationships with the researchers, either positive or negative, that can affect the objectivity of their review and decision concerning a particular protocol. But the strength of the system is that the entire board can help minimize the impact of any potential relationship conflict; the entire board participates in the decision to approve or disapprove a protocol. Institutional IRBs are confronted with institutional conflict of interest issues. Research that is approved and conducted will benefit the institution of which many members are employees. Although rarely does pressure come from the institution to approve a particular study, the research effort at the institution benefits all members of that hospital or school, both in reputation and financially. Thus, a subconscious pressure may be placed on the board to approve protocols that must be guarded against.

on this committee is not part of their academic citizenship responsibility but rather is an extracurricular activity. Independent IRBs have oversight for a research protocol that may be conducted at many sites nationally or internationally. Board members and IRB staff do not work with the researchers, and frequently they are not familiar with the location or site at which the research will be conducted. Although the relationship conflicts of interest do not exist, the board members and staff are not familiar with the researcher and are not aware of the skills and ethics of a particular researcher beyond what might be available through review of curriculum vitae. The more subtle characteristics of the researcher are unknown to the IRB when it makes a decision to approve or disapprove a protocol. Independent IRBs must deal with a different type of institutional conflict of interest. These IRBs receive direct income from a given protocol review. Although the payment does not hinge on approval of the protocol, if an IRB repeatedly disapproves protocols from a given sponsor, then that sponsor is likely to use another independent IRB, which decreases the overall income of the IRB. Thus, a subconscious pressure may be placed on the board to approve protocols that must be guarded against.

5.2 Independent Review Boards

2. Code of Federal Regulations, Title 21, Chapter 1 [21 CFR 50], Food and Drug Administration, DHHS; Part 50—Protection of Human Subjects. Updated 1 April, 2007.

Independent IRBs traditionally are not affiliated with an academic institution or hospital performing the research, although they may be affiliated with a contract research organization or site management organization that performs research (see ‘‘Contract Research Organization’’). As suggested by their name, independent IRBs are usually independent of the researchers and independent of any other committees that may review the research. Their members are frequently paid, rather than being volunteers, because participation

REFERENCES 1. Code of Federal Regulations, Title 45A [45 CFR 46], Department of Health and Human Services; Part 46—Protection of Human Subjects. Updated 1 October, 1997.

FURTHER READING E. A. Bankert and R. J. Amdur, Institutional Review Board: Management and Function. Sudbury, MA: Jones and Bartlett Publishers, Inc., 2006. IRB: A Review of Human Subjects Research—a periodical dedicated to IRB issues. Published by The Hastings Center, Garrison, NY.

6

INSTITUTIONAL AND INDEPENDENT REVIEW BOARDS

IRB Forum, An internet discussion board focused on IRB issues. Available: http://www. irbforum.org/. USDHHS, Office of Human Research Protection Policy Guidance. Available: http://www. hhs.gov/ohrp/policy/index.html#topics. USFDA Guidances, Information Sheets, and Important Notices on Good Clinical Practice in FDA-Regulated Clinical Trials. Available: http://www.fda.gov/oc/gcp/guidance.html.

INSTITUTIONAL REVIEW BOARDS (IRB)

review of activities commonly conducted by research institutions. In addition to possessing the professional competence needed to review specific activities, an IRB must be able to ascertain the acceptability of applications and proposals in terms of institutional commitments and regulations, applicable law, standards of professional conduct and practice, and community attitudes. Therefore, IRBs must be composed of people whose concerns are in relevant areas. Currently, the FDA does not require IRB registration. The institutions where the study is to be conducted should be contacted to determine whether they have their own IRB. If the study is conducted at a site that does not have its own IRB, the investigators should be queried to see if they are affiliated with an institution with an IRB that would be willing to act as the IRB for that site in the study. Independent IRBs can be contracted to act as the IRB for a site without its own IRB. An IRB can be established in accordance with 21 Code of Federal Regulation (CFR) 56, and IRBs must comply with all applicable requirements of the IRB regulation and the Investigational Device Exemption (IDE) regulation (21 CFR Part 812) in reviewing and approving device investigations involving human testing. The sFDA does periodic inspections of IRB records and procedures to determine compliance with the regulations. The FDA Office of Health Affairs provides guidelines, including an ‘‘IRB Operations and Clinical Requirements’’ list, to help IRBs carry out their responsibilities for protection of research subjects. The topic of IRBs is also addressed in the Federal Register (March 13, 1975) and the Technical Amendments concerning ‘‘Protection of Human Subjects’’ (45 CFR Part 46).

Institutional Review Boards (IRBs) ensure the rights and the welfare of people who participate in clinical trials both before and during the trial. At hospitals and research institutions throughout the country, IRBs make sure that participants are fully informed and have given their written consent before studies ever begin. The U.S. Food and Drug Administration (FDA) monitors IRBs to protect and ensure the safety of participants in medical research. The purpose of IRB review is to ensure, both in advance and by periodic review, that appropriate steps are taken to protect the rights and welfare of human participants in research. To accomplish this purpose, IRBs use a group process to review research protocols and the related materials such as informed consent documents and investigator brochures. If an IRB determines that an investigation involves a significant risk device, it must notify the investigator and, if appropriate, the sponsor. The sponsor may not begin the investigation until approved by the FDA. Under FDA regulations, an IRB is an appropriately constituted group that has been formally designated to review and monitor biomedical research involving human subjects. In accordance with FDA regulations, an IRB has the authority to approve, require modifications in (to secure approval), or disapprove research. This group review serves an important role in the protection of the rights, safety, and welfare of human research subjects. An IRB must be composed of no less than five experts and lay people with varying backgrounds to ensure a complete and adequate This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/handbook/irb.htm, http://www.fda.gov/cder/about/smallbiz/humans. htm, and http://www.fda.gov/cdrh/devadvice/ide/ irb.shtml) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

INTEGRATED DATABASE

Concurrent with these methods for data integration was the development of principles of data warehousing. The classic approach to data warehousing involves three steps: data extraction, transformation, and loading. It is within the second step of transformation that data integration is performed. Here data is altered, if necessary, so that data values in the data warehouse are represented in a standard and consistent format. A simple example of this alteration is the transformation of laboratory values into standard international units. Often the term data warehouse is used interchangeably with integrated data. However, in a theoretical sense, they are not the same because data warehousing encompasses the entire process of designing, building, and accessing data in real time. Integrated data, however, may be a component of a data warehouse, but it can also exist as a separate entity and represents a harmonized view of data from multiple sources.

SUSAN J. KENNY Inspire Pharmaceuticals, Inc., Durham, North Carolina,

1 THEORY AND CONSTRUCTION OF INTEGRATED DATABASES Data integration is the problem of combining data that resides at different sources and providing the user with a unified view of these data (1). Integrated databases are widely used in many professions, including health care, epidemiology, postmarketing surveillance, and disease registries. The content of this article is restricted to the concepts, applications, and best practice as applied specifically to clinical trials. 1.1 The Purpose of Integrated Databases The primary purpose for creating an integrated database of clinical trial data is to have a database with a greater sample size so that potential differences among subpopulations of interest can be evaluated. These important differences may exist in the individual studies, but the smaller sample size may not yield enough statistical power to identify them adequately.

1.3 Constructing an Integrated Database The goal of an integrated database is to give the user the ability to interact with one large system of information that in actuality came from multiple sources. Thus, users must be provided with a homogeneous view of the data despite the reality that the data may have been represented originally in several forms. Before the creation of an integrated database, it must be established that the data to be integrated were collected in a similar manner so that the resulting integrated data accurately reflects the true associations between data elements. If studies used substantially different case report forms, had different inclusion/exclusion criteria, used different endpoint measurements, or had vastly different measures of statistical variability, then the use of an integrated database that is built from disparate data may lead to spurious or statistically biased conclusions. The first task in constructing an integrated database is to create a desired schema to describe the structure of the integrated database, the variables it will contain, and

1.2 The History of Integrated Databases The practice of combining heterogeneous data sources into a unified view is not new. With the introduction of computers into most business sectors in the 1960s, the use of electronic databases to store information became a standard practice. This practice led to an expansion of knowledge in the academic areas of database technology and computer science. Database technology was introduced in the late 1960s to support business applications. As the number of these applications and data repositories grew, the need for integrated data became more apparent. The first approach to integration was developed around 1980 (2), and over the past 20 years a rich history of research in the area of data integration and a rapid sophistication of methods has developed.

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

INTEGRATED DATABASE

how the values of these variables will be represented. A schema is akin to a blueprint that will guide the construction of the desired outcome. As a simple example, a schema to integrate the demographic data from three different clinical trials might specify that there will be one record per subject and that variables for age, age group category, sex, race, and survival status will be included. Associated with the database schema is a metadata dictionary to describe the attributes for each variable. The attributes would include the variable name, variable label, variable type (character or numeric), and the precision for numeric variables. In addition, a list of the acceptable values for each of the character variables and a range of acceptable values for numeric variables would be provided. The need for management of metadata is not limited to integrated databases, and various authors have presented approaches to sound practices of this important step (3,4). One of the most important steps in creating an integrated database is to specify clearly how variable values would be harmonized, or remapped, in the event that the originally collected values were not similar. This harmonization ranges from the simplistic, such as conversion of height to a standard unit, to the more difficult, such as the remapping of the recording of the states of disease progression. The recoding of adverse event data across several studies so that all verbatim terms are coded using the same version of the coding dictionary is another example of a type of harmonization that commonly occurs. It is important to document clearly the transformations because the integrity of the integrated data rests on the validation of this process. As new data is added to an existing integrated database, this documentation may need to be updated to reflect the remapping decision for any new and not previously seen data values. A consequence of harmonization is the loss of detail on an individual study basis. Depending on the extent of harmonization, it may not be possible to regenerate the original results from individual studies. Therefore, a trade-off occurs between standardization and the retention of important study details. Well-documented metadata should

support the efforts of the user of the integrated database to trace the path from the original data to the integrated, mapped data and clearly understand how important differences may have occurred. 1.4 The Benefit of Standards for Integration In an ideal setting, the knowledge of whether data from an individual clinical study eventually will be included in an integrated database would be known at the start of the study. However, this case is not practical because the decision to include data in an integrated database will likely depend on the success of the individual study and on the success of the entire product development. Therefore, the integration of data is most often done retrospectively, after the data has been collected and perhaps long after the study has been completed. This situation increases the likelihood that variables and values will need to be remapped to harmonize them with other data sources. In recent years, several initiatives to standardize clinical trial data have developed. The adoption of standards should serve to streamline any integration effort because it will reduce the variability in the individual study metadata. The Clinical Data Interchange Standards Consortium (CDISC) was founded in 1999 with the mission to ‘‘develop and support global, platform-independent data standards that enable information system interoperability to improve medical research and related areas of healthcare’’ (5). CDISC has developed standard metadata models for the representation of clinical trial data. The use of CDISC standards for each individual clinical trial at the beginning of the trial should result in a substantial reduction in the effort needed to integrate multiple studies into one integrated database. In a similar vein, Health Level Seven (HL7) is an accredited standards developing organization that operates in the healthcare arena with a mission to create standards for the exchange, management, and integration of electronic healthcare information. HL7 is working toward the creation of a standard vocabulary that will enable the exchange of clinical data and information so that a shared, well-defined, and unambiguous knowledge of the meaning of the data

INTEGRATED DATABASE

transferred is found (6). Both of these standards organizations have been referenced in the U.S. Food and Drug Administration’s (FDA’s) Critical Path Initiatives List (7,8) and undoubtedly will play an increased role in clinical trial data management in the future and result in the streamlining of integration efforts. Leading by example, the National Cancer Institute established the National Cancer Institute Center for Bioinformatics (NCICB) (9) in 2001 with the purpose to develop and provide interoperable biomedical informatics infrastructure tools and data that are needed by the biomedical communities for research, prevention, and care. Recognizing the problem of integrating data when multiple ways are available to describe similar or identical concepts, NCICB has embarked on developing a repository of common data elements, associated metadata, and a standard vocabulary. These efforts provide a robust example of how data integration can be streamlined and a solid vision for the future management of clinical trial data that can be applied to many therapeutic areas. 2 APPLICATION OF INTEGRATED DATABASES IN CLINICAL TRIAL RESEARCH Integrated databases have been used in clinical trial research for several years. In the classic sense, integrated databases are created for both regulatory purposes and postmarketing surveillance. With the rapid expansion of the field of bioinformatics, the use of integrated databases has expanded beyond this classic application. 2.1 Classic Applications The integration of data, especially the data related to patient safety, is required by most regulatory agencies when the drug is submitted for marketing approval. For submissions made to the U.S. Food and Drug Administration, the guidelines recommend that an application have sections entitled Integrated Summary of Efficacy and Integrated Summary of Safety (ISS) (10). In the ISS section, safety data from all available studies are integrated so that analyses of measures of

3

safety can be performed for the entire patient population exposed to the drug compound. Integrated databases, therefore, are often initially created as part of a new drug application. Using the integrated database of adverse events, sponsor companies can determine the most commonly occurring adverse events across the entire patient population. This information becomes part of the package insert and is used as a reference by informed patients, prescribing physicians, and marketing campaigns. In addition to defining the most common adverse events, the use of the integrated database to evaluate the serious adverse effects that would be too rare to be detected in a single study is of special interest. Often a sponsor may not be aware of potentially serious adverse effects or subpopulations that may have a differential response to treatment until the data is integrated and the sample size is large enough to detect these important differences. Using the integrated database, both sponsor companies and regulatory agencies can explore the safety profile across various subpopulations, such as age groups, racial or ethnicity groups, and gender. By employing data standards within a study project, data integration can be done as each clinical trial is completed rather than all at once just before submitting a new drug application. Sponsor companies profit from creating an integrated database as studies complete because it can reduce the time to submission, and it can highlight important differences between treatment groups or subpopulations earlier in the product development. In addition to creating integrated databases to fulfill regulatory requirements, many sponsors maintain an integrated database that contains results from late phase and postmarketing trials. The medical literature is replete with published examples of the use of integrated databases to monitor the safety and tolerability of a marketed drug. Many of these articles describe the use of integrated data specifically for postmarketing safety surveillance (11–16), and the sample size can be quite large when the drug compound under study is widely prescribed (17). The use of integrated databases

4

INTEGRATED DATABASE

is not restricted to late phase or postmarketing trials and can serve useful purposes in designing new trials and data collection instruments (18). 2.2 Recent Advances As the collection of data from clinical trials becomes less reliant on paper and increasingly collected via electronic means, the types of data that can be integrated has expanded, especially data from digital images. Constructing a database of images is inherently different than a database of textual values, and this difference impacts the design of the database and use of the data. New database strategies have been developed to address these issues (19,20). A recent study (21) has enumerated the issues associated with building an integrated database that contains the results of quantitative image analysis obtained from digital magnetic resonance images (MRIs). The authors present a workflow management system that uses a centralized database coupled with database applications that allow for seamless integration of the analysis of MRIs. This process could be generalized and applied to any clinical trial that uses image-based measurements. In an effort to improve drug safety programs, the FDA has recently embarked on several projects to discover better methods to predict cardiovascular risk of drugs. In two of these projects, the FDA has collaborated with nongovernment entities, Mortara Instruments, Inc. and Duke Clinical Research Institute, to design and build a repository to hold digital electrocardiograms. The use of this integrated warehouse will facilitate regulatory review and research and will aid in the development of tools to be used for the evaluation of cardiac safety of new drugs (22). The explosion of knowledge of the areas of genomics and proteomics has fostered the development of integrated databases that can be used to support clinical research. Matthew and colleagues (23) give a thorough review of the challenges that are being addressed in this new era of computational biology as applied to translational cancer research. By creating an integrated database

of gene expression and drug sensitivity profiles, Tsunoda and colleagues (24) could identify genes with expression patterns that showed significant correlation to patterns of drug responsiveness. Other authors have developed integrated databases of ‘‘omics’’ information that enables researchers working in the field of anticancer drug discovery to explore and analyze information in a systematic way (25). An example of the software infrastructure used to build biological data warehouse for the integration of bioinformatics data and for the retrieval of this information has been described and made freely available to the public (26). REFERENCES 1. M. Lenzerini, Data Integration: a theoretical perspective. Proceedings of the 21st ACM-SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems. 2002: 243–246. 2. A. Hurson and M. Bright, Multidatabase systems: An advanced concept in handling distributed data. Advances in Computers. 1991; 32: 149–200. 3. C. Brandt, R. Gadagkar, C. Rodriguez, and P. Nadkarni, Managing complex change in clinical study metadata. J. Am. Med. Inform. Assoc. 2004; 11(5): 380–391. 4. A. Vaduva and T. Vetterli, Meta data management for data warehousing: an overview. Int. J. Cooperative Inform. Syst. 2001; 10(3): 273–298. 5. Clinical Data Interchange Standards Consortium (CDISC). Available: www.cdisc.org. 6. Health Level Seven (HL7). Available: www.hl7.org. 7. U. S. Food and Drug Administration (FDA). Challenge and opportunity on the critical path to new medical products. March 2004. Available: http://www.fda.gov/ oc/initiatives/criticalpath/whitepaper.pdf. 8. U. S. Food and Drug Administration (FDA). Critical path opportunities list. March 2006. Available: http://www.fda.gov/oc/ initiatives/criticalpath/reports/opp list.pdf. 9. National Cancer Institute Center for Bioinformatics (NCICB). Available: http:// ncicb.nci.nih.gov/. 10. U. S. Food and Drug Administration (FDA). Guideline for the format and content of the clinical and statistical sections of an

INTEGRATED DATABASE application. July 1988. Available: http:// www.fda.gov/cder/guidance/statnda.pdf. 11. D. Payen, A. Sablotzk, P. Barie, G. Ramsay, S. Lowry, M. Williams, S. Sarwat, J. Northrup, P. Toland, F. V. McLBooth, International integrated database for the evaluation of severe sepsis and drotrecogin alfa (activated) therapy: analysis of efficacy and safety in a large surgical cohort. Surgery. 2006; 140(5): 726–739. 12. A. Gottlieb, C. Leonardi, B. Goffe, J. Ortonne, P. van der Kerhof, R. Zitnik, A. Nakanishi, and A. Jahreis, Etanercept monotherapy in patients with psoriasis: a summary of safety, based on an integrated multistudy database. J. Am. Acad. Dermatology. 2006; 54(3 Suppl 2): S92–100. 13. D. Hurley, C. Turner, I. Yalcin, L. Viktrup, and S. Baygani, Duloxetine for the treatment of stress urinary incontinence in women: an integrated analysis of safety. Eur. J. Obstet. Gynecol. Reprod. Biol. 2006; 125(1): 120–128. 14. R. Fleischmann, S. Baumgartner, M. Weisman, T. Liu, B. White, and P. Peloso, Long term safety of etanercept in elderly subjects with rheumatic diseases. Ann. Rheum. Dis. 2006; 65(3): 379–384. 15. C. Martin and C. Pollera, Gemcitabine: safety profile unaffected by starting dose. Int. J. Clin. Pharmacol. Res. 1996; 16(1): 9–18. 16. J. Rabinowitz, I. Katz, P. De Deyn, H. Brodaty, A. Greenspan, and M. Davidson, Behavioral and psychological symptoms in patients with dementia as a target for pharmacotherapy with risperidone. J. Clin. Psychiatry. 2004; 65(10): 1329–1334. 17. J. Shepherd, D. Vidt, E. Miller, S. Harris, and J. Blasetto,. Safety of rosuvastatin: Update of 16,876 rosuvastatin-treated patients in a multinational clinical trial program. Cardiology. 2007; 107(4): 433–443. 18. A. Farin and L. Marshall, Lessons from epidemiologic studies in clinical trials of traumatic brain injury. Acta. Neurochir. Suppl. 2004; 89: 101–107. 19. H. Tagare, C. Jaffe, and J. Duncan, Medical image databases: a content-based retrieval approach. J. Am. Med. Inform. Assoc. 1997; 4(3): 184–198. 20. C. Feng, D. Feng, and R. Fulton, Contentbased retrieval of dynamic PET functional images. IEEE Trans. Inf. Technol. Biomed. 2000; 4(2): 152–158. 21. L. Liu, D. Meir, M. Polgar-Turcsanyi, P. Karkocha, R. Bakshi, C. R. Guttman, Multiple

22.

23.

24.

25.

26.

5

sclerosis medical image analysis and information management. J. Neuroimaging. 2005; 15(4 Suppl): 103S–117S. U. S. Food and Drug Administration. The Future of Drug Safety – Promoting and Protecting the Health of the Public. January 2007. Available: http://www. fda.gov/oc/reports/iom013007.pdf. J. Matthew, B. Taylor, G. Bader, S. Pyarajan, M. Antoniotti, A. Chinnaiyan, C. Sander, J. Buarkoff, and B. Mishra, From bytes to bedside: data integration and computational biology for translational cancer research. PLOS Computational Biology. 2007; 3(2): 153–163. D. Tsunoda, O. Kitahara, R. Yanagawa, H. Zembutsu, T. Katagiri, K. Yamazaki, Y. Nakamura, and T. Yamori, An integrated database of chemosensitivity to 55 anticancer drugs and gene expression profiles of 39 human cancer cell lines. Cancer Res. 2002; 62(4): 1139–1147. F. Kalpakov, V. Poroikov, R. Sharipov, Y. Kondrakhin, A. Zakharov, A. Lagunin, L. Milanesi, and A. Kel, CYCLONET – an integrated database on cell cycle regulation and carcinogenesis. Nucleic Acids Res. 2007; 35(database issue): D550–556. S. Shah, Y. Huang, T. Xy, M. Yuen, J. Ling, and B. Ouellette, Atlas – a data warehouse for integrative bioinformatics. BMC Biominformatics. 2005; 21(6): 34.

CROSS-REFERENCES Clinical Data Management Integrated Summary of Safety Information Integrated Summary of Effectiveness Data Bioinformatics Data Mining of Health System Data

INTENTION-TO-TREAT ANALYSIS

that it is only possible to perform an analysis using a post-hoc selected subset. The essential concern with such subset selection, or equivalently post-hoc exclusions, is that the resulting subset may be susceptible to various forms of bias (3,5–7). A review is provided in Reference 8. Many who champion the efficacy subset analysis approach argue that statistical techniques may be applied to provide an unbiased analysis under certain assumptions (9). The essential statistical issue is the extent to which it can be assumed that missing data and omitted data do not introduce a bias under the missing information principle (10) (i.e., that missing/omitted data are ignorable).

JOHN M. LACHIN The George Washington University, Washington, DC

A clinical trial of a new therapy (agent, intervention, diagnostic procedure, etc.) has many objectives. One objective is to evaluate whether the therapy has the intended biological or physiologic effect, which is often termed pharmacologic efficacy in relation to a new pharmaceutical agent. Another is to evaluate the pragmatic use of the therapy in clinical practice, which is often termed simply effectiveness. The intention-to-treat principle refers to a strategy for the design, conduct, and analysis of a clinical trial aimed at assessing the latter, which is the pragmatic effectiveness of a therapy in clinical practice. An analysis for the assessment of pharmacologic efficacy generally excludes subjects who either did not comply with the assigned therapy or could not tolerate it because of adverse effects. Such analyses are often termed per-protocol, efficacy subset, or evaluable subset analyses and involve posthoc subset selection or post-hoc exclusions of randomized subjects. Conversely, for the assessment of effectiveness in an intent-totreat analysis, follow-up data for all randomized subjects should be obtained and included in the analysis. This design is the essence of the intention-to-treat principle. 1

1.2 Ignorable Missing Data Missing data refers to data that are hypothetically obtainable from a subject enrolled in a trial, but that are not obtained. The hypothetically obtainable data consists of every possible observation that could be obtained from a subject from the point of initial screening and randomization to the prespecified scheduled end of follow-up for that subject. In some trials, the prespecified end of study for a subject is a fixed period, such as 1 year of treatment and follow-up. In other trials, the prespecified end may depend on when the subject enters the trial, such as the case where patient entry is staggered over one period but there is a prespecified date on which all treatment and follow-up will end. If recruitment is conducted over a 3-year period and the total study duration is 5 years, then the first subject randomized hypothetically can contribute 5 years of data collection whereas the last can only contribute 2 years. Thus, for every clinical trial design, each randomized subject has an associated hypothetical complete set of data that could be collected, and the aggregate over all subjects is the set of hypothetically obtainable data. Data may be missing for many possible reasons or mechanisms, some of which may be ignorable, occur randomly, or occur completely by chance. Missing data that develop from an ignorable mechanism are called

MISSING INFORMATION

1.1 Background The intention-to-treat principle evolved from the evaluation by regulatory officials at the Food and Drug Administration (FDA) (1), as well as scientists from the National Institutes of Health (2) and academia (3), of clinical trials in which post-hoc subset selection criteria were applied (4). In some cases, the data are collected but are excluded at the time of analysis. More often, the protocol allows subjects to terminate follow-up, and thus allows subject data to be excluded (not collected), such

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

INTENTION-TO-TREAT ANALYSIS

missing completely at random (MCAR) (10), in the sense that the unobserved value of the missing observation is statistically independent of other potentially observable information, most importantly including the treatment group assignment. In the context of survival analysis, the equivalent assumption is censoring at random. Few missing data mechanisms satisfy the MCAR criterion on face value. One is administratively missing data whereby data from a subject cannot be obtained because of administrative curtailment of follow-up. This situation usually applies in the setting of staggered patient entry with a fixed study end date, such as the last patient entered in the above example for whom at most 2 years of data could be collected before study end, and for whom the assessments during years 3–5 would be administratively missing. Missing data may occur for many other reasons. Some subjects may die while participating in the study. Others may be withdrawn from therapy because of poor compliance, lack of evidence of a therapeutic effect, or an adverse effect of therapy, and concurrently withdrawn from the study (i.e., with no additional follow-up). Others may be lost to follow-up (so-called dropouts) because of withdrawal of consent, moving away, incarceration, loss of interest, and so on. The fundamental issue is whether these mechanisms can be claimed to be ignorable. Of these, losses to follow-up are often considered to be missing completely at random, but missing data from such subjects would only be ignorable when effects of treatment, either positive or negative, played no role in the decision to curtail follow-up. This situation might be plausible if examination of the timing and characteristics of such losses are equivalent among the groups, and the characteristics of losses versus those who were not lost to follow-up are equivalent. If differences are detected, then it is possible or even likely that the missing data from such subjects is not ignorable. If baseline covariate differences are detected between those lost to follow-up versus those not, then a sensitivity analysis that compares groups adjusted for those covariates might provide a less-biased comparison of the treatment groups.

On-study deaths in some circumstances may also be claimed to be ignorable, as in a study of a disease or condition (e.g., topical skin therapy) that has a negligible risk of mortality. However, even in such cases, deaths may not be ignorable if the treatment itself may adversely affect vital organs, such as through hepatotoxicity (liver toxicity). The latter could be a hypothetical concern for any agent that is known to be metabolized and excreted by the liver. Drugs have been discovered with none of the other preclinical (i.e., animal) findings or clinical findings (e.g., elevated liver enzymes) that would trigger suspicion of hepatotoxicity, but in later trials or clinical practice are shown to pose a risk of possibly fatal hepatotoxicity in some patients. Clearl, subjects withdrawn from treatment and follow-up because of a specific adverse effect would only be ignorable when the adverse effect can be claimed to be statistically independent of the treatment assignment. This result can virtually never be proven. Other subjects withdrawn from treatment and follow-up because of insufficient therapeutic effect are clearly not ignorable. The fundamental issue is that in all of these cases, it cannot be proven that the missing data or the mechanism(s) for missing data are ignorable or missing completely at random. 1.3 Conditionally Ignorable Missing Data Many statistical methods require the assumption that missing data are missing completely at random to provide an unbiased analysis. However, other methods provide an unbiased analysis under the assumption that missing data are missing at random (MAR). This explanation is somewhat of a misnomer. Under MAR, it is assumed that the missing data are in fact nonignorable in the sense that the probability of being missing may depend on the unobserved value. However, MAR assumes that this dependence is reflected in other information that has been observed. Thus under MAR, it is assumed that the missing data are conditionally independent of the unobserved value, conditioning on the other information that has been observed including treatment assignment.

INTENTION-TO-TREAT ANALYSIS

Clearly, this is a big assumption. For example, a longitudinal analysis that uses a mixed model implicitly assumes that the data that are missing at a follow-up visit are a function of the baseline characteristics and other follow-up observations that preceded it. The treatment group comparison can then be claimed to be unbiased if this relationship applies (i.e., the structural and random model components are correctly specified), and the important covariates have been measured and observed. However, these assumptions cannot be verified.

In conclusion, although statistical methods can provide an unbiased analysis of virtually any data structure with missing data when certain assumptions apply, and although those assumptions, either MCAR or MAR conditionally, can be tested and rejected, those assumptions can never be proven to apply, and the resulting analysis can never be proven to be unbiased. Lachin (8) summarized the issue by stating that the only incontrovertibly unbiased study is one in which all randomized patients are evaluated and included in the analysis, assuming that other features of the study are also unbiased. This is the essence of the intent-to-treat philosophy. Any analysis which involves post hoc exclusions of information is potentially biased and potentially misleading.

1.4 Potential for Bias If the data that are missing develop from nonignorable mechanisms, then the missing data can introduce substantial bias in the results. Lachin (8) presents a model for the assessment of the possible bias for an analysis of the difference between two proportions and the pursuant inflation in the type I error probability α. As the proportion of subjects excluded from the analysis increases, the maximum possible bias and the resulting α increase, as expected. Furthermore, as the total sample size increases, the possible bias and α with a given fraction of missing data increases. Consider the simplest case in which all control group subjects are followed, but a fraction of the treated group has missing data, for example 20%. For a sample size of 200, a bias of 0.05 leads to an α = 0.14086 whereas for N = 800, the same bias leads to an α = 0.38964. Various analyses can be performed to assess whether the missing at random (conditionally) assumption applies, such as comparing the characteristics of those with missing versus observed data, in aggregate and within groups, and comparing the characteristics of those with missing data between groups. The null hypothesis of such tests is that the missing data are conditionally ignorable within the context of a particular model. The alternative hypothesis is that they are not ignorable, and the resulting analysis is biased. Thus, such tests can reject the null hypothesis of ignorable missing data in favor of the alternative of nonignorable missing data, but such tests cannot prove that missing data are ignorable.

3

2

THE INTENTION-TO-TREAT DESIGN

Thus, an intention-to-treat design makes every attempt to ensure complete follow-up and collection of outcome data for every participant from the time of randomization to the scheduled completion of study, regardless of other developments such as noncompliance or adverse effects of therapy. The International Conference on Harmonization (ICH) document Guidance on Statistical Principles for Clinical Trials (11), provides the following description of the intention-to-treat principle: The principle that asserts that the effect of a treatment policy can best be assessed by evaluating on the basis of the intention to treat a subject (i.e. the planned treatment regimen) rather than the actual treatment given. It has the consequence that subjects allocated to a treatment group should be followed up, assessed and analyzed as members of that group irrespective of their compliance with the planned course of treatment.

This guidance also states (Section 5.2.1): The intention-to-treat principle implies that the primary analysis should include all randomized subjects. Compliance with this principle would necessitate complete follow-up of all randomized subjects for study outcomes.

4

INTENTION-TO-TREAT ANALYSIS

The ICH the Guidance on General Considerations for Clinical Trials (12, Section 3.2.2) also states: The protocol should specify procedures for the follow-up of patients who stop treatment prematurely.

These principles are the essence of the intent-to-treat design. To conduct a study that provides high confidence that it is unbiased, the extent of missing data must be minimized. 2.1 Withdrawal from Treatment versus Withdrawal from Follow-up Every study protocol should include a provision for the investigator to withdraw a subject from the randomly assigned therapy because of possible adverse effects of therapy. However, the intent-to-treat design requires that such subjects should not also be withdrawn from follow-up. Thus, the protocol should distinguish between withdrawal from treatment versus withdrawal from follow-up. To the extent possible, every subject randomized should be followed as specified in the protocol. In fact, it would be advisable that the study protocol not include provision for the investigator to withdraw a subject from the study (i.e., follow-up). The only exceptions to this policy might be the death or incapacitation of a subject, or the subject’s withdrawal of patient consent. In some studies, time to a major clinical event is the primary outcome, such as a major cardiovascular adverse event or overall survival. In these cases, even if the subject withdraws consent to continue follow-up visits, it would also be important to ask the patient to consent to ascertainment of major events and/or vital status. Furthermore, in long-term studies, patients may withdraw consent, and then later be willing to resume follow-up and possibly the assigned treatment where not contraindicated. To allow patient participation to the extent possible, subjects should not be labeled as ‘‘dropouts’’ while the study is underway. Subjects who withdraw consent or who do not maintain contact may be termed inactive, temporarily, with the understanding that

any such subject may later become active. The designation of ‘‘dropout’’ or ‘‘lost to followup’’ should only be applied after the study has been completed. 2.2 Investigator and Subject Training/Education Unfortunately, many investigators have participated in clinical trials that were not designed in accordance with this principle. Even though the protocol may state that all patients should continue follow-up to the extent possible, investigators may fail to comply. In studies that have successfully implemented continued follow-up, extensive education of physicians, nurse investigators, and patients has been implemented. In the Diabetes Control and Complications Trial (1983–1993), the patients and investigators received intensive patient education on the components of the trial and the expectation that they would continue followup (13). Of the 1441 subjects randomized into the study, during the 10 years of study only 32 subjects were declared inactive at some point in time. Of these subjects, 7 later resumed treatment and follow-up. Among the 1330 surviving subjects at study end, only 8 subjects did not complete a study closeout assessment visit. During the trial, 155 of the 1441 patients deviated from the originally assigned treatment for some period (were noncompliant). Virtually all subjects continued to attend follow-up assessment visits and most resumed the assigned therapy later. The DCCT was unusual because two multifaceted insulin therapies were being compared in subjects with type 1 diabetes who must have insulin therapy to sustain life. Therefore, withdrawal from insulin therapy was not an option. In a study comparing a drug versus placebo, the issues are different. The Diabetes Prevention Program (14) included comparison of metformin versus placebo for preventing the development of overt diabetes among patients with impaired glucose tolerance. Metformin is an approved antihyperglycemic therapy for treatment of type 2 diabetes with known potential adverse effects that require discontinuation of treatment in about 4% of patients, principally

INTENTION-TO-TREAT ANALYSIS

because of gastrointestinal effects. The following is the text provided to the investigators to explain to the patient why the study desired to continue follow-up after therapy was withdrawn due to an adverse effect: When we designed the study we knew that a fraction of patients would not be able to tolerate metformin. You were told this when you agreed to participate in the study. However, we cannot tell beforehand which participants will be able to take metformin, and which will not. In order to answer the DPP study question as to whether any treatment will prevent diabetes, every participant randomized into the study is equally important. Thus, even though you will not be taking a metformin pill, it is just as important for us to know if and when you develop diabetes as it is for any other participant. That’s why it is just as important to the study that you attend your outcome assessment visits in the future as it was when you were taking your pills.

2.3 The Intent-to-Treat Analysis An intent-to-treat analysis refers to an analysis that includes all available data for all randomized subjects. However, for an intent-totreat analysis to comply with the intentionto-treat principle, all ‘‘available’’ data should represent a high fraction of all potentially observable data. Thus, an analysis of all randomized subjects, in which a high fraction have incomplete assessments and missing data, deviates from the intention-to-treat principle in its design and/or implementation, and thus it is possibly biased. 2.4 Intent-to-treat Subset Analysis In many studies, multiple analyses are conducted in different sets of subjects. The intention-to-treat ‘‘population’’ is often defined to include all subjects randomized who had at least one dose of the study medication. However, unless the protocol specifies systematic follow-up of all subjects, and a high fraction of the potentially obtainable data is actually obtained, then an analysis of the intention-to-treat population is simply another post-hoc selected subgroup analysis that is susceptible to bias because of nonignorable or informatively missing data.

5

2.5 LOCF Analysis In an attempt to reconstruct a complete data set from incomplete data, a simplistic approach that is now commonly employed is an analysis using the last observation carried forward (LOCF) for each subject with missing follow-up measurements. This method is popular because it makes it seem as though no substantial data are missing. However, the technique is statistically flawed (15,16). LOCF values would not be expected to provide the same level of information as values actually measured and would not be expected to follow the same distribution. Furthermore, such values will distort the variance/covariance structure of the data so that any confidence intervals or P-values are biased in favor of an optimistic assessment in that the sample size with LOCF values is artificially inflated and the variance of the measures is artificially deflated. The LOCF method has no formal statistical basis and has been soundly criticized by statisticians. 2.6 Structurally Missing Data In some studies, the primary outcome is the observation of a possibly right-censored event time and a secondary outcome is a mechanistic or other longitudinal measure. Often, the follow-up of a subject is terminated when the index event occurs, that causes all subsequent mechanistic measurements to be missing structurally. For example, in the Diabetes Prevention Program, the primary outcome was the time to the onset of type 2 diabetes, and measures of insulin resistance and insulin secretory capacity were obtained up to the time of diabetes or the end of study (or loss to follow-up). Thus, these mechanistic measures were missing beyond the time of diabetes, and it is not possible to conduct a straightforward intention-to-treat analysis to describe the long-term effect of each initial therapy on these mechanistic measures (e.g., the difference between groups at say 4 years in the total cohort of those entered into the study). 2.7 Worst Rank Analyses In some cases, it may be plausible to assume that subjects with missing data because of

6

INTENTION-TO-TREAT ANALYSIS

a prior index event are actually worse (or better) than all those who continue followup for a particular measure. For example, subjects who have died can be assumed to have a worse quality of life than any subject who survives. In this case, the subjects with structurally missing data because of such an index event can be assigned a ‘‘worst rank,’’ (i.e., a rank worse than that of any of the measures actually observed) (17). 3 EFFICIENCY OF THE INTENT-TO-TREAT ANALYSIS 3.1 Power The intent-to-treat design, which is necessary to conduct a true intent-to-treat analysis, requires the follow-up of all patients, which includes those who failed to comply with the therapy or those who were withdrawn from therapy by personal choice or by the investigator. Thus, the treatment effect observed may be diluted compared with a setting in which all subjects receive the therapy and are fully compliant. However, that comparison is specious. Virtually every agent or intervention will not be applied optimally in every subject in clinical practice. Therefore, an analysis aimed at the treatment effectiveness under optimal conditions is not relevant to clinical practice. Such an assessment, however, is of interest as a reflection of pharmacologic activity, which is the underlying mechanism by which a treatment is purported to have a clinical effect. This mechanism is the justification for the so-called ‘‘efficacy subset’’ or ‘‘perprotocol’’ analysis often conducted in pharmaceutical trials. The phrase ‘‘per-protocol’’ is used when the protocol itself specifies the post-hoc exclusions of patients and patient data based on compliance, adverse effects, or other factors that indicate suboptimal treatment. Such a subset analysis is highly susceptible to bias. Nevertheless, it is instructive to compare the power of an intention-to-treat design and analysis versus an efficacy subset analysis when it is assumed that the latter is unbiased. Using the test for proportions, Lachin (8) showed that there is a trade-off between the increasing sample size in the intentto-treat analysis versus the larger expected

treatment effect in the efficacy subset analysis. However, in some settings the intentto-treat design and analysis may be more powerful, especially when the treatment may have some long-term beneficial effects that persist beyond the period of therapy. For example, if the treatment arrests progression of the disease, then a subject treated for a period of time who is then withdrawn may still be expected to have a more favorable outcome long-term compared with a subject originally assigned to control. In this case, the intention-to-treat analysis can have substantially more power than the efficacy subset analysis, and it has the advantage that it is far less susceptible to bias. These issues were borne out in the analysis of the study of tacrine in the treatment of Alzheimer’s disease (18), in which 663 subjects were entered and 612 completed follow-up. However, only 253 continued on treatment. Thus, the intent-to-treat analysis of the 612 evaluated could be compared with the efficacy subset analysis of the 253 ‘‘ontherapy completers’’ as shown in Reference 8. For some outcomes, the intention-to-treat analysis produced results that were indeed significant whereas the efficacy subset analysis was not. 3.2 Sample Size In the planning of a study, it is common to adjust the estimate of the sample size for losses to follow-up. For a simple analysis of means or proportions, the sample size computed assuming complete follow-up is inflated to allow for losses. For example, to adjust for 20% losses to follow-up (subjects with missing outcome data), the sample size is inflated by the factor 1/0.8, or by 25%. Such an adjustment allows for the loss of information because of losses to followup or missing data. It does not adjust for the potential bias introduced by such losses if the mechanism for the missing data is informative.

4 COMPLIANCE ADJUSTED ANALYSES The efficacy subset analysis is a crude attempt to assess the treatment effect had

INTENTION-TO-TREAT ANALYSIS

all subjects remained fully compliant. However, if an intention-to-treat design is implemented, then noncompliance and the degree of compliance become outcome measures, which make it possible to conduct analyses that assess treatment group differences in the primary outcomes while taking into account the differences in compliance. Analyses could also be conducted to estimate the treatment group difference for any assumed degree of compliance in the treatment groups (19–21). However, these methods can only be applied when study outcomes are measured in all subjects, or a representative subset, which includes those who are noncompliant or who are withdrawn from therapy. These methods cannot be applied in cases where follow-up is terminated when a subject is withdrawn from therapy because of noncompliance or other factors. 5

CONCLUSION

The intention-to-treat principle encourages the collection of complete follow-up data to the extent possible under an intentionto-treat design and the inclusion of all data collected for each subject in an intentionto-treat analysis. An analysis of a so-called Intention-to-treat population of all subjects randomized, but without systematic followup, is simply another type of post-hoc subset analysis that is susceptible to bias. REFERENCES 1. R. Temple and G. W. Pledger, The FDA’s critique of the Anturane Reinfarction Trial. N. Engl. J. Med. 1980; 303: 1488–1492. 2. D. L. DeMets, L. M. Friedman, and C. D. Furberg, Counting events in clinical trials (letter to the editor). N. Engl. J. Med. 1980; 302: 924. 3. L. M. Friedman, C. D. Furberg, and D. L. DeMets, Fundamentals of Clinical Trials, 3rd ed. New York: Springer; 1998. 4. D. L. Sackett and M. Gent, Controversy in counting and attributing events in clinical trials. N. Engl. J. Med. 1979; 301: 1410–1412. 5. G. S. May, D. L. DeMets, L. M. Friedman, C. Furberg, E. Passamani, The randomized clinical trial: bias in analysis. Circulation 1981; 64: 669–673.

7

6. P. Armitage, Controversies and achievements in clinical trials. Control. Clin. Trials 1984; 5: 67–72. 7. R. Peto, M. C. Pike, P. Armitage, et al. Design and analysis of randomized clinical trials requiring prolonged observation of each patient. I. Introduction and design. Br. J. Cancer 1976; 34: 585–612. 8. J. M. Lachin, Statistical Considerations in the Intent-to-treat Principle. Control. Clin. Trials 2000; 21: 167–189. 9. L. B. Sheiner and D. B. Rubin, Intention-totreat analysis and the goals of clinical trials. Clin. Pharmacol. Ther. 1995; 57: 6–15. 10. R. J. A. Little and D. B. Rubin, Statistical Analysis with Missing Data. New York: Wiley; 1987. 11. Food and Drug Administration, International Conference on Harmonization: Guidance on statistical principles for clinical trials. Federal Register September 16, 1998; 63: 49583–49598. 12. Food and Drug Administration, International Conference on Harmonization: Guidance on general considerations for clinical trials. Federal Register December 17, 1997; 62: 66113–66119. 13. Diabetes Control and Complications Trial Research Group, Implementation of a multicomponent process to obtain informed consent in the Diabetes Control and Complications Trial. Control. Clin. Trials 1989; 10: 83–96. 14. Diabetes Prevention Program Research Group, The Diabetes Prevention Program: design and methods for a clinical trial in the prevention of type 2 diabetes. Diabetes Care 1999; 22: 623–634. 15. G. Veberke, G. Molenberghs, L. Bijnens, and D. Shaw, Linear Mixed Models in Practice. New York: Springer, 1997. 16. F. Smith, Mixed-model analysis of incomplete longitudinal data from a high-dose trial of tacrine (cognex) in Alzheimer’s patient. J. Biopharm. Stat. 1996; 6: 59–67. 17. J. M. Lachin, Worst rank score analysis with informatively missing observations in clinical trials. Control. Clin. Trials 1999; 20: 408–422. 18. M. J. Knapp, D. S. Knopman, P. R. Solomon, et al., A 30-week randomized controlled trial of high-dose Tacrine in patients with Alzheimer’s disease. J. Am. Med. Assoc. 1994; 271: 985–991. 19. J. Rochon, Supplementing the intent-to-treat analysis: Accounting for covariates observed postrandomization in clinical trials. J. Am. Stat. Assoc. 1995; 90: 292–300.

8

INTENTION-TO-TREAT ANALYSIS

20. B. Efron and D. Feldman, Compliance as an explanatory variable in clinical trials. J. Am. Stat. Assoc. 1991; 86: 9–25. 21. J. W. Hogan and N. M. Laird, Intention-totreat analyses for incomplete repeated measures data. Biometrics 1996; 52: 1002–1017.

FURTHER READING Principles of Clinical Trial Design and Analysis: D. G. Altman, K. F. Schulz, D. Moher, M. Egger, D. Davidoff, D. Elbourne, P. C. Gøtzsche, and T. Lang, for the CONSORT Group, The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann. Intern. Med. 2001; 134: 663–694. P. Armitage, Controversies and achievements in clinical trials. Control. Clin. Trials 1984; 5: 67–72. D. L. DeMets, Statistical issues in interpreting clinical trials. J. Intern. Med. 2004; 255: 529–537 P. W. Lavori and R. Dawson, Designing for intent to treat. Drug Informat. J. 2001; 35: 1079–1086 G. S. May, D. L. DeMets, L. M. Friedman, C. Furberg, and E. Passamani, The randomized clinical trial: bias in analysis. Circulation 1981; 64: 669–673. R. Peto, M. C. Pike, P. Armitage, et al., Design and analysis of randomized clinical trials requiring prolonged observation of each patient. I. Introduction and design. Br. J. Cancer 1976; 34: 585–612. D. Schwartz and J. Lellouch, Explanatory and pragmatic attitudes in therapeutic trials. J. Chron. Dis. 967; 20: 637–648. Case Studies G. Chene, P. Moriat, C. Leport, R. Hafner, L. Dequae, I. Charreau, JP. Aboulker, B. Luft, J. Aubertin, J. L. Vilde, and R. Salamon, Intention-to-treat vs. on-treatment analyses from a study of pyrimethamine in the primary prophylaxis of toxoplasmosis in HIV-infected patients. ANRS 005/ACTG 154 Trial Group. Control. Clin. Trials 1998; 19: 233–248. The Coronary Drug Project Research Group, Influence of adherence to treatment and response of cholesterol on mortality in the Coronary Drug Project. N. Engl. J. Med. 1980; 303: 1038–1041. P. W. Lavori, Clinical trials in psychiatry: should protocol deviation censor patient data (with Discussion). Neuropsychopharm 1992; 6: 39–63.

P. Peduzzi, J. Wittes, K. Detre, and T. Holford, Analysis as-randomized and the problem of non-adherence: An example from the Veterans Affairs Randomized Trial of Coronary Artery Bypass Surgery. Stat. Med. 1993; 15: 1185–1195. C. Redmond, B. Fisher, H. S. Wieand, The methodologic dilemma in retrospectively correlating the amount of chemotherapy received in adjuvant therapy protocols with disease-free survival. Cancer Treat. Rep. 1983; 67: 519–526. Alternative Views J. H. Ellenberg, Intent-to-treat analysis versus astreated analysis. Drug Informat. J. 1996; 30: 535–544. M. Gent and D. L. Sackett, The qualification and disqualification of patients and events in longterm cardiovascular clinical trials. Thrombos. Haemostas. 1979; 41: 123–134. E. Goetghebeur and T. Loeys, Beyond Intention-totreat. Epidemiol. Rev. 2002; 24: 85–90. Y. J. Lee, J. H. Ellenberg, D. G. Hirtz, and K. B. Nelson, Analysis of clinical trials by treatment actually received: is it really an option? Stat. Med. 1991; 10: 1595-605. R. J. A. Little, Modeling the drop-out mechanism in repeated-measure studies. J. Am. Stat. Assoc. 1995; 90: 1112–1121. L. B. Sheiner, Is intent-to-treat analysis always (ever) enough? Br. J. Clin. Pharmacol. 2002; 54: 203–211.

CROSS-REFERENCES Adequate and Well-Controlled Trial Adherence Analysis Data Set Analysis Population Completer Analysis Casual Inference CONSORT Diabetes Control and Complications Trial Diabetes Prevention Program Effectiveness Evaluable Population Full Analysis Set Good Statistics Practice International Conference on Harmonization (ICH) Last Observation Carried Forward (LOCF) Lost-to-Follow-Up Missing Values

INTENTION-TO-TREAT ANALYSIS Per Protocol Set Analysis Protocol Deviations Protocol Deviators Protocol Violators Responder Analysis Subset Withdrawal from Study Withdrawal from Treatment

9

INTERACTION MODEL

1

GRAPHICAL INTERACTION MODELS

What is meant by association between two variables? The most general response to this question is indirect. Two variables are dissociated if they are conditionally independent given the rest of the variables in the multivariate framework in which the two variables are embedded. Association then simply means that the two variables are not dissociated. Association in this sense is, of course, not a very precise statement. It simply means that conditions exist under which the two variables are not independent. Analysis of association will typically have to go beyond the crude question of whether or not association is present, to find out what characterizes the conditional relationship—for instance, whether it exists only under certain conditions, whether it is homogeneous, or whether it is modified by outcomes on some or all the conditioning variables. Despite the inherent vagueness of statements in terms of unqualified association and dissociation, these statements nevertheless define elegant and useful models that may serve as the natural first step for analyses of association in multivariate frames of inference. These so-called graphical models are defined and described in the subsections that follow.

SVEND KREINER University of Copenhagen, Copenhagen, Denmark

Interaction models for categorical data are loglinear models describing association among categorical variables. They are called interaction models because of the analytic equivalence of loglinear Poisson regression models describing the dependence of a count variable on a set of categorical explanatory variables and loglinear models for contingency tables based on multinomial or product multinomial sampling. The term is, however, somewhat misleading, because the interpretation of parameters from the two types of models are very different. Association models would probably be a better name. Instead of simply referring the discussion of interaction and association models to the section on loglinear models, we will consider these models from the types of problems that one could address in connection with analysis of association. The first problem is a straightforward question of whether or not variables are associated. To answer this question, one must first define association and dissociation in multivariate frameworks and, secondly, define multivariate models in which these definitions are embedded. This eventually leads to a family of so-called graphical models that can be regarded as the basic type of interaction or association. The second problem concerns the properties of the identified associations. Are associations homogeneous or heterogeneous across levels of other variables? Can the strength of association be measured and in which way? To solve these problems, one must first decide upon a natural measure of association among categorical variables and, secondly, define a parametric structure for the interaction models that encapsulates this measure. Considerations along these lines eventually lead to the family of hierarchical loglinear models for nominal data and models simplifying the basic loglinear terms for ordered categorical data.

1.1 Definition A graphical model is defined by a set of assumptions concerning pairwise conditional independence given the rest of the variables of the model. Consider, for instance, a model containing six variables, A to F. The following set of assumptions concerning pairwise conditional independence defines four constraints for the joint distribution Pr(A, B, C, D, E, F). The family of probability distributions satisfying these constraints is a graphical model: A ⊥ C|BDEF ⇔ Pr(A, C|BDEF) = Pr(A|BDEF) Pr(C|BDEF), A ⊥ D|BCEF ⇔ Pr(A, D|BCEF) = Pr(A|BCEF) Pr(D|BCEF),

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

INTERACTION MODEL

Graphical models for multidimensional tables were first discussed by Darroch et al. (5). Since then, the models have been extended both to continuous and mixed categorical and continuous data and to regression and block recursive models. Whittaker (9), Edwards (7), Cox & Wermuth (4), and Lauritzen (8) present different accounts of the theory of graphical models. The sections below summarize some of the main results from this theory. Figure 1. An interaction graph

1.2 The Separation Theorem B ⊥ E|ACDF ⇔ Pr(B, E|ACDF) = Pr(B|ACDF) Pr(E|ACDF), C ⊥ E|ABDF ⇔ Pr(C, E|ABDF) = Pr(C|ABDF) Pr(E|ABDF).

Interaction models defined by conditional independence constraints are called ‘‘graphical interaction models’’, because the structure of these models can be characterized by so-called interaction graphs, where variables are represented by nodes connected by undirected edges if and only if association is permitted between the variables. The graph shown in Figure 1 corresponds to the set of conditional independence constraints above, because there are no edges connecting A to C, A to D, B to E, and C to E. Interaction graphs are visual representations of complex probabilistic structures. They are, however, also mathematical models of these structures, in the sense that one can describe and analyze the interaction graphs by concepts and algorithms from mathematical graph theory and thereby infer properties of the probabilistic model. This connection between probability theory and mathematical graph theory is special to the graphical models. The key notion here is conditional independence, as discussed by Dawid (5). While the above definition requires that the set of conditioning variables always includes all the other variables of the model, the results described below imply that conditional independence may sometimes be obtained if one conditions with certain subsets of variables.

The first result connects the concept of graph separation to conditional independence. First, we present a definition: a subset of nodes in an undirected graph separate two specific nodes, A and B, if all paths connecting A and B intersect the subset. In Figure 1, (B, D, F) separate A and B, as does (B, E, F). E and C are separated by both (A, D, F) and (B, D, F). The connection between graph separation and conditional independence is given by the following result, sometimes referred to as the separation theorem. 1.2.1 Separation Theorem. If variables A and B are conditionally independent given the rest of the variables of a multivariate model, A and B will be conditionally independent given any subset of variables separating A and B in the interaction graph of the model. The four assumptions on pairwise conditional independence defining the model shown in Figure 1 generate six minimal separation hypotheses: A ⊥ C|BDF,

A ⊥ C|BEF,

A ⊥ D|BEF,

B ⊥ E|ADF,

C ⊥ E|ADF,

C ⊥ E|BDF.

1.3 Closure and Marginal Models It follows from the separation theorem that graphical models are closed under marginalization, in the sense that some of the independence assumptions defining the model transfer to marginal models. Collapsing, for instance, over variable C of the model shown in Figure 1 leads to a

INTERACTION MODEL

3

been deleted define a hierarchial loglinear model with parameters corresponding to each of the completely connected subsets of nodes in the graph. The interaction graph for the model shown in Figure 1 has four cliques, BCDF, ABF, AEF, and DEF, corresponding to a loglinear model defined by one four-factor interaction and three three-factor interactions. 1.5 Separation and Parametric Collapsibility

Figure 2. An interaction graph obtained by collapsing the model defined by Figure 1 over variable C

graphical model defined by conditional independence of A and D and B and E, respectively, because the marginal model contains separators for both AD and BE (Figure 2). 1.4 Loglinear Representation of Graphical Models for Categorical Data No assumptions have been made so far requiring variables to be categorical. If all variables are categorical, however, the results may be considerably strengthened both with respect to the type of model defined by the independence assumptions of graphical models and in terms of the available information on the marginal models. The first published results on graphical models (5) linked graphical models for categorical data to loglinear models: A graphical model for a multidimensional contingency table without structural zeros is loglinear with generators defined by the cliques of the interaction graph.

The result is an immediate result of the fact that any model for a multidimensional contingency table has a loglinear expansion. Starting with the saturated model, one removes all loglinear terms containing two variables assumed to be conditional independent. The loglinear terms remaining after all the terms relating to one or more of the independence assumptions of the model have

While conceptually very simple, graphical models are usually complex in terms of loglinear structure. The problems arising from the complicated parametric structure are, however, to some degree to be compensated for by the properties relating to collapsibility of the models. Parametric collapsibility refers to the situation in which model terms of a complete model are unchanged when the model is collapsed over one or more variables. Necessary conditions implying parametric collapsibility of loglinear models are described by Agresti [1, p. 151] in terms which translate into the language of graphical models: Suppose variables of a graphical model of a multidimensional contingency table are divided into three groups. If there are no edges connecting variables the first group with connected components of the subgraph of variables from the third group, then model terms among variables of the first group are unchanged when the model is collapsed over the third group of variables.

Parametric collapsibility is connected to separation in two different ways. First, parametric collapsibility gives a simple proof of the separation theorem, because a vanishing two-factor term in the complete model also vanishes in the collapsed model if the second group discussed above contains the separators for the two variables. Secondly, separation properties of the interaction graph may be used to identify marginal models permitting analysis of the relationship between two variables. If one first removes the edge between the two variables, A and B, and secondly identifies separators for A and B in the graph, then the model is seen to be parametric collapsible on to the model containing A

4

INTERACTION MODEL

and B and the separators with respect to all model terms relating to A and B. The results are illustrated in Figure 3, where the model shown in Figure 3(a) is collapsed on to marginal models for ABCD and CDEF. The separation theorem is illustrated in Figure 3(b). All terms relating to A and B vanish in the complete model. The model satisfies the condition for parametric collapsibility, implying that these parameters also vanish in the collapsed model. The second property for the association between E and F is illustrated in Figure 3(c). C and D separate E and F in the graph from which the EF edge has been removed. It follows, therefore, that E and F cannot be linked to one and the same connected component of the subgraph for the variables over which the table has been collapsed. The model is therefore parametric collapsible on to CDEF with respect to all terms pertaining to E and F. 1.6 Decomposition and Reducibility Parametric collapsibility defines situations in which inference on certain loglinear terms may be performed in marginal tables because these parameters are unchanged in the marginal tables. Estimates of, and test statistics for, these parameters calculated in the marginal tables will, however, in many cases differ from those obtained from the complete table. Conditions under which calculations give the same results may, however, also be stated in terms of the interaction graphs. An undirected graph is said to be reducible if it partitions into three sets of nodes—X, Y, and Z—if Y separates the nodes of X from those of Z and if the nodes of Y are completely connected. If the interaction graph meets the condition of reducibility, it is said to decompose into two components, X + Y and Y + Z. The situation is illustrated in Figure 4, which decomposes into two components, ABCD and CDEF. It is easily seen that reducibility above implies parametric collapsibility with respect to the parameters of X and Z, respectively. It can also be shown, however, that likelihoodbased estimates and test statistics obtained by analysis of the collapsed tables are exactly the same as those obtained from the complete table.

Figure 3. Collapsing the model given in (a) illustrates the separation theorem for A and B (b), and parametric collapsibility with respect to E and F (c)

1.7 Regression Models and Recursive Models So far, the discussion has focused on models for the joint distribution of variables. The models can, however, without any problems, be extended first to multidimensional regression models describing the conditional distribution of a vector of dependent variables given another vector of explanatory variables and, secondly, to block recursive systems of variables. In the first case, the model will be based on independence assumptions relating to either two dependent variables

INTERACTION MODEL

5

associations. For categorical data, the natural measures of association are measures based on the so-called cross product ratios (2). The question therefore reduces to a question of whether or not cross product ratios are constant across different levels of other variables, thus identifying loglinear models as the natural framework within which these problems should be studied.

3

Figure 4. An interaction graph of a reducible model

or one dependent and one independent variable. In the second case, recursive models have to be formulated as a product of separate regression models for each recursive block conditionally given variables in all prior blocks. To distinguish between symmetric and asymmetric relationships edges between variables in different recursive blocks, interaction graphs are replaced by arrows. 2 PARAMETRIC STRUCTURE: HOMOGENEOUS OR HETEROGENEOUS ASSOCIATION The limitations of graphical models for contingency tables lie in the way in which they deal with higher-order interactions. The definition of the graphical models implies that higher-order interactions may exist if more than two variables are completely connected. It is therefore obvious that an analysis of association by graphical models can never be anything but the first step of an analysis of association. The graphical model will be useful in identifying associated variables and marginal models where associations may be studied, but sooner or later one will have to address the question of whether or not these associations are homogeneous across levels defined by other variables and, if not, which variables modify the association. The answer to the question of homogeneity of associations depends on the type of measure that one uses to describe or measure

ORDINAL CATEGORICAL VARIABLES

In the not unusual case of association between ordinal categorical variables, the same types of argument apply against the hierarchical loglinear models as against the graphical models. Loglinear models are basically interaction models for nominal data; and, as such, they will give results that are too crude and too imprecise for ordinal categorical data. The question of whether or not the association between two variables is homogeneous across levels of conditioning variables can, for ordinal variables, be extended to a question of whether or not the association is homogeneous across the different levels of the associated variables. While not abandoning the basic loglinear association structure, the answer to this question depends on the further parameterization of the loglinear terms of the models. We refer to a recent discussion of these problems by Clogg & Shihadeh (3).

4

DISCUSSION

The viewpoint taken here on the formulation of interaction models for categorical data first defines the family of graphical models as the basic type of models for association and interaction structure. Loglinear models are, from this viewpoint, regarded as parametric graphical models, meeting certain assumptions on the nature of associations not directly captured by the basic graphical models. Finally, different types of models for ordinal categorical data represent yet further attempts to meet assumptions relating specifically to the ordinal nature of the variables.

6

INTERACTION MODEL

REFERENCES 1. Agresti, A. (1990). Categorical Data Analysis. Wiley, New York. 2. Altham, P. M. E. (1970). The measurement of association of rows and columns for an r × s contingency table, Journal of the Royal Statistical Society, Series B 32, 63–73. 3. Clogg, C. & Shihadeh, E. S. (1994). Statistical Models for Ordinal Variables. Sage, Thousand Oaks. 4. Cox, D. R. & Wermuth, N. (1996). Multivariate Dependencies. Models, Analysis and Interpretation. Chapman & Hall, London. 5. Darroch, J. N., Lauritzen, S. L. & Speed, T. P. (1980). Markov fields and log-linear models for contingency tables, Annals of Statistics 8, 522–539.

6. Dawid, A. P. (1979). Conditional independence in statistical theory, Journal of the Royal Statistical Society, Series B 41, 1–15. 7. Edwards, D. (1995). Introduction to Graphical Modelling. Springer-Verlag, New York. 8. Lauritzen, S. L. (1996). Graphical Models. Oxford University Press, Oxford. 9. Whittaker, J. (1990). Graphical Models in Applied Multivariate Statistics. Wiley, Chichester.

INTERIM ANALYSES

The alternative, and possibly the most familiar, purpose for interim analyses in a clinical trial is to allow interim assessment of treatment differences. Valid interim comparisons of the treatments and their use in deciding whether to stop a trial will be the main focus of the rest of this article. When using the term interim analysis hereafter, it is assumed that treatment comparisons are being conducted. The traditional approach to conducting phase III clinical trials has been to calculate a single fixed sample size in advance of the study, which depends on specified values of the significance level, power, and the treatment advantage to be detected. Data on all patients are then collected before any formal analyses are performed. Such a framework is logical when observations become available simultaneously, as in an agricultural field trial; but it may be less suitable for medical studies, in which patients are recruited over time and data are available sequentially. Here, results from patients who enter the trial early on are available for analysis while later patients are still being recruited. It is natural to be interested in such results. ‘‘However, the handling of treatment comparisons while a trial is still in progress poses some tricky problems in medical ethics, practical organisation and statistical analysis’’ (see Reference 3, Section 10.1), as discussed below. In methodological terms, the approach presented in this article is known as the frequentist approach and is the most widely used framework for clinical trials. An alternative school of thought, not discussed here but mentioned for completeness, is the Bayesian approach as described by, for example, Spiegelhalter et al. (4).

SUSAN TODD The University of Reading Medical and Pharmaceutical Statistics Research Unit Reading, Berkshire, United Kingdom

1

INTRODUCTION

The term ‘‘interim analysis’’ can, in its broadest sense, be used in relation to any evaluation of data undertaken during an ongoing trial. Whether examination of the data presents ethical and analytical challenges depends on the purpose of the analysis. Some routine monitoring of trial progress, usually blinded to treatment allocation, is often undertaken as part of a phase III trial. This monitoring can range from simple checking of protocol compliance and the accurate completion of record forms to monitoring adverse events in trials of serious conditions so that prompt action can be taken. Such monitoring may be undertaken in conjunction with a data and safety monitoring board (DSMB), established to review the information collected. As no direct comparison of the treatments in terms of their benefit is undertaken at such interim analyses, special methodology is not required. A second purpose of interim analyses is to undertake a sample size review, when the purpose of the interim analysis is to estimate one or more nuisance parameters (for example, σ 2 in the case of normally distributed data), and this information is used to determine the sample size required for the remainder of the trial. Sample size re-estimation based on the estimation of nuisance parameters, particularly on the variance of normally distributed observations, was proposed by Gould and Shih (1). A review of the methodology is given by Gould (2). A sample size review can be undertaken using data pooled over treatments in order to avoid any breaking of blindness. Simulations reported in the papers cited above show that there is negligible effect on the statistical properties of the overall test and so it is usually ignored.

2 OPPORTUNITIES AND DANGERS OF INTERIM ANALYSES The most compelling reason for monitoring trial data for treatment differences at interim analyses is that, ethically, it is desirable to terminate or modify a trial when evidence has emerged that one treatment is clearly

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

INTERIM ANALYSES

superior to the other, which is particularly important when life-threatening diseases are being studied. Alternatively, the data from a trial may support the conclusion that the experimental treatment and the control do not differ by some pre-determined clinically relevant magnitude, in which case it is likely to be desirable, both ethically and economically, to stop the study for futility and divert resources elsewhere. Finally, if information in a trial is accruing more slowly than expected, perhaps because the actual event or response rate observed in a trial is less than that anticipated when the trial was planned, then extension of enrollment may be appropriate, until a large enough sample has been recruited. Unfortunately, multiple analyses of accumulating data lead to problems in the interpretation of results (5). The main problem occurs when significance testing is undertaken at the various interim looks. Even if the treatments are actually equally effective, the more often one analyzes the accumulating data, the greater the chance of eventually and wrongly detecting a difference. Armitage et al. (6) were the first to numerically compute the extent to which the type I error probability (the probability of incorrectly declaring the experimental treatment as different from the control) is increased over its intended level if a standard hypothesis test is conducted at that level at each of a series of interim analyses. They studied the simplified setting of a comparison of normal data with known variance and set the significance level (or type I error probability) for each analysis to be 5%. If just one analysis is performed, then the error rate is 5% as planned. If one interim analysis and one final analysis are performed, the error rate rises to 8%. If four interim analyses and a final analysis are undertaken, this figure is 14%. This inflation continues if more interim analyses are performed. In order to make use of the advantages of monitoring the treatment difference, methodology is required to maintain the overall type I error rate at the planned level. The term ‘‘sequential trial’’ is most commonly used to describe a trial that uses this special methodology. Such trials implement pre-specified stopping rules to accurately maintain error rates.

A second problem associated with the conduct of interim analyses that involve treatment comparisons concerns the final analysis of the trial. When data are inspected at interim looks, the analysis appropriate for fixed sample size studies is no longer valid. Quantities such as P-values, point estimates, and confidence intervals are still well defined, but new methods of calculation are required. If a traditional analysis is performed at the end of a trial that stops because the experimental treatment is found better than control, the P-value will be too small (too significant), the point estimate too large, and the confidence interval too narrow. Again, special techniques are required. 3 THE DEVELOPMENT OF TECHNIQUES FOR CONDUCTING INTERIM ANALYSES It was in the context of quality control experiments that special methods for conducting interim analyses were first implemented. Manufactured items are often inspected one at a time, with a view to accepting or rejecting a batch in terms of its quality. Double sampling, or in the terms of this article, the use of a single interim analysis, was introduced to quality control by Dodge and Romig (7). The Second World War then provided the setting for the development of the sequential probability ratio test by Wald (8) in the United States and by Barnard (9) in the United Kingdom. In this procedure, an interim analysis is conducted after the inspection of every individual item. Sequential medical plans for comparing two sets of binary responses were introduced by Bross (10), and Kilpatrick and Oldham (11) applied the sequential t-test to a comparison of bronchial dilators. By 1960, enough accumulated theory and practice existed for the first edition of Armitage’s book on Sequential Medical Trials. During the 1970’s work in this area prescribed designs whereby traditional test statistics, such as the t-statistic or the chisquared statistic, were monitored after each patient’s response was obtained. Examples can be found in the book by Armitage (12). The principal limitation was, obviously, the need to perform an interim analysis so frequently, which was eventually overcome to

INTERIM ANALYSES

allow interim analyses to be performed after groups of patients. Work by Pocock (13) and O’Brien and Fleming (14) allowed inspections after the responses from predefined sets of patients. A more flexible approach leading on from these early repeated significance tests, referred to as the alpha-spending method, was proposed by Lan and DeMets (15) and extended by Kim and DeMets (16). An alternative flexible method, sometimes referred to as the boundaries approach, encompasses a collection of designs based on straight-line boundaries, which builds on work that has steadily accumulated since the 1940s. This approach is discussed by Whitehead (17), and the best known and most widely implemented design within this framework is the triangular test (18). 4 METHODOLOGY FOR INTERIM ANALYSES In his 1999 paper, Whitehead (19) lists the key ingredients required to conduct a trial incorporating interim analyses for assessing treatment differences: • A parameter that expresses the advan-

tage of the experimental treatment over control, which is an unknown population characteristic about which hypotheses will be posed and of which estimates will be sought. • A statistic that expresses the advantage of experimental over control apparent from the sample of data available at an interim analysis, and a second statistic that expresses the amount of information about the treatment difference contained in the sample. • A stopping rule that determines whether the current interim analysis should be the last, and, if so, whether it is to be concluded that the experimental is better than or worse than control or that no treatment difference has been established. • A method of analysis, valid for the specific design used, giving a P-value and point and interval estimates at the end of the trial.

3

The first two ingredients are common to both fixed sample size and sequential studies, but are worth revisiting for completeness. The second two are solutions to the particular problems of preserving error rates and obtaining a valid analysis on completion of a study in which interim analyses have been conducted. Any combination of choices for the four ingredients is possible, but, largely for historical reasons, particular combinations preferred by authors in the field have been extensively developed, incorporated into software, and used in practice. Each of the four ingredients will now be considered in turn. 4.1 The Treatment Effect Parameter As with a fixed sample size study, the first stage in designing a phase III clinical trial incorporating interim analyses is to establish a primary measure of efficacy. The authority of any clinical trial will be greatly enhanced if a single primary response is specified in the protocol and is subsequently found to show significant benefit of the experimental treatment. The choice of response should depend on such criteria as clinical relevance, familiarity to clinicians, and ease of obtaining accurate measurements. An appropriate choice for the associated parameter measuring treatment difference can then be made, which should depend on such criteria as interpretability, for example, whether a measurement based on a difference or a ratio is more familiar, and precision of the resulting analysis. If the primary response is a continuous measure such as the reduction in blood pressure after 1 month of antihypertensive medication, then the difference in true (unknown) means is of interest. If a binary variable is being considered, such as the occurrence (or not) of deep vein thrombosis following hip replacement, the log-odds ratio may be the parameter of interest. Finally, suppose that in a clinical trial the appropriate response is identified as survival time following treatment for cancer, then a suitable parameter of interest might be the log-hazard ratio. 4.2 Test Statistics for Use at Interim Analyses At each interim analysis, a sequential test monitors a statistic that summarizes the

4

INTERIM ANALYSES

current difference between the experimental treatment and control, in terms of the chosen parameter of interest. If the value of this statistic lies outside some specified critical values, the trial is stopped and appropriate conclusions can be drawn. The timing of the interim looks can be defined directly in terms of number of patients, or more flexibly in terms of information. It should be noted that the test statistic measuring treatment difference may increase or decrease between looks, but the statistic measuring information will always increase. Statisticians have developed flexible ways of conducting sequential trials allowing for the number and the timing of interim inspections. Whitehead (17) describes monitoring a statistic measuring treatment difference known in technical terms as the efficient score and schedules the interim looks in terms of a second statistic approximately proportional to study sample size known as observed Fisher’s information. Jennison and Turnbull (20) employ a direct estimate of the treatment difference itself as the test statistic of interest and record inspections in terms of a function of its standard error. Other choices are also possible. 4.3 Stopping Rules at Interim Analyses As highlighted above, a sequential test compares the test statistic measuring treatment difference with appropriate critical values at each interim analysis. These critical values form a stopping rule or boundary for the trial. At any interim analysis in the trial, if the boundary is crossed, the study is stopped and an appropriate conclusion drawn. If the statistic stays within the boundary, then not enough evidence exists to come to a conclusion at present and a further interim look should be taken. It is possible to look after every patient or to have just one or two interim analyses. When interims are performed after groups of patients, it may be referred to as a ‘‘group sequential trial.’’ A design incorporating inspections after every patient may be referred to as a ‘‘fully sequential test.’’ The advantage of looking after every patient is that a trial can be stopped as soon as an additional patient response results in the boundary being crossed. In

contrast, performing just one or two looks reduces the potential for stopping, and hence delays it. However, the logistics of performing interim analyses after groups of subjects are far easier to manage. In practice, planning for between four and eight interim analyses appears sensible. Once it had been established that a problem existed with inflating the type I error when using traditional tests and the usual fixed sample size critical values, designs were suggested that adjusted for this problem. It is the details of the derivation of the stopping rule that introduces much of the variety of sequential methodology. In any trial, the important issues to focus on are the desirable reasons for stopping or continuing a study at an interim analysis. Reasons for stopping may include evidence that the experimental treatment is obviously better than the control, evidence that the experimental treatment is already obviously worse than the control, or alternatively it may be established that little chance exists of showing that the experimental treatment is better than the control. Reasons for continuing may include belief that a moderate advantage of the experimental treatment is likely and it is desired to estimate this magnitude carefully or, alternatively, evidence may exist that the event rate is low and more patients are needed to achieve power. Criteria such as these will determine the type of stopping rule that is appropriate for the study under consideration. Stopping rules are now available for testing superiority, non-inferiority, equivalence, and even safety aspects of clinical trials. Some designs even aim to deal with both efficacy and safety aspects in a combined stopping rule, but these designs are more complex. As an example, consider a clinical trial conducted by the Medical Research Council Renal Cancer Collaborators between 1992 and 1997 (21). Patients with metastatic renal carcinoma were randomly assigned to treatment with either the hormone therapy, oral medroxyprogesterone acetate (MPA), or the biological therapy, interferon-α. The primary endpoint in the study was survival time and the treatment difference was measured by the log-hazard ratio. It was decided that if a difference in 2-year survival from 20%

INTERIM ANALYSES

on MPA to 32% on interferon-α (log-hazard ratio −0.342) was present, then a significant treatment difference at the two-sided 5% significance level should be detected with 90% power. The use of interferon-α was experimental and this treatment is known to be both costly and toxic. Consequently, its benefits over MPA needed to be substantial to justify its wider use. A stopping rule was required to satisfy the following two requirements: early stopping if data showed a clear advantage of interferon-α over MPA and early stopping if data showed no worthwhile advantage of interferon-α, which suggested use of an asymmetric stopping rule. Such a rule handles both of these aspects. The design chosen was the triangular test (17), similar in appearance to the stopping rule in Fig. 1. Interim analyses were planned every 6 months from the start of the trial. The precise form of the stopping rule was defined, as is the sample size in a fixed sample size trial, by consideration of the significance level, power, and desired treatment advantage, with reference to the primary endpoint. 4.4 Analysis following a Sequential Trial Once a sequential trial has stopped, a final analysis should be performed. The interim analyses determine only whether stopping should take place; they do not provide a complete interpretation of the data. An appropriate final analysis must take account of the fact that a sequential design was used. Unfortunately, many trials that have been terminated at an interim analysis are finally reported with analyses that take no statistical account of the inspections made (22). In a sequential trial, the meaning and interpretation of data summaries such as significance levels, point estimates, and confidence intervals remain as for fixed sample size trials. However, various alternative valid methods of calculation have been proposed. These methods can sometimes lead to slightly different results when applied to the same set of data. The user of a computer package may accept the convention of the package and use the resulting analysis without being concerned about the details of calculation. Those who wish to develop a deeper understanding

5

of statistical analysis following a sequential trial are referred to Chapter 5 of Whitehead (17) and Chapter 8 of Jennison and Turnbull (23). The generally accepted method is based on orderings methodology, whereby potential outcomes in a trial are ordered by degree of support for the alternative hypothesis. The original, and most successful, form of ordering was introduced by Fairbanks and Madsen (24) and explored further by Tsiatis et al. (25). 5 AN EXAMPLE: STATISTICS FOR LAMIVUDINE The effectiveness of lamivudine for preventing disease progression among patients with chronic hepatitis B and advanced fibrosis or cirrhosis is unknown and prompted the conduct of a large survival study (26). An efficient trial methodology for reaching a reliable conclusion with as few subjects as possible was required. The annual rate of disease progression was assumed to be 20% for placebo and a reduction of one-third (to 13.3%) for the lamivudine group was taken as a clinically relevant treatment effect. It was desired to detect this with power 0.9. A significance level of 0.05 was specified. When the objectives of the trial were considered in detail, interim analyses using an appropriate stopping rule were planned. The methodology followed was the boundaries approach, as discussed by Whitehead (17), and the triangular test was selected as an appropriate design. Clinically compensated chronic hepatitis B patients with histologically confirmed fibrosis or cirrhosis were randomized 2:1 to receive lamivudine (100 mg/day) or placebo for up to 5 years. Overall, 651 patients were randomized at 41 sites in nine Asia-Pacific countries, 436 to lamivudine and 215 to placebo. These people were then followed up for evidence of disease progression. An independent data and safety monitoring board was established to study the progress of the trial at interim analyses. At the first interim analysis, 36 patients had experienced disease progression. The first point plotted on Fig. 1 (x) represents those data. The statistic Z signifies the advantage seen so far on lamivudine and is the efficient score for the log-hazard ratio.

6

INTERIM ANALYSES

Z 25 20 15 10 5 0 −5

10

20

30

40

50

60

70

80

−10

90 V

−15 Figure 1. Sequential plot for the trial of lamivudine. Z: Efficient score statistic measuring the advantage of lamivudine over placebo. V: Statistic measuring the information on which the comparison is based.

Here, Z is calculated adjusting for 5 covariates, center, gender, baseline fibrosis staging score, baseline Childs-Pugh score, and baseline ALT values. An unadjusted Z would be the usual log-rank statistic for survival data. The statistic V measures the information on which the comparison is based, which is the variance of Z and is approximately one-quarter the number of events. The inner dotted boundaries, known as the Christmas tree correction for discrete looks, form the stopping boundary: reach this boundary and the trial is complete. Crossing the upper boundary results in a positive trial conclusion. At the second interim analysis, a total of 67 patients had experienced disease progression. The upper boundary was reached and the trial was stopped. When the results from data that had accumulated between the second interim analysis and confirmation of stopping were added, the final ‘‘overunning’’ point was added to the plot. A final overrunning analysis of the data was conducted using the computer package PEST. The P-value obtained was 0.0008, a highly significant result in favor of lamivudine. By using a series of interim looks, the design allowed a strong positive conclusion to be drawn after only 71 events had been observed.

6

INTERIM ANALYSES IN PRACTICE

Interim analyses are frequently being implemented in clinical trials. Early examples can

be found in the proceedings of two workshops, one on practical issues in data monitoring sponsored by the U.S. National Institutes of Health (NIH) held in 1992 (27) and the other on early stopping rules in cancer clinical trials held at Cambridge University in 1993 (28). The medical literature also demonstrates the use of interim analyses. Examples of such studies include trials of lithium gamolenate in pancreatic cancer (29), of viagra in erectile disfunction (30), and of implanted defibrilators in coronary heart disease (31), as well as the example detailed in Section 5 above and many others. Two books dealing exclusively with the implementation of sequential methods in clinical trials are those by Whitehead (17) and Jennison and Turnbull (23). In addition, three major commercial software packages are currently available. The package PEST (32) is based on the boundaries approach. The package EaSt (33) implements the alpha-spending boundaries of Wang and Tsiatis (34) and Pampallona and Tsiatis (35). An addition to the package S-Plus is the S + SeqTrial module (36). If a trial is to be a comparison of two treatments in respect to a single primary endpoint, with the objective of discovering whether one treatment is superior, non-inferior, or equivalent to the other, then it is extremely likely that a suitable sequential method exists, which means that infeasibility and unfamiliarity are no longer valid reasons for avoiding interim analyses and stopping rules in such trials, if ethical or economic purposes would be served by them.

INTERIM ANALYSES

When planning to include interim analyses in any clinical trial, the implications of introducing a stopping rule need to be thought out carefully in advance of the study. In addition, all parties involved in the trial should be consulted on the choice of clinically relevant difference, specification of an appropriate power requirement, and the selection of a suitable stopping rule. As part of the protocol for the study, the operation of any sequential procedure should be clearly described in the statistical section. Decision making as part of a sequential trial is both important and time sensitive. A decision taken to stop a study not only affects the current trial, but also is likely to affect future trials planned in the same therapeutic area. However, continuing a trial too long puts participants at unnecessary risk and delays the dissemination of important information. It is essential to make important ethical and scientific decisions with confidence. Wondering whether the data supporting interim analyses are accurate and up-to-date is unsettling and makes the decision process harder. It is, therefore, necessary for the statistician performing the interim analyses to have timely and accurate data. Unfortunately, a trade-off exists—it takes time to ensure accuracy. Potential problems can be alleviated if data for interim analyses are reported separately from the other trial data, as part of a fast-track system. Less data means that they can be validated more quickly. If timeliness and accuracy are not in balance, not only may real-time decisions be made on old data, but more seriously, differential reporting may lead to inappropriate study conclusions. If a DSMB is appointed, one of their roles should be to scrutinize any proposed sequential stopping rule before the start of the study and to review the protocol in collaboration with the trial Steering Committee. The procedure for undertaking the interim analyses should also be finalized in advance of the trial start-up. The DSMB would then review results of the interim analyses as they are reported. Membership of the DSMB and its relationship with other parties in a clinical trial has been considered in Reference 27 and by Ellenberg et al. (37). It is important that the interim results of an ongoing trial are not

7

circulated widely as it may have an undesirable effect on the future progress of the trial. Investigators’ attitudes will clearly be affected by whether a treatment looks good or bad as the trial progresses. It is usual for the DSMB to be supplied with full information and, ideally, the only other individual to have knowledge of the treatment comparison would be the statistician who performs the actual analyses. 7

CONCLUSIONS

Use of interim analyses in phase III clinical trials is not new, but it is probably the more recent theoretical developments, together with availability of software, which have precipitated their wider use. The methodology is flexible as it enables choice of a stopping rule from a number of alternatives, allowing the trial design to meet the study objectives. One important point is that a stopping rule should not govern the trial completely. If external circumstances change the appropriateness of the trial or assumptions made when choosing the design are suspected to be false, it can and should be overridden, with the reasons for doing so carefully documented. Methodology for conducting a phase III clinical trial sequentially has been extensively developed, evaluated, and documented. Error rates can be accurately preserved and valid inferences drawn. It is important that this fact is recognized and that individuals contemplating the use of interim analyses conduct them correctly. Regulatory authorities, such as the Food and Drug Administration (FDA), do not look favorably on evidence from trials incorporating unplanned looks at data. In the United States, the FDA (38) published regulations for NDAs, which included the requirement that the analysis of a phase III trial ‘‘assess . . . the effects of any interim analyses performed.’’ The FDA guidelines were updated by publication of ‘‘ICH Topic E9: Notes for Guidance on Statistical Principles for Clinical Trials’’ (39). Section 3 of this document discusses group sequential designs and Section 4 covers trial conduct including trial monitoring, interim analysis, early stopping, sample size adjustment, and the role of an independent DSMB. With such

8

INTERIM ANALYSES

acknowledgement from regulatory authorities, interim analyses are likely to become even more commonplace. REFERENCES 1. A. L. Gould and W. J. Shih, Sample size re-estimation without unblinding for normally distributed data with unknown variance. Commun. Stat. – Theory Meth. 1992; 21: 2833–3853. 2. A. L. Gould, Planning and revising the sample size for a trial. Stat. Med. 1995; 14: 1039–1051. 3. S. J. Pocock, Clinical Trials: A Practical Approach. New York: Wiley, 1983. 4. D. J. Spiegelhalter, K. R. Abrams, and J. P. Myles, Bayesian Approaches to Clinical Trials and Health-Care Evaluation. Chichester: Wiley, 2004. 5. K. McPherson, Statistics: the problem of examining accumulating data more than once. N. Engl. J. Med. 1974; 28: 501–502. 6. P. Armitage, C. K. McPherson, and B. C. Rowe, Repeated significance tests on accumulating data. J. Royal Stat. Soc. Series A 1969; 132: 235–244. 7. H. F. Dodge and H. C. Romig, A method of sampling inspection. Bell Syst. Tech. J. 1929; 8: 613–631. 8. A. Wald, Sequential Analysis. New York: Wiley, 1947. 9. G. A. Barnard, Sequential test in industrial statistics. J. Royal Stat. Soc. 1946;(Suppl 8): S1–S26. 10. I. Bross, Sequential medical plans. Biometrics 1952; 8: 188–205. 11. G. S. Kilpatrick and P. D. Oldham, Calcium chloride and adrenaline as bronchial dilators compared by sequential analysis. Brit. Med. J. 1954; ii: 1388–1391. 12. P. Armitage, Sequential Medical Trials, 2nd ed. Oxford: Blackwell, 1975. 13. S. J. Pocock, Group sequential methods in the design and analysis of clinical trials. Biometrika 1977; 64: 191–199. 14. P. C. O’Brien and T. R. Fleming, A multiple testing procedure for clinical trials. Biometrics 1979; 35: 549–556. 15. K. K. G. Lan and D. L. DeMets, Discrete sequential boundaries for clinical trials. Biometrika 1983; 70: 659–663. 16. K. Kim and D. L. DeMets, Design and analysis of group sequential tests based on the type I

error spending rate function. Biometrika 1987; 74: 149–154. 17. J. Whitehead, The Design and Analysis of Sequential Clinical Trials, rev. 2nd ed. Chichester: Wiley, 1997. 18. J. Whitehead, Use of the triangular test in sequential clinical trials. In: J. Crowley (ed.), Handbook of Statistics in Clinical Oncology. New York: Dekker, 2001, pp. 211–228. 19. J. Whitehead, A unified theory for sequential clinical trials. Stat. Med. 1999; 18: 2271–2286 20. C. Jennison and B. W. Turnbull, Group sequential analysis incorporating covariate information. J. Amer. Stat. Assoc. 1997; 92: 1330–1341. 21. Medical Research Council Renal Cancer Collaborators, Interferon-α and survival in metastatic renal carcinoma: early results of a randomised controlled trial. Lancet 1999; 353: 14–17. 22. K. M. Facey and J. A. Lewis, The management of interim analyses in drug development. Stat. Med. 1998; 17: 1801–1809. 23. C. Jennison and B. W. Turnbull, Group Sequential Methods with Applications to Clinical Trials. Boca Raton, FL: Chapman and Hall/CRC, 2000. 24. K. Fairbanks and R. Madsen, P values for tests using a repeated significance test design. Biometrika 1982; 69: 69–74. 25. A. A. Tsiatis, G. L. Rosner, and C. R. Mehta, Exact confidence intervals following a group sequential test. Biometrics 1984; 40: 797–803. 26. Y. F. Liaw et al. on behalf of the CALM study group, Effects of lamivudine on disease progression and development of hepatocellular carcinoma in advanced chronic hepatitis B: a prospective placebo-controlled clinical trial. N. Engl. J. Med. 2004; 351: 1521–1531. 27. S. Ellenberg, N. Geller, R. Simon, and S. Yusuf (eds.), Proceedings of ‘Practical issues in data monitoring of clinical trials’. Stat. Med. 1993; 12: 415–616. 28. R. L. Souhami and J. Whitehead (eds.), Workshop on early stopping rules in cancer clinical trials. Stat. Med. 1994; 13: 1289–1499. 29. C. D. Johnson et al., Randomized, dose-finding phase III study of lithium gamolenate in patients with advanced pancreatic adenocarcinoma. Brit. J. Surg. 2001; 88: 662–668. 30. F. A. Derry et al., Efficacy and safety of oral sildenafil (viagra) in men with erectile dysfunction caused by spinal cord injury. Neurology 1998; 51: 1629–1633.

INTERIM ANALYSES 31. A. J. Moss et al., Improved survival with implanted defibrillator in patients with coronary disease at high risk of ventricular arrhythmia. N. Engl. J. Med. 1996; 335: 1933–1940. 32. MPS Research Unit, PEST 4: Operating Manual. Reading, UK: The University of Reading, 2000. 33. Cytel Software Corporation, EaSt 2000: A Software Package for the Design and Interim Monitoring of Group-Sequential Clinical Trials. Cambridge, MA: Cytel Software Corporation, 2000. 34. S. K. Wang and A. A. Tsiatis, Approximately optimal one-parameter boundaries for group sequential trials. Biometrics 1987; 43: 193–199. 35. S. Pampallona and A. A. Tsiatis, Group sequential designs for one-sided and twosided hypothesis testing with provision for early stopping in favor of the null hypothesis. J. Stat. Plan. Inference 1994; 42: 19–35.

9

36. MathSoft Inc., S-Plus 2000. Seattle, WA: MathSoft Inc., 2000. 37. S. Ellenberg, T. R. Fleming, and D. L. DeMets, Data Monitoring Committees in Clinical Trials: A Practical Perspective. Chichester: Wiley, 2002. 38. US Food and Drug Administration Guideline for the Format and Content of the Clinical and Statistical Sections of an Application. Rockville, MD: FDA. http://www.fda.gov/cder/guidance/statnda.pdf 1988. 39. International Conference on Harmonisation Statistical Principles for Clinical Trials, Guideline E9. http://www.ich. org/LOB/media/MEDIA485.pdf 1998.

INTERIM CLINICAL TRIAL/STUDY REPORT An Interim Clinical Trial/Study Report is a report of intermediate results and their evaluation based on analyses performed during the course of a trial.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

INTERNATIONAL CONFERENCE ON HARMONISATION OF TECHNICAL REQUIREMENTS FOR REGISTRATION OF PHARMACEUTICALS FOR HUMAN USE (ICH) The International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) is a cooperative effort between the drug regulatory authorities and innovative pharmaceutical company professional organizations of the European Union, Japan, and the United States to reduce the need to duplicate the testing conducted during the research and development of new medicines. Through harmonization of the technical guidelines and requirements under which drugs for human use are approved within the participating nations, ICH members seek more economical use of human, animal, and material resources and the elimination of delay in availability of new drugs, while maintaining the quality, safety, and effectiveness of regulated medicines.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/handbook/) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

International Studies of Infarct Survival (ISIS) The ISIS began in 1981 as a collaborative worldwide effort to evaluate the effects of several widely available and practical treatments for acute myocardial infarction (MI). The ISIS Collaborative Group randomized more than 134 000 patients into four large simple trials assessing the independent and synergistic effects of beta-blockers, thrombolytics, aspirin, heparin, converting enzyme inhibitors, oral nitrates, and magnesium in the treatment of evolving myocardial acute infarction (Table 1). More than 20 countries participated in these trials, which were coordinated worldwide by investigators in Oxford, England.

ISIS-1: Atenolol in Acute MI [1] Beta-blocking agents reduce the heart rate and blood pressure, as well as their product, inhibit the effects of catecholamines, and increase thresholds for ventricular fibrillation. Thus, it is not surprising that beta-blockers were among the first agents to be evaluated in randomized trials of evolving acute MI. Even by 1981, the available trials of beta-blocking agents for acute infarction were too small to demonstrate a significant benefit. However, based on an overview of the available evidence (see Meta-analysis of Clinical Trials), it was judged that the prevention of even one death per 200 patients treated with beta-blockers (see Number Needed to Treat (NNT)) would represent a worthwhile addition to usual care. Unfortunately, detecting such an effect would require the randomization of over 15 000 patients. It was toward this end that the First International Study of Infarct Survival (ISIS-1) trial was formed. Table 1 Trial

In a collaborative effort involving 245 coronary care units in 11 countries, the ISIS-1 trial randomized 16 027 patients with suspected acute MI to a regimen of intravenous atenolol versus no betablocker therapy. Patients assigned to active treatment received an immediate intravenous injection of 5–10 mg atenolol, followed by 100 mg/day orally for seven days. Similar agents were avoided in those assigned at random to no beta-blocker therapy unless it was believed to be medically indicated. As in the subsequent ISIS collaborations, all other treatment decisions were at the discretion of the responsible physician. During the seven-day treatment period in which atenolol was given, vascular mortality was significantly lower in the treated group (3.89% vs. 4.57%, P < 0.04), representing a 15% mortality reduction. Almost all of the apparent benefit was observed in days 0 to 1 during which time there were 121 deaths in the atenolol group as compared with 171 deaths in the control group. The early mortality benefit attributable to atenolol was maintained at 14 days and at the end of one year follow-up (10.7% atenolol vs. 12.0% control). Treatment did not appear to decrease infarct size substantially, although the ability of a large and simple trial such as ISIS-1 to assess such a reduction was limited. Despite its large size, the 95% confidence limits of the risk reductions associated with atenolol in ISIS-1 were wide and included relative risk reductions between 1% and 25%. However, an overview that included ISIS1 and 27 smaller completed trials of beta-blockade suggested a similar sized mortality reduction (14%). When a combined endpoint of mortality, nonfatal cardiac arrest and nonfatal reinfarction was considered from all available trials, the 10%–15% reduction persisted with far narrower confidence limits. Taken together, these data suggest that early treatment of 200 acute MI patients with beta-blocker therapy

The International Studies of Infarct Survival (ISIS) Year completed

ISIS-1 ISIS-2

1985 1988

ISIS-3

1991

ISIS-4

1993

Agents studied Atenolol vs. control Streptokinase vs. placebo Aspirin vs. placebo Streptokinase vs. tPA vs. APSAC Aspirin + SC heparin vs. aspirin Captopril vs. placebo oral mononitrate vs. placebo Magnesium vs. control

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

Patients randomized 16 027 17 187 41 299 58 050

2

International Studies of Infarct Survival (ISIS)

would lead to avoidance of one reinfarction, one cardiac arrest, and one death during the initial seven-day period. Unfortunately, beta-blocker use in the setting of acute MI remains suboptimal with utilization rates ranging between 30% in the US to less than 5% in the UK. This underutilization appears related in part to poor physician education. In the GUSTO1 trial, beta-blockers were encouraged by the study protocol and almost 50% of all patients received the drugs without any apparent increase in adverse effects.

ISIS-2: Streptokinase and Aspirin in Acute MI [2] As with beta-blockers, data from randomized trials of thrombolytic therapy completed prior to 1985 did not yield truly reliable results. Indeed, the largest of the early studies enrolled 750 patients, a totally inadequate sample size to detect the most plausible 20%–25% reduction in mortality. Given this situation, the Second International Study of lnfarct Survival (ISIS-2) was designed to test directly in a single randomized, double-blind, placebo-controlled trial (see Blinding or Masking) the risks and benefits of streptokinase and aspirin in acute MI. To accomplish this goal, the ISIS-2 collaborative group randomized 17 187 patients presenting within 24 hours of symptom onset using a 2 × 2 factorial design to one of four treatment groups: 1.5 million units of intravenous streptokinase over 60 minutes; 162.5 mg/day of oral aspirin for 30 days; both active treatments; or neither. In brief, the primary endpoint (see Outcome Measures in Clinical Trials) of the trial, total vascular mortality, was reduced 25% by streptokinase alone (95% Cl, −32 to −18, P < 0.0001) and 23% by aspirin alone (95% CI, −30% to −15%, P < 0.00001). Patients allocated to both agents had a 42% reduction in vascular mortality (95% Cl, −50 to −34, P < 0.00001), indicating that the effects of streptokinase and aspirin are largely additive. When treatment was initiated within six hours of the onset of symptoms, the reduction in total vascular mortality was 30% for streptokinase, 23% for aspirin, and 53% for both active agents. For aspirin, the mortality benefit was similar when the drug was started 0–4 hours (25%), 5–12 hours (21%), or 13–24 hours (21%) after the onset of

clinical symptoms. Aspirin use also resulted in highly significant reductions for nonfatal reinfarction (49%) and nonfatal stroke (46%). As regards side-effects, for bleeds requiring transfusion, there was no significant difference between the aspirin and placebo groups (0.4% vs. 0.4%), although there was a small absolute increase of minor bleeds among those allocated to aspirin (0.6%, P < 0.01). For cerebral hemorrhage, there was no difference between the aspirin and placebo groups. For streptokinase, those randomized within four hours of pain onset experienced the greatest mortality reduction, although statistically significant benefits were present for patients randomized throughout the 24 hour period. As expected, there was an excess of confirmed cerebral hemorrhage with streptokinase (7 events vs. 0; 2P < 0.02), all of which occurred within one day of randomization. Reinfarction was slightly more common among those assigned streptokinase alone, but this difference was not statistically significant. Furthermore, aspirin abolished the excess reinfarction attributable to streptokinase. In addition to demonstrating the independent as well as synergistic effects of streptokinase and aspirin, ISIS-2 also supplied important information concerning which patients to treat. Because the ISIS-2 entry criteria were broad, the trial included the elderly, patients with left bundle branch block, and those with inferior as well as anterior infarctions. In each of these subgroups, clear mortality reductions were demonstrated. Thus, in addition to changing radically the premise that thrombolysis should be avoided in patients already on aspirin, the ISIS-2 trial was largely responsible for widening the eligibility criteria for patients who would benefit from thrombolytic therapy.

ISIS-3: Streptokinase vs. APSAC vs. tPA and Subcutaneous Heparin vs. No Heparin in Acute MI [3] While ISIS-2 (streptokinase), the first Gruppo Italiano per lo Studio della Sopravvivenza nell’Infarto miocardico (GISSI-1, streptokinase), the APSAC Intervention Mortality Study (AIMS, anisoylated plasminogen streptokinase activator complex [APSAC]), Anglo–Scandinavian Study of Early Thrombolysis (ASSET, tissue plasminogen activator [tPA]) and ISIS-2 all documented clear mortality benefits for

International Studies of Infarct Survival (ISIS) thrombolysis, they did not provide information that allowed for directly comparing these agents. It was also unclear whether patients given aspirin would further benefit from the addition of heparin. These questions were the focus of the Third International Study of Infarct Survival (ISIS-3). In brief, the ISIS-3 collaborative group randomized 41 299 patients to streptokinase, APSAC, and tPA. Patients presenting within 24 hours of the onset of evolving acute MI and with no clear contraindication to thrombolysis were assigned randomly to IV streptokinase (1.5 MU over one hour), IV tPA (duteplase, 0.50 million U/kg over four hours), or IV APSAC (30 U over three minutes). All patients received daily aspirin (162.5 mg), with the first dose crushed or chewed in order to achieve a rapid clinical antithrombotic effect. In addition, half were randomly assigned to receive subcutaneous heparin (12 500 IU twice daily for seven days), beginning four hours after randomization. ISIS-3 demonstrated no differences in mortality between the three thrombolytic agents. Specifically, among the 13 780 patients randomized to streptokinase, there were 1455 deaths (10.5%) within the initial 35-day follow-up period as compared with 1448 deaths (10.6%) among the 13 773 patients randomized to APSAC and 1418 deaths (10.3%) among the 13 746 randomized to tPA. Long-term survival was also virtually identical for the three agents at both three and six months. With regard to in-hospital clinical events, cardiac rupture, cardiogenic shock, heart failure requiring treatment, and ventricular fibrillation were similar for the three agents. For nonfatal reinfarction, there was a reduction with tPA, while streptokinase and APSAC allocated patients had higher rates of allergy and hypotension requiring treatment. Streptokinase produced fewer noncerebral bleeds than either APSAC or tPA. While there were no major differences between thrombolytic agents in terms of lives saved or serious in-hospital clinical events, significant differences were found in ISIS-3 for rates of total stroke and cerebral hemorrhage. Specifically, there were 141 total strokes in the streptokinase group as compared with 172 and 188 in the APSAC and tPA groups, respectively. For cerebral hemorrhage there were 32 events (two per 1000) in the streptokinase group as compared with 75 (five per 1000) in the APSAC group and 89 (seven per 1000) in the tPA group. While

3

the absolute rates for cerebral hemorrhage for all three agents was low, this apparent advantage for streptokinase was highly statistically significant (P < 0.0001 for streptokinase vs. APSAC, P < 0.00001 for streptokinase vs. tPA). With regard to the addition of delayed subcutaneous heparin to thrombolytics there was no reduction in the prespecified endpoint of 35 day mortality. During the scheduled seven day period of heparin use, there were slightly fewer deaths in the aspirin plus heparin group compared with the aspirin group alone, a difference of borderline significance. There was, however, a small but significant excess of strokes deemed definite or probable cerebral hemorrhages among those allocated aspirin plus heparin (0.56% vs. 0.40%, P < 0.05). In contrast, reinfarction was more common among those randomized to aspirin alone as compared with those receiving aspirin plus subcutaneous heparin.

ISIS-4: Angiotensin Converting Enzyme Inhibition, Nitrate Therapy, and Magnesium in Acute MI [4] In 1991 the ISIS collaboration chose to investigate several other promising but unproven approaches to the treatment of acute MI. Specifically, the Fourth International Study of Infarct Survival (ISIS-4) sought to examine treatment strategies that would benefit both high- and low-risk patients presenting with acute MI, not simply those who are eligible for thrombolysis. To attain this goal, the ISIS collaborative group chose to study three promising agents: a twice daily dose of the angiotensin converting enzyme (ACE) inhibitor captopril for 30 days, a once daily dose of controlled release mononitrate for 30 days, and a 24-hour infusion of intravenous magnesium. As was true in each of the preceding ISIS trials, the available data were far too limited to allow reliable clinical recommendations concerning these therapies. For example, while ACE inhibiting agents had been shown to be successful in reducing mortality in patients with congestive heart failure and in patients a week or two past acute infarction, it was unclear whether these agents provided a net benefit for all patients in the setting of evolving acute MI. Similarly, while nitrates were often used in evolving MI because of their ability to reduce myocardial

4

International Studies of Infarct Survival (ISIS)

afterload and potentially limit infarct size, barely 3000 patients had received intravenous nitroglycerin in randomized trials and even fewer patients had been studied on oral nitrate preparations. Finally, because of its effects on calcium regulation, arrhythmia thresholds, and tissue preservation, magnesium therapy had often been considered as an adjunctive therapy for acute infarction even though no data from a randomized trial of even modest size had been available. Based on statistical overviews, the ISIS investigators estimated that each of these therapies had the potential to reduce mortality in acute infarction by as much as 15%–20%. However, because many patients presenting with acute infarction were treated with thrombolytic therapy and aspirin, the estimated mortality rates at one month were estimated to be as low as 7%–8%. Thus, to assess reliably whether these potentially important clinical effects were real required the randomization of a very large number of patients, perhaps as many as 60 000. To achieve this goal, a 2 × 2 × 2 factorial design was employed in which patients were randomized first to captopril or captopril placebo, then to mononitrate or mononitrate placebo, and then to magnesium or magnesium control. Thus, it was possible in the trial for any given patient to receive all three active agents, no active agents, or any combination.

Captopril Use of the ACE inhibitor captopril was associated with a significant 7% decrease in five-week mortality (2088 [7.19%] deaths among patients assigned to captopril vs. 2231 [7.69%] deaths among those assigned to placebo), which corresponds to an absolute difference of 4.9 ± 2.2 fewer deaths per 1000 patients treated for one month. The absolute benefits appeared to be larger (possibly as high as 10 fewer deaths per 1000) in some higher-risk groups, such as those presenting with heart failure or a history of MI. The survival advantage appeared to be maintained at 12 months. In terms of side-effects, captopril produced no excess of deaths on days 0–1, even among patients with low blood pressure at entry. It was associated with an increase of 52 patients per 1000 in hypotension considered severe enough to require termination of study treatment, of five per 1000 in reported cardiogenic shock, and of five per 1000 in some degree of renal dysfunction.

Mononitrate Use of mononitrate was not associated with any significant improvements in outcomes. There was no significant reduction in overall five-week mortality, nor were there reductions in any subgroup examined (including those not receiving short-term nonstudy intravenous or oral nitrates at entry). Continued follow-up did not indicate any later survival advantage. Somewhat fewer deaths on days 0–1 were reported among individuals allocated to active treatment, which is reassuring about the safety of using nitrates early in evolving acute MI. The only significant side-effect of the mononitrate regimen was an increase in hypotension of 15 per 1000 patients.

Magnesium As with mononitrate, use of magnesium was not associated with any significant improvements in outcomes, either in the entire group or any subgroups examined (including those treated early or late after symptom onset or in the presence or absence of fibrinolytic or antiplatelet therapies, or those at high risk of death). Further follow-up did not indicate any later survival advantage. In contrast to some previous small trials, there was a significant excess of heart failure with magnesium of 12 patients per 1000, as well as an increase of cardiogenic shock of five patients per 1000 during or just after the infusion period. Magnesium did not appear to have a net adverse effect on mortality on days 0–1. In terms of side-effects, magnesium was associated with an increase of 11 patients per 1000 in hypotension considered severe enough to require termination of the study treatment, of three patients per 1000 in bradycardia, and of three patients per 1000 in a cutaneous flushing or burning sensation. Because of its size, ISIS-4 provided reliable evidence about the effects of adding each of these three treatments to established treatments for acute MI. Collectively, GISSI-3, several smaller studies, and ISIS-4 have demonstrated that, for a wide range of patients without clear contraindications, ACE inhibitor therapy begun early in evolving acute MI prevents about five deaths per 1000 in the first month, with somewhat greater benefits in higher-risk patients. The benefit from one month of ACE inhibitor therapy persists for at least the first year. Oral nitrate

International Studies of Infarct Survival (ISIS) therapy, while safe, does not appear to produce a clear reduction in one-month mortality. Finally, intravenous magnesium was ineffective at reducing one-month mortality.

Conclusion Because of their simplicity, large size, and strict use of mortality as the primary endpoint, the ISIS trials have played a critical substantive role in establishing rational treatment plans for patients with acute MI. Methodologically, they have clearly demonstrated the utility of large simple randomized trials. Three principles guided the design and conduct of the ISIS trials. The first was the belief that a substantial public health benefit would result from the identification of effective, widely practical treatment regimens that could be employed in almost all medical settings, as opposed to those that can be administered only at specialized tertiary care facilities. For this reason, the ISIS investigations focused on strategies to decrease mortality which, in and of themselves, did not require cardiac catheterization or other invasive procedures for either diagnostic or therapeutic purposes. The second principle was that the benefits of truly effective therapies would be applicable to a wide spectrum of patients with diverse clinical presentations. Thus, the entry criteria for the ISIS trials were intentionally broad and designed to mimic the reality all health care providers encounter when deciding whether or not to initiate a given treatment plan. This is one reason that the ISIS trials focused on evolving acute MI in the view of the responsible physician. The third and perhaps most important principle was that most new therapies confer small to moderate benefits, on the order of 10%–30%. While such benefits on mortality are clinically very meaningful, these effects can be detected reliably only by randomized trials involving some tens of thousands of patients. Thus, the ISIS protocols were

5

streamlined to maximize randomization and minimize interference with the responsible physician’s choice of nonprotocol therapies and interventions. Nonetheless, by selectively collecting the most important entry and follow-up variables that relate directly to the efficacy or adverse effects of the treatment in question, the ISIS trials yielded reliable data for providing a rational basis for patient care. By limiting paperwork and not mandating protocol-driven interventions, the ISIS approach proved to be remarkably cost-effective. Indeed, the large ISIS trials were conducted at a small fraction of the usual cost of other smaller trials which, because of their inadequate sample sizes, failed to demonstrate either statistically significant effects or informative null results.

References [1]

[2]

[3]

[4]

ISIS-1 (First International Study of lnfarct Survival) Collaborative Group (1986). Randomised trial of intravenous atenolol among 16027 cases of suspected acute myocardial infarction: ISIS-1, Lancet 2, 57–65. ISIS-2 (Second International Study of lnfarct Survival) Collaborative Group (1998). Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17187 cases of suspected acute myocardial infarction: ISIS-2, Lancet 2, 349–360. ISIS-3 (Third International Study of lnfarct Survival) Collaborative Group (1992). ISIS-3: Randomised comparison of streptokinase vs. tissue plasminogen activator vs. anistreplase and of aspirin plus heparin vs. aspirin alone among 41299 cases of suspected acute myocardial infarction: ISIS-3. Lancet 339, 75–70. ISIS-4 (Fourth International Study of Infarct Survival) Collaborative Group (1995). ISIS-4: a randomised factorial trial assessing early oral captopril, oral mononitrate, and intravenous magnesium sulphate in 58 050 patients with suspected acute myocardial infarction: ISIS4. Lancet 345, 669–685.

CHARLES H. HENNEKENS & P.J. SKERRETT

Xij = ξ i +  ij where in the first treatment group, T1, the mean of ξ i were µ1 and in the other treatment group, T2, µ2 . The assumptions underlying the two-sample t-test hold. With N subjects randomly assigned to T1 and T2, the power to detect a deviation from the null hypothesis of equivalence of T1 and T2 using outcome measure O is the effect size:

INTER-RATER RELIABILITY HELENA CHMURA KRAEMER Stanford University, Palo Alto, California

1

DEFINITION

The classic model that underlies reliability stipulates that a rating of unit i, i = 1,2, . . . can be expressed as Xij = ξ i +  ij , where ξ i is that unit’s ‘‘true value’’ (i.e., that free of rater error), and εij is error made by the jth independent rater sampled from the population of raters (1). The inter-rater reliability coefficient is defined as:

δ(ρ) =

(µ1 − µ2 ) √ (µ1 − µ2 ) √ = ρ = ρδ (σx2 )1/2 σξ

(1)

Because the sample size necessary to achieve specified power for a two-sample t-test is inversely related to the square of the effect size, the sample size necessary to achieve given power is inversely related to the reliability coefficient. Thus, in a situation in which one would need 100 subjects per group with a perfectly reliability measure, one would need 200 subjects if the measure had a reliability of .5, and 500 subjects if the measure had a reliability of .2. Similarly, suppose two variables X and Y were to be correlated. Then, the product moment correlation coefficient between X and Y can be shown to be:

ρ = σ 2 ξ/σ 2 X where σ 2 ξ is the variance of the ξ i in the population of interest, and σ 2 X that of a single observation per subject. Thus, in a sense, reliability relates to signal-to-noise ratio where σ 2 ξ relates to ‘‘signal’’ and σ 2 X combines ‘‘signal’’ and ‘‘noise.’’ According to this definition, the reliability coefficient is zero if and only if subjects in the population are homogeneous in whatever X measures. This situation should almost never pertain when considering measures for use in randomized clinical trials. Consequently, testing the null hypothesis that ρ = 0 is virtually never of interest, although admittedly it is often observed in the research literature. Instead, the tasks of greatest interest to clinical research are (1) obtaining a confidence interval for ρ, (2) judging the adequate of ρ, and (3) considering how to improve ρ.

Corr(X, Y) = Corr(ξ , ν)(ρX ρY )1/2 Where ξ and ν are the ‘‘true values’’ of X and Y, and ρ X and ρ Y are their reliabilities, the attenuation of correlation is always associated with unreliability of measurement. In this case, the sample size needed to achieve a certain power in testing whether the observed correlation deviates from zero is proportional to the product of the two reliability coefficients. However, if the only effect of unreliability of measurement were to decrease the power of statistical tests, researchers could compensate for unreliability by increasing sample size. Increasing the sample size is not always feasible, and if feasible, it is not easy. Increasing sample size increases the time and cost of performing clinical trials, increasing the difficulty of obtaining adequate funding, and delaying the process of settling important clinical questions. Yet, all these processes

2 THE IMPORTANCE OF RELIABILITY IN CLINICAL TRIALS Concern about inter-rater reliability in randomized clinical trials stems from the fact that a lack of reliability results in a lack of precision in estimation of population parameters as well as a loss of power in testing hypotheses. For example, suppose the response to treatment in a clinical trial were given by

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

INTER-RATER RELIABILITY

could be managed. However, the effect of unreliability of measurement also leads to attenuation of effect sizes. Thus, it is possible to increase sample size enough to declare the treatment difference statistically significant, only to have the effect size indicate that the treatment difference is unlikely to be of clinical significance.

randomly allocated to positions in the analysis to assure that any disagreement between raters observing one subject is attributed to error, and raters must be ‘‘blinded’’ to each other’s ratings to ensure independence of errors. It is the design of the reliability study that assures that what is estimated adequately indicates reliability.

3 HOW LARGE A RELIABILITY COEFFICIENT IS LARGE ENOUGH?

5 ESTIMATE OF THE RELIABILITY COEFFICIENT—PARAMETRIC

No standards guarantee that inter-rater reliability is ‘‘large enough,’’ although ‘‘rules of thumb’’ have been suggested. For example, ρ > .8 might be considered ‘‘almost perfect,’’ .6 to .8 ‘‘substantial, .4 to .6 ‘‘moderate,’’ .2 to .4 ‘‘low,’’ and under .2 ‘‘unacceptable’’ (2). In fact, relatively few clinical measurements have reliability above .8, and many with reliability under .4 (3,4). Moreover, many medical measures, even those in common use, have never been assessed for reliability.

When it is assumed that ξ i and  ij are independent of each other, the intraclass correlation coefficient (q.v.) is generally used to estimate ρ. In practice, the easiest approach is to use a two-way ANOVA (N subjects by M raters). Then

4 DESIGN AND ANALYSIS OF RELIABILITY STUDIES

1 + (M − 1)r 1 + (M − 1)ρ ≈ F(M−1),(M−1)(N−1) 1−r 1−ρ

The design to assess the inter-rater reliability of a measurement requires sampling N subjects from the population of interest, then sampling M independent (‘‘blinded’’) raters of each subject at the same time (to avoid additional variance caused by inconsistency of the subject’s responses over time) from the population of ratings to which the results are meant to apply. It is important to realize that the reliability of any measure may vary from one clinical population to another. For example, a diagnostic procedure may be reliable in a clinical population but unreliable in a community population. At the same time, a diagnostic procedure may be reliable when done by well-trained physicians in a clinical population but unreliable when done by technicians (or vice versa) in the same population. Consequently, having a representative sample of patients from the relevant population and a representative sample of raters from that relevant population is crucial. Because of the practical difficulty of having multiple simultaneous raters of a subject at one time, most often M equals 2. Raters are

Easier to use in applications is an extension of Fisher’s z-transformation. Here for M ≥ 2,

r=

Fs − 1 Fs + M − 1

where FS is the F-statistic to test for the effect of subjects (5). Under these assumptions:


zM (r1 ) = .5 ln

1 + (M − 1)r1 1 − r1



is approximately normally distributed with mean zM (ρ) and variance 1/(h − 1) where: h=

2(M − 1)(N − 1) M

In particular, when M = 2, this means that
.5 ln

1+r 1−r



is approximately normally distributed with mean 
1+ρ .5 ln 1−ρ and variance 1/(N − 2). If a product moment correlation coefficient between the two ratings exists, then the variance of the z-transformation would have been 1/(N − 3), which

INTER-RATER RELIABILITY

indicates the minor loss of power associated with using an intraclass correlation rather than a product moment correlation coefficient applied to data generated in a reliability design. With the above information, confidence intervals for the intraclass correlation coefficient can be computed, presented, and evaluated for adequacy. The intra-class correlation coefficient has generated some confusion. Several different forms of the intraclass do not, in general, lead to the same conclusion (6–9). However, in a reliability study, when the multiple raters per subject are randomly assigned to the m positions, all various forms do estimate the same population parameter, although not necessarily equally efficiently. The problem related to choice of intra-class correlation coefficient therefore need not confuse reliability estimation. 6 ESTIMATION OF THE RELIABILITY COEFFICIENT—NONPARAMETRIC The above distribution theory for the intraclass correlation coefficient is robust to deviations from the normality assumptions, but if there is serious deviation for ordinal X, a nonparametric alternative is related to Kendall’s Coefficient of Concordance, W (10). If the N ratings in the jth position (j = 1, 2, . . . , M) are rank-ordered (tied ranks averaged), application of the above ANOVA leads to a nonparametric inter-rater reliability coefficient rKI = (MW − 1)/(M − 1). The distribution of rKI is well approximated by that of r when some nonlinear transformation of X ij exists that satisfies the normality assumptions (11). Otherwise, bootstrap methods (12–14) might be used to obtain confidence intervals for the reliability coefficient. 7 ESTIMATION OF THE RELIABILITY COEFFICIENT—BINARY On example of what would be considered a very serious deviation from the assumptions that underlie the parametric intraclass correlation coefficient occurs if X ij is binary (0 or 1), for then the within subject variance

3

usually depends on the true value. In that situation, the intraclass kappa coefficient estimates the inter-rater reliability coefficient (15). To compute the intraclass kappa coefficient, the above ANOVA may be applied to the 0/1 data (16). It is the distribution of the resulting reliability coefficient (the intraclass kappa) that is affected, not its computation. For M > 2, bootstrap methods are recommended to obtain confidence intervals, for the distribution depends on unknown higher order moments of ξ i = Ej (X ji ). For M = 2, the asymptotic distribution of the intraclass kappa is known (16), but unless sample size is very large, bootstrap methods are still preferred.

8 ESTIMATION OF THE RELIABILITY COEFFICIENT—CATEGORICAL Finally, suppose that X ij represents more than two nonordered categories (ordered categories are ordinal data for which rKI is more suited.). An intraclass kappa appropriate for this situation estimates 

Pk (1 − Pk )κk /



Pk (1 − Pk )

where Pk is the probability of falling into category k, k = 1, 2, . . . , K, and κ k is the intraclass kappa coefficient for the binary variable in which each subject is classified either in category k or not in category k (17). However, as can be observed from this formula, it is possible that the multicategory intraclass kappa is near zero when some individual categories are nearly perfectly reliable, or conversely, that the multicategory intraclass kappa is high one when some rarer categories have near zero reliability. For this reason, for multicategory X ij , it is recommended that the intraclass kappa be computed and evaluated for each of the categories separately (18). Then, if some individual categories are unreliable and other acceptably reliable, attention can be focused on improving the reliability of the flawed categories, either by redefinition, perhaps by combining categories that are ill distinguished from each other, or by retraining raters.

4

INTER-RATER RELIABILITY

9 STRATEGIES TO INCREASE RELIABILITIY (SPEARMAN-BROWN PROJECTION) What if the reliability coefficient estimated by the above methods is not high enough to satisfy the standards set by the designers of the randomized clinical trial? Then, several strategies might be considered to improve the inter-rater reliability of a measure prior to its use in a RCT: better training of raters and clarification of the measurement system. Alternatively, one can always propose to average multiple independent ratings for each subject in the RCT (common practice with assay procedures in which three independent assay results are often averaged per tissue sample.). Under the classic model, the reliability of the average of m ratings per subject, ρ m , is given by the Spearman-Brown projection formula: ρm =

mρ (m − 1)ρ + 1

Thus, for example, to raise the reliability of a measure from ρ to ρ m , one would need the average of m independent raters for each subject, where: m=

10

ρm (1 − ρ) (1 − ρm )ρ

OTHER TYPES OF RELIABILITIES

Inter-observer reliability is one of several types of reliabilities that might be of interest, depending on which sources of error are of interest in determining the accuracy of a measurement. Two others of considerable importance are test-retest reliability and intra-rater reliability. For test-retest reliability, the multiple ratings per subjects are sampled over a span of time short enough that the subject characteristic of interest is unlikely to change but long enough that the subject inconsistency in expressing that characteristic can be included in the error (e.g., over several weeks for the diagnosis of a long-lasting condition). Once again, to ensure the independence of errors within each subject, the ratings are

preferably made by different raters. Generally, because the error included in testretest reliability combines both rater error and error caused by subject inconsistency, test-retest reliability is generally lower than is inter-observer reliability that includes only rater errors. Because inconsistency of subject’s expression of the characteristic of interest is often the major source of error, and thus of attenuation of power in statistical hypothesis testing, or attenuation of effect size in estimation, test-retest reliability provides greater assurance of accuracy of measurement than does inter-observer reliability. For intra-rater reliability, the multiple ratings per subject are obtained by multiple independent measurements made by the same rater on the same observation. For example, one might sample N cancer patients, obtain a tumor tissue sample from each, divide that sample into M subsamples, each mounted separately for pathological examination, without any subject labels. Then, one rater would examine all unlabeled MN tissue samples in random order and classify each into the type or stage of cancer. The question is to what extent the rater classifies the M tissue samples from each patient the same. The error of interest now develops only from the inconsistencies within a rater, which is one component of the inconsistencies from one rater to another that is reflected in inter-observer unreliability. Consequently, one would expect that intra-observer reliability would be higher than would be interobserver reliability, which, in turn, would be higher than test-retest reliability. The discrepancies between these reliability coefficients provide clues as to the sources of error of measurement: from inconsistencies within each rater, from inconsistencies between one rater and another, and from inconsistencies in the patient’s expression of the characteristic.

REFERENCES 1. F. M. Lord and M. R. Novick, Statistical Theories of Mental Test Scores. Reading, MA: Addison-Weslely Publishing Company, Inc., 1968.

INTER-RATER RELIABILITY 2. J. R. Landis and G. G. Koch, The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159–174.

CROSS-REFERENCES

3. L. M. Koran, The reliability of clinical methods, data and judgments, part 1. N. Engl. J. Med. 1975; 293: 642–646.

Boostrapping Categorical Variables Correlation Intraclass Correlation Coefficient

4. L. M. Koran, The reliability of clinical methods, data and judgments, part 2. N. Engl. J. Med. 1975; 293: 695–701. 5. E. A. Haggard, Intraclass Correlation and the Analysis of Variance. New York: Dryden Press, 1958. 6. J.J. Bartko, The intraclass correlation coefficient as a measure of reliability. Psychol. Reports 1966; 19: 311. 7. J. J. Bartko, Corrective note to: The Intraclass Correlation Coefficient as a Measure of 1974; 1974: 34. 8. J. J. Bartko, On various intraclass correlation reliability coefficients. Psychol. Reports 1976; 83: 762–765. 9. P. E. Shrout and J. L. Fleiss, Intraclass correlations: uses in assessing rater reliability. Psychol. Bull. 1979; 86: 420–428. 10. M. Kendall and J. D. Gibbons, Rank Correlation Methods, 5th ed. New York: Oxford University Press, 1990. 11. H. C. Kraemer, The small sample non-null properties of Kendall’s Coefficient of Concordance for normal populations. J. Am. Stat. Assoc. 1976; 71: 608–613. 12. C. E. Lunneforg, Estimating the correlation coefficient: the bootstrap. Psychol. Bull. 1985; 98: 209–215. 13. J. L. Rasmussen, Estimating correlation coefficients: bootstrap and parametric. Psychol. Bull. 1987; 101: 136–139. 14. M. J. Strube, Bootstrap Type I error rates for the correlation coefficient: an examination of alternate procedures. Psychol. Bull. 1988; 104: 290–292. 15. H. C. Kraemer, V. S. Periyakoil, and A. Noda tutorial in biostatistics: kappa coefficients in medical research. Stats. Med. 2002; 21: 2109–2129. 16. J. L. Fleiss, Statistical Methods For Rates and Proportions. New York: John Wiley & Sons, 1981. 17. H. C. Kraemer, Ramifications of a population model for k as a coefficient of reliability. Psychometrika 1979; 44: 461–472. 18. H. C. Kraemer, Measurement of reliability for categorical data in medical research. Stat. Methods Med. Res. 1992; 1: 183–199.

Analysis of Variance (ANOVA)

Kappa Statistic Reliability Analysis Repeated Measurements Type II Error (False Negative)

5

INTERVAL CENSORED

are still in use. For instance, a potentially biased analysis would result from using the Kaplan–Meier estimator or the Cox regression model after transforming the interval censored observations into the right censoring situation. Replacing intervals with their middle or maximal point, however, approximates the true results only in exceptional cases, for instance, when the observed intervals are generally small and when the accuracy needed in the specific problem is low.

THOMAS A GERDS and CAROLINA MEIER-HIRMER Institute for Medical Biometry and Medical Informatics, University Hospital Freiburg Center for Data Analysis and Modeling Freiburg, Germany

1

CENSORING

In this article, the response is always the time of an event, the occurrence of which becomes known at examination times. Some special cases shall be distinguished. One speaks of left censoring, if the event of interest occurred before the examination time and of right censoring if the event did not occur until the examination time. The situation with only one examination for each patient is called ‘‘case 1’’ interval censoring, and the resulting observations are often called current status data. Left and right censoring can be generalized to ‘‘case k’’ interval censoring for situations in which the information from exactly k examinations is available for each patient (k is a positive integer). Since in the clinical practice the number of examination times is typically different among patients, most frequently one has to deal with ‘‘mixed case’’ interval censoring. This term refers to situations in which some observations are exact event times, some are right or left censored, and others are really censored to intervals. It is important to emphasize that the name ‘‘interval censoring’’ is often used to generally describe data consisting of such a mixture. Using artificial data, it is demonstrated in Fig. 1 and Table 1 how interval censored observations can be obtained from the longitudinal data of the examination process. Note that although patients 2 and 4 have the same event time, the observed intervals differ considerably. A note of caution: Complicated censoring schemes arise in medical practice as well as in other fields (see the statistical literature). But the connection between the theoretical results and the applications is not yet well developed in all cases nor easily available. Moreover, computer programs are not generally available. As a result, ad hoc methods that may cause biased conclusions

2

CLASSIFICATION AND EXAMPLES

In this section, the types of interval censoring are classified and illustrated more formally. For the special cases of left and right censoring, see the article on survival analysis. ‘‘Case 1’’ interval censoring: The observations are also called current status data. Here the information of one examination is available for each patient, and it is known if the event occurred before or after the date of examination. As a result, the observations are either left censored or right censored. Crosssectional studies in clinical or epidemiological projects often result in current status data. Another example are tumor incidence data in animal experiments where independent individuals are exposed to carcinogens. Typically, a valid histological diagnosis of the onset of a tumor is only possible after death. Thus, the day of death is the only examination time revealing whether a tumor has grown. The limitation of detecting a tumor only if it exceeds a critical size is a technical reason for censoring. For tumor incident experiments, the censoring occurs because of the inability to measure the event of interest exactly (occult tumors). For cross-sectional studies, the censoring is introduced by the study design. ‘‘Case k’’ interval censoring: For each patient, the results from k examinations are available. As the same number of examinations are almost never available for all patients, ‘‘case k’’

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

INTERVAL CENSORED

Examination process I

II

III

IV V VI

Right

1

Interval 2

Left 3

Interval 4

Exact Time

5

0

12

30

42

54

Figure 1. Interval-censored observations (thick lines) corresponding to the artificial data in Table (1). Filled dots in the intervals represent the (unobservable) true event times. The respective type of censoring is marked on the left axis of the diagram. Although observations 2 and 4 have the same true event time, the observed intervals differ considerably.

Table 1. Illustration of How to Obtain Interval-Censored Observations from Hypothetical Examination Processes Examination Process Time

0

12

30

42

48

Event

NO

NO

NO

NO

NO

Time

0

12

42

–

–

Event

NO

NO

YES

–

–

Time

0

–

–

–

–

Event

YES

–

–

–

–

Time

0

12

30

54

–

Event

NO

NO

NO

YES

–

Time

0

30

–

–

–

Event

NO

YES

–

–

–

Patient 1

Patient 2

Patient 3

Patient 4

Patient 5

interval censoring (for k greater than one) occurs sparsely in the medical praxis. An example is the supervision of children learning to speak where learned words are collected on a fixed number of questionnaires for each child. However, if single questionnaires are missing for some children, and hence the number of available examinations differs, the data have to be considered as ‘‘mixed case’’ interval censored. ‘‘Mixed case’’ interval censoring: For each patient, the event status is known at a differing number of examination

True Event Time

Censored Observation

60

[48, ∞)

36

[12, 42]

− 10

( − ∞, 0]

36

[30, 54]

30

30

times. Therefore, ‘‘mixed case’’ interval censored data typically consist of a mixture of interval-censored and rightcensored observations. Sometimes they include left-censored observations and even exact time points, if the event happens to occur at the day of examination. An example for mixed case interval censoring is given by breast cancer studies where the event of interest is the first occurrence of breast retraction. Shortly after therapy, which can be a combination of radiotherapy and chemotherapy, the time interval between two adjacent examinations is

INTERVAL CENSORED

typically small but lengthens as the recovery progresses. The exact time of retraction is only known to fall into the interval between two visits or after the last examination time. Double interval censoring: If the variable of interest is the duration between two events, then both the start point and the endpoint of the duration can be interval censored. A well-known example is the incubation time between HIV-infection and AIDS diagnosis: The infection time is interval censored between the last negative and the first positive antibody test, and the time to AIDS diagnosis is right censored, when AIDS is not diagnosed within the study time. Double censoring: Double censoring refers to situations in which the observations are either exact event times, left censored, or right censored. Double censoring is therefore a special case of mixed case interval censoring. Interval censored covariates: In the framework of multistate models, the influence of an intermediate event on the main outcome variable is sometimes of interest. For instance, if the endpoint of interest is death, then the recurrence of a tumor can be an influential intermediate event. In this example, the occurrence of the intermediate event is an interval-censored covariate. Situations in which the time of the intermediate event is interval censored occur frequently in such frameworks, in particular for illnessdeath models.

3 STUDY DESIGN AND STATISTICAL MODELING Considering interval-censored data resulting from periodic follow-up in a clinical trial or longitudinal study, the information is generally the greater the smaller the intervals are between adjacent examinations. The length of the observed intervals evidently influences the power of statistical methods, and it is important for the significance of statistical

3

analysis to gather as much information as possible. However, acquisition on a dense time schedule or even continuous in time can be prohibited for various reasons. Financial costs or stress of patients are typical factors that limit the accuracy of measurement. The statistical model building for censored data proceeds in two steps: In a first step, the survival model is specified, that is, a family of probability distributions that includes the underlying survival function. The second step deals with the dependence of the event time on the censoring scheme. Sometimes it is important to specify a probabilistic model also for the distribution of the censoring process. For convenience, it is often assumed that the censoring is independent of the event time. Survival models can be parametric, semiparametric, or nonparametric. Besides the differences of the statistical methods used for the three model classes, one must be aware of the well-known tradeoff between bias and variance, which may differ among the three approaches: On the one hand, if the parametric model assumptions are violated, the statistical inference can be misleading due to biased estimates. On the other hand, with semiparametric or nonparametric models, the power of the statistical methods is typically low for small or moderate sample sizes that are often present in clinical studies. In the presence of covariates, a suitable survival regression model has to be specified. For instance, the proportional hazards model, the proportional odds model, and the accelerated failure time model are frequently used regression models that have extensions to interval-censored data structures. The task of modeling the censoring mechanism has a similar impact: Strong assumptions on the distribution of the censoring mechanism can result in biased inference, whereas allowing general censoring schemes may lead to low statistical power of the statistical procedures. As noted, most statistical procedures for interval-censored data assume that the examination times are independent of the event time. This assumption is satisfied for externally scheduled examination times. However, there are situations in which the examination process is not independent of the

4

INTERVAL CENSORED

event time. The random censorship assumption is often violated when the data arise from a serial screening and the timing of screening depends on the patient’s health status. Or, for time-to-infection data, if the infection can be suspected after certain undertakings or accidents, cool-headed patients would likely urge a laboratory testing. Then, the infection time and the examination time are not independent. 4

STATISTICAL ANALYSIS

Several characteristics of survival distributions are of interest in clinical trials: the survival probability function, the difference or ratio of the survival probabilities in two (treatment) groups, and the influence of covariates on the survival probabilities, to name the most important ones. Under the burden of complicated censoring schemes, for each of these cases, a valid estimation method is needed. The methods for interval-censored data are nonstandard and need advanced techniques. Consistent estimators may be available only under restrictive assumptions on the censoring mechanism, and the distributional properties of estimates, confidence intervals, or valid testing procedures are only approximately constructed. To this end, one should note that the development of statistical methods and their distributional properties for interval-censored data are at present not fully developed and research is an ongoing process. In particular, examples with appropriate statistical treatment of dependent interval censoring are only very recent; see References 1 and 2. There is also ongoing work in mathematical statistics. The mathematically interested reader is referred to References 3 and 4 and the references given therein. In the remaining section, some established statistical methods are named for which it is assumed that the censoring is independent of the event time. The inference in parametric survival models is relatively straightforward. As a consequence of the independence assumption, likelihood based methods are applicable; see Reference 5. In particular, the maximum likelihood estimator has the usual properties, as are consistency and the usual convergence

rate n1/2 , where n is the sample size. Software for the parametric inference should be available for most standard statistic programs. For the nonparametric estimation of the survival probability function, the so-called nonparametric maximum likelihood estimator (NPMLE) can be used. The estimator is related to the familiar Kaplan–Meier estimator, which is the nonparametric maximum likelihood estimator for right-censored data. However, the Kaplan–Meier estimator cannot be applied directly and only in exceptional cases to interval-censored data; see the note of caution at the end of Section (1). NPMLE for interval-censored data is not uniquely defined: Any function that jumps the appropriate amount in the so-called equivalence sets represents a NPMLE; see Reference 6. Briefly, the equivalence sets are found by ordering all unique left-hand limits and all unique right-hand limits of all observed intervals in a dataset; see Reference 7 for details. Outside the equivalence sets, the nonparametric estimator defines constant survival probability and the graph is horizontal in these parts. Although the graph of NPMLE is formally undefined in the equivalence sets, some authors visualize NPMLE as if it was a step function, some interpolate between the horizontal parts, and others leave the graph indefinite outside the horizontal parts. Technical differences occur with the computation of NPMLE for the different types of censoring: For ‘‘case 1’’ interval-censored data, NPMLE is given by an explicit formula, (8). For the other types of interval censoring, NPMLE has to be computed recursively; see References 6, and (8–10). For instance, the self-consistency equations developed by Turnbull (9) yield an algorithm that is a special case of the EM-algorithm. Turnbull’s algorithm is implemented in some major statistical software packages (SAS, Splus). The more recently suggested algorithms achieve improvement concerning stability and computational efficiency (8, 11, 12). Unlike the Kaplan–Meier estimator, the NPMLE for interval-censored data converges at a rate slower than n1/2 , where n is the sample size. In particular, the distribution of this survival function estimator cannot be approximated by a Gaussian distribution. However, at least for ‘‘case 1’’ and ‘‘case 2’’

INTERVAL CENSORED

5

Semiparametric models are prominent for the analysis of regression problems in survival analysis. The frequently used regression models that have extensions for intervalcensored data structures are the proportional hazards model (14, 15), the proportional odds model (16), and the accelerated failure time model (3). These model classes have semiparametric and parametric subclasses. The main difference is that the estimators of the survival curve in the semiparametric models perform as NPMLE; i.e., the convergence rate is slow, and the distribution is not

interval censoring, the asymptotic distribution of NPMLE has been derived in Reference 8. In some cases, the bootstrap provides an alternative method for approximating the distribution of NPMLE (3). By using such tools, confidence intervals for the survival probability at a fixed time can be constructed. Nonparametric tests for the two group comparison have been proposed in References 12 and 13.

Table 2. Breast Cancer Retraction Data in Two Treatment Arms. The Time of First Occurrence of Breast Retraction Lies Between the Left and the Right Endpoint of the Intervals; Right-Censored Observations have ∞ as the Right Endpoint Treatment 1 (n = 46)

[0, 5], [0, 7], [0, 8], [4, 11], [5, 11], [5, 12], [6, 10], [7, 14], [7, 16], [11, 15], [11, 18], [17, 25], [17, 25], [18, 26], [19, 35], [25, 37], [26, 40], [27, 34], [36, 44], [36, 48], [37, 44], [15, ∞), [17, ∞), [18, ∞), [22, ∞), [24, ∞), [24, ∞), [32, ∞), [33, ∞), [34, ∞), [36, ∞), [36, ∞), [37, ∞), [37, ∞), [37, ∞), [38, ∞), [40, ∞), [45, ∞), [46, ∞), [46, ∞), [46, ∞), [46, ∞), [46, ∞), [46, ∞), [46, ∞), [46, ∞)

Treatment 2 (n = 48)

[0, 5], [0, 22], [4, 8], [4, 9], [5, 8], [8, 12], [8, 21], [10, 17], [10, 35], [11, 13] [11, 17], [11, 20], [12, 20], [13, 39], [14, 17], [14, 19], [15, 22], [16, 20] [16, 24], [16, 24], [16, 60], [17, 23], [17, 26], [17, 27], [18, 24], [18, 25] [19, 32], [22, 32], [24, 30], [24, 31], [30, 34], [30, 36], [33, 40], [35, 39] [44, 48], [11, ∞), [11, ∞), [13, ∞), [13, ∞), [13, ∞), [21, ∞), [23, ∞) [31, ∞), [32, ∞), [34, ∞), [34, ∞), [35, ∞), [48, ∞)

Patients

Treatment 1

0

5

10

15

20

25

30

35

40

45

50

55

60

∞

40

45

50

55

60

∞

Months

Patients

Treatment 2

0

5

10

15

20

25

30

35

Months Figure 2. Graphical representation of the breast deterioration data in two treatment groups. Right-censored observations are shown at the end of each treatment group.

6

INTERVAL CENSORED

approximately Gaussian. In case of a parametric survival model, the usual properties retain for the maximum likelihood estimators of the covariate effects. The hypothesis of zero covariate effects can then be tested, e.g., under the proportional hazard assumption (14); see also References 17 and 18 for related test statistics. 5

It was suggested that additional chemotherapy leads to earlier breast retraction. The data consist of a mixture of interval-censored and right-censored observations. A complete listing taken from Reference 7 is presented in Table 2. A special diagram has been introduced in Reference 12 for the graphical representation of interval censored data; see Fig. 2. In each treatment group, the interval-censored part of the data is sorted by the length of the observed interval and the right-censored part by the time of the last examination. Figure 3 compares the parametric estimate of the survival curve in the Weibull survival model to NPMLE. In addition, the Kaplan–Meier estimator was computed by treating the center of the 56 closed intervals, where the right endpoint is not ∞, as if these observations were exact. The graph of the Kaplan–Meier estimator, which is only an ad hoc method in this situation, is also displayed in Fig. 3. All estimation methods show late differences among the survival probabilities in the treatment groups. The graphs of the competing methods are quite close for treatment 1, but they differ for treatment 2. To test the hypothesis of no treatment effect, the nonparametric tests proposed in References 12 and 13 are compared with the

WORKED EXAMPLE

1.0

In this section, an illustration of the mainstream statistical methods for interval censoring is given. The data are taken from the overview article (7), where the reader also finds comprehensive statistical analysis and a comparison of statistical methods. The source of the data is clinical studies on the cosmetic effect of different treatments of breast cancer (19, 20). Women with early breast cancer received a breastconserving excision followed by either radiotherapy alone or a combination of radiotherapy and chemotherapy. The event of interest was the first occurrence of breast retraction. The time interval between two adjacent examinations was in the mean 4–6 months, stretching wider as the recovery progresses. In what follows, treatment 1 corresponds to radiotherapy alone and treatment 2 to a combination of radiotherapy and chemotherapy.

0.0

0.2

Survival Probability 0.4 0.6 0.8

NPMLE Weibull Kaplan−Meier

0

10

20

30

40

50

60

Months Figure 3. Comparison of estimated survival curves in the treatment groups of the breast deterioration data. The respective upper curves belong to treatment 1 for all methods of estimation.

INTERVAL CENSORED Table 3. Test Results for the Treatment Effect in the Breast Deterioration Data Test

P-value

Finkelstein (14) ¨ Dumbgen et al. (12) Sun (13) Parametric

0.004 0.0028 0.0043 0.0012

11.

12.

13.

test under the proportional hazard assumption given in Reference 14, and with the test in a fully parametric survival model (Weibull family). The resulting P-values presented in Table 3 are different. However, in this example, all methods show significant differences among the treatment arms. REFERENCES 1. R. A. Betensky, On nonidentifiability and noninformative censoring for current status data. Biometrika 2000; 218–221. 2. D. M. Finkelstein, W. B. Goggins, and D. A. Schoenfeld, Analysis of failure time data with dependent interval censoring. Biometrics 2002; 298–304. 3. J. Huang and J. A. Wellner. Interval censored survival data: A review of recent progress. Proc. First Seattle Symposium in Biostatistics: Survival Analysis, 1997. 4. J. Sun, Encyclopedia of Biostatistics: Interval Censoring, P. Armitage and T. Colton (eds.), New York: Wiley, 2002: 2090–2095. 5. J. P. Klein and M. L. Moeschberger, Survival Analysis—Techniques for Censored and Truncated Data. Statistics in Biology an Health. New York: Springer, 1997. 6. R. Peto, Experimental survival curves for interval-censored data. Appl. Stat. 1973; 22: 86–91. 7. J. C. Lindsey and L. M. Ryan, Tutorial in biostatistics—methods for interval-censored data. Stat. Med. 1998; 17: 219–238. 8. P. Groeneboom and J. A. Wellner, Information Bounds and Nonparametric Maximum Likelihood Estimation, vol. 19 of DMV-Seminar. ¨ New York: Birkhauser, 1992. 9. B. W. Turnbull, The empirical distribution function with arbitrarily grouped, censored and truncated data. J. R. Stat. Soc. Series B 1976; 38: 290–295. 10. A. P. Dempster, N. M. Laird, and D. B. Rubin, Maximum likelihood from incomplete data via

14.

15.

16.

7

the EM algorithm (with discussion). J. R. Stat. Soc. Series B 1977; 39: 1–38. G. Jongbloed, The iterative convex minorant algorithm for nonparametric estimation. J. Comput. Graphical Stat. 1998; 7: 310–321. ¨ L. Dumbgen, S. Freitag, and G. Jongbloed, Estimating a unimodal distribution from interval-censored data. Technical report. University of Bern, 2003. J. Sun, A nonparametric test for intervalcensored failure time data with application to AIDS studies. Stat. Med. 1996; 15: 1387–1395. D. M. Finkelstein. A proportional hazards model for interval-censored failure time data. Biometrics 1986; 42: 845–854. J. Huang, Efficient estimation for the proportional hazards model with interval censoring. Ann. Stat. 1996; 24: 540–568. A. J. Rossini and A. A. Tsiatis, A semiparametric proportional odds regression model for the analysis of current status data. J. Amer. Stat. Assoc. 1996; 91: 713–721.

17. M. P. Fay. Rank invariant tests for interval censored data under the grouped continous model. Biometrics 1996; 52: 811–822. 18. G. R. Petroni and R. A. Wolfe. A two-sample test for stochastic ordering with intervalcensored data. Biometrics 1994; 50: 77–87. 19. G. F. Baedle, S. Come, C. Henderson, B. Silver, and S. A. H. Hellman, The effect of adjuvant chemotherapy on the cosmetic results after primary radiation treatment for early stage breast cancer. Int. J. Radiation Oncol. Biol. Phys. 1984; 10: 2131–2137. 20. G. F. Baedle, J. R. Harris, B. Silver, L. Botnick, and S. A. H. Hellman, Cosmetic results following primary radiation therapy for early beast cancer. Cancer 1984; 54: 2911–2918.

INTRACLASS CORRELATION COEFFICIENT

to different dietary interventions. This form of randomization at a higher level than the individual is known as a cluster-randomized (or group randomized) design. Although randomization occurs at this higher level, observations are usually still made at the level of the individual. Cluster trials can adopt a completely randomized design (in which clusters are randomized to each intervention without restriction), a stratified design (in which similar clusters are grouped into strata and randomization to the interventions takes place within each stratum), or a paired design (in which clusters are paired and one cluster from each pair is randomized to each intervention). Adoption of a cluster-randomized design, however, has implications for the design, conduct, and analysis of such studies. A fundamental assumption of an individually randomized trial is that the outcome for an individual patient is completely unrelated to that for any other patient (i.e., they are ‘‘independent’’). This assumption no longer holds when cluster randomization is adopted, because patients within any one cluster are more likely to respond in a similar manner. For example, the management of patients in a single hospital is more likely to be consistent than management across several hospitals. This correlation among individuals within clusters has to be taken into account when planning and analyzing the cluster trial design. The statistical measure of this correlation is known as the intraclass or intracluster correlation coefficient (ICC). In this article, the measurement and impact of the ICC is discussed. The most common trial design—the completely randomized design—is assumed throughout.

MARION K. CAMPBELL University of Aberdeen, Aberdeen, UK

JEREMY M. GRIMSHAW Ottawa Health Research Institute, Ottawa, Canada

GILDA PIAGGIO World Health Organisation, Geneva, Switzerland

1

INTRODUCTION

The randomized controlled trial is the design of choice for evaluating healthcare interventions (1,2). Most commonly, randomization takes place at the level of the patient, in which individual patients are allocated to the different arms of the trial. Randomization by individual minimizes bias in study results and provides maximal efficiency regarding the precision of estimates and power of statistical tests. In some situations, however, allocation at an individual patient basis is problematic, for example, if there is a risk of contamination of interventions across trial groups (i.e., the unintentional transfer of elements of one intervention to participants allocated to another arm of the trial) (3). For example, if some patients in an inpatient ward were allocated randomly to receive charts to record symptoms to trigger particular actions and some patients were not, then it would difficult to prevent those patients given the charts from talking to those patients not assigned charts. This latter group might then begin to note their symptoms, even without charts, thereby potentially distorting the results of the trial. In such situations, randomizing at some ‘‘higher’’ level may be desirable to obtain a more reliable estimate of the treatment effect (4). In this case, randomization by hospital ward would have been more appropriate. Similarly, when trials that evaluate dietary interventions are being designed, families are often randomized as an entire unit to avoid the possibility of different family members being assigned

2 THE INTRACLUSTER (OR INTRACLASS) CORRELATION COEFFICIENT Individuals within a cluster are usually more similar than individuals from different clusters. Because individuals within a cluster receive the same intervention and those who receive different interventions belong to different clusters, randomizing clusters implies

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

INTRACLASS CORRELATION COEFFICIENT

a loss in efficiency compared with randomizing individuals. The variability between units that receive the same intervention, which is the basis for estimating the uncontrolled variation, is thus greater than that between individuals who receive the same intervention had the individuals been randomized. The similarity between individuals within clusters is measured by the intracluster correlation coefficient (which is commonly referred to by the Greek letter rho, ρ). 2.1 Definition of the ICC The ICC can be defined in general as the correlation between pairs of subjects within clusters, and thus it can vary between −1/(j−1) and 1, where j is the number of clusters (5). However, the most commonly used definition of the ICC, which is appropriate within the cluster trial context, is based on the analysis of variance and specifically on the relationship of the between to within-cluster variance. Using this definition, the ICC is calculated as the proportion of the total variation in the outcome that can be attributed to the difference between clusters: ρ=

σb2 σb2

+ σw2

where σb2 is the between-cluster variance component, and σw2 is the within-cluster variance component (6). In this context, therefore, the ICC takes a value of between 0 and 1. 2.2 Accounting for the ICC in Sample-Size Calculations The presence of clustering within trial data has a direct impact on sample-size requirements for a trial, primarily because standard sample-size calculations assume that data within the trial are independent (which is not the case in a cluster trial). The size of the ICC has a direct impact on the efficiency of the sample size within the clustered design. For example, for a completely randomized cluster trial design, to achieve the equivalent power of a patient-randomized trial, standard sample-size estimates require to be inflated by a factor: 1 + (n − 1)ρ

to accommodate for the clustering effect, where n is the average cluster size, and ρ is the estimated ICC (assuming the clusters are of a similar size) (7). (It is important to note that cluster size can be defined as either the number of available individuals with the cluster or the number of individuals within the cluster being selected to take part in the trial.) This inflation factor is commonly known as the ‘‘design effect’’ or the ‘‘variance inflation factor’’ (8). The impact of randomization by cluster on the required sample size can be substantial, and as both the ICC and the cluster size influence the calculation, the design effect can be substantial even when the ICC seems to be numerically small. Table 1 shows the example of a trial designed to detect a change from 40% to 60% in a dichotomous outcome, with 80% power and a 5% significance level. For an individually randomized trial, a sample size of 194 patients would be required to detect this level of difference. If, however, cluster randomization was required because of the potential for contamination, samplesize estimates would have to be inflated to account for the ICC and the cluster size. Table 1 shows that, even at relatively modest levels of ICC (say ρ = 0.01), a doubling of the original estimate would not be unusual. Sample-size calculators have been developed to aid in sample size calculations for trials, which also allow trade-offs between cluster sizes and number of clusters to be assessed (9,10). 2.3 Methods for Estimating the ICC The aim of this section is to provide a brief overview for the reader of the most common approaches to estimating the ICC, providing references for additional detail and methods. Some of these estimation procedures can return ICC values of less than zero, but in such cases the ICC estimate should be truncated at zero. For the estimation of an ICC from continuous data, the most common method adopted is the analysis of variance (ANOVA) approach (6,11,12). This method draws on the fact that, as alluded to above, an ICC can be described as the relationship between the two components of variation in outcome in

INTRACLASS CORRELATION COEFFICIENT

3

Table 1. Impact of the ICC on Sample Size Requirements Example:

To detect a change from 40% to 60%, with 80% power and a 5% significance level, assuming different levels of ICC and cluster size (under individual randomization a total of 194 patients would be required).

Cluster size

ICC

10 Number of clusters (total number of individuals) required

20 Number of clusters (total number of individuals) required

50 Number of clusters (total number of individuals) required

100 Number of clusters (total number of individuals) required

0.01 0.05 0.10 0.15

22 (220) 30 (300) 38 (380) 46 (460)

12 (240) 20 (400) 30 (600) 38 (760)

6 (300) 14 (700) 24 (1200) 34 (1700)

4 (400) 12 (1200) 22 (2200) 32 (3200)

a cluster trial— the variation among individuals within a cluster and the variation between clusters. These two sources of variation can be identified separately using a standard ANOVA approach—by adopting a one-way ANOVA, with the cluster-level variable as a random factor (13). For the ANOVA approach, assuming a two-level model (i.e., cluster and individual) the appropriate model can expressed as: yij = µ + βj + eij where yij is the response of the ith individual in the jth cluster, µ is the overall mean response, β j is the random effect of the jth cluster, and eij is the random error. It is assumed that the β j are independently and identically distributed with zero mean and constant variance σb2 , and that the eij are similarly identically and independently distributed with zero mean and constant variance σw2 . The following variance component estimates are then identified from the ANOVA table: (MSB − MSW) σˆ b2 = n0 and σˆ w2 = MSW where MSB is the between-cluster mean square, MSW is the within-cluster mean square, and n0 is the ‘‘average’’ cluster size.

The ‘‘average’’ cluster size is calculated using:
 n2j 1 n0 = N− J−1 N where J is the number of clusters, N is the total number of individuals, and nj is the number of individuals in the jth cluster. This version of the ‘‘average’’ cluster size is used over the arithmetic average, as the arithmetic mean estimate can lead to underestimation of the between-cluster variance component when the cluster sizes vary. The ICC can then be estimated as: ρˆ =

σˆ b2 σˆ b2

+ σˆ w2

Most random effects modeling packages also produce estimates for these betweencluster σˆ b2 and within-cluster σˆ w2 components of variation directly as part of their standard output. These estimates can then be used directly in the formula to estimate the ICC. Random effects modeling also allows estimation of the ICC when covariates need to be accounted for in the model. Donner and Koval (11) note that using this derivation of the ICC requires no assumption that the dependent data be normally distributed. Donner and Koval (6) also describe an alternative approach to calculating the ICC from continuous data that can be calculated by assuming that the observations yij , are distributed about the same mean µ and variance σ 2 in such a way that two observations

4

INTRACLASS CORRELATION COEFFICIENT

within the same cluster have a common correlation ρ. This result adopts a maximum likelihood estimator (MLE) approach. Donner and Koval note that in samples of equal size n, this expression of the ICC is equivalent to the Pearson product-moment correlation as computed over all pairs of observations that can be constructed within clusters. In this article, Donner and Koval compared the performance of ANOVA and maximum likelihood estimates using Monte Carlo simulation. The results of the simulation suggested that for low to moderate ICCs (< 0.5), the ANOVA method performed well and provided accurate estimates of the ICC, especially when single-member clusters were excluded from the analysis. However, if the ICC was large (> 0.5), or completely unknown, then the maximum likelihood approach tended to produce more accurate estimates. For the estimation of an ICC from binary data, the ANOVA approach is again most commonly used, as it has also been shown not to require any strict distributional assumptions (14,11,12). Although the ANOVA estimate of the ICC may be used with dichotomous data, however, standard methods to calculate confidence intervals around this estimate do not hold in the dichotomous case. Moore and Tsiatis (15) and Feng and Grizzle (16) describe an alternative approach for use in the binary case—the moment estimator for dichotomous data. The moment estimator is based on an extension of the generalized Pearson statistic to estimate overdispersion. Ridout et al. (17) reviewed the performance of several methods for calculating ICCs from binary data and showed that the ANOVA method performed well. Feng and Grizzle (16) also showed that the ANOVA method and the moment estimator method were similar, but they noted that bias increases as the estimate of ρ increases and as the prevalence nears zero or one (estimates were lower than they should be).

3

THE MAGNITUDE OF THE ICC

As has been described above, robust estimates of ICCs are required for informed sample size calculations to be made.

3.1 What is the Magnitude of the ICC? Estimates on ICC size are available from several settings and for several outcomes. The ICC estimate varies depending on the type of cluster under examination (e.g., community, hospital, family, and so on). As the definition of the ICC suggests, ICCs are likely to be high if the variation between clusters is high and the variation within clusters is low. Gulliford et al. (18) published a range of ICCs suitable for use in community intervention studies based on a systematic analysis of the 1994 English Health Survey. This research showed that for community intervention trials where communities were at a regional or district level ICCs were typically lower than 0.01. However, as the definition of ‘‘community’’ was varied, so the typical range of ICC values varied. Where communities were defined at postcode/zipcode level, ICCs were generally lower than 0.05. When community was defined at a household level, however, ICCs ranged up to 0.3. This finding highlights well the contribution of the between-to-within cluster variation to the estimate of the ICC. In small units, such as households, within-cluster variation tends to be small so the between-cluster variation dominates, which leads to a higher estimate of ICC. For larger clusters, however, withincluster variation is relatively large; hence, the effect of the between-cluster variation is lessened, which leads to a smaller ICC estimate. Todd et al. (19) also showed how different design features could affect the size of the ICC in community-intervention trials. They showed that effective matching of clusters resulted in a marked lowering of the ICC. As the community trials referred to by Todd et al. (19) all involved large cluster sizes, however, reported ICCs, were generally lower than 0.01 irrespective of the chosen design. Individual community-intervention studies have also published several ICCs. For example, Hannan et al. (20) published ICCs from the Minnesota Health Program—these ranged typically from 0.002 to 0.012. A range of ICCs from other settings has also been identified. Murray and Blitstein (21) highlighted more than 20 papers that had reported ICCs from a range of settings,

INTRACLASS CORRELATION COEFFICIENT

which included workplace and school-based studies. Smeeth and Ng (22) provided a comprehensive analysis of ICCs for cluster trials that involved older people in primary care and observed ICCs that ranged from very small ( 0. • The error ε ij is a normally distributed random variable with mean 0, and variance σˆ2 > 0, and it is independent of tj . A small value for σ2 will lead to a small variation between replicate scores, which in turn should lead to a high intrarater reliability. Therefore, the theoretical parameter that represents the intrarater reliability γ is defined as follows: γ =

σt2 σt2 + σ2

(6)

which is one of the parameters studied by McGraw and Wong (8). It follows from Equation (5) that the denominator of γ is the total variation in the scores, and Equation (6) is the most popular form of ICC found in the

4

INTRARATER RELIABILITY

statistical literature and represents the portion of total variance that is accounted for by the between-subject variance. The statistic γˆ of Equation (3) is a sample-based approximation of γ widely accepted in the statistical literature. We will now present a method for evaluating the precision of the statistic γˆ . How close is γˆ to γ ? To answer this question we will construct a 95% confidence interval around γ ; that is a range of values expected to contain the unknown γ with 95% certainty. Constructing a 95% confidence interval for γ requires the calculation of the 2.5th and 97.5th percentiles of the F distribution with m−1 and m(n−1) degrees of freedom. These two percentiles are denoted by F0.975,m−1,m(n−1) and F0.025,m−1,m(n−1) , respectively, where 0.975 = 1–(1–0.95)/2 and 0.025 = (1–0.95)/2. Although textbooks’ statistical tables provide these percentiles, they are also readily obtained from MS Excel as follows: ‘‘ =FINV(0.025,m−1,m*(n−1))’’ gives the 97.5th percentile, whereas ‘‘FINV(0.975,m−1,m*(n−1))’’ for the 2.5th percentile. Let F be defined as follows: Fo =

BSVo MST = WSV/n MSE

(7)

The 95% confidence interval for γ is obtained by noting that: P(L95 ≤ γ ≤ U95 ) = 0.95 L95 =

F0 /F0.025,m−1,m(n−1) − 1 and (m − 1) + F/F0.025,m−1,m(n−1)

U95 =

F0 /F0.975,m−1,m(n−1) − 1 (m − 1) + F/F0.975,m−1,m(n−1)

(8)

The 95% confidence interval width is given by: (9) W95 = U95 − L95 The design of an intrarater reliability study must aim at minimizing the magnitude of the confidence interval width. The optimal values for the number m of subjects and the number n of replicates per subject are those that minimize W 95 of Equation (9).

1.3 Optimizing the Design of the Intrarater Reliability Study An intrarater reliability study is well designed if the number of observations is determined to minimize the confidence interval length and the experimental error is kept small. This section addresses the following two questions: 1. What is the optimal number m of subjects, and number n of replicates per subject? 2. Can the intrarater reliability study involve two raters or more? 1.3.1 Sample Size Determination. Finding the optimal number of subjects and replicates per subject based the confidence interval length is the approach Giraudeau and Mary (9) used to propose guidelines for planning a reproducibility study. Let ω95 be the expected width of the 95% confidence interval associated with γ . Note that σ2 Fo /(σ2 + mσt2 ) = Fo /(1 + mγ /(1 − γ )) where Fo is defined by Equation (7) and follows the F distribution with m−1 and m(n−1) degrees of freedom. Because W 95 defined by Equation (9) is a function of Fo , its expected value ω95 is a function of γ . The relationship between ω95 and γ is depicted in Figs. 1, 2, and 3 for various values of m and n. The ω95 values are calculated using a Monte-Carlo simulation approach because of the difficulty to derive a mathematical expression of the probability distribution of W95 . For values of γ that vary from 0 to 1 by step of 0.05, and for various combinations of (m,n) we simulated 10,000 observations from the F distribution with m−1 and m(n−1) degrees of freedom, and calculated 10,000 confidence intervals using Equation (8). The mean length of the 10,000 intervals was used as an estimate for ω95 . Each of the three figures contains two plots, and each plot shows how different values of m and n affect the relationship between γ and ω95 for a fixed total number of observations mn. For the two plots of Fig. 1, the total sample sizes mn are 20 and 40. For Fig. 2, the total sample sizes are 60 and 80, whereas Fig. 3’s plots are based on the sample sizes of 100 and 120. All three figures tend to indicate

Expected Interval Width

INTRARATER RELIABILITY

1. .75 .50

Expected Interval Width

0

.25

.50 .75 Intra Rater Reliability

1.

.75

.50 (m,n) = (20,2) (m,n) = (10,4) (m,n) = (8,5)

.25

0

Expected Interval Width

(m,n) = (10,2) (m,n) = (5,4)

.25 0

Figure 1. Expected width of the 95% confidence interval as a function of γ for m and n values that correspond to mn = 20 and mn = 40.

5

0

.25

.50 .75 Intra Rater Reliability

1.0

0.6 0.5 0.4 (m,n) = (30,2) (m,n) = (20,3) (m,n) = (15,4) (m,n) = (12,5)

0.3 0.2 0.1 0 0

.25

.50

.75

1.0

.75

1.0

Figure 2. Expected width of the 95% confidence interval as a function of γ for m and n values that correspond to mn = 60 and mn = 80.

Expected Interval Width

Intra Rater Reliability 0.6 0.5 0.4 0.3

(m,n) = (40,2) (m,n) = (20,4) (m,n) = (16,5) (m,n) = (10,8)

0.2 0.1 0

0

.25

.50 Intra Rater Reliability

INTRARATER RELIABILITY

Expected Interval Width

6

0.6 0.5 0.4 0.3

0.1

Expected Interval Width

0

Figure 3. Expected width of the 95% confidence interval as a function of γ for m and n values that correspond to mn = 100 and mn = 120.

(m,n) = (50,2) (m,n) = (25,4) (m,n) = (20,5) (m,n) = (10,10)

0.2

0

.25

.50 .75 Intra Rater Reliability

1.0

0.5 0.4 0.3 (m,n) = (60,2) (m,n) = (40,3) (m,n) = (30,4) (m,n) = (24,5)

0.2 0.1 0

that for high intrarater reliability coefficients (i.e., greater than 0.5), and a fixed total number of observations mn, using 2, 3, or at most 4 replicates per subjects is sufficient to obtain the most efficient intrarater reliability coefficient. Having more than four replicates is likely to lead to a loss of precision. For smaller γ values, the recommendation is to use four or five replicates per subject. One would also note that if the ‘‘true’’ value of the intrarater reliability is smaller than 0.80, then its estimation will generally not be very precise. 1.3.2 Blocking the Rater Factor. If two raters or more are used in a completely randomized intrarater reliability experiment, the resulting coefficient will be inaccurate. In a completely randomized design, subjects and replicates are assigned randomly to different raters. Consequently, the rater effect will increase the experimental error, which thereby decreases the magnitude of the intrarater reliability coefficient. This problem can be resolved by designing the experiment so that the rater effect can be measured and separated from the experimental error. A design that permits this method requires each rater to rate all subjects and provide the same number of replicates per

0

.25

.50 .75 Intra Rater Reliability

1.0

subject. Under this design referred to as Randomized Block Design (RBD), the data is gathered by block (i.e., by rater in this case) with random rating within the block, and it would be organized as shown in Table 4, where γ is the number of raters. In Table 4, yijk represents the kth replicate observation on subject i provided by rater j. The intrarater reliability coefficient γˆ under an RBD design is still defined by Equation (3) with the exception that the within-subject variation (WSV) and the between-subject variation (BSV) are defined as follows: 1

2 Sij , and mr m

WSV =

r

i=1 j=1

BSV = BSVo − where S2ij = BSVo =

1 n−1

1 m−1

WSV nr n
(yijk − yij· )2 and k=1

m
(yi·· − y··· )2 ;

(10)

i=1

yij is the average of all n scores rater j assigned to subject i, yi·· is the average of

INTRARATER RELIABILITY

7

Table 4. Intrarater Reliability Data on m Subjects with r Raters and n Replicates per Subject and per Rater Subjects 1 2 .. . i . .. m

Rater 1

···

Rater j

···

Rater r

y111 , . . . y11n y211 , . . . y21n .. . yill , . . . yiln . .. ymll , . . . ymln

... ... .. . ... . .. ...

y1j1 , . . . y1jn y2j1 , . . . y2jn .. . yijl , . . . yijn . .. ymjl , . . . ymjn

... ... .. . ... . .. ...

y1r1 , . . . y1rn y2rl , . . . y2rn .. . yirl , . . . yirn . .. ymrl , . . . ymrn

nr scores assigned to subject i, and y··· the overall mean scores. Note that the WSV is obtained by averaging the mr sample variances calculated at the cell level. Equation (10) offers the advantage of removing any influence of inter-rater variation when calculating the intrarater reliability. The number of replicates in an RBD design may vary by rater and by subject. We assumed it to be fixed in this section for the sake of simplicity. Although a single rater is sufficient to carry out an intrarater reliability experiment, the use of multiple raters may be recommended for burden reduction or for convenience. The techniques and the inferential framework discussed in this section work well for continuous data, such as the cholesterol level, but they are not suitable for nominal data. In the next section, I present some techniques specifically developed for nominal data. 2

NOMINAL SCALE SCORE DATA

Although the ICC is effective for quantifying the reproducibility of continuous data, nominal data raise new statistical problems that warrant the use of alternative methods. Rating subjects on a nominal scale amounts to classifying them into one of q possible response categories. The discrete nature of that data has the following two implications: 1. The notion of reproducibility is exact. A response category is reproduced when the initial and the replicate categories are identical, and unlike continuous data, nominal data are not subject to random measurement errors.

2. A rater may classify a subject on two occasions into the exact same category by pure chance with a probability that is non-negligible. Table 5 shows the distribution of 100 individuals with identified pulmonary abnormalities examined by a medical student on two occasions. On both occasions, the medical student found the same 74 individuals with pulmonary abnormalities and the same 15 individuals without any abnormalities. However, the student disagreed with himself on 11 individuals. These data, which are a modification of the results reported by Mulrow et al. (10), shows how analysts may organize intrarater reliability data, and it is used later in this section for illustration purposes. For intrarater reliability experiments based on two replicates per subject, analysts may organize the observations as shown in Table 6, where m is the number of subjects rated, and mkl the number of subjects classified in category k on the first occasion and in category l on the second occasion. If the experiment uses three replicates per subject or more, then a more convenient way to organize the data is shown in Table 7 where n is the number of replicates per subject, and nik the number of times subject i is classified into category k. 2.1 Intrarater Reliability: Single Rater and Two Replications When ratings come from a simple experiment based on a single rater, two replicates per subject, and two categories such as described in Table 5, the kappa coefficient of Cohen (2) or an alternative kappa-like statistic may

8

INTRARATER RELIABILITY

Table 5. Distribution of 100 Subjects with Respect to the Presence of Pulmonary Abnormalities Observed on two Occasions by a Medical Student First

Second Observation

Observation

Present

Present Absent Total

Absent

Total

1 15 16

75 25 100

74 10 84

be used to estimate the intrarater reliability coefficient. The medical student who generated Table 5 data could have obtained some of the 89 matches by pure chance because of the small number of response categories limited to two. Consequently, 89% will overestimate student’s self-consistency. Cohen’s (2) solution to this problem was a chance-corrected agreement measure γˆκ , which is known in the literature as kappa, and it is defined as follows: pa − pe γˆκ = (11) 1 − pe where for Table 5 data pa = (74 + 15)/100 = 0.89 is the overall agreement probability, and pe = 0.75 × 0.84 + 0.21 × 0.16 = 0.6636 is the chance-agreement probability. Consequently, the kappa coefficient that measures the medical student intrarater reliability is γˆ = 0.673. According to the Landis and Koch (11) benchmark, a kappa value of this magnitude is deemed substantial. In a more general setting with m subjects, two replicates per subject, and an arbitrary number q of response categories (see Table 6), the kappa coefficient of Cohen (2) is still defined by Equation (11), except the overall

agreement probability pa and the chanceagreement probability pe that are respectively defined as follows: pa =

q q

pkk , and pe = pk+ p+k k=1

(12)

k=1

where pkk = mkk /m, P+k = m+k /m, and Pk+ = mk+ /m The overall agreement probability is the proportion of subjects classified into the exact same category on both occasions (i.e., the diagonal of Table 6). The kappa coefficient will at times yield unduly low values when the ratings suggest high reproducibility. Cicchetti and Feinstein (12), as well as Feinstein and Cicchetti (13) have studied these unexpected results known in the literature as the kappa paradoxes. Several alternative more paradox-resistant coefficients are discussed by Brennan and Prediger (14). A Brennan-Prediger alternative denoted by γˆGI , which is often referred to as the G-Index (GI) and should be considered by practitioners, is defined as follows: γˆGI =

pa − 1/q . 1 − 1/q

(13)

Applied to Table 5 data, the BrennanPrediger coefficient becomes γˆGI = (0.89 −

Table 6. Distribution of m Subjects by Response Category and Replication Number. First-Replication Category 1 .. . k . .. q Total

Second-Replication Response Category 1

···

k

···

q

Total

m11 .. . mk1 . .. mq1 m+1

... .. . ... . .. ... ...

m1k .. . mkk . .. mqk m+k

... .. . ... .. . ... ...

m1q .. . mkq . .. mqq m+k

m+ .. . mk+ . .. mq+ m

INTRARATER RELIABILITY

0.5)/(1 − 0.5) = 0.78, which is slightly higher than the kappa. Aickin (15) presents an interesting discussion about kappa-like Intrarater reliability coefficients and suggests the use of his α coefficient. The α coefficient is based on a sound theory and uses the maximum likelihood estimation of some of its components obtained with a computationally intensive iterative algorithm. Gwet (16) proposed the AC1 statistic as a simple way to resolve the kappa paradoxes. The AC1 coefficient is defined as follows: γˆAC1 =

pa − p(1) e

reliability coefficient. The techniques discussed in this section generalize those of the previous section, and they are suitable for analyzing Table 7 data that involve three replicates or more per subject. All kappa-like statistics presented in the previous section can still be used with Table 7 data. However, the overall agreement probability pa is defined as the probability that two replicates random by chosen from the n replicates associated with a randomly selected subject, are identical. More formally pa is defined as follows:   q m 1

nik (nik − 1) (16) pa = m n(n − 1)

(14)

1 − p(1) e

where pa is defined by Equation (12), and the chance-agreement probability is as follows:

i=1

k=1

pk = (pk+ + p+k )/2

k=1

Concerning the calculation of chance-agreement probability, several versions have been proposed in the literature, most of which are discussed by Conger (17) in the context of inter-rater reliability, rather than in the context of intrarater reliability. Fleiss (18) suggested that chance-agreement probability be estimated as follows:

1
= pk (1 − pk ), where q−1 q

p(1) e

9

(15)

For Table 5 data, p1 = (0.75 + 0.84)/2 = 0/795, p2 = (0.25 + 0.16)/2 = 0.205, and p2 = 1−p1 . Consequently, Gwet’s chanceagreement probability is P(1) e = 2 × 0.795 × 0.205 = 0.32595. The AC1 statistic is then given by γˆAC1 = (0.89 − 0.32595)/(1 − 0.32595) ∼ = 0.84, which is more in line with the observed extent of reproducibility. Gwet (16) extensively discusses the statistical properties of the AC1 statistic as well as the origins of the kappa paradoxes.

p(F) e =

q


p2k where pk =

k=1

m 1
nik m n

(17)

i=1

Note that pk represents the relative number of times that a subject is classified into category k. Fleiss’ generalized kappa is then given by: (F) γˆF = (pa − p(F) e )/(1 − pe ).

2.2 Intrarater Reliability: Single Rater and Multiple Replications

Conger (17) criticized Fleiss’ generalized kappa statistic for not reducing to Cohen’s kappa when the number of replicates is

Using more than two replicates per subject can improve the precision of an intrarater

Table 7. Frequency Distribution of mn Observations by Subject and Response Category. Response Category Subject

1

···

k

···

q

Total

1 .. . i . .. m Total

n11 .. . ni1 . .. nn1 n+1

... .. . ... . .. ... ...

n1k .. . nik . .. nnk n+k

... .. . ... .. . ... ...

n1q .. . niq . .. nnq n+q

n .. . n . .. n mn

10

INTRARATER RELIABILITY

limited to two and proposed the following chance-agreement probability: p(C) e =

q


p2k −

k=1

q
s2k /n

occur in practice, and it should be dealt with using special methods that eliminate the impact of inter-rater variation. The general approach consists of averaging various probabilities calculated independently for each rater as previously discussed.

(18)

k=1

where s2k is the sample variance

2.3 Statistical Inference

1
s2k = (θjk − θ ·k )2 n−1 n

(19)

For an intrarater reliability coefficient to be useful, it must be computed with an acceptable level of precision; this notion can be defined and measured only within a formal framework for statistical inference. This section gives an overview of the main inferential approaches proposed in the literature and provides references for more inquiries. Several authors have proposed frameworks for statistical inference based on various theoretical models. Kraemer et al. (19) review many models that have been proposed in the literature. Kraemer (20) proposed a model under which the kappa coefficient can be interpreted as an intraclass correlation. Donner and Eliasziw (21), Donner and Klar (22), and Aickin (15) have proposed different models that may be useful in different contexts. This model-based approach poses two important problems for practitioners. The first problem stems from the difficulty of knowing which model is most appropriate for a particular situation. The second problem is the dependency of inferential procedures on the validity of the hypothesized model. Fortunately, a different approach to inference based on finite population sampling and widely used in the social sciences can resolve both problems.

j=1

where θ jk = mjk /m is the percent of subjects classified into category k on the jth occasion, and θ ·k is the average of these n values. To compute the variances s2k , k = 1, · · · , q, it could be useful to organize ratings as in Table 8. Both the Fleiss and Conger versions of kappa are vulnerable to the paradox problem previously discussed, and they yield reasonable intrarater reliability coefficients only when pk , the propensity for classification in category k, remain fairly constant from category to category. The generalized version of the AC1 statistic of Gwet (16) is a more paradox-resistant alternative to kappa, and it is based on Equation (14) with the exception that the chance-agreement probability is defined as follows: 1
pk (1 − pk ) q−1 q

p(1) e =

(20)

k=1

where pk is defined as in Equation (17). The situation where intrarater reliability data are collected by multiple raters may

Table 8. Frequency Distribution of mn Observations by Replicate Number and Response Category. Response Category Replication

1

···

k

···

q

Total

1 .. . j . .. n Total

m11 .. . mj1 . .. mn1 m+1

... .. . ... . .. ... ...

m1k .. . mjk . .. mnk m+k

... .. . ... .. . ... ...

m1q .. . miq . .. mnq m+q

m .. . m . .. m mn

INTRARATER RELIABILITY

The randomization approach or designbased inference is a statistical inference framework in which the underlying random process is created by the random selection of m subjects out of a predefined finite population of M subjects of interest. This approach is described in textbooks such as Kish (23) and Cochran (24), and it has been used extensively in the context of inter-rater reliability assessment by Gwet (16,25). The variances of many intrarater reliability coefficients presented in the second section can be found in Gwet (16,25). The first two sections present various approaches for evaluating the reproducibility of continuous and nominal data. These approaches are not recommended for ordinal or interval data, although ordinal clinical measurements such as the presence (no, possible, probable, definite) of a health condition as read on a radiograph, are commonplace. The objective of the next section is to present a generalization of kappa susceptible for use with ordinal and interval data. 3

ORDINAL AND INTERVAL SCORE DATA

Berry and Mielke (26) as well as Janson and Olsson (27,28) have generalized the kappa statistic to handle ordinal and interval data. In addition to being applicable to ordinal and interval data, these extensions can analyze multivariate data of subjects rated on multiple characteristics. Although Berry and Mielke (26) deserve credit for introducing the notions of vector score and Euclidean distance behind these extensions, Janson and Olsson (27) improved and expanded them substantially. Let us consider a simple intrarater reliability study in which a rater must rate all five subjects (m = 5) on two occasions (n = 2) on a three-level nominal scale (q = 3). If the rater classifies subject 1 into category 2 on the first occasion, then the corresponding score can be represented as a vector a11 = (0,1,0), with the second position of digit ‘‘1’’ indicating the category number where the subject is classified. The vector score associated with the classification of subject 1 into category 3 on the second occasion is a12 = (0,0,1). The squared Euclidean distance between a11 and a12 is

11

obtained by summing all three squared differences between the elements of both vectors and is given by: d2 (a11 , a12 ) = (0 − 0)2 + (0 − 1)2 + (1 − 0)2 = 2 Following Janon and Olsson (26), Cohen’s kappa coefficient can re-expressed as follows: 1
2 d ( ai1 , ai2 ) 5 5

γˆJO = 1 −

i=1 5 5

1

2 d ( ai1 , aj2 ) 52

(21)

i=1 j=1

The kappa coefficient as written in Equation (21) depends solely on several distance functions. Its generalization relies on the distance function’s ability to handle ordinal and interval data. If the scoring is carried out on a three-level ordinal scale, then each score will represent a single rank instead of threedimensional vector of 0s and 1s. If the categories in Table 6 are ordinal, then Equation (21) can be adapted to that data and yield the following more efficient kappa coefficient: q  q 

γˆJO = 1 −

pkl (k − l)2

k=1i=1 q q  

(22)

pk+ p+l (k − l)2

k=1l=1

To illustrate the use of kappa with ordinal scores, let us consider Table 9 data, which are a modification of the initial chest radiograph data that Albaum et al. (29) analyzed. A radiologist has examined 100 initial chest radiographs on two occasions to determine the presence of a radiographic pulmonary infiltrate. The four levels of the measurement scale for this experiment are ‘‘No,’’ ‘‘Possible,’’ ‘‘Probable,’’ and ‘‘Definite.’’ Because classifications of radiographs into the ‘‘Probable’’ and ‘‘Definite’’ categories agree more often than those in the ‘‘No’’ and ‘‘Definite’’ categories, the use of classic kappa of Cohen (2) will dramatically underestimate the intrarater reliability. Cohen’s kappa for Table 9 is given by γˆk = (0.57 − 0.3151)/(1 − 0.3151) ≈ 0.37. The generalized version of kappa based on Equation

12

INTRARATER RELIABILITY

Table 9. Distribution of 100 Subjects by Presence of Radiographic Pulmonary inFiltrate and Assessment Time Radiographic Assessment

Radiographic Assessment in Time 2

in Time 1

No

Possible

Probable

Definite

TOTAL

No Possible Probable Definite TOTAL

6 2 2 1 11

7 7 4 4 22

2 6 7 7 22

1 2 5 37 45

16 17 18 49 100

(22) yields an intrarater reliability coefficient of γˆJO = 1 − 0.89/2.41 = 0.63. This generalized version of kappa yields an intrarater reliability coefficient substantially higher and accounts for partial agreement in a more effective way. 4

CONCLUDING REMARKS

This article introduces the notion of intrarater reliability for continuous, nominal, ordinal, as well as interval data. Although the intraclass correlation coefficient is the measure of choice for continuous data, kappa and kappa-like measures defined by Equations (11), (13), (14), and (18)–(20) are generally recommended for nominal data. The extension of kappa to ordinal data is more efficient than classic kappa when the data is ordinal, and it is an important addition to the kappa literature. The literature on inter-rater reliability is far more extensive than that on intrarater reliability, particularly for discrete data, which is explained partially by the tendency researchers have to underestimate the importance of data reproducibility. Although many techniques were developed to measure interrater reliability, very few specifically address the problem of intrarater reliability. In this article, we have adapted some inter-rater reliability estimation procedures so they can be used for computing intrarater reliability coefficients. Unlike inter-rater reliability experiments that involve multiple raters, multiple subjects, and a single replicate per subject, intrarater reliability experiments typically involve a single rater and several replicates per subject. Consequently interrater reliability methods have been modified

by considering the replicates as ratings from different independent raters. Several authors, such as Fleiss and Cohen (30), Kraemer (20), and others, have attempted to interpret kappa as well as other kappa-like reliability measures as a form of intraclass correlation under certain conditions. The main justification for this effort stems from the need to link kappa to a population parameter and to create a framework for statistical inference. So far, no clear-cut theory can establish such a link in a broad setting. The connection of kappa to the intraclass correlation is unnecessary to have a good statistical inference framework. A satisfactory solution to this problem is the use of the finite population inference framework discussed in Gwet (16, 25). REFERENCES 1. S. M. Grundy, Second report of the expert panel on detection, evaluation, and treatment of high blood cholesterol in adults (Adult Treatment Panel II). 1993; National Institutes of Health, NIH Publication No. 93-3095. 2. J. Cohen, A coefficient of agreement for nominal scales. Educat. Psychol. Measurem. 1960; 20: 37–46. 3. R. L. Ebel, Estimation of the reliability of ratings. Psychometrika 1951; 16: 407–424. 4. J. J. Barko, The intraclass correlation coefficient as a measure of reliability. Psychol. Reports 1966; 19: 3–11. 5. P. E. Shrout and J. L. Fleiss, Intraclass correlations: uses in assessing rater reliability. Psychol. Bull. 1979; 86: 420–428. 6. J. M. Lachin, The role of measurement reliability in clinical trials. Clin. Trials 2004; 1: 553–566. 7. National Health and Nutrition Examination Survey. 2005.

INTRARATER RELIABILITY 8. K. O. McGraw and S. P. Wong, Forming inferences about some intraclass correlation coefficients. Psychol. Methods 1996; 1: 30–46. 9. B. Giraudeau and J. Y. Mary, Planning a reproducibility study: how many subjects and how many replicates per subject for an expected width of the 95 per cent confidence interval of the intraclass correlation coefficient. Stats. Med. 2001; 20: 3205–3214. 10. C. D. Mulrow, B. L. Dolmatch, E. R. Delong, J. R. Feussner, M. C. Benyunes, J. L. Dietz, S. K. Lucas, E. D. Pisano, L. P. Svetkey, B. D. Volpp, R. E. Ware, and F. A. Neelon, Observer variability in the pulmonary examination. J. Gen. Intern Med. 2007; 1: 364–367. 11. J. R. Landis and G. G. Koch, The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159–174. 12. D. V. Cicchetti and A. R. Feinstein, High agreement but low kappa: II. Resolving the paradoxes. J. Clin. Epidemiol. 1990; 43: 551–558. 13. A. R. Feinstein and D. V. Cicchetti, High agreement but low kappa: I. The problems of two paradoxes. J. Clin. Epidemiol. 1990; 43: 543–549. 14. Brennan, RL, and Prediger, DJ. Coefficient kappa: some uses, misuses, and alternatives. Educat. Psychol Measurem. 1981; 41: 687–699. 15. M. Aickin, Maximum likelihood estimation of agreement in the constant predictive probability model, and its relation to Cohen’s kappa. Biometrics 1990; 46: 293–302. 16. K. L. Gwet, Computing inter-rater reliability and its variance in the presence of high agreement. Br. J. Mathemat. Stat. Psychol. 2008. In press. 17. A. J. Conger, Integration and generalization of kappas for multiple raters. Psychol. Bull. 1980; 88: 322–328. 18. J. L. Fleiss, Measuring nominal scale agreement among many raters. Psychol. Bull. 1971; 76: 378–382. 19. H. C. Kraemer, V. S. Periyakoil, and A. Noda, Kappa coefficients in medical research. Stats. Med. 2002; 21: 2109–2129.

13

20. H. C. Kraemer, Ramifications of a population model for κ as a coefficient of reliability. Psychometrika 1979; 44: 461–472. 21. A. Donner, M. A. Eliasziw, A hierarchical approach to inferences concerning interobserver agreement for multinomial data. Stats. Med. 1997; 16: 1097–1106. 22. Donner, A, Klar, N. The statistical analysis of kappa statistics in multiple samples. Journal of Clinical Epidemiology. 1996; 49(9): 1053–1058. 23. K. Kish, Survey Sampling. New York: Wiley, 1965. 24. W. G. Cochran, Sampling Techniques, 3rd ed. New York: Wiley, 1977. 25. K. L. Gwet, Variance estimation of nominalscale inter-rater reliability with random selection of raters. Psychometrika. 2008. In press. 26. K. J. Berry and P. W. Mielke Jr., A generalization of Cohen’s Kappa agreement measure to interval measurement and multiple raters. Educat. Psychol. Measurem. 1988; 48: 921–933. 27. H. Janson and U. Olsson, A measure of agreement for interval or nominal multivariate observations. Educat. Psychol. Measurem. 2001; 61: 277–289. 28. H. Janson and U. Olsson, A measure of agreement for interval or nominal multivariate observations by different sets of judges. Educat. Psychol. Measurem. 2004; 64: 62–70. 29. M. N. Albaum, L. C. Hill, M. Murphy, Y. H. Li, C. R. Fuhrman, C. A. Britton, W. N. Kapoor, and M. J. Fine, PORT Investigators. Interobserver reliability of the chest radiograph in community-acquired pneumonia. CHEST 1996; 110: 343–350. 30. J. L. Fleiss and J. Cohen, The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educat. Psychol. Measurem. 1973; 33: 613–619.

FURTHER READING D. C. Montgomery, Design and Analysis of Experiments. New York: John Wiley & Sons, 2004. D. T. Haley, Using a New Inter-rater Reliability Statistic, 2007. Available: http://computingreports.open.ac.uk/index.php/2007/200716. ANOVA Using MS Excel. Available: http:// higheredbcs.wiley.com/legacy/college/mann/ 0471755303/excel manual/ch12.pdf.

14

INTRARATER RELIABILITY

CROSS-REFERENCES Inter-Rater Reliability Intraclass Correlation Coefficient Kappa Statistic Weighted Kappa Analysis of Variance (ANOVA)

INVESTIGATIONAL DEVICE EXEMPTION (IDE)

apply to devices in commercial distribution. Sponsors do not need to submit a PMA or Premarket Notification 510(k), register their establishment, or list the device while the device is under investigation. Sponsors of IDE’s are also exempt from the Quality System (QS) Regulation except for the requirements for design control.

An Investigational Device Exemption (IDE) refers to the regulations under 21 CFR (Code of Federal Regulations) 812. An approved IDE means that the Institutional Review Board (IRB) [and the Food and Drug Administration (FDA) for significant risk devices] has approved the sponsor’s study application and that all requirements under 21 CFR 812 are met. An IDE allows the investigational device to be used in a clinical study to collect safety and effectiveness data required to support a Premarket Approval (PMA) application or a Premarket Notification [510(k)] submission to FDA. Clinical studies are most often conducted to support a PMA. Only a small percentage of 510(k)’s require clinical data to support the application. Investigational use also includes clinical evaluation of certain modifications or new intended uses of legally marketed devices. All clinical evaluations of investigational devices, unless exempt, must have an approved IDE before the study is initiated. Clinical evaluation of devices that have not been cleared for marketing requires: • an IDE approved by an IRB. If the study

• • • •

involves a significant risk device, the IDE must also be approved by the FDA; informed consent from all patients; labeling for investigational use only; monitoring of the study and; required records and reports.

An approved IDE permits a device to be shipped lawfully for the purpose of conducting investigations of the device without complying with other requirements of the Food, Drug, and Cosmetic Act (Act) that would This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cdrh/devadvice/ide/index. shtml) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

INVESTIGATIONAL NEW DRUG APPLICATION PROCESS (IND)

an investigation and under whose immediate direction the investigational drug is administered or dispensed. A physician might submit a research IND to propose studying an unapproved drug or an approved product for a new indication or in a new patient population. 2. Emergency Use IND allows the FDA to authorize use of an experimental drug in an emergency situation that does not allow time for submission of an IND in accordance with 21 Code of Federal Regulations (CFR) Section 312.23 or Section 312.24. It is also used for patients who do not meet the criteria of an existing study protocol or when an approved study protocol does not exist. 3. Treatment IND is submitted for experimental drugs that show promise in clinical testing for serious or immediately life-threatening conditions while the final clinical work is conducted and the FDA review takes place.

In the United States, current federal law requires that a drug be the subject of an approved marketing application before it is transported or distributed across state lines. Because a sponsor will probably want to ship the investigational drug to clinical investigators in many states, it must seek an exemption from that legal requirement. The Investigational New Drug (IND) process (Fig. 1) is the means through which the sponsor technically obtains this exemption from the U.S. Food and Drug Administration (FDA). During a new drug’s early preclinical development, the sponsor’s primary goal is to determine whether the product is reasonably safe for initial use in humans and if the compound exhibits pharmacologic activity that justifies commercial development. When a product is identified as a viable candidate for further development, the sponsor then focuses on collecting the data and information necessary to establish that the product will not expose humans to unreasonable risks when used in limited, early-stage clinical studies. The FDA’s role in the development of a new drug begins after the drug’s sponsor (usually the manufacturer or potential marketer) has screened the new molecule for pharmacologic activity and acute toxicity potential in animals, and wants to test its diagnostic or therapeutic potential in humans. At that point, the molecule changes in legal status under the Federal Food, Drug, and Cosmetic Act and becomes a new drug subject to specific requirements of the drug regulatory system. There are three IND types:

There are two IND categories: 1. Commercial 2. Research (noncommercial) The IND application must contain information in three broad areas: 1. Animal Pharmacology and Toxicology Studies. Preclinical data to permit an assessment as to whether the product is reasonably safe for initial testing in humans. Also included are any previous experience with the drug in humans (often foreign use). 2. Manufacturing Information. Information pertaining to the composition, manufacturer, stability, and controls used for manufacturing the drug substance and the drug product. This information is assessed to ensure that the company can adequately produce and supply consistent batches of the drug.

1. Investigator IND is submitted by a physician, who both initiates and conducts This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/regulatory/applications/ ind page 1.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

INVESTIGATIONAL NEW DRUG APPLICATION PROCESS (IND)

Applicant (Drug Sponsor)

IND

Review by CDER

Medical

Pharmacology/ Toxicology

Chemistry

Sponsor Submits New Data

Safety Review

Safety Acceptable for Study to Proceed ?

Statistical

No

No

Clinical Hold Decision Yes

Yes Complete Reviews

Reviews Complete and Acceptable ?

Notify Sponsor

No

Sponsor Notified of Deficiencies

Yes No Deficiencies

Study Ongoing*

* While sponsor answers any deficiencies Figure 1. Flowchart of the Investigational New Drug process.

3. Clinical Protocols and Investigator Information. Detailed protocols for proposed clinical studies to assess whether the initial-phase trials will expose study participants to unnecessary risks. Also, information must be provided on the qualifications of clinical investigators— professionals (generally physicians) who

oversee the administration of the experimental agent—to assess whether they are qualified to fulfill their clinical trial duties. Finally, commitments must be made to obtain informed consent from the research participants, to obtain review of the study by an institutional review board, and to adhere to the investigational new drug regulations.

INVESTIGATIONAL NEW DRUG APPLICATION PROCESS (IND)

Once the IND has been submitted, the sponsor must wait 30 calendar days before initiating any clinical trials. During this time, the FDA has an opportunity to review the IND for safety to ensure that research participants will not be subjected to unreasonable risk.

3

INVESTIGATIONAL PRODUCT

The investigator should ensure that the investigational product(s) are used only in accordance with the approved protocol. The investigator, or a person designated by the investigator/institution, should explain the correct use of the investigational product(s) to each subject and should check, at intervals appropriate for the trial, that each subject is following the instructions properly. The sponsor is responsible for supplying the investigator(s)/institution(s) with the investigational product(s). The sponsor should not supply an investigator/institution with the investigational product(s) until the sponsor obtains all required documentation [e.g., approval/favorable opinion from IRB (Institutional Review Board)/IEC (Independent Ethics Committee) and regulatory authority(ies)]. The sponsor should ensure that written procedures include instructions that the investigator/institution should follow for the handling and storage of investigational product(s) for the trial and documentation thereof. The procedures should address adequate and safe receipt, handling, storage, dispensing, retrieval of unused product from subjects, and return of unused investigational product(s) to the sponsor [or alternative disposition if authorized by the sponsor and in compliance with the applicable regulatory requirement(s)]. The sponsor should

An Investigational Product is a pharmaceutical form of an active ingredient or placebo being tested or used as a reference in a clinical trial, including a product with a marketing authorization when used or assembled (formulated or packaged) in a way different from the approved form, when used for an unapproved indication, or when used to gain further information about an approved use. Responsibility for investigational product(s) accountability at the trial site(s) rests with the investigator/institution. Where allowed/required, the investigator/ institution may/should assign some or all of the duties of the investigator/institution for investigational product(s) accountability at the trial site(s) to an appropriate pharmacist or another appropriate individual who is under the supervision of the investigator/institution. The investigator/institution and/or a pharmacist or other appropriate individual, who is designated by the investigator/institution, should maintain records of the delivery of the product to the trial site, the inventory at the site, the use by each subject, and the return to the sponsor or alternative disposition of unused product(s). These records should include dates, quantities, batch/serial numbers, expiration dates (if applicable), and the unique code numbers assigned to the investigational product(s) and trial subjects. Investigators should maintain records that document adequately that the subjects were provided the doses specified by the protocol and should reconcile all investigational product(s) received from the sponsor. The investigational product(s) should be stored as specified by the sponsor and in accordance with applicable regulatory requirement(s).

• Ensure timely delivery of investigational

product(s) to the investigator(s), • Maintain records that document ship-

ment, receipt, disposition, return, and destruction of the investigational product(s), • Maintain a system for retrieving investigational products and documenting this retrieval (e.g., for deficient product recall, reclaim after trial completion, and expired product reclaim), and • Maintain a system for the disposition of unused investigational product(s) and for the documentation of this disposition.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

INVESTIGATIONAL PRODUCT

The sponsor should • Take steps to ensure that the investi-

gational product(s) are stable over the period of use, and • Maintain sufficient quantities of the investigational product(s) used in the trials to reconfirm specifications, should this become necessary, and to maintain records of batch sample analyses and characteristics. To the extent that stability permits, samples should be retained either until the analyses of the trial data are complete or as required by the applicable regulatory requirement(s), whichever represents the longer retention period.

INVESTIGATOR

The investigator(s) should be qualified by education, training, and experience to assume responsibility for the proper conduct of the trial; should meet all the qualifications specified by the applicable regulatory requirement(s); and should provide evidence of such qualifications through up-to-date curriculum vitae and/or other relevant documentation requested by the sponsor, the IRB/IEC, and/or the regulatory authority(ies). The investigator should be thoroughly familiar with the appropriate use of the investigational product(s) as described in the protocol, in the current Investigator’s Brochure, in the product information, and in other information sources provided by the sponsor. The investigator should be aware of, and should comply with, GCP and the applicable regulatory requirements. The investigator/institution should permit monitoring and auditing by the sponsor and inspection by the appropriate regulatory authority(ies). The investigator should maintain a list of appropriately qualified persons to whom the investigator has delegated significant trial-related duties. The investigator should be able to demonstrate (e.g., based on retrospective data) a potential for recruiting the required number of suitable subjects within the agreed recruitment period. The investigator should have sufficient time to properly conduct and complete the trial within the agreed trial period. The investigator should have available an adequate number of qualified staff and adequate facilities for the foreseen duration of the trial to conduct the trial properly and safely. The investigator should ensure that all persons assisting with the trial are adequately informed about the protocol, the investigational product(s), and their trialrelated duties and functions. During and following the participation of a subject in a trial, the investigator/institution should ensure that adequate medical care is provided to the subject for any adverse events, including clinically significant laboratory values, related to the trial. The investigator/institution should inform a subject when medical care is needed for intercurrent illness(es) of which the investigator becomes

An Investigator is a person responsible for conducting the clinical trial at a trial site. If a trial is conducted by a team of individuals at a trial site, the investigator is the responsible leader of the team and may be called the principal investigator. The sponsor is responsible for selecting the investigator(s)/institution(s). If a coordinating committee and/or coordinating investigator(s) are to be used in multicenter trials, their organization and/or selection are the responsibility of the sponsor. Before entering an agreement with an investigator/institution to conduct a trial, the sponsor should provide the investigator(s)/institution(s) with the protocol and an up-to-date Investigator’s Brochure and should provide sufficient time for the investigator/institution to review the protocol and the information provided. The sponsor should obtain the agreement of the investigator/institution • To conduct the trial in compliance with

Good Clinical Practice (GCP), with the applicable regulatory requirement(s), and with the protocol agreed to by the sponsor and given approval/favorable opinion by the IRB (Institutional Review Board)/IEC (Independent Ethics Committee), • To comply with procedures for data recording/reporting, • To permit monitoring, auditing, and inspection, and • To retain the essential documents that should be in the investigator/institution files until the sponsor informs the investigator/institution that these documents no longer are needed. The sponsor and the investigator/institution should sign the protocol or an alternative document to confirm this agreement. This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

INVESTIGATOR

aware. It is recommended that the investigator inform the primary physician of the subject about the participation of the subject in the trial if the subject has a primary physician and if the subject agrees to the primary physician being informed. Although a subject is not obliged to give his/her reason(s) for withdrawing prematurely from a trial, the investigator should make a reasonable effort to ascertain the reason(s), while fully respecting the rights of the subject.

INVESTIGATOR/INSTITUTION

market segment with a 38% share of total clinical trial grants. Nearly two thirds of the 9,000 investigative sites that conduct clinical trials in the United States annually fall into the part-time segment. Select sites in this segment are conducting as many as 25 or 30 clinical trials each year whereas other sites may conduct less than one trial annually. As industry sponsors continue to favor conducting their clinical trials in actualuse, community-based settings, and given the relatively low clinical trial operating costs within part-time sites, this segment is well positioned to continue to grow the faster than its peers (5). As Figure 1 shows, AMCs receive nearly all clinical research grants from government agencies and from foundations, and approximately 36% of all clinical trial grants from private sector pharmaceutical, biotechnology, and medical device companies (1, 6). The dedicated investigative site segment captures 26% of the market for clinical trial grants from industry. Dedicated sites derive nearly 100% of their annual revenue from clinical trial grants. This segment has undergone profound structural changes over the past decade and now includes a wide variety of private entities that include site management organizations, stand-alone dedicated research facilities, and managed site networks. These various entities will be discussed shortly (1).

KENNETH A. GETZ Tufts Center for the Study of Drug Development Boston, Massachusetts

Overall spending on clinical research grants to investigative sites exceeded $20 billion in 2007. Government agencies and foundations contribute $9.9 billion—approximately 50%—to the total amount spent on clinical research grants. However, these sponsors support a broad variety of activities that include patient-oriented research, epidemiologic and behavioral studies, outcomes research, and health services research (1). In 2007, government agencies and foundations spent approximately $1.9 billion on clinical trials of new medical interventions. The lion’s share of spending on clinical trials came from private sector sources that include pharmaceutical, biotechnology, and medical device manufacturers. Industry spent more than $10 billion on clinical trial grants to investigative sites in 2007 (2). Given the high failure rates of new molecular entities—which include both traditional chemical entities and biologics—under clinical investigation, the U.S. Department of Commerce estimates that industry sponsors spend more than $1 billion in capitalized costs to bring a single new drug through the 17-year R&D cycle and into the market place (3, 4). This figure includes the sponsor’s costs to develop drugs that fail to demonstrate efficacy and present a poor safety profile. 1

2

SEGMENT PROFILE: AMCs

Until the early 1990s, AMCs were the primary and the predominant home of industrysponsored clinical trials. As these programs became larger, more complex, and costly, industry sponsors grew tired of the inherent inefficiencies in working with academia that include protracted contractual and budget negotiations, bureaucratic and slow-moving institutional review boards (IRBs) and higher relative cost associated with poorer performance. (7) A select group of institutions— which include the University of Kansas Medical Center, Columbia-Presbyterian Medical Center, Johns Hopkins University School

LANDSCAPE SIZE AND STRUCTURE

Today, the market for clinical research grants is composed of three primary segments: academic medical centers (AMCs), part-time investigative sites, and dedicated investigative sites. The part-time segment consists of physicians in private, community settings that derive most (85% or more) of their annual revenue from clinical practice. In 2007, this highly fragmented and transient group has surpassed AMCs as the largest

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

INVESTIGATOR/INSTITUTION

Privatization of US Clinical Investigators

70% 51%

41%

36%

Figure 1. Privatization of US clinical investigators.

of Medicine, University of Rochester Medical School, New York University Medical Center, and George Washington Medical Center—centralized their administrative clinical research functions to compete with community-based investigative sites. Institutional culture, philosophy, and politics made it difficult to implement and to establish these offices. Central clinical trial offices serve faculty and study personnel by freeing up investigator and staff time; improving business development and marketing; and accelerating contract, budget, and IRB approval processes. Specific initiatives by central offices included the following: (1) building databases of a given institutions clinical research capabilities and patient populations to respond quickly to sponsor inquiries, (2) working with the IRB to expedite the approval process, (3) obtaining and negotiating new contracts on behalf of faculty, (4) training study coordinators and investigators, (5) providing patient recruitment support, and (6) tracking study performance. Today, most (70%) of the nearly 130 AMCs have established central offices to assist in supporting clinical research activities at their institutions (6). Still, AMCs continue to lose market share of clinical trial grants from industry. This loss is caused by several factors: many academic-based investigative sites

report that they receive poor returns on infrastructure investment, minimal institutional support, and limited incentive given renewed emphasis on the more prestigious governmentfunded clinical research programs as NIH funding surged (1). During the most recent 5-year period between 2002 and 2007, for example, total NIH funding grew nearly 4% annually to $28.8 billion FY2007. Moving forward, however, NIH funding growth is expected to level off and the NIH is beginning to phase out its long-established General Clinical Research Center (GCRC) program to make room for the new Clinical and Translational Science Awards (CTSA) model. The result will be an intensification of competition specifically for NIH clinical research grants (2). Many AMCs are turning their sights again toward industry-sponsored clinical trials as a vital source for much-needed financial resources in the near future. Fortunately, many clinical research professionals within pharmaceutical and biotechnology companies remain interested in forging, and in building, stronger clinical research relationships with academia. Whereas many AMCs are considering whether to make a more concerted push to expand their involvement in industry-funded clinical research activity, others have been implementing new strategies and tactics designed to improve their ability to compete

INVESTIGATOR/INSTITUTION

for industry study grants that include the implementation of services to expedite study start-up (e.g., contract and budget approvals, IRB review and approval), to provide higher quality training of study investigators and staff, and to accelerate the collection and management of clinical research data. Rising financial pressures will prompt many AMCs to expand their involvement in industry-funded clinical trials over the next several years. The NIH Roadmap for Medical Research and its push to establish Clinical and Translational Science Awards will serve as a catalyst in driving up AMC’s commitment to industry-sponsored clinical trials (6). By 2012, the NIH’s CTSA initiative will provide approximately $500 million annually to 60 AMCs to encourage more effective interaction and communication between clinical research and medical practice. 3 SEGMENT PROFILE: PART-TIME INVESTIGATIVE SITES Part-time investigative sites are a vital component of the study conduct services engine, yet this group of investigative sites is the most difficult to define given its highly fragmented nature. Investigative sites in this segment are very transient with extremely variable clinical research infrastructure and experience. Top sites in this segment conduct as many as 25 or 30 clinical trials each year, whereas bottom sites may do less than one annually. Although 40% of this segment has full-time coordinators, 31% of the segment does not have a single research nurse on staff. Turnover is the highest among investigators in this segment. One of every three part-time investigators who file the FDA-required 1572 form discontinues their involvement once the trial has ended (8). Sponsors have long favored placing a significant portion of their phase II and III clinical studies in actual-use settings. With rapidly expanding pipelines of promising new chemical entities, part-time sites can expect to see increasing numbers of annual clinical trial opportunities in the future. Additionally, industry anticipates that the demand for clinical research centers will exceed supply within the next several years. Given their

3

more flexible involvement in conducting clinical trials, part-time investigative sites play a valuable role in providing much needed capacity. With typically low overhead and minimal fixed costs, part-time sites can play a broader role as a variable study conduct resource that can transition flexibly into and out of clinical projects on demand. In essence, part-time sites may find larger opportunities to act as contract study conduct partners with dedicated clinical research centers. The dramatic growth in post-approval research spending represents another important trend that will increase the demand for part-time investigators. Post-approval studies are now conducted by sponsor companies for a wide variety of purposes: to extend knowledge about efficacy, safety and outcomes within actual use settings, to position newly launched drugs against standard and competing therapies, to prime the market and stimulate drug use, to observe prescribing behaviors and patient experiences, and to extend drug life cycles by targeting new populations and new markets. The FDA is also increasing its pressure on companies to continue the testing of new therapies after they are launched. Typically, phase IV research programs are conducted in community-based, clinical practice settings. Part-time sites report modest growth in year-over-year revenue and relatively high profitability, which is largely caused by the flexible and transient way that part-time sites approach conducting clinical trials. Although many very experienced part-time investigators with established clinical trial operations exist, as a group this segment is composed largely of investigative sites with wide variability in operating structure, experience, and performance. Annual revenue reported by part-time investigative sites ranges from $25,000 annually to as much as $1,000,000– $2,000,000 (1). Most (83%) part-time investigative sites report that they primarily focus on one or two specialty research areas. Few part-time sites consider themselves multispecialty research centers. Nearly half of all part-time investigative sites operate out of a group or network of

4

INVESTIGATOR/INSTITUTION

practicing physicians. Many sites in this segment report that several physicians within their provider network have been involved in conducting at least one clinical trial during the past five years. On average, part-time sites have one study coordinator on staff. The average age of a principal investigator in a part-time investigative site setting is 50 years. Most investigators are getting involved in industry-sponsored clinical research at later stages in their professional careers. These individuals are more established and more financially secure. Less than 10% of part-time principal investigators who conduct industry-sponsored clinical trials are under the age of 40. An estimated 45% of the more than 40,000 active clinical investigators— some 16,000 to 18,000 people—in any given year are conducting their clinical trials on a part-time basis in independent research centers (1). Operating profits among part-time sites vary widely—and it is a function of research infrastructure. Some of the largest part-time sites report operating profits as low as 5–7% of net revenue. As expected, part-time sites that conducted only a few clinical trials in 2006 report operating profits as high as 60–70% of grant revenue. Almost 20 cents of every clinical grant award dollar is profit for the part-time site. Minimal infrastructure and integrated and shared resources help part-time sites maintain higher relative operating margins. In addition, these sites are well positioned to handle the typical cash flow problems that result from late sponsor and CRO (contract research organization) payments. Part-time sites have a steady clinical practice revenue stream. But it is certainly not the economics alone that attracts practicing physicians to conduct clinical trials. These physicians are drawn to the science of new drug development, and they want to offer their patients access to medicines on the frontiers of medical knowledge. Both small and large part-time research centers have unique needs that must be addressed if this segment is to meet increasing demand and performance standards. To reach a comparable level of performance across this segment, experienced and new

entrant part-time sites will need to improve their abilities to start studies and enroll patients, to comply with evolving federal guidelines and regulations, to use emerging electronic solutions technologies more effectively, and to improve their operating efficiencies. At this time, part-time sites largely learn to conduct clinical research through actual experience. A more systematic and uniform approach, which is integrated into health professional training and supported by research sponsors, will likely be necessary. Ultimately, this study conduct services segment holds great promise in meeting growing demand and in offering much needed capacity. 4 SEGMENT PROFILE: DEDICATED INVESTIGATIVE SITES An estimated 500 dedicated investigative sites operate in the United States currently. Approximately two thirds of this total is stand-alone research centers. Close to 35% of all dedicated sites is part of a network (1). Dedicated sites are typically multispecialty and are relatively sophisticated in terms of their approach to managing their clinical research activity— from financial management to staffing and recordkeeping. They conduct an average of 30 clinical research studies per year. Similar to parttime community-based investigative sites, dedicated sites interact with centralized IRBs for human subject protection oversight. Most dedicated sites—nearly 90%— report that they are profitable with net operating margins of 11–15%. Dedicated investigative sites do not have a clinical practice with which to draw patients for their clinical trials. Instead, dedicated sites rely heavily on advertising and promotion to attract study volunteers. Approximately 40% of dedicated sites report that they have a full or part-time patient recruitment specialist on staff (1). In the mid-1990s, a group of dedicated sites formed to meet growing sponsor demands for clinical trial cost and time efficiency and improved data. The conceptual promise offered by these new Site Management Organizations (SMOs) was compelling:

INVESTIGATOR/INSTITUTION • Centralized clinical research operations • Standardized contracts and operating

procedures • Trained and accredited staff • New technologies to manage informa-

tion and to track performance management of patient recruitment and retention Systematic management of clinical data Streamlined regulatory and legal review and approval processes Reduced fixed costs to offer more competitive pricing Applied business and management principles

• Systematic • • • •

But since their introduction 15 years ago, most SMOs have struggled to deliver on these conceptual promises, and they have been through a wide variety of incarnations. The first SMOs emerged under a cloud of scandal when Future HealthCare, which was an early entrant, was indicted for manipulating its financial records to influence the capital markets. The mid-1990s saw a wave of new entrants— owned-site and affiliation model SMOs offering single- and multispecialty expertise—including Affiliated Research Centers (ARC), Clinical Studies Limited (CSL), Collaborative Clinical Research, Hill Top Research, Health Advance Institute (HAI), InSite Clinical Trials, Protocare, Integrated Neuroscience Consortium (INC), Rheumatology Research International (RRI), and several hybrid CRO-SMOs such as Clinicor, MDS Harris, and Scirex In the late 1990s, SMOs entered their most active period of venture capital fund raising and aggressively pursued expansion and diversification strategies. Collaborative Clinical raised $42 million in a 1996 initial public offering. ARC, INC, InSite, and HAI each closed rounds of venture capital financing. Phymatrix, which is a $200 million physician’s practice management group, acquired CSL for $85 million. Having raised more than $14 million in 1999, Radiant Research purchased peer SMO Hill Top Research. And nTouch Research (formerly Novum) raised $8 million in venture capital funding to double the size of its investigative site network through the acquisition of HAI.

5

By the early 2000s, many SMOs had exited the market whereas others diversified their services even more, venturing into traditional CRO and patient recruitment offerings. To name but a few, after a public offering, Collaborative Clinical Research renamed itself Datatrak and announced that it would be exiting the SMO business to become a provider of electronic clinical trial technology solutions. Integrated Neuroscience Consortium, MDS, and RRI focused attention on offering CRO services; The Essential Group (formerly ARC) abandoned its SMO services to focus on offering contract patient recruitment services; in 2005 nTouch Research was acquired by CRO Accelovance; InSync Research shut down its operations in 2000 after selling four of its seven sites to Radiant Research; Clincare, which had hoped to expand its regional network of eight owned sites, also exited the business in 2000. In late 2001, ICSL sold its SMO assets to Comprehensive Neuroscience. And in 2003, Radiant Research completed the acquisition of Protocare—another top five SMO—thereby expanding Radiant’s network to nearly 60 sites. Radiant Research has clearly demonstrated that it is an outlier—it is growing and operating profitably among traditional SMOs that have either exited or struggled to survive in this market. Today, Radiant owns and operates more than 30 dedicated sites, and it has 550 employees including 225 study coordinators. With the exception of Radiant Research, industry observers have concluded that traditional SMOs failed to demonstrate and deliver the value of centralized investigative site operating controls. SMO management structures were cumbersome and challenged the organization’s ability to operate profitably for a sustainable period of time. Traditional SMOs struggled to compete for sufficient levels of new business, to manage positive cash flows, and they failed to achieve revenue and earnings growth that would satisfy their investors. SMO insiders express the failures of traditional SMOs differently: Many believe that research sponsors failed to use the SMO properly by neglecting to empower the SMO to manage an entire study.

6

INVESTIGATOR/INSTITUTION

The past several years has observed strong growth in new structures among managed site networks. The Tufts Center for the Study of Drug Development has been tracking this development. These new players tend to operate regionally, and they have extremely lean centralized operations. They are composed of small networks of sites—on average five—that are connected loosely through minimal standardized management services provided across the network. Basic management services include contract, budget, and regulatory assistance. Although many decentralized site networks have established exclusive arrangements with their investigators, they encourage autonomy and direct interaction with study sponsors. As a result, they address sponsor’s ‘‘investigator-centric’’ preferences while offering minimal—although essential—operating support. Examples of decentralized site networks include Pivotal Research, Benchmark Research, ResearchAcrossAmerica, and RxTrials. Decentralized site networks are building momentum. They generated $215 million in study grant revenue in 2006, and they are growing 9.3% annually. They seem well positioned to capture growing market share over the next several years while containing operating costs (1).

5 THE LANDSCAPE MOVES OVERSEAS Recently, a dramatic shift has occurred in the use of ex–U.S.-based investigative sites. As Figure 2 indicates, major pharmaceutical and biotechnology companies have been placing an increasing proportion of their clinical trials at offshore locations throughout Central and Eastern Europe, Latin America, and parts of Asia. Several sponsors report that they now routinely place most of their clinical trials among ex–U.S.-based investigative sites. This shift has major implications not only for U.S.-based investigative sites, but also for sponsors and for CROs who are looking to optimize their relationships with an increasingly global network of investigators. It is widely accepted that conducting trials overseas is far less expensive and a healthy supply of Western-trained physicians are located there, who are eager to serve as investigators and to study coordinators. Drug shipment issues have largely been overcome and the adoption of e-clinical technology solutions helped to address operating support issues that used to bottleneck overseas projects. Most importantly, the abundance of treatment-na¨ıve patients abroad has translated into speedier clinical trial recruitment and improved retention rates. In the developing world, vast numbers of patients suffer

An Increasingly Global Mix of FDA- Regulated Investigators

Figure 2. An increasingly global mix of FDA-regulated clinical investigators.

INVESTIGATOR/INSTITUTION

from diseases that range from the familiar to the exotic. These patients are often eager to enroll in clinical trials at the request of their physicians and to gain access to free medical care, tests, and complimentary therapies that they could otherwise not afford. In an analysis of 1572 filings with the FDA, the Tufts Center for the Study of Drug Development has found that most investigative sites still hail from the United States, but the geographic mix is changing rapidly. A decade ago, less than 1 in 10 FDA-regulated investigators was based outside the U.S.; in 2006, 40% of FDA-regulated investigators was ex-U.S. The Tufts Center also found that, whereas the overall number of principal investigators within the United States has been growing by 5.5% annually since 2002, the number of FDA-regulated principal investigators based outside the US has been growing by more than 15% annually during that same time period (8). Major pharmaceutical companies have already tipped the balance in favor of ex-U.S. investigative sites, or they are planning to do so within the next couple of years. Wyeth, for example, reports that it does more than half of its clinical trials among ex–U.S.-based investigative sites. In 2007, nearly 60% of all investigative sites recruited by Novartis were ex-U.S. Mid-sized P&G Pharmaceuticals, which now conducts more than one third of its trials among ex-U.S. sites, says that it is routinely looking to place trials overseas, often as replacements for traditionally U.S.-based investigators. And GlaxoSmithKline said that it conducted 29% of its clinical trials abroad in 2006andhopes to increase that figure to 50% of its clinical trials by the end of 2006. Merck is conducting significantly more studies internationally than in the past. More than 40% of its investigative sites are now based outside the U.S. in regions that include Central and Eastern Europe, Latin America, Australia, New Zealand, and, increasingly, the Asia-Pacific (8). As clinical trial volume and scope continues to increase, an increasingly global community of investigative sites seems ready to meet drug development capacity needs. For U.S.-based investigative sites, this trend signals that competition for a smaller number

7

of trials will intensify domestically. Research sponsors have already been turning to more sophisticated criteria (e.g., historical performance metrics and details on access to specific patient populations) to justify the higher relative costs of U.S.-based clinical investigators. Competition among ex-U.S. investigators is also intensifying. As these sites take on more operating infrastructure, they will need to sustain a larger stream of clinical trial activity. Clinical research managers who represent different global regions within biopharmaceutical companies must compete with their own colleagues to gain a larger share of their organization’s finite clinical trial work. At this time, Eastern Europe and Latin America tend to be the favored regions. Growing use of ex-U.S. based investigative sites is also expected to increase outsourcing to contract service providers well positioned within markets abroad and offering significantly lower operating expenses. Given the added complexity of simultaneously conducting clinical trials internationally, several sponsors report a heavier reliance on CROs to meet more aggressive development timelines. Global Phase I clinical trials activity is one area that is beginning to migrate back to the United States and Canada largely due to a weaker US dollar combined with tighter global regulatory requirements. During the next several years, Phase I investigative sites can expect intensifying competition and consolidation among CRO-owned and community-based investigator-owned facilities. The shift toward increasing use of investigative sites outside the U.S., particularly in Phase II and III clinical research studies, will likely attract public scrutiny. With mounting political pressure, the FDA is expected to weigh-in with new restrictions and reforms. Media and policy-makers have already identified this trend as a potentially explosive issue. This issue will fuel the fire of an already damaged and eroded relationship with the public. Despite data suggesting that regardless of where clinical trials are conducted, the U.S. market typically gains early access to new medical innovations, and the media tends to depict the growing prevalence of ex–U.S.-based investigative sites as

8

INVESTIGATOR/INSTITUTION

profit-motivated and exploitive of vulnerable global markets. It is critical for research sponsors to educate the public and policymakers proactively about the full rationale behind broad-based global development programs. New drug approval in the United States requires that sponsors submit an application that contains substantial evidence of effectiveness based on the results of adequate and well-controlled studies. A few cases in which a new drug has been approved for the U.S. market are based on data from well-controlled studies conducted solely by investigative sites abroad. With regard to the exact mix of U.S.-based and ex–U.S.-based investigative sites, the FDA remains elusive. No minimum number of U.S. patients is required in a New Drug Application. Sponsors may design and conduct their studies in any manner provided that they demonstrate, via prospectively planned analyses, that the ex-U.S. patients are similar to U.S. patients, both in terms of pretreatment disease characteristics and treatment outcomes. Now and in the near future, sponsors and the FDA will continue to evaluate their studies for their applicability to patient populations and medical care settings in the United States. Select diseases and their respective patient affinity groups may vie to keep some clinical trials within the United States given wide variations in diet, lifestyle, and health care consumption. Among these diseases are experimental treatments for age-related illnesses (such as Alzheimer’s, Parkinson’s, and ALS) gastro-intestinal and endocrine disorders. The investigative site landscape is poised to continue to evolve substantially. Research sponsors increasingly seek more efficient and productive collaborations with study conduct providers. As the cost and the duration of clinical research studies continue to increase, ample opportunities exist for investigative sites to improve. Streamlined operating processes, improved financial controls, increased adoption of integrated clinical research data collection and management technologies, the implementation of more effective patient recruitment and retention practices are but a few of the key areas in which successful investigative sites will excel and differentiate.

REFERENCES 1. J. DiMasi et al., The price of innovation: new estimates of drug development Costs. J. Health Econ. 2003; 22: 151–185 2. J. DiMasi, Risk in new drug development: approval success rates for investigational drugs. Clin. Pharmacol. Ther. 2001; 69: 297–307. 3. K. Getz, The Market for Site Management Organizations and Clinical Trial Conduct Services. Dorland Medical and Healthcare Marketplace. 2003; 18: 216–218 4. K. Getz, Number of active investigators in FDA-regulated clinical trials drop. The Impact Report, The Tufts Center for the Study of Drug Development at Tufts University. 2005; 7(3). 5. K. Getz, The Evolving SMO in the United States. New York: Marcel Dekker, 2004. 6. H. Moses et al., Financial anatomy of biomedical research. JAMA 2005; 1333–1342. 7. R. Rettig, The industrialization of clinical research. Health Aff. 2000; 129–146. 8. N. Sung et al., Central challenges facing the national clinical research enterprise. JAMA 2003; 289: 1278–1287.

INVESTIGATOR’S BROCHURE

of importance to the investigator. If a marketed product is being studied for a new use (i.e., a new indication), then an IB specific to that new use should be prepared. The IB should be reviewed at least annually and revised as necessary in compliance with the written procedures of the sponsor. More frequent revision may be appropriate depending on the stage of development and the generation of relevant new information. However, in accordance with Good Clinical Practice (GCP), relevant new information may be so important that it should be communicated to the investigators and possibly to the Institutional Review Boards (IRBs)/Independent Ethics Committees (IECs) and/or regulatory authorities before it is included in a revised IB. Generally, the sponsor is responsible for ensuring that an up-to-date IB is made available to the investigator(s), and the investigators are responsible for providing the up-to-date IB to the responsible IRBs/IECs. In the case of an investigator-sponsored trial, the sponsor-investigator should determine whether a brochure is available from the commercial manufacturer. If the investigational product is provided by the sponsorinvestigator, then he or she should provide the necessary information to the trial personnel. In cases where preparation of a formal IB is impractical, the sponsor-investigator should provide, as a substitute, an expanded background information section in the trial protocol that contains the minimum current information described in this guideline.

The Investigator’s Brochure (IB) is a compilation of the clinical and nonclinical data on the investigational product(s) that are relevant to the study of the product(s) in human subjects. Its purpose is to provide the investigators and others involved in the trial with the information to facilitate their understanding of the rationale for, and their compliance with, many key features of the protocol, such as the dose, dose frequency/interval, methods of administration, and safety monitoring procedures. The IB also provides insight to support the clinical management of the study subjects during the course of the clinical trial. The information should be presented in a concise, simple, objective, balanced, and nonpromotional form that enables a clinician or potential investigator to understand it and make his/her own unbiased risk–benefit assessment of the appropriateness of the proposed trial. For this reason, a medically qualified person generally should participate in the editing of an IB, but the contents of the IB should be approved by the disciplines that generated the described data. This guideline delineates the minimum information that should be included in an IB and provides suggestions for its layout. It is expected that the type and extent of information available will vary with the stage of development of the investigational product. If the investigational product is marketed and its pharmacology is widely understood by medical practitioners, then an extensive IB may not be necessary. Where permitted by regulatory authorities, a basic product information brochure, package leaflet, or labeling may be an appropriate alternative, provided that it includes current, comprehensive, and detailed information on all aspects of the investigational product that might be This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

KAPLAN–MEIER PLOT

In its original form, the K–M estimator is applicable to right censored data, meaning that, for some patients, it is only known that their true survival time exceeds a certain censoring time. Right censoring is exclusively considered here, and extensions of the K–M estimator to other types of censoring are merely mentioned at the end of this article. Different types of censoring are described elsewhere in the encyclopedia. Nearly all textbooks on survival analysis discuss the K–M estimator in detail. The classic book by Kalbfleisch and Prentice (2), a standard reference on survival analysis for many years, increased the popularity of the K–M estimator substantially. The more practically oriented reader may refer to the textbooks by Marubini and Valsecchi (3) or Klein and Moeschberger (4). Rigorous derivations of the statistical properties of the Kaplan–Meier estimator are provided, for example, in the book by Andersen et al. (5). The practical analysis can be performed using any sound statistical software package. For instance, SAS provides the procedure PROC LIFETEST, SPSS the procedure KM, R/S-Plus the procedure SURVFIT, and STATA the procedure STS GRAPH. Next, the K–M estimator is described including an estimation of its variance and the calculation of confidence intervals. The application of the plot is illustrated using data from a particular clinical study. Then, the estimation of the median survival time and of survival probabilities of specific time points is discussed. Furthermore, practical notes are provided on the use of the K–M estimator for the analysis of clinical survival data and its interpretation. Finally, a few additional topics are considered.

¨ NORBERT HOLLANDER

University Hospital of Freiburg Institute of Medical Biometry and Medical Informatics Freiburg, Germany

1

INTRODUCTION

The Kaplan–Meier plot or Kaplan–Meier curve is the standard method to describe and compare graphically the overall survival of groups of patients. This method is applied to survival time, which is defined as time from treatment start or date of randomization to death or last date of follow up, and, more general, to any type of time-to-event data, such as time-to-disease progression, time-to-detoriation, or time-to-first toxicity. Sometimes, the time-to-event is also called failure time. In the sequel, the term ‘‘survival time’’ is used as a synonym for any time-to-event. In addition to its broad applicability, the Kaplan–Meier plot has further advantages: It displays without bias nearly all information concerning survival time obtained in the dataset. Furthermore, with the Kaplan–Meier plot, the outcome of a trial can easily be explained to clinicians. Therefore, the Kaplan–Meier plot is omnipresent in clinical trials literature. No trial investigating survival time should be published without showing the curve. The Kaplan–Meier plot is the graphical presentation of the nonparametric Kaplan–Meier estimator (K–M estimator) of the survival curve. The K–M estimator generalizes the empirical distribution function of a sample in the presence of censoring. It’s name is derived from the seminal paper of E. L. Kaplan and P. Meier entitled ‘‘Nonparametric estimation from incomplete observations’’ in the Journal of the American Statistical Association (1). As a result of the importance of the Kaplan–Meier plot, this paper has become one of the most cited statistical papers. Sometimes, the K–M estimator is also denoted as product-limit estimator, which was the name originally used by Kaplan and Meier.

2

ESTIMATION OF SURVIVAL FUNCTION

Consider the situation in which the survival time is studied for a homogenous population. The survival function S(t) = Pr(T > t) is the basic quantity to describe time-to-event data, where S(t) is defined as the probability of an individual surviving time t. Censoring is

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

KAPLAN–MEIER PLOT

usually assumed to be independent from the occurrence of the event (e.g., death), which assumes that the additional knowledge of censoring before any time t does not alter the risk of failure at t. Let n denote the sample size of a study (= number of patients) and assume that deaths are observed at k different time points with k ≤ n. To calculate the K–M estimator, the data has to be arranged as in Table 1: The observed death times are ordered by ascending values t1 < t2 < . . .< tk . The symbol dj denotes the number of deaths at time point tj (dj ≥1) and rj is the number of patients ‘‘at risk’’ just before time point tj (j = 1, . . ., k). The latter contains all patients without an observed event just before time point tj . ‘‘At risk’’ just before tj are, therefore, all patients dying at tj or thereafter and those censored at tj or thereafter. Censored observations are given in Table 1 indirectly only by the number of patients ‘‘at risk,’’ between tj−1 and tj , rj−1 − rj − dj−1 censored survival times exist. The K–M estimator for the survival distribution function S(t) is then obtained by ˆ = S(t)


 tj ≤t

1−

dj rj

 j = 1, . . . k

where rj is the number of patients at risk (i.e., aliveand not censored) just before time tj and denotes the product over all observed tj ≤t

death times tj less than or equal to t. The variance of the K–M estimator is commonly estimated by the so-called Greenwood formula: ˆ ˆ 2 Var(S(t)) = S(t)

 tj ≤t

dj rj (rj − dj )

Table 1. Arrangement of Time-to-Event Data Time points

Number of patients ‘‘at risk’’

Number of deaths

t1 t2 . .. tk

r1 r2 . .. rk

d1 d2 . .. dk

where
tj ≤t denotes the sum over all observed death times tj less than or equal to t. This formula allows the calculation of the standard ˆ at each time point tj . deviation of S(t) In large samples, the K–M estimator, evaluated at a given time t, is approximately normally distributed so that a standard 100(1 − α)% pointwise confidence interval for S(t) can be calculated by  ˆ ˆ ± z1−α/2 Var(S(t)) S(t)

(1)

where z1−α/2 denotes the 1 − α/2 quantile of the standard normal distribution. The confidence interval, which is routinely calculated by several statistical packages, is appropriate for studies with n = 200 or more patients with k > 100 observed deaths (6). However, for small sample sizes (e.g., n = 50), Equation (1) serves as a rough approximation only and may be improved (see Practical Note 7). If the upper confidence bound is larger than one, it is generally set equal to one and the lower confidence bound is set equal to zero if it is negative. 2.1 An Example The computation of the K–M estimator is illustrated with the data on remission durations of leukemia patients. This dataset is taken from a controlled clinical trial published by Freireich et al. (7) and has been used previously by several authors (e.g., References 3 and 4) for illustration. In the clinical trial, it was investigated whether patients who had achieved complete remission after some induction treatment could benefit from further treatment. Here, only the patients treated with 6mercaptopurine (6-MP) chemotherapy are considered. The time-to-event of interest is the time from complete remission to relapse. For the 21 patients of the 6-MP group, the observed remission durations or rightcensored times are recorded in weeks. These times were, rearranged in ascending order: 6,6,6,6*,7,9*,10,10*,11,11*,13,16,17*,19*, 20*,22, 23,25*, 32*,34* and 35*, where * indicates a censored observation. Table 2 illustrates the single steps of the computation, and Fig. 1 displays the resulting Kaplan–Meier plot including the standard

0.50

0.75

1.00

3

0.25

symbol 'l' denotes a censored observation 95 % confidence Interval for the median survival time tnz

0.00

Survival Probability (= Probability of staying in remission)

KAPLAN–MEIER PLOT

12 weeks

0

5

~ t0.5 = 23 weeks

15 20 25 10 Time since complete remission (weeks)

30

∞

35

Figure 1. Kaplan–Meter plot with standard pointwise 95% confidence interval for the leukemla data

pointwise 95% confidence intervals derived from Equation (1). Estimated survival probˆ abilities S(t), describing, in this example, the probability of staying in complete remission, are obtained for each time point t since complete remission. At time point t = 0, all patients are in complete remission and the estimated survival probability is, therefore, ˆ S(0) = 1. After 12 weeks, for example, the estimated probability of being still in ˆ remission is S(12) = 0.753, with a 95% confidence interval of [0.564; 0.942]. 2.2 Practical Notes 1. The K–M estimator of the survival curve is a step function with jumps at the observed uncensored survival times. The size of these jumps depends not only on the number of events observed at each event time tj , but also on the pattern of the censored observations before tj . If and only if all observed times are uncensored, the K–M estimator is equal to one minus the empirical distribution function and jumps by the amount 1/n if n is the sample size. Note that, in the latter special case, the K–M

plot and the respective software are still applicable. 2. If both an event and a censored observation are observed at exactly the same time point, the event is assumed to have occurred first. This assumption is crucial for the mathematical analysis but convenient from the practical point of view. 3. To calculate the K–M estimator, the time is partitioned into a sequence of time intervals, where the number and lengths of intervals are determined by the observed uncensored survival times in the data. In the example provided, the intervals are 0 < t ≤ 6 weeks, 6 < t ≤ 7 weeks, 7 < t ≤ 10 weeks, and so on. A similar approach for estimating survival curves is the so-called life-table estimator, also known as actuarial method. With this approach, the time is partitioned into a data-independent fixed sequence of intervals that are almost always of equal lengths. For instance, in lifetable analysis for human populations, the length of each interval is usually

4

KAPLAN–MEIER PLOT

1 year. Once the set of intervals has been chosen, the construction of the estimator follows the basic idea used for the Kaplan–Meier estimator. More details on various types of life-table estimators can be found in the book by Lee (8). In clinical trials in which individual data on survival time are available, the K–M estimator should be preferred because it is more precise. 4. The K–M estimator is a product of conditional probabilities. For each time interval (see Practical Note 3), the probability of being a survivor at the end of the interval is estimated on the condition that the subject was a survivor at the beginning of the interval. This conditional probability is denoted as ‘‘factor contributed to the K–M estimator’’ ˆ in Table 2. The survival probability S(t) at some time point t is calculated as the product of the conditional probabilities of surviving each time interval up to time point t, which is done recursively as described in the second footnote of Table 2. 5. The K–M estimator is well defined for all time points less than the largest observed survival or censoring time tlast . If tlast corresponds to an event

(i.e., an uncensored observation), then the estimated survival curve is zero beyond this point. If tlast corresponds to a censored observation, as in the above example, where tlast = 35 weeks, the K–M estimator should be considered as undefined beyond tlast , because when this last survivor would have died if the survivor had not been censored is not known. Therefore, a correct Kaplan–Meier plot should end at this last observed time point and the curve should not be prolonged to the right (right-censored!). 6. To improve the interpretability of results, the K–M plot should contain information on the censoring distribution, which is especially relevant if many patients are censored after the last observed death, thereafter the estimated survival probability remains constant until the last censored observation. In Fig. 1, censored observations are marked by a symbol; 4 patients are censored after the last observed death with censoring times of 25*, 32*, 34*, and 35* weeks. Considering the K–M plot without this information, the authors only would have known that the last observed censored time is 35*

Table 2. Illustration of the Computation of the Kaplan-Meier Estimator with Standard Pointwise 95% Confidence Bounds for the Leukemia Data of Freireich et al. (7)

j 1 2 3 4 5 6 7

Number of Factor KM-estimator Ordered patients Number of contributed for all patients to the t in the distinct at risk§ failure just before dying at KM-estimator interval time tj at t [tj , tj+1 ) times time tj j   dj ˆ # tj rj dj 1− S(t) rj 6 7 10 13 16 22 23

21 17 15 12 11 7 6

3 1 1 1 1 1 1

18/21 16/17 14/15 11/12 10/11 6/7 5/6

18/12 = 0.857 0.857 × 16/17 = 0.807 0.807 × 14/15 = 0.753 0.753 × 11/12 = 0.690 0.690 × 10/11 = 0.627 0.627 × 6/7 = 0.538 0.538 × 5/6 = 0.448

Standard deviation based on Greenwood variance estimator  ˆ Var(S(t))

Standard pointwise 95% confidence interval for S(t)

0.0764 0.0869 0.0963 0.1068 0.1141 0.1282 0.1346

[0.707; 1.000] [0.636; 0.977] [0.564; 0.942] [0.481; 0.900] [0.404; 0.851] [0.286; 0.789] [0.184; 0.712]

§ Note that not only d but also the censored observations 6*,9*,10*,11*,17*,19*,20*,25*,32*,34* and 35* change the set j of patients at risk # S ˆ (0) = 1, for t the K-M estimator ˆ (t) is calculated recursively: at time point t = 0 all patients are in remission leading to S 1     ˆ ˆ is S(t 1 ) = S(0) ·

d 1− 1 r1

d2 ˆ ) = S(t ˆ , S(t 2 1 ) · 1− r 2

etc.

KAPLAN–MEIER PLOT

weeks and that the remaining 3 censored observations (this number would usually be unknown too) were somewhere between 23 weeks and 35 weeks. Thus, without any information on the censoring distribution, the K–M plot should be interpreted until the last observed death only. For large samples, information concerning the censoring distribution could be given by adding the number of patients ‘‘at risk’’ for equidistant time points to the time axis. 7. For small samples, better pointwise confidence intervals can be constructed ˆ using transformations of S(t) such as ˆ log(− log(S(t))) (6). Based on this transformation, the resulting 100(1 − α)% confidence interval is not symmetric about the K–M estimator of the survival function. In the example provided, one would obtain a 95% confidence interval for the probability of being still in remission after 12 weeks, S(12), of [0.503; 0.923] in contrast to that reported above, which was [0.564; 0.942] around 0.753. 8. It should be noted that alternatives exist to the Greenwood formula for variance estimation (see, for example, Reference 9). Using these alternatives may effect pointwise confidence intervals!

2.3 Median Survival Time For the presentation of survival data with censoring, the entire estimated survival ˆ curve, S(·), together with standard errors or confidence intervals, is usually the best choice. Nevertheless, summary statistics such as location estimates are sometimes also useful. Beside the presentation of survival probabilities for a specific time point t, the median, or 0.5 quantile, is a useful estimator of location. The median survival time t˜0.5 is determined by the relation S(t˜0.5 ) = 0.5. Notice that the K–M estimator is a step function and, hence, does not necessarily attain the value 0.5. Therefore, t˜0.5 is usually defined ˆ as the smallest time t, for which S(t) ≤ ˆ 0.5 (i.e., the time t where S(t) jumps from

5

a value greater than 0.5 to a value less or equal to 0.5). A commonly used method to calculate confidence intervals for the median survival time t˜0.5 has been proposed by Brookmeyer and Crowley (10), which is based on a modification of the construction of pointwise confidence intervals for S(t) described above. Based on the standard confidence interval in Equation (1), a 100(1 − α)% confidence interval for t˜0.5 is the set of all time points t that satisfy the following condition: ˆ − 0.5 S(t) ≤ z1−α/2 −z1−α ≤   S(t) ˆ Var The resulting confidence interval for t˜0.5 can easily be obtained from the pointwise confidence intervals for S(t): The lower confidence bound is the smallest value for which the lower confidence bound of S(t) is less than or equal to 0.5, and the upper confidence bound for t˜0.5 is the smallest t-value for which the upper confidence bound of S(t) is less than or equal to 0.5. Therefore, one can easily determine the corresponding confidence interval from the graph of the Kaplan–Meier Plot and its pointwise 95% confidence band (see Fig. 1). The median survival time (i.e., the median duration from complete remission to relapse) for leukaemia patients tretated with 6-MP is 23 weeks. The corresponding lower confidence bound is 13 weeks. As the upper confidence bound for S(t) does not fall below the value 0.5, one may set the upper confidence bound for t˜0.5 to infinity (= ∞). 2.4 More Practical Notes 9. If the estimated survival probability is larger than 0.5 at any time point t (which may occur when considering patients with a good prognosis), the median survival time t˜0.5 cannot be calculated. 10. Instead of the median t˜0.5 , the K–M estimator can also be used to provide estimates of other quantiles of the survival time distribution. Recall that the p-quantile t˜p is estimated by ˆ the smallest time t at which S(t) is less than or equal to 1-p. Analogous to t˜0.5 , the corresponding confidence

6

KAPLAN–MEIER PLOT

interval for t˜p can be obtained from the pointwise confidence interval of S(t). 11. Using a transformation to calculate pointwise confidence intervals for S(t) (see Practical Note 7), the confidence interval for t˜p should be based on the same transformation and, therefore, may differ from the approach described above. In the example provided, the 95% confidence interval for t˜0.5 does not change. 12. Use of the median survival time has obvious advantages as compared with the mean, which is highly sensitive to the right tail of the survival distribution, where estimation tends to be imprecise because of censoring. The ∞ mean survival time µ = 0 S(t) dt can naturally be estimated by substituting ˆ the K–M estimator S(t) for S(t). This estimator is appropriate only when tlast corresponds to a death (see Practical Notes 2) and censoring is light. Consequently, it should be used with caution.

3

ADDITIONAL TOPICS 1. The pointwise confidence intervals described above are valid only at a fixed single time point t0 (in the example, t0 = 12 weeks was considered). Although these intervals are plotted by many statistical software packages in addition to the K–M estimator as a function of time (see Fig. 1), the curves cannot be interpreted as a confidence band with, for example, 95% confidence that the entire survival function lies within the band. Proper confidence bands can be √ derived from the weak convergence of n(Sˆ − S)/S to a mean zero Gaussian martingale. Two important types of such confidence bands are the bounds of Hall and Wellner (11) and the equal precision bands (12). 2. Besides right-censoring other kinds of data incompleteness, such as left truncation, may be present in survival analysis. In epidemiologic applications, for

example, individuals are often not followed from time zero (e.g., the birth, if age is the relevant time scale), but only from a later entry time (conditional on survival until this entry time). For such left-truncated data, the usual K–M estimator may be calculated with a modified risk set, but it is of little practical use, because the estimates ˆ S(t) have large sampling errors. More useful is the estimation of the conditional survival distribution function (see, for example, Reference 4). 3. Left-censoring occurs when some individuals have experienced the event of interest before the start of the observation period. For samples that include both left-censoring and right–censoring, a modified K–M estimator of the survival function has been suggested by Turnbull (13). 4. Besides the event of interest (e.g., relapse) and right-censored observations, other events (e.g., non-diseaserelated deaths) may occur. Thus, for patients dying before relapse, the event of interest, namely relapse, could not be observed. Therefore, it is said that the two events, death and relapse, compete with each other. In such socalled competing risk situations, the occurrence of the competing risk (here the non-disease-related death) is often inadequately treated as right-censored observation, and the survival function for the event of interest is estimated by the K–M estimator. However, as illustrated, for example, by Schwarzer et al. (14), the use of the K–M estimator is not appropriate in competing risk situations: Calculating the K–M estimator for both Sˆ (time to the event of first interest) and Sˆ (time to non-diseaserelated death) where the corresponding competing risk (death in the first case, event of interest in the second case) is treated as right-censored observation the sum of both survival probabilities for a fixed time point t0 may be larger than 1. However, as the two events are mutually exclusive, the sum must not exceed 1. For competing risks, the estimation of survival functions based on

KAPLAN–MEIER PLOT

cumulative incidences is appropriate, however (14, 15).

REFERENCES 1. E. L. Kaplan and P. Meier, Non-parametric estimation from incomplete observations. J. Amer. Stat. Assoc. 1958; 53: 457–481. 2. J. Kalbfleisch and R. Prentice, The Statistical Analysis of Failure Time Data. New York: Wiley, 1980. 3. E. Marubini and M. G. Valsecchi, Analysing Survival Data from Clinical Trias and Observational Studies. Chichester: Wiley, 1995. 4. J. P. Klein and M. L. Moeschberger, Survival Analysis: Techniques for Censored and Truncated Data. New York: Springer, 1997. 5. P. K. Andersen, O. Borgan, R. D. Gill, and N. Keiding, Statistical Methods Based on Counting Processes. New York: Springer, 1992. 6. Ø. Borgan and K. Liestøl, A note on confidence intervals and bands for the survival function based on transformations. Scand. J. Stat. 1990; 17: 35–41. 7. E. J. Freireich et al., The effect of 6-mercaptopurine on the duration of steroid-induced remissions in acute leukaemia: a model for evaluation of other potentially useful therapy. Blood 1963; 21: 699–716. 8. E. T. Lee, Statistical Methods for Survival Data Analysis. New York: Wiley, 1992. 9. O. O. Aalen and S. Johansen, An empirical transition matrix for nonhomogeneous Markov chains based on censored observations. Scand. J. Stat. 1978; 5: 141–150. 10. R. Brookmeyer and J. J. Crowley, A confidence interval for the median survival time. Biometrics 1982; 38: 29–41. 11. W. Hall and J. A. Wellner, Confidence bands for a survival curve from censored data. Biometrika 1980; 67: 133–143. 12. V. N. Nair, Confidence bands for survival functions with censored data: a comparative study. Technometrics 1984; 14: 265–275. 13. B. W. Turnbull, Nonparametric estimation of the survivorship function with doubly censored data. J. Amer. Stat. Assoc. 1974; 69: 169–173. 14. G. Schwarzer, M. Schumacher, T. B. Maurer, and P. E. Ochsner, Statistical analysis of failure times in total joint replacement. J. Clin. Epidemiol. 2001; 54: 997–1003.

7

15. T. A. Gooley, W. Leisenring, J. Crowley, and B. E. Storer, Estimation of failure probabilities in the presence of competing risks: new representations of old estimators. Stat. Med. 1999; 18: 695–706.

KAPPA

Table 1.

RICHARD J. COOK University of Waterloo, Waterloo, Ontario, Canada

In medical research it is frequently of interest to examine the extent to which results of a classification procedure concur in successive applications. For example, two psychiatrists may separately examine each member of a group of patients and categorize each one as psychotic, neurotic, suffering from a personality disorder, or healthy. Given the resulting data, questions may then be posed regarding the diagnoses of the two psychiatrists and their relationship to one another. The psychiatrists would typically be said to exhibit a high degree of agreement if a high percentage of their diagnoses concurred, and poor agreement if they often made different diagnoses. In general, this latter outcome could arise if the categories were ill-defined, the criteria for assessment were different for the two psychiatrists, or their ability to examine these criteria differed sufficiently, possibly as a result of different training or experience. Poor empirical agreement might therefore lead to a review of the category definitions and diagnostic criteria, or possibly retraining with a view to improving agreement and hence consistency of diagnoses and treatment. In another context, one might have data from successive applications of a test for dysplasia or cancer from cervical smears. If the test indicates normal, mild, moderate, or severe dysplasia, or cancer, and the test is applied at two time points in close proximity, ideally the results would be the same. Variation in the method and location of sampling as well as variation in laboratory procedures may, however, lead to different outcomes. In this context, one would say that there is empirical evidence that the test is reliable if the majority of the subjects are classified in the same way for both applications of the test. Unreliable tests would result from the sources of variation mentioned earlier. Again, empirical evidence of an unreliable test may lead to refinements of the testing procedure.

T2 = 1

T2 = 2

Total

T1 = 1 T1 = 2

x11 x21

x12 x22

x1· x2·

Total

x·1

x·2

x·· = n

1 THE KAPPA INDEX OF RELIABILITY FOR A BINARY TEST For convenience, consider a diagnostic testing procedure generating a binary response variable T indicating the presence (T = 1) or absence (T = 2) of a particular condition. Suppose this test is applied twice in succession to each subject in a sample of size n. Let T k denote the outcome for the kth application with the resulting data summarized in the two-by-two table (Table 1). where xij denotes the frequency at which
T 1 = i and T 2 = j, xi· = 2j=1 xij , and x·j =
2 i=1 xij , i = 1, 2, j = 1, 2. Assuming that test results on different subjects are independent, conditioning on n leads to a multinomial distribution for the outcome of a particular table with  f (x; p) =

x11

 2  2 xij n pij , x12 x21 x22 i=1 j=1

x = (x11 , x12 , x21 , x22 )
, p = (p11 , p12 , p21 , and p22 = 1 − p11 p22 )
,

− p12 − p21 . Let pi· = 2j=1 pij and p·j = 2i=1 pij . Knowledge of p would correspond to a complete understanding of the reliability of the test. Since knowledge of p is generally unattainable and estimation of p does not constitute a sufficient data reduction, indices of reliability/agreement typically focus on estimating one-dimensional functions
of p. A natural choice is p0 = 2i=1 pii , the probability of
raw agreement, which is estimated as pˆ 0 = 2i=1 xii /n. If p0 = 1, then the test is completely reliable since the probability of observing discordant test results is zero. Similarly, if pˆ 0 is close to unity, then it suggests that the outcomes of the two

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

KAPPA

applications concurred for the vast majority of the subjects. However, several authors have expressed reluctance to base inferences regarding reliability on the observed level of raw agreement (see (3) and references cited therein). The purported limitations of pˆ 0 as a measure of reliability stem from the fact that p0 reflects both ‘‘chance’’ agreement and agreement over and above that which would be expected by chance. The agreement expected by chance, which we denote by pe , is computed on the basis of the marginal distribution, defined by p1· and p·1 , and under the assumption that the outcomes of the two tests are independent conditional on the
true status. Specifically, pe = 2i=1 pi· p·i is
estimated by pˆ e = 2i=1 x1· x·1 /n2 . To address concerns regarding the impact of nonnegligible chance agreement, Cohen (3) defined the index kappa which takes the form κ=

p0 − pe , 1 − pe

and indicated that it can be interpreted as reflecting ‘‘the proportion of agreement after chance agreement is removed from consideration’’. This can be seen by noting that p0 − pe is the difference in the proportion of raw agreement and the agreement expected by chance, this being the agreement arising due to factors not driven by chance. If p0 − pe > 0, then there is agreement arising from nonchance factors; if p0 − pe = 0, then there is no additional agreement over that which one would expect based on chance; and if p0 − pe < 0, then there is less agreement than one would expect by chance. Furthermore, 1 − pe is interpreted by Cohen (3) as the proportion ‘‘of the units for which the hypothesis of no association would predict disagreement between the judges’’. Alternatively, this can be thought of as the maximum possible agreement beyond that expected by chance. An estimate of κ, denoted κ, ˆ is referred to as the kappa statistic and may be obtained by replacing p0 and pe with their corresponding point estimates, giving κˆ =

pˆ 0 − pˆ e . 1 − pˆ e

(1)

Table 2. T 2 = 1 T 2 = 2 T 2 = 3 . . . T 2 = R Total T1 = 1 T1 = 2 T1 = 3 .. . T1 = R

x11 x21 x31 .. . xR1

x12 x22 x32 .. . xR2

x13 x23 x33 .. . xR3

Total

x·1

x·2

x·3

... ...

x1R x2R x3R .. . xRR

x1· x2· x3· .. . xR·

...

x·R

x·· = n

... ... ...

2 THE KAPPA INDEX OF RELIABILITY FOR MULTIPLE CATEGORIES When the classification procedure of interest has multiple nominal categories, assessment of agreement becomes somewhat more involved. Consider a diagnostic test with R possible outcomes and let T k denote the outcome of the kth application of the test, k = 1, 2. Then T k takes values on {1, 2, 3, . . ., R} and interest lies in assessing the extent to which these outcomes agree for k = 1 and k = 2. An R × R contingency table may then be constructed (see Table 2), where again xij denotes the frequency with which the first application of the test led to outcome i and the second led to outcome j, i = 1, 2, . . ., R, j = 1, 2, . . . , R. A category-specific measure of agreement may be of interest to examine the extent to which the two applications tend to lead to consistent conclusions with respect to outcome r, say. In this problem there is an implicit assumption that the particular nature of any disagreements are not of interest. One can then collapse the R × R table to a 2 × 2 table constructed by crossclassifying subjects with binary indicators such that T k = 1 if outcome r was selected at the kth application, T k = 2 otherwise, k = 1, 2. A category-specific kappa statistic can then be constructed in the fashion indicated earlier. This can be repeated for each of the R categories giving R such statistics. In addition to these category-specific measures, however, an overall summary index of agreement is often of interest. The kappa statistic in (1) is immediately generalized for the R × R (R > 2) table as follows. Let pij denote the probability of T 1

KAPPA

= i and T 2 = j, one of the R2 multino
mial probabilities, pi· = R j=1 pij , and p·j =
R p , i = 1, 2, . . . , R, j = 1, 2, . . . , R. Then, ij i=1 as before, pˆ ij = xij /n, pˆ i· = xi· /n, pˆ ·j = x·j /n,

pˆ 0 = R ˆ ii , pˆ e = R ˆ i· pˆ ·i , and the overi=1 p i=1 p all kappa statistic takes the same form as in (1). This overall kappa statistic can equivalently be written as a weighted average of category-specific kappa statistics (6). The kappa statistic has several properties that are widely considered to be attractive for measures of agreement. First, when the level of observed agreement, reflected by pˆ 0 , is equal to the level of agreement expected by chance (pˆ e ), κˆ = 0. Secondly, κˆ takes on its maximum value of 1 if and only if there is perfect agreement (i.e. pˆ 0 = 1 arising from a diagonal table). Thirdly, the kappa statistic is never less than −1. The latter two features require further elaboration, however, as the actual upper and lower limits on κˆ are functions of the marginal frequencies. In particular, κˆ takes on the value 1 only when the marginal frequencies are exactly equal and all off-diagonal cells are zero. Values less than 1 occur when the marginal frequencies are the same but there are different category assignments in the table or, more generally, when the marginal frequencies differ (when the marginal frequencies differ there are necessarily nonzero diagonal cells and hence some disagreements). It is natural then to expect the kappa statistic for such a table to be less than unity. Cohen (3) shows that the maximum possible value of κˆ takes the form x·· κˆ M =

R 

min(xi· , x·i ) −

i=1

R  i=1

x2·· −

R 

If the marginal frequencies for the two tests are uncorrelated (as measured by the product–moment correlation of the margins (3)), then the lower bound for κˆ is κˆ L = −(R − 1)−1 . When the marginal frequencies are negatively correlated, κˆ L > − (R − 1)−1 . However, when the marginal frequencies are positively correlated, κˆ L < −(R − 1)−1 . It is only as the number of categories reduces to two, the correlation of the marginal frequencies approaches 1, and the variances of the marginal frequencies increase, that κˆ L approaches −1 (3). Having computed a kappa statistic for a given contingency table it is natural to want to characterize the level of agreement in descriptive terms. Landis & Koch (11) provide ranges that suggest, beyond what one would expect by chance, 0.75 < κˆ typically represents excellent agreement, 0.40 < κˆ < 0.75 fair to good agreement, and κˆ < 0.40 poor agreement. While there is some appeal to this convenient framework for the interpretation of κ, ˆ caution is warranted. Frequently, it will be of interest to construct confidence intervals for the index kappa. Fleiss et al. (8) derive an approximate large sample estimate for the variance of κ, ˆ v ar(κ), ˆ as



R 

pˆ ii [1 − (pˆ i· + pˆ ·i )(1 − κ)] ˆ 2

i=1

+ (1 − κ) ˆ 2

xi· x·i

 i

,

pˆ ij (pˆ ·i + pˆ j· )2

j=i

(2)

xi· x·i

3

− [κˆ − pˆ e (1 − κ)] ˆ 2

 [x..(1 − pˆ e )2 ], (3)

i=1

and argues that this is intuitively reasonable since differences in the marginal frequencies necessarily lead to a reduction in the level of agreement and hence κ. ˆ Cohen then suggests that if one is interested in assessing the proportion of the agreement permitted by the margins (correcting for chance), then one computes κ/ ˆ κˆ M . We return to the topic of marginal frequencies and their influence on the properties of κ later in the article.

and Fleiss (6) recommends carrying out tests (see Hypothesis Testing) and constructing confidence intervals by assuming approximate normality of (κˆ − κ)/[ var(κ)] ˆ 1/2 and proceeding in the standard fashion. For tests regarding the null hypothesis H0 : κ = 0, an alternate variance estimate may be derived from (3) by substituting 0 for κ, ˆ and pˆ i· pˆ ·j for

4

KAPPA

pˆ ij , giving ˆ v ar0 (κ)  R   pˆ i· pˆ ·i [1 − (pˆ i· + pˆ ·i )]2 + pˆ i· pˆ ·j = k=1

 × (pˆ ·i + pˆ j· )2 − p2e

i=j

[x·· (1 − pˆ e )2 ],

(4)

with tests carried out as described above. 3

THE WEIGHTED KAPPA INDEX

The discussion thus far has focused on situations in which the test serves as a nominal classification procedure (e.g. as in the psychiatric diagnosis example at the beginning of the article). In such settings, since there is no natural ordering to the outcomes, any disagreements are often considered to be equally serious and the methods previously described are directly applicable. In some circumstances with nominal scales, however, certain types of disagreements are more serious then others and it is desirable to take this into account. Furthermore, when the outcome is ordinal (as in the cervical cancer screening example), it is often of interest to adopt a measure of agreement that treats disagreements in adjacent categories as less serious than disagreements in more disparate categories. For the test based on cervical smears designed to classify the condition of the cervix as healthy, mildly, moderately, or severely dysplastic, or cancerous, if on one occasion the test suggested mild dysplasia and on another moderate, this type of disagreement would be considered less serious than if a cervix previously diagnosed as cancerous was subsequently classified as mildly dysplastic. In general, the seriousness reflects clinical implications for treatment and the consequences of wrong decisions. Weighted versions of the kappa statistic were derived by Cohen (4) to take into account the additional structure arising from ordinal measures or from nominal scales in which certain types of disagreement are of more importance than others. In particular, the objective of adopting a weighted kappa

statistic is to allow ‘‘different kinds of disagreement’’ to be differentially weighted in the construction of the overall index. We begin by assigning a weight to each of the R2 cells; let wij denote the weight for cell (i, j). These weights may be determined quite arbitrarily but it is natural to restrict 0 ≤ wij ≤ 1, set wii to unity to give exact agreement maximum weight, and set 0 ≤ wij < 1 for i = j, so that all disagreements are given less weight than exact agreement. The selection of the weights plays a key role in the interpretation of the weighted kappa statistic and also impacts the corresponding variance estimates, prompting Cohen (4) to suggest these be specified prior to the collection of the data. Perhaps the two most common sets of weights are the quadratic weights, with wij = 1 − (i − j)2 /(R − 1)2 , and the so-called Cicchetti weights, with wij = 1−|i − j|/(R − 1) (1,2). The quadratic weights tend to weight disagreements just off the main diagonal more highly than Cicchetti weights, and the relative weighting of disagreements farther from the main diagonal is also higher with the quadratic weights. Clearly, these two weighting schemes share the minimal requirements cited above. The weighted kappa statistic then takes the form κˆ (w) =

pˆ (w) ˆ (w) e 0 −p 1 − pˆ (w) e

,

(5)



R = R ˆ ij and pˆ (w) where pˆ (w) e = i=1 j=1 wij p
R
R
R 0 wij pˆ i· pˆ ·j . If wi· = j=1 pˆ ·j wij and i=1
j=1 w·j = R p i=1 ˆ i· wij , then the large-sample variance of κˆ (w) is estimated by v ar(κˆ (w) )  R  R  = pˆ ij [wij − (wi· + w·j )(1 − κˆ (w) )]2 i=1 j=1

 − [κˆ (w) − pˆ (w) ˆ (w) )]2 e (1 − κ 2 [x2·· (1 − pˆ (w) e ) ]

(6)

and, as before, tests and confidence intervals may be carried out and derived in the standard fashion assuming asymptotic normality var(κˆ (w) )]1/2 . As of the quantity (κˆ (w) − κ (w) )/[

KAPPA

in the unweighted case, a variance estimate appropriate for testing H0 :κ (w) = 0 may be derived by substituting pˆ i· pˆ ·j for pˆ ij , and 0 for κˆ (w) in (6). We note in passing that the weighted kappa with quadratic weights has been shown to bear connections to the intraclass correlation coefficient. Suppose that with an ordinal outcome the categories are assigned the integers 1 through R from the ‘‘lowest’’ to ‘‘highest’’ categories, respectively, and assignment to these categories is taken to correspond to a realization of the appropriate integer value. Fleiss & Cohen (7) show that the intraclass correlation coefficient computed by treating these integer responses as coming from a Gaussian general linear model for a two-way analysis of variance, is asymptotically equivalent to the weighted kappa statistic with quadratic weights.

4 THE KAPPA INDEX FOR MULTIPLE OBSERVERS Thus far we have restricted consideration to the case of two applications of the classification procedure (e.g. two successive applications of a diagnostic test, two physicians carrying out successive diagnoses, etc.). In many situations, however, there are multiple (>2) applications and interest lies in measuring agreement on the basis of several applications. Fleiss (5) considered the particular problem in which a group of subjects was examined and classified by a fixed number of observers, but where it was not necessarily the same set of observers carrying out the assessments for each patient. Moreover, Fleiss (5) assumed that it was not possible to identify which observers were involved in examining the patients. For this problem, we require some new notation. Let M denote the number of subjects, N denote the number of observers per subject, and R denote the number of categories as before. Therefore, NM classifications are to be made. Let nij denote the number of times the ith subject was assigned to the jth category. A measure of overall raw agreement for the assignments on the ith

5

subject is given by R  nij (nij − 1)

qˆ i =

j=1

N(N − 1)

,

which can be interpreted as follows. N  With  N posobservers per subjects there are 2   nij sible pairs of assignments. There are 2 which agree on category   j and hence a total
R nij number of pairs of assignments j=1 2 which concur altogether for the ith subject. Thus, (7) simply represents the proportion of all paired assignments on the ith subject for which there was agreement on the category. The overall measure of raw observed agreement over all subjects is then given by
ˆ i , which equals qˆ 0 = M −1 M i=1 q M  R 

qˆ 0 =

n2ij

i=1 j=1

MN(N − 1)

−

1 . N−1

(8)

As before, however, some agreement would be expected among the observers simply by chance and the kappa statistic in this setting corrects for this. The expected level of agreement is computed by noting that M  nij

pˆ j =

i=1

MN

is the sample proportion of all assignments
R made to category j, with ˆ j = 1. So if j=1 p pairs of observers were simply assigning subjects to categories at random and independently one can estimate that they would be expected to agree according to pˆ e =

R 

pˆ 2j ,

(9)

j=1

then the kappa statistic is computed by correcting for chance in the usual way as κˆ =

qˆ 0 − pˆ e . 1 − pˆ e

(10)

6

KAPPA

The sample variance for (10) is derived by Fleiss et al. (9) to be v ar(κ) ˆ



2 R R    pj (1 − pj ) − pj (1 − pj ) = 2 j=1

j=1

 (1 − 2pj )



 MN(N − 1)

R 

2 pj (1 − pj )

j=1

(11) and is typically used for tests or interval estimation in the standard fashion. When the same set of raters assesses all subjects and individual raters scores are known, it is not possible to use the results of Fleiss (5) without ignoring the rater-specific assignments. For this context, Schouten (13) proposed the use of indices based on weighted sums of pairwise measures of observed and expected levels of agreement. In particular, for a given pair of raters and a given pair of categories, observed and expected measures of agreement may be computed as earlier. Then, for each pair of raters, a measure of overall observed agreement may be obtained by taking a weighted average of such measures over all pairwise combinations of categories. Given a corresponding measure of expected agreement, an overall kappa statistic can be computed in the usual fashion. Schouten (13) then described how to obtain kappa statistics reflecting agreement over all observers, agreement between a particular observer and the remaining observers, and agreement within and between subgroups of observers. 5

GENERAL REMARKS

MaClure & Willett (12) provide a comprehensive review and effectively highlight a number of limitations of the kappa statistics. In particular, they stress that for ordinal data derived from categorizing underlying continuous responses, the kappa statistic depends heavily on the often arbitrary category definitions, raising questions about interpretability. They also suggest that the

use of weights, while attractive in allowing for varying degrees of disagreement, introduces another component of subjectivity into the computation of kappa statistics. Perhaps the issue of greatest debate is the so-called prevalence, or base-rate, problem of kappa statistics. Several other authors have examined critically the properties and interpretation of kappa statistics (10,14,15), and the debate of the merits and demerits continues unabated. Despite the apparent limitations, the kappa statistic enjoys widespread use in the medical literature and has been the focus of considerable statistical research. REFERENCES 1. Cicchetti, D. V. (1972). A new measure of agreement between rank ordered variables, Proceedings of the American Psychological Association 7, 17–18. 2. Cicchetti, D. V. & Allison T. (1973). Assessing the reliability of scoring EEG sleep records: an improved method, Proceedings and Journal of the Electro-physiological Technologists’ Association 20, 92–102. 3. Cohen, J. (1960). A coefficient of agreement for nominal scales, Educational and Psychological Measurement 20, 37–46. 4. Cohen, J. (1968). Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit, Psychological Bulletin 70, 213–220. 5. Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters, Psychological Bulletin 76, 378–382. 6. Fleiss, J. L. (1981). Statistical Methods for Rates and Proportions, 2nd Ed. Wiley, New York. 7. Fleiss, J. L. & Cohen, J. (1973). The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability, Educational and Psychological Measurement 33, 613–619. 8. Fleiss, J. L., Cohen, J. & Everitt, B. S. (1969). Large sample standard errors of kappa and weighted kappa, Psychological Bulletin 72, 323–327. 9. Fleiss, J. L., Nee, J. C. M. & Landis, J. R. (1979). Large sample variance of kappa in the case of different sets of raters, Psychological Bulletin 86, 974–977. 10. Kraemer, H. C. & Bloch, D. A. (1988). Kappa coefficients in epidemiology: an appraisal of a

KAPPA reappraisal, Journal of Clinical Epidemiology 41, 959–968. 11. Landis, J. R. & Koch, G. G. (1977). The measurement of observer agreement for categorical data, Biometrics 33, 159–174. 12. MaClure, M. & Willett, W. C. (1987). Misinterpretation and misuse of the kappa statistic, American Journal of Epidemiology 126, 161–169.

7

13. Schouten, H. J. A. (1982). Measuring pairwise interobserver agreement when all subjects are judged by the same observers, Statistica Neerlandica 36, 45–61. 14. Thompson, W. D. & Walter S. D. (1988). A reappraisal of the kappa coefficient, Journal of Clinical Epidemiology 41, 949–958. 15. Thompson, W. D. & Walter S. D. (1988). Kappa and the concept of independent errors, Journal of Clinical Epidemiology 41, 969–970.

KEFAUVER–HARRIS DRUG AMENDMENTS

advertising from the Federal Trade Commission to the FDA.

In 1962, news reports about how Food and Drug Administration (FDA) Medical Officer Frances O. Kelsey, M.D., Ph.D., had kept the drug thalidomide off the U.S. market aroused public interest in drug regulation. Thalidomide had been marketed as a sleeping pill by the German firm Chemie Grunenthal, and it was associated with the birth of thousands of malformed babies in Western Europe. ‘‘In the years before 1962, Senator Estes Kefauver had held hearings on drug costs, the sorry state of science supporting drug effectiveness, and the fantastic claims made in labeling and advertising,’’ Temple says. ‘‘Well-known clinical pharmacologists explained the difference between well-controlled studies and the typical drug study. With the [Food, Drug and Cosmetic] FD&C Act ‘in play’ because of thalidomide, Congress had the opportunity to make major changes.’’ In October 1962, Congress passed the Kefauver–Harris Drug Amendments to the Federal FD&C Act. Before marketing a drug, firms now had to not only prove safety, but also they had to provide substantial evidence of effectiveness for the product’s intended use. Temple says, ‘‘That evidence had to consist of adequate and well-controlled studies, a revolutionary requirement.’’ ‘‘Also critically, the 1962 amendments required that the FDA specifically approve the marketing application before the drug could be marketed, another major change.’’ The Kefauver–Harris Drug Amendments also asked the Secretary to establish rules of investigation of new drugs, which include a requirement for the informed consent of study subjects. The amendments also formalized good manufacturing practices, required that adverse events be reported, and transferred the regulation of prescription drug This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/fdac/features/2006/106 cder.html) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

LANS-DEMETS: ALPHA SPENDING FUNCTION

Assume that the procedure is a one-sided test but that the process can be easily generalized to introduce two one-sided symmetric or asymmetric boundaries. In general, a test statistic Z(j), j= 1, 2, 3, . . . ,J is computed at each successive interim analysis. In the large sample case and under the null hypothesis, the Z(j)s are standard N(0,1). At each analysis, the test statistic is compared with a critical value Zc (j). The trial would continue as long as the test statistic does not exceed the critical value. That is, continue the trial as long as

DAVID L. DEMETS University of Wisconsin-Madison Madison, Wisconsin

K. K. GORDON LAN Johnson & Johnson Raritan, New Jersey

The randomized control clinical trial (RCT) is the standard method for the definitive evaluation of the benefits and risks of drugs, biologics, devices, procedures, diagnostic tests, and any intervention strategy. Good statistical principles are critical in the design and analysis of these RCTs (1,2). RCTs also depend on interim analysis of accumulating data to monitor for early evidence of benefit, harm, or futility. This interim analysis principle was established early in the history of RCTs (3) and was implemented in early trials such as the Coronary Drug Project (4,5). Evaluation of the interim analysis may require the advice of an independent data monitoring committee (DMC) (6,7), including certain trials under regulatory review (8,9). However, although ethically and scientifically compelling, interim repeated analysis of accumulating data has the statistical consequence of increased false positive claims unless special steps are taken. The issue of sequential analysis has a long tradition (10,11) and has received special attention for clinical trials (12,13). In particular, increasing the frequency of interim analysis can substantially increase the Type I error rate if the same criteria are used for each interim analysis (13). This increase was demonstrated in the Coronary Drug Project, which used sequential analysis for monitoring several treatment arms compared with a placebo (4). Most of the classical sequential methods assumed continuous analysis of accumulating data, a practice not realistic for most RCTs. Rather than continuous monitoring, most clinical trials review accumulating data periodically after additional data has been collected.

Z(j) < Zc (j) for j = 1, 2, 3, . . . ., J-1 Otherwise, the trial might be considered for termination. We would fail to reject the null hypothesis if Z(j) < Zc (j) for all j (j = 1,2, . . . ,J). We would reject the null hypothesis if at any interim analysis, Z(j) ≥ Zc (j) for j = 1, 2, . . . , J Peto et al. (14) recommended using a very conservative critical value for each interim analysis, say a standardized value of Zc (j) = 3.0 for all j (j = 1,2, . . . ,J), such the impact on the overall Type I error would be minimal. In 1977, Pocock (15) published a paper based on the earlier work of Armitage and colleagues (13) that formally introduced the idea of a group sequential approach. This modification developed a more conservative critical value than the na¨ıve (e.g., 1.96 for one-sided Type I error of 0.025) to be used at each analysis such that the overall Type I error was controlled. For example, if a total of 5 interim analyses were to be conducted with an intended Type I error of 0.025, then Zc (j) = 2.413 would be used at each interim analysis (j = 1,2, . . . ,5). Note that this final critical value is much larger than the standard critical value. In 1979, O’Brien and Fleming (16) introduced an alternative group sequential boundary for evaluating interim analysis. In this approach, the critical values change with each interim analysis, starting with a very conservative (i.e., large) value and shrinking to a final value close to the nominal critical value at the scheduled com-

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

LANS-DEMETS: ALPHA SPENDING FUNCTION

pletion. The exact√ form for each critical value is Zc (j) = ZOBF (J) (J/j). In this case, for the same 5 interim analyses and an overall Type I error of 0.025, the ZOBF (5) value is 2.04, which √ makes the 5 critical values 2.05 (5/j) for j = 1,2, . . . ,5 or (4.56, 3.23, 2.63, 2.28, and 2.04). Both of these latter models assume an equal increment in information between analyses and that the number of interim analyses J is fixed in advance. These three group sequential boundaries have been widely used, and examples are shown in Fig. 1. In fact, the OBF group sequential method was used in the Betablocker Heart Attack Trial (BHAT) (17), which terminated early because of an overwhelming treatment benefit for mortality. In 1987, Wang and Tsiatis generalized the idea of Pocock and O’Brien-Fleming and introduced a family of group sequential boundaries. For given α, J and a shape parameter φ, a constant C will be chosen so that Z(j) ≥ C(J/j)φ for some j = 1,2, . . . ,J is equal to α. The choice of φ = 0.5 yields the OBF boundary, and φ = 0 yields the Pocock boundary. 1 ALPHA SPENDING FUNCTION MOTIVATION The BHAT trial was an important factor in the motivation for the alpha spending function approach to group sequential monitoring. BHAT was a cardiovascular trial that evaluated a betablocker class drug to reduce mortality following a heart attack (17). An independent DMC reviewed the data periodically, using the OBF group sequential boundaries as a guide. A beneficial mortality trend emerged early in the trial and continued to enlarge with subsequent evaluations. At the sixth of a planned seven interim analyses, the logrank test statistic crossed the OBF boundary. After careful examination of all aspects, the DMC recommended that the trial be terminated, approximately 1 year earlier than planned. However, although the OBF sequential boundaries were used, the assumptions of these models were not strictly met. The increment in the number of deaths between DMC meetings was not equal. Furthermore, additional interim analyses were contemplated although not done. This experience suggested the need for more flexible

sequential methods for evaluating interim results. Neither the number nor the timing of interim analyses can be guaranteed in advance. A DMC may need to add additional interim analyses as trends that suggest benefit or harm emerge. As described by Ellenberg, Fleming, and DeMets (7), many factors must be considered before recommendations for early termination are made, and an additional interim analysis may be necessary to confirm or more fully evaluate these issues. Thus, the need for a flexible group sequential method seemed compelling. 2 THE ALPHA SPENDING FUNCTION The initial (or classical) group sequential boundaries are formed by choosing boundary values such that the sum of the probability of exceeding those critical values during the interim analyses is exactly the specified alpha level set in the trial design, assuming the null hypothesis of no intervention effect. That is, the total available alpha is allocated or ‘‘spent’’ over the prespecified times of interim analyses. The alpha spending function proposed by Lan and DeMets (18) allocated the alpha level over the interim analyses by a continuous monotone function, α(t), where t is the information fraction, 0 ≤ t ≤ 1. Here t could be the fraction of target patients recruited (n/N) or the fraction of targeted deaths observed (d/D) at the time of the interim analysis. In general, if the total information for the trial design is I, then at the j-th analysis, the information fraction tj = Ij /I. The total expected information I should have been determined by the trial design if properly done. The function α(t) is defined such that α(0) = 0 and α(1) = α. Boundary values Zc (j), which correspond to the α-spending function α(t), can be determined successfully so that under the null P0 {Z(1) ≥ Zc (1), or Z(2) ≥ Zc (2), or . . . , or Z(j) ≥ Zc (j)} = α(tj )

(1)

where {Z(1), . . . ,Z(j),} represent the test statistics from the interim analyses 1, . . . , . The specification of α(t) will create a boundary of critical values for interim test statistics,

LANS-DEMETS: ALPHA SPENDING FUNCTION

3

Group Sequential Boundaries 6 5

Alpha Spending OBF Type OBF

Z Score

4 3 2 Figure 1. Upper boundary values corresponding to the α 1 (t*) spending function for α – 0.05 at information fraction t* = 0.25, 0.50, 0.75, and 1.0 and for a truncated version at a critical value of 3.0.

1 0

and we can specify functions that approximate O’Brien–Fleming or Pocock boundaries as follows: √ α1 (t) = 2 − 2 − ( Zα/2 ´ / t) O’Brien–Fleming Type α2 (t) = α1n (1 + (e − 1)t) Pocock Type where  denotes the standard normal cumulative distribution function. The shape of the alpha spending function is shown in Fig. 2 for both of these boundaries. Two other families of spending functions proposed (19,20) are α(θ , t) = α tθ for θ > 0 α(γ , t) = α[(1 − e−γ t )/(1 − e−γ )], for γ = 0 The increment α(tj ) – α(tj−1 ) represents the additional amount of alpha or Type I error probability that can be used at the jth analysis. In general, to solve for the boundary values Zc (j), we need to obtain the multivariate distribution of Z(1), Z(2), . . . ,Z(J). In the cases to be discussed, the distribution is asymptotically multivariate normal with covariance structure  = (σ jk ) where σjk = cov (Z(j), Z(k)) √ √ = (tj /tk ) = (iJ /iK ) for j ≤ k where ij and ik are the amount of information available at the j-th and k-th data

0

0.2

0.4 0.6 Information Fraction

0.8

1.0

monitoring, respectively. Note that at the jth data monitoring, ij and ik are observable and σ jk is known even if I (total information) is unknown. However, if I is not known during interim analysis, we must estimate I by Iˆ and tj by tj = xj /Iˆ so that we can estimate α(tj ) by α(tˆ). If these increments have an independent distributional structure, which is often the case, then derivation of the values of the Zc (j) from the chosen form of α(t) is relatively straightforward using Equation (1) and the methods of Armitage et al. (21,22). If the sequentially computed statistics do not have an independent increment structure, then the derivation of the Zc (j) involves a more complicated numerical integration and sometimes is estimated by simulation. However, as discussed later, for the most frequently used test statistics, the independent increment structure holds. This formulation of the alpha spending function provides two key flexible features. Neither the timing nor the total number of interim analyses have to be fixed in advance. The critical boundary value at the j-th analysis only depends on the information fraction tj and the previous j-1 information fractions, t1 ,t2 , . . . ,tj−1 , and the specific spending function being used. However, once an alpha spending function has been chosen before the initiation of the trial, that spending function must be used for the duration of the trial. A DMC can change the frequency of the interim analyses as trends emerge without appreciably affecting the overall α level (23,24). Thus,

4

LANS-DEMETS: ALPHA SPENDING FUNCTION

Spending Functions .05 .04

Alpha

α2 (t∗) .03 α3 (t∗)

.02

α1 (t∗)

.01 0 0

0.2

0.4 0.6 Information Fraction

0.8

it is difficult to abuse the flexibility of this approach. The timing and spacing of interim analyses using the alpha spending function approach have been examined (19,25–27). For most trials, two early analyses with less than 50% of the information fraction are adequate. An early analysis say at 10% is often useful to make sure that all of the operational and monitoring procedures are in order. In rare cases, such early interim reviews can identify unexpected harm such as in the Cardiac Arrhythmia Suppression Trial (28) that terminated early for increased mortality at 10% of the information fraction using an alpha spending function. A second early analysis at 40 or 50% information fraction can also identify strong convincing trends of benefit as in two trials that evaluated beta blocker drugs in chronic heart failure (29,30). Both trials terminated early at approximately 50% of the information fraction with mortality benefits. Computation of the alpha spending function can be facilitated by available software on the web (www.biostat.wisc.edu/landemets)

1.0

Figure 2. Comparison of spending functions α 1 (t*), α 2 (t*), and α 3 (t*) at information fractions t* = 0.2, 0.4, 0.6, 0.8, and 1.0.

or by commercial software packages (www. cytel.com/Products/East/default.asp). 3 APPLICATION OF THE ALPHA SPENDING FUNCTION Initial development of group sequential boundaries was for comparison of proportions or means (15,16,26). In these cases, the increments in information are represented by additional groups of subjects and their responses to the intervention. For comparing means or proportions, the information fraction t can be estimated by the n/N, the observed sample size divided by the expected sample size. However, later work expanded the use to other common statistical procedures. Tsiatis and colleagues (31,32) demonstrated that sequential logrank test statistics and the general class of rank statistics used in censored survival data had the independent increment structure that made the application to group sequential boundary straightforward. Later, Kim and Tsiatis (33)

Table 1. Comparison of boundaries using spending functions with Pocock (P) and O’Brien–Fleming (OBF) methods (α = 0.05, t* = 0.2, 0.4, 0.6, 0.8, and 1.0) t*

α 1 (t*)

OBF

α 2 (t*)

P

0.2 0.4 0.6 0.8 1.0

4.90 3.35 2.68 2.29 2.03

4.56 3.23 2.63 2.28 2.04

2.44 2.43 2.41 2.40 2.36

2.41 2.41 2.41 2.41 2.41

LANS-DEMETS: ALPHA SPENDING FUNCTION

demonstrated that the alpha spending function approach for sequential logrank tests was also appropriate. In this case, the information fraction is approximated by d/D, the number of observed events or deaths divided by the expected or design for number of events or deaths (34). Application of the alpha spending function for logrank tests has been used in several clinical trials (e.g., 28–30). Group sequential procedures including the alpha spending function have also been applied to longitudinal studies using a linear random effects model (35,36). Longitudinal studies have also been evaluated using generalized estimating equations (37). In a typical longitudinal clinical trial, subjects are added over time, and more observations are gathered for each subject during the course of the trial. One statistic commonly used is to evaluate the rate of change by essentially computing the slope of the observations for each subject and then taking a weighted average of these slopes over the subjects in each intervention arm. The sequential test statistics for comparison of slopes using the alpha spending function must take into account their distribution. If the information fraction is defined in terms of the Fisher information (i.e., inverse of the variance for the slopes), then the increments in the test statistic are independent, and the alpha function can be applied directly (38). The total expected information may not be known exactly, but it often can be estimated. Wu and Lan (36) provide other approaches to estimate the information fraction in this setting. Scharfstein and Tsiatis (39) demonstrated that any class of test statistics that satisfies specific likelihood function criteria will have this property and thus can be used directly in this group sequential setting. 4 CONFIDENCE INTERVALS AND ESTIMATION Confidence intervals for an unknown parameter θ following early stopping can be computed by using the same ordering of the sample space described by Tsiatis et al. (32) and by using a process developed by Kim and DeMets (25,40) for the alpha spending function procedures. The method can be briefly

5

summarized as follows: A 1–γ lower confidence limit is the smallest value of θ for which an event at least as extreme as the one observed has a probability of at least γ . A similar statement can be made for the upper limit. For example, if the first time the Z-value exists the boundary at tj with the observed Z*(j) ≥ Zc (j), then the upper θ U and lower θ L confidence limits are θ U = sup {θ : Pθ {Z(1) ≥ Zc (1), or · · · , or Z( j − 1) > Zc ( j − 1), or Z( j) ≥ Z∗ (j)} ≤ 1 − γ }} and θ L = sup {θ : Pθ {Z(1) ≥ Zc (1), or · · · , or Z( j − 1) ≥ Zc ( j − 1), or Z( j) ≥ Z∗ (j)} ≥ 1 − γ }} Confidence intervals obtained by this process will have coverage closer to 1 – γ than na¨ıve ˆ confidence intervals using θˆ ± Zγ /2 SE(θ). As an alternative to computing confidence intervals after early termination, Jennison and Turnbull (41) have allocated the calculation of repeated confidence intervals. This calculation is achieved by inverting a sequential test to obtain the appropriate coefficient Z∗α/2 in the general form for the confidence interval, θˆ ± Z∗α/2 SE(θˆ ). This inversion can be achieved when the sequential test is based on an alpha spending function. If we compute the interim analyses at the tj , obtaining corresponding critical values Zc (j), then the repeated confidence intervals are of the form θˆk ± Zc ( j)SE(θˆj ) where θˆj is the estimate for the parameter θ at the j-th analysis. Methodology has also been developed to obtain adjusted estimates for the intervention effect (42–47). Clinical trials that terminate early are prone to exaggerate the magnitude of the intervention effect. These methods shrink the observed estimate closer to the null. The size of the adjustments may depend on the specific sequential boundary employed. Conservative boundaries such as that proposed by Peto or O’Brien and Fleming

6

LANS-DEMETS: ALPHA SPENDING FUNCTION

generally require less adjustment, and the na¨ıve point estimate and confidence intervals may be quite adequate. Another issue is the relevance of the estimate to clinical practice. The population sample studied is usually not a representative sample of current and future practice. Subjects were those who passed all of the inclusion and exclusion criteria and volunteered to participate. Early subjects may differ with later subjects as experience is gained with the intervention. Thus, the intervention effect estimate may represent populations like the one studied, the only solid inference, but not as relevant to how the intervention will be used. Thus, complex adjustments may not be as useful. 5

TRIAL DESIGN

If any trial is planning to have interim analyses for monitoring for benefit or harm, then that plan must be taken into account in the design. The reason is that group sequential methods will impact the final critical value, and thus power, depending on which boundary is used. For the alpha spending function approach, the specific alpha function must be chosen in advance. In addition, for planning purposes, the anticipated number of interim analyses must be estimated. This number does not have to be adhered to in the application, but it is necessary for the design. Variance with this number in the application will not practically affect the power of the trial. Thus, the design strategy for the alpha spending function is similar to that strategy described by Pocock for the initial group sequential methods (15). The key factor when the sample size is computed is to take into consideration the critical value at the last analysis when the information fraction is 1.0. One simple approach is to use this new critical value in the standard sample size formula. This estimate will reasonably approximate a more exact approach described below. To illustrate, consider a trial that is comparing failure rates of successive groups of subjects. Here, H0 : pC − pT = 0 HA : pC − pT = δ = 0

where pC and pT denote the unknown response rates in the control and newtreatment groups, respectively. We would estimate the unknown parameter by pˆ C and pˆ T , the observed event rates in our trial. For a reasonably large sample size, we often use the following test statistics, Z=


pˆ C − pˆ T ˆ − p)(1/m ˆ p(1 C + 1/mT )

to compare event rates where pˆ is the combined event rate across treatment groups. For sufficiently large n where n = mC = mT , this statistics has an approximate standard normal distribution with mean and unit variance under the null hypothesis H0 : = 0. In this case, assuming equal sample size (n) per group in each arm,
n(pC − pT )/ 2p(1 − p)
√ = nδ/ 2p(1 − p)

=

√

where p = (pC + pT )/2 It follows that n=

2 2 p(1 − p) δ2

To design our studies, we evaluate the previous equation for n, the sample size per treatment per sequential group. Because the plan is to have J groups each of size 2n, the total sample size 2N equals 2nJ. Now, to obtain the sample size in the context of the alpha spending function, we proceed as follows: 1. For planning purposes, estimate the number of planned interim analyses J at equally spaced increments of information (i.e., 2n subjects). It is also possible to specify unequal increments, but equal space is sufficient for design purposes. 2. Obtain the boundary values of the K interim analyses under the null hypothesis H0 to achieve a prespecified overall alpha level, α, for a specific spending function α(t). 3. For the boundary obtained, obtain the value of to achieve a desired power (1–β).

LANS-DEMETS: ALPHA SPENDING FUNCTION

4. Determine the value of n that determines the total sample size 2N = 2nJ. 5. Having computed these design parameters, one may conduct the trial with interim analysis to be done based on the information fraction tj approximated by tj = Number of subjects observed/2N at the jth analysis (38). The number of actual interim analyses may not be equal to J, but the alpha level and the power will be affected only slightly (26). As a specific example, consider using an O’Brien–Fleming-type alpha spending function α 1 (t) with a one-sided 0.025 alpha level and 0.90 power at equally spaced increments at t = 0.2, 0.4, 0.6, 0.8 and 1.0. Using previous publications (16) or available computer software, we obtain boundary values 4.56, 3.23, 2.63, 2.28, and 2.04. Using these boundary values and available software, we find that = 1.28 provides the desired power of 0.90. If we specify pC = 0.6, pT = 0.4 (p = 0.5) under the alternative hypothesis, then we can obtain a sample size as follows. For = 1.28, n=

2(1.28)2 (0.5)(0.5) = 20.5, (0.2)2

and we have a total sample size of 2(21)5 = 210 subjects. We can then proceed to conduct interim analysis times at information fraction tj equal to the observed number of subjects divided by 210. Similar formulations can be developed for the comparison of means, repeated measures, and survival analysis (48). However, for most applications, the standard sample size formulas with the new alpha spending function final critical value will be a very good approximation. 6

CONCLUSIONS

The alpha spending function approach for group sequential interim analysis has provided the necessary flexibility that allows

7

data monitoring committees to fulfill their task. DMCs can adjust their analysis as data accumulates and trends emerge. As long as the alpha spending function is specified in advance, there is little room for abuse. Many trials sponsored by industry and government have successfully used this approach. Although the decision to terminate any trial early, for benefit or harm, is a very complex decision process, the alpha spending function can be an important factor in that process. REFERENCES 1. L. Friedman, C. Furberg, and D. DeMets, Fundamentals of Clinical Trials. Littleton, MA: John Wright – PSG Inc., 1981. 2. S. J. Pocock, Clinical Trials: A Practical Approach, New York: Wiley, 1983. 3. Heart Special Project Committee. Organization, review and administration of cooperative studies (Greenberg Report): a report from the Heart Special Project Committee to the National Advisory Council, May 1967. Control. Clin. Trials 1988; 9:137–48. 4. P. L. Canner, Monitoring treatment differences in long–term clinical trials. Biometrics 1977; 33:603–615. 5. Coronary Drug Project Research Group. Practical aspects of decision making in clinical trials: The Coronary Drug Project as a case study. Control. Clin. Trials 1982; 9:137–148. 6. D. L. DeMets, L. Friedman, C. D. Furberg, Data Monitoring in Clinical Trials: A Case Studies Approach. 2005. Springer Science + Business Media, New York, NY. 7. S. Ellenberg, T. Fleming and D. DeMets, Data Monitoring Committees in Clinical Trials: A Practical Perspective. West Sussex, UK: John Wiley & Sons, Ltd., 2002. 8. ICH Expert Working Group: International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH Harmonised Tripartite Guideline. Statistical principles for clinical trials. Stats. Med. 1999; 18:1905–1942. 9. U.S. Department of Health and Human Services. Food and Drug Administration. Docket No. 01D–0489. Guidance for Clinical Trial Sponsors on the Establishment and Operations of Clinical Trial Data Monitoring Committees. Federal Register 66:58151–58153, 2001. Available:

8

LANS-DEMETS: ALPHA SPENDING FUNCTION http://www.fda.gov/OHRMS/DOCKETS/98fr/ 112001b.pdf.

10. F. J. Anscombe, Sequential medical trials. Journal of the American Statistical Association 1963; 58:365–383. 11. I. Bross, Sequential medical plans. Biometrics 1952; 8:188–205. 12. P. Armitage, Sequential Medical Trials, 2nd ed. New York: John Wiley and Sons, 1975. 13. P. Armitage, C. K. McPherson, and B. C. Rowe, Repeated significance tests on accumulating data. J. Royal Stat. Soc. Series A 1969; 132:235–244. 14. R. Peto, M. C. Pike, P. Armitage, et al., Design and analysis of randomized clinical trials requiring prolonged observations of each patient. 1. Introduction and design. Br. J. Cancer 1976; 34:585–612. 15. S. J. Pocock: Group sequential methods in the design and analysis of clinical trials. Biometrika 1977; 64:191–199. 16. P. C. O’Brien and T. R. Fleming, A multiple testing procedure for clinical trials. Biometrics 1979; 35:549–556. 17. Beta–Blocker Heart Attack Trial Research Group. A randomized trial of propranolol in patients with acute myocardial infarction. I Mortality results. J. Amer. Med. Assoc. 1982; 247:1707–1714. 18. K. K. G. Lan and D. L. DeMets, Discrete sequential boundaries for clinical trials. Biometrika 1983; 70:659–663. 19. K. Kim and D. L. DeMets, Design and analysis of group sequential tests based on the type I error spending rate function. Biometrika 1987; 74:149–154.

45(3):1017–1020. 25. K. Kim and D. L. DeMets, Confidence intervals following group sequential tests in clinical trials. Biometrics 1987; 4:857–864. 26. K. Kim and D. L. DeMets, Sample size determination for group sequential clinical trials with immediate response. Stats. Med. 1992; 11:1391–1399. 27. Z. Li and N. L. Geller, On the choice of times for date analysis in group sequential trials. Biometrics 1991; 47:745–750. 28. Cardiac Arrhythmia Suppression Trial (CAST) Investigators. Preliminary report: Effect of endainide and flecainide on mortality in a randomized trial of arrhythmia suppression after myocardial infarction. N. Engl. J. Med. 1989; 321:406–412. 29. MERIT–HF Study Group. Effect of metoprolol CR/XL in chronic heart failure: Metoprolol CR/XL randomised intervention trial in congestive heart failure. Lancet 1999; 353:2001–2007. 30. M. Packer, A. J. S. Coats, M. B. Fowler, H. A. Katus, H. Krum, P. Mohacsi, J. L. Rouleau, M. Tendera, A. Castaigne, C. Staiger, et al. for the Carvedilol Prospective Randomized Cumulative Survival (COPERNICUS) Study Group. Effect of Carvedilol on survival in severe chronic heart failure. New Engl. J. Med. 2001; 334:1651–1658. 31. A. A. Tsiatis, Repeated significance testing for a general class of statistics used in censored survival analysis. J. Am. Stat. Assoc. 1982; 77:855–861. 32. A. A. Tsiatis, G. L. Rosner, and C. R. Mehta, Exact confidence intervals following a group sequential test. Biometrics 1984; 40:797–803.

20. I. K. Hwang, W. J. Shih, and J. S. DeCani, Group sequential designs using a family of type I error probability spending function. Stats. Med. 1990; 9:1439–45.

33. K. Kim and A. A. Tsiatis, Study duration for clinical trials with survival response and early stopping rule. Biometrics 1990; 46:81–92.

21. K. K. G. Lan and D. L. DeMets, Group sequential procedures: Calendar versus information time. Stats. Med. 1989; 8:1191–1198.

34. K. K. G. Lan and J. Lachin, Implementation of group sequential logrank tests in a maximum duration trial. Biometrics 1990; 46:759–770.

22. D. M. Reboussin, D. L. DeMets, K. M. Kim, and K. K. G. Lan, Computations for group sequential boundaries using the Lan–DeMets spending function method. Control. Clin. Trials 2000; 21:190–207.

35. J. W. Lee and D. L. DeMets, Sequential comparison of changes with repeated measurement data. J. Am. Stat. Assoc. 1991; 86:757–762.

23. M. A. Proschan, D. A. Follman, and M. A. Waclawiw. Effects of assumption violations on type I error rate in group sequential monitoring. Biometrics 1992; 48:1131–1143. 24. K. K. G. Lan and D. L. DeMets, Changing frequency of interim analyses in sequential monitoring. Biometrics 1989 Sept;

36. M. C. Wu and K. K. G. Lan, Sequential monitoring for comparison of changes in a response variable in clinical trials. Biometrics 1992; 48:765–779. 37. S. J. Gange and D. L. DeMets, Sequential monitoring of clinical trials with correlated categorical responses. Biometrika 1996; 83:157–167.

LANS-DEMETS: ALPHA SPENDING FUNCTION 38. K. K. G. Lan, D. M. Reboussin, D. L. DeMets: Information and information fractions for design and sequential monitoring of clinical trials. Communicat. Stat.– Theory Methods 1994; 23:403–420. 39. D. O. Scharfstein, A. A. Tsiatis, and J. M. Robins, Semiparametric efficiency and its implication on the design and analysis of group–sequential studies. J. Am. Stat. Assoc. 1997; 92:1342–1350. 40. K. Kim, Point estimation following group sequential tests. Biometrics 1989; 45:613–617. 41. C. Jennison and B. W. Turnbull: Interim analyses: The repeated confidence interval approach. J. Royal Stat. Soc., Series B 1989; 51:305–361. 42. S. S. Emerson and T. R. Fleming, Parameter estimation following group sequential hypothesis testing. Biometrika 1990; 77:875–892. 43. M. D. Hughes and S. J. Pocock, Stopping rules and estimation problems in clinical trials. Stats. Med. 1981; 7:1231–1241. 44. Z. Li and D. L. DeMets, On the bias of estimation of a Brownian motion drift following group sequential tests. Stat. Sinica 1999; 9:923–937. 45. J. C. Pinheiro and D. L. DeMets: Estimating and reducing bias in group sequential designs with Gaussian independent structure. Biometrika 1997; 84:831–843. 46. D. Siegmund, Estimation following sequential tests. Biometrika 1978; 65:341–349. 47. J. Whitehead, On the bias of maximum likelihood estimation following a sequential test. Biometrika 1986; 73:573–581. 48. D. L. DeMets and K. K. G. Lan, The alpha spending function approach to interim data analyses. In: P. Thall (ed.), Recent Advances in Clinical Trial Design and Analysis. Dordrecht, The Netherlands: Kluver Academic Publishers, 1995, pp. 1–27.

FURTHER READING M. N. Chang and P. C. O’Brien, Confidence intervals following group sequential tests. Control. Clin. Trials 1986; 7:18–26. T. Cook and D. L. DeMets, Statistical Methods in Clinical Trials. Boca Raton, FL: CRC Press/Taylor & Francis Co., 2007. D. L. DeMets, Data monitoring and sequential analysis – An academic perspective. J. Acq. Immune Def. Syn. 1990; 3(Suppl 2): S124–S133.

9

D. L. DeMets, R. Hardy, L. M. Friedman, and K. K. G. Lan, Statistical aspects of early termination in the Beta–Blocker Heart Attack Trial. Control. Clin. Trials 1984; 5:362–372. D. L. DeMets and K. K. G. Lan, Interim analysis: the alpha spending function approach. Stats. Med. 1994; 13:1341–1352. D. L. DeMets, Stopping guidelines vs. stopping rules: A practitioner’s point of view. Communicat. Stats.–Theory Methods 1984; 13:2395–2417. T. R. Fleming and D. L. DeMets, Monitoring of clinical trials: issues and recommendations. Control. Clin. Trials 1993; 14:183–197. J. L. Haybittle, Repeated assessment of results in clinical trials of cancer treatment. Brit. J. Radiol. 1971; 44:793–797. K. K. G. Lan, W. F. Rosenberger, and J. M. Lachin: sequential monitoring of survival data with the Wilcoxon statistic. Biometrics 1995; 51:1175–1183. J. W. Lee, Group sequential testing in clinical trials with multivariate observations: a review. Stats. Med. 1994; 13:101–111. C. L. Meinert, Clinical Trials: Design, Conduct, and Analysis. New York: Oxford University Press, 1986. S. Piantadosi. Clinical Trials: A Methodologic Perspective, 2nd ed. Hoboken, NJ: John Wiley and Sons, Inc., 2005. S. J. Pocock. Statistical and ethical issues in monitoring clinical trials. Stats. Med. 1993; 12:1459–1469. S. J. Pocock. When to stop a clinical trial. Br. Med. J. 1992; 305:235–240. E. Slud and L. J. Wei, Two–sample repeated significance tests based on the modified Wilcoxon statistic. J. Am. Stat. Assoc. 1982; 77:862–868. A. Wald, Sequential Analysis. New York: John Wiley and Sons, 1947.

LARGE, SIMPLE TRIALS

impact of daily aspirin reduces the risk of myocardial infarction. When carefully evaluated with precise estimates of effect and applied judiciously for maximum impact, relatively inexpensive interventions can produce disproportionate health benefits. Rare effects, especially those that are differential by treatment group, are difficult to detect or to distinguish from background rates in conventional studies. The demonstrated efficacy of a new drug or other medical product is established at a time when the limited clinical experience leaves the safety profile not yet fully defined. A large, simple trial with sufficient power to detect orders of magnitude differences in rare event rates can be very important (7). Serious adverse events that are not of concern at the start of the study may be explored using large health-care databases. Mitchell and Lesko (8) suggested that such was the case for the use of pediatric ibuprofen as an antipyretic. If such a difference is estimated reliably, health policy and practices can be developed to manage the risk while preserving the benefit. The need for larger trials to detect adverse event rates as low as 1 in 1000 or less has been emphasized for the full evaluation of new vaccines (9–11), highlighting the limitations of information derived from smaller trials with the ability to detect rates of 1 in 100 or higher. Similarly, large, simple trials have been used to address primary prevention questions (12). Large, simple trials have posed practical treatment questions relevant to a wide range of patients. If personalized effects applicable only to a single individual, characterized by gene expression analysis, are considered as one end of a spectrum, then large, simple trial issues would be at the opposite end of that spectrum. Cardiovascular disease research has been an active area for the application of large, simple clinical trials (13–15).

MARY A. FOULKES Center for Biologics Evaluation and Research U.S. Food and Drug Administration Rockville, Maryland

1

LARGE, SIMPLE TRIALS

Since the launch of the first randomized clinical trials in the middle of the twentieth century, immediate, major, and ubiquitous impacts on clinical practice and public health policy have been the result of very large randomized trials. For example, within hours of the results announced from the first oral polio vaccine field trial, the clarity of those results led to the regulatory approval of the vaccine and then to the rapid immunization of millions. That field trial was large (almost half a million children), simple (capture of incident cases of paralytic polio through public health records), randomized, and double blind (1, 2). The conclusions of that trial were unambiguous, leading to a seismic revolution in the fight against polio. This article describes all the characteristics that differentiate large, simple trials, sometimes called ‘‘megatrials’’ (3), from other types of trial, and we address important clinical questions that can have substantive impact: sizing the trial appropriate to the question, reliance on the bias control of randomization, minimizing the data collection, and other trial processes. 2 SMALL BUT CLINICALLY IMPORTANT OBJECTIVE The clinical effect size of interest dictates the number of subjects needed (or events observed) in any randomized, controlled trial. To identify a relatively small or moderate but clinically meaningful effect, the number of subjects needed (or number of events observed) in the trial may be very large (4–6). There are numerous instances where clinical effects that are limited can be used to substantial benefit from a population perspective. For example, in cardiovascular disease, the demonstration of a small but meaningful

2.1 Minimal Detectable Difference The difference to be estimated between the treatment groups needs to be clinically important when balanced against potential side effects, and practical enough to alter clinical practice. These differences could have major

Wiley Encyclopedia of Clinical Trials, Copyright  2007, John Wiley & Sons, Inc.

1

2

LARGE, SIMPLE TRIALS

public health impact when the interventions might ultimately be available to millions of patients. For example, the VITATOPS trial (16) aims to investigate whether a 15% relative reduction in the rate of stroke, myocardial infarction, or death from any vascular cause in patients with recent transient ischemic attack or stroke can result from a multivitamin intervention. The VITATOPS trial is expected to enroll approximately 8,000 patients, with a trial completion in 2010. It is important for the progress of medicine and public health that small to moderate (but real) benefits be reliably distinguished from no benefit, and this distinction requires large trials. 2.2 Sample Size Based on the minimal detectable difference, the sample size should be sufficient to reliably estimate the difference between treatment groups. When these differences are assumed to be relatively small, the resulting sample size estimates are large, in the tens of thousands of patients. The large sample size is necessary ‘‘not only to be correct but also to be convincing,’’ argues Peto (17). The sample size estimation process should focus on minimizing the risk of a false-negative result, a type II error. A trial of sufficient size will also limit the width of the confidence interval for the effect estimate, reducing uncertainty in applying the trial results. 2.3 Cost and Time Annual health-care expenditures, both direct and indirect, can be substantially impacted by small but important health-care advances that influence large numbers of people, as with cardiovascular disease, diabetes, or many other diseases. Effective preventive measures can also have major health-care cost benefits. The resource costs of a large trial, including time spent on protocol mandated follow-up procedures and data collection, can be minimized. If trial designs are stripped to the essentials, once the intervention is delivered, the time and cost of the trial are a function of the primary outcome measure. For a large, simple trial to be something other than a conventional trial

multiplied by orders of magnitude in sample size, cost, time, and effort, every action and every requirement of the trial should be questioned and only maintained as part of the trial if the contribution to the end result is substantial. This implies judicious inclusion, if at all, of extraneous substudies, ancillary studies, and other nonessential additions. Another costly aspect of trials that could be avoided in large, simple trials is the overhead associated with reliance on tertiary care institutions or clinical research institutions as the clinical sites for a trial. These may provide ideal sites for early clinical studies of complex interventions, or for studies requiring extensive clinical work-up to establish eligibility, but the usual health-care settings are the sites for large, simple trials. These usual health-care settings are also the ultimate site of application of large, simple trial results. 3 ELIGIBILITY Simple eligibility will allow easy, rapid identification and enrollment of eligible patients. The eligibility screen should be easily interpretable not only to those enrolling patients into the trial, but ultimately to those who may use the results if applicable. To be most widely interpretable, such eligibility requirements should not rely on the highest end ‘‘high-tech’’ diagnostics, which might not be widely available or may be completely unavailable outside the trial. 3.1 Simple, Broad In contrast to many clinical trials with very specific, thoroughly documented eligibility criteria, a large, simple trial engulfs a broad swath of patients with the general indication (18). Patients may enter from many different sites, from many different treating clinicians and practices, and from many different health-care systems, so the eligibility criteria need to be clear, interpretable, and consistent. Minimal testing should be required to determine eligibility. The exclusions should be a function of contraindications to any of the interventions. Patients thought to be at high risk of adverse events or with an independent life-threatening disease are generally

LARGE, SIMPLE TRIALS

excluded. In the Fourth International Study of Infarct Survival (ISIS-4) trial, for example, all patients presenting within 24 hours of a suspected acute myocardial infarction (with no clear indications for or contraindications to any one of the trial treatments) were eligible (19). 3.2 Multicenter and Multinational To rapidly enroll large numbers of patients, these trials are necessarily multicenter trials. They are often conducted in many countries simultaneously. This lends to the generalizability of results, but it also requires that definitions, eligibility screening, intervention delivery, and data capture be coordinated and harmonized multinationally. The uncertainty principle (20) as it applies to substantial uncertainty about which intervention is more beneficial can easily vary across countries, across health-care systems, and over time. Although internal validity would not be impacted, variation across countries and over time may impact the enrollment and the ultimate generalizability of results. One example of accommodation of such variations across countries was in the Options in Management with Antiretrovirals (OPTIMA) trial (21), a factorial design in which some countries enrolled to both factors and some to only one factor. 4

RANDOMIZED ASSIGNMENT

The strength of a large, simple trial over any similar-sized comparison of differently treated groups of individuals is the a priori assignment by chance to treatment group. Randomization ensures that the treatment groups are similar in both measured and unmeasured factors such that the treatment can be assumed to be the primary difference between the groups. Randomization addresses selection bias in the assignment of patients to groups. The strength of randomization has been widely articulated (22) and is a major strength in inferences from largescale randomized evidence and large, simple trials in particular. Randomization also provides a method of controlling confounding, of both known and unknown confounders. With or without stratification, randomization

3

is relied on to control confounding where the simple character of the trial would be compromised by other methods such as eligibility restriction. 4.1 Simple Process The actual randomization process should be as simple as possible. This process has benefited substantially from global telecommunications, networks, and interactive websites. It is possible for a single, central organization to manage and execute the randomization process. The randomization process is often the only point of capture for baseline information, so only the most important and clearly defined baseline factors (e.g., gender) should be recorded at this point. 4.2 Blinding Many clinical trials are blinded, but the incorporation of blinding into the conduct of a large, simple trial must be carefully weighed. To effect blinding, the interventions must be sufficiently similar that the large, simple nature of the trial is not compromised. If a blinded trial might require an extra intravenous line, as might have been the case had the Global Utilization of Streptokinase and t-PA for Occulted Arteries (GUSTO) trial (23) been blinded, the large size and the simple conduct might have been impossible. The Francis Field Trial (1), on the other hand, was blinded and involved delivery of identical placebo or polio vaccine. The large, simple nature of that trial was not impinged by blinding. Blinding of the outcome evaluation or adjudication, however, does not depend on blinding the patients to their randomized treatment groups. The integrity of trial results can be strengthened by evaluation by blinded reviewers. When the primary outcome is allcause mortality, blinded reviewers and adjudication are unnecessary. 4.3 Intervention Administered/Compliance The typical interventions chosen for evaluation in a large, simple trial are practical to deliver, such as the oral polio vaccine, and relatively short-term, such as a few hours of intravenous thrombolytic agent infusion.

4

LARGE, SIMPLE TRIALS

Longer term or self-administered interventions bring up issues of compliance. As with all trials, achieving and maintaining high compliance is important; the less compliance, the more the intervention effects begin to coincide. The usual methods of maintaining and improving compliance—minimizing the length and complexity of intervention, selecting expected compliant patients, maintaining frequent follow-up contact, and designing simple measures of compliance—should be applied to the extent possible while maintaining the large, simple character of the trial (24). 4.4 Patient Management All other aspects of patient management, other than the randomized comparison, should mimic local clinical practice and should be at the discretion of the treating clinician. Although this may introduce wide variation in the ancillary treatments, the results over a very large group of patients and clinicians will reflect the heterogeneity in practice. There should be no additional record keeping, laboratory tests, or other investigations and no extra follow-up visits. 5

OUTCOME MEASURES

The primary trial endpoint or outcome measure in a large, simple trial should be one that is directly relevant to the subject, that is easily and objectively measured with minimal resources. The outcome measure for a given trial substantially affects how large and simple the trial can feasibly be. It also affects the reliability of the effect estimates and the relevancy of the trial results. 5.1 Objectively Determined Survival, or mortality, is often the primary outcome measure in large, simple trials. It has many advantages: it is an easy to ascertain, objectively determined, binary, absorbing state. With the availability of healthcare databases and other national databases, information on survival status, including date, location, and cause of death, can often be captured without active follow-up efforts. Outcomes other than mortality can also be objectively determined.

5.2 Minimal Missing The amount and detail of the data collected, particularly after baseline, should be limited. The clear necessity of each data item will help to ensure that the participating patients, clinicians, and others will consistently provide the limited data requested, thus minimizing the amount of essential data that is missing. 5.3 Minimal Follow-up Depending on the primary endpoint of the trial, frequent, lengthy, labor-intensive, and high-tech direct medical assessments are not necessary in clinical trials. If the relevant primary endpoint can be adequately captured by indirect contact, clinical follow-up efforts can be minimized. Mail and telephone follow-up contacts have been employed, and e-mail could similarly be used. Lubsen and Tijssen (25) have argued that simple protocols with minimal follow-up contact are only appropriate when the mechanism of action is well understood and when the sideeffect profile is well understood. In general, minimal follow-up contact does not provide information on intermediate outcomes and does not contribute to greater understanding the mechanism of action. 5.4 Limited and Automated Data Capture The availability of health-care databases and other sources of automated data further simplify the capture of outcome data. The use of automated means to transmit data as effortlessly as possible also contributes to the completeness of the resulting data. The Francis Field Trial (1) used the existing system of public health surveillance and the required reporting of incident cases. National health systems can be used to provide hospitalization or mortality information. In the United Kingdom, this is facilitated when trial participants are ‘‘flagged’’ in the National Health Service (NHS) central register. Horsburgh (26) has described the infrastructure requirements for a public health surveillance system to capture a common disease outcome (e.g., tuberculosis). The practice of trying to answer as many questions as possible within a single trial is the antithesis of the simplicity required for

LARGE, SIMPLE TRIALS

a large, simple trial. A trial that attempts to address quality of life, economic issues, and other questions simultaneously is not feasible on a large-sample scale. The capture of data items extraneous to the primary central question must be avoided (27). The design and conduct of a large, simple trial must be diligently and vigilantly preserved; otherwise, the trial will become too complex to be either large or simple. Maintaining the simplicity of the follow-up and data capture procedure, without a routine comprehensive assessment of adverse events, and relying on randomization to control biases, the differences between treatment groups in rates of unexpected and rare adverse events can be evaluated by using large health-care databases, such those used in pharmcoepidemiology. These health-care databases could include the U.S. Centers for Medicare and Medicaid Services (CMS) Database, the U.K. and Italian General Practice Research Databases, the German MediPlus Database, various health maintenance organization databases, and many more. Serious adverse events, hospitalizations, and deaths can be captured by these sources rather than by sacrificing the simplicity of the large simple trial. 5.5 Subgroups The expectation in a large, simple trial is that the primary effect may vary by subgroups but is not likely to be reversed (28). The direction and consistency of effect across subgroups is one rationale for avoiding the collection of data to identify subgroups. Gray et al. (28) concluded that ‘‘most studies could be much larger, and of so much greater scientific value, if they had wider entry criteria and collected ten times less data.’’ Trial designers for many years have been warning that ‘‘marked heterogeneity of the treatment comparison in different strata can arise by chance more easily than would intuitively be expected’’ (29). The pitfalls of subgroup analyses are widely known (30); the ISIS-2 trial report provides a classic example, an analysis of effects across astrological subgroups (31). Qualitative rather than quantitative differences, though unexpected and uncommon, should be of concern.

5

Large, simple trials can include planned, small substudies, for example, at only certain sites, to address more specific hypotheses and requiring more complex follow-up information. These substudies would have no impact on the overall simplicity of the trial for the majority of patients. 5.6 Impact on Practice An additional outcome measure from a large, simple trial is the timely and pervasive impact on clinical practice. When a clear and convincing result to important questions shows a real effect that benefits the patient, that result should be widely reported and disseminated. The external influences on clinical decisions (e.g., financial) often respond rapidly to these results, accelerating the uptake of the results. The Gruppo Italiano per lo Studio della Streptochinasi nell’Infarto Miocaridico 1 (GISSI-1) trial result, for example, became common (near universal) practice in Italian coronary care units within 1 year of reporting (32). Contrast this rapid pace of uptake with control of hypertension: despite numerous trials showing the positive effects of blood pressure control, according to a recent survey of the U.S. population, control of hypertension is stagnant at 34% (33). Because most individuals with elevated blood pressure are not being adequately treated, the demonstration within randomized controlled trials of a treatment effect is only a piece of a larger, more complex cluster of conditions. Even trials that have begun to enroll can benefit from further simplification, as seen in a trial of trastuzumab (Herceptin) (34). The simplification in this case involved broadening the eligibility criteria to more closely reflect the actual target patient population, streamlining study procedures (reducing frequencies and durations of procedures), and reconsidering and eliminating the placebo (and any need for blinding). As the result of these simplifications, the enrollment rate accelerated, risks of infections and other complications of multiple infusions dropped, and the resources to enroll and follow each patient were cut. These trial design and conduct aspects can be simplified in many trials, making a large trial feasible.

6

6

LARGE, SIMPLE TRIALS

CONCLUSIONS

By design, bias and confounding are minimized in randomized trials, and substantial power to detect moderate-sized effects is afforded by large sample sizes. These attributes lead to relevant, usable information that is generalizable to a broad population. Large, simple trials are not appropriate to every clinical question, but they can certainly contribute to clinical practice and public health policy. Practice and policy decisions to be addressed in the future will include decision-making based on knowledge of small but important clinical effects. Large, simple trials have a place in the full range of clinical study designs. REFERENCES 1. T. Francis Jr., Evaluation of the 1954 Field Trial of Poliomyelitis Vaccine: Final Report. 1957. Ann Arbor, MI: Edwards Brothers/National Foundation for Infantile Paralysis; School of Public Health, University of Michigan, 1957. 2. T. Francis, Jr., An evaluation of the 1954 Poliomyelitis Vaccine Trials—summary report. Am J Public Health. 1955; 45: 1–63. 3. D. Heng, Megatrials of drug treatment: strengths and limitations. Ann Acad Med Singap. 2000; 29: 606–609. 4. S. Yusuf, R. Collins, and R. Peto. Why do we need some large, simple randomized trials? Stat Med. 1984; 3: 409–420. 5. C. Baigent, The need for large-scale randomized evidence. Br J Clin Pharmacol. 1997; 43: 349–353. 6. R. M. Califf and D. L. DeMets, Principles from clinical trials relevant to clinical practice: part I. Circulation. 2002; 106: 1015–1021. 7. R. Temple, Meta-analysis and epidemiologic studies in drug development and postmarketing surveillance. JAMA. 1999; 281: 841–844. 8. A. A. Mitchell and S. M. Lesko, When a randomized controlled trial is needed to assess drug safety. The case of paediatric ibuprofen. Drug Saf. 1995; 13: 15–24. 9. B. L. Strom, How the US drug safety system should be changed. JAMA. 2006; 295: 2072–2075. 10. S. S. Ellenberg, M. A. Foulkes, K. Midthun, and K. L. Goldenthal, Evaluating the safety of

new vaccines: summary of a workshop. Am J Public Health. 2005; 95: 800–807. 11. J. Clemens, R. Brenner, M. Rao, N. Tafari, and C. Lowe, Evaluation of new vaccines for developing countries. Efficacy or effectiveness? JAMA. 1996; 275: 390–397. 12. J. E. Buring and C. H. Hennekens, The contributions of large, simple trials to prevention research. Prev Med. 1994; 23: 595–598. 13. C. D. Furberg, S. Yusuf, and T. J. Thom, Potential for altering the natural-history of congestive heart-failure—need for large clinical trials. Am J Cardiol. 1985; 55: A45–A47. 14. S. Yusuf and R. Garg, Design, results, and interpretation of randomized, controlled trials in congestive-heart-failure and leftventricular dysfunction. Circulation. 1993; 87: 115–121. 15. M. D. Flather, M. E. Farkouh, and S. Yusuf. Large simple trials in cardiovascular disease: their impact on medical practice. In: R. M. Califf, B. Mark, and G. S. Wagner (eds.), Acute Coronary Care. St. Louis, MO: Mosby, 1995, pp. 131–144. 16. VITATOPS Trial Study Group. The VITATOPS (Vitamins to Prevent Stroke) Trial: rationale and design of an international, large, simple, randomized trial of homocysteine-lowering multivitamin therapy in patients with recent transient ischaemic attack or stroke. Cerebrovasc Dis. 2002; 13: 120–126. 17. R. Peto, Clinical trial methodology. Biomedicine. 1978; 28(Special): 24–36. 18. S. Yusuf, P. Held, K. K. Teo, et al. Selection of patients for randomized controlled trials: implications of wide or narrow eligibility criteria. Stat Med. 1990; 9: 73–86. 19. ISIS-4 (Fourth International Study of Infarct Survival) Collaborative Group. A randomised factorial trial assessing early oral captopril, oral mononitrate, and intravenous magnesium sulphate in 58,050 patients with suspected acute myocardial infarction. Lancet. 1995; 345: 669–685. 20. R. Peto and C. Baigent, Trials: the next 50 years. Large scale randomized evidence of moderate benefits. BMJ. 1998; 317: 1170–1171. 21. T. C. Kyriakides, A. Babiker, and J. Singer, et al. An open-label randomized clinical trial of novel therapeutic strategies for HIV-infected patients in whom antiretroviral therapy has failed: rationale and design of OPTIMA trial. Control Clin Trials. 2003; 24: 481–500.

LARGE, SIMPLE TRIALS 22. R. Peto, R. Collins, and R. Gray, Largescale randomized evidence—large, simple trials and overviews of trials. J Clin Epidemiol. 1995; 48: 23–40. 23. Global Utilization of Streptokinase and t-PA for Occulted Arteries (GUSTO) Investigators. An international randomized trial comparing four thrombolytic strategies for acute myocardial infarction. N Engl J Med. 1993; 329: 673–682. 24. C. Hennekens and J. E. Buring, Need for large sample sizes in randomized trials. Pediatrics. 1987; 79: 569–571. 25. J. Lubsen and J. G. P. Tijssen, Large trials with simple protocols: indications and contraindications. Control Clin Trials. 1989; 10: 151S–160S. 26. C. R. Horsburgh, A large, simple trial of a tuberculosis vaccine. Clin Infect Dis. 2000; 30 (Suppl 3): S213–216. 27. S. Yusuf, Randomised controlled trials in cardiovascular medicine: past achievements, future challenges. BMJ. 1999; 319: 564–568. 28. R. Gray, M. Clarke, R. Collins, and R. Peto, Making randomized trials larger: a simple solution? Eur J Surg Oncol. 1995; 21: 137–139. 29. R. Peto, M. C. Pike, P. Armitage, N. E. Breslow, D. R. Cox, et al. Design and analysis of randomized clinical trials requiring prolonged observation of each patient. II. Analysis and examples. Br J Cancer. 1977; 35: 1–39. 30. S. Yusuf, J. Wittes, J. Probstfield, and H. A. Tyroler, Analysis and interpretation of treatment effects in subgroups of patients in randomized clinical trials. JAMA. 1991; 266: 93–98. 31. ISIS-2 (Second International Study of Infarct Survival) Collaborative Group. Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17,187 cases of suspected acute myocardial infarction: ISIS-2. Lancet. 1988; 2: 349–360. 32. G. Tognoni, M. G. Franzosi, S. Garattini, and A. Maggioni, The case of GISSI in changing the attitudes and practice of Italian cardiologists. Stat Med. 1990; 9: 17–27. 33. A. V. Chobanian G. L. Bakris, H. R. Black, W. C. Cushman, L. A. Green, et al., for the National Heart, Lung, and Blood Institute Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure; National High Blood Pressure Education Program Coordinating Committee. The seventh report of the Joint National Committee on Prevention, Detection, Evaluation,

7

and Treatment of High Blood Pressure: the JNC 7 report. JAMA. 2003; 289: 2560–2572. 34. T. R. Fleming, Issues in the design of clinical trials: insights from the transuzumab (Herceptin) experience. Semin Oncol. 1999; 26: 102–107.

FURTHER READING Selected Examples of Large, Simple Trials ALLHAT Collaborative Research Group. Major cardiovascular events in hypertensive patients randomized to doxazosin vs chlorthalidone: the antihypertensive and lipid-lowering treatment to prevent heart attack trial (ALLHAT). JAMA. 2000; 283: 1967–1975. M. L. Barreto, L. C. Rodrizues, S. S. Cunha, S. Pereira, et al. Design of the Brazilian BCGREVAC trial against tuberculosis: a large, simple randomized community trial to evaluate the impact on tuberculosis of BCG revaccination at school age. Control Clin Trials. 2002; 23: 540–553. S. Black, H. Shinefield, M. A. Fireman, et al. Efficacy, safety and immunogenicity of the heptavalent pneumococcal conjugate vaccine in children. Pediatr Infect Dis J. 2000; 19: 187–195. Chinese Acute Stroke Trial (CAST) Collaborative Group. CAST: a randomised placebo-controlled trial of early aspirin use in 20,000 patients with acute ischaemic stroke. Lancet. 1997: 349: 1641–1649. R. Collins, R. Peto, and J. Armitage. The MRC/BHF Heart Protection Study: preliminary results. Int J Clin Pract. 2002; 56: 53–56. CRASH Trial Pilot Study Collaborative Group. The MRC CRASH Trial: study design, baseline data, and outcome in 1000 randomised patients in the pilot phase. Emerg Med J. 2002; 19: 510–514. Digitalis Investigation Group (DIG). The effect of digoxin on mortality and morbidity in patients with heart failure. N Engl J Med. 1997; 336: 525–533. Gruppo Italiano per lo Studio della Streptochinasi nell’Infarto miocaridico (GISSI). Effectiveness of intravenous thrombolytic treatment in acute myocardial infarction. Lancet. 1986; 1: 397–402. P. Edwards, M. Aragno, L. Balica, and R. Cottingham, Final results of the MRC CRASH, a randomized placebo-controlled trial of intravenous corticosteroid in adults with head injury—outcomes at 6 months. Lancet. 2005; 365: 1957–1959.

8

LARGE, SIMPLE TRIALS

Gruppo Italiano per lo Studio della Streptochinasi nell’Infarto Miocaridico (GISSI). GISSI-2: a factorial randomized trial of alteplase versus streptokinase and heparin versus no heparin among 12,490 patients with acute myocardial infarction. Lancet. 1990; 336: 65–71. HOPE Study Investigators. The HOPE (Heart Outcomes Prevention Evaluation) Study: the design of a large, simple randomized trial of an angiotensin-converting enzyme inhibitor (ramipril) and vitamin E in patients at high risk of cardiovascular events. Cardiovasc Med. 1996; 12: 127–137. International Stroke Trial Collaborative Group. International Stroke Trial (IST): a randomised trial of aspirin, subcutaneous heparin, both or neither among 19,435 patients with acute ischaemic stroke. Lancet. 1997; 349: 1569–1581. ISIS-1 (First International Study of Infarct Survival) Collaborative Group. Randomised trial of intravenous atenolol among 16,027 cases of suspected acute myocardial infarction: ISIS-1. Lancet. 1986; 2: 57–66. ISIS-3 (Third International Study of Infarct Survival) Collaborative Group. ISIS-3: a randomised trial of streptokinase vs tissue plasminogen activator vs anistreplase and of aspirin plus heparin vs aspirin alone among 41,299 cases of suspected acute myocardial infarction. Lancet. 1992; 339: 753–770.

S. M. Lesko and A. A. Mitchell, An assessment of the safety of pediatric ibuprofen. A practitionerbased randomized clinical trial. JAMA. 1995; 273: 929–933. MAGIC Steering Committee. Rationale and design of the magnesium in coronaries (MAGIC) study: a clinical trial to reevaluate the efficacy or early administration of magnesium in acute myocardial infarction. Am Heart J. 2000; 139: 10–14. SOLVD Investigators. Studies of left ventricular dysfunction (SOLVD)—rationale, design and methods: two trials that evaluate the effect of enalapril in patients with reduced ejection fraction. Am J Cardiol. 1990; 66: 315–322. E. Van Ganse, J. K. Jones, N. Moore, J. M. Le Parc, R. Wall, and H. Schneid, A large simple clinical trial prototype for assessment of OTC drug effects using patient-reported data. Pharmacoepidemiol Drug Saf. 2005; 14: 249–255.

CROSS-REFERENCES Effect size Eligibility Expanded safety trials Hypothesis Sample size

LINEAR MODEL

variation, and goodness of fit. Understanding the basic structure of a (General) Linear Model is extremely helpful in the practical application of statistical models and the interpretation of their results. This article presents the Linear Model in its general form, discusses its three major subtypes, namely Linear Regression, Analysis of Variance (ANOVA), and Analysis of Covariance (ANCOVA), and explains how extensions are related to Linear Model and how they are interrelated between themselves. Therefore, the aim is to provide a comprehensive overview. For specific modeling approaches, refer to the respective articles on Linear Regression, ANOVA, and Generalized Linear Models.

DR. IRIS BURKHOLDER DR. LUTZ EDLER German Cancer Research Center Department of Biostatistics Heidelberg, Germany

1

INTRODUCTION

Linear Models for simple and multiple regression, Analysis of Variance (ANOVA), and Analysis of Covariance (ANCOVA) have broad and often direct application in clinical trials. Various extensions and generalizations of the Linear Model have been developed more recently often motivated by medical research problems. Most prominent are the loglinear models for the evaluation of count data; the logistic regression model for the evaluation of the influence of clinical factors on a dichotomous outcome as, for example, cancer incidence; and, last but not least, the Cox regression model for the analysis of survival data. The Linear Model, also called General Linear Model (GLM) when extended to incorporate classes of predictive factors, is built on the assumption of response data following a Gaussian (Normal) distribution. In contrast, extensions of the GLM no longer require this assumption, but are either based on other distributions or on allowing more general classes of distributions. The Generalized Linear Model—not to be confused with the General Linear Model—became an extremely valuable and versatile modeling approach for more general types of data (distributions from the exponential family) including the General Linear Model as a special case. Further extensions have been models for repeated measurements and the General Estimation Equation (GEE) approach. It is important to note that central features of the Linear Model were retained in those extensions. Features of similarity and analogy include such important components as model building, variable selection, maximum likelihood technique, residual analysis, explained

2

LINEAR MODEL

The Linear Model represents the relationship between a continuous response variable y and one or more predictor variables X in the form: y = Xβ + ε

(1)

where • y is a vector of observations of the

response variable • X is the design matrix determined by

the predictors • β is a vector of unknown parameters to

be estimated • ε is a vector of random disturbances,

independent of each other and usually having a normal distribution with mean zero and variance σ 2 . Remarks: 1. As an example, consider as response variable y the blood pressure (mmHg) of a patient and as predictor variables x1 = gender, x2 = age, x3 = smoking status, and x4 = treatment status (beta-blocker therapy). The variables gender, smoking status, and treatment status are measured on a qualitative

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

LINEAR MODEL

scale, whereas age is a quantitative variable. 2. It is very common to specify by subscripts the dimension of the quantities of model Equation (1)

where • yi denotes the ith observed response • β 0 is a constant intercept • β j , j = 1, . . . , p are the unknown regres-

sion parameters

ynx1 = Xnxp βpx1 + εnx1

• xij , j = 1, . . . , p are the values of the p

indicating that y is a column vector of length n of n single observations (y1 , . . ., yn ) and X is a matrix (Xij )i=1,...,n j=1,...,p with n columns and p rows. Notice, n is the sample size of the respective clinical study. 3. Model Equation (1) assumed ε ∼ N(0, σ 2 ) as normal distribution with mean zero and the same variance σ 2 for each observation. More general models exist that allow different variance σi2 for the individual observations (heteroscedasticity). Then, one writes ε = εnx1 ∼ N(0, ) where 0 = (0, . . . , 0) is a vector of length n with all components equal to zero and where  is a diagonal matrix with σ12 , . . . , σn2 in the diagonal and all other elements equal to zero ( = diag(σ12 , . . . , σn2 )). Three subtypes of classic Linear Models can be distinguished by the composition of the design matrix X (see Fig. 1). The Linear Model leads to – Regression analysis, if X contains only quantitative values (covariates), – Analysis of variance, if X contains only categorical variables, – Analysis of covariance, if X contains quantitative as well as categorical variables.

explanatory variables • ε i is the ith error term.

Remark: 1. If all p explanatory variables are zero (i.e., if they add no information) Equation (2) reduces to the constant regression y = β0 + ε. In simple regression yi = β0 + β1 x1 + ε, β 0 is exactly the intercept on the y-axis and β 1 the slope of the regression line. 2. Details about estimation of the regression parameters and interpretation of the parameters can be found in the article on Linear Regression Analysis. 2.1.1 Historical Example of Simple Linear Regression. The British scientist Sir Francis Galton (1) was studying the inheritance of physical characteristics. In particular, he wondered if he could predict a boy’s adult height based on the height of his father. He plotted the heights of fathers and the heights of their sons for a number of father-son pairs and then tried to fit a straight line through the data. Let • y be the height of the first, fully grown,

son (continuous response) and • x be the height of the father (the contin-

uous predictor). 2.1 Simple and Multiple Regression Linear regression is a technique for modeling a continuous response or outcome variable as a linear combination of one (simple regression) or several potential (multiple regression) continuous explanatory factors. The general model equation for p explanatory variables is given by yi = β0 + β1 xi1 + . . . + βp xip + εi , i = 1, . . . , n

(2)

One can say that, in mathematical terms, Galton wanted to determine the intercept constant β 0 and the regression parameter β 1 . He observed that when choosing a group of parents of a given height, the mean height of their children was closer to the mean height of the population than is the given height. In other words, tall parents tend to be taller than their children and short parents tend to be shorter. Galton termed this phenomenon ‘‘regression towards mediocrity’’

LINEAR MODEL

meaning ‘‘going back towards the average.’’ The mean height of the children was closer to the mean height of all children than the mean height of their parents was to the mean height of all parents. In mathematical terms, unless X and Y are exactly linearly related for a given value of X, the predicted value of Y is always fewer standard deviations from its mean than is X from its mean. Regression toward the mean appears unless X and Y are perfectly correlated, so it always occurs in practice. 2.2 Analysis of Variance (ANOVA) The Analysis of variance is used to uncover the main and interaction effects of categorical independent variables (factors) X on the dependent variable y. A main effect is the direct effect of an independent variable on the dependent variable. An interaction effect is the joint effect of two or more independent variables on the dependent variable. Depending on the number of factors used in an ANOVA model, one distinguishes one-way ANOVA and multi-way (p-way) ANOVA models. The one-way ANOVA investigates the effects of one categorical factor on the dependent variable. If that factor has k different values (features), one can test for differences of the dependent variable between the groups defined by those different features. A dichotomous categorical variable can be included as predictor variable in the ANOVA model by creating a dummy variable that is 0 if the characteristic is absent and 1 if it is present. For example, a dummy variable representing the sex of patients in a clinical trial can take the value 0 for females and 1

3

for males. The coefficient then represents the difference in mean between male and female. Variables with more than two categories cannot simply be inserted as one factor in the ANOVA, unless it can be assumed that the categories are ordered in the same way as their codes, and that adjoining categories are in some sense the same distance apart (i.e., the variable possesses a metric scale). As this assumption is very strong, a set of dummy variables is created instead to represent the predictive factor. The number of dummy variables is the number of categories of that factor minus one. For example, in patients with nonsmall-cell lung cancer, three different histologic tumor types are usually distinguished: adenocarcinoma, squamous cell carcinoma, and large cell carcinoma. The variable histologic tumor type could be included in an ANOVA model by creating two dummy variables: histo1 = 1 if patient has adenocarcinoma, 0 otherwise, and histo2 = 1 if patient has squamous cell carcinoma, 0 otherwise.

Obviously, the patient has large cell carcinoma if histo1 = 0 and histo2 = 0. Therefore, the two dummy variables histo1 and histo2 characterize each patient of that study. The mathematical model that describes the relationship between the response and a categorical factor for the one-way ANOVA is

Linear modeling

Simple and multiple regression

Analysis of Variance ANOVA

Analysis of Covariance ANCOVA

only quantitative covariates

only categorical variables

quantitative covariates and categorical variables

Figure 1. Differences between the three subtypes of Linear Models: regression, analysis of variance, and analysis of covariance

4

LINEAR MODEL

given by

where yij = µ + τi + εij

(3)

• yij represents the kth observation on the

where • yij represents the jth observation on the

ith level of the categorical factor • µ is the common effect for the whole population • τ i denotes the effect of the ith level • ε ij is the random error present in the jth observation on the ith level and the errors are assumed to be normally and independently distributed with mean zero and variance σ 2 . 2.2.1 Example: One-way ANOVA. In four groups of patients, measures of gut permeability are obtained (2) and the differences between the groups should be investigated. In the one-way ANOVA model, let • yij the continuous response variable indi-

cating the result of the permeability test for the jth individual in the ith patient group (i = 1, . . . 4) • τ i denotes the effect of the ith patient group (i = 1, . . . 4). The basic idea of a one-way ANOVA consists in considering the overall variability in the data and partitioning the variability into two parts: between-group and within-group variability. If between-group variability is much larger than within-group variability, it suggests that differences between the groups are real, not just random noise. Two-way ANOVA analyzes one dependent variable in terms of the categories (groups) formed by two independent factors, which generalises to p-way ANOVA, which deals p independent factors. It should be noted that as the number of independent factors increases, the number of potential interactions proliferates. Two independents have one single first-order interaction only. The two-way ANOVA model that describes the relationship between the response, two categorical factors, and their interaction is given by yijk = µ + τi + δj + (τ δ)ij + εijk

(4)

• • • •

•

ith level of the first categorical factor and on the jth level of the second categorical factor µ is the common effect for the whole population τ i denotes the effect of the ith level of the first categorical factor δ j denotes the effect of the jth level of the second categorical factor (τ δ)ij represents the interaction between the ith level of the first categorical factor and the jth level of the second categorical factor εijk is the random error present in the kth observation on the ith level of the first categorical factor and on the jth level of the second categorical factor and the errors are assumed to be normally and independently distributed with mean zero and variance σ 2 .

2.2.2 Example: Interaction in Two-way ANOVA. Consider an experiment to determine the effect of exercise and diet on cholesterol levels. Both factors are categorical variables with two levels. For each factor, a dummy variable is created; for example, the dummy variable exercise is 0 if no exercise is done and is 1 if the patient does exercise, and the dummy variable is 0 if patient is on normal diet and is 1 if the patient is on low-fat diet. Besides the two main effects of the two factors of interest, the design matrix X contains a column representing the interaction between exercise and diet. In this example, the interaction column could be formed by combining the two singles columns of the main effects. The model equation is, therefore, given by • y the continuous response describing the

cholesterol level • µ the common effect for the whole popu-

lation • τ 1 the effect of no exercise and τ 2 the

effect of exercise • δ 1 the effect of normal diet and δ 2 the

effect of low-fat diet

LINEAR MODEL • (τ δ) the interaction effects: (τ δ)11 no

exercise—normal diet, (τ δ)12 no exercise—low-fat diet, (τ δ)21 exercise—normal diet and (τ δ)22 exercise—low-fat diet. Three independent factors give rise to three first-order interactions and one secondorder interaction; four independent factors have six first-order interactions, three second-order interactions, and one thirdorder interaction. As the number of interactions increases, it may become extremely difficult to interpret the model outcome. More information about one-way and twoway ANOVA and hypothesis testing could be found in the article on Analysis of Variance ANOVA. 2.3 Analysis of Covariance (ANCOVA) The analysis of covariance (ANCOVA) is a technique that combines elements of regression and variance analysis. It involves fitting a model where some elements are effects corresponding to levels of factors and interactions, in the manner of analysis of variance, and some others are regression-style coefficients. ANCOVA compares regression within several groups. Of main interest is the explanation of the relationship between the response and the quantitative variable within each group. The general model equation is given by y = Xα + Zβ + ε

(5)

where • y is the vector of observations of the

5

2.3.1 Example. The relationship between salt intake and blood pressure may be investigated for male and female patients. ANCOVA can be used to analyze whether the relationship between salt intake and blood pressure holds for both sexes. The model equation is given by • the continuous response y describing the

blood pressure • the design matrix X containing the categorical covariate sex, and • the matrix Z containing the continuous covariate of daily salt intake. ANCOVA allows one to test more precisely whether the treatment (group) means are equal. It can also be used to study the linear relationship between the dependent and the quantitative variable within each group. 3 GENERALIZATIONS OF THE LINEAR MODEL

Generalized Linear Models are an extension of the linear modeling that allows models to be fit to data that follow probability distributions other than the normal distribution, such as the Poisson, Binomial, Multinomial, and so on. Three extensions that have broad range of application (see Fig. 2) are represented here: the loglinear model for Poisson-distributed data, the logistic regression analysis applicable for binomial-distributed data, and the Cox regression for survival data. For a complete discussion, consult the article on Generalized Linear Models.

response variable • X is the matrix of dummy (0,1) variables • α is the parameter vector of the general

mean and the effects corresponding to levels of factors and their interactions • Z contains the values of the covariates (‘‘regression part’’) • β is the parameter vector for the regression-style coefficients of the covariates • ε is the error term and it is assumed that the errors are identical independent normal N(0,σ 2 ) distributed.

3.1 Loglinear Models Loglinear analysis is an extension of the two-way contingency table where the conditional relationship between two categorical variables is analyzed by taking the natural logarithm of the cell frequencies within a contingency table assuming Poissondistributed data. No distinction is made between dependent and independent variables and, therefore, loglinear models can only analyze the association between variables. Loglinear modeling is an analogue to

6

LINEAR MODEL

Generalized Linear Models

Loglinear Model

Logistic Regression

Cox Regression

Poisson distributed data

Binomial distributed data

Survival data

Figure 2. Extensions of the classical Linear Model (Generalized Linear Models)

multiple regression for categorical variables, and it can be applied to involve not only two but also three or more variables corresponding to a multiway contingency analysis. The loglinear model equation for a 2 × 2 × 2 contingency table is given by B C AB ln(Fijk ) = µ + λA i + λj + λk + λij BC ABC + λAC ik + λjk + λijk

(6)

where • ln(F ijk ) is the natural logarithm of the

expected cell frequency of the cases for cell ijk in the 2 × 2 × 2 contingency table where i indexes the first variable A, j the second variable B, and k the third variable C of this three-dimensional table • µ is the overall mean of the natural logarithm of the expected frequencies • λA i is the main effect for variable A • λB j is the main effect for variable B • λC k is the main effect for variable C • λAB ij is the two-way interaction effect for

variables A and B • λAC ik is the two-way interaction effect for variables A and C • λBC jk is the two-way interaction effect for variables B and C • λABC is the three-way interaction effect ijk for variables A, B, and C. 3.1.1 Example. Suppose one is interested in the relationship between heart disease, sex, and body weight. The continuous variable body weight is broken down into two discrete categories: not over weight and over weight. The variables heart disease and sex

are dichotomous. Then, the three-dimensional contingency looks like the results in the table below (where some fictitious numbers have been inserted in the table): Heart Disease Total no yes Body Weight

Sex

Not over weight

Male Female

Total Over weight

Male Female

Total

40 20

30 30

70 50

60

60

12

10 5

65 25

75 30

15

90

105

The basic strategy in loglinear modeling involves fitting models to the observed frequencies in the cross tabulation of categorical variables. The model can then be represented by a set of expected frequencies that may or may not resemble the observed frequencies. The most parsimonious model that fits the data is chosen and effect parameters for the variables and their interactions could be estimated (3). 3.2 Logistic Regression The logistic regression model is used when the response variable y of interest takes in two variables. Possible situations include studies in which subjects are alive or dead or have or do not have a particular characteristic (e.g., a specific disease). Denote the event y = 1 when the subject has the characteristic of interest and y = 0 when the subject does

LINEAR MODEL

not. In simple logistic regression, the subject has a single predictor variable x, which can take any form (continuous, discrete, or dichotomous). The logistic regression model relates then the probability P(y = 1) to the response via P(y = 1) =

exp(β0 + β1 x) 1 + exp(β0 + β1 x)

(7)

This model has a convenient representation in terms of the odds of the event y = 1 as odds(y = 1) =

P(y = 1) = exp(β0 + β1 x). (8) P(y = 0)

which means the log odds is simply the linear function β 0 + β 1 x, where the parameter β 1 is of primary interest. This often-called slope parameter controls the degree of association between the response and the predictor variable.

7

• β = (β 1 , . . ., β k ) is the vector of the k

unknown regression coefficients • Xi = (xi1 , . . ., xik ) is the vector of the

covariates for the patient i. Cox regression allows one to estimate the regression coefficients that best predict the observed survival. More details on estimation procedures are given in the article on Cox Regression. REFERENCES 1. F. Galton, Regression toward mediocrity in hereditary stature. J. Anthropologic. Inst. 1886; 15: 246–263. 2. M. Bland, An Introduction to Medical Statistics. Oxford: Oxford University Press, 1995. 3. D. Knoke and P. J. Burke, Log-Linear Models. Newberry Park, CA: Sage Publications, 1980. 4. D. R. Cox, Regression models and life tables (with discussion). J. Royal Stat. Soc. B 1972; 74: 187–220.

3.3 Cox Regression In many applications of survival analysis of clinical trials, the interest focuses on how covariates may effect the outcome. The proportional hazard regression model of Cox (4) has become the most important model for survival data. The basic endpoint of a patients, say i, is his/her survival Time T i . No specific distribution is assumed for this response variable, instead one considers his/her hazard function hi (t), which describes the instantaneous risk of failure of time t. Formally, hi (t) = lim

t→0

1 P(t ≤ T ≤ t + t|T ≥ t) (9) t

for example, the probability of failure of time t given no failure up to this time point. In its simplest form, but for many practical uses sufficient is a hazard function for a patient i given by hi (t) = h0 (t) exp(β1 xi1 + · · · + βk xik )

(10)

where • h0 (t) is a constant describing the hazard

function for patients with all covariates equal to 0

FURTHER READING A. Agresti, An Introduction to Categorical Data Analysis. New York: John Wiley & Sons, 1996. J. M. Bland and D. G. Altman, Regression towards the mean. Brit. Med. J. 1994; 308: 1499. P. McCullagh and J. A. Nelders, Generalized Linear Models. New York: Chapman and Hall, 1983. S. R. Searle, Linear Models. New York: John Wiley & Sons, 1971.

LOGISTIC REGRESSION

The logit transformation, defined in terms of π (x), is as follows:

STANLEY LEMESHOW




Ohio State University Columbus, OH, USA

g(x) = ln

 π (x) = β0 + β1 x. 1 − π (x)

(2)

DAVID W. HOSMER Jr The importance of this transformation is that g(x) has many of the desirable properties of a linear regression model. The logit, g(x), is linear in its parameters, may be continuous, and may range from −∞ to +∞ depending on the range of x. The second important difference between the linear and logistic regression models concerns the conditional distribution of the outcome variable. In the linear regression model we assume that an observation of the outcome variable may be expressed as y = E(Y|x) + ε. The quantity ε is called the error and expresses an observation’s deviation from the conditional mean. The most common assumption is that ε follows a normal distribution with mean zero and some variance that is constant across levels of the independent variable. It follows that the conditional distribution of the outcome variable given x is normal with mean E(Y|x), and a variance that is constant. This is not the case with a dichotomous outcome variable. In this situation we may express the value of the outcome variable given x as y = π (x) + ε. Here the quantity ε may assume one of two possible values. If y = 1, then ε = 1 − π (x) with probability π (x), and if y = 0, then ε = −π (x) with probability 1 − π (x). Thus, ε has a distribution with mean zero and variance equal to π (x)[1 − π (x)]. That is, the conditional distribution of the outcome variable follows a binomial distribution with probability given by the conditional mean, π (x).

University of Massachusetts, Amherst, MA, USA

The goal of a logistic regression analysis is to find the best fitting and most parsimonious, yet biologically reasonable, model to describe the relationship between an outcome (dependent or response variable) and a set of independent (predictor or explanatory) variables. What distinguishes the logistic regression model from the linear regression model is that the outcome variable in logistic regression is categorical and most usually binary or dichotomous. In any regression problem the key quantity is the mean value of the outcome variable, given the value of the independent variable. This quantity is called the conditional mean and will be expressed as E(Y|x), where Y denotes the outcome variable and x denotes a value of the independent variable. In linear regression we assume that this mean may be expressed as an equation linear in x (or some transformation of x or Y), such as E(Y|x) = β0 + β1 x. This expression implies that it is possible for E(Y|x) to take on any value as x ranges between −∞ and +∞. Many distribution functions have been proposed for use in the analysis of a dichotomous outcome variable. Cox & Snell (2) discuss some of these. There are two primary reasons for choosing the logistic distribution. These are: (i) from a mathematical point of view it is an extremely flexible and easily used function, and (ii) it lends itself to a biologically meaningful interpretation. To simplify notation, let π (x) = E(Y|x) represent the conditional mean of Y given x. The logistic regression model can be expressed as π (x) =

exp(β0 + β1 x) . 1 + exp(β0 + β1 x)

1 FITTING THE LOGISTIC REGRESSION MODEL Suppose we have a sample of n independent observations of the pair (xi , yi ), i = 1, 2, . . . , n, where yi denotes the value of a dichotomous outcome variable and xi is the value of the independent variable for the ith subject. Furthermore, assume that the outcome

(1)

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

LOGISTIC REGRESSION

variable has been coded as 0 or 1 representing the absence or presence of the characteristic, respectively. To fit the logistic regression model (1) to a set of data requires that we estimate the values of β 0 and β 1 , the unknown parameters. In linear regression the method used most often to estimate unknown parameters is least squares. In that method we choose those values of β 0 and β 1 that minimize the sum of squared deviations of the observed values of Y from the predicted values based upon the model. Under the usual assumptions for linear regression the least squares method yields estimators with a number of desirable statistical properties. Unfortunately, when the least squares method is applied to a model with a dichotomous outcome the estimators no longer have these same properties. The general method of estimation that leads to the least squares function under the linear regression model (when the error terms are normally distributed) is maximum likelihood. This is the method used to estimate the logistic regression parameters. In a very general sense the maximum likelihood method yields values for the unknown parameters that maximize the probability of obtaining the observed set of data. To apply this method we must first construct a function called the likelihood function. This function expresses the probability of the observed data as a function of the unknown parameters. The maximum likelihood estimators of these parameters are chosen to be those values that maximize this function. Thus, the resulting estimators are those that agree most closely with the observed data. If Y is coded as 0 or 1, then the expression for π (x) given in (1) provides (for an arbitrary value of β  = (β 0 , β 1 ), the vector of parameters) the conditional probability that Y is equal to 1 given x. This will be denoted Pr(Y = 1|x). It follows that the quantity 1 − π (x) gives the conditional probability that Y is equal to zero given x, Pr(Y = 0|x). Thus, for those pairs (xi , yi ), where yi = 1, the contribution to the likelihood function is π (xi ), and for those pairs where yi = 0, the contribution to the likelihood function is 1 − π (xi ), where the quantity π (xi ) denotes the value of π (x) computed at xi . A convenient way to express the contribution to the likelihood function for

the pair (xi , yi ) is through the term ξ (xi ) = π (xi )yi [1 − π (xi )]1−yi .

(3)

Since the observations are assumed to be independent, the likelihood function is obtained as the product of the terms given in (3) as follows: l(β) =

n 

ξ (xi ).

(4)

i=1

The principle of maximum likelihood states that we use as our estimate of β the value that maximizes the expression in (3). However, it is easier mathematically to work with the log of (3). This expression, the log likelihood, is defined as

L(β) = ln[l(β)]  = {yi ln[π (xi )] + (1 − yi )ln[1 − π (xi )]}. (5) To find the value of β that maximizes L(β) we differentiate L(β) with respect to β 0 and β 1 and set the resulting expressions equal to zero. These equations are as follows: n 

[yi − π (xi )] = 0

(6)

xi [yi − π (xi )] = 0,

(7)

i=1

and

n  i=1

and are called the likelihood equations. In linear regression, the likelihood equations, obtained by differentiating the sum of squared deviations function with respect to β, are linear in the unknown parameters, and thus are easily solved. For logistic regression the expressions in (6) and (7) are nonlinear in β 0 and β 1 , and thus require special methods for their solution. These methods are iterative in nature and have been programmed into available logistic regression software. McCullagh & Nelder (6) discuss the iterative methods used by most programs. In particular, they show that the

LOGISTIC REGRESSION

solution to (6) and (7) may be obtained using a generalized weighted least squares procedure. The value of β given by the solution to (6) and (7) is called the maximum likelihood ˆ Similarly, π(x estimate, denoted as β. ˆ i ) is the maximum likelihood estimate of π (xi ). This quantity provides an estimate of the conditional probability that Y is equal to 1, given that x is equal to xi . As such, it represents the fitted or predicted value for the logistic regression model. An interesting consequence of (6) is that n  i=1

yi =

n 

π(x ˆ i ).

i=1

That is, the sum of the observed values of y is equal to the sum of the predicted (expected) values. After estimating the coefficients, it is standard practice to assess the significance of the variables in the model. This usually involves testing a statistical hypothesis to determine whether the independent variables in the model are ‘‘significantly’’ related to the outcome variable. One approach to testing for the significance of the coefficient of a variable in any model relates to the following question. Does the model that includes the variable in question tell us more about the outcome (or response) variable than does a model that does not include that variable? This question is answered by comparing the observed values of the response variable with those predicted by each of two models; the first with and the second without the variable in question. The mathematical function used to compare the observed and predicted values depends on the particular problem. If the predicted values with the variable in the model are better, or more accurate in some sense, than when the variable is not in the model, then we feel that the variable in question is ‘‘significant’’. It is important to note that we are not considering the question of whether the predicted values are an accurate representation of the observed values in an absolute sense (this would be called goodness of fit). Instead, our question is posed in a relative sense. For the purposes of assessing the significance of an independent variable we compute

3

the value of the following statistic:  likelihood without the variable . likelihood with the variable (8) Under the hypothesis that β 1 is equal to zero, the statistic G will follow a chi-square distribution with one degree of freedom. The calculation of the log likelihood and this generalized likelihood ratio test are standard features of any good logistic regression package. This makes it possible to check for the significance of the addition of new terms to the model as a matter of routine. In the simple case of a single independent variable, we can first fit a model containing only the constant term. We can then fit a model containing the independent variable along with the constant. This gives rise to a new log likelihood. The likelihood ratio test is obtained by multiplying the difference between the log likelihoods of the two models by −2. Another test that is often carried out is the Wald test, which is obtained by comparing the maximum likelihood estimate of the slope parameter, βˆ1 , with an estimate of its standard error. The resulting ratio 

G = −2 ln

W=

βˆ1 ,  βˆ1 ) se(

under the hypothesis that β 1 = 0, follows a standard normal distribution. Standard errors of the estimated parameters are routinely printed out by computer software. Hauck & Donner (3) examined the performance of the Wald test and found that it behaved in an aberrant manner, often failing to reject when the coefficient was significant. They recommended that the likelihood ratio test be used. Jennings (5) has also looked at the adequacy of inferences in logistic regression based on Wald statistics. His conclusions are similar to those of Hauck & Donner. Both the likelihood ratio test, G, and the Wald test, W, require the computation of the maximum likelihood estimate for β 1 . For a single variable this is not a difficult or costly computational task. However, for large data sets with many variables, the iterative computation needed to obtain the maximum likelihood estimates can be considerable.

4

LOGISTIC REGRESSION

The logistic regression model may be used with matched study designs. Fitting conditional logistic regression models requires modifications, which are not discussed here. The reader interested in the conditional logistic regression model may find details in [4, Chapter 7]. 2 THE MULTIPLE LOGISTIC REGRESSION MODEL Consider a collection of p independent variables which will be denoted by the vector x = (x1 , x2 , . . . , xp ). Assume for the moment that each of these variables is at least interval scaled. Let the conditional probability that the outcome is present be denoted by Pr(Y = 1|x) = π (x). Then the logit of the multiple logistic regression model is given by g(x) = β0 + β1 x1 + β2 x2 + · · · + βp xp ,

Assume that we have a sample of n independent observations of the pair (xi , yi ), i = 1, 2, . . . , n. As in the univariate case, fitting the model requires that we obtain estimates of the vector β  = (β 0 , β 1 , . . . , β p ). The method of estimation used in the multivariate case is the same as in the univariate situation, i.e. maximum likelihood. The likelihood function is nearly identical to that given in (4), with the only change being that π (x) is now defined as in (10). There are p + 1 likelihood equations which are obtained by differentiating the log likelihood function with respect to the p + 1 coefficients. The likelihood equations that result may be expressed as follows: n  [yi − π (xi )] = 0

(9)

i=1

and

in which case π (x) =

3 FITTING THE MULTIPLE LOGISTIC REGRESSION MODEL

exp[g(x)] . 1 + exp[g(x)]

(10)

If some of the independent variables are discrete, nominal scaled variables such as race, sex, treatment group, and so forth, then it is inappropriate to include them in the model as if they were interval scaled. In this situation a collection of design variables (or dummy variables) should be used. Most logistic regression software will generate the design variables, and some programs have a choice of several different methods. In general, if a nominal scaled variable has k possible values, then k − 1 design variables will be needed. Suppose, for example, that the jth independent variable, xj has kj levels. The kj − 1 design variables will be denoted as Dju and the coefficients for these design variables will be denoted as β ju , u = 1, 2, . . . , kj − 1. Thus, the logit for a model with p variables and the jth variable being discrete is kj −1

g(x) = β0 + β1 x1 + · · · +

 u=1

βju Dju + βp xp .

n 

xij [yi − π (xi )] = 0,

i=1

for j = 1, 2, . . . , p. As in the univariate model, the solution of the likelihood equations requires special purpose software which may be found in many packaged programs. Let βˆ denote the solution to these equations. Thus, the fitted values for the multiple logistic regression model are π(x ˆ i ), the value of the expression in (13) computed using βˆ and xi . Before proceeding further we present an example that illustrates the formulation of a multiple logistic regression model and the estimation of its coefficients. 4 EXAMPLE To provide an example of fitting a multiple logistic regression model, consider the data for the low birth weight study described in Appendix 1 of Hosmer & Lemeshow (4). The code sheet for the data set is given in Table 1. The goal of this study was to identify risk factors associated with giving birth to a low birth weight baby (weighing less than 2500 g). In this study data were collected on

LOGISTIC REGRESSION

5

Table 1. Code Sheet for the Variables in the Low Birth Weight Data Set Variable

Abbreviation

Identification code Low birth weight (0 = birth weight ≥2500 g, 1 = birth weight 37.94] < 0.0001 (see Table 3). Rejection of the null hypothesis (that all of the coefficients are simultaneously equal to zero) has an interpretation analogous to that in multiple linear regression; we may

6

LOGISTIC REGRESSION

Table 3. Estimated Coefficients for a Multiple Logistic Regression Model Using All Variables From the Low Birth Weight Data Set Number of obs. = 189 χ 2 (9) = 37.94 Prob > χ 2 = 0.0000

Logit estimates

Log likelihood = −98.36 Variable

Coeff.

Std. error

AGE LWT SMOKE HT UI RACE 1 RACE 2 FTV01 PTL01 cons

−0.035 −0.015 0.815 1.824 0.702 1.202 0.773 0.121 1.237 0.545

0.039 0.007 0.420 0.705 0.465 0.534 0.460 0.376 0.466 1.266

conclude that at least one, and perhaps all p coefficients are different from zero. Before concluding that any or all of the coefficients are nonzero, we may wish to look at the univariate Wald test statistics, se(βˆj ). These are given in the fourth Wj = βˆj / column (labeled z) in Table 3. Under the hypothesis that an individual coefficient is zero, these statistics will follow the standard normal distribution. Thus, the value of these statistics may give us an indication of which of the variables in the model may or may not be significant. If we use a critical value of 2, which leads to an approximate level of significance (two-tailed) of 0.05, then we would conclude that the variables LWT, SMOKE, HT, PTL01 and possibly RACE are significant, while AGE, UI, and FTV01 are not significant. Considering that the overall goal is to obtain the best fitting model while minimizing the number of parameters, the next logical step is to fit a reduced model, containing only those variables thought to be significant, and compare it with the full model containing all the variables. The results of fitting the reduced model are given in Table 4. The difference between the two models is the exclusion of the variables AGE, UI, and FTV01 from the full model. The likelihood ratio test comparing these two models is obtained using the definition of G given in (8). It has a distribution that is chi-square

z

Pr|z|

−0.920 −2.114 1.939 2.586 1.511 2.253 1.681 0.323 2.654 0.430

0.357 0.035 0.053 0.010 0.131 0.024 0.093 0.746 0.008 0.667

[95% conf. interval] −0.111 −0.029 −0.009 0.441 −0.208 0.156 −0.128 −0.615 0.323 −1.937

0.040 −0.001 1.639 3.206 1.613 2.248 1.674 0.858 2.148 3.027

with three degrees of freedom under the hypothesis that the coefficients for the variables excluded are equal to zero. The value of the test statistic comparing the models in Tables 3 and 4 is G = −2[(−100.24) − (−98.36)] = 3.76 which, with three degrees of freedom, has a P value of P[χ 2 (3) > 3.76] = 0.2886. Since the P value is large, exceeding 0.05, we conclude that the reduced model is as good as the full model. Thus there is no advantage to including AGE, UI, and FTV01 in the model. However, we must not base our models entirely on tests of statistical significance. Numerous other considerations should influence our decision to include or exclude variables from a model.

6 INTERPRETATION OF THE COEFFICIENTS OF THE LOGISTIC REGRESSION MODEL After fitting a model the emphasis shifts from the computation and assessment of significance of estimated coefficients to interpretation of their values. The interpretation of any fitted model requires that we can draw practical inferences from the estimated coefficients in the model. The question addressed is: What do the estimated coefficients in the model tell us about the research questions that motivated the study? For most models this involves the estimated coefficients for the independent variables in the model.

LOGISTIC REGRESSION

7

Table 4. Estimated Coefficients for a Multiple Logistic Regression Model Using the Variables LWT, SMOKE, HT, PTL01 and RACE From the Low Birth Weight Data Set Number of obs. = 189 χ 2 (6) = 34.19 Prob > χ 2 = 0.0000

Logit estimates

Log likelihood = 100.24 Variable

Coeff.

Std. error

LWT SMOKE HT RACE 1 RACE 2 PTL01 cons

−0.017 0.876 1.767 1.264 0.864 1.231 0.095

0.007 0.401 0.708 0.529 0.435 0.446 0.957

The estimated coefficients for the independent variables represent the slope or rate of change of a function of the dependent variable per unit of change in the independent variable. Thus, interpretation involves two issues: (i) determining the functional relationship between the dependent variable and the independent variable, and (ii) appropriately defining the unit of change for the independent variable. For a linear regression model we recall that the slope coefficient, β 1 , is equal to the difference between the value of the dependent variable at x + 1 and the value of the dependent variable at x, for any value of x. In the logistic regression model β 1 = g(x + 1) − g(x). That is, the slope coefficient represents the change in the logit for a change of one unit in the independent variable x. Proper interpretation of the coefficient in a logistic regression model depends on being able to place meaning on the difference between two logits. Consider the interpretation of the coefficients for a univariate logistic regression model for each of the possible measurement scales of the independent variable. 7 DICHOTOMOUS INDEPENDENT VARIABLE Assume that x is coded as either 0 or 1. Under this model there are two values of π (x) and equivalently two values of 1 − π (x). These values may be conveniently displayed in a 2 × 2 table, as shown in Table 5.

z

Pr|z|

−2.407 2.186 2.495 2.387 1.986 2.759 0.099

0.016 0.029 0.013 0.017 0.047 0.006 0.921

[95% conf. interval] −0.030 0.091 0.379 0.226 0.011 0.357 −1.781

−0.003 1.661 3.156 2.301 1.717 2.106 1.970

The odds of the outcome being present among individuals with x = 1 is defined as π (1)/[1 − π (1)]. Similarly, the odds of the outcome being present among individuals with x = 0 is defined as π (0)/[1 − π (0)]. The odds ratio, denoted by ψ, is defined as the ratio of the odds for x = 1 to the odds for x = 0, and is given by ψ=

π (1)/[1 − π (1)] . π (0)/[1 − π (0)]

(11)

The log of the odds ratio, termed log odds ratio, or log odds, is  π (1)/[1 − π (1)] = g(1) − g(0), ln(ψ) = ln π (0)/[1 − π (0)] which is the logit difference, where the log of the odds is called the logit and, in this example, these are  π (1) g(1) = ln 1 − π (1) 

and g(0) = ln

π (0) . 1 − π (0)

Using the expressions for the logistic regression model shown in Table 5 the odds ratio is    exp(β0 + β1 ) 1 1 + exp(β0 + β1 ) 1 + exp(β0 )   ψ =  1 exp(β0 ) 1 + exp(β0 ) 1 + exp(β0 + β1 )

8

LOGISTIC REGRESSION

Table 5. Values of the Logistic Regression Model When the Independent Variable is Dichotomous Independent variable X x=1 y=1 Outcome variable

y y=0

1 − π (1) =

Total

=

1 1 + exp(β0 + β1 ) 1.0

exp(β0 + β1 ) = exp(β1 ). exp(β0 )

Hence, for logistic regression with a dichotomous independent variable ψ = exp(β1 ),

x=0

exp(β0 + β1 ) π (1) = 1 + exp(β0 + β1 )

(12)

and the logit difference, or log odds, is ln(ψ) = ln[exp(β1 )] = β1 . This fact concerning the interpretability of the coefficients is the fundamental reason why logistic regression has proven such a powerful analytic tool for epidemiologic research. A confidence interval (CI) estimate for the odds ratio is obtained by first calculating the endpoints of a confidence interval for the coefficient β 1 , and then exponentiating these values. In general, the endpoints are given by

  βˆ1 ) . exp βˆ1 ± z1−α/2 × se( Because of the importance of the odds ratio as a measure of association, point and interval estimates are often found in additional columns in tables presenting the results of a logistic regression analysis. In the previous discussion we noted that the estimate of the odds ratio was ψˆ = exp(βˆ1 ). This is correct when the independent variable has been coded as 0 or 1. This type of coding is called ‘‘reference cell’’ coding. Other coding could be used. For example, the variable may be coded as −1 or +1. This type of coding is termed ‘‘deviation from means’’ coding. Evaluation of the

π (0) =

expβ0 1 + expβ0

1 − π (0) =

1 1 + expβ0

1.0

logit difference shows that the odds ratio is calculated as ψˆ = exp(2βˆ1 ) and if an investigator were simply to exponentiate the coefficient from the computer output of a logistic regression analysis, the wrong estimate of the odds ratio would be obtained. Close attention should be paid to the method used to code design variables. The method of coding also influences the calculation of the endpoints of the confidence interval. With deviation from means coding, the estimated standard error needed for con βˆ1 ), which fidence interval estimation is se(2  βˆ1 ). Thus the endpoints of the confiis 2 × se( dence interval are 

 βˆ1 ) . exp 2βˆ1 + z1−α/2 × 2 × se( In summary, for a dichotomous variable the parameter of interest is the odds ratio. An estimate of this parameter may be obtained from the estimated logistic regression coefficient, regardless of how the variable is coded or scaled. This relationship between the logistic regression coefficient and the odds ratio provides the foundation for our interpretation of all logistic regression results. 8 POLYTOMOUS INDEPENDENT VARIABLE Suppose that instead of two categories the independent variable has k > 2 distinct values. For example, we may have variables that denote the county of residence within a state, the clinic used for primary health care within a city, or race. Each of these variables has a fixed number of discrete outcomes and the scale of measurement is nominal.

LOGISTIC REGRESSION

Suppose that in a study of coronary heart disease (CHD) the variable RACE is coded at four levels, and that the cross-classification of RACE by CHD status yields the data presented in Table 6. These data are hypothetical and have been formulated for ease of computation. The extension to a situation where the variable has more than four levels is not conceptually different, so all the examples in this section use k = 4. At the bottom of Table 6 the odds ratio is given for each race, using white as the reference group. For example, for hispanic the estimated odds ratio is (15 × 20)/(5 × 10) = 6.0. The log of the odds ratios are given in the last row of Table 6. This display is typical of what is found in the literature when there is a perceived referent group to which the other groups are to be compared. These same estimates of the odds ratio may be obtained from a logistic regression program with an appropriate choice of design variables. The method for specifying the design variables involves setting all of them equal to zero for the reference group, and then setting a single design variable equal to one for each of the other groups. This is illustrated in Table 7. Use of any logistic regression program with design variables coded as shown in Table 7 yields the estimated logistic regression coefficients given in Table 8. A comparison of the estimated coefficients in Table 8 with the log odds in Table 6 ˆ shows that ln[ψ(black, white)] = βˆ11 = 2.079, ˆ ln[ψ(hispanic, white)] = βˆ12 = 1.792, and ˆ ln[ψ(other, white)] = βˆ13 = 1.386. In the univariate case the estimates of the standard errors found in the logistic regression output are identical to the estimates obtained using the cell frequencies from the contingency table. For example,

9

the estimated standard error of the estimated coefficient for design variable (1), βˆ11 , is 0.6325 = (1/5 + 1/20 + 1/20 + 1/10)1/2 . A derivation of this result appears in Bishop et al. (1). Confidence limits for odds ratios may be obtained as follows:  βˆij ). βˆij ± z1−α/2 × se( The corresponding limits for the odds ratio are obtained by exponentiating these limits as follows:  βˆij )]. exp[βˆij ± z1−α/2 × se(

9

CONTINUOUS INDEPENDENT VARIABLE

When a logistic regression model contains a continuous independent variable, interpretation of the estimated coefficient depends on how it is entered into the model and the particular units of the variable. For purposes of developing the method to interpret the coefficient for a continuous variable, we assume that the logit is linear in the variable. Under the assumption that the logit is linear in the continuous covariate, x, the

Table 7. Specification of the Design Variables for RACE Using White as the Reference Group RACE (code)

D1

White (1) Black (2) Hispanic (3) Other (4)

0 1 0 0

Design variables D2 0 0 1 0

D3 0 0 0 1

Table 6. Cross-Classification of Hypothetical Data on RACE and CHD Status for 100 Subjects CHD status

White

Black

Hispanic

Other

Total

Present Absent Total

5 20 25

20 10 30

15 10 25

10 10 20

50 50 100

ˆ Odds ratio (ψ) 95% CI ˆ ln(ψ)

1.0

8.0 (2.3, 27.6) 2.08

6.0 (1.7, 21.3) 1.79

4.0 (1.1, 14.9) 1.39

0.0

10

LOGISTIC REGRESSION

Table 8. Results of Fitting the Logistic Regression Model to the Data in Table 6 Using the Design Variables in Table 7 Variable

Coeff.

RACE 1 RACE 2 RACE 3 cons

2.079 1.792 1.386 −1.386

Variable

Odds ratio

RACE 1 RACE 2 RACE 3

8 6 4

Std. error 0.632 0.645 0.671 0.500

equation for the logit is g(x) = β 0 + β 1 x. It follows that the slope coefficient, β 1 , gives the change in the log odds for an increase of ‘‘l’’ unit in x, i.e. β 1 = g(x + 1) − g(x) for any value of x. Most often the value of ‘‘1’’ will not be biologically very interesting. For example, an increase of 1 year in age or of 1 mmHg in systolic blood pressure may be too small to be considered important. A change of 10 years or 10 mmHg might be considered more useful. However, if the range of x is from zero to one, as might be the case for some created index, then a change of 1 is too large and a change of 0.01 may be more realistic. Hence, to provide a useful interpretation for continuous scaled covariates we need to develop a method for point and interval estimation for an arbitrary change of c units in the covariate. The log odds for a change of c units in x is obtained from the logit difference g(x + c) − g(x) = cβ 1 and the associated odds ratio is obtained by exponentiating this logit difference, ψ(c) = ψ(x + c, x) = exp(cβ 1 ). An estimate may be obtained by replacing β 1 with its maximum likelihood estimate, βˆ1 . An estimate of the standard error needed for confidence interval estimation is obtained by multiplying the estimated standard error of βˆ1 by c. Hence the endpoints of the 100(1 − α)% CI estimate of ψ(c) are  βˆ1 )]. exp[cβˆ1 ± z1−α/2 cse( Since both the point estimate and endpoints of the confidence interval depend on the choice of c, the particular value of c should be clearly specified in all tables and calculations.

z

P|z|

3.288 2.776 2.067 −2.773

0.001 0.006 0.039 0.006

[95% conf. interval] 0.840 0.527 0.072 −2.367

3.319 3.057 2.701 −0.406

[95% conf. interval] 2.32 1.69 1.07

27.63 21.26 14.90

10 MULTIVARIATE CASE Often logistic regression analysis is used to adjust statistically the estimated effects of each variable in the model for differences in the distributions of and associations among the other independent variables. Applying this concept to a multiple logistic regression model, we may surmise that each estimated coefficient provides an estimate of the log odds adjusting for all other variables included in the model. The term confounder is used by epidemiologists to describe a covariate that is associated with both the outcome variable of interest and a primary independent variable or risk factor. When both associations are present the relationship between the risk factor and the outcome variable is said to be confounded. The procedure for adjusting for confounding is appropriate when there is no interaction. If the association between the covariate and an outcome variable is the same within each level of the risk factor, then there is no interaction between the covariate and the risk factor. When interaction is present, the association between the risk factor and the outcome variable differs, or depends in some way on the level of the covariate. That is, the covariate modifies the effect of the risk factor. Epidemiologists use the term effect modifier to describe a variable that interacts with a risk factor. The simplest and most commonly used model for including interaction is one in which the logit is also linear in the confounder for the second group, but with a

LOGISTIC REGRESSION

different slope. Alternative models can be formulated which would allow for other than a linear relationship between the logit and the variables in the model within each group. In any model, interaction is incorporated by the inclusion of appropriate higher order terms. An important step in the process of modeling a set of data is to determine whether or not there is evidence of interaction in the data. Tables 9 and 10 present the results of fitting a series of logistic regression models to two different sets of hypothetical data. The variables in each of the data sets are the same: SEX, AGE, and CHD. In addition to the estimated coefficients, the log likelihood for each model and minus twice the change (deviance) is given. Recall that minus twice the change in the log likelihood may be used to test for the significance of coefficients for variables added to the model. An interaction is added to the model by creating a variable that is equal to the product of the value of the sex and the value of age. Examining the results in Table 9 we see that the estimated coefficient for the variable SEX changed from 1.535 in model 1 to 0.979 when AGE was added in model 2. Hence, there is clear evidence of a confounding effect owing to age. When the interaction term ‘‘SEX × AGE’’ is added in model 3 we see that the change in the deviance is only 0.52 which, when compared with the chi-square distribution with one degree of freedom, yields a P value of 0.47, which clearly is not significant. Note that the coefficient for sex changed from 0.979 to 0.481. This is not surprising since the inclusion of an interaction term, especially when it involves a continuous variable, will usually produce fairly marked changes in the estimated coefficients of dichotomous variables involved in the interaction. Thus, when an interaction term is present in the model we cannot assess confounding via the change in a coefficient. For these data we would prefer to use model 2 which suggests that age is a confounder but not an effect modifier. The results in Table 10 show evidence of both confounding and interaction due to age. Comparing model 1 with model 2 we see that the coefficient for sex changes from 2.505 to 1.734. When the age by sex interaction is added to the model we see that the deviance

11

is 4.06, which yields a P value of 0.04. Since the deviance is significant, we prefer model 3 over model 2, and should regard age as both a confounder and an effect modifier. The net result is that any estimate of the odds ratio for sex should be made with respect to a specific age. Hence, we see that determining if a covariate, X, is an effect modifier and/or a confounder involves several issues. Determining effect modification status involves the parametric structure of the logit, while determination of confounder status involves two things. First, the covariate must be associated with the outcome variable. This implies that the logit must have a nonzero slope in the covariate. Secondly, the covariate must be associated with the risk factor. In our example this might be characterized by having a difference in the mean age for males and females. However, the association may be more complex than a simple difference in means. The essence is that we have incomparability in our risk factor groups. This incomparability must be accounted for in the model if we are to obtain a correct, unconfounded estimate of effect for the risk factor. In practice, the confounder status of a covariate is ascertained by comparing the estimated coefficient for the risk factor variable from models containing and not containing the covariate. Any ‘‘biologically important’’ change in the estimated coefficient for the risk factor would dictate that the covariate is a confounder and should be included in the model, regardless of the statistical significance of the estimated coefficient for the covariate. On the other hand, a covariate is an effect modifier only when the interaction term added to the model is both biologically meaningful and statistically significant. When a covariate is an effect modifier, its status as a confounder is of secondary importance since the estimate of the effect of the risk factor depends on the specific value of the covariate. The concepts of adjustment, confounding, interaction, and effect modification may be extended to cover the situations involving any number of variables on any measurement scale(s). The principles for identification and inclusion of confounder and interaction variables into the model are the same regardless

12

LOGISTIC REGRESSION

Table 9. Estimated Logistic Regression Coefficients, Log Likelihood, and the Likelihood Ratio Test Statistic (G) for an Example Showing Evidence of Confounding But no Interaction Model 1 2 3

Constant

SEX

AGE

−1.046 −7.142 −6.103

1.535 0.979 0.481

0.167 0.139

SEX × AGE

Log likelihood

0.059

−61.86 −49.59 −49.33

G 24.54 0.52

Table 10. Estimated Logistic Regression Coefficients, Log Likelihood, and the Likelihood Ratio Test Statistic (G) for an Example Showing Evidence of Confounding and Interaction Model 1 2 3

Constant

SEX

AGE

−0.847 −6.194 −3.105

2.505 1.734 0.047

0.147 0.629

of the number of variables and their measurement scales. Much of this article has been abstracted from (4). Readers wanting more detail on any topic should consult this reference. REFERENCES 1. Bishop, Y. M. M., Fienberg, S. E. & Holland, P. (1975). Discrete Multivariate Analysis: Theory and Practice. MIT Press, Boston. 2. Cox, D. R. & Snell, E. J. (1989). The Analysis of Binary Data, 2nd Ed. Chapman & Hall, London.

SEX × AGE

Log likelihood

G

0.206

−52.52 −46.79 −44.76

11.46 4.06

3. Hauck, W. W. & Donner, A. (1977). Wald’s Test as applied to hypotheses in logit analysis, Journal of the American Statistical Association 72, 851–853. 4. Hosmer, D. & Lemeshow, S. (2000). Applied Logistic Regression. 2nd Ed. Wiley, New York. 5. Jennings, D. E. (1986). Judging inference adequacy in logistic regression, Journal of the American Statistical Association 81, 471–476. 6. McCullagh, P. & Nelder, J. A. (1983). Generalized Linear Models. Chapman & Hall, London.

LOG-RANK TEST

The null hypothesis can also be stated using the hazard function. The null hypothesis is equivalent to comparing the hazard rates

ALAN HOPKINS Theravance Inc., South San Francisco, California

H0 : λ1 (t) = λ2 (t) = . . . = λK (t), for t ≥ 0 (1) The alternative hypothesis usually of interest is that the survival function for one group is stochastically larger or smaller than the survival functions for the other groups. H a : Sk ≥ Sk (t), or Sk (t) ≤ Sk (t), for some k, k with strict inequality for some t.

Calculation of the log-rank test is described, and an example is provided. Conditions under which the test is valid are discussed and testing of assumptions is addressed. Generalizations of the log-rank test are described including its relationship to the Cox regression model. Sample size calculation for the log-rank test is discussed, and computing software is described. Suppose T is a continuous, non-negative random variable representing survival times from point of randomization. The distribution of censored event times can be estimated by the survival function S(t) The survival function estimates the probability of surviving to time t so S(t) = Pr(T > t). The hazard function is directly related to the survival function through the relationship λ(t) = − d log S(t)/dt. The hazard rate is the conditional probability of death in a small interval [t, t + dt) given survival to the beginning of the interval. With censored data, we cannot observe events on all subjects, so we only observe a censoring time. Thus, for each subject, we have data on observation time, an indicator whether an event was observed at the last observation time, and treatment group. With this information, we can compare hazard functions in intervals containing observed death times and calculate a global test statistic for comparing the observed survival distributions among several groups. The log-rank test was originally proposed by Mantel (1), and it is equivalent under certain circumstances to the Cox (2) regression model.

0.0.2 Assumptions. The log-rank test is nonparametric in the sense that no underlying probability distribution is assumed for the survival function. The log-rank test assumes the censoring process is unrelated to survival times or to the treatment groups themselves (independent censoring) and that the survival times are from the same distribution for subjects recruited early or late in the clinical trial (stationarity). Observations are assumed to be independent. Special methods are required if recurrent events are observed on a single individual (e.g., multiple infections.) 0.0.3 Inference. Let t1 < t2 < · · · < tL represent the ordered distinct failure times in the combined groups of observations. At time tt , dij events are observed in the jth sample out of Rij individuals at risk just prior to ti . K
Here di = dij represents the total number j=1

of deaths at ti and Ri =

K


Rij the number of

j=1

subjects at risk at ti . We can represent the data at time ti as shown in Table 1. The test of hypothesis (1) is based on weighted deviations of the estimated hazard functions for each group from the overall estimated hazard rate among all data combined. If the null hypothesis is true, then an estimator of the expected hazard rate in the jth population under H0 is the pooled sample estimator of the hazard rate di /Ri . An estimate of the hazard rate for the jth sample

0.0.1 Hypothesis. Suppose we have K groups and we denote the survival functions associated with each group as S1 (t), . . . , SK (t). The null hypothesis can be expressed as H0 : S1 (t) = S2 (t) = . . . = SK (t), for t ≥ 0

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

LOG-RANK TEST Table 1. Layout and Notation for the K-Group Log-Rank Test Time ti

Group 1

Deaths Survivors

Group 2

...

di1

di2

...

diK

di

Ri2 − di2

...

RiK − diK

Ri − di

Ri1

Ri2

...

RiK

Ri

is dij /Rij . To compare survival distributions, we take a weighted average of deviations across all failure times. The test is based on statistics of the form: L  i=1

   di W(ti ) dij − Rij , j = 1, . . . , K Ri

where W(t) is a positive weight function. vj (t) is the sum over all event times of the difference in observed and conditionally expected events for group j. This quantity has a product hypermultinomial distribution with covariance matrix: Vjg =

At Risk

Ri1 − di1

Total

vj =

Group K

   L  Rig Rij Ri − di W(ti )2 di δjg − Ri Ri Ri − 1 i=1

j, g = 1, . . . , K where δ jg = 1 when j = g and 0 otherwise. Let v = (v1 , v2 , . . . , vK )T . Then a test statistic for hypothesis (1) is the quadratic form X 2 = vT V− v

(2)

where V− is a generalized inverse. The components of v are linearly dependent and sum to zero so the variance-covariance matrix has maximum rank K − 1. The overall test statistic can be constructed using any K − 1 components of v and corresponding elements of the variance-covariance matrix. Therefore, if the last row and column of V is deleted to give VK−1 and vK−1 = (v1 , v2 , . . . , vK−1 )T , then the overall log-rank test statistic is −1 vK−1 X 2 = vTK−1 VK−1

(3)

where VK−1 −1 is an ordinary inverse. The distribution of the weighted log-rank statistic is chi-squared with K − 1 degrees of freedom. Using W(ti ) = 1 gives the widely

used log-rank test. Alternative weights will be discussed in a later section. Since the log-rank statistic as presented here sums across multiple failure times, the tables used are not independent, which precludes use of standard methods to derive the asymptotic distribution of the statistic. The asymptotic properties of the log-rank test were rigorously developed using counting process techniques. For details of this approach, see Fleming and Harrington (3) or Kalbfleisch and Prentice (4). 0.0.4 A Special Case (K = 2) and W (t i ) = 1. Often a clinical trial consists of only two treatment groups. In this case, the computations are simplified. We may write the two-sample log-rank test as    L
di di1 − Ri1 Ri i=1 ZLR =     L R
Ri1 Ri − di i1 di 1− Ri Ri − 1 i=1 Ri which has approximately a standard normal distribution under the null hypothesis for large samples. 0.0.5 Relationship of the Log-rank Statistic to the Cox Regression Model. The log-rank test is closely related to the Cox proportional hazards regression model. Let zT = (z1 , . . . , zp ) represent p covariates on a given subject. In the case of the log-rank test, z would be indicator variables for treatment groups. The proportional hazards regression model is λ(t|z) = λ0 (t)exp(β T z), where λ0 (t) is the baseline hazard corresponding to zT = (0, . . . , 0) and β is a vector of regression coefficients. The likelihood for the Cox regression model is simply L

L(β) = 
i=1

exp(β T zi ) j∈Di

exp(β T zj )

(4)

LOG-RANK TEST

where Di is the set of subjects at risk at time ti . The efficient score for Equation (4) is given by U(β) = ∂/∂β log L(β), and its covariance by the inverse of I(β) = − ∂ 2 /∂β 2 log L(β). Then the score statistic is U’(0)I−1 (0)U(0), which has a chi-squared distribution with p − 1 degrees of freedom. This statistic is equivalent to the log-rank test when there are no tied survival times. 0.0.6 Power. The log-rank test is most powerful when the survival curves are proportional. This occurs when one survival function is consistently greater than the other over the study period. The log-rank test is the most powerful nonparametric test to detect proportional hazards alternatives. If the hazard functions cross, then there may be very little power to detect differences between the survival curves. One easy way to assess the proportionality assumption is to plot the Kaplan-Meier survival curves. If the survival curves cross, then the proportionality assumption is not met. Alternatively, a plot the estimated survival curves on a log(-log) scale gives a constant vertical shift of the two curves by an amount equal to the log of the hazards if the hazards are proportional. A more rigorous approach to checking the proportionality assumption is to use a statistical test based on a Cox regression model. Proportionality fails when there is an interaction between treatments and time. Introduction of a time-dependent interaction can be used to test formally for nonproportional hazards with the Cox regression model. Therneau and Grambsch (5) describe using residuals from Cox regressions to identify deviations from the proportional hazards assumption. 1 EXAMPLE: DISEASE-FREE SURVIVAL FOR ACUTE MYELOGENOUS LEUKEMIA AFTER BONE MARROW TRANSPLANTATION Klein and Moeschberger (6) provide a dataset containing 101 patients who received bone marrow transplantation after chemotherapy for acute myelogenous leukemia. Transplants were either allogenic (from the patient’s sibling) or autologous (from the patients own marrow harvested prior to chemotherapy).

3

The event time was based on relapse or death, whichever occurred first. The R software (7) package KMsurv contains this dataset called alloauto. Each patient in the dataset has a sequence number, a leukemia-free survival time (in months), an indicator for censoring (0 = yes, 1 = no), and an indicator for type of bone marrow transplant (1 = allogenic and 2 = autologous). There are 101 subjects in the dataset and 50 leukemia relapses. Of the 101 patients, 50 had allogenic transplants and 51 had autologous transplantation. An R script for this example is in Table 2. survfit calculates the Kaplan-Meier curve. The plot command gives the Kaplan-Meier curves shown in Fig. 1. The allogenic transplant survival is initially higher than the autologous transplants. This trend reverses itself as the survival functions cross at about 12 months casting doubt on the proportional hazards assumption. Vertical dashes on the survival functions represent censored observations. Finally, the survdiff command calculates the log-rank test shown in Table 3. Although there is separation of the two survival curves late in the time axis, the log-rank test does not yield a P < 0.05. Differences in the survival functions summed over time decrease the magnitude of the logrank statistic when the survival functions cross. The sum of the quantities (O-E)ˆ2/E in Table 3 is a conservative approximation to the actual log-rank chi-squared statistic and is produced in the output for information purposes only. 1.0.7 The Stratified Log-Rank Test. Sometimes one may know that survival is not proportional among all subjects but is related to a nuisance factor. Heterogeneous populations can sometimes be stratified into homogeneous groups for analysis purposes eliminating the nuisance source of variation. Stratified analysis can be applied to the log-rank test. This process is appropriate for situations where the proportional hazards assumption breaks down in known subgroups. For example, the hazard rate may be different for a disease outcome depending on disease burden at baseline. In that case, it may be possible to define strata within which the proportional hazards assumption is more

4

LOG-RANK TEST Table 2. R Script for Kaplan-Meier Plot and Log-Rank Test

0.8 0.6 0.4 0.2

allogenic autologous

0.0

Leukemia-Free Survival Probability

1.0

library(survival) data(alloauto, package = ‘‘KMsurv’’) my.fit 0 with power 1 − β requires Pr(U ≥ cα |θ = 0) =α and Pr(U ≥ cα |θ =θ1 ) = 1 − β. This determines the information content or the information horizon of the study as If ixed =

2
zα + zβ θ12

θˆ − θ a
 ∼ N (0, 1) , ˆ Var θ

1 TWO PARADIGMS: DURATION VERSUS INFORMATION

(1)

and the critical value of the test as 
zα + zβ zα , cα = θ1

According to Lan and DeMets (1), there are two paradigms for the design of clinical trials with time-to-event data: duration trials and information trials. They are distinguished

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

MAXIMUM DURATION AND INFORMATION TRIALS

by the way the end of the study is defined. In the former, the study is concluded when data are accumulated over a fixed duration of follow-up on a specified number of subjects, a sample size in a traditional sense. In the latter, a study is concluded when a prespecified amount of statistical information has been obtained. For example, with the logrank test for comparison of time-toevent data in chronic disease clinical trials, the operating characteristics of the study do not depend directly on the number of subjects enrolled in the study, which is a function of the duration of the accrual period in calendar time and the enrollment rate of subjects into the study, but rather depend on the number of events of interest, which is directly proportional to the statistical information. Thus, the study design often specifies the duration of the accrual and follow-up periods to ensure that a necessary number of events are observed during the study. Either the accural duration or the follow-up duration is fixed, and the necessary duration of the other period is determined so that the required number of events are ultimately observed during the study. Determination of the study duration for a fixed-sample study has been investigated by Bernstein and Lagakos (2) and Rubinstein et al. (3). With the duration trial, the information of the test statistic at study closure is random, and there is no guarantee that the required amount of statistical information specified in equation 1 will be obtained if the duration design is strictly adhered to in the analysis. With the information trial, however, the statistical information specified for the design in equation 1 is obtained exactly, and as a result the operating characteristic of the statistical test can be maintained exactly as specified in the design. The calendar time of study closure will be random. 2 SEQUENTIAL STUDIES: MAXIMUM DURATION VERSUS INFORMATION TRIALS If it is desirable to monitor the data periodically during the course of a study, group sequential designs or designs based on triangular tests can be used. Group sequential methods such as by Pocock (4) and O’Brien

and Fleming (5) were developed to maintain the type I error probability at a desired overall significance level despite repeated significance tests. Both methods assume that the number of interim analyses is specified in advance and that the interim analyses are performed after equal increments of statistical information. For monitoring of time-toevent data, one needs the flexibility because these two assumptions are often not met. This flexibility can be achieved by using the error spending function introduced by Lan and DeMets (6). When computed at calendar time s, the efficient score is a

U (s) ∼ N (θ I (s) , I (s)) , as in the fixed-sample study. With the logrank test, which is the efficient score under the proportional hazards model, θ is the log hazard ratio of control λc to experimental λe ; that is, θ = log(λc /λe ), and I(s) is the asymptotic variance of the logrank statistic. The asymptotic variance I(s) is closely related to
the  expected number of events ε(s) by I (s  σz2 ε (s), where σz2 is the variance of treatment indicator Z. As will be shown later, the expression for ε(s) can be derived based on some parametric assumptions regarding the length of the accrual period in calendar time, the enrollment rate, and the distribution for time to event. Assume that patient accrual is uniform during the accrual period (0, sa ) with a constant accrual rate A, the average number of patients accrued per unit time; that allocation to treatment is by simple randomization, with possibly unequal allocation between two treatments, control (c) and experimental (e); that events occur with constant hazard rates λv ,v = c,e; and that random censoring occurs with common constant hazard rate ν. Finally, assume, at least tentatively, that accumulating data will be analyzed after equal increments of information for a maximum of K times. When designing a group sequential study, it is reasonable to assume a prespecified maximum number of analyses K at equal increments of statistical information. With a group sequential design, it is known that the information horizon of the study has to be

MAXIMUM DURATION AND INFORMATION TRIALS

inflated from that for the fixed-sample study as compensation for the possibility of early stopping for treatment difference. Given K,α, and β, and a group sequential design, the amount of required inflation in sample size is called the inflation factor F by Kim et al. (7). Given the information horizon for the corresponding fixed-sample study determined by equation 1, the maximum information, (i.e., the information content for the group sequential study), is determined by Imax = If ixed × F.

(2)

Given the maximum information for the group sequential design, the necessary maximum expected number of events to be observed by the end of study at calendar time sK (i.e., at the last analysis K), is determined by εmax  Imax /σz2 . Then the length of study sK = sa + sf is determined to satisfy ε(1) (sK ) = (1 − µz ) εe (sK ) + µz εc (sK ) = εmax , where µz is the mean of treatment indicator Z for the experimental treatment and εv (s) is the expected number of events by time s when all patients are given treatment v. In other word, the maximum duration of the study is −1 determined by sK = ε(1) (εmax ) subject to εmax −1 ≤ sa ≤ ε(1) (εmax ) . A These inequalities ensure that the accrual duration is long enough— but no longer than necessary— for the required maximum number of events. The expected number of events εv (s) can be evaluated by double integration with respect to the density function for time to event and the uniform density function for patient entry. For example, under exponential time with hazard rates λv , v = c, e, and exponential random censoring with common constant hazard rate ν, then the expected number of events by time s if all the patients in the study are given treatment v are εv (s) = A

λv λv

  exp{−λv (s − sa )+ } − exp(−λv s) s ∧ sa − , λv

3

where λv  = λv + ν, s∧&sa is the smaller of s and sa , and x+ = x if x is positive and 0 otherwise.Therole of exponential distribution is simply to provide a calculation for the expected number of events. The above formula can be generalized to other time to event distributions, and the arguments extend naturally to other parametric proportional hazards models after suitable transformation on the time scale. As noted above, once the study duration is fixed, the maximum expected number of events by the end of the trial can be estimated as ε(1) (sa + sf ). Hence, one may choose to fix the trial duration, sa + sf , or, equivalently, the total number of events to be observed. Although interim analyses are scheduled at regular intervals in calendar times, they depend on the information. By analogy with the fixed-sample study, there are also two paradigms for design of group sequential trials (1): the maximum duration trial in which the maximum duration of the study is fixed, or the maximum information trial in which the maximum information of the study is fixed. As such, the two designs again differ in how the end of the study is defined. Design procedure for maximum duration trials with time data has been investigated by Kim and Tsiatis (8), and that for maximum information trials has been proposed by Kim et al. (9). A maximum duration design specifies the end of the study in terms of the study duration, and a maximum information design specifies the end of the study in terms of the maximum information. In a maximum duration trial, the study is concluded either due to early stopping for treatment difference or when the follow-up reaches the prespecified calendar time. In a maximum information trial, the study is concluded either due to early stopping for treatment difference or when a prespecified maximum information is reached. Sequential clinical trials with time-toevent data are often designed as maximum duration trials in which a specified number of subjects enrolled during the accrual period are evaluated over the follow-up period. As such, the maximum duration—the accrual duration plus the follow-up duration—of a clinical trial is fixed at the design stage.

4

MAXIMUM DURATION AND INFORMATION TRIALS

Because the maximum information to be observed during the study is unknown at the time of an interim analysis, the information times have to be estimated. When monitoring a clinical trial with time-to-event data using the logrank test, the information time at an interim analysis is proportional to the maximum expected number of events to be observed during the study and is estimated by the number of events observed at the time of interim analysis divided by the maximum number of events expected by the end of the study. The denominator of this fraction is a random quantity and must be estimated. There are at least two candidates for the denominator, one under the null hypothesis of no treatment difference and the other under the specified alternative hypothesis; therefore, there are at least two candidates for the information time: tˆj =

Number of observed events at interim analysis , Maximum expected number of events Hj at the end of trial

j = 0, 1. This problem is due to the fact that the asymptotic variance of the logrank statistic depends on treatment effect, a situation similar to the comparison of proportions. This uncertainty about information horizon complicates interim analyses as it necessitates some adjustments in determining group sequential boundaries to maintain the type I error probability at a specified significance level. Kim et al. (9) proposed the following convention for estimation of the information time:  ε /ε if k < K and dk ≤ Dj , ˆtk,j = k max,j 1 otherwise where εk denotes the observed number of events at calendar time sk of the k-th interim analysis for k = 1, . . . ,K, and εmax,j denotes the information horizon under Hj for j = 0,1. By setting the information time equal to 1 at the last analysis, the type I error probability is always maintained using the error spending function. In maximum information trials, the information horizon is determined under the

specified alternative hypothesis to achieve a desired power given other design parameters, and the information time is estimated by tˆ1 and is always unbiased. The net effect is that computation of group sequential boundaries becomes straightforward; furthermore, not only the significance level but also the power of group sequential tests is maintained exactly as specified during the design of the study. Therefore, from a statistical point of view, a maximum information trial design is preferable. The end of the study is, however, defined as the random calendar time when the information horizon is realized. 3 AN EXAMPLE OF A MAXIMUM INFORMATION TRIAL For many years, radiotherapy has been the treatment of choice for patients with regional stage III non-small cell lung cancer, giving a median survival of 9 to 11 months and a 3-year survival rate of less than 10%. In attempts to improve survival in these patients, clinical researchers in the early 1980s considered the possibility that radiotherapy alone might not be sufficient to eradicate micrometastatic disease. At that time, there was also some evidence that platinum-based chemotherapy was beneficial in terms of survival in patients with more advanced disease. Therefore, various systemic approaches to the treatment of stage III patients were proposed, including chemotherapy in conjunction with radiotherapy. The Cancer and Leukemia Group B 8433 (CALGB 8433) study was developed in 1983tocompare the standard control treatment with radiotherapy alone with an experimental treatment using two courses of combination chemotherapy given before radiotherapy in these patients. The primary objective of the CALGB 8433 study was to compare survival with the experimental treatment with that with the control treatment. Under the proportional hazards assumption, survival data can be modeled based on the hazard function λ (t|z) = λ0 (t) exp (θ z) , where λ0 (t) represents the baseline hazard at time t, z is a treatment indicator, and

MAXIMUM DURATION AND INFORMATION TRIALS

θ is the log hazard ratio. The null hypothesis of no treatment difference is H0 :θ = 0, and the alternative hypothesis that the two treatments differ with respect to survival is H 1 :θ = 0. The sample size was obtained to achieve 80% power (1 − β) to detect a log hazard ratio, θ = 0.405, using the logrank test at a twosided significance level of α = 0.05. This log hazard ratio represents a 50% difference in median survival between the two treatments. Therefore, a total of 190 deaths on the two arms was required: εf ixed = =

2
4 zα/2 + zβ θ12 4 (1.96 + 0.84)2 = 190. (log 1.5)2

The fixed sample size was determined by assuming that the final analysis of the study would take place after 80% of the patients had died, so 190/0.8 or approximately 240 patients were required. Based on previous experience in the same patient population, about 60 patients were expected to be accrued each year. Therefore, the study was anticipated to have about 4 years of accrual with possibly 6 months to 1 year of additional follow-up to obtain 80% deaths among 240 patients. When this study was developed initially in 1983, this fixed sample size was used, and there was no provision for early stopping for treatment difference. The CALGB policies for interim monitoring for possible

early termination were amended in 1986, coinciding with the emergence of treatment differences in CALGB 8433. At the time of the first interim monitoring, a conservative error spending function α4∗ (t) = αt1.5 by Kim and DeMets (10) was chosen for formal sequential tests to take advantage of its flexible nature. This error spending function was known to generate group sequential boundaries similar to O’Brien and Fleming (5), but not quite as conservative early on. Also, because formal interim monitoring was going to be used, it was decided that the final analysis would be performed with more ‘‘information’’ than for the fixed-sample size analysis in order to maintain the same power. This group sequential test at 80% power has an inflation factor of F = 1.05 over the corresponding fixed-sample size design. Therefore, the maximum number of deaths required by the end of study was inflated to εmax = 190×1.05 = 200 in accordance with equation 2 and the proportionality between the number of events and the information. Table 1 summarizes the monitoring process in the study. Based on the the ground sequential test and the error spending function α4∗ (t), the study was closed in March of 1987. Although only 28% of the total number of deaths were obtained, 163 (68%) of the total number of patients were accrued by the time of study termination. More importantly, the results of this trial were published 2 years earlier than originally anticipated.

Table 1. Summary of the monitoring process in CALGB 8433. Nominal P-value Analysis date Sep 1985 Mar 1986 Aug 1986 Oct 1986 Mar 1987

5

Percent of information

Logrank P-value

Truncated O’Brien-Fleminga

Pocock

α4∗ (t)

5% 8% 18% 22% 29%

NA 0.021 0.0071 0.0015 0.0015b

0.0013 0.0013 0.0013 0.0013 0.0013

0.0041 0.0034 0.0078 0.0061 0.0081

0.0006 0.0007 0.0027 0.0026 0.0042

a The standard O’Brien-Fleming boundary would give a nominal P-value less than 0.0001 at each interim analysis shown. b The P-value of 0.0008 from the Cox model was used in the decision for early termination of the study. Note: NA, not applicable.

6

MAXIMUM DURATION AND INFORMATION TRIALS

REFERENCES 1. K. K. G. Lan and D. L. DeMets, Group sequential procedures: calendar versus information time. Stat Med. 1989; 8: 1191–1198. 2. D. Bernstein and S. W. Lagakos, Sample size and power determination for stratified clinical trials. J Stat Comput Simulat. 1978; 8: 65–73. 3. L. V. Rubinstein, M. H. Gail, and T. J. Santner, Planning the duration of a comparative clinical trial with loss to follow-up and a period of continued observation. J Chronic Dis. 1981; 34: 469–479. 4. S. J. Pocock, Group sequential methods in the design and analysis of clinical trials. Biometrika. 1977; 64: 191–199. 5. O’Brien, P. C. and T. R. Fleming, A multiple testing procedure for clinical trials. Biometrics. 1979; 35: 549–556. 6. K. K. G. Lan and D. L. DeMets, Discrete sequential boundaries for clinical trials. Biometrika. 1983; 70: 659–663.

7. K. Kim, A. A. Tsiatis, and C. R. Mehta, Computational issues in information-based group sequential clinical trials J Jpn Soc Comput Stat 2003; 15: 153–167. 8. K. Kim and A. A. Tsiatis, Study duration and power consideration for clinical trials with survival response and early stopping rule. Biometrics. 1990; 46: 81–92. 9. K. Kim, H. Boucher, and A. A. Tsiatis, Design and analysis of group sequential logrank tests in maximum duration versus information trials. Biometrics. 1995; 51: 988–1000. 10. K. Kim and D. L. DeMets, Design and analysis of group sequential tests based on the type I error spending rate function Biometrika 1987; 74: 149–154.

CROSS-REFERENCES

MAXIMUM TOLERABLE DOSE

point that is acceptable to the patient, based on several severity grading scales of adverse events, such as the Common Toxicity Criteria Grades developed by the National Cancer Institute of the United States or those developed by the World Health Organization. For instance, in oncology, DLT is usually defined as any nonhematological grade III or grade IV toxicity (except alopecia, nausea, vomiting, or fever, which can be rapidly controlled with appropriate measures); absolute neutrophil count < 500/ml for at least 7 days; febrile neutropenia (absolute neutrophil count < 500/ml for at least 3 days and fever above 38.5◦ C for 24 hours); or thrombocytopenia grade IV. Nevertheless, as stated above, definition of dose-limiting toxicities widely differs across Phase I (1).

SYLVIE CHEVRET Inserm France

The ‘‘Maximum Tolerable Dose’’ (MTD), also known as the ‘‘Maximum Tolerated Dose’’ or the ‘‘Maximally Tolerated Dose’’, is used both in animal toxicology and in early phases of clinical trials, mostly in life-threatening diseases such as cancer and AIDS. In toxicology, the MTD has been defined operationally as the highest daily dose of a chemical that does not cause overt toxicity in laboratory mice or rats. Similarly, in Phase I cancer clinical trials, the MTD of a new agent has been defined as the highest dose level of a potential therapeutic agent at which the patients have experienced an acceptable level of dose-limiting toxicity (DLT) or that does not cause unacceptable side effects. In both settings, these definitions refer on imprecise notions such as ‘‘overt,’’ ‘‘acceptable,’’ or ‘‘unacceptable’’ toxicity. Moreover, definition of dose-limiting toxicities has been found highly variable across published Phase I studies (1). This lack of consensus is observed also when dealing with the methods for establishing the MTD, with wide variations in the designs and the statistical methods of estimation. Finally, the concept of MTD itself has become criticized and even controversial. We summarize the main concepts underlying the choice of MTD as the main endpoint in cancer Phase I clinical trials and the main approaches used in establishing the MTD of a new drug. More discussion with regards to its currently reported limits is provided in the last section with some proposed alternate endpoints. 1

1.1 Underlying Assumptions Determining the optimal dose of a new compound for subsequent testing in Phase II trials is the main objective of cancer Phase I clinical trials. With cytotoxic agents, this dose typically corresponds to the highest dose associated with an acceptable level of toxicity, based on the underlying assumption that stems from the work of Skipper et al. (3) that the higher the dose, the greater the likelihood of drug efficacy. In addition to the relationship between dose and antitumor response, cytotoxic agents also exhibit a dose–toxicity relationship. Thus, dose-related toxicity is regarded as a surrogate for efficacy. In other words, it is assumed that dose-response curves for toxicity and efficacy are parallel or, simply expressed, ‘‘the more pain, the more gain.’’ These findings have yielded the concept of maximum tolerable or tolerated dose (MTD) as the main objective of Phase I trials. Of note, the recommended dose level for more Phase II trials is either considered as synonymous with the MTD or, in two thirds of the Phase I trials as reported recently (1), chosen as one dose level below the selected MTD.

BASIC CONCEPTS AND DEFINITIONS

The MTD is often defined as the dose that produces an ‘‘acceptable’’ level of toxicity (2); the dose that, if exceeded, would put patients at ‘‘unacceptable’’ risk for toxicity; or the dose that produces a certain frequency of (medically unacceptable) reversible, dose-limiting toxicity (DLT) within the treated patient population. DLT includes host effects up to the

2

HOW IS THE MTD ESTABLISHED?

In Phase I oncology trials conducted over the past few decades, the MTD has usually been

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

MAXIMUM TOLERABLE DOSE

estimated by the traditional 3 + 3 escalation rule, which traces back to the last 1960s (4). The maximum tolerated dose is determined thus by testing increasing doses on different groups of people until the highest dose with acceptable side effects is found (see ‘‘Phase I trials’’ for more details). 2.1 The MTD is Interpreted Differently According to the Design Numerous Phase I dose finding clinical trials are conducted everyday to find the ‘‘maximum tolerated dose’’ (MTD) of a cancer treatment. Although various innovative statistical designs for Phase I clinical trials have been proposed in the literature, the traditional 3 + 3 design is still widely used because of its algorithm-based simplicity in logistics for the clinical investigators to carry out. Actually, based on a recent review of Phase I trials of single agent cytotoxics published between 1993 and 1995, the MTD was usually defined as the dose level at which > 2/6 patients experienced DLT, but several studies required 3–4/6 patients (1). Such algorithm- or rulebased Phase I designs treat the MTD as being identifiable from the data so that in this setting, the MTD is a statistic rather than a parameter (5). Indeed, no statistical estimation after these trials has been maintained, although statistical quantities of interest have been recently estimated from such trial data (6). By design, the MTD estimate relies heavily on the actual cohort of patients treated and the order in which they enter the study, with poor properties. It seemed in the early 1990s that the MTD should be thought of as a population characteristic rather than as a sample statistic. The MTD as a percentile of the dose–toxicity relationship seemed the mostly commonly used definition of the MTD in so-called model-based designs. Most proposals consisted in establishing a mathematical model for the probability of DLT over the dose scale, iteratively fitted from the data after each patient inclusion using Bayesian inference (7,8). Comparisons show that the Bayesian methods are much more reliable than the conventional algorithm for selecting the MTD (9). NonBayesian methods were also proposed, with close properties (10).

3 EXTENDING THE MTD? 3.1 Maximum Tolerated Schedule Most Phase I clinical trials are designed to determine a maximum tolerated dose for one initial administration or treatment course of a cytotoxic experimental agent. Toxicity usually is defined as the indicator of whether one or more particular adverse events occur within a short time period from the start of therapy. However, physicians often administer an agent to the patient repeatedly and monitor long-term toxicity caused by cumulative effects. A new method for such settings has been proposed, the goal of which is to determine a maximum tolerated schedule (MTS) rather than a conventional MTD (11,12). 3.2 Overdose Control More recently, some researchers considered the percentile definition of MTD as inadequate in addressing the ethical question of protecting the patients from severe toxicity. This consideration leads to the suggestion by Babb and colleagues (13) that the MTD should be chosen with toxicity not exceeding the tolerable toxicity with a high probability, which imposes a safety constraint on the overdose control. 3.3 Most Successful Dose Objective responses observed in Phase I trials are important for determining the future development of an anticancer drug (14). Thus, much recent interest has developed in Phase I/II dose finding designs in which information on both toxicity and efficacy is used. These designs concern, for instance, dose finding in HIV in which both information on toxicity and efficacy are almost immediately available. Recent cancer studies are beginning to fall under this same heading in which toxicity can be evaluated quickly and, in addition, we can rely on biological markers or other measures of tumor response. Unlike the classic Phase I dose finding design in which the aim is to identify the MTD, the Phase I/II dose finding study aims to locate the most successful dose (MSD) (i.e., the dose that maximizes the product of the probability of seeing no toxicity together with the probability of seeing

MAXIMUM TOLERABLE DOSE

a therapeutic response). For a dose finding study in cancer, the MSD, among a group of available doses, is that dose at which the overall success rate is the highest (15,16). Close proposals have been also published, based on a bivariate modeling of toxicity and efficacy (17,18). 3.4 Patient-Specific Optimal Dose Because Phase I trials are small studies, the maximum tolerated dose of a new drug may not be established precisely for any individual. New paradigms for the clinical evaluation of new cancer therapies have been proposed. One proposal entails adjusting the search for the optimal dose on the basis of measurable patient characteristics to obtain a personalized treatment regimen (19). Accelerated titration (i.e., rapid intrapatient drug dose escalation) designs also seem to provide a substantial increase in the information obtained with regards to interpatient variability or cumulative toxicity (20). 3.5 Different Optimal Doses: The Example of Targeted Agents in Cancer Advances in molecular biology have led to a new generation of anticancer agents that inhibit aberrant and cancer-specific proliferative and antiapoptotic pathways. These agents may be cytostatic and may produce relatively minimal organ toxicity compared with standard cytotoxics. Thus, these new, targeted anticancer agents have posed challenges to the current Phase I paradigm of dose selection based on toxicity. Indeed, traditional trial designs and endpoints may not be adequate for developing contemporary targeted drugs. Notably, increasing the drug dose to toxicity seems unnecessary for drug effect, which makes the use of MTD as a surrogate of effective dose inappropriate in the Phase I setting. To accommodate these new drugs, the concept of an optimal biologic dose defined as a dose that reliably inhibits a drug target or achieves a target plasma concentration has been reported as desirable and appropriate for the Phase I study of mechanism-based, relatively nontoxic novel agents. This concept should rely on pharmacodynamic data in addition to toxicity (21).

3

In these early-phase dose finding clinical trials with monotone biologic endpoints, such as biological measurements, laboratory values of serum level, and gene expression, a specific objective is to identify the minimum dose that exhibits adequate drug activity and shifts the mean of the endpoint from a zero dose to the so-called minimum effective dose (22). Stepwise test procedures for dose finding have been well studied in the context of nonhuman studies in which the sampling plan is done in one stage (23). This development has fueled interest in alternatives to toxicity as a surrogate endpoint in Phase I clinical trials, although no consensus has been reached. Indeed, the optimal biologic dose rarely formed the basis of dose selection. This situation is exemplified in a recent overview of 60 Phase I trials that involved 31 single agents representative of the most common targets of interest in the oncology literature: 60% still used toxicity, whereas only 13% used pharmacokinetic data as endpoints for selection of the recommended Phase II dose (24). Finally, the selected dose should incorporate the fact that wide variations will be found in steady-state drug levels in patients (25). 4

CONCLUDING REMARKS

Phase I clinical trials are designed to identify an appropriate dose for experimentation in Phase II and III studies. Assuming that efficacy and toxicity are closely related, the highest dose level of a potential therapeutic agent at which the patients have experienced an acceptable level of dose-limiting toxicity defined the main objective of these trials. Efforts in Phase I studies over the past several years have focused on efficient estimation of maximum tolerated dose. Recently, the MTD has been controversial, in part because of difficulties in extrapolating findings to the whole treatment course and in applying population findings to individuals. Some extensions have been proposed to better address these issues. Otherwise, improved understanding of the biology of cancer has led to the identification of new molecular targets and the development of pharmacologic agents that hold

4

MAXIMUM TOLERABLE DOSE

promise for greater tumor selectivity than traditional cytotoxic agents. This development has avoided the use of the MTD as a surrogate endpoint for efficacy. Nevertheless, increased research efforts should be spent on the prospective evaluation and validation of novel biologic endpoints and innovative clinical designs so that promising targeted agents can be effectively developed to benefit the care of cancer patients. A need exists for improved definition of optimal biologic dose (26). Finally, allowing accelerated drug development through combined Phase I/II and Phase II/III clinical trial designs is a promising research area in the near future.

REFERENCES 1. S. Dent and E. Eisenhauer, Phase I trial design: Are new methodologies being put into practice? Ann. Oncol. 1996; 7(6): 561–566. 2. B. Storer, Phase I trials, in T. Redmond (ed.), Biostatistics in clinical trials, C.a.C. Chichester: John Wiley & Sons, 2001,:. pp. 337–342. 3. H. Skipper, F. J. Schabel, and W. Wilcox, Experimental evaluation of potential anticancer agents XIII: on criteria and kinetics associated with ‘‘curability’’ of experimental leukemia. Cancer Chemother. Rep. 1964; 35: 1. 4. M. Schneiderman, Mouse to man: statistical problems in bringing a drug to clinical trial. Proc. of the fifth Berkeley symposium on mathematical statistics and prob. in fifth Berkeley symposium on mathematical statistics and probability. Berkeley, CA: University of California Press, 1967. 5. W. F. Rosenberger and L. M. Haines, Competing designs for phase I clinical trials: a review. Stat. Med. 2002; 21(18): 2757–2770. 6. Y. Lin and W. Shih, Statistical properties of the traditional algorithm-based designs for phase I cancer clinical trials. Biostatistics. 2001; 2(2): 203–215. 7. J. O’Quigley, M. Pepe, and L. Fisher, Continual reassessment method: a practical design for phase 1 clinical trials in cancer. Biometrics. 1990; 46(1): 33–48. 8. J. Whitehead and D. Williamson, Bayesian decision procedures based on logistic regression models for dose-finding studies. J. Biopharm. Stat. 1998; 8(3): 445–467.

9. P. Thall and S. Lee, Practical model-based dose-finding in phase I clinical trials: methods based on toxicity. Int. J. Gynecol. Cancer. 2003; 13(3): 251–261. 10. J. O’Quigley, and L. Shen, Continual reassessment method: a likelihood approach. Biometrics. 1996; 52(2): 673–684. 11. T. Braun et al., Simultaneously optimizing dose and schedule of a new cytotoxic agen. Clin. Trials. 2007; 4(2): 113–124. 12. T. Braun, Z. Yuan, and P. Thall, Determining a maximum-tolerated schedule of a cytotoxic agent. Biometrics. 2005; 61(2): 335–343. 13. J. Babb, A. Rogatko, and S. Zacks, Cancer phase I clinical trials: efficient dose escalation with overdose control. Stat. Med. 1998; 17(10): 1103–1120. 14. I. Sekine et al., Relationship between objective responses in phase I trials and potential efficacy of non-specific cytotoxic investigational new drugs. Ann. Oncol. 2002; 13(8): 1300–1306. 15. J. O’Quigley, M. Hughes, and T. Fenton, Dosefinding designs for HIV studies. Biometrics. 2001; 57(4): 1018–1029. 16. S. Zohar and J. O’Quigley, Optimal designs for estimating the most successful dose. Stat. Med. 2006; 25(24): 4311–4320. 17. P. F. Thall, E. H. Estey, and H.-G. Sung, A new statistical method for dose-finding based on efficacy and toxicity in early phase clinical trials. Investigational New Drugs. 1999; 17: 155–167. 18. Y. Zhou et al., Bayesian decision procedures for binary and continuous bivariate doseescalation studies. Pharm. Stat. 2006; 5(2): 125–133. 19. A. Rogatko et al., New paradigm in dosefinding trials: patient-specific dosing and beyond phase I. Clin. Cancer Res. 2005; 11(15): 5342–5346. 20. R. Simon et al., Accelerated titration designs for phase I clinical trials in oncology. J. Natl. Cancer Inst. 1997; 89(15): 1138–1147. 21. D. Kerr, Phase I clinical trials: adapting methodology to face new challenges. Ann. Oncol. 1994; 5(S4): 67–70. 22. S. Kummar et al., Drug development in oncology: classical cytotocixs and molecularly targeted agents. Br. J. Clin. Pharmacol. 2006; 62(1): 15–26. 23. M. Polley and Y. Cheung, Two-stage designs for dose-finding trials with a biologic endpoint using stepwise tests. Biometrics. 2007.

MAXIMUM TOLERABLE DOSE 24. W. Parulekarand E. Eisenhauer, Phase I trial design for solid tumor studies of targeted, noncytotoxic agents: theory and practice. J. Natl. Cancer Inst. 2004; 96(13): 990–997. 25. A. Adjei, The elusive optimal biologic dose in phase I clinical trials. J. Clin. Oncol. 2006; 24(25): 4054–4055. 26. B. Ma, C. Britten, and L. Siu, Clinical trial designs for targeted agents. Hematol. Oncol. Clin. North Am. 2002; 16(5): 1287–1305.

CROSS-REFERENCES Phase I trials Therapeutic Dose Range

5

METADATA

2.1 A Metadata Example The importance of metadata can be illustrated by the following example adapted from actual events. Consider the two datasets shown in Table 1. Each dataset contains the same variables but has different data values. With only cryptic variable names and no additional metadata, it would be almost impossible to determine what the data values represent without additional documentation. If we add context by stating that these datasets represent rocket burn instructions for placing a satellite in orbit and that the last three variables represent distance, speed, and force, then a knowledgeable rocket scientist may be able to infer what the data points could represent. In fact, that inference, based on inadequate metadata, led to a very costly mistake. The Mars Climate Orbiter was launched in December 1998 and was scheduled to go into orbit around Mars 9 months later. The $150 million satellite failed to reach orbit because the force calculations were ‘‘low by a factor of 4.45 (1 pound force = 4.45 Newtons), because the impulse bit data contained in the AMD file was delivered in lb-sec instead of the specified and expected units of Newton-sec’’ (2). In other words, a data file in English units (pounds) was input into a program that expected metric units (Newtons). Table 2 illustrates the same two datasets with additional metadata, including meaningful variable names, labels, and units. Note the units used in each dataset are now clearly identified in the column header metadata. Although it cannot be said with certainty that this metadata would have prevented the error, it would have increased the probability that the error would have been found by project personnel. The use of machine readable or ‘‘parsable’’ metadata would allow computer programs to be used to check for compatible units.

DAVID H. CHRISTIANSEN Christiansen Consulting Boise, Idaho

1

INTRODUCTION

Metadata (also ‘‘meta data’’ or ‘‘meta-data’’) are commonly defined as ‘‘data about data.’’ Clinical trial metadata are concerned with describing data originating from or related to clinical trials, including datasets and statistical analyses performed on the datasets. Clinical trial metadata may be in the form of a separate written document, may be linked electronically to a document or dataset, or may be integrated into a dataset as part of the definition of the data fields or variables. Metadata may be accessed by statisticians performing analyses on the data and by other scientists reviewing or using the data and results. Increasingly, metadata included in computerized datasets (machine-readable metadata) can also be used by statistical software and other computer applications to present or use the data in an appropriate manner, based on the metadata description. For machine-readable metadata to be used by a computer application, standards for the format and content must exist for both the metadata and the application that reads the metadata. Metadata are an important component of the documentation required for regulatory submissions and should provide a clear and concise description of the data collected and the analyses performed.

2

HISTORY/BACKGROUND

The term metadata was coined in 1969 by Jack E. Kelly. Although ‘‘Metadata’’ was copyrighted in 1986 by The Metadata Company (1), the generic ‘‘metadata’’ is commonly used in many disciplines, including computer science, database administration, geographic science, and clinical data management.

2.2 Geospatial Data Geographic science is an example of a discipline with well-defined metadata. In 1990, the Office of Management and Budget established the Federal Geographic Data

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

METADATA

Table 1. Example Datasets with No Metadata Dataset 1 Event

Time

D

S

F

Begin End

9/23/99 02:01:00 9/23/99 02:17:23

121,900,000

12,300 9,840

143.878

Dataset 2 Event

Time

D

S

F

Start Finish

19990923 05:01:00 19990923 05:17:23

196,200,000

5.5 4.4

640

Table 2. Mars Climate Orbiter Burn Instructions with Metadata Dataset 1 - Mars Orbit Insertion Burn in English Units Event

M/D/Y HH:MM:SS Pacific Daylight Time (Earth Receive Time, 10 min. 49 sec. Delay)

Begin End

9/23/99 02:01:00 9/23/99 02:17:23

Distance (miles) 121,900,000

Speed (miles/hr) 12,300 9,840

Force (Pounds) 143.878

Dataset 2 - Mars Orbit Insertion Burn in Metric Units Event Start Finish

YYYYMMDD EDT (Earth Receive Time, 10 min. 49 sec. Delay) 19990923 05:01:00 19990923 05:17:23

Committee as an interagency committee to promote the coordinated development, use, sharing, and dissemination of geospatial data on a national basis (3). In addition to detailed metadata and tools, the group also provides informational and educational materials that may be useful for motivating the development and use of metadata. For example, the U.S. Geologic Survey in a document titled ‘‘Metadata in Plain Language’’ (4) poses the following questions about geospatial data:

1. What does the dataset describe? 2. Who produced the dataset? 3. Why was the dataset created? 4. How was the dataset created? 5. How reliable are the data; what problems remain in the dataset? 6. How can someone get a copy of the dataset? 7. Who wrote the metadata?

Distance (km) 196,200,000

Speed (km/sec) 5.5 4.4

Force (Newtons) 640

Although these questions may have a different context and different emphasis in clinical trials, they can provide background information for our discussion of clinical trial metadata. 2.3 Research Data and Statistical Software Research data management refers to the design, collection, editing, processing, analyzing, and reporting of data that results from a research project such as a clinical trial. The characteristics and requirements for research data management activities are different from those of a commercial data management system. Because clinical trials are experimental by nature, the resulting datasets are unique, contain many different variables, and require a high degree of studyspecific documentation, including metadata. Conversely, commercial data systems, such as payroll or credit card billing, are characterized by very stable systems that perform well-defined functions on a relatively small

METADATA

3

number of standard variables. These differences make it difficult to use commercial data systems, such as databases and reporting systems, for research data. These unique requirements drove the development of statistical software specifically for research data over the last 30 years. The most commonly used systems all have some type of metadata at both the dataset and variable levels. These metadata allow the researchers conducting the studies to describe the structure and content of the datasets clearly. The ability to describe the data clearly and unambiguously is important in the analysis and reporting of the study results, both within an organization and for regulatory review and approval of new treatments.

3

2.4 Electronic Regulatory Submission

A dataset is a computer file structured in a predictable format that can be read and processed by a computer application or program, typically a statistical system such as SAS (SAS Institute, Cary, NC), S-Plus (Insightful Corporation, Seattle, WA), or SPSS (SPSS Corporation, Chicago, IL). In the discussion here, SAS terminology will be used, but similar concepts and attributes exist in all statistical languages and systems. Dataset-level metadata describes the content, structure, and use of the dataset by describing its physical and logical attributes. Some of these attributes relate to the dataset itself, whereas others are dependent on the context in which the dataset is used. The description and use of these attributes as they relate to regulatory submissions are shown below.

The International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) (5) has developed a Common Technical Document (CTD) that ‘‘addresses the organization of the information to be presented in registration applications for new pharmaceuticals (including biotechnology-derived products) in a format that will be acceptable in all three ICH regions (Japan, Europe and United States)’’ (6). The CTD outline is, in fact, metadata for the regulatory submission. It provides information about the submission in a structured format that facilitates both the creation and review of the submission. Because the requirements for submission of clinical data vary from country to country, the CTD does not specifically address electronic submission of clinical data. The U.S. Food and Drug Administration (FDA), however, defined metadata requirements for clinical trial datasets submitted as part of its treatment approval processes in 1999 (7). The Clinical Data Interchange Standard Consortium (CDISC) enhanced the FDA metadata by adding metadata attributes at the dataset and variable levels (8). Since that time, the FDA and CDISC have collaborated on more standards, resulting in the FDA referencing the CDISC standard for clinical domain data as an acceptable format (9,10). The FDA and CDISC currently are working on a similar standard for analysis datasets (11,12).

DATASET METADATA

In the context of clinical trials and regulatory submissions, the most useful metadata refer to the datasets that contain the trial results. The datasets can be generally classified as those data collected during the execution of the trial (tabulation data or observed data) or data derived from the observed data for the purpose of display or analysis (analysis data). Dataset metadata can be classified at various levels as described below. The metadata attributes described are discussed by one or more of the following standards or organizations: CDISC Tabulation (10), CDISC Analysis (11), FDA (13), and SAS (14). 3.1 Dataset-Level Metadata

3.1.1 Dataset Name. – Unique dataset name for this file. FDA and CDISC have naming conventions for some clinical domain and analysis datasets. (FDA, CDISC, and SAS) 3.1.2 Description (Label). – A more detailed description of the content of the dataset. (FDA, CDISC, and SAS) 3.1.3 Location. – The relative physical file location in an electronic submission. (FDA and CDISC) 3.1.4 Structure. – The shape of the dataset or the level of detail represented by each

4

METADATA

row or record. Structure can range from very horizontal (one record per subject) to very vertical (one record per subject per visit per measurement). It is recommended that structure be defined as ‘‘one record per . . . ’’ rather than the ambiguous terms normalized or denormalized, horizontal or vertical, tall or short, skinny or fat, and so forth. (CDISC) 3.1.5 Purpose. – Definition of the type of dataset as tabulation or analysis. (CDISC) 3.1.6 Key Fields. – Variables used to uniquely identify and index records or observations in the dataset. (CDISC). 3.1.7 Merge Fields. – A subset of key fields that may be used to merge or join SAS datasets. (CDISC analysis) 3.1.8 Analysis Dataset Documentation. – Written documentation that includes descriptions of the source datasets, processing steps, and scientific decisions pertaining to creation of the dataset. Analysis dataset creation programs may also be included. (CDISC Analysis) 3.1.9 Rows. – The number of data records or observations in the dataset. (SAS) 3.1.10 Columns. – The number of variables or fields in the dataset. For most analysis datasets, each column represents the measurement value of some characteristic of the subject such as sex, age, and weight at a visit, for example. (SAS) 3.2 Variable-Level Metadata Each column or variable in a dataset has certain attributes that describe the content and use of the variable. These variable attributes are usually consistent for all observations within a dataset. That is, each column has the same attributes for all rows of the dataset. This rectangular data structure is required for statistical analysis by the vast majority of statistical analysis software and is the natural structure for CDISC Analysis datasets. Many CDISC Tabulation datasets, however, have a structure of one record per measurement. Because different measurements have

different attributes, metadata at the variable level is not adequate. See the section on value-level metadata below for a discussion of this issue. 3.2.1 Variable Name. – Unique name for the variable. A variable name should be consistent across datasets and studies within a regulatory submission. The data values of a variable should also be consistent in definition and units across all datasets within a submission. For example, if AGE is recorded in years in one study and months in another, then either AGE must be converted to, say, years in the second study or a different variable name, say AGEMON, must be used for age in months. (FDA, CDISC, and SAS) 3.2.2 Variable Label. – A more detailed description of the variable. This description may be used by some software and review tools. (FDA, CDISC, and SAS) 3.2.3 Type. – Description of how the data values for this variable are stored. Current conventions specify only a character string (CHAR) or a numeric value (NUM). These two types are consistent with SAS and other software, but additional types such as floating point, integer, binary, and date/time are used by some systems. (FDA, CDISC, and SAS) 3.2.4 Length. – The number of bytes allocated to store the data value. For CHAR variables, this number is the length of the character string. SAS and other software define this attribute as the number of bytes allocated to store the numeric value in some internally defined form, typically floating point hexadecimal. This length is not the number of digits used to display a number. See Format below. (SAS) 3.2.5 Format. – Description of how a variable value is displayed. Formats can be general, defining the width of a display field and the number of decimal digits displayed for a number. They can also provide code list values (1 = ‘‘Female’’; 2 = ‘‘Male’’) or display numerically coded values such as SAS dates in a human readable form. (FDA, CDISC, and SAS)

METADATA

CDISC Tabulation metadata defines this code list attribute as ‘‘Controlled Terms.’’ These terms are standard values for specific variables in certain clinical domains. 3.2.6 Role. – Description of how a variable may be used in a particular dataset. A variable may be assigned multiple roles. The role attribute is used by CDISC in two distinct ways. First, CDISC Tabulation datasets have a set of specific roles designed to describe a variable in the context of the proposed FDA JANUS database (15). • Identifier variables, usually keys,

identify the study, subject, domain, or sequence number of the observation. • Topic variables specify the focus of the observation, typically the name of a measurement or lab test in a one record per measurement structured dataset. • Timing variables specify some chronological aspect of a record such as visit number or start date. • Qualifier variables define a value with text, units, or data quality. Qualifiers are often used to store variables not otherwise allowed in tabulation datasets. Role attributes in CDISC Analysis datasets have a different emphasis than those in the tabulation models. Analysis roles focus on providing information useful to the statistical review rather than specification for the still-to-be-developed JANUS database. Because the primary goal of analysis roles is clear communication of the statistical analysis performed by the sponsor, the values of the role attribute are open-ended and can be extended as needed to meet this goal. The following roles have been identified by FDA and CDISC through the Analysis Dataset Model group (16). It should be noted that these definitions are still under development and may be subject to change. • Selection variables are frequently

used to subset, sort, or group data for reporting, displaying, or analysis. Common selection variables include treatment group, age, sex, and race. Specific study designs, drug indications,

5

and endpoints may have specific selection variables as well. For example, a hypertension trial may identify baseline blood pressure measurements as selection variables of interest. Flag variables identifying analysis populations such as ‘‘per protocol’’ or ‘‘intent to treat’’ are also commonly identified as selection variables. • Analysis variables relating to major study objectives or endpoints may be identified to assist reviewers. This identification may be especially useful in complex studies in which it may not be clear from the variable name which variable is the primary endpoint. • Support variables are identified as useful for background or reference. For example, the study center identifier may be used to group subjects by center, but the study name or investigator name would provide supporting information. • Statistical roles such as covariate, censor, endpoint, and so forth may also be useful for specific analyses and study designs. These roles may be added as needed to improve clear communication between the study authors and the reviewers.

3.2.7 Origin. – Describes the point of origin of a variable. CDISC Tabulation datasets allow CRF ‘‘derived’’ and ‘‘sponsor defined’’ values of origin. This attribute usually refers to the first occurrence of a value in a clinical study and does not change if a variable is added to another file such as an analysis dataset. 3.2.8 Source. – In a CDISC Analysis dataset, source provides information about how a variable was created and defines its immediate predecessor. For example, an analysis dataset may be created by merging two or more source datasets. The variable SEX from the demographics domain dataset DM would have a source of ‘‘DM.SEX’’. This convention of specifying the immediate predecessor defines an audit trail back to the original value, no matter how many generations of datasets are created. Derived variables in

6

METADATA

analysis datasets may include a code fragment to define the variable or may hyperlink to more extensive documentation or programs.

requires additional metadata for each value of the vital signs test as shown in Table 4. The value-level metadata defines the attributes needed to transpose the tabulation dataset into a CDISC Analysis dataset, typically with a structure of one record per subject. This dataset, shown in Table 5, contains the data points from the tabulation dataset, but in a different structure. Values of SEX and the derived variable Body Mass Index (BMI) have been added to illustrate that analysis dataset can include data from several sources. Note that now each measurement is represented by a variable, and each variable can have its own metadata attributes. This dataset structure is preferred by most statisticians and can be used directly by most statistical packages. ?>

3.3 Value-Level Metadata Many CDISC Tabulation (SDTM) datasets have a structure of one record per subject per time point per measurement. This structure means that different measurements having different attributes such as name, label, type, and format vary from record to record. To provide clear communication of the content and use of such a dataset, each measurement name (test code) must have its own metadata. For example, the CDISC Tabulation model defines a Vital Signs Dataset with measurements for height, weight, and frame size. Table 3 illustrates selected portions of such a dataset and its metadata. Some CDISC attributes have been changed or omitted to simplify the example. Note that the data values of HEIGHT, WEIGHT, and FRMSIZE are stored in the same column, and no useful metadata identifies format, variable type, role, or origin. This dataset is difficult to understand and cannot be used with statistical software without additional programming. This file structure

3.4 Item-Level Metadata Item-level metadata refers to the attributes of an individual data value or cell in a rectangular data matrix. That is, it refers to a value for the crossing of a specific variable (column) and observation (row). Item-level metadata are typically used to describe the quality of that particular measurement, for example, a partial date in which the day portion is missing and is imputed or a lipid measurement

Table 3. Vital Signs Tabulation Dataset Vital Signs Findings–1 Record/subject/measurement Name

USUBJID

VSTESTCD

VSTEST

Label Type Format Origin Role

Subject ID Char

VS Short Name Char

VS Name Char

Sponsor Identifier

CRF Topic

Sponsor Qualifier

00001 00001 00001

HEIGHT WEIGHT FRMSIZE

Height in cm Weight in kg Frame Size

VSSTRESN

VSSTRESC

Numeric Result Numeric ? CRF Qualifier

Character Result Char ? CRF Qualifier

165 56.1 Small

Table 4. Vital Signs Tabulation Value-Level Metadata Vital Signs Value-Level Metadata for VSTESTCD Value (VSTESTCD)

Label (VSTEST)

Type

Format or Controlled Terms

Origin

HEIGHT WEIGHT FRMSIZE

Height in cm Weight in kg Frame size

Num Num Char

3.0 5.1 Small, Medium, Large

VSSTRESN VSSTRESN VSSTRESC

METADATA

7

Table 5. Vital Signs Analysis Dataset Vital Signs Analysis - 1 Record/Subject Name

USUBJID

SEX

Label Type Format

Subject ID Char

Sex Char Female, Male

Origin Source

Role

Sponsor CRF VS.USUBJID DM.SEX

Identifier 00001

Qualifier, Selection Female

HEIGHT

WEIGHT

BMI

FRMSIZE

Num 3.0

Num 5.1

Num 5.2

CRF VS.VSSTRESN (where VS.VSTESTCD = ‘‘HEIGHT’’) Analysis

CFR VS.VSSTRESN (where VS.VSTESTCD = ‘‘WEIGHT’’) Analysis

Derived WEIGHT/ (0.01* HEIGHT)∗∗ 2

Analysis

Char Small, Medium, Large CRF VS.VSSTRESC (where VS.VSTESTCD = ‘‘FRMSIZE’’) Analysis

165

56.1

20.61

Small

in which a frozen sample was accidentally allowed to thaw. In each of these cases, a value exists, but it may be considered to be of lesser quality than a regular measurement. Because of the complex and expensive nature of clinical trials, it is not always practical to discard such measurements, nor is it scientifically valid to treat them as complete. The identification of these data items is especially important for accurate statistical analysis in regulatory submissions. In a discussion of quality assurance for submission of analysis datasets, one FDA statistical director stated that it was desirable to provide a ‘‘clear description of what was done to each data element: edits, imputations, partial missing . . . .’’ (17). Historically, the concept of status attributes for data items was used in the 1970s by the Lipids Research Clinic Program to identify missing values and record the editing status of each variable (18). Clinical research data management systems may also have similar features, but this information is not easily exported to statistical reporting and analysis programs. Currently, the most common method of identifying itemlevel quality involves the addition of a separate variable that contains the status value. These ‘‘status flags’’ are cumbersome to maintain and can increase the size of datasets considerably. The development of statistical datasets using the eXtensible Markup Language (XML) (19) have the potential to provide the more flexible structure required

for integrating item-level metadata into clinical datasets, but the tools required do not exist at this time. An audit file system for regulatory statistical reviewers was proposed by an FDA statistical director as a ‘‘file describing the changes or edits made during the data management or cleaning of the data,’’ that is, provide a link or audit trial between the original and submitted data values by providing metadata related to the edits, including the following attributes (20). • Patient, observation, visit, variable, and

other identifiers • Original and submitted values • Qualifiers describing the change such as

who, when, and why • Edit codes describing the action taken

such as empty (not recorded), completed, replaced, confirmed, suspicious but not collectable. It is interesting to note that many of the edit codes proposed by the FDA director are similar to the 1970s system described above. It should also be noted that current data management systems do have audit trails, but they typically cannot extract and submit the data in a form that is useable by a regulatory reviewer. 4

ANALYSIS RESULTS METADATA

Analysis results metadata define the attributes of a statistical analysis performed

8

METADATA

on clinical trial data. Analyses may be tables, listings, or figures included in a study report or regulatory submission. Analyses may also be statistical statements in a report, for example, ‘‘The sample size required to show a 20% improvement in the primary endpoint is 200 subjects per treatment arm’’ or ‘‘The active treatment demonstrated a 23% reduction in mortality (p = 0.023) as compared to placebo.’’ Analysis results metadata are designed to provide the reader or reviewer with sufficient information to evaluate the analysis performed. Inclusion of such metadata in FDA regulatory submissions was proposed in 2004 (21) and is included in the CDISC Analysis Data Model V 2.0 (22). By providing this information in a standard format in a predictable location, reviewers can link from a statistical result to metadata that describes the analysis, the reason for performing the analysis, and the datasets and programs used to generate the analysis. Note that analysis results metadata are not part of an analysis dataset but that one attribute of analysis results metadata describes the analysis datasets used in the analysis. • Analysis Name. – A unique identifier

•

•

•

•

for this analysis. Tables, figures, and listing may incorporate the name and number (Fig. 4 or Table 2.3). Conventions for this name may be sponsorspecific to conform to Standard Operating Procedures (SOPs). Description. – Additional text describing the analysis. This field could be used to search for a particular analysis or result. Reason. – Planned analyses should be linked to the Statistical Analysis Plan (SAP). Other reasons would include data driven, exploratory, requested by FDA, and so forth. Dataset(s). – Names of datasets used in the analysis. If datasets are part of a submission, then a link to the dataset location should be provided. Documentation. – A description of the statistical methodology, software used for computation, unexpected results, or any other information to provide the

reviewer with a clear description of the analysis performed. Links to the SAP, external references, or analysis programs may also be included.

5 REGULATORY SUBMISSION METADATA 5.1 ICH Electronic Common Technical Document In addition to the Common Technical document described earlier, the ICH has also developed a specification for an electronic Common Technical Document (eCTD), thus defining machine-readable metadata for regulatory submissions. This eCTD is defined to serve as an interface for industry-to-agency transfer of regulatory information while at the same time taking into consideration the facilitation of the creation, review, lifecycle management, and archival of the electronic submission (23). The eCTD uses XML (24) to define the overall structure of the document. The purpose of this XML backbone is two-fold: ‘‘(1) to manage meta-data for the entire submission and each document within the submission and (2) to constitute a comprehensive table of contents and provide corresponding navigation aids’’ (25). Metadata on submission level include information about submitting and receiving organization, manufacturer, publisher, ID and kind of the submission, and related data items. Examples for metadata on document level are versioning information, language, descriptive information such as document names, and checksums used to ensure accuracy. 5.2 FDA Guidance on eCTD Submissions The FDA has developed a guidance for electronic submission based on the ICH eCTD backbone. As discussed earlier, the ICH does not define detailed specification for submission of clinical data within the eCTD, but it does provide a ‘‘place-holder’’ for such data in the guideline E3 ‘‘Structure and Content of Clinical Study Reports’’ as Appendix 16.4 with the archaic term ‘‘INDIVIDUAL PATIENT DATA LISTINGS (US ARCHIVAL LISTINGS)’’ (26). The FDA eCTD specifies that submitted datasets should be organized as follows (27): Individual Patient Data Listings (CRTs)

METADATA • Data tabulations

– Data tabulations datasets – Data definitions – Annotated case report form • Data listing

– Data listing datasets – Data definitions – Annotated case report form

9

developments may include protocol authoring tools, Statistical Analysis Plan templates, eCRF and CRF automated database design, automated analysis and reporting, and submission assembly. This tool development by government, drug developers, and software providers will contribute to drug development and approval by enhancing the clear communication of the content and structure of clinical trial data and documents.

• Analysis datasets

– – – –

Analysis datasets Analysis programs Data definitions Annotated case report form

• Subject profiles • IND safety reports

The FDA Study Data Specification document (28) defines tabulation and analysis datasets and refers to the CDISC Data Definition Specification (Define.XML) for machinereadable dataset metadata in XML (29). This machine-readable metadata and the ICH eCDT are key elements in providing clear communication of the content and structure of clinical datasets and regulatory submissions. These metadata standards allow the regulatory agencies, software developers, and drug developers to create and use standard tools for creating, displaying, and reviewing electronic submissions and clinical datasets. The FDA has developed a viewer to use the ICH eCTD backbone to catalog and view the components of a submission, thus providing FDA reviewers with a powerful tool to view, manage, and review submissions. Software developers have used the Define.XML standard metadata to develop tools for compiling and viewing patient profiles and viewing tabulation datasets. SAS Institute has developed software to generate XML-based datasets with Define.XML metadata (30) and viewing tools for review and analysis. These first steps demonstrate the power of having clearly defined metadata for clinical research. The adoption and additional specification of these metadata standards will provide the basis for the development of a new generation of tools for review and analysis. Future

REFERENCES 1. U.S. Trademark Registration No. 1,409,260. 2. Mars Climate Orbiter Mishap Investigation Board Phase I Report, 1999:13. Available:ftp. hq.nasa.gov/pub/pao/reports/1999/MCO report. pdf. 3. The Federal Geographic Data Committee. Available:www.fgdc.gov/. 4. U.S. Geologic Survey, Metadata in Plain Language. Available:geology.usgs.gov/tools/ metadata/tools/doc/ctc/. 5. International Conference and Harmonisation. Available: www.ich.org. 6. International Conference and Harmonisation, Organization of the Common Technical Document for the Registration of Pharmaceuticals for Human Uses M4, 2004. Available: www.ich.org/LOB/media/MEDIA554.pdf. 7. Providing Regulatory Submissions in Electronic Format — NDAs, FDA Guidance, 1999. 8. D. H. Christiansen and W. Kubick, CDISC Submission Metadata Model, 2001. Available: www.cdisc.org/standards/SubmissionMetadata ModelV2.pdf. 9. FDA Study Data Specification, 2006. Available: www.fda.gov/cder/regulatory/ersr/Studydatav1.3.pdf. 10. CDISC Study Data Tabulation Model Version 1.1, 2005. Available: www.cdisc.org/models/sds/ v3.1/index.html. 11. CDISC Analysis Data Model: Version 2.0, 2006. Available: www.cdisc.org/pdf/ADaMdocument v2.0 2 Final 2006-08-24.pdf. 12. FDA Future Guidance List, p. 2. Available: www.fda.gov/cder/guidance/CY06.pdf. 13. Providing Regulatory Submissions in Electronic Format — Human Pharmaceutical Product Applications and Related Submissions Using the eCTD Specifications, FDA Guidance, 2006. Available: www.fda.gov/cder/ guidance/7087rev.pdf.

10

METADATA

14. SAS 9.1.3 Language Reference: Concepts. Cary, NC: SAS Institute Inc., 2005, pp. 475– 476. 15. JANUS Project Description, NCI Clinical Research Information Exchange, 2006. Available: crix.nci.nih.gov/projects/janus/. 16. D. H. Christiansen and S. E. Wilson, Submission of analysis datasets and documentation: scientific and regulatory perspectives. PharmaSUG 2004 Conference Proc., San Diego, CA, 2004: Paper FC04, p. 3. 17. S. E. Wilson, Submission of analysis datasets and documentation: regulatory perspectives. PharmaSUG 2004 Conference Proc., San Diego, CA, 2004. 18. W. C. Smith, Correction of data errors in a large collaborative health study. Joint Statistical Meetings presentation, Atlanta, GA, 1975. 19. SAS/ 9.1.3 XML LIBNAME Engine: User’s Guide. Cary, NC: SAS Institute Inc., 2004. 20. S. E. Wilson, Clinical data quality: a regulator’s perspective. DIA 38th Annual Meeting presentation, Chicago, IL, 2002: 20–21. Available: www.fda.gov/cder/present/DIA62002/default. htm. 21. D. H. Christiansen and S. E. Wilson, submission of analysis datasets and documentation: scientific and regulatory perspectives. PharmaSUG 2004 Conference Proc., San Diego, CA, 2004: Paper FC04, p. 5. 22. CDISC Analysis Data Model: Version 2.0, 2006, p. 22. Available: www.cdisc.org/pdf/ ADaMdocument v2.0 2 Final 2006-08-24. pdf. 23. ICH M2 EWG Electronic Common Technical Document Specification V 3.2, 2004, p. 1. Available: http://estri.ich.org/eCTD/eCTD Specification v3 2.pdf. 24. World Wide Web Consortium (W3C) Extensible Markup Language (XML). Available: http://www.w3.org/XML/.

25. ICH M2 EWG Electronic Common Technical Document Specification V 3.2, 2004: Appendix 1, p. 1-1. Available: http://estri.ich. org/eCTD/eCTD Specification v3 2.pdf. 26. ICH Structure and Content of Clinical Study Reports E3, p. 29. Available: http:// www.ich.org/LOB/media/MEDIA479.pdf. 27. Providing Regulatory Submissions in Electronic Format — Human Pharmaceutical Product Applications and Related Submissions Using the eCTD Specifications, FDA Guidance, 2006. Available: http://www.fda.gov/cder/ guidance/7087rev.pdf. 28. FDA Study Specifications. Available: http:// www.fda.gov/cder/regulatory/ersr/Studydatav1.3.pdf. 29. CDISC Case Report Tabulation Data Definition Specification (define.xml). Available: http://www.cdisc.org/models/def/v1.0/CRT DDSpecification1 0 0.pdf. 30. SAS 9.1.3 XML LIBNAME Engine: User’s Guide. Cary NC: SAS Institute Inc., 2004, pp. 27, 39–43.

CROSS-REFERENCES Biostatistics Statistical Analysis Plan Electronic Submission of NDA International Conference on Harmonisation (ICH) Good Programming Practice

METHODS FOR CONDUCT OF RIGOROUS GROUP-RANDOMIZATION

of GRTs. These challenges generally arise from factors pertaining to the group-directed intervention and the nature of the group. The group-targeted interventions in GRTs are commonly prevention interventions (disease prevention or risk-factor prevention), and they are commonly targeted at relatively healthy populations. These two features have big consequences for GRTs. First, the rate of endpoint (disease, or risk factor) occurrence is relatively low. As a result, GRTs generally require a moderate number of groups (10–50) and typically have large numbers of study participants (1,000–10,000 +). GRTs are also typically long (5–20 + years), which makes follow-up of study participants and maintaining research relationships with collaborating organizations especially challenging. Also, the large size and long duration typical of most GRTs means an expensive trial and thus a low tolerance for degradation of scientific rigor. In addition, the healthy nature of the study population in most GRTs means that (1) research participation may have low salience or importance to study participants, and (2) the intervention must be designed to be easy to do. As a result of (1), there are special challenges for achieving high response rates for endpoint data collection, and for intervention compliance among those in the experimental organizations. As a result of (2), intervention contamination in the control organizations can easily happen, and so avoiding it is a challenge. It is critical for scientific rigor that these challenges be overcome. To do so, the challenges must first be recognized. Then, appropriate methods for addressing them must be planned in advance. Finally, these methods must be meticulously implemented throughout the trial. For each of the four requirements for trial execution—maintaining the research collaboration, maintaining the randomized assignment, achieving high location rate, and achieving high response rate—this article summarizes the challenges and same principles for addressing them and provides

ARTHUR V. PETERSON Jr. Member, Public Health Sciences, Fred Hutchinson Cancer Research Center, and Professor of Biostatistics, University of Washington.

1

INTRODUCTION

Just as with trials randomized by the individual, the randomized assignment in group-randomized trials (GRTs) provides two invaluable advantages that enable unambiguous conclusions to be reached from the trial: avoidance of selection and accidental biases (and consequent biases in the reported effect sizes), and provision of a basis for the statistical inferences that draw conclusions from the study. Moreover, the basis for statistical inference that randomization provides does not rely on any distributional or other assumptions. To realize the invaluable advantages that randomization enables, careful attention to trial conduct is needed from beginning to end of the GRT. For example, both sample size determination in the design phase at the start of the GRT and choice of statistical methods in the analysis phase at the end of the GRT must account for the intraclass correlation and the imprecision of its estimate. Likewise, achieving the scientific rigor enabled by the randomization in GRTs requires attention during the period of trial execution. There are four key requirements: (1) maintaining research collaboration with each recruited group for the duration of the trial, (2) maintaining the randomization assignment of intervention or control condition for each group throughout the trial, (3) achieving a high location rate for study participants at endpoint, and (4) achieving a high response rate at endpoint data collection. Special challenges to meeting these requirements are inherent to the very nature

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

METHODS FOR CONDUCT OF RIGOROUS GROUP-RANDOMIZATION

examples of methods for applying these principles in the conduct of GRTs.

2 MAINTAINING THE RESEARCH COLLABORATIONS FOR THE DURATION OF THE TRIAL Maintaining the research collaborations with participating organizations is essential for maintaining scientific rigor. Because an organization’s dropping out of the study is not determined at random, but it is instead determined by selection due to circumstances, it introduces selection bias into the results and weakens the (randomization-generated) basis for statistical inference. Moreover, the consequences of one organization dropping out from a GRT are far more severe than one study participant dropping out from an individual-randomized trial. Because the number of randomized units in a GRT is small or moderate (10–50 organizations), compared with the typical 100–400 individuals in an individual-randomized trial, loss of even one organization can severely impact the integrity of the trial. Thus, strong efforts are needed throughout the trial to avoid organizational dropout and its associated degradation of scientific rigor. Presented in Table 1 are challenges inherent in GRTs for maintaining research collaboration with participating organizations and principles for addressing the challenges. Of special note is the need to minimize, by

design, the research burden on the organization. Making participation easy for the organization directly addresses challenge #1—participation in research is not a goal of collaborating organizations. It also makes it more possible to overcome challenges #2 and #4—changes in personnel and organizations’ priorities over time. Examples of methods for maintaining research collaboration include the following: (1) Learn about the organizations in advance; identify key stakeholders; and demonstrate knowledge and interest in stakeholders and in their organization, needs, priorities, and challenges. (2) Conduct well-organized, efficient, and highly motivating recruitment meeting(s) with key stakeholders in each collaborating organization. Not only does this meeting serve to motivate potential collaborators, but also it provides them with first-hand evidence that the research group is capable and competent. (3) Begin maintaining the collaboration during recruitment, for example, by emphasizing the research question and randomized design so that any decision to participate is an informed and lasting one, and by working as a team to prepare for collaboration in the event that the organization decides to join the study. (4) Make excellent customer service an integral part of all trial procedures. For example, respond quickly to organizations’ inquiries and requests, and do what you say you will do. (5) Be visible: Communicate regularly with collaborating organizations to update

Table 1. Maintenance of Research Collaborations with Participating Organizations Requirement: Maintain the research collaborations (support for all research activities) with all participating organizations for the duration of the trial Challenges 1. Participation in research is not a goal of collaborating organizations, which are busy with their own priorities and mandates. 2. Organizations have multiple stakeholders to be kept informed and satisfied. 3. Organizations’ priorities or circumstances change over time. 4. Turnover of personnel within the collaborating organizations over time.

Principles 1. Design the trial and activities to minimize the research burden on the organization. 2. Sell the research. 3. Emphasize the essential role of the organizations as research partners. 4. Meet their needs to be informed, motivated, valued, and to have expectations met. 5. Make maintaining the research collaborations the top priority for the research group.

METHODS FOR CONDUCT OF RIGOROUS GROUP-RANDOMIZATION

progress, to alert them to upcoming activities, and to express appreciation for their support. Offer to present in-person to key stakeholders on the trial and its progress. (6) Incorporate the principles of maintaining collaborative relationships into all project activities and protocols. (7) Identify and develop simple, low-cost procedures, such as periodic progress-report letters and Secretary Day cards, that serve the sole purpose of building and maintaining collaborative relationships. (8) Coordinate and document all contacts with collaborating organizations. 3 MAINTAINING THE RANDOMIZED ASSIGNMENT The essential scientific advantages that randomization enables—avoidance of bias and a basis for statistical inference—depend on maintaining each organization’s randomized assignment to either experimental or control. Thus, strong efforts are warranted throughout the trial to minimize at all times the risk of straying from the randomized assignment. Conformance to the randomized assignment could be violated by (1) an organization’s nonacceptance, or poor understanding, of the concept of randomized assignment at the start of the collaboration, during recruitment; (2) an organization’s nonacceptance of the actual assignment when randomization is performed and the assignment is communicated; (3) intervention contamination in organizations assigned to the control condition; and (4) implementation noncompliance in organizations assigned to the experimental condition. Accordingly, maintaining the randomized assignment requires attention to each aspect. Presented in Table 2 are, for each of these four aspects, the challenges inherent in GRTs for maintaining the randomization and the principles for addressing the challenges. Of special note is the importance of learning about changes in key personnel, and motivating the new stakeholders about the crucial role of randomization. A change in key personnel could conceivably cause challenges to occur in all four aspects of maintaining the randomized assignment. GRT managers must be proactive to overcome such challenges.

3

Examples of methods for maintaining randomized assignment for the duration of the trial include (1) during recruitment, sell the randomized assignment as the crux of the experimental design and the key to the research’s ability to attain scientific rigor and unambiguous conclusions at the trial’s end. (For example, ‘‘Because of the randomized assignment, any difference in outcome between the control and experimental groups can be attributed to the intervention, and not to other factors that differ among participating organizations.’’) (2) Have the principal investigator be the one to notify the organization, so that the importance of the randomization is emphasized. (3) Emphasize at every verbal and written contact the importance of the research and the critical role of the randomized assignment in obtaining unambiguous study conclusions. (4) Control access to intervention materials. (5) Make provision whenever practical for control organizations to receive intervention materials after the trial is finished. (6) Avoid choosing as the experimental unit entities that communicate easily with each other, for example, classrooms within a school. (7) In all communications with key personnel in control organizations, reinforce the essential role of control organizations in the research and express gratitude for their support of this often less-than-glamorous role. (8) Try to design the intervention to also meet the existing goals of the organization. (9) Incorporate behavior change theory and principles into the provider training to motivate providers to want to implement the intervention. (10) Meet with implementers periodically to provide visible support and appreciation of their key role and to obtain their critical feedback on the quality of the intervention and its implementation. (11) Publicly acknowledge and thank implementers for their contributions. (12) Use both provider self-reports and project staff observations to monitor provider implementation. 4

LOCATING STUDY PARTICIPANTS

Because losses to follow-up (failure to obtain endpoint data from individual study participants in participating organizations) are

4

METHODS FOR CONDUCT OF RIGOROUS GROUP-RANDOMIZATION

Table 2. Maintenance of the Randomized Assignment Requirement: Maintain the randomized assignment of experimental/control condition for each participating organization for the duration of the study Challenges

Principles

Aspect #1: Acceptance (at recruitment) of the concept of randomized assignment 1. Randomization is not the usual way organizations make decisions. 2. Each organization tends to have its own preference/hope for the experimental condition that would be best for them. 3. Benefits of the control condition may not be clearly evident to participating organizations. 4. Changes in organizations and key personnel may threaten the understanding and/or support of the randomized assignment.

1. Motivate randomization as the key element for the study’s scientific rigor. 2. Emphasize that it is unknown whether the intervention is effective or not; indeed, the whole purpose of the trial is to find that out. 3. Emphasize importance of both intervention and control organizations to the success of the research. 4. Learn about changes in key personnel and inform them promptly about the trial.

Aspect #2: Acceptance (at randomization) of the communication of the actual randomized assignment 1. Organization stakeholders of the randomization may anticipate the randomization result. 2. Changing personnel and circumstances could heighten preference.

5. Make the performance of the randomization a big deal (it is), and witnessed. 6. Inform organization immediately. 7. Apply principles 1-4 above, again.

Aspect #3: Avoidance of contamination in control organizations 1. Participation in study, even as control, may enhance interest in the scientific question, and hence the intervention. 2. Prevention interventions are necessarily easy to do, and so may be easy for control to adopt.

8. Minimize the opportunity for purloining the intervention. 9. Reinforce the essential role of control organizations in the research.

Aspect #4: Implementation compliance in intervention organizations 1. Implementers within the organizations may be reluctant to add to their existing activities. 2. Intervention activities may mean some extra work, often unrewarded by organizations.

selected (not random), each loss degrades the scientific integrity of the study: It introduces bias into the results and degrades the basis for inference that the randomization enabled. Thus, to maximize the scientific

10. Make the intervention easy to do, enjoyable, and useful to them. 11. Motivate and train providers to implement the intervention. 12. Maintain strong, collaborative relationships with implementers as research partners.

rigor of the study, informed and aggressive measures must be taken to minimize loss to follow-up. Because successfully following up a study participant requires both locating the study

METHODS FOR CONDUCT OF RIGOROUS GROUP-RANDOMIZATION

participant and, once located, obtaining the endpoint data (compliant response to the endpoint survey), both need to be addressed and successfully accomplished. The first requirement is covered in this section; the second in the next section. Successfully locating a study participant is defined as obtaining an up-to-date address and/or telephone number (where the study participant can be contacted for collecting the trial’s endpoint data). Presented in Table 3 are the challenges in locating study participants and principles for addressing the challenges. Examples of methods for applying these principles (see Refs. 1–3) include (1) Collect at the start of the trial the names, addresses, and telephone numbers of study participants, their parents, and close friends or relatives who would know their whereabouts if they moved away. (2) First activities for updating addresses: periodic ‘‘softtracking’’ letters sent to study participants (which ask for no reply from the participant) that use the U.S. Postal Service’s ‘‘Address Service Requested’’ endorsement. (3) Use the National Change of Address Service to obtain

5

new addresses of study participants and parents, close friends, and relatives. (4) Contact parents, friends, and relatives, as needed. (5) Keep detailed records of all directory information changes and all contact attempts: date and time, whom contacted, relationship to study participant, and response/information obtained. (6) Use Directory Assistance (from more than one telephone company). (7) Use online people-search engines on the Internet. (8) Use Telephone Append & Verify, a service arranged by the U.S. Postal Service in cooperation with telephone companies to provide phone numbers that go with specific addresses. (9) If all else fails, use publicly or privately available databases. Deserving special emphasis is method (4)—Contact parents, friends, and relatives. Although timeconsuming, it is the most likely to yield accurate information. 5 HIGH RESPONSE RATE TO ENDPOINT SURVEY Together with achieving high location rates, achieving high response rates to the endpoint survey is necessary for achieving successful follow-up of study participants. In contrast

Table 3. Locating Study Participants at Endpoint Requirement: Locate at endpoint study participants enrolled at the start Challenges 1. Long-term nature of GRTs (prevention studies, rare endpoints) requires follow-up over many years. 2. Our society is highly mobile. 3. Participation is not a high priority in the lives of study participants. 4. Organizations may not keep track of new addresses of those who leave. 5. The USPS will forward mail to a new address for only 6 months following a move, and will provide a new address to a sender for only 12 months. 6. Telephone companies provide new telephone numbers for only a limited period of time (6 months-1 year). 7. Our society is skeptical about contact from unknown callers.

Principles 1. Collect locator information at the outset of the trial. 2. Update locator information periodically throughout the trial. 3. Use a friendly (yet professional) approach with study participants at all times in all circumstances. 4. Keep record for each study participant of all tracking attempts. 5. Don’t limit the number of tracking attempts. 6. Use multiple modes (mail, telephone, email, cell phones).

6

METHODS FOR CONDUCT OF RIGOROUS GROUP-RANDOMIZATION

to the other requirements for the conduct of GRTs, the challenges for achieving high response rates are common to both grouprandomized trials and individual-randomized trials. The challenges, and principles and methods for addressing them, have been well established over many years [see Don Dillman’s useful text (4)]. Shown in Table 4 are the challenges inherent in GRTs and other trials for achieving high response rates for the endpoint survey, and principles for addressing these challenges. Some examples of methods for achieving these follow. To build trust, when contacting participants, always (1) provide name of research organization, (2) remind the participant about his/her past participation, and (3) assure participants that their participation in the survey is voluntary and confidential. To make surveys easy to do, (4) keep the questionnaire short 50% of all human genes are subjected to alternative splicing (37). Although the machinery of splicing has been well known, how splice sites are selected to generate alternative transcripts remains poorly understood. For most alternative splicing events, their functional significance remains to be determined. In recent years, many researchers are characterizing alternative splicing using microarrays with probes from exons, introns, and intron–exon junctions. The basic idea is that the discordance among multiple probes

from the same gene indicates difference in transcripts themselves across samples. For example, skipping of an exon in one sample will cause the probes that hybridize to the skipped exon or the related exon–intron junctions to show different fold change compared with other probes from the same gene. Hu et al. (38) used a custom oligo array of 1600 rat genes with 20 pairs of probes for each gene to survey various rat tissues. A total of 268 genes were detected to have alternative splicing, and 50% of them were confirmed by RT-PCR. Clark et al. (39) monitored the global RNA splicing in mutants of yeast RNA processing factors using exon, intron, and exon-junction probes. Two of 18 mutants tested were confirmed by RT-PCR. In addition, they could cluster the mutants based on the indices for intron exclusion and junction formation to infer the function of associated mutant factors. In humans, Johnson et al. (40) analyzed the global alternative splicing of 10,000 multiexon genes in 52 normal tissues using probes from every exon–exon junction and estimated that at least 74% of human multiexon genes are alternatively spliced. They validated a random sample of 153 positives using RT-PCR and successfully confirmed 73. Other similar experiments were also conducted in humans to investigate alternative splicing in tumor cell line (41) and various tissues (42) but on a smaller scale. 3.3 Exon Array Most current designs of short oligo arrays have probes concentrated at the 3
end of genes because of the better EST support and commonly used reverse transcription methods starting from the 3
end. For sequenced genomes, the gene structure is annotated. Every exon instead of the 3
end can be represented on the array, which is called exon array. Exon array has a better representation of the genes. It provides not only information on the gene expression level but also an opportunity to detect alternative splicing by examining the relative expression of the exons across samples. If the relative expression of different exons of the same gene does not agree, then alternative splicing is indicated. It can detect some types of alternative

MICROARRAY

splicing such as exon skipping, but it is not sensitive to some types of alternative splicing, such as intron retention. In addition to detecting gene expression and alternative splicing, exon arrays can also be used to detect gene copy number difference through comparative genomic hybridization (43).

3.4 Tiling Array—Including Methylation Arrays The current knowledge of genes and gene expression is mainly based on the study of expressed mRNA and computational analysis of the EST collections. The full genome sequences of several organisms and the advance of microarray technology provides a mean, which are also called tiling arrays, to survey the whole genome for gene expression (44). The tiling arrays contain probes that cover the whole genome (or a few chromosomes) in an overlap, head-to-tail, or with small gaps. In theory, the expression of any region in the genome can be read out from the expression intensity of the probes. Tiling arrays have been developed for several organisms, including human (45,46), Drosophila (47), Arabidopsis (48,49), and rice (50). The results from the tiling experiments showed that tiling arrays can detect much more expressed regions in the genome than what have been known; however, because of the lack of reproducibility, many false positives may be observed (51,52). In addition, tiling arrays can be used to survey the genome for copy numbers using DNA instead of mRNA as hybridization target (53). The similar idea was also used to detect the epigenetic modification of the genome, DNA methylation (54,55), and histone acetylation (56). For detecting DNA methylations, arrays contain probes that correspond to each fragment of the methylation sensitive restriction enzymes. After digestion with these enzymes, the different fragments from hypermethylated or hypomethylated regions are enriched by specific adapters and PCR schemes. The enriched and samples are then hybridized to tiling arrays or arrays that cover just GpC islands in the genome (57).

5

3.5 SNP Chip A major task in dissecting the genetic factors that affect complex human disease is genotyping each individual in the study population. The most common polymorphism across individuals is the single nucleotide polymorphism (SNP). Special microarrays are specially designed for genotyping SNPs, such as Affymetrix 10k, 100k, 500k SNP chips, and Illumina SNP chips. These chips have probes centered on the SNP site and have perfect match probes for each allele as well as corresponding mismatch probes (58–60). Each array platform can genotype 10,000 or 500,000 SNP in a single hybridization. Although each hybridization is relatively expensive, the cost per genotype is low (61,62). 3.6 ChIP-on-Chip Microarrays can also be used to characterize the bindings of protein to DNA in defining gene regulation elements in the genome. The DNA sequences with proteins bound are enriched using chromatin immunoprecipitation (ChIP) and are then compared with regular DNA samples to detect the enrichment of some DNA elements. These DNA elements are associated with protein binding and tend to be involved in regulation of gene expression and other features related to the binding protein. The microarrays used in ChIP-chip studies are genomic arrays such as tiling arrays. However, because of the large number of arrays that it takes to cover the genome, some special arrays are designed to cover certain regions of the genome, such as the promoter regions and arrays, which use DNA from large probes (several kb or mb BAC clones). However, the resolution of the protein binding position decreases as the probe size increases. For reviews, see References 63–65. 3.7 Protein Arrays So far, the most application of microarrays in biological research are DNA arrays that analyze mRNA expression and DNA genotyping. However, the functional units for most genes are proteins instead of mRNA, which is just a messenger. Researchers have put forth a

6

MICROARRAY

large effort to increase the throughput for characterizing the assays with protein, such as the proteome technologies. Considerable efforts have been spent in developing protein arrays; however, because of the challenging nature of protein, there has been less success in producing and applying protein arrays. Nonetheless, the research and development of protein arrays is still an active field with the hope of similar success as DNA microarrays in the near future (66,67). Based on the specific use, two major types of protein arrays are available. One type is analytical array, in which antibodies are arrayed on solid surface to detect the corresponding proteins in the hybridization soup (68–70). Another type is more functional oriented to detect protein–protein, protein–DNA, and protein–small molecule interaction. For the protein–protein interaction arrays, proteins are arrayed on the surface, and they can interact and bind to their interaction partners in the hybridization solution (71,72). Protein–DNA interaction arrays are used for detecting the binding sites of some proteins in the genome (73,74). The protein–small molecule interaction arrays are used to identify substrates for protein (75), drug targets (76), and immune response to proteins (77–79). 4

DEFINE OBJECTIVES OF THE STUDY

In the next few sections, we will discuss the steps of a microarray study and what an investigator needs to be aware of before, during, and after starting a clinical trial with microarrays. The first step of any experiment, microarray or otherwise, is to define a hypothesis. Although it may seem perfectly obvious to state this, the objectives of a study are likely to be met if that study is designed in such a way that it is consistent with meeting those objectives, and a study can only be designed to meet the objectives if the objectives can be articulated clearly prior to initiating the study. It is not uncommon for investigators who have conducted a microarray study to be unable to state the objective(s) of their study clearly (e.g., to identify genes that are differentially expressed in

response to a treatment or genes that may be responsible for differential response to treatments). Often, investigators have inadvertently allowed themselves to go down the slippery slope from hypothesis-generating studies to studies that might be termed ‘‘objective generating.’’ By objective generating, we mean studies in which the investigator has initiated a microarray study without a clear conception of the exact objectives of the study in the sole hope that, by some mysterious process, the mass of data from the microarray study will make all things clear. We do not believe that such outcomes are likely. However, that is not to say that the experimental objectives may not include the generation of new objectives/hypotheses on interesting pathways/genes that may be highlighted as a result of the microarray study or be very broadly defined. Thus, we urge investigators to articulate clearly what they hope to obtain from microarray studies so that the studies can be designed to meet those objectives from the beginning; in other words, researchers should make a hypothesis. 5 EXPERIMENTAL DESIGN FOR MICROARRAY After stating a hypothesis, the most important step of an experiment is the experimental design. If an experiment is designed well, then the choice of analytical methods will be defined and the analysis plan will follow. Several essential principles must be in mind when designing a microarray experiment. These include avoidance of experimental artifact; elimination of bias via use of a simultaneous control group; randomization and (potentially) blinding; and reduction of sampling error via use of replication, balance design, and (where appropriate) blocking. 5.1 Avoidance of Experimental Artifacts Microarrays and most laboratory techniques are liable to nonbiological sources of variation including day, chip lot, reagent lot, day of extraction, clinic samples came from and personnel (post doc effects). In many cases, these sources of variation are larger than

MICROARRAY

the biological variation (80–84). If nonbiological and biological differences are confounded, then the experiment can be essentially meaningless. Thus, careful consideration and identification of all factors must be taken before starting a study. These factors must then be eliminated by making the experimental conduct more homogeneous or controlling by randomization and blocking. 5.2 Randomization, Blocking, and Blinding Although blocking can be used to control for measured or known confounding factor, such as the number of samples that can be run in a day, randomization of varieties/samples/groups and random sampling of populations is very useful for reducing the effect of unmeasured confounding factors (81,85,86), such as difference in weather and interruptions in sample delivery. Microarray experiments can require multiple levels of randomization and blocking to help minimize unanticipated biases from known factors. For example, if only four samples can be processed in a day and there are two experimental groups, then two samples from each treatment group can be run each day (blocking), but the two samples are randomly selected from all samples in a the experimental group. Proper randomization and blocking can greatly reduce the bias of studies. Blinding should, of course, be a part of the conduct of any clinical trial, but blinding may also be appropriate on array analysis at the sample collection and processing steps. Unintentional biases can be introduced by individuals collecting samples; for example, margins may be cleaned more carefully in one group compared to another. In addition, more care may be paid to the processing of one treatment group over another. All of these may cause bias, and if possible blinding should be used in microarray experiments. 5.3 Replication Individuals within a population vary, and all measurement tools such as microarrays measure with some error; thus, a single sample cannot be used to make generalizable inferences about a group or population. Replication of microarrays is needed at several levels.

7

5.3.1 Types of Replication. Replication in the context of microarrays can be incorporated at several levels: (R1) gene-to-gene; genes can be spotted multiple times per array, (R2) array-to-array; mRNA samples can be used on multiple arrays and each array is hybridized separately, and (R3) subject-tosubject; mRNA samples can be taken from multiple individuals to account for inherent biological variability. The first two types of replication are referred to as technical replication. The first type measures withinarray variation whereas the second type measures between-array variation. These types of replication are important for assessing the measurement error and reproducibility of microarray experiments and are extremely useful for improving precision of measurements. On the other hand, the third type of replication allows us to assess the biological variation within populations and thereby to make biologically interesting observations. R1 Technical replicates can not substitute for biological replicates (R3). Although R2 technical replicates have a specific role when the cost of samples is far larger than arrays, an experiment cannot be run only with R2 replicates and biological generalizable results cannot be obtained (87). 5.3.2 Replication, Power, and Sample Size. Sample size has a major impact on how confidently genes can be declared either differentially (sensitivity and power) or not differentially expressed (80,88) (specificity). Sample sizes can be determined in a variety of ways. One way is traditional statistical power analysis programs such as PS, which contains the following: power (1-beta), significance (alpha), a measure of variation (Standard deviation), and a detectable difference (delta). As an example at 80% power at a Bonferroni corrected significance level α = 0.05 to detect a 1/2 standard deviation (SD) reduction requires a sample size of over 250 per group, which is not normally achievable in microarray experimentations for budgetary reasons. Another approach we believe is more appropriate would be to choose sample size based on control of the false discovery rate (FDR) (89) and the expected discovery rate (EDR). The FDR is an estimate of the expected proportion of

8

MICROARRAY

genes declared significant that are in fact not differentially expressed [i.e., that are ‘‘false discoveries’’ (90,91)]. The EDR is the expected proportion of genes in which true differences between conditions exist that are found to be significantly different. This approach has been developed and applied in the PowerAtlas (92–94) (www.poweratlas.org). In addition, the PowerAtlas allows an investigator to either upload their own pilot data or to choose from among over 1000 public microarray experiments to use as pilot data for estimating sample size. 5.4 Practice, Practice, Practice No experimenter runs every step of a microarray experiment perfectly the first time. A learning curve is observed for all steps, and the learning process is a confounding factor. Training at all steps is necessary from sample collection to RNA processing, hybridization, and analysis. Thus, all the individual steps should be practiced before running an experiment, and new people who will handle samples need to be trained to sufficient standards before they run ‘‘real’’ samples. Resources spent on training are not wasted, and training samples should not be included in a ‘‘real’’ experiment. 5.5 Strict Experimental Practices Because microarray experiments are liable to many nonbiological sources of error, it is critical to conduct microarray studies that follow very strictly defined protocols. For example, know exactly what types of samples are and are not acceptable, what cut of the samples is needed, what protocol will be used to extract samples, what analyses will be used, what is good quality RNA, and so on before a study is started. Consider a microarray study like a clinical trial in which the researcher must perform a full disclosure of all steps before starting a clinical trial. Deviations from these protocols are strongly discouraged for fear of introducing biases. 6

DATA EXTRACTION

Once a microarray experiment has been conducted and the image of an array is obtained,

several steps must occur to convert the image to analyzable data, and the methods are specific to each technology. 6.1 Image Processing from cDNA and Long Oligo Arrays Image processing consists of three steps: addressing, segmentation, and information extraction. 6.1.1 Gridding/Addressing. Typically, a microarray is an ordered array of spots with constant separation between the row and column; grids or spots must be the same throughout the microarray. Addressing is the process of finding the location of spots on the microarray or assigning coordinates to each spot on the microarray. However, the spotting is rarely perfect; variations must be dealt with. Although software usually does a good job, manual review and occasional intervention results in better data, but it is a very time-consuming process. 6.1.2 Segmentation. Segmentation is the most important and also the most difficult part of the image analysis. In this process, each image pixel is classified as either signal or the background noise. The popular methods of segmentation used fixed circles, adaptive circles, adaptive shapes, or the histogram method. The first two methods provide the means to separate the circular spots from the background by clearly defining the boundaries of the spots. A variety of comparisons of the methods has been published with no clear winner (95,96). 6.1.3 Information Extraction. In the final step of image analysis, the mean and median values of the spot intensities and the background spot intensities are calculated. Usually, correlations between spot intensities, percentage of the spots without any signal, their distribution and signal-to-noise ratio (SNR) for each spot, and variation of the pixel intensities are also calculated. The spot intensities are then measured as the sum of intensities of all the pixels inside the spot.

MICROARRAY

6.2 Image Analysis of Affymetrix GeneChip Microarrays Affymetrix GeneChips are the most widely used oligonucleotide arrays. Unlike the other systems, the Affymetrix system represents each gene as 11–20 probe pairs. Each probe pair is composed of a 25 base pair perfect match (PM) probe that represents the gene’s known sequence and a mismatch (MM) probe that differs from the PM by the middle base. The expression level is some function of the averages of difference in intensities of PM and MM over the 20 sequences. Several algorithms have been developed for averaging the probe pairs to yield a final quantification. These include Dchip (97), GCRMA-EB and GCRMA-MLE (98), MAS5 (99), PDNN (100), and RMA (101,102). all of which have different measurement properties, and it is not yet clear which is best (103). Other technologies such as Illumina and NimbleGen have their own image analysis steps as well. 6.3 Normalization of DNA Data One of the early and near universal steps in microarray study is the use of a technique called either normalization or transformation. The normalization has at least two purposes: to adjust microarray data for effects that develop from variation in the technology rather than from biological differences between the RNA samples or between the printed probes (104) and ‘‘aiding in the analysis of the data by bending the data nearer the Procrustean bed of the assumptions underlying conventional analyses’’ (105), which will allow for reliable statistical and biological analyses. The former is really more adjusting for measured covariates such as dye biases whereas the later is the true meaning of normalization. Either way, a wide variety of methods has been developed for all meanings of normalization including several varieties of linear models (106), loess (104), quantilequantle (107), log2 , and others (108,109). Normalization is usually required in cDNA microarray experiments to reduce dye biases. This area still requires active research, and it is not clear which methods are appropriate for each chip and experimental design.

7

9

MICROARRAY INFORMATICS

For many investigators, microarrays will involve the highest data storage, analysis, and informatics hurdle they will face in their research careers. Microarrays generate huge amounts of data, which can make data storage and handling difficult. In addition, data reporting standards are being enforced by many journals for publications (110,111), and the NIH has started to make data sharing mandatory for certain grants. Microarray experiments generate volumes of data that many biological researchers may not be accustomed to. A single Affymetrix chip will generate about 50 MB of data. After initial processing, each chip will provide thousands to tens of thousands of numbers per array for analysis. After analysis, summary statistics, such as changes in expression and associated significance probabilities, will be available for all genes on the chips. Sorting through significance tests for tens of thousands of genes ‘‘manually’’ and trying to deduce a biological meaning is a Sisyphusian task because of the dimensionality of the data the speed at which new information is generated. These data can be overwhelming. Before an experiment has begun, consideration should be paid to how data are stored, viewed, and interpreted (112). 8

STATISTICAL ANALYSIS

Three types of single gene analyses are typically conducted on microarray data. Class prediction involves building models to predict which group to which samples should be assigned. This method is often used in clinical trials, for example, to develop profiles that predict poor prognosis of cancer (21) or to differentiate among pathologically similar samples (113). The second set of analyses is class discovery, which involves the unsupervised analysis of data to identify previously unknown relationships between genes or samples. The final type is class differentiation, which usually involves the inferential statistical analysis. 8.1 Class Prediction Analysis We use the term prediction to define the construction of a model that uses gene expression

10

MICROARRAY

experiments to classify objects into preexisting known classes, to develop indexes that can serve as a biomarker, or to predict to which class a sample should be assigned (114,115). Many methods can be used to construct such scores (116–118), and it is not clear which technique is best, but the goal of all methods is to find the best compromise between complexity and simplicity in the model. As the predicted model becomes more and more complex by using more and more sample information, the predicted ability in the sample in hand will increase; however, the sample data contain not only information about the true structure of the data but also it contains ‘‘noise’’ because of sample variation. Thus, great care must be taken in the model building (119–121). To build the models that will predict new samples well, one must build cross-validated models. Crossvalidating requires that one have sufficient data to hold some models from the estimation process so that one can subsequently check how well the prediction is using the held back data. For cross validation to be accurate, the held back data used in the cross validation must have not been used in the selection of the structure of the model used for prediction or which parameters go into that model (122,123). This validation has often been violated in microarray research. 8.2 Class Discovery Analysis Since Eisen et al. (4) first applied hierarchical clustering to microarray data analysis in 1998, cluster analysis has emerged as a prevalent tool for the exploration and visualization of microarray data. A variety of cluster methods are available, which include hierarchical and nonhierarchical methods. Among hierarchical methods are agglomerative and divisive methods. For the hierarchical methods, different ways can be used to measure distance between genes, which includes Person’s correlation, Euclidian distance, and Kendall’s Tau as well as a variety of methods for linking genes based on their distance including average, single, complete, and median. Several nonhierarchical methods exist including K-nearest neighbors, selforganizing Maps and related techniques such

a support vector machine and singular value decomposition. Each method and approach has its own positive and negative aspects that should be evaluated. Clustering is a commonly used tool for microarray data analysis; but unlike other statistical methods, no theoretical foundations provide the correct answer. This problem leads directly to several related criticisms of cluster analysis. First, the cluster algorithms are guaranteed to produce clusters from data, no matter what kind of data has been used. Two, different methods can produce drastically different results, and the search for the best choice among them has just begun (124,125). Third, no valid method is available to establish the number of clusters in nonhierarchical cluster analysis. Therefore, cautions are required for performing such analysis, and one should avoid overinterpreting the results; however, cluster analysis is good to provide exploratory descriptive analysis and concise displays of complex data. 8.3 Class Differentiation Analysis One of the main tasks in analyzing microarray data is to determine which genes are differentially expressed between two or more groups of samples. This type of analysis is a conventional hypothesis test. For making inference, virtually any statistical method can be used including t-test, analysis of variance, and linear models. A variety of Bayesian methods and information borrowing approaches such as Cyber-t (126) and SAM (90) has been developed. Because of the small sample sizes, it is often useful to employ variance shrinkage based methods for more robust estimation (127,128). 8.3.1 Adjusting for Multiple Testing. Because each microarray can contain thousands of genes, some adjustment for multiple testing is required to avoid many false positive results. One way is to control the family-wise error rate (FWE), which is the probability of wrongly declaring at least one gene as differentially expressed. A Bonferroni correction is a method to adjust P-values from independent tests. Permutation methods can be used to control FWE in presence of nonindependent tests (129). Another approach

MICROARRAY

to address the problem of multiple testing is the false discovery rate (FDR), which is the proportion of false positives among all genes initially identified as being differentially expressed (89,130). In addition, a variety of Bayesian and alternative FDR methods has been developed (91). 9

ANNOTATION

The gene-by-gene statistical analysis of microarray data is not the end of a study by any stretch of the imagination. The next step is the annotation of the expression data. The amount of information about the functions of genes is beyond what any one person can know. Consequently, it is useful to pull in information on what others have discovered about genes to interpret an expression study fully and correctly. A variety of tools such as array manufacturer’s web sites, KEGG (Kyoto Encyclopedia of Genes and Genomes)(131,132), Gene Index(133,134), Entrez Gene, Medminer (135), DAVID (Database for Annotation, Visualization and Integrated Discovery), and Gene Ontology (136). Each database and tool has slightly different data, and one should use multiple databases when annotating. Also be aware databases can be different with respect to the same information. 10 PATHWAY, GO, AND CLASS-LEVEL ANALYSIS TOOLS Analysis of microarray experiments should not stop at a single gene, but rather several approaches can be used to get a picture beyond a single gene. These tools are called by a variety of names including pathway analysis, gene class testing, global testing, entrez testing, or GO (Gene Ontology analysis). The goal of all these tools is to relate the expression data to other attributes such as cellular localization, biological process, molecular function or a pathway for individual genes or groups of related genes. The most common way to analyze a gene list functionally is to gather information from the literature or from databases that cover the whole genome. In recent years, many

11

tools have been developed to assess the statistical significance of association of a list of genes with GO annotations terms and new ones are released regularly (137). Extensive discussion has occurred of the most appropriate methods for the class level analysis of microarray data (138–140). The methods and tools are based on different methodological assumptions. Two key points must be considered: (1) does the method use gene sampling or subject sampling and (2) whether the method use competitive or self-contained procedures. The subject sampling methods are preferred, and the competitive versus selfcontained debate continues. Gene sampling methods base their calculation of the P-value for the gene set on a distribution in which the gene is the unit of sampling, whereas the subject sampling methods take the subject as the sampling unit. The latter, which is based on the subjects not the genes, is typically the unit of randomization in a study (141–143). Competitive tests, which encompass most existing tools, test whether a gene class, defined by a specific GO term or pathway or similar, is overrepresented in the list of genes differentially expressed compared to a reference set of genes. A self-contained test compares the gene set with a fixed standard that does not depend on the measurements of genes outside the gene set. Goeman et al. (144,145), Mansmann and Meister (141) and Tomfohr et al. (143) applied the selfcontained methods. These methods are also implemented in SAFE and Globaltest. Another important aspect of ontological analysis regardless of the tool or statistical method is the choice of the reference gene list against which the list of differentially regulated genes is compared. Inappropriate choice of reference genes may lead to false functional characterization of the differentiated gene list. Khatri and Draghici (146) pointed out that only the genes represented on the array should be used as reference list instead of the whole genome as it is a common practice. In addition, correct, up-to-date, and complete annotation of genes with GO terms is critical; the competitive and gene samplebased procedures tend to have better and more complete databases. GO allows annotation of genes at different levels of abstraction due to the directed acyclic graph structure

12

MICROARRAY

of the GO. In this hierarchical structure, each term can have one or more child terms as well as one or more parent terms. For instance, the same gene list is annotated with a more general GO term such as ‘‘cell communication’’ at a higher level of abstraction, whereas the lowest level provides a more specific ontology term such as ‘‘intracellular signaling cascade.’’ It is important to integrate the hierarchical structure of the GO in the analysis because various levels of abstraction usually give different P-values. The large number (hundreds or thousands) of tests performed during ontological analysis may lead to spurious associations just by chance. Correction for multiple testing is a necessary step to take. Other analyses look beyond single genes such as coexpression (147), network analysis (148,149), and promoter and transcriptional regulations (150,151). 11 VALIDATION OF MICROARRAY EXPERIMENTS A plethora of factors, which include biological and technical factors, inherent characteristics of different array platforms, and processing and analytical steps can affect results of a typical microarray experiment (152). Thus, several journals now require some sort of validation for a paper to be published. Sometimes, it is possible to confirm the outcome without doing any additional laboratorybased analysis. For example, array results can be compared with information available in the literature or in expression databases such as GEO (153). However, such in silico validation is not always possible or appropriate. Thus, other techniques such as RT-PCR, SAGE (154), and proteomics are used. However, many studies merely conduct technical validation of microarray results. This method may have been appropriate before the result of MAQC established the validity of expression studies. Thus, in our opinion, if microarray studies are well planned, then a valid and technical validation of the array results is not needed, but rather the verification that investigators should pursue should advance their hypotheses rather than arbitrarily technical validation of certain genes.

12 CONCLUSIONS When coupled with good experiment design, a high-quality analysis, and thorough interpretation, microarray technology has matured to the point where it can generate incredibly valuable information. Microarrays can be used to provide greater understanding of the disease being studied, to develop profiles to predict response to compounds, and to predict side effects or poor outcomes. In the near future, microarrays may be used determine which treatments a person may best respond to. We hope this article will help investigators in their use of microarrays. 13 ACKNOWLEDGEMENT The contents here were developed over many years in discussion with many investigators at UAB and around the world including David Allison, Steve Barnes, Lang Chen, Jode Edwards, Gary Gadbury, Issa Coulibaly, Kyoungmi Kim, Tapan Mehta, Prinal Patal, Mahyar Sabripour, Jelai Wang, Hairong Wei, Richard Weindruch, Stanislav Zakharkin, and Kui Zhang. The work could not have been conducted without their thoughts and input. GPP and XQ were supported by NIH grants AT100949, AG020681, and ES 012933 and GPP by NSF grant 0501890. REFERENCES 1. M. Chee, R. Yang, E. Hubbell, A. Berno, Z. C. Hunag, D. Stern, J. Winkler, D. J. Lockhart, M. S. Morris, and S. P. A. Fodor, Accessing genetic information with high-density DNA arrays. Science 1996; 274: 610–614. 2. D. J. Lockhart, H. Ding, M. Byrne, M. T. Follettie, M.V. Gallo, M. A. S. Chee, M. Mittmann, C. Wang, M. Kobayashi, H. Horton, and E. L. Brown, Expression monitoring by hybridization to high-density oligonucleotide arrays. Nat. Biotech. 1996; 14: 1675–1680. 3. C.-K. Lee, R. G. Kloop, R. Weindruch, and T. A. Prolla, Gene expression profile of aghing and its restriction by caloric restriction. Science 1999; 285: 1390–1393. 4. M. B. Eisen, P.T. Spellman, P.O. Brown, and D. Botstein, Cluster analysis and display of genome-wide expression patterns.

MICROARRAY Proc. Natl. Acad. Sci. U.S.A. 1998; 95: 14863–14868. 5. C. M. Perou, S. S. Jeffrey, R. M. van de, C. A. Rees, M. B. Eisen, D. T. Ross, A. Pergamenschikov, C. F. Williams, S. X. Zhu, J. C. Lee, D. Lashkari, D. Shalon, P. O. Brown, and D. Botstein Distinctive gene expression patterns in human mammary epithelial cells and breast cancers. Proc. Natl. Acad. Sci. U.S.A. 1999; 96: 9212–9217. 6. M. A. Ginos, G. P. Page, B. S. Michalowicz, K. J. Patel, S. E. Volker, S. E. Pambuccian, F. G. Ondrey, G. L. Adams, and P. M. Gaffney, Identification of a gene expression signature associated with recurrent disease in squamous cell carcinoma of the head and neck. Cancer Res. 2004; 64: 55–63. 7. Y. Higami, T. D. Pugh, G. P. Page, D.B. Allison, T. A. Prolla, and R. Weindruch, Adipose tissue energy metabolism: altered gene expression profile of mice subjected to longterm caloric restriction. FASEB J. 2003; 8: 415–417. 8. S. O. Zakharkin, K. Kim, T. Mehta, L. Chen, S. Barnes, K.E. Scheirer, R. S. Parrish, D. B. Allison, and G. P. Page, Sources of variation in Affymetrix microarray experiments. BMC Bioinformat. 2005; 6: 214. 9. J. C. Lacal, How molecular biology can improve clinical management: the MammaPrint experience. Clin. Transl. Oncol. 2007; 9: 203. 10. S. Mook, L. J. van’t Veer, Rutgers E.J., Piccart-Gebhart M.J., and F. Cardoso, Individualization of therapy using Mammaprint: from development to the MINDACT Trial. Cancer Genom. Proteom. 2007; 4: 147–155. 11. J. Zhao, J. Roth, B. Bode-Lesniewska, M. Pfaltz, P.U. Heitz, and P. Komminoth, Combined comparative genomic hybridization and genomic microarray for detection of gene amplifications in pulmonary artery intimal sarcomas and adrenocortical tumors. Genes Chromos. Cancer 2002; 34: 48–57. 12. K. L. Gunderson, F. J. Steemers, G. Lee, L. G. Mendoza, and M. S. Chee, A genomewide scalable SNP genotyping assay using microarray technology. Nat. Genet. 2005; 37: 549–554. 13. L. Cekaite, O. Haug, O. Myklebost, M. Aldrin, B. Ostenstad, M. Holden, A. Frigessi, E. Hovig, and M. Sioud, Analysis of the humoral immune response to immunoselected phage-displayed peptides by a microarray-based method. Proteomics 2004; 4: 2572–2582.

13

14. C. Gulmann, D. Butler, E. Kay, A. Grace, and M. Leader, Biopsy of a biopsy: validation of immunoprofiling in gastric cancer biopsy tissue microarrays. Histopathology 2003; 42:70–6. 15. T. C. Mockler, S. Chan, A. Sundaresan, H. Chen, S. E. Jacobsen, and J. R. Ecker, Applications of DNA tiling arrays for wholegenome analysis. Genomics 2005; 85: 1–15. 16. V. K. Mootha, C. M. Lindgren, K. F. Eriksson, A. Subramanian, S. Sihag, J. Lehar, P. Puigserver, E. Carlsson, M. Ridderstrale, E. Laurila, N. Houstis, M. J. Daly, N. Patterson, J. P. Mesirov, T. R. Golub, P. Tamayo, B. Spiegelman, E. S. Lander, J. N. Hirschhorn, D. Altshuler, and L. C. Groop, PGC-1alpharesponsive genes involved in oxidative phosphorylation are coordinately downregulated in human diabetes. Nat. Genet. 2003; 34: 267–273. 17. L. Cabusora, E. Sutton, A. Fulmer, and C. V. Forst, Differential network expression during drug and stress response. Bioinformatics 2005; 21: 2898–2905. 18. J. M. Naciff, M. L. Jump, S. M. Torontali, G. J. Carr, J. P. Tiesman, G. J. Overmann, and G. P. Daston, Gene expression profile induced by 17alpha-ethynyl estradiol, bisphenol A, and genistein in the developing female reproductive system of the rat. Toxicol. Sci. 2002; 68: 184–199. 19. Y. Tang, D. L. Gilbert, T. A. Glauser, A. D. Hershey, and F. R. Sharp, Blood gene expression profiling of neurologic diseases: a pilot microarray study. Arch. Neurol. 2005; 62: 210–215. 20. J. P. Ioannidis, Microarrays and molecular research: noise discovery? Lancet 2005; 365: 454–455. 21. L. J. ’t Veer, H. Dai, M. J. van de Vijver, Y. D. He, A. A. Hart, M. Mao, H. L. Peterse, K. K. van der, M. J. Marton, A. T. Witteveen, G. J. Schreiber, R. M. Kerkhoven, C. Roberts, P. S. Linsley, R. Bernards, and S. H. Friend, Gene expression profiling predicts clinical outcome of breast cancer. Nature 2002; 415: 530–536. 22. M. J. van de Vijver, Y. D. He, L. J. van’t Veer, H. Dai, A. A. Hart, D. W. Voskuil, G. J. Schreiber, J. L. Peterse, C. Roberts, M. J. Marton, M. Parrish, D. Atsma, A. Witteveen, A. Glas, L. Delahaye, T. van der Velde, H. Bartelink, S. Rodenhuis, E. T. Rutgers, S. H. Friend, and R. Bernards, A gene-expression signature as a predictor of survival in breast cancer. N. Engl. J. Med. 2002; 347: 1999–2009.

14

MICROARRAY

23. Glas A. M., A. Floore, L. J. Delahaye, A. T. Witteveen, R. C. Pover, N. Bakx, J. S. LahtiDomenici, T. J. Bruinsma, M. O. Warmoes, R. Bernards, L. F. Wessels, and L. J. van’t Veer, Converting a breast cancer microarray signature into a high-throughput diagnostic test. BMC Genom. 2006; 7: 278. 24. M. Buyse, S. Loi, L. van’t Veer, G. Viale, M. Delorenzi, A. M. Glas, M. S. d’Assignies, J. Bergh, R. Lidereau, P. Ellis, A. Harris, J. Bogaerts, P. Therasse, A. Floore, M. Amakrane, F. Piette, E. Rutgers, C. Sotiriou, F. Cardoso, and M. J. Piccart, Validation and clinical utility of a 70-gene prognostic signature for women with node-negative breast cancer. J. Natl. Cancer Inst. 2006; 98: 1183–1192.

31. W. Tong, A. B. Lucas, R. Shippy, X. Fan, H. Fang, H. Hong, M. S. Orr, T. M. Chu, X. Guo, P. J. Collins, Y. A. Sun, S. J. Wang, W. Bao, R. D. Wolfinger, S. Shchegrova, L. Guo, J. A. Warrington, and L. Shi, Evaluation of external RNA controls for the assessment of microarray performance. Nat. Biotechnol. 2006; 24: 1132–1139. 32. J. O. Borevitz, D. Liang, D. Plouffe, H. S. Chang, T. Zhu, D. Weigel, C. C. Berry, E. Winzeler, and J. Chory, Large-scale identification of single-feature polymorphisms in complex genomes. Genome Res. 2003; 13: 513–523.

25. W. Zhang, I. Shmulevich, and J. Astola, Microarray Quality Control. 2004. John Wiley & sons, Inc., Hoboken, NJ.

33. X. Cui, J. Xu, R. Asghar, P. Condamine, J. T. Svensson, S. Wanamaker, N. Stein, M. Roose, and T. J. Close, Detecting singlefeature polymorphisms using oligonucleotide arrays and robustified projection pursuit. Bioinformatics 2005; 21: 3852–3858.

26. M. Schena, D. Shalon, R. W. Davis, and P. O. Brown, Quantitative monitoring of gene expression patterns with a complementary DNA microarray. Science 1995; 270: 467–470.

34. N. Rostoks, J. Borevitz, P. Hedley, J. Russell, S. Mudie, J. Morris, L. Cardle, D. Marshall, and R. Waugh, Single-feature polymorphism discovery in the barley transcriptome. Genome Biol. 2005; 6:R54.

27. P. K. Tan, T. J. Downey, E. L. Spitznagel Jr, P. Xu, D. Fu, D. S. Dimitrov, R. A. Lempicki, B. M. Raaka, and M. C. Cam, Evaluation of gene expression measurements from commercial microarray platforms. Nucleic Acids Res. 2003; 31: 5676–5684.

35. E. A. Winzeler, C. I. Castillo-Davis, G. Oshiro, D. Liang, D. R. Richards, Y. Zhou, and D. L. Hartl, Genetic diversity in yeast assessed with whole-genome oligonucleotide arrays. Genetics 2003; 163: 79–89.

28. T. A. Patterson, E. K. Lobenhofer, S. B. Fulmer-Smentek, P. J. Collins, T. M. Chu, W. Bao, H. Fang, E. S. Kawasaki, J. Hager, I. R. Tikhonova, S. J. Walker, L. Zhang, P. Hurban, F. de Longueville, J. C. Fuscoe, W. Tong, L. Shi, and R. D. Wolfinger, Performance comparison of one-color and two-color platforms within the Microarray Quality Control (MAQC) project. Nat. Biotechnol. 2006; 24: 1140–1150.

36. J. Ronald, J. M. Akey, J. Whittle, E. N. Smith, G. Yvert, and L. Kruglyak, Simultaneous genotyping, gene-expression measurement, and detection of allele-specific expression with oligonucleotide arrays. Genome Res. 2005; 15: 284–291. 37. P. A. Sharp, The discovery of split genes and RNA splicing. Trends Biochem. Sci. 2005; 30: 279–281. 38. G. K. Hu, S. J. Madore, B. Moldover, T. Jatkoe, D. Balaban, J. Thomas, and Y. Wang, Predicting splice variant from DNA chip expression data. Genome Res. 2001; 11: 1237–1245.

29. L. Shi, L. H. Reid, W. D. Jones, R. Shippy, J. A. Warrington, S. C. Baker, P. J. Collins, F. de Longueville, E. S. Kawasaki, and K. Y. Lee, The MicroArray Quality Control (MAQC) project shows inter- and intraplatform reproducibility of gene expression measurements. Nat. Biotechnol. 2006; 24: 1151–1161.

39. T. A. Clark, C. W. Sugnet, M. Ares Jr., Genomewide analysis of mRNA processing in yeast using splicing-specific microarrays. Science 2002; 296: 907–910.

30. R. Shippy, S. Fulmer-Smentek, R. V. Jensen, W. D. Jones, P. K. Wolber, C. D. Johnson, P. S. Pine, C. Boysen, X. Guo, E. Chudin, et al., Using RNA sample titrations to assess microarray platform performance and normalization techniques. Nat. Biotechnol. 2006; 24: 1123–1131.

40. J. M. Johnson, J. Castle, P. Garrett-Engele, Z. Kan, P. M. Loerch, C. D. Armour, R. Santos, E. E. Schadt, R. Stoughton, and D. D. Shoemaker, Genome-wide survey of human alternative pre-mRNA splicing with exon junction microarrays. Science 2003; 302: 2141–2144.

MICROARRAY 41. A. Relogio, C. Ben Dov, M. Baum, M. Ruggiu, C. Gemund, V. Benes, R. B. Darnell, and J. Valcarcel, Alternative splicing microarrays reveal functional expression of neuronspecific regulators in Hodgkin lymphoma cells. J. Biol. Chem. 2005; 280: 4779–4784. 42. K. Le, K. Mitsouras, M. Roy, Q. Wang, Q. Xu, S. F. Nelson, and C. Lee, Detecting tissue-specific regulation of alternative splicing as a qualitative change in microarray data. Nucleic Acids Res. 2004; 32:e180. 43. P. Dhami, A. J. Coffey, S. Abbs, J. R. Vermeesch, J. P. Dumanski, K. J. Woodward, R. M. Andrews, C. Langford, and D. Vetrie, Exon array CGH: detection of copy-number changes at the resolution of individual exons in the human genome. Am. J. Hum. Genet. 2005; 76: 750–762. 44. T. C. Mockler, S. Chan, A. Sundaresan, H. Chen, S. E. Jacobsen, and J. R. Ecker, Applications of DNA tiling arrays for wholegenome analysis. Genomics 2005; 85: 1–15. 45. P. Kapranov, S. E. Cawley, J. Drenkow, S. Bekiranov, R. L. Strausberg, S. P. Fodor, and T. R. Gingeras, Large-scale transcriptional activity in chromosomes 21 and 22. Science 2002; 296: 916–919. 46. D. Kampa, J. Cheng, P. Kapranov, M. Yamanaka, S. Brubaker, S. Cawley, J. Drenkow, A. Piccolboni, S. Bekiranov, G. Helt, H. Tammana, and T. R. Gingeras, Novel RNAs identified from an in-depth analysis of the transcriptome of human chromosomes 21 and 22. Genome Res. 2004; 14: 331–342. 47. M. Hild, B. Beckmann, S. A. Haas, B. Koch, V. Solovyev, C. Busold, K. Fellenberg, M. Boutros, M. Vingron, F. Sauer, J. D. Hoheisel, and R. Paro, An integrated gene annotation and transcriptional profiling approach towards the full gene content of the Drosophila genome. Genome Biol. 2003; 5:R3. 48. K. Yamada, J. Lim, J. M. Dale, H. Chen, P. Shinn, C. J. Palm, A. M. Southwick, H. C. Wu, C. Kim, M. Nguyen, et al., Empirical analysis of transcriptional activity in the arabidopsis genome. Science 2003; 302: 842–846. 49. V. Stolc, M. P. Samanta, W. Tongprasit, H. Sethi, S. Liang, D. C. Nelson, A. Hegeman, C. Nelson, D. Rancour, S. Bednarek, E. L. Ulrich, Q. Zhao, R. L. Wrobel, C. S. Newman, B. G. Fox, G. N. Phillips Jr, J. L. Markley, and M. R. Sussman, Identification of transcribed sequences in Arabidopsis thaliana by using high-resolution genome

15

tiling arrays. Proc. Natl. Acad. Sci. U.S.A. 2005; 102: 4453–4458. 50. L. Li, X. Wang, V. Stolc, X. Li, D. Zhang, N. Su, W. Tongprasit, S. Li, Z. Cheng, J. Wang, and X. W. Deng, Genome-wide transcription analyses in rice using tiling microarrays. Nat. Genet. 2006; 38: 124–129. 51. J. M. Johnson, S. Edwards, D. Shoemaker, and E. E. Schadt, Dark matter in the genome: evidence of widespread transcription detected by microarray tiling experiments. Trends Genet. 2005; 21: 93–102. 52. T. E. Royce, J. S. Rozowsky, P. Bertone, M. Samanta, V. Stolc, S. Weissman, M. Snyder, and M. Gerstein, Issues in the analysis of oligonucleotide tiling microarrays for transcript mapping. Trends Genet. 2005; 21: 466–475. 53. A. E. Urban, J. O. Korbel, R. Selzer, T. Richmond, A. Hacker, G. V. Popescu, J. F. Cubells, R. Green, B. S. Emanuel, M. B. Gerstein, S. M. Weissman, and M. Snyder, High-resolution mapping of DNA copy alterations in human chromosome 22 using high-density tiling oligonucleotide arrays. Proc. Natl. Acad. Sci. U.S.A. 2006; 103: 4534–4539. 54. A. Schumacher, P. Kapranov, Z. Kaminsky, J. Flanagan, A. Assadzadeh, P. Yau, C. Virtanen, N. Winegarden, J. Cheng, T. Gingeras, and A. Petronis, Microarray-based DNA methylation profiling: technology and applications. Nucleic Acids Res. 2006; 34: 528–542. 55. X. Zhang, J. Yazaki, A. Sundaresan, S. Cokus, S. W. L. Chan, H. Chen, I. R. Henderson, P. Shinn, M. Pellegrini, S. E. Jacobsen, and J. J S. Ecker, Genome-wide highresolution mapping and functional analysis of DNA methylation in arabidopsis. Cell 2006; 126: 1189–1201. 56. C. L. Liu, T. Kaplan, M. Kim, S. Buratowski, S. L. Schreiber, N. Friedman, and O. J. Rando, Single-nucleosome mapping of histone modifications in S. cerevisiae. PLoS Biol. 2005; 3:e328. 57. A. Schumacher, P. Kapranov, Z. Kaminsky, J. Flanagan, A. Assadzadeh, P. Yau, C. Virtanen, N. Winegarden, J. Cheng, T. Gingeras, and A. Petronis, Microarray-based DNA methylation profiling: technology and applications. Nucleic Acids Res. 2006; 34: 528–542. 58. G. C. Kennedy, H. Matsuzaki, S. Dong, W. Liu, J. Huang, G. Liu, X. Su, M. Cao, W.

16

MICROARRAY Chen, J. Zhang, et al., Large-scale genotyping of complex DNA. Nat. Biotech. 2003; 21: 1233–1237.

59. G. C. Kennedy, H. Matsuzaki, S. Dong, W. Liu, J. Huang, G. Liu, X. Su, M. Cao, W. Chen, J. Zhang, et al., Large-scale genotyping of complex DNA. Nat. Biotech. 2003; 21: 1233–1237. 60. H. Matsuzaki, S. Dong, H. Loi, X. Di, G. Liu, E. Hubbell, J. Law, T. Berntsen, M. Chadha, H. Hui, et al., Genotyping over 100,000 SNPs on a pair of oligonucleotide arrays. Nat. Methods 2004; 1: 109–111. 61. S. John, N. Shephard, G. Liu, E. Zeggini, M. Cao, W. Chen, N. Vasavda, T. Mills, A. Barton, A. Hinks, S. Eyre, et al., Whole-genome scan, in a complex disease, using 11,245 single-nucleotide polymorphisms: comparison with microsatellites. Am. J. Hum. Genet. 2004; 75: 54–64. 62. C. I. Amos, W. V. Chen, A. Lee, W. Li, M. Kern, R. Lundsten, F. Batliwalla, M. Wener, E. Remmers, D. A. Kastner, L. A. Criswell, M. F. Seldin, and P. K. Gregersen, High-density SNP analysis of 642 Caucasian families with rheumatoid arthritis identifies two new linkage regions on 11p12 and 2q33. Genes Immun. 2006; 7: 277–286. 63. Buck MJ, Lieb JD. ChIP-chip: considerations for the design, analysis, and application of genome-wide chromatin immunoprecipitation experiments. Genomics 2004; 83: 349–360. 64. J. Wu, L. T. Smith, C. Plass, and T. H. M. Huang, ChIP-chip comes of age for genomewide functional analysis. Cancer Res. 2006; 66: 6899–6902. 65. M. L. Bulyk, DNA microarray technologies for measuring protein-DNA interactions. Curr. Opin. Biotechnol. 2006; 17: 422–430. 66. C. S. Chen and H. Zhu, Protein microarrays. Biotechniques 2006; 40: 423, 425, 427. 67. P. Bertone and M. Snyder Advances in functional protein microarray technology. FEBS J. 2005; 272: 5400–5411. 68. B. B. Haab, M. J. Dunham, and P. O. Brown, Protein microarrays for highly parallel detection and quantitation of specific proteins and antibodies in complex solutions. Genome Biol. 2001; 2:RESEARCH0004. 69. A. Sreekumar, M. K. Nyati, S. Varambally, T. R. Barrette, D. Ghosh, T. S. Lawrence, and A. M. Chinnaiyan, Profiling of cancer cells using protein microarrays: discovery of novel radiation-regulated proteins. Cancer Res. 2001; 61: 7585–7593.

70. B. Schweitzer, S. Roberts, B. Grimwade, W. Shao, M. Wang, Q. Fu, Q. Shu, I. Laroche, Z. Zhou, V. T. Tchernev, J. Christiansen, M. Velleca, and S. F. Kingsmore, Multiplexed protein profiling on microarrays by rollingcircle amplification. Nat. Biotechnol. 2002; 20: 359–365. 71. H. Zhu, M. Bilgin, R. Bangham, D. Hall, A. Casamayor, P. Bertone, N. Lan, R. Jansen, S. Bidlingmaier, T. Houfek, et al., Global analysis of protein activities using proteome chips. Science 2001; 293: 2101–2105. 72. M. Arifuzzaman, M. Maeda, A. Itoh, K. Nishikata, C. Takita, R. Saito, T. Ara, K. Nakahigashi, H. C. Huang, A. Hirai, et al., Large-scale identification of protein-protein interaction of Escherichia coli K-12. Genome Res. 2006; 16: 686–691. 73. S. W. Ho, G. Jona, C. T. L. Chen, M. Johnston, and M. Snyder, Linking DNA-binding proteins to their recognition sequences by using protein microarrays. Proc. Natl. Acad. Sci. U.S.A. 2006; 103: 9940–9945. 74. D. A. Hall, H. Zhu, X. Zhu, T. Royce, M. Gerstein, and M. Snyder, Regulation of gene expression by a metabolic enzyme. Science 2004; 306: 482–484. 75. T. Feilner, C. Hultschig, J. Lee, S. Meyer, R. G. H. Immink, A. Koenig, A. Possling, H. Seitz, A. Beveridge, D. Scheel, et al., High throughput identification of potential arabidopsis mitogen-activated protein kinases substrates. Molec. Cell. Proteom. 2005; 4: 1558–1568. 76. H. Du, M. Wu, W. Yang, G. Yuan, Y. Sun, Y. Lu, S. Zhao, Q. Du, J. Wang, S. Yang, et al., Development of miniaturized competitive immunoassays on a protein chip as a screening tool for drugs. Clin. Chem. 2005; 51: 368–375. 77. A. Lueking, O. Huber, C. Wirths, K. Schulte, K. M. Stieler, U. Blume-Peytavi, A. Kowald, K. Hensel-Wiegel, R. Tauber, H. Lehrach, et al., Profiling of alopecia areata autoantigens based on protein microarray technology. Molec. Cell. Proteom. 2005; 4: 1382–1390. 78. W. H. Robinson, C. DiGennaro, W. Hueber, B. B. Haab, M. Kamachi, E. J. Dean, S. Fournel, D. Fong, M. C. Genovese, H. E. de Vegvar, et al., Autoantigen microarrays for multiplex characterization of autoantibody responses. Nat. Med. 2002; 8: 295–301. 79. A. Lueking, A. Possling, O. Huber, A. Beveridge, M. Horn, H. Eickhoff, J. Schuchardt, H. Lehrach, and D. J. Cahill, A nonredundant human protein chip for antibody

MICROARRAY

80.

81.

82.

83.

84.

85.

86.

87.

88.

89.

90.

91.

screening and serum profiling. Molec. Cell. Proteom. 2003; 2: 1342–1349. M. T. Lee, F. C. Kuo, G. A. Whitmore, and J. Sklar, Importance of replication in microarray gene expression studies: statistical methods and evidence from repetitive cDNA hybridizations. Proc. Natl. Acad. Sci. U.S.A. 2000; 97: 9834–9839. M. K. Kerr and G. A. Churchill, Statistical design and the analysis of gene expression microarray data. Genet. Res. 2001; 77: 123–128. K. Mirnics, Microarrays in brain research: the good, the bad and the ugly. Nat. Rev. Neurosci. 2001; 2: 444–447. K. R. Coombes, W. E. Highsmith, T. A. Krogmann, K. A. Baggerly, D. N. Stivers, and L. V. Abruzzo, Identifying and quantifying sources of variation in microarray data using high-density cDNA membrane arrays. J. Comput. Biol. 2002; 9: 655–669. Y. Woo, J. Affourtit, S. Daigle, A. Viale, K. Johnson, J. Naggert, and G. Churchill, A comparison of cDNA, oligonucleotide, and Affymetrix GeneChip gene expression microarray platforms. J. Biomol. Tech. 2004; 15: 276–284. D. Rubin, Practical implications of modes of statistical inference for causal effects and the critical role of the assignment mechanism. Biometrics 1991; 47: 1213–1234. M. K. Kerr and G. A. Churchill, Experimental design for gene expression microarrays. Biostatistics 2001; 2: 183–201. M. F. Oleksiak and G. A. Churchill, and D. L. Crawford, Variation in gene expression within and among natural populations. Nat. Genet. 2002; 32: 261–266. D. B. Allison and C. S. Coffey, Two-stage testing in microarray analysis: what is gained? J. Gerontol. A Biol. Sci. Med. Sci. 2002; 57:B189–B192. Y. Benjamini and Y. Hochberg, Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. Royal Stat. Soc. Series B 1995; 57: 289–300. V. G. Tusher, R. Tibshirani, and G. Chu, Significance analysis of microarrays applied to the ionizing radiation response. Proc. Natl. Acad. Sci. U.S.A. 2001; 98: 5116–5121. D. Allison, G. Gadbury, M. Heo, J. R. Fernandez, C. K. Lee, T. A. Prolla, and R. Weindruch, A mixture model approach for the analysis of microarray gene expression data. Computat. Stat. Data Anal. 2002; 39: 1–20.

17

92. G. L. Gadbury, G. Xiang, J. Edwards, G. Page, and D. B. Allison, The role of sample size on measures of uncertainty and power. In D. B. Allison, J. W. Edwards, T. M. Beasley, and G. Page (eds.), DNA Microarrays and Related Genomics Techniques. Boca Raton, FL: CRC Press, 2005, pp. 51–60. 93. G. Gadbury, G. Page, J. Edwards, T. Kayo, R. Weindruch, P. A. Permana, J. Mountz, and D. B. Allison, Power analysis and sample size estimation in the age of high dimensional biology. Stat. Meth. Med. Res. 2004; 13: 325–338. 94. G. P. Page, J. W. Edwards, G. L. Gadbury, P. Yelisetti, J. Wang, P. Trivedi, and D. B. Allison, The PowerAtlas: a power and sample size atlas for microarray experimental design and research. BMC Bioinformat. 2006; 7: 84. 95. R. Nagarajan, Intensity-based segmentation of microarray images. IEEE Trans. Med. Imag. 2003; 22: 882–889. 96. Q. Li, C. Fraley, R. E. Bumgarner, K. Y. Yeung, and A. E. Raftery, Donuts, scratches and blanks: robust model-based segmentation of microarray images. Bioinformatics 2005. 97. C. Li and W. W. Hung, Model-based analysis of oligonucleotide arrays: model validation, design issues and standard error application. Genome Biol. 2001; 2: 32–35. 98. Z. Wu and R. A. Irizarry, Stochastic models inspired by hybridization theory for short oligonucleotide arrays. J. Comput. Biol. 2005; 12: 882–893. 99. E. Hubbell, W. M. Liu, and R. Mei, Robust estimators for expression analysis. Bioinformatics 2002; 18: 1585–1592. 100. L. Zhang, L. Wang, A. Ravindranathan, and M. F. Miles, A new algorithm for analysis of oligonucleotide arrays: application to expression profiling in mouse brain regions. J. Mol. Biol. 2002; 317: 225–235. 101. R. A. Irizarry, B. M. Bolstad, F. Collin, L. M. Cope, B. Hobbs, T. P. Speed, Summaries of Affymetrix GeneChip probe level data. Nucleic Acids Res. 2003; 31:e15. 102. R. A. Irizarry, B. Hobbs, F. Collin, Y. D. Beazer-Barclay, K. J. Antonellis, U. Scherf, T. P. Speed, Exploration, normalization, and summaries of high density oligonucleotide array probe level data. Biostatistics 2003; 4: 249–264. 103. K. Shedden, W. Chen, R. Kuick, D. Ghosh, J. Macdonald, K. R. Cho, T. J. Giordano, S. B. Gruber, E. R. Fearon, J. M. Taylor, and

18

MICROARRAY S. Hanash, Comparison of seven methods for producing Affymetrix expression scores based on False Discovery Rates in disease profiling data. BMC Bioinformat. 2005; 6: 26.

104. G. K. Smyth and T. Speed, Normalization of cDNA microarray data. Methods 2003; 31: 265–273. 105. J. Tukey, On the comparative anatomy of transformation. Ann. Mathemat. Statist. 1964; 28: 602–632. 106. R. D. Wolfinger, G. Gibson, E. D. Wolfinger, L. Bennett, H. Hamadeh, P. Bushel, C. Afshari, and R. S. Paules, Assessing gene significance from cDNA microarray expression data via mixed models. J. Comput. Biol. 2001; 8: 625–637. 107. B. M. Bolstad, R. A. Irizarry, M. Astrand, and T. P. Speed, A comparison of normalization methods for high density oligonucleotide array data based on variance and bias. Bioinformatics 2003; 19: 185–193. 108. B. P. Durbin, J. S. Hardin, D. M. Hawkins, and D. M. Rocke, A variance-stabilizing transformation for gene-expression microarray data. Bioinformatics 2002; 18(suppl 1):S105–S110. 109. B. P. Durbin and D. M. Rocke, Variancestabilizing transformations for two-color microarrays. Bioinformatics 2004; 20: 660–667. 110. C. A. Ball, G. Sherlock, H. Parkinson, P. Rocca-Sera, C. Brooksbank, HC. Causton, D. Cavalieri, T. Gaasterland, P. Hingamp, F. Holstege, et al., Standards for microarray data. Science 2002, 298: 539. 111. C. A. Ball, G. Sherlock, H. Parkinson, P. Rocca-Sera, C. Brooksbank, H. C. Causton, D. Cavalieri, T. Gaasterland, P. Hingamp, F. Holstege, et al., An open letter to the scientific journals. Bioinformatics 2002; 18: 1409. 112. K. H. Cheung, K. White, J. Hager, M. Gerstein, V. Reinke, K. Nelson, P. Masiar, R. Srivastava, Y. Li, J. Li, J. Li, et al., YMD: A microarray database for large-scale gene expression analysis. Proc. AMIA Symp. 2002; 140–144. 113. C. Baer, M. Nees, S. Breit, B. Selle, A. E. Kulozik, K. L. Schaefer, Y. Braun, D. Wai, and C. Poremba, Profiling and functional annotation of mRNA gene expression in pediatric rhabdomyosarcoma and Ewing’s sarcoma. Int. J. Cancer 2004; 110: 687–694. 114. R. L. Somorjai, B. Dolenko, and R. Baumgartner, Class prediction and discovery using

gene microarray and proteomics mass spectroscopy data: curses, caveats, cautions. Bioinformatics 2003; 19: 1484–1491. 115. M. D. Radmacher, L. M. McShane, and R. Simon, A paradigm for class prediction using gene expression profiles. J. Comput. Biol. 2002; 9: 505–511. 116. M. Ringner and C. Peterson, Microarraybased cancer diagnosis with artificial neural networks. Biotechnology 2003(suppl): 30–35. 117. U. M. Braga-Neto and E. R. Dougherty, Is cross-validation valid for small-sample microarray classification? Bioinformatics 2004; 20: 374–380. 118. C. Romualdi, S. Campanaro, D. Campagna, B. Celegato, N. Cannata, S. Toppo, G. Valle, and G. Lanfranchi, Pattern recognition in gene expression profiling using DNA array: a comparative study of different statistical methods applied to cancer classification. Hum. Mol. Genet. 2003; 12: 823–836. 119. R. Simon and M. D. Radmacher, and K. Dobbin, Design of studies using DNA microarrays. Genet. Epidemiol. 2002; 23: 21–36. 120. R. Simon, M. D. Radmacher, K. Dobbin, and L. M. McShane, Pitfalls in the use of DNA microarray data for diagnostic and prognostic classification. J. Natl. Cancer Inst. 2003; 95: 14–18. 121. R. Simon, Diagnostic and prognostic prediction using gene expression profiles in highdimensional microarray data. Br. J. Cancer 2003; 89: 1599–1604. 122. C. Ambroise and G. J. McLachlan, Selection bias in gene extraction on the basis of microarray gene-expression data. Proc. Natl. Acad. Sci. U.S.A. 2002; 99: 6562–6566. 123. R. Simon, M. D. Radmacher, K. Dobbin, and L. M. McShane, Pitfalls in the use of DNA microarray data for diagnostic and prognostic classification. J. Natl. Cancer Inst. 2003; 95: 14–18. 124. N. R. Garge, G. P. Page, A. P. Sprague, B. S. Gorman, and D. B. Allison, Reproducible clusters from microarray research: whither? BMC Bioinformat. 2005; 6(suppl 2):S10. 125. S. Datta and S. Datta, Comparisons and validation of statistical clustering techniques for microarray gene expression data. Bioinformatics 2003; 19: 459–466. 126. P. Baldi and A. D. Long, A Bayesian framework for the analysis of microarray expression data: regularized t -test and statistical inferences of gene changes. Bioinformatics 2001; 17: 509–519.

MICROARRAY 127. D. B. Allison, X. Cui, G. P. Page, and M. Sabripour, Microarray data analysis: from disarray to consolidation and consensus. Nat. Rev. Genet. 2006; 7: 55–65. 128. X. Cui, J. T. Hwang, J. Qiu, N. J. Blades, and G. A. Churchill, Improved statistical tests for differential gene expression by shrinking variance components estimates. Biostatistics 2005; 6: 59–75. 129. P. H. Westfall, D. V. Zaykin, and S. S. Young, Multiple tests for genetic effects in association studies. Methods Molec. Biol. 2002; 184: 143–168. 130. Y. Benjamini, D. Drai, G. Elmer, N. Kafkafi, and I. Golani, Controlling the false discovery rate in behavior genetics research. Behav. Brain Res. 2001; 125: 279–284. 131. M. Kanehisa, S. Goto, M. Hattori, K.F. Aoki-Kinoshita, M. Itoh, S. Kawashima, T. Katayama, M. Araki, and M. Hirakawa, From genomics to chemical genomics: new developments in KEGG. Nucleic Acids Res. 2006; 34:D354–D357. 132. X. Mao, T. Cai, J. G. Olyarchuk, and L. Wei, Automated genome annotation and pathway identification using the KEGG Orthology (KO) as a controlled vocabulary. Bioinformatics 2005; 21: 3787–3793. 133. Y. Lee, R. Sultana, G. Pertea, J. Cho, S. Karamycheva, J. Tsai, B. Parvizi, F. Cheung, V. Antonescu, J. White, et al., Crossreferencing eukaryotic genomes: TIGR Orthologous Gene Alignments (TOGA). Genome Res. 2002; 12: 493–502. 134. Y. Lee, J. Tsai, S. Sunkara, S. Karamycheva, G. Pertea, R. Sultana, V. Antonescu, A. Chan, F. Cheung, and J. Quackenbush, The TIGR Gene Indices: clustering and assembling EST and known genes and integration with eukaryotic genomes. Nucleic Acids Res. 2005; 33:D71–D74. 135. L. Tanabe, U. Scherf, L. H. Smith, J. K. Lee, L. Hunter, and J. N. Weinstein, MedMiner: an Internet text-mining tool for biomedical information, with application to gene expression profiling. Biotechnology 1999; 27: 1210–1217. 136. M. Ashburner, C. A. Ball, J. A. Blake, D. Botstein, H. Butler, J. M. Cherry, A. P. Davis, K. Dolinski, S. S. Dwight, J. T. Eppig, et al., Gene ontology: tool for the unification of biology (In Process Citation). Nat. Genet. 2000; 25: 25–29. 137. P. Khatri and S. Draghici, Ontological analysis of gene expression data: current tools,

138.

139.

140.

141.

142.

143.

144.

145.

146.

147.

148.

149.

19

limitations, and open problems. Bioinformatics 2005; 21: 3587–3595. J. J. Goeman and P. Buhlmann, Analyzing gene expression data in terms of gene sets: methodological issues 2. Bioinformatics 2007; 23: 980–987. I. Rivals, L. Personnaz, L. Taing, and M. C. Potier, Enrichment or depletion of a GO category within a class of genes: which test? Bioinformatics 2007; 23: 401–407. D. B. Allison, X. Cui, G. P. Page, and M. Sabripour, Microarray data analysis: from disarray to consolidation and consensus. Nat. Rev. Genet. 2006; 7: 55–65. U. Mansmann and R. Meister, Testing differential gene expression in functional groups. Goeman’s global test versus an ANCOVA approach. Methods Inf. Med. 2005; 44: 449–453. VK. Mootha, CM. Lindgren, KF. Eriksson, A. Subramanian, S. Sihag, J. Lehar, P. Puigserver, E. Carlsson, M. Ridderstrale, E. Laurila, et al., PGC-1alpha-responsive genes involved in oxidative phosphorylation are coordinately downregulated in human diabetes. Nat. Genet. 2003; 34: 267–273. J. Tomfohr, J. Lu, and T. B. Kepler, Pathway level analysis of gene expression using singular value decomposition. BMC Bioinformat. 2005; 6: 225. J. J. Goeman, S. A. van de Geer, F. de Kort, and H. C. van Houwelingen, A global test for groups of genes: testing association with a clinical outcome. Bioinformatics 2004; 20: 93–99. J. J. Goeman, J. Oosting, A. M. CletonJansen, J. K. Anninga, H. C. van Houwelingen, Testing association of a pathway with survival using gene expression data. Bioinformatics 2005; 21: 1950–1957. P. Khatri and S. Draghici, Ontological analysis of gene expression data: current tools, limitations, and open problems. Bioinformatics 2005; 21: 3587–3595. P. Zimmermann, M. Hirsch-Hoffmann, L. Hennig, and W. Gruissem, GENEVESTIGATOR. Arabidopsis microarray database and analysis toolbox. Plant Physiol. 2004; 136: 2621–2632. F. A. de la, P. Brazhnik, and P. Mendes, Linking the genes: inferring quantitative gene networks from microarray data. Trends Genet. 2002; 18: 395–398. S. Imoto, T. Higuchi, T. Goto, K. Tashiro, S. Kuhara, and S. Miyano, Combining microarrays and biological knowledge for estimating

20

MICROARRAY

gene networks via Bayesian networks. J. Bioinform. Comput. Biol. 2004; 2: 77–98. 150. Z. S. Qin, L. A. McCue, W. Thompson, L. Mayerhofer, C. E. Lawrence, and J. S. Liu, Identification of co-regulated genes through Bayesian clustering of predicted regulatory binding sites. Nat. Biotechnol. 2003. 151. B. Xing and M. J. van der Laan, A statistical method for constructing transcriptional regulatory networks using gene expression and sequence data. J. Comput. Biol. 2005; 12: 229–246. 152. D. Murphy, Gene expression studies using microarrays: principles, problems, and prospects. Adv. Physiol. Educ. 2002; 26: 256–270. 153. R. Edgar, M. Domrachev, and A. E. Lash, Gene Expression Omnibus: NCBI gene expression and hybridization array data repository. Nucleic Acids Res. 2002; 30: 207–210. 154. R. Tuteja and N. Tuteja, Serial analysis of gene expression (SAGE): unraveling the bioinformatics tools. BioEssays 2004; 26: 916–922.

MINIMUM EFFECTIVE DOSE (MINED)

1 INDIVIDUAL AND POPULATION DOSE RESPONSE CURVES

NEAL THOMAS and NAITEE TING

Figure 1 distinguishes between individual dose-response relationships (the three thinner curves representing three different individual subjects) and the (single, thicker) population average dose-response relationship. Because of intersubject variability, different subjects may respond to the same drug in different ways (2). The definitions of the MinED in the statistics literature focus on the population dose-response curve. The most common definition of the MinED is the lowest dose that produces a specified average difference in response with placebo or other comparator treatment. In the case of a binary outcome, the specified average response is a proportion. Bretz, Hothorn, and Hsu (3) define a MinED as the lowest dose that achieves a specified ratio of the mean response divided by the comparator mean response. None of the available literature appear to define the MinED with time-toevent outcomes analyzed by methods such as Cox regression. Figure 2 displays a typical monotonically increasing dose response curve (left side of the plot) with a hypothetical MinED and MaxED indicated by vertical lines. The MaxED is placed at a dose that nearly achieves the maximum mean response. There does not appear to be a broadly accepted definition of the MaxED, but it is common in pharmacometric practice to target no more than 90% of the maximum effect because dose-response curves often increase slowly to the maximum effect, so much higher doses are required to yield small additional improvements in efficacy. The MinED is the dose that produces a minimum clinically important difference (MCID) from the control. The curve on the right side of Figure 2 is a population dose-response curve for some measure of toxicity. A somewhat optimistic setting is displayed, with toxicity increasing for doses above the MinED. Because of subject variability around the average dose response, even in this setting some patients may not achieve the MinED without experiencing

Global Research and Development Pfizer Inc. New London, Connecticut

The International Conference on Harmonization (ICH) E4 Guideline for Industry (1), the primary source of regulatory guidance for dose response studies, provides a concise definition of a minimum effective dose (MinED or MED): Historically, drugs have often been initially marketed at what were later recognized as excessive doses (i.e., doses well onto the plateau of the dose-response curve for the desired effect), sometimes with adverse consequences (e.g., hypokalemia and other metabolic disturbances with thiazide-type diuretics in hypertension). This situation has been improved by attempts to find the smallest dose with a discernible useful effect or a maximum dose beyond which no further beneficial effect is seen, but practical study designs do not exist to allow for precise determination of these doses. Further, expanding knowledge indicates that the concepts of minimum effective dose and maximum useful dose do not adequately account for individual differences and do not allow a comparison, at various doses, of both beneficial and undesirable effects. Any given dose provides a mixture of desirable and undesirable effects, with no single dose necessarily optimal for all patients.

The definition of the MinED as the ‘‘smallest dose with a discernible useful effect’’ requires that we further define a discernible effect and a useful effect. The MinED is implicitly defined by population summaries, as indicated by the latter portion of the ICH passage. Methodological development of the MinED has focused on operational definitions of ‘‘discernible’’ and ‘‘useful,’’ and the distinction between population and individual effects. Although less commonly discussed, the guidelines assign similar importance to establishing a ‘‘maximum useful dose,’’ which is sometimes called the maximum effective dose (MaxED).

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

MINIMUM EFFECTIVE DOSE (MINED)

Response

2

Dose

Response

Figure 1. Individual and average dose-response curves.

Toxicity

Efficacy

MCID

MinED

MoxED Dose

Figure 2. Dose-response for efficacy and toxicity.

adverse events. In less optimistic settings, the MinED cannot be evaluated because of toxicities occurring at lower doses.

2 OPERATIONAL DEFINITIONS OF USEFUL EFFECT It is difficult to achieve consensus about the smallest magnitude of an effect that is useful. The useful effect is often based on

precedent or convention within a therapeutic area. There can be different sources for the precedent, including the estimated effect of a well-established therapy that has been accepted as useful. For example, in studying chronic eye diseases such as age-related macular degeneration or diabetic macular edema, a loss (or gain) of three lines (or, equivalently, 15 letters) in vision based on the Early Treatment Diabetic Retinopathy Study (ETDRS) chart is a common standard for

MINIMUM EFFECTIVE DOSE (MINED)

visual acuity (4). Another example would be a two-point mean difference on the International Prostate Symptoms Scale for a useful effect for benign prostate hyperplasia (5). Another useful concept is the minimum clinically important difference (MCID). This is the magnitude of the treatment response exceeding the placebo response that is large enough to be perceived by a subject. It is a commonly used concept in outcomes research. There are two general methods to study this issue (6, 7): (1) the anchor-based method, which correlates response of a clinical endpoint to responses to questions about perceptions of improvement (or worsening); and (2) the distribution-based method, which calculates the between-person standard deviation unit and uses a multiple of this unit as the MCID. A similar problem arises when selecting a tolerance interval width for noninferiority and equivalence trials, which typically arise when comparing two active drugs rather than a drug and placebo. Achieving consensus on acceptable difference between active drugs is also difficult in this setting, as has been discussed elsewhere (8, 9). 3 OPERATIONAL DEFINITIONS OF DISCERNIBLE EFFECTS A discernible effect has been interpreted as a statistically significant difference between a dose group and placebo by several investigators. Ruberg (10) provides another way of restating the ‘‘smallest dose with a discernible useful effect’’: ‘‘The MinED is the lowest dose producing a clinically important response that can be declared statistically, significantly different from the placebo response.’’ This definition of a discernible effect involving statistical significance depends on the size and design of the dose-response study. Two general approaches have been used to establish statistical differences: (1) pairwise testing of doses versus placebo, with adjustment for multiple comparisons; and (2) dose response modeling, with reporting sometimes based on Bayesian probabilities rather than P-values. Sequential applications of trend tests corrected for multiple comparisons are a compromise between the pairwise

3

test-based approaches and estimation-based approaches. The testing-based approaches appear to be more commonly used. Pairwise comparisons of each dose group to placebo can be performed that preserve type I error without requiring any assumptions about the ordering of dose groups. To maintain a prespecified type I error, the alpha level of each test must be adjusted (multiple comparison adjustment). Dunnett (11) is a highly referenced method in this setting. Numerous other multiple comparison methods have been proposed that have improved power for some implicitly defined dose-response curves (12–14). These methods are based on sequential hypothesis testing. They typically begin with tests for the highest dose. The lowest statistically significant dose that is also clinically useful is selected as the MinED. Testing (and estimation) methods have also been developed that assume an increasing order of response with dose but do not assume a specific form for the doseresponse curve. These methods, such as that of Williams (15, 16), are called isotonic inference, isotonic regression, or order-restricted inference (10, 17–20). Tests based on contrasts (weighted combinations of dose group mean responses) derived from dose-response curves have also been proposed (10, 21, 22). These approaches, like those from isotonic inference, can increase power to achieve statistical significance by using information about the likely shape of the dose-response curve. The contrast tests are also implemented as sequential tests, which proceed as long as statistical significance is achieved. The MinED is derived from the contrast that is statistically significant, and includes the lowest maximum dose. The maximum dose from the selected contrast is the MinED. Bauer (23) and Hsu and Berger (13) note that these methods can have inflated type I error for an ineffective high dose when the dose-response curve is not monotonic. Hsu and Berger (13) show that sequential tests that preserve the familywise error rate are in the form of directional pairwise comparisons with the comparator group.

4

MINIMUM EFFECTIVE DOSE (MINED)

As noted in Tamhane et al. (22), all of the testing procedures select a MinED from only those doses included in the dose-response study design; as a consequence, they can be very dependent on the selection the doses included in the design. They noted this fact and offered a different naming convention: ‘‘what any test procedure finds is a minimum detectable dose (MinDD).’’ Most of the procedures noted here require higher doses to be statistically significant before lower doses can be declared significant, and, as a consequence, they tend to report higher values for the MinED. The MinED can also be selected by inversion of an estimated dose-response curve to find a dose with expected response equal to a target value, typically the smallest useful dose (24). The inversion of the dose-response curve may be preceded by a global test for a dose-response trend. A significant global trend test, however, is not equivalent to statistical significance of individual doses, as is typically required in most testing approaches. Bretz, Pinheiro, and Branson (25) describe several criteria for selecting a MinED based on inversion of the dose-response curve. The MinED estimated from a dose-response curve is likely to be a dose not included in the study. Ruberg (10) describes another method of estimating the MinED using the logistic Emax curve. This approach is based on methods for determining the minimum detectable concentration in biological assays (26, 27). Unlike all of the other methods noted here, this approach is based on selecting a dose that could be consistently differentiated from placebo in future trials of a specified size. REFERENCES 1. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH Harmonised Tripartite Guideline: E4 DoseResponse Information to Support Drug Registration. Current Step 4 version, March 10, 1994. Available at: http://www.ich.org/LOB/ media/MEDIA480.pdf 2. N. Holford and L. Sheiner, Understanding the dose-effect relationship: clinical application of pharmacokinetic–pharmacodynamic models. Clin Pharmacokinet. 1981; 6: 429–453.

3. F. Bretz, L. Hothorn, and J. Hsu, Identifying effective and/or safe doses by stepwise confidence intervals for ratios. Stat Med. 2003; 22: 847–858. 4. U.S. Food and Drug Administration, Center for Drug Evaluation and Research. Joint meeting of the ophthalmic drugs subcommittee of the dermatologic and ophthalmic drugs advisory committee and the endocrine and metabolic drugs advisory committee, March 11, 1998. http://www.fda.gov/cder/foi/ adcomm/98/jnt doac emdac 031198 ag ques. pdf 5. F. Desgrandchamps, Importance of individual response in symptom score evaluation. Eur Urol. 2001; 40(Suppl 3): 2–7. 6. G. H. Guyatt, D. Osoba, A. W. Wu, K. W. Wyrwich, G. R. Norman, and the Clinical significance consensus meeting group. Methods to explain the clinical significance of health status measures. Mayo Clin Proc. 2002; 77: 371–383. 7. M. A. G. Sprangers, C. M. Moinpour, T. J. Moynihan, D. L. Patrick, D. A. Revicki, and the Clinical Significance Consensus Meeting Group. Assessing meaningful change in quality of life over time: a users’ guide for clinicians, Mayo Clin Proc. 2002; 77: 561–571. 8. Committee for Medicinal Products for Human Use (CHMP), European Medicines Agency. Guideline on the Choice of the Non-Inferiority Margin. London, UK, July 27, 2005. Available at: http://www.emea.europa.eu/pdfs/human/ ewp/215899en.pdf 9. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH Harmonised Tripartite Guideline: E10 Choice of Control Group and Related Issues in Clinical Trials. Current Step 4 version, July 20, 2000. Available at: http://www.ich.org/LOB/media/ MEDIA486.pdf 10. S. Ruberg, Dose response studies. II. Analysis and interpretation. J Biopharm Stat. 1995; 5: 15–42. 11. C. Dunnett, A multiple comparison procedure for comparing several treatments to control. J Am Stat Assoc. 1955; 50: 1096–1121. 12. Y. Hochberg, and A. Tamhane, Multiple Comparison Procedures. New York: Wiley, 1987. 13. J. Hsu and R. Berger, Stepwise confidence intervals without multiplicity adjustment for dose-response and toxicity studies. J Am Stat Assoc. 1999; 94: 468–482. 14. A. Tamhane and B. Logan, Multiple comparison procedures in dose response studies. In:

MINIMUM EFFECTIVE DOSE (MINED) N. Ting (ed.), Dose-Finding in Drug Development. New York: Springer, 2006, pp 172–183. 15. D. Williams, A test for differences between treatment means when several dose levels are compared with a zero dose level. Biometrics. 1971; 27: 103–117. 16. D. Williams, The comparison of several dose levels with a zero dose control. Biometrics. 1972; 28: 519–531. 17. T. Robertson, F. Wright, and R. Dykstra, Order Restricted Inference, New York: Wiley, 1988. 18. D. J. Bartholomew, Isotonic inference. In: Encyclopedia of Statistical Sciences, Vol. 4. New York: Wiley, 1983, pp 260–265. 19. C. Hirotsu, Isotonic inference. In: Encyclopedia of Biostatistics, Vol. 3. New York: Wiley, 1999, pp 2107–2115. 20. J. W. McDonald, Isotonic regression. In: Encyclopedia of Biostatistics, Vol. 3. New York: Wiley, 1999, pp 2115–2116. 21. S. Ruberg, Contrasts for identifying the minimum effective dose. J Am Stat Assoc. 1989; 84: 816–822. 22. A. Tamhane, Y. Hochberg, and C. Dunnett, Multiple test procedures for dose finding. Biometrics. 1996; 52: 21–37. 23. P. Bauer, A note on multiple testing procedures in dose finding. Biometrics. 1997; 53: 1125–1128. 24. T. G. Filloon, Estimating the minimum therapeutically effective dose of a compound via regression modelling and percentile estimation (Disc: p933-933). Stat Med. 1995; 14: 925–932.

5

25. F. Bretz, J. Pinheiro, and M. Branson, Combining multiple comparisons and modeling techniques in dose-response studies. Biometrics. 2005; 61: 738–748. 26. M. Davidian, R. J. Carroll, and W. Smith, Variance functions and the minimum detectable concentration in assays. Biometrika. 1988; 75: 549–556. 27. D. Rodbard, Statistical estimation of the minimal detectable concentration (‘‘sensitivity’’) of radioligand assays. Anal Biochem. 1978; 90: 1–12.

CROSS-REFERENCES

MINISTRY OF HEALTH, LABOUR AND WELFARE (MHLW) OF JAPAN

R&D promotion (i.e., guardian) is an important feature of the Japanese system. This reflects some historical mishaps related to blood products and human immunodeficiency virus in the 1990s, in which the Ministry was severely criticized for its unclear decisionmaking process. Japan has universal health insurance, and all clinical trials submitted for investigational new drug notification (IND), with the exception of phase I studies involving healthy volunteers, must be done under the health insurance scheme. The financial rules for reimbursement and copayment in IND-submitted trials are determined by the Health Insurance Bureau of the MHLW. The MHLW is responsible for national policy making and final decisions on IND and NDA approvals. Day-to-day scientific evaluation of actual INDs and NDAs, safety review and pharmaco-vigilance after marketing approval, and good manufacturing practice (GMP) inspections are handled by an external agency, the Pharmaceuticals and Medical Devices Agency (PMDA).

SHUNSUKE ONO Graduate School of Pharmaceutical Sciences University of Japan Tokyo, Japan

As an important component of pharmaceutical research and development (R&D) activities, clinical trials in Japan are implemented under various regulations of the Pharmaceutical Affairs Law (PAL) and related ordinances and guidelines issued by the Ministry of Health, Labour and Welfare (MHLW). 1 MINISTRY OF HEALTH, LABOUR AND WELFARE (MHLW) The Ministry of Health, Labour and Welfare (MHLW), which was originally established in 1938, is a government agency responsible for public health, social security, and labor issues in Japan. In 2001, two ministries, the Ministry of Health and Welfare and the Ministry of Labour, were merged into the current MHLW in an effort to reorganize and streamline government ministries. The MHLW consists of the Minister’s Secretariat, 11 bureaus, and affiliated institutions, councils, local branch offices, and external organizations. For the updated organizational structure, see the Ministry’s website at http://www.mhlw.go.jp/english/index.html (in English). Issues related to pharmaceuticals and medical devices are under the control of two bureaus in the MHLW, the Pharmaceutical and Food Safety Bureau (PFSB) and the Health Policy Bureau (HPB). The PFSB is the office in charge of enactment, amendment, and implementation of the Pharmaceutical Affairs Law (PAL), the backbone of Japanese pharmaceutical regulation. The objectives of the PFSB are the implementation of regulatory rules and related guidelines rather than the promotion of R&D policies. A division in the HPB, the Research and Development Division, is responsible for R&D policy issues in health care. The philosophy of strict separation between regulations (i.e., police) and

2 PHARMACEUTICALS AND MEDICAL DEVICES AGENCY (PMDA) The PMDA is an incorporated administrative agency established in 2004. It is not a government agency per se but rather was established based on the law for the Incorporated Administrative Agency. It is therefore a quasi-governmental agency, and its objectives and operations are specifically determined by the law and related ordinances. The PMDA took over the operations and services of three previous agencies: the Pharmaceuticals and Medical Devices Evaluation Center (PMDEC), the Organization for Pharmaceutical Safety and Research (OPSR), and part of the Japan Association for the Advancement of Medical Equipment (JAAME) (1). 2.1 Objectives of the PMDA The PMDA has three major objectives. 1. Consultation, review, and related activities. The PMDA conducts scientific

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

MINISTRY OF HEALTH, LABOUR AND WELFARE (MHLW) OF JAPAN

review for submitted NDAs and INDs, and conveys its review results to the MHLW, the final decision-making agency, in the form of review reports. It also offers consultation services for the industry and clinical researchers on a fee-for-service basis (see section 2.4). 2. Postmarketing safety operations. The PMDA collects, analyzes, and disseminates information on the quality, efficacy, and safety of pharmaceuticals and medical devices on the market. 3. Adverse health-effect relief services. The PMDA provides services related to payment for medical expenses and disability pensions to patients affected by adverse drug reactions. These services were originally performed by the OPSR. Before the PMDA was established in 2004, review and consultation were provided in two separate organizations. The PMDEC undertook scientific review activities to evaluate NDAs and INDs submitted to the MHLW, and the OPSR provided consultation services for the industry for predetermined fees. The NDA and IND review activities are official responsibilities, for which delegation to the PMDA is stipulated by the PAL, but the consultation services are not directly based on the PAL. This is in apparent contrast to the regulatory apparatus in the United States, where the sponsors can ask for various types of meetings as part of the overall review (2). However, the basic services that the regulatory agencies in both countries offer to sponsors (i.e., scientific advice on a timely basis) are the same.

The PMDA website (http://www.pmda. go.jp/) provides more detailed information on the scope of its operations. 2.2 Organization of the PMDA Efficacy, safety, and quality issues of pharmaceuticals (including over-the-counter drugs) and medical devices are handled by eleven review-related offices in the PMDA. The Offices of New Drugs I, II, III, and IV, and the Offices of Biologics I and II are the offices responsible for conducting scientific review for NDAs and INDs of pharmaceutical products. In each office, several review teams are organized, each focusing on specific therapeutic categories (Table 1). As of October 2007, there were ten review teams operating in the PMDA. A review team consists of project managers and experts in chemistry, pharmacology and pharmacokinetics, toxicology, physicians, and biostatisticians. The total number of review staff including the reviewers and support staff involved in review activities is about 200, one-tenth of that of the U.S. Food and Drug Administration (as of 2007). Although the human resources for review have been expanding gradually, there is still a significant gap between the United States and Japan. The lack of sufficient review resources has often been attributed to insufficient opportunities for consultation services and delay in new drug approval times. 2.3 IND and NDA Review by the PMDA For each IND and NDA, a specific review team is assigned at the time of submission. Assignment is based on the therapeutic

Table 1. Review offices and therapeutic fields in the Pharmaceuticals and Medical Devices Agency (PDMA) PMDA office

Therapeutic area or type of products

New Drugs I (three teams)

Anticancer, anti-infective, dermatology, additives for pharmaceutical products, and gastroenterology Cardiovascular disease, Alzheimer disease, Parkinson disease, obstetrics and gynecology, and urology Central and peripheral nervous system, and ophthalmology Respiratory disease, allergy, and autoimmune disease, Blood products, etc. Vaccines and tissue engineering

New Drugs II (two teams) New Drugs III (one team) New Drugs IV (two teams) Biologics I (one team) Biologics II (one team)

MINISTRY OF HEALTH, LABOUR AND WELFARE (MHLW) OF JAPAN

3

area and class of the drug. When biological products are assigned to a nonbiologic team, several reviewers from the Office of Biologics always join the review team to scrutinize the quality issues. The focus of review is different between INDs and NDAs. The IND review is to check the validity of initiating clinical trials, paying particular attention to safety concerns. The PAL requires that the IND review be finished within 30 days. The scope of the NDA review, on the other hand, is much broader, and it commonly takes one or more years for nonpriority review products. Figure 1 presents the review process for typical pharmaceutical NDAs. Review results prepared by review teams as well as the summary documents (i.e., Common Technical Documents [CTD] Module 2) submitted by the drug companies are published publicly on an Internet website in Japanese (http://www.info.pmda.go.jp/).

clinical development plans with Japanese regulators. Any sponsor (i.e., drug companies or physicians in sponsor-investigator trials) can apply for a consultation. Applications must be submitted 3 months ahead of the meeting. Due to insufficient review capacities with the agency, applications in crowded therapeutic areas are prioritized according to a point table. Some applicants with low priority points (e.g., a trial for an additional new indication) sometimes must wait several months in the queue. Though the PMDA explains that this ‘‘rationing’’ of services is inevitable with its current lack of sufficient review resources, the industry insists that this situation should be improved as soon as possible by adopting a fee system as in the U.S. Prescription Drug User Fee Act (PDUFA). Several types of consultations are offered to satisfy the needs in different development stages. Common times of consultation include before phase I, pre-early phase II, pre-late 2.4 Consultation Services Provided by the phase II, after phase II, and before NDA PMDA submission. The sponsors can ask for complementary consultation(s) for a lower fee if The PMDA provides various consultation serfurther discussions or advice are considered vices on a fee-for-service basis. The fees vary, necessary. depending on the type of consultation. For The sponsor and PMDA discuss design example, the fee for a consultation after a issues (e.g., choice of endpoints, sample size, phase II trial (i.e., the planning stage for statistical analysis), conduct issues (e.g., good phase III studies) is about 6 million yen (U.S. $55,000). The consultation services were started clinical practice [GCP] compliance), and all other issues pertinent to the clinical trials of in 1997 to meet the needs of the industry. interest. To clarify questions and concerns, Before that time, there were no official opportunities in which the industry could discuss preliminary discussions can be held several scientific and regulatory issues in specific times by facsimile and over the phone. The

PAFSC Special members Support

PAC & Committee

Support

Consult

PMDA Compliance review Team review

Figure 1. New Drug Application (NDA) review process in Japan.

NDA submission

Expert meeting

Interview

Hearing

Review Report

Interview

Applicant (Pharmaceutical company)

MHLW PMSB

Advise

Final decision on approval

Approval

4

MINISTRY OF HEALTH, LABOUR AND WELFARE (MHLW) OF JAPAN

PMDA and the sponsor commonly hold a final meeting to conclude the discussions. In cases in which all the questions and concerns are settled before the final meeting, the meeting can be canceled. The results of discussions are documented by the reviewers of the PMDA. The sponsor can comment on the results before they are finalized. All consultation result documents must be attached to an NDA submission in CTD Module 1. 3 REGULATORY STRUCTURE FOR CLINICAL TRIALS Most clinical trials in Japan, both industry sponsored and investigator sponsored, are done under the regulation of the PAL. For clinical trials outside the scope of the PAL, the MHLW issued a general ethics guideline for clinical research in 2003. 3.1 Definition of Clinical Trials The PAL requires submission of IND notification for a clinical trial that could be part of a clinical data package in an NDA. By definition, ‘‘sponsor’’ would include the physician who plans to conduct the clinical trials as the sponsor-investigator, but most of the trials for which IND notifications are submitted are sponsored by pharmaceutical companies. The regulatory scheme of sponsor-investigator trials was introduced comparatively recently. The PAL was amended to incorporate the definition of sponsor-investigator trials in

2002, and domestic GCP guidelines were also amended in 2003 to reflect the changes in the PAL. Before that, there were, of course, clinical trials implemented spontaneously by clinical researchers. However, they were not given official regulatory status and thus were not allowed to be used for Japanese NDA submissions. Since the introduction of the sponsor-investigator trials, 21 sponsor-investigator IND notifications have been submitted (as of November 2006). In contrast to the IND-related requirements in the United States, the definition of the Japanese IND requirement puts more emphasis on the intention for future NDA submission of pharmaceutical products. This leads to limited IND submissions from academic researchers, who are not necessarily interested in pharmaceutical NDAs. The number of recently submitted IND notifications is shown in Figure 2. The reasons for the drastic changes observed are explained in section 3.3. The PAL stipulates that IND notifications for substances for which clinical trials are done for the first time in Japan should be reviewed by the PMDA within 30 days. A review team assigned to the IND notification checks the IND; if they identify critical problems that could harm study participants, they issue an order to hold the trial. In such a case, the sponsor cannot start the trial until the problems are settled. This mechanism of clinical hold is similar to that of the United States.

1400 1200

First INDs All INDs

1000 800 600 400 200

Ye ar 19 93 19 94 19 95 19 96 19 97 19 98 19 99 20 00 20 01 20 02 20 03 20 04

0

Figure 2. Annual Investigational New Drug (IND) notifications submitted to the Ministry of Health, Labour and Welfare (MHLW).

MINISTRY OF HEALTH, LABOUR AND WELFARE (MHLW) OF JAPAN

3.2 GCP and Related Regulations in Japan The conduct of clinical trials for which IND notifications are submitted must satisfy the requirements of the GCP guideline and other pertinent ordinances. The Japanese GCP guideline is a Ministerial ordinance under the PAL, and serious violations of the guideline could be eligible for criminal punishment. It should be noted that the scope of penalties in the GCP is only for the sponsors; all of the other participants on the investigator’s side (e.g., physicians as investigators or subinvestigators, heads of hospitals, and research nurses) are exempt from the penalties. This exemption from punishment is an outstanding feature of Japanese GCP regulation. However, physicians could be punished in sponsor-investigator trials if a severe violation was committed when they played the role of the sponsor. The PMDA is responsible for implementation of on-site GCP inspections (see section 3.5). Lack of punishment on the investigator’s side makes it virtually impossible to publish a blacklist of investigators who have been involved in serious GCP violations in the past. 3.3 Drastic Changes in Japanese Clinical Trial Market As Figure 2 shows clearly, a drastic decline in the number of domestic commercial trials has been observed. This was caused by several factors on both the demand and supply sides (3). The MHLW’s health insurance policies in 1980s to 1990s, which included tightfisted price-setting rules for newly approved drugs and historical price cutting for existing drugs, seemed to have a negative impact on R&D activities in Japan (4). In addition to the economic disincentives, the globalization of R&D activities also has reduced the demand for Japanese trials. The International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) E5 guideline, called the ‘‘bridging guideline,’’ was accepted by the United States, the European Union, and Japan, and was implemented in 1998; this accelerated the trend toward global use of foreign clinical data (5). At the same time, the

5

MHLW abolished the longstanding requirements for the inclusion of Japanese pharmacokinetic studies, a Japanese dose-response study, and at least one Japanese confirmatory phase III study in the clinical data package. All these changes significantly reduced the demand for Japanese trials. Drastic changes also occurred on the supply side of clinical trials. After several years of discussions by the ICH expert working group, ICH-GCP was implemented in Japan in 1997. The new GCP guideline, for the first time in Japanese history, introduced Western components of clinical trials such as intensive monitoring and auditing through source data verification, support staff such as research nurses and pharmacists, contract research organizations, and independent data monitoring committees. These changes on the supply side, in conjunction with the changes on the demand side, caused a reduction in the number of trials in the Japanese market. The cost of Japanese trials skyrocketed accordingly. Most academic institutions and national hospitals adopted similar feesetting tables, but the fees are far from similar because overhead costs vary significantly among institutions (see section 3.4). 3.4 Expensive Japanese Trials for the Industry The Japanese pharmaceutical R&D environment, and the clinical environment in particular, has been discussed in light of launch delays in Japan. Several health-care providers have been associated with the delay in the background. Because virtually all Japanese clinical trials (except typical phase I trials) are done under the universal health insurance scheme, investigators as well as study participants face significant insurance-related red-tape situations in contracts, in-house procedures, and payment. For the sponsor, the cost of Japanese trials (i.e., payment from a drug company to a hospital) is a serious concern. It has been reported that the payments per subject are much higher in Japan than in the United States, European countries, and most Asian countries. In Japan, the sponsors are not allowed to pay investigators directly because the investigators (i.e., physicians) are regarded as employees of the hospital.

6

MINISTRY OF HEALTH, LABOUR AND WELFARE (MHLW) OF JAPAN

Instead, clinical trial fees go directly to the hospitals. As previously mentioned, in many public and academic hospitals, a clinical trial fee for each trial is determined based on a point-table and a matrix to calculate the fee basis, taking into consideration the design and conditions of the trial. The basic components of point-tables are similar nationwide, but actual fees vary to a great extent because of variable overhead costs. In general, hospitals have much a stronger power over the fee-setting process because they are the customers for the drug companies’ marketed products. The close tie between R&D activities and postmarketing business characterizes Japanese clinical development. Also, the limited availability of clinical experts in some therapeutic fields makes it possible for them to exert pricing powers as monopolists or oligopolists in providing clinical trials. All these features seem to be associated with the exceptionally high prices of clinical trials in Japan. 3.5 Quality of Japanese Trials Until the ICH-GCP was introduced, Japanese trials had a generally poor reputation for quality. Even before the ICH-GCP introduction, GCP guidance had been issued in 1989, but it was not a legally binding ordinance based on the PAL. Western components of clinical trials such as source data verification (SDV) monitoring and auditing, research nurses, and strict rules for informed consent were introduced in Japan for the first time in 1997, along with the ICH-GCP. The actual reports of GCP inspections are not publicly available, but the PMDA routinely publishes summary findings for educational purposes and provides materials on the quality of Japanese trials (6). Before the introduction of the ICH-GCP, the most obvious deficiencies were errors in case report forms (CRFs). Some errors were serious (e.g., fraudulent discrepancies from medical records), but most of them were trivial such as typographical errors. Surprisingly, SDV was not officially done before the ICH-GCP in Japan because there was no regulatory basis to allow SDV with appropriate protection of personal information. The ICH-GCP

made it possible for drug companies to conduct intensive monitoring through SDV. Also, research nurses and pharmacists employed at clinical institutions started operations to support investigators and drug companies. These improvements in the research environment reduced CRF deficiencies drastically and thus increased the accuracy of the data. Some types of deficiencies have not decreased, however. According to the published summary of inspection results, the number of protocol deviations has not declined since the ICH-GCP introduction (6). Japanese investigators participate in clinical trials as employees of hospitals and within the restrictions of health insurance regulations. Their motivation is not necessarily high because clinical trials have historically been considered low-priority chores that their supervisors (e.g., professors, directors) order them to do to acquire research grants from drug companies. Academic incentives are not effective for many physicians because they know Japanese commercial trials are rarely accepted in prestigious medical journals. This mindset has been changing, but only gradually. Recent discussions on trial quality often focus on burdens related to handling documents. For example, there has been widespread misinterpretation of the scope of essential documents. Too much attention to paper handling is likely to increase the cost of monitoring activities and the cost of clinical trials, accordingly. 4 GOVERNMENT EFFORTS TO INVITE NEW DRUGS TO JAPAN For several reasons, some drugs already marketed in the United States and European Union are not introduced in Japan (4). Even when they are finally introduced in Japan, significant delays in market introduction are quite common. This situation, called ‘‘new drug lag,’’ has attracted a great deal of public attention, especially in cancer therapy. That many drugs and therapeutic regimens already approved in Western countries are unavailable or are not approved in Japan has caused Japanese policy-makers to establish regulatory schemes to bring those drug therapies to Japan.

MINISTRY OF HEALTH, LABOUR AND WELFARE (MHLW) OF JAPAN

4.1 Study Group on Unapproved Drugs (MHLW since January 2005) The Study Group on Unapproved Drugs started in January 2005 to oversee the situation of Japanese new drug lag, to evaluate the need for introducing unapproved drugs in Japan, and to lead these drugs to the clinical development stage. Ten formal meetings have been held since the group’s establishment, and development and approval strategies for 33 drugs have been discussed (as of November 2006). Of the 33 drugs, 20 drugs were cancer drugs, and nine were pediatric drugs. The study group also makes recommendations about clinical trials for safety confirmation that should be performed for drugs that have already been submitted for NDA because such trials could provide opportunities for patients to access these unapproved drugs during the NDA review. Compassionate use of investigational or unapproved drugs has not been legitimized in Japan. 4.2 Study Group on Cancer Combination Therapy (MHLW since January 2004) The Study Group on Cancer Combination Therapy specifically focuses on unapproved use (e.g., indications) or unapproved regimens of cancer drugs for which Japanese approval has already been obtained for some other indication(s) or regimen. Because drug companies would have to bear substantial costs to conduct clinical trials to expand these indications, they would rather maintain the indications unchanged and leave the decision on how to use the drugs to individual oncologists. As a result, many cancer drugs have been used for unapproved conditions or in unapproved regimens in Japan. Lack of flexibility in Japanese health insurance reimbursement decisions makes the situation more serious. For pharmaceutical products, reimbursement is strictly based on the conditions (i.e., indication, dose, regimen, patient type) of approval under the PAL. Japanese insurance bodies basically are not allowed to make their own decisions on reimbursement under the universal health-care insurance, a marked contrast to the diversified decisions by the U.S. insurance bodies.

7

To fill the gap between Japan and the United States/European Union, this MHLW study group investigates the current Japanese situation and also collects clinical evidence available in foreign countries. When the investigation shows that a given drug use is considered ‘‘standard’’ in the therapeutic field, the MHLW issues its approval based on those prior investigations and foreign evidence. In 2005, expanded indications were approved for 30 cancer drugs. 5 GOVERNMENT EFFORTS TO PROMOTE CLINICAL TRIALS IN JAPAN The MHLW and related government agencies have been implementing strategies to boost the number and quality of Japanese clinical trials. 5.1 Three-Year Clinical Trial Promotion Plan (MHLW and MEXT, 2003–2005) The MHLW and the Ministry of Education, Culture, Sports, Science and Technology (MEXT) jointly executed a set of programs to promote Japanese clinical trials, Three-Year Clinical Trial Promotion Plan. Setting up several large networks for clinical trials was one of the important objectives of this plan. Research grants to establish networks and implement some sponsorinvestigator trials as model programs were managed by the Center for Clinical Trials of the Japan Medical Association. As of September 2006, there are 1,212 clinical institutions participating in the networks. Training programs for clinical research coordinators (e.g., research nurses and pharmacists) were provided regularly under this plan. Several symposiums were also held to raise public awareness of clinical trials because it is believed that Japanese patients who receive medical services under the universal health insurance without substantial financial burdens are not necessarily interested in clinical trials. The national hospitals and research centers directly managed by the MHLW have been promoting harmonization of trial contract documents. These streamlining efforts also are intended to reduce the administrative costs of trials.

8

MINISTRY OF HEALTH, LABOUR AND WELFARE (MHLW) OF JAPAN

The MHLW plans to continue these activities for the next 5 years beginning with the fiscal year 2007. 5.2 Study Group on Faster Access to Innovative Drugs (MHLW, 2006) The MHLW convened an expert panel, the Study Group on Faster Access to Innovative Drugs, to discuss the bottlenecks to the introduction of new drugs in Japan in October 2006. This study group will focus on the principles of new drug approval, postmarketing safety measures, and enhancement of the PMDA review. It is expected that the discussion results will be reflected in upcoming amendments of existing regulations. REFERENCES 1. Y. Fujiwara and K. Kobayashi, Oncology drug clinical development and approval in Japan: the role of the pharmaceuticals and medical devices evaluation center (PMDEC). Crit Rev Oncol Hematol. 2002; 42: 145–155. 2. Center for Drug Evaluation and Research, U.S. Food and Drug Administration. Training and Communications: Formal meetings between CDER and CDER’s external constituents. MAPP 4512.1. Available at: http://web.archive.org/web/20050921092516 /http://www.fda.gov/cder/mapp/4512-1.pdf 3. S. Ono and Y. Kodama, Clinical trials and the new good clinical practice guideline in Japan. Pharmacoeconomics. 2000; 18: 125–141.

4. P. M. Danzon, The impact of price regulation on the launch delay of new drugs—evidence from twenty-five major markets in the 1990s. Health Econ. 2005; 14: 269–292. 5. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH Harmonised Tripartite Guideline: E5(R1) Ethnic Factors in the Acceptability of Foreign Clinical Data. Step 4 version, February 5, 1998. Available at: http://www.ich.org/LOB/media/ MEDIA481.pdf 6. K. Saito, Y. Kodama, S. Ono, M. Mutoh, S. Kawashima, and A. Fujimura, Quality of Japanese trials estimated from Good Clinical Practice auditing findings. Am J Ther. 2006; 13: 127–133.

FURTHER READING For more detailed information about Japanese regulations, see Japan Pharmaceutical Manufacturers Association (JPMA), Pharmaceutical Administration and Regulations in Japan, March 2006. Available from http://www.jpma.or.jp/english/parj/0607.html

CROSS-REFERENCES Good clinical practice (GCP) International Conference on Harmonization (ICH) Investigational New Drug Application (IND) Regulatory authorities Food and Drug Administration (FDA)

MIN TEST

the name by Gleser (2), and was further studied by Berger (3–5) and others. Saikali and Berger (6) have pointed out that the Min test and the simple IUT are alternative names for the same test. Note that the elementary tests are performed at the same α level as the desired overall size of the test, and the test is valid even if the individual test statistics are correlated. Here, validity means that the probability of a type 1 error is at most α. The conclusion that the test has the claimed probability of a type 1 error follows only if inference is restricted to accepting or rejecting the specified global null and alternative hypotheses previously discussed. No consideration in calculating the size of the test is given to other errors that could arise if other inferences are drawn. For example, suppose that the global null is not rejected but one or more of the component tests reject their elementary null Hoi . If it were desired to reach conclusions about the corresponding parameters, it would be necessary to take into account the possibility of additional errors in order to control the familywise error rate. There are many ways to control the multiplicity problem but reporting on significant findings in the computation of the components of the Min test is not one of them. After failing to find min[ti − ciα ] ≥ 0 for all i, the only statistically valid statement is that there is insufficient evidence to conclude that all of the parameters are positive. The test statistics ti can be parametric, such as a t-test, or it could be nonparametric, such as a rank test. The parameters appearing in the hypotheses may represent a single outcome measure obtained under different experimental conditions, or the parameters could arise from a multivariate outcome. Both cases occur when testing whether a combination comprising several treatments is efficacious.

EUGENE M LASKA MORRIS J MEISNER Nathan Kline Institute for Psychiatric Research New York University School of Medicine Orangeburg, New York

Multiple parameter testing arises in many if not most clinical trials. There are very few diseases for which treatments are directed at a single targeted endpoint. In many cases, the evidence required to demonstrate that a new treatment is effective requires the simultaneous comparison of many outcome measures. This translates to formal hypotheses involving inequalities among multiple parameters. One common case is testing whether the parameters are all positive. This arises, for example, when it is desired to show that a combination treatment is superior to each component; that a treatment is equivalent or noninferior to a control; or that a dose of a combination is synergistic. In such cases, the Min test is the test of choice. We introduce this test as well as present and discuss its properties and illustrative applications. 1

THE MIN TEST

The Min test is used to test a null hypothesis such as H o : υ i ≤ 0 for at least one i verses H 1 : υ i > 0 for all i, i = 1, 2, . . . , K. This is sometimes called the sign-testing problem. If ti is an α-level test of υ i ≤ 0 that rejects H oi : υ i ≤ 0 if the statistic is larger than its critical value ciα , then the Min test rejects the global null hypothesis Ho if the min[ti − ciα ] > 0. Alternatively, if pi is the observed P-value of the i-th test statistic, then the Min test rejects the null if the largest value of pi < α. This test is also called the simple intersection union test (simple IUT or the SIUT) because the form of the null hypothesis is the union of K ‘‘elementary’’ hypotheses, and the alternative is the intersection of the complement of the elementary hypotheses. The simple IUT was first described by Lehman (1), was given

1.1 Power The statistical power (probability of rejecting the null hypothesis) of an α-level Min test for K = 2 depends on the primary parameter δ, the minimum of υ 1 and υ 2 , and on the nuisance parameter γ , the difference between

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

MIN TEST

υ 1 and υ 2 . If the parameters υ i are means, given any fixed value of δ, the power of the α-level Min test increases as a function of the absolute value of γ . Therefore, given any value of the primary parameter, the α-level Min test has the smallest power when the nuisance parameter is zero, that is, when υ1 = υ2. In determining the sample size for a study, in the absence of prior knowledge to the contrary, a prudent statistician will assume that the means are equal to ensure that there is sufficient power for rejecting the null hypothesis. For example, for two normal random variables with effect sizes υ 1 /σ = υ 2 /σ = 0.5, to achieve 90% power to reject the null, the sample size per group is 84 based on a t-test and 97 per group based on a Wilcoxon test. If υ 1 /σ were larger than 0.5, then γ would be positive, and the power of the Min test would be greater than 0.90. The size of a test, α, is the supremum of the power taken over all values of the parameters that lie in the null space. For the sign-testing problem, where the parameters are the means of a bivariate normal, the parameter values that define α occur in the limit of (υ 1 , 0) and (0, υ 2 ) as υ 1 or υ 2 approaches positive infinity. A test is said to be biased if there is a parameter point in the alternative hypothesis space (so that the null should be rejected) for which the power to reject is smaller than the size of the test. Thus, biased tests reject the null more frequently when the null is true than when it is false. Unfortunately, for parameters in the alternative space that lie in the neighborhood of the origin, the size of the Min test is very small. Therefore, the test is biased, and the power is low near the origin. The power of the Min test has been considered under different distributional assumptions by many investigators, including Laska and Meisner (7, 8), Patel (9), Hung et al. (10), Hung (11), Wang and Hung (12), Horn et al. (13) Sidik and Jonkman(14), and Kong et al. (15, 16). The tables or formulas in their publications may be consulted to obtain sample sizes for specific circumstances. 1.2 Optimality Properties If the test statistics are restricted to the class of monotone functions of the test statistics

of the elementary hypotheses (T 1 , T 2 ), then, under some mild regularity conditions, the uniformly most powerful test of Ho is the Min test (1, 7). A test is said to be monotone in (T 1 , T 2 ) if (a, b) are values of the test statistics that lie in the critical region implies that (a’, b’) also lies in the critical region whenever a ≥ a and b ≥ b. That is, larger values of the test statistic provide greater evidence in support of rejection. The sign-testing problem was considered by Inada (17) for bivariate normal and Sasabuchi (18) for multivariate normal; they found that the Min test is the likelihood ratio test (LRT) under the assumption of equal sample sizes per group and known variance. They specified the null as H o :{υ 1 = 0, υ 2 > 0} or {υ 1 > 0, υ 2 = 0} rather than H o : υ 1 ≤ 0 or υ 2 ≤ 0, but later Berger (4) showed that the LRT is the same for both formulations of the null. When the sample sizes are unequal, however, Saikali and Berger (6) showed that the two tests are not the same. The rejection region of the LRT is a proper subset of the rejection region of the Min test, which is therefore uniformly more powerful. However, the Min test is biased and has low power when the parameters are close to the origin, and the size of the test α is attained only in the limit. In an effort to remedy this situation, Berger (4), Liu and Berger (19), and McDermott and Wang (20) enlarged the rejection region near the origin without changing the α level and obtained tests that are uniformly more powerful than the Min test. But this result does not come without a price. In particular, the new tests are not monotone. Laska and Meisner (7) have argued that in a regulatory setting, such a property would be unacceptable. Imagine the consequences of one randomized, controlled clinical trial with less evidence for rejection of the null being declared a pivotal study by the regulatory body while another study with more evidence does not reach the nominal level of statistical significance. Additionally, these ‘‘improved’’ new tests all contain points in the rejection region that are arbitrarily close to the origin, which of course is not good evidence against the null hypothesis. Perlman and Wu (21) have argued strongly against the use of these new tests. Hung (22) has cautioned that their value in increasing power must be carefully

MIN TEST

weighed against the shortcomings already discussed. 2 COMMON APPLICATIONS OF THE MIN TEST The Min test is useful in many areas of application, but its most common use has been in testing whether a combination treatment is efficacious. Such treatments are widely used, and, even though their components may be effective when used individually, it does not follow that the combination consisting of these components is useful. To market a combination treatment, the U.S. Food and Drug Administration (U.S. FDA 21CFR300.50) requires demonstration that ‘‘each component makes a contribution to the claimed effects . . . ’’ (23, 24). In addition, the European Agency for the Evaluation of Medical Products (EMEA) also requires that the benefit/risk assessment of the combination equals or exceeds the ratio of each of its components (25). 2.1 Combination Treatments: Single Endpoint There are many ways to interpret the concept of ‘‘contribution.’’ If all of the components of the combination treat the same symptom, such as pain, then each component ‘‘contributes’’ is interpreted as requiring that the combination be superior to each of its components in terms of effectiveness. Clearly, if the combination is not superior to one of its ingredients, then, other issues such as adverse events aside, the combination has no advantage over the ingredient. Suppose there are K components in the combination treatment. Then here, υ i is the difference between the mean effect of the combination and the mean effect of component treatment i. If ti is a one-sided test that rejects H oi : υ i ≤ 0 if the statistic is larger than its critical value ciα , then the Min test may be used to demonstrate that the components of a combination contribute to the claimed effects. 2.2 Combination Treatments: Multiple Endpoints A combination treatment may be designed to treat many symptoms, such as pain and

3

sleeplessness. For simplicity, suppose there are two components in the combination treatment, each of which may have an effect on J outcome measures. Let υ ij be the difference between the mean effect of the combination and the mean effect of component treatment i on the j-th outcome measure. The notion of contribution has multiple interpretations. Laska et al. (26) gave the definitions for uniformly best, comparable to the best, and admissible, and Tamhane and Logan (27) provided the definition for locally best: 1. Uniformly best. The combination treatment is superior to each of its component treatments on all J endpoints. 2. Locally best. For each endpoint, the combination treatment is at least as good as (noninferior to) the component treatment that is best for that endpoint and superior to each treatment on at least one endpoint. 3. Comparable to the best. For each endpoint, the combination treatment is at least as good as (noninferior to) the component treatment that is best for that endpoint. 4. Admissible. The combination treatment is superior to each component treatment on at least one endpoint. Laska et al. (26) gave a general theory for these hypothesis formulations and applied the theory when the random variables are multivariate normal and when the tests of the elementary hypotheses are rank tests. If tij is a one-sided test that rejects Hoij : υ ij ≤ 0 when the statistic is larger than its critical value cijα , then the Min test may be used to demonstrate that the combination is uniformly best. Under mild conditions, it too is uniformly most powerful (UMP)among the class of monotone functions of the test statistics tij . Tamhane and Logan (27) describe a test for demonstrating locally best. The alternative hypothesis is the intersection of the superiority alternative and the noninferior alternative. For each outcome j, a noninferiority margin ej must be chosen to meaningfully reflect clinical indifferences. If sij is

4

MIN TEST

an α/J-level test of υ ij ≤ 0 that rejects H oij : υ ij ≤ 0 if the statistic is larger than its critical value c*ijα , and tij is an α-level test of υ ij − ≤ ej , that rejects Hoij : υij − ≤ ej if the statistic is larger than its critical value cijα , then the test rejects Ho if simultaneously the Min test given by min[tij − cijα ] and max[sij − c∗ ijα/J ] are both positive. The test uses Roy’s (28) union intersection (UI) principle and the Min test if the endpoint on which the combination is superior is not specified in advance. If j∗ is the endpoint specified in advance for testing superiority, then the test is a Min test composed of min[tij − cijα ]>0 and max[sij∗ − c∗ ij∗ α ] > 0. To test the hypothesis of comparable to the best, Laska et al. (26) show that the Min test once again is a UMP test among the class of monotone functions of the test statistics tij . The component test tij rejects Hoij : υ − ≤ ej if the corresponding test statistic is larger than its critical value. These same tests were later used by Tamhane and Logan (27) in the noninferiority portion of their test for locally best. For the admissibility hypothesis, Laska et al. (26) compare the combination with each component treatment to determine if there is at least one endpoint where it is superior to that component. Inference about each component i involves J comparisons, so the size of these J tests must be adjusted to achieve an α-level test for the component. The proposed adjustment was based on the Bonferroni method, so the test for component i is max[sij − c∗ ijα/J ] > 0. The test in Laska et al. (26) then is min[max[sij − c∗ ijα/J ]] > 0. Westfall et al. (29) observed that the power of the test could be improved by replacing Bonferroni with Holm (30) and still more by using the Simes (31) and Hommel (32) approaches instead of Bonferroni. It is interesting to note that some of the definitions of contribution listed above have inclusion relations. If a combination is uniformly best, then it is locally best. Similarly, a locally best combination is comparable to the best, and it is admissible. There is no inclusion relationship between the latter two. Thus, in any application, based on closed testing, the admissibility hypothesis may be tested first; and if it is not rejected, the locally

best hypothesis may be tested; and if it is not rejected, then uniformly best can be tested. These tests are all conducted at the stated α level without adjustment for multiplicity. Alternatively, the comparable to the best hypothesis may be tested; and if not rejected, the locally best hypothesis may be tested; and if not rejected, uniformly best may be tested. This may not be prudent unless a specific endpoint for locally best is chosen in advance; otherwise, power is lost in seeking the endpoint where the combination is best. This power loss occurs because part of the overall type 1 error in testing locally best is spent in the contrast for each of the endpoints. Thus, if the goal is to test for uniformly best, there is no cost to either first testing for admissibility or first testing for comparable to the best. 2.3 Combination Treatments: Multiple Doses, Univariate Effect Particularly for antihypertensive treatments but for other indications as well, both the United States and the European Union require that dose-ranging studies be conducted, preferably using a factorial design. In such a design, patients are randomly allocated to receive one of the doses of drug A and one of the doses of drug B, which includes the possibility of the zero dose, placebo. Hung et al. (33, 34) considered the statistical problem of identifying which dose combinations have the property that both components make a contribution. These investigators recognized that such a design also yields information on the dose response of the combination. Just as for combination drugs treating multiple symptoms, the definition of ‘‘contributing’’ has many interpretations. Hung et al. (33) described contributing in both a weak and a strong sense. A combination exhibits global superiority in a weak sense if the average effect of the combination taken over all of the nonzero doses is superior to the average effect of each of the component treatments taken over its corresponding individual non-zero doses. A combination exhibits global superiority in a strong sense if there is at least one non-zero dose combination that is superior to both of its components. Notice that demonstration of weak superiority does

MIN TEST

not guarantee that there is a dose combination that is superior to its components. Hung et al. (10) developed two tests for global superiority. The α-level AVE test is an average of the Min test statistics examining the contribution of each combination relative to its components taken over all of the non-zero dose combinations under study. The MAX test examines the largest Min test statistic. Both are one-sided α-level tests. Hung (35) extended the tests to incomplete and unbalanced factorial designs in which not all dose combinations are studied and the sample sizes of the dose pairs are not equal. For the local problem of identifying which explicit dose pairs are contributing, they recommend using simultaneous Min tests adjusted according to the Hochberg (36) procedure to maintain the familywise error rate at α. Alternative multiple testing methods that protect the familywise error rate for identifying effective dose combinations were investigated by Buchheister and Lehmacher (37) and Lehmacher (38). They proposed procedures based on linear contrast tests and on closed testing procedures. 2.4 Synergy In some contexts, it is desirable to determine whether the components of a combination are additive, synergistic, or antagonistic at specific doses. These concepts specify whether the effect of the combination is, respectively, the same as, greater than, or less than expected on the basis of the effects of its components (39). Another concept is the notion of therapeutic synergy. A combination is said to be therapeutically synergistic if its effect is larger than the maximum effect achievable by any of its components over all doses in its therapeutic range. Laska et al. (40) proposed a simple approach to assessing synergy in a combination of two drugs that does not require modeling a response surface or marginal doseresponse curves. Their experimental design and test are based on the concept of an isobologram (39) to establish sufficient conditions for synergy at a specified dose (x1 , x2 ) of the combination. An isobole is the set of dose pairs from the two treatments, all of which have the same expected responses.

5

The design calls for studying the combination and a single dose of each drug. Suppose that x1e and x2e are the doses of drug 1 and drug 2, respectively, that produce the same level of response, say e. The potency ratio at e is defined to be ρ(e) = x2e /x1e . In many instances, ρ(e) is well studied, and an estimate r is available. Then, to investigate whether the combination is synergistic at (x1 , x2 ), N subjects are randomly assigned to each of three dose combinations: (x1 + x2 /r, 0), (0, r x1 + x2 ), and (x1 , x2 ). The Min test is used to see if the following two conditions hold simultaneously: 1. The response at the combination dose (x1 , x2 ) is greater than the response to drug 1 alone at dose (x1 + x2 /r, 0). 2. The response at the combination dose of interest is greater than the response to drug 2 alone at dose (0, r x1 + x2 ). If both these conditions are true, then synergy can be claimed at the combination dose of interest. Recently, Feng and Kelly (41) generalized this approach to three or more drugs and studied the power of the procedure. To test if a combination is therapeutically synergistic at a given dose pair (x1 , x2 ), its effect must be shown to be superior to the maximum effect achieved by its components at any dose. Suppose that xe1∗ and xe2∗ are the known doses of drug 1 and drug 2 that produce the maximum level of response, say e1 * and e2 *, respectively. Then, component i at dose xei∗ and the combination must be studied, and the Min test is an optimal test. 2.5 Bioequivalence, Equivalence and Noninferiority Two formulations of the same drug are said to be bioequivalent if they are absorbed into the blood and become available at the drug action site at about the same rate and concentration. Suppose υ T = mean blood concentration of a test treatment and υ S = mean blood concentration of a standard treatment. Then, to demonstrate bioequivalence the null hypothesis H0 : υT − υS ≥ δ or υT − υS ≤ −δ must be rejected in favor of HA : − δ < υT − υS < δ

6

MIN TEST

which is equivalent to HA : − δ < υT − υS and υT − υS < δ Here, δ > 0 is a prespecified tolerance limit. More generally, this representation of equivalence may be used for any clinically relevant outcome measure where δ demarks a zone of clinical indifference. Bioequivalence refers to measures of blood concentration of the drug, whereas equivalence refers to outcomes that measure the impact on the therapeutic target. The Min test may be used to test the null hypothesis. In the bioequivalence context, the test is often referred to as the TOST because it is composed of two one-sided α-level tests (42). Kong et al. (15) considered a multivariate setting and studied a method for demonstrating equivalence or noninferiority of two treatments on each endpoint under the assumption of multivariate normality. The test statistic is a Min test that takes into account a noninferiority margin ej for each outcome j that reflects meaningful clinical indifference. The form of the test and the form for testing whether a combination treatment is uniformly best are the same. After developing the distributional properties, the investigators used a simulation to examine the power of the test under different configurations of the means and covariance matrix. More recently, Kong et al. (16) considered a similar scenario in which the endpoints are distributed binomially.

REFERENCES 1. E. L. Lehmann, Testing multiparameter hypotheses. Ann Math Stat. 1952; 23: 541–552. 2. L. J. Gleser, On a theory of intersection-union tests [abstract]. Inst Math Stat Bull. 1973; 2: 233. 3. R. L. Berger, Multiparameter hypothesis testing and acceptance sampling. Technometrics. 1982; 24: 295–300. 4. R. L. Berger, Uniformly more powerful tests for hypotheses concerning linear inequalities and normal means. J Am Stat Assoc. 1989; 84: 192–199.

5. R. L. Berger, Likelihood ratio tests and intersection-union tests In: S. Panchapakesan and N. Balakrishnan (eds.), Advances in Statistical Decision Theory and Applications Boston: Birhauser, 1997, pp. 225–237. 6. K. G. Saikali and R. L. Berger, More powerful tests for the sign testing problem. J Stat Plan Inference. 2002; 107: 187–205. 7. E. M. Laska and M. Meisner, Testing whether an identified treatment is best. Biometrics. 1989; 45: 1139–1151. 8. E. M. Laska and M. Meisner, Testing whether an identical treatment is best: the combination problem In: Proceedings of the Biopharmaceutical Section of the American Statistical Association Alexandria, VA: American Statistical Association, 1986, pp. 163–170. 9. H. I. Patel, Comparison of treatments in a combination therapy trial. J Biopharm Stat. 1991; 1: 171–183. 10. H. M. Hung, G. Y. Chi, and R. J. Lipicky, Testing for the existence of a desirable dose combination. Biometrics. 1993; 49: 85–94. 11. H. M. Hung, Two-stage tests for studying monotherapy and combination therapy in twoby-two factorial trials. Stat Med. 1993; 12: 645–660. 12. S. J. Wang and H. M. Hung, Large sample tests for binary outcomes in fixed-dose combination drug studies. Biometrics. 1997; 53: 498–503. 13. M. Horn, R. Vollandt, and C. W. Dunnett, Sample size determination for testing whether an identified treatment is best. Biometrics. 2000; 56: 879–881. 14. K. Sidik and J. N. Jonkman, Sample size determination in fixed dose combination drug studies. Pharm Stat. 2003; 2: 273–278. 15. L. Kong, R. C. Kohberger, and G. G. Koch, Type I error and power in noninferiority/equivalence trials with correlated multiple endpoints: an example from vaccine development trials. J Biopharm Stat. 2004; 14: 893–907. 16. L. Kong, G. G. Koch, T. Liu, and H. Wang, Performance of some multiple testing procedures to compare three doses of a test drug and placebo. Pharm Stat. 2005; 4: 25–35. 17. K. Inada, Some bivariate tests of composite hypotheses with restricted alternatives. Rep Fac Sci Kagoshima Univ (Math Phys Chem). 1978; 11: 25–31. 18. S. Sasabuchi, A test of a multivariate normal mean with composite hypotheses determined by linear inequalities. Biometrika. 1980; 67: 429–439.

MIN TEST 19. H. Liu and R. L. Berger, Uniformly more powerful, one sided tests for hypotheses about linear inequalities. Ann Stat. 1995; 72: 23–55. 20. M. P. McDermott and Y. Wang, Construction of uniformly more powerful tests for hypotheses about linear inequalities. J Stat Plan Inference. 2002; 107: 207–217. 21. M. D. Perlman and L. Wu, The emperor’s new tests (with discussion). Stat Sci. 1999; 14: 355–369. 22. H. M. Hung, Combination drug clinical trial In: S. C. Chow (ed.), Encyclopedia of Biopharmaceutical Statistics, Revised and Expanded 2nd ed. New York: Marcel Dekker, 2003, pp. 210–213. 23. U.S. Food and Drug Administration, Department of Health and Human Services. Code of Federal Regulations, Title 21 Food and Drugs. Volume 5, Part 300.50: Fixedcombination prescription drugs for humans. Revised as of April 1, 1999. Available at: http://www.access.gpo.gov/nara/cfr/waisidx 99 /21cfr300 99.html 24. H. M. Leung and R. O’Neill, Statistical assessment of combination drugs: a regulatory view In: Proceedings of the Biopharmaceutical Section of the American Statistical Association Alexandria, VA: American Statistical Association, 1986, pp. 33–36. 25. European Agency for the Evaluation of Medicinal Products, Human Medicines Evaluation Unit, Committee for Proprietary Medicinal Products (CPMP). Note for Guidance on Fixed Combination Medicinal Products. CPMP/EWP/240/95. April 17, 1996. Available at: http://www.emea.europa.eu/ pdfs/human/ewp/024095en.pdf 26. E. M. Laska, D. I. Tang, and M. Meisner, Testing hypotheses about an identified treatment when there are multiple endpoints. J Am Stat Assoc. 1992; 87: 825–831. 27. A. C. Tamhane and B. R. Logan, A superiority-equivalence approach to one-sided tests on multiple endpoints in clinical trials. Biometrika. 2004; 91: 715–727. 28. S. N. Roy, On a heuristic method of test construction and its use in multivariate analysis. Ann Math Stat. 1953; 24: 220–238. 29. P. H. Westfall, S. Y. Ho, and B. A. Prillaman, Properties of multiple intersection-union tests for multiple endpoints in combination therapy trials. J Biopharm Stat. 2001; 11: 125–138. 30. S. Holm, A simple sequentially rejective test procedure. Scand J Stat. 1979; 6: 65–70.

7

31. R. J. Simes, An improved Bonferroni procedure for multiple tests of significance. Biometrika. 1986; 73: 751–754. 32. G. Hommel, A comparison of two modified Bonferroni procedures. Biometrika. 1988; 75: 383–386. 33. H. M. Hung, T. H. Ng, G. Y. Chi, and R. J. Lipicky, Response surface and factorial designs for combination antihypertensive drugs. Drug Inf J. 1990; 24: 371–378. 34. H. M. Hung, G. Y. Chi, and R. J. Lipicky, On some statistical methods for analysis of combination drug studies. Commun Stat Theory Methods. 1994; A23: 361–376. 35. H. M. Hung, Evaluation of a combination drug with multiple doses in unbalanced factorial design clinical trials. Stat Med. 2000; 19: 2079–2087. 36. Y. Hochberg, A sharper Bonferroni procedure for multiple tests of significance. Biometrika. 1988; 75: 800–802. 37. B. Buchheister and W. Lehmacher, Establishing desirable dose combinations with multiple testing procedures In: N. Victor, M. Blettner, L. Edler, R. Haux, P. Knaup-Gregori, et al. (eds.), Medical Informatics, Biostatistics and Epidemiology for Efficient Health Care and Medical Research Munchen, Germany: Urban & Vogel, 1999, pp. 18–21. 38. B. Buchheister and W. Lehmacher, Multiple testing procedures for identifying desirable dose combinations in bifactorial designs. GMS Med Inform Biom Epidemiol. 2006; 2 (2): Dec07. 39. M. C. Berenbaum, What is synergy?. Pharmacol Rev. 1989; 41: 93–141. 40. E. M. Laska. M. Meisner, and C. Siegel, Simple designs and model-free tests for synergy. Biometrics. 1994; 50: 834–841. 41. P. Feng and C. Kelly, An extension of the model-free test to test synergy in multiple drug combinations. Biometric J. 2004; 3: 293–304. 42. R. L. Berger and J. C. Hsu, Bioequivalence trials, intersection-union tests and equivalence confidence sets. Stat Sci. 1996; 11: 283– 319.

FURTHER READING Office of Combination Products, U.S. Food and Drug Administration Website, at http://www.fda.gov/oc/combination/

8

MIN TEST

CROSS-REFERENCES Bioavailability Combination therapy Multiple comparisons Multiple endpoints Noninferiority

MISSING DATA

a nonrandom process is nonignorable. Thus, under ignorable dropout, one can literally ignore the missingness process and nevertheless obtain valid estimates of, say, the treatment. The above definitions are conditional on including the correct set of covariates into the model. An overview of the various mechanisms, and their (non-)ignorability under likelihood, Bayesian, or frequentist inference, is given in Table 1. Let us first consider the case in which only one follow-up measurement per patient is made. When dropout occurs in a patient, leaving the investigator without follow-up measures, one is usually forced to discard such a patient from analysis, thereby violating the intention to treat (ITT) principle, which stipulates that all randomized patients should be included in the primary analysis and according to the randomization scheme. Of course, the effect of treatment can be investigated under extreme assumptions, such as, for example, a worst-case and a best-case scenario, but such scenarios are most often not really helpful. The focus of this article will be on analysis techniques for repeated measurements studies. Early work regarding missingness focused on the consequences of the induced lack of balance of deviations from the study design (2, 3). Later, algorithmic developments took place, such as the expectation-maximization algorithm (EM) (4) and multiple imputation (5). These have brought likelihood-based ignorable analysis within reach of a large class of designs and models. However, they usually require extra programming in addition to available standard statistical software. In the meantime, however, clinical trial practice has put a strong emphasis on methods such as complete case analysis (CC) and last observation carried forward (LOCF) or other simple forms of imputation. Claimed advantages include computational simplicity, no need for a full longitudinal model analysis (e.g., when the scientific question is in terms of the last planned measurement occasion only), and for LOCF, compatibility

GEERT MOLENBERGHS Universiteit Hasselt Center for Statistics Diepenbeek, Belgium

EMMANUEL LESAFFRE Catholic University of Leuven Leuven, Belgium

1

INTRODUCTION

Data from longitudinal studies in general, and from clinical trials in particular, are prone to incompleteness. As incompleteness usually occurs for reasons outside of the control of the investigators and may be related to the outcome measurement of interest, it is generally necessary to reflect on the process governing incompleteness. Only in special but important cases is it possible to ignore the missingness process. When patients are examined repeatedly in a clinical trial, missing data can occur for various reasons and at various visits. When missing data result from patient dropout, the missing data have a monotone pattern. Nonmonotone missingness occurs when there are intermittent missing values as well. The focus here will be on dropout. Reasons typically encountered are adverse events, illness not related to study medication, uncooperative patient, protocol violation, ineffective study medication, loss to follow-up, and so on. When referring to the missing-value, or nonresponse, process, we will use the terminology of Little and Rubin (1). A nonresponse process is said to be missing completely at random (MCAR) if the missingness is independent of both unobserved and observed data and missing at random (MAR) if, conditional on the observed data, the missingness is independent of the unobserved measurements. A process that is neither MCAR nor MAR is termed nonrandom (MNAR). In the context of likelihood inference, and when the parameters describing the measurement process are functionally independent of the parameters describing the missingness process, MCAR and MAR are ignorable, whereas

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

MISSING DATA

with the ITT principle. However, a CC analysis assumes MCAR, and the LOCF analysis makes peculiar assumptions about the (unobserved) evolution of the response, underestimates the variability of the response, and ignores the fact that imputed values are no real data. On the other hand, a likelihood-based longitudinal analysis requires only MAR, uses all data (obviating the need for both deleting and filling in data), and is consistent with the ITT principle. Furthermore, it can also be shown that the incomplete sequences contribute to estimands of interest (treatment effect at the end of the study), even early dropouts. For continuous responses, the linear mixed model is popular and is a direct extension of analysis of variance (ANOVA) and MANOVA approaches, but more broadly valid in incomplete data settings. For categorical responses and count data, so-called marginal (e.g., generalized estimating equations, GEEs) and randomeffects (e.g., generalized linear mixed-effects models, GLMMs) approaches are in use. Although GLMM parameters can be fitted using maximum likelihood, the same is not true for the frequentist GEE method, but modifications have been proposed to accommodate the MAR assumption (6). Finally, MNAR missingness can never be fully ruled out based on the observed data only. It is argued that, rather than going either for discarding MNAR models entirely or for placing full faith on them, a sensible compromise is to make them a component of a sensitivity analysis. 2

METHODS IN COMMON USE

We will focus on two relatively simple methods that have been and still are in extensive use. A detailed account of simple methods to handle missingness is given in Verbeke and Molenberghs (7, 8).

2.1 Complete Case Analysis A complete case analysis includes only those cases for analysis for which all measurements were recorded. This method has obvious advantages. It is very simple to describe, and because the data structure is as would have resulted from a complete experiment, standard statistical software can be used without additional work. Furthermore as the entire estimation is performed on the same subset of completers, there is a common basis for inference. Unfortunately, the method suffers from severe drawbacks. First, there is nearly always a substantial loss of information. The impact on precision and power is dramatic. Furthermore, such an analysis will only be representative for patients who remain on study. Of course a complete case analysis could have a role as an auxiliary analysis, especially if a scientific question relates to it. A final important issue about a complete case analysis is that it is only valid when the missingness mechanism is MCAR. However, severe bias can result when the missingness mechanism is MAR but not MCAR. This bias can go both ways, i.e., either overestimating or underestimating the true effect. 2.2 Last Observation Carried Forward A method that has received a lot of attention (9–11) is the last observation carried forward (LOCF). As noted, in the LOCF method, whenever a value is missing, the last observed value is substituted. For the LOCF approach, the MCAR assumption is necessary but not sufficient for an unbiased estimate. Indeed, it further assumes that subjects’ responses would have been constant from the last observed value to the endpoint of the trial. These conditions seldom hold (8). In a clinical trial setting, one might believe that the response profile changes as soon as a patient goes off treatment and even

Table 1. Overview of Missing Data Mechanisms Acronym

Description

Likelihood/Bayesian

Frequentist

MCAR MAR MNAR

missing completely at random missing at random missing not at random

ignorable ignorable non-ignorable

ignorable non-ignorable non-ignorable

MISSING DATA

that it would flatten. However, the constant profile assumption is even stronger. Therefore, carrying observations forward may bias estimates of treatment effects and underestimate the associated standard errors (8, 12–16). Further more this method artificially increases the amount of information in the data, by treating imputed and actually observed values on equal footing. Despite its shortcomings, LOCF has been the longstanding method of choice for the primary analysis in clinical trials because of its simplicity, ease of implementation, and the belief that the potential bias from carrying observations forward leads to a ‘‘conservative’’ analysis in comparative trials. An analysis is called conservative when it leads to no treatment difference, whereas in fact there is a treatment difference. However, reports of anti-conservative or liberal behavior of LOCF are common (17–21), which means that a LOCF analysis can create a treatment effect when none exists. Thus, the statement that LOCF analysis has been used to provide a conservative estimate of treatment effect is unacceptable. Historically, an important motivation behind the simpler methods was their simplicity. Indeed, the main advantage, shared with complete case analysis, is that complete data software can be used. However, with the availability of commercial software tools, such as, for example, the SAS procedures MIXED and NLMIXED and the SPlus and R nlme libraries, this motivation no longer applies. It is often quoted that LOCF or CC, although problematic for parameter estimation, produces randomization-valid hypothesis testing, but this is questionable. First, in a CC analysis, partially observed data are selected out, with probabilities that may depend on post-randomization outcomes, thereby undermining any randomization justification. Second, if the focus is on one particular time point, e.g., the last one scheduled, then LOCF plugs in data. Such imputations, apart from artificially inflating the information content, may deviate in complicated ways from the underlying data (17). Third, although the size of a randomizationbased LOCF test may reach its nominal size under the null hypothesis of no difference

3

in treatment profiles, there will be other regions of the alternative space where the power the LOCF test procedure is equal to its size, which is completely unacceptable. 3 AN ALTERNATIVE APPROACH TO INCOMPLETE DATA A graphical illustration is first provided, using an artificial example, of the various simple methods that have been considered, and then so-called direct likelihood analysis is discussed. 3.1 Illustration of Simple Methods Take a look at an artificial but insightful example, depicted in Fig. 1, which displays the results of the traditional methods, CC and LOCF, next to the result of an MAR method. In this example, the mean response is supposed to be linear. For both groups (completers and dropouts), the slope is the same, but their intercepts differ. Patients with incomplete observations dropped out half way through the study; e.g., because they reached a certain level of the outcome. It is obviously an MAR missingness mechanism. Using a method, valid under the MAR assumption, yields the correct mean profile, being a straight line centered between the mean profiles of the completers and incompleters. If one would perform a CC analysis, the fitted profile would coincide with the mean profile of the complete cases (bold line). Next, under LOCF, data are imputed (dashed line). The resulting fitted profile will be the bold dashed line. Clearly, both traditional methods produce an incorrect result. Furthermore, in a traditional available case analysis (AC), one makes use of the information actually available. One such set of estimators could be the treatment-specific mean at several designed measurement occasions. With a decreasing sample size over time, means later in time would be calculated using less subjects than means earlier in time. Figure 1 shows a dramatic instance of this approach, due to the extreme nature of this illustrative example. The key message is that such an approach cannot remove major sources of bias.

MISSING DATA

10

4

8

9

Unobserved

7

LOCF ‘data’

6

LOCF

4

MAR

5

Inc.Obs.

2

3

AC

1

CC

0

Comp.Obs. 0

1

2

3

4

5

6

7

8

9

10

Figure 1. Artificial situation, illustrates the results of the traditional MCAR methods—CC and LOCF—next to the result of the direct likelihood method.

3.2 Direct Likelihood Analysis For continuous outcomes, Verbeke and Molenberghs (8) describe likelihood-based mixedeffects models, which are valid under the MAR assumption. Indeed, for longitudinal studies, where missing data are involved, a mixed model only requires that missing data are MAR. As opposed to the traditional techniques, mixed-effects models permit the inclusion of subjects with missing values at some time points (both dropout and intermittent missingness). This likelihood-based MAR analysis is also termed likelihood-based ignorable analysis or, as used in the remainder of this article, a direct likelihood analysis. In such a direct likelihood analysis, the observed data are used without deletion nor imputation. In so doing, appropriate adjustments are made to parameters at times when data are incomplete, due to the within-patient correlation. Thus, even when interest lies, for example, in a comparison between the two treatment groups at the last occasion, such a full longitudinal analysis is a good approach, because

the fitted model can be used as the basis for inference at the last occasion. In many clinical trials, the repeated measures are balanced in the sense that a common (and often limited) set of measurement times is considered for all subjects, which allows the a priori specification of a ‘‘saturated’’ model. For example, a full group-by-time interaction for the fixed effects combined with an unstructured covariance matrix. Such a model specification is sometimes termed mixed-effects model repeated-measures analysis (MMRM) (11). Thus, MMRM is a particular form of a linear mixed model, relevant for acute phase confirmatory clinical trials, fitting within the direct likelihood paradigm. Moreover, this direct likelihood MMRM analysis of variance (ANOVA) and multivariate analysis of variance (MANOVA) approaches, but more generally valid when they are incomplete. This response is a strong answer to the common criticism that a direct likelihood method is making strong assumptions. Indeed, its coincidence with MANOVA for data without missingness shows that the assumptions

MISSING DATA

made are very mild. Therefore, it constitutes a very promising alternative for CC and LOCF. When a relatively large number of measurements is made within a single subject, the full power of random effects modeling can be used (8). The practical implication is that a software module with likelihood estimation facilities and with the ability to handle incompletely observed subjects manipulates the correct likelihood, providing valid parameter estimates and likelihood ratio values. A few cautionary remarks are warranted. First, when at least part of the scientific interest is directed toward the nonresponse process, obviously both processes need to be considered. Under MAR, both questions can be answered separately, which implies that a conventional method can be used to study questions in terms of the outcomes of interest, such as treatment effect and time trend, whereafter a separate model can be considered to study missingness. Second, likelihood inference is often surrounded with references to the sampling distribution (e.g., to construct measures of precision for estimators and for statistical hypothesis tests (22)). However, the practical implication is that standard errors and associated tests, when based on the observed rather than the expected information matrix and given that the parametric assumptions are correct, are valid. Thirdy, it may be hard to rule out the operation of an MNAR mechanism. This point was brought up in Section 1 and will be discussed further in Section 5. 4 ILLUSTRATION: ORTHODONTIC GROWTH DATA As an example, we use the orthodontic growth data, introduced by Potthoff and Roy (23) and used by Jennrich and Schluchter (24). The data have the typical structure of a clinical trial and are simple yet illustrative. They contain growth measurements for 11 girls and 16 boys. For each subject, the distance from the center of the pituitary to the maxillary fissure was recorded at ages 8, 10, 12, and 14. Figure 2 presents the 27 individual profiles. Little and Rubin (1) deleted 9 of the [(11 + 16) × 4] measurements, rendering 9

5

incomplete subjects, which even though it is a somewhat unusual practice has the advantage of allowing a comparison between the incomplete data methods and the analysis of the original, complete data. Deletion is confined to the age 10 measurements, and rougly speaking, the complete observations at age 10 are those with a higher measurement at age 8. Some emphasis will be placed on ages 8 and 10, the typical dropout setting, with age 8 fully observed and age 10 partially missing. The simple methods and direct likelihood method from Sections 2 and 3 are now compared using the growth data. For this purpose, a linear mixed model is used, assuming unstructured mean, i.e., assuming a separate mean for each of the eight age × sex combinations, together with an unstructured covariance structure, and using maximum likelihood (ML) as well as restricted maximum likelihood (REML). The mean profiles of the linear mixed model using maximum likelihood for all four datasets, for boys, are given in Fig. (3). The girls’ profiles are similar and hence not shown. Next to this longitudinal approach, a full MANOVA analysis and a univariate ANOVA analysis will be considered, i.e., one per time point. For all of these analyses, Table 2 shows the estimates and standard errors for boys at ages 8 and 10, for the original data and all available incomplete data, as well as for the CC and the LOCF data. First, the group means for the boys in the original dataset in Fig. (3) are considered; i.e., relatively a straight line is observed. Clearly, there seems to be a linear trend in the mean profile. In a complete case analysis of the growth data, the 9 subjects that lack one measurement are deleted, resulting in a working dataset with 18 subjects. This result implies that 27 available measurements will not be used for analysis, a severe penalty on a relatively small dataset. Observing the profiles for the CC dataset in Fig. (3), all group means increased relative to the original dataset but mostly so at age 8. The net effect is that the profiles overestimate the average length. For the LOCF dataset, the 9 subjects that lack a measurement at age 10 are completed by imputing the age 8 value. It is clear

6

MISSING DATA

Orthodontic Growth Data Profiles and Means 34

Distance

30

26

22

18

14 6

8

10

12

14

16

Age in Years Figure 2. Orthodontic growth data. Raw and residual profiles. (Girls are indicated with solid lines. Boys are indicated with dashed lines.)

22

24

Distance

26

28

Original Data CC LOCF Direct Likelihood (Fitted) Direct Likelihood (Observed)

8

10

12

14

Age Figure 3. Orthodontic growth data. Profiles for the original data, CC, LOCF, and direct likelihood for boys.

MISSING DATA

that this procedure will affect the apparently increasing linear trend found for the original dataset. Indeed, the imputation procedure forces the means at ages 8 and 10 to be more similar, thereby destroying the linear relationship. Hence, a simple, intuitively appealing interpretation of the trends is made impossible. In case of direct likelihood, two profiles can now be observed: one for the observed means and one for the fitted means. These two coincide at all ages except age 10. As mentioned, the complete observations at age 10 are those with a higher measurement at age 8. Due to the within-subject correlation, they are the ones with a higher measurement at age 10 as well, and therefore, the fitted model corrects in the appropriate direction. The consequences of this are very important. Although it is believed that the fitted means do not follow the observed means all that well, this nevertheless is precisely what should be observed. Indeed, as the observed means are based on a nonrandom subset of the data, the

fitted means take into account all observed data points, as well as information on the observed data at age 8, through the measurements that have been taken for such children, at different time points. As an aside, note that, in case of direct likelihood, the observed average at age 10 coincides with the CC average, whereas the fitted average does not coincide with anything else. Indeed, if the model specification is correct, then a direct likelihood analysis produces a consistent estimator for the average profile, as if nobody had dropped out. Of course, this effect might be blurred in relatively small datasets due to small-sample variability. Irrespective of the small-sample behavior encountered here, the validity under MAR and the ease of implementation are good arguments that favor this direct likelihood analysis over other techniques. Now compare the different methods by means of Table 2, which shows the estimates and standard errors for boys at age 8 and 10, for the original data and all available

Table 2. Orthodontic Growth Data. Comparison of Analyses Based on Means at Completely Observed Age 8 and Incompletely Observed Age 10 Measurement Method

Boys at Age 8

Boys at Age 10

Original Data Direct likelihood, ML Direct likelihood, REML MANOVA ANOVA per time point

22.88 (0.56) 22.88 (0.58) 22.88 (0.58) 22.88 (0.61)

23.81 (0.49) 23.81 (0.51) 23.81 (0.51) 23.81 (0.53)

All Available Incomplete Data Direct likelihood, ML Direct likelihood, REML MANOVA ANOVA per time point

22.88 (0.56) 22.88 (0.58) 24.00 (0.48) 22.88 (0.61)

23.17 (0.68) 23.17 (0.71) 24.14 (0.66) 24.14 (0.74)

Complete Case Analysis Direct likelihood, ML Direct likelihood, REML MANOVA ANOVA per time point

24.00 (0.45) 24.00 (0.48) 24.00 (0.48) 24.00 (0.51)

24.14 (0.62) 24.14 (0.66) 24.14 (0.66) 24.14 (0.74)

Last Observation Carried Forward Analysis Direct likelihood, ML Direct likelihood, REML MANOVA ANOVA per time point

7

22.88 (0.56) 22.88 (0.58) 22.88 (0.58) 22.88 (0.61)

22.97 (0.65) 22.97 (0.68) 22.97 (0.68) 22.97 (0.72)

8

MISSING DATA

incomplete data, as well as for the CC data and the LOCF data. Table 2 shows some interesting features. In all four cases, a CC analysis gives an upward biased estimate, for both age groups. This result is obvious, because the complete observations at age 10 are those with a higher measurement at age 8, as shown before. The LOCF analysis gives a correct estimate for the average outcome for boys at age 8. This result is not surprising because there were no missing observations at this age. As noted, the estimate for boys of age 10 is biased downward. When the incomplete data are analyzed, we see from Table 2 that direct likelihood produces good estimates. The MANOVA and ANOVA per time point analyses give an overestimation of the average of age 10, like in the CC analysis. Furthermore, the MANOVA analysis also yields an overestimation of the average at age 8, again the same as in the CC analysis. Thus, direct likelihood shares the elegant and appealing features of ANOVA and MANOVA for fully observed data, but it is superior with incompletely observed profiles. 5

SENSITIVITY ANALYSIS

When there is residual doubt about the plausibility of MAR, one can conduct a sensitivity analysis. Although many proposals have been made, this is still an active area of research. Obviously, several MNAR models can be fitted, provided one is prepared to approach formal aspects of model comparison with due caution. Such analyses can be complemented with appropriate (global and/or local) influence analyses (25). Another route is to construct pattern-mixture models, where the measurement model is considered, conditional upon the observed dropout pattern, and to compare the conclusions with those obtained from the selection model framework, where the reverse factorization is used (26, 27). Alternative sensitivity analyses frameworks are provided by Robins, et al. (28), Forster and Smith (29) who present a Bayesian sensitivity analysis, and Raab and Donnelly (30). A further paradigm, useful for sensitivity analysis, is so-called shared parameter models, where common latent or

random effects drive both the measurement process as well as the process governing missingness (31, 32). Nevertheless, ignorable analyses may provide reasonably stable results, even when the assumption of MAR is violated, in the sense that such analyses constrain the behavior of the unseen data to be similar to that of the observed data. A discussion of this phenomenon in the survey context has been given in Rubin, et al. (33). These authors first argue that, in well-conducted experiments (some surveys and many confirmatory clinical trials), the assumption of MAR is often to be regarded as a realistic one. Second, and very important for confirmatory trials, an MAR analysis can be specified a priori without additional work relative to a situation with complete data. Third, although MNAR models are more general and explicitly incorporate the dropout mechanism, the inferences they produce are typically highly dependent on the untestable and often implicit assumptions built in regarding the distribution of the unobserved measurements given the observed ones. The quality of the fit to the observed data need not reflect at all the appropriateness of the implied structure governing the unobserved data. Based on these considerations, it is recommended, for primary analysis purposes, the use of ignorable likelihood-based methods or appropriately modified frequentist methods. To explore the impact of deviations from the MAR assumption on the conclusions, one should ideally conduct a sensitivity analysis (8). 6 CONCLUSION In conclusion, a direct likelihood analysis is preferable because it uses all available information, without the need neither to delete nor to impute measurements or entire subjects. It is theoretically justified whenever the missing data mechanism is MAR, which is a more relaxed assumption than MCAR, necessary for simple analyses (CC, LOCF). There is no statistical information distortion, because observations are neither removed (such as in CC analysis) nor added (such as in LOCF analysis). Software is available, such that no

MISSING DATA

additional programming is involved to perform a direct likelihood analysis. It is very important to realize that, for complete sets of data, direct likelihood, especially with the REML estimation method, is identical to MANOVA (see Table 2). Given the classic robustness of MANOVA, and its close agreement with ANOVA per time point, this provides an extra basis for direct likelihood. Indeed, it is not as assumption-driven as is sometimes believed. This, in addition with the validity of direct likelihood under MAR (and hence its divergence from MANOVA and ANOVA for incomplete data), provides a strong basis for the direct likelihood method. 7

ACKNOWLEDGMENTS

The authors gratefully acknowledge support from Fonds Wetenschappelijk OnderzoekVlaanderen Research Project G.0002.98 ‘‘Sensitivity Analysis for Incomplete and Coarse Data’’ and from Belgian IUAP/PAI network ‘‘Statistical Techniques and Modeling for Complex Substantive Questions with Complex Data.’’ We are thankful to Eli Lilly for kind permission to use their data. REFERENCES 1. R. J. A. Little, and D. B. Rubin, Statistical Analysis with Missing Data. New York: John Wiley & Sons, 2002. 2. A. Afifi, and R. Elashoff, Missing observations in multivariate statistics I: Review of the literature. J. Am. Stat. Assoc. 1996; 61: 595–604. 3. H. O. Hartley, and R. Hocking, The analysis of incomplete data. Biometrics, 1971; 27: 7783–7808. 4. A. P. Dempster, N. M. Laird, and D. B. Rubin, Maximum likelihood from incomplete data via the EM algorithm (with discussion). J. Roy. Stat. Soc. Series B, 1977; 39: 1–38. 5. D. B. Rubin, Multiple Imputation for Nonresponse in Surveys. New York: John Wiley & Sons, 1987. 6. J. M. Robins, A. Rotnitzky, and L. P. Zhao, Analysis of semiparametric regression models for repeated outcomes in the presence of missing data. J. Am. Stat. Assoc. 1995; 90: 106–121.

9

7. G. Verbeke, and G. Molenberghs, Linear Mixed Models in Practice: A SAS-Oriented Approach. Lecture Notes in Statistics 126. New York: Springer-Verlag, 1997. 8. G. Verbeke, and G. Molenberghs, Linear Mixed Models for Longitudinal Data. New York: Springer-Verlag, 2000. 9. O. Siddiqui, and M. W. Ali, A comparison of the random-effects pattern mixture model with last observation carried forward (LOCF) analysis in longitudinal clinical trials with dropouts. J. Biopharm. Stat. 1998; 8: 545–563. 10. C. H. Mallinckrodt, W. S. Clark, R. J. Carroll, and G. Molenberghs, Assessing response profiles from incomplete longitudinal clinical trial data under regulatory considerations. J. Biopharm. Stat. 2003; 13: 179–190. 11. C. H. Mallinckrodt, T. M. Sanger, S. Dube, D. J. Debrota, G. Molenberghs, R. J. Carroll, W. M. Zeigler Potter, and G. D. Tollefson, Assessing and interpreting treatment effects in longitudinal clinical trials with missing data. Biol. Psychiatry, 2003; 53: 754–760. 12. R. D. Gibbons, D. Hedeker, I. Elkin, D. Waternaux, H. C. Kraemer, J. B. Greenhouse, M. T. Shea, S. D. Imber, S. M. Sotsky, and J. T. Watkins. Some conceptual and statistical issues in analysis of longitudinal psychiatric data. Arch. Gen. Psychiatry, 1993; 50: 739–750. 13. A. Heyting, J. Tolboom, and J. Essers. Statistical handling of dropouts in longitudinal clinical trials. Stat. Med. 1992; 11: 2043–2061. 14. P. W. Lavori, R. Dawson, and D. Shera. A multiple imputation strategy for clinical trials with truncation of patient data. Stat. Med. 1995; 14: 1913–1925. 15. C. H. Mallinckrodt, W. S. Clark, and R. D. Stacy. Type I error rates from mixed-effects model repeated measures versus fixed effects analysis of variance with missing values imputed via last observation carried forward. Drug Inform. J. 2001; 35(4): 1215–1225. 16. C. H. Mallinckrodt, W. S. Clark, and R. D. Stacy. Accounting for dropout bias using mixed-effects models. J. Biopharm. Stat. 2001; 11(1 & 2): 9–21. 17. M. G. Kenward, S. Evans, J. Carpenter, and G. Molenberghs. Handling missing responses: Time to leave Last Observation Carried Forward (LOCF) behind, Submitted for publication. 18. G. Molenberghs, H. Thijs, I. Jansen, C. Beunckens, M. G. Kenward, C. Mallinckrodt,

10

19.

20.

21.

22.

23.

24.

MISSING DATA and R. J. Carroll. Analyzing incomplete longitudinal clinical trial data. Biostatistics, 2004; 5: 445–464. C. H. Mallinckrodt, J. G. Watkin, G. Molenberghs, and R. J. Carroll. Choice of the primary analysis in longitudinal clinical trials. Pharm. Stat. 2004; 3: 161–169. R. J. A. Little, and L. Yau. Intent-to-treat analysis in longitudinal studies with dropouts. Biometrics 1996; 52: 1324–1333. G. Liu and A. L. Gould. Comparison of alternative strategies for analysis of longitudinal trials with dropouts. J. Biopharm. Stat. 2002; 12: 207–226. M. G. Kenward, and G. Molenberghs, Likelihood based frequentist inference when data are missing at random. Stat. Sci. 1998; 12: 236–247. R. F. Potthoff, and S. N. Roy, A generalized multivariate analysis of variance model useful especially for growth curve problems. Biometrika 1964; 51: 313–326. R. I. Jennrich, and M. D. Schluchter, Unbalanced repeated measures models with structured covariance matrices. Biometrics 1986; 42: 805–820.

25. G. Verbeke, G. Molenberghs, H. Thijs, E. Lesaffre, and M. G. Kenward, Sensitivity analysis for non-random dropout: A local influence approach. Biometrics 2001; 57: 7–14. 26. B. Michiels, G. Molenberghs, L. Bijnens, T. Vangeneugden, and H. Thijs. Selection models and pattern-mixture models to analyze longitudinal quality of life data subject to dropout. Stat. Med. 2002; 21: 1023–1041.

27. H. Thijs, G. Molenberghs, B. Michiels, G. Verbeke, and D. Curran, Strategies to fit pattern-mixture models. Biostatistics 2002; 3: 245–265. 28. J. M. Robins, A. Rotnitzky, and D. O. Scharfstein, Semiparametric regression for repeated outcomes with non-ignorable non-response. J. Am. Stat. Assoc. 1998; 93: 1321–1339. 29. J. J. Forster, and P. W. Smith, Model-based inference for categorical survey data subject to non-ignorable non-response. J. Roy. Stat. Soc. Series B 1998; 60: 57–70. 30. G. M. Raab, and C. A. Donnelly, Information on sexual behaviour when some data are missing. Appl. Stat. 1999; 48: 117–133. 31. M. C. Wu, and K. R. Bailey, Estimation and comparison of changes in the presence of informative right censoring: Conditional linear model. Biometrics 1989; 45: 939–955. 32. M. C. Wu, and R. J. Carroll, Estimation and comparison of changes in the presence of informative right censoring by modeling the censoring process. Biometrics 1988; 44: 175–188. 33. D. B. Rubin, H. S. Stern, and V. Vehovar, Handling ‘‘don’t know’’ survey responses: The case of the Slovenian plebiscite. J. Am. Stat. Assoc. 1995; 90: 822–828.

MONITORING

however, in exceptional circumstances, the sponsor may determine that central monitoring in conjunction with procedures such as investigators’ training and meetings, and extensive written guidance can assure appropriate conduct of the trial in accordance with GCP. Statistically controlled sampling may be an acceptable method for selecting the data to be verified. The monitor(s), in accordance with the sponsor’s requirements, should ensure that the trial is conducted and documented properly by carrying out the following activities when relevant and necessary to the trial and the trial site:

The purposes of trial monitoring are to verify that: • The rights and well-being of human sub-

jects are protected. • The reported trial data are accurate,

complete, and verifiable from source documents. • The conduct of the trial is in compliance with the currently approved protocol/amendment(s), with Good Clinical Practice (GCP), and with applicable regulatory requirement(s).

• Acting as the main line of communica-

tion between the sponsor and the investigator. • Verifying that the investigator has adequate qualifications and resources and these remain adequate throughout the trial period, and that the staff and facilities, which include laboratories and equipment, are adequate to conduct the trial in a safe and proper manner, and these remain adequate throughout the trial period. • Verifying, for the investigational product(s): • That storage times and conditions are acceptable, and that supplies are sufficient throughout the trial. • That the investigational product(s) are supplied only to subjects who are eligible to receive it and at the protocol specified dose(s). • That subjects are provided with necessary instruction on properly using, handling, storing, and returning the investigational product(s). • That the receipt, use, and return of the investigational product(s) at the trial sites are controlled and documented adequately. • That the disposition of unused investigational product(s) at the trial sites complies with applicable regulatory

Selection and Qualifications of Monitors • Monitors should be appointed by the

sponsor. • Monitors should be trained appropri-

ately and should have the scientific and/ or clinical knowledge needed to monitor the trial adequately. A monitor’s qualifications should be documented. • Monitors should be thoroughly familiar with the investigational product(s), the protocol, the written informed consent form and any other written information to be provided to subjects, the sponsor’s Standard Operation Procedures (SOP), GCP, and the applicable regulatory requirement(s). The sponsor should ensure that the trials are adequately monitored. The sponsor should determine the appropriate extent and nature of monitoring. The determination of the extent and the nature of monitoring should be based on considerations such as the objective, purpose, design, complexity, blinding, size, and endpoints of the trial. In general, a need exists for on-site monitoring, before, during, and after the trial; This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

MONITORING

requirement(s) and is in accordance with the sponsor’s authorized procedures. • Verifying that the investigator follows

the approved protocol and all approved amendment(s), if any. • Verifying that written informed consent

was obtained before each subject’s participation in the trial. • Ensuring that the investigator receives

the current Investigator’s Brochure, all documents, and all trial supplies needed to conduct the trial properly and to comply with the applicable regulatory requirement(s). • Ensuring that the investigator and the

investigator’s trial staff are adequately informed about the trial. • Verifying that the investigator and the

investigator’s trial staff are performing the specified trial functions, in accordance with the protocol and any other written agreement between the sponsor and the investigator/institution, and have not delegated these functions to unauthorized individuals. • Verifying that the investigator is enroll-

ing only eligible subjects. • Reporting the subject recruitment rate. • Verifying that source data/documents

and other trial records are accurate, complete, kept up-to-date, and maintained. • Verifying that the investigator provides

all the required reports, notifications, applications, and submissions, and that these documents are accurate, complete, timely, legible, dated, and identify the trial. • Checking the accuracy and complete-

ness of the Case Report Form (CRF) entries, source data/documents, and other trial-related records against each other. The monitor specifically should verify that: • The data required by the protocol are reported accurately on the CRFs and are consistent with the source data/ documents.

• Any dose and/or therapy modifications

are well documented for each of the trial subjects. • Adverse events, concomitant medications, and intercurrent illnesses are reported in accordance with the protocol on the CRFs. • Visits that the subjects fail to make, tests that are not conducted, and examinations that are not performed are clearly reported as such on the CRFs. • All withdrawals and dropouts of enrolled subjects from the trial are reported and explained on the CRFs. • Informing the investigator of any CRF

entry error, omission, or illegibility. The monitor should ensure that appropriate corrections, additions, or deletions are made, dated, explained (if necessary), and initialed by the investigator or by a member of the investigator’s trial staff who is authorized to initial CRF changes for the investigator. This authorization should be documented. • Determining whether all adverse events (AEs) are appropriately reported within the time periods required by GCP, the protocol, the IRB (Institutional Review Board)/IEC (Independent Ethic’s Committee), the sponsor, the applicable regulatory requirement(s), and indicated in the International Conference on Harmonisation (ICH) Guideline for Clinical Safety Data Management: Definitions and Standards for Expedited Reporting. • Determining whether the investigator is maintaining the essential documents. • Communicating deviations from the protocol, SOPs, GCP, and the applicable regulatory requirements to the investigator and taking appropriate action designed to prevent recurrence of the detected deviations. The monitor(s) should follow the sponsor’s established written SOPs as well as those procedures that are specified by the sponsor for monitoring a specific trial.

MONITORING

Monitoring Report • The monitor should submit a written report to the sponsor after each trial-site visit or trial-related communication. • Reports should include the date, site, name of the monitor, and name of the investigator or other individual(s) contacted. • Reports should include a summary of what the monitor reviewed and the monitor’s statements concerning the significant findings/facts, deviations and deficiencies, conclusions, actions taken or to be taken, and/or actions recommended to secure compliance. • The review and follow-up of the monitoring report by the sponsor should be documented by the sponsor’s designated representative.

3

MONOTHERAPY

therapy with therapy delayed for an appropriate length of time such that every patient has the opportunity to receive the drug and yet a sufficient comparison period exists to assess the effect on a short-term outcome (as well as the opportunity to compare the effect of the different durations imposed by the delayed start of treatment on longer-term outcomes). The comparisons of immediate versus delayed therapy may also be appropriate when it is unclear when to start therapy, a concept explored in the Concorde trial (3). Once an effective therapy is available for a disease, it becomes much more difficult to assess new therapies as it is usually no longer ethical to compare them with an untreated group (which may receive placebo). However, this is not always the case; for example, it may be appropriate to evaluate a new bronchodilator for acute asthma by comparison with a placebo as long as treatment is available if therapy is not effective. An alternative way to evaluate certain new monotherapy regimens is in a crossover trial. However, this approach is only practicable if the disease is chronic and returns to a similar baseline when therapy is stopped and no prolonged effect of the therapy exists after it is stopped. To evaluate a new monotherapy when an existing therapy exists and an untreated group is not ethically acceptable, the randomized comparison is usually with the existing standard therapy. The new therapy may be expected to be more effective than the existing therapy or of similar efficacy but with less toxicity, and this fact will influence whether the trial is designed as a superiority or equivalence (or non-inferiority trial). In either case, the trial will need to be larger than if the new drug was compared with no treatment, as the difference between the efficacy of the new and old drug is likely to be much smaller than between drug and no treatment. An example of a monotherapy trial comparing a new drug with existing therapy is the evaluation of the second antiHIV drug didanosine (ddI), which was compared directly with ZDV (4). Such trials are, wherever possible, blinded (or masked) either by making the two drugs indistinguishable

JANET DARBYSHIRE London, United Kingdom

1

DEFINITION

A trial that evaluates a monotherapy would most often test a single drug, but the definition could be extended to include other interventions such as immunotherapy or even a different modality such as radiotherapy that contain a single element. The issues around trial design are essentially the same and, therefore, this section focuses on monotherapy trials that evaluate a drug. The simplest and most efficient way to evaluate a new agent is to compare it with no treatment or a placebo in a classic randomized parallel group design, assuming that its safety and activity have already been demonstrated in Phase I and II trials. If the drug is to be clinically useful, the effect compared with no treatment will be relatively large and, therefore, the trial may not need to include many patients. The duration of the trial will be driven by the disease, in particular its natural history, and the primary outcome measure. A classic example of a new monotherapy was the drug zidovudine (ZDV), the first drug to be evaluated for the treatment of HIV infection. In patients with advanced HIV infection, ZDV was shown to be substantially and significantly better than placebo in preventing death over a short period (1). However, subsequent trials, also placebo-controlled, of ZDV at a much earlier stage of HIV infection showed that the effect was short-lived because of the emergence of resistance to ZDV (2). In a highly fatal disease or a disease with a high incidence of other clear-cut clinical endpoints, the benefits of the first potent therapy may even be clearly demonstrated without a trial, the often quoted ‘‘penicillinlike’’ effect. In such circumstances, it may be difficult to withhold a new drug that looks promising from individuals, and one approach could be to compare immediate

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

MONOTHERAPY

or, if this is not possible, employing the double dummy technique. The selection of the comparison therapy is crucial. For example, if a new antibiotic is being evaluated, the choice of an inappropriate comparator may make the new drug appear better than it really is. Alternatively, if the comparator happens to be more potent against the likely infecting organisms, a drug that is useful in other infections may appear ineffective. In certain circumstances, it may be necessary to assess a new drug that will ultimately be used only as part of a combination therapy regimen. In these circumstances, the new drug may be given in addition to the conventional therapy (which itself may be a single therapy or a combination). Again, the basic comparison is with a group that only received the conventional therapy and, wherever possible, the trial is blinded or masked. Many examples of this comparison exist, such as the ESPRIT trial, which is evaluating Interleukin-2 (IL-2) on a background of antiretroviral therapy (5) and the first trial of a drug from a new class in HIV infection, ritonavir, a protease inhibitor (6). One of the difficulties of assessing a drug by adding it to or substituting it for one component of a combination regimen, is that additive, synergistic, or antagonistic activity may exist between the drugs. Further, an active new drug may add little to a potent treatment and therefore be rejected as inactive. New drugs that will never be used as monotherapy, such as in HIV and tuberculosis (TB), are sometimes evaluated for the first time as monotherapy by using short-term studies based on laboratory markers. These studies are used to determine whether the new drug has any activity ideally comparing them with other drugs concurrently or alternatively using historical controls. For both TB and HIV, issues around such designs exist because of concerns about the emergence of drug resistance, and therefore, innovative designs are needed to address this increasingly important issue. An efficient approach to the evaluation of two drugs, which also explores the effect of giving them together, is the factorial design. This design relies on the assumption that there will not be an interaction between

them. If a possibility of an interaction exists, whether positive or negative, then it may be an important finding, and the factorial design is still the optimal approach to evaluating it. However, the sample size will need to be sufficiently large to adequately assess interactions. If monotherapy regimens fail, a number of options exist that may need to be explored depending on the disease and the available drugs and issues such as cross-resistance or interactions. For example, in early HIV trials, a new drug was often added to ZDV or, alternatively, patients were switched from one monotherapy to another. No single optimal approach to decide how to use new drugs exists, but a clear need exists to consider all the relevant disease and drug-related factors. REFERENCES 1. M. A. Fischl, D. D. Richmann, M. H. Grieco, M. S. Gottlieb, P. A. Volberding, O. L. Laskin et al., The efficacy of azidothymidine (AZT) in the treatment of patients with AIDS and AIDS related complex. A double-blind, placebo controlled trial. N. Engl. J. Med. 1987; 317: 185– 191. 2. HIV Trialists’ Collaborative Group, Zidovudine, didanosine, and zalcitabine in the treatment of HIV infection: meta-analyses of the randomised evidence. Lancet 1999; 353: 2014–2025. 3. Concorde Coordinating Committee, Concorde: MRC/ANRS randomised double-blind controlled trial of immediate and deferred zidovudine in symptom-free HIV infection. Lancet 1994; 343: 871–882. 4. J. O. Kahn, S. W. Lagakos, D. D. Richman, A. Cross, C. Pettinelli, S-H. Liou et al., A controlled trial comparing continued zidovudine with didanosine in human immunodeficiency virus infection. N. Engl. J. Med. 1992; 327: 581–587. 5. S. Emery, D. I. Abrams, D. A. Cooper, J. H. Darbyshire, H. C. Lane, J. D. Lundgren, and J. D. Neaton, The evaluation of subcutaneous Proleukin (interleukin-2) in a randomized international trial: rationale, design, and methods of ESPRIT. Controlled Clin. Trials 2002; 23: 198–220. 6. D. W. Cameron, M. Heath-Chiozzi, S. Danner, C. Cohen, S. Kravcik, C. Maurath et al., Randomised placebo controlled trial of ritonavir in advanced HIV-1 disease. Lancet 1998; 351(9102): 543–549.

MOTHER TO CHILD HUMAN IMMUNODEFICIENCY VIRUS TRANSMISSION TRIALS

in developed countries and two trials of interventions feasible in resource-limited settings (Table 1). For each trial, the objectives, study design, results, and conclusions are summarized, along with implications for future patients and questions for additional study.

DAVID E. SHAPIRO Center for Biostatistics in AIDS Research, Harvard School of Public Health Boston, Massachusetts

1 THE PEDIATRIC AIDS CLINICAL TRIALS GROUP 076 TRIAL

Mother-to-child transmission (MTCT) of human immunodeficiency virus type 1 (HIV) can occur through three major routes: through the placental barrier during gestation (in utero and antepartum), via contact with maternal bodily fluids during labor and delivery (intrapartum), and by ingestion of breast milk after delivery (postpartum); most MTCT is believed to occur close to the time of or during childbirth (1). Infants who have a positive HIV test within the first 72 hours of life are presumed to have been infected in utero, and those who are HIV-negative within the first 72 hours and HIV-positive thereafter are presumed to have been infected close to or during delivery (or via early breastfeeding, if the infant breastfeeds). Prevention of MTCT (PMTCT) is one of the most successful areas in HIV clinical research. Interventions to prevent MTCT have been developed for two different settings: the United States and other developed countries with access to medications, medical infrastructure, and safe replacement feeding so that breastfeeding can be discouraged; and resource-limited countries with limited access to medications and clean water, and where breastfeeding is critical (2). In the absence of intervention, the risk of MTCT is approximately 15–25% during pregnancy through delivery (3), and an additional 0.9% per month during breastfeeding (4). With combination antiretroviral therapy during pregnancy and delivery, MTCT risk can be reduced to below 2% in the absence of breastfeeding (5,6); several clinical trials of interventions to reduce MTCT during breastfeeding are ongoing (7). This article describes four important PMTCT randomized trials, including two trials of key interventions used

The seminal Pediatric AIDS Clinical Trials Group (PACTG) 076 trial, which was sponsored by the U.S. National Institutes of Health (NIH), was the first Phase III trial to establish the efficacy of an antiretroviral drug to prevent MTCT of HIV. As of October 2007, the PACTG 076 zidovudine (ZDV) regimen is still recommended in the United States and other developed countries (8). 1.1 Objectives Based on animal models of retroviral infection, it was hypothesized that ZDV might prevent MTCT either by reducing the circulating HIV viral load in the mother and thereby reducing exposure of the fetus to HIV in utero and during delivery, or by accumulating therapeutic concentrations in the fetus and infant that could provide protection during and after exposure to HIV, or both (9). Phase I studies in pregnant women suggested that ZDV was safe when used for short periods and that it crossed the placenta well (9). The primary objectives of the trial were to assess the efficacy and safety of ZDV for the prevention of MTCT of HIV. 1.2 Study Design The PACTG 076 study was a randomized, double-blind, placebo-controlled trial conducted in the United States and France from 1991 to 1994. HIV-infected pregnant women, who were between 14- and 34-weeks gestation, had CD4+ T-lymphocyte counts exceeding 200/mm3 and no indications for antiretroviral therapy were enrolled. The study regimen consisted of oral ZDV five times daily during pregnancy, intravenous ZDV during labor and delivery, and oral ZDV

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

MOTHER TO CHILD HUMAN IMMUNODEFICIENCY VIRUS TRANSMISSION TRIALS

Table 1. Summary of the Study Interventions in Four Important PMTCT Randomized Trials Study (site; years enrolled) PACTG 076 (US, France; 1991–4)

EMDC (Europe; 1993–1998)

Antepartum

Intrapartum

Postpartum: mother

Arm 1: ZDV (oral, Arm 1: ZDV 5x/day from 14 (intravenous) weeks gestation) Arm 2: placebo Arm 2: placebo

No antiretrovirals

Nonstudy antiretrovirals (mainly ZDV)

Non-study Non-study antiretrovirals antiretrovi(mainly ZDV) rals (mainly ZDV)

Arm 1: elective cesarean delivery

Postpartum: infant Arm 1: ZDV (oral, 4x/day for 6 weeks) Arm 2: placebo

Arm 2: vaginal delivery HIVNET 012 (Uganda; 1997–1999)

No antiretrovirals

Arm 1: NVP (oral, single-dose)

No antiretrovirals

Arm 2: ZDV (oral) Arm 3: placebo (stopped Feb. 1998) MASHI (Botswana; ZDV (oral, 2x/day 2001–2003) from 34 weeks gestation)∗

∗ In

First No antiretrorandomization∗∗ : virals∗ NVP (oral, single-dose) versus placebo ZDV (oral)

Arm 1: NVP (oral, single dose on day 2–3 of age) Arm 2: ZDV (oral, 2x/day for 1 week) Arm 3: placebo (stopped Feb. 1998)

Second randomization: Breastfeeding with ZDV (oral, 3x/day until age 6 months) versus formula feeding NVP∗∗ (oral, single-dose) plus ZDV (oral, 2x/day for 4 weeks)

the revised design, women received combination antiretroviral therapy if required per Botswana national guidelines. the initial design, the first randomization was to maternal/infant single dose NVP versus maternal/infant placebo.

∗∗ In

to the infant for 6 weeks. The women were followed until 6 months postpartum, and the infants were followed until 18 months of age. The primary efficacy outcome measure was the MTCT rate at 18 months of age, as estimated by the Kaplan-Meier method to permit inclusion of infants who had incomplete follow-up. The target sample size was 748 mother–infant pairs (636 assessable) to provide 80% power to detect a reduction in MTCT from 30% in the placebo arm to 20% in the ZDV arm, with a two-sided, 0.05 Type I error. The trial was to be monitored by an independent Data and Safety Monitoring Board (DSMB) at least annually for study progress and safety; three interim efficacy analyses were planned, with an O’Brien– Fleming stopping boundary. In

February 1994, the first interim efficacy analysis revealed a dramatic reduction in MTCT in the ZDV arm. The interim results were released immediately, enrollment was discontinued, all study participants who received the blinded study drug were offered ZDV, and study follow-up was completed as planned (9). 2 RESULTS In the final efficacy analysis based on complete follow-up of 402 assessable mother– infant pairs, MTCT rates were 7.6% and 22.6% in the ZDV and placebo groups, respectively (10), which represented a 66% reduction in overall MTCT risk and confirmed

MOTHER TO CHILD HUMAN IMMUNODEFICIENCY VIRUS TRANSMISSION TRIALS

the results of the interim efficacy analysis (9). ZDV significantly reduced MTCT both in utero and intrapartum (11). However, ZDV reduced maternal viral load only slightly (median reduction, 0.24 log), and after adjustment for the baseline maternal viral load and CD4+ cell count, the reduction in viral load from baseline to delivery was not significantly associated with MTCT risk (10). ZDV was well tolerated and minimal shortterm toxic effects were observed, other than significantly lower hemoglobin levels in ZDVexposed infants during the first 6 weeks of life (9,12). No differences between groups were observed with respect to adverse pregnancy outcomes, uninfected infant growth and immune parameters, or HIV disease progression (12,13). The development of viral mutations conferring resistance to ZDV was rare (14). 2.1 Conclusions and Questions for Additional Study The PACTG 076 results suggested that a maternal–infant ZDV regimen could dramatically reduce MTCT of HIV with little short-term toxicity in pregnant women with mildly symptomatic HIV disease. Within 2 months of the release of the initial PACTG 076 results, the U.S. Public Health Service (USPHS) issued interim guidance to support the use of the PACTG 076 ZDV regimen, and 4 months later, they issued more extensive guidance (15,16). Epidemiologic studies have subsequently demonstrated large decreases in MTCT with incorporation of the PACTG 076 regimen into general clinical practice (8). Two important questions for additional study included the regimen’s long-term safety and whether it would be efficacious in women with more advanced HIV disease. Long-term follow-up studies of the PACTG 076 mothers (17) and infants (18,19) observed no major adverse effects of ZDV within the first few years after delivery, although subsequently a possible association of in utero ZDV or ZDV/lamivudine (3TC) exposure with mitochondrial dysfunction among uninfected infants has been found in some studies but not others (8). The efficacy of ZDV among women with more advanced HIV disease was demonstrated in PACTG 185, which was a randomized trial in the United States of passive

3

immunization for the prevention of MTCT of HIV among pregnant HIV-infected women who had CD4+ counts below 500/mm3 and who were receiving ZDV for maternal health. Enrollment was discontinued after the first interim efficacy analysis revealed an unexpectedly low MTCT rate of 4.8%, which substantially increased the sample size required to achieve the primary objective (20). Questions for additional study regarding modifications of the PACTG 076 regimen were in opposite directions in developed and resource-limited countries. In developed countries, the primary question was whether intensifying the PACTG 076 regimen by adding other antiretrovirals to reduce viral load even more would increase efficacy. A nonrandomized study in France showed that adding 3TC to the PACTG 076 regimen starting at 32-weeks gestation could reduce MTCT to below 2% (21), and other studies subsequently observed similar results with other combination antiretroviral regimens (5,6). In resource-limited settings, however, the PACTG 076 regimen was already too complex and expensive to be feasible, so the key question was whether the regimen could be shortened or simplified, yet still reduce MTCT risk. The relative importance of the maternal and infant components of the ZDV regimen could not be determined from PACTG 076 because mothers and infants received the same treatment assignment. Interestingly, a 2 × 2 factorial design, which could have addressed this question, was considered for PACTG 076 but ultimately rejected because of concerns that the maternal and infant components would not have independent influence on MTCT risk, which would reduce the statistical power of the factorial design (22). Subsequently, a randomized, placebo-controlled trial in Thailand found that a simplified, short-course (SC) maternal ZDV regimen (oral ZDV twice daily starting from 36-weeks gestation and oral ZDV during labor) with no infant treatment could reduce MTCT by 50%, from 19% to 9% (23). Another Thai trial with a 2 × 2 factorial design compared long versus short maternal ZDV (starting from 28- or 36-weeks gestation) and long versus short infant ZDV (for 6 weeks or 3 days). At the first interimefficacy analysis, the short–short arm was discontinued because it was inferior to the

4

MOTHER TO CHILD HUMAN IMMUNODEFICIENCY VIRUS TRANSMISSION TRIALS

long–long arm (10.5% versus 4.1% MTCT), but at the end of the trial the long–long, long–short, and short–long arms had equivalent MTCT (6.5% versus 4.7% versus 8.6%); the rate of in utero MTCT was higher with short versus long antepartum ZDV, which suggested that longer treatment of the infant cannot substitute for longer treatment of the mother (24). A trial in Africa among breastfeeding HIV-infected women showed that a short-course combination regimen of ZDV and 3TC starting from 36-weeks gestation, orally during delivery, and for 1 week after delivery to the mother and infant, reduced MTCT at age 6 weeks by approximately 63% compared with placebo (25). 3 THE EUROPEAN MODE OF DELIVERY TRIAL Because most MTCT is thought to occur during labor and delivery, interventions at the time of delivery could potentially reduce MTCT risk substantially, especially for women who have little or no prenatal care, are diagnosed as HIV-infected late in pregnancy, or have high viral load near the time of delivery. In the early 1990s, when results of some observational studies but not others suggested that elective cesarean-section (ECS) delivery before membrane rupture and before labor could reduce MTCT compared with vaginal delivery, the European Mode of Delivery Collaboration (EMDC) conducted a randomized trial that demonstrated the efficacy of ECS (26). 3.1 Objectives It was hypothesized that ECS might reduce MTCT risk by avoiding direct contact with maternal vaginal secretions and infected blood in the birth canal and by reducing influx of maternal blood during uterine contractions. The primary objectives of the EMDC trial were to assess the relative risks and benefits of ECS versus vaginal delivery overall and in subgroups defined according to ZDV use and viral load (26). 3.2 Study Design HIV-infected pregnant women who were at 34–36-weeks gestation and had no indication for or contraindication to ECS were

enrolled in Italy from 1993 to 1998, and in France, the United Kingdom, Spain, Switzerland, and Sweden from 1995 to 1998. Women were randomized to ECS at 38-weeks gestation or vaginal delivery. Women assigned to the ECS group who went into labor before 38 weeks-gestation were delivered by cesarean section if labor was diagnosed before the start of the second stage. Women assigned to vaginal delivery waited for spontaneous labor unless a clinical decision was made for cesarean section. The primary efficacy outcome measure was MTCT by age 18 months. The original planned sample size was about 450 women, based on an anticipated MTCT rate of 15% in the vaginal delivery group and an estimated 50% reduction associated with ECS. With the publication of the PACTG 076 results, the assumed MTCT rate in the vaginal delivery group was decreased to 8%, which increased the required sample size to 1200 women. No interim analyses were planned. However, in March 1998, when the initially planned sample size was reached, and with increasing evidence from observational studies of a protective effect of ECS that was greater than previously suggested, an interim efficacy analysis was conducted and enrollment was discontinued because of a significant difference in MTCT between the ECS and vaginal delivery groups (26). 3.3 Results In all, 70% of women in the ECS group and 58% in the vaginal-delivery group received antiretroviral therapy during pregnancy, generally the PACTG 076 ZDV regimen. Overall, 88.3% of the women assigned to the ECS group delivered by cesarean section (4.3% of which were emergency procedures), and 11.7% of women delivered vaginally. In this study, 73.2% of the women assigned to the vaginal delivery group delivered vaginally and 26.8% delivered by cesarean section (54% of which were emergency procedures). In an intent-to-treat analysis, MTCT rates were 1.8% in the ECS group and 10.5% in the vaginal delivery group (P < 0.001). MTCT rates according to actual mode of delivery were 2.4% with ECS, 10.2% with vaginal delivery, and 8.8% with emergency cesarean section (after membrane rupture or

MOTHER TO CHILD HUMAN IMMUNODEFICIENCY VIRUS TRANSMISSION TRIALS

onset of labor). Few postpartum complications occurred and no serious adverse events occurred in either group (26).

3.4 Conclusions and Questions for Additional Study The results of both the intent-to-treat and as-delivered analyses suggested that ECS significantly lowers MTCT without a significantly increased risk of complications in women who received no antiretroviral therapy or only ZDV during pregnancy. Interpretation was somewhat complicated, however, because more women in the ECS group received ZDV, and substantial numbers of women did not deliver according to their assigned mode. Nonetheless, after publication of the results of the EMDC trial and an individual patient data meta-analysis of 15 prospective cohort studies that also observed a protective effect of ECS (27), the American College of Obstetrics and Gynecology (ACOG) recommended that ECS be offered to all HIVinfected pregnant women, and the ECS rate among HIV-infected women in the United States and Europe increased substantially (28). Because the EMDC trial was conducted before the advent of viral-load testing and combination antiretroviral therapy, one important question for additional study was whether ECS would be worthwhile among women with low viral loads or who receive combination antiretroviral regimens. Subsequent observational studies have suggested that MTCT risk is very low in such women, and current ACOG and USPHS guidelines recommend ECS only for women with viral load greater than 1000 copies per milliliter, for whom benefits with respect to reduction of transmission risk generally outweigh the increased risk of maternal and infant morbidity and the cost of ECS delivery (28). Another unanswered question was how soon after onset of labor or rupture of membranes the benefit of ECS is lost. These questions are unlikely to be answered by randomized clinical trials because of the large sample sizes required (28).

5

4 THE HIV NETWORK FOR PREVENTION TRIALS 012 TRIAL The PACTG 076 ZDV regimen, ECS, and even short-course ZDV or ZDV/3TC regimens were too expensive or complex for resourcelimited countries, and MTCT via breastfeeding remained a problem. The landmark, NIHsponsored HIV Network for Prevention Trials (HIVNET) 012 trial demonstrated the efficacy of a very simple and inexpensive regimen: a single dose of nevirapine (SD-NVP) given to the mother during labor and to the infant within 72 hours after birth (29). 4.1 Objectives It was hypothesized that giving SD-NVP to the woman during labor could protect the infant from infection during delivery and during the first 1–2 weeks of life via breastfeeding, because NVP has potent antiviral activity, is rapidly absorbed when given orally, passes quickly through the placenta, and has a long half-life in pregnant women and babies (29). The primary objectives were to determine the safety and rates of MTCT and infant HIV-free survival after exposure to SD-NVP or SC-ZDV during labor and the first week of life. 4.2 Study Design The HIVNET 012 trial was conducted in Uganda and enrolled HIV-infected pregnant women who were at greater than 32-weeks gestation and not currently receiving antiretroviral or HIV immunotherapy from November 1997 to April 1999 (29). The trial was originally designed as a 1500 patient, double-blind, placebo-controlled trial to determine the efficacy of SD-NVP and SC-ZDV (during labor and to the infant for 1 week), but enrollment to the placebo arm was discontinued after release of the Thai shortcourse ZDV trial results (23), after only 49 women had been enrolled in HIVNET 012. Enrollment into the open-label SD-NVP and SC-ZDV arms was continued to provide preliminary screening data on efficacy to select one of the two regimens for inclusion in a redesigned, future Phase III efficacy trial, in which the comparator would be a standard antiretroviral regimen to be chosen based

6

MOTHER TO CHILD HUMAN IMMUNODEFICIENCY VIRUS TRANSMISSION TRIALS

on the anticipated results of other continuing perinatal trials. The sample size of 500 assessable mother–infant pairs was chosen to provide 80% probability to choose SD-NVP or SC-ZDV correctly if the true difference in MTCT rates between arms were 0% or 8%, respectively. The primary efficacy endpoints were MTCT and HIV-free survival at 6–8 weeks, 14–16 weeks, and 18 months of age. The study was monitored by an independent DSMB and interim-efficacy analyses (with an O’Brien–Fleming stopping boundary) were to be performed approximately annually during the projected 3-year study duration. Postpartum follow-up was originally planned to be 18 months for infants and 6 weeks for mothers, but it was subsequently extended to 5 years (30). 4.3 Results In the final efficacy analysis based on 617 assessable mother–infant pairs, the estimated risks of MTCT in the SD-NVP and SC-ZDV groups were 11.8% and 20.0% by age 6–8 weeks, respectively (P = 0.0063) (30). The cumulative MTCT rates in both groups increased by ages 14–16 weeks and 18 months because of continued breastfeeding, but the absolute and relative reductions in MTCT risk with SD-NVP (8.2% and 41%, respectively) were sustained through age 18 months. Results for HIV-free survival were similar. Both regimens were well tolerated with few serious side effects (30). Mutations in HIV that conferred resistance to NVP were detected at 6–8 weeks postpartum in 25% of mothers (31) and 46% of assessable HIV-infected infants exposed to NVP, but these mutations were no longer detected after 12–24 months postpartum (32). 4.4 Conclusions and Questions for Additional Study The HIVNET 012 results suggested that SD-NVP was efficacious and safe. In most resource-limited countries, especially in subSaharan Africa, national PMTCT programs subsequently were built around the HIVNET 012 regimen (33). Extensive controversy about whether the results of HIVNET 012 were valid developed in 2002, after Boehringer Ingelheim (BI),

which is the manufacturer of NVP, decided to pursue a U.S. Food and Drug Administration (FDA) labeling change to include PMTCT using HIVNET 012 as a registrational trial (34). As a result, reviews of the safety data that were far more in-depth than would ordinarily occur for a NIH-sponsored trial that was not intended to support an FDA submission were conducted, including a pre-FDA inspection audit by an NIH contractor who found some deficiencies in study conduct. The findings led to a comprehensive and lengthy remonitoring effort by NIH, withdrawal of BI’s FDA application because of inability to meet time constraints, and ultimately a U.S. Institutes of Medicine review of HIVNET 012, which concluded that no reason could be found to retract the publications or alter the conclusions of the trial (34). Two important areas for additional study suggested by the HIVNET 012 results included the efficacy of combining SD-NVP with other regimens and the implications and prevention of NVP resistance after SDNVP. The efficacy of combining SD-NVP with other regimens depends on their duration and potency. When the mother receives antiretrovirals during pregnancy, adding SDNVP can increase the efficacy of short-course regimens: The PHPT-2 trial in Thailand showed that adding SD-NVP to SC-ZDV in a nonbreastfeeding population could reduce the MTCT risk to 2% (35); when this combination regimen was used in a breastfeeding setting, a somewhat greater MTCT risk of 6.5% was observed (36). However, the PACTG 316 trial suggested that SD-NVP does not seem to provide any additional efficacy when added to the standard antiretroviral regimens used in developed countries (at a minimum, the full PACTG 076 ZDV regimen, often combined with at least two other antiretrovirals). The trial was stopped for futility because of low MTCT rates of 1.6% with placebo and 1.4% with SD-NVP (6). The efficacy of SD-NVP plus SC-ZDV in infants of mothers who did not receive antiretrovirals during pregnancy was assessed in two clinical trials in Malawi; SD-NVP plus SC-ZDV provided greater efficacy than SD-NVP alone when the mother did not receive any antiretrovirals during labor (37) but not when the mother received SD-NVP during labor (38).

MOTHER TO CHILD HUMAN IMMUNODEFICIENCY VIRUS TRANSMISSION TRIALS

The high prevalence of NVP resistance mutations after SD-NVP is of concern because it could compromise the effectiveness of (1) SD-NVP for prevention of MTCT in subsequent pregnancies and (2) future antiretroviral treatment for HIV disease, which, in resource-limited settings, often includes drugs in the same class [non-nucleoside reverse transcriptase inhibitor (NNRTI)] as NVP (39). Initial data from secondary analyses or follow-up studies of clinical trials suggest that SD-NVP remains effective in subsequent pregnancies and NNRTI-based treatment may still be effective after SDNVP exposure, particularly if sufficient time has elapsed; randomized trials of the latter question in women and children are in progress (39). One approach to reducing NVP resistance after SD-NVP would be to avoid the maternal NVP dose; in the perinatal component of the 2 × 2 factorial MASHI trial in Botswana, equivalent MTCT rates with and without the maternal NVP dose were observed when mothers received SCZDV and infants received SD-NVP and SCZDV (40). Another approach that has been studied is adding additional antiretrovirals during and after delivery, under the hypothesis that NVP resistance emerges because NVP remains present in subtherapeutic concentrations for several days or weeks because of its long half-life (41). A clinical trial in South Africa showed that adding ZDV/3TC for 3 or 7 days after delivery to cover the NVP ‘‘tail’’ can reduce the prevalence of NVP resistance mutations after SD-NVP but did not seem to have an effect on MTCT. Randomized trials are ongoing to assess whether ‘‘tail therapy’’ for longer durations or with more potent antiretrovirals could reduce the prevalence of NVP resistance even more (39). 5

THE MASHI TRIAL

Breastfeeding accounts for up to half of all MTCT in resource limited-settings, but it also provides important benefits, such as protection against other causes of infant mortality and morbidity when replacement feeding such as infant formula or animal milk is not safely available (e.g., because of a lack of clean water), culturally acceptable,

7

or affordable (7). As of October 2007, several clinical trials of antiretroviral, immunologic, or behavioral interventions to reduce breastfeeding MTCT are ongoing but only a few have been completed (7). The postpartum component of the NIH-funded MASHI trial (42) is described to illustrate some key issues in this active area of research. 5.1 Objectives A previous randomized clinical trial of breastfeeding versus formula feeding in Kenya found that formula feeding could prevent an estimated 44% of MTCT without leading to excess infant mortality, and it was therefore associated with improved HIV-free survival (43). However, study participants were required to have access to clean water, which is often limited outside urban areas, and it did not receive any antiretroviral treatment or prophylaxis, which could provide protection from MTCT during breastfeeding (42). The primary objective of the postpartum component of the MASHI trial was to compare the efficacy and safety of formula feeding versus breastfeeding plus extended infant ZDV for the prevention of postpartum MTCT. 5.2 Study Design The MASHI trial was a 2 × 2 factorial randomized trial that enrolled HIV-infected women at 33–35-weeks gestation in Botswana from March 2001 to October 2003 (42). All women received SC-ZDV during pregnancy and labor, and all infants received 1 month of ZDV. The two randomization factors were (1) SD-NVP versus placebo and (2) breastfeeding plus infant ZDV until 6 months of age versus formula feeding. Initially, mothers and infants were both randomized to SD-NVP or placebo; however, almost midway through enrollment, after release of the PHPT-2 trial results (35), the MASHI design was revised to give all infants SD-NVP and randomize only the mothers to SD-NVP or placebo (42). This modification coincided with the availability of antiretroviral therapy in Botswana for qualifying HIV patients, so the original and revised designs can be viewed as two 2 × 2 factorial trials. The primary efficacy outcome measures were MTCT by

8

MOTHER TO CHILD HUMAN IMMUNODEFICIENCY VIRUS TRANSMISSION TRIALS

age 7 months and HIV-free survival by age 18 months, and the primary safety outcome measure was the rate of adverse events by 7 months of age. The planned sample size was 1,200 mother-infant pairs, to provide 80% power to detect a 7% difference in MTCT by age 7 months and 90% power to detect a 10% difference in HIV-free survival by 18 months of age, based on reference rates of 17% and 79%, respectively, a two-sided type I error of 0.05, and an annual loss-to-follow-up rate of 10%. The trial was monitored by an independent DSMB and two interim efficacy analyses were planned, with an O’Brien–Fleming stopping boundary (42). 5.3 Results The results of the SD-NVP component of the trial were described previously. Breastfeeding with infant ZDV was associated with higher MTCT (9% versus 5.6%, P = 0.04) but lower mortality (4.9% vs. 9.3%, P = 0.003) by age 7 months compared with formula feeding, which resulted in comparable HIV-free survival at 18 months (84.9% vs. 86.1%, P = 0.60). The most common causes of infant deaths were diarrhea and pneumonia. A statistically significant interaction occurred between the feeding strategies and the original maternal/infant SD-NVP/placebo intervention, with a greater increase in MTCT with breastfeeding plus ZDV and SDNVP (P = 0.04), and a greater decrease in mortality with breastfeeding plus ZDV and placebo (P = 0.03); no significant interaction was observed in the revised study (42). 5.4 Conclusions and Questions for Additional Study The results of the MASHI trial suggested that formula feeding was associated with a lower risk of MTCT but a higher risk of early mortality compared with breastfeeding plus infant ZDV; that is, formula-fed infants escaped HIV infection but then died of other causes, which led to similar rates of HIV-free survival at 18 months in the two groups (42). The MASHI study demonstrated the risks of formula feeding but did not definitively support infant ZDV to prevent breastfeeding MTCT, which highlighted the need for additional study (42).

Prevention of breastfeeding MTCT in a safe manner remains one of the major challenges and the most active areas in MTCT research. Several randomized trials are ongoing or planned to assess the efficacy and safety of other interventions to prevent postpartum MTCT without adversely affecting infant or maternal health, which includes antiretroviral and immune interventions to reduce MTCT risk during breastfeeding, and interventions to make replacement feeding safer (7). The issues are complex and tension often exists between what is best for the mother and what is best for the baby. For example, many pregnant and postpartum HIV-infected women are asymptomatic and have relatively high CD4+ cell counts, and therefore would not require antiretroviral therapy for their HIV disease if they were not pregnant or breastfeeding; the safety of giving such mothers combination antiretrovirals for the duration of breastfeeding is unknown (33). A preventive infant vaccine would be a very attractive approach to both reducing the risk of MTCT while providing nutrition and protection against other infectious causes of morbidity and mortality via breastfeeding (44), but scientists believe that a vaccine is most likely at least 10 years away (45). REFERENCES 1. Kourtis AP, Lee FK, Abrams EJ, Jamieson DJ, Bulterys M. Mother-to-child transmission of HIV-1: timing and implications for prevention. Lancet Infect Dis. 2006; 6: 726–732. 2. Minkoff H. For whom the bell tolls. Am. J. Obstet. Gynecol. 2007; 197:S1–S2. 3. De Cock KM, Fowler MG, Mercier E, de Vincenzi I, Saba J, Hoff E, Alnwick DJ, Rogers M, Shaffer N. Prevention of mother-to-child HIV transmission in resource-poor countries: translating research into policy and practice. JAMA 2000; 283: 1175–1182. 4. Breastfeeding and HIV International Transmission Study Group, Coutsoudis A, Dabis F, Fawzi W, Gaillard P, Haverkamp G, Harris DR, Jackson JB, Leroy V, Meda N, Msellati P, Newell ML, Nsuati R, Read JS, Wiktor S. Late postnatal transmission of HIV-1 in breast-fed children: an individual patient data metaanalysis. J. Infect. Dis. 2004; 189: 2154–2166.

MOTHER TO CHILD HUMAN IMMUNODEFICIENCY VIRUS TRANSMISSION TRIALS 5. Cooper ER, Charurat M, Mofenson L, Hanson IC, Pitt J, Diaz C, Hayani K, Handelsman E, Smeriglio V, Hoff R, Blattner W; Women and Infants’ Transmission Study Group. Combination antiretroviral strategies for the treatment of pregnant HIV-1-infected women and prevention of perinatal HIV-1 transmission. J. Acquir. Immune Defic. Syndr. 2002; 29: 484–494. 6. Dorenbaum A, Cunningham CK, Gelber RD, Culnane M, Mofenson L, Britto P, Rekacewicz C, Newell ML, Delfraissy JF, CunninghamSchrader B, Mirochnick M, Sullivan JL; International PACTG 316 Team. Two-dose intrapartum/newborn nevirapine and standard antiretroviral therapy to reduce perinatal HIV transmission: a randomized trial. JAMA 2002; 288: 189–198. 7. Kourtis AP, Jamieson DJ, de Vincenzi I, Taylor A, Thigpen MC, Dao H, Farley T, Fowler MG. Prevention of human immunodeficiency virus-1 transmission to the infant through breastfeeding: new developments. Am. J. Obstet. Gynecol. 2007; 197(Suppl 3):S113–S122. 8. Public Health Service Task Force. Recommendations for use of antiretroviral drugs in pregnant HIV-1 infected women for maternal health and interventions to reduce perinatal HIV-1 transmission in the United States. October 12, 2006 update. http://AIDSinfo.nih.gov. 9. Connor EM, Sperling RS, Gelber R, Kiselev P, Scott G, O’Sullivan MJ, VanDyke R, Bey M, Shearer W, Jacobson RL, Jimenez E, O’Neill E, Bazin B, Delfraissy J-F, Culnane M, Coombs R, Elkins M, Moye J, Stratton P, Balsley J; Pediatric AIDS Clinical Trials Group Protocol 076 Study Group. Reduction of maternal-infant transmission of human immunodeficiency virus type 1 with zidovudine treatment. Pediatric AIDS Clinical Trials Group Protocol 076 Study Group. N. Engl. J. Med. 1994; 331: 1173–1180. 10. Sperling RS, Shapiro DE, Coombs RW, Todd JA, Herman SA, McSherry GD, O’Sullivan MJ, Van Dyke RB, Jimenez E, Rouzioux C, Flynn PM, Sullivan JL. Maternal viral load, zidovudine treatment, and the risk of transmission of human immunodeficiency virus type 1 from mother to infant. Pediatric AIDS Clinical Trials Group Protocol 076 Study Group. N. Engl. J. Med. 1996; 335: 1621–1629. 11. Shapiro DE, Sperling RS, Coombs RW. Effect of zidovudine on perinatal HIV-1 transmission

12.

13.

14.

15.

16.

17.

18.

9

and maternal viral load. Pediatric AIDS Clinical Trials Group 076 Study Group. Lancet. 1999; 354: 156; author reply 157–158. Sperling RS, Shapiro DE, McSherry GD, Britto P, Cunningham BE, Culnane M, Coombs RW, Scott G, Van Dyke RB, Shearer WT, Jimenez E, Diaz C, Harrison DD, Delfraissy JF. Safety of the maternal-infant zidovudine regimen utilized in the Pediatric AIDS Clinical Trial Group 076 Study. AIDS. 1998; 12: 1805–1813. McSherry GD, Shapiro DE, Coombs RW, McGrath N, Frenkel LM, Britto P, Culnane M, Sperling RS. The effects of zidovudine in the subset of infants infected with human immunodeficiency virus type-1 (Pediatric AIDS Clinical Trials Group Protocol 076). J. Pediatr. 1999; 134: 717–724. Eastman PS, Shapiro DE, Coombs RW, Frenkel LM, McSherry GD, Britto P, Herman SA, Sperling RS. Maternal viral genotypic zidovudine resistance and infrequent failure of zidovudine therapy to prevent perinatal transmission of human immunodeficiency virus type 1 in pediatric AIDS Clinical Trials Group Protocol 076. J. Infect. Dis. 1998; 177: 557–564. Jamieson DJ, Clark J, Kourtis AP, Taylor AW, Lampe MA, Fowler MG, Mofenson LM. Recommendations for human immunodeficiency virus screening, prophylaxis, and treatment for pregnant women in the United States. Am. J. Obstet. Gynecol. 2007; 197(3 Suppl):S26–S32. Centers for Disease Control and Prevention. Recommendations of the US Public Health Service Task Force on the use of zidovudine to reduce perinatal transmission of human immunodeficiency virus. MMWR Recomm. Rep. 1994; 43: 1–20. Bardeguez AD, Shapiro DE, Mofenson LM, Coombs R, Frenkel LM, Fowler MG, Huang S, Sperling RS, Cunningham B, Gandia J, Maupin R, Zorrilla CD, Jones T, O’Sullivan MJ; Pediatrics AIDS Clinical Trials Group 288 Protocol Team. Effect of cessation of zidovudine prophylaxis to reduce vertical transmission on maternal HIV disease progression and survival. J. Acquir. Immune Defic. Syndr. 2003; 32: 170–181. Culnane M, Fowler M, Lee SS, McSherry G, Brady M, O’Donnell K, Mofenson L, Gortmaker SL, Shapiro DE, Scott G, Jimenez E, Moore EC, Diaz C, Flynn PM, Cunningham B, Oleske J. Lack of long-term effects of in utero exposure to zidovudine among uninfected children born to HIV-infected women. Pediatric

10

MOTHER TO CHILD HUMAN IMMUNODEFICIENCY VIRUS TRANSMISSION TRIALS AIDS Clinical Trials Group Protocol 219/076 Teams. JAMA. 1999; 281: 151–157.

19. Hanson IC, Antonelli TA, Sperling RS, Oleske JM, Cooper E, Culnane M, Fowler MG, Kalish LA, Lee SS, McSherry G, Mofenson L, Shapiro DE. Lack of tumors in infants with perinatal HIV-1 exposure and fetal/neonatal exposure to zidovudine. J. Acquir. Immune Defic. Syndr. Hum. Retrovirol. 1999; 20: 463–467. 20. Stiehm ER, Lambert JS, Mofenson LM, Bethel J, Whitehouse J, Nugent R, Moye J Jr, Glenn Fowler M, Mathieson BJ, Reichelderfer P, Nemo GJ, Korelitz J, Meyer WA 3rd, Sapan CV, Jimenez E, Gandia J, Scott G, O’Sullivan MJ, Kovacs A, Stek A, Shearer WT, Hammill H. Efficacy of zidovudine and human immunodeficiency virus (HIV) hyperimmune immunoglobulin for reducing perinatal HIV transmission from HIV-infected women with advanced disease: results of Pediatric AIDS Clinical Trials Group protocol 185. J. Infect. Dis. 1999; 179: 567–575. 21. Mandelbrot L, Landreau-Mascaro A, Rekacewicz C, Berrebi A, Benifla JL, Burgard M, Lachassine E, Barret B, Chaix ML, Bongain A, Ciraru-Vigneron N, Crenn-Hebert C, Delfraissy JF, Rouzioux C, Mayaux MJ, Blanche S; Agence Nationale de Recherches sur le SIDA (ANRS) 075 Study Group. Lamivudine-zidovudine combination for prevention of maternal-infant transmission of HIV-1. JAMA. 2001; 285: 2083–2093. 22. Gelber RD, Lindsey JC, MaWhinney S. Clinical Trials to Reduce the Risk of Maternal-Infant Transmission of HIV Infection. In: AIDS Clinical Trials. Finkelstein DM, Schoenfeld DA, eds. 1995. John Wiley & Sons, New York. pp. 287–302. 23. Shaffer N, Chuachoowong R, Mock PA, Bhadrakom C, Siriwasin W, Young NL, Chotpitayasunondh T, Chearskul S, Roongpisuthipong A, Chinayon P, Karon J, Mastro TD, Simonds RJ. Short-course zidovudine for perinatal HIV-1 transmission in Bangkok, Thailand: a randomised controlled trial. Bangkok Collaborative Perinatal HIV Transmission Study Group. Lancet 1999; 353: 773–780. 24. Lallemant M, Jourdain G, Le Coeur S, Kim S, Koetsawang S, Comeau AM, Phoolcharoen W, Essex M, McIntosh K, Vithayasai V. A trial of shortened zidovudine regimens to prevent mother-to-child transmission of human immunodeficiency virus type 1. Perinatal HIV Prevention Trial (Thailand) Investigators. N. Engl. J. Med. 2000; 343: 982–991.

25. Petra Study Team. Efficacy of three shortcourse regimens of zidovudine and lamivudine in preventing early and late transmission of HIV-1 from mother to child in Tanzania, South Africa, and Uganda (Petra study): a randomised, double-blind, placebo-controlled trial. Lancet 2002; 359: 1178–1186. 26. No authors listed. Elective caesarean-section versus vaginal delivery in prevention of vertical HIV-1 transmission: a randomised clinical trial. The European Mode of Delivery Collaboration. Lancet 1999; 353: 1035–1039. 27. No authors listed. The mode of delivery and the risk of vertical transmission of human immunodeficiency virus type 1--a meta-analysis of 15 prospective cohort studies. The International Perinatal HIV Group. N. Engl. J. Med. 1999; 340: 977–987. 28. Jamieson DJ, Read JS, Kourtis AP, Durant TM, Lampe MA, Dominguez KL. Cesarean delivery for HIV-infected women: recommendations and controversies. Am. J. Obstet. Gynecol. 2007; 197(Suppl 3):S96–100. 29. Guay LA, Musoke P, Fleming T, Bagenda D, Allen M, Nakabiito C, Sherman J, Bakaki P, Ducar C, Deseyve M, Emel L, Mirochnick M, Fowler MG, Mofenson L, Miotti P, Dransfield K, Bray D, Mmiro F, Jackson JB. Intrapartum and neonatal single-dose nevirapine compared with zidovudine for prevention of mother-to-child transmission of HIV-1 in Kampala, Uganda: HIVNET 012 randomised trial. Lancet. 1999; 354: 795–802. 30. Jackson JB, Musoke P, Fleming T, Guay LA, Bagenda D, Allen M, Nakabiito C, Sherman J, Bakaki P, Owor M, Ducar C, Deseyve M, Mwatha A, Emel L, Duefield C, Mirochnick M, Fowler MG, Mofenson L, Miotti P, Gigliotti M, Bray D, Mmiro F. Intrapartum and neonatal single-dose nevirapine compared with zidovudine for prevention of mother-to-child transmission of HIV-1 in Kampala, Uganda: 18-month follow-up of the HIVNET 012 randomised trial. Lancet 2003; 362: 859–868. 31. Eshleman SH, Guay LA, Mwatha A, Brown ER, Cunningham SP, Musoke P, Mmiro F, Jackson JB. Characterization of nevirapine resistance mutations in women with subtype A vs. D HIV-1 6-8 weeks after single-dose nevirapine (HIVNET 012). J. Acquir. Immune Defic. Syndr. 2004; 35: 126–130. 32. Eshleman SH, Mracna M, Guay LA, Deseyve M, Cunningham S, Mirochnick M, Musoke P, Fleming T, Glenn Fowler M, Mofenson LM, Mmiro F, Jackson JB. Selection and fading of resistance mutations in women and infants receiving nevirapine to prevent HIV-1 vertical

MOTHER TO CHILD HUMAN IMMUNODEFICIENCY VIRUS TRANSMISSION TRIALS transmission (HIVNET 012). AIDS 2001; 15: 1951–1957. 33. Dao H, Mofenson LM, Ekpini R, Gilks CF, Barnhart M, Bolu O, Shaffer N. International recommendations on antiretroviral drugs for treatment of HIV-infected women and prevention of mother-to-child HIV transmission in resource-limited settings: 2006 update. Am. J. Obstet. Gynecol. 2007; 197(Suppl 3):S42–S55. 34. Institute of Medicine of the National Academy of Sciences. Review of the HIVNET 012 Perinatal HIV Prevention Study. 2005. National Academies Press. Washington, DC. pp. 1–10. http://www.nap.edu/catalog.php?record id= 11264#toc.

41.

42.

35. Lallemant M, Jourdain G, Le Coeur S, Mary JY, Ngo-Giang-Huong N, Koetsawang S, Kanshana S, McIntosh K, Thaineua V; Perinatal HIV Prevention Trial (Thailand) Investigators. Single-dose perinatal nevirapine plus standard zidovudine to prevent mother-tochild transmission of HIV-1 in Thailand. N. Engl. J. Med. 2004; 351: 217–228. 36. Dabis F, Bequet L, Ekouevi DK, Viho I, Rouet F, Horo A, Sakarovitch C, Becquet R, Fassinou P, Dequae-Merchadou L, Welffens-Ekra C, Rouzioux C, Leroy V; ANRS 1201/1202 DITRAME PLUS Study Group. Field efficacy of zidovudine, lamivudine and singledose nevirapine to prevent peripartum HIV transmission. AIDS 2005; 19: 309–318. 37. Taha TE, Kumwenda NI, Gibbons A, Broadhead RL, Fiscus S, Lema V, Liomba G, Nkhoma C, Miotti PG, Hoover DR. Short postexposure prophylaxis in newborn babies to reduce mother-to-child transmission of HIV-1: NVAZ randomised clinical trial. Lancet 2003; 362: 1171–1177. 38. Taha TE, Kumwenda NI, Hoover DR, Fiscus SA, Kafulafula G, Nkhoma C, Nour S, Chen S, Liomba G, Miotti PG, Broadhead RL. Nevirapine and zidovudine at birth to reduce perinatal transmission of HIV in an African setting: a randomized controlled trial. JAMA 2004; 292: 202–209. 39. McConnell MS, Stringer JS, Kourtis AP, Weidle PJ, Eshleman SH. Use of single-dose nevirapine for the prevention of mother-tochild transmission of HIV-1: does development of resistance matter? Am. J. Obstet. Gynecol. 2007; 197(Suppl 3):S56–S63. 40. Shapiro RL, Thior I, Gilbert PB, Lockman S, Wester C, Smeaton LM, Stevens L, Heymann SJ, Ndung’u T, Gaseitsiwe S, Novitsky V, Makhema J, Lagakos S, Essex M. Maternal single-dose nevirapine versus placebo as

43.

44.

45.

11

part of an antiretroviral strategy to prevent mother-to-child HIV transmission in Botswana. AIDS. 2006; 20: 1281–1288. Cressey TR, Jourdain G, Lallemant MJ, Kunkeaw S, Jackson JB, Musoke P, Capparelli E, Mirochnick M. Persistence of nevirapine exposure during the postpartum period after intrapartum single-dose nevirapine in addition to zidovudine prophylaxis for the prevention of mother-to-child transmission of HIV-1. J. Acquir. Immune Defic. Syndr. 2005; 38: 283–288. Thior I, Lockman S, Smeaton LM, Shapiro RL, Wester C, Heymann SJ, Gilbert PB, Stevens L, Peter T, Kim S, van Widenfelt E, Moffat C, Ndase P, Arimi P, Kebaabetswe P, Mazonde P, Makhema J, McIntosh K, Novitsky V, Lee TH, Marlink R, Lagakos S, Essex M; Mashi Study Team. Breastfeeding plus infant zidovudine prophylaxis for 6 months vs formula feeding plus infant zidovudine for 1 month to reduce mother-to-child HIV transmission in Botswana: a randomized trial: the Mashi Study. JAMA. 2006; 296: 794–805. Nduati R, John G, Mbori-Ngacha D, Richardson B, Overbaugh J, Mwatha A, NdinyaAchola J, Bwayo J, Onyango FE, Hughes J, Kreiss J. Effect of breastfeeding and formula feeding on transmission of HIV-1: a randomized clinical trial. JAMA 2000; 283: 1167–1174. Fowler MG, Lampe MA, Jamieson DJ, Kourtis AP, Rogers MF. Reducing the risk of motherto-child human immunodeficiency virus transmission: past successes, current progress and challenges, and future directions. Am. J. Obstet. Gynecol. 2007; 197(Suppl 3):S3–S9. Stark K. Ending of HIV vaccine trial jolts industry. Philadelphia Inquirer. Oct 7, 2007. http://www.philly.com/philly/business/ 10296647.html.

FURTHER READING NIH AIDSinfo website. http://aidsinfo.nih.gov. UNAIDS website. http://www.unaids.org. CDC HIV/AIDS website. http://www.cdc.gov/hiv. IMPAACT website. http://impaact.s-3.com. I-Base website. http://www.i-base.info.

CROSS-REFERENCES AIDS Clinical Trials Group (ACTG) Human Immunodeficiency Virus (HIV) Trials Data and Safety Monitoring Board

12

MOTHER TO CHILD HUMAN IMMUNODEFICIENCY VIRUS TRANSMISSION TRIALS Interim Analysis Factorial Designs

MULTICENTER TRIAL

from the different centers performing functions in relation to a particular study protocol. The centers are clinics, coordinating centers, and other resource centers as needed, such as central laboratories and reading centers, for conduct of the trial. Broadly, the complement of multicenter trial is single center trial, defined as (1):

CURTIS L. MEINERT The Johns Hopkins University Center for Clinical Trials Bloomberg School of Public Health Baltimore, Maryland

1

DEFINITION

1. A trial performed at or from a single site: (a) Such a trial, even if performed in association with a coalition of clinics in which each clinic performs its own trial, but in which all trials focus on the same disease or condition (e.g., such a coalition formed to provide preliminary information on a series of different approaches to the treatment of hypertension by stress control or reduction); (b) A trial not having any clinical centers and a single resource center (e.g., the Physicians’ Health Study). 2. A trial involving a single clinic; with or without satellite clinics or resource centers. 3. A trial involving a single clinic and a center to receive and process data. 4. A trial involving a single clinic and one or more resource centers. ant: multicenter trial Usage note: Note that the usual line of demarcation between single and multicenter is determined by whether or not there is more than one treatment or data collection site. Hence, a trial having multiple centers may still be classified as a single-center trial if it has only one treatment or data collection site.

The term ‘‘multicenter trial’’ is variously defined: multicenter trial (1): 1. A trial involving two or more clinical centers, a common study protocol, and a data center, data coordinating center, or coordinating center to receive, process, and analyze study data. 2. A trial involving at least one clinical center or data collection site and one or more resource centers. 3. A trial involving two or more clinics or data collection sites. syn: collaborative trial (not recommended), cooperative trial (not recommended) ant: singlecenter trial Usage note: Preferred to collaborative trial or cooperative trial for reasons indicated in usage notes for those two terms. See single-center trial for comments on line of demarcation between single and multicenter trials. multicentre study (2): A study carried out at more than one study centre. multicentre trial (3): A clinical trial conducted according to a single protocol but at more than one site, and therefore carried out by more than one investigator.

Generally, in the language of trials, trials are assumed to be single center unless otherwise indicated. Typically, a multicenter trial involves two or more administratively distinct clinics as well as a coordinating center or data center and, perhaps, other resource centers as well. However, trials exist, such as the Physician’s Health Study (PHS; see Section 4 for sketch), that can be classified as single center or multicenter, depending on the characteristics used for classification. People in the PHS were recruited, enrolled, and followed by mail and telephone and, hence, no clinical centers existed. However, a coordinating center and a differentiated organizational structure did exist and, hence, from an organizational perspective, the trial can be classified as multicenter. (The trial is not indexed as multicenter by the National Library of Medicine (NLM) indexers and,

The National Library of Medicine (medical subject headings—annotated alphabetic list 2003) defines multicenter trials as controlled studies that are planned and carried out by several cooperating institutions to assess certain variables and outcomes in specified patient populations, for example, a multicenter study of congenital anomalies in children. A multicenter trial involves two or more administratively distinct study centers and a leadership structure involving investigators

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

MULTICENTER TRIAL

therefore, is regarded as single center under their system of classification.) 2

RELATED TERMS

Various other terms, as taken from Meinert (1), of relevance to multicenter structures are: multi-protocol: (adj) Of or relating to a research activity involving two or more protocols (e.g. as in a multi-study or program project. multi-site: (adj) 1. Relating to or having more than one site. 2. multicenter multi-study: (adj) Of, relating to, or consisting of multiple studies. ant: singlestudy rt: multi-trial Usage note: Use multi-trial if all studies are trials. See note for study n. multi-study: (n) A study having two or more separate and distinct studies performed under the same umbrella organizational structure; multi-trial when studies are trials. Usage note: Not to be confused with a study having a series of substudies. multi-study design: (n) A design involving two or more studies; performed on the same or different study populations. ant: single-study design rt: multi-trial design, multi-study structure Usage note: See multi-study. multi-study network: (n) A network of centers organized to conduct a series of studies. multi-study structure: (n) An umbrella organizational structure created and maintained to initiate and carry out a series of related studies involving the same or different study populations. ant: single-study structure Usage note: Use multi-trial if all studies are trials. See multi-trial structure and study n for added comments. multi-trial: (adj) Of, relating to, or consisting of multiple trials. ant: single-trial rt: multi-study multi-trial: (n) A study having two or more separate and distinct trials performed under the same umbrella organizational structure.

multi-trial design: (n) A design involving two or more trials performed on the same or different people and involving different treatment protocols. ant: single-trial design Usage note: See multi-trial structure. multi-trial structure: (n) An organizational structure created or maintained to initiate and carry out a series of related trials involving the same or different study populations. ant: singletrial structure Usage note: Most useful when characterizing past or existing structures as to observed function or planned intent. Note that a structure created as a single-trial structure may ultimately be classified as a multi-trial structure if it serves the purpose of directing two or more trials, even if not originally created for that purpose. program project: (n) A collection of interrelated research activities having a common organizational structure and bearing on the same general question or issue (e.g., a collection of basic and applied research projects aimed at providing a better understanding of atherosclerosis, especially such a collection funded from a single source). sponsor: (n) 1. A person or agency that is responsible for funding a designated function or activity; sponsoring agency. 2. A person or agency that plans and carries out a specified project or activity. 3. The person or agency named in an Investigational New Drug Application or New Drug Application; usually a drug company or person at such a company, but not always (as with an INDA submitted by a representative of a research group proposing to carry out a phase III or phase IV drug trial not sponsored by a drug company). 4. A firm or business establishment marketing a product or service. study: (adj) Of or relating to one or something being evaluated or studied. Usage note: Used primarily as a general descriptor, as in study candidate, when other more precise terms are not appropriate or have undesirable connotations

MULTICENTER TRIAL

(as in trial candidate). See usage note for study n for additional notes. study: (n) 1. An experimental or nonexperimental investigation or analysis of a question, process, or phenomenon. 2. Any one of a variety of activities involving the collection, analysis, or interpretation of data. 3. Clinical trial, especially in a setting where there is a desire or need to de-emphasize the experimental nature of the investigation. 4. An investigation involving both a trial and nonexperimental investigation (as in the Coronary Artery Surgery Study, comprised of a clinical trial and a followup study) (4, 5). Usage note: Widely and loosely used. Avoid in favor of more informative, less generic, terms whenever possible (e.g. use trial rather than study when appropriate). As a label, limit use to defined sets of activities involving data collection not ordinarily characterized with a more informative, design-specific term (such as trial or followup study), or where a general term is needed to characterize a collection of activities involving a number of different designs (e.g. referring to a mix of trials and observational studies). Avoid as a synonym for a specified kind of investigation having a recognized term (as in referring to an investigation as a trial in the title or abstract of a paper and study in the body of the paper). umbrella organizational structure: (n) [multicenter multistudies] An organizational structure created to support and direct a series of separate and distinct studies; multi-study structure.

3

HISTORY

The accomplishment of scientific goals by collaboration involving disparate groups and disciplines is hardly unique to trials. Increasingly, as goals expand and projects become more complex, the only viable route to accomplishment is through the ‘‘pooling’’ of effort. Evidence of this trend extends to virtually all sciences, marked most recently by human genome projects.

3

Multicenter trials started to come into prominence in the 1940s. An early example is a trial of patulin, a metabolic product of Penicillium patulum Bainer, as a possible preventive for the common cold. It was carried out in the fall of 1943 and spring of 1944 and involved a study population of more than 100,000 persons assembled from multiple factory and postal worksites in the United Kingdom (6). One of the first multicenter trials involving a chronic disease was of streptomycin treatment of pulmonary tuberculosis (see Section 4 for sketch of this trial and others). One of the largest multicenter trials ever undertaken was carried out largely in 1954 and was done to test effectiveness of the Salk vaccine in preventing polio. All told, the study involved some 1.8 million children recruited from schools across the United States, with 650,000 of them being randomized to receive injections of the vaccine or matching placebo injections and the others serving as a nonrandomized comparison group. To a major degree, emergence of the multicenter trial coincides with transition of a society burdened with infectious diseases to one afflicted with chronic diseases and from trials involving small sample sizes and short-term treatment and follow up to trials involving large sample sizes and long-term treatment and follow up. The realization that even small benefits of treatment in reducing mortality or morbidity translates into huge reductions when projected to the general population served to underscore the need for sample sizes far beyond the reach of any one study site. By the 1960s, it was clear that the industrialized world was facing an ‘‘epidemic’’ of heart disease and of increasing rates of cancer of most types. It was clear, as well, that if treatments were to be tested for their potential in reducing or reversing the rise, that it would have to be via large-scale long-term multicenter trials. This reality gave rise to working parties in the United Kingdom and collaborative groups in the United States in the late 1950s and early 1960s to undertake multicenter treatment and prevention trials for cancer and heart disease. The ‘‘multicenter era’’ came into prominence in the late 1970s and 1980s in

4

MULTICENTER TRIAL

the United States due, in no small measure, to the efforts of Donald Fredrickson (1924–2002; Director of NIH 1975–1981) and Tom Chalmers (1917–1995). Both men were major proponents of trials and especially the long-term multicenter trial. The mindset underlying trials has progressed, albeit slowly, from one in which trials were seen as a measure of last resort to one in which they are seen increasingly as the option of choice. No doubt exists that single center trials are administratively less complicated than multicenter trials; but it is also true that, on average, multicenter trials are more robust and more rigorously designed and conducted than their single center counterparts. The move to multicenter trials has been accomplished by an ever more active role of sponsors in initiating multicenter trials. The NIH started to assume the role of initiator in the 1970s with the advent of requests for applications (RFAs; grant funding) and requests for proposals (RFPs; contract funding) for multicenter trials. To a large degree, the traditional investigator-initiated multicenter trial, at least in the United States via NIH funding, has become ‘‘endangered’’ with the advent in the mid-1990s of the requirement that investigators need permission to submit proposals in excess of $500,000 per year (direct cost). Permission is unlikely if the proposed trial is not high on the agenda of an institute. The transition to a ‘‘global economy’’ led to the creation of the International Conference of Harmonisation (ICH) in 1990 (www.ich.org). This economy has caused drug firms to take progressively more directive approaches to initiating and conducting trials they sponsor and has caused them to want study sites located in various parts of the world. For a comprehensive compilation of landmark trials published over time, visit the James Lind Library website at: jameslindlibrary.org/trial records/published. html.

4 EXAMPLES Streptomycin Treatment of Pulmonary Tuberculosis Trial Period of conduct: 1947–1948 Study population: Patients with acute bilateral pulmonary tuberculosis and unsuitable for collapse therapy Trial type: Treatment trial Locale of conduct: United Kingdom Sponsor/funding: UK Medical Research Council Sample size: 109 (not including 2 people who died in the preliminary observation period) Test treatment: Bed rest (minimum of 6 months) + streptomycin (intramuscular; 2g/day in 4 injections at 6 hour intervals) Control treatment: Bed rest (minimum of 6 months) Outcome measure: Death; radiographic changes as determined by masked readers Enrolling sites: 6 Method of assignment: Randomized blocks; stratified by clinic and gender; numbered envelopes Assignment ratio (observed): 55(S):52(C) Finding/conclusion: Streptomycin treatment beneficial in reducing the risk of death and in effecting improved chest radiographs Primary publication: Streptomycin in Tuberculosis Trials Committee, Streptomycin treatment of pulmonary tuberculosis. Br. Med. J. 1948; 2: 769–782.

MULTICENTER TRIAL

5

University Group Diabetes Program (UGDP)

Physicians’ Health Study (PHS)

Period of conduct: 1960–1975 Study population: Adults (males and females) with non-insulin-dependent, adult-onset, diabetes Trial type: Treatment/prevention trial Locale of conduct: United States Sponsor/funding: National Institutes of Health Sample size: 1,027 Test treatments (4): Antidiabetic diet + 1.5 gm tolbutamide/day; diet + 100 mg phenformin/day, diet + 10, 12, 14, or 16 units of insulin lente/day depending on body surface; diet + insulin lente daily dosed to control blood sugar Control treatment: Antidiabetic diet + placebo matched to tolbutamide or phenformin (no placebo injections) Outcome measure: Renal, CV, eye, and peripheral vascular morbidity and all cause mortality Clinics: 12 Coordinating center: University of Minnesota, School of Public Health, through 1963; University of Maryland, School of Medicine, thereafter Method of assignment: Randomized blocks; stratified by clinic and gender; centrally administered Assignment ratio: 1:1:1:1:1 Finding/conclusion: Tolbutamide and phenformin treatments stopped due to safety concerns; insulin treatments no more effective than placebo in prolonging life or reducing morbidity Primary publications: University Group Diabetes Program Research Group, A Study of the effects of hypoglycemic agents on vascular complications in patients with adult-onset diabetes: I. Design, methods and baseline characteristics. Diabetes 1970; 19(Suppl 2): 747–783. University Group Diabetes Program Research Group, A Study of the effects of hypoglycemic agents on vascular complications in patients with adult-onset diabetes: II. Mortality results. Diabetes 1970; 19(Suppl 2): 785–830. University Group Diabetes Program Research Group, A Study of the effects of hypoglycemic agents on vascular complications in patients with adult-onset diabetes: V. Evaluation of Phenformin therapy. Diabetes 1975; 24(Suppl 1): 65–184.

Period of conduct: 1982–1995 Study population: Male physicians, aged 40 to 84, registered in the American Medical Association, free of overt evidence of heart disease Trial type: Prevention trial Locale of conduct: United States Sponsor/funding: National Institutes of Health Sample size: 22,071 Test treatments: Aspirin (325 mg/day) and beta carotene (50 mg/day) Test treatment groups: Aspirin and beta carotene (5,517), aspirin and beta carotene placebo (5,520), aspirin placebo and beta carotene (5,519) Control treatment: Aspirin placebo and beta carotene placebo (5,515) Period of treatment: 46–77 months depending on when enrolled Method of treatment administration and followup: Mail and telephone Primary outcome measure: Fatal and nonfatal MI Study control center: Harvard Medical School Method of assignment: Blocked randomization Finding/conclusion: In regard to aspirin: Regular doses of aspirin were found to significantly reduce the risk of both fatal and nonfatal MI Primary publication: Steering Committee of the Physicians’ Health Study Research Group, Final report of the aspirin component of the ongoing physicians’ health study. NEJM 1989; 321:129–135.

6

MULTICENTER TRIAL

Pediatric AIDS Clinical Trial Group: Protocol 076 (ACTG 076) Period of conduct: 1991–1994 Study population: Pregnant, HIV-infected, women; 14 to 34 weeks gestation on entry Trial type: Prevention trial Locale of conduct: Primarily United States Sponsor/funding: National Institutes of Health Sample size: 477 Test treatment: Zidovudine (100 mg orally 5 times daily plus intrapartum zidovudine (2 mg/kg of body weight given intravenously for 1 hour followed by 1 mg/kg/hr until delivery; plus zidovudine for the newborn (2 mg/kg daily every 5 hours for 6 weeks, beginning 8 to 12 hours after birth Control treatment: Matching placebo Primary outcome measure: HIV infection of newborn Study clinics: 59 Method of assignment: Randomization, stratified by gestation time (2 groups) Finding/conclusion: ‘‘In pregnant women with mildly symptomatic HIV disease and no prior treatment with antiretroviral drugs during the pregnancy, a regimen consisting of zidovudine given ante partum and intra partum to the mother and to the newborn for six weeks reduced the risk of maternal-infant transmission by approximately two thirds’’ Primary publication: E. M. Connor, R. S. Sperling, R. Gelber, P. Kiseley, G. Scott, et al. for the Pediatric AIDS Clinical Trials Group Protocol 076 Study Group, Reduction of maternal-infant transmission of human immunodeficiency virus type 1 with zidovudine treatment. NEJM 1994; 331:1173–1180.

Asymptomatic Carotid Surgery Trial (ACST) Period of conduct: 1993–2003 Study population: Asymptomatic patients with substantial carotid narrowing Trial type: Prevention trial Locale of conduct: 30 different countries Sponsor/funding: Medical Research Council of the United Kingdom Sample size: 3,120 Test treatment: Immediate carotid endarterectomy (CEA) (1,560) Control treatment: Deferral of CEA (1,560) Period of followup: Variable depending on time of randomization; continuous from randomization to common closing date Primary outcome measure: Stroke (fatal or nonfatal) Study clinics: 126 (located in 30 different countries) Coordinating center/data center: Clinical Trial Service Unit, Oxford, United Kingdom Method of assignment: Minimized randomization using age, sex, history of hypertension, and several other variables Finding/conclusion: ‘‘In asymptomatic patients younger than 75 years of age with carotid diameter reduction about 70% or more on ultrasound, immediate CEA halved the net 5-year stroke risk from about 12% to about 6%. Half this benefit involved disabling or fatal stokes.’’ Primary publication: MRC Asymptomatic Carotid Surgery Trial (ACST) Collaborative Group, Prevention of disabling and fatal stroke by successful carotid endarterectomy in patients without recent neurological symptoms: randomised controlled trial. Lancet 2004; 363:1491–1502.

5 ORGANIZATIONAL AND OPERATIONAL FEATURES The key feature of multicenter trials is a differentiated organizational structure involving at least two administratively distinct centers and a defined infrastructure structure serving to bind centers and associated personnel into a cohesive whole. The key elements in the organizational and operational structure of multicenter trials are as follows: Center Director: The scientific head of a study center. Executive Committee (EC): A committee in multicenter studies responsible for

MULTICENTER TRIAL

direction of the day-to-day affairs of the study. One of the key committees in the organizational structure of a multicenter trial. Usually consists of the officers of the study and perhaps others selected from the steering committee and typically headed by the chair or vice-chair of the steering committee and reporting to that committee. Principal Investigator (PI): Broadly, the scientific head of study; generally best avoided in favor of more operationally explicit terminology in multicenter trials because of potential for confusion in favor of terms such as Center Director and Study Chair. Traditionally, in single center trials, the PI is the person in charge of the trial and is usually also the one responsible for enrollment and study of patients in the trial. As a result, the term is often reserved in multicenter trials for persons heading study clinics, leaving those heading the coordinating center or other resource center with some lesser designation implying ‘‘nonprincipal’’ investigatorship. However, even if uniformly applied to all center directors, the term is still best avoided, except in settings where the term is used to refer to a single individual (e.g., the one in investigator-initiated trials who is the recognized head by virtue of initiative in the funding effort). The term should be avoided in egalitarian settings in which, in effect, multiple ‘‘principal’’ investigators exist such as those represented in sponsor or governmentinitiated trials. Research Group: The entire set of personnel involved in the conduct of a research project; in multicenter trials includes center directors and support staff, representatives from the sponsoring agency, and study committee members. syn: investigative team, investigative group, study group (not a recommended syn), study staff (not a recommended syn) Steering Committee (SC): A committee of an organization responsible for directing or guiding the activities of that

7

organization. In multicenter trials, the committee responsible for conduct of the trial and to which other study committees report. Usually headed by the study chair and consisting of persons designated or elected to represent study centers, disciplines, or activities. One of the key committees in multicenter structures. Study Center: An operational unit in the structure of a study, especially a multicenter structure, separate and distinct from other such units in the structure, responsible for performing specified functions in one or more stages of the study (e.g., a clinical center or resource center, such as coordinating center). Study Chair: Chair of the steering committee. Study Officers: The set of persons holding elected or designated offices in a study; in multicenter trials, generally the study chair and vice-chair and the heads or directors of key resource centers, such as the coordinating center, and project office. Treatment Effects Monitoring Committee: A standing committee in the structure of most trials responsible for the periodic review of accumulated data for evidence of adverse or beneficial treatment effects during the trial and for making recommendations for modification of a study treatment, including termination, when appropriate. One of the key committees in the organizational structure of a multicenter trial. Usually constituted such that voting privileges are restricted to members not directly involved in the execution of the trial and not associated with participating centers or sponsors of the trial. syn: Data Monitoring Committee (DMC), Data and Safety Monitoring Committee (DSMC) and Safety Monitoring Committee (SMC); sometimes also Ethical Committee or Ethics Committee but not recommended usage because of likely confusion with institutional review boards (IRBs) and the like.

8

6

MULTICENTER TRIAL

COUNTS

The table below gives counts of all trials, randomized clinical trials, all multicenter clinical trials, and randomized multicenter clinical trials from 1980 forward, as indexed by the National Library of Medicine and counted via PubMed (August 2003). The indexing for multicenter trials was largely nonexistent prior to 1980 and probably spotty in the early 1980s. For example, the UGDP publications, while indexed as ‘‘clinical trial,’’ are not indexed as ‘‘multicenter trial’’. Randomized multicenter trials published since 1980 account for 9% of all randomized trials and 52% of all multicenter trials. About 40% of all randomized multicenter trials are in cancer or cardiovascular disease. The counts of single and multicenter trials (based on indexing by NLM) published in BMJ, Lancet, JAMA, and NEJM in 2002 and the 25th, 50th (median), and 75th percentile points of the sample distributions for the two classes of trials is given in the table below. Not surprisingly, multicenter trials are larger than single center trials. 7

ACKNOWLEDGMENTS

Thanks to Susan Tonascia, Ann Ervin, and Betty Collison for help in producing this piece. REFERENCES 1. C. L. Meinert, Clinical Trials Dictionary: Terminology and Usage Recommendations. Baltimore, MD: The Johns Hopkins Center for Clinical Trials, 1996. 2. S. Day, Dictionary for Clinical Trials. Chichester: Wiley, 1999. 3. International Conference on Harmonisation, E6 Good Clinical Practice. Washington, DC: U.S. Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research (CDER), and the Center for Biologics Evaluation and Research (CBER), April 1996. 4. CASS Principal Investigators and their associates, Coronary Artery Surgery Study (CASS): a randomized trial of coronary artery bypass surgery: comparability of entry characteristics

and survival in randomized patients and nonrandomized patients meeting randomization criteria. J. Am. Coll. Cariol. 1984; 3: 114–128. 5. CASS Principal Investigators and their associates, National Heart, Lung, and Blood Institute Coronary Artery Surgery Study (CASS): a multicenter comparison of the effects of randomized medical and surgical treatment of mildly symptomatic patients with coronary artery disease, and a registry of consecutive patients undergoing coronary angiography. Circulation 1981; 63(monograph 79)(Part II): I1–I-81. 6. Patulin Clinical Trials Committee, Clinical trial of patulin in the common cold. Lancet 1944; 2: 373–375.

FURTHER READING B. G. Greenberg (chair), A Report from the Heart Special Project Committee to the National Advisory Heart Council: Organization, Reviews and Administration of Cooperative Studies, 1967. Controlled Clin. Trials 1988; 9: 137–148. C. R. Klimt, Principles of multi-center clinical studies. In: J. P. Boissel and C. R. Klimt (eds.), Multi-Center Controlled Trials. Principles and Problems. Paris: INSERM, 1979. Cancer Research Campaign Working Party, Trials and tribulations: thoughts on the organization of multicentre clinical studies. Br. Med. J. 1980; 280: 918–920. J. Y. Lee, J. E. Marks, and J. R. Simpson, Recruitment of patients to cooperative group clinical trials. Cancer Clin. Trials 1980; 3: 381–384. J. M. Lachin, J. W. Marks, J. L. Schoenfield, and the NCGS Protocol Committee and the National Cooperative Gallstone Study Group, Design and methodological considerations in the National Cooperative Gallstone Study: a multicenter clinical trial. Controlled Clin. Trials 1981; 2: 177–229. C. L. Meinert, Organization of multicenter clinical trials. Controlled Clin. Trials 1981; 1: 305–312. Coronary Drug Project Research Group, The Coronary Drug Project: methods and lessons of a multicenter clinical trial. Controlled Clin. Trials 1983; 4: 273–541. S. J. Pocock, Clinical Trials: A Practical Approach. New York: Wiley, 1983. D. G. Weiss, W. O. Williford, J. F. Collins, and S. F. Bingham, Planning multicenter clinical trials: a biostatistician’s perspective. Controlled Clin. Trials 1983; 4: 53–64.

MULTICENTER TRIAL

All trials Rz

Total 1980–85 1986–90 1991–95 1996–00 2001 2002 Total

27,806 39,973 65,534 96,667 19,254 18,469 267,703

5,388 11,625 16,777 25,006 5,790 5,761 70,347

19.4 29.1 25.6 25.9 30.1 31.2 26.3

No. of trials Single center Multi-center BMJ Lancet JAMA NEJM

29 62 27 31 149

Multicenter trials Total Rz % Rz

% Rz

35 47 38 54 174

501 3,039 2,901 4,229 961 825 12,456

194 1,091 1,473 2,547 602 525 6,432

38.7 35.9 50.8 60.2 62.6 63.6 51.6

9

RZ MC by disease Cancer CV 25 219 213 334 77 72 940

81 344 428 583 130 103 1,669

Sample size Single center 25% Mdn 132 41 84 41 82

C. L. Meinert and S. Tonascia, Clinical Trials: Design, Conduct, and Analysis. New York: Oxford University Press, 1986. C. L. Meinert, In defense of the corporate authorship for multicenter trials. Controlled Clin. Trials 1993; 14: 255–260. L. M. Friedman, C. D. Furberg, and D. L. Demets, Fundamentals of Clinical Trials, 3rd ed. Multicenter Trials. New York: Springer, 1998, pp. 345–356. B. O’Brien, L. Nichaman, J. E. H. Brouhe, D. L. Levin, P. C. Prorok, and J. K. Gohagan for the PLCO Project Team, Coordination and management of a large multicenter screening trial: the Prostate, Lung, Colorectal and Ovarian (PLCO) Cancer Screening Trial. Controlled Clin. Trials 2000; 21: 310S–328S. Asthma Clinical Research Network, The Asthma Clinical Research Network. Controlled Clin. Trials 2001; 22: 117S–251S.

272 204 230 123 219

75%

25%

Multicenter Mdn

75%

1,090 1,004 1,382 287 775

211 201 481 239 281

540 920 1,182 446 690

1,859 2,309 2,475 1,232 2,037

MULTINATIONAL (GLOBAL) TRIAL

multinational trial it is important to carefully consider such factors as the number of countries and ratios of patients per center and centers per country. Several investigators have discussed some of the practical difficulties associated with multinational trials (1–3). One important difficulty is that medical practice can vary among countries. There might be different diagnostic criteria or differences in medical terminology that lead to a different understanding of what constitutes the disease in question or how a study endpoint is defined. For example, if the study endpoint is hospitalization for the disease in question, there may be substantial differences among countries in the criteria for a hospital admission. In addition, the standard of care for that hospital admission may vary considerably among countries, and the variations in the standard of care may interact with the study treatment, leading to the potential for countryby-treatment interaction. Another aspect of medical practice relates to the choice of the control treatment. When the standard of care for the disease varies from country to country, a decision must be made between choosing a common control treatment across the protocol and choosing the control treatment deemed most relevant within each country. The logistical aspects of providing study supplies for a multinational trial might also be somewhat difficult. Import/export policies for experimental treatments vary from country to country, and they may be time consuming to navigate. Even when the treatments are locally available, the formulations might differ, leading to questions regarding equivalent bioavailability. Another logistical difficulty has to do with the analysis of laboratory samples. Differences in laboratory assays can make a pooled analysis of local laboratory results difficult to interpret. The use of a central laboratory solves this problem, but it may be difficult to ship biologic samples across national borders. The International Conference on Harmonization (ICH) was formed to develop common regulatory standards for clinical research across Europe, North America, and Japan. Despite considerable progress, there

STEVEN M. SNAPINN Amgen, Inc. Thousand Oaks, California

Multicenter trials have long been common in clinical research (see also the article on multicenter trials). The main reasons for conducting a trial at multiple centers is the ability to recruit a large number of patients in a relatively short period of time, and the ability to generalize the results more broadly. Multinational trials are those in which the centers are in multiple countries; Suwelack and Weihrauch (1) define them as ‘‘prospective clinical trials with investigational centers from more than one country with a common protocol foreseeing the combination of clinical data from all centers for a joint evaluation.’’ The reasons for conducting a multinational trial are similar to those for a multicenter trial; however, there are also some special considerations. 1

PRACTICAL ISSUES

The need to recruit a large number of patients is often the main driving force for multinational trials, but there may be particular reasons to recruit patients from specific parts of the world. For example, the disease might be particularly prevalent in specific countries, or in some cases it might be necessary to recruit diseased patients who are treatment-na¨ıve, necessitating selection of countries where access to the treatment is currently limited. It can also be advantageous to demonstrate the safety and efficacy of a therapy under a variety of conditions to gain confidence that the conclusions of the trial are generalizable. For this reason, it might be desirable to conduct the trial in countries with different lifestyles, diets, or cultures. However, it should be recognized that these differences, as well as differences in patient characteristics, may lead to increased variability, which may decrease the power of the trial or complicate the ability to reach a clear conclusion. For this reason, when designing a

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

MULTINATIONAL (GLOBAL) TRIAL

are still several legal and regulatory aspects to consider. For example, different countries may have different requirements for informed consent or for reporting serious adverse events to health authorities. When submitting the study protocol for institutional review board review, different countries may assess potential risks differently and so may require conflicting changes. Cultural differences may also cause some difficulties. When the Case Report Form (CRF) needs to be translated into the local languages, there is always the potential for misunderstandings. This is particularly true for the collection of information on patientreported outcomes, including information on quality of life, where nuances in the wording of questions can have a major effect. Although much information on the CRF can be collected through checkboxes, there is often the need to collect some information as free text, which naturally leads to additional potential for misunderstanding. 2 COUNTRY-BY-TREATMENT INTERACTION Multinational trials may provide broadly generalizable results, but this requires that the results be consistent across patient subsets. However, substantial country-by-treatment interaction has been noted in several clinical trials, leading to difficulties in interpretation. One such study was the Metoprolol CR/XL Randomized Intervention Trial in Congestive Heart Failure (MERIT-HF), described by Wedel et al. (4), which compared the βblocker metoprolol with placebo with respect to total mortality in 3991 patients with heart failure. The trial was stopped early by the steering committee due to a highly significant reduction in total mortality (hazard ratio = 0.66; P = 0.00009). After completion of the trial, the investigators carried out a number of subgroup analyses. Although the overall result was consistent among most subgroups, there was one notable exception: Among the 1071 patients in the United States, the hazard ratio was 1.05. After carefully examining the potential causes of the interaction, the investigators cautioned against overinterpretation of subgroup results and concluded that the best estimate of the treatment

effect for any subgroup, including country subgroups, should be the overall observed effect. O’Shea and DeMets (5) and O’Shea and Califf (6) reviewed other cardiovascular trials that have found country-by-treatment interactions. For example, the Beta-blocker Heart Attack Trial (BHAT) compared propranolol with placebo in 3837 patients who had survived a myocardial infarction; the overall result was highly positive (the mortality rates were 9.8% with placebo and 7.2% with propranolol), but there was wide variation in the size and direction of the effect among study centers. The Flolan International Randomized Survival Trial (FIRST) compared epoprostenol with placebo in patients with heart failure; although mortality was greatly reduced by epoprostenol among patients from Europe, there was no apparent benefit in patients from North America. The Platelet Glycoprotein IIb/IIIa in Unstable Angina: Receptor Suppression Using Integrilin Therapy (PURSUIT) trial compared the platelet inhibitor eptifibatide with placebo in nearly 11,000 patients with an acute coronary syndrome; there was an absolute reduction of 3.3% in the primary endpoint among 3827 patients in North America, but the reductions in other regions ranged only from 1.0% to −1.4%. The Global Utilization of Streptokinase and t-PA for Occluded Coronary Arteries I (GUSTO-I) trial studied the effects of four thrombolytic strategies in 41 thousand patients and found a statistically significant country-by-treatment interaction when comparing the U.S. and non-U.S. patients. Finally, the Global Use of Strategies To Open Occluded Coronary Arteries IV (GUSTO-IV) trial studied the use of abciximab in 7800 patients with an acute coronary syndrome; though there was no overall effect of abciximab, the drug appeared to be beneficial among patients in North America. Although it is not clear if these interactions are due to chance, O’Shea and Califf (7) discuss some of the differences among countries that might lead to interactions like these. Examining the results of over a dozen cardiovascular trials, they found significant differences in patient characteristics. One notable difference was that patients enrolled in the United States were heavier and taller

MULTINATIONAL (GLOBAL) TRIAL

than other patients, and were more likely to have diabetes. They also found important differences in the use of concurrent therapies. For example, compared with patients from other countries, patients in the United States were consistently more likely to be taking a β-blocker. In PURSUIT, women from the United States were considerably more likely to be taking hormone replacement therapy. In addition, patients from the United States in the acute coronary syndrome trials presented to the hospital earlier than patients in other countries. 3

COST EFFECTIVENESS EVALUATION

Some clinical trials include a health economic or cost-effectiveness evaluation (see also the article on cost-effectiveness analysis). Although the situation is conceptually similar to that of the evaluation of clinical effects, the generalizability of costeffectiveness results is more controversial (8–10). Differences in medical practice and patient characteristics have the potential to cause country-by-treatment interaction with regard to clinical effects, but the biologic effect of the treatment is expected to be consistent among countries. With respect to economic data, on the other hand, it is often assumed that systematic differences among countries will prevent pooling the results. This is due to perceived differences in practice patterns, payment systems, and relative prices of resources. Despite the common assumption that economic results will vary among countries, it is reasonable to include an assessment of the degree of similarity. Cook et al. (9) discussed the issues involved in combining economic data from multinational trials, and pointed out that one can use the same tools to assess interaction for clinical and economic endpoints. They concluded that, in the absence of interaction, the pooled estimate of cost effectiveness should be considered representative of the participating countries. Reed et al. (10) summarized a workshop held on this topic and discussed the terminology for multination economic evaluations; this terminology depends on whether pooled

3

or country-specific estimates of clinical efficacy and resource utilization are used. Also concerned by between-country heterogeneity in costs, Pinto et al. (11) proposed univariate and multivariate shrinkage estimators for costs and effects from multinational trials. REFERENCES 1. D. Suwelack and T. R. Weihrauch, Practical issues in design and management of multinational trials. Drug Inf J. 1992; 26: 371–378. 2. H. Maier-Lenz, Implementating multicenter, multinational clinical trials. Drug Inf J. 1993; 27: 1077–1081. 3. H. T. Ho and S. C. Chow, Design and analysis of multinational clinical trials. Drug Inf J. 1998; 32: 1309S–1316S. 4. H. Wedel, D. DeMets, P. Deedwania, B. Fagerberg, S. Goldstein, et al., for the MERIT-HF Study Group. Challenges of subgroup analyses in multinational clinical trials: experiences from the MERIT-HF trial. Am Heart J. 2001; 142: 502–511. 5. J. C. O’Shea and D. L. DeMets, Statistical issues relating to international differences in clinical trials. Am Heart J. 2001; 142: 21–28. 6. J. C. O’Shea and R. M. Califf, International differences in treatment effects in cardiovascular clinical trials. Am Heart J. 2001; 141: 875–880. 7. J. C. O’Shea and R. M. Califf, International differences in cardiovascular clinical trials. Am Heart J. 2001; 141: 866–874. 8. S. D. Sullivan, B. Liljas, M. Buxton, C. J. Lamm, P. O’Byrne, et al., Design and analytic considerations in determining the costeffectiveness of early intervention in asthma from a multinational clinical trial. Control Clin Trials. 2001; 22: 420–437. 9. J. R. Cook, M. Drummond, H. Glick, and J. F. Heyse, Assessing the appropriateness of combining economic data from multinational clinical trials. Stat Med. 2003; 22: 1955–1976. 10. S. D. Reed, K. J. Anstrom, A. Bakhai, A. H. Briggs, R. M. Califf, et al., Conducting economic evaluations alongside multinational clinical trials: toward a research consensus. Am Heart J. 2005; 149: 434–443. 11. E. M. Pinto, A. R. Willan, and B. J. O’Brien, Cost-effectiveness analysis for multinational clinical trials. Stat Med. 2005; 24: 1965–1982.

4

MULTINATIONAL (GLOBAL) TRIAL

CROSS-REFERENCES Case report form Cost-effectiveness analysis Foreign clinical data International Conference on Harmonization (ICH) Multicenter trial

MULTIPLE COMPARISONS

µ1 , . . . ,µk . Fisher’s least significant difference (LSD) procedure (1) declares the pair µi and µj different if the t-statistic that compares µi and µj is significant at level 0.05 and the F-statistic that compares all means is also significant at level 0.05. Fisher’s LSD protects the FWE under the global null hypothesis µ1 = . . . = µk because, to declare at least one pair different, we must reach a P-value of 0.05 or less for the F-statistic, and that has probability 0.05 under the global null hypothesis. On the other hand, suppose that µ1 = . . . = µk−1 = µ, but µk is so much larger than µ that the F-statistic is virtually guaranteed to reach statistical significance. In that case, Fisher’s LSD is tantamount to pairwise comparisons of means at level 0.05. But the FWE for 0.05-level pairwise comparisons of the equal means µ1 , . . . ,µk−1 is clearly larger than 0.05. Therefore, Fisher’s LSD provides weak control but not strong control of the FWE.

MICHAEL A. PROSCHAN National Institute of Allergy and Infectious Diseases Bethesda, Maryland

Multiple comparisons arise in several ways in clinical trials; common examples include multiple arms, multiple endpoints, subgroup analyses, and monitoring over multiple time points. At first, it seems that trying to answer several questions in the same trial is very efficient, especially if another large trial to answer the same questions is unlikely. The problem is that if the comparisonwise error rate (also known as the per-comparison error rate)—the expected proportion of false positives—is 0.05, then the chance of at least one false positive, which is known as the familywise error rate (FWE) or the experimentwise error rate, can be substantially greater than 0.05. In fact, with enough comparisons, the FWE can be close to 1. To control the FWE at level 0.05, one must require stronger evidence for each individual comparison; this is called a multiple comparison adjustment.

2 CRITERIA FOR DECIDING WHETHER ADJUSTMENT IS NECESSARY Whether and how much multiplicity adjustment is needed is controversial. Some have argued that such adjustments are never needed (2), whereas others have used several case studies to argue that without adjusting for all comparisons made in a clinical trial, one cannot be confident of the results (3). The view of many in clinical trials is somewhere between these two positions, namely that adjustment is needed in certain settings but not in others. It is important to note that multiplicity adjustment is needed only when a trial is considered successful when at least one comparison is statistically significant, not when all comparisons must be statistically significant to declare a success. For example, regulatory agencies such as the Food and Drug Administration (FDA) require proof that a combination drug is superior to each of its constituents. If two constituents exist, A and B, then the null hypotheses are H0A : the combination is no better than constituent A, and H0B : the combination is no better than constituent B. To declare success, we must reject

1 STRONG AND WEAK CONTROL OF THE FWE Two different levels of control of the FWE exist. The first level, which is called weak control, means that under the global null hypothesis that each separate null hypothesis is true, the probability of at least one type 1 error is no greater than α. But the global null hypothesis is a very strong assumption; it is more likely that some null hypotheses are true and others are not. We would like the probability of rejecting at least one true null hypothesis to be α or less, where that probability is computed without necessarily assuming that the other null hypotheses are true. If this holds, then the FWE is said to be controlled strongly. To understand why strong control is indeed stronger than weak control, consider a trial with k arms in which we are interested in all pairwise comparisons of means

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

MULTIPLE COMPARISONS

both H0A and H0B ; it is not sufficient to reject at least one of H0A and H0B . If each comparison is made at level 0.05, then the probability of a successful trial—the probability of the intersection of the separate rejection events—will be no greater than 0.05. Therefore, no multiple comparison adjustment is needed in this situation (4). A similar setting is when a drug must show benefit on several different characteristics to be declared beneficial. A helpful way to distinguish whether adjustment is needed is to ask whether a successful trial requires rejection of hypothesis 1 AND hypothesis 2 . . . .AND hypothesis k. If so, then no adjustment is needed. If AND is replaced by OR, then adjustment may be needed. In the sequel, we restrict attention to the latter setting. Proschan and Waclawiw (5), who observed that some settings exist in which nearly everyone agrees that a multiple comparison adjustment is needed, tried to determine what those settings had in common. One factor is the sheer number of comparisons. Situations in which the number of comparisons is often larger than two include interim monitoring and subgroup analyses. Another factor for deciding whether a multiple comparison adjustment is needed is whether one stands to benefit from the multiplicity. For instance, even if investigators have a good reason for changing the primary endpoint in the middle of a trial, their decision will be met with skepticism, especially if no treatment effect existed for the original endpoint but an effect existed for the new primary endpoint. Situations in which one stands to benefit from unadjusted multiple comparisons include monitoring, subgroup analyses, multiple primary endpoints, and comparison of multiple doses of a drug to a placebo. By contrast, no single entity stands to gain from multiple unadjusted comparisons of drugs made by different manufacturers. The most important and difficult factor is the relatedness of the questions being asked and their answers (6, p. 31–35). Clinical trialists tend to feel more compelled to adjust for a family of related (e.g., heart disease and stroke) than unrelated (e.g., cancer and heart disease) hypotheses. Situations in which the questions are related include interim monitoring (in which case the same question is

asked at different times), subgroup analyses, and comparisons of different doses of the same drug to a control. One could debate the relatedness of comparisons of drugs made by different manufacturers to the same control group. One might consider that whether drug A is better than control is a completely separate question than whether drug B is better than control. In fact, the two questions could have been answered in separate trials, in which case no multiple comparison adjustment would have been made (2, 7). But the test statistics in separate trials are statistically independent, so learning that a type 1 error was made in one trial does not erode our confidence in conclusions of other trials. Learning that a type 1 error was made in a trial that compares multiple treatments to the same control does erode our confidence in the results of comparisons of the other treatments with control because it calls into question whether the common control arm was aberrant. Thus, having the same control de facto makes the answers to the questions related (8). In other cases, the questions and answers are more clearly unrelated. Sometimes, two completely different interventions that could have been tested in separate trials are, for efficiency reasons, tested in the same trial using a factorial design. The Women’s Angiographic Vitamin and Estrogen trial (9) in post-menopausal women compared hormone replacement with placebo and vitamins with placebo with respect to diameters of coronary arteries measured using an angiogram. It was considered unlikely that an interactive effect of the two interventions would occur, so a factorial design was used. Unlike comparisons of several arms to the same control, comparisons of main effects in a factorial design with no interaction are asymptotically statistically independent. Therefore, multiple comparison adjustments are often not used for main effect comparisons in a factorial trial (8). Table 1 shows the multiplicity criteria (columns) applied to common clinical trial settings (rows). Darker rows indicate settings that suggest a greater need for adjustment. Monitoring and subgroups are the darkest, and this is consistent with the general feeling that these situations call for some sort of

MULTIPLE COMPARISONS

3

Table 1. Guidelines applied to four common sources of multiplicity in clinical trials Large Number of Comparisons?

Single Entity Benefits?

Related Family?

Multiple Arms Doses vs. same control Different drugs, same control Different drugs, factorial Primary Endpoints Related Unrelated Subgroups Monitoring It usually applies. It may or may not apply, depending on the circumstances. The reader should fill in the gray squares with white or black ones according to the circumstances of the specific application. It usually does not apply.

adjustment (specific adjustment techniques are discussed later). Multiple primary endpoints and multiple doses of a drug compared with the same control are also dark rows, suggesting that separate comparisons at level 0.05 are probably not appropriate. 3 IMPLICIT MULTIPLICITY: TWO TAILED TESTING Before moving to specific adjustment methods in different clinical trial settings, we apply the relatedness criterion to an issue we do not usually associate, although we should, with multiple comparisons—two-tailed testing. Consider, for example, a trial that compares a drug with placebo with respect to mortality. Although the alternative hypothesis for a two-tailed test is that the mortality rate differs between treatment and placebo, we always want to know the direction of the effect. Therefore, we are really trying to answer two separate questions with respect to mortality: (1) Is treatment better than placebo? (2) Is treatment worse than placebo? The two conclusions and their implications could not be more dissimilar, so why do researchers routinely lump them by controlling the probability of any difference? It makes more sense to specify error rates for the two types of errors separately; we may or may not want to use α = 0.025 for each. Consider the Cardiac Arrhythmia Suppression Trial (CAST), which tested the

hypothesis that suppression of cardiac arrhythmias in arrhythmic patients with a previous heart attack would reduce the rate of sudden arrhythmic deaths and cardiac arrests. When it was discovered that two drugs, encainide and flecainide, actually increased the rate of sudden death and cardiac arrest compared with placebo (10), investigators eliminated these two drugs and continued with the third drug, moricizine. The continuation trial was called CAST II (11). Investigators used level 0.025 for declaring superiority and level 0.05 for declaring harm in CAST II, even though the combined type 1 error rate, 0.025 + 0.05 = 0.075, exceeds the conventional level of significance in clinical trials. Because of the asymmetry of harm and benefit coupled with the experience of CAST 1, it made sense to exceed the conventional two-tailed error rate of 0.05 4 SPECIFIC MULTIPLE COMPARISON PROCEDURES We briefly review some common multiplicity adjustment techniques in clinical trials. Readers should consult Miller (6) or Hochberg and Tamhane (12) for a full treatment of multiple comparison methods. 4.1 Multiple Arms Certain multiarmed trials exist in which a multiple comparison adjustment is typically not made. One example is when an arm

4

MULTIPLE COMPARISONS

is added for the purpose of ‘‘validating’’ a trial. For example, the purpose of the Glucosamine/Chondroitin Arthritis Intervention Trial (13) was to determine whether glucosamine and/or chondroitin were superior to placebo for the relief of pain of osteoarthritis of the knee. It was already known that celecoxib relieves pain, so a celecoxib arm was included to validate the trial; if the celecoxib and placebo arms did not differ, then some would question the validity of the trial. Because celecoxib is being included for a very different purpose, no reason exists to lump the comparison of celecoxib to placebo with the comparisons of glucosamine and/or chondroitin to placebo. In the section, ‘‘Criteria for deciding whether adjustment is necessary,’’ we cited another case in which adjustment for multiple comparisons is not typically used—a factorial trial in which an interaction is considered very unlikely, such as when the mechanisms of action and/or outcomes of interest are different. Such a trial is often thought of as two separate trials combined for efficiency reasons only. In many trials, more interest is focused on certain comparisons than on others, as pointed out by Cook and Farewell (14) and Hughes (15). For instance, interest often centers on comparisons with a common control arm. The mere fact that comparisons are made with the same control group was cited in the section on ‘‘Criteria’’ as a reason to consider a multiplicity adjustment, especially if more than two comparisons are made. Dunnett (16) developed a classic method for doing so. Let Ti be the t-statistic that compares arm i to the control, but replace the pooled variance of the two arms by the pooled variance across all k arms and use (k + 1)(n − 1) instead of 2(n − 1) degrees of freedom. Let Pi be the corresponding P-value, i = 1,2, . . . ,k. Treatment i is declared different from the control if Pi ≤ ck , where ck is selected so that P(P1 ≤ ck or P2 ≤ ck or . . . or Pk ≤ ck ) = α. It is easy to see that Dunnett’s method protects the FWE not only under the global null hypothesis, but also when only some means differ from control. Thus, Dunnett’s method provides strong control of the type 1 error rate for comparisons with control.

A more powerful modification of Dunnett’s method that still strongly controls the type 1 error rate is the following step down method (see Example 4.2 of Reference 12). Order the P-values so that P(1) < P(2) < . . . < P(k) . Step 1 tests whether any of the k treatments differ from control by comparing P(1) with ck . If P(1) > ck , then stop and declare no treatment superior to control, whereas if P(1) ≤ ck , then declare the associated treatment different from control and proceed to step 2. If we proceed to step 2, then we either already made a type 1 error or we did not. If we did not, then at most k − 1 type 1 errors are possible, so we ‘‘step down’’ and compare the remaining k − 1 treatments to control. That is, we compare P(2) with ck−1 , where ck−1 is the critical value for P-values that compare k − 1 active arms to control. If we find, after stepping down, that P(2) > ck−1 , then we stop and declare no remaining treatments different from control. On the other hand, if P(2) ≤ ck−1 , then we declare that treatment different from control, and then ‘‘step down’’ again by comparing P(3) to ck−2 , and so on. The step down version of Dunnett’s method clearly has more power than the original, but it has a drawback. Suppose that the k different treatments are made by different companies. If one company’s drug is extremely good, then another company’s drug will not have to meet the same burden of proof to be declared superior to the control. This failure to use a ‘‘level playing field’’ for all comparisons may be objectionable. The section entitled, ‘‘Crtieria for deciding whether adjustment is necessary’’ mentioned another setting that involves comparisons with the same control—that of different doses of the same drug. Table 1 suggests that it is inappropriate to compare each dose to control at level α as the primary analysis. But, often the first question is whether the drug worked, and the dose-response relationship is examined only if the answer is yes. One useful method is a hierarchical procedure whereby one first establishes that the comparison of the active arms combined to the control is significant at level α, then compares each dose to control at level α.

MULTIPLE COMPARISONS

4.2 Multiple Endpoints The multiple endpoint problem is complex because the endpoints might all be primary or a combination of primary and secondary. First consider multiple primary endpoints. As mentioned in the section on Criteria and as suggested by Table 1, if the endpoints are thought to be unrelated, then one could defend not using any adjustment, although most statisticians would feel uneasy if the study had more than two primary endpoints. An attractive method to adjust within a group of similar endpoints is to use a perpatient summary that combines the different endpoints. An example is O’Brien’s rank test (17), which first ranks patients on each of the k outcomes, and then averages ranks across outcomes for a given patient. These per-patient summary measures are then analyzed using a t-test or permutation test. Cook and Farewell (14) note that this method works well when many outcome variables reasonably measure the same phenomenon that treatment is expected to benefit. Follmann (18) showed that if relatively few endpoints exist and treatment has a strong effect on one outcome but not others, then the Bonferroni method, which uses significance level α/k for each endpoint, is attractive. It controls the FWE strongly and is very simple to use, but it is conservative. Bonferroni’s method works well when the number of endpoints is not too large and the test statistics are not highly correlated. For example, Table 2 shows the FWE for two, five, or ten z-statistics with correlation ρ when no adjustment is used (upper panel) and when the Bonferroni adjustment is used (lower panel). The type 1 error rate is substantially inflated with no multiple comparison adjustment, even if ρ is high. The FWE using the Bonferroni adjustment is only slightly less than 0.05 for two uncorrelated z-statistics, which indicates very slight conservatism. As the correlation becomes close to 1, the FWE is about half of 0.05 because the two test statistics are essentially one as the correlation approaches 1. Nonetheless, even for a fairly high correlation of 0.7, the degree of conservatism is not excessive. The conservatism is much more substantial with a greater number of comparisons. For example, with 10

5

z-statistics whose pairwise correlations are all 0.7, the actual type 1 error rate is 0.029 instead of 0.05. More powerful variants of the Bonferroni procedure maintain strong control of the FWE. One variant (19) is a step-down procedure similar to the step-down version of Dunnett’s method described above. If the smallest of the k P-values is less than or equal to α/k, then we declare that endpoint significant and proceed to the second step, whereas if the smallest P-value exceeds α/k, then we stop. If we proceed to the second step, then we compare the second smallest P-value to α/(k − 1). If the second smallest P-value exceeds α/(k − 1), then we stop; if it is less than or equal to α/(k − 1), then we declare that endpoint significant and compare the third smallest P-value to α/(k − 2), and so on. Any time we fail to reach statistical significance, we stop testing, so any larger P-values cannot be declared significant. This procedure, which still strongly controls the type 1 error rate, is more powerful than the classical Bonferroni method, which requires each P-value to be α/k or less to be declared significant. Another Bonferroni variant is Hochberg’s (20) modification of Simes’ (21) method, though as Dmitrienko and Hsu (22) point out, Hochberg’s procedure does not control the FWE in all settings. The Hochberg modification first compares the largest P-value to α; if the largest P-value is α or less, then all endpoints are declared significant. If the largest P-value is larger than α, then the second largest P-value is compared with α/2. If the second largest P-value is α/2 or less, then it and smaller P-values are declared significant. If the second largest P-value exceeds α/2, then the third largest P-value is compared with α/3, and so on. Finally, if all other P-values are larger than their thresholds, then the smallest P-value is declared significant if its P-value is α/k or less. For example, with two endpoints, if both attain P-values of 0.05 or less, then they are both declared significant; if one P-value exceeds 0.05, then the other is still declared significant if its P-value is 0.025 or less. The discussion thus far has not dealt with secondary endpoints. It is difficult to formulate a single strategy that covers all cases

6

MULTIPLE COMPARISONS

Table 2. FWE of unadjusted (upper panel) and Bonferroni-adjusted (lower panel) two-tailed tests for two, five, or ten comparisons using Z-statistics with the same pairwise correlation, ρ FWE with no adjustment ρ k 2 5 10

0 .098 .226 .401

.10 .097 .224 .394

.20 .096 .218 .377

.30 .095 .209 .351

.40 .093 .197 .321

.50 .091 .183 .287

.60 .088 .167 .251

.70 .083 .148 .213

.80 .078 .127 .173

.90 .070 .102 .128

1 .050 .050 .050

.70 .043 .035 .029

.80 .040 .030 .023

.90 .036 .023 .017

1 .025 .010 .005

FWE with Bonferroni adjustment ρ 2 5 10

0 .049 .049 .049

.10 .049 .049 .049

.20 .049 .048 .047

.30 .048 .047 .045

.40 .048 .045 .042

because, as D’Agostino (23) points out, there are many different purposes for secondary endpoints. One purpose might be, after showing that the treatment works, to understand its mechanism of action, in which case a multiplicity adjustment is probably not needed. In other situations, it is less clear how much adjustment is needed. Some people would be content to use level α for each secondary endpoint, arguing that the designation ‘‘secondary outcome’’ instills in the reader the proper amount of caution when interpreting results. Others suggest adjusting for all secondary outcomes or all secondary outcomes plus the primary outcome (24) using the Bonferroni method. The problem is that trials are usually powered for the primary outcome, so even with no multiple comparison adjustment, power for secondary outcomes may be low. To undermine power even more by adjusting for all secondary outcomes makes it very unlikely to reach statistical significance. A reasonable middle ground is to consider any secondary outcome finding suggestive if it reaches level α when no adjustment is made and more definitive if it remains significant after adjustment for multiplicity. 4.3 Subgroup Analyses The subgroup row of Table 1 is completely dark, which suggests that some sort of multiplicity adjustment or at least cautionary language is needed for subgroup conclusions,

.50 .047 .042 .039

.60 .045 .039 .034

especially if the overall effect is not significant. That is what happened in the rpg120 HIV Vaccine Study (25), which was the first Phase 3 trial of an HIV vaccine in humans. No significant difference in HIV acquisition was found between vaccine and placebo arms overall, although an effect seemed to exist in nonwhite participants. One is tempted to conclude that the trial would have reached statistical significance if it had enrolled only nonwhites. The problem is that by chance alone we can often find one or more subgroups in which a benefit of treatment seems to exist, and other subgroups in which no benefit seems to exist. Peto (26) showed that with two equally-sized subgroups constituted completely at random, if the overall effect reaches a P-value of about 0.05, then the probability is about 1/3 that one subgroup will have a statistically significant treatment effect that is more than three times as large as the other subgroup’s treatment effect, which fails to reach statistical significance. Thus, one can be misled by subgroup results even in the simplest possible setting of only two equally sized subgroups. In practice, many subgroups of varying sizes often exist, which compounds the problem. ISIS-2 Investigators (27) illustrated the difficulty in interpreting subgroup effects in a factorial trial of aspirin and streptokinase on mortality. The trial showed a strong benefit of aspirin. In an effort to show how misleading subgroup results can be, investigators

MULTIPLE COMPARISONS

facetiously noted a slightly adverse effect of aspirin in patients born under the astrological signs of Gemini or Libra, but a strong positive effect of aspirin for patients of other astrological signs (P < 0.00001). The ISIS2 example highlights that the magnitude of multiplicity can be larger than it seems. Astrological sign sounds like it should comprise 12 components, but the example above combined two noncontiguous signs, Gemini and Libra. It is hard to know the actual magnitude of multiplicity because if the authors could not demonstrate their point by combining two astrological signs, then they may have tried combining three or more signs. The same sort of phenomenon has arisen from post-hoc combinations of different classes of heart failure, for example. Similarly, if subgroups are formed by somewhat arbitrary cutpoints on a numeric scale, then one may be inclined to experiment with different cutpoints until one is found that highlights subgroup differences. The true extent of multiplicity may be unknowable, in which case it is impossible to separate a real effect from the play of chance. The discussion so far has stressed the need for extreme caution in interpreting subgroup effects, but too much caution will cause us never to find real subgroup effects. After all, even with no adjustment for multiplicity, power in subgroups is often low. For this reason, a widely accepted compromise is first to test whether a subgroup by treatment interaction exist. If the interaction is not statistically significant, then one estimates the treatment effect in the subgroup by the overall treatment effect. If the interaction test is significant, then one reports the treatment effects observed in the different subgroups. It is important to keep in mind that although the treatment effect may differ numerically across subgroups (a quantitative interaction), it is unusual for the treatment to benefit one subgroup and harm another (a qualitative interaction) (26). 4.4 Interim Monitoring The monitoring row of Table 1 is completely dark, which suggests that adjustment is needed for monitoring. A popular and flexible method is to use a spending function

7

α*(t), which dictates the cumulative amount of type 1 error to be spent by time t of the trial, where t is measured in terms of relative information; t = 0 and 1 at the beginning and end of the trial, respectively. Making α*(1) = α ensures that the FWE will be controlled. The boundary at a given interim analysis depends on its time and the times of previous interim analyses, and it is computed using numerical integration. Neither the number nor the timing of interim analyses need be specified in advance. The properties of spending function boundaries depend on the particular spending function selected. Desirable spending functions spend relatively little alpha early, and then make up for that by steeply increasing for t close to 1. This causes early boundaries to be large, and the boundary at the end of the trial to be close to what it would have been without monitoring. A more thorough treatment of spending functions and other monitoring boundaries may be found in References 28 and 29. This article focused mostly on phase III clinical trials. Generally speaking, less emphasis is placed on multiplicity adjustment in earlier phase trials because such trials are used to decide whether more definitive phase III testing is justified rather than to declare the treatment beneficial.

REFERENCES 1. G. W. Snedecor and W. G. Cochran, Statistical Methods, 7th ed. Ames, IA: The Iowa State University Press, 1980. 2. K. Rothman, No adjustments are needed for multiple comparisons. Epidemiology 1990; 1: 43–46. 3. L. A. Moy´e, Multiple Analyses in Clinical Trials. New York: Springer-Verlag, 2003. 4. E. M. Laska and M. J. Meisner, Testing whether an identified treatment is best. Biometrics 1989; 45: 1139–1151. 5. M. A. Proschan and M. A. Waclawiw, Practical guidelines for multiplicity adjustment in clinical trials. Control. Clin. Trials 2000; 21: 527–539. 6. R. G. Miller, Simultaneous Statistical Inference. New York: Springer-Verlag, 1981. 7. D. B. Duncan, Multiple range and Multiple F-tests. Biometrics 1955; 11: 1–42.

8

MULTIPLE COMPARISONS 8. M. Proschan and D. Follmann, Multiple comparisons with control in a single experiment versus separate experiments: why do we feel differently? Am. Stat. 1995; 49: 144–149. 9. D. D. Waters, E. L. Alderman, J. Hsia, et al., Effects of hormone replacement therapy and antioxidant vitamin supplements on coronary atherosclerosis in postmenopausal women: a randomized controlled trial. J. Am. Med. Assoc. 2002; 288: 2432–2440.

10. Cardiac Arrhythmia Suppression Trial Investigators. Preliminary report: effect of encainide and flecainide on mortality in a randomized trial of arrhythmia suppression after myocardial infarction. N. Engl. J. Med. 1989; 321: 406–412. 11. Cardiac Arrhythmia Suppression Trial II Investigators. Effect of the antiarrhythmic agent moricizine on survival after myocardial infarction. N. Engl. J. Med. 1992; 327: 227–233. 12. Y. Hochberg and A. C. Tamhane, Multiple Comparison Procedures. New York: Wiley, 1987. 13. D. O. Clegg, D. J. Reda, C. L. Harris, et al., Glucosamine, chondroitin sulfate, and the two in combination for painful knee osteoarthritis. N. Engl. J. Med. 2006; 354: 795–808. 14. R. J. Cook, and V. T. Farewell, Multiplicity considerations in the design and analysis of clinical trials. J. R. Stat. Soc. A 1996; 159: 93–110. 15. M. D. Hughes, Multiplicity in clinical trials. In: P. Armitage, and T. Colton, eds. Encyclopedia of Biostatistics. New York: Wiley, 1998. 16. C. Dunnett, A multiple comparisons procedure for comparing several treatments with a control. J. Am. Stat. Assoc. 1957; 50: 1096–1121. 17. P. C. O’Brien, Procedures for comparing samples with multiple endpoints. Biometrics 1984; 40: 1079–1087. 18. D. A. Follmann, Multivariate tests for multiple endpoints in clinical trials. Stats. Med. 1995; 14: 1163–1176. 19. S. Holm, A simple sequentially rejective multiple test procedure. Scand. J. Stats. 1979; 6: 65–70. 20. Y. Hochberg, A sharper Bonferroni procedure for multiple tests of significance. Biometrika 1988; 75: 800–802. 21. R. J. Simes, An improved Bonferroni procedure for multiple tests of significance. Biometrika 1986; 73: 751–754. 22. A. Dmitrienko and J. C. Hsu, Multiple testing in clinical trials. In: S. Kotz, N. Balakrishnan,

C. B. Read, B. Vidakovic, and N. Johnson, eds. Encyclopedia of Statistical Sciences. New York: Wiley, 2006. 23. R. B. D’Agostino, Controlling alpha in a clinical trial: the case for secondary endpoints. Stats. Med. 2000; 19: 763–766. 24. C. E. Davis, Secondary endpoints can be validly analyzed, even if the primary endpoint does not provide clear statistical significance. Control. Clin. Trials 1997; 18: 557–560. 25. The rgp120 HIV Vaccine Study Group. Placebo-controlled, phase III trial of a recombinant glycoprotein 120 vaccine to prevent HIV-1 infection. J. Infect. Dis. 2005; 191: 654–665. 26. R. Peto, Clinical trials. In P. Price and K. Sikora, eds. Treatment of Cancer. London: Chapman and Hall, 1995. 27. ISIS-2 Collaborative Group. Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17,187 cases of suspected acute myocardial infarction: ISIS-2. Lancet 1988; 2: 349–360. 28. C. Jennison and B. W. Turnbull, Group Sequential Methods with Applications to Clinical Trials. Boca Raton, FL: Chapman and Hall/CRC Press, 2000. 29. M. A. Proschan, K. K. Lan, and J. T. Wittes, Statistical Monitoring of Clinical Trials: A Unified Approach. New York: Springer, 2006.

MULTIPLE ENDPOINTS

(2) multivariate global tests. Multiple testing approaches, such as the well-known Bonferroni procedure, make individual assessments for each endpoint under investigation. They are particularly advantageous if the endpoints can be classified before the study according to some hierarchy, as this natural ordering can then be explicitly used in the multiple testing approach. In contrast, global tests aim at assessments across all endpoints based on measuring the distance between multivariate populations. These methods include, for example, the classical multivariate analysis of variance (MANOVA) F-tests, which are particularly powerful if treatment differences are expressed through the combined effect of the endpoints. In the following, the main attributes underpinning multiple testing and global testing methodologies are briefly described.

FRANK BRETZ and MICHAEL BRANSON Norvatis Pharma AG Basel, Switzerland

1

INTRODUCTION

A common problem in pharmaceutical research is the comparison of two treatments for more than one outcome measure, which in the clinical context are often referred to as endpoints. A single observation or measurement is often not sufficient to describe a clinically relevant treatment benefit. In respiratory studies, for example, several endpoints (such as FEV1 , respiratory symptoms, and health-related quality of life) are frequently considered to determine a treatment-related benefit. In cardiovascular trials, possible endpoints include time to myocardial infarction, congestive heart failure, stroke, and so on. In such instances, the experimenter is interested in assessing a potential treatment effect while accounting for all multiple outcome measures. The aspects of a highly multidimensional and complex syndrome are usually assessed by means of various symptoms or ordinal items in scales. In order to map these observations on one (or a few) ordinal scale(s) that best represents the severity (or improvement) of the disease, one can try to reduce the dimensionality by combining the univariate projections of this syndrome to a single (or a few) measure(s) of efficacy. This approach is also in agreement with the ICH E9 guideline (1), in which it is recommended to use a single (primary) endpoint, where possible, thus reflecting the need to efficiently reduce the dimensionality. If, nevertheless, certain aspects cannot be combined into one scale or index because they describe inherently different dimensions, or if various measures need to be taken in order to capture the entire benefit range, multiple endpoints are unavoidable. Statistical test procedures for the analysis of multiple endpoints can roughly be classified into (1) multiple testing approaches and

2

MULTIPLE TESTING METHODS

If not stated otherwise, the comparison of two treatments j = 1, 2 with k > 1 endpoints being measured on nj patients are considered. No assumptions are made about the distribution of the measurements in this section. The interest lies in testing the k null hypotheses H i : θ 1i = θ 2i (the two treatments do not differ in effect) against the one-sided alternative hypotheses K i : θ 1i > θ 2i (treatment 1 is better than treatment 2), where θ ji denotes the mean effect of treatment j for endpoint i = 1, . . ., k. Note that the following results are directly extendable to the two-sided testing situation. A main concern when testing multiple hypotheses is the increased likelihood of rejecting incorrectly at least one true null hypothesis [familywise error rate, FWER; see Hochberg and Tamhane (2) for a detailed overview]. In particular, within the regulated environment in which drug development takes place, it is often required to control the FWER at a prespecified level α-frequently, but not restricted to, 2.5% onesided. A standard approach in this context is the Bonferroni procedure. Let pi denote the P-value for the ith hypothesis, Hi , as

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

MULTIPLE ENDPOINTS

obtained from applying an appropriate twosample test. If pi < α/k, H i can be rejected at FWER ≤ α, which follows from Bonferroni’s inequality 
 {pi < α/k} FWER = P i∈T  P(pi < α/k) ≤ α ≤ i∈T

where T denotes the set of all true null hypotheses. Several improvements of the Bonferroni procedure exist. The closure principle (CP) of Marcus et al. (3) is a general and powerful multiple testing procedure, which includes many of these improvements as special cases. For simplicity, its use is illustrated for k = 3 endpoints (see Fig. 1). Starting from the set of hypotheses of interest H 1 , H 2 , H 3 , all possible intersection hypotheses H12 , H 23 , H 13 , H 123 are created, where Hij = Hi ∩ Hj , 1 ≤ i, j ≤ 3, are the intersection hypotheses of two hypotheses at a time. For example, H23 states that endpoints 2 and 3 do not differ for both treatments. The overall intersection hypothesis H 123 is the global null hypothesis of no treatment effect across all three endpoints. According to the CP, a hypothesis H i is rejected at FWER α if Hi itself and all hypotheses formed by intersection with H i are each rejected at (local) level α. For example, in order to reject H1 in Fig. 1, one has to reject H 1 itself as well as H12 , H 13 , H 123 , where the choice of the individual level-α tests is free. Application of the Bonferroni approach to each hypothesis, for example, leads to the stepwise procedure of Holm (4): Let p(1) ≤ . . . ≤ p(k) denote the ordered unadjusted P-values with the associated hypotheses H (1) , . . ., H (k) . Then, H (i) is rejected if p(j) < α/(k − j + 1), j = 1, . . ., i (i.e., if all hypotheses H(j) preceding H (i) are also rejected). As a result of its stepwise nature, Holm’s procedure is more powerful than the Bonferroni procedure. A second important application of the CP is the allocation of the FWER α to a preordered sets of hypotheses (5). Chi (6), for example, advocated the definition of relevant decision paths before conducting the clinical trial so that the inherent multiplicity problem caused by multiple endpoints is reduced

by sequentially testing different sets of endpoints in a clinically relevant and meaningful way. It is, therefore, common practice in clinical trials to classify the endpoints into primary, co-primary, and secondary endpoints (6) and to test them in a prespecified order of importance, thus reflecting the need to efficiently reduce the multiplicity. Different strategies of allocating the FWER α between the sets of hypotheses are possible (6, 7), most of which can ultimately be derived by applying the CP. Fixed sequence methods test the individual endpoints in a prespecified order, each at level α, where the nonrejection of a hypothesis at any step stops the test procedure (5). The fixed sequence approach is only applicable, when a complete hierarchy of the endpoints is available and a hypothesis is only of interest and thus tested if the higher prioritized hypotheses have all been rejected beforehand (8). Gatekeeping methods first test a preliminary hypothesis and, in case of a rejection, continue testing additional hypotheses (9). This approach may be required if a single primary endpoint is to be tested for significance before further secondary endpoints are to be analyzed. More general gatekeeping procedures are introduced in (10). An important point to note is that of requiring statistical significance for all primary (multiple) endpoints before being able to claim ‘‘success.’’ In this context, no formal adjustment of the FWER is necessary (i.e., each endpoint is tested separately at level α) (8). What should be carefully evaluated and understood is the impact on the power for such a hypothesis testing strategy. Other multiplicity adjustments exist that improve on Bonferroni’s inequality, in particular the Simes test (11) and its stepwise extension (12). All of the methods considered so far, however, do not account for the inherent correlation between different endpoints. Exploiting the association between the endpoints can considerably improve the methods above. Under the standard ANOVA assumptions and if the correlations are known (or at least well approximated), the endpointspecific t-tests are jointly multivariate t distributed (13) and efficient integration routines can be used for the design and analysis

MULTIPLE ENDPOINTS

3

H123

Figure 1. CP hypothesis tree for three endpoints

H12

H13

H23

H1

H2

H3

of related clinical trials (14). If the correlations are unknown (which is the typical case in practice), a standard alternative is resampling the data to approximate the joint distribution of the test statistics (15, 16). Different resampling schemes are available, each of which has its own advantages. The key idea is to permute, a large number of times, the entire patient-specific observation vectors such that the complete information for a single patient, across the endpoints, is maintained. Such procedure retains the (stochastic) dependencies between the endpoints under the null hypothesis when no real differences exist among the treatments. For each resampling step, the test statistics are computed based on the resampled data. Then, the originally observed test statistics are compared with the resampled test statistics via the resampling distribution, where extreme values indicate a significant treatment effect. The resampling methods are available for a variety of multiple comparison procedures, in particular for the CP based on t-statistics (15). In addition, extensions to the comparison of multiple treatments are possible and software is readily available such as PROC MULTTEST in SAS.

3

MULTIVARIATE GLOBAL TESTS

The classic approach for comparing two multivariate normal populations with an unknown common covariance matrix is Hotelling’s T 2 test (17)

T2 =

n1 n2 (Y1 − Y2 ) S−1 (Y1 − Y2 ) n1 + n2

where Yj denotes the vector of sample means,  nj S = j l=1 (Yjl − Yj )(Yjl − Yj ) /ν denotes the sample covariance matrix with ν = n1 + n2 − 2, and Yjl denotes the observation vector for patient l under treatment j = 1, 2. Under the null hypothesis of no treatment effect across all endpoints, (n1 + n2 − k − 1)T 2 is k(n1 + n2 − 2)Fν1 , ν2 distributed with ν 1 = k and ν 2 = n1 + n2 − k − 1 degrees of freedom. Note that T 2 can be regarded as a multivariate generalization of the twosample t test to the multivariate setting. The test statistic T 2 is the squared maximum of univariate t test statistics of all linear combinations of the endpoints. In addition, the T 2 test has several optimality properties. Among others, it is uniformly the most powerful of all tests invariant to scale transformations (18). Several extensions to situations with more than two treatments exist, the most notable being Wilks’ , Pillai’s trace, Hotelling-Lawley trace, and Roy’s maximum root, all of which are approximately F distributed (18). All of the afore-mentioned global tests were developed for the two-sided alternative and thus lack power in practical applications involving one-sided directional decisions. A variety of one-sided tests in the multivariate setting exists, although many problems remain unsolved [see Tamhane and Logan (19) for a recent review of these methods]. Kudˆo (20) derived the exact likelihood ratio (LR) test for the test problem H: θ = 0 vs. K: θ ≥ 0 with at least one θ i > 0, when the covariance structure is known. Perlman (21) extended the LR test to situations with an unknown covariance structure. However, the exact distribution of the LR test is not free

4

MULTIPLE ENDPOINTS

of the unknown nuisance parameters. Moreover, the LR test is biased, so that research to circumvent these difficulties is still ongoing (22, 23). O’Brien (24) proposed a different solution by restricting the alternative space K, where the standardized treatment differences (θ 1i − θ 2i )/σ i , i = 1, . . ., k, are assumed to be all positive and of the same magnitude. Using ordinary least square (OLS) and generalized least square (GLS) methods, O’Brien (24) showed that the resulting LS statistics are standardized weighted sums of the individual −1 t statistics ti = (Y 1i − Y 2i )/ s2i (n−1 1 + n2 ),

where Y ji , j = 1, 2, and s21 denote, respectively, the mean response and pooled sample variance of endpoint i = 1, . . ., k. More  specifi ˆ cally, the OLS statistic is tOLS = i ti / 1
R1, ˆ where R denotes the pooled sample correlation matrix. Alternatively, the GLS method  ˆ −1 t/ 1
R ˆ −1 1 weights the t statistGLS = 1
R ˆ where t is the vector of t tics according to R, statistics from above. Note that by construction, tGLS may include negative weights for certain correlation structures so that it is possible to reject H in favor of positive treatment effects when, in fact, the opposite is true (25). As the exact distribution of tOLS and tGLS is unknown, approximations have to be used. O’Brien (24) proposed the use of a t distribution with ν(= n1 + n2 − 2) degrees of freedom. For small sample sizes, the LS tests can be either conservative or liberal, depending on the parameter constellations. Improved small sample approximations can be found in Logan and Tamhane (26). For large sample sizes, the limiting standard normal distribution can be used. ¨ ¨ In contrast, Lauter (27) and Lauter et al. (28) derived exact one-sided level-α tests without restricting the multivariate alternative region. They gave conditions under which choosing data-dependent weight vectors w, lead to the test statistics   wi ti n1 n2 tw =  i n1 + n2 w
Rw ˆ being t distributed with ν degrees of freedom under H. Let Y i denote the total variablewise mean over both samples. A common choice is then to set wi = νii−1 , where

 nj νii = 2j=1 l=1 (Yijl − Y i )2 is the ith diagonal element of the total product matrix, leading to the standardized score (SS) test. Logan and Tamhane (26) compared the power of the OLS and the SS tests. They found out by analytical methods that (1) if a single endpoint exists with a positive effect, the OLS test is more powerful than the SS test; and (2) if all endpoints have a positive effect with the same magnitude (which is the underlying assumption of the LS tests), both tests are equally powerful. Moreover, the authors conducted a simulation study for a variety of scenarios, showing that both the OLS test and the SS test behave similar, in terms of power, throughout most of the alternative region. Alternatively to the SS test, the first principal component calculated from the total product matrix can be used, leading to the PC test (27, 28). The PC test has higher power than the OLS test over a wide range of the alternative, such as, for example, if some variables have an effect and others not.

4 CONCLUSIONS In this article, some of the existing methods to analyze multiple endpoints have briefly been reviewed. The main application of these methods are clinical trials focusing on relatively few efficacy endpoints, where a strong control of the type I error rate is mandatory. It is important to reiterate that the multiple testing methods, as discussed above, are P-value based approaches, providing a high degree of flexibility in tailoring the testing strategy to suit the particular application. Such procedures are not restricted to particular data types, and are applicable for the analysis of, for example, normal, binary, count, ordinal, and time-to-event endpoints. An interesting and evolving method in this context is the use of the CP in adaptive designs, which allows the user to select and confirm endpoints within a single trial (29, 30). A further approach is to apply multivariate tests to the intersection hypotheses of the CP (31). Such a hybrid approach thus combines the advantages of multiple testing methods (assessment of the individual hypotheses) and multivariate global test

MULTIPLE ENDPOINTS

(taking the multivariate nature of data into account.) Other applications may require different statistical methods than reviewed in this article. Longitudinal data, for example, may include (nonlinear) mixed-effects models to describe the stochastic dependencies between the time points (32). The analysis of multivariate time-to-event outcomes has been the topic of much research, and standard implementations based on a counting-process formulation (33, 34) can be easily implemented in standard statistical software (35). Safety (adverse event) data may require analysis using novel multiple testing techniques based on controlling the false discovery rate (36, 37) or by, for example, using hierarchical Bayes models as proposed by Berry and Berry (38). Finally, high-dimensional screening studies with thousands of endpoints, such as gene expression profiling, for example, often use completely different statistical tools again (39, 40). REFERENCES 1. ICH Tripartite Guideline E9. (1998). International conference on harmonization; guidance on statistical principles for clinical trials. (online). Available: http://www.fda.gov/80/cder/guidance/. 2. Y. Hochberg and A. C. Tamhane, Multiple Comparison Procedures. New York: Wiley, 1987. 3. R. Marcus, E. Peritz, and K. R. Gabriel, On closed testing procedures with special reference to ordered analysis of variance. Biometrika 1976; 63: 655–660. 4. S. Holm, A simple sequentially rejective multiple test procedure. Scand. J. Statist. 1979; 6: 65–70. 5. W. Maurer, L. A. Hothorn, and W. Lehmacher, Multiple comparisons in drug clinical trials and preclinical assays: a-priori ordered hypotheses. In: J. Vollman (ed.), Biometrie in der Chemische-Pharmazeutichen Industrie, vol. 6. Stuttgart, Germany: Fischer Verlag, 1995. 6. G. Chi, Multiple testings: Multiple comparisons and multiple endpoints. Drug Inform. J. 1998; 32: 1347S–1362S. 7. P. H. Westfall and A. Krishen, Optimally weighted, fixed sequence and gatekeeper multiple testing procedures. J. Stat. Plan. Infer. 2001; 99: 25–40.

5

8. CPMP Points to Consider. (2002). PtC on multiplicity issues in clinical trials. CPMP/EWP/908/99. (online). Available: http://www.emea.eu.int/pdfs/human/ewp/ 090899en.pdf. 9. P. Bauer, J. R¨ohmel, W. Maurer, and L. A. Hothorn, Testing strategies in multi-dose experiments including active control. Stat. Med. 1998; 17: 2133–2146. 10. A. Dmitrienko, W. W. Offen, and P. H. Westfall, Gatekeeping strategies for clinical trials that do not require all primary effects to be significant. Stat. Med. 2003; 22: 2387–2400. 11. R. J. Simes, An improved Bonferroni procedure for multiple tests of significance. Biometrika 1986; 73: 751–754. 12. Y. A. Hochberg, Sharper Bonferroni procedure for multiple tests of significance. Biometrika 1988; 75: 800–802. 13. F. Bretz, A. Genz, and L. A. Hothorn, On the numerical availability of multiple comparison procedures. Biometric. J. 2001; 43: 645–656. 14. A. Genz and F. Bretz, Comparison of methods for the computation of multivariate probabilities. J. Comput. Graph. Stat. 2002; 11: 950–971. 15. P. H. Westfall and S. S. Young, Resamplingbased multiple testing. New York: Wiley, 1993. 16. J. F. Troendle, A permutational step-up method of testing multiple outcomes. Biometrics 1996; 52: 846–859. 17. H. Hotelling, The generalization of Student’s ratio. Ann. Math. Stat. 1931; 2: 360–378. 18. T. W. Anderson, An Introduction to Multivariate Statistical Analysis, 3rd ed. New York: Wiley, 2003. 19. A. C. Tamhane and B. R. Logan, Multiple endpoints: an overview and new developments. Technical Report, 43. Milwaukee, WI: Division of Biostatistics, Medical College of Wisconsin, 2003. 20. A. Kudˆo, A multivariate analogue of the oneside test. Biometrika 1963; 15: 403–418. 21. M. D. Perlman, One-sided testing problems in multivariate analysis. Ann. Stat. 1969; 40: 549–567. 22. M. D. Perlman and L. Wu The Emperor’s New Tests (with discussion). Statistical Science 1999; 14: 355–381. 23. M. S. Srivastava, Methods of multivariate statistics. New York: Wiley, 2002. 24. P. C. O’Brien, Procedures for comparing samples with multiple endpoints. Biometrics 1984; 40: 1079–1089.

6

MULTIPLE ENDPOINTS

25. S. J. Pocock, N. L. Geller, and A. A. Tsiatis, The analysis of multiple endpoints in clinical trials. Biometrics 1987; 43: 487–498. 26. B. R. Logan and A. C. Tamhane, On O’Brien’s OLS and GLS tests for multiple endpoints. In: Y. Benjamini, F. Bretz, and S. Sarkar (eds.), New Developments in Multiple Comparison Procedures. IMS Lecture Notes - Monograph Series 47. 2004, pp. 76–88. ¨ 27. J. Lauter, Exact and F test for analyzing studies with multiple endpoints. Biometrics 1996; 52: 964–970. ¨ 28. J. Lauter, E. Glimm, and S. Kropf, Multivariate tests based on left-spherically distributed linear scores. Ann. Stat. 1998; 26: 1972–1988.

33. R. L. Prentice, B. J. Williams, and A. V. Peterson, On the regression analysis of multivariate failure time data. Biometrika 1981; 68: 373–379. 34. L. J. Wei, D. Y. Lin, and L. Weissfeld, Regression analysis of multivariate incomplete failure time data by modeling marginal distributions. J. Amer. Stat. Assoc. 1993; 84: 1065–1073. 35. T. M. Therneau and P. M. Grambsch, Modeling survival data: Extending the Cox Model. New York: Springer, 2000. 36. Y. Benjamini and Y. Hochberg, Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. Royal Stat. Soc. B 1995; 57: 289–300.

29. M. Kieser, P. Bauer, and W. Lehmacher, Inference on multiple endpoints in clinical trials with adaptive interim analyses. Biometric. J. 1999; 41: 261–277. 30. G. Hommel, Adaptive modifications of hypotheses after an interim analysis. Biometric. J. 2001; 43: 581–589. 31. W. Lehmacher, G. Wassmer, and P. Reitmeir. Procedures for two-sample comparisons with multiple endpoints controlling the experimentwise error rate. Biometrics 1991; 47: 511–521.

37. D. V. Mehrotra and J. F. Heyse, Use of the false discovery rate for evaluating clinical safety data. Stat. Meth. Med. Res. 2004; 13: 227–238 38. S. M. Berry and D. A. Berry, Accounting for multiplicities in assessing drug safety: a three-level hierarchical mixture model. Biometrics 2004; 60: 418–426. 39. T. Speed, Statistical Analysis of Gene Expression Microarray Data. Boca Raton, FL: CRC Press, 2003.

32. J. Pinheiro and D. Bates, Mixed-Effects Models in S and S-PLUS. New York: Springer, 2000.

40. G. Parmigiani, E. S. Garett, R. A. Irizarry, and S. L. Zeger, The Analysis of Gene Expression Data. New York: Springer, 2003.

MULTIPLE EVALUATORS

and measure agreement? What is the overall level of agreement? How much bias and variance is there among evaluators? In short, the goal of an agreement study is two-fold. The first goal is to determine if the measurements from multiple evaluators agree with each other. If not, then the second goal is to identify where the difference(s) occur and correct/calibrate them if possible. The agreement problem covers a broad range of data, and examples can be found in different disciplines. The scale of a measurement can be continuous, binary, nominal, or ordinal. The rest of this article is organized as follows. In the next section, the approaches for assessing agreement of continuous data are reviewed. We classify these approaches into three categories. The first category is the hypothesis testing-type approach; the second category is an index approach; and the third category is the interval-type approach. In the section entitled, ‘‘Agreement for categorical data,’’ the approaches for assessing agreement of categorical data are reviewed. A summary is provided in the last section.

JASON J. Z. LIAO and ROBERT C. CAPEN Merck Research Laboratories, West Point, Pennsylvania

1

INTRODUCTION

In medical and other related sciences, clinical or experimental measurements usually serve as a basis for diagnostic, prognostic, therapeutic, and performance evaluations. The measurement can be from multiple systems, processes, instruments, methods, raters, and so forth, but for the sake of simplicity, we refer to them as ‘‘evaluators’’ throughout this article. As technology continues to advance, new methods/instruments for diagnostic, prognostic, therapeutic, and performance evaluations become available. Before a new method or a new instrument is adopted for use in measuring a variable of interest, its agreement relative to other similar evaluators needs to be assessed. Measurements of agreement are needed to assess the reliability of multiple raters (or the same rater over time) in a randomized clinical trial (RCT). These measurements of agreement can be used for assessing the reliability of the inclusion criteria for entry into an RCT, validating surrogate endpoints in a study, determining that the important outcome measurements are interchangeable among the evaluators, and so on. An agreement study involving multiple evaluators can happen in all phases of drug development or other medical-related experimental settings. A typical design for an agreement study involves sampling n subjects from the population, sampling d evaluators from a population of evaluators, and obtaining r replicate determinations/evaluator. Each evaluator is ‘‘blinded’’ to all others to ensure independence of measurement error; the measurements represent the range of ratings proposed to be used interchangeably in the RCT. Various questions regarding agreement can be posed, such as the listings by Bartko (1). Can the measurements from evaluators be used interchangeably? How does one define

2

AGREEMENT FOR CONTINUOUS DATA

Let Y ij be the continuous measurement made by the j-th evaluator on the i-th subject (i = 1, . . . , n, and j = 1, . . . , d). The goal is to assess the agreement among the measurements made by the d evaluators. In the case of d = 2, the paired measurements of a perfect agreement from the two evaluators fall exactly on the identity line Y 2 = Y 1 , for example, the 45◦ line through the origin. The existing approaches can be classified into three categories. An appropriate transformation, such as logarithm, of the data may be recommended and used to better meet the assumptions under a specified approach. Therefore, Y ij are the final chosen reportable values used in the agreement evaluations. 2.1 Hypothesis Testing Approach The first approach of this type is the paired t-test to compare the mean value of two measurements for d = 2, and the F-test to compare the mean value of d(d > 2) mea-

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

MULTIPLE EVALUATORS

surements (1). The second approach in this category is a hypothesis of the parameters from, for example, different types of regression settings (2–9). All these hypothesis-type approaches heavily depend on the residual variance, which can reject a reasonably good agreement when the residual errors are small (more precision) but fail to reject a poor agreement when the residual errors are large (less precision) (10). Other critiques of these approaches can be found in Reference 11. Thus, any kind of approach based on a hypothesis testing strategy is not recommended for assessing agreement. 2.2 An Index Approach Two commonly used approaches comprise this category: the intraclass correlation coefficient (ICC) and the concordance correlation coefficient (CCC). Fleiss (12) gave an overview of ICC as a measure of agreement among multiple evaluators. In defining ICC, a useful two-way random effects model is given by Yij = µ + si + rj + eij , i = 1, . . . , n, and j = 1, . . . , d,

(1)

where µ is a fixed parameter and where si , rj , and eij are independent random effects with mean 0 and variances σs2 , σr2 , and σe2 , respectively. Usually d = 2. The term si is the subject effect, rj is the systematic bias (i.e., the systematic error), and eij is the measurement error. To assess the agreement among multiple evaluators by using the concept of intraclass correlation coefficient, the index is defined as ρu =

σs2 σs2 + σr2 + σe2

(2)

In Equation (1), the denominator of ρ u is the variance of Y ij (unconditionally on j), and the numerator of ρ u is the covariance between Y ij1 and Y ij2 (unconditionally on j1 = j2). Thus, ICC defined in Equation (2) is the correlation between any two arbitrary measurements made on the same subject.

Another measuring index is the ICC defined as follows: ρc =

σs2

σs2 + σe2

(3)

ρ c in Equation (3) does not take systematic error into account in assessing agreement under Equation (1). Contrasting this with the ICC defined in Equation (2), the denominator of ρ c in Equation (3) is the variance of Y ij conditionally on j. Thus, ρ c is the correlation between Y ij1 and Y ij2 conditionally on j1 = j2, which is the usual product–moment correlation between two specific measurements made on the same subject. Note that ρ u = ρ c if we assume a model without systematic error, that is σr2 = 0 in Equation (1), which is also referred to as the one-way random effects model (13). The one-way random model is as follows: Yij = µ + si + eij , i = 1, . . . , n, and j = 1, . . . , d,

(4)

Again, ICC from Equation (4) is the simple correlation between Y ij1 and Y ij2 . Therefore, we can estimate the ICCs using the Pearson correlation coefficient. However, the estimation of ICCs based on applying analysis of variance (ANOVA) is the most commonly used and recommended approach (13). Among the various methods proposed for calculating the confidence intervals of the ICCs, the one proposed by Cappelleri and Ting (14) almost exactly maintains the nominal coverage probability for typical agreement studies (15). As pointed out by Rousson et al. (16), the main difference between these two ICCs is that the value of ρ u is attenuated when systematic error exists, whereas ρ c is not. Another difference is that ρ u does not depend on the ordering of the d measurements, whereas ρ c does. For assessing interevaluator agreement, Rousson et al. (16) recommended using ρ u because a systematic bias clearly indicates a failure in agreement and should be taken into account. In contrast, they recommended using ρ c for test–retest agreement because the first and the second

MULTIPLE EVALUATORS

trial in this situation are clearly not interchangeable. In assessing agreement for two evaluators, Lin (10) claimed that we should divide the assessment into two steps. First, a precision step assesses how close the observations are to the regression line. Second, an accuracy step assesses how close the regression line and the target line (the line of identity) are to each other. Using squared differences under the assumption that the observations are from a bivariate normal distribution and µ2 , and covariance matrix with mean µ1 
σ12 ρσ1 σ2 , Lin (10) proposed a concorρσ1 σ2 σ22 dance correlation coefficient (CCC) for d = 2 (denoted as CCC = ρCa ) as follows: CCC = 1 −

E(Y2 − Y1 )2 E[(Y2 − Y1 )2 |ρ = 0]

= ρCa =

2ρσ1 σ2 σ12 + σ22 + (µ1 − µ2 )2

(5)

This index combines the measure of precision (ρ) and measure of accuracy 
Ca =

2σ1 σ2 σ12 +σ22 +(µ1 −µ2 )2

together. Lin (10) esti-

mated his CCC by replacing the parameters with the corresponding samples values and recommended using the Fisher  Z- transformation Z(r) = tanh−1 (r) = 12 ln 1+r 1−r for inference. However, some concerns for using CCC exist (17–20). Note that the CCC is a special case of the formula for ICC (19, 21). Therefore, two improved CCCs have been proposed in the literature (22, 23). Many critiques argue against using the index approaches. In assessing agreement, all indices assumed observations from a distribution (usually normal) with a fixed mean and constant covariance. However, the fixed mean assumption is often not satisfied in practice (24–26). In addition, usually a single index is not good enough to measure the degree of agreement (27). When an index indicates a poor agreement, no indication shows what is wrong. When poor agreement occurs, it is important to determine the degree of bias (fixed and/or proportional biases). No current agreement index can provide this information. As pointed out by Bland and Altman (11, 28), a correlation method is very sensitive

3

to the range of the measurements available in the sample. The greater this range, the higher the correlation. In addition, it is not related to the actual scale of measurement or to the size of error that might be clinically allowable. Related to this point, Liao and Lewis (20) gave examples where nested experiments lead to conflicted conclusions. Because any index approach is very sensitive to sample heterogeneity, therefore, Atkinson and Nevill (18) suggested no index approach should be used to assess agreement. 2.3 An Interval Approach The commonly used approach in this category was proposed for d = 2 evaluators by Bland and Altman (28). Let Di , Y i2 − Y i1 , i = 1, . . . , n. Assuming the Di s are approximately normally distributed, Bland and Altman (11, 28) proposed using the 95% interval of the observed differences [D − 2SD , D + 2SD ]

(6)

which they called ‘‘limits of agreement’’ to n  measure agreement, where D = n1 Di and

S2D

=

1 n−1

n 

i=1

(Di −

D)2 .

These limits are

i=1

then compared with scientifically acceptable boundaries. The interval in Equation (6) will ensure that 95% of all differences will fall into it under the assumption of normality. This method does not depend on the range of the sample, and the limits of agreement give some indication whether the discrepancy is acceptable in practice by comparing the limits with a scientifically justifiable boundary. As a supplement, they also proposed a mean–difference graphic that plots the difference Di against the mean of the two measurements M i = (Y i1 + Y i2 )/2 along with the 95% limits of the difference, for example, the limits of agreement. They claimed that this graphical tool could be used to investigate the assumptions and possible trends in systematic error (bias) and/or in measurement error, thus, leading to a possible transformation of the responses being compared. These trends can be tested using an appropriate technique, such as the Spearman rank correlation between |Di | and M i .

4

MULTIPLE EVALUATORS

The Bland and Altman approach is a favorite of medical researchers with over 11,000 citations. However, some concerns about this method exist. First, it creates a problem of interpretation when a mixture of fixed, proportional bias and/or proportional error occurs (29). Second, it is only good for the test–retest situation (16). Third, the level and type of bias between the two sets of measurements cannot be fully assessed, and covariate information cannot be incorporated into this approach (25, 26). Fourth, the mean–difference plot gives artifactual bias information in measurements differing only in random error (30). A new approach that overcomes these concerns was proposed in References 25 and 26. 3

AGREEMENT FOR CATEGORICAL DATA

rect for such marginal disagreement (35), it is preferable that it be investigated as a potential source of interevaluator difference (36). Existing approaches for categorical data generally focus on assessing agreement through the calculation of a kappa-like index or by modeling the pattern of agreement in the data. 3.1 Measuring Agreement between Two Evaluators 3.1.1 Kappa. Suppose two evaluators are to classify n subjects into m mutually exclusive and exhaustive categories. Following Banerjee et al. (32), let pkk be the proportion of subjects placed into category k(k = 1, 2, . . . , m) by both evaluators and define m  pkk as the observed proportion of p0 = k=1

Since the development of the kappa coefficient (31), a considerable amount of research has been performed in the area of interevaluator agreement for categorical data (see, for example, Reference 32). Cohen originally developed kappa as a measure of chancecorrected agreement between two evaluators for nominal ratings (two or more ratings that cannot be logically ordered, such as positive/negative, schizophrenic/bi-polar/ neurotic/depressed, etc.). However, kappa itself is affected by the prevalence of the illness (33), an issue that will be discussed in more detail in the section entitled, ‘‘Issues in kappa.’’ Understanding the underlying assumptions behind the methods as well as their limitations is critical when analyzing categorical agreement data. For example, in the development of kappa, Cohen assumed independence between the evaluators. When quantifying the level of agreement, this assumption might be questionable when the evaluations from one clinician are known in advance by a second clinician, or if the evaluations are done over time, without blinding, by the same clinician. A well-publicized shortcoming of kappa is its inability to differentiate between two components of disagreement (34). Although it is possible to cor-

agreement. A portion of this overall agreement will be due to chance alone. Call this portion pc . Kappa is defined as the achieved agreement beyond chance relative to the potential agreement beyond chance (33). p0 − pc (7) κ= 1 − pc Cohen (31) defined chance agreement in a natural way as pc =

m 

pk· p·k

(8)

k=1

where pk· is the proportion of subjects placed into the kth category by the first evaluator and p·k is the proportion of subjects placed into the kth category by the second evaluator. When disagreements between evaluators are not equally weighted, the use of kappa is problematic. 3.1.2 Weighted Kappa. Cohen (37) generalized the definition of kappa to include situations when disagreements are not equally weighted. Let njk be the number of subjects in the (j, k)th cell of an m × m table. The total m  number of subjects is n = njk . Define j, k = 1

MULTIPLE EVALUATORS

wjk to be the weight corresponding to the (j, k)th cell. Then, weighted kappa, which measures the proportion of weighted agreement corrected for chance (32), is defined as κw =

p0w − pcw 1 − pcw

(9)

where p0w =

m m  1  wjk njk = wjk pjk n j, k = 1

pcw =

1 n2

m  j, k = 1

j, k = 1

wjk nj· n·k =

m 

wjk pj· p·k (10)

j, k = 1

Three common choices for the weights are provided in Equation (11). Weights can also be selected based on disagreement (37).  wjk =

1, j = k (Unweighted) 0, j =  k

wjk = 1 −

|j − k| (Linear Weights) (m − 1)2

wjk = 1 −

(j − k)2 (Quadratic Weights) (m − 1)2 (11)

Hypothesis testing and confidence interval construction follow similarly to the unweighted case (38–40), although a large number of subjects are generally required to construct confidence intervals for weighted kappa even for a modest number of categories (32). Note that for 2 × 2 tables, use of either linear or quadratic weights is equivalent to calculating unweighted kappa. For more than two categories, each weighing scheme assigns less weight to agreement as the categories get farther apart. The greatest disparity in ratings is assigned an agreement weight of 0. In general, for more than two categories the use of quadratic weights is common (33). 3.1.3 Agreement Measures for Ordinal Data. In many settings, the potential ratings that can be assigned to a subject form a natural ordering. Such ordinal scales often arise because a truly continuous variable is discretized. For example, the degree of

5

motion of a particular joint, which could be measured, might instead be categorized as ‘‘unrestricted,’’ ‘‘slightly restricted,’’ ‘‘moderately restricted,’’ and ‘‘highly restricted’’ (33). Although the kappa statistics mentioned above can, in theory, be extended to ordinal data, doing so will generally result in a loss of statistical power (41, 42) and could lead to misleading results (43). Various alternative methods to kappa exist for assessing agreement for ordered categories (44). The approach developed by Rothery (45) is a nonparametric analog to the ICC and has a straightforward interpretation (44). 3.2 Extensions to Kappa and Other Approaches for Modeling Patterns of Agreement 3.2.1 Extensions to Kappa. Since the development of kappa more than 45 years ago, most research has focused on the case of two evaluators using a binary scale to classify a random sample of n subjects. Both Banerjee et al. (32) and Kraemer et al. (43) provide nice discussions on various extensions to kappa. We refer the reader to their work for more details. Extension to the case of multiple evaluators was treated in Reference 46. Fleiss assumed that each of n subjects was rated by an equal number (d > 2) of evaluators into one of m mutually exclusive and exhaustive nominal categories, where, for each subject, the set of evaluators was randomly selected from a ‘‘population’’ of evaluators. Davies and Fleiss (47) developed a chance-corrected statistic for measuring agreement assuming, among other things, that a common set of evaluators rated all subjects. Kraemer (48) allowed for a different number of ratings per subject as well as for the same evaluator to place a subject into more than one category. By considering the placement of each observation in the rank ordering of the m categories, Kraemer could derive a kappa-like statistic as a function of the Spearman rank correlation coefficient. Kraemer et al. (43) warned that the use of kappa when more than two nominal categories exist is suspect regardless of the number of evaluators. 3.2.2 Modeling Patterns of Agreement. Up to this point, the focus has been centered on

6

MULTIPLE EVALUATORS

assessing agreement through the calculation of a single index. Alternatively, one can model the pattern of agreement in the data. Various sophisticated strategies have been developed for doing this modeling, and a discussion of them is beyond the scope of this article. The most common approaches include the use of log-linear models (49–51), latent-class models (50, 52–56), and generalized estimating equations (57–60). 3.3 Issues with Kappa Although the use of kappa statistics is widespread, appearing in the psychology, education, physical therapy, medical, and psychiatry literature (to name a few), its usefulness as a metric to gauge interevaluator agreement is not universally accepted (61). A thorough discussion of what kappa is designed to do is provided in Kraemer et al. (43). In this section, we will describe some more common issues surrounding the use of kappa. It is not meant to be an exhaustive list. 1. Trait prevalence. When the prevalence of a disease or trait is rare, the kappa statistic can be misleading. Feinstein and Cicchetti (62) discussed this paradox and ways to resolve it. Sim and Wright (33) defined a prevalence index in the context of a 2 × 2 table as |p11 − p22 |. When the prevalence index is high, chance agreement is very likely and kappa will be attenuated as a result. 2. Evaluator bias. Bias measures the extent that two evaluators disagree on the proportion of ‘‘positive’’ (or ‘‘negative’’) cases (33). This bias is different than bias relative to a ‘‘gold standard.’’ However, a small level of bias, as determined by the bias index of Sim and Wright (33), does not imply a large value for kappa. In fact, any time the proportion of subjects in the (1,2) cell is the same as the proportion of subjects in the (2,1) cell, their bias index will be 0. A calculation of both the bias and prevalence index should accompany any calculation of kappa. 3. Kappa does not distinguish among types and sources of disagreement. By placing less weight on agreement for

categories that are more separated, this issue can be, to some degree, overcome. Still, different weighting schemes will lead to different values for kappa, so using any type of ad hoc rule for determining the strength of agreement (see, for example, Reference 63) is inappropriate. 4. Nonindependent ratings. An important assumption in the development of kappa is that evaluators generate ratings independently of one another. The independence assumption is often satisfied in practice by appropriate care in the design of the agreement study. Dependent ratings will generally lead to inflated values of kappa. Two typical settings where the independence assumption is questionable are where (1) one evaluator is aware of the ratings of the other evaluator and (2) the same evaluator rates the same set of subjects, without blinding, on two different occasions. In the latter setting, Streiner and Norman (64) suggest that a time interval of 2–14 days is reasonable but depends on the attribute being evaluated. 5. Ordinal data. Calculating kappa on data that are continuous but have been categorized is a practice that should be avoided. It is much better to preserve the original scale and apply measures appropriate for continuous data to assess agreement (see the section entitled ‘‘Agreement for continuous data’’). When it is impossible to obtain the continuous measure or when the data are ordinal in their original form, the weighted form of kappa is often used to assess agreement. This approach is questionable because it is generally not as powerful as other measures of agreement specifically developed for ordinal data (e.g., see Reference 45). 4 SUMMARY AND DISCUSSION In this article, we have reviewed approaches for assessing measurement agreement among multiple evaluators. For continuous data, the index approach and the interval approach are

MULTIPLE EVALUATORS

commonly used for assessing the agreement. Because of its simplicity and its intuitive and practical interpretability, we particularly recommend using the interval approach instead. For the common situation of two evaluators and two nominal categories, we primarily discussed kappa and weighted kappa statistics and mentioned several issues surrounding their usefulness to assess agreement. Trying to describe the degree of agreement among two or more evaluators by a single index is overly optimistic. Even for the simplest case of two evaluators and binary categories, besides reporting the value for a kappa statistic, one should report the proportion of overall agreement and the proportions of agreement specific to each category. For multiple evaluators, the strategy for two evaluators can be adopted for pairs of evaluators. If the underlying attribute is continuous, one can use log-linear models, latent class models, or generalized estimating equations to investigate, among other things, interevaluator agreement. A similar strategy can be used for the case of nominal or ordinal scale data, although for ordinal data, one should avoid the use of kappa. The design of an agreement study is very important. At the very least, design aspects that must be considered include the procedures for selecting the subjects and the evaluators. The population from which the subjects are selected must be well–defined, and the evaluators must be well trained/validated. We advocate collecting r replicate evaluations for each subject by each evaluator. If the r replicate evaluations are to be completed over time, then the time interval must be chosen to minimize the possibility of correlated ratings while still ensuring that the attribute being observed has not meaningfully changed in the population or sample. A consequence of violating the latter requirement is that the ratings would become time-dependent (i.e., not interchangeable). Other important issues regarding the general design of agreement studies are discussed in References 11, 13, 44, 65, and 66. For the particular issue of sample size, see References 15, 67, and 68, for continuous data, and see References 33, 69, and 70 for categorical data.

5

7

ACKNOWLEDGMENTS

The authors thank Dr. Christy Chuang-Stein and two referees for their valuable comments and suggestions that improved this article. REFERENCES 1. J. J. Bartko, Measurements of agreement: a single procedure. Stat. Med. 1994; 13: 737–745. 2. W. E. Deming, Statistical Adjustment of Data. New York: Wiley, 1943. 3. U. Feldmann, B. Schneider, H. Klinkers, and R. Haeckel, A multivariate approach for the biometric comparison of analytical methods in clinical chemistry. J. Clin. Chem. Clin. Biochem. 1981; 19: 121–137. 4. U. Feldmann and B. Schneider, Bivariate structural regression analysis: a tool for the comparison of analytic methods. Methods Informat. Med. 1987; 6: 205–214. 5. H. Passing and W. A. Bablok, A new biometrical procedure for testing the equality of measurements from two different analytical methods. J. Clin. Chem. Clin. Biochem. 1983; 21: 709–720. 6. E. L. Bradley and L. G. Blackwood, Comparing paired data: a simultaneous test of means and variances. Am. Stat. 1989; 43: 234–235. 7. K. Linnet, Estimation of the linear relationship between the measurements of two methods with proportional errors. Stat. Med. 1990; 9: 1463–1473. 8. B. Zhong and J. Shao, Testing the agreement of two quantitative assays in individual means. Commun. Stat. Theory Meth. 2002; 31: 1283–1299. 9. B. Zhong and J. Shao, Evaluating the agreement of two quantitative assays with repeated measurements. J. Biopharmaceut. Stat. 2003; 13: 75–86. 10. L. I-K. Lin, A concordance correlation coefficient to evaluate reproducibility. Biometrics 1989; 45: 255–268. 11. J. M. Bland and D. G. Altman, Statistical methods for assessing agreement between two methods of clinical measurement. Stat. Methods Med. Res. 1999; 8: 135–160. 12. J. L. Fleiss, The Design and Analysis of Clinical Experiments. New York: John Wiley & Sons, 1986. 13. N. J.-M. Blackman, Reproducibility of clinical data I: continuous outcomes. Pharmaceut. Stat. 2004a; 3: 99–108.

8

MULTIPLE EVALUATORS

14. J. C. Cappelleri and N. Ting, A modified large sample approach to approximate interval estimation for a particular intraclass correlation coefficient. Stat. Med. 2003; 22: 1861–1877. 15. Y. Saito, T. Sozu, C. Hamada, and I. Yoshimura, Effective number of subjects and number of raters for inter-rater reliability studies. Stat. Med. 2006; 25: 1547–1560. 16. V. Rousson, T. Gasser, and B. Seifert, Assessing intrarater, interrater and test-retest reliability of continuous measurements. Stat. Med. 2002; 21: 3431–3446. 17. R. Muller and P. Buttner, A critical discussion of intraclass correlation coefficients. Stat. Med. 1994; 13: 2465–2476. 18. A. Atkinson and A. Nevill, Comment on the use of concordance correlation to assess the agreement between two variables. Biometrics 1997; 53: 775–777. 19. C. A. E. Nickerson, A note on ‘a concordance correlation coefficient to evaluate reproducibility.’ Biometrics 1997; 53: 1503–1507. 20. J. J. Z. Liao and J. Lewis, A note on concordance correlation coefficient. PDA J. Pharmaceut. Sci. Technol. 2000; 54: 23–26. 21. J. L. Carrasco and L. Jover, Estimating the generalized concordance correlation coefficient through variance components. Biometrics 2003; 59: 849–858. 22. J. J. Z. Liao, An improved concordance correlation coefficient. Pharmaceut. Stat. 2003; 2: 253–261. 23. M. P. Fay, Random marginal agreement coefficients: rethinking the adjustment for chance when measuring agreement. Biostatistics 2005; 6: 171–180. 24. J. J. Z. Liao, Agreement for curved data. J. Biopharmaceut. Stat. 2005; 15: 195–203. 25. J. J. Z. Liao, R. C. Capen, and T. L. Schofield, Assessing the reproducibility of an analytical method. J. Chromat. Sci. 2006a; 44: 119–122. 26. J. J. Z. Liao, R. C. Capen, and T. L. Schofield, Assessing the concordance of two measurement methods. ASA Proceedings on Section of Biometrics CD, 2006b. 660–667. 27. R. A. Deyo, P. Diehr, and D. L. Patrick, Reproducibility and responsiveness of health status measures: statistics and strategies for evaluation. Control. Clin. Trial 1991; 12: 142S–158S. 28. J. M. Bland and D. G. Altman, Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 2: 307–310.

29. J. Ludbrook, Comparing methods of measurement. Clin. Exp. Pharmacol. Physiol. 1997; 24: 193–203. 30. W. G. Hopkins, Bias in Bland-Altman but not regression validity analyses. Sportscience 2004; 8: 42–46. 31. J. Cohen, A coefficient of agreement for nominal scales. Educ. Psychol. Meas. 1960; 20: 37–46. 32. M. Banerjee, M. Capozzoli, L. McSweeney, D. Sinha, Beyond kappa: a review of interrater agreement measures. Canadian J. Stat. 1999; 27: 3–23. 33. J. Sim and C. C. Wright, The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys. Ther. 2005; 85: 257–268. 34. T. P. Hutchinson, Kappa muddles together two sources of disagreement: tetrachoric correlation is preferable. Res. Nurs. Health 1993; 16: 313–315. 35. R. Zwick, Another look at interrater agreement. Psychol. Bull. 1988; 103: 374–378. 36. J. Cohen, Weighted kappa: nominal scale agreement with provisions for scaled disagreement or partial credit. Psychol. Bull. 1968; 70: 213–220. 37. J. L. Fleiss, J. Cohen, and B. S. Everitt, Large sample standard errors of kappa and weighted kappa. Psychol. Bull. 1969; 72: 323–327. 38. D. A. Bloch and H. C. Kraemer, 2 × 2 Kappa coefficients: measures of agreement or association. Biometrics 1989; 45: 269–287. 39. D. V. Cicchetti and J. L. Fleiss, Comparison of the null distributions of weighted kappa and the C ordinal statistic. Appl. Psychol. Meas. 1977; 1: 195–201. 40. J. L. Fleiss and D. V. Cicchetti, Inference about weighted kappa in the non-null case. Appl. Psychol. Meas. 1978; 2: 113–117. 41. A. Donner and M. Eliasziw, Statistical implications of the choice between a dichotomous or continuous trait in studies of interobserver agreement. Biometrics 1994; 50: 550–555. 42. E. Bartfay and A. Donner, The effect of collapsing multinomial data when assessing agreement. Internat. J. Epidemiol. 2000; 29: 1070–1075. 43. H. C. Kraemer, V. S. Periyakoil, and A. Noda, Kappa coefficient in medical research. Stat. Med. 2002; 21: 2109–2129. 44. N. J.-M. Blackman, Reproducibility of clinical data II: categorical outcomes. Pharmaceut. Stat. 2004b; 3: 109–122.

MULTIPLE EVALUATORS 45. P. Rothery, A nonparametric measure of intraclass correlation. Biometrika 1979; 66: 629–639. 46. J. L. Fleiss, Measuring nominal scale agreement among many raters. Psychol. Bull. 1971; 76: 378–382. 47. M. Davies and J. L. Fleiss, Measuring agreement for multinomial data. Biometrics 1982; 38: 1047–1051. 48. H. C. Kraemer, Extension of the kappa coefficient. Biometrics 1980; 36: 207–216. 49. M. A. Tanner and M. A. Young, Modeling agreement among raters. J. Am. Stat. Assoc. 1985; 80: 175–180. 50. A. Agresti, Modelling patterns of agreement and disagreement. Stat. Methods Med. Res. 1992; 1: 201–218. 51. P. Graham, Modelling covariate effects in observer agreement studies: the case of nominal scale agreement. Stat. Med. 1995; 14: 299–310. 52. W. R. Dillon and N. Mulani, A probabilistic latent class model for assessing inter-judge reliability. Multivar. Behav. Res. 1984; 19: 438–458. 53. M. Aickin, Maximum likelihood estimation of agreement in the constant predictive model, and its relation to Cohen’s kappa. Biometrics 1990; 46: 293–302. 54. J. S. Uebersax, and W. M. Grove, Latent class analysis of diagnostic agreement. Stat. Med. 1990; 9: 559–572. 55. Y. Qu, M. Tan, and M. H. Kutner, Random effects models in latent class analysis for evaluating accuracy of diagnostic tests. Biometrics 1996; 52: 797–810. 56. S. L. Hui and X. H. Zhou, Evaluation of diagnostic tests without gold standards. Stat. Methods Med. Res. 1998; 7: 354–370. 57. S. R. Lipsitz, and G. M. Fitzmaurice, Estimating equations for measures of association between repeated binary responses. Biometrics 1996; 52: 903–912. 58. J. M. Williamson, A. K. Manatunga, and S. R. Lipsitz, Modeling kappa for measuring dependent categorical agreement data. Biostatistics 2000; 1: 191–202. 59. N. Klar, S. R. Lipsitz, and J. G. Ibrahim, An estimating equations approach for modelling kappa. Biomet. J. 2000; 42: 45–58. 60. E. Gonin, S. R. Lipsitz, G. M. Fitzmaurice, and G. Molenberghs, Regression modelling of weighted κ by using generalized estimating equations. Appl. Stat. 2000; 49: 1–18.

9

61. J. S. Uebersax, Diversity of decision-making models and the measurement of interrater agreement. Psychol. Bull. 1987; 101: 140–146. 62. A. R. Feinstein and D. V. Cicchetti, High agreement but low kappa I: the problems of two paradoxes. J. Clin. Epidemiol. 1990; 43: 543–548. 63. J. R. Landis and G. G. Koch, The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159–174. 64. D. L. Streiner and G. R. Norman, Health Measurement Scales: A Practical Guide to their Development and Use, 3rd ed. Oxford, UK: Oxford University Press, 2003. 65. H. C. Kraemer, Evaluating Medical Tests. Objective and Quantitative Guidelines. Newbury Park, CA: Sage, 1992. 66. P. Graham and R. Jackson, The analysis of ordinal agreement data: beyond weighted kappa. J. Clin. Epidemiol. 1993; 9: 1055–1062. 67. D. G. Bonett, Sample size requirements for estimating intraclass correlations with desired precision. Stat. Med. 2002; 21: 1331–1335. 68. J. J. Z. Liao, Sample size calculation and concordance assessment for an agreement study. ENAR presentation, 2004. 69. A. Donner, Sample size requirements for the comparison of two or more coefficients of interobserver agreement. Stat. Med. 1997; 15: 1157–1168. 70. S. D. Walter, M. Eliasziw, and A. Donner, Sample size and optimal designs for reliability studies. Stat. Med. 1998; 17: 101–110.

CROSS-REFERENCES Interrater reliability Reproducibility Kappa statistic weighted Kappa

MULTIPLE RISK FACTOR INTERVENTION TRIAL (MRFIT)

1 TRIAL DESIGN 1.1 Overview The objective of MRFIT was to test whether a multifactor intervention would result in lower CHD mortality among men, baseline ages 35–57 years, at higher risk for CHD as measured by three established major risk factors: blood pressure (BP), serum cholesterol, and cigarette smoking. Eligible highrisk men were randomized to either the ‘‘Special Intervention’’ (SI) group, which targeted cessation of smoking, lowering serum cholesterol, and lowering BP, or to the ‘‘Usual Care’’ (UC) group; the latter were referred to their personal physicians or a community care clinic and did not receive any study intervention.

LYNN E. EBERLY School of Public Health, University of Minnesota, Minneapolis, Minnesota

JEREMIAH STAMLER Feinberg School of Medicine, Northwestern University, Chicago, Illinois

LEWIS H. KULLER School of Public Health, University of Pittsburgh, Pittsburgh, Pennsylvania

JAMES D. NEATON School of Public Health, University of Minnesota, Minneapolis, Minnesota

1.2 Design of the Multifactor Intervention for the SI Group

The Multiple Risk Factor Intervention Trial (MRFIT) was a nationwide randomized trial on primary prevention of coronary heart disease (CHD) death, sponsored by the National Heart, Lung, and Blood Institute within the United States National Institutes of Health, and it was conducted during 1972–1982. The study cohort consisted of men (baseline ages 35–57 years) at higher risk for CHD death, but with no clinical evidence of cardiovascular disease (CVD); women were not included because of substantially lower CHD rates. The intervention tested was multifactor: intensive counseling for smoking cessation, dietary advice particularly to lower serum cholesterol, and stepped-care pharmacologic treatment for hypertension (primarily with diuretics). The primary outcome was CHD mortality, with an observation period of 6 years minimum and 7 years on average (1,2). Since the end of active follow-up in 1982, the 361,662 men who were screened during MRFIT recruitment, which includes the 12,866 who were randomized, have been followed for mortality date and cause through Social Security Administration and National Death Index searches (3,4).

1.2.1 Cigarette Smoking. Each cigarette smoker was counseled individually by a MRFIT physician immediately after randomization. Dosage reduction (low tar, low nicotine cigarettes) was not recommended (5,6). 1.2.2 Nutrition. The nutrition intervention did not stipulate a structured diet, but instead it aimed to encourage lifelong shopping, cooking, and eating patterns concordant with saturated fat intake < 10% of calories, dietary cholesterol < 300 mg/day, polyunsaturated fat ∼ 10% of calories, with a fare moderate in total fat and calories to facilitate prevention and control of obesity and elevated BP (7). 1.2.3 Hypertension. Antihypertensive drugs were prescribed using a stepped-care protocol beginning with the use of an oral diuretic (either hydrochlorothiazide or chlorthalidone); other drugs were sequentially added if the BP goal (either a 10 mm Hg reduction in diastolic BP, or diastolic BP no more than 89 mm Hg) was not reached. Before drug prescription, weight loss was attempted for overweight men (8).

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

MULTIPLE RISK FACTOR INTERVENTION TRIAL (MRFIT)

1.2.4 Simultaneous Efforts. Shortly after randomization, each man was invited (with spouse or friend) to a 10-week series of weekly group discussions covering all three risk factors; common educational materials were used at all sites. After this initial phase, individual counseling by an intervention team (behavioral scientist, nutritionist, nurse, physician, and general health counselor) and measurement of risk factors was provided every 4 months; specific risk factor goals were set for each person. Men could be examined at more frequent intervals based on their changes in risk factors and intervention team recommendations (1,9). 1.3 Sample Size and Power Calculations The MRFIT design stipulated recruitment of at least 12,000 men ages 35 to 57 years; the men had to be at elevated risk of CHD mortality but without clinical signs of CVD at baseline. The randomized sample size for MRFIT of 12,866 was estimated to provide 88% power to detect a 26.6% reduction in risk of CHD death over 6 years (29.0/1000 men in UC compared with 21.3/1000 men in SI) using a one-sided test of proportions and a type I error of 5%. Behind this calculation were several steps with key assumptions about level of CHD risk in the UC group and anticipated intervention effects. First, a multiple logistic regression model for 6-year CHD death—as a function of serum cholesterol, diastolic BP, and cigarettes smoked/day—was developed using Framingham Heart Study data. Second, variables that represented the screening levels of diastolic BP, serum cholesterol, and cigarettes/day for the MRFIT randomized men were entered into the logistic risk model to project the 6-year CHD death rate of 29.0/1000 men in UC. Third, a reduction of the CHD death rate to 21.3/1000 men in SI was projected based on the following assumptions about reductions of screening risk factor levels for SI men: (1) a 10% reduction in serum cholesterol for those at a level of 220 mg/dL or more (no reduction for others); (2) a 10% reduction in diastolic BP for those at a level of 95 mm Hg or more (no reduction for others); (3) a 25% reduction for smokers of 40 or more cigarettes/day, a 40% reduction for

smokers of 20–39/day, and a 55% reduction for smokers of µP . According to the UI testing principle, the global hypothesis H0I is tested by examining each of its components individually, rejecting H0I if at least one of the components is rejected. Tests of homogeneity that one learns in elementary statistics courses, such as the F-test, tend to be UI tests. The following is a brief overview of popular methods for constructing UI tests. 3.1 Single-Step Tests Based on Univariate P-Values ˇ ak ´ These tests (e.g., the Bonferroni and Sid tests) are intuitive, easy to explain to non-statisticians and, for this reason, are frequently used in clinical applications. The Bonferroni adjustment for testing H0i amounts to computing an adjusted P-value ˇ ak-adjusted ´ given by kpi . Similarly, the Sid P-value for H0i is equal to 1 − (1 − pi )k . The adjusted P-values are then compared with α and the global hypothesis H0I is rejected if at least one adjusted P-value is no greater than α. Another example of a test based on univariate P-values is the Simes test (8). The adjusted Simes P-value for the global hypothesis H0I is k min(p[1] , p[2] /2, . . . , p[k] /k), where p[1] , . . . , p[k] are ordered P-values (i.e., p[1] ≤ . . . ≤ p[k] ). It is easy to see from this definition that the Simes test is uniformly more powerful than the Bonferroni test in the sense that the former rejects H0I every time the latter does. Although the Simes test has a power advantage over the Bonferroni test, one needs to remember that the Simes test does not always preserve the overall Type I error rate. It is known that the size of this test does not exceed α when p1 , . . . , pk are independent or positively dependent (9). It is important to keep in mind that tests based

3

on univariate P-values ignore the underlying correlation structure and become very conservative when the test statistics are highly correlated or the number of null hypotheses is large (e.g., in clinical trials with multiple outcome variables). 3.2 Parametric Single-Step Tests The power of simple tests based on univariate P-values can be improved considerably when one can model the joint distribution of the test statistics T1 , . . . , Tk . Consider, for example, the problem of comparing k doses of an experimental drug with a control in a one-sided manner. Assuming that T1 , . . . , Tk follow a multivariate normal distribution and larger treatment differences are better, Dunnett (10) derived a multiple test that rejects H0i if Ti ≥ d, where d is the 100(1 − α)% percentile of max(T1 , . . . , Tk ). Dunnett’s method also yields a set of simultaneous one-sided confidence intervals for the true mean treatment differences δ1 , . . . , δk :  δi > δˆi − ds 2/n,

i = 1, . . . , k

where s is the pooled sample standard deviation and n is the common sample size per treatment group. 3.3 Resampling-Based Single-Step Tests A general method for improving the performance of tests based on univariate P-values was proposed by Westfall and Young (3). Note first that the adjusted P-value for H0i is given by P{min(P1 , . . . , Pk ) ≤ pi }. In this equation, P1 , . . . , Pk denote random variables that follow the same distribution as p1 , . . . , pk under the assumption that the global hypothesis H0I is true. The joint distribution of the P-values is unknown and can be estimated using permutation or bootstrap resampling. The advantage of using resampling-based testing procedures is that they account for the empirical correlation structure of the individual P-values and, thus, are more powerful than the Bonferroni and similar tests. Furthermore, unlike the Dunnett test, the resampling-based approach does not rely on distributional assumptions. When carrying out resampling-based tests, it is important

4

MULTIPLE TESTING IN CLINICAL TRIALS

to ensure that the subset pivotality condition is met. This condition guarantees that the resampling-based approach preserves the FWER at the nominal level. The subset pivotality condition is met in most multiple testing problems for which pivotal quantities exist; however, it may not be satisfied in the case of binary variables, for example; see Reference 3 for more details. 4

CLOSED TESTING

A cornerstone of multiple hypotheses testing has been the closed testing principle of Marcus et al. (11). The principle has provided a foundation for a variety of multiple testing methods and has found a large number of applications in multiple testing problems occurring in clinical trials. Examples of such applications include procedures for multiple treatment comparisons and multiple outcome variables (12, 13), testing a dose-response relationship in dose ranging trials (14), and gatekeeping strategies for addressing multiplicity issues developing in clinical trials with multiple primary and secondary endpoints (15, 16). The closed testing principle is based on a hierarchical representation of the multiplicity problem in question. To illustrate, consider the null hypotheses HL and HH from the dose-finding trial example. In order to derive a closed test for this multiple testing problem, construct the closed family of null hypotheses by forming all possible intersections of the null hypotheses. The closed family contains HL , HH , and HL ∩ HH . The next step is to establish implication relationships in the closed family. A hypothesis that contains another hypothesis is said to imply it; for example, HL ∩ HH implies both H L and HH . The closed testing principle states that an FWER-controlling testing procedure can be constructed by testing each hypothesis in the closed family using a suitable level α test. A hypothesis in the closed family is rejected if its associated test and all tests associated with hypotheses implying it are significant. For example, applying the closed testing principle to the dose-finding trial example, statistical inference proceeds as follows:

• If HL ∩ HH is accepted, the closed test

has to accept HL and HH because HL ∩ HH implies HL and HH . • If HL ∩ HH is rejected, but not HL or HH , the inference is at least one of the two alternative hypotheses is true, but which one cannot be specified. • If HL ∩ HH and HH are rejected but HL is accepted, one concludes that HH is false (i.e., µH > µP ). Similarly, if HL ∩ HH and HL are rejected but HH is accepted, the null hypothesis HL is declared to be false (i.e., µL > µP ). • Lastly, if HL ∩ HH , HL and HH are rejected, the inference is that µL > µP and µH > µP . Now, in order to construct a multiple testing procedure, one needs to choose a level α significance test for the individual hypotheses in the closed family. Suppose, for example, that the individual hypotheses are tested using the Bonferroni test. The resulting closed testing procedure is equivalent to the stepwise testing procedure proposed by Holm (17). The Holm procedure relies on a sequentially rejective algorithm for testing the ordered null hypotheses H[01] , . . . , H[0k] corresponding to the ordered P-values p[1] ≤ . . . ≤ p[k] . The procedure first examines the null hypothesis associated with the most significant P-value (i.e., H[01] ). This hypothesis is rejected if p[1] ≤ α/k. Further, H[0i] is rejected if p[j] ≤ α/(k − j + 1) for all j = 1, . . . , i. Otherwise, the remaining null hypotheses H[0i] , . . . , H[0k] are accepted and testing ceases. Note that H[01] is tested at the α/k level and the other null hypotheses are tested at successively higher significance levels. As a result, the Holm procedure rejects at least as many (and possibly more) null hypotheses as the Bonferroni test from which it was derived. This example shows that, by applying the closed testing principle to a single-step test, one can construct a more powerful stepwise test that maintains the FWER at the same level. The same approach can be adopted to construct stepwise testing procedures based on other single-step tests. For example, the popular Hochberg and Hommel testing procedures can be thought of as closed testing

MULTIPLE TESTING IN CLINICAL TRIALS

versions of the Simes test (18, 19). It is worth noting that the Hommel procedure is uniformly more powerful than the Hochberg procedure, and both of the two procedures preserve the FWER at the nominal level only when the Simes test does (i.e., under the assumption of independence or positive dependence). In the parametric case, an application of the closed testing principle to the Dunnett test results in the stepwise Dunnett test defined as follows. Consider again the comparison of k doses of an experimental drug with a control in a one-sided setting. Let T[1] , . . . , T[k] denote the ordered test statistics (T[1] ≤ . . . ≤ T[k] ) and di be the 100(1 − α)% percentile of max(T1 , . . . , Ti ), i = 1, . . . , k. The stepwise Dunnett test begins with the most significant statistic and compares it with dk . If T[k] ≥ dk , the null hypothesis corresponding to T[k] is rejected and the second most significant statistic is examined. Otherwise, the stepwise algorithm terminates and the remaining null hypotheses are accepted. It is easy to show that the derived stepwise test is uniformly more powerful than the single-step Dunnett test. An important limitation of the closed testing principle is that it does not generally provide the statistician with a tool for constructing simultaneous confidence intervals for parameters of interest. For instance, it is not clear how to set up simultaneous confidence bounds for the mean differences between the k dose groups and control group within the closed testing framework. The closed testing principle can also be used in the context of resampling-based multiple tests to set up stepwise testing procedures that account for the underlying correlation structure. 5

PARTITION TESTING

The partitioning principle introduced in References 20 and 21 can be viewed as a natural extension of the principle of closed testing. The advantage of using the partitioning principle is two-fold: Partitioning procedures are sometimes more powerful than procedures derived within the closed testing framework and, unlike closed testing procedures, they are easy to invert in order to set up simultaneous confidence sets for parameters of

5

interest. To introduce the partitioning principle, consider k null hypotheses tested in a clinical trial and assume that H0i states that θ ∈ i , where θ is a multidimensional parameter and i is a subset of the parameter space. Partition the union of 1 , . . . , k into disjoint subsets ∗J , J ⊂ {1, . . . , k}, which can be interpreted as the part of the parameter space in which exactly H0i , i ∈ J are true and the remaining null hypotheses are false. Now define null hypotheses corresponding to the constructed subsets (i.e., HJ∗ : θ ∈ ∗J ) and test them at level α. As these null hypotheses are mutually exclusive, at most one of them is true. Therefore, although no multiplicity adjustment is made, the resulting multiple test controls the FWER at the α level. To illustrate the process of carrying out partitioning tests, consider the null hypotheses HL : µL ≤ µP and HH : µH ≤ µP from the dose-finding trial example. The union of HL and HH is partitioned into three hypotheses: H1∗ : µL ≤ µP

and

µH ≤ µP

H2∗

: µL ≤ µP

and

µH > µP

H3∗

: µL > µP

and

µH ≤ µP

Testing each of the three hypotheses with a level α significance test results in the following decision rule: • If H1∗ is accepted, neither HL nor HH can

be rejected, otherwise infer that µL > µP or µH > µP . • If H1∗ and H2∗ are rejected, one concludes that µL > µP . Likewise, rejecting H1∗ and H3∗ implies that µH > µP . • Finally, if H1∗ , H2∗ and H3∗ are rejected, the inference is that µL > µP and µH > µP . Although these decision rules appears to be similar to the closed testing rules, it is important to point out that the partitioning principle does not deal with the hypotheses in the closed family (i.e., HL , HH and HL ∩ HH ) but rather with hypotheses H1∗ , H2∗ , and H3∗ defined above. As a result of the choice of mutually exclusive null hypotheses, partitioning tests can be inverted to derive a confidence region for

6

MULTIPLE TESTING IN CLINICAL TRIALS

the unknown parameter θ . Recall that the most general method for constructing a confidence set from a significance test is defined as follows. For each parameter point θ0 , test H0 : θ = θ0 using an level-α test and then consider the set of all parameter points θ0 for which H0 : θ = θ0 is accepted. The obtained set is a 100(1 − α) confidence set for the true value of θ . This procedure corresponds to partitioning the parameter space into subsets consisting of a single parameter point and can be used for constructing simultaneous confidence limits associated with various stepwise tests. Consider, for example, confidence limits for the mean treatment differences between k dose groups and a control group (20). If the largest mean difference is not significant (T[k] < dk ), the one-sided limits for the true mean differences δ1 , . . . , δk are given by  δi > δˆi − dk s 2/n,

i = 1, . . . , k

and testing stops. Otherwise, one infers that δ[k] > 0 and examines the second largest difference. At the jth step of the stepwise test, the one-sided limits for δ[1] , . . . , δ[k−j+1] are  δ[i] > δˆ[i] − dk s 2/n,

i = 1, . . . , k − j + 1

if the corresponding test statistic is not significant (T[k−j+1] < dk−j+1 ) and δ[k−j+1] > 0 otherwise. Comparing the resulting testing procedure to the stepwise Dunnett test derived in Section 4 using the closed testing principle, it is easy to see that the partitioning principle extends the closed testing framework by enabling clinical researchers to set up confidence limits for treatment-control differences. The partitioning principle can also be used for constructing confidence sets in a much more general context [e.g., confidence intervals for fixed-sequence testing methods occurring in dose-finding studies and other clinical applications (22)]. REFERENCES 1. European Committee for Propriety Medicinal Products, Points to consider on multiplicity issues in clinical trials. September 19, 2002. 2. Y. Hochberg and A. C. Tamhane, Multiple Comparison Procedures. New York: Wiley, 1987.

3. P. H. Westfall and S. S. Young, ResamplingBased Multiple Testing: Examples and Methods for P-Value Adjustment. New York: Wiley, 1993. 4. J. C. Hsu, Multiple Comparisons: Theory and Methods. London: Chapman & Hall, 1986. 5. Y. Benjamini and Y. Hochberg, Controlling the false discovery rate—a practical and powerful approach to multiple testing. J. Royal Stat. Soc. Series B 1995; 57: 289–300. 6. H. Finner and M. Roter, On the false discovery rate and expected Type I errors. Biometr. J. 2001; 43: 985–1005. 7. S. N. Roy, On a heuristic method for test construction and its use in multivariate analysis. Ann. Stat. 1953; 24: 220–238. 8. R. J. Simes, An improved Bonferroni procedure for multiple tests of significance. Biometrika 1986; 63: 655–660. 9. S. Sarkar and C. K. Chang, Simes’ method for multiple hypothesis testing with positively dependent test statistics. J. Amer. Stat. Assoc. 1997; 92: 1601–1608. 10. C. W. Dunnett, A multiple comparison procedure for comparing several treatments with a control. J. Amer. Stat. Assoc. 1955; 50: 1096–1121. 11. R. Marcus, E. Peritz and K. R. Gabriel, On closed testing procedure with special reference to ordered analysis of variance. Biometrika 1976; 63: 655–660. 12. P. Bauer, Multiple testings in clinical trials. Stat. Med. 1991; 10: 871–890. 13. W. Lehmacher, G. Wassmer and P. Reitmeir, Procedures for two-sample comparisions with multiple endpoints controlling the experimentwise error rate. Biometrics 1991; 47: 511–521. 14. D. M. Rom, R. J. Costello and L. T. Connell, On closed test procedures for dose-response analysis. Stat. Med. 1994; 13: 1583–1596. 15. A. Dmitrienko, W. Offen and P. H. Westfall, Gatekeeping strategies for clinical trials that do not require all primary effects to be significant. Stat. Med. 2003; 22: 2387–2400. 16. P. H. Westfall and A. Krishen, Optimally weighted, fixed sequence, and gate-keeping multiple testing procedures. J. Stat. Plan. Inference 2001; 99: 25–40. 17. S. Holm, A simple sequentially rejective multiple test procedure. Scand. J. Stat. 1979; 6: 65–70. 18. Y. Hochberg, A sharper Bonferroni procedure for multiple significance testing. Biometrika 1988; 75: 800–802.

MULTIPLE TESTING IN CLINICAL TRIALS 19. G. Hommel, A stagewise rejective multiple test procedure based on a modified Bonferroni test. Biometrika 1988; 75: 383–386. 20. G. Stefansson, W-C. Kim and J. C. Hsu, On confidence sets in multiple comparisons. In: S. S. Gupta and J. O. Berger, eds. Statistical Decision Theory and Related Topics IV. New York: Academic Press, 1988, pp. 89–104. 21. H. Finner and K. Strassburger, The partitioning principle: a powerful tool in multiple decision theory. Ann. Stat. 2002; 30: 1194–1213. 22. J. C. Hsu and R. L. Berger, Stepwise confidence intervals without multiplicity adjustment for dose-response and toxicity studies. J. Amer. Stat. Assoc. 1989; 94: 468–482.

FURTHER READING A. Dmitrienko, G. Molenberghs, C. Chuang-Stein, W. Offen. Analysis of Clinical Trials Using SAS: A Practical Guide. Carry, NC (Chapter 2, ‘‘Multiple comparisons and multiple endpoints’’). 2005.

7

MULTISTAGE GENETIC ASSOCIATION STUDIES

Our primary focus is on interactions rather than main effects, which has been the main thrust of most of the literature on multistage designs. In a pharmaceutical trial, there is generally a wealth of prior physiological and biochemical knowledge about the pathway(s) targeted by the agent under study, so the aim of a genetic study will be to characterize the role of particular functional polymorphisms across multiple genes within critical steps in the pathway. The exposure variable (here, treatment) may be studied with a randomized controlled trial, rather than by observational epidemiologic studies with their attendant potential for confounding and other biases. The study design is likely to involve unrelated individuals rather than families, particularly in therapeutic trials where there would be few families with multiple cases eligible to be treated concurrently and even fewer opportunities to assign members of the same family to different treatments.

DUNCAN THOMAS DAVID CONTI Department of Preventive Medicine University of Southern California Los Angeles, CA

We consider the design of genetic association studies within the context of clinical research, where the ultimate question might be to identify and characterize genes that modify the response to pharmacologic agents, either preventive or therapeutic, or other interventions. This includes both existing clinical trials primarily designed to determine treatment effects, expanding them with secondary goals aimed at characterizing the influence of genes, as well as nested challenge or treatment experiments within existing population-based studies, exploiting extensive information (possibly genetic) that may already have been collected. In either case, the questions are similar. Why is it that some people respond favorably to a particular treatment and others do not? Why do some experience a particular adverse effect and others do not? Why is one treatment better for some people, and a different treatment better for someone else? Could something in a person’s genetic makeup explain such differences and help inform a future of personalized medicine? There is, of course, a vast literature on the design of genetic association studies and pharmacogenetic trials, which is summarized in other articles in this volume (see the articles on genetic association analysis and pharmacogenomics) as well as previously in the literature (1–3). Here, we focus on the design of multistage association studies, where statistical sampling can be used in various ways to improve cost-efficiency. General works are available on multistage designs (4–7) and the context of genetic studies (8–10). In particular, we focus on two-stage designs in the pharmacogenetic context. This focus has several implications.

It is also worth considering the various genetic contexts in which such research might be undertaken. One might have in mind a particular candidate gene containing one or more known functional polymorphisms that are known to play a role in the metabolism or other action of the agent under study. Or one might know the function of the gene, but not have identified any (or all) functional polymorphisms, and would like to fully characterize the spectrum of variation within the gene and its potential impact on the outcome under study. This may entail single nucleotide polymorphism (SNP) discovery through resequencing, or extensive genotyping of all known polymorphisms in a sample, before proceeding with typing only a subset of variants in the main study. More generally, one may have some understanding

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

MULTISTAGE GENETIC ASSOCIATION STUDIES

of an entire pathway involving multiple genes and possibly environmental factors affecting their expression or substrates, or even a complex network of interconnected pathways; in either case, one wishes to build a model for the entire system of genes. Whether one is studying a single gene or an entire pathway, it may be helpful to measure certain intermediate biomarkers (gene expression, methylation, metabolomic or proteomic measures) or additional clinical phenotypes. As these may be expensive or difficult to obtain, one might consider using only a subsample for this purpose. If no particular candidate genes are known, one might consider a genomewide search for possible associations, or at least a search for variants within a candidate region (say from a previous linkage scan); in either case, one would be looking for variants that might either be directly causal or in linkage disequilibrium with other causal variants. Each of these contexts provides different design challenges that might be addressed using some form of multistage design. 1 REASONS TO CONSIDER MULTISTAGE DESIGNS The main reason to consider multistage sampling designs is cost. No subsampling can possibly yield a more powerful study than obtaining complete data on all variables on all subjects, unless the cost of doing so would require a smaller overall sample size (1). But some measurements may be more informative or more costly than others, so some trade-off could lead to a more cost-efficient design. Here, two situations must be distinguished, one where the total sample size is fixed (say, by a previously completed clinical trial or population-based sample), and one where it is not. In the former case, one may simply be interested in getting a reasonably precise answer at a reasonable cost, recognizing that additional genetic measurements

or treatment assignments may not improve precision enough to justify the cost of making them on everybody. In this case, one might have an opportunity to conduct a larger main study by saving costs on some measurements through subsampling. A second reason is there may be opportunities to improve the statistical efficiency of the design by using stratified sampling, based on one or more sources of information already collected. These could include treatment arm, outcomes, covariates, genotypes (or surrogates for them), or various combinations of such factors. Depending on the sampling design used, adjustments may be necessary to allow for the sampling scheme in the statistical analysis to obtain unbiased estimates. For example, if one were to overrepresent both cases (adverse outcomes) and active treatment arm subjects, then a na¨ıve analysis would induce a spurious association between treatment and outcome and likely distort the assessment of the modifying effect of genotypes or biomarkers measured on the subsample. Fortunately, allowing for such biased sampling schemes is relatively easy via weighting subjects by the inverse of their sampling probabilities in the analysis (11) or by including the logarithms of these weights as ‘‘offsets’’ (covariates with regression coefficients fixed at unity) in a logistic regression analysis (12). As an extreme example, when investigating a dichotomous outcome within a previously performed randomized control trial, one might only select the ‘‘cases’’ as the subset for further genotyping and then perform the appropriate case-only analysis, yielding an increase in power for detection of gene × treatment interactions (13–17). In many cases, the substudy data only may be sufficient to answer the immediate questions, but some types of more integrative questions might require combining the information from the main study and substudy in some manner. For example, one might use substudy data to build a model for the dependence of biomarkers on genes, treatment, and possibly other factors, and then incorporate these predicted biomarkers into an analysis of the main study in which data on treatments, genotypes, and outcomes (but not biomarkers) were available on everybody (18).

MULTISTAGE GENETIC ASSOCIATION STUDIES

Much of our previous discussion and literature has assumed that the genetic association study is to be conducted as an add-on to a previous clinical trial, with the aim of studying the genetic determinants of response to treatment. An alternative approach might be to add a clinical trial to some preexisting observational study in which the genetic variation has already been characterized (for instance, an etiologic cohort or case-control study), and use the genetics as the basis for a sampling scheme to assess the outcome of some treatment in targeted subgroups. The potential benefit of this lies in being able to balance the genotypic distribution to obtain greater statistical efficiency when testing for gene–treatment interactions. For example, if one were interested in investigating how treatment response varies in relation to a previously measured rare genetic variant, one could overrepresent mutation carriers relative to their original cohort frequencies. However, one may quickly reach the limits for an efficient block design if the number of independent variants under investigation is large (see the article on stratification). Finally, whatever general sampling and analysis scheme is used, it may be possible to predict what the optimal sampling fractions would be for a range of plausible model parameters and costs (19–23). Under some circumstances, such an analysis may reveal that the most efficient design, weighing both statistical efficiency and cost, does not involve any subsampling with differential assessment of variables across samples but rather a complete assessment of all variables on a smaller main study. 2 EXAMPLE 1: CANDIDATE GENE ASSOCIATION STUDY USING TAG SNPs Suppose one has identified a candidate gene that is hypothesized to have an effect on the outcome under study or, in a clinical trial setting, to modify the effect of treatment. If one or more functional polymorphisms have already been identified in this gene and if one were persuaded that the hypothesized effects of the gene could be tested using only these known variants, there would probably be no need for a multistage sampling

3

design. Assuming that the cost for genotyping these known variants is not prohibitive, one would simply genotype these variants in a single sample from the relevant population; this might be, for example, cases and controls for a disease trait, a simple or stratified random sample of the population for a quantitative trait, or possibly the entire sample from a preexisting study such as a clinical trial. The design and analysis of such studies is discussed elsewhere in this volume (see the articles on genetic association analysis and pharmacogenomics). A negative result from such a study could, however, arise if the targeted polymorphism(s) was not the relevant one and would not exclude the possibility that some other variants in the gene might have shown an effect. Complete characterization of a gene—including all rare variations— conventionally entails full sequencing of the entire gene (not just coding regions, but potential regulatory regions, splice-site junctions, and highly conserved intronic regions). Because previous genomic databases could not be relied on to include all rare variants, a large sample would be required to detect them, although there do exist procedures for detecting rare variants in DNA pools (24). If such large samples are required, such a monumental task would be difficult to justify unless one had a strong hypothesis that rare variants were likely to be important (25–28). If, on the other hand, one wished instead to exhaustively characterize only the common variation in the gene, then it would not be necessary to fully sequence every subject, and a staged approach might be considered. The fact that most common variants are in strong linkage disequilibrium (LD) with other nearby variants means that most of them can be effectively predicted by combinations of a modest number of ‘‘tagging SNPs’’ (29–34). In this case, a relatively small sample of individuals might be sufficient to characterize the LD structure of the gene and select this subset of tag SNPs, and only this subset is then tested in the main study. However, in determining the subsample needed, one must also take into account the uncertainty of LD levels and patterns across the genome and the effect this uncertainty may have on characterization of

4

MULTISTAGE GENETIC ASSOCIATION STUDIES

the underlying genetic diversity. Often the SNP selection sample is not part of the main study and is selected to represent the source population (mainly controls, if the trait is rare), so the information on the untagged variants is not used in the analysis of the main association study. Because the International HapMap Project is designed to provide a freely available catalog of the genetic similarities and difference across human populations, this resource serves as a primary example of SNP selection samples and often remains independent of the main study in the final analysis. However, the SNP selection sample could be used to develop a prediction model for each of the untagged variants, which could then be tested for association in the main study (23). Such analyses could be done at the genotype or haplotype level. Suppose, for example, that one wished to test the association of a disease trait Y with a particular untyped SNP polymorphism G using a set T of tagging SNPs. The substudy (in which both G and T are measured) yields an estimate of pα (G|T), where α denotes a vector of LD information to be estimated from the substudy. The main study would then be analyzed using likelihood contributions of the form for each subject: pG (Y|T) = g pβ (Y|G = g)pα (G = g|T) where β is the relative risk parameter of interest. The single-imputation strategy (35) approximates this likelihood by using as the risk factor for each individual in the main study his or her expected allele (or haplotype) ‘‘dosage,’’ E(G|T). Although more complex, the likelihood-based analysis, explicitly incorporating contributions from the substudy subjects, has the advantage of properly allowing for the uncertainty in the estimates of the parameters α in the genotype prediction model, and supports a broader range of analyses, such as the estimation of the location of an unobserved causal variant. A haplotype-based analysis is similar, except that the postulated causal genotype G is replaced by a vector of diplotypes (pairs of haplotypes) H, and the likelihood requires an additional summation over all possible diplotypes that are compatible with the observed unphased tag-SNP genotypes T.

This likelihood can be fitted using either the expectation-maximization (E-M) algorithm (36) or Markov-chain Monte Carlo (MCMC) methods (37). This basic approach could be extended to a multistage design, depending on the extent of ancillary knowledge from genomic databases. For example, one might fully sequence a small sample to search for previously undetected variants (particularly if one were studying a population that was underrepresented in the HapMap or previous resequencing databases); cases with extreme phenotypes are sometimes used for this purpose to identify previously unrecognized variants that are more likely to be causal (38). A somewhat larger sample of controls might then be genotyped at all common variants (from the investigator’s own first sample as well as other databases) to investigate the LD structure and choose tagSNPs, which would then be genotyped in the main study. In a study of gene-treatment interactions, one could optimize a two-stage design by selecting the subsample and stratifying on outcome, treatment, or a correlate of the causal variant (23). For example, one might overrepresent subjects with a positive family history or known carriers of a particular functional polymorphism. 3 EXAMPLE 2: PATHWAY-BASED STUDY INVOLVING BIOMARKERS Suppose one wished to investigate an entire pathway or network of genes, such as those known to be involved in the metabolism of a particular drug or involved in repair of DNA damage induced by ionizing radiation. One could simply genotype all these loci in the entire study population and build an empirical model for the main effects and interactions (gene–gene or gene–treatment, for example) (39) or a physiologically based pharmacokinetic model incorporating these genes as modifiers of the respective metabolic rates (40). Such models might be usefully informed by having additional biomarker data on some of the intermediate steps in the process, such as metabolite concentrations in blood or urine (41). These measurements may be expensive or difficult to collect, however, possibly

MULTISTAGE GENETIC ASSOCIATION STUDIES

requiring multiple samples over time because of the variability in the measurements or biases due to inducing factors, measurement factors (e.g., time of day), or the underlying disease process (making measurements on cases suspect due to the possibility of ‘‘reverse causation’’). One might therefore consider making the biomarker measurements on only a sample, perhaps only for controls. Suppose one is primarily interested in studying the relation of some intermediate metabolite X on outcome Y, where X is postulated to be modified by treatment T and genes G, and the biomarker Z is a flawed surrogate for X. Then one might have measurements of (T,G,Z) on the subsample S and of (T,G,Y) on the main study M. The likelihood is then formed by marginalizing over the unobserved X:
 p(Zi |Xi = x)p(Xi = x|Gi , Ti ) dx iS

×




p(Yj |Xj = x)p(Xj = x|Gj , Tj ) dx

jM

As in Example (1), one might seek to optimize the study design by stratified sampling of S based on information about (T,G,Y) already obtained from the main study. Or, rather than using such a model-dependent likelihood analysis, the technique of Mendelian randomization (42–47) provides an indirect ‘‘nonparametric’’ test of the effect of X on Y by comparing instead the effects of G on both Y and Z separately. Davey Smith and Ibrahim (42) have argued that, under appropriate circumstances, inferences from Mendelian randomization about the effect of an environmentally modifiable intermediate variable can have the same strength of causality as from a randomized controlled trial. 4 EXAMPLE 3: GENOME-WIDE ASSOCIATION STUDY Now, suppose there were no specific genes or pathways with strong prior hypotheses, but one wished instead to scan the entire genome for variants that influenced the trait or response to treatment. This possibility was first seriously suggested a decade ago (48),

5

but it is only very recently that advances in high-volume genotyping technologies have made it possible at a reasonable cost (49–52). Most recent genome-wide association (GWA) studies have adopted some variant of a multistage genotyping strategy first proposed by Sobell et al. (53) and formalized in a series of papers by Satagopan et al. (54–56) and others (57–60). The basic idea of all these approaches is to use relatively inexpensive (per genotype) commercial genotyping technology for dense SNP genotyping (about 300 thousand to 1 million SNPs and increasing rapidly) of a sample of the study population and, on the basis of the observed associations in this sample; one selects a small subset of the most promising SNPs for genotyping in a second sample, using a more expensive (per genotype) customized assay. The two samples are then combined in the final analysis, with critical values at stages 1 and 2 chosen to yield a predetermined level of genome-wide significance (e.g., 0.05 expected false positives across the entire genome) (61). Variants of this general idea include targeted sampling of subjects for the first stage, use of DNA pooling in the first stage followed by individual genotyping 62, 63, adding additional SNPs at the second stage to localize the LD signal (60), or using prior information such as linkage scans or genomic annotation data to prioritize the selection of SNPs for the second stage 64, 65. Needless to say, with hundreds of thousands of markers under consideration, the multiple comparisons problem is serious (see the article on multiple comparisons in this work). A variety of ways of analyzing the data have been proposed, generally using some form of Bonferroni correction or false discovery rate approaches (66–68). Even more daunting are analyses of all possible haplotype associations (69) or all possible gene–gene interactions (70). Given the infancy of this approach, there are many unresolved design and analysis issues, but there are many opportunities to exploit multistage sampling designs in new and creative ways. As in the previous example, of course, one could stratify the sampling of subjects by various combinations of disease, treatment (or other interacting factors), or race. In addition, there are various ways to refine the selection of markers to

6

MULTISTAGE GENETIC ASSOCIATION STUDIES

advance to stage 2, particularly if there are multiple types of tests under consideration (main effects of SNPs, haplotype associations, gene–gene or gene–treatment interactions, etc.) or prior knowledge to be exploited. 5

PERSPECTIVES

Although multistage sampling designs have been well established in statistics for decades and are increasingly used in epidemiology and genetics, this remains an underdeveloped research area, particularly in the pharmacogenetics context. Its utility for exploring pathways appears to be particularly promising, especially in the potential incorporation of pharmacokinetic/pharmacodynamic principles and approaches (71–75). This field is in a rapid state of development, driven in part by the potential for drug development. The interested reader should consult several recent reviews (17, 76–80) for further discussion of these issues. REFERENCES 1. R. C. Elston, R. M. Idury, L. R. Cardon, and J. B. Lichter, The study of candidate genes in drug trials: sample size considerations. Stat Med. 1999; 18: 741–751. 2. L. R. Cardon, R. M. Idury, T. J. Harris, J. S. Witte, and R. C. Elston, Testing drug response in the presence of genetic information: sampling issues for clinical trials. Pharmacogenetics. 2000; 10: 503–510. 3. P. J. Kelly, N. Stallard, and J. C. Whittaker, Statistical design and analysis of pharmacogenetic trials. Stat Med. 2005; 24: 1495–1508. 4. N. Breslow and K. Cain, Logistic regression for two-stage case-control data. Biometrika. 1988; 75: 11–20. 5. N. E. Breslow and N. Chatterjee, Design and analysis of two-phase studies with binary outcome applied to Wilms tumor prognosis. Appl Stat. 1999; 48: 457–468. 6. K. Cain and N. Breslow, Logistic regression analysis and efficient design for twostage studies. Am J Epidemiol. 1988; 128: 1198–1206. 7. J. E. White, A two stage design for the study of the relationship between a rare exposure and a rare disease. Am J Epidemiol. 1982; 115; 119–128.

8. R. C. Elston, D. Y. Lin, and G. Zheng. Multistage sampling for genetic studies. Annu Rev Genomics Hum Genet. 2007; 8: 327–342. 9. K. D. Siegmund, A. S. Whittemore, and D. C. Thomas, Multistage sampling for disease family registries. J Natl Cancer Inst Monogr. 1999; 26: 43–48. 10. A. S. Whittemore and J. Halpern, Multi-stage sampling in genetic epidemiology. Stat Med. 1997; 16: 153–167. 11. D. Horvitz and D. Thompson, A generalization of sampling without replacement from a finite population. J Am Stat Assoc. 1952; 47: 663–685. 12. B. Langholz and L. Goldstein. Conditional logistic analysis of case-control studies with complex sampling. Biostatistics. 2001; 2: 63–84. 13. W. Piegorsch, C. Weinberg, and J. Taylor, Non-hierarchical logistic models and caseonly designs for assessing susceptibility in population-based case-control studies. Stat Med. 1994; 13: 153–162. 14. M. Khoury and W. Flanders, Nontraditional epidemiologic approaches in the analysis of gene-environment interaction: case-control studies with no controls! Am J Epidemiol. 1996; 144: 207–213. 15. S. Greenland, A unified approach to the analysis of case-distribution (case-only) studies. Stat Med. 1999; 18: 1–15. 16. W. J. Gauderman, Sample size requirements for matched case-control studies of geneenvironment interaction. Stat Med. 2002; 21: 35–50. 17. J. Little, L. Sharp, M. J. Khoury, L. Bradley, and M. Gwinn, M. The epidemiologic approach to pharmacogenomics. Am J Pharmacogenomics. 2005; 5: 1–20. 18. D. C. Thomas, Multistage sampling for latent variable models. Lifetime Data Anal. In press. 19. D. Spiegelman and R. Gray, Cost-efficient study designs for binary response data with Gaussian covariate measurement error. Biometrics. 1991; 47: 851–869. 20. D. Spiegelman, R. J. Carroll, and V. Kipnis, Efficient regression calibration for logistic regression in main study/internal validation study designs with an imperfect reference instrument. Stat Med. 2001; 20: 139–160. 21. S. Greenland, Statistical uncertainty due to misclassification: implications for validation substudies. J Clin Epidemiol. 1988; 41: 1167–1174.

MULTISTAGE GENETIC ASSOCIATION STUDIES 22. W. D. Flanders and S. Greenland, Analytic methods for two-stage case-control studies and other stratified designs. Stat Med. 1991; 10: 739–747. 23. D. Thomas, R. Xie, and M. Gebregziabher, Two-Stage sampling designs for gene association studies. Genet Epidemiol. 2004; 27: 401–414. 24. L. T. Amundadottir, P. Sulem, J. Gudmundsson, A. Helgason, A. Baker, A., et al., A common variant associated with prostate cancer in European and African populations. Nat Genet. 2006; 38: 652–658. 25. J. K. Pritchard, Are rare variants responsible for susceptibility to complex diseases? Am J Hum Genet. 2001; 69: 124–137. 26. J. K. Pritchard and N. J. Cox, The allelic architecture of human disease genes: common disease-common variant . . . or not? Hum Mol Genet. 2002; 11: 2417–2423. 27. W. Y. Wang, H. J. Cordell, and J. A. Todd, Association mapping of complex diseases in linked regions: estimation of genetic effects and feasibility of testing rare variants. Genet Epidemiol. 2003; 24: 36–43. 28. N. S. Fearnhead, J. L. Wilding, B. Winney, S. Tonks, S. Bartlett, et al. Multiple rare variants in different genes account for multifactorial inherited susceptibility to colorectal adenomas. Proc Natl Acad Sci USA. 2004; 101: 15992–15997. 29. D. O. Stram, Tag SNP selection for association studies. Genet Epidemiol. 2004; 27: 365–374. 30. D. O. Stram, Software for tag single nucleotide polymorphism selection. Hum Genomics. 2005; 2: 144–151. 31. D. Thompson, D. Stram, D. Goldgar, and J. S. Witte, Haplotype tagging single nucleotide polymorphisms and association studies. Hum Hered. 2003; 56: 48–55. 32. G. C. Johnson, L. Esposito, B. J. Barratt, A. N. Smith, J. Heward, et al. Haplotype tagging for the identification of common disease genes. Nat Genet. 2001; 29: 233–237. 33. Z. Lin and R. B. Altman, Finding haplotype tagging SNPs by use of principal components analysis. Am J Hum Genet. 2004; 75: 850–861. 34. Z. Meng, D. V. Zaykin, C. F. Xu, M. Wagner, and M. G. Ehm, Selection of genetic markers for association analyses, using linkage disequilibrium and haplotypes. Am J Hum Genet. 2003; 73: 115–130. 35. D. O. Stram, C. L. Pearce, P. Bretsky, M. Freedman, J. N. Hirschhorn, et al., Modeling and E-M estimation of haplotype-specific

7

relative risks from genotype data for a casecontrol study of unrelated individuals. Hum Hered. 2003; 55: 179–190. 36. L. Excoffier, G. Laval, and D. Balding, Gametic phase estimation over large genomic regions using an adaptive window approach. Hum Genomics. 2003; 1: 7–19. 37. T. Niu, Z. S. Qin, X. Xu, and J. S. Liu, Bayesian haplotype inference for multiple linked singlenucleotide polymorphisms. Am J Hum Genet. 2002; 70: 157–169. 38. C. A. Haiman, D. O. Stram, M. C. Pike, L. N. Kolonel, N. P. Burtt, et al., A comprehensive haplotype analysis of CYP19 and breast cancer risk: the Multiethnic Cohort. Hum Mol Genet. 2003; 12: 2679–2692. 39. R. J. Hung, P. Brennan, C. Malaveille, S. Porru, F. Donato, et al., Using hierarchical modeling in genetic association studies with multiple markers: application to a case-control study of bladder cancer. Cancer Epidemiol Biomark Prev. 2004; 13: 1013–1021. 40. D. V. Conti, V. Cortessis, J. Molitor, and D. C. Thomas, Bayesian modeling of complex metabolic pathways. Hum Hered. 2003; 56: 83–93. 41. P. Tonolio, P. Boffetta, D. E. K. Shuker, N. Rothman, B. Hulka, and N. Pearce, Application of Biomarkers in Cancer Epidemiology. Lyon, France: IARC Scientific, 1997. 42. G. Davey Smith and S. Ebrahim, ‘‘Mendelian randomization’’: can genetic epidemiology contribute to understanding environmental determinants of disease? Int J Epidemiol. 2003; 32: 1–22. 43. Brennan, P. Commentary: Mendelian randomization and gene-environment interaction. Int J Epidemiol. 2004; 33: 17–21. 44. G. Davey Smith and S. Ebrahim, What can mendelian randomisation tell us about modifiable behavioural and environmental exposures? BMJ. 2005; 330: 1076–1079. 45. G. Davey Smith and S. Ebrahim, Mendelian randomization: prospects, potentials, and limitations. Int J Epidemiol. 2004; 33: 30–42. 46. D. C. Thomas and D. V. Conti, Commentary: the concept of ‘‘mendelian randomization.’’ Int J Epidemiol. 2004; 33: 21–25. 47. J. Little and M. J. Khoury, Mendelian randomization: a new spin or real progress? Lancet. 2003; 362: 390–391. 48. N. Risch and K. Merikangas, The future of genetic studies of complex human diseases. Science. 1996; 273: 1616–1617.

8

MULTISTAGE GENETIC ASSOCIATION STUDIES

49. D. C. Thomas, R. W. Haile, and D. Duggan, Recent developments in genomewide association scans: a workshop summary and review. Am J Hum Genet. 2005; 77: 337–345. 50. D. C. Thomas, Are we ready for genomewide association studies? Cancer Epidemiol Biomarkers Prev. 2006; 15: 595–598. 51. W. Y. S. Wang, B. J. Barratt, D. G. Clayton, and J. A. Todd, Genome-wide association studies: theoretical and practical concerns. Nat Rev Genet. 2005; 6: 109–118. 52. J. N. Hirschhorn and M. J. Daly, Genomewide association studies for common disease and complex traits. Nat Rev Genet. 2005; 6: 95–108. 53. J. L. Sobell, L. L. Heston, and S. S. Sommer, Novel association approach for determining the genetic predisposition to schizophrenia: case-control resource and testing of a candidate gene. Am J Med Genet. 1993; 48: 28–35. 54. J. M. Satagopan, D. A. Verbel, E. S. Venkatraman, K. E. Offit, and C. B. Begg, Two-stage designs for gene-disease association studies. Biometrics. 2002; 58: 163–170. 55. J. M. Satagopan and R. C. Elston, Optimal two-stage genotyping in population-based association studies. Genet Epidemiol. 2003; 25: 149–157. 56. J. M. Satagopan, E. S. Venkatraman, and C. B. Begg, Two-stage designs for gene-disease association studies with sample size constraints. Biometrics. 2004; 60: 589–597. 57. A. Saito and N. Kamatani, Strategies for genome-wide association studies: optimization of study designs by the stepwise focusing method. J Hum Genet. 2002; 47: 360–365. 58. S. K. Service, L. A. Sandkuijl, and N. B. Freimer, Cost-effective designs for linkage disequilibrium mapping of complex traits. Am J Hum Genet. 2003; 72: 1213–1220. 59. H. Wang and D. O. Stram, Optimal twostage genome-wide association designs based on false discovery rate. Comput Stat Data Anal. 2006; 5: 457–465. 60. H. Wang, D. C. Thomas, I. Pe’er, and D. O. Stram, Optimal two-stage genotyping designs for genome-wide association scans. Genet Epidemiol. 2006; 30: 356–368. 61. A. D. Skol, L. J. Scott, G. R. Abecasis, and M. Boehnke, Joint analysis is more efficient than replication-based analysis for two-stage genome-wide association studies. Nat Genet. 2006; 38: 209–213. 62. A. Bansal, D. van den Boom, S. Kammerer, C. Honisch, G. Adam, et al., Association testing by DNA pooling: an effective initial

screen. Proc Natl Acad Sci USA. 2002; 99: 16871–16874. 63. S. Wang, K. K. Kidd, and H. Zhao, On the use of DNA pooling to estimate haplotype frequencies. Genet Epidemiol. 2003; 24: 74–82. 64. K. Roeder, S. A. Bacanu, L. Wasserman, and B. Devlin, Using linkage genome scans to improve power of association in genome scans. Am J Hum Genet. 2006; 78: 243–252. 65. A. S. Whittemore, A Bayesian false discovery rate for multiple testing. Appl Stat. 2007; 34: 1–9. 66. D. Y. Lin, Evaluating statistical significance in two-stage genomewide association studies. Am J Hum Genet. 2006; 78: 505–509. 67. Thomson, G. Significance levels in genome scans. Adv Genet. 2001; 42: 475–486. 68. J. Zhao, E. Boerwinkle, and M. Xiong, An entropy-based statistic for genomewide association studies. Am J Hum Genet. 2005; 77: 27–40. 69. S. Lin, A. Chakravarti, and D. J. Cutler, Exhaustive allelic transmission disequilibrium tests as a new approach to genomewide association studies. Nat Genet. 2004; 36: 1181–1188. 70. J. Marchini, P. Donnelly, and L. R. Cardon, Genome-wide strategies for detecting multiple loci that influence complex diseases. Nat Genet. 2005; 37: 413–417. 71. M. Eichelbaum, M. Ingelman-Sundberg, and W. E. Evans, Pharmacogenomics and individualized drug therapy. Annu Rev Med. 2006; 57: 119–137. 72. T. W. Gant and S. D. Zhang, In pursuit of effective toxicogenomics. Mutat Res. 2005; 575: 4–16. 73. I. Cascorbi, Genetic basis of toxic reactions to drugs and chemicals. Toxicol Lett. 2006; 162: 16–28. 74. R. H. Howland, Personalized drug therapy with pharmacogenetics—Part 1: pharmacokinetics. J Psychosoc Nurs Ment Health Serv. 2006; 44: 13–16. 75. R. H. Howland, Personalized drug therapy with pharmacogenetics—Part 2: pharmacodynamics. J Psychosoc Nurs Ment Health Serv. 2006; 44: 13–16. 76. C. M. Ulrich, K. Robien, and R. Sparks, Pharmacogenetics and folate metabolism—a promising direction. Pharmacogenomics. 2002; 3: 299–313. 77. C. M. Ulrich, K. Robien, and H. L. McLeod, Cancer pharmacogenetics: polymorphisms,

MULTISTAGE GENETIC ASSOCIATION STUDIES pathways and beyond. Nat Rev Cancer. 2003; 3: 912–920. 78. Z. Feng, R. Prentice, and S. Srivastava, Research issues and strategies for genomic and proteomic biomarker discovery and validation: a statistical perspective. Pharmacogenomics. 2004; 5: 709–719. 79. U. Mahlknecht and S. Voelter-Mahlknecht, Pharmacogenomics: questions and concerns. Curr Med Res Opin. 2005; 21: 1041–1048.

9

80. A. C. Need, A. G. Motulsky, and D. B. Goldstein, Priorities and standards in pharmacogenetic research. Nat Genet. 2005; 37: 671–681.

NATIONAL CANCER INSTITUTE (NCI)

of disparities among underserved groups and gaps in quality cancer care, helping to translate research results into better health for groups at high risk for cancer, including cancer survivors and the aging population. As the leader of the National Cancer Program, the NCI provides vision and leadership to the global cancer community, conducting and supporting international research, training, health information dissemination, and other programs. Timely communication of NCI scientific findings help people make better health choices and advise physicians about treatment options that are more targeted and less toxic.

The National Cancer Institute is the world’s largest organization solely dedicated to cancer research. The NCI supports researchers at universities and hospitals across the United States and at NCI Designated Cancer Centers, a network of facilities that not only study cancer in laboratories but also conduct research on the best ways to rapidly bring the fruits of scientific discovery to cancer patients. In the NCI’s own laboratories, almost 5000 principal investigators, from basic scientists to clinical researchers, conduct earliest phase cancer clinical investigations of new agents and drugs. Recent advances in bioinformatics and the related explosion of technology for genomics and proteomics research are dramatically accelerating the rate for processing large amounts of information for cancer screening and diagnosis. The largest collaborative research activity is the Clinical Trials Program for testing interventions for preventing cancer, diagnostic tools, and cancer treatments, allowing access as early as possible to all who can benefit. The NCI supports over 1300 clinical trials a year, assisting more than 200,000 patients. The NCI’s scientists also work collaboratively with extramural researchers to accelerate the development of state-of-the-art techniques and technologies. In addition to direct research funding, the NCI offers U.S. cancer scientists a variety of useful research tools and services, including tissue samples, statistics on cancer incidence and mortality, bioinformatic tools for analyzing data, databases of genetic information, and resources through NCI-supported Cancer Centers, Centers of Research Excellence, and the Mouse Models of Human Cancer Consortium. NCI researchers are also seeking the causes This article was modified from the website of the National Institutes of Health (http://www.nih.gov /about/almanac/organization/NCI.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NATIONAL CENTER FOR TOXICOLOGICAL RESEARCH (NCTR)

The NCTR’s research program involves basic and applied research specifically designed to define biological mechanisms of action underlying the toxicity of products regulated by the FDA. This research is aimed at understanding critical biological events in the expression of toxicity and at developing methods to improve assessment of human exposure, susceptibility, and risk. The NCTR conducts research through eight divisions. The NCTR research divisions study biochemical and molecular markers of cancer, nutritional modulation of risk and toxicity, developmental toxicity, neurotoxicity, quantitative risk assessment, transgenics, applied and environmental microbiology, and solid-state toxicity. Each division works with the others to support the FDA’s mission to bring safe and efficacious products to the market rapidly and to reduce the risk of adverse health effects from products on the market.

The National Center for Toxicological Research (NCTR) conducts peer-reviewed scientific research and provides expert technical advice and training that enables the U.S. Food and Drug Administration (FDA) to make sound, science-based regulatory decisions. The research is focused toward understanding critical biological events in the expression of toxicity and toward developing methods and incorporating new technologies to improve the assessment of human exposure, susceptibility, and risk through the characterization of present models and the development of new models. The aim of the program is to: • Conduct

peer-reviewed scientific research that provides the basis for FDA to make sound, science-based regulatory decisions, and to promote U.S. health through the FDA’s core activities of premarket review and postmarket surveillance.

• Conduct fundamental and applied re-

search aimed at understanding critical biological events and to determine how people are affected adversely by exposure to products regulated by FDA. • Develop methods to measure human

exposure to products that have been adulterated or to assess the effectiveness and/or the safety of a product. • Provide the scientific findings used by

the FDA product centers for premarket application review and produce safety assurance to the scientific community for the betterment of public health. This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/nctr/overview/mission.htm and http://www.fda.gov/oc/oms/ofm/accounting/cfo /2002/NCTR.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NATIONAL COOPERATIVE GALLSTONE STUDY

consisting of a standard X-ray of the abdominal region after injestion of iodine that is absorbed and secreted into bile to allow visualization of the gallbladder and its contents on the X-ray. Gallstones formed from pure cholesterol tend to be buoyant and seem to be ‘‘floating’’ on the X-ray. Gallstones containing calcium are termed ‘‘calcified.’’ Gallstones can be benign for a period of years, but they may also cause periodic episodes of pain (biliary colic) caused by bile duct obstruction that occurs when a gallstone has either escaped from the gallbladder or is formed in a bile duct. Obstruction in the cystic duct that connects the gallbladder to the larger common duct leads to debilitating episodes of cholecystitis or gallbladder inflamation that often requires emergency surgery. Gallstones can also promote bacterial infection of the bile in the gallbladder or bile ducts (cholangitis) requiring emergency therapy and when untreated may lead to sepsis. In 1970, the only available treatment for gallstone disease was radical cholecystectomy or surgical removal. Unlike the laparoscopic cholecystectomy practiced today, the radical procedure required an extensive hospital stay and a period of rehabilitation. Several different bile acids are secreted into bile in humans, with the primary bile acid being cholic acid and its derivatives. A small preliminary study showed that feeding chenodoxycholic acid (chenodiol or just cheno, for short) daily via a pill could restore cholesterol solubility in bile. It was hypothesized that prolonged treatment of cholesterol gallstones with chenodiol would desaturate cholesterol in bile and would literally cause the gallstones to dissolve. However, chenodiol, then as now, could not be synthesized from raw chemicals and must be extracted from the bile of animals at slaughter; then it must be purified into crystals, formulated, and then encapsulated. There was no industrial sponsor for the compound. Today chenodiol would be characterized as an ‘‘orphan’’ drug. Thus, the National Cooperative Gallstone Study was organized by the NIAMDD to conduct various studies leading up to a full-scale study of the

John M. Lachin The George Washington University Washington, DC

The National Cooperative Gallstone Study (NCGS) was conducted from 1973 to 1984 with funding from the National Institute of Arthritis, Metabolism and Digestive Diseases (NIAMDD), now the National Institute of Diabetes, Digestive and Kidney Diseases (NIDDK). The ultimate objective was to conduct a Phase III clinical trial of the effectiveness and safety of exogenous bile acid supplementation for treatment of cholesterol gallstones. The design and methods are presented in Reference 1 and the principal results in Reference 2. The function of the gallbladder is to serve as a reservoir for the storage of bile that is secreted by the liver into the gallbladder. In response to a meal, especially a fatty meal, the gallbladder contracts and deposits the bile into the intestine to assist with digestion. Bile consists of bile acids and lipids, among other elements. Hepatic excretion into bile is also one of the principal means by which lipids are excreted. One of the earliest lipidlowering drugs, cholestyramine, is a bile acid sequestrant that promotes clearance of lipids from circulating blood through secretion into bile. Bile is an aqueous medium, and bile acids and phospholipids (lecithin) maintain cholesterol and other lipids in solution. However, if bile becomes super-saturated with lipids, which means that the concentration of bile acids is inadequate to keep the available cholesterol in solution, then the cholesterol begins to precipitate, forming cholesterol gallstones (cholelithiasis). A minority of gallstones may also contain other precipitants, such as calcium, bilirubinate, or other pigments. Today, gallstones are diagnosed by an ultrasound examination. At the time that the NCGS was conducted, gallstones were diagnosed using an oral cholecyctogram (OCG)

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

NATIONAL COOPERATIVE GALLSTONE STUDY

effectiveness and safety of bile acid supplementation with chenodiol for treatment of gallstones. 1

THE NCGS ORGANIZATION

The NCGS was initiated by the NIAMDD through issuance of a Request For Proposals (RFP). Through an open competition, the Cedars-Sinai Medical Center of Los Angeles, CA, was named the Coordinating Center with Leslie Schoenfeld as Director and Jay Marks as Deputy Director (among others), both gastroenterologists, and Gloria Croke as Project Coordinator. Cedars-Sinai contracted with The Biostatistics Center of George Washington University (GWU) to serve as the Biostatistical Unit (BSU) of the Coordinating Center under the codirection of Jerome Cornfield and Larry Shaw. Cedars-Sinai and GWU jointly submitted the winning application. After Cornfield’s death, John Lachin became codirector. Supplemental RFPs issued by CedarsSinai led to the appointment of other central units, including a Central Radiology Unit, Central Serum Laboratory, Central Bile Laboratory, and a Central Morphology and Electron Microscopy Laboratory. 2

2.2 The Hepatic Morphology Study Because of the potential for hepatotoxicity, the FDA mandated that the first 100 subjects treated with chenodiol should undergo liver biopsy before and during treatment for up to two years. Since the NCGS planned to employ a low and a high dose of chenodiol (375 and 750 mg/day, respectively), an initial study of 128 subjects randomly allocated to these doses was conducted in which all subjects had a liver biopsy at baseline and again at 9 and 24 months of treatment. All biopsies were independently assessed using light microscopy by two pathologists at different institutions using a common protocol for the assessment of morphologic abnormalities (5). The study included masked duplicate readings to assess interpathologist and intrapathologist reliability of the detailed morphologic assessments. In addition, biopsies were also read by electron microscopy by one morphologist (6). The study did not include an untreated control group. However, both the light and electron microscopic evaluations showed an increase in the prevalence of lesions over time that were consistent with intrahepatic cholestasis and were unlikely to be a reflection of the natural history of gallstone disease.

STUDIES

2.1 Animal Toxicology

2.3 The Full-Scale Study

Since the objective was to conduct a largescale Phase III study using chenodiol, the U.S. Food and Drug Administration (FDA) required that various animal toxicology studies be conducted. The program of toxicology studies, including studies in monkeys, was organized by the Pharmacology Toxicology Committee appointed by the Coordinating Center. These studies showed that chenodiol was associated with liver enzyme increases, and resulted in hepatic injury in rhesus monkeys (3). Subsequently, however, it was shown that the injury in the monkey was attributed to biotransformation of chenodiol to lithocholic acid and that humans were protected from such injury through sulfation of the lithocholic acid that neutralizes its toxicity (4).

When the morphology (biopsy) study was fully enrolled, the full-scale study was launched (1). A total of 916 subjects were enrolled in 10 clinical centers in the United States and randomly assigned to receive either placebo (n = 305), the low dose of 375 mg/day (n = 360), or the high dose of 750 mg/day (n = 305), double-masked. Subjects had predominantly cholesterol gallstones present on a baseline OCG and were generally in good health. Subjects were to be treated for two years with frequent monitoring of symptoms, frequent laboratory testing including lipids and liver enzymes, periodic follow-up OCGs, and periodic collection of gallbladder bile through duodenal intubation.

NATIONAL COOPERATIVE GALLSTONE STUDY

The results (2) showed a dose-dependent effect on complete confirmed gallstone dissolution, the primary outcome; the cumulative incidence over two years being 13.5% with the high dose, 5.2% with the low dose, and 0.8% with placebo. The cumulative incidence of partial (or complete) dissolution by at least 50% occurred in 40.8%, 23.6%, and 11% in the three groups, respectively. Although statistically significant, these modest effects were of marginal clinical significance. Clinically significant hepatotoxicity occurred in 3% of those treated with the high dose and 0.4% of those treated with the low dose or placebo. Gallstone dissolution with treatment occurred more frequently among women, those with lower body weight, those with small or floating gallstones, and those with higher cholesterol levels at baseline. There were no differences among the treatment groups in the incidence of biliary symptoms (pain, cholangitis, cholecyctitis) or the need for surgery (cholecystectomy). The investigators concluded that highdose chenodiol treatment for up to two years was an ‘‘appropriate therapy for dissolution of gallstones in selected patients who are informed of the risks and benefits’’ (2). The investigators’ evaluation of the study is presented in Reference 7. 2.4 The Recurrence Study Patients who achieved complete dissolution in the main study were then offered the opportunity to receive additional doublemasked therapy with either the low dose or placebo. The cumulative incidence of gallstone recurrence was 27% after up to 3.5 years of treatment with no difference between treatment groups (8). 2.5 Open-Label Treatment The protocol and informed consent stated that subjects randomly assigned to placebo would be offered the opportunity to receive treatment with chenodiol at the conclusion of the study. The final phase consisted of openlabel treatment for up to two years of those subjects originally treated with placebo who opted to receive active therapy.

3

3 MAJOR ISSUES 3.1 Clinical Centers The initial complement of 10 clinical centers was selected after an open competition (9). Early in the study it was clear that two of the clinical centers could not meet the recruitment targets and were promptly replaced. Recruitment, however, proved difficult for all clinical centers, and aggressive steps were taken to achieve the recruitment goals (10). 3.2 The Biospy Study Prior to initiation of the full-scale NCGS, the FDA required that a preliminary Phase II study be conducted with liver biopsies before and after treatment with chenodiol. The NCGS investigators strongly objected to the implementation of this recommendation as part of the main study, in part because a needle liver biopsy was considered to have substantial risk and would be unethical to administer to patients treated with placebo, and in part because it would not be scientifically possible to differentiate the effects of therapy from the natural history of gallstone disease progression in the absence of an untreated (and biopsied) control group. Substantial negotiations among the NIAMDD, the NCGS investigators, and the FDA led to the compromise that the major study could be launched after the enrollment of the uncontrolled biopsy study was completed (9). 3.3 Costs Over the 8-year period from 1973 to 1981 (not including the open-label follow-up), the total cost was $11.2 million. Of this cost, 43% was allocated to the clinical centers, 13% to the Coordinating Center, 19% to the Biostatistical Unit, 10% to the central laboratories and units, 5% to ancillary studies, 4% to animal toxicology studies, 3% to the purchase of drug, and 3% to committee expenses (7). 4 BIOSTATISTICAL CONSIDERATIONS 4.1 Randomization The NCGS randomization sequence was stratified by clinical center and was implemented using a technique (1) developed by Larry Shaw, the BSU co-PI with Cornfield,

4

NATIONAL COOPERATIVE GALLSTONE STUDY

that later became known as ‘‘big-stick’’ randomization (11). This technique is a variation on the biased-coin design (12) but less restrictive in that complete randomization (a coin toss) is employed except where the imbalance between the numbers assigned to each group reaches an unacceptable level. In the NCGS, this level was defined as a deviation from perfect balance assigned to any one group that was greater than one standard deviation under the corresponding binomial (1). For example, for equal allocations to two groups with probability 12 , after n allo√ cations, the standard deviation is 12 n. On the next allocation, one tosses a coin if the number assigned to both groups falls within √ 1 2 (n ± n); otherwise, the next allocation is assigned to whichever group has the smaller number. 4.2 Sample Size Evaluation The sample size for the biopsy study was dictated by the FDA that requested sequential biopsy on 100 subjects. To ensure that close to this number received 24 months of treatment and were biopsied, the number enrolled was increased to 128. Since there was no untreated control group, the power of the study was then described in terms of the change from baseline to follow-up in the proportion of subjects with an abnormality (1). The sample size for the three group main study was based on the test for the difference in proportions achieving gallstone dissolution using pairwise tests for each of the two dose groups versus control (1). The power of the study was also assessed in terms of the ability to detect a treatment group by covariate interaction or heterogeneity of the treatment group effect between two strata. For the comparison of means of K active treatment groups versus a control, the Dunnett (13) procedure is widely used for which the most powerful comparisons are provided √ using the ‘‘square root rule’’ in which K subjects are assigned to the control group for every one subject assigned to a treatment group. For the case of two active treatments, as in the NCGS, this would yield alloca√ tion ratios of 2 : 1 : 1 or 1.41:1:1 for control and the two active treatment groups, respectively. However, it was not known whether

these allocations were also optimal for comparisons of proportions among K active treatment groups versus control. Lachin (14) explored the power of such tests and determined that there was no uniformly optimal allocation ratio for tests for proportions. Rather the optimal ratio depended on the nature of the underlying null and alternative hypotheses, and on average, equal allocations seemed to be robust. 4.3 Outcome Evaluation The primary outcome of the study was the presence or absence of gallstones on an OCG and, if present, the number, total volume, and total surface area. The Central Radiology Unit developed a novel technique for providing these measurements. The gallbladder and gallstones were visualized on a standard flat plate X-ray of the abdomen taken with the patient recumbent. Thus, the actual distance of the gallbladder and stones from the table surface (and film plane) varied as a function of patient girth. To allow for the resulting magnification differential, each clinical center was provided with a half-inch ball bearing that was taped to the patient’s side at the level of the gallbladder. The resulting image of the ball bearing allowed the area of the gallbladder and gallstones on the X-ray to be calibrated to that of the image of the ball bearing (15). A computer program was then written to allow the reader to identify the ‘‘edge’’ of the image of the gallbladder and individual gallstones using a computer monitor, and to then compute areas. Biliary bile acids were measured in a Central Bile Laboratory and other biochemistries measured in a Central Serum Laboratory. 4.4 External Quality Assurance The reproducibility and reliability of all outcome assessments was monitored to assure high-quality data. For the Central Radiology Unit, individual X-rays were randomly selected for masked duplicate re-reading within the unit (2). For the central laboratories, split duplicate specimens were collected in the clinical centers and then shipped to the laboratories for a masked duplicate assessment. The detailed procedures and descriptions of the statistical calculations of reliability are presented in Reference 16.

NATIONAL COOPERATIVE GALLSTONE STUDY

4.5 External Monitoring of the BSU Finally, an external monitoring committee was appointed to site visit the Biostatistical Unit (BSU) periodically, and over a period of years, it reviewed and critiqued virtually all functions of the unit. Details are presented in Reference 17. REFERENCES 1. J. M. Lachin, J. Marks, and L. J. Schoenfield, the Protocol Committee and the NCGS Group. Design and methodological considerations in the National Cooperative Gallstone Study: A multi-center clinical trial. Controlled Clinical Trials, 1981; 2: 177–230. 2. L. J. Schoenfield, J. M. Lachin, the Steering Committee and the NCGS Group. National Cooperative Gallstone Study: A controlled trial of the efficacy and safety of chenodeoxycholic acid for dissolution of gallstones. Ann. Internal Med. 1981; 95: 257–282. 3. R. Okun, L. I. Goldstein, G. A. Van Gelder, EI. Goldenthal, F. X. Wazeter, R. G. Giel, National Cooperative Gallstone Study: nonprimate toxicology of chenodeoxycholic acid. J. Toxicol. Environ. Health, 1982; 9: 727–741. 4. J. W. Marks, S. O. Sue, B. J. Pearlman, G. G. Banorris, P. Varady, J. M. Lachin, and L. J. Schoenfield. Sulfation of lithocholate as a possible modifier of chenodeoxycholic acidinduced elevations of serum transaminase in patients with gallstones. J. Clin. Invest. 1981; 68: 1190–1196. 5. R. L. Fisher, D. W. Anderson, J. L. Boyer, K. Ishak, G. Klatskin, J. M. Lachin, and M. J. Phillips, and the Steering Committee for the NCGS Group. A prospective morphologic evaluation of hepatic toxicity of chenodeoxycholic acid in patients with cholelithiasis: The National Cooperative Gallstone Study. Hepatology 1982; 2: 187–201. 6. M. J. Phillips, R. L. Fisher, D. W. Anderson, S. Lan, J. M. Lachin, J. L. Boyer, and the Steering Committee for the NCGS Group. Ultrastructural evidence of intrahepatic cholestasis before and after chenodeoxycholic acid (CDCA) therapy in patients with cholelithiasis: The National Cooperative Gallstone Study (NCGS). Hepatology 1983; 3: 209–220. 7. L. J. Schoenfield, S. M. Grundy, A. F. Hofmann, J. M. Lachin, J. L. Thistle, and M. P. Tyor, for the NCGS. The National Cooperative Gallstone Study viewed by its investigators. Gastroenterology 1983; 84: 644–648.

5

8. J. W. Marks, S. P. Lan, The Steering Committee, and The National Cooperative Gallstone Study Group. Low dose chenodiol to prevent gallstone recurrence after dissolution therapy. Ann. Intern. Med. 1984; 100: 376–381. 9. J. Marks, G. Croke, N. Gochman, A. F. Hofmann, J. M. Lachin, L. J. Schoenfield, and M. P. Tyor, and the NCGS Group. Major issues in the organization and implementation of the National Cooperative Gallstone Study (NCGS). Controlled Clinical Trials 1984; 5: 1–12. 10. G. Croke. Recruitment for the the National Cooperative Gallstone Study. Clin. Pharm. Ther, 1979; 25: 691–674. 11. J. F. Soares and C. F Wu. Some restricted randomization rules in sequential designs. Communications in Statistics Theory and Methods 1982; 12: 2017–2034. 12. B. Efron. Forcing a sequential experiment to be balanced. Biometrika 1971; 58: 403–417. 13. C. W. Dunnett. A multiple comparison procedure for comparing several treatments with a control. J. Am Stat Assoc. 1955; 50: 1096–1121. 14. J. M. Lachin. Sample size determinations for r x c comparative trials. Biometrics, 1977; 33: 315–324. 15. E. C. Lasser, J. R. Amberg, N. A. Baily, P. Varady, J. M. Lachin, R. Okun, and L. J. Schoenfield. Validation of a computer-assisted method for estimating the number and volume of gallstones visualized by cholecystography. Invest. Radiol 1981; 16: 342–347. 16. R. L. Habig, P. Thomas, K. Lippel, D. Anderson, and J. M. Lachin. Central laboratory quality control in the National Cooperative Gallstone Study. Controlled Clinical Trials 1983: 4: 101–123. 17. P. L. Canner, L. C. Gatewood, C. White, J. M. Lachin, and L. J. Schoenfield. External monitoring of a data coordinating center: Experience of the National Cooperative Gallstone Study. Controlled Clinical Trials 1987: 8: 1–11.

FURTHER READING J. W. Marks, G. G. Bonorris, A Chung, M. J. Coyne, R. Okun, J. M. Lachin, and L. J. Schoenfield. Feasibility of low-dose and intermittent chenodeoxycholic acid therapy of gallstones. Am. J. Digest. Dis. 1977; 22: 856–860. J. J. Albers, S. M. Grundy, P. A. Cleary, D. M. Small, J. M. Lachin, L. J. Schoenfield, and the

6

NATIONAL COOPERATIVE GALLSTONE STUDY

National Cooperative Gallstone Study Group. National Cooperative Gallstone Study. The effect of chenodeoxycholic acid on lipoproteins and apolipoproteins. Gastroenterology, 1982; 82: 638–646. A. F. Hofmann, S. M. Grundy, J. M. Lachin, S. P. Lan, et al. Pretreatment biliary lipid composition in white patients with radiolucent gallstones in the National Cooperative Gallstone Study. Gastroenterology 1982; 83: 738–752. A. F. Hofmann and J. M. Lachin. Biliary bile acid composition and cholesterol saturation. Gastroenterology 1983; 84: 1075–1077. J. M. Lachin, L. J. Schoenfield, and the National Cooperative Gallstone Study Group. Effects of dose relative to body weight in the National Cooperative Gallstone Study: A fixed-dose trial. Controlled Clinical Trials 1983; 4: 125–131. S. M. Grundy, S. Lan, J. Lachin, the Steering Committee and the National Cooperative Gallstone Study Group. The effects of chenodiol on biliary lipids and their association with gallstone dissolution in the National Cooperative Gallstone Study (NCGS). J. Clin. Invest. 1984; 73: 1156–1166. J. L. Thistle, P. A. Cleary, J. M. Lachin, M. P. Tyor, T. Hersh, The Steering Committee, and The National Cooperative Gallstone Study Group. The natural history of cholelithiasis: The National Cooperative Gallstone Study. Ann. Intern. Med. 1984; 101: 171–175.

F. Stellard, P. D. Klein, A. F. Hofmann, and J. M. Lachin. Mass spectrometry identification of biliary bile acids in bile from patients with gallstones before and during treatment with chenodeoxycholic acid. J. Lab. Clin. Med. 1985; 105: 504–513.

CROSS-REFERENCES Biased-coin Randomization Clinical Trial/Study Conduct Multiple Comparisons Orphan Drugs Phase III Trials Placebo-Controlled Trial Quality Assurance Quality Control Reproducibility Sample Size for Comparing Proportions (Superiority and Non-Inferiority)

NATIONAL EYE INSTITUTE (NEI) The U.S. National Eye Institute (NEI) conducts and supports research, training, health information dissemination, and other programs with respect to blinding eye diseases, visual disorders, mechanisms of visual function, preservation of sight, and the special health problems of individuals who are visually impaired or blind. Vision research is supported by the NEI through research grants and training awards made to scientists at more than 250 medical centers, hospitals, universities, and other institutions across the United States and around the world. The NEI also conducts laboratory and patient-oriented research at its own facilities located on the U.S. National Institutes of Health (NIH) campus in Bethesda, Maryland. Another part of the NEI mission is to conduct public and professional education programs that help prevent blindness and reduce visual impairment. To meet these objectives, the NEI has established the National Eye Health Education Program, a partnership of more than 60 professional, civic, and voluntary organizations and government agencies concerned with eye health. The program represents an extension of the NEI’s support of vision research, where results are disseminated to health professionals, patients, and the public.

This article was modified from the website of the National Institutes of Health (http://www. nih.gov/about/almanac/organization/NEI.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

• Coordinates relevant activities with

NATIONAL HEART, LUNG, AND BLOOD INSTITUTE (NHLBI)

other research institutes and all federal health programs in the covered areas, including the causes of stroke. • Conducts educational activities, including development and dissemination of materials for health professionals and the public in the covered areas, with emphasis on prevention. • Maintains continuing relationships with institutions and professional associations, and with international, national, state, and local officials as well as voluntary agencies and organizations working in the covered areas. • Oversees management of the Women’s Health Initiative study.

The National Heart, Lung, and Blood Institute (NHLBI): • Provides leadership for a national pro-

gram in diseases of the heart, blood vessels, lungs, and blood; sleep disorders; and blood resources. • Plans, conducts, fosters, and supports

an integrated and coordinated program of basic research, clinical investigations and trials, observational studies, and demonstration and education projects related to the causes, prevention, diagnosis, and treatment of heart, blood vessel, lung, and blood diseases, and sleep disorders conducted in its own laboratories and by scientific institutions and individuals supported by research grants and contracts. • Plans and directs research in development, trials, and evaluation of interventions and devices related to the prevention of diseases and disorders in the covered areas and the treatment and rehabilitation of patients suffering from such conditions.

1 DIVISION OF CARDIOVASCULAR DISEASES (DCVD) The Division of Cardiovascular Diseases (DCVD) promotes opportunities to translate promising scientific and technological advances from discovery through preclinical studies to networks and multisite clinical trials. It designs, conducts, supports, and oversees research on the causes and prevention and treatment of diseases and disorders such as atherothrombosis, coronary artery disease, myocardial infarction and ischemia, heart failure, arrhythmia, sudden cardiac death, adult and pediatric congenital heart disease, cardiovascular complications of diabetes and obesity, and hypertension.

• Conducts research on clinical use of

blood and all aspects of the management of blood resources. • Supports research training and career

development of new and established researchers in fundamental sciences and clinical disciplines to enable them to conduct basic and clinical research related to heart, blood vessel, lung, and blood diseases; sleep disorders; and blood resources through individual and institutional research training awards and career development awards.

2 DIVISION OF PREVENTION AND POPULATION SCIENCES (DPPS) The Division of Prevention and Population Sciences (DPPS) supports and provides leadership for population-based and clinic-based research on the causes, prevention, and clinical care of cardiovascular, lung, and blood diseases and sleep disorders. Research includes a broad array of epidemiologic studies to describe disease and risk factor patterns in populations and to identify risk factors for

This article was modified from the website of the National Institutes of Health (http://www. nih.gov/about/almanac/organization/NHLBI.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

NATIONAL HEART, LUNG, AND BLOOD INSTITUTE (NHLBI)

disease; clinical trials of interventions to prevent disease; studies of genetic, behavioral, sociocultural, and environmental influences on disease risk and outcomes; and studies of the application of prevention and treatment strategies to determine how to improve clinical care and public health. 3

DIVISION OF LUNG DISEASES (DLD)

The Division of Lung Diseases (DLD) supports basic research, clinical trials, national pulmonary centers, technological development, and application of research findings. Activities focus on understanding the structure and function of the respiratory system, increasing fundamental knowledge of mechanisms associated with pulmonary disorders, and applying new findings to evolving treatment strategies for patients.

NATIONAL HUMAN GENOME RESEARCH INSTITUTE (NHGRI) The National Human Genome Research Institute (NHGRI), which was originally established in 1989 as the National Center for Human Genome Research, became an institute of the U.S. National Institutes of Health (NIH) in 1997. The NHGRI led the NIH’s contribution to the International Human Genome Project, which successfully completed the sequencing of the 3 billion base pairs that make up the human genome in April 2003. The NHGRI mission has evolved over the years to encompass a broad range of studies aimed at understanding the structure and function of the human genome and its role in health and disease. To that end, the NHGRI supports the development of resources and technology that will accelerate genomeresearch and its application to human health as well as the study of the ethical, legal, and social implications of genome research. The NHGRI also supports the training of investigators and the dissemination of genome information to the public and to health professionals. The NHGRI is organized into three main divisions: the Office of the Director, which provides guidance to scientific programs and oversees general operations; the Division of Extramural Research, which supports and administers genomic research; and the Division of Intramural Research, which comprises the in-house genetics research laboratories. Research guidance and guidance related to NHGRI grants comes from the National Advisory Council for Human Genome Research, which meets three times a year. Members include representatives from health and science disciplines, public health, social sciences, and the general public. Portions of the council meetings are open to the public. This article was modified from the website of the National Institutes of Health (http://www. nih.gov/about/almanac/organization/NHGRI.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NATIONAL INSTITUTE OF ALLERGY AND INFECTIOUS DISEASE (NIAID)

clinical trials of hematopoietic stem cell transplantation for treating severe autoimmune disorders.

The National Institute of Allergy and Infectious Diseases (NIAID) conducts and supports research to study the causes of allergic, immunologic, and infectious diseases, and to develop better means of preventing, diagnosing, and treating these illnesses. The NIAID runs a pediatric allergy clinic at the U.S. National Institutes of Health (NIH) Clinical Center that serves as a focal point for translational research conducted in collaboration with NIAID intramural laboratories and clinical trials of novel therapies.

3

1

4

In collaboration with industry, academia, and other government agencies, the NIAID has established research programs to facilitate drug development, including capacity databases to screen compounds for their potential use as therapeutic agents, facilities to conduct preclinical testing of promising drugs, and clinical trials networks to evaluate the safety and efficacy of drugs and therapeutic strategies in humans.

GENETICS AND TRANSPLANTATION

The NIAID’s basic immunology and genetics research seeks to define the effects of gene expression on immune function and to determine the manner in which the products of gene expression control the immune response to foreign substances such as transplanted organs and cells. Research programs in genetics and transplantation include human leukocyte antigen (HLA) region genetics in immune-mediated diseases, the genomics of transplantation, and clinical trials in organ transplantation. 2

DRUG RESEARCH AND DEVELOPMENT

ANTIMICROBIAL RESISTANCE

The NIAID-supported clinical trials networks with capacity to assess new antimicrobials and vaccines relevant to drug-resistant infections include the Adult AIDS Clinical Trials Groups, the Bacteriology and Mycology Study Group, the Collaborative Antiviral Study Group, and Vaccine and Treatment Evaluation Units. 5 DIVISION OF MICROBIOLOGY AND INFECTIOUS DISEASES The Division of Microbiology and Infectious Diseases (DMID) supports extramural research to control and prevent diseases caused by virtually all human infectious agents except human immunodeficiency virus (HIV). The DMID supports a wide variety of projects spanning the spectrum from basic biomedical research through applied research to clinical trials to test the safety and efficacy of new disease prevention strategies.

IMMUNE-MEDIATED DISEASES

The NIAID conducts and supports basic, preclinical, and clinical research on immunemediated diseases, including asthma and allergic diseases, autoimmune disorders, primary immunodeficiency diseases, and the rejection of transplanted organs, tissues, and cells. Efforts are underway to evaluate the safety and efficacy of tolerance induction strategies for treating immune-mediated diseases as well as to assess the efficacy through This article was modified from the website of the National Institutes of Health (http://www. nih.gov/about/almanac/organization/NIAID.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NATIONAL INSTITUTE OF ARTHRITIS AND MUSCULOSKELETAL AND SKIN DISEASES (NIAMS) The U.S. National Institute of Arthritis and Musculoskeletal and Skin Diseases (NIAMS) was established in 1986. The mission of NIAMS is to support research into the causes, treatment, and prevention of arthritis and musculoskeletal and skin diseases; the training of basic and clinical scientists to carry out this research; and the dissemination of information on research progress in these diseases. The Institute also conducts and supports basic research on the normal structure and function of joints, muscles, bones, and skin. Basic research involves a wide variety of scientific disciplines, including immunology, genetics, molecular biology, structural biology, biochemistry, physiology, virology, and pharmacology. Clinical research includes rheumatology, orthopaedics, dermatology, metabolic bone diseases, heritable disorders of bone and cartilage, inherited and inflammatory muscle diseases, and sports and rehabilitation medicine. The Institute’s Genetics and Clinical Studies Program supports genetic studies in rheumatic diseases, both in animal models and in humans; clinical trials and complex clinical studies, including epidemiology, outcomes, and prevention of rheumatic and related diseases; and research on Lyme disease and infection-related arthritis.

This article was modified from the website of the National Institutes of Health (http://www.nih. gov/about/almanac/organization/NIAMS.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NATIONAL INSTITUTE OF BIOMEDICAL IMAGING AND BIOENGINEERING (NIBIB) The mission of the U.S. National Institute of Biomedical Imaging and Bioengineering (NIBIB) is to improve health by leading the development of and accelerating the application of biomedical technologies. The Institute integrates the physical and engineering sciences with the life sciences to advance basic research and medical care: • Supporting research and development

• • • •

•

•

of new biomedical imaging and bioengineering techniques and devices to fundamentally improve the detection, treatment, and prevention of disease. Enhancing existing imaging and bioengineering modalities. Supporting related research in the physical and mathematical sciences. Encouraging research and development in multidisciplinary areas. Supporting studies to assess the effectiveness and outcomes of new biologics, materials, processes, devices, and procedures. Developing technologies for early disease detection and assessment of health status. Developing advanced imaging and engineering techniques for conducting biomedical research at multiple scales.

This article was modified from the website of the National Institutes of Health (http://www. nih.gov/about/almanac/organization/NIBIB.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

• Human growth and development is a

NATIONAL INSTITUTE OF CHILD HEALTH AND HUMAN DEVELOPMENT (NICHD)

lifelong process that has many phases and functions. Much of the research in this area focuses on cellular, molecular, and developmental biology to build understanding of the mechanisms and interactions that guide a single fertilized egg through its development into a multicellular, highly organized adult organism. • Learning about the reproductive health of women and men and educating people about reproductive practices is important to both individuals and societies. Institute-supported basic, clinical, and epidemiologic research in the reproductive sciences seeks to develop knowledge that enables women and men to overcome problems of infertility and to regulate their fertility in ways that are safe, effective, and acceptable for various population groups. Institute-sponsored behavioral and social science research in the population field strives to understand the causes and consequences of reproductive behavior and population change. • Developing medical rehabilitation interventions can improve the health and the well-being of people with disabilities. Research in medical rehabilitation seeks to develop improved techniques and technologies for the rehabilitation of individuals with physical disabilities resulting from diseases, disorders, injuries, or birth defects.

The mission of the National Institute of Child Health and Human Development (NICHD) is to ensure that every person is born healthy and wanted; that women suffer no harmful effects from the reproductive process; and that all children have the chance to fulfill their potential to live healthy and productive lives, free from disease or disability; and to ensure the health, productivity, independence, and well-being of all people through optimal rehabilitation. In pursuit of this mission, the NICHD conducts and supports laboratory research, clinical trials, and epidemiologic studies that explore health processes; examines the impact of disabilities, diseases, and defects on the lives of individuals; and sponsors training programs for scientists, doctors, and researchers to ensure that NICHD research can continue. NICHD research programs incorporate the following concepts: • Events that happen before and through-

out pregnancy as well as during childhood have a great impact on the health and the well-being of adults. The Institute supports and conducts research to advance knowledge of pregnancy, fetal development, and birth for developing strategies that prevent maternal, infant, and childhood mortality and morbidity; to identify and promote the prerequisites of optimal physical, mental, and behavioral growth and development through infancy, childhood, and adolescence; and to contribute to the prevention and amelioration of mental retardation and developmental disabilities.

The Institute also supports research training across all its programs, with the intent of adding to the cadre of trained professionals who are available to conduct research in areas of critical public health concern. In addition, an overarching responsibility of the NICHD is to disseminate information that emanates from Institute research programs to researchers, practitioners, other healthcare professionals, and the public.

This article was modified from the website of the National Institutes of Health (http://www. nih.gov/about/almanac/organization/NICHD.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NATIONAL INSTITUTE OF DENTAL AND CRANIOFACIAL RESEARCH (NIDCR)

The center has six components: Clinical Trials Program, Dental Practice-Based Research Networks, Epidemiology Research Program, Health Disparities Research Program, Health Promotion and CommunityBased Research Program, and the Basic and Applied Behavioral/Social Science Research Program.

The mission of the National Institute of Dental and Craniofacial Research (NIDCR) is to improve oral, dental, and craniofacial health through research, research training, and the dissemination of health information. We accomplish our mission by:

2 DIVISION OF EXTRAMURAL ACTIVITIES, SCIENTIFIC REVIEW BRANCH

• Performing and supporting basic and

The Scientific Review Branch of the Division of Extramural Activities coordinates the initial scientific peer review of applications for the following mechanisms of support: center research grants, program project grants, small research grants, research conference grants, institutional training grants, shortterm training and fellowship grants, Physician Scientist Awards for Dentists, Dentist Scientist Awards, requests for applications issued by the NIDCR, certain investigatorinitiated clinical trials, cooperative agreements, and all proposals for research and development contracts.

clinical research. • Conducting and funding research training and career development programs to ensure an adequate number of talented, well-prepared, and diverse investigators. • Coordinating and assisting relevant research and research-related activities among all sectors of the research community. • Promoting the timely transfer of knowledge gained from research and its implications for health to the public, health professionals, researchers, and policymakers. 1

CENTER FOR CLINICAL RESEARCH

The Center for Clinical Research (CCR) supports and conducts patient-oriented and population-based research, including clinical trials, practice-based networks, epidemiology, and health-disparity research in all areas of program interest to the NIDCR. Providing statistical support for Institute centers and divisions, the CCR develops and supports programs to foster diversity in the scientific workforce as well as clinical research activities aimed at the health of vulnerable and special needs populations. This article was modified from the website of the National Institutes of Health (http://www. nih.gov/about/almanac/organization/NIDCR.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NATIONAL INSTITUTE OF DIABETES AND DIGESTIVE AND KIDNEY DISEASES (NIDDK)

of surrogate markers for use in clinical trials for the prevention or treatment of type 2 diabetes, cellular therapies for the treatment of type 2 diabetes, and improving the care of patients with type 2 diabetes. The Type 1 Diabetes Clinical Trials Program supports large, multicenter clinical trials conducted under cooperative agreements or contracts. For example, the Diabetes Prevention Trial Type 1 (DPT-1) was aimed at determining whether it was possible to prevent or delay the onset of type 1 diabetes in individuals who are at immunologic, genetic, and/or metabolic risk. The program also supports future clinical trials that are part of the Type 1 Diabetes TrialNet, which are intervention studies to prevent or slow the progress of type 1 diabetes, and natural history and genetics studies in populations screened for or enrolled in these studies. The program also supports the Epidemiology of Diabetes Interventions and Complications (EDIC) study, an epidemiologic follow-up study of the patients previously enrolled in the Diabetes Control and Complications Trial (DCCT). The Type 2 Diabetes Clinical Trials Program supports large, multicenter clinical trials conducted under cooperative agreements or contracts. For example, the Diabetes Prevention Program (DPP) focuses on testing lifestyle and pharmacologic intervention strategies in individuals at genetic and metabolic risk for developing type 2 diabetes to prevent or delay the onset of this disease. The Gene Therapy and Cystic Fibrosis Centers Program supports three types of centers: Gene Therapy Centers (P30), Cystic Fibrosis Research Centers (P30), and Specialized Centers for Cystic Fibrosis Research (P50). Gene Therapy Centers provide shared resources to a group of investigators to facilitate development of gene therapy techniques and to foster multidisciplinary collaboration in the development of clinical trials for the treatment of cystic fibrosis and other genetic metabolic diseases. Cystic Fibrosis Research Centers (P30) and Specialized Centers for Cystic Fibrosis Research (P50) provide resources and support research on many

The National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) conducts and supports research on many of the most serious diseases affecting public health. The institute supports much of the clinical research on the diseases of internal medicine and related subspecialty fields as well as many basic science disciplines. The Institute’s Division of Intramural Research encompasses the broad spectrum of metabolic diseases such as diabetes, obesity, inborn errors of metabolism, endocrine disorders, mineral metabolism, digestive and liver diseases, nutrition, urology and renal disease, and hematology. Basic research studies include: biochemistry, biophysics, nutrition, pathology, histochemistry, bioorganic chemistry, physical chemistry, chemical and molecular biology, and pharmacology. The NIDDK extramural research is organized into four divisions: Diabetes, Endocrinology and Metabolic Diseases; Digestive Diseases and Nutrition; Kidney, Urologic and Hematologic Diseases; and Extramural Activities. The Institute supports basic and clinical research through investigator-initiated grants, program project and center grants, and career development and training awards. The Institute also supports research and development projects and large-scale clinical trials through contracts. The Clinical Research in Type 2 Diabetes Program focuses on patient-oriented research (i.e., clinical studies and small clinical trials) related to pharmacologic interventions and/or lifestyle interventions to prevent or treat type 2 diabetes, including studies relevant to new drug development, development This article was modified from the website of the National Institutes of Health (http://www. nih.gov/about/almanac/organization/NIDDK.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

NATIONAL INSTITUTE OF DIABETES AND DIGESTIVE AND KIDNEY DISEASES (NIDDK)

aspects of the pathogenesis and treatment of cystic fibrosis. The Clinical Trials in Digestive Diseases Program supports patient-oriented clinical research focusing on digestive diseases. Small clinical studies (pilot), planning grants, or phase III clinical trials may be appropriate to this program. The small clinical studies focus on research that is innovative and/or potentially of high impact that will lead to full-scale clinical trials. Phase III clinical trials usually are multicenter and involve several hundred participants who are randomized to two or more treatments, one of which is usually a placebo. The aim of the trial is to provide evidence for support of, or a change in, health policy or standard of care. The interventions/treatments may include pharmacologic, nonpharmacologic, and behavioral interventions given for disease prevention, prophylaxis, diagnosis, or therapy. The Clinical Trials in Liver Disease Program supports patient-oriented clinical research in liver diseases to evaluate one or more experimental intervention(s) in comparison with a standard treatment and/or placebo control among comparable groups of patients. Experimental interventions may include pharmacologic, nonpharmacologic, and behavioral interventions given for disease prevention, prophylaxis, diagnosis, or therapy. Either pilot studies or phase III trials may be appropriate. The Obesity Prevention and Treatment Program supports research that focuses on the prevention and treatment of obesity and the overweight condition in humans. Prevention includes primary and secondary approaches to prevent the initial development of obesity through control of inappropriate weight gain and increases in body fat, weight maintenance among those at risk of becoming overweight, and prevention of weight regain once weight loss has been achieved. Treatment includes clinical trials evaluating approaches to lose weight or maintain weight loss, including, but not limited to, behavioral, pharmacologic, and surgical approaches. Look AHEAD: Action for Health in Diabetes is a clinical trial recruiting 5000 obese individuals with type 2 diabetes into an 11.5 year study that investigates the long-term

health consequences of interventions designed to achieve and sustain weight loss. The primary outcome of the trial is cardiovascular events: heart attack, stroke, and cardiovascular death. The Clinical Trials in Nutrition Program supports clinical research on nutrition and eating disorders, focusing on metabolic and/or physiologic mechanisms. Small clinical studies (pilot), planning grants, or phase III clinical trials may be appropriate to this program. The small clinical studies focus on research that is innovative and/or potentially of high impact that will lead to full-scale clinical trials. Phase III clinical trials usually are multicenter and involve several hundred participants who are randomized to two or more treatments, one of which is a placebo.

NATIONAL INSTITUTE OF ENVIRONMENTAL HEALTH SCIENCE (NIEHS) The mission of the U.S. National Institute of Environmental Health Science (NIEHS) is to reduce the burden of human illness and disability by understanding how the environment influences the development and progression of human disease. To have the greatest impact on preventing disease and improving human health, the NIEHS focuses on basic science, disease-oriented research, global environmental health, and multidisciplinary training for researchers: • Funding

extramural research and training via grants and contracts to scientists, environmental health professionals, and other groups worldwide. • Conducting intramural research at the NIEHS facility and in partnership with scientists at universities and hospitals. • Providing toxicological testing and test validation through the National Toxicology Program. • Maintaining outreach and communications programs that provide reliable health information to the public and scientific resources to researchers.

This article was modified from the website of the National Institutes of Health (http://www.nih. gov/about/almanac/organization/NIEHS.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NATIONAL INSTITUTE OF GENERAL MEDICAL SCIENCES (NIGMS)

Certain NIGMS training programs address areas in which there is a particularly serious need for well-prepared scientists. One of these, the Medical Scientist Training Program, provides investigators who can bridge the gap between basic and clinical research by supporting research training leading to the combined M.D.–Ph.D. degree. Other programs train scientists to conduct research in the rapidly growing field of biotechnology and at the interface between the fields of chemistry and biology. The NIGMS also has a Pharmacology Research Associate Program, in which postdoctoral scientists receive training in the NIH or Food and Drug Administration (FDA) laboratories and clinics. In recent years, the NIGMS has launched initiatives in such cutting-edge areas as structural genomics (the Protein Structure Initiative), pharmacogenetics, collaborative research initiatives (which includes ‘‘glue grants’’), and computational modeling of infectious disease outbreaks. The NIGMS is also participating in the NIH Roadmap for Medical Research, a series of far-reaching initiatives that seek to transform the nation’s biomedical research capabilities and speed the movement of research discoveries from the bench to the bedside. Each year, NIGMS-supported scientists make major advances in understanding fundamental life processes. In the course of answering basic research questions, these investigators also increase our knowledge about the mechanisms and pathways involved in certain diseases. Other grantees develop important new tools and techniques, many of which have medical applications. In recognition of the significance of their work, a number of NIGMS grantees have received the Nobel Prize and other high scientific honors.

The National Institute of General Medical Sciences (NIGMS) was established in 1962, and in fiscal year 2006, its budget was $1.9 billion. The vast majority of this money funds grants to scientists at universities, medical schools, hospitals, and research institutions throughout the country. At any given time, the NIGMS supports over 4400 research grants— about 10% of the grants that are funded by the U.S. National Institutes of Health as a whole—and the NIGMS also supports approximately 25% of the trainees who receive assistance from the NIH. Primarily, the NIGMS supports basic biomedical research that lays the foundation for advances in disease diagnosis, treatment, and prevention. The NIGMS is organized into divisions and a center that support research and research training in basic biomedical science fields. One division has the specific mission of increasing the number of underrepresented minority biomedical and behavioral scientists. The Institute places great emphasis on the support of individual, investigatorinitiated research grants. It funds a limited number of research center grants in selected fields, including structural genomics, trauma and burn research, and the pharmacologic sciences. In addition, NIGMS funds several important resources for basic scientists. The Institute’s training programs help provide the most critical element of good research: well-prepared scientists. The NIGMS research training programs recognize the interdisciplinary nature of biomedical research today and stress approaches to biological problems that cut across disciplinary and departmental lines. Such experience prepares trainees to pursue creative research careers in a wide variety of areas. This article was modified from the website of the National Institutes of Health (http://www.nih. gov/about/almanac/organization/NIGMS.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NATIONAL INSTITUTE OF MENTAL HEALTH (NIMH)

mental health clinical sciences and other disciplines urgently needed in studies of mental illness and the brain. Another important part of this research is to eliminate the effects of disparities in the availability of and access to high quality mental health services. These disparities, which impinge on the mental health status of all Americans, are felt in particular by many members of ethnic/cultural, minority groups, and by women, children, and elderly people.

Mental disorders occur across the lifespan, from very young childhood into old age, and in the United States, mental disorders collectively account for more than 15% of the overall ‘‘burden of disease’’— a term that encompasses both premature death and disability associated with mental illness. The mission of the National Institute of Mental Health (NIMH) is to reduce the burden of mental illness and behavioral disorders through research on mind, brain, and behavior. Investments made over the past 50 years in basic brain and behavioral science have positioned NIMH to exploit recent advances in neuroscience, molecular genetics, behavioral science and brain imaging; to translate new knowledge about fundamental processes into researchable clinical questions; and to initiate innovative clinical trials of new pharmacologic and psychosocial interventions, with emphasis on testing their effectiveness in the diagnostically complex, diverse group of patients typically encountered in front-line service delivery systems. Investigators funded by NIMH also seek new ways to translate results from basic behavioral science into research relevant to public health, including the epidemiology of mental disorders, prevention and early intervention research, and mental health service research. Diverse scientific disciplines contribute to the design and evaluation of treatments and treatment delivery strategies that are relevant and responsive to the needs of persons with and at risk for mental illness. In this era of opportunity, NIMH is strongly committed to scientific programs to educate and train future mental health researchers, including scientists trained in molecular science, cognitive and affective neuroscience,

1

MECHANISMS OF SUPPORT

NIMH provides leadership at a national level for research on brain, behavior, and mental illness. Under a rigorous and highly competitive process, the institute funds research projects and research center grant awards and contracts to individual investigators in fields related to its areas of interest and to public and private institutions. NIMH also maintains and conducts a diversified program of intramural and collaborative research in its own laboratories and clinical research units at the National Institutes of Health (NIH). NIMH’s informational and educational activities include the dissemination of information and education materials on mental illness to health professionals and the public; professional associations; international, national, state, and local officials; and voluntary organizations working in the areas of mental health and mental illness. 2

AUTISM STAART CENTERS

NIMH supports interdisciplinary research centers through an NIH cooperative agreement in the Studies to Advance Autism Research and Treatment (STAART) Program, in cooperation with the National Institute of Child Health and Human Development (NICHD), the National Institute of Neurological Disorders and Stroke (NINDS), the National Institute on Deafness and Other Communication Disorders (NIDCD), and the National Institute of Environmental Health

This article was modified from the website of the National Institutes of Health (http://www.nih.gov/ about/almanac/organization/NIMH.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

NATIONAL INSTITUTE OF MENTAL HEALTH (NIMH)

Science (NIEHS). By evaluating and treating patients as well as enrolling them in clinical trials, each center helps expand the research base on the causes, diagnosis, early detection, prevention, and treatment of autism. 3 DIVISION ON SERVICES AND INTERVENTION RESEARCH (DSIR) The Division on Services and Intervention Research (DSIR) supports two critical areas of research: intervention research to evaluate the effectiveness of pharmacologic, psychosocial (psychotherapeutic and behavioral), somatic, rehabilitative, and combination interventions on mental and behavior disorders; and mental health services research on organization, delivery (process and receipt of care), related health economics, delivery settings, clinical epidemiology, and the dissemination and implementation of evidencebased interventions into service settings. The division also provides biostatistical analysis and clinical trials operations expertise for research studies, analyzes and evaluates national mental health needs and community research partnership opportunities, and supports research on health disparities. 3.1 DSIR Clinical Trials Operations and Biostatistics Unit The Clinical Trials Operations and Biostatistics Unit serves as the operations focal point for collaborative clinical trials on mental disorders in adults and children. The unit is responsible for overseeing both contractsupported and cooperative agreement–supported multisite clinical trial protocols as well as special projects undertaken by NIMH. In addition, the unit manages overarching matters related to clinical trials operations such as the coordination of the ancillary protocols across the large trials and the implementation of NIMH policy for dissemination of public access datasets. The unit also consults Institute staff and grantees/contractors on biostatistical matters related to appropriateness of study design, determination of power and sample size, and approaches to statistical analysis of data from NIMH-supported clinical trials.

NATIONAL INSTITUTE OF NEUROLOGICAL DISORDERS AND STROKE (NINDS)

hundreds of scientists in training, and provide career awards that offer a range of research experience and support for faculty members at various levels. The purposes and goals of the Extramural Division, Clinical Trials are:

The National Institute of Neurological Disorders and Stroke (NINDS), created by the U.S. Congress in 1950, has occupied a central position in the world of neuroscience for more than 50 years. The mission of NINDS is to reduce the burden of neurologic disease in every age group and segment of society. To accomplish this goal, the Institute supports and conducts research on the healthy and diseased nervous system; fosters the training of investigators in the basic and clinical neurosciences; and seeks better understanding, diagnosis, treatment, and prevention of neurologic disorders. Scientists in the Institute’s laboratories and clinics in Bethesda, Maryland, conduct research in the major areas of neuroscience and on many of the most important and challenging neurologic disorders, and collaborate with scientists in several other NIH institutes. The NINDS vision is:

• To promote the development of clinical

•

• • •

interventions for neurologic disorders and stroke. To stimulate the translation of findings in the laboratory to clinical research and clinical interventions. To ensure measures for protection of human subjects and safety monitoring. To encourage innovation in clinical research methodology. To support the development of neurology clinical researchers with training in biostatistics, epidemiology, and clinical trial methodology.

• To lead the neuroscience community in

shaping the future of research and its relationship to brain diseases. • To build an intramural program that is the model for modern collaborative neuroscience research. • To develop the next generation of basic and clinical neuroscientists through inspiration and resource support. • To seize opportunities to focus our resources to rapidly translate scientific discoveries into prevention, treatment, and cures. The Institute’s extramural program supports thousands of research project grants and research contracts. Institutional training grants and individual fellowships support This article was modified from the website of the National Institutes of Health (http://www.nih. gov/about/almanac/organization/NINDS.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NATIONAL INSTITUTE OF NURSING RESEARCH (NINR)

(NIH) campus, the NINR’s Division of Intramural Research (DIR) focuses on health promotion and symptom management and also provides research training opportunities.

The U.S. National Institute of Nursing Research (NINR) seeks to promote and to improve the health of individuals, families, communities, and populations by supporting and conducting clinical and basic research and research training on health and illness across the lifespan. The NINR research focus encompasses health promotion and disease prevention, quality of life, health disparities, and end-of-life issues. In keeping with the importance of nursing practice in various settings, the NINR seeks to extend nursing science by integrating the biological and behavioral sciences, employing new technologies to research questions, improving research methods, and developing the scientists of the future. The NINR supports basic research on preventing, delaying the onset, and slowing the progression of disease and disability. This includes finding effective approaches to achieving and sustaining a healthy lifestyle, easing the symptoms of illness, improving quality of life for patients and caregivers, eliminating health disparities, and addressing issues at the end of life. The NINR also fosters collaborations with many other disciplines in areas of mutual interest such as long-term care for older people, the special needs of women across the lifespan, genetic testing and counseling, biobehavioral aspects of the prevention and treatment of infectious diseases, and the impact of environmental influences on risk factors for chronic illnesses. In support of finding sound scientific bases for changes in clinical practice, the NINR’s major emphasis is on clinical research, and NINR programs are conducted primarily through grants to investigators across the country. On the National Institutes of Health This article was modified from the website of the National Institutes of Health (http://www.nih.gov/ about/almanac/organization/NINR.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NATIONAL INSTITUTE ON AGING (NIA)

neuroendocrine system, and immune system in aging; and the development of clinical trials and novel interventions to treat these pathologies. The Dementias of Aging Branch supports studies of etiology, pathophysiology, epidemiology, clinical course/natural history, diagnosis and functional assessment, drug discovery, drug development and clinical trials, prevention, behavioral management, and intervention in the cognitive changes associated with the dementias of later life (e.g., mild cognitive impairment, vascular dementia, frontotemporal dementia, Lewy body dementia), especially Alzheimer’s disease.

The U.S. National Institute on Aging (NIA) was established by the U.S. Congress in 1974 to ‘‘conduct and support of biomedical, social, and behavioral research, training, health information dissemination, and other programs with respect to the aging process and diseases and other special problems and needs of the aged.’’ The NIA maintains several branches with varying emphases. The Clinical Trials Branch plans and administers clinical trials on age-related issues that require extensive specialized clinical trials expertise. Examples of current and possible future types of interventions for trials are: • Interventions to prevent or to treat geri-

•

•

•

•

atric syndromes, disability, and complications of comorbidity or polypharmacy. Trials to detect age-related or comorbidity-related differences in responses to interventions against conditions found in middle age and old age. Interventions for problems associated with menopause and other midlife and late-life changes. Interventions that may affect rates of progression of age-related declines in function in early life and midlife. Interventions with protective effects against multiple age-related conditions.

The Integrative Neurobiology Section of the Neurobiology of Aging Branch supports research on the neural mechanisms underlying age-related changes in endocrine functions; neurodegenerative diseases of aging associated with conventional and unconventional infectious agents (e.g., prions); interactions of the central nervous system, This article was modified from the website of the National Institutes of Health (http://www. nih.gov/about/almanac/organization/NIA.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NATIONAL INSTITUTE ON ALCOHOL ABUSE AND ALCOHOLISM (NIAAA) The mission of the National Institute on Alcohol Abuse and Alcoholism (NIAAA) is to provide leadership in the national effort to reduce alcohol-related problems by: • Conducting and supporting research in

a wide range of scientific areas including genetics, neuroscience, and epidemiology to examine the health risks and benefits of alcohol consumption, prevention, and treatment. • Coordinating and collaborating with other research institutes and federal programs on alcohol-related issues. • Collaborating with international, national, state, and local institutions, organizations, agencies, and programs engaged in alcohol-related work. • Translating and disseminating research findings to health-care providers, researchers, policymakers, and the public. The Institute’s efforts to fulfill its mission are guided by the NIAAA vision to support and promote, through research and education, the best science on alcohol and health for the benefit of all by: • Increasing the understanding of normal

and abnormal biological functions and behavior relating to alcohol use. • Improving the diagnosis, prevention, and treatment of alcohol use disorders. • Enhancing quality health care.

This article was modified from the website of the National Institutes of Health (http://www.nih. gov/about/almanac/organization/NIAAA.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NATIONAL INSTITUTE ON DEAFNESS AND OTHER COMMUNICATION DISORDERS (NIDCD) The U.S. National Institute on Deafness and Other Communication Disorders (NIDCD) conducts and supports research and research training on disorders of hearing and other communication processes, including diseases affecting hearing, balance, smell, taste, voice, speech, and language. The NIDCD sponsors: • Research performed in its own laborato-

ries and clinics. • A program of research grants, individ-

ual and institutional research training awards, career development awards, center grants, conference grants, and contracts to public and private research institutions and organizations. • Cooperation and collaboration with professional, academic, commercial, voluntary, and philanthropic organizations concerned with research and training that is related to deafness and other communication disorders, disease prevention and health promotion, and the special biomedical and behavioral problems associated with people having communication impairments or disorders. • The support of efforts to create devices that substitute for lost and impaired sensory and communication functions. • Ongoing collection and dissemination of information to health professionals, patients, industry, and the public on research findings in these areas.

This article was modified from the website of the National Institutes of Health (http://www.nih. gov/about/almanac/organization/NIDCD.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

• The relationship of drug abuse to the

NATIONAL INSTITUTE ON DRUG ABUSE (NIDA)

acquisition, transmission, and clinical course of human immunodeficiency virus/acquired immunodeficiency syndrome (HIV/AIDS), tuberculosis, and other diseases and the development of effective prevention/intervention strategies.

The National Institute on Drug Abuse (NIDA) provides national leadership for research on drug abuse and addiction by supporting a comprehensive research portfolio that focuses on the biological, social, behavioral, and neuroscientific bases of drug abuse on the body and brain as well as its causes, prevention, and treatment. The NIDA also supports research training, career development, public education, and research dissemination efforts. Through its Intramural Research Program as well as grants and contracts to investigators at research institutions around the country and overseas, NIDA supports research and training on:

The Center for Clinical Trials Network (CCTN) supports and leads a network of 17 Regional Research Training Centers (RRTCs) and 240 Community Treatment Programs (CTPs) in a bi-directional effort to bridge the gap between the science of drug treatment and its practice through the study of scientifically based treatments in real world settings. The Clinical Trials Network (CTN) serves as a resource and forum for: • Multisite efficacy and effectiveness tri-

• The neurobiological, behavioral, and

social mechanisms underlying drug abuse and addiction.

•

• The causes and consequences of drug

abuse, including its impact on society and the morbidity and mortality in selected populations (e.g., ethnic minorities, youth, women).

•

• The relationship of drug use to problem

•

behaviors and psychosocial outcomes such as mental illness, unemployment, low socioeconomic status, and violence.

•

•

• Effective prevention and treatment ap-

proaches, including a broad research program designed to develop new treatment medications and behavioral therapies for drug addiction.

als of promising medications and behavioral interventions. Researchers who use the CTN as a platform for studies supported outside of the CCTN. NIDA-supported training using predoctoral and postdoctoral and career awards mechanisms. Secondary analyses of its rich database. Rapid response to emerging public health needs. Systematic transfer of research findings, both positive and negative, to treatment programs, clinicians, and patients.

• The mechanisms of pain and the search

for nonaddictive analgesics. • The relationship of drug abuse to cul-

tural and ethical issues such as health disparities. This article was modified from the website of the National Institutes of Health (http://www.nih. gov/about/almanac/organization/NIDA.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NATIONAL INSTITUTES OF HEALTH (NIH)

alcohol to combating heart disease. In every state across the country, the NIH supports research at hospitals, universities, and medical schools. The NIH is training the current and next generation of researchers to ensure that the capability to advance medical science remains strong. Many of these scientists-in-training will go on to become leading medical researchers and educators at universities; medical, dental, nursing, and pharmacy schools; schools of public health; nonprofit health research foundations; and private medical research laboratories around the country. As a federal agency, the NIH considers many different perspectives in establishing research priorities. A very competitive peer-review system identifies and funds the most promising and highest quality research to address these priorities. This research includes studies that ultimately touch the lives of all people. Currently, with the support of the American people, the NIH annually invests over $28 billion in medical research. More than 83% of the NIH’s funding is awarded through almost 50,000 competitive grants to more than 325,000 researchers at over 3000 universities, medical schools, and other research institutions in every state and around the world. About 10% of the NIH’s budget supports projects conducted by nearly 6000 scientists in its own laboratories, most of which are on the NIH campus in Bethesda, Maryland. The NIH’s own scientists and scientists working with support from the NIH grants and contracts have been responsible for countless medical advances, and more than 100 of these scientists have received Nobel Prizes in recognition of their work.

The National Institutes of Health (NIH), a part of the U.S. Department of Health and Human Services, is the primary federal agency for conducting and supporting medical research. Headquartered in Bethesda, Maryland, the NIH is composed of 27 Institutes and Centers and has more than 18,000 employees on the main campus and at satellite sites across the country. The NIH provides leadership and financial support to researchers in every state and throughout the world. Helping to lead the way toward important medical discoveries that improve people’s health and save lives, NIH scientists investigate ways to prevent disease as well as the causes, treatments, and even cures for common and rare diseases. From the time of its founding in 1887 as the Laboratory of Hygiene at the Marine Hospital in Staten Island, New York, the National Institutes of Health has played an important role in improving health in the United States. Many important health and medical discoveries of the last century resulted from research supported by the NIH. In part because of NIH research, our citizens are living longer and better: life expectancy at birth was only 47 years in 1900; by 2000, it was almost 77 years. The NIH translates research results into interventions and communicates research findings to patients and their families, health-care providers, and the general public. The National Institutes of Health supports and conducts medical research to understand how the human body works and to gain insight into countless diseases and disorders, from rare and unusual diseases to more familiar ones like the common cold. It supports a wide spectrum of research, from learning how the brain becomes addicted to This article was modified from the website of the National Institutes of Health (http://www.nih. gov/about/NIHoverview.html) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NATIONAL INSTITUTES OF HEALTH STROKE SCALE (NIHSS)

for sensory function (pupillary response and plantar response) were initially included, but were later removed.

GUSTAVO A. ORTIZ and RALPH L. SACCO MD MS FAAN FAHA, MD

1

Miller School of Medicine University of Miami Miami, Florida

CHARACTERISTICS OF THE SCALE

The NIHSS assesses level of consciousness, gaze, visual fields, facial weakness, motor performance of the extremities, sensory deficit, coordination (ataxia), language (aphasia), speech (dysarthria), and hemi-inattention (neglect) (Tables 1, 2, and 3. Figs. 1 and 2). For all parameters, a value of 0 is normal; so, the higher the score, the worse the neurological deficit. The differences between the levels are not subtle. For example, motor performance for the upper extremity was graded as 0 = normal, 1 = drift (with arms outstretched), 2 = inability to resist gravity for 10 seconds (with arms outstretched), 3 = no effort against gravity, and 4 = no movement. The lowest possible overall NIHSS score is 0 or ‘‘normal,’’ and the highest possible score is 42. The scale was designed to be done quickly and easily at the bedside to provide a rapid and standardized assessment of neurological function in the early periods after a stroke. When first evaluated in 65 patients, it was administered in a mean of 6.6 ± 1.3 minutes and was completed in all patients, regardless of stroke severity (2).

The correlation of NIHSS scores with MRI and clinical findings has been used for assessment of ischemic penumbra, in an effort to identify patients with tissue at risk of infarction for thrombolytic or neuroprotective drugs. Currently, the NIHSS is the most widely used scale for trials related to thrombolytic therapy. Clinical measurement of the severity of cerebral infarction became an important subject of investigation in the late 1980s with the introduction of new therapies for acute stroke. Traditional measures such as mortality or long-term functional status were not as well suited for the assessment of acute stroke therapies in which the immediate effects of stroke had to be quantified. The evaluation tools used to quantify a clinical condition are usually referred to as clinimetric instruments (1). ‘‘Stroke scales’’ are clinimetric instruments used to quantify neurological deficits, functional outcome, global outcome, or health-related quality of life in patients after a stroke. The National Institutes of Health Stroke Scale (NIHSS) is a systematic assessment tool designed to measure the neurological deficits most often seen with acute stroke. It was developed by Thomas Brott, MD and colleagues at the University of Cincinnati (Ohio), and its description was first reported in 1989 (2). The examination format was developed to assess neurologic signs in the distribution of each of the major arteries of the brain. Exam items for the NIHSS were adapted from the Toronto Stroke Scale, the Oxbury Initial Severity Scale (3,4) and the Cincinnati Stroke Scale which graded speech, drift of the affected arm, drift of the affected leg, and grip strength. Two items from the Edinburgh-2 Coma scale were also used to assess mental status (5). Other categories

1.1 Reliability Reliability, an index of the reproducibility and precision of the NIHSS, was first evaluated among twenty-four patients with acute cerebral infarction who were examined twice within a 24-hour interval. Each examination was performed by a neurologist, whereas the other three examination team members (neurology house officer, neurology nurseclinician, and emergency department nurseclinician) observed. Then, each team member independently performed scoring. Inter-rater agreement between these four examiners was high (mean k = 0.69); and intra-rater agreement was also excellent, especially when the rater was a neurologist (k = 0.77) (2). Another study of 20 patients confirmed that the NIH

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

NATIONAL INSTITUTES OF HEALTH STROKE SCALE (NIHSS)

Table 1. The National Institutes of Health Stroke Scale∗ Test 1a. Level of Consciousness (LOC):

1b. LOC Questions:

1c. LOC Commands:

2. Best Gaze:

3. Visual: Visual fields (upper and lower quadrants) 4. Facial Palsy: Ask– or use pantomime to encourage

5. Motor Arm: 5a. Left Arm 5b. Right Arm

6. Motor Leg: 6a. Left Leg 6b. Right Leg

7. Limb Ataxia:

8. Sensory:

Scale Definition 0 = Alert; keenly responsive. 1 = Not alert; however, arousable by minor stimulation to obey, answer, or respond. 2 = Not alert; requires repeated stimulation to attend, or is obtunded and requires strong or painful stimulation to make movements (not stereotyped). 3 = Responds only with reflex motor or autonomic effects or totally unresponsive, flaccid, and are flexic. 0 = Answers both questions correctly. 1 = Answers one question correctly. 2 = Answers neither question correctly. 0 = Performs both tasks correctly. 1 = Performs one task correctly. 2 = Performs neither task correctly. 0 = Normal. 1 = Partial gaze palsy; gaze is abnormal in one or both eyes, but forced deviation or total gaze paresis is not present. 2 = Forced deviation, or total gaze paresis not overcome by the oculocephalic maneuver. 0 = No visual loss. 1 = Partial hemianopia. 2 = Complete hemianopia. 3 = Bilateral hemianopia (blind including cortical blindness). 0 = Normal symmetrical movements. 1 = Minor paralysis (flattened nasolabial fold; asymmetry on smiling). 2 = Partial paralysis (total or near-total paralysis of lower face). 3 = Complete paralysis of one or both sides (absence of facial movement in the upper and lower face). 0 = No drift; limb holds 90 (or 45) degrees for full 10 seconds. 1 = Drift; limb holds 90 (or 45) degrees, but drifts down before full 10 seconds; does not hit bed or other support. 2 = Some effort against gravity; limb cannot get to or maintain (if cued) 90 (or 45) degrees; drifts down to bed, but has some effort against gravity. 3 = No effort against gravity; limb falls. 4 = No movement. UN = Amputation or joint fusion, explain: 0 = No drift; leg holds 30-degree position for full 5 seconds. 1 = Drift; leg falls by the end of the 5-second period but does not hit bed. 2 = Some effort against gravity; leg falls to bed by 5 seconds but has some effort against gravity. 3 = No effort against gravity; leg falls to bed immediately. 4 = No movement. UN = Amputation or joint fusion, explain: 0 = Absent. 1 = Present in one limb. 2 = Present in two limbs. UN = Amputation or joint fusion, explain: 0 = Normal; no sensory loss. 1 = Mild-to-moderate sensory loss; patient feels pinprick is less sharp or is dull on the affected side, or there is a loss of superficial pain with pinprick, but patient is aware of being touched. 2 = Severe to total sensory loss; patient is not aware of being touched in the face, arm, and leg.

NATIONAL INSTITUTES OF HEALTH STROKE SCALE (NIHSS)

3

Table 1. (continued) Test

Scale Definition

9. Best Language:

0 = No aphasia; normal. 1 = Mild-to-moderate aphasia; some obvious loss of fluency or facility of comprehension, without significant limitation on ideas expressed or form of expression. Reduction of speech and/or comprehension, however, makes conversation about provided materials difficult or impossible. For example, in conversation about provided materials, examiner can identify picture or naming card content from patient response. 2 = Severe aphasia; all communication is through fragmentary expression; great need for inference, questioning, and guessing by the listener. Range of information that can be exchanged is limited; listener carries burden of communication. Examiner cannot identify materials provided from patient response. 3 = Mute, global aphasia; no usable speech or auditory comprehension. 0 = Normal. 1 = Mild-to-moderate dysarthria; patient slurs at least some words and, at worst, can be understood with some difficulty. 2 = Severe dysarthria; patient speech is so slurred as to be unintelligible in the absence of or out of proportion to any dysphasia, or is mute/anarthric. UN = Intubated or other physical barrier, explain: 0 = No abnormality. 1 = Visual, tactile, auditory, spatial, or personal inattention or extinction to bilateral simultaneous stimulation in one of the sensory modalities. 2 = Profound hemi-inattention or extinction to more than one modality; does not recognize own hand or orients to only one side of space.

10. Dysarthria:

11. Extinction and Inattention (Formerly Neglect):

∗ For all parameters, a value of 0 is normal; so, the higher the score, the worse the neurological deficit. The differences between the levels are not subtle. (Adapted from ‘‘NIH Stroke Scale’’ at www.ninds.nih.gov.)

Table 2. Evaluation of Language: Reading∗ You know how. Down to earth. I got home from work. Near the table in the dining room. They heard him speak on the radio last night. ∗ The patient is asked to read these sentences. (Reproduced from www.ninds.nih.gov.)

Table 3. Evaluation of Dysarthria: the Patient is Asked to Read and Say These Words (Reproduced From www.ninds.nih.gov) MAMA TIP – TOP FIFTY – FIFTY THANKS HUCKLEBERRY BASEBALL PLAYER

4

NATIONAL INSTITUTES OF HEALTH STROKE SCALE (NIHSS)

Stroke Scale has substantial or moderate inter-rater reliability for nine of its items (6). The NIHSS was first used in a pilot study of t-PA administered between 0 and 180 minutes from stroke onset (7); and a modified, 13-item form of the scale was later used in the definitive NINDS trial of t-PA (8). In preparation for the latter trial, 162 investigators were trained and certified in the use of the scale by using a two-camera videotape method, an approach that enhanced intra-rater and inter-rater reliability (9). Moderate to excellent agreement was found on most items (unweighted k > 0.60), but facial paresis and ataxia showed poor agreement (unweighted k < 0.40). The scale was also demonstrated to be reliable when performed by non-neurologists and nonphysicians, after training via the videotape method was considered a prerequisite for its use (9–12). 1.2 Validity The validity, or extent to which the NIHSS measured what it purported to measure, was initially assessed by determining reliability and accuracy (13). Accuracy was determined by correlating the NIHSS scores with two other measures of stroke severity: computed tomography (CT) measurement of volume of infarction and clinical outcome at 3 months. 1. Correlation with size of infarction: The Spearmen correlation between the total NIHSS score and the computed tomography lesion at 7 days was 0.74. The initial neurological deficit of the patient as measured by the scale also correlated with the 7–10 day CT lesion volume (correlation coefficient = 0.78) (14). 2. Correlation with clinical outcome: Stroke scale scores at 7 days corresponded to eventual clinical outcome, including placement in the community, with a Spearman correlation of 0.71 (P = 0.0001). The admission stroke scale scores also correlated with the eventual patient functional outcome at 3 months, with a Spearman correlation of 0.53 (P = 0.0001) (2). In 1999, Dr. Lyden et al. with the NINDS tPA Trial investigators validated the scale

as an outcome measure, using data from the NINDS tPA Stroke Trial (15). In this study, it was shown that the internal scale structure remained consistent in treated and placebo groups and when administered serially over time. Also, the correlations between the scale and other clinical outcome scales (Barthel Index, Rankin Scale, and Glasgow Outcome Scale) at 3 months were significant (P < 0.001). The study supported the validity of the scale for use in future treatment trials as an outcome measure. Other studies using the NIHSS have also shown good construct validity. In a post hoc analysis by stroke subtype of 1268 patients enrolled in an acute stroke trial, baseline NIHSS scores strongly predicted outcome at 7 days and at 3 months. An excellent outcome was achieved by almost two thirds of patients with a score of 3 or less at day 7; however, very few patients with baseline scores of more than 15 had excellent outcomes after 3 months (16). Other alternative measures of neurological outcome, such as activities of daily living (ADL) scales, have also been correlated with the NIHSS (17). 2 CORRELATION OF NIHSS SCORES WITH MRI STUDIES MRI scans permitted a more accurate measurement of stroke lesion volume and allowed for a better assessment of the correlation between NIHSS and stroke size. Lovblad et al. in 1997 found a significant correlation between Diffusion Weighted Imaging (DWI) lesion size and both initial (r = 0.53, P = 0.0003) and chronic NIHSS score (r = 0.68, P < 0.0001) in 50 patients with acute middle cerebral artery ischemic stroke (< 24-hour duration) (18). The correlation was also demonstrated in earlier strokes (within 6.5 hours of symptom onset), by comparing DWI or Perfusion Weighted Imaging (PWI) to assess lesion size (19). In eight of nine DWIpositive patients in this series, a strong linear correlation existed between 24-hour NIHSS and DWI volume (r = 0.96, P < 0.001), and PWI correlated better with 24-hour NIHSS than with DWI. The difference was attributed primarily to one patient, who had a substantial perfusion delay in PWI, whereas DWI

NATIONAL INSTITUTES OF HEALTH STROKE SCALE (NIHSS)

5

Figure 1. Evaluation of language: Comprehension. The patient is asked to describe what is happening in this picture. (Reproduced from www.ninds.nih.gov).

Figure 2. Evaluation of language: Naming. The patient is asked to name the objects shown in this ‘‘naming sheet.’’ (Reproduced from www.ninds.nih.gov).

showed no abnormality. Clinically, the deficit was substantial (NIHSS = 24). On a subsequent 24-hour MRI scan, a DWI abnormality developed that closely matched her initial perfusion deficit (19). This study led to the

development of the concept of PWI/DWI mismatch. Following the initial report by Dr. Tong, MRI has been used extensively to characterize the ischemic penumbra by identifying regions of brain with reduced blood flow

6

NATIONAL INSTITUTES OF HEALTH STROKE SCALE (NIHSS)

(defined by PWI) and regions with irreversible neuronal injury (approximated by DWI). A mismatch between the volume of abnormality on diffusion and perfusion imaging seems to indicate viable penumbral tissue (PWI/DWI mismatch). PWI/DWI mismatch has been shown to correlate with favorable outcomes following late treatment with intravenous rtPA up to 6 hours from symptom onset (20–23). However, PWI is a complex, time-consuming, and not well-standardized technique, with limited availability. Based on the fact that abnormal PWI volume has a higher correlation with the stroke severity (evaluated clinically with the NIHSS) than the DWI volume (19,24), Davalos et al. in 2004 proposed the concept of clinical-DWI mismatch (CDM) (25). CDM defined as NIHSS ≥ 8 and ischemic volume on DWI of ≤ 25 ml was associated with a high probability of infarct growth and early neurological deterioration. Subsequently, it was suggested that CDM may identify patients with tissue at risk of infarction for thrombolytic or neuroprotective drugs. CDM was later shown to predict the presence of PWI/DWI mismatch with a specificity of 93% (95% confidence interval (CI), 62%–99%) and a positive predictive value of 95% (95% CI, 77%–100%) but a sensitivity of only 53% (95% CI, 34%–68%). Efforts are being made to evaluate a computed tomography–NIHSS mismatch, by using the recently applied Alberta Stroke Program Early CT score (ASPECTS), but results are still controversial (26,27). 3

USE OF THE NIHSS

Originally, the NIHSS was developed for use in multicenter acute stroke trials. It proved reliable and valid for the measurement of stroke-related deficits at baseline, as well as for measuring outcome. Currently, the NIHSS is the most widely used scale for trials related to thrombolytic therapy. With the approval of intravenous thrombolysis for the treatment of acute stroke, the use of the NIHSS in the clinical care of patients increased. The ease of administration of the NIHSS, which can be effectively performed by physicians and nonphysicians

after a relatively simple training, led to more widespread acceptance. The NIHSS is used to provide an accurate assessment of the deficits of a patient with stroke during initial presentation. This initial clinical assessment can be easily transmitted to other health-care providers involved in the care of the patient, thus expediting the decision-making process in the emergency setting. The scale is also a sensitive measurement tool for serial monitoring of patients with stroke, quantifying changes in the neurological examination. An increase of 2 points or more on the NIHSS has been suggested to be associated to a relevant worsening of the stroke, although this specific cutoff has not been independently validated (28). In the clinical setting, the NIHSS predicts post-acute care disposition among stroke patients and may facilitate the process of hospital discharge (29,30). In particular, stroke patients with NIHSS scores ≤ 5 can be expected to go home, those with scores > 13 most often go to a nursing facility, and those with intermediate scores most often go to acute rehabilitation (29). 4 STANDARDIZED TRAINING FOR THE USE OF THE NIHSS The American Stroke Association, in conjunction with the American Academy of Neurology (AAN) and the National Institute of Neurological Disorders and Stroke (NINDS), developed a free, CME/CE-certified online training program for healthcare professionals (targeted to emergency physicians, neurologists, nurses, clinical research raters, and medical students) to learn or review how to administer the NIHSS for acute stroke assessment. Available at the American Stroke Association webpage (www. strokeassociation.org) by clicking on the link to Professional Education Center, it requires signing up and creating a profile with username and password before taking the course. Video training is also available on DVD. The course consists of six test groups, A through F. After successfully completing the program, participants can print their CME/CE credit for test groups A and B directly from the website and can also print a certificate of

NATIONAL INSTITUTES OF HEALTH STROKE SCALE (NIHSS)

completion for test groups A, B, C, D, E, or F (31). 5

scale are graded numerically, with higher values representing more severe deficits, these are ordinal-level, not interval-level, data (33). The total score is obtained by adding all these individual rankings and may be misleading (34). It has been suggested that clinically it is more useful to think of the NIHSS as a way to quantify serial neurologic findings and measure change within an individual patient over time.

LIMITATIONS OF THE SCALE

Several limitations of the NIHSS are as follows: 1. The inter-rater reliability of the scale is not homogeneous for all the items. It has been shown that the assessments of limb ataxia and facial weakness have a lower agreement between examiners compared with the other items, with calculated k values that were not significantly different from chance. However, the calculated values of k were significantly greater than expected by chance for 11 of 13 items, indicating substantial agreement for 5 items, moderate agreement for 4 items, and fair agreement for 2 items. This rating system compared favorably with other scales (6). 2. The NIHSS has a differential characterization of right versus left hemisphere strokes. Of the 42 possible points on the NIHSS, 7 are directly related to measurement of language function (orientation questions, 2; commands, 2; and aphasia, 3), and only 2 points are for neglect. It has been shown that for a given NIHSS, the total lesion volume for patients with right (usually nondominant for language) hemisphere strokes is statistically larger than the lesion volume for patients with left (usually dominant for language) hemisphere strokes. The difference reflects the differential weighting of the NIHSS with regard to language function, as compared with hemineglect (32). 3. Brainstem infarcts may not be adequately characterized by the NIHSS. Cranial nerves are not fully assessed in the scale; therefore, it is possible that life-threatening brainstem or cerebellar infarctions may result in lower scores that underestimate the clinical severity of the stroke. 4. The total NIHSS score has limited value. Although all items in the

7

6

FUTURE DIRECTIONS

A modified NIHSS (mNIHSS) with 11 items, derived from previous clinimetric studies of the NIHSS, has been proposed. Poorly reproducible or redundant items (level of consciousness, face weakness, ataxia, and dysarthria) were deleted, and the sensory items were collapsed into 2 responses. 10 of the 11 items show excellent reliability, and 1 item shows good reliability (35,36). Other shortened scales are being proposed for prehospital clinical assessment of stroke, but additional studies are needed to assess their value in the screening of stroke by the paramedic services (37). Video telemedicine is being proposed as an option to facilitate cerebrovascular specialty consults to underserved areas. It seems feasible to perform NIHSS remotely using computer-based technology, in an effort to increase the rate of rt-PA administration (38–40). REFERENCES 1. K. Asplund, Clinimetrics in stroke research. Stroke 1987; 18: 528–530. 2. T. Brott, et al., Measurements of acute cerebral infarction: a clinical examination scale. Stroke 1989; 20: 864–870. 3. Oxbury, J.M., R.C. Greenhall, and K.M. Grainger, Predicting the outcome of stroke: acute stage after cerebral infarction. Br. Med. J. 1975; 3: 125–127. 4. R. Cote, et al., The Canadian Neurological Scale: a preliminary study in acute stroke. Stroke 1986; 17: 731–737. 5. K. Sugiura, et al., The Edinburgh-2 coma scale: a new scale for assessing impaired consciousness. Neurosurgery 1983; 12: 411–415.

8

NATIONAL INSTITUTES OF HEALTH STROKE SCALE (NIHSS)

6. L. B. Goldstein, C. Bertels, and J. N. Davis, Interrater reliability of the NIH stroke scale. Arch. Neurol. 1989; 46: 660–662. 7. E. C. Haley Jr., et al., Urgent therapy for stroke. Part II. Pilot study of tissue plasminogen activator administered 91–180 minutes from onset. Stroke 1992; 23: 641–645. 8. NINDS, Tissue plasminogen activator for acute ischemic stroke. The National Institute of Neurological Disorders and Stroke rt-PA Stroke Study Group. N. Engl. J. Med. 1995; 333: 1581–1587. 9. P. Lyden, et al., Improved reliability of the NIH Stroke Scale using video training. NINDS TPA Stroke Study Group. Stroke 1994; 25: 2220–2226. 10. M. A. Albanese, et al., Ensuring reliability of outcome measures in multicenter clinical trials of treatments for acute ischemic stroke. The program developed for the Trial of Org 10172 in Acute Stroke Treatment (TOAST). Stroke 1994; 25: 1746–1751. 11. L. B. Goldstein and G. P. Samsa, Reliability of the National Institutes of Health Stroke Scale. Extension to non-neurologists in the context of a clinical trial. Stroke 1997; 28: 307–310. 12. S. Schmulling, et al., Training as a prerequisite for reliable use of NIH Stroke Scale. Stroke 1998; 29: 1258–1259. 13. R. Cote, et al., Stroke assessment scales: guidelines for development, validation, and reliability assessment. Can. J. Neurol. Sci. 1988; 15:261–265 14. T. Brott, et al., Measurements of acute cerebral infarction: lesion size by computed tomography. Stroke 1989; 20: 871–875. 15. P. Lyden, et al., Underlying structure of the National Institutes of Health Stroke Scale: results of a factor analysis. NINDS tPA Stroke Trial Investigators. Stroke 1999; 30: 2347–2354. 16. H. P. Adams Jr., et al., Baseline NIH Stroke Scale score strongly predicts outcome after stroke: a report of the Trial of Org 10172 in Acute Stroke Treatment (TOAST). Neurology 1999; 53: 126–131. 17. P. W. Duncan, et al., Measurement of motor recovery after stroke. Outcome assessment and sample size requirements. Stroke 1992; 23: 1084–1089. 18. K. O. Lovblad, et al., Ischemic lesion volumes in acute stroke by diffusion-weighted magnetic resonance imaging correlate with clinical outcome. Ann Neurol, 1997; 42: 164–170. 19. D. C. Tong, et al., Correlation of perfusion- and diffusion-weighted MRI with NIHSS score in

acute (< 6.5 hour) ischemic stroke. Neurology 1998; 50: 864–870. 20. G. W. Albers, et al., Magnetic resonance imaging profiles predict clinical response to early reperfusion: the diffusion and perfusion imaging evaluation for understanding stroke evolution (DEFUSE) study. Ann. Neurol. 2006; 60: 508–517. 21. J. Rother, et al., Effect of intravenous thrombolysis on MRI parameters and functional outcome in acute stroke < 6 hours. Stroke 2002; 33: 2438–2445. 22. M. W. Parsons, et al., Diffusion- and perfusionweighted MRI response to thrombolysis in stroke. Ann. Neurol. 2002; 51: 28–37. 23. G. Thomalla, et al., Outcome and symptomatic bleeding complications of intravenous thrombolysis within 6 hours in MRI-selected stroke patients: comparison of a German multicenter study with the pooled data of ATLANTIS, ECASS, and NINDS tPA trials. Stroke 2006; 37: 852–858. 24. T. Neumann-Haefelin, et al., Diffusion- and perfusion-weighted MRI. The DWI/PWI mismatch region in acute stroke. Stroke 1999; 30: 1591–1597. 25. A. Davalos, et al., The clinical-DWI mismatch: a new diagnostic approach to the brain tissue at risk of infarction. Neurology 2004; 62: 2187–2192. 26. H. Tei, S. Uchiyama, and T. Usui, Clinicaldiffusion mismatch defined by NIHSS and ASPECTS in non-lacunar anterior circulation infarction. J Neuro. 2007; 254: 340–346. 27. S. R. Messe, et al., CT-NIHSS mismatch does not correlate with MRI diffusion-perfusion mismatch. Stroke 2007; 38: 2079–2084. 28. B. C. Tilley, et al., Use of a global test for multiple outcomes in stroke trials with application to the National Institute of Neurological Disorders and Stroke t-PA Stroke Trial. Stroke 1996; 27: 2136–2142. 29. D. Schlegel, et al., Utility of the NIH Stroke Scale as a predictor of hospital disposition. Stroke 2003; 34: 134–137. 30. D. J. Schlegel, et al., Prediction of hospital disposition after thrombolysis for acute ischemic stroke using the National Institutes of Health Stroke Scale. Arch. Neurol. 2004; 61: 1061–1064. 31. American Stroke Association, NIH Stroke Scale Training Online, 2007. Available: www.strokeassociation.org. 32. D. Woo, et al., Does the National Institutes of Health Stroke Scale favor left hemisphere

NATIONAL INSTITUTES OF HEALTH STROKE SCALE (NIHSS) strokes? NINDS t-PA Stroke Study Group. Stroke 1999; 30: 2355–2359. 33. A. R. Feinstein, B. R. Josephy, and C. K. Wells, Scientific and clinical problems in indexes of functional disability. Ann. Intern. Med. 1986; 105: 413–420. 34. T. J. Steiner and F. Clifford Rose, Towards a model stroke trial. The single-centre naftidrofuryl study. Neuroepidemiology 1986; 5: 121–147. 35. B. C. Meyer, et al., Modified National Institutes of Health Stroke Scale for use in stroke clinical trials: prospective reliability and validity. Stroke 2002; 33: 1261–1266. 36. P. D. Lyden, et al., A modified National Institutes of Health Stroke Scale for use in stroke clinical trials: preliminary reliability and validity. Stroke 2001; 32: 1310–1317. 37. D. L. Tirschwell, et al., Shortening the NIH Stroke scale for use in the prehospital setting. Stroke 2002; 33: 2801–2806. 38. B. C. Meyer, et al., Prospective reliability of the STRokE DOC wireless/site independent telemedicine system. Neurology 2005; 64: 1058–1060. 39. S. R. Levine and M. Gorman, ‘‘Telestroke’’: the application of telemedicine for stroke. Stroke 1999; 30: 464–469. 40. S. R. Levine, and K. M. McConnochie, Telemedicine for acute stroke: when virtual is as good as reality. Neurology 2007; 69: 819–820.

9

NATIONAL LIBRARY OF MEDICINE (NLM) The U.S. National Library of Medicine (NLM) is the world’s largest research library of the health sciences and serves scientists, health professionals, and the public. The Library has a statutory mandate from Congress to apply its resources broadly to the advancement of medical and health-related sciences. To that end, it collects, organizes, and makes available biomedical information to investigators, educators, practitioners, and the public and carries out programs designed to strengthen existing and develop new medical library services in the United States. It conducts research in health communications, supports medical informatics, and provides information services and sophisticated tools in the areas of molecular biology and toxicology/environmental health. The NLM also creates Web-based services for the general public containing information from the National Institutes of Health (NIH) and other reliable sources.

This article was modified from the website of the National Institutes of Health (http://www.nih.gov/ about/almanac/organization/NLM.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NEW DRUG APPLICATION (NDA) PROCESS For decades, the regulation and control of new drugs in the United States has been based on the New Drug Application (NDA). Since 1938, every new drug has been the subject of an approved NDA before U.S. commercialization. The NDA application is the vehicle through which drug sponsors formally propose that the U.S. Food and Drug Administration (FDA) approve a new pharmaceutical for sale and marketing in the United States. The data gathered during the animal studies and human clinical trials of an Investigational New Drug (IND) become part of the NDA. The goals of the NDA are to provide enough information to permit an FDA reviewer to reach the following key decisions: • Whether the drug is safe and effective

in its proposed use(s), and whether the benefits of the drug outweigh the risks. • Whether the drug’s proposed labeling (the package insert) is appropriate, and what it should contain. • Whether the methods used in manufacturing the drug and the controls used to maintain the drug’s quality are adequate to preserve the drug’s identity, strength, quality, and purity. The documentation required in an NDA is supposed to tell the drug’s whole story, including what happened during the clinical tests, what the ingredients of the drug are, what the results of the animal studies were, how the drug behaves in the body, and how it is manufactured, processed, and packaged.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/regulatory/applications/ nda.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

N-OF-1 RANDOMIZED TRIALS

hypertension, nocturnal leg cramps, attention deficit/hyperactivity disorder, fibromyalgia, Parkinson’s disease, asthma, and chronic obstructive pulmonary disease, among many others.

ANDREW L. AVINS Kaiser-Permanente, Northern California Division of Research and Departments of Medicine and Epidemiology & Biostatistics, University of California, San Francisco

1

GOAL OF N-OF-1 STUDIES

An N-of-1 study may be initiated for several reasons. Most commonly, the primary motivation is to better define the effectiveness of therapy for a symptomatic condition, though the effect of therapy on an asymptomatic endpoint (e.g., blood pressure) may also be investigated with appropriate measurements. N-of-1 studies may be valuable when patients are hesitant to use a therapy and desire more information about their response and when the medical provider and patient disagree about the response to a therapy. Another common use is to understand whether a specific therapy is the cause of an undesirable symptom in a patient. Determining the optimal dose of a medication is a matter that also lends itself well to study by N-of-1 methodology. Because it is often impossible or impractical to conduct typical randomized trials for patients with rare conditions, N-of-1 trials may provide an opportunity for defining optimal therapy for these patients. As will be noted, however, conditions and therapies amenable to N-of-1 trials have specific attributes that render many conditions or treatments unsuitable for study by this design. One of the most valuable situations in which to perform N-of-1 studies is when the clinical response to an intervention is known to be highly variable, there is uncertainty about the patient’s response, and the intervention is expensive or associated with serious adverse effects. In this situation, greater insight into a patient’s true response is particularly helpful in formulating rational treatment decisions. For example, Zucker et al. (8) describe the use of an N-of-1 trial to help fibromyalgia patients make decisions about using the combination of amitriptyline and fluoxetine versus amitriptyline alone; a prior trial had found that only 63% of patients

JOHN NEUHAUS Department of Epidemiology & Biostatistics, University of California, San Francisco

Comparative clinical trials in individual patients, commonly known as ‘‘N-of-1’’ trials, have been used for many years in the field of psychology and have found increasing applications for biomedical problems. Such trials can be structured to provide information generalizable to large groups of patients, but the primary application of N-of-1 trials is to help individual patients make more evidencebased decisions about specific aspects of their medical care (1). In clinical practice, patients frequently undergo ‘‘therapeutic trials’’ in which a health-care provider recommends a course of action that the patient tries and decides whether to continue, modify, or terminate. An N-of-1 study has the same goal as a therapeutic trial but brings a more structured and less biased approach to evaluating a therapeutic response (1–3). Such trials are also known as ‘‘single-patient trials,’’ ‘‘individualized medication effectiveness trials,’’ and ‘‘individual patient trials’’ (4–6). An N-of-1 trial is generally structured as a randomized, double-blind, controlled, multiple-period crossover trial in a single patient, often of a pharmacologic agent (1, 3, 6, 7). In these studies, a patient is assigned to different periods on the active and the comparator treatments, with outcome measurements obtained during each of these periods. At the end of the trial, the patient’s responses during each of the treatment conditions are compared, and the information is used to make a decision about the likelihood that one of the therapeutic options is superior. Such trials have been successfully used in such disparate conditions as osteoarthritis,

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

N-OF-1 RANDOMIZED TRIALS

responded to the combination therapy (and both drugs are associated with a wide range of side effects). 2

REQUIREMENTS

Similar to other crossover studies, N-of-1 trials are possible or practical in only a limited number of settings, as both the condition being treated and the therapy being tested must meet certain conditions in order for the trial to be successful (1, 3, 6, 7, 9, 10). 2.1 Patient’s Motivation Both the adherence to the study protocol and the usefulness of the data obtained are closely related to the motivation of the patient. Hence, there should be a clearly perceived need for conducting the N-of-1 study on the part of both the patient and the health-care provider. There should be uncertainty or disagreement about which choice of therapy (if any) is best and a mutual understanding of the value of conducting an N-of-1 trial to answer the question (2). Given the difficulty and expense of conducting an N-of-1 trial, there should also be a clear understanding of how (and whether) the patient will use the information from the trial to make a decision about therapy. A lack of motivation may seriously undermine the likelihood of success of the trial (see section 5.1). 2.2 Conditions Suitable for Study Because N-of-1 trials are generally structured as multiple-period crossover trials, only certain medical conditions are amenable to study with this methodology. Generally, appropriate illnesses must be chronic or recurring. Conditions whose status changes rapidly are difficult to investigate, though increasing the number of crossover periods may mitigate this problem. Similarly, recurrent illnesses with long asymptomatic intercurrent periods are difficult to study; this problem can sometimes be addressed by increasing the length of the treatment periods, though this approach may reduce the patient’s motivation and adherence to the protocol. Studies of short, self-limited illnesses are unlikely to be informative.

2.3 Therapies Suitable for Study For a successful trial, the study therapy must act quickly or a longer treatment period will be required to fairly evaluate treatment effectiveness. Similarly, it is desirable that once treatment is stopped the patient’s condition quickly returns to its baseline state, or prolonged washout periods will be required. Obviously, a potentially curative treatment is inappropriate for any crossover study. 2.4 Combining Results of Many N-of-1 Trials When many patients with the same diagnosis participate in N-of-1 trials with the same intervention, the results of these studies can be combined to estimate a population treatment effect, much like a formal multiparticipant crossover study. If such a series of N-of-1 trials is contemplated, care should be taken to ensure that they are structured in a way that permits a valid synthesis of the data. These considerations include appropriate definitions of the disease state under study and applicable eligibility criteria, a treatment protocol that provides enough similarity to make a synthesis meaningful, and outcome assessments that are measured with sufficient uniformity to permit combining the data. Using Bayesian methods, the data from an accumulating series of N-of-1 trials can be used to refine the estimates of the likelihood of a response for an individual patient as well as providing a generalizable estimate of effect for a population. Zucker et al. (8, 11) provide a detailed methodology for this technique, including the structure of a hierarchical Bayesian model and an example of this method in a series of N-of-1 studies for testing pharmacologic therapy for fibromyalgia. 3 DESIGN CHOICES AND DETAILS FOR N-OF-1 STUDIES Numerous options exist for the design and implementation of N-of-1 studies. Most variants have been tried successfully, and it is clear that there is no single design that is optimal for all situations; the creativity and insight of the investigator will dictate the best approach among the available

N-OF-1 RANDOMIZED TRIALS

alternatives. Issues that must be addressed include randomization, number and frequency of crossover periods, run-in and washout periods, blinding, outcome assessment, and choice of outcome measure.

3

alternative is to use strictly alternating treatments, with the first treatment chosen at random, though such a design may be more susceptible to unblinding of the treatments. 3.2 Number and Length of Treatment Periods

3.1 Randomization It is not absolutely required that the various treatment periods in an N-of-1 study be randomized, but most authorities agree that randomization is greatly preferable when it can be done. When randomization is employed, there are two major options: an unconstrained randomized design, or a blocked randomized design (2). In the former, the treatment periods are completely randomized without regard to ordering (i.e., without blocking). This method has the advantage of satisfying the underlying assumption of some statistical tests commonly used in single-patient experiments and is easy to implement. However, it suffers from the risk of generating treatment patterns that make causal inference difficult, being more susceptible to period effects. For example, in an N-of-1 study of two therapies with three treatment periods each, an unconstrained design may result in three periods on the first treatment followed by three treatments periods on the second treatment (i.e., AAABBB). Should the second treatment appear superior, one may be left wondering if it is the better therapy or if the patient’s symptoms were beginning to remit spontaneously. In typical crossover studies, this problem is addressed by randomizing participants to different interventions for the initial treatment period, but this option is not available in the N-of-1 design with a single participant. An alternative to the unconstrained randomized design is the blocked randomized design. In this design (using the paradigm of the two-treatment comparison), the treatment periods are randomized in pairs (i.e., blocks of two) so that each treatment is subsequently compared with the other treatment after every two treatment periods, though the specific ordering of the comparison is random. For example, a blocked randomization design might result in treatment ordering of ABBABA, BABABA, or BAABBA. A related

The choice of the number and length of the treatment periods will be a function of the particular characteristics of the illness and therapies under study, the tolerance of the patient for longer trial lengths (and burden of data collection), the need to obtain the clinical information more quickly, the availability of resources for conducting the trial, and the specific outcomes studied. No definitive advice is applicable to all situations. Of course, in general, the longer the trial and the greater the number of crossover periods, the more information will be obtained, and the more stable the resultant estimates of effect will be. 3.3 Run-In and Washout Periods One of the difficulties in all crossover designs is the potential for carryover effects—the possibility that the effect of one treatment has not entirely dissipated when the next treatment is instituted. For example, a crossover comparison of two treatments, A and B, will actually be a comparison of A followed by a comparison of B combined with some effect of A if the effects of treatment A persist into the B-treatment period. Several approaches are available for addressing the carryover effect. A ‘‘run-in’’ period (an initial period off all active treatment, which may be a placebo-assigned period) is often advocated to wash out the effect of any remaining active therapies the patient may have used. A conditiondependent crossover rule may be specified; that is, after treatment A is stopped, the patient’s clinical condition is allowed to return to baseline before starting treatment B. A time-dependent crossover rule is an alternative: a specific amount of time is required to pass before the patient is crossed over to the next therapy. One implementation method for the time-dependent crossover design is to discard data from the first part of each treatment period and analyze only

4

N-OF-1 RANDOMIZED TRIALS

the data from the latter part of each treatment period, timing the data-discard period to an adequate washout period for the prior treatment. For example, Nikles et al. (4) performed a set of N-of-1 studies of medical therapy for osteoarthritis; the treatment periods were 2 weeks long, but the data for only the second week of treatment in each period was used for analysis. Additional advantages to this latter technique are that participants are never off the study drug (which may improve adherence) and using the active treatment for a period of time before true data collection starts allows for some stability of drug levels and drug effects before the clinical response is measured. Another variant is an open-label run-in period to assess the tolerability of the study treatment, ensure that some response is evident, and/or perform an initial dose-finding analysis; this information may help shorten the overall duration of the trial (7, 9). 3.4 Blinding One of the great advantages of a formal Nof-1 study over a typical therapeutic trial is the potential to blind (or ‘‘mask’’) the therapies under study. Successful blinding can help distinguish the placebo effect from the specific treatment effect of a trial agent and potentially provide more meaningful data, depending on the needs of the patient and clinician. Blinding a pharmaceutical N-of-1 trial, however, requires the formulation of a credible placebo (for placebo-controlled trials) or formulation of similar-appearing medications for active-controlled trials. Producing these products is often difficult and can be an insurmountable problem for clinicians who are not supported by an infrastructure to help conduct these studies. Some institutions have created services to support clinicians interested in conducting N-of-1 trials (see section 5.3). 3.5 Choice of Outcome Measures The N-of-1 trial is unique in that the choice of outcome measures is often far less a function of credibility to the research community or the general population of patients than of personal importance to the individual patient

and clinician. For those trials that are conducted to better inform a single patient about the effectiveness or side effects of a particular intervention, it is critical that the outcomes chosen for the study be the ones that will have the greatest impact on the decision to continue, terminate, or change therapy. Therefore, although a large-scale randomized trial of a therapy for osteoarthritis, for example, might choose well-validated measures of pain, stiffness, and physical function, the outcome chosen for an N-of-1 trial of osteoarthritis might be simple ordinal scales of very specific issues such as the degree of resting pain and/or the ability to dance, ride a bicycle, or walk to the grocery store. Alternatively, simple dichotomous choices or ordinal preference scores may also be used (e.g., ‘‘Which did you prefer overall, treatment with A or B?’’ and ‘‘How much better was your preferred treatment than the other one?’’). For example, Pope, et al. (12) conducted a set of N-of-1 trials of a nonsteroidal inflammatory drug for patients with osteoarthritis. Each trial consisted of 2-week treatment pairs; after each pair, the patient was asked to select the preferred study medication. If he or she was unable to do so, up to two additional blinded treatment pairs were attempted. For multiple N-of-1 trials, a more uniform choice of endpoints may be required. However, the number of patients treated with these designs is generally small, and the emphasis may still be on the relevance of the response to the individual patients. In these situations, very specific outcomes or simple choice outcomes may still be chosen, but it is desirable that the scale on which the outcomes are measured be uniform to permit easier aggregation of the data (10). 3.6 Outcome Assessment Like any clinical trial, an unbiased assessment of outcomes is critical. For fully blinded trials, achieving this objective is relatively straightforward. For incompletely blinded trials, an independent outcome assessment method should be employed whenever possible to guard against the biases that may be held by the clinician-investigator. In many trials, multiple assessments of the outcomes over time are often employed

N-OF-1 RANDOMIZED TRIALS

to increase the power of the analysis (e.g., by use of a symptom diary). This additional burden on the patient should be discussed in detail before conducting the trial to enhance adherence. Additional work by highly motivated patients, however, may not be entirely undesirable. For example, in a series of Nof-1 studies of children with attention deficit/ hyperactivity disorder, follow-up interviews with patients and their parents found very positive impressions of the experience of participating in the N-of-1 studies; these included the data collection aspect, which was viewed favorably by many patients as a means of feeling more invested in the study and providing more information (13). 4

STATISTICAL ISSUES

One of the more controversial and underdeveloped aspects of N-of-1 trials is the issue of the statistical design and analysis. Indeed, some investigators have argued that statistical analysis is often superfluous, being irrelevant to the purpose of the trial. Should statistical analysis be desired, data features such as correlation between repeated measures and unequal variances by treatment period must be addressed. 4.1 Should Statistical Analysis of an N-of-1 Study be Conducted at All? It has been suggested that a formal statistical analysis of data from an N-of-1 study need not be undertaken when the goal is to help a patient make a simple therapeutic decision (14). The argument is that the decision is unavoidable. In this context, issues of statistical significance are immaterial: the patient should simply choose the therapy that appears to be most beneficial. Because the decision will be based on evidence gathered in a structured (and, possibly, blinded) fashion, it will be better informed than any alternative, regardless of P-values and confidence intervals. Critical to this argument is the need to present the data in a manner understandable to the patient; graphical methods are often preferred, though variability in interpretation of graphical presentations can prove problematic (15).

5

This line of reasoning is not endorsed by all investigators and several methods of analysis have been proposed, as will be discussed. Even if one accepts that formal statistical testing is not relevant for some clinical applications, there may still be times when it assists decision making. For example, consider a patient at high risk for coronary heart disease who may strongly desire the protective effects of a cholesterol-lowering medication but is concerned about recent gastrointestinal upset that she believes may be caused by the drug. For this patient, there may be a need for a high degree of certainty that any attribution of the symptoms to the drug not be the result of random error, so statistical design and analysis would be helpful for this patient’s decision making. Clearly, if the goal of conducting a series of N-of-1 trials is to generalize beyond the patients in the studies, a formal statistical analysis is required. 4.2 Statistical Analysis Options for the Multiple N-of-1 Study The statistical analysis of N-of-1 studies focuses on estimation of the magnitude and significance of changes in the expected value of the response between periods of treatment change and must accommodate correlation between the repeated measures of the study subject. The repeated responses gathered in an N-of-1 study form a time series, or more accurately an interrupted time series (16), and data analysts could analyze such data using classic time series methods (17). However, the lengths of typical N-of-1 series tend to be shorter than a typical time series, and popular approaches commonly involve linear model methods instead of classic time series approaches. Rochon (18) described the following useful linear model approach to analyze data from an N-of-1 study. Let yit denote the response observed at the tth time point (t = 1, . . . , T) during the i-th treatment period (i = 1, . . . , I). For notational convenience, we assume equal treatment period lengths, but the methods extend easily to settings with unequal period lengths. Let yTi = (yi1 , . . . , yiT) denote the set of T responses gathered in the i-th treatment period and Xi denote a Txp

6

N-OF-1 RANDOMIZED TRIALS

matrix of p explanatory variables. Rochon proposes the linear model, yi = XTi β + ei , where ei = (ei1 , . . . ,eiT ) is a vector of error terms representing unexplained variation in the repeated response. This linear model is quite general and can describe features such as treatment effects, time, and carryover effects. Because the N-of-1 studies gather repeated measures on the same study subject, it is typically unreasonable to assume that the error terms are uncorrelated. Rather, one assumes that the repeated measures are correlated, often with an autoregressive structure. For example, as in Rochon (18), one might assume that the errors follow a first order autoregressive process, eit = ρi ei,t−1 + uit where ρ i is the autoregressive correlation parameter, with |ρ i | < 1 and uit as mutually independent random variables with mean zero and variance σi2 (1 − ρi2 ). Rochon and others found that both the variability and longitudinal correlation of responses from N-of-1 studies seemed to vary with treatment period (18). To address this, useful models allow separate variance and correlation parameters for each of the treatment periods. One typically estimates the model parameters and variability using maximum likelihood methods and can express the estimated magnitude of treatment effects using standard confidence interval methods. Finally, one can test that observed treatment effects are greater than expected by chance using standard hypothesis testing methods such as likelihood ratio tests and associated significance probabilities. Spiegelhalter (14) noted that patients can use such significance probabilities to assess the confidence associated with statements about the superiority of one treatment over another. 5

OTHER ISSUES

5.1 Do N-of-1 Trials Achieve Their Goals? One standard for assessing the success of Nof-1 studies is the extent to which patients accept the results of the study and adhere to the recommended therapy (which may be

no therapy at all). Several authors have measured the proportion of patients who accepted the results of their N-of-1 studies and adopted the therapy recommended by their trials in follow-up. Clearly, large numbers of patients and physicians who participate in N-of-1 studies use the information in their clinical decision making, as evidenced by substantial proportions of patients who ultimately change their choice of therapy (19–22). However, on follow-up, several investigators have found many patients ultimately elect a treatment strategy that is inconsistent with the results of their N-of-1 trial (10, 23, 24). It is unclear why rates of post-trial adherence are not higher. In a study of N-of-1 trials of quinine for the treatment of nocturnal leg cramps, the investigators found that all 10 patients completing the trials elected to use quinine after their studies, despite the fact that the drug was clearly effective only in three. The authors attributed this, in part, to the fact that they did not sufficiently discuss the implications of inconclusive or negative findings with patients prior to initiating the trials (24). Another potential explanation for low rates of adopting the apparently superior therapy after an N-of-1 trial may be that many of these trials may have been instigated by the clinician or investigator, not the patient. It is difficult to glean this aspect of trial initiation from the available publications but, as noted above, it is important that the patient perceive a value to the Nof-1 study or their adherence to its results may be diminished. This problem may also account, in part, for the relatively high rates of withdrawal from many studies (10, 23, 24), though many of these ‘‘withdrawals’’ may simply be early recognition of effectiveness or toxicity, resulting in early termination of the trial. Finally, even when no statistical testing is performed, many patients may intuit that observed differences in the trial were of small degree or the results unstable and act accordingly. Greater understanding of how N-of-1 trial results are understood and used by patients is clearly needed. Despite the inconsistent rates of adopting the superior therapy observed in some studies of N-of-1 trials, many studies attest to high levels of patient satisfaction with

N-OF-1 RANDOMIZED TRIALS

the procedure. Even when the patient’s final decision is not entirely consistent with the information gleaned from the study, most patients report that the trial was useful and provided helpful information as they made their therapeutic decisions (8, 13, 20, 21, 23). Two groups have formally compared patients concurrently randomized to an N-of-1 study versus usual care for therapeutic decision making. Mahon et al. (25) randomized 31 patients with chronic obstructive pulmonary disease with an unclear response to theophylline to take part in an N-of-1 study or usual care. At 6 months, there was less theophylline use among the patients allocated to the N-of-1 trials with no apparent decline in their pulmonary symptoms. In a later study, the same investigators randomized 68 comparable patients using a similar design. Over a 1-year follow-up, the authors found no significant differences between groups in symptoms or theophylline use (26). Finally, Pope et al. (12) randomized 51 patients with osteoarthritis to an N-of-1 study or usual care for deciding whether a non-steroidal anti-inflammatory drug was helpful for their arthritis symptoms. Over the 6-month followup period, the investigators found no differences in use of the drug and apparent response. All studies found that the N-of-1 trials increased total medical-care costs for the duration of the trial period. 5.2 Ethics of N-of-1 Trials All N-of-1 trials, regardless of technique or intent, are intervention studies. As such, the ethics of conducting an N-of-1 trial must be considered carefully. Certainly, a set of multiple N-of-1 trials designed primarily to provide generalizable data is little different from other crossover studies and the full complement of ethical safeguards is likely to be required in this context. Such safeguards include full written informed consent and approval by an institutional review board (IRB). It is less clear that all ethical protections required in typical research studies are required for N-of-1 studies that are designed to help an individual patient make a more informed personal therapeutic decision. Some authors contend that written, informed

7

consent and IRB approval are not necessary, since these studies are simply more structured methods of conducting therapeutic trials, a procedure performed regularly in routine clinical practice (27). 5.3 N-of-1 Trial Support Services Single-patient studies are not commonly performed. Numerous barriers exist to wider application of these methods: unfamiliarity on the part of patients and clinicians, hesitancy to conduct ‘‘studies’’ in the clinical setting, and additional burdens of time and trouble to carry out the study to make ‘‘simple’’ clinical decisions. One of the greatest barriers to wider implementation is the lack of infrastructure for performing these studies: placebos (or indistinguishable active drugs) must be formulated, a randomization scheme must be prepared (to which the clinician and patient are blind), and data must be collected and presented and/or analyzed. Few clinicians have access to these services or expertise (28). Recognizing this difficulty, some academic institutions have established ‘‘N-of-1 Services’’ that support clinicians who desire to use these methods. Investigators at McMaster University and the University of Washington have published their experiences with such services in the 1980s and 1990s (3, 20, 29, 30). Both groups of investigators found the service to be feasible and the results useful to patients and clinicians. A more ambitious effort was organized by Nikles et al. (22) in Australia. These investigators established an N-of-1 trial support service that was available to clinicians throughout the country who desired to carry out a single-patient trial in their practice. The success of this venture was documented in the completion of a large number of trials in conditions such as attention deficit/hyperactivity disorder and osteoarthritis (4, 13, 22). 5.4 Novel Uses of N-of-1 Studies The utility of N-of-1 studies enables a wide variety of applications. In addition to the common use for informing clinical decisions for individual patients, N-of-1 studies have found value in other settings.

8

N-OF-1 RANDOMIZED TRIALS

In one recent randomized, double-blind, parallel-comparison clinical trial, a participant considered withdrawing from the study because he was concerned that the study medication raised his blood pressure. Rather than conduct a simple therapeutic trial of withdrawing the study medication, the investigators offered the participant the option of taking part in an N-of-1 study (using the study medication and known placebo), which he selected. The N-of-1 study showed that, relative to placebo, his assigned study medication had minimal effects on his blood pressure, and he elected to continue in the trial of the study medication (31). It is notable that such N-of-1 studies are relatively simple to perform because there are no problems with the availability of placebo, a research pharmacist, data collection infrastructure, and personnel for presenting and analyzing the data. Guyatt et al. (32) describe another potential use of the multiple N-of-1 method: as a means of gathering critical information about a new drug in development. Such trials can be designed to provide a wide range of information useful for planning larger, fully powered phase III studies. The kinds of information gleaned from such studies include defining the population of patients most likely to benefit from the drug, determining the optimal dose, estimating the rapidity of the drug’s onset of action and loss of treatment effect, identifying the types of outcomes most responsive to the drug, and estimating the magnitude of a potential treatment effect. The investigators provided an example of the use of a tricyclic antidepressant medication for the treatment of fibromyalgia. The same technique has also been used in the initial study of a device for treatment of Tourette’s syndrome (33). 6

CONCLUSIONS

Although N-of-1 trials have been employed in the mental health field for many decades, interest has grown in wider applications of the method in recent years. Because the focus of this type of study is on the individual patient, there are several unique aspects to the design, conduct, and interpretation of the

results of these studies. Serious infrastructure barriers to wider implementation often exist, as do issues of adherence and optimizing the value of the N-of-1 trial for its intended purpose. Analytic issues continue to evolve, and, as clinicians and patients become more familiar with the technique, N-of-1 trials may play a more prominent role in the application of evidence-based, individualized therapeutics. Patient satisfaction with N-of1 trials is high, but the incremental benefit of this methodology over standard practice remains uncertain. REFERENCES 1. G. Guyatt, D. Sackett, D. W. Taylor, J. Chong, R. Roberts, and S. Pugsley, Determining optimal therapy—randomized trials in individual patients. N Engl J Med. 1986; 314: 889–892. 2. R. Jaeschke, D. Cook, and D. L. Sackett, The potential role of single-patient randomized controlled trials (N-of-1 RCTs) in clinical practice. J Am Board Fam Pract. 1992; 5: 227–229. 3. E. B. Larson, N-of-1 clinical trials. A technique for improving medical therapeutics. West J Med. 1990; 152: 52–56. 4. C. J. Nikles, M. Yelland, P. P. Glasziou, and C. Del Mar, Do individualized medication effectiveness tests (N-of-1 trials) change clinical decisions about which drugs to use for osteoarthritis and chronic pain? Am J Ther. 2005; 12: 92–97. 5. P. M. Peloso, Are individual patient trials (Nof-1 trials) in rheumatology worth the extra effort? J Rheumatol. 2004; 31: 8–11. 6. B. Spilker, Single-patient clinical trials. In: Guide to Clinical Trials. Philadelphia: Lippincott-Raven, 1996, pp. 277–282. 7. D. J. Cook, Randomized trials in single subjects: the N of 1 study. Psychopharmacol Bull. 1996; 32: 363–367. 8. D. R. Zucker, R. Ruthazer, C. H. Schmid, J. M. Feuer, P. A. Fischer, et al., Lessons learned combining N-of-1 trials to assess fibromyalgia therapies. J Rheumatol. 2006; 33: 2069–2077. 9. G. Guyatt, D. Sackett, J. Adachi, R. Roberts, J. Chong, D., et al., A clinician’s guide for conducting randomized trials in individual patients. CMAJ. 1988; 139: 497–503. 10. A. C. Wegman, D. A. van der Windt, W. A. Stalman, and T. P. de Vries, Conducting research in individual patients: lessons learnt from two

N-OF-1 RANDOMIZED TRIALS series of N-of-1 trials. BMC Fam Pract. 2006; 7: 54. 11. D. R. Zucker, C. H. Schmid, M. W. McIntosh, R. B. D’Agostino, H. P. Selker, and J. Lau, Combining single patient (N-of-1) trials to estimate population treatment effects and to evaluate individual patient responses to treatment. J Clin Epidemiol. 1997; 50: 401–410. 12. J. E. Pope, M. Prashker, and J. Anderson, The efficacy and cost effectiveness of N of 1 studies with diclofenac compared to standard treatment with nonsteroidal antiinflammatory drugs in osteoarthritis. J Rheumatol. 2004; 31: 140–149. 13. C. J. Nikles, A. M. Clavarino, and C. B. Del Mar, Using N-of-1 trials as a clinical tool to improve prescribing. Br J Gen Pract. 2005; 55: 175–180. 14. D. J. Spiegelhalter, Statistical issues in studies of individual response. Scand J Gastroenterol Suppl. 1988; 147: 40–45. 15. K. J. Ottenbacher and S. R. Hinderer, Evidence-based practice: methods to evaluate individual patient improvement. Am J Phys Med Rehabil. 2001; 80: 786–796. 16. T. D. Cook TD and D. T. Campbell, Quasiexperimentation: Design And Analysis Issues. Boston: MA: Houghton-Mifflin, 1979. 17. P. J. Diggle, Time Series: A Biostatistical Introduction. Oxford: Clarendon Press, 1990. 18. J. Rochon, A statistical model for the ’’N-of-1’’ study. J Clin Epidemiol. 1990; 43: 499–508. 19. G. H. Guyatt and R. Jaeschke, N-of-1 randomized trials—where do we stand? West J Med. 1990; 152: 67–68. 20. E. B. Larson, A. J. Ellsworth, and J. Oas, Randomized clinical trials in single patients during a 2-year period. JAMA. 1993; 270: 2708–2712. 21. L. March, L. Irwig, J. Schwarz, J. Simpson, C. Chock, and P. Brooks, N of 1 trials comparing a non-steroidal anti-inflammatory drug with paracetamol in osteoarthritis. BMJ. 1994; 309: 1041–1046. 22. C. J. Nikles, G. K. Mitchell, C. B. Del Mar, A. Clavarino, and N. McNairn, An N-of-1 trial service in clinical practice: testing the effectiveness of stimulants for attention-deficit/hyperactivity disorder. Pediatrics. 2006; 117: 2040–2046. 23. M. A. Kent, C. S. Camfield, and P. R. Camfield, Double-blind methylphenidate trials: practical, useful, and highly endorsed by families. Arch Pediatr Adolesc Med. 1999; 153: 1292–1296.

9

24. R. Woodfield, F. Goodyear-Smith, and B. Arroll, N-of-1 trials of quinine efficacy in skeletal muscle cramps of the leg. Br J Gen Pract. 2005; 55: 181–185. 25. J. Mahon, A. Laupacis, A. Donner, and T. Wood, Randomised study of N of 1 trials versus standard practice. BMJ. 1996; 312: 1069–1074. 26. J. L. Mahon, A. Laupacis, R. V. Hodder, D. A. McKim, N. A. Paterson, et al., Theophylline for irreversible chronic airflow limitation: a randomized study comparing N of 1 trials to standard practice. Chest. 1999; 115: 38–48. 27. L. Irwig, P. Glasziou, and L. March, Ethics of N-of-1 trials. Lancet. 1995; 345: 469. 28. K. B. Saunders, N of 1 trials. BMJ. 1994; 309: 1584. 29. G. H. Guyatt, J. L. Keller, R. Jaeschke, D. Rosenbloom, J. D. Adachi, and M. T. Newhouse, The N-of-1 randomized controlled trial: clinical usefulness. Our three-year experience. Ann Intern Med. 1990; 112: 293–299. 30. J. L. Keller, G. H. Guyatt, R. S. Roberts, J. D. Adachi, and D. Rosenbloom, An N of 1 service: applying the scientific method in clinical practice. Scand J Gastroenterol Suppl. 1988; 147: 22–29. 31. A. L. Avins, S. Bent, and J. M. Neuhaus, Use of an embedded N-of-1 trial to improve adherence and increase information from a clinical study. Contemp Clin Trials. 2005; 26: 397–401. 32. G. H. Guyatt, A. Heyting, R. Jaeschke, J. Keller, J. D. Adachi, and R. S. Roberts, N of 1 randomized trials for investigating new drugs. Control Clin Trials. 1990; 11: 88–100. 33. J. L. Houeto, C. Karachi, L. Mallet, B. Pillon, J. Yelnik, et al., Tourette’s syndrome and deep brain stimulation. J Neurol Neurosurg Psychiatry. 2005; 76: 992–995.

CROSS-REFERENCES Crossover design Randomization Placebo-controlled trial Washout period Blinding

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

Non-compartmental analysis (NCA) is a standard technique to determine the pharmacokinetics (PK) of a drug. After drug intake, the concentration time profiles (e.g., in plasma or serum) are recorded and used to characterize the absorption, distribution, and elimination of the drug. Less frequently, concentrations in blood, saliva, other body fluids, or amounts excreted unchanged in urine are used instead of or in addition to plasma or serum concentrations. NCA is the most commonly used method of PK data analysis for certain types of clinical studies like bioequivalence, dose linearity, and food effect trials. The common feature of non-compartmental techniques is that no specific compartmental model structure is assumed. The most frequently applied method of NCA is slope height area moment analysis (SHAM analysis) (1,2). For SHAM analysis, the area under the concentration time curve (AUC) is most commonly determined by numerical integration or by curve fitting. Numerical integration is the non-compartmental method of choice for analysis of concentration time data after extravascular input because absorption kinetics are often complex. In comparison to compartmental modeling, numerical integration has the advantage that it does not assume any specific drug input kinetics. For intravenous bolus data, fitting the concentration time curves by a sum of exponentials is the non-compartmental method of choice. This introductory article presents some standard applications of NCA of plasma (or serum) concentration data, as those applications are most commonly used. References to NCA of urinary excretion data and more advanced topics of NCA are provided. This article focuses on exogenous compounds and does not cover endogenous molecules. Our focus is (1) to describe studies and objectives for which NCA is well suited, (2) to provide and discuss assumptions of NCA, and (3) to present a practical guide for performing an NCA by numerical integration and basic approaches to choosing appropriate blood sampling times.

¨ JuRGEN B. BULITTA

Department of Pharmaceutical Sciences, School of Pharmacy and Pharmaceutical Sciences, State University of New York at Buffalo, Buffalo, New York

NICHOLAS H. G. HOLFORD Department of Pharmacology and Clinical Pharmacology, University of Auckland, Auckland, New Zealand

Objectives of NCA are often assessing dose proportionality, showing bioequivalence, characterizing drug disposition, and obtaining initial estimates for pharmacokinetic models. Specific results and applications of NCA are as follows: (1) The area under the concentration time curve (e.g., in plasma or serum) describes the extent of systemic drug exposure; the peak concentration and its timing indicate the rate of drug input (absorption), and (2) NCA provides estimates for clearance, volume of distribution, terminal half-life, mean residence time, and other quantities. Application 1 serves purely descriptive purposes and requires almost no assumptions. Importantly, application 2 does rely on several assumptions that are similar to the assumptions for compartmental modeling. Standard NCA requires frequent (blood) samples to estimate pharmacokinetic parameters reliably. Numerical integration is most commonly used for NCA after extravascular input. Fitting of disposition curves by a sum of exponential functions, for example is the method of choice for intravenous bolus input. NCA of an adequately designed clinical trial can provide robust estimates for extent of drug exposure and rate of absorption and other quantities. NCA estimates for clearance and volume of distribution rely on several assumptions that have to be critically considered for appropriate interpretation of NCA results.

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

1 TERMINOLOGY 1.1 Compartment A compartment is a hypothetical volume that is used to describe the apparent homogeneous and well-mixed distribution of a chemical species. ‘‘Kinetically homogeneous’’ assumes that an instantaneous equilibration of a chemical compound (drug or metabolite) is found between all components of the compartment. 1.2 Parameter A parameter is a primary constant of a quantitative model that is estimated from the observed data. For example, clearance (CL) and volume of distribution at steady state (Vss) are the two most important PK parameters. 1.3 Fixed Constant Some models include fixed constants that are known (or assumed) a priori and are not estimated. Examples are stoichiometric coefficients or π . 1.4 Statistic A statistic is a derived quantity that is computed from observed data or estimated model parameters. Examples: The average plasma concentration is a statistic because it is computed from observed concentrations. Another statistic is the AUC. Under certain assumptions, the AUC is F · Dose / CL with F being the fraction of drug that reaches the systemic circulation. The average clearance and its standard deviation are two statistics that can be calculated from individual clearance estimates of several subjects. 1.5 Comment A one-compartment PK model can be defined by any two parameters, for example, by CL and Vss, by CL and t1/2 , or by Vss and t1/2 . NCA estimates CL and t1/2 and derives Vss from other NCA statistics. However, physiologically, the parameterization by CL and Vss is more informative than the other two parameterizations because CL and Vss characterize the physiology of the body and the

physicochemical properties of the drug. Halflife is determined by the primary PK parameters clearance and volume of distribution and should be called a statistic (or secondary parameter). For more complex PK models, Vss is calculated by residence time theory (see below and Reference 3). 2 OBJECTIVES AND FEATURES OF NON-COMPARTMENTAL ANALYSIS Non-compartmental techniques can be applied to PK analyses of studies with a variety of objectives. Some of those objectives are: 1. Characterization of drug disposition. 2. PK analysis of various types of studies, including: • Bioavailability and bioequivalence studies, • Food-effect studies, • PK interaction studies, and • Dose-proportionality studies. 3. Supporting development of pharmaceutical formulations by characterizing drug absorption profiles. 4. Obtaining initial estimates for compartmental modeling. NCA has appealing features, and the majority of studies in literature only report results from NCA. Some specific results from NCA and its features are: 1. NCA can provide robust estimates for important PK parameters like clearance and volume of distribution, if the assumptions of NCA are adequately met. 2. Descriptive measures for the rate of absorption like peak concentration (Cmax) and its timing (Tmax) can be directly derived from the observed data. 3. No need exists to specify the full structure of the compartmental model. 4. NCA and plots of the observed concentration time data may provide valuable insights into the PK characteristics of a drug and can be very helpful for testing model assumptions and for building compartmental models.

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

5. Standard NCA can be learned quickly and is often straightforward to apply. 6. NCA requires minimum decision making by the user, can be highly standardized, and most often yields consistent results between different users. However, it is important to be aware that NCA may be suboptimal or more difficult to apply in situations that need: 1. to analyze sparse data (insufficient to characterize the shape of the profile before the log-linear terminal phase), 2. to derive PK parameters from concentration time profiles of complex dosage regimens, 3. to simulate other-than-the-studied dosage regimens [powerful methods do exist for this task, but they are underused (4)], or 4. to study drugs with mixed-order (‘‘saturable’’) elimination or time-dependent PK. 2.1 Advanced Non-Compartmental Techniques Various advanced non-compartmental techniques have been developed. A detailed presentation of these concepts is beyond the scope of this article. Some situations for which those advanced methods are superior to standard NCA are listed below. Non-compartmental methods to analyze data with saturable elimination (Michaelis– Menten kinetics) were developed by several authors (5–13). Methods for the analysis of metabolite data (14–18), reversible metabolism (19–27), maternal–fetal disposition (28), enterohepatic recirculation (29), nonlinear protein binding (30) or capacity limited tissue-distribution (31), organ clearance models (19,32), target-mediated drug disposition (33,34), and for modeling a link compartment (35) are available. Noncompartmental methods for sparse sampling (36–43) are often applied in preclinical PK and toxicokinetics. Veng-Pedersen presented (4,44) an overview of linear system analysis that is a powerful tool to determine the absorption, distribution, and elimination

3

characteristics and to predict concentrations for other dosage regimens. Although these advanced methods have been developed, they are not as widely used as standard NCA, and some of these methods are not available in standard software packages. 3 COMPARISON OF NON-COMPARTMENTAL AND COMPARTMENTAL MODELS NCA is often called ‘‘model independent,’’ although this phrase is misleading and often misinterpreted (45–47). The common feature of non-compartmental techniques is that they do not assume a specific compartmental model structure. NCA only can be described as (fully) model independent when it is exclusively used for descriptive purposes, for example, for recording the observed peak concentration (Cmax) and its associated time point (Tmax). Importantly, such a descriptive model-independent approach cannot be applied for extrapolation purposes. The AUC from time zero to infinity (AUC0−∞ ) can be interpreted as a descriptive measure of systemic drug exposure (e.g., in plasma) for the studied dosage regimen. This interpretation requires the assumption of a monoexponential decline in concentrations during the terminal phase (or another extrapolation rule). However, the AUC is commonly used to calculate F, for example, in bioavailability and bioequivalence studies. As the AUC is determined by F · dose / clearance, the use of AUC to characterize F assumes that clearance is constant within the observed range of concentrations and over the time course of the data used to calculate the AUC. Therefore, all assumptions required for determination of clearance (see below) are implicitly made, if AUC is used to characterize F. Figure 1 compares the standard noncompartmental model and a compartmental model with two compartments. An important assumption of NCA is that drug is only eliminated from the sampling pool. The two models shown in Fig. 1 differ in the specification of the part of the model from which no observations were drawn (nonaccessible part of the system). In this example,

4

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

Figure 1. Comparison of the standard non-compartmental model and a compartmental model with two compartments.

the models differ only in the drug distribution part. For compartmental modeling, the user has to specify the complete model structure including all compartments, drug inputs, and elimination pathways. In Fig. 1 (panel b), drug distribution is described by one peripheral compartment that is in equilibrium with the central compartment. The ‘‘non-compartmental’’ model does not assume any specific structure of the nonaccessible part of the system. Any number of loops can describe the distribution, and each loop can contain any number of compartments. A more detailed list of the assumptions of standard NCA is shown below.

4 ASSUMPTIONS OF NCA AND ITS REPORTED DESCRIPTIVE STATISTICS NCA parameters like clearance and volume of distribution only can be interpreted as physiological or pharmacological properties of the body and the drug of interest, if the assumptions of NCA are adequately met. NCA relies on a series of assumptions. Violating these assumptions will cause bias in some or all NCA results. This bias has been quantified and summarized by DiStefano and Landaw (45,46) for several examples. Table 1 compares several key assumptions (36–50) between standard NCA and compartmental modeling. One key assumption of standard NCA is linear drug disposition [see Veng-Pedersen (44,50), Gillespie (47), and DiStefano and Landaw (46) for details]. Although advanced methods to account for

nonlinear drug disposition have been developed [see, e.g., Cheng and Jusko (8)], these techniques are seldom applied. As shown in Table 1, the assumptions for a compartmental model with first-order disposition (‘‘linear PK’’) are similar to the assumptions of standard NCA. Compartmental models offer the flexibility of specifying various types of nonlinear disposition. This process is straightforward, if a compartmental model is specified as a set of differential equations. Below is a discussion of the assumptions of NCA and compartmental modeling. 4.1 Assumptions 1 (see Table 1): Routes and Kinetics of Drug Absorption Compartmental modeling uses a parametric function (e.g., first-order or zero-order kinetics) for drug absorption, whereas NCA does not assume a specific time course of drug input. 4.2 Assumptions 2 to 4 (see Table 1): Drug Distribution Standard NCA does not assume a specific model structure for the nonaccessible part of the system (‘‘distribution compartments’’). It only assumes that drug distribution is linear, for example, neither the rate nor the extent of distribution into the peripheral compartment(s) is saturable. The user has to specify the complete structure and kinetics of drug transfer between all distribution compartments for compartmental modeling. Nonlinear drug disposition can be incorporated into compartmental models. Most compartment models assume linear

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

Table 1. Comparison of Assumptions Between Standard NCA and Compartmental Modeling

5

6

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

distribution. In this case, the assumptions for drug distribution are similar for noncompartmental and compartmental methods. 4.3 Assumption 5 (see Table 1): Routes of Drug Elimination Standard NCA assumes that all elimination occurs only from the sampling pool. Most compartmental models also assume that drug is eliminated only from the sampling (central) compartment (no elimination from any peripheral compartment). This assumption seems reasonable because the liver and the kidneys as the two main elimination organs are highly perfused organs and therefore in a rapid equilibrium with plasma (or serum), which is most commonly used for PK analysis. DiStefano and Landaw (45,46) discussed this assumption and the implications of its violation in detail. Other routes of elimination can be specified for compartmental modeling. Nakashima and Benet (51) derived formulas for linear mammillary compartmental models with drug input into any compartment and drug elimination from any compartment. 4.4 Assumption 6 (see Table 1): Kinetics of Drug Elimination Standard NCA assumes that clearance is not saturable. As metabolism and renal tubular secretion are potentially saturable, the results of standard NCA for drugs with saturable elimination need to be interpreted cautiously. This cautious interpretation is especially important when plasma concentrations exceed the Michaelis– Menten constant of the respective saturable process. Several NCA methods that can account for saturable elimination are quoted above. However, those methods are not implemented in software packages like WinNonlinTM Pro or Thermo Kinetica , which makes them unavailable to most users. Compartmental modeling offers a great flexibility for specifying the elimination pathways and the kinetics of the elimination process (e.g., first-order, zero-order, mixedorder, target-mediated, etc.)

4.5 Assumptions 7 and 8 (see Table 1): Sampling Times and Monoexponential Decline For adequately chosen sampling times, the assumption that the last three (or more) observed concentrations characterize a monoexponential decline of concentrations during the terminal phase is often reasonable [see Weiss for details (52)]. Standard NCA requires that the samples be collected during the whole concentration time profile. This time period is usually one dosing interval at steady state or at least three terminal halflives after a single dose (see below for NCA of sparse data). For studies with frequent sampling (12 or more samples per profile) at appropriately chosen time points, the AUC usually characterizes the total drug exposure well. For single-dose studies, it should be ensured that the AUC from time zero until the last observed concentration (AUC0−last ) comprises at least 80% of AUC0−∞ . Blood sampling times should be chosen so that at least two or more observations will be observed before the peak concentration in all subjects. If the peak concentration is at the time of the first observation, then regulatory authorities like the Food and Drug Administration (FDA) (53,54) may require that a study be repeated because the safety of the respective formulation could not be established. In this situation, NCA cannot provide a valid measure for peak concentrations. Blood sampling time schedules can be optimized to select the most informative sampling times for estimation of compartmental model parameters. The latter approach is more efficient as it requires (much) fewer blood samples per patient compared with NCA and is especially powerful for population PK analyses. 4.6 Assumption 9 (see Table 1): Time Invariance of Disposition Parameters This assumption is made very often for both NCA and compartmental modeling. The disposition parameters are assumed to be timeinvariant (constant) within the shortest time interval required to estimate all PK parameters of interest. PK parameters may differ between two dosing intervals because

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

two different sets of PK parameters can be estimated for each dosing interval both by NCA and compartmental methods. Caution is indicated for data analysis of drugs with inducible metabolism or when comedication may affect hepatic metabolism of the drug. In summary, the assumptions of compartmental models with linear PK and of the standard NCA model are similar, if NCA is used to derive PK parameters like CL, Vss, and F (as is most often done). The complete model structure and kinetics of all drug transfer processes must be defined for compartmental models, whereas NCA requires fewer assumptions on the model structure. Both standard compartmental models and the standard NCA model assume linear PK [see Veng-Pedersen (44,50), Gillespie (47), and DiStefano and Landaw (46) for more details]. It is easier to specify nonlinear drug disposition for compartmental models than for NCA. 4.7 Assumptions of Subsequent Descriptive Statistics NCA does not directly make any assumption about the distribution of NCA statistics between different subjects. However, most authors report the average and standard deviation for the distribution of NCA statistics and thereby assume that the average is an appropriate measure for the central tendency and that the standard deviation characterizes the variability adequately. The distribution of Tmax is especially problematic because it is determined primarily by the discrete distribution of the nominal sampling times. It is often not possible to decide whether the between-subject variability follows a normal distribution, log-normal distribution, or another distribution. Therefore, it is often helpful to report the median and representative percentiles (e.g., 5% to 95% percentile and 25% to 75% percentile) to describe the central tendency and variability of data by nonparametric statistics in addition to the average and standard deviation. If only data on a few subjects is available, then the 10% and 90% percentiles or the interquartile range are usually more appropriate than the 5% and 95% percentiles. Reporting only the

7

median and range of the data does not provide an accurate measure for dispersion, especially for studies with a small sample size. 5

CALCULATION FORMULAS FOR NCA

This section presents numerical integration methods to determine the AUC that is most commonly used to analyze plasma (or serum) concentrations after extravascular dosing. Additionally, fitting of concentration time curves by a sum of exponential functions is described. The latter is the non-compartmental method of choice for iv bolus input. The concentration time curves after iv bolus input are called disposition curves because drug disposition comprises drug distribution and elimination but not the absorption process. Both methods are noncompartmental techniques because they do not assume a specific model structure. It is important to note that several compartmental models will result in bi-exponential concentration time profiles. It is appealing that the numerical integration of a standard NCA can be performed with a hand calculator or any spreadsheet program. Nonlinear regression to describe disposition curves can be done by standard software packages like ADAPT, WinNonlinTM Pro, Thermo Kinetica , SAAM II, EXCEL and many others. Performing an NCA with a hand calculator or self-written spreadsheets is good for learning purposes. However, the use of validated PK software for NCA of clinical trials is vital for projects to be submitted to regulatory authorities. Audit trail functionality is an important advantage of validated PK software packages. 5.1 NCA of Plasma or Serum Concentrations by Numerical Integration Numerical integration is the most commonly used method to determine the AUC and other moments of the plasma (or serum) concentration time curve after extravascular input. This method is also often applied for infusion input, if the duration of infusion is nonnegligible. Numerical integration requires a function neither to describe the whole concentration time curve nor to describe the rate of

8

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

drug absorption (e.g., first-order or zero-order input). The Cmax and Tmax are directly recorded from the observed concentration time data. NCA assumes that the concentration during the terminal phase declines monoexponentially. Subsequently, linear regression on semilogarithmic scale is used to determine the slope (−λz ) of the concentration time curve during the terminal phase. Terminal half-life is calculated as t1/2 =

ln(2) λz

A guide for the most appropriate selection of observations for estimation of terminal halflife is described below. Figure 2 illustrates the calculation of terminal half-life for one subject. In this example, the last five observations were used to estimate the terminal slope. The area under the plasma concentration time curve is most often determined by the trapezoidal method. The plasma concentration time profile is usually linearly (or logarithmically) interpolated between two observations. The formulas apply to linear and logarithmic interpolation, and Ci denotes the ith observed concentration: Linear interpolation: Ci + Ci+1 2 Ci + Ci+1 = (ti+1 − ti ) · 2 Logarithmic interpolation: = t · AUCi+1 i

= t · AUCi+1 i

Ci+1 − Ci 
Ci+1 ln Ci

Figure 3 shows the calculation of trapezoids for data after a single oral dose by the linear interpolation rule. The sum of the individual trapezoids yields the AUC from time zero to the last quantifiable concentration: AUC0−last =

n−1 

AUCi+1 (for n observations) i

i=1

Linear interpolation is usually applied, if concentrations are increasing or constant

(Ci+1 ≥ Ci ), and logarithmic interpolation is often used, if concentrations are decreasing (Ci+1 < Ci ). Note that the log-trapezoidal rule is invalid when Ci+1 = Ci and when Ci is zero. Plasma concentration time profiles are curved to the right during an intravenous infusion at a constant rate (see Fig. 4) and also tend to be curved to the right for extravascular administration. Therefore, linear interpolation to calculate the AUC is a reasonable approximation if Ci+1 ≥ Ci . After the end of infusion, drug concentrations often decline mono-, bi-, or triexponentially (see Fig. 4). The number of exponential phases can be assessed visually, when drug concentrations are plotted on log scale versus time. Therefore, assuming an exponential decline (equivalent to a straight line on log-scale) is often a good approximation for the AUC calculation if Ci+1 < Ci . Figure 4 shows that linear interpolation approximates the true plasma concentration time curve slightly better than the logarithmic interpolation during the infusion. After the end of infusion, logarithmic interpolation approximates the true curve better then linear interpolation—as expected. The difference between linear and logarithmic interpolation tends to be small, if blood samples are frequently drawn. Differences are expected to be larger for lessfrequent blood sampling. Linear and logarithmic interpolation have been compared with interpolation by higher-order polynomials and splines (55,56). For PK data, the linear trapezoidal rule for increasing or constant concentrations and the log-trapezoidal rule for decreasing concentrations is usually a good choice (19) because those methods are stable and reasonably accurate even for sparse data. Although higher-order polynomials and spline functions performed better for some datasets, these methods are potentially more sensitive to error in the data and are not available in most standard software packages. The AUC from the last concentration (Clast ) to time infinity (AUClast−∞ ) is extrapolated assuming a log-linear decline of concentrations during the terminal phase: C . AUClast−∞ = λlast z Standard software packages like WinNonlinTM Pro, Thermo Kinetica ,

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

9

Figure 2. Determination of terminal half-life after a single oral dose.

and others provide two options for specifying Clast : One can use the last observed concentration as Clast . Alternativley, the concentration predicted by log linear regression at the time of the last observation may be used as Clast to estimate λz . The current FDA guideline for bioavailability and bioequivalence studies (53) recommends that the last ‘‘measured’’ (observed) concentration should be used. The FDA guideline does not provide a reason why this method is proposed. Theoretically, the last predicted concentration will be more precise than an observed value if the assumptions for estimation of the terminal half-life are met. Use of the last observed concentration may yield a large bias, if this concentration was an ‘‘outlier’’ with a high value, because it would cause a longer half-life and therefore would affect both terms in the equation for AUClast−∞ . The AUC from time zero to infinity (AUC0−∞ ) is calculated as: AUC0−∞ = AUC0−last + AUClast−∞ The residual area is calculated as 1 − AUC0−last / AUC0−∞ . Residual areas above

Figure 3. Calculation of the AUC by the linear trapezoidal rule.

20% are considered too large for reliable estimation of AUC0−∞ and therefore also for CL. The uncertainty in the calculated area under the first moment concentration time curve from time zero to infinity (AUMC0−∞ ) is even larger compared with the uncertainty in AUC0−∞ for larger residual areas. The (apparent) total clearance (CL/F) is calculated from the administered dose and AUC0−∞ as: Dose CL = F AUC0−∞ For intravenous administration, the extent of drug reaching the systemic circulation (F) is 100% by definition, and the ratio of dose and AUC0−∞ yields total clearance (CL). As F is mathematically not identifiable after extravascular administration without an intravenous reference dose, only CL/F can be derived after extravascular administration. The volume of distribution during the terminal phase (Vz) can be calculated from CL and λz . As shown by several authors (57,58), Vz is not an independent parameter that characterizes drug disposition because it depends on, for example, the estimate for

10

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

Figure 4. Interpolation between plasma concentration time points by the linear and logarithmic trapezoidal method for a 2-h infusion (sampling times: 0, 0.5, 2, 2.5, 4, and 6 h).

clearance. Volume of distribution at steady state (Vss) is a better measure for volume of distribution than Vz. The estimate for clearance does not affect the estimate of Vss, if drug is only eliminated from the sampling pool. Vss can always be calculated from data that allow one to calculate Vz (see below). Gobburu and Holford (58) pointed out that the finding of an altered Vz, for example, in a special patient population may lead to the inappropriate conclusion of changing the loading dose in this patient population. Because of the potential misuse of Vz, the use of Vz should be discouraged (58). Statistical moment theory (59) is used to calculate the mean residence time (MRT). The MRT is calculated from AUMC0−∞ and AUC0−∞ . The AUMC is calculated as:

Linear interpolation: ti · Ci + ti+1 · Ci+1 2 Logarithmic interpolation: AUMCi+1 = t · i

= t · AUMCi+1 i

ti+1 · Ci+1 − ti · Ci
 Ci+1 ln Ci

− t2 ·

Ci+1 − Ci 2
Ci+1 ln Ci

The AUMC from time zero to the last observed concentration (AUMC0−last ) and from time zero to infinity (AUMC0−∞ ) are calculated

as: AUMC0−last =

n−1 

AUMCi+1 i

i=1

(for n observations) AUMC0−∞ = AUMC0−last +

Clast · tlast Clast + λz λz 2

For steady-state data (dosing interval: τ ), the AUMC0−∞ can be calculated as (60): AUMC0−∞ = AUMCSS,0−τ + τ · AUCSS,τ −∞ The mean residence time (MRT) is calculated as: AUMC0−∞ MRT = AUC0−∞ [The AUMC0−last and AUC0−last , should not be used for calculation of MRT, because such a calculation yields systematically biased (low) estimates for the MRT.] For an intravenous bolus administration, the MRT is equal to the mean disposition residence time (MDRT). The MDRT is the average time a drug molecule stays in the systemic circulation (44). The MDRT is often called MRTiv (iv indicates iv bolus input). We prefer to write MDRT to refer specifically to drug disposition. If peripheral venous blood is sampled, then the mean transit time of drug molecules from arterial blood to the sampling site needs to be considered for calculation of MDRT, at least for drugs with a short MDRT [see Weiss (3,61) and Chiou (62) for details].

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

Vss is calculated as: Vss = MDRT · CL For high-clearance drugs, calculation of Vss is more complex (3,19,32,63). The MDRT determines the accumulation ratio (RA ), which is defined as the average amount of drug in the body at steady state (after multiple administration) divided by the bioavailable maintenance dose (F · MD). The RA is MDRT divided by the dosing interval (3). Furthermore, MDRT determines the accumulation time and the washout time as described by Weiss (64,65). For noninstantaneous drug input (e.g., extravascular dose or constant rate infusion), the ratio of AUMC0−∞ and AUC0−∞ yields the mean total residence time (MTRT), which is the sum of the mean input time (MIT) and MDRT. For constant rate (intravenous) infusion with a given duration of infusion (Tinf ), the MIT equals Tinf / 2. Therefore, the MDRT can be calculated as: MDRT = MTRT − MIT =

Tinf AUMC0−∞ − AUC0−∞ 2

The MIT after extravascular administration is more difficult to calculate without an intravenous reference. The MIT can be determined as the difference of MTRT and MDRT, when both an extravascular and intravenous dose are given to the same subject on different occasions. To calculate Vss after extravascular administration, the MIT needs to be subtracted from the MTRT (66,67). The MIT can be approximated by Tmax/2 if the input process is close to zero-order [common for drugs in class I of the biopharmaceutical classification system (BCS) because of a zeroorder release of drug from stomach; BCS class I drugs have high solubility and high permeability]. If the process seems to be first-order, then MIT can be approximated by Tmax/(3·ln(2)) because Tmax commonly occurs at around three times the first-order absorption half-life. The extent of bioavailability (F) is typically determined by giving a single oral dose and a single intravenous dose on two different occasions (with an appropriate washout

11

period) to the same subject. From these data, CL, Vss, MDRT, MIT, and F can be calculated by the formulas shown above. The ratio of the AUC after oral and intravenous dosing characterizes F: F=

AUC0−∞,oral AUC0−∞,iv

The MIT values for each subject can be calculated as the difference of MTRT and MDRT. Assuming a first-order absorption (without lag time), MIT is the inverse of the firstorder absorption rate constant (1/ka). The MIT can be correlated with the mean dissolution time determined in vitro to establish an in vitro/in vivo correlation, for example, to compare the release characteristics of various modified release formulations (68,69). 5.2 NCA of Plasma or Serum Concentrations by Curve Fitting For NCA of concentration time curves after iv bolus administration, the results from numerical integration are very sensitive to the timing of samples and to errors in the data (e.g., measurement error or undocumented deviations in the sampling time). For such data, use of analytical functions to fit the disposition curves is the method of choice. For monotonously decreasing concentrations, a sum of exponential functions is often a good choice to describe disposition curves. No a priori reason exists for choosing a sum of exponential functions. Some authors used other functions like gamma curves (52,70–72). Most commonly, disposition curves (concentration CD (t) at time t) are described by a sum of n exponential functions: CD (t) =

n 

Ci · e−λi ·t

i=1

The Ci is the intercept of each exponential phase, and −λi is the associated slope on semilogarithmic scale. The kth moment (MOCk ) of the disposition curve can be calculated as (3): MOCk = k!

n  i=1

λi

Ci (k+1)

for k = 0, 1, 2, 3, . . .

12

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

regression is beyond the scope of this article. See Gabrielsson and Weiner (73) and the manuals of the respective software packages for details. 5.3 NCA with Plasma or Serum Concentrations and Amounts in Urine

Figure 5. Disposition curve after iv bolus administration with three exponential functions.

As AUC is the area under the zeroth-moment concentration time curve and AUMC is the area under the first-moment concentration time curve, this formula yields: AUC0−∞ =

n  Ci , λi

AUMC0−∞ =

i=1

MDRT =

n  Ci i=1

λ2i

,

n  Ci λi 2

AUMC0−∞ i=1 = n  Ci AUC0−∞ λi i=1

Figure 5 shows an example of a triexponential disposition curve. The following parameter values were used for simulation: C1 = 50 mg/L, C2 = 40 mg/L, C3 = 10 mg/L, λ1 = 2.77 h−1 , λ2 = 0.347 h−1 , and λ3 = 0.0578 h−1 . The disposition curve shows three different slopes on a semilogarithmic plot versus time. The contribution of the individual exponential functions is indicated. Fitting the concentration time profiles by a sum of exponential functions may be helpful to interpolate between observed concentrations. This sum of exponentials could be used, for example, as forcing function for a pharmacodynamic model. The parameters of this sum of exponential functions can be estimated by software packages like ADAPT, WinNonlinTM Pro, Thermo Kinetica , SAAM II, and many others. Use of an appropriate error variance model (or of an adequate weighting scheme) is highly recommended. A thorough discussion of nonlinear

Urinary excretion rates can be used instead of plasma concentrations to characterize the PK profile of drugs excreted (primarily) by the kidneys. The majority of studies use plasma (or serum) concentrations to determine the PK. If the excretion of drug into urine is studied in addition to plasma concentrations, then renal clearance (CLR ) and nonrenal clearance/F can be calculated in addition to total clearance. The total amount excreted unchanged in urine from time zero up to the last collected urine sample (Turine ) and the AUC from time zero to Turine are used to calculate CLR :

CLR =

Amount excreted unchanged in urine until time Turine AUC0−Turine in plasma

Importantly, this formula yields renal clearance (and not renal clearance/F) both for intravenous and for extravascular administration because the amount recovered in urine is known. Nonrenal clearance (CLNR ) is calculated as the difference between total (CL) and renal clearance (CLR ) for intravenous administration. Apparent nonrenal clearance is calculated as the difference between CL/F and CLR for extravascular administration. If F is less than 100%, then this difference will not be equal to CLNR after intravenous administration divided by F because the estimate for CLR does not include F. In addition to clearance, MDRT and therefore volume of distribution at steady state can be calculated from urinary excretion data as described by Weiss (3). The urinary excretion rate is calculated as amount of drug excreted unchanged in urine per unit of time. A urinary excretion rate versus time plot is one method to estimate terminal elimination half-life based on urine data [see Rowland and Tozer (74) for details].

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

The calculated half-life from urinary excretion data should be similar to the half-life calculated from plasma data. 5.4 Superposition Methods and Deconvolution Superposition methods assume that the concentration time profiles of each dose can be added if two doses are given at the same or at different times because the concentrations achieved by each dose do not influence each other (47). A linear pharmacokinetic system has this property. This situation allows one to simulate plasma concentration time profiles after multiple doses by adding up concentration time profiles after various single doses at the respective dosing times. Such a nonparametric superposition module is implemented, for example, in WinNonlinTM Pro, and a similar module (on convolution) is available in Thermo Kinetica . In pharmacokinetics, deconvolution is often used to determine the in vivo release profile, for example, of a modified release formulation. The disposition curve after iv bolus administration (impulse response of the system) can be used in combination with the plasma concentration time curve of an extravascular formulation to determine the in vivo release profile. The Wagner–Nelson method (75–78) that is based on a onecompartment model and the Loo–Riegelman method that can account for multiple disposition compartments (79,80) have been applied extensively. These methods subsequently were greatly extended (44,50,73,81–83). A detailed description of those algorithms is beyond the scope of this article. Convolution/deconvolution methods are available in WinNonlinTM Pro and in Thermo Kinetica . 6 GUIDELINES TO PERFORM AN NCA BASED ON NUMERICAL INTEGRATION Before running any PK analysis, it is very helpful to prepare various plots of the observed concentration time data. Typically, the observed individual concentration time data are plotted on linear and semilogarithmic scale (i.e., concentrations on log-scale vs. time on linear scale). A log2 scale is often a good choice to visualize concentration versus

13

time data because half-life can be visualized more easily by such a plot. These plots usually are prepared for all subjects at the same time (‘‘spaghetti plots’’) and for each subject individually. An intraindividual comparison of the observed data for different treatments is often helpful. If different doses were given to the same subject, then plotting dose-normalized concentrations is helpful to assess dose linearity visually according to the superposition principle. Calculation of descriptive statistics in addition to the individual observed data provides useful summary statistics. The average ± standard deviation and the median (25%–75% percentiles) or other representative percentiles are often plotted versus time for each treatment. These observed data plots might already reveal that some assumptions like first-order elimination might not be appropriate. Actual, instead of nominal, protocol sampling times should always be recorded and used for PK analysis. For the very common case that numerical integration by the trapezoidal rule is applied, the two most important user decisions are (1) determination of the terminal phase and (2) choice of the integration rule for the AUC. The choice of the most appropriate AUC calculation method is discussed above. Some guidelines for determining the terminal phase are shown below. 6.1 How to Select Concentration Data Points for Estimation of Terminal Half-Life The following rules provide some practical guidelines for determination of the terminal phase of the concentration time profile: 1. Define a maximum number of data points used to specify the terminal phase. During the terminal phase, drug distribution is in equilibrium. As drug distribution is usually not in equilibrium at Tmax, the terminal phase should not contain Cmax and the data points directly after Cmax. Use of at least three points and not more than five to seven points to define the terminal phase is often appropriate.

14

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

2. Select the ‘‘optimal’’ number of data points according to a statistical criterion for the goodness of fit. Programs like WinNonlinTM Pro, Thermo Kinetica , and others provide an interactive graphical interface that shows the r2 and the r2 -adjusted value (the latter is called G-criterion in Thermo Kinetica ) for the chosen data points. The highest r2 -adjusted (or r2 ) value can be used as a statistical criterion for the best fit (ideal value: 1.0). The r2 adjusted criterion might be preferred to the r2 -criterion because the r2 -adjusted criterion considers the number of data points used to derive λz , whereas the r2 -criterion does not. If the last three, four, and five data points all yield an r2 -value of 0.98, for example, the r2 adjusted criterion is highest for the last five data points. With more data points being used to estimate λz , the probability of this correlation occurring by chance decreases, and, thus, several data points may be preferable to estimate λz . Other criteria for the number of data points may also be appropriate. Proost (84) compared various criteria to estimate λz by simulating profiles for a one-compartment model and concluded that various criteria (including the r2 and r2 -adjusted criterion) had comparable bias and precision for the estimate of λz . Irrespective of the criterion chosen, it seems reasonable to ensure an appropriate selection of data points to estimate the terminal half-life by visual inspection of the observations and the predicted regression line in each subject. 3. If half-life becomes systematically shorter with more data points being used for estimation of λz , then the use of only the last three, four, (or five) data points to define the terminal phase seems reasonable. This situation often occurs for drugs that need to be described by a sum of two, three, or more exponential functions, when the distribution is non-instantaneous.

4. If half-life becomes systematically longer with more data points being used for estimation of λz , then the use of only the last three, four, (or five) data points to define the terminal phase seems reasonable. One possible reason for such an observation would be a mixed-order (Michaelis-Menten) elimination that violates the assumptions of standard NCA (see Table 1). 5. It is reasonable to use at least three data points for estimation of λz . Exception: If the third last point is Cmax, only the last two points should be used to estimate λz or λz should not be estimated in this subject. 6. Exclusion of data points for estimation of λz seems warranted only if a problem with the respective sample has been documented, for example, by the clinical site or by the analytical laboratory. Probably no set of guidelines is applicable to all datasets. This set of rules may need to be revised, if the precision of the bioanalytical assay is low at low concentrations. Terminal half-life sometimes cannot be well determined by NCA for extended-release formulations or for data after multiple dosing, especially if the dosing interval is shorter than about twice the terminal half-life. It may be very helpful to report the median instead of the arithmetic mean to describe the central tendency of terminal half-life, if NCA indicates extremely long half-lives for some subjects. Compartmental modeling may be more powerful in these situations (85). The concentrations during the terminal phase used in linear regression on semilogarithmic scale often span a wide range (sometimes more than a factor of 100 between the highest and lowest concentration). This consideration is important for choosing an adequate weighting scheme. Importantly, unweighted linear regression (uniform weighting) on semilogarithmic scale is approximately equal to assuming a constant coefficient of variation error model. Therefore, unweighted linear regression on semilogarithmic scale is usually an adequate weighting scheme for NCA.

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

6.2 How to Handle Samples Below the Quantification Limit The best way to handle observations reported as being below the limit of quantitation is to ask the chemical analyst to supply the measured values. It may be necessary to tell and convince the analytical team that reporting the measured concentration value for samples ‘‘below the quantification limit’’ (BQL) contributes valuable information. These low concentrations can be adequately weighted when compartmental modeling techniques are applied. No good PK or regulatory reason exists not to use these observations. However, very good statistical theory supports the idea that discarding these values will bias the results of subsequent PK analysis (86–94). Concentrations at the first time points for extravascular administration and during the terminal phase are often reported to be BQL by the analytical laboratory. A thorough discussion of handling BQL samples in a PK data analysis is beyond the scope of this article. Population PK modeling offers more powerful methods for dealing with BQL samples. As a practical guide for NCA, the following procedure can be applied, if the only available information is that the respective samples were reported to be BQL: 1. All BQL samples before the first quantifiable concentration (including the predose sample) are set to zero (see predose, 0.5 and 1 h sample in Fig. 6). This setting will underestimate the AUC of the trapezoids before the first quantifiable concentration slightly, but the bias is usually small. An unreasonable option is to ignore the BQL samples before the first quantifiable concentration and to calculate the first trapezoid from time zero to the first quantifiable concentration because this calculation would yield a much larger overestimation of the AUC compared with the small underestimation described above. 2. BQL samples that were drawn after the first quantifiable concentration and before the last quantifiable concentration are usually ignored. This approach assumes that these samples were in

15

fact lost or never drawn (see 3.5 h sample in Fig. 6). 3. BQL samples after the last quantifiable concentrations are usually all ignored. A less common and potentially suboptimal method for NCA is to set the first BQL sample after the last quantifiable concentration (see 6 h sample in Fig. 6) to a fixed value. Typically, half of the reported quantification limit is used. All subsequent BQL samples are ignored for NCA (see 7 h and 8 h sample in Fig. 6). If the terminal phase is adequately descri bed by the last quantifiable concentrations and if the residual area is below 5%, then ignoring all BQL samples after the last quantifiable concentration seems to be most reasonable. Imputing the first BQL sample at 6 h (see Fig. 6) to be half of the quantification limit (or any other prespecified value) is likely to yield (slightly) biased estimates for terminal half-life in an NCA. Importantly, such an imputed sample should not be used for calculation of residual area. 6.3 NCA for Sparse Concentration Time Data NCA methods for sparse concentration time data have been developed (36–43). These methods are often applied for preclinical PK, toxicokinetic, and animal studies for which usually only sparse data are available. Bailer (36) originally proposed a method that uses concentration time data from destructive sampling (one observation per animal) to estimate the average AUC up to the

Figure 6. Example of a plasma concentration time profile with various samples reported to be below the quantification limit (BQL) that is reported to be 1 mg/L.

16

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

last observation time and to derive the standard error of the average AUC. Assume, for example, that 20 animals are sacrificed at 5 different time points (4 animals at each time point). The average and variance of the concentration at each time point are calculated, and the linear trapezoidal rule is used to derive the average AUC. Bailer provided a formula for the standard error of the average AUC. If various treatments are compared, then the average AUC can be compared statistically between different treatments by use of this standard error. Extensions to the Bailer method were developed (41–43) and were recently implemented into version 5 of WinNonlinTM Pro. Sparse sampling methods for NCA will also become available in the next version of Thermo Kinetica . Subsequently, bootstrap resampling techniques for sparse data (one observation per animal) were developed and evaluated (37–39). The so-called pseudoprofile-based bootstrap (38) creates pseudoprofiles by randomly sampling one concentration from, for example, the four animals at each time point. One concentration is drawn at each of the five time points. These five concentrations represent one pseudoprofile. Many pseudoprofiles are randomly generated, and an NCA is performed for each pseudoprofile. Summary statistics are computed to calculate the between subject variability and standard error of the NCA statistics of interest [see Mager and Goller (38) for details]. 6.4 Reporting the Results of an NCA The PK parameters and statistics to be reported for an NCA depend on the objectives of the analysis and on the types of data. Regulatory guidelines list several PK parameters and statistics to be reported from an NCA (53,54). A detailed report on the design, analysis, and reporting of clinical trials by NCA has been published by the Association for Applied Human Pharmacology (AGAH) (54). Valuable guidelines can also be found on pages 21 to 23 of the FDA guidelines for bioavailability and bioequivalence studies (53). It is often helpful for subsequent analyses to report the average ± SD (or geometric mean and %CV) and median (percentiles) of

the following five NCA statistics: AUC, Cmax, Tmax, t1/2 , and AUMC. Reporting the results on CL and Vss provides insight into the drug disposition. If the average and SD of those five NCA statistics are reported, then the so-called back analysis method (95) can be applied to convert NCA statistics into compartmental model parameters. Alternatively, the individual NCA statistics in each subject can be used instead of average and SD data. One has to decide based on literature data, for example, before applying the back analysis method, if a one- or two-compartment model is likely to be more appropriate for the drug of interest. The back analysis method provides estimates for the mean and variance of the model parameters for a one- or two-compartment model. The resulting PK parameter estimates can be used, for example, to build a population PK model and to run clinical trial simulations. The back analysis method allows one to run such simulations based on NCA results without the individual concentration time data. However, if individual concentration time data are available, population PK analysis is the preferred method of building a population PK model. 6.5 How to Design a Clinical Trial that is to be Analyzed by NCA It is possible to optimize the design of clinical trials (e.g., of bioequivalence studies) that are to be analyzed by NCA. However, those methods rely on compartment models in a population PK analysis and clinical trial simulation and are not covered in this article. Some practical guidelines for selection of appropriate sampling time points are provided below. This article focuses on NCA analysis by numerical integration for the most common case of extravascular dosing. The methods presented here are not as powerful as optimization of the study design by clinical trial simulation but may provide a basic guide. Before planning the sampling time schedule, one needs at least some information on the blood volume required for drug analysis and on the sensitivity of the bioanalytical assay. Some knowledge on the expected average concentration time profile of the drug

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

available is also assumed. The FDA (53) recommends 12 to 18 blood samples (including the predose sample) to be taken per subject and per dose. Cawello (96) and the AGAH working group recommend in total 15 blood samples to be drawn. Five samples (including the predose sample) should be drawn before Tmax. Another five samples are to be drawn between Tmax and one half-life after Tmax, and another five samples should be drawn up to five half-lives after Tmax. If one assumes that the median Tmax is at 1.5 h and that the average terminal halflife is 5 h, then this recommendation would yield, for example, the following sampling times: 0 h (predose), 0.33, 0.67, 1.00, 1.50, 2.00, 2.50, 3.00, 4.00, 6.00, 8.00, 12.0, 16.0, 24.0, and 32.0 h post dose. Cawello (96) recommends that each of the three intervals described above should contain four samples, if only twelve samples can be drawn in total. This recommendation would yield, for example, the following sampling times: 0 h (predose), 0.50, 1.00, 1.50, 2.00, 3.00, 4.00, 6.00, 10.0, 16.0, 24.0, and 32.0 h post dose. Such a sampling schedule may need to be modified, for example, for drugs with a large variability in terminal half-life or in Tmax. If the variability in Tmax is large, then more frequent sampling may be required during the first hour post dose in this example. Overall, this sampling schedule should provide low residual areas and a robust estimate for the terminal half-life. 7

CONCLUSIONS AND PERSPECTIVES

NCA is an important component of the toolbox for PK analyses. It is most applicable for studies with frequent sampling. Numerical integration, for example, by the trapezoidal rule, is most commonly used to analyze data after extravascular dosing. Fitting of disposition curves by a sum of exponential functions is the non-compartmental method of choice for analysis of concentrations after iv bolus dosing. Non-compartmental methods for handling sparse data were developed and are available in standard software packages. Standard NCA is straightforward and can be conveniently applied also by nonprofessional users.

17

It is important to recognize the assumptions and limitations of standard NCA. Almost all applications of NCA require a series of assumptions that are similar to the assumptions required for compartmental modeling. Violation of the assumptions for NCA will result in biased parameter estimates for some or all PK parameters. From a regulatory perspective, NCA is appealing as it involves minimal decision making by the drug sponsor or drug regulator. NCA can provide robust estimates of many PK parameters for studies with frequent sampling. For these studies, NCA and compartmental modeling often complement each other, and NCA results may be very helpful for compartmental model building. Therefore, NCA may be very valuable for PK data analysis of studies with frequent sampling, irrespective of whether the study objectives require subsequent analysis by compartmental modeling. 8

ACKNOWLEDGMENT

We thank one of the reviewers and Dr. William J. Jusko for comments on this ¨ manuscript. Jurgen Bulitta was supported by a postdoctoral fellowship from Johnson & Johnson. REFERENCES 1. O. Caprani, E. Sveinsdottir, and N. Lassen, SHAM, a method for biexponential curve resolution using initial slope, height, area and moment of the experimental decay type curve. J. Theor. Biol. 1975; 52: 299–315. 2. N. Lassen and W. Perl, Tracer Kinetic Methods in Medical Physiology. New York: Raven Press, 1979. 3. M. Weiss, The relevance of residence time theory to pharmacokinetics. Eur. J. Clin. Pharmacol. 1992; 43: 571–579. 4. P. Veng-Pedersen, Noncompartmentallybased pharmacokinetic modeling. Adv. Drug Deliv. Rev. 2001; 48: 265–300. 5. P. Veng-Pedersen, H. Y. Cheng, and W. J. Jusko, Regarding dose-independent pharmacokinetic parameters in nonlinear pharmacokinetics. J. Pharm. Sci. 1991; 80: 608–612.

18

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

6. A. T. Chow and W. J. Jusko, Application of moment analysis to nonlinear drug disposition described by the Michaelis-Menten equation. Pharm. Res. 1987; 4: 59–61. 7. H. Y. Cheng and W. J. Jusko, Mean residence time of drugs in pharmacokinetic systems with linear distribution, linear or nonlinear elimination, and noninstantaneous input. J. Pharm. Sci. 1991; 80: 1005–1006. 8. H. Y. Cheng and W. J. Jusko, Mean residence time concepts for pharmacokinetic systems with nonlinear drug elimination described by the Michaelis-Menten equation. Pharm. Res. 1988; 5: 156–164. 9. H. Cheng, W. R. Gillespie, and W. J. Jusko, Mean residence time concepts for non-linear pharmacokinetic systems. Biopharm. Drug Dispos. 1994; 15: 627–641. 10. H. Cheng, Y. Gong, and W. J. Jusko, A computer program for calculating distribution parameters for drugs behaving nonlinearly that is based on disposition decomposition analysis. J. Pharm. Sci. 1994; 83: 110–112. 11. P. Veng-Pedersen, J. A. Widness, L. M. Pereira, C. Peters, R. L. Schmidt, and L. S. Lowe, Kinetic evaluation of nonlinear drug elimination by a disposition decomposition analysis. Application to the analysis of the nonlinear elimination kinetics of erythropoietin in adult humans. J. Pharm. Sci. 1995; 84: 760–767. 12. M. Weiss, Mean residence time in non-linear systems? Biopharm. Drug Dispos. 1988; 9: 411–412. 13. D. J. Cutler, A comment regarding mean residence times in non-linear systems. Biopharm. Drug Dispos. 1989; 10: 529–530. 14. H. Y. Cheng and W. J. Jusko, An area function method for calculating the apparent elimination rate constant of a metabolite. J. Pharmacokinet. Biopharm. 1989; 17: 125–130. 15. A. T. Chow and W. J Jusko, Michaelis-Menten metabolite formation kinetics: equations relating area under the curve and metabolite recovery to the administered dose. J. Pharm. Sci. 1990; 79: 902–906. 16. M. Weiss, Use of metabolite AUC data in bioavailability studies to discriminate between absorption and first-pass extraction. Clin. Pharmacokinet. 1990; 18: 419–422. 17. M. Weiss, A general model of metabolite kinetics following intravenous and oral administration of the parent drug. Biopharm. Drug Dispos. 1988; 9: 159–176. 18. J. B. Houston, Drug metabolite kinetics. Pharmacol. Ther. 1981; 15: 521–552.

19. W. J. Jusko, Guidelines for collection and analysis of pharmacokinetic data. In: M. E. Burton, L. M. Shaw, J. J. Schentag, and W. E. Evans (eds.), Applied Pharmacokinetics & Pharmacodynamics, 4th ed. Philadelphia, PA: Lippincott Williams and Wilkins, 2005. 20. H. Y. Cheng and W. J. Jusko, Mean residence times and distribution volumes for drugs undergoing linear reversible metabolism and tissue distribution and linear or nonlinear elimination from the central compartments. Pharm. Res. 1991; 8: 508–511. 21. H. Y. Cheng and W. J. Jusko, Mean residence time of drugs showing simultaneous first-order and Michaelis-Menten elimination kinetics. Pharm. Res. 1989; 6: 258–261. 22. H. Y. Cheng and W. J. Jusko, Mean interconversion times and distribution rate parameters for drugs undergoing reversible metabolism. Pharm. Res. 1990; 7: 1003–1010. 23. H. Y. Cheng and W. J. Jusko, Mean residence times of multicompartmental drugs undergoing reversible metabolism. Pharm. Res. 1990; 7: 103–107. 24. H. Y. Cheng and W. J. Jusko, Mean residence time of oral drugs undergoing first-pass and linear reversible metabolism. Pharm. Res. 1993; 10: 8–13. 25. H. Y. Cheng and W. J. Jusko, Pharmacokinetics of reversible metabolic systems. Biopharm. Drug Dispos. 1993; 14: 721–766. 26. S. Hwang, K. C. Kwan, and K. S. Albert, A liner mode of reversible metabolism and its application to bioavailability assessment. J. Pharmacokinet. Biopharm. 1981; 9: 693–709. 27. W. F. Ebling, S. J. Szefler, and W. J. Jusko, Methylprednisolone disposition in rabbits. Analysis, prodrug conversion, reversible metabolism, and comparison with man. Drug Metab. Dispos. 1985; 13: 296–304. 28. M. N. Samtani, M. Schwab, P. W. Nathanielsz, and W. J. Jusko, Area/moment and compartmental modeling of pharmacokinetics during pregnancy: applications to maternal/fetal exposures to corticosteroids in sheep and rats. Pharm. Res. 2004; 21: 2279–2292. 29. M. S. Roberts, B. M. Magnusson, F. J. Burczynski, and M. Weiss, Enterohepatic circulation: physiological, pharmacokinetic and clinical implications. Clin. Pharmacokinet. 2002; 41: 751–790. 30. H. Cheng and W. R. Gillespie, Volumes of distribution and mean residence time of drugs with linear tissue distribution and binding and nonlinear protein binding. J. Pharmacokinet. Biopharm. 1996; 24: 389–402.

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS 31. F. M. Gengo, J. J. Schentag, and W. J. Jusko, Pharmacokinetics of capacity-limited tissue distribution of methicillin in rabbits. J. Pharm. Sci. 1984; 73: 867–873. 32. R. Nagashima, G. Levy, and R. A. O’Reilly, Comparative pharmacokinetics of coumarin anticoagulants. IV. Application of a threecompartmental model to the analysis of the dose-dependent kinetics of bishydroxycoumarin elimination. J. Pharm. Sci. 1968; 57: 1888–1895. 33. D. E. Mager, Target-mediated drug disposition and dynamics. Biochem. Pharmacol. 2006; 72: 1–10. 34. D. E. Mager and W. J. Jusko, General pharmacokinetic model for drugs exhibiting targetmediated drug disposition. J. Pharmacokinet. Pharmacodyn. 2001; 28: 507–532. 35. H. Cheng and W. J. Jusko, Disposition decomposition analysis for pharmacodynamic modeling of the link compartment. Biopharm. Drug Dispos. 1996; 17: 117–124. 36. A. J. Bailer, Testing for the equality of area under the curves when using destructive measurement techniques. J. Pharmacokinet. Biopharm. 1988; 16: 303–309. 37. H. Mager and G. Goller, Analysis of pseudoprofiles in organ pharmacokinetics and toxicokinetics. Stat. Med. 1995; 14: 1009–1024. 38. H. Mager and G. Goller, Resampling methods in sparse sampling situations in preclinical pharmacokinetic studies. J. Pharm. Sci. 1998; 87: 372–378. 39. P. L. Bonate, Coverage and precision of confidence intervals for area under the curve using parametric and non-parametric methods in a toxicokinetic experimental design. Pharm. Res. 1998; 15: 405–410. 40. J. R. Nedelman and E. Gibiansky, The variance of a better AUC estimator for sparse, destructive sampling in toxicokinetics. J. Pharm. Sci. 1996; 85: 884–886. 41. J. R. Nedelman, E. Gibiansky, and D. T. Lau, Applying Bailer’s method for AUC confidence intervals to sparse sampling. Pharm. Res. 1995; 12: 124–128. 42. J. R. Nedelman and X. Jia, An extension of Satterthwaite’s approximation applied to pharmacokinetics. J. Biopharm. Stat. 1998; 8: 317–328. 43. D. J. Holder, Comments on Nedelman and Jia’s extension of Satterthwaite’s approximation applied to pharmacokinetics. J. Biopharm. Stat. 2001; 11: 75–79. 44. P. Veng-Pedersen, Stochastic interpretation of linear pharmacokinetics: a linear system

19

analysis approach. J. Pharm. Sci. 1991; 80: 621–631. 45. J. J. DiStefano, 3rd. Noncompartmental vs. compartmental analysis: some bases for choice. Am. J. Physiol. 1982; 243: R1–6. 46. J. J. DiStefano 3rd, E. M. Landaw, Multiexponential, multicompartmental, and noncompartmental modeling. I. Methodological limitations and physiological interpretations. Am. J. Physiol. 1984; 246: R651–664. 47. W. R. Gillespie, Noncompartmental versus compartmental modelling in clinical pharmacokinetics. Clin. Pharmacokinet. 1991; 20: 253–262. 48. D. J. Cutler, Linear systems analysis in pharmacokinetics. J. Pharmacokinet. Biopharm. 1978; 6: 265–282. 49. J. L. Stephenson, Theory of transport in linear biological systems: I. Fundamental integral equation. Bull. Mathemat. Biophys. 1960; 22: 1–7. 50. P. Veng-Pedersen, Linear and nonlinear system approaches in pharmacokinetics: how much do they have to offer? I. General considerations. J. Pharmacokinet. Biopharm. 1988; 16: 413–472. 51. E. Nakashima and L. Z. Benet, An integrated approach to pharmacokinetic analysis for linear mammillary systems in which input and exit may occur in/from any compartment. J. Pharmacokinet. Biopharm. 1989; 17: 673–686. 52. Weiss M. Generalizations in linear pharmacokinetics using properties of certain classes of residence time distributions. I. Log-convex drug disposition curves. J. Pharmacokinet. Biopharm. 1986; 14: 635–657. 53. Food and Drug Administration (CDER). Guidance for Industry: Bioavailability and Bioequivalence Studies for Orally Administered Drug Products - General Considerations, 2003. 54. Committee for Proprietary and Medicinal Products (CPMP). Note for Guidance on the Investigation of Bioavailability and Bioequivalence. CPMP/EWP/QWP/1401/98, 2001. 55. K. C. Yeh and K. C. Kwan, A comparison of numerical integrating algorithms by trapezoidal, Lagrange, and spline approximation. J. Pharmacokinet. Biopharm. 1978; 6: 79–98. 56. Z. Yu and F. L. Tse, An evaluation of numerical integration algorithms for the estimation of the area under the curve (AUC) in pharmacokinetic studies. Biopharm. Drug Dispos. 1995; 16: 37–58.

20

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

57. W. J. Jusko and M. Gibaldi, Effects of change in elimination on various parameters of the two-compartment open model. J. Pharm. Sci. 1972; 61: 1270–1273. 58. J. V. Gobburu and N. H. Holford, Vz, the terminal phase volume: time for its terminal phase? J. Biopharm. Stat. 2001; 11: 373–375. 59. K. Yamaoka, T. Nakagawa, and T. Uno, Statistical moments in pharmacokinetics. J. Pharmacokinet. Biopharm. 1978; 6: 547–558. 60. I. L. Smith and J. J. Schentag, Noncompartmental determination of the steady-state volume of distribution during multiple dosing. J. Pharm. Sci. 1984; 73: 281–282. 61. M. Weiss, Definition of pharmacokinetic parameters: influence of the sampling site. J. Pharmacokinet. Biopharm. 1984; 12: 167–175. 62. W. L. Chiou, The phenomenon and rationale of marked dependence of drug concentration on blood sampling site. Implications in pharmacokinetics, pharmacodynamics, toxicology and therapeutics (Part I). Clin. Pharmacokinet. 1989; 17: 175–199. 63. M. Weiss, Nonidentity of the steady-state volumes of distribution of the eliminating and noneliminating system. J. Pharm. Sci. 1991; 80: 908–910. 64. M. Weiss, Model-independent assessment of accumulation kinetics based on moments of drug disposition curves. Eur. J. Clin. Pharmacol. 1984; 27: 355–359. 65. M. Weiss, Washout time versus mean residence time. Pharmazie 1988; 43: 126–127. 66. D. Perrier and M. Mayersohn, Noncompartmental determination of the steady-state volume of distribution for any mode of administration. J. Pharm. Sci. 1982; 71: 372–373. 67. H. Cheng and W. J. Jusko, Noncompartmental determination of the mean residence time and steady-state volume of distribution during multiple dosing. J. Pharm. Sci. 1991; 80: 202–204. 68. D. Brockmeier, H, J. Dengler, and D. Voegele, In vitro-in vivo correlation of dissolution, a time scaling problem? Transformation of in vitro results to the in vivo situation, using theophylline as a practical example. Eur. J. Clin. Pharmacol. 1985; 28: 291–300. 69. D. Brockmeier, In vitro-in vivo correlation, a time scaling problem? Evaluation of mean times. Arzneimittelforschung 1984; 34: 1604–1607. 70. M. E. Wise, Negative power functions of time in pharmacokinetics and their implications. J. Pharmacokinet. Biopharm. 1985; 13:

309–346. 71. K. H. Norwich and S. Siu, Power functions in physiology and pharmacology. J. Theor. Biol. 1982; 95: 387–398. 72. G. T. Tucker, P. R. Jackson, G. C. Storey, and D. W. Holt, Amiodarone disposition: polyexponential, power and gamma functions. Eur. J. Clin. Pharmacol. 1984; 26: 655–656. 73. J. Gabrielsson and D. Weiner, Pharmacokinetic and Pharmacodynamic Data Analysis, Concepts and Applications. 4th ed. Stockholm, Sweden: Swedish Pharmaceutical Press, 2007. 74. M. Rowland and T. N. Tozer, Clinical Pharmacokinetics: Concepts and Applications. Philadelphia, PA: Lippincott Williams & Wilkins, 1995. 75. J. G. Wagner and E. Nelson, Per cent absorbed time plots derived from blood level and/or urinary excretion data. J. Pharm. Sci. 1963; 52: 610–611. 76. J. G. Wagner and E. Nelson, Kinetic analysis of blood levels and urinary excretion in the absorptive phase after single doses of drug. J. Pharm. Sci. 1964; 53: 1392–1403. 77. J. G. Wagner, Modified Wagner-Nelson absorption equations for multiple-dose regimens. J. Pharm. Sci. 1983; 72: 578–579. 78. J. G. Wagner, The Wagner-Nelson method applied to a multicompartment model with zero order input. Biopharm. Drug Dispos. 1983; 4: 359–373. 79. J. C. Loo and S. Riegelman, New method for calculating the intrinsic absorption rate of drugs. J. Pharm. Sci. 1968; 57: 918–928. 80. J. G. Wagner, Pharmacokinetic absorption plots from oral data alone or oral/intravenous data and an exact Loo-Riegelman equation. J. Pharm. Sci. 1983; 72: 838–842. 81. F. N. Madden, K. R. Godfrey, M. J. Chappell, R. Hovorka, and R. A. Bates, A comparison of six deconvolution techniques. J. Pharmacokinet. Biopharm. 1996; 24: 283–299. 82. D. Verotta, Comments on two recent deconvolution methods. J. Pharmacokinet. Biopharm. 1990; 18: 483–489; discussion 489–499. 83. D. P. Vaughan and M. Dennis, Mathematical basis and generalization of the LooRiegelman method for the determination of in vivo drug absorption. J. Pharmacokinet. Biopharm. 1980; 8: 83–98. 84. J. H. Proost, Calculation of half-life - PharmPK Discussion, 2005. Available: http://www.boomer.org/pkin/PK05/ PK2005095.html.

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS 85. A. Sharma, P. H. Slugg, J. L. Hammett, and W. J. Jusko, Estimation of oral bioavailability of a long half-life drug in healthy subjects. Pharm. Res. 1998; 15: 1782–1786. 86. Beal SL. Ways to fit a PK model with some data below the quantification limit. J Pharmacokinet Pharmacodyn 2001; 28: 481–504. 87. V. Duval and M. O. Karlsson, Impact of omission or replacement of data below the limit of quantification on parameter estimates in a two-compartment model. Pharm. Res. 2002; 19: 1835–1840. 88. H. Jacqmin-Gadda, R. Thiebaut, G. Chene, and D. Commenges, Analysis of left-censored longitudinal data with application to viral load in HIV infection. Biostatistics 2000; 1: 355–368. 89. H. S. Lynn, Maximum likelihood inference for left-censored HIV RNA data. Stat. Med. 2001; 20: 33–45. 90. J. P. Hing, S. G. Woolfrey, D. Greenslade, and P. M. Wright, Analysis of toxicokinetic data using NONMEM: impact of quantification limit and replacement strategies for censored data. J. Pharmacokinet. Pharmacodyn. 2001; 28: 465–479. 91. A. Samson, M. Lavielle, and F. Mentre, Extension of the SAEM algorithm to left-censored data in nonlinear mixed-effects model: application to HIV dynamics model. Computat. Stat. Data Anal. 2006; 51: 1562–1574. 92. R. Thiebaut, J. Guedj, H. Jacqmin-Gadda, et al., Estimation of dynamical model parameters taking into account undetectable marker values. BMC Med. Res. Methodol. 2006; 6: 38. 93. J. Asselineau, R. Thiebaut, P. Perez, G. Pinganaud, and G. Chene, Analysis of leftcensored quantitative outcome: example of procalcitonin level. Rev. Epidemiol. Sante Publique 2007; 55: 213–220. 94. S. Hennig, T. H. Waterhouse, S. C. Bell, et al., A d-optimal designed population pharmacokinetic study of oral itraconazole in adult cystic fibrosis patients. Br. J. Clin. Pharmacol. 2007; 63: 438–450. 95. C. Dansirikul, M. Choi, and S. B. Duffull, Estimation of pharmacokinetic parameters from non-compartmental variables using Microsoft Excel. Comput. Biol. Med. 2005; 35: 389–403. 96. W. Cawello, Parameters for Compartment-free Pharmacokinetics - Standardisation of Study Design, Data Analysis and Reporting. Aachen, Germany: Shaker Verlag, 1999.

21

FURTHER READING D. Z. D’Argenio, Advanced Methods of Pharmacokinetic and Pharmacodynamic Systems Analysis. New York: Plenum Press, 1991. D. Foster, NIH course ‘‘Principles of Clinical Pharmacology’’ lecture (including video) on compartmental versus noncompartmental analysis. Available: http://www.cc.nih.gov/training/training. html. M. Weiss, The relevance of residence time theory to pharmacokinetics. Eur. J. Clin. Pharmacol. 1992; 43: 571–579. W. J. Jusko, Guidelines for collection and analysis of pharmacokinetic data. In: M. E. Burton, L. M. Shaw, J. J. Schentag, and W. E. Evans (eds.), Applied Pharmacokinetics & Pharmacodynamics. 4th ed. Philadelphia, PA: Lippincott Williams and Wilkins, 2005.

CROSS-REFERENCES Pharmacokinetic Study Bioavailability Bioequivalence Population Pharmacokinetic Methods Analysis of Variance (ANOVA)

NONCOMPLIANCE Noncompliance with the protocol, Standards of Procedure (SOP), Good Clinical Practice (GCP), and/or applicable regulatory requirement(s) by an investigator/institution, or by member(s) of the sponsor’s staff should lead to prompt action by the sponsor to secure compliance. If the monitoring and/or auditing personnel identify(ies) serious and/or persistent noncompliance on the part of an investigator/institution, the sponsor should terminate the investigator’s/institution’s participation in the trial. When an investigator’s/institution’s participation is terminated because of noncompliance, the sponsor should notify promptly the regulatory authority(ies).

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NONINFERIORITY TRIAL

loss in effectiveness from the control may still lead one to conclude that the experimental is effective (i.e., relative to no use of the experimental treatment). However, the term ‘‘noninferior’’ is in some sense misleading. It is, in fact, ‘‘inferior,’’ and what it exactly means is ‘‘not unacceptably inferior.’’ The unacceptable extent of inferiority (e.g., loss of the effect of the active control) would need to be defined. This extent is the so-called noninferiority margin.

H. M. JAMES HUNG Division of Biometrics I, Office of Biostatistics, Office of Translational Sciences, Center for Drug Evaluation and Research U.S. Food and Drug Administration Silver Spring, Maryland

SUE-JANE WANG ROBERT O’NEILL Office of Translational Sciences, Center for Drug Evaluation and Research U.S. Food and Drug Administration Silver Spring, Maryland

1 ESSENTIAL ELEMENTS OF NONINFERIORITY TRIAL DESIGN Literature on noninferiority trials or activecontrolled trials in general is abundant (2–16). Basically, there are two types of noninferiority trial design: with or without a placebo. When the placebo arm is present in the noninferiority trial, the efficacy of the experimental treatment can be evaluated via a direct comparison with the placebo. The comparison of the experimental treatment with the active control elucidates the extent of effectiveness in a relative sense. However, when the placebo is absent, the direct comparison between the experimental treatment and the active control is the only comparison available. It not only elucidates the extent of inferiority or superiority of the experimental treatment over the active control, but it also is expected to serve as a bridge for an indirect assessment of the experimental treatment’s efficacy (relative to a placebo). That is, the indirect inference pertains to the important question of how the experimental treatment would have fared against a placebo had the placebo been in the trial. In the absence of a concurrent placebo, the indirect inference for the experimental treatment’s efficacy entails use of the active control’s effect (relative to a placebo) from the historical placebo-controlled trials. For a simple reason, if the true effect of the experimental relative to the active control and the true effect of the active control versus the placebo are known, one can obtain the effect of the experimental treatment versus the placebo by transitivity.

Traditionally, the effect of an experimental treatment in treating a disease is mostly evaluated in a placebo-controlled clinical trial with the objective of demonstrating that the experimental treatment is more effective than the placebo with or without the standard-of-care medical treatment on the background. This clinical objective drives the formulation of a statistical hypothesis for testing. The trial is designed to provide statistical evidence to reject the null hypothesis that there is no treatment difference in the therapeutic effect and consequently to accept the intended alternative hypothesis that the experimental treatment is more effective than the placebo. This is often referred to as superiority testing. When a placebo can no longer be used ethically in the trial (e.g., with a life-saving drug on the market), a treatment regimen that has been proven to be effective and safe can be selected as a control, often called active or positive control, for an experimental treatment to be compared against (1). In an active-controlled trial, demonstration of superiority over the active control certainly establishes the effectiveness of the experimental treatment. Alternatively, proving the efficacy of the experimental treatment can be based on the so-called noninferiority testing, by which the experimental treatment will be shown to be not much inferior to the active control. Because the effectiveness of the active control has been established, a small amount of

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

NONINFERIORITY TRIAL

Usage of the historical data to obtain the effect of the active control presents several critical issues that directly influence the interpretability of a noninferiority trial. First, the effect of the active control versus a placebo in the historical trials is, at best, only an estimate. Derivation of the historical estimate requires meta-analysis, which is in itself controversial in practice. For instance, can the existing historical trials produce an approximately unbiased estimate of the active control’s effect, given that the negative trials are often not reported (this is the problem of publication bias)? And how does one handle large intertrial variability? Thus, the meta-analysis is often suspected of having a tendency of overestimating a treatment effect. The historical estimates in the individual historical trials are often unstable. Selection of the historical trials relevant to the noninferiority trial setting is a subjective process. Statistical methods for handling between-trial heterogeneity may depend on strong untestable assumptions. For instance, the validity of the resulting interval estimator by some meta-analysis method may be questionable (16–18). Secondly, there is a question of whether the historical estimate deemed valid in the historical trial patient population is applicable to the noninferiority trial setting; this is the question about the so-called ‘‘constancy’’ assumption (19–26). In practice, it is always doubtful that the constancy assumption is satisfied because often there are significant differences between the noninferiority trial and the historical trial settings, such as the differences in patient population, study endpoints, background or concomitant medications, or disease status. Violation of the constancy assumption will induce a bias that is the difference in the effect of the selected active control between the historical trial setting and the noninferiority setting. When the active control is much less effective in the noninferiority trial setting than in the historical trial setting, the bias may substantially increase the risk of falsely concluding noninferiority and the risk of falsely asserting that the experimental treatment is effective. Third, the quality of the noninferiority trial may be problematic because of inevitable

issues of medication nonadherence and noncompliance. The quality affects the ability of the noninferiority trial to distinguish an effective treatment from a less effective or ineffective treatment; this ability is referred as assay sensitivity in the International Conference on Harmonization (ICH) E10 guidelines (19, 20). It is not difficult to understand that if most of the patients in both treatment groups do not take the study drugs they are assigned, then the two treatment groups will be more alike than they should. That is, a noninferiority trial that lacks assay sensitivity may find an inferior treatment to be noninferior to the active control, which will lead to erroneous conclusions about noninferiority and the experimental treatment’s efficacy. The two important determinations as laid out in the ICH E-10 document that deduce the presence of assay sensitivity are (1) historical evidence of sensitivity to drug effects (i.e., that similarly designed trials in the past regularly distinguished effective treatments from less effective or ineffective treatments) and (2) appropriate trial conduct (i.e., that the conduct of the trial did not undermine its ability to distinguish effective treatments from less effective or ineffective treatments). The absence of a placebo arm from the noninferiority trial makes it impossible to verify or even check the assay sensitivity and the constancy assumptions with the trial data. Hence, the extent of the potential bias attributed to violation of these assumptions cannot be quantified in practice. 2 OBJECTIVES OF NONINFERIORITY TRIALS In the context of therapeutic effectiveness, three possible objectives have often been entertained in noninferiority studies. First, noninferiority in effectiveness traditionally means ‘‘not unacceptably inferior’’ in the sense that the effectiveness of the experimental treatment is ‘‘clinically indifferent’’ from that of the active control. Second, as a minimum requirement, the noninferiority trial must be able to assert that the experimental treatment is efficacious; for instance, the experimental treatment would have been

NONINFERIORITY TRIAL

more effective than the placebo had a placebo been in the noninferiority trial. Third, in many cases, it is often stipulated to establish that the experimental treatment preserves at least a certain portion of the effect of the selected active control in the noninferiority trial. On one hand, consideration of effect retention could arise from a part of value assessment of the relative efficacy, such as how much of loss in clinical benefits is acceptable with use of the experimental treatment in lieu of the active control to exchange for a better safety profile. On the other hand, retention of the active control’s effect may be necessary to create a cushion for the noninferiority inference to be valid beyond the many sources of unquantifiable bias. The biases can occur due to the uncertainty in the estimates of the active control’s effect in historical trials, application of such estimates to the noninferiority trial, or a lack of assay sensitivity. The three possible objectives are interrelated, though the term ‘‘noninferiority’’ may arguably be inappropriate when the primary objective is to demonstrate superiority over a putative placebo or to demonstrate the retention of a certain portion of the activecontrol effect. From the design perspective, they have tremendously different influences on the design specifications. For one, the clinical hypothesis and the corresponding statistical hypothesis for these objectives may be quite different. Consequently, statistical analysis, the decision tree, and inferential interpretation can be quite different. In practice, it is a mandatory first step to clearly define the primary objective of the noninferiority trial. The study objective is also a key factor determining the choice of the noninferiority margin. 3

MEASURE OF TREATMENT EFFECT

In addition to the study objective, selection of a measure for quantifying treatment effect is another key determination for defining the noninferiority margin. Let T, C, and P label, respectively, the experimental treatment, the selected active control, and the placebo that are studied or not studied in the noninferiority trial. To simplify the notation, let T, C,

3

and P also denote the targeted parameters of the experimental treatment, active control, and placebo, respectively. For instance, for the continuous response variable, if the effect of interest is expressed in terms of a mean change of a response variable from baseline, then T, C, and P are the mean parameters associated with the experimental treatment, active control, and placebo, respectively. For a binary outcome variable of event or not, T, C, and P may be the probabilities of an event associated with the three respective treatments. For a time-to-event variable, T, C, and P may be the respective hazard functions. The effect of a treatment can be measured on a variety of scales. Difference and ratio are two widely used scales. For a continuous response variable, the effect of the experimental treatment relative to the placebo is often defined on the difference scale, T − P. As an example, if the test drug decreases a mean of 6 mm Hg in sitting diastolic blood pressure after some point of treatment administration and the placebo decreases a mean of 2 mm Hg, then the effect of the experimental treatment is often measured as T − P = 4 mm Hg. On the ratio scale, the effect of the experimental treatment relative to the placebo is often defined as (1 − T/P). For instance, if the probability of death at some point during the study in the patients who receive the experimental treatment is 0.30 (or 30%) and the probability of death associated with the placebo is 0.40 (40%), then the effect of the experimental treatment on the risk scale is often measured by 1 − T/P = 1 − 0.30/0.40 = 25%; that is, the experimental treatment yields a 25% reduction in the mortality risk. On the difference scale, the experimental treatment reduces the probability of death by 10% from 40% to 30%. For a time-to-event parameter, the treatment effect is often quantified using a hazard ratio, if this ratio is approximately constant over time. For notational convenience, let T/P denote a risk ratio that can be a relative risk or an odds ratio for a binary outcome variable and a hazard ratio for a time-to-event variable. For a binary outcome variable or time-toevent variable, statistical inference is often based on the natural logarithm of a risk ratio statistic, rather than the risk ratio itself,

4

NONINFERIORITY TRIAL

because the former is better approximated by a Gaussian (or normal) distribution. On the log risk ratio scale, the treatment effect is defined as a difference of the log parameter, such as log probability of death or log hazard. 4

NONINFERIORITY MARGIN

Determination of a noninferiority margin is probably the most difficult task in designing a noninferiority trial (26–32). For demonstration that the experimental treatment is superior to the selected control, the objective of the trial is to rule out a zero difference between the two treatments and thus reject the null hypothesis of no treatment difference. A superiority margin is selected mostly to size the trial, not for being ruled out; therefore, the superiority margin does not need to be prespecified to define the hypothesis. In contrast, for demonstration of noninferiority, a noninferiority margin usually needs to be specified to define the statistical hypothesis before commencement of the noninferiority trial. The statistical null hypothesis to reject is that the treatment difference is at least as large as the specified margin against the experimental treatment. The noninferiority margin is an acceptable threshold in the value of the targeted parameter of the response variable between the experimental treatment and the selected positive control under comparison. At the Drug Information Association Annual Meeting in 2000, Dr. Robert Temple introduced the idea that this margin should be a statistical margin or a clinical margin, whichever is smaller. The clinical margin is determined on the basis of clinical judgments that are mostly subjective. Selection of the statistical margin depends upon the objective of the noninferiority trial. Suppose that the true probability of death is 30% associated with the selected positive control and 40% with the placebo. Thus, the control reduces 10% in the probability of death. If the noninferiority inference is set out to demonstrate that the experimental treatment is efficacious or effective (i.e., superior to a placebo), then the probability of death with the experimental treatment must be shown to be smaller than 40%; that is, the noninferiority margin for the experimental treatment versus the control can be set

to −10%. If the objective is to show that the experimental treatment preserves at least a certain portion, 50%—say, of the active control’s effect—then the probability of death with the experimental treatment must be shown to be smaller than 35%. Thus, the noninferiority margin for achieving a 50% retention on the difference scale is −5%. For a 75% retention, the noninferiority margin on the difference scale is 2.5% [= (1 − 0.75) × 10%]. These calculations can easily be articulated as follows. To obtain a noninferiority margin for the difference T − C, all that is needed is to examine the P-C axis between the control effect of 10% and the zero control effect. The margin needed for showing a 50% retention of (P − C) is the midpoint 5% between the 10% effect and zero effect. The margin defining a 75% retention of (P − C) is the one-quarter point above zero. Similar arguments can be made for risk ratio. To generate a noninferiority margin for the risk ratio T/C where the control C is the denominator, work on P/C, the risk ratio of placebo versus the control C, so that the two ratios to be compared have the same denominator. In the above example, P/C = 4/3 and the null effect of P versus C is P/C = 1. The noninferiority margin for the risk ratio T/C can then be generated by working on P/C between 4/3 and 1. For 50% retention of the control’s effect on the relative risk, the noninferiority margin is 1.17, which is the midpoint between 4/3 and 1. For 75% retention, the noninferiority margin for T/C is 1.08, which is the one-quarter point above one. In general, mathematical arguments can be made as follows. To retain X% of the control’s effect, we would need the effect of the experimental treatment relative to the placebo, had it been in the trial, to be greater than X times the control’s effect. That is, an X% retention of the control’s effect on the risk ratio amounts to showing (1 − T/P) > X(1 − C/P), equivalently, T/C < X + (1 − X)(P/C),

(1)

which indeed is the weighted average of one and the ratio P/C.

NONINFERIORITY TRIAL

For retention on the log risk ratio scale, the same arguments as those for the difference (P − C) can be made to construct the noninferiority margin for log(T/C) = log(T) − log(C). Thus, for 50% retention on the log risk ratio, the noninferiority margin for log(T/C) is the midpoint between log(4/3) and zero. By inverting log risk ratio to risk ratio, we can obtain that the noninferiority margin for T/C is the geometric mean between P/C = 4/3 and 1 to retain 50% of the control’s effect on the log risk ratio scale. For 75% retention on the log risk ratio scale, the noninferiority margin for T/C is (4/3)1/4 , raising (4/3) to the one-quarter power. For rendering a noninferiority margin on the risk ratio scale, the percent retention on the log risk ratio is convertible to that on the risk ratio and vice versa, given that the effect of active control can be estimated (16). For instance, if the relative risk C/P of the control to the placebo is 0.75, then preservation of 50% of the control effect on the risk ratio scale is equivalent to preservation of 46% of the control effect on the logarithm risk ratio. It is worth mentioning that at any level of retention on the log risk ratio the resulting noninferiority margin is always smaller than the margin for the same level of retention on the risk ratio scale. Thus, preservation of 50% of the control’s effect on the log risk ratio results in a smaller margin than preservation of 50% on the risk ratio. The statistical margin derived using the concept of retention of the control effect cannot always properly characterize clinical indifference. For example, if the selected positive control is highly effective with a relative risk C/P = 0.12, say, for a clinical adverse event (i.e., the control yields a 88% reduction of the risk of having the clinical event), then the statistical margin can be as large as 4.67 for a 50% preservation of the control’s effect and 2.47 for a 80% retention, by setting X to 50% and 80%, respectively, on the right-hand side of inequality (1). With these margins, an experimental treatment that yields at least a one-and-a-half-fold increase in risk relative to the control can still be concluded to be not inferior to the control. Such a large statistical margin cannot possibly be adequate for asserting the clinical indifference that the experimental treatment is as effective as or

5

not much worse than the positive control. For showing clinical indifference, a margin is required to define the clinical indifference. The margin determination discussed above is predicated on the knowledge of the true value of the effect parameter at stake. In practice, the true value is rarely known, and thus the effect of the active control must be estimated from trial data to determine the noninferiority margin. If a placebo is present in a noninferiority trial, the effect of the active control may be better estimated from this trial. In the absence of a concurrent placebo, the estimate will have to come from the external historical trials. In either case, the bias and the variance of the estimate must be properly incorporated in the margin determination. 5 STATISTICAL TESTING FOR NONINFERIORITY As mentioned earlier, the noninferiority margin must be selected and fixed in advance so that the noninferiority hypothesis to test is well defined before designing the trial. For example, if noninferiority is defined as retaining a 50% of the active control’s effect, one conventional approach employs the worst limit of a 95% confidence interval of the historical estimate of the control’s effect (i.e., relative to placebo) as a conservative estimate of the control’s effect in the noninferiority trial and then generates the statistical margin as previously described in section 4. Taking the smaller of the statistical margin and the clinical margin will then determine the noninferiority margin. In most cases, use of a conservative estimate of the active-control effect derived from some kind of meta-analyses of historical trials to define the noninferiority margin is necessary because of statistical uncertainty around the estimate and the unverifiable and yet often doubtful assumptions that must be made in making inferences from the noninferiority study. Once the noninferiority margin is determined, the widely used statistical method for noninferiority testing employs a 95% or higher confidence interval for the experimental treatment versus the selected active control from the noninferiority trial. If this

6

NONINFERIORITY TRIAL

interval rules out the predefined margin, then noninferiority defined by the margin can be concluded. This is in contrast with the superiority testing that depends on the same confidence interval to rule out the null value of no treatment difference. The probability of type I error of falsely concluding noninferiority associated with this confidence interval method is no more than 2.5%, conditional on the estimated noninferiority margin. When a placebo is present in the noninferiority trial, some type of the noninferiority hypothesis, such as percent retention hypothesis, can be tested directly with a more statistically efficient test method (33). The test can be constructed by dividing a relevant sum of the estimate of relative effect of the experimental treatment to the control and the estimate of the control’s effect by the standard error of the sum. Both estimates are derived from the noninferiority trial. For example, the 25% retention test on the log risk ratio scale is constructed by dividing the sum of the estimate of log (T/C) of the noninferiority trial and 75% times the estimate of log(C/P) by the standard error of this sum. A P-value can then be generated from the test. A sufficiently small P-value can indicate that the experimental treatment retains more than 25% of the control’s effect. If the placebo is absent from the noninferiority trial, by the same kind of combination, this test arguably can still be constructed in the same way but the estimate of the active control’s effect has to come from the historical trials. However, this test method is controversial (20–24, 34–39) because it assumes no bias with the historical estimate of the control’s effect when applied to the noninferiority trial—that is, when the constancy assumption holds, which is almost always very much doubtful. There are no data that can verify this assumption. Thus, this test method constructed by incorporating the historical estimate of the active control’s effect is rarely useful in practice. 6 MEDICATION NONADHERENCE AND MISCLASSIFICATION/MEASUREMENT ERROR Interpretation of the results of randomized clinical trials is primarily according to the

intent-to-treat principle, based on which all randomized patients are analyzed as per the treatment to which they are randomized. This analysis is intended to avoid selection bias, which may confound with a valid clinical interpretation. In superiority trials, medication nonadherence generally biases the intent-to-treat analysis toward the null hypothesis of no treatment difference; thus, statistical tests for superiority in intent-totreat analyses tend to be conservative. In noninferiority trials, nonadherence may bias intent-to-treat analyses in either a conservative or nonconservative direction (40–42), and thus it may undermine the clinical interpretability. Misclassification or measurement error also may generate bias. On-treatment or per-protocol analyses include only patients who are adherent to the assigned study treatment and protocol. This analysis is intended to address the question of what the true causal effect of the experimental treatment would have been had all patients adhered to the assigned treatment. In some cases, the on-treatment analyses may apparently be able to account for nonadherence when it is correctly measured, but these analyses require the unverifiable assumption that there is no unmeasured confounding caused by the factors such as selection of the patients for analyses. Hence, nonadherence when related to study outcome can also bias on-treatment analyses in either a conservative or nonconservative direction. Medication nonadherence and misclassification or measurement error may generate bias in both intent-to-treat analyses and on-treatment analyses conservatively or nonconservatively in noninferiority trials. The amount of bias is generally not estimable. Therefore, with serious nonadherence or such errors, most often no valid clinical interpretation of noninferiority trials can be made. Ontreatment or per-protocol analysis is unlikely to be able to rescue the study. 7 TESTING SUPERIORITY AND NONINFERIORITY As already discussed, in an active-control trial, the study goal can be superiority or noninferiority. In order for noninferiority to be

NONINFERIORITY TRIAL

entertained, the noninferiority margin must be fixed and prespecified in advance. The same 95% or higher confidence interval can be used to test both superiority and noninferiority simultaneously or in any sequential order (24–26), with the overall type I error rate associated with testing both objectives of no larger than a two-sided 5% level. The type I error rate for superiority and for noninferiority are each no larger than a two-sided 5% level. However, if the noninferiority margin is not prespecified, this confidence interval approach may still be problematic, particularly when the margin is influenced by the internal noninferiority trial data (43–48). Furthermore, medication noncompliance and misclassification or measurement error may still make the type I error rate for the prespecified noninferiority invalid. Therefore, to achieve noninferiority, the trial design requires the highest quality. To entertain testing for superiority and noninferiority, it is imperative to plan the study for noninferiority testing and the sample size to ensure sufficient power for both superiority and noninferiority, so defined (48). 8

CONCLUSION

It is quite clear that the effect of an experimental treatment should be evaluated, if all possible, by conducting a ‘‘showing superiority’’ trial. Showing noninferiority over a selected active control can be too difficult to provide statistical evidence for assessing the effect of the experimental treatment, particularly when the placebo cannot be used in the trial. Many factors determine the interpretability of a noninferiority trial that does not have a placebo arm. First, a noninferiority margin must be selected and fixed in advance when designing a noninferiority trial. The margin determination depends on the trial objective. Second, the critical assumptions of assay sensitivity and constancy and the judgment of ‘‘clinical indifference’’ also play key roles in the margin determination. Third, statistical uncertainty in the historical estimate of the active control’s effect also needs to be properly incorporated in the margin determination. The historical trials must have assay sensitivity. In contrast with showing superiority, the noninferiority trial must have very

7<