Experimental Design for Formulation
ASA-SIAM Series on Statistics and Applied Probability The ASA-SIAM Series on Statistics and Applied Probability is published jointly by the American Statistical Association and the Society for Industrial and Applied Mathematics. The series consists of a broad spectrum of books on topics in statistics and applied probability. The purpose of the series is to provide inexpensive, quality publications of interest to the intersecting membership of the two societies.
Editorial Board Robert N. Rodriguez SAS Institute Inc., Editor-in-Chief
Douglas M. Hawkins University of Minnesota
David Banks Duke University
Susan Holmes Stanford University
H. T. Banks North Carolina State University
Lisa LaVange Inspire Pharmaceuticals, Inc.
Richard K. Burdick Arizona State University Joseph Gardiner
Gary C. McDonald Oakland University and National Institute of Statistical Sciences
Michigan State University
Francoise Seillier-Moiseiwitsch University of Maryland—Baltimore County
Smith, W. F, Experimental Design for Formulation Baglivo, J. A., Mathematica Laboratories for Mathematical Statistics: Emphasizing Simulation and Computer Intensive Methods Lee, H. K. H., Bayesian Nonparametrics via Neural Networks O'Gorman, T. W., Applied Adaptive Statistical Methods: Tests of Significance and Confidence Intervals Ross, T. ]., Booker, J. M., and Parkinson, W. J., eds., Fuzzy Logic and Probability Applications: Bridging the Cap Nelson, W. B., Recurrent Events Data Analysis for Product Repairs, Disease Recurrences, and Other Applications Mason, R. L and Young, J. C., Multivariate Statistical Process Control with Industrial Applications Smith, P. L., A Primer for Sampling Solids, Liquids, and Cases: Based on the Seven Sampling Errors of Pierre Cy Meyer, M. A. and Booker, j. M., Eliciting and Analyzing Expert judgment: A Practical Guide Latouche, G. and Ramaswami, V., Introduction to Matrix Analytic Methods in Stochastic Modeling Peck, R., Haugh, L, and Goodman, A., Statistical Case Studies: A Collaboration Between Academe and Industry, Student Edition Peck, R., Haugh, L., and Goodman, A., Statistical Case Studies: A Collaboration Between Academe and Industry Barlow, R., Engineering Reliability Czitrom, V. and Spagon, P. D., Statistical Case Studies for Industrial Process Improvement
Experimental Design for Formulation Wendell F. Smith Pittsford, New York
siam. Society for Industrial and Applied Mathematics Philadelphia, Pennsylvania
American Statistical Association Alexandria, Virginia
The correct bibliographic citation for this book is as follows: Smith, Wendell F., Experimental Design for Formulation, ASA-SIAM Series on Statistics and Applied Probability, SIAM, Philadelphia, ASA, Alexandria, VA, 2005. Copyright © 2005 by the American Statistical Association and the Society for Industrial and Applied Mathematics. 10987654321 All rights reserved. Printed in the United States of America. No part of this book may be reproduced, stored, or transmitted in any manner without the written permission of the publisher. For information, write to the Society for Industrial and Applied Mathematics, 3600 University City Science Center, Philadelphia, PA 19104–2688. No warranties, express or implied, are made by the publisher, authors, and their employers that the programs contained in this volume are free of error. They should not be relied on as the sole basis to solve a problem whose incorrect solution could result in injury to person or property. If the programs are employed in such a manner, it is at the user's own risk and the publisher, authors and their employers disclaim all liability for such misuse. Trademarked names may be used in this book without the inclusion of a trademark symbol. These names are used in an editorial context only; no infringement of trademark is intended. Library of Congress Cataloging-in-Publication Data Smith, Wendell F. (Wendell Franklyn), 1931Experimental design for formulation / Wendell F. Smith. p. cm. - (ASA-SIAM series on statistics and applied probability) Includes bibliographical references and index. ISBN 0-89871-580-6 (pbk.) 1. Experimental design. I. Title. II. Series. QA279.S64 2005 519.5'7-dc22
siam.
is a registered trademark.
2004065317
Contents List of Figures
ix
List of Tables
xiii
Preface
xvii
I
Preliminaries
1
1
Introduction 1.1 The Experimental Design Process 1.2 Resources
3 3 6
2
Mixture Space
9
3
Models for a Mixture Setting 3.1 Model Assumptions 3.2 Linear Models 3.2.1 Intercept Forms 3.3 Quadratic Models 3.3.1 Intercept Forms 3.4 Cubic and Quartic Schefte Models 3.4.1 Special Forms 3.5 Choosing a Model
II 4
Design
15 16 19 21 23 25 27 29 31 33
Designs for Simplex-Shaped Regions 4.1 Constraints and Suhspaces 4.2 Some Design Considerations 4.3 Three Designs 4.3.1 Simplex Lattice Designs 4.3.2 Simplex Centroid Designs 4.3.3 Simplex-Screening Designs v
35 35 45 47 47 50 52
vi
Contents 4.4 4.5
Designs for Three Components Coding Mixture Variables
55 57
5
Designs for Non-Simplex-Shaped Regions 5.1 Strategy Overview 5.2 Algorithm Overview 5.3 Creating a Candidate List 5.3.1 XVERT 5.3.2 CONSIM 5.4 Choosing Design Points 5.4.1 Designs Based on Classical Two-Level Screening Designs 5.4.2 D-Optimality Criterion 5.4.3 A-Optimality Criterion Design Study
61 62 65 67 67 70 72 73 76 84 87
6
Design Evaluation 6.1 Properties of the Least-Squares Estimators 6.2 Leverage
95 95 100
7
Blocking Mixture Experiments 7.1 Symmetrically Shaped Design Regions 7.2 Asymmetrically Shaped Design Regions Appendix 7A. Mates for Latin Squares of Order 4 and 5
119 119 131 146
III Analysis
151
8
Building Models in a Mixture Setting 8.1 Partitioning Total Variability. Sequential Sums of Squares 8.2 The ANOVA Table. Partial Sums of Squares 8.3 Summary Statistics 8.3.1 The R2 Statistic 8.3.2 The Adjusted R2 Statistic 8.3.3 PRESS and R2 for Prediction Case Study
153 156 165 172 172 175 176 179
9
Model Evaluation 9.1 Scaling Residuals 9.2 Plotting Residuals 9.2.1 Checking Assumptions 9.2.2 Outlier Detection 9.3 Measuring Influence 9.3.1 Cook's Distance 9.3.2 DFFITS 9.3.3 DFBETAS Case Study
183 183 186 187 192 193 194 198 199 202
Contents
vii
10
Model Revision 10.1 Remedial Measures for Outliers 10.2 Variable Selection 10.3 Partial Quadratic Mixture Models 10.4 Transformation of the Response Case Study
205 205 218 227 235 249
11
Effects 1.1 Orthogonal Effects 1.2 Cox Effects 1.3 Piepel Effects 1.4 Calculating/Displaying Effects 1.5 Inferences Case Study
257 257 261 264 267 269 271
12
Optimization 12.1 Graphical Optimization 12.2 Numerical Optimization 12.3 Propagation of Error
277 279 281 290
IV Special Topics
297
13
Including Process Variables 13.1 Models 13.2 Designs 13.3 Collecting Data 13.4 Analysis 13.5 Related Applications Case Study
299 299 303 308 310 314 315
14
Collinearity 14.1 Definition and Impact 14.2 Warnings and Diagnostics 14.3 Dealing with Collinearity Case Study
325 325 332 341 347
Bibliography
351
Index
363
This page intentionally left blank
List of Figures 2.1 2.2 2.3 2.4 2.5
A two-component simplex A three-component simplex A four-component simplex Simplex coordinate system for a 3-simplex A set of coordinate axes for mixture-related variables
10 11 12 14 14
3.1 3.2 3.3 3.4
A two-component linear response surface Two three-component linear response surfaces A two-component quadratic response surface Curvature modeled by X 1X2X3 and X\X2Xi terms
19 21 23 30
4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8 4.9 4.10 4.11 4.12 4.13 4.14 4.15 4.16
A single lower bound in a 3-and a 4-simplex Two lower bounds in a 3- and a 4-simplex Three lower bounds in a 3-simplex One and two upper bounds in a 3-simplex Three upper bounds in a 3-simplex Three upper bounds in a 3-simplex Constrained region defined by Eqs. 4.15 Constrained region defined by Ui < 1/3, / = 1, 2, 3, 4 {4, 2} and {3, 3} simplex lattice designs Augmented {3, 2} simplex lattice design {3, 3} and {4, 2} simplex centroid designs Full (q = 3) and partial (q = 4) simplex-screening designs Diazepam solubility experiment. Screening plot {3, 3} and augmented {3, 2} simplex lattice designs Constrained region defined by Eqs. 4.15 A pseudocomponent simplex
36 37 37 40 41 43 44 45 48 50 52 53 55 56 59 60
5.1 5.2 5.3 5.4 5.5 5.6
Irregular polygonal-shaped subregion Irregular polygonal-shaped subregion Designs A, B, and C. Joint confidence regions Geometry of a determinant Cox directions in a 3-simplex Design A. Univariate and joint confidence regions
61 62 77 79 83 86
ix
List of Figures
x
5.7
Surfactant experiment
6.1 6.2 6.3 6.4 6.5 6.6
Poultry-feed example 1. Design points Poultry-feed example 2. Design points Poultry-feed examples. Cox-effect directions Poultry-feed example 2. Standard errors of prediction Shrunken regions Poultry-feed example 2. Variance dispersion graph
106 110 Ill 112 113 113
7.1 7.2 7.3
Projection design for q = 2 Projection design for q = 3 Projection design f o r q = 3
134 137 141
8.1 8.2 8.3 8.4 8.5 8.6 8.7 8.8 8.9 8.10 8.11
Hot-melt adhesive experimental setting Hot-melt adhesive experiment Adhesive experiment. Screening plot for viscosity Adhesive experiment. Null-model response surface Adhesive experiment. Null-model response surface Adhesive experiment. Linear-model response surface Adhesive experiment. Quadratic-model response surface Adhesive experiment. Sequential SSs tree Two linear response surfaces Adhesive experiment. Design points Adhesive experiment. Summary statistics vs. model terms
154 155 156 157 158 158 161 162 164 169 176
9.1 9.2 9.3 9.4 9.5 9.6 9.7 9.8 9.9 9.10 9.11 9.12
Studentized-residuals plot Studentized-residuals plot Adhesive experiment. Dotplot of studentized residuals Adhesive experiment. Normal probability plot Prototypical normal probability plots Adhesive experiment. Simulation envelope Adhesive experiment. Index plots Low-vs. high-influence data point Design A. Joint confidence regions Adhesive experiment. DFBETAS for viscosity data Adhesive experiment. DFBETAS for green strength data Adhesive experiment. Design points
187 188 189 189 190 192 193 194 195 201 203 203
10.1 X, Y, and residual outliers 10.2 Adhesive experiment. Box plots of responses 10.3 Adhesive experiment. Index plot of leverages 10.4 Adhesive experiment. Index plot R-student 10.5 Adhesive experiment. Cox-vs. Piepel-effect directions 10.6 Adhesive experiment. Trace plots for GS3 response 10.7 Huber influence and weight functions 10.8 Ramsay influence and weight functions
88
206 207 208 209 210 211 213 214
Jst of Figures
xi
10.9 10.10 10.11 10.12 10.13 10.14 10.15 10.16 10.17 10.18 10.19 10.20 10.21 10.22 10.23 10.24 10.25 10.26
Adhesive experiment. Contour plots for GS3 LDLD experiment. Box-Cox plot LDLD experiment. Index plots of R-student and Cook's D LDLD experiment. Index plots of .R-student and Cook's D LDLD experiment. Plot of studentized residuals LDLD experiment. Lambda plot LDLD experiment. Plots of studenti/.ed residuals Proportions as responses Logit and arcsine square-root transformations Logit transformation of data used for Fig. 10.16 DMBA experiment. Diagnostic plots Adhesive experiment. Box-Cox plot Adhesive experiment. Lambda plot Adhesive experiment. Contour and trace plots Adhesive experiment. Ln(viscosily) response surface Adhesive experiment. Viscosity response surface Adhesive experiment. Contour and trace plots, PQM model Adhesive experiment. Viscosity response surface
217 238 240 240 241 241 243 244 245 247 248 250 250 251 252 253 254 255
11.1 11.2 11.3 11.4 11.5 1 1.6 11.7 11.8
Orthogonal effects Orthogonal effects, constrained region Adhesive experiment. Cox- vs. Piepel-effect directions Cox effects Unrealizable total Cox effects Piepel effects Response trace plots. Cox- vs. Piepel-effect directions Hald cement experiment. Response trace plot
258 259 261 262 265 265 268 273
12.1 12.2 12.3 12.4 12.5 12.6 12.7 12.8 12.9
Chromatography experiment. Contour and overlay plots Desirability function Design-Expert's ramps and JMP's Profiler Coating experiment. Response trace plots Coating experiment. Contour plot Coaling experiment. Response trace plot Adhesive experiment. Contour and 3D surface plots Adhesive experiment. Propagation of error Adhesive experiment. 3D desirability surface plots
280 282 286 287 288 288 292 292 295
13.1 13.2 13.3 13.4 13.5 13.6 13.7 13.8
Mixture and process-variable designs Mixture-process variable design KCV vs. D-optimal design KCV vs. D-optimal design. Standard error plots KCV vs. D-optimal design. Variance dispersion contour plots Mixture-process variable design Finishing product experiment. Design Finishing product experiment. Ln(hydrophilicity) contours
303 304 304 306 307 308 315 324
xii
List of Figures
14.1 14.2 14.3 14.4 14.5
Scatter plot displaying pairwise collinearity DMBA-induced mammary gland tumors design Hypothetical design DMBA experiment. Response trace plots DMBA experiment. 3D surface plot
326 328 338 345 346
List of Tables 2.1
Simplexes contained wilhin simplexes
13
3.1 3.2
Number of terms in some Scheffe polynomials Number of terms in some special Scheffe polynomials
29 30
4.1 4.2 4.3 4.4 4.5
Some designs to support the model Y = [B1 Y\ + Simplex lattice designs. Point types Simplex-screening designs. Point types Diazepam solubility experiment Hypothetical simplex-screening design
5.1 5.2 5.3 5.4 5.5 5.6 5.7 5.8 5.9 5.10 5.11
Boundaries in constrained regions Point-generation and point-selection algorithms Alloy example. XVKRT design Iron ore sinter experiment. Candidate lists Designs A, B, and C. Three two-component, six-point designs Designs A, B, and C. X'X and |X'X| matrices Six-point designs based on the D-criterion Alloy example. D-optimal designs Designs A, B, and C. (X'X)" 1 matrix and tr(X'X)– 1 Surfactant experiment. Candidate points Surfactant experiment. Cii values for two designs
6.1 6.2 6.3 6.4 6.5 6.6 6.7 6.8 6.9
Alloy example. Cii values for a quadratic model cii values for some Scheffe models Designs A, B, and C. Covariance and correlation matrices Surfactant experiment. Correlation matrix of coefficients Leverages for two designs and two models Leverages for a four component simplex centroid design Poultry-feed example 1. Leverages Poultry-feed example 1. Prediction-oriented criteria Poultry-feed example 2. Prediction-oriented criteria
97 98 99 99 103 104 107 108 110
7.1 7.2
Hypothetical design. Blocking arrangement A Hypothetical design. Blocking arrangement B
120 121
xiii
B2
F2
46 50 53 54 57 63 65 69 73 77 79 81 82 84 87 92
xiv
List of Tables
7.3 7.4 7.5 7.6 7.7 7.8 7.9 7.10 7.11 7.12 7.13 7.14 7.15 7.16 7.17 7.18 7.19 7.20 7.21 7.22 7.23
Correlation matrices of coefficients for blocking A vs. B Standard Latin squares for q =4 Orthogonally blocked mixture design for q = 4 Standard Latin squares for q = 5 Effect of run order on a blocked q — 3 D-optimal design Effect of blocking on coefficient variances Projection design. Example 1 Projection design. Example 2 Projection design. Example 2 (cont'd) Projection design. Example 3 Blocked factorial and fractional factorial designs Blocked central composite designs Pattern #1 tor q = 4 Pattern #2 for q =4 Pattern #3 for q =4 Pattern #1 for 4 = 5 Pattern #2 for q =5 Pattern #3 for q = 5 Pattern #4 for q =5 Pattern #5 for 4 = 5 Pattern #6 for 4 = 5
8.1 8.2 8.3 8.4 8.5 8.6 8.7 8.8 8.9
Hot-melt adhesive experiment 156 Adhesive experiment. Sequential SSs for viscosity 161 Adhesive experiment. Partial SSs for viscosity 166 Effect of order of entry on sequential SSs 167 Adhesive experiment. Viscosity data 169 Adhesive experiment. Parameter estimates for viscosity 171 Adhesive experiment. Ordinary and PRESS residuals for viscosity . . . . 178 Adhesive experiment. Sequential SSs for GS3 179 Adhesive experiment. Partial SSs for GS3 182
9.1 9.2 9.3 9.4 9.5
Adhesive experiment. Adhesive experiment. Adhesive experiment. Adhesive experiment. Adhesive experiment.
10.1 10.2 10.3 10.4 10.5 10.6 10.7 10.8
Simple regression example. Influence diagnostics Adhesive experiment. Effect of point deletion Adhesive experiment. Robust regression of GS3 Adhesive experiment. OLS and IRLS parameter estimates Surfactant experiment Surfactant experiment. Analysis for lather units Adhesive experiment. Sequential SSs for GS3 (cont'd) Adhesive experiment. Hierarchy effects on the GS3 analysis
Studentized residuals for viscosity Influence diagnostics for viscosity DFBETAS for viscosity DFBETAS binary representation Influence diagnostics for GS3
122 125 129 130 133 133 135 138 140 142 143 144 146 146 146 147 147 148 148 149 149
188 198 200 201 202 206 210 215 217 219 220 222 222
List of Tables
xv
10.9 10.10 10.11 10.12 10.13 10.14 10.15
Concrete mixture experiment Concrete mixture experiment. Scheffe vs. PQM models Common power transformations LDLD experiment LDLD experiment. Effect of transformation DMBA-induced mammary gland tumors experiment Adhesive experiment. Summary statistics for various models
232 234 236 239 242 247 256
11.1 11.2
Comparison of gradients vs. effects Hald cement data
268 272
12.1 12.2 12.3 12.4
Chromatography experiment Coating experiment Adhesive experiment. One-minute green strength data (GS1) Adhesive experiment. Simulations for GS1
280 284 291 296
13.1 13.2 13.3 13.3 13.4
Design-Expert Fit Summary table Finishing product experiment. Fit Summary table Finishing product experiment cont'd Finishing product experiment. Parameter estimates
313 316 321 322 323
14.1 14.2 14.3 14.4 14.5 14.6 14.7 14.8 14.9 14.10 14.11 14.12 14.13 14.14 14.15 14.16 14.17 14.18
Hypothetical component proportions and correlation matrix 326 DMBA-induced mammary gland tumors experiment (conl'd) 327 Collinearity in the DMBA-induced tumor data, quadratic Scheffe model . 329 DMBA experiment. Parameter estimates 330 DMBA experiment. Simulated responses 330 DMBA experiment. Parameter estimates (simulated responses) 331 DMBA experiment. Correlation matrix of regressors 333 DMBA experiment. Correlation matrix of coefficients 334 DMBA experiment. Variance-decomposition proportions 337 Hypothetical experiment. Component proportions 339 Hypothetical experiment. Variance-decomposition proportions 340 Hypothetical experiment. Variance-decomposition proportions 340 Hypothetical experiment. Auxiliary regressions 340 Hypothetical experiment. Variance inflation factors 341 Hypothetical experiment. Coefficient estimates (pseudos) 342 Hypothetical experiment. Coefficient estimates (reals) 342 DMBA experiment. Regressor respecification 344 Hypothetical experiment. Ill conditioning in ratio models 347
This page intentionally left blank
Preface Few things impact our everyday lives more than those products that are manufactured by mixing ingredients together. From the time we arise in the morning until we retire at night, we depend on formulated products. Some examples are Adhesives Beverages Biological solutions Cements Ceramic glazes Cleaning agents Combination vaccines Cosmetics Construction materials Dyes Fiber finishes Floor coverings Floor finishes Foams Food ingredients Froth flotation reagents Gasketing materials Glasses
Herbicides Hydrogels Inks Paints Personal care products Pesticides Petroleum products Pharmaceuticals Photoconductors Photoresists Polymer additives Polymers Powder coatings Protective coatings Rubber Sealants UV curable coatings Water treatment chemicals
At the time of this writing there is only one hook in the English language dedicated to experimental design for formulation [29]. The probable reason for this is because the subject is a specialized subset of experimental design in general. As a consequence the topic is either relegated to a single chapter in books on regression or the design of experiments [49,79, 107 ] or simply to sections within chapters [93, 94, 102]. This text has evolved from a short course that I have taught since 1995 for the American Chemical Society. Because the sponsoring organization for the course is the American Chemical Society, the majority of students have been chemists. However, this book is intended for a much broader audience that would include students and researchers in the physical sciences, engineering disciplines, or statistics. The book is intended to provide a practical step-by-step guide to the design and analysis of experiments involving formulations. It contains many examples selected from a wide variety of fields along with output from several popular computing packages. Formulas underlying the computer output will be explained, and proper interpretation of the output will be emphasized. There is more than enough material in this book for it to be used as the xv ii
xviii
Preface
supporting text for a course at the senior undergraduate or beginning graduate level. With selective abridgement, the text is also suitable for a two- or three-day short course. The prerequisites for this book are relatively modest. Previous exposure to a first course in statistics and an introductory course on experimental design will be assumed. The reader should be comfortable with hypothesis tests, confidence intervals, the normal, t, and F distributions, and factorial, fractional factorial, and central composite designs. It is also assumed that the reader has some knowledge of matrix algebra. The matrix formulation of ordinary least squares ( L S ) is well covered in texts on linear regression and will not be repeated here. At the same time, we will draw generously on the results of this approach to OLS. Statistical proofs are largely absent, as they can be found in texts on linear regression or experimental design. Topics are ordered according to a sequence of steps one normally follows in a designed experiment — hypothesizing a model, designing an experiment to support the model, collecting data, and finally fitting a model and interpreting the results. The four major parts of the text are as follows: • Preliminaries (Chapters 1-3). This section covers topics that one needs to know before beginning to design a mixture experiment. This includes a description of mixture space and an explanation of the model types commonly used in a mixture setting. • Design (Chapters 4–7). These chapters cover the design of mixture experiments, design evaluation, and the modification of designs following evaluation. In addition, the blocking of mixture experiments is discussed in this section. • Analysis (Chapters 8-12). This section covers model fitting, model evaluation, and the modification of models following evaluation. Other topics include the concept of an effect in a mixture setting and elementary optimization methods. • Special Topics (Chapters 13 and 14). It is sometimes of interest to combine mixture and nonmixture variables (often called process variables) in a designed experiment. This topic is covered in Chapter 13. Chapter 14 explains the concept of collinearity and the possible problems that can result from its presence. The beginning student in this area need not embark on a complete read-through. To get started as quickly as possible designing and analyzing mixture experiments, a beginning practitioner should be comfortable with the material in Chapters 1–6, 8, 9, 10 (Sections 1 and 2), 11 (Sections 1–4), and 12 (Sections 1 and 2). In addition, one could save the material on robust regression (Section 10.1, pages 212–218) for a later reading. This book would not have been written were it not for my friend Henry Altland, retired from Corning Incorporated. "Hank" Altland has been encouraging me for years to write a book on this subject, and so in 2002 I decided to undertake the project. For the past two years he has provided timely feedback and assistance in the preparation of the manuscript, for which I am most grateful. In 1984 I took an American Chemical Society short course titled Sequential Simplex Optimization. The course changed my professional life from a focus on photochemistry to a focus on the statistical design and analysis of mixture experiments. Stanley N. Deming,
Preface
xix
Professor (now Professor Emeritus) of Analytical Chemistry, University of Houston, was one of the teachers of that course, and we have continued a personal and professional relationship over the past 20 years. Stan provided extremely valuable critiques of the manuscript, which led to significant improvements in both clarity and content. When I realized the important role that formulation plays in the development of color photographic products, I arranged to have John A. Cornell, Professor (now Professor Emeritus) of Statistics, University of Florida, come to Eastman Kodak Company as a consultant. For several years I served as John's host while John served as my mentor, and as a result much of what I learned I owe to him. John provided an extremely valuable critique of the manuscript that led to several improvements. Patrick Whitcomb, Principal, Slat-Ease, Inc., also provided helpful feedback in the preparation of this book. Pat founded Stat-Ease, Inc. in 1985, and since that time Stat-Ease's product Design-Expert has enjoyed an ever-widening user base. Pat teaches several short courses on the design of experiments and has had considerable experience in the design and analysis of mixture experiments. Several of the examples in this book were checked by Pat using Design-Expert Version 7. Finally, I would like to thank Linda Thiel, Acquisitions Editor, Society for Industrial and Applied Mathematics (SI AM). Linda encouraged me to submit a preliminary manuscript to SIAM for review and has provided advice and help at several points during the preparation of the final manuscript.
This page intentionally left blank
Parti
Preliminaries
This page intentionally left blank
Chapter 1
Introduction
1.1
The Experimental Design Process
Many scientific experiments can he broken down into three stages: the planning of the experiment, the implementation of the experiment, and the analysis and interpretation of the data that are collected. When the planning stage involves the statistical design of experiments (DOE), when the implementation stage entails a randomization scheme and possible blocking, and when the interpretation stage utilizes the statistical analysis of data, then these three stages comprise the experimental design process. While the phrase "experimental design process" tends to put the emphasis on the design (planning) stage, one should keep in mind that it encompasses the planning, implementation, and analysis of an experiment using valid statistical principles. Aspects of all three stages will be addressed in this book. In the course of planning an experiment, one must decide what conditions are to be varied (the treatments or factors) and what response or responses are to be measured. One then hypothesizes that any responses to be observed and measured are functionally related to the levels (or values) of a factor or factors. This can be represented analytically as
where /(factor levels) is to be read "some mathematical function of the levels of a factor or factors". A formulation is nothing more than a mixture, being composed of two or more components. Component proportions are not independent of one another — if the proportion of one component is increased, then the proportion of one or more of the other components must decrease if the total weight (or amount) of the mixture remains the same. The proportions of mixture components could be thought of as factor levels, although the word factors is usually reserved for nonmixture variables that are often (although not necessarily) independent of one another. Factors are sometimes called "process variables". Examples are time, temperature, pressure, coating speed, etc. In a mixture setting, then, the responses to be observed and measured are functionally related to the component proportions, and we can rewrite Eq. 1.1 as
3
4
Chapter!.
Introduction
The responses that we will be primarily concerned with in this book will be measured responses — those that can be arranged on a continuous scale from smallest to largest, can be negative or positive, and have a consistent unit of measurement (that is, a difference of one unit has the same meaning wherever the difference occurs) [91]. Responses that are counts (taking only nonnegative integer values) or that are dichotomous (taking one or the other of two values) require special regression models (such as the generalized linear model [108]) and will not be considered in this book. We will, however, consider cases where the response (the left side of Eq. 1.2) is a proportion — the quotient obtained when the magnitude of apart is divided by the magnitude of the whole. For example, if the part were the number of successes (or failures) and the whole were the number of attempts, then the response would be the proportion of successes (or failures) out of the number of attempts. An example of such an experimental setting is the proportion out of 30 rats exhibiting DMBA (7,12-dimethylbenz(a)anthracene)-induced mammary gland tumors as a function of the relative proportions of fat, carbohydrate, and fiber in an isocaloric diet [17]. Although the outcome on a "per-rat" basis is dichotomous (either tumor or no tumor), when the number of rats exhibiting tumors is divided by the number of rats in a group (30), the result is a proportion. In this case the left and right sides of Eq. 1.2 are in units of proportion. Another example of an experimental setting that will be covered in this book is the dependence of textile hydrophilicity (the response) on the method of finishing [ 16]. Textile hydrophilicity was measured as a function of the relative proportions of three fabric softeners as well as the total amount of the softeners and the amount of a resin. Because softener amount and resin amount are factors (nonmixture variables), the model equation might be of the form
or perhaps of the form
Experiments in which component proportions and factor levels are combined and varied are called mixture-process variable (MPV) experiments [30, 32, 63, 75]. A unique process variable is the total amount of a mixture. Experiments in which the total amount of a mixture is varied as well as the relative proportions of the mixture components are called mixture-amount (MA) experiments [ 128, 129]. The textile hydrophilicity example is really a MA-MPV experiment, because the amount of the softeners (the mixture components) is varied as well as the resin level. While most of the examples in this book are mixture experiments, MA and MPV designs and models will be discussed in Chapter 13. Although not the subject of this book, readers should be aware that there are experimental settings that are modeled by equations of the form
In these models the mixture compositions are on the left side of the equal sign because the compositions are the response. An example of this would be an experiment to determine
1.1. The Experimental Design Process
5
the relative proportions of sand, silt, and clay in sediments as a function of water depth (the factor) in a lake. Experiments modeled by equations of the form of Eq. 1.3 are the subject of the text The Statistical Analysis of Compositional Data by Aitchison [1 ]. There are four general goals to be achieved using model equations of the form of Eq. 1.2: 1. Use the model to gain insight as to why the mixture compositions behave as they do. (E.g., Is there synergistic or antagonistic blending among the components?) The model is used as a tool for understanding. 2. Use the model to determine the mixture composition(s) where the response is near a maximum, a minimum, or a target value. The model is used as a predictive tool. 3. Use the model to determine the mixture composition(s) where the effect of mixing measurement error is minimal. Mixing measurement error arises through imprecise measurement of the amounts of the mixture components. Such errors lead to actual mixture proportions that are different from the aim mixture proportions. If the formulation is going to be manufactured, this could be an important consideration. 4. Use the model to determine the mixture composition(s) where the effects of external uncontrollable variables, such as temperature and humidity, are minimized. As with item 3, if the formulation will be manufactured, then this may also be an important consideration. The first three goals are addressed in this book, and leading references to the fourth are given in Section 13.5. Model equation 1.2 implies that the response to be observed and measured is functionally related to the composition of the formulations by a model. Generally one does not have enough knowledge about the system to write a theoretical model, and so we fall back on an empirical model that we hope will be locally satisfactory. The empirical models most commonly used are polynomial functions of graduating degree (such as linear, quadratic, cubic, and quartic). See Box, Hunter, and Hunter, Chapter 9 of 113) for a discussion of empirical vs. theoretical models and Box and Draper, Chapter 12 of [ 12| for a discussion of the links between the two. The reader may wonder if the cart has been put before the horse because we are talking about models before we have even considered the design of the experiment. The reason for this is that to properly design an experiment, one must have some idea of the model that the design is intended to support. By support it is meant that there are enough experiments to adequately fit the model plus additional experiments to provide some measure of experimental error. A measure of experimental error is needed to make inferences about the statistical significance of the model as well as of the terms in the model at the analysis stage. It is for these reasons that models for a mixture setting are described in one of the earlier chapters of this book. At the beginning of an investigation, and without prior subject-matter knowledge, one would have no idea about the functional relationship between the response and the mixture variables. What is often done at this point is to hypothesi/.e a linear polynomial model, sometimes called a screening model. Fitting the data to a screening model helps to sort
6
Chapter 1. Introduction
out which components of the mixture have an effect on the response and which do not. Components that have no effect on the response can be held constant in a future experiment, thus reducing the number of variables. Second-degree and higher-order polynomial models — commonly called responsesurface models (even though linear models also generate response surfaces) — require more experiments than linear polynomial models. Second-order response surfaces have stationary points: tops of "mountains", bottoms of "valleys", and "saddle" points. For this reason optimization is usually carried out using response-surface models. One might hypothesize a second-degree (quadratic) model at the outset of an investigation because of prior subject-matter knowledge or possibly because experimentation is inexpensive. Subjectmatter knowledge could have arisen from having run a screening experiment and discovering that there was lack of fit. A test for lack of fit, explained in Chapter 8, requires that a design include replicates, multiple experiments carried out at the same set of conditions. How the experiment is to be conducted must also be thought about at the design stage because the way that the experiment is carried out determines the calculations required to make meaningful statistical inferences. Randomization and blocking provide two strategies for handling unwanted variability. Unplanned systematic variability will lead to distortion (bias) of the estimates in a fitted model. Randomization does not remove this systematic variability, but it converts it into random, or chance-like, variability. Variability arising from unwanted step changes, such as day-to-day or batch-to-batch variations, can be minimized by blocking. Blocking transforms nuisance variables that are known or suspected of undergoing discrete changes into factors of the design. Randomization and blocking are not mutually exclusive and are often used together. At the analysis stage, linear regression using ordinary least squares will be used to fit models to data. On fitting a model it is not unusual to conclude that it is either overspecified or underspecified. If the former, then there are terms in the model that have no statistical significance. In this case they may be eliminated, leading to a simpler, more parsimonious model. On the other hand, a lack-of-fit test may suggest that the model is underspecified, in which case additional, usually higher-order, terms are needed. To support the additional terms the design may need augmentation, in which case new data will need to be collected and the results reanalyzed. Thus the sequence "plan —> conduct —> analyze" is often an iterative process.
1.2 Resources At the time of this writing, the only other book in the English language dedicated to the subject matter in this book is Experiments with Mixtures by John Cornell [29]. A few texts, such as Draper and Smith [49], Khuri and Cornell [79], and Myers and Montgomery [107], do contain chapters introducing and discussing design and analysis of mixture experiments. Cornell's excellent third edition thoroughly covers the literature up to 2002. Because of its thorough coverage the book is highly recommended to practitioners who need a single reference with virtually complete coverage of the literature. Experimental Design for Formulation does not purport to cover all of the subjects in Cornell's treatise. Its purpose is to provide the industrial scientist or engineer, who may have limited knowledge of statistics, with the basic tools needed to put these methods into practice.
1.2.
7
Resources
It is the author's belief that anyone who aspires to become a practitioner of these methods should have at least one book on linear regression analysis within arm's reach. Books on linear regression have proliferated in recent years, and many excellent texts are available. Some well-known texts are Draper and Smith [49], Montgomery, Peck, and Vining [100], and Myers [104]. If one is completely new to model fitting, then Lunneborg's Modeling Experimental and Observational Data [91 ] provides an excellent introduction. Inevitably one will need software to implement the methods in this book. There is a plethora of products to choose from. Examples of products with DOE functionality are JMP, MINITAB, SAS (including the ADX Interface), S-PLUS, STATGRAPHICS Plus, and STATISTICA.1 Design-Expert and HCHIP are examples of dedicated DOE products with mixture and mixture-process variable capabilities. The software package MIXSOFT is a collection of FORTRAN routines for the statistical design and analysis of mixture experiments and other experiments with constrained experimental spaces. For the advanced practitioner, products such as GAUSS, MATLAB, and SAS/IML are tools for doing statistical computations using matrices and vectors. Information about most of these products can be found on the Web, and in some cases product reviews are available. Only a subset of these products will be cited in the text. Arc is a computer program written in the Xlisp-Stat language that is designed to be used with the book Applied Regression Including Computing and Graphics by Cook and Weisberg [24]. Both Arc and Xlisp-Stat can be downloaded for free from http://www.stat.umn.edu/arc/ To use Arc, you do not need to know how to program in Xlisp-Stat. A useful feature of Arcis a dialog window for calculating probabilities (p values) from the values of t, x2- and F tests. A Web site that can be very useful is StatLib. This is a system for distributing statistical software, datasels, and information by electronic mail, FTP, and WWW. It also provides links to other Web sites of interest to the statistics community. The site is maintained by the Department of Statistics at Carnegie Mellon University; the URL is http://lib.stat.cmu.edu/ A representative overview of scientific journals with articles on mixture experiments can be obtained by perusing the Bibliography. The greatest percentage of these have appeared in Technornetrics and the Journal of Quality Technology. Taken together these two journals constitute approximately 50% of the journal entries in the Bibliography. Much of the early work in this area appeared in Technornetrics, with the lion's share of presentday work appearing in the Journal of Quality Technology. "41 Years of Technornetrics" is a set of four CDs that includes a fully searchable archive of all Technometrics articles from 1959 to 2000 in PDF format. This is available from the American Statistical Association at http://www.amstat.org/publications/ All software product names mentioned in this hook are tiademarked.
8
Chapter 1. Introduction
A useful overview that readers should find helpful is Piepel and Cornell, "Mixture experiment approaches: Examples, discussion, and recommendations" [130]. The same authors have compiled A Catalog of Mixture Experiment Examples [131]. The Catalog contains a comprehensive bibliography with over 400 entries plus numerous tables. Between the tables and the bibliography one can quickly track down published articles based on a variety of search criteria, such as the number of mixture components, the types of constraints, the number of design points, the fitted model(s), etc. The Catalog is available as a Word file via email from
[email protected].
Chapter 2
Mixture Space
The first subject that needs to be addressed is to define the experimental space within which one is going to carry out experiments. This space, known as the mixture simplex, is defined by the following two constraints: 1. The summation or equality constraint:
2. The nonnegativity constraint:
The symbol q is used throughout the mixture literature to represent the number of mixture components. The symbol X, is used to symbolize component / as well as its proportion in a mixture. The latter may be expressed in units of proportion by weight, volume, or mole fraction. Most often, however, the units are proportions by weight. One could, of course, express the total in terms of ounces, pounds, grams, etc., as long as the total always adds up to the same amount. For example, if one were formulating a cake recipe, the amounts of the individual components (flour, sugar, butter, etc.) could be expressed in terms of ounces as long as the total always adds up, say, to 16 ounces. Design-Expert allows one to express a formulation in terms of actuals (e.g., ounces, pounds, grams) or in terms of reals (component proportions). Although the words "actual" and "real" are synonyms, it is convenient to adopt this nomenclature to distinguish between the two methods of expressing the composition of a mixture. Let us begin by using the mixture constraints to define the 2-simplex, that is, the simplex for q = 2 components. If we were working with two factors (nonmixture variables) instead of two mixture components, we could define our experimental space in terms of a set of x,y axes (see Fig. 2.1). Along the horizontal Xi axis, the point where X\ = 1.0 (labeled a) may be viewed as a "mixture" in which the proportions of X\ and Xi are 1.0 9
10
Chapter 2. Mixture Space
Figure 2.1. A two-component simplex. and 0, respectively. Similarly, along the vertical X2 axis, the point where X2 = 1.0 (labeled b) can be viewed as another "mixture" where the proportions of X\ and X2 are 0 and 1.0, respectively. If we pass a line through these two points, with no restrictions on the length of the line, then anywhere along this line it will be true that X\ + X2= 1.0 and the equality constraint will be satisfied. However, this will be true even when X\ or X2 takes on negative values (dashed lines). Applying the nonnegativity constraint restricts the line to that which is bolder in Fig. 2.1. The steps that we have taken to define the 2-simplex can be summarized as follows: • The summation constraint restricted the two-dimensional factor space to a one-dimensional line. • The equality part of the nonnegativity constraint defined two bounding lines. • The two constraints reduced the factor space to a one-dimensional simplex. Note that we have gone from a two-dimensional factor space to a one-dimensional mixture space, and as a result we have lost one degree of freedom. In general we shall see that mixture space for a q-component mixture is always a q — 1 dimensional simplex. As we are capable of illustrating in three dimensions, this means that we can also illustrate three- and four-component simplexes. Moving on to the 3-simplex, consider a set of orthogonal x, y, z axes labeled A^, XT, and X3, (Fig. 2.2). If we tick off points a, b, and c on these axes where X\ = X2 = X3= 1, and then pass a plane through these points, we will have an unbounded plane. If we now apply the nonnegativity constraint, then this plane becomes bounded by three additional planes — the X1,X2 AS.X3, and X\X^ planes — leading to the three edges ab, be, and ac, respectively. The resulting 3-simplex is an equilateral triangle.
Chapter 2. Mixture Space
11
Figure 2.2. A three-component simplex. Because the 3-simplex is so easy to draw, we will use three-component mixtures for many of the examples in this hook. For q > 3, we can generali/.e the steps that are taken to define a simplex. • The summation constraint restricts the factor space to a q — \ dimensional plane (q = 3) or hyperplane (q > 3). • The equality part of the nonnegativity constraint creates q bounding planes (q = 3)orboundinghyperplanes (q > 3) of dimension q-\. • The two constraints reduce the factor space to a regular q — 1 dimensional simplex. In the case of four components, we cannot really conceptualize four-dimensional factor space. However, if we could and if we applied the stepwise procedure, our resulting 4-simplex would look like Fig. 2.3, i.e., a tetrahedron. As far as illustrations go, this is all we can do. We can infer from Figs. 2.1-2.3 a few properties of higher-order simplexes. Property 2.1 Simplexes are modular — all of the boundaries are simplexes. For example, the three-dimensional tetrahedron is hounded by four two-dimensional simplexes (triangles), six one-dimensional simplexes (edges), and four zero-dimensional simplexes (vertices), for a total of 14. Property 2.2 Because each vertex is connected to every other vertex, the number of ddimensional simplexes in a q-component simplex is
12
Chapter 2. Mixture Space
Figure 2.3. A four-component simplex. For example, the total number of one-dimensional edges (d = 1) in a q-component simplex is
This property leads directly to Property 2.3. Property 2.3
The total number of simplexes bounding any simplex is
For example, a 10-component simplex is 9-dimensional and is bounded by 210 — 2 = 1022 simplexes of lower dimensions. The number of simplexes of each dimension is tabulated in Table 2.1. Each vertex of a simplex represents a pure component. Binary blends occur on the one-dimensional edges. Ternary blends occur on the two-dimensional faces or constraint planes (the triangle in Fig. 2.2 and the faces of the tetrahedron in Fig. 2.3). Four-component blends are located within the tetrahedron. It is fair to ask why one might want to count the number of edges in a simplex. One reason is that the edges are good places to locate design points if one is interested in detecting binary blending behavior. For example, if one were interested in detecting quadratic blending of two components, then the vertices and the midpoints of the edges would be good places to locate design points. If one were interested in detecting cubic blending of binary blends, then the vertices and the 1/3 – 2/3 blends would be good places to locate design points. Thus the total number of design points needed to detect binary
Chapter 2. Mixture Space
13
Table 2.1. Lower-dimensional simplexes contained within a 10-compotient .simplex Type Vertices Edges Constraint planes
Total
Dimension 0 1 2 3 4 5 6 7 8
Number 10 45 120 210 252 210 120 45 10 1022
blending is equal to the number of vertices plus some multiple of the number of edges. Similar arguments would apply for detecting and measuring ternary blending. These arguments assume that one is able to explore the whole simplex, which will not always be the case. Further consideration will be given to design points and their location in Chapters 4 and 5. A word should be said about the simplex coordinate system that is customarily used with mixture simplexes [29J. One-hundred percent of a component is always located at a vertex. Zero percent of a component is always located on the opposite q — 2 dimensional subsimplex. For two components, 100% of component X\ is located at one end of the 2-simplex (a vertex). Zero percent of component X\ (or 100% of X2) is located on the q — 2 = 0 dimensional subsimplex that does not contain Xi, which is the opposite end of the 2-simplex (a vertex). In the case of three components (Fig. 2.4), 100% of component Xi is located at the top vertex. Zero percent of component X\ is located on the opposite q — 2 = 1 dimensional subsimplex, which is the bottom edge of the triangle. Proceeding from this edge, the horizontal dotted lines represent increasing amounts (in increments of 0.2) of component X|. The proportions of Xi are indicated along the right side of the triangle. In a similar fashion, the proportions of Xi are indicated along the left side of the triangle and correspond to the dotted lines sloping downward to the right. The proportions of X\ are indicated along the bottom of the triangle and correspond to the dotted lines sloping upward to the right. At any point in the plane, the proportions sum to one. In the 4-simplex illustrated in Fig. 2.3, 100% of component X[ is located at the top vertex. Zero percent of component Xi is located on the opposite q — 2 = 2 dimensional subsimplex, the triangular base of the tetrahedron. The coordinate lines in the 3-simplex are replaced by coordinate planes in the 4-simplex. In a 5- or higher-simplex, the planes become hyperplanes. It was stated above that the simplex coordinate system is the system that is most often used. It should be pointed out that one could pass a set of orthogonal axes through a simplex and represent the component proportions in terms of coordinates based on the set of
14
Chapter 2. Mixture Space
Figure 2.4. Simplex coordinate system for a 3-simplex.
Figure 2.5. A set of coordinate axes for mixture-related variables. orthogonal axes. This is illustrated for the case where q = 3 in Fig. 2.5. Here the origin of the orthogonal x, y axes, labeled W1 and W2, is located at the overall centroid of the simplex (where (X\, X2, X3) = (1/3, 1/3, 1/3)). The origin could have been located elsewhere, for example at a vertex. In the general case, the W,, / = 1 , . . . , (q — 1), will be linear combinations of the X,-, z = 1 , . . . , q. Cornell [25] refers to the W,- as mixture-related variables (MRVs). Designing with MRVs is beyond the intended scope of this book. For details see Cornell's text [29].
Chapter 3
Models for a Mixture Setting
Many types of regression models can be used in a mixture setting, hut we shall focus in this chapter on Scheffe canonical polynomials. According to Webster, canonical means "reduced to the simplest or clearest schema possible." Scheffe polynomials are by far the most commonly encountered mixture model forms in technical articles and books as well as in software packages that have mixture capabilities. Certain reparameterized forms of the Scheffe models can be useful, and these will be discussed as well. The models discussed in this chapter certainly do not embrace all of the models that can be used in a mixture setting, but they do cover those that are most frequently used. Cornell and Gorman [33] discuss other mixture model forms. First, a word should be said about the meaning of linear in the context of regression models, as it has two meanings. The term "linear model", as opposed to a "nonlinear model", is one that can be written in the form where E ( Y ) is the expected value of K, the a s are called parameters or coefficients, and the Z,, i; = 0, 1, . . . , p, are predictor variables, regressor variables, or simply regressors. In the sense used here, a linear model is said to be linear in the parameters. For example, the following are linear regression models:
whereas the regression models
15
16
Chapter 3. Models for a Mixture Setting
are nonlinear because they cannot be written in the form of Eq. 3.1. In all six expressions, the as, Bs, and ys are the parameters. In the case of the three linear models, Eqs. 3.2-3.4, and with reference to Eq. 3.1,
Thus the Zs can be algebraic (as in Eq. 3.4) or transcendental (as in Eqs. 3.2 and 3.3). Particularly relevant to our purposes is Eq. 3.4, which is an example of a polynomial model. A polynomial model is of the form
where the exponents are whole numbers (i.e., members of the set { 0, 1, 2, 3, ...}). The value of the highest power in a polynomial model is called the order or degree of the model. Equation 3.4, for example, is a second-order or second-degree polynomial. Polynomials of degree one are called linear polynomials, of degree 2 quadratic, of degree 3 cubic, and of degree 4 quartic. Thus the adjective "linear" is used here in a sense that differs from its use to describe a linear vs. nonlinear model. All of the models used in this text will be linear in the parameters, so when the adjective "linear" is used it should be taken to mean "first-order". More generally, the degree or order of a polynomial model is usually taken as equal to the largest sum of exponents appearing in any term. For example, consider the polynomial model
This is a third-degree polynomial because the exponents of the terms in the second line of Eq. 3.5 (the terms of highest order) sum to three. The terms containing x\x2 and x\x2 are mixed third-order terms.
3.1 Model Assumptions For ease in speech and writing, we often make reference to "models" without being specific about what we mean by a model. To be precise, a model is composed of two parts, the model equation and any assumptions that need to be made about the terms in the model equation [150]. Looking ahead to Chapter 8, we shall eventually be fitting response data to regression models using the method of ordinary least squares (OLS). In so doing, certain assumptions will be made, and these assumptions will be part of the OLS regression models. The situation will be illustrated using a simple linear regression "model", for which we can write where E(Y) is the expected value of Y conditional on a specific value of X (sometimes written E(Y\X) or simply n). E(Y) is equal to the average, or mean, value expected after
3.1. Model Assumptions
17
an infinite number of samplings at the specific value of X. Although this expression was loosely referred to as a "model", it is more precisely an expectation function. For the ah observation, the expected value E(Y,) is, of course, not the same as a single observed value, YJ. The difference between the two is given by
which on rearrangement leads to
As the model is applicable to all observations, we can drop the subscript / and write
Generalizing, this can be written
which we now take as our model equation [ 148, 1501. The last term represents a disturbance and is a recognition of the fact that our observations will not fit the model exactly. Although each X is assumed to be measured without error, € is a random variable that has some assumed probability distribution. The distributional properties of E are assumed to be passed to the Ys. The observed response is thus also a random variable with a probability distribution at each specific value of X. When model 3.6 is fitted to data, we write the fitted model as
where the symbol Y is read "Y hat" to symbolize the fitted value of Y, and (I0 and a\ are the least-squares estimators of the intercept and slope, respectively. Note that the u s in Hq. 3.6 have been replaced with a s. In OLS, the estimators are linear combinations of the observations, F,. As a result, the estimators are also random variables and also have a probability distribution. The differences between the observed values Y, and the corresponding fitted values YJ are called residuals. A residual is symbolized by a lower-case Roman e, to distinguish it from the conceptual errors, which are symbolized by a lower-case Greek epsilon, €. Thus we have
To complete the description of our model(s), we need to specify the assumptions about the €i s. These assumptions are embodied in the Gauss-Markov conditions, which are
18
Chapters. Models for a Mixture Setting
Condition 3.8 implies that no necessary explanatory variables have been left out of the model, the result of which would lead to a systematic bias in the disturbances. Condition 3.9 implies that all of the observed responses (K/) are equally unreliable and is known as the homogeneous variance assumption. This assumption is usually checked after a model is fitted to the data. When this condition is violated, the problem is handled by a variance stabilizing transformation of the response, or in some cases by weighted least squares. Condition 3.10 means the disturbances are pairwise independent of one another. Leastsquares fitting of models when the disturbances are not independent of one another is handled by generalized least squares. Weighted least squares will be discussed in Chapter 10 in the context of robust regression, but generalized least squares will not be discussed at all. The reader is referred to texts on linear regression, such as Draper and Smith [49], Montgomery, Peck, and Vining [100], or Myers [105], for information about generalized least squares. Taken together, the Gauss-Markov conditions imply that the disturbances are independently and identically distributed, often abbreviated €, ~ i.i.d. An important result about the quality of the least-squares estimators is the Gauss-Markov theorem, which states the following: Under the G-M conditions, the least-squares estimators are unbiased and have minimum variance among all unbiased linear estimators. The estimators are called best linear unbiased estimators (BLUE) because • "best" implies minimum variance (precise), • "linear" means the estimated coefficients are linear combinations of the K/, and • "unbiased" means £"(«/) = «/• Note that the Gauss-Markov conditions do not say anything about the e, being normally distributed. The assumption of normality is in fact not required for least-squares estimation unless one cares to engage in hypothesis testing and confidence-interval estimation, which is most often the case. Like the homogeneous variance assumption, the normality assumption is usually checked after the model is fit. Thus the complete assumptions are that the e, are normally and independently distributed with mean 0 and variance a2, abbreviated e, ~ NID(0, a2). In the sections to follow, the reader will encounter a variety of expectation functions, but in the interest of simplicity, these shall be referred to as "models" or "model functions". Despite their variety, all of the models can be succinctly represented by the general linear model
Let n be the number of observations (i.e., mixtures, formulations, experiments,...) and p be the number of parameters in the model. Y is then a n x 1 vector of observed responses, X is a n x p matrix of regressor variables, ft is a p x 1 vector of coefficients, and € is a n x 1 vector of disturbances.
19
3.2. Linear Models
3.2 Linear Models Assume that the following disturbance-free data have been collected for a two-component mixture: Xl
X2
Y
1.0 0.8 0.6 0.4 0.2 0.0
0.0 0.2 0.4 0.6 0.8 1.0
10 12 14 16 18 20
Fig. 3.1 displays the response, Y, as a function of the composition of the mixture. The plotted line is the first-order (linear) response surface for the two-component mixture.
Figure 3.1. A two-component linear response surface. Let us conjecture that this surface can be described by the linear regression function Eq. 3.12: A model that will fit this data is
That this model fits the data is easily verified by substituting values for X1 and X2 into Eq. 3.13. Unfortunately, this equation is not unique. Another model that will fit the data equally well is With a little reflection it becomes apparent that there are an infinite number of models of the form of Eq. 3.12 that will fit this data. What we have encountered is a situation where the regression function is overparameterized — there are more parameters than can be estimated uniquely.
20
Chapter 3. Models for a Mixture Setting
For this particular example, the reason for the problem is that there are actually three explanatory variables in Eq. 3.12. Two of these are explicit, while one is implicit. We could rewrite Eq. 3.12 as
where 1 is an implied (albeit constant) regressor. IfwenotethatX1+X2 = 1—that is, there is an exact linear dependency between the regressors — then we have an overparameterized model. To correct the problem we can carry out the following algebra:
The parameters of Eq. 3.15 may be written in terms of the parameters of Eq. 3.12 as
As a specific example, consider the fitted model Eq. 3.13. The necessary algebraic transformations would be the following:
The reader may care to show that Eq. 3.14 may be transformed to Eq. 3.16 as well. Generalizing, we can state the following: A first-order polynomial in q factors has q + / terms, of which one term must be deleted to obtain a full-rank mixture model. A full-rank model, whether it is a mixture or a nonmixture model, is one in which there are no linear dependencies among the regressors, and as a result the parameter estimates will be unique. Note that the bold-face statement specifies only that one term must be deleted, not that the intercept must specifically be deleted. This is discussed further in the subsection to follow. The model function Eq. 3.15 is referred to in the mixture-experiment literature as a linear Scheffe polynomial. The general form of a linear Scheffe polynomial may be written
The number of terms in a linear Scheffe polynomial is the same as the number of components in the mixture, namely q.
3.2. Linear Models
21
As a second example, consider the linear Scheffe model of the form
The response surface for this model is shown in Fig. 3.2 (left). It is important to realize that linear coefficients in Scheffe models are estimates of the response at each vertex — not estimates of the effects of the components. This important distinction is not helped by some software products that label coefficients in linear Scheffe models "effects". To see the difference, consider the surface on the right in Fig. 3.2, for which the model is
For the same mixture composition, a response in the right illustration is displaced from a response in the left illustration by +10. This is true for any composition, not just the vertices.
Figure 3.2. Two three-component linear response surfaces. The effect of a component is manifested by a gradient (slope) in some specified direction. Whatever direction one might choose in Fig. 3.2, slopes and therefore effects are identical for the two surfaces.
3.2.1
Intercept Forms
Previously it was stated that we must delete one term from a q-factor linear regression model to obtain a full-rank mixture model. For the specific case of model function E,q. 3.12, page 19, either the a\X\ term or the 0-2X2 term could have been selected for deletion. To see this, make the substitution
in Eq. 3.12. Algebraic rearrangement will then lead to an intercept form for the mixture model.
22
Chapter 3. Models for a Mixture Setting
The parameters in Eq. 3.18 have the following meaning:
Alternatively, had we made the substitution
in Eq. 3.12, then the reparameterized model function would have become
In this case the parameters have the meaning
Regression functions Eqs. 3.18 and 3.19 are called intercept mixture models. The intercept (yo) in these models is the estimated response at the vertex of the mixture variable that has been algebraically eliminated from the model. Another way to think about the intercept forms of mixture models is to derive them starting from the full-rank Scheffe model Eq. 3.15. For example, model function Eq. 3.18 may be derived as follows:
Using this type of algebra, and starting with the Scheffe model Eq. 3.16, page 20, one can derive the equivalent models
"Equivalent" means that models Eqs. 3.16, 3.20, and 3.21 will each reproduce the response surface in Fig. 3.1. However, the meanings of the parameter estimates in the three models are not equivalent. Let us write Eq. 3.20 and Eq. 3.21 as
where go and g/ are least-squares estimates of yo and y,,i = 1, 2. The value of go estimates the response at the X\ vertex when i = 2 and at the X2 vertex when i = 1. The value of g,-, i = 1,2, estimates the difference between the response at the vertex of the mixture variable missing from the model and at the /th vertex.
23
3.3. Quadratic Models
3.3 Quadratic Models In a manner that is similar to Section 3.2, assume that the following disturbance-free data have been collected for a two-component mixture: Xi 1.0 0.8 0.6 0.4 0.2 0.0
X2 0.0 0.2 0.4 0.6 0.8 1.0
Y 10.00 12.64 14.96 16.96 18.64 20.00
Fig. 3.3 displays the response, F, as a function of the composition of the mixture.
Figure 3.3. A two-component quadratic response surface.
The data in the table can be fitted to the quadratic response function Eq. 3.22.
A model that fits the data is
Another model that fits the data equally well is
In fact, there are an infinite number of models of the form Eq. 3.22 that will lit this data. As in the linear case, the reason for the problem is that model Eq. 3.22 is overparameterized for the two-component mixture setting. To see this, it is convenient to augment the
24
Chapter 3. Models for a Mixture Setting
X data in the previous table as follows: (1)
Xi
X2
.0 .0 .0 .0 .0 .0
1.0 0.8 0.6 0.4 0.2 0.0
0.0 0.2 0.4 0.6 0.8 1.0
X{X2 0.00 0.16 0.24 0.24 0.16 0.00
X1/2 1.00 0.64 0.36 0.16 0.04 0.00
X\ 0.00 0.04 0.16 0.36 0.64 1.00
(1) is the implied regressor for the intercept. Inspection of the columns reveals that there are three linear dependencies:
If substitutions are made for the three terms on the left in model 3.23, then an equivalent full-rank mixture model may be derived as follows:
In the same fashion, model 3.24 may be transformed into Eq. 3.25 as well. Let us now generalize the situation. The general form of a q -factor quadratic regression rr\r\f\f*\
ic
To reparameterize these models to full-rank mixture models, we make the following substitutions. For the intercept we write
which has the effect of converting the intercept into a sum of first-order terms. Second, wherever there is a squared term, we write
which has the effect of converting all squared terms into a single linear term plus a string of crossproduct terms. Collecting the linear and crossproduct terms leads to the general form
3.3. Quadratic Models
25
of the quadratic Scheffe polynomial.
Although the terms ^//X/X/ look like interaction terms, they are referred to as quadratic blending terms in the mixture-experiment literature, and the coefficients, /?/,-, are referred to as quadratic or nonlinear blending coefficients. When ft,-j > 0 and a high response value is desirable, we say that the blending between components / and j is synergistic; otherwise we say that it is antagonistic. When fi/j < 0 and a low response value is desirable, we say that the blending between components / and j is synergistic, and otherwise we say that it is antagonistic. The number of terms in a quadratic Scheffe polynomial can be calculated as follows:
Thus the minimum number of design points needed to support a quadratic Scheffe polynomial
is q(q + l)/2. Because quadratic models are so common in a mixture setting, it is worth committing this formula to memory. We now state the following: A second-order polynomial in q factors has (q + 1) (q + 2)12 terms, of which q + 1 must be deleted to obtain a full-rank mixture model. A sufficient but not a necessary condition is to delete the constant term and the q pure quadratic terms. Deletion of other terms can lead to intercept forms of the quadratic model, and these are now discussed.
3.3.1
Intercept Forms
Referring again to the two-factor regression function Rq. 3.22, page 23, this function has [(2 + I )(2 + 2)]/2 = 6 terms, of which 2 + 1 = 3 must be deleted to obtain a full-rank mixture model. If we retain the intercept, then we may retain two more regressors from the remaining five. We cannot choose X\ and X2 together, as this will lead to an exact dependency and will not model the quadratic curvature, and so we have a choice of nine possible models, all of which will be full rank. As q gets larger and larger, the number of possible intercept mixture models increases very rapidly. However, there are a limited number of useful intercept models, two of which we shall focus on here. The first is what is known as a slack-variable model. For the particular case where q = 2, the quadratic slack-variable model function takes the form
26
Chapter 3. Models for a Mixture Setting
When i = 1 we say that X2 is slack, and when / = 2 we say that X\ is slack. The slack-variable forms of model 3.25 are
The first equation (X2 slack) can be derived starting from the full-rank Scheffe model 3.25 as follows:
The model for X\ slack is derived in a similar manner. For q > 2 there will be additional terms in the model function of the form
with terms in the qth (slack) component absent. Thus the general form of a quadratic slack-variable model function is
The number of terms contributed by each type of term is
which, when added together, lead to a total of q(q -\- l)/2 terms, the same as the quadratic Scheffe model function. The slack-variable model is occasionally used in situations where one of the mixture components is present in very large amount while the other q — 1 components are present in much smaller amounts. Varying the proportions of the components present in small amounts will have little effect on the proportion of the component present in large amount, and so the latter is viewed as taking up the slack. The second quadratic intercept model function that occasionally is useful is the following:
Inspection of this function reveals that it differs from the quadratic Scheffe function Eq. 3.28, page 25, in that one of the linear terms in Eq. 3.28 has been replaced by an intercept. This
3.4. Cubic and Quartic Scheffe Models
27
procedure whereby one of the linear terms is replaced with an intercept can be applied to Scheffe polynomials of any order (linear, quadratic, or higher order). Casting model Eq. 3.25, page 24, in this form leads to the equations
The first equation is derived as follows:
In these intercept forms the meaning of the linear coefficients differs from their meaning in the Scheffe model form, but the higher-order terms retain their meaning. These models are useful when regression software does not output correct regression statistics for Scheffe models. The reasons for this are discussed in Chapter 8.
3.4
Cubic and Quartic Scheffe Models
Using an approach similar to that described for the linear and quadratic Scheffe models, one can derive higher-order Scheffe models. The terms in a cubic Scheffe model can be exemnlihed hv the model for a = 3
Just as the quadratic Scheffe model can be viewed as an augmented linear model, the cubic model can be viewed as an augmented quadratic model. The cubic terms are of two types. The Xi,X j ( X i — X j) terms model cubic blending of binaries. The coefficients of these terms are symbolized by yy/s to distinguish them from the coefficients of the quadratic terms, the jS/ys. The XjXjXk terms model cubic blending of ternaries. Although when q = 3 there are many more terms of the type X/X/(X/ — X,-) than of the type X , X j X k , beyond q = 5 the reverse is true. For example, when q — 8 there are 56 terms of the type X / X j X k but "only" 28 terms of the type X / X / C X , - X / ) . In presenting the general forms for cubic and quartic Scheffe polynomial, the following "shorthand" notation will be used:
28
Chapter 3. Models for a Mixture Setting
The general form for a cubic Scheffe polynomial is then
while the general form of the quartic Scheffe polynomial is
Terms of the form X\ XjX^, X,-X2jXk, and X/X/X^ can be included only when q > 3, while terms of the form XjXjX^Xi can be included only when q > 4. Recall that the number of terms in a quadratic Scheffe polynomial is q(q + 1 )/2. This can be expressed as
It can be shown that the number of terms in a cubic Scheffe polynomial is
and that the number of terms in a quartic Scheffe polynomial is
Clearly, a pattern is evolving. This pattern can be summarized by the following combinatorial expression:
Equation 3.33 can be used to calculate the number of terms in a Scheffe polynomial of any order m and for any q.
3.4. Cubic and Quartic Scheffe Models
3.4.1
29
Special Forms
A motivation for truncated forms of the cubic and quartic polynomial functions arises because, as q increases even moderately, the number of terms that must be supported by a design increases dramatically. This can be seen in Table 3.1.
Table 3.1. Number of terms in some Scheffe polynomials
q
Linear
Quadratic
Cubic
Quartic
2 3 4 5 6 7 8
2 3 4 5 6 7 8
3 6 10 15 21 28 36
4 10 20 35 56 84 120
5 15 35 70 126 210 330
Truncated forms of the cubic and quarlic models are called special cubic and special quartic models. Being truncated, these polynomials will not model as much complex curvature in a response surface as the full model forms. The general form of the special cubic polynomial is
and that for the special quartic polynomial is
For the specific case where q — 3 these polynomials take the forms
Chapter 3. Models for a Mixture Setting
30
X3
Figure 3.4. Curvature modeled by XiX2X3 (left) and X\X2X^ (right). The cubic term in the special cubic polynomial and the three quartic terms in the special quartic polynomial model are useful for modeling curvature of a response surface in the interior of the triangle. As illustrated in Fig. 3.4, a term such as XjXjX^ models peaks or valleys that are symmetrically located with respect to the centroid of the XjXjXk simplex. Terms such as X2XjXk also model peaks and valleys, but these are offset from the centroid along the Xi component axis (cf. page 53). The number of terms in some special cubic and special quartic polynomials are summarized in Table 3.2. The number of terms in special cubic models is q(q2 + 5)/6, while the number of terms in special quartic models is q(q2 — 2q + 3)/2. Although the special models contain fewer terms than the full models, we are still confronted with a large number of terms for moderately large q. Table 3.2. Number of terms in some special Scheffe polynomials
q
Special Cubic
Speial Quartic
3 4 5 6 7 8
7 14 25 41 63 92
9 22 45 81 133 204
2
For further discussions of Scheffe models and examples, see Scheffe [146], Gorman and Hinman [59], and several papers by Lambrakis [87, 88, 89]. Lambrakis [88] presents an equation for the general form of a Scheffe model of any order and for any number of mixture components.
3.5. Choosing a Model
31
3.5 Choosing a Model Assume for the moment that we begin an investigation without the complications of including process variables or mixture amount, and that we desire a design that will adequately support a Scheffe model. But where does one begin? Linear, quadratic, higher order? At the beginning of an investigation we have no knowledge of the functional relationship between the response and the mixture variables — or even that one exists. Perhaps, for example, the best "model" may turn out to be simply the average response. On the other hand, if an investigator is entering a research area in which scientists have built up experience about which model is most apt to describe the data, then he or she may well benefit from subject-matter knowledge. There is no substitute for knowing how similar experiments have turned out. When there is no preexisting information, what is often done is to hypothesize a linear polynomial model, sometimes called a screening model. There are at least two reasons for this. First, experimentation is usually expensive. Supporting a first-order model requires fewer observations than supporting a second- or higher-order model. With a properly designed experiment, a formal statistical test may be performed to check the adequacy of the linear model. If the fitted model is not adequate, then we need to consider augmenting the model and possibly the design. A second reason for beginning with a linear model is that such a model can help us determine if there are components that have no effect on the response or if there are components that have the same effect on the response. Components that have no effect can be disregarded in the analysis (after renormalization of the proportions of the components that do have an effect), leading to a more parsimonious model (and perhaps a more parsimonious formulation). Furthermore, in future experiments the proportion of a component with no effect could be held constant, thus reducing the number of variables. Components that have the same effect can be combined in the analysis, again leading to a more parsimonious model. Second-degree and higher-order models are commonly called response-surface models. Optimization is usually carried out using response-surface models. One might hypothesize a second-order model at the beginning of an investigation because of subject-matter knowledge or perhaps because experimentation is inexpensive. A higher-degree model can always be reduced to a lower-degree model, provided certain statistical criteria are met. Once a decision has been made about the order of the model, the next step is to develop a suitable design to support the model. Chapters 4 and 5 discuss designing in a mixture setting, while Chapter 6 discusses design evaluation — an exercise worth engaging in before starting what might be an expensive experimental program.
This page intentionally left blank
Part II
Design
This page intentionally left blank
Chapter 4
Designs for Simplex-Shaped Regions
This chapter and the next divide the subject of design into two parts, depending on the shape of the design region. Before discussing designs, then, we need first to consider those conditions that determine the possible shapes that design regions may assume in a mixture setting.
4.1 Constraints and Subspaces The shape of a design region in a mixture setting is determined by the constraints that are imposed on the component proportions. Mixture constraints can be divided into two broad categories: single-component and multicomponent constraints. Single-component constraints are of the form where Li, and Ui are lower and upper bounds, respectively, on the proportion Xi of component i. Equation 4.1 has two single-component constraints, a lower- and an upper-bound constraint. Multicomponent constraints are of the form
where Lk and Uk are lower and upper bounds, respectively, for the Kth two-sided constraint, and where some of the a^ may be zero. Ratio constraints are also common in formulation work. Consider, for example, the rntio rnnstmint
With a little algebra this can be rewritten as the multicomponent constraint:
In terms of Hq. 4.2, Lk = 0, ak\ = ak2 — \, and ak->, = ak4 — ak5 — — 1. 35
36
Chapter 4. Designs for Simplex-Shaped Regions
The set of single-component constraints
might be considered the trivial case. These constraints, in combination with the summation constraint Eq. 2.1, page 9, lead to q-simplexes. Consider the following modified constraint on X\:
Here we have a nonzero lower bound on component X\ only. The cases for q — 3 and q — 4 are illustrated in Fig. 4.1.
Figure 4.1. A single lower bound in a 3- and a 4-simplex. In the 3-simplex the shaded constrained region lies above the line at X\ =0.1; in the 4simplex, it lies above the triangle (3-simplex) at X\ =0.1. In both cases the constrained region is still simplex-shaped. Let us now add a second nonzero lower bound.
Fig. 4.2 illustrates the result. Again, the shaded constrained regions are still shaped like a simplex. Finally, let us add a third nonzero lower bound.
4.1. Constraints and Subspaces
37
Figure 4.2. Two lower bounds in a 3- and a 4-simplex.
Figure 4.3. Three lower bounds in a 3-simplex.
Fig. 4.3 illustrates the result tort/ = 3. (The diagram for q = 4isrnessy and is not included.) Again the constrained region is simplex-shaped. From these examples, the following property is interred. Property 4.1 When there are only lower bounds on component proportions, the suhregion of interest is always shaped like a simplex. As a consequence, the simplex designs discussed in this chapter apply not only to the full simplex but also to mixtures that have only lower-bound constraints. Let us take a closer look at the hound in Eq. 4.3 and Fig. 4.1. If the minimum proportion of component Xt is 0.1, then the maximum proportion of any other components cannot exceed 0.9. Any proportion X,, / = 2, 3 , . . . , q, greater than 0.9 is unattainable. When q = 3 (for example), upper bounds of 1.0 on X2 and X$ are said to be inconsistent [119,123].
38
Chapter 4. Designs for Simplex-Shaped Regions
When Eq. 4.3 applies and q = 3, the complete set of constraints would be written
The adjusted bounds on Xj and X^ are referred to as implied constraints. The set of lower bounds in Eqs. 4.4 leads to q implied constraints no matter what the value of q. U\ must be adjusted to 0.8 to account for L^ = 0.2, Ui must be adjusted to 0.9 to account for LI = 0.1, and U,•, i = 3, . . . , q, must be adjusted to 0.7 to account for Lj + L2 = 0.3. Cf. Fig. 4.2. When Eqs. 4.5 apply and q = 3, the complete set of constraints would be written
t/i is adjusted to 0.5 to account for L-L + LT, — 0.5, £/> is adjusted to 0.6 to account for . LI + L3 = 0.4, and U3 is adjusted to 0.7 to account for Lj + L2 - 0.3. Eqs. 4.5 apply and q > 3, then U, = 0.4 for z = 4 , . . . , q because L\ + L2 + L3 = 0.6. These considerations lead us to Property 4.2 [119]. Property 4.2
Inconsistent constraints occur whenever
or whenever
Property 4.2 implies that U-, and Li, are expressed in terms of component proportions (the reals; p. 9). If Uit and Li, are expressed in terms of actuals (such as grams, ounces, or pounds), then Eq. 4.6 should be reexpressed as
and Eq. 4.7 as
Note that in the case of Eqs. 4.6 and 4.8, consistency can be achieved by lowering either U; or the Lj, j = /, and that in the case of Eqs. 4.7 and 4.9, consistency can be achieved by
4.1. Constraints and Subspaces
39
raising Li, or theUj j = i. Software characteristically lowers U-, in the former case and raises L, in the latter case. When inconsistent constraints are entered in PC computing packages, range adjustments are automatically made so that the greater-than symbol in Eqs. 4.6 and 4.8 and/or the less-than symbol in Eqs. 4.7 and 4.9 become equal signs. Some software packages will print a message when these types of range adjustments are made. Design-Expert, for example, prints the message
whereas MINITAB prints the message
Inconsistent constraints are not always obvious and can lull the unwary into believing that he or she is exploring a broader range than is actually the case. Here is an example from the food industry. Soo, Sander, and Kess 1161] studied the effects of composition and processing on the textural quality of cooked shrimp patties. The mixture components were shrimp, isolated soy protein (ISP), sodium chloride, and sodium tritolylphosphate (STP). The following ranges were specified by the authors:
The effective range of one component is ~ 50% smaller than its stated range. Identifying the inconsistent constraint(s) will elevate one's appreciation for the fact that software packages usually take care of this automatically. What shapes might we expect when there are only upper bounds? Let us begin as before with a simple case where there is only one upper bound that is less than 1.0 and a = 3:
These constraints are consistent because neither Eq. 4.6 nor 4.7 is violated. The situation is illustrated in Fig. 4.4 (left). This figure is the same as Fig. 4.1 (left), but now the constraint is interpreted as an upper-bound rather than a lower-bound constraint. As a result, the shaded constrained region lies below the line at X\ = 0.1 and is shaped like a trapezoid. Within the constrained region, X2 and X3 are free to range between 0.0 and 1.0. Adding a second upper constraint that is less than 1.0, such as
40
Chapter 4. Designs for Simplex-Shaped Regions
Figure 4.4. One and two upper bounds in a 3-simplex.
leads to a subspace that is now a parallelogram (shaded corner of the triangle in Fig. 4.4 (right)). The lower bound on X3 has been adjusted to maintain consistency, and it is easy to see from the figure that X3, is no longer free to range to 0.0. Clearly, we cannot state a simple property about the shape of the design region when there are upper bounds. The shape will depend on the nature of the particular bound(s). Before adding a third upper constraint and continuing with this example, it is useful to digress for a moment. Assume that there are the same two upper-bound constraints on X\ and X2, but now q — 4. In this case it may come as a bit of a surprise that the following constraints are consistent because neither Eqs. 4.6 nor 4.7 is violated:
This result is general no matter what the upper bounds on X\ and X2 and is true as well if we replace X 4 with X / , i = 4, 5, . . . , q. Continuing with the three-component example, let us add a third upper bound that is less than 1.0:
where the lower bound on X3 has been reset to 0.0. The picture now looks like Fig. 4.5. In the center triangle, all three constraints are violated. In each of the three trapezoidal regions, two constraints are violated. And in each of the three parallelograms, one of the three constraints is violated. There is no shading because nowhere in the large triangle are all three constraints satisfied simultaneously. Thus we have an empty constrained region and the constraints are again said to be inconsistent [119, 123]. This leads to Property 4.3.
4.1. Constraints and Subspaces
41
Figure 4.5. Three upper bounds in a 3-simplex. Property 4.3
Upper bounds are inconsistent whenever
and lower bounds are inconsistent whenever
If, lor example, the bounds
were specified for a q = 4 design, then the equality part of Eq. 4.10 would apply. The constrained region would not be a region at all but rather the single mixture ( X \ , X 2 , X 3 , X 4 ) =(0.1,0.2,0.3,0.4). Had the U/ in Eq. 4.10 or the L\ in Rq. 4 . 1 1 been expressed in terms of actuals, then Eq. 4.10 would become
and Eq. 4.11 would become
Software packages handle the inconsistencies exemplified by Property 4.3 in different ways. .JMP outputs an empty data table but no message. Design-Expert and MINITAB print
42
Chapter 4. Designs for Simplex-Shaped Regions
error messages only. For example, if Eq. 4.10 or 4.12 applies, Design-Expert prints the message
whereas if Eq. 4.11 or 4.13 applies, then the word "maximums" is replaced by the word "minimums" and "less than" by "greater than". Again, if Eq. 4.10 or Eq. 4.12 applies, MINITAB prints the message "The total for high values ([total printed]) is less or equal to the total for the mixture ([total printed]). Increase the high values or decrease the mixture total"
and if Eq. 4.11 or 4.13 applies, then the message is "The total for the mixture ([total printed]) is less than or equal to the sum of the lower bounds ([sum printed]). Increase the mixture total"
The implications of these error messages are that (a) the burden of adjustment is on the user, and (b) adjustments may be made to either the relevant bounds or the total so that the < sign in Eq. 4.10 becomes a > sign, or the > sign in Eq. 4.11 becomes a < sign. If the user chooses to adjust the total, then the total no longer is equal to 1.0, and one is automatically designing in the actuals rather than the reals. To illustrate, assume the following constraints are specified in MINITAB:
Because the sum of the lower bounds (1.2) exceeds 1.0 and Eq. 4.11 is violated, an error message is printed. "The total for the mixture (1.000000) is less than or equal to the sum of the lower bounds (1.200000). Increase the mixture total"
One way to correct the situation is to increase the default total (1.0) to something that is greater than 1.2, such as 1.25, in which case one is specifying the constraints in terms of the actuals. MINITAB will output a design and print the following tables in the session window:
Comp A B C
Amount Lower Upper 0.30000 0.35000 0.40000 0.45000 0.50000 0.55000
Proportion Lower Upper 0.24000 0.28000 0.32000 0.36000 0.40000 0.44000
Pseudocomponent Lower Upper 0.00000 1.00000 0.00000 1.00000 0.00000 1. 00000
The upper bounds in the Amount columns are adjusted to be consistent with a total amount of 1.25. For example, the upper bound on X\ (0.35), when added to the lower bounds for X2 and XT,, is equal to 1.25. The numbers in the Proportion columns are simply normalized
4.1. Constraints and Subspaces
43
values of the numbers in the Amount table. In this particular example, the numbers in the Amount column have been divided by 1.25 to give the numbers in the Proportion columns. In terms of proportions, the upper bound on Xi (0.28), when added to the lower bounds for X2 and X3, is equal to 1.00. Pseudocomponents are explained in the last section of this chapter. Under what circumstances might upper bounds lead to simplex-shaped design regions? Fig. 4.6 shows an example for q = 3 and the set of constraints
Figure 4.6. Three upper bounds in a 3-simplex. The inverted subsimplex has been called a U-simplex, in contrast to the constrained region in Fig. 4.3, page 37, which is sometimes called an L-simplex. If any one of the three upper bounds were increased beyond 0.5, the constrained region would no longer be simplexshaped. This leads to Property 4.4 [36]. Property 4.4 If there are only upper-bound constraints, then a .U-simplex will lie within the full simplex whenever
As additional examples, consider the three sets of constraints:
The set on the left does not lead to a simplex-shaped subregion because there are not only lower-bound or only upper-bound constraints. The set in the middle has only upper-bound
44
Chapter 4. Designs for Simplex-Shaped Regions
constraints, but Eq. 4.14 is violated. The set on the right has only upper-bound constraints and does not violate Eq. 4.14, and therefore will lead to a simplex-shaped subregion. Software will adjust the last set so that the bounds are consistent:
The small inverted triangle in Fig. 4.7 shows the constrained region defined by the constraints in Eqs. 4.15. The sides of this triangle are delineated by the upper-bound constraints, while the vertices are defined by the lower-bound constraints. The compositions of vertices 1, 2, and 3 are (X1, X2, X3) = (0.1, 0.4, 0.5), (0.3, 0.2, 0.5), and (0.3, 0.4, 0.3), respectively.
Figure 4.7. Constrained region defined by Eqs. 4.15. Figure 4.8 shows an example of a q = 4 U-simplex. In this case the upper bounds have been set to
The vertices numbered 1–4 are located at the centroids of each of the four triangular constraint planes and have 0% of components X\, X2, X^, and X4, respectively. The design space is therefore an inverted tetrahedron. Although sets of constraints that satisfy Property 4.4 lead to simplex-shaped design regions and consequently designs in this chapter apply, some computing packages do not recognize this and defer to the procedures discussed in the next chapter. Exceptions include Design-Expert Version 7 and MIXSOFT. More will be said about this in Section 4.5. Piepel [119] also discusses checking constraints on linear combinations of components, such as Eq. 4.2. The procedures are much more complex and are beyond the intended scope of this text. The interested reader is referred to the discussion by Piepel.
4.2. Some Design Considerations
45
Figure 4.8. Constrained region defined by Ui < 1/3, / — 1,2, 3, 4.
4.2 Some Design Considerations Box and Draper [ 1 1 , 12] list several properties of a good experimental design, many of which have been discussed by other authors (see, for example, Atkinson and Donev [31, Myers [106], and Myers and Montgomery [107]). These properties can be grouped according to the design or analysis stage where they are implemented, checked, or modified. Some can be implemented at the design stage — before any data are collected — but others cannot be checked and possibly adjusted until after data are collected and an analysis is performed. Consequently the list below has been divided into two broad groups, depending on the stage (design or analysis) at which the property is checked. Furthermore, the design stage has been divided into two subgroups, reflecting the order of implementation in most statistical software packages. 1. Design stage. (a) Initial goals: i. Generate a satisfactory distribution of information throughout the region of interest. ii. Provide sufficient design points to allow a test for model lack of fit. iii. Provide replicate design points to allow an estimate of pure experimental error. iv. Allow experiments to be performed in blocks, v. Allow designs to be built up sequentially, vi. Be cost-effective — we do not have an infinite amount of time or money. (b) Design evaluation stage: i. Check for the presence of high influence points, ii. Is the design robust to the presence of outliers in the data? iii. Is there a good distribution (however that may be defined) of prediction variances?
46
Chapter 4. Designs for Simplex-Shaped Regions
2. Analysis checks and modifications: (a) Homogeneous variance assumption. (b) Normally distributed residuals. (c) Outliers. (d) Transformation of the response, when necessary. (e) Augmentation of the design and model, with blocking, when necessary. The aims under l(a) and l(b) are the subject of this chapter and Chapters 5-7; those under 2 are covered in Chapter 9. To clarify points i-iii under 1 (a), assume that one wants to design an experiment to support the linear Scheffe model (4.16)
Y = PiXi+02X2
which contains only two unknown parameters. Table 4.1 gives several designs that one might consider. The minimum number of design points required to support a polynomial model is equal to the number of unknown parameters in the model. Thus to support model 4.16 requires only two design points, and design a is adequate. The idea of fitting a two-term model to two design points is not ideal, and it would be much better to include additional points, such as a 50:50 blend (design b) or perhaps a 50:50 blend plus two additional design points midway between the 50:50 blend and the vertices (design c). Table 4.1. Some designs to support the model Y = B1 Y\ + B2 X2
Design a b c d e f ;
Niumber of de;sign pts. ofcompositic)n' 0.75,0.25 0.5,0.5 0.25,0.75 0,1 1,0 1 1 1 1 1 1 1 1 1 1 2 1 2 2 2 2 1 2 1 2 2
Degrees o f freedoim for Residuals LOF PE 0 0 0 1 1 0 0 3 3 1 2 3 3 4 1 3 3 6
Compositions are expressed as X\, X2
For every data point that we collect, we "earn" a degree of freedom. For every parameter that we estimate, we "spend" a degree of freedom. If we earn more degrees of freedom than we spend estimating the model parameters, then we have degrees of freedom left over, which we call residual degrees of freedom. Thus in designs a-f we earn 2, 3, 5, 5, 6, and 8 degrees of freedom. In each case, however, we spend only two of these estimating the parameters, fi\ and fa, in model 4.16. As a result we have residual degrees of freedom
4.3. Three Designs
47
as given in column 7 of Table 4.1. Degrees of freedom play an important role when fitting models to data and will be discussed in detail in Chapters 8 and 9. Designs c and d both have three residual degrees of freedom. However, there is clearly a difference between these two designs. Design c has five discrete design points, whereas design d has only three. As a result we can subclassify the residual degrees of freedom into two subcategories, those for lack of fit (LOF) and those for pure experimental error (PE). The residual degrees of freedom in design c are all lack-of-fit degrees of freedom because we have five discrete design points, and 5 — 2 = 3. In design d we have one lack-of-fit degree of freedom because we have three discrete design points, and 3 — 2 — I; the remaining residual degrees of freedom would be classified as pure-error degrees of freedom, because they arise from replication. To obtain an estimate of pure error, replicates must be included in a design. Columns 8 and 9 of Table 4.1 give the breakdown of the residual degrees of freedom into lack of fit and pure error for the six designs. Having degrees of freedom for lack of lit and pure error is desirable. This is because it allows for a formal statistical test, called a lack-of-fit test, to be carried out that provides information about the adequacy of the model. If lack of fit is statistically significant, then one needs to consider a higher-order model. Details of this test are reserved for discussion in the chapters on analysis. 1
4.3
Three Designs
In this section, three designs for simplex-shaped design regions are presented. The first two, the simplex lattice and simplex centroid designs, were both introduced by Scheffe in the 1950s [146] and 1960s [147]. Both designs are available in Design-Expert, JMP, MINITAB, and MIXSOFT. The third design, the simplex screening design, was introduced about 20 years later by Snee and Marquardt [ 159]. Screening designs are offered by Design-Expert and JMP. Despite differences in software packages, these designs may be created in most packages with a minimum of fuss.
4.3.1
Simplex Lattice Designs
A simplex lattice design always has a descriptor of the form {q, m}. The q within the curly braces has the usual meaning, i.e., the number of mixture components. The m within the curly braces describes the order of the model that is supported by the design. Thus a {4,2} simplex lattice design will support a q = 4 second-order Scheffe mixture model. The treatment combinations for a { q , m ] simplex lattice design consist of all mixtures whose proportions are members of the set
' As a rule of thumb, for a mixture setting Design-Expert 11631 recommends LOF and PE degrees of freedom each equal to the number of components plus one (q + !) up to a maximum of five each.
48
Chapter 4. Designs for Simplex-Shaped Regions
To illustrate, the {4,2} simplex lattice design would consist of the following design points: *i 2/2 0/2 0/2 0/2 1/2 1/2 1/2 0/2 0/2 0/2
X2
X3
X4
0/2 2/2 0/2 0/2 1/2 0/2 0/2 1/2 1/2 0/2
0/2 0/2 2/2 0/2 0/2 1/2 0/2 1/2 0/2 1/2
0/2 0/2 0/2 2/2 0/2 0/2 1/2 0/2 1/2 1/2
X2 0
*3
1 0 0 1/2 0 0 1/2 1/2 0
0 1 0 0 1/2 0 1/2 0 1/2
or equivalently Xi
1 0 0 0 1/2 1/2 1/2 0 0 0
0
X, 0
0 0
1
0 0 1/2 0 1/2 1/2
There are two categories of points in this design — the vertices and the midpoints of the edges (edge centroids). The design is displayed on the left in Fig. 4.9.
Figure 4.9. {4, 2} and {3, 3} simplex lattice designs.
4.3. Three Designs
49
As a second example, the design points for the {3,3} simplex lattice design are tabulated as follows:
x, 1
X2 0
0 0 1/3 2/3 1/3 2/3 0 0 1/3
0 2/3 1/3 0 0 1/3 2/3 1/3
1
x> 0 0
1
0 0 2/3 1/3 2/3 1/3 1/3
This design is displayed on the right in Fig. 4.9. Both designs in Fig. 4.9 are 10-point designs. The number of design points in a {q, m] simplex lattice design is equal to the number of terms in ag-component Scheffe model of degree m (cf. Eq. 3.33, page 28). Because of this, Scheffe models of degree m are sometimes referred to as \q, m} Scheffe models when they are associated with the corresponding {q,m} lattice designs [87, 88, 146J. If the design on the left in Fig. 4.9 were used to support the {4,2} Scheffe model, then the design is said to be saturated — there are no degrees of freedom for estimating lack of tit. The same can be said about the design on the right if it were used to support the {3,3} Scheffe model. Note that the {3,3} design has one complete mixture [29] — a formulation in which all of the components are present — whereas the {4,2} design does not. Whenever m < q, a simplex lattice design will consist of mixtures containing up to m components, and consequently there will be no complete mixtures; if m = q, there will be one complete mixture; and if m > q, there will be more than one complete mixture. In a {3,4} lattice design, for example, there will be three complete mixtures. Table 4.2 shows, for 3 < q < 6 and 2 < m < 4, the number of different types of design points in several {q, m} lattice designs. Underlined values are complete mixtures. Note that among the 12 designs in Table 4.2, there are few complete mixtures and that the design points tend to "pile up" on the lower-dimensional subsimplexes. Thus the distribution of information is weighted towards the boundaries of the simplex [26]. For this reason, plus the fact that the designs are saturated designs when used to support the corresponding {q, m} Scheffe models, the designs are often augmented with q +1 additional complete blends. One of these blends is always the overall centroid, while the remaining q are axial check blends — design points located halfway between the vertices and the overall centroid. Design-Expert and MINITAB provide this feature as an option. Fig. 4.10 displays the {3,2} lattice design (tilled circles) augmented by the center point and axial check blends (open circles). Augmenting { q , 2 } lattice designs in this manner leads to designs that are sometimes called simplex response-surface designs. Such designs support quadratic Scheffe models with q + 1 additional degrees of freedom for lack of lit. It is still necessary to replicate design points to obtain degrees of freedom for pure error. Software handles this in different ways. .IMP replicates the entire design. DesignExpert picks the highest leverage points to replicate, and as the vertices often are high
50
Chapter 4. Designs for Simplex-Shaped Regions
Table 4.2. Simplex lattice designs. Point types Blends
q
3
4 5 6
m 2 3 4 2 3 4 2 3 4 2 3 4
pure
3 3 3 4
4 4 5 5 5 6 6 6
2 3 6 9 6 12 18 10 20 30 15 30 45
3
4
1 3 4 12
1
10 30
5
20 60
15
Total blends 6 10 15 10 20 35 15 35 70 21 56 126
Figure 4.10. Augmented {3,2} simplex lattice design. leverage points, they are usually selected.2 MINITAB allows the user to choose points to be replicated. In all cases the resulting design can be further modified by the user.
4.3.2
Simplex Centroid Designs
For given q, there is only one simplex centroid design. The design consists of all mixtures located at the centroid of each simplex contained within a ^-component simplex — all the pure "blends" (1, 0, 0, . . . , 0), all the binary blends (1/2, 1/2, 0, . . . , 0), all the ternary blends (1/3, 1/3, 1/3, 0, . . . ) , . . . , plus the overall centroid (\/q, \/q, . . . , l/q). As the number of vertices can be viewed as q components taken one at a time, the number of 2
Leverage is discussed in Section 6.2.
4.3. Three Designs
51
binary blends as q components taken two at a time, and so on, the number of points in a full simolex centroid desien is then
The number of design points in a simplex centroid design increases rapidly with q. Assume, for example, that one desires to support a cj — 6 quadratic Scheffe polynomial, which has 21 terms. If one used a simplex centroid design to support this model, there would be 63 — 21 =42 degrees of freedom for lack of lit. One might think that it would be a better idea to use this design to support a q — 6 cubic Scheffe model, which has 56 terms. Unfortunately, this model has terms of the form X i , X j ( X i — X /), and these are equal to zero in a simplex centroid design because either X,-X/ or (X-, — X j) will always be equal to zero. Problems also arise with terms such as X/X^X/ — X/)2 and XjXjX^. For these reasons, the following special polynomials, which have the same number of terms as the simplex centroid designs, are used with these designs:
As pointed out by Piepel [123|, there may be instances where one may care to fit a truncated form of this model, in which case it would be desirable to generate a p-level fraction of the full design. Piepel proposed truncating the series 4.18 at the( q/p) term but always including the overall centroid. For example, if one wanted a fractional design plan to support terms in model 4.19 up through X , X / , then the number of points in the q = 6 simplex centroid design that one may choose to include could be
or perhaps
Neither design is ideal, as the 22-point design suffers from having only one degree of freedom for lack of fit, whereas the 42-point design has an excess of lack-of-fit degrees of freedom. However, with the addition of replicates and axial check blends to the 22-point design, one could perform a formal lack-of-fit test and if necessary augment the special polynomial model up through terms in X,X /X/^. This would, of course, require the addition of the (6/3) = 20 additional ternary blends, which are the two-dimensional centroids. A /;-level fraction of a ^-component simplex centroid design is called a {q, p] simplex centroid design. The 22- and 42-point designs above are {6,2} and {6,3} designs, respectively. One must be careful to distinguish the meaning of the second number in the curly braces, as it has a different meaning with the centroid designs than it does with the
52
Chapter 4. Designs for Simplex-Shaped Regions
lattice designs. A {q, 3} centroid design, for example, has edge centroids, whereas a {q, 3} lattice design has design points on the edges located at the 1/3,2/3 and 2/3,1/3 blends of the components that define the edges. Both designs include the vertices and the two-dimensional centroids. Fig. 4.11 shows pictures of the full (i.e., {3,3}) simplex centroid design and a 2-level fraction of the centroid design for four components. The overall centroid in the {4,2} design is indicated by a filled square. This design has 11 design points, equal to the sum of terms in the series
The full simplex centroid design for four components would have 24 — 1 = 15 runs. The additional four runs would be located at the centroids of the two-dimensional constraint planes (triangles).
Figure 4.11. {3, 3} and {4, 2} simplex centroid designs. Neither Design-Expert nor MINITAB has an option for specifying a p-level fraction of a simplex centroid design. One must generate the full design composed of 2^ — 1 formulations and then delete unwanted observations from the data table. IMP and MIXSOFT, on the other hand, provide an option to choose a p-level fraction of the design. Design-Expert and MINITAB have options for augmenting the designs with axial check blends, while JMP and MIXSOFT do not.
4.3.3
Simplex-Screening Designs
In the initial stages of an experimental program, and in the absence of subject-matter knowledge, simplex-screening designs deserve serious consideration by the formulator. As will be clear with the example to be presented, a single two-dimensional plot of the observed responses can provide an initial visual indication of the effects of the components on the response. This is before a model is fit to the data. For a given q there is only one screening design, and this consists of the set of points in Table 4.3. End points are the q blends with q — \ components present at \QQ/(q — 1)%.
53
4.3. Three Designs
Thus when q = 3, the end points are the three blends with two components present at 50%. See Fig. 4.12 (left) for a picture of this design. When q — 4, the end points are the four blends with three components present at 331/3% — the centroids of the four triangular constraint planes. Fig. 4.12 (right) shows four of the 13 design points — those that are on the X\ axis. The design point at the bottom of the tetrahedron, which is one of the four end points, is the centroid of the (X2, X3, X^ constraint plane, where X\ — 0. The end points are sometimes called constraint-plane centroids (154] even though when q > 4 the planes are really hyperplanes. Table 4.3. Simplex-screening designs. Point types Type Vertices Overall centroid Axial check blends End points Total
Number
