ALSO FROM COLD SPRING HARBOR LABORATORY PRESS An Illustrated Chinese-English Guide for Biomedical Scientists At the Bench: A Laboratory Navigator At the Helm: A Laboratory Navigator Lab Dynamics: Management Skills for Scientists
I EXPERIMENTAL DESIGN FO IOLOGI TS
Lab Math: A Handbook of Measurements, Calculations, and Other Quantitative Skills for Use at the Bench Lab Ref: A Handbook of Recipes, Reogents, and Other Reference Tools for Use at the Bench Lab Ref Volume 2: A Handbook of Recipes, Reagents, ond Other Reference Tools for Use at the Bench Laboratory Research Notebook Safety Sense: A Laboratory Guide
David J. Glass Novartis Institutes for Biomedical Research
COLD SPRING HARBOR Cold Spring Harbor, New York
BORATORY PRESS
Experimental Design for Biologists All righrs reserved @ 2007 by Cold Spring Harbor Laboratory Press, Cold Spring Harbor, New York Primed in rhe lJnired Stares of America
Publisher
John Inglis
Acquisition Editor
David Crotrv
Development Director
Jan /ugemine
Developmental Editors
Sian Curtis, Maria Smit, and Virginia Peschke
Project Coordinator
Inez Sialiano
Permissions Coordinator
Maria Fairchild
Production Editor
Rena Steuer
Desktop Editor
Compser Inc. and Susan Schaefer
Production Manager
Denise Weiss
Cover Designer
Lewis Agrell
For my nieces Molly jane and Madeline Rose who provide a reason to ensure that the future will yield great discoveries and to my teachers Charles Cantor, Lex van der Ploeg, Arg Efstratiadis, Steve Goff, and George Yancopoulos
Photograph of Karl Popper flLacinda Douglas-:v!cnzics/National Porrr:1ir Gallerv, London. Phnmo-ranl" of Francis Bacon and Rene Desortcs reproduced, wirh permission, from the Collections of the
Librarv of Congress Caraloging-in-Puhlication Data Glass, David J. Experimcnral design for biologists I David]. Glass. hihliographical references ~nd index. !SBN-13: 978-0-87969-735-8 (aile paper) l. Biology--Marhcm~ricai models. 2. Expcrimemol design. I. Tide, QH323.5.G565 2006 570.72'4--dc22 2006023665
10 9 8
::tbove address.
internal or persor.al use. or rhe inrernal of specific diems, is grantL:1horarory Press, provided thar the appropriate is paid direcdy to rhe Rosewood Drive, Danvers, rv1A 01923 (978-750-8400) for phcJroc·op\!fng items for cduorional classroom me, cnnract CCC ar the ar CCC Online at hrrp:i/www.cnpyrighr.com/.
Ali Cold Spring Harbor Loborarory Press public~rions may he direcrlv from Cold Spring Harbor Lahorarory Press, 500 5unrn·side Blvd., Wnndhnry, New York 11797-2924. Phone: 1-800-843-4388 in Conrinenral U.S. and Canada. All other loc1tiom: (516) 422-4100, FAX: (5!6) 422-4097. E-mail:
[email protected]. For a complete catalog of all Cold Spring Harbor I aboraron· Press puhlictrions, visit our \Vorld \Vide \Veb sire hrrp:/\vww.cshlpress.com/.
with gratitude and the hope that a little has rubbed off
Contents
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Defining the Experimental Program ................. 2
~
ix
........... .
The Hypothesis as a Framework for Scientific Projects: Is Critical Rationalism Critical? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
7
3
Scientific Settings in Which a Hypothesis Is Not Practical . . . . . . . . . . . . . . 1 7
4
The Problem/Question as a Framework for Scientific Projects: An invitation for Inductive Reasoning ............................. 21
5
What Constitutes an Acceptable Answer to an Experimental Question? ... 33
6
How Experimental Conclusions Are Used to Represent Reality: Model Building . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
7
Establishing a System for Experimentation . . . . . . . . . . . . . . . . . . . . . . . . . 51
8
Designing the Experiment: Definitions, Time Courses, and Experimental Repeats .......................................... 57
9
Validating a Model: The Ability to Predict the Future . . . . . . . . . . . . . . . . . 67
10
Designing the Experimental Project: A Biological Example ............. 75
11
Experimental Repetition: The Process of Acquiring Data to Model Future Outcomes ................................................. 103
12
The Requirement for the Negative Control ........................ 11 7
13
The Requirement for the Positive Control ......................... 1 33
14
Method and Reagent Controls ................................. 149
15
Subject Controls ............................................ 157
16
Assumption Controls ........................................ 1 71 vii
viii
Contents
17
Experimentalist Controls: Establishing a Claim to an Objective Perspective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
18
A Description of Biological Empiricism ........................... 191
19
A Short Synopsis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
Preface
Index ........................................................ 201
THIS BOOK IS A PRODUCT OF SHORT COURSE on Experimental Design that I developed while still working at Regeneron Pharmaceuticals and I would like to express my thanks to all who gave feedback on that course. Like many ideas, the decision to turn the course into a book came about after a little too much wine with dinner-a dinner with fellow scientists Dan Finley, Fred Goldberg, and Allan Weissman-at which we were discussing both the odd fact that experimental design was not commonly taught to prospective biologists in graduate school, and the obvious discontinuities between the demands of Critical Rationalism as written and the way that science was actually practiced. Of course, this book still would not have been produced were it not accepted by a publisher. Therefore, I am extremely grateful to David Crotty at Cold Spring Harbor Laboratory Press tor agreeing to take the project on behalf of the Press. Sian Curtis then edited the manuscript in an extremely able fashion, with the help of Ginger Peschke and Maria Smit. The project -was overseen by Jan ~A.rgentine and her colleagues at the Press. Thanks very much to Sian and Jan tor their steady feedback and enthusiasm. Thanks so much also to Rena Steuer for expert production guidance, to Susan Schaefer tor typesetting, and to Denise Weiss for her design expertise. A great deal of help in writing this was lent by Kumar Dharmarajan. Kumar is a former student and intern in my laboratory, who was a medical student at Columbia when much of the book was written. He was thus able to rake on the role of rhe "prospective audience" and gave invaluable feedback on each chapter, identifYing passages rhat were unclear and asking questions rhat helped in the rewriting. Thanks very much to him fur spending so much time on this project. Brian Clarke in my lab also read a large chunk of the manuscript and highlighted some sections that needed clarification. Woody Fu is a former student who now has many jobs, including artist and carmonist. He provided three cartoons for this book, and they came out so well that I am sorry we did not make greater use of his talents. Thank you very much to the fulks at .Novartis, where I now work, tor supporting this project. Novartis a great emphasis on continuing education, and it was a
ix
x
Preface
comfort to find that this project invoked enthusiasm in the company. Special thanks to statisticians Leah Martell and Mathis Thoma tor taking ~:he time to read a large portion of the text for potential statistical misstatements. However, having s~id that, if there are any remaining problems, they are my responsibility alone. It would be great to hear from readers, including feedback regarding problems with the text so that these could be corrected in fumre editions. e-mail address is expdesignbiology@ gmail.com/. Feel free to drop me a line. DAVID
J.
1 Defining the perimentai Program
GL'\SS
in the basics of the medical crart: how to perform a physical examination, interpret symptoms, approach a patient, and perform a differential diagnosis. At law school, the initial course load is designed to steep the novitiate in the forensic approach: how to write a brief~ follow legal procedures and prepare contracts, how lawyers are expected to conduct themselves in various situations, and how to think like a lawyer. Now here's an amazing fact: At this writing, in graduate schools throughout the United States, budding Ph.D. candidates receive almost no formal instruction in experimental design, despite rhe fact that the effective design, conduct, and interpretation of experiments is critical co their success as scientists. A quick survey of the curricula for Ph.D. students in biology, biochemistry, and related disciplines, in even the most established universities, reveals courses devoted almost entirely to factual information. There is no emphasis on rhe formal experimental processes required to unmask the previously unknown, and no discussion of how these processes derive from various-and sometimes dissenting-theories of epistemology.' Currently, the one notable exception to this lack of procedural education is training in statistics, although such training is sporadic and varies greatly depending on the scienrific discipline. However, aithough an understanding of statistics can be tremendously helpful in the development of dfective experimental design, both apprentices and more experienced scientists would benefit from a broader understanding of different mental frameworks and their implications. Bur isn't a formal procedural education dispensable tor a practicing scientist (who is, after all, more concerned with empirical data than with philosophical theory)? Is it not obvious what is required ro pertorm an experiment? Isn't rhe scientific method well established, developed over hundreds of years of philosophical reasoning, and distilled into consistent sciemific practice? The answer ro all of these questions is ''no.''
AT MEDICAL SCHOOL, STUDENTS ARE TRAINED
of the grounds of l
