Measurement, Analysis, and Control Using Jmp: Quality Techniques for Manufacturing

Measurement, Analysis, ® and Control Using JMP Quality Techniques for Manufacturing Jack E. Reece The correct biblio...

Author: Jack Reece

617 downloads 1132 Views 8MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

Measurement, Analysis, ® and Control Using JMP Quality Techniques for Manufacturing

Jack E. Reece

The correct bibliographic citation for this manual is as follows: Reece, Jack E. 2007. Measurement, Analysis, and Control Using JMP®: Quality Techniques for Manufacturing. Cary, NC: SAS Institute Inc. Measurement, Analysis, and Control Using JMP®: Quality Techniques for Manufacturing Copyright © 2007, SAS Institute Inc., Cary, NC, USA ISBN 978-1-59047-885-1 All rights reserved. Produced in the United States of America. For a hard-copy book: No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, or otherwise, without the prior written permission of the publisher, SAS Institute Inc. For a Web download or e-book: Your use of this publication shall be governed by the terms established by the vendor at the time you acquire this publication. U.S. Government Restricted Rights Notice: Use, duplication, or disclosure of this software and related documentation by the U.S. government is subject to the Agreement with SAS Institute and the restrictions set forth in FAR 52.227-19, Commercial Computer Software-Restricted Rights (June 1987). SAS Institute Inc., SAS Campus Drive, Cary, North Carolina 27513. 1st printing, July 2007 SAS® Publishing provides a complete selection of books and electronic products to help customers use SAS software to its fullest potential. For more information about our e-books, e-learning products, CDs, and hardcopy books, visit the SAS Publishing Web site at support.sas.com/pubs or call 1-800-727-3228. ®

SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. ® indicates USA registration. Other brand and product names are registered trademarks or trademarks of their respective companies.

Contents Foreword xi Acknowledgments xiii Introduction xv

Part 1

Characterizing the Measurement Process 1

Chapter 1 Basic Concepts of Measurement Capability

3

Introduction 4 Figure-of-Merit Statistics 5 The P/T Ratio 5 The SNR 6 RR Percent 6 Six Sigma Quality of Measurement 6 Capability Potential versus P/T and SNR 7 Precision versus Accuracy in a Measurement Tool or Process 11 To Calibrate or Not 16 Understanding Risk 16 JMP Sample Size Calculations and Power Curves 18 Uncertainty in Estimating Means and Standard Deviations 21 Confidence Interval for the Mean 22 Confidence Interval for the Standard Deviation 24 Components of Measurement Error 28 Repeatability Error 28 Reproducibility Error 29 Linearity of Measurement Tools 29 Random versus Fixed Effects 30

iv Contents

Chapter 2 Estimating Repeatability, Bias, and Linearity 17 Introduction 33 Evaluating a Turbidity Meter 33 The Data 33 Determining Bias for Stabcal < 0.1 36 The Paired Analysis 43 Measurement Tool Capability: P/T for Stabcal < 0.1 45 Determining Bias for Gelcal < 10 45 Determining P/T for Gelcal < 10 46 Lessons Learned 47 An Oxide Thickness Measurement Tool 47 Examining the Data 47 Excluding the Questionable Values 51 Generating a Multivari Chart 53 Lessons Learned 56 Repeatability of an FTIR Measurement Tool 56 Examining the First Test 57 Conducting a Second Test 59 Estimating Potential Bias 61 P/T and Capability Analysis 63 Repeatability and Linearity of a Resistance Measurement Tool 66 Reorganizing the Data 67 A Shortcut Method for Evaluating a Measurement Tool 75 Linearity of This Measurement Tool 76 Lessons Learned 77 Using Measurement Studies for Configuring a Measurement Tool 78 Examining the Data 78 Manipulating the Data: Combining the Tables 82 Evaluating the Combinations 85 Lessons Learned 90 No Calibration Standard Available 91 Examining the Data 92 Matched Pair Analysis 93

Contents

Lessons Learned 94 Exploring Alternative Analyses 94 Capability Analyses 95 Summarizing Data 96 Variance Component Analysis: Using Gage R&R 96 Using Regression Analysis 98 Lessons Learned 99 A Repeatability Study Including Operators and Replicates 99 Estimating a Figure of Merit: P/T 100 Variability Charts: Gage R&R 102 Fitting a Model 104 Lessons Learned 106 Summary of Repeatability Studies 108

Chapter 3 Estimating Reproducibility and Total Measurement Error 109 Introduction 111 Planning a Measurement Study 112 Stating the Objective 112 Identifying Potential Sources of Variation 112 Gathering the Standard Objects 114 Scheduling the Trials 114 Generating a Data Entry Form 115 Summary of Preparations for a Measurement Study 120 Analysis of Measurement Capability Studies: A First Example 120 Looking at the Data 121 Generating a Figure of Merit 123 Other Analyses 125 Lessons Learned and Summary of the First Example 127 A More Detailed Study 128 Rearranging and Examining the Data 129 Measurement Capability 131 Summary and Lessons Learned 135 Turbidity Meter Study 135 Examining the Data 136

v

vi Contents

Estimating Measurement Capability 138 Summary and Lessons Learned 139 A Thin Film Gauge Study 140 Adding a Variable 141 Reordering the Table and Examining the Data 142 Estimating Measurement Capability 143 Fitting Regression Models 147 Lessons Learned 149 A Resistivity Study 149 Examining the Data 150 Estimating a Figure of Merit 153 Fitting a Regression Model 153 Lessons Learned 154 A Final Example 155 Looking at the Data 156 Data Structure and Model Fitting 157 Comparing the Variability Due to OPERATOR 159 Summary and Lessons Learned 160 Summary of Measurement Capability Analyses 162

Part 2

Analyzing a Manufacturing Process 163

Chapter 4 Overview of the Analysis Process

165

Introduction 165 How Much Data? 166 Expected Results from Passive Data Collection 167 Performing a Passive Data Collection 167 Planning the Experiment 167 Collecting the Data 168 Analyzing the Data 168 Drawing Conclusions and Reporting the Results 169

Contents

vii

Chapter 5 Analysis and Interpretation of Passive Data Collections 171 Introduction 172 A Thermal Deposition Process 172 Looking at the Data: Initial Analysis of Supplier-Recommended Monitor Wafers 174 Analysis of the Team-Designed Sampling Plan 176 Reporting the Results 179 Lessons Learned 180 Identifying a Problem with a New Processing Tool 181 Looking at the Data: Estimating Sources of Variation 181 An Alternative Analysis 185 Lessons Learned 188 Deposition of Epitaxial Silicon 189 Determining a Sampling Plan 189 Analyzing the Passive Data Study 191 Lessons Learned 196 A Downstream Etch Process 196 Overview of the Investigation 196 Passive Data Collections 197 Lessons Learned 207 Chemical Mechanical Planarization 208 Polishing Oxide Films 208 Polishing Tungsten Films 217 Polishing a Second Type of Oxide Film 222 Summary of Passive Data Collections 230

viii Contents

Part 3

Developing Control Mechanisms 231

Chapter 6 Overview of Control Chart Methodology

233

Introduction 234 General Concepts and Basic Statistics of Control Charts 235 Types of Data 235 The Normal Distribution 236 How Many Samples? 236 Examination of Data 237 Types of Control Charts and Their Applications 237 Charts for Variables Data 237 Charts for Attributes Data 245 Special Charts for Variables Data 249 Trend Analysis 252 The Western Electric Rules 253 The Westgard Rules 254 Implementing Trend Rules 255 Capability Analysis 256 The Cp Statistic 256 The Cpk Statistic 258 The Cpm Statistic 260 Generating Capability Statistics in JMP 261 Control Charts Involving Non-Normal Data 261 Summary 261

Chapter 7 Control Chart Case Studies

263

Introduction 264 Measurement Tool Control 264 Some Scenarios for Measurement Tool Control Charts 266 Replicated Measurements at a Single Sample Site 267 Summary of the First Example 275 Measurements across an Object—No Replicates 275 Summary of the Second Example 279 A Measurement Study with Sample Degradation 280

Contents

Summary of Control Charting Issues for Measurement Tools 286 Scenarios for Manufacturing Process Control Charts 287 A Single Measurement on a Single Object per Run 289 An Application for XBar and S Charts? 294 A Process with Incorrect Specification Limits 302 Multiple Observations on More Than a Single Wafer in a Batch 305 Dealing with Non-Normal Data 316 The SEMATECH Approach 316 Monitoring D0 317 Cleaning Wafers (Delta Particles) 322 Defective Pixels 328 Summary of Control Chart Case Studies 332

References Index

337

335

ix

x Contents

Foreword The material presented here resulted from some 30 years of experience in a manufacturing environment. Although trained as an organic chemist, the author had the good fortune to associate with a number of skilled, practical-minded statisticians during his career, initially while a process engineer at Minnesota Mining and Manufacturing Company, St. Paul, MN, and later at SEMATECH (now International SEMATECH) in Austin, TX, while part of the Statistical Methods Group in that organization. At Minnesota Mining and Manufacturing Company, Don Marshal, a graduate of statistical training at the University of Wisconsin, Madison, introduced the author to practical applied statistics. During his career at SEMATECH, the author worked with academic statisticians including Dr. Peter John, University of Texas, Austin, and Dr. George Milliken, Kansas State University, Manhattan. These statisticians provided considerable insight and support for understanding and dealing with a variety of issues in semiconductor manufacturing, not the least of which was understanding and quantifying sources of variation in processes. Statistical colleagues who were temporarily assigned at SEMATECH from various semiconductor member companies also expanded the author’s exposure to practical statistical methods. The material that follows draws heavily on the author’s experience within SEMATECH and after retirement, when he worked as a private consultant on a variety of manufacturing operations. Although the material does rely heavily on examples from semiconductor manufacturing, the principles applied, the experiments conducted, and the analyses described also apply widely in all manufacturing environments. The author is particularly indebted to Dr. George Milliken for his support in critiquing this material, offering advice and counsel, and making certain that statistical principles were preserved in the analyses described, particularly in those areas that involved mixed models involving random and fixed effects. The capabilities of JMP 6.0 support a wide variety of analysis and graphics techniques critical to understanding and quantifying variation in the manufacturing environment.

Jack E. Reece, Ph.D. April 2006

xii

Postscript Jack E. Reece died in October of 2006 before he was able to finish editing his book on Measurement, Analysis, and Control in Manufacturing. As a tribute to Jack and his surviving wife, Janet, it was a great honor to do some final editing on his manuscript. It is important to me to recognize Jack’s great contributions to the JMP community and to those who monitor manufacturing processes. George A. Milliken Milliken Associates, Inc. Manhattan, KS 66502 December 2006

Acknowledgments SAS Press acknowledges the contributions of two individuals who ensured that the author’s vision was carried out in a way that would have met his approval. George Milliken offered to revise the final manuscript with the updates that he and the author discussed before the author’s death. George reviewed final galleys to ensure the accuracy and completeness of the publication. Annie Dudley Zangi, a JMP developer, spent countless hours as an in-house technical reviewer for the various drafts of the book. Because of the unique circumstances associated with this book’s publication, Annie took a lead role as our on-site technical contact, answering our questions and working in cooperation with George to achieve the desired results for this book. George and Annie, thank you from the SAS Press Team.

xiv

Introduction For a manufacturing activity to remain competitive, engineers must rigorously apply statistics methodologies that enable them to understand the sources and consequences of uncontrolled process variation. This book addresses the following major issues related to this activity:

Characterizing the measurement process Analyzing process performance Developing appropriate control mechanisms for monitoring measurement and performance

The Six Sigma DMAIC methodology provides a successful mechanism for implementing techniques that Define, Measure, Analyze, Improve, and Control manufacturing processes. An alternative methodology used at SEMATECH (now International SEMATECH of Austin, TX) is called the Qualification Plan. The Qualification Plan provides direction for implementing rigorous statistical methods to understand and to characterize processes and associated manufacturing tools in the semiconductor industry. Although this plan focuses on the manufacturing of semiconductor devices, the methodology is applicable to most manufacturing processes. Figure 1 shows how the SEMATECH Qualification Plan parallels the concepts of Six Sigma and extends beyond them. The four major divisions of this figure represent the Six Sigma concepts, as represented by the letters on the left. The flowchart shows the Qualification Plan activities and how they relate to those concepts.

xvi Introduction

Figure 1 The SEMATECH Qualification Plan and Its Relationship to Six Sigma

At the top of the figure, the first activity in the Qualification Plan corresponds to the Six Sigma Define step (D). Obviously, no project should proceed without clearly defined goals and some level of planning to ensure coordination among groups and availability of required materials, equipment, and personnel. The next critical step is characterization of measurement tools, which corresponds to the Six Sigma Measure step (M). Once

Introduction xvii

measurement tools have been qualified, the next step determines the performance of the particular process being investigated. This step corresponds to the Six Sigma Analyze and Control steps (A, C). If the data collected at this point suggests that the process is stable, then an engineer may generate appropriate initial control charts. If process performance is not acceptable, then the team enters Active Process Development and Process Optimization, which correspond to the Six Sigma Improve and Control steps (I, C). These steps involve identifying and manipulating factors controlling the process in order to improve that process, either by placing it on a particular target, minimizing its variation, or both. The expected outcome at this stage is a process that meets or exceeds specifications. Once the work has yielded the desired process, a team can implement suitable control mechanisms to monitor it. The far right section of the figure illustrates the IRONMAN process for improving the reliability of new equipment, developed largely at Motorola Corporation. IRONMAN is an acronym for “Improving the Reliability of New Machines at Night.” Its name suggests the methodology used to accomplish this goal without interfering with routine manufacturing operations. These reliability improvements are not covered in this book. The Marathon portion of Figure 1 is an activity in which processes operate undisturbed for an extended period of time (usually at least 30 batches of material). This activity can develop basic information for creating control mechanisms. In addition, it generates Cost of Ownership data related to new manufacturing tools. This data is critical to assisting companies in making intelligent acquisition decisions. This book is divided into three parts that are directly related to the concepts illustrated in Figure 1. “Part 1—Characterizing the Measurement Process” corresponds to Gauge Studies and Measurement (M). “Part 2—Analyzing a Manufacturing Process” corresponds to PDC or Capability Demo and Analysis and Control (A, C). “Part 3— Developing Control Mechanisms” corresponds to Control (C). Each part relates these activities to JMP platforms with appropriate demonstrations. Most of the examples rely on case studies, largely in the semiconductor manufacturing area. Data collection in that industry can be both sophisticated and potentially overwhelming due to volume. Therefore, the discussions spend some time explaining how to examine potentially large volumes of data and how to reduce that data to more manageable levels. Even though the examples focus heavily on semiconductors, the principles described and the methods used apply generally to the study of any process or manufacturing activity. A few examples come from other activities and a very few represent simulations. In these cases, the examples are identified as such. This book assumes that the reader has some knowledge of the JMP software system, but does not assume extensive statistical knowledge. Each time the material introduces a particular use of JMP capabilities, the example includes considerable “how to” information for carrying out the task. This book is not an exhaustive demonstration of JMP software capabilities, nor is it a tutorial for the general use of JMP. Its intention is to

xviii Introduction

demonstrate good methods for interpreting data making full use of graphics as well as formal analyses. Later sections require the use of JMP’s capability to handle mixed models—models involving both fixed and random effects. When appropriate, the book explains those concepts to enable understanding. Finally, this book’s companion Web site (http://support.sas.com/reece) includes the data tables used, some with extensive scripts illustrating the steps taken in analysis. These tables are not typically included with the sample data installed with JMP.

Historical Notes The Qualification Plan methodology originated over 20 years ago at Intel Corporation (a founding member of SEMATECH) as part of their “burn-in” program for qualifying new equipment for manufacturing. As a member of the SEMATECH Statistical Methods Group, the author guided and supported engineers in implementing this methodology. Since its birth in the mid 1980s, SEMATECH has considered its most important mission to be helping its member companies by generating information regarding semiconductor tool performance. Therefore, adapting Intel’s program to those objectives was logical and intuitive. SEMATECH’s member-supported programs have always had a firm basis in applied statistical methods to ensure that any information generated is unequivocal and trustworthy, because member companies base expansion and acquisition plans on it.

Suggested Reading For those readers not comfortable with their basic knowledge of the JMP software system, a useful and informative reference is JMP Start Statistics, Third Edition, by John Sall, Lee Creighton, and Ann Lehman. To find this book and others like it, go to the SAS Web site at support.sas.com. In addition, careful reading of the examples in the JMP User’s Guide furnished with the software will help the new user understand the capabilities and applications of JMP. For detailed information, consult specific topics in the JMP Statistics and Graphics Guide, also furnished with the software. The most recent revisions and expansions of JMP documentation are in the PDF files furnished with the installation disk. This book reflects the use of JMP 6.

1

P a r t

Characterizing the Measurement Process Chapter

1

Basic Concepts of Measurement Capability

Chapter

2

Estimating Repeatability, Bias, and Linearity

Chapter

3

Estimating Reproducibility and Total Measurement Error 109

3 31

2 Measurement, Analysis, and Control Using JMP: Quality Techniques for Manufacturing

C h a p t e r

1

Basic Concepts of Measurement Capability Introduction 4 Figure-of-Merit Statistics 5 The P/T Ratio 5 The SNR 6 RR Percent 6 Six Sigma Quality of Measurement 6 Capability Potential versus P/T and SNR 7 Precision versus Accuracy in a Measurement Tool or Process 11 To Calibrate or Not 16 Understanding Risk 16 JMP Sample Size Calculations and Power Curves 18 Uncertainty in Estimating Means and Standard Deviations 21 Confidence Interval for the Mean 22 Confidence Interval for the Standard Deviation 24


Components of Measurement Error 28 Repeatability Error 28 Reproducibility Error 29 Linearity of Measurement Tools 29 Random versus Fixed Effects 30

Introduction Standing between every manufacturing process and the observer is some form of measurement process. The observer never sees the actual manufacturing process clearly or separately—a measurement process or device always intervenes to provide the data necessary for interpreting process performance. Ideally the impact of the variation in the measurement device or process is negligible, but that might not be the case. A manufacturing process could be satisfying all Six Sigma requirements, but a measurement tool could be obscuring that truth. Equation 1.1 illustrates the relationship among perceived process variance, actual process variance, and the measurement tool contributions to the variance. 2

σˆ perceived

= σ actual process + σ measurement tool 2

process

2

1.1

Figure 1.1 provides a graphic illustration of this equation. Obviously, if the contribution due to variation in the measurement tool is extremely small, then the perceived process variation is approximately the same as the actual process variation. Therefore, establishing a figure-of-merit statistic that identifies sources of variation for a measurement tool or process helps the observer decide whether the contribution of measurement error materially affects any interpretation of the manufacturing process.

Chapter 1: Basic Concepts of Measurement Capability 5

Figure 1.1 Contribution of Measurement Error to Perceived Process Error

Figure-of-Merit Statistics The P/T Ratio The precision/tolerance ratio (P/T) is one convenient figure of merit used throughout this book for describing the capability of a measurement device. Equation 1.2 illustrates its calculation based on the inverse of the capability potential (Cp), where USL and LSL are the upper and lower specification limits, respectively, for the process in question—not the measurement process.

⎛ 6σ measurement ⎟⎞ ⎜⎝ USL − LSL ⎟⎟⎠

P / T = 100 ⎜⎜

1.2

If the P/T ratio for a measurement tool is ≤ 30 (see “Capability Potential versus P/T and SNR” in this chapter), then its contribution to the perceived process variation is negligible.


The SNR In many cases, the specification limits for a process are arbitrary assignments, subject to change. An alternative figure of merit for a measurement tool is the signal-to-noise ratio (SNR). Equation 1.3 illustrates its calculation.

SNR =

σˆ process σˆ measurement

1.3

Because the perceived variation for the process probably contains a contribution from measurement error, a more elaborate calculation of SNR is applicable (Equation 1.4).

SNR =

2 σˆ 2process − σˆ measurement

σˆ measurement

1.4

RR Percent When a measurement study involves only two factors, such as operator and part, JMP can prepare a report on the study that includes RR percent. This statistic compares the measurement variation to the total variation in the data and calculates a percent Gage R&R. Barrentine (1991) suggests guidelines for acceptable RR percent. This statistic is approximately 1/SNR (discussed in the previous section). See the JMP Statistics and Graphics Guide for more information. Such a simple metrology study occurs very seldom in the examples used here, so this book contains little additional discussion of this topic.

Six Sigma Quality of Measurement Another approach to assessing the capability of a measurement process uses the capability analysis option associated with the display of distributions in JMP. A pop-up menu option on the distribution report generates a capability analysis of data collected from a measurement process (assuming the user has removed from that data all unusual values that are due to an assignable cause). Part of that capability report includes the parts per million (ppm) of observations beyond the specification limits (in this case one uses the specification limits for the process being monitored, not the specification limits associated with the measurement tool). This approach is analogous to the calculation of the P/T ratio illustrated earlier. Also reported is Sigma Quality (provided the user has specified specification limits). Values of Sigma


Quality ≥ 6 indicate that the measurement tool is capable of handling the measurement task being examined. A section in Chapter 2 illustrates this approach.

Capability Potential versus P/T and SNR To refresh the reader’s memory, Equation 1.5 illustrates the calculation of capability potential (Cp).

Cp = where

USL − LSL 6σ

1.5

USL and LSL, respectively, are the upper and lower specification limits for a process, and σ is the observed process standard deviation.

This statistic, discussed further in Chapter 6, estimates how much of the output of a process fits between defined specification limits. A Cp value of 1 predicts that some 99.73% of observations will fit within the specifications (see Chapter 6 for calculations supporting this statement). The JMP data tables referenced in the discussions in this and in following sections are available on the companion Web site for this book at http://support.sas.com/reece. Sample data tables are arranged by chapters in the discussion. To open a table using the JMP Starter window, select the File category, and then click Open Data Table (left panel in Figure 1.2). Alternatively, select File¼Open in the menu bar at the top of the JMP window (right panel in Figure 1.2). Select the Chapter 1 directory, then select the data table you want and open it (Figure 1.3).

Figure 1.2 Opening a Data Table in JMP


Figure 1.3 Selecting a Data Table

Figure 1.4 contains the table Capability vs Measurement Error.jmp. Entries in this table are simulations of the contributions of measurement error to an observed theoretical capability potential (Cp). In the left panel of the display, the symbol to the left of the name of a column indicates that the contents of that column have numeric or continuous modeling properties. The symbol to the right of the last four column names indicates that each column has an associated formula to compute its contents. To see that formula, right-click that symbol to reveal a menu (left panel of Figure 1.5). On that menu, select Formula to reveal the embedded calculation (right panel of Figure 1.5).

Figure 1.4 Capability vs Measurement Error.jmp


Figure 1.5 Revealing a Column Formula

To explore how the observed capability potential (Cp) of a process varies with the measurement error encountered, generate overlay plots of observed capability (y) versus either the P/T ratio (x) or the SNR. Figure 1.6 illustrates setting up the graph for capability versus SNR using the menu bar at the top of the JMP window. Notice that the X axis has been converted to logarithmic scale during the setup.

Figure 1.6 Setting Up an Overlay Plot for Observed Capability versus SNR

Figure 1.7 displays the overlay plot for observed capability versus P/T ratio; Figure 1.8 displays the overlay plot for observed capability versus SNR. Each graph received considerable modification to improve the displays.


Figure 1.7 Observed Capability versus P/T Ratio

Figure 1.7 suggests that so long as P/T < 30, the measurement process does not materially affect the perception of process performance. Similarly, in Figure 1.8, as long as SNR is >~3, the same is true. Obviously, having a measurement of P/T 5 is highly desirable.

Figure 1.8 Observed Capability versus SNR


Precision versus Accuracy in a Measurement Tool or Process To minimize the confusion that might exist between the terms “accuracy” and “precision,” consider the following definitions. Precision, as applied to a measurement tool or process, is a measure of the total amount of variation in that tool or process. Intuitively, the more precise a measurement tool or process, the more desirable it becomes. Accuracy is the difference between a standard or true value and the average of several repeated measurements using a particular tool or process. A measurement tool or process can be precise and accurate, or it can have any combination of those factors, including neither of them. Figure 1.9 illustrates these concepts using a marksmanship model.

Figure 1.9 Precision versus Accuracy (Marksmanship Model)

The target labeled “Tool 1” represents a measurement process that is both precise and accurate. That labeled “Tool 2” represents a measurement process that is not particularly precise, but does give accurate results on average—on average “we got ‘em.” The process or tool suffers from large variation, but no bias.


The image for “Tool 3” shows considerable precision, but the aim is off; the tool has considerable bias in its measurements and is not accurate. “Tool 4” is a disaster. Not only are the results not accurate, but they lack precision as well. The JMP data table Measurement Tools.jmp, found in the directory Chapter 1 on the companion Web site for this book at http://support.sas.com/reece and illustrated in Figure 1.10, contains simulated data from the four measurement tools just described. This table makes extensive use of the formula capabilities in JMP to simulate data.

Figure 1.10 Excerpt of Measurement Tools.jmp

To create a table like this: 1. Generate a new, blank table in JMP. 2. To this table add 100 rows using any of several approaches. The approach in this example was to use the Add Rows option under the Rows menu on the menu bar and change the default number of rows from 20 to 100. 3. Change the title of the first column to x by clicking in the column heading and typing the new heading. 4. Figure 1.11 illustrates adding a formula for that column. Because the table currently contains only one column, all actions affect that column. To access the options in the upper left panel, right-click the column and select Formula. In the window that


appears (upper right panel) select Row¼Count. To start filling in the formula, select Change Sign in the formula editor window; enter 3 in the first highlighted box. Follow that entry with another 3 and 100 to complete the formula. Clicking OK in the editor window completes the process and writes the results to the table in the first column.

Figure 1.11 Creating a Count of Entries in the First Column of Measurement Tools.jmp

5. The next steps add additional columns to the table and label them as shown in Figure 1.10. The author chose to right-click in the empty space to the right of the first column to produce the options in the left panel of Figure 1.12. Selecting Add Multiple Columns brings up the window shown in the right panel of Figure 1.12. Completing this window as shown adds four new columns to the original table.


Figure 1.12 Adding Multiple Columns to Measurement Tools.jmp

6. Creating a formula for each of the four Tool columns using the Normal Density function (left panel of Figure 1.13) provides the entries in those columns. By default this function does not provide for entering the desired mean and standard deviation of a normal density function. To add that capability, press the comma key twice to bring up the right panel in Figure 1.13. Select the x column to fill in the first box, and then enter the appropriate mean and standard deviation for the column Tool 1. The means and standard deviations for each of the columns Tool 1 through Tool 4 are as follows: Column

Mean

Standard Deviation

Tool 1

0

0.25

Tool 2

0

0.5

Tool 3

4

0.25

Tool 4

4

0.50


Figure 1.13 Generating Distribution Data for Measurement Tools.jmp

Figure 1.14 shows an overlay plot for the simulated distribution data in each column versus the values of x. This figure has undergone considerable modification to improve its appearance, including adding a reference line to the X axis as well as annotating the curves to identify the columns that produced them.

Figure 1.14 Precision versus Accuracy in Measurement Tools


To Calibrate or Not When you have a measurement device, it is important to make sure it is actually measuring the object correctly. A calibration study is to determine how well a device carries out the measurement process. In some situations, traceable standards might be available to calibrate a measurement process. The National Institute of Science and Technology (NIST), formerly the National Bureau of Standards (NBS), certifies a variety of standards. Or a measurement tool might have some calibration procedure embedded in its software. The problem is how to decide whether or not to calibrate an instrument at some point in its use. Obviously, uncalibrated instruments could indicate that a manufacturing process is producing material off target when it really is not.

Understanding Risk Any decision made regarding calibrating an instrument contains risk. Figure 1.15 shows a truth table to help define the situations one might encounter. Although the average engineer or researcher probably does not realize it, making a decision about whether or not to calibrate a measurement tool involves generating two hypotheses—one the exact opposite of the other.

Figure 1.15 Truth Table

The null hypothesis, usually abbreviated H0, states that “the measurement tool does not require calibration.” Its direct opposite, designated the alternate hypothesis or alternative hypothesis, states that “the measurement tool requires calibration.” An investigator gathers data and analyzes it to determine whether or not the facts support the alternate hypothesis. The result of that analysis dictates whether he or she rejects or fails to reject the null hypothesis.


Here are some trivial mnemonics (not necessarily statistically rigorous) that help keep the α and β risk straight:

ART: αlpha – Reject the null hypothesis when it is True.

BAF: βeta – Accept the null hypothesis when it is False.

The power of a test is 1 – β. Another trivial method for helping to understand the types of risk is to liken α risk to seeing a ghost. Alternatively, β risk is akin to stepping off a curb in front of an oncoming truck. From Figure 1.15, if an engineer decides to calibrate a measurement tool when it does not require it (rejecting H0 when it is actually true), he or she commits an α error. On the other hand, deciding not to calibrate a measurement tool when it actually requires it produces a β error. Obviously, prudent investigators want to keep both risks small in their work, although some level of risk is always present. Calibrating a measurement tool unnecessarily might or might not be a serious problem, depending on the complexity and cost of the operation. If the combination of complexity and cost is large, then proceeding cautiously is good advice. If complexity and cost are trivial, then unnecessary calibrations will not necessarily produce a serious problem. The nature of the problem is that the observer must compare a computed average to a standard value within the bounds of α and β risks and in the presence of some variation in observations. That is, the sample size required to detect a difference between an observed average and a standard value is directly proportional to the α and β risks allowed and to the ratio of the inherent variation in the observations to the size of the difference to detect. Diamond (1989) provides an expression for estimating the sample size for this problem shown in Equation 1.6. 2 ⎛σ2 ⎞ ⎛ ⎞ N = ⎜ z α + zβ ⎟ ⎜ 2 ⎟ ⎝ 2 ⎠ ⎝δ ⎠

where

1.6

zα/2 and zβ are values from the unit normal distribution corresponding to the risks accepted, σ is the variation in the data, and δ is the chosen difference to detect.

Rarely, if ever, will an investigator know precisely what the variation in the data will be before conducting any experiments; that value must be estimated from the experimental data after running the experiments. A very useful approach is to decide what difference relative to the inherent variation in the data is acceptable and solve the equation from that perspective.


JMP Sample Size Calculations and Power Curves JMP contains a very useful utility to help an experimenter make reasonable decisions about sample size requirements for a variety of scenarios. The situation described here is actually the simplest among many. As shown in Figure 1.16, select Sample Size and Power on the DOE menu. Then select the first option, One Sample Mean, on the window that appears, because the problem under consideration is to compare an observed average of several observations to a target value.

Figure 1.16 Accessing Sample Size Calculations

Selecting One Sample Mean displays the window shown in Figure 1.17. The system sets a default α risk at 0.05, but the user can specify any value by editing the table. To use this system most effectively, set Error Std Dev to 1, and then specify the Difference to detect as some fraction or multiple of the error. As the window indicates, supplying two values calculates the third, whereas entering only one value (for example, a fraction or multiple of the unknown error) produces a plot of the other two.


Figure 1.17 Opening Window of Sample Size Calculation

Figure 1.18 shows Sample Size vs Risk and Delta.jmp (also found in the Chapter 1 directory on the companion Web site for this book at http://support.sas.com/reece) created by exercising the option of filling in Difference to detect and Power for a number of scenarios, given a value of 1 for Error Std Dev. This approach to determining sample size can be very useful in exploring various levels of risk and differences to detect what effect each has on the number of samples required to satisfy the conditions.

Figure 1.18 Sample Size vs Risk and Delta.jmp


An alternative approach for exploring sample size requirements specifies only the Difference to detect in Figure 1.19 in order to produce a plot showing how the other two parameters (Sample Size and Power) vary under set conditions of Alpha risk and Difference to detect, given an expected Error Std Dev.

Figure 1.19 Plots of Power versus Sample Size, Given Alpha and a Difference to Detect Relative to Error


Uncertainty in Estimating Means and Standard Deviations Anyone who has had a basic course in statistical process control (SPC) or perhaps in some level of measurement capability instruction might have been struck by the large numbers of observations usually recommended. In SPC one must estimate the grand average of process output as well as an estimate of variation in that process. Similarly, in characterizing the capability of a measurement tool or process, one must estimate several possible sources of variation. Figure 1.20 shows MN, STDEV CI.jmp (also found in the Chapter 1 directory on the companion Web site for this book at http://support.sas.com/reece). Throughout the table, the observed mean is 10 and the observed standard deviation is 1, but the sample sizes used to determine these observations vary from 2 to 100. The lower and upper 95% confidence interval (CI) boundaries are simulations based on established statistical concepts.

Figure 1.20 MN, STDEV CI.jmp


Confidence Interval for the Mean Equation 1.7 supplies the entries for the lower and upper 95% CI bounds for the mean (third and fourth columns of data in Figure 1.20).

⎛ s ⎞ ⎟ ⎝ n⎠

y ± tc ⎜ where

1.7

y is the observed average; tc is a critical value of the Student’s t based on n – 1 degrees of freedom (supported as t quantile in JMP); s is the observed standard deviation; and n is the sample size. The reader can display the formulas associated with data columns three and four to see the actual JMP implementation of this equation in each case.

Here are the major steps in creating the formula for the lower 95% CI bound for the mean. 1. After selecting the appropriate table column and choosing to generate a formula, select the Observed Mean column and add an element to it (Figure 1.21).

Figure 1.21 Adding an Element to the Observed Mean Column


2. Under the Probability option, select t Quantile (Figure 1.22). Figure 1.23 shows the resulting formula.

Figure 1.22 Selecting t Quantile

Figure 1.23 The Formula after Step 2

3. Select the multiplication operator from the choices provided in the formula window. 4. With the new box highlighted, select the division operator. The numerator in this expression is the Observed Std Dev, while the denominator is the square root of Sample Size (Figure 1.24).


5. The DF in the formula is Sample Size – 1; the p value is a number representing the fraction of a student’s t distribution remaining in the left tail of that distribution as the lower bound of the 95% confidence interval—0.025. Figure 1.25 shows the completed formula.


Figure 1.25 The Completed Formula

The formula for the upper 95% CI boundary for the mean is identical to that just described except for the value entered for p. The value for this expression is 0.975, which reflects the area under the student’s t distribution left of that upper boundary. Therefore, the area under the curve between the two limits is 0.975 – 0.025 = 0.95.

Confidence Interval for the Standard Deviation In Figure 1.20, data columns six and seven, respectively, provide the upper and lower 95% CI bounds for the standard deviation. Equation 1.8 provides these values based on the varying sample sizes.

⎛ ν s2 ⎜ ⎜ χν2,1−α ⎝ 2 where

⎞ ⎛ 2 ⎟ ≤ σ2 ≤ ⎜ νs ⎟ ⎜ χν2,α ⎠ ⎝ 2

⎞ ⎟ ⎟ ⎠

1.8

ν represents the degrees of freedom in the sample (n – 1); s2 is the 2 square of the observed standard deviation; σ is the true population 2 variance; and χ is a value from the chi-square distribution such that 1 – α/2 or α/2 of the distribution remains to the left of that value (α = 0.05 for a 95% confidence interval). The reader can display the formulas associated with data columns six and seven to see the actual JMP implementation.


The formulas embedded in these columns compute the lower and upper 95% confidence boundaries of the standard deviation (given as 1 in the table) based on the number of observations used to estimate it. Here are the steps used to create these formulas. 1. To create a formula with the creation of a division as the initial entry, select the ÷ symbol in the formula window. Convert that fraction to its square root using the √ symbol in the formula window (Figure 1.26).

Figure 1.26 Beginning the Formula

2. Add Sample Size –1 to the numerator to compute the degrees of freedom in the estimate. Multiply this entry by the square of observed standard deviation. Use the xy option in the formula window (Figure 1.27).



3. From the Probability menu, select ChiSquare Quantile (Figure 1.28).

Figure 1.28 Selecting ChiSquare Quantile

4. Fill in the appropriate value for p and DF in the expression. As shown in Equation 1.8 and as implemented in JMP, p refers to a value on the horizontal axis of the 2 distribution such that a fraction of the area under the χ curve lies to the left of that value. For a 95% confidence interval, α = 0.05, so 1 – α/2 = 0.975. The DF for the estimate is one less than the sample size. For the upper boundary of the confidence interval, the p value is 0.025.

Figure 1.29 The Completed Formula

Figures 1.30 and 1.31 are overlay plots created in JMP that illustrate how the confidence intervals for the mean and standard deviation, respectively, respond to changes in the sample size used to estimate them.


Figure 1.30 Mean CI versus Sample Size

Figure 1.31 StDev CI versus Sample Size

Notice in both figures that the horizontal axes are logarithmic and that the vertical axis in Figure 1.31 is logarithmic. In Figure 1.30 the uncertainty in estimating a mean (confidence interval) begins to stabilize between sample sizes of 10 to 20. However, Figure 1.31 shows that considerable uncertainty exists in the estimate of a standard deviation until the sample size is 30 or more. This does not mean that an investigator cannot estimate these parameters with fewer samples. It means that the uncertainty in


estimates of either means or standard deviations based on small samples might be unacceptable, particularly in the case of standard deviations. A measurement capability study primarily estimates variation in the measurement process; therefore, the investigator should try to accumulate at least 30 independent or replicate samples for each source of variation in that measurement process in order to have more sound estimates of the contributions of each.

Components of Measurement Error The observed error in a process is the sum of the actual process variation and a measurement error, as shown in Equation 1.1 previously. Measurement error itself has two identifiable and measurable components as well: repeatability and reproducibility, as shown in Equation 1.9.

σ measurement = σ repeatability + σ reproducibility 2

2

2

1.9

Repeatability Error Repeatability error is the simplest measurement error to estimate, because it represents the ability of the measurement process to repeat values in a short period of time. In the semiconductor industry, for example, an investigator using an automated tool could generate data for this estimation in a matter of minutes. All one has to do is place the object to be measured in the tool and press the Measure button a number of times. Since this error is a variance, the experimenter should collect at least 30 readings in short order (Figure 1.31) in order to obtain a reliable estimate. In other cases involving operators measuring individual parts, this error represents the ability of operators to repeat their measurements on those parts, perhaps over a period of time. The important point is that this error is an estimate of the variation of the measurement system under conditions of minimum perturbation. A guideline for the level of allowable repeatability error is for the P/T ratio to be ≤5. Many measurement experiments allow estimation of repeatability error along with estimation of reproducibility error and total measurement error. Conducting a simple preliminary experiment to estimate repeatability alone can pay dividends in that such a study might detect an inherent weakness or problem with the measurement system and do it relatively inexpensively. The sections “An Oxide Thickness Measuring Tool” and “Repeatability of an FTIR Measurement Tool” in Chapter 2 illustrate case studies where this was indeed the case.


Reproducibility Error Reproducibility error includes all other variables chargeable to the measurement system, such as day-to-day variation or operator-to-operator variation. Again this type of study estimates standard deviations or variances, so an investigator planning this investigation must pay particular attention to generating enough data such that each measurement factor has enough degrees of freedom (replicates) associated with it to produce a reliable estimate. Therefore, a complete and robust measurement study to estimate total measurement error and to separate repeatability and reproducibility errors can require weeks to complete. This is not to say that the measurement study should dominate the work of individuals running a process. Rather, over a significant period of time, the study should include enough measurement episodes to enable reliable estimation of variances. Generating a matrix of trials before starting the study and analyzing the result of filling in the measurement data with random numbers will provide information about the degrees of freedom associated with each factor of interest. In the semiconductor industry, an investigator usually measures several points on a wafer. These points will yield different values because coating thicknesses, for example, will vary depending on location on the wafer surface. The variations among these points is not normally charged to the measurement process, because they are an artifact of the object being measured and not part of the measurement process itself. But repeated measurements of those same locations do contribute to measurement variability.

Linearity of Measurement Tools As it applies to measurement processes, linearity is a measure of how stable measurement error and bias are over some range of values being measured. For example, a particular process might measure items of widely varying dimensions. The question to answer is whether or not the measurement error and bias is constant over that range. If it is not, then this type of study will make users aware of any additional limitations of their measurement process. Graphical representations can generate the required information. The section “Repeatability and Linearity of a Resistance Measurement Tool” in Chapter 2 provides an example that illustrates some of these principles.


Random versus Fixed Effects As applied to measurement systems, random effects are sources of variation that might include repeated measurements, variation over time, variation due to operators, variation due to different measurement tools, or even variation due to supposedly identical objects. Classical statistical methods consider observations of such events as examples from a large population of possible events—such as an infinite number of repetitions of an event, an infinite number of days for a study, or an infinite number of operators doing a particular task. Realistically, a measurement study samples replications, time, and operators and assumes they constitute a sample from a larger population. Particularly in the case of operators, usually no more than a few are available or trained for a particular task. Therefore, even though the entire population of operators might be involved in a study, logic requires treating any contribution from them as a source of nuisance variation or random noise initially. If, for example, one or more operators demonstrably produce results different from the group, then further study of each operator becomes warranted to establish a cause and a possible correction. A later section illustrates this point. In a measurement system, a factor is a fixed effect when that factor that is not normally considered a source of noise in the measurement system. In the semiconductor industry, examples of fixed effects would be the differences in individual measurement locations on a wafer, or differences between wafers with distinctly different properties such as film thicknesses. Ironically, factors designated fixed in one case (such as locations on a wafer surface) can become random in another, depending on the context of their analysis. For example, an experimental study of a manufacturing process might seek to minimize the variation of measurements found across a wafer surface. Although the investigation could continue to consider each wafer location a fixed effect, the objective of such a study generally is to minimize the contribution of those differences. Therefore, a logical approach considers the measurement sites sources of random error or nuisance variation in a process. In the author’s experience, whether to consider a particular effect random or fixed depends on the context of its effect on a process, whether a measurement process or a manufacturing process. JMP has excellent utilities for handling both types of effects, and later sections will make every effort to explain how and why to assign particular effects to a specific designation of random or fixed.

C h a p t e r

2

Estimating Repeatability, Bias, and Linearity Introduction 33 Evaluating a Turbidity Meter 33 The Data 33 Determining Bias for Stabcal < 0.1 36 The Paired Analysis 43 Measurement Tool Capability: P/T for Stabcal < 0.1 45 Determining Bias for Gelcal < 10 45 Determining P/T for Gelcal < 10 46 Lessons Learned 47 An Oxide Thickness Measurement Tool 47 Examining the Data 47 Excluding the Questionable Values 51 Generating a Multivari Chart 53 Lessons Learned 56 Repeatability of an FTIR Measurement Tool 56 Examining the First Test 57


Conducting a Second Test 59 Estimating Potential Bias 61 P/T and Capability Analysis 63 Repeatability and Linearity of a Resistance Measurement Tool 66 Reorganizing the Data 67 A Shortcut Method for Evaluating a Measurement Tool 75 Linearity of This Measurement Tool 76 Lessons Learned 77 Using Measurement Studies for Configuring a Measurement Tool 78 Examining the Data 78 Manipulating the Data: Combining the Tables 82 Evaluating the Combinations 85 Lessons Learned 90 No Calibration Standard Available 91 Examining the Data 92 Matched Pair Analysis 93 Lessons Learned 94 Exploring Alternative Analyses 94 Capability Analyses 95 Summarizing Data 96 Variance Component Analysis: Using Gage R&R 96 Using Regression Analysis 98 Lessons Learned 99 A Repeatability Study Including Operators and Replicates 99 Estimating a Figure of Merit: P/T 100 Variability Charts: Gage R&R 102 Fitting a Model 104 Lessons Learned 106 Summary of Repeatability Studies 108

Chapter 2: Estimating Repeatability, Bias, and Linearity 33

Introduction The section “Repeatability Error” in Chapter 1 suggested that a simple short repeatability study could identify problems with a measurement process before an investigator commits to a more complicated longer-term study. In addition, this type of study can determine the level of bias in a measurement tool, revealing just how accurate it actually is. Finally, repeatability studies carried out on a variety of parts or examples can provide information for evaluating the linearity of a measurement process. In this chapter, linearity is defined as the ability of the measurement tool to maintain a figure of merit (P/T) over the range of samples. Most of the examples in this chapter present actual case studies taken from semiconductor manufacturing; the discussion notes the few exceptions. The examples demonstrate various analysis platforms in JMP, but the discussions of these platforms are not extensive; for more information, consult JMP documentation.

Evaluating a Turbidity Meter Providing purified water to customers in a city or other municipality is an important task because public health is at risk. Water from surface sources or shallow wells usually contains suspended matter that might include harmful organisms. Water purification systems filter raw water either through beds of sand or through progressively smaller membranes down to 1 μ or less. The method used to clarify the raw water depends mainly on the volume of water being processed. Small systems (approximately 10K gallons of water/day) usually use membrane filters in replaceable cartridges, whereas larger systems require much larger filter beds to accommodate the volumes required. Regardless of the method used, the Environmental Protection Agency (EPA) sets stringent limits on the clarity of water entering a distribution system. Turbidity meters pass a beam of light through a sample cell, measure the scatter that occurs due to suspended material, and report the results in terms of nephelometric turbidity units (NTU). Very clear water has turbidity < 0.2 NTU; the EPA allows up to 2 NTU. Turbidity standards from the National Institute of Science and Technology (NIST) are available over a wide range of values. The example in this section considers results from the calibration of a turbidity meter used by a small water system.


The Data Figure 2.1 presents an excerpt from Turbidity 2100P Repeat and Calibration.jmp (found in the Chapter 2 directory on the companion Web site for this book at http://support.sas.com/reece). The first column in this table has the designation Numeric, Nominal while the other columns have the designation Numeric, Continuous.

Figure 2.1 Excerpt from Turbidity 2100P Repeat and Calibration.jmp

The operator had a set of traceable standards at various levels of turbidity available for calibrating the instrument (labeled Stabcal…). At the outset of the study, he elected to carry out a calibration study with the standard labeled < 0.1 order to estimate the repeatability or bias of the instrument. NOTE: Documentation furnished with the turbidity measurement tool described an elaborate calibration procedure using NIST traceable standards. The details of that calibration procedure are specific to this device and are not part of this discussion. In addition, a set of secondary standards (labeled Gelcal…) was also available, so he elected to test the repeatability of the instrument slightly beyond the acceptable limit of turbidity set by the EPA and to get an estimate of linearity of the device over that range.


A useful first step examines the distributions of both sets of data using the Distribution option on the Analyze menu as shown in the left panel of Figure 2.2. Selecting this option brings up the right panel in the figure. Working with both Stabcal… columns in the data table at the same time is convenient at this point and facilitates some of the decisions. After the user selects OK, the results shown in Figure 2.3 are displayed.

Figure 2.2 Generating Distributions from Data in Figure 2.1

The displays of the two data distributions suffer from inadequate measurement units (Wheeler and Lyday 1989). Each observation has no more than two significant figures, so the appearances of the histograms suggest discrete observations, even though the data is continuous.


Figure 2.3 Distributions of Stabcal … Columns from Figure 2.1

Under the heading Moments, note that JMP reports summary statistics for each group of data. Included in that list are the upper and lower bounds for the 95% confidence intervals for the means. The label on the sample vial stated that the turbidity value should be < 0.1%. The confidence intervals for the observations from the uncalibrated instrument do not include 0.1, so the true mean of this group of observations is not likely > 0.1% at 95% confidence. The confidence interval for the mean of the measurements taken after calibration has an upper limit slightly above 0.1, so the true mean of this group could be > 0.1%, although not by a large amount.

Determining Bias for Stabcal < 0.1 To determine whether the measurement tool exhibits bias in either case, one selects the menu hot spot (the inverted triangle) at the top of each distribution graph (upper panel of Figure 2.4). The menu that appears shows additional options for examining the data represented by a particular histogram (lower panel of Figure 2.4).


Figure 2.4 Distribution Options

The Test Mean option assumes that the distribution of observations is normal (bellshaped). Given the appearances of the histograms in Figure 2.3, this assumption might or might not be valid. To test the distribution of the observations, select Fit Distribution ¼ Normal (Figure 2.5).

Figure 2.5 Fitting a Distribution to Data

The system superimposes a curve on the histograms based on the mean and standard deviation parameters given in the summary statistics noted in Figure 2.3. The results for one of the distributions appear in Figure 2.6. The parameters μ and σ exactly match those found under the Moments title in the original report shown in Figure 2.3. Given the appearance of the curve relative to the histogram bars, one might conclude that a normal curve does not fit this data particularly well. To test the fit, one selects the menu hot spot


on the Fitted Normal title bar. This option brings up an additional menu that enables one to define the goodness of fit for the normal distribution curve on this data (top panel in Figure 2.7). Selecting the Goodness of Fit option as shown produces a report with an interpretation of the results (lower panel in Figure 2.7). NOTE: JMP supports two different techniques for testing the goodness of fit of a distribution curve to data. The one used here, the Shapiro-Wilk W Test applies to sample sizes ≤ 2000. For larger samples the system applies the KSL test.

Figure 2.6 Fitting a Normal Curve to a Distribution

The report in Figure 2.7 provides information on how to interpret the test. The null hypothesis (HO) assumes that the data are from a normal distribution. The alternative hypothesis (HA) contradicts that statement. The W value reported is the risk one assumes in rejecting the null hypothesis—that is, the normal curve does not fit the data very well. Performing the same tests on the data obtained after the calibration process produces similar results. The default methods for statistical tests of a mean assume that the data came from a normal distribution. Some doubt thus exists about the distribution of the data being considered here, given the appearance of the histograms and results of the goodness-of-fit tests. Therefore, including a nonparametric (distribution-free) test when comparing each mean of the distribution to its target value is a prudent choice.


NOTE: The Wilcoxon Signed Rank test does not assume that the data fits a normal distribution, but it does require that the data be distributed reasonably symmetrically about the mean. For more information, see Hollander and Wolfe (1973) or Natrella (1966).

Figure 2.7 Goodness-of-Fit Test for Normal Distribution Curve

Figure 2.8 shows the process for conducting a test of the mean. The upper panel shows the option to use from the menu associated with each distribution graph. The middle panel illustrates an intermediate dialog box in which the user defines the target mean and has the opportunity to include a distribution-free test as well as the test that assumes a normal distribution. Given the information presented to this point, selecting the distribution-free test (nonparametric, Wilcoxon Signed Rank) is a prudent choice. The lower panel in Figure 2.8 presents the report generated by JMP for this exercise. The identity of the sample that produced the data (in this case before the calibration step) indicated that its turbidity value should be < 0.1 NTU. Therefore, the correct test selected before the actual analysis should be to determine whether the data supports the conclusion that the mean is < 0.1.


Figure 2.8 Testing the Mean of the Data before Calibration

The JMP report actually tests three different sets of hypotheses and reports them in the included table. The first portion of the report summarizes the statistics calculated for the sample. The Test Statistic line in the report provides the test statistics (Student’s t and Signed Rank) calculated from the data.


Equation 2.1 illustrates the computation of the Student’s t statistic, which assumes the data has a normal distribution. (For the computation of the Signed Rank statistic, consult the references previously given.)

tobserved = where

y −μ 0 s n

2.1

tobserved is the test statistic; y is the sample average; μ 0 is the hypothesized mean value, s is the sample standard deviation, and n is the number of observations.

Following that entry the subsequent rows test three pairs of hypotheses: The row Prob > |t| provides a so-called two-tailed test with the underlying null hypothesis (HO) that the true mean of the distribution of data is the same as the hypothesized value. The alternative (HA) is that the true mean is not the same as the hypothesized value. In the row Prob > t the null hypothesis states that the true mean of the data is less than or equal to (≤) the hypothesized value; the alternative states that the mean is greater than the hypothesized value. Finally, in the row Prob < t, the null hypothesis states that the true mean of the data is greater than or equal to (≥) the hypothesized value; the alternative is that the mean is less than the hypothesized value. As stated previously, before collecting data and generating this report, the analyst must select the hypothesis to test. This means that only one of the three tests reported is truly appropriate for the given problem. In any form of data comparison, the actual test is whether the data supports the alternative hypothesis. The probability values in each row represent the α risk one accepts in accepting the alternative and rejecting the null hypothesis. In Figure 2.8, the analysis results indicate that the mean of the observations from the uncalibrated sample is different from the hypothesized value and is less than the hypothesized value. The graphic at the end of the report displays a normal distribution with the hypothesized mean and standard deviation based on the dispersion in the data. The single line to the left of the normal curve represents the observed mean of the data. That the theoretical distribution does not overlap this line suggests that the observed mean is indeed < 0.1.


Figure 2.9 Testing the Mean of the Data after Calibration

Figure 2.9 shows the report based on the data collected after the calibration of the instrument. Again, the correct statement for the null hypothesis is that the mean of the sample is < 0.1. In this case, deciding that the observed sample average of 0.09833 is < 0.1 (accepting the alternative hypothesis while rejecting the null) requires an α risk of 0.08, assuming a normal distribution and 0.13 using the nonparametric test. Statistical convention prefers that these risks do not exceed 0.05 for this type of comparison. To restate: the label on the standard sample used in the measurement study states that the mean observed value should be < 0.1 NTU. The results of these analyses suggest that after calibration the observed mean is not different from 0.1 NTU at α = 0.085. The actual computed mean is below the indicated value, but the confidence interval for that calculated mean extends above 0.1. These results suggest that the calibration actually changed the behavior of the instrument. However, even the calibrated instrument does not exhibit enough bias for concern. Validation of the instrument at this low turbidity level is important because the majority of water samples examined at this facility had turbidity readings < 0.5 NTU.


The Paired Analysis Before and after measurements yield paired data. As such these measurements must occur on the same experimental unit and must differ only in the conditions applied to the measurement. The arrangement of the observations for uncalibrated versus calibrated turbidity numbers in Figure 2.1 suggests a matched pair structure for the data. A paired data structure requires that one measures something under one set of conditions and then measures the same thing under another set of conditions. This particular case involved a single standard sample cell having the specified turbidity. The experimenter first measured that cell using the uncalibrated instrument. After calibration according to the specifications in the operating manual of the instrument, the experimenter measured the same standard cell again. Therefore, the data structure satisfies the requirements for paired data. The previous section suggested that the two means were different, and JMP provides a platform for this specific form of paired data analysis. Figure 2.10 shows access to this platform. The left panel illustrates selecting the Matched Pairs method of analysis, and the right panel illustrates describing the paired data to the system.

Figure 2.10 Setting Up a Matched Pairs Analysis

A matched pair comparison actually tests the null hypothesis that the average difference between the two groups is 0. The alternative hypothesis is that the difference is not 0. This hypothesis statement requires a two-tailed test represented by the line in the report Prob > |t|. Figure 2.11 shows the results of this analysis. In this figure the horizontal line at 0 represents the null hypothesis of 0 difference. The vertical line is the grand mean of all the data being compared. The horizontal solid line above the 0 line is the average difference found between the two groups; the dotted lines represent the 95% confidence interval for that observed difference. If the confidence interval for the difference includes the 0 line, then one concludes that the two groups are


not significantly different from one another at α = 0.05. Since these intervals are well above the 0 line and since the test Prob > |t| is much less than 0.05, one concludes that the two sets of data have different means—that is, the calibration affected the observations on the measurement tool.

Figure 2.11 Output of a Matched Pair Comparison

An alternative analysis of this data generates a column of differences between the two conditions and compares that difference to 0. The section “Estimating Potential Bias” in this chapter illustrates that approach for a different example.


Measurement Tool Capability: P/T for Stabcal < 0.1 The computed standard deviation for the calibrated tool using the < 0.1 NTU standard is 0.00648. No specification limits exist for this measurement, so generating a P/T ratio has questionable validity. However, if one arbitrarily assigns a tolerance range of 0.1 to this measurement, then Equation 2 .2 results: P / T = 100

⎛ 6 * σˆ ⎞ ⎝ USL − LSL ⎠

= 100

⎛ 6 * 0.00648 ⎞ ⎝ 0.1 ⎠

=

3.888 0.1

= 38.88

2.2

Clearly, this figure of merit exceeds both the overall P/T level discussed earlier and the recommendation made earlier in this chapter that P/T for a simple repeatability study should not exceed 5.

Determining Bias for Gelcal < 10 Repeating the analysis applied initially to the first two columns to the column headed Gelcal < 10 After Calibration in Figure 2.1 and testing the mean produces Figure 2.12. This data has three significant figures, so the histogram has a more reasonable appearance. However, fitting a normal distribution to the data and then checking its fit reveals that the distribution is not normal (Goodness-of-Fit Test report segment). Therefore, including the nonparametric test of the mean is appropriate in this case. Because the sample that produced the data had the label “< 10 NTU,” the correct analysis tests the null hypothesis that the sample average is ≤ 10. The line Prob < t in the report provides the appropriate analysis and indicates the sample average is indeed < 10. The graphic at the bottom of the report illustrates how the observed sample average lies well outside what one might predict had the data come from a distribution with a true mean of 10.


Figure 2.12 Analysis of Gelcal < 10 Data

Determining P/T for Gelcal < 10 In this case as well, generating a P/T test might or might not be valid, because no specifications exist. Arbitrarily applying a tolerance range of ± 0.5 produces a P/T value of 6.72. Notice that the standard deviations of the data from the Stabcal and Gelcal samples are not materially different (0.006 versus 0.01, respectively). This fact suggests that within the range tested, the variation in instrument readings is reasonably linear. Testing the bias of the instrument against a traceable standard in the higher range would define bias more precisely over this range, but the investigation did not collect that data.


Lessons Learned If anything, this study showed that the measurement tool might have precision problems when measuring very low turbidity levels. Unfortunately, that is the range of most interest in preparing suitably pure potable water. Finding that a measurement tool has precision problems for the lower range of its utility is not unusual. Notice that the standard deviations observed for the two ranges investigated were not extremely different; this result is fortunate. The standard deviation has a much larger impact for the very low turbidity levels than it does for the more moderate levels. Contacts with the supplier of this device revealed that the performance seen here was typical of this unit. Therefore, the technician decided to continue the study with a longer-term investigation of the device to make certain that it at least gave stable readings over time.

An Oxide Thickness Measurement Tool Semiconductor manufacturing involves depositing a number of layers of materials on a silicon wafer and selectively removing portions of them to create electrical circuits. Depositing or growing silicon oxide on these wafers provides insulating layers to separate multiple circuit levels and to provide the correct electrical properties. This example describes an initial study to determine the repeatability of a measurement tool dedicated to measuring silicon oxide thickness. In this study, no traceable standards were available, so direct estimation of bias was not possible.

Examining the Data The measurement tool routinely measured a 49-point pattern across the 8-inch diameter silicon wafer, so the investigators decided to use that programmed sampling plan rather than to create another one. In conducting this study, the investigators repeated the measurement pattern 31 times before removing the object from the measurement tool. Figure 2.13 shows an excerpt of the raw data table Oxide Repeatability.jmp. The data table contains over 1500 rows, so critical examination of the entries is extremely important before doing any formal calculations or making any decisions.


Figure 2.13 Excerpt of Oxide Repeatability.jmp

Examining the Distribution Figure 2.14 shows the distribution of the raw data from this table. The outlier box plot included by default suggests that a number of recorded points are unusual. The quantile list below the histogram indicates that 50% of the data is between about 105.54 and 107.95, but there are values > 400 as well as some = 0. In fact, approximately 97% of the data lie between 104.02 and 155.84. This fact strongly suggests that something is wrong with the measurement tool or the object being measured. One approach for determining the source(s) of the unusual values is to use the Brush tool on the graphic to select the unusually high and unusually low points, and then create a subset of the original table containing only the highlighted entries. Although this method can suggest the source or sources of the unusual values, the author prefers a graphical method such as an overlay plot.


Figure 2.14 Distribution of Data in Oxide Repeatability.jmp

Creating an Overlay Plot The left panel in Figure 2.15 illustrates accessing the overlay graph type, and the right panel illustrates the setup used to create it.


Figure 2.15 Accessing the Overlay Plot

Plotting the measured thickness versus measurement site is but one way to generate this graphic. However, it is the most useful in this case. Figure 2.16 shows the result.

Figure 2.16 Overlay Plot of THICK versus Measurement Site

Creating an overlay plot of thickness versus measurement site using all the data in Figure 2.13 further illuminates the problem. This plot (after modification of the horizontal axis) shows that some values collected at sites 37 and 38 are unusual, along with the single odd observation at site 1. Some detective work after the study had ended revealed an anomaly


centered at site 38 on the substrate. In semiconductor manufacturing, each silicon wafer receives a unique identification number that is scribed by a laser on its surface. Microscopic examination of the wafer surface revealed debris from the laser scribing operation scattered about the surface near measurement site 38. Presumably some of the debris might have affected site 37 also. Because these two sites have questionable utility for measurements, a sensible approach excludes them from further consideration. In addition, because a single measurement at site 1 is also unusual, prudence dictates excluding it as well.

Excluding the Questionable Values Writing an exclusionary clause for this table will eliminate the suspicious points from further consideration. Access to row selection is available either on the Rows menu at the top of the JMP window or from the menu hot spot for Rows on the table display. To select specific rows, one selects Row Selection ¼ Select Where (Figure 2.17).

Figure 2.17 Selecting Rows from a Table

In the next window (Figure 2.18), one identifies the rows to select and makes sure to select the option to select the rows if any condition specified is met. This simplifies the process and does not require writing out complicated AND or OR Boolean expressions.


Figure 2.18 Defining Selection Criteria

With the selection criteria in place, one selects Exclude/Unexclude on the Rows menu to remove these rows from consideration. Generating a new overlay plot using all the surviving data points in the original table (Figure 2.19) shows the result of the exclusions. The undulating nature of the graphic is characteristic of how thickness measurements vary across a wafer surface. NOTE: Alternatively, the user might select Hide/Unhide from the Rows menu to hide the selected points and generate the graphic using that data. Either approach (Exclude/Unexclude, Hide/Unhide, or the combination) can produce the same graphic. The author has found that excluding points is a simpler approach. However, the scattering of large values throughout the measurement sites is not normal and suggests some further problem might exist.


Figure 2.19 Overlay Plot of Data after Excluding Suspicious Values

Generating a Multivari Chart In JMP, the Variability/Gage Chart option on the Graph menu at the top of the JMP window includes extremely useful graphing options as well as powerful analysis and interpretation routines related to gauge studies and to characterizing the variation in a process. Later sections will explore these analysis options. Figure 2.20 shows the selection and launching of the Variability/Gage Chart platform. For this type of graph, one selects the variable that varies the least first, followed in order of hierarchy with others. In this data table, DATE has only two values, whereas TRIALNO has 31.


Figure 2.20 Launching and Defining a Variability Chart

NOTE: For this particular example, the order in which one defines the X, Grouping variables is not very important. This order does become important when analyzing the data and determining the variance components. Figure 2.21 shows the result of generating the graph. By default, JMP plots the raw data as requested and includes all data points, a range bar, and the mean for each cell. Also included is a standard deviation graph for the groupings that can help identify unusual excursions in the data. Figure 2.21 includes only the means of each group and suppresses display of the range bars and data points to facilitate the interpretation that follows.


Figure 2.21 Variability Graph for Oxide Repeatability.jmp

Notice the partitioning of the data according to the date of the trial. For each trial, the experimenters placed an object in the measurement tool, carried out the 49-point measurement, and then removed it. Time ran out during the first day of the work, so they restarted the study at Trial 24 immediately upon returning to work the next day. Clearly something happened to the measurement process during that interval, because the cell mean increased several units and the standard deviation increased as well. Some further detective work by the investigators found that when the measurement tool lay dormant for some period, a screen saver activated to protect the phosphors in the computer screen associated with the measurement tool. Whenever the screen saver came on, the system also shut down the light that illuminated the measurement stage. This light has a critical color temperature, and some time elapses after it comes on until it reaches that color temperature. Therefore, at least some of the measurements taken on the second day are suspect.


Lessons Learned The first important lesson derived from this study was that an analyst must pay careful attention to the condition of the data associated with particular measurement points on the object being considered. In this case, microscopic debris generated by the laser scribing process that places the identification number on the wafer affected some measurement sites. The second important lesson concerned the measurement device itself. The activation of the screen saver for the computer screen on the measurement tool also shut down the lamp used to make the optical measurements being examined. The solution to this problem was to disable the screen saver to prevent this event. The engineering team applied this change to all similar devices used in this facility. Rather than try to determine which measurements on the second day were not useful, the engineers disabled the screen saver and repeated the study another day. The results of the second study showed that this device was quite capable of making precise measurements as required.

Repeatability of an FTIR Measurement Tool This example shows how an initial short repeatability study identified a problem with a measurement tool. This study involved taking 30 readings in quick succession on the center point of a wafer without removing the wafer from the measurement device. It detected a problem with a Fourier Transform Infrared (FTIR) measurement device used to determine polysilicon thickness on a silicon wafer. Figure 2.22 presents an excerpt of the data in FTIR Repeatability.jmp.


Figure 2.22 Excerpt of FTIR Repeatability.jmp

Examining the First Test In this case the engineers conducted the first repeatability test on the device by placing a substrate in the machine, closing the access door, and measuring the center of the object 30 times without removing it from the tool. Visual examination of the raw data for Test 1 suggests a drift to higher values. An overlay plot of Test 1 versus Observation confirms an apparent increase in values. To explore this evidence further, the user fits a line to the data and evaluates the slope of that line statistically. One changes the Modeling Type of the Observation column from Nominal to Continuous, and then selects Fit Y by X on the Analyze menu (Figure 2.23).


Figure 2.23 Creating a Bivariate Fit of Test 1 versus Observation

Figure 2.24 shows the results. Fitting a line to the data (the Fit Line option on the menu) indicates a substantial and statistically significant drift from Observation = 1 to Observation = 30. The Analysis of Variance section of the report confirms that the model of Test 1 versus Observation explains a statistically significant portion of the variation in the Test 1 data. The test for significance is the Fisher’s F Ratio. The null hypothesis being tested states that the variance (mean square) due to the Model (0.011037) is less than or equal to that for the Error (0.000025). The alternative hypothesis asserts that the variance of the Model is greater than that of the Error. The Prob > F portion of this report provides the α risk if one rejects the null hypothesis and asserts that the two variances are different—that is, the slope of the line is different from 0.


Figure 2.24 Examining the Results of Test 1

NOTE: The F Ratio listed (447.5554) is the ratio Model Mean Square/Error Mean Square; this value is the square of the t Ratio reported for the coefficient of Observation in the bottom line of the report.

Conducting a Second Test The question then becomes, Why did the measurements drift so much in such a relatively short time? Subsequent investigation suggested that purging of the system with dry nitrogen to remove water vapor in the measurement chamber might be responsible. The drift might be the result of the change in moisture content in the chamber with time. Two approaches might solve the problem: either allow a significant amount of time for the chamber atmosphere to equilibrate or suspend the flow of nitrogen. The engineers opted for the second choice because of production time constraints, recognizing that the values obtained might contain a slight bias. The second set of data collected without the nitrogen


purge is in the column Test 2 in Figure 2.22. Treating this column the same as Test 1 produced the bivariate fit shown in Figure 2.25.

Figure 2.25 Examining the Results of Test 2

The fitted line in this figure has a statistically significant negative slope, but it is not as large as the positive slope observed in the first case. After examining the situation, the engineers decided that this drift did not pose a problem for future measurements involving measuring five or nine sites on each wafer—that is, it did not have engineering significance.


Estimating Potential Bias The average value from the second test appeared larger than the average in the first test. In this case, the engineers measured the center of a wafer 30 times under one condition, and then measured it again under another condition. To compare these two averages, one uses the Matched Pairs platform again. The section “The Paired Analysis” in this chapter illustrates how to access this analysis platform. The analysis here is analogous in that one compares Test 1 to Test 2 and determines whether the differences in the values observed average 0. Two methods exist for conducting this test. The first is in the referenced section, in which one compares two paired sets of data. The second is for the user to generate a new column of differences in the data table and test whether the mean of the differences is different from 0. A useful exercise is to generate the column of differences and then generate the distributions of all three columns and test to see whether the distributions are normal. If one is not, then incorporating a nonparametric test into comparing the two columns is prudent. Figure 2.26 shows the distributions and the fitted normal distribution for each of these three columns. Under each distribution are a Probability statistic and its interpretation. As in the section “The Paired Analysis,” the null hypothesis for the test is that the distribution is normal, whereas the alternative hypothesis is that the distribution is not normal. The probabilities reported are the risk one takes in rejecting the null hypothesis. In each of these cases, the probabilities of being wrong in assuming that the distributions are not normal are all high. Therefore, one decides that the distributions are normal in each case, and so fails to reject the null hypothesis.

Figure 2.26 Distributions of Test 1, Test 2, and Difference


Figure 2.27 shows the output from the Matched Pairs platform involving Test 1 and Test 2 data. Although the difference in means between the two methods is statistically significant, the engineers decided that given the tolerance (specification limits) of the process, the observed difference did not have engineering significance. An alternative analysis of this data uses the column of differences described earlier and compares the average of that column to 0. Figure 2.28 shows this analysis. The hypothesis being tested is HO: The mean of the differences is 0. The alternative is HA: The mean of the differences is not 0. This is a two-tailed test, so the appropriate statistic to consider in the report is Prob > |t|. This value provides the α risk the investigator assumes in rejecting the null hypothesis.

Figure 2.27 Matched Pair Test for Test 1, Test 2


Figure 2.28 Testing If the Mean of the Differences is Equal to 0

This report says that the investigator should reject the null hypothesis of no difference and accept the alternative that the mean of the differences is different from 0. The statistics are identical in this case, as seen by the values of the t-test and the t-ratio.

P/T and Capability Analysis To compute the P/T ratio for this experiment, one first summarizes the data for Test 2, generating the mean and standard deviation of this column. Figure 2.29 shows access to the summarizing environment. First one selects Summary on the Tables menu; then in the dialog box that appears one selects the column or columns to summarize, followed by the statistics desired from the Statistics menu. Clicking OK generates a new table containing the statistics. To this table, one adds a new column, Repeatability P/T, and generates a formula based on Equation 1.2. The tolerance value is ± 5% of the mean. Figure 2.30 shows the formula generated and the final table.


Figure 2.29 Summarizing Test 2

Figure 2.30 Formula for Generating P/T Ratio and Result

The value 13.63 is somewhat larger than the target value of 5 for a repeatability study, but the tool might be adequate; only a longer-term study to estimate the total measurement error can answer that question completely. The high value observed should serve as a warning. An alternative means for judging the measurement tool is to carry out a capability analysis and develop a Sigma Quality level for the measurement process. To compute these values, one creates a distribution of the data for Test 2 using the Distribution option on the Analyze menu; then one chooses Capability Analysis from the pop-up menu associated with the histogram that is generated. Using a target value of the mean of the data with the ± 5% limits for the tolerance completes the dialog box in Figure 2.31.


Figure 2.31 Setting Up a Capability Analysis for Test 2

By default, the system uses the standard deviation computed from the data (Long Term Sigma) to prepare the graphics. Unless some other compelling reason exists, leaving this default unchanged is appropriate. Figure 2.32 shows the output.

Figure 2.32 Capability Analysis of Test 2 Data

The Sigma Quality output at the right of the report is a measure of how likely it is that this measurement tool might produce observations outside the ± 5% tolerance limits for the response. The capability indices reported are a measure of how many distributions of observations could likely fit between the arbitrarily defined specification limits. For both


Sigma Quality and Capability, the larger the value the better the measurement tool. In a general sense, capability indices ≥ 3 or Sigma Quality values ≥ 6 are highly desirable. Chapter 1 contained the statement that the P/T ratio calculated earlier in this example is actually an inverse of capability (Cp). Multiplying the inverse of Cp by 100 produces 13.63, in close agreement with the P/T value in Figure 2.30.

NOTE: Display limitations prevent the system from reporting precisely what the PPM level of defects is in the report generated by the capability analysis. By trial -9 and error, the author determined that the value had to be approximately 1x10 PPM using the formula given in the documentation (JMP Statistics and Graphics Guide, Release 6).

Repeatability and Linearity of a Resistance Measurement Tool In this example, several operations in a semiconductor manufacturing facility shared the same measurement tool. These operations produced a series of films with a wide variation in surface resistivities, expressed as Ω/ or ohms/square. As a prelude to a general complete measurement capability study of this tool, the engineers collected a series of wafers with deposited films representative of what one might expect to measure on this device. They loaded the 13 wafer types collected into a single cassette in no particular order. After reprogramming the measurement tool to measure a point at the center of the wafer repeatedly (30 times) before unloading and loading the next wafer, the engineers cycled the cassette through the measurement tool three times. Therefore, this study provides the opportunity to test whether the loading/unloading operation contributed materially to the variability of the tool.


Figure 2.33 Excerpt of Resistivity repeatability.jmp

Because the samples had resistivities ranging over approximately three orders of magnitude (0.03 to 14.5 Ω/), the study also presents an opportunity to judge linearity of the measurement device. This study defines linearity as the ability of the measurement tool to maintain a figure of merit (P/T) over the range of samples. A statement of linearity capability should also include some measure of bias. However, measuring bias implies that traceable standards are available, and that was not the case here. The samples are representative of the manufacturing process and not standards. Figure 2.33 presents an excerpt of the table Resistivity repeatability.jmp.

Reorganizing the Data The data organization illustrated in Figure 2.33 is typical of the structure often generated by automated data collection programs; the structure is not the “case by variable” or flat file format required by most statistical analysis platforms. Therefore, a first requirement is to reorganize this table using the appropriate routine within JMP. Figure 2.34 shows the sequence of events one should follow with a demonstration of the proper method for describing the stacking operation to JMP.


Figure 2.34 Setup for Reorganizing Resistivity repeatability.jmp

The 30 columns with numeric headings in the original table represent the 30 observations taken on each wafer at its center. The observations record the resistivity of the object. Therefore, in filling out the screen description of the stacking operation, the data in those 30 columns become one column which should have the label Resistivity. The column headings become another new column in the table with the label Observation. Figure 2.35 presents an excerpt of the new table generated from the original, Resistivity repeatability stack.jmp. The Stack routine assumes that the column headings in the original table are text entries and assigns the new column Observation a character data type. To facilitate some graphing operations, one should change that designation to Numeric, Nominal.


Figure 2.35 Excerpt of Resistivity repeatability stack.jmp

The graphing capabilities in JMP allow the investigator to examine the data collected from all 13 substrate types fairly conveniently. For example, to view the data distributions for each sample, one assigns the BY option to WAFTYPE in the Distribution option on the Analyze platform. This generates histograms for all groups with one command. Examination of these graphs indicates that none of the sets of data have any unusual values. Similarly, the Variability/Gage Chart option on the Graphs menu also generates 13 sets of graphs with one command. Figure 2.36 shows how one might prepare to generate those graphs. Figure 2.37 is one of the 13 graphs generated from this setup. This graphic provides another convenient method for examining the data from each group to identify important trends or unusual values as demonstrated in a previous example.


Figure 2.36 Setting Up a Group of Variability Charts

Figure 2.37 A Typical Variability Chart from This Example

The pop-up menu on the variability chart offers a variety of options for examining the data in more detail, including computation of Variance Components and evaluating gage capability. Figure 2.38 shows this menu.


Figure 2.38 Pop-up Menu on Variability Chart

The Variance Components option enables the user to select a variety of model options for the calculation. If the data collection is balanced (the same number of observations in each group), the system uses a method known as Expected Mean Squares (EMS) for the calculation. If the data collection lacks balance, then the system uses a method called Restricted Maximum Likelihood (REML). Figure 2.38 indicates the user has chosen the Variance Components option. This choice brings up the dialog box in Figure 2.39, where the user can specify an appropriate model. In this case, observations “belong” to a particular repetition, so the proper choice is Nested as shown. This action produces the report in Figure 2.40, which reveals the sources of variation in the data (using the variability chart in Figure 2.37). The Within source of variation is zero with zero degrees of freedom. This occurs because specifying Observation(Rep) is like specifying the residual of the model. One should not use an over-specified model, but instead should restructure the variability chart by not including Observation, as in Figure 2.41. The results are in Figure 2.42, where the Within has replaced Observation(REP). The Main Effect option was selected since there was just one effect in the model.


Figure 2.39 Specifying a Variance Component Model

Figure 2.40 Variance Components from Data for WAFTYPE = TISI/POLY1

The report indicates that the variation among observations contributes slightly more than 98% of the variability in the data for this set of observations. Interestingly enough, the act of loading and unloading the object (REP) in conducting the measurements contributed very little variation in the data.


Figure 2.41 Rerunning the Model without Observation in the Model

Figure 2.42 Variance Components from Data for WAFTYPE = TISI/POLY1 without OBSERVATION Being Specified

The Gage Study option on the menu evaluates the measurement tool only for two levels of variability and interprets the data in a classical Operator/Part methodology. The examples in this material focus on more powerful general methods that have proved extremely successful in the semiconductor industry. But to illustrate the Gage R&R report for this set of data, assume that the variance component calculation was not run. After selecting the Gage Studies option in Figure 2.38, followed by the Gage R&R option from the menu, the user must define a model for the variation as illustrated in Figure 2.39 as an intermediate step.


Defining the same model as before produces the dialog box in Figure 2.43. The tolerance interval represents an arbitrary ± 10% interval around the mean of the data.

Figure 2.43 Defining the Tolerance Interval for the Measurement Study

Figure 2.44 shows the Gage R&R report for this example. Notice that the system treats the first X, Grouping variable, REP, as Operator and the second, Observation, as Part. This behavior is a limitation of this analysis method. Note also that the variance components reported are identical to those shown in Figure 2.42. In this case, the variation contributed due to the repeated observations has the label Repeatability, whereas the effect of loading and unloading the object has become Part-to-Part. The % Gage R&R statistic reported is a figure of merit for the measurement tool. The section “To Calibrate or Not” in Chapter 1 mentioned this statistic in passing. The value listed, 99.2, is “not adequate” on this scale. Values less than 10 are “excellent.” The approach for analyzing measurement tools in the semiconductor industry differs from this report in that this study was a repeatability study. Determining a figure of merit for the measurement tool for each object measured requires repeating the process for each variability chart produced in this example.


Figure 2.44 Gage R&R Report for Data in Figure 2.37

A Shortcut Method for Evaluating a Measurement Tool In conducting measurement studies during his tenure at SEMATECH, the author devised a shortcut method for evaluating measurement tools. Dr. Peter W. M. John of the University of Texas, Austin, evaluated this technique at the author’s request while Dr. John was assisting with statistical consulting for the Statistical Methods Group at SEMATECH. He found that the method described generally underestimated the true variation in a measurement system by a small percentage, depending on the complexity and structure of the data set (John, personal communication, August 1992), but that it was adequate to characterize a measurement process that was not marginal in its performance. Basically, the technique summarizes the data across all sources of variation that are chargeable to a measurement process and generates a mean and a standard deviation. Then one uses the equation for P/T (Equation 1.2) to establish a figure of merit for the process. Figure 2.45 shows the summary statistics found for the 13 substrate types in this study and the appropriate calculation of P/T, assuming a ± 10% tolerance about the calculated mean.


Figure 2.45 Data in Resistivity repeatability.jmp Summarized by WAFTYPE

To evaluate the precision of this method, compare the value for the standard deviation for TISI/POLY1 to that computed more precisely using variance components methods in Figure 2.40. The value in line 11 of the table in Figure 2.45 is 0.00968 and blends contributions from the loading/unloading step with the repeated observations. The total standard deviation reported in Figure 2.40 is 0.00971. The shortcut method, therefore, underestimates the variation by less than 1% in this case.

Linearity of This Measurement Tool So far as the author is concerned, one useful method for determining linearity of a measurement tool over some range of values for substrates is to compute the P/T figure of merit for the tool in each case and compare these values. Figure 2.45 provides that information in numeric format; alternatively, the investigator can graph them for a visual comparison. Figure 2.46 is an overlay plot with the mean resistivity plotted on the left axis and the P/T ratio plotted on the right axis. The figure shows the mean resistivity and P/T value for each substrate type. The reference line on the right axis represents the desirable target for this figure of merit for repeatability studies (P/T = 5).


Figure 2.46 Graph of Mean and P/T by Substrate Types

Careful examination of the most important violators of this boundary condition showed that the samples had been scratched or otherwise damaged. Replacement of these objects with better examples produced results similar to the other samples. As noted earlier, this technique does not address the bias issue because the samples do not represent traceable standards.

Lessons Learned This experiment demonstrated the capability of a particular resistivity measurement tool to accommodate an extremely wide range of mean resistivity values with precision. This example also illustrated a valuable shortcut method for generating the P/T figure of merit and demonstrated its close agreement with more sophisticated methods for computing variance components.


Using Measurement Studies for Configuring a Measurement Tool A new tool had been purchased for measuring the resistivity of coated films in a particular semiconductor operation. With this new tool came four heads or four-point probes to use for measuring surface resistivities as ohms/square or Ω/ values mentioned in the previous example. Engineers who planned to use this machine wanted to determine which of the probes, if any, was better suited to measure the variety of films produced in this operation. They gathered four representative wafers whose expected resistivities varied from about 20 to 700 Ω/ and conducted repeatability measurements on each substrate using each head. Presumably, they measured resistivities of all substrates with a single probe before changing to another probe, but that information is not available. The pattern of measurement on each substrate was 49 points, repeated 34 to 54 times.

Examining the Data Four tables contain the measurements from this study: Resistivity Head B Full.jmp, Resistivity Head C Full.jmp, Resistivity Head D Full.jmp, and Resistivity Head E Full.jmp. Given the nature of this experiment, each table is fairly large in that each contains up to 11000+ rows and 5 columns. Figure 2.47 presents a small excerpt from one of them that is representative of all.

Figure 2.47 Excerpt of Resistivity Head B Full.jmp


Particularly when dealing with data tables this large, the axiom “When all else fails, look at the data!” certainly applies. Graphical examination of the data might immediately identify problem areas and usually suggests solutions. In this case, the variability chart is particularly useful. Figure 2.48 shows how the author approached this problem for each of the original data tables in this example. Limiting the X, Grouping variable to SITE and grouping the charts by levels of Nom. Resist. provides four useful and readable charts per table. Any other information in the X, Grouping option begins to obscure information on the computer screen. NOTE: The Overlay Plot is another option for generating graphs grouped by Nom. Resist. for each table, but the author prefers to use the Variability Chart because that platform automatically scales the Y axis for each graph depending on the data in that graph. This scaling makes interpretation somewhat easier without extensive editing of the graphics produced.

Figure 2.48 Setting Up Variability Charts

Figure 2.49 shows the four charts generated from Resistivity Head B Full.jmp with the displays of range bars, cell means, and the standard deviation chart suppressed. As seen in an earlier example, all the substrates exhibit erratic measurements at site 38, and some show values of 0 at various sites. Whether this latter situation is a problem with the probe head or with the substrate is unknown at this point.


Figure 2.49 Variability Charts Generated from Resistivity Head B Full.jmp


Because these erratic measurements represent unusual or suspicious observations, a prudent choice is to eliminate them from the data table, starting with the Row Selection option as shown in Figure 2.50. This operation selects some 288 rows in the table being examined. Choosing to delete the selected rows removes them from the data table. Generating the distributions for the modified data table grouped by nominal resistivity suggests a few more unusual points from the outlier box plots associated with each distribution. Brushing these outliers identifies another 171 points that are suspicious. Choosing to delete these new points from Resistivity Head B Full.jmp eliminates all the unusual values and still leaves over 8500 observations in this table out of an original 9000+.

Figure 2.50 Selecting Suspicious Observations

Regenerating the variability chart from the modified table reveals that the standard deviations for the observations in each nominal resistivity group are much more stable. Generating the variability chart from Resistivity Head C Full.jmp indicates no truly suspicious values in that table. Generating the corresponding distributions (Figure 2.51) identifies a few outliers in the 20 and 350 groups. Brushing those points identifies only 38 points, leaving more than 8400 unchanged. None of the unusual values at Site 38 or any 0 values are present. Did measuring the wafers repeatedly with Head B remove the sources of the unusual values, or does Head C simply perform better? With the information currently available, the observer cannot make a decision.


Figure 2.51 Distributions from Resistivity Head C Full.jmp

Similarly, the data in Head D Resistivity Full.jmp also showed only three unusual values; brushing and removing them left over 11000 observations in the table. The data in Head E Resistivity Full.jmp had 15 zero values; the generation of distributions revealed an additional 85 outliers. Brushing and deleting all suspicious observations left over 7000 points in this table.

Manipulating the Data: Combining the Tables Each of the four tables was now free of the most suspicious data points. The ultimate objective of the study was to determine just how well each probe head could measure the assortment of wafers. While not absolutely necessary, combining the four tables into a single table by appending rows of each table to a new table is a useful approach. The Concatenate option on the Tables menu provides this option (Figure 2.52).


Figure 2.52 Concatenate Tables

The expected result of using this utility is that the system will generate a new table by appending the rows of selected tables to form a new table. However, for this to work as expected, the column headings must be absolutely the same in each table. If any differences exist in the names, then the new table will have a column for every version of column names that it finds. Activating this option brings up the window in Figure 2.53. The current active table is the first table in the list, and all other open tables appear in the Opened Data Table section on the left.

Figure 2.53 Setting Up a Concatenation


From the list of opened data tables, the user selects the ones of interest and chooses Add. When the list is complete, one selects Concatenate. Figure 2.54 is an excerpt of the resulting table Resistivity Heads Combined.jmp.

Figure 2.54 Excerpt of Resistivity Heads Combined.jmp

The new table has over 35,000 rows and contains all the surviving data from the original four tables.


Evaluating the Combinations The objective of this study was to identify which of the four probe heads available for this measurement tool gave the most consistent results across a spectrum of expected resistivities. The shortcut method for determining the P/T figure of merit illustrated in the section “A Shortcut Method for Evaluating a Measurement Tool” in this chapter is particularly useful here. This study contained four sources of variation: probe head, nominal resistivity, replication, and measurement site. Of these four sources, only the replication variation is truly chargeable to the measurement process. Therefore, a first step in computing a figure of merit is to summarize the table containing all of the data for RS while grouping the calculations by HEAD, Nom. Resist, and SITE. Failing to include wafer site among the grouping variables effectively charges the measurement tool for the inherent variation among sites on the substrate. Figure 2.55 shows the setup of the summarization; Figure 2.56 is an excerpt of the results.

Figure 2.55 First Step in Summarizing the Filtered and Combined Data

Notice that the Figure 2.55 requests the variance of RS rather than the standard deviation as in the example in “A Shortcut Method for Evaluating a Measurement Tool.”


Figure 2.56 Excerpt of the First Summary Table of Filtered Data (Resistivity Heads—First Summary.jmp)

The next step pools the variance among the sites and determines the grand mean of the observations by Head and Nom. Resist. Pooling the variances among sites effectively averages the individual site variances. This process assumes that the sample sizes associated with each site are equal and that each variance is approximately equal. Although this assumption is not precisely true of each site, most of the sites have 40 or more observations in them, so the assumption is a reasonable approximation. Figure 2.57 shows the setup of the second summarization step.


Figure 2.57 Setting Up the Second Summarization Step

Figure 2.58 presents the result of the second step and includes an additional column for Repeatability P/T for each combination of HEAD and Nom. Resist. Figure 2.59 displays the formula used to compute P/T, because the summary included the pooled variance rather than a standard deviation. The highlighted portion shows that heads C and D gave very similar results in all categories and produced figures of merit nearly independent of the nature of the substrate being measured.


Figure 2.58 Second Summarization of Filtered Data: Calculation of P/T (Resistivity Heads—Second Summary.jmp)

Figure 2.59 Formula for P/T Column

To examine these results graphically, one can construct a bar chart of P/T as a function of Head and Nom. Resist. The upper left panel in Figure 2.60 shows access to this environment. The upper right panel in Figure 2.60 shows how to use the raw data in a column for the chart. Finally, the lower panel shows the completed dialog box for creating the bar chart.


Figure 2.60 Setting Up a Chart of P/T versus HEAD and Nom. Resist.

Figure 2.61 shows the chart generated. The results from HEAD = C and HEAD = D are 1 clearly more linear across the varying levels of nominal resistivity than those from HEAD = B and HEAD = E. Either HEAD = C and HEAD = D would be most useful in configuring this tool for routine use. Since they are so similar, a possible solution places one of them in service while holding the other in reserve in case of damage. Since the tolerances of the resistivities in each case are ± 10% of the mean response, little if any bias exists between these two probe heads. Notice that HEAD = B and HEAD = E might be useful for the lower resistivity films (20 and 110), but they appear inappropriate for the more resistive films (350 and 700). In fact, considerable bias seems to exist with E for the latter two levels of resistivity, because the mean values observed appear somewhat different from those observed for the other three (Figure 2.58).

1

In this context, “linear” is used to describe the relationship between the different levels of the factor, in order. In this case, the relationship is between different heads, as each head’s resistivity increases.


Figure 2.61 Bar Chart of P/T versus HEAD and Nom. Resist.

Lessons Learned This was a very large repeatability study that contained far more observations than actually required. Since the experiments measured 49 sites on each substrate, 10 or fewer replications would have produced adequate sample sizes for estimating the final pooled variances. In fact, because the probe heads actually make light contact with the surface being measured, such extensive replication might actually damage the substrates. A later example discussing control mechanisms demonstrates how data behave when substrates degrade. Although the investigators conducting this study probably collected too much data, they did find that two of the probe heads were essentially interchangeable over the range of values likely to be measured. The other two could have some utility for certain levels, but were not as flexible as the two chosen.


No Calibration Standard Available If a traceable or verifiable standard for a particular measurement exists, then measuring that standard periodically will help decide whether a measurement tool has remained calibrated. Control charts maintained on a measurement tool (discussed in Part 3) will also help detect measurement tool drift, even if no standard is available. An additional method one might use (although it does not replace implementing and maintaining control charts) is to check a reference substrate in the measurement tool periodically. Figure 2.62 shows an excerpt of Check Calibration, no standard.jmp. In this case, engineers had preserved a particular substrate as their “golden wafer” and did not use it in routine monitoring of the measurement tool. Rather, after a set of initial readings, they measured the same locations again after 14 days.

Figure 2.62 Excerpt of Check Calibration, no standard.jmp


Examining the Data Useful first steps include generating the column of differences shown, and then checking the distributions of the data for normality. Figure 2.63 shows the distributions of each column and includes fitting each distribution with a normal curve and checking the goodness of fit for each.

Figure 2.63 Distributions of Data Based on Calibration, no standard.jmp


The goodness-of-fit tests indicate that each distribution is indistinguishable from a normal distribution. This means that conducting the matched pair analysis of comparing the mean of the differences to 0 does not require nonparametric calculations.

Matched Pair Analysis The nature of this experiment again fits the requirements for a matched pair analysis. The report for this analysis shown in Figure 2.64 indicates that the measurement tool has shifted some six units, or about 1%, and the amount is statistically significant. As before, the appropriate statistical inference occurs in the line Prob > |t|. Recall that the most common matched pair test uses the null hypothesis that the average difference between the pairs is 0. The alternative hypothesis states that the average difference is different from 0—a two-tailed test. The p-value given is the α risk taken if one rejects the null and accepts the alternative. The difference found might not have engineering significance at this point, because typical tolerance levels for a measurement of this type are ± 10%. Any continuing trend would be of concern, however.

Figure 2.64 Matched Pair Analysis Report


Lessons Learned This approach to monitoring the calibration of a measurement system is at best stopgap. It should not and cannot take the place of a formal control chart application, but it can provide initial information until engineers can formally implement a control methodology.

Exploring Alternative Analyses In the examples given to this point, the author has tried to focus attention on reasonable and relatively quick methods to determine the capability of a measurement tool. This example is a very simple repeatability test of another type of device used to measure the thickness of thin films in semiconductor manufacture—an ellipsometer. Other types of thickness-measuring equipment such as those discussed in previous sections produce variation too large to allow reliable measurements of films with thicknesses much below 100 Å. The purpose of this example is to show how apparently different approaches to analysis produce very similar results. Figure 2.65 shows an excerpt of Ellipsometer repeat.jmp. In this study, a single operator measured the center position of two wafers with different nominal thicknesses.

Figure 2.65 Excerpt of Ellipsometer repeat.jmp


Capability Analyses Generating the distributions of each wafer, and then requesting capability analyses of the data is one way to decide whether a measurement tool is capable of minimum measurement error for a particular substrate. In this case, the target value for WAFER=1 is 74 Å with lower and upper specifications, respectively, of 73 Å and 75 Å. Similarly, the target value for WAFER=2 is 85 Å with lower and upper specifications, respectively, of 84 Å and 86 Å. Figure 2.66 shows the report generated for these two sets of data. The example in the section “P/T and Capability Analysis” in this chapter discussed the meaning of the Sigma Quality statistic and its derivation. With values for both wafers well above 6 in each case, the tool is certainly capable of providing precise measurements in this range of use.

Figure 2.66 Capability Analyses from Ellipsometer repeat.jmp


Figure 2.67 Summarization of Ellipsometer Data: P/T Calculation

Summarizing Data The section “Evaluating the Combinations” in this chapter showed how to summarize data when a measurement study involves multiple sites on more than one object. The process generates a mean and variance for each group in a first step, then generates means of these results in a second step, and ultimately computes a P/T ratio. This case involves two objects, but all the measurements occurred at a single position on each wafer. Therefore, a simpler and more direct approach for calculating P/T applies and is analogous to that discussed in “P/T and Capability Analysis.” Figure 2.67 shows the result of summarizing the data using WAFER as the grouping variable and adding a column with a formula to compute P/T. By this method of analysis, this measurement tool is also initially quite capable of making this measurement.

Variance Component Analysis: Using Gage R&R The Variability/Gage Chart option on the Graphs menu can compute variance components and can include a Gage R&R report for relatively simple situations. In this example Replicate is nested within WAFER—replicate 1 for wafer 1 is different from replicate 1 for wafer 2. First one generates a chart with WAFER as the grouping variable. On the pop-up menu associated with the chart, one selects Variance Components and designates the variables Main Effects. Figure 2.68 shows the results of the analysis. The variance component for Within is the pooled variance across both wafers.


Figure 2.68 Variability Chart and Variance Component Analysis from Ellipsometer repeat.jmp

Figure 2.69 Report from Gage R&R Environment

The Gage R&R environment within the Variability Chart platform assumes that a measurement study involves operators and parts. This example does not involve an operator and parts as assumed by this option, but WAFER corresponds to the part and Replicate corresponds to the operator. Figure 2.69 shows the output of the Gage R&R analysis of this data. The value for % Gage R&R is well under the limits specified in the JMP software documentation. The report further indicates that repeatability error is negligible, and reproducibility error is minor. In this case, the reproducibility error is actually the error due to replication; the author would call this repeatability error, given the manner of


conducting the experiment. This approach applies to almost none of the other examples in this book, because they involve other than Operator and Part variables.

Using Regression Analysis A more elaborate method for computing variance components is to select the Fit Model option on the Analyze menu. This selection activates an extremely flexible regression environment in which the user can specify exactly what he or she believes is an appropriate model. Figure 2.70 shows activating the platform and setting up the model.

Figure 2.70 Setup for Regression Analysis of Ellipsometer Data

In the window shown on the right in Figure 2.70, one selects the variables (responses and predictors) from the Select Columns section at the left and places them in the model. To add a response, select that variable and then click Y. To add predictors, one selects them and clicks Add. For this analysis, selecting WAFER first is essential. Then one uses the pop-up menu at the Attributes option and designates WAFER as a Random effect. When you make this designation, the system automatically recommends REML or restricted maximum likelihood for Method. Because this data is balanced (same number of observations per WAFER), using this more powerful technique is not necessary. Figure 2.71 shows the variance components that result using REML. WAFER should be declared to be a fixed effect, but with no random effect terms in the model, there is no table of estimates of the variance components. WAFER is declared to be random just to obtain the variance component estimate and statistics for the Residual.


Figure 2.71 Variance Component Results by Regression Analysis (Fit Model)

Note also the close agreements between this method and those previously demonstrated.

Lessons Learned At least for this simple repeatability example, a variety of analysis approaches provide the same answer. The author prefers to use the summarization approach with calculation of P/T initially. Examples in later sections in this chapter will show that when this technique determines that the P/T figure of merit is marginal, then using the Fit Model approach to estimate variance components can possibly identify the source of a problem. However, it is critical that one carefully examines the raw data from any study for unusual values or trends before one uses any of these techniques.

A Repeatability Study Including Operators and Replicates Figure 2.72 contains an excerpt of the table Repeatability, Oper, Rep.jmp. In this study, two operators measured four locations on each of three wafers eight times without removing the wafer from the measurement tool. Note: The operators and wafers form a two-way table with wafers nested within operators. That is, each operator worked on a different set of wafers. The variable SITE is nested within WAFER, and MEASUREMENT within SITE.


Figure 2.72 Excerpt of Repeatability, Oper, Rep.jmp

Comments in previous examples have noted that a true repeatability study involves only measuring sites repeatedly under conditions of minimum perturbation of the measurement process. This study is more elaborate than that, but it still does not qualify (in the author’s opinion) as a true total measurement-error study, because it occurred over a relatively short time and cannot provide a satisfactory estimate of the variability in the measurement process over time. Examining the data distributions and creating a series of variability charts showed no unusual data points, although the data did exhibit considerable scatter in some cases.

Estimating a Figure of Merit: P/T Of the variables in the table, only two are truly chargeable to the measurement process— OPER and Replicate. As part of the measurement process, they contribute to variation in the process and are random effects. The others (WAFER, SITE) are fixed effects that are not part of the measurement process error, because one would expect wafers and sites on wafers to differ from one another even though the process that created the wafers intended to produce identical results. Figure 2.73 shows the first summary of this data, grouping it by WAFER and SITE. This initial summary is analogous to that shown previously in the section “Evaluating the Combinations” in this chapter.


Figure 2.73 First Summary of Repeatability, Oper, Rep.jmp with Generation of Means and Variances

Figure 2.74 shows the second summary of this table to generate the grand mean of the observations for each wafer and the pooled variances among the sites on each wafer. In this table, WAFER was the only grouping variable. Added to Figure 2.74 is the calculation of the P/T figure of merit for repeatability. The formula for this column appears in the left panel of Figure 2.74. Because this repeatability study was relatively short, the observed value of P/T is much too high to allow use of this measurement tool at this point. Therefore, the investigator must execute more sophisticated analyses in an effort to identify the cause(s) of variation.

Figure 2.74 Second Summary of Oper, Rep.jmp with Addition of Computed P/T Repeatability


Variability Charts: Gage R&R Although this example contains more than just two variables, proper manipulation of the setup for the Variability Chart platform can allow use of the Gage R&R platform. Figure 2.75 shows the setup applied to start this platform again for analysis. This study did involve operator and parts, but it also included different sites on the part and replicated measurements of those sites. Therefore, one includes SITE as a grouping variable and leaves Replicate undefined in the setup.

Figure 2.75 Setting Up a Gage R&R Analysis

This setup produces four Variability Charts arranged by the site on the object being measured. Figure 2.76 is a representative example of the charts produced. Note how the range bars for one operator appear somewhat longer than for the other; this immediately suggests that the variation obtained by one operator is somewhat larger than the other.


Figure 2.76 Variability Chart of SITE = T

Activating the Gage R&R option from the pop-up menu on each chart produces analysis summaries for each site. In each summary, the contribution of reproducibility error (OPER) dominates the analysis, suggesting that something about the operators’ handling of the measurement tool is responsible for the high variability in the response. Figure 2.77 is the analysis summary for SITE = T. Focusing on the Variance Components for Gage R&R section, we see the variance components summarized by the Gage methods.


Figure 2.77 Gage R&R Report for SITE = T

Part-to-Part variation (differences among the three wafers) is not charged to the measurement tool. Of the variation charged to the measurement tool (Var Component = 1.07), over 60% or so is due to Reproducibility (~0.64 estimates contributions due to OPER). The repeatability value is the variation due to replication in this analysis.

Fitting a Model Close examination of the structure of the original data table suggests the relationship among the variables. Although each operator measured each wafer, a nesting relationship exists among the replications, operators, and sites. Specifically, the term for Replication is Replication[WAFER, OPER, SITE], but it is confounded with the residual of the model and does not need to be included in the specification. In addition, the correct term for SITE is SITE[WAFER], because each site measured “belongs” to a particular wafer— that is, SITE = C on one wafer is different from the other SITE = C observations on the other wafers. Figure 2.78 shows the model defined for this example. First one selects the variables from the Select Columns section on the left and either adds them to the model or designates them as the response. Selecting all four of the variables with Nominal modeling type and adding them to the list of model effects in one step is convenient.


Figure 2.78 Specifying a Model for Repeatability, Oper, Rep.jmp

Of the model effects chosen, OPER is a random effect that contributes to the variation of the measurement tool, in addition to the residual, which is equivalent to the replication effect. One selects OPER, and then chooses Random Effect from the pop-up menu associated with Attributes. To create the nesting for SITE, one selects this variable in the Construct Model Effects list and WAFER from the Select Columns list. One then selects Nest to create the form shown in Figure 2.78. Running the model produces the important details, which are the estimates of the variance components of the random effects shown in Figure 2.79.

Figure 2.79 Variance Components from Repeatability, Oper, Rep.jmp


The important result from this analysis is that OPER contributes more than half of the variability seen in the data, whereas the contribution from Replicate is less. Notice the enormous range of the confidence interval reported for OPER; this range results from this variable having only 1 degree of freedom in the analysis.

Lessons Learned Further examination of the measurement tool involved in this study revealed a potential source of the problem. To use it properly, an operator had to align a flat portion of the object being measured very precisely with an index line on the tool’s measurement stage. Any lack of attention that might allow a misalignment of the object changed the actual position being measured on the object, thereby increasing its variability. However, graphical examination of the data showed that neither operator was truly to blame; both had difficulty under some circumstances in aligning wafers properly. Summarizing the raw data again, but using OPER and WAFER for grouping variables with SITE as a subgroup, produces the variance summary shown in Figure 2.80.

Figure 2.80 Summarizing Data by WAFER and OPER with SITE as a Subgroup

On the Graph menu, the Chart option enables an investigator to examine the contributions of each operator more closely. Figure 2.81 shows the setup for the chart used in this example. In this example, one chooses the four Variance(…) columns and specifies to display their data under the Statistics menu. OPER defines the horizontal axis, whereas WAFER separates the charts generated. Figure 2.82 shows the results. Although data collected by OPER = 1 might show less variation than that collected by OPER = 2, both operators had problems with variability.


Figure 2.81 Defining a Chart to Explore OPER Variability

Therefore, the solution to this problem was to retrain each operator and impress each of them with the importance of precision in placement of the wafers on the measurement stage. Then the study was repeated.

Figure 2.82 Comparisons of Operator Performance


Summary of Repeatability Studies A repeatability study best serves as an early warning that some part of the measurement process might not be suitable for an extended study if the P/T calculated for the repeatability error is much larger than 5.0. Repeatability studies can be relatively simple, as in an experiment where an observer measures a single object 30 or so times. However, more complicated studies, such as that described for the oxide study in the section “An Oxide Thickness Measurement Tool” can reveal properties of a measurement tool and a substrate that are not otherwise obvious. The key element in any repeatability study is the repeated measurements of characteristics with little or no perturbation of the measurement system while doing the measurements. Repeatability error attempts to measure the absolute best a measurement system can do, assuming that one could use it with minimum system upset over a short time. That is why the standard for P/T for repeatability is so low. More complicated studies to estimate total measurement error include additional sources of variation and are the subject of the next chapter.

C h a p t e r

3

Estimating Reproducibility and Total Measurement Error Introduction 111 Planning a Measurement Study 112 Stating the Objective 112 Identifying Potential Sources of Variation 112 Gathering the Standard Objects 114 Scheduling the Trials 114 Generating a Data Entry Form 115 Summary of Preparations for a Measurement Study 120 Analysis of Measurement Capability Studies: A First Example 120 Looking at the Data 121 Generating a Figure of Merit 123 Other Analyses 125 Lessons Learned and Summary of the First Example 127


A More Detailed Study 128 Rearranging and Examining the Data 129 Measurement Capability 131 Summary and Lessons Learned 135 Turbidity Meter Study 135 Examining the Data 136 Estimating Measurement Capability 138 Summary and Lessons Learned 139 A Thin Film Gauge Study 140 Adding a Variable 141 Reordering the Table and Examining the Data 142 Estimating Measurement Capability 143 Fitting Regression Models 147 Lessons Learned 149 A Resistivity Study 149 Examining the Data 150 Estimating a Figure of Merit 153 Fitting a Regression Model 153 Lessons Learned 154 A Final Example 155 Looking at the Data 156 Data Structure and Model Fitting 157 Comparing the Variability Due to OPERATOR 159 Summary and Lessons Learned 160 Summary of Measurement Capability Analyses 162

Chapter 3: Estimating Reproducibility and Total Measurement Error 111

Introduction This book could be considered at odds with some of the literature on measurement capability studies. Chapter 1 of this book indicated that perceived process variation (Equation 1.1) is a combination of two elements: repeatability error and reproducibility error. As the term is used here, repeatability error is that variance associated with operating a measurement process under conditions of minimum perturbation over a relatively short time. Generally speaking, repeatability error is an estimate of measurement tool error under the best possible measurement conditions; it represents approximately the best a measurement device can accomplish. Reproducibility error is a variance associated with more normal use of a measurement process—variation among operators, day-to-day, or perhaps part-to-part under some circumstances. This error represents an estimate of variation that accounts for variations in the measurement process due to any and all outside sources other than repeatability. When combined, these two variances estimate the overall performance of the total measurement system. Common practices in semiconductor manufacturing might combine these studies into a relatively long-lived study and determine the overall or total measurement error associated with a system. In the previous chapter, the author made the point, however, that sometimes a separate and relatively short repeatability study can identify immediate problems with a measurement device that one should correct before embarking on a more involved study. Most of the examples given in this chapter follow a brief repeatability study and might include a repeatability segment combined with a reproducibility segment in order to compute total measurement error. These larger studies can also include a variety of different objects to measure, thus leading to additional estimates of bias and linearity in a measurement process. In a practical sense, the investigator is most concerned with the total measurement error in a system. If that variation is acceptable, then no further investigation of the measurement process is necessary, provided one maintains suitable control mechanisms that monitor the measurement process. Only when total measurement error is excessive does the investigator need to separate the total into its components to identify the source of the excessive variation and to plan corrective action. Chapter 1 introduced the P/T ratio as a figure of merit for a measurement tool and stated that values ≤ 30 for this ratio generally indicate that the variation contributed by the measurement system does not significantly affect the perceived variation in the process.


Planning a Measurement Study Because a measurement study is a disciplined investigation of a process, the elements of planning an experiment of this type are analogous to those involved in planning a formal experiment intended to understand or to optimize a manufacturing process. In this case, the initial objective is not so much to improve the measurement process as it is to quantify it. Each measurement study should contain at least the following elements:

stating the objective

identifying potential sources of variation

gathering the standard objects

scheduling the trials

generating a data entry form

Stating the Objective One should take the time to define the objective of any study in enough detail that later examination of any documents associated with the study will differentiate it from other studies. A poor objective statement: Run gage capability study on XYZ measurement tool. A better objective statement: Identify and quantify sources of variation in measuring silicon oxide films using the Prometrix 500, ID xxxxx.

Identifying Potential Sources of Variation The key to understanding any measurement process is understanding how the process works. For example:

Does the tool require manual positioning of a single object for measurement or does it position objects automatically?

Does the tool allow loading of a cassette of wafers with automated wafer handling?


What range of responses (thickness, resistivity, refractive index, etc.) should the plan include?

How many operators use this tool? What measurement operations might be operator-sensitive?

Does the measurement process degrade the object being measured in any way?

How many sites should the plan consider on each object being measured?

Are external, traceable standards available?

Are appropriate internal standards available?

How long can this experiment continue without materially affecting normal manufacturing routines?

What parameters does the tool collect? Is automated data handling available?

If a measurement tool is totally automated, such that an operator removes a container of samples from a processing tool and simply places it in the measurement tool without making any adjustments or decisions, then finding an operator effect in that measurement process is unlikely. This fact does not mean that one should not include operators as a variable; it means that the likelihood of finding a major effect due to operators is small in this case. Consider carefully how many operators are likely to use this tool; in some cases operators might work specified shifts, so execution of shifts might confound any operator effect. Considering the option of measuring more than a single response on each object is certainly reasonable, depending on the complexity of the data being gathered. Previous examples in Chapter 2 illustrated measuring two or more items in a single study. Sometimes a measurement tool might damage the object being measured in some manner as a study repeats measurements over some period of time. Such a possibility dictates starting a study with multiple objects that have very similar properties. Then one stores and preserves all but one of the objects for future use and continues the study using only one. If the one being used degrades, then one substitutes a new one, taking into account any difference between objects in the analysis. In many manufacturing operations, standards traceable to NIST might be available. If so, make certain to have more than one such standard in case the one being used becomes damaged or contaminated. Otherwise, a study must use samples taken from production as relative reference points. A study of measurement error cannot be so involved or long-lived that it materially affects manufacturing production. A measurement study estimates variances, so the plan should provide at least 30 observations of each source of variation. Less than that number (refer to Chapter 1) leads to lower confidence in the estimates being generated. Because a good study should involve a significant time interval, the author has always


recommended that those executing the plan should scatter measurement episodes over a period of four to six weeks, without necessarily conducting portions of the study on consecutive days. Nor does each operator (if operator is a variable) have to measure each item every day. The important concept is to generate enough samples of each source of variation to allow confident estimates of the variation being observed. Finally, the most modern measurement devices have computer interfaces that automatically log the observations as collected. If this logging is available, then one source of error—data entry—can be avoided.

Gathering the Standard Objects Each measurement study might involve one or more standards that represent the expected spread of parameter values needed to use the tool. Because many measurement processes involve physical contact with the sample surface, degradation of that surface over the course of the actual study and later control chart measurements is possible. Therefore, at the start of a measurement study, one should gather more than one sample of each type of standard (at least three). Preferably, these should be traceable primary standards. If such standards are not available, then one should choose representative samples from the manufacturing operation. At the start of the study, each operator should measure all standards, including the duplicates. After the first day, all but one of each standard should be secured, and the study continued on a single example of each. If and when the sample being used is lost or damaged, one should replace it with one of those held in reserve. Minor differences will exist in the actual parameter values among similar standards, but the variance of the measurement tool should be independent of small differences among samples. Subsequent analyses can adjust for any large differences found by using techniques similar to blocking in more conventional process experiments.

Scheduling the Trials A measurement capability study estimates variances. As such it must contain adequate sample sizes for variables to provide reasonable precision in that estimate (minimum confidence interval width). An effective measurement capability study does not require every operator to measure each standard object every day. Although the study should involve a reasonable time period to establish measurement tool stability and control (up to several weeks), measurements need not occur every day during that period.


Rather, the objective should be to have up to 30 measurement episodes occur per operator over this period. That means each operator should work through the measurement study protocol no more than about 30 times during the time of study.

Generating a Data Entry Form In conducting measurement studies, the author has found that generating a JMP data table containing a column for each variable involved and a column for each response measured is a useful and flexible approach. Alternatively, the investigator might generate a more formal matrix of settings using the DOE options in JMP. The matrices associated with a measurement study actually belong to a class of full factorial designs. Because each factor in the design matrix might involve many more than two levels, the best design matrix supported by JMP is the Custom Design, as it allows essentially any number of levels for factors. Figure 3.1 shows how to open this platform along with the window that appears.

Figure 3.1 Custom Design Platform and Opening Window


For this example, assume that the measurement study will require 30 days (or measurement times), with three operators making thickness measurements at five sites on each of four objects with different nominal thicknesses. The operators repeat each set of observations four times. The simple factorial combination of all these factors indicates that the study will generate as many as 30 x 3 x 5 x 4 x 3 or 5400 observations if each operator makes all the required observations on each object as specified. To set up this experiment using the Custom Design feature in JMP, one starts by considering the actual structure of the experiment. Figure 3.2 should help understand this structure for a given measurement day.

Figure 3.2 Measurement Study Proposed Structure

On each of 30 measurement days (not necessarily the same days for each operator), the three operators should process each of the four objects for study three times, reading five sites each time. NOTE: Because of space limitations, the figure does not show the five sites under each replicate for each operator. Also, only three of the four replications are shown.


This structure makes logical sense in that this is the way the operators will execute the experiment. Therefore, in setting up this experiment in JMP, one defines the factors in this order: Day, Operator, Object ID, Rep, and Measurement Site. The Custom Design platform will thus provide for each setting of each factor in the resulting matrix. Each variable is declared Categorical. Figure 3.3 shows the final description of this experiment to JMP, including the definition of the response to measure Thickness.

Figure 3.3 Describing the Measurement Study to JMP

The levels for Day are 1, 2…30. When an investigator conducts the measurements, he or she should fill in the actual date in that column using the format mm/dd/yyyy. JMP recognizes this date format and allows changing the column information for this variable when the experiment has concluded and all data has been entered. NOTE: Explaining each nuance of each window in JMP DOE is beyond the scope of this book. This discussion attempts to provide enough information to allow successful generation of the required matrix for creating a data entry form for a measurement study of this type. Clicking Continue on the Custom Design window continues the process for generating the design matrix and brings up the window in Figure 3.4. JMP uses an optimization algorithm by default to generate the design matrix, producing a 120-row matrix if allowed. However, for a measurement study, one selects the Grid option to generate the expected factorial combination of settings for each factor.


Figure 3.4 Choosing the Number of Rows and Generating the Design Grid

Clicking Make Design activates the underlying algorithm and generates the matrix. In this case, the process takes a few minutes because of the size of the matrix grid. JMP always randomizes a design matrix, because that is the best practice in generating experimental designs. However, randomization will complicate execution of this experiment. At the bottom of the display of the design matrix (left panel in Figure 3.5) is an option to sort the resulting design table before creating the table for data entry. Using the option arrow associated with Run Order, one can choose to sort the current design matrix and the table generated from left to right. Clicking Make Table completes the process.

Figure 3.5 Sorting the Design Matrix, Option to Simulate Responses


An additional option available on the pop-up menu on this window simulates responses for this experiment. This option enables the user to experiment with the model associated with the generated table to explore the degrees of freedom available for each parameter being estimated. This activity is beyond the scope of this discussion and will not be demonstrated. Figure 3.6 is an excerpt from the table Dummy Measurement Study.jmp generated in this example. Note the sorting of the variables.

Figure 3.6 Excerpt of Dummy Measurement Study.jmp

The user can shorten this table somewhat by selecting Split on the Tables menu; this option is the reverse of the Stacking option demonstrated in previous sections. One selects Thickness as the column to split, with Measurement Site providing the split column labels. This produces a 1080-row table with the observations arranged horizontally after the Replicate ID. However, to analyze the data from this study, one must use a table like that in Figure 3.6. NOTE: In addition, the author saved descriptions of the responses and factors in this example as Dummy Study Responses.jmp and Dummy Study Factors.jmp, respectively, using that option on the menu shown in the right panel of Figure 3.5.


Summary of Preparations for a Measurement Study Previous sections have tried to emphasize the importance of proper planning in setting up a measurement study. One of the main concerns is to generate a study that provides adequate degrees of freedom (DF) for each factor investigated to allow more precise estimation of the variances involved. The author has never used a design-generation approach in setting up a measurement study. Instead, his approach has always been to generate a table with the appropriate variables identified. All the examples in this chapter use that approach.

Analysis of Measurement Capability Studies: A First Example As stated in previous sections, the most important differences between the repeatability studies previously discussed and a study intended to establish overall capability of a measurement tool are the length of time of the study and the number of variables included. Any measurement study provides an estimate of a tool’s capability, but only a study that continues for a reasonable length of time can indicate whether a tool will be dependable in manufacturing use. Figure 3.7 shows an excerpt from 4 objects 9 sites.jmp, which shows data from a simple form of an extended study. In this study, a single operator measured 9 locations on each of four objects with deposited films on their surfaces (nominally 800, 1200, 1800, and 2100 Å thick) over a period of approximately one month. Approximately 15 measurement days occurred during the period of the study; measurements did not necessarily occur on consecutive days.


Figure 3.7 Excerpt of Data from 4 objects 9 sites.jmp

The data table contains two fixed effects—WAFER ID and Measurement Site; the only random effect in the study was variation with time. This study produces an estimate of the overall error in the measurement system, but the analysis combines any error due to repeatability with reproducibility because the data contains no replications on any measurement day.

Looking at the Data The distributions of the data grouped by WAFER ID appear in Figure 3.8. Brushing the apparent outlier values associated with WAFER ID = 1200 and examining the original data table showed that the outlier values were all from measurement site 1. Generating a variability chart using DATE and Measurement Site as the X, Grouping variables, and segregating the data by WAFER ID (Figure 3.9) produced four charts.


Figure 3.8 Distributions of Data from 4 objects 9 sites.jmp Grouped by WAFER ID

Figure 3.9 Setting Up Variability Charts from 4 objects 9 sites.jmp


Reproduced in Figure 3.10 is the variability chart for WAFER ID = 1200. This chart confirms that measurement site 1 generated the unusual values on each day. Historically, this measurement site was not known to produce such values due to the debris from laser scribing of the object, as was the case in a previous example. In addition, the unusual points differed from the bulk of the points by only 1% or so. No compelling reason existed to exclude them from the analysis at this point, so they remained in the data.

Figure 3.10 Variability Chart for WAFER ID = 1200 from 4 objects 9 sites.jmp

Generating a Figure of Merit As stated earlier, generating an immediate figure of merit such as P/T for a measurement tool can provide satisfactory information about its performance. If the measurement tool passes this initial test easily, then further analysis is not really necessary. Summarizing the data using the two fixed effects as grouping variables and calculating the mean and variance of THICK produces the table in Figure 3.11 (excerpt of result). This table is available as 4 objects 9 sites first summary.jmp.


Figure 3.11 First Summary of Raw Data for This Example

The next step summarizes the summary table to produce a grand mean per WAFER ID along with the pooled variance of the observations. Added to Figure 3.12 is the calculation of P/T for each WAFER ID in this study. The formula for this calculation appears in the lower panel of Figure 3.12. This table is available as 4 objects 9 sites second summary.jmp.

Figure 3.12 Second Summary of Data with Computation of P/T


Because the effect of P/T on process capability is negligible when P/T < 30, this study shows that this tool is quite capable of measuring the samples chosen. In addition, the fact that the P/T ratio is reasonably stable throughout this range of measurements suggests that no linearity problems exist with the system either. At this point a more complicated or sophisticated analysis of this data is not necessary.

Other Analyses An alternative analysis that applies in this case is to determine the Sigma Quality level of this system by generating the capability analyses associated with each group of data, assuming that the specification limits are ± 10% of the observed mean. Figure 3.13 shows the worst case from this study corresponding to WAFER ID = 800. This data indicates an overall Sigma Quality of 6.694. From the report table, one sees that approximately 1 measurement in 10 million would be likely to fall outside the upper specification limits. All the other samples had Sigma Quality > 9.

Figure 3.13 Capability Analysis for WAFER ID = 800


Another view of the data involves summarizing the raw data and generating the mean thickness using DATE and WAFER ID as grouping variables. The summary table is available as 4 Objects 9 sites by (DATE, WAFER ID).jmp. Plotting the mean thickness versus DATE, grouped by WAFER ID, and fitting a line to the result can help detect any drift in the observations. Figure 3.14 shows setting up the Fit Y by X platform found on the Analyze menu for this exercise.

Figure 3.14 Setting Up the Fit Y by X Platform

Figure 3.15 contains the results of fitting a line to each graph. In the figure, the slopes of the lines for samples 800 and 1200 are statistically significant, indicating that the measurement tool was drifting slightly during the study. However, the drift in the mean was somewhat less than 1% in the worst case (800). Although such a drift does not have engineering significance at this point, monitoring the average in all cases with control charts is certainly advisable.


Figure 3.15 Bivariate Plots of Mean(THICK) versus DATE with Fitted Lines

Lessons Learned and Summary of the First Example This study lacked sophistication, but it did demonstrate that the measurement tool being studied was capable of the analyses being performed over the range used and that the variability of the measurement tool was stable over the range of samples tested. However, engineers should continue to monitor the statistically significant drift observed in the measurements for two of the samples and should include the other samples as well.


A More Detailed Study The most rigorous form of total measurement study has the following characteristics:

It requires a reasonable amount of time (usually several weeks of observations).

It might include some form of short-term measurement replication to observe how repeatability might or might not change during the period of the study.

It might include more than one standard wafer (incorporating some aspects of a linearity study into the total study).

Over the period of the study, one need not measure each of the standard objects every day. A sensible approach is to have each operator measure each of the objects in the set according to some pre-established protocol on at least 15 or 20 occasions during the period of the study. For example, if a measurement study is to last six weeks (42 working days, assuming a 7-day schedule), then each operator should measure the samples on at least 20 different days, or approximately every other day. One should collect information on at least 10 different days during the period to give a reasonable estimate of how the measurement tool (mean observation) varies with time. Figure 3.16 shows an excerpt of data from such a comprehensive measurement study (Oxide Study.jmp). In this case, the study lasted only two weeks and involved the operators on each of three shifts (during this period a single operator worked each particular shift). The study involved measuring 49 sites on a single wafer five times each shift. The data display is similar to that in a previous example, with the observations for the 49 sites arranged in separate columns as is typical for an automated data collection program.

Figure 3.16 Excerpt of Data from Oxide Study.jmp


Rearranging and Examining the Data As illustrated earlier, the first step is to convert the data table to a flat file such that each variable has its own column. Variables in the original table include DATE, SHIFT, REP, MEASUREMENT SITE, and THICKNESS. In this case, the variable MEASUREMENT SITE is in the column headings 1 to 49, whereas THICKNESS is the data entered in those columns. The option to Stack on the Tables menu is the utility to use. Figure 3.17 shows how to set up this operation.

Figure 3.17 Setup to Stack the Table Oxide Study.jmp

Figure 3.18 is an excerpt of Oxide study stack.jmp as produced by the system. In setting up this action, the Stacked Data Column is THICK, and the Source Label Column is SITE. The system default treats SITE as a character variable, because it came from column headings; one should change that to Numeric, but leave the modeling type Nominal.


Figure 3.18 Excerpt of Oxide study stack.jmp

The investigator can use any of several options for examining the data in this table. Generating a distribution of the data from so large a sample produces histograms and outlier plots that are extremely hard to interpret. Brushing the points on the outlier graph or on the histogram identifies measurement sites that produce extreme values. In the author’s opinion, the best and most efficient approach generates a variability chart using THICK as the Y variable and SITE as the X-grouping variable. Figure 3.19 shows the graph generated in this manner.

Figure 3.19 MultivariGraph for THICK, grouped by SITE


This example again shows the problem associated with SITE = 38 on the surface of the object. As stated in an earlier example, the most likely cause of this anomaly is debris from the laser-scribing process used to place an ID number on the wafer surface. Therefore, the most prudent approach is to remove SITE = 38 data from the database and proceed with the analysis. Removing these points uses the same process demonstrated in the section “Excluding the Questionable Values” in Chapter 2. Creating a selection statement for SITE = 38 and then excluding these rows from the data table removes 145 data entries. Creating a second multivarigraph using the same parameters as for the graph in Figure 3.19 shows that the maximum standard deviation for repeated measurements on any SITE is about 2, compared to approximately 25 previously.

Figure 3.20 Initial Summary of Oxide study stack with selection and exclusion.jmp

Measurement Capability An initial summary of the modified data (SITE = 38 excluded) produces the results in Figure 3.20, showing the mean and variance of THICK by SITE. Earlier examples illustrated a second summarization followed by calculation of the P/T ratio for the measurement tool. An alternative approach creates the column means for the columns Mean(THICK) and Variance(THICK). Figure 3.21 shows how to create the formulas to generate a column mean, a pooled variance, and P/T. Figure 3.23 shows the finished table.


Figure 3.21 Generating Formulas for Grand Mean, Pooled Variance, and P/T Columns

The P/T value computed in Figure 3.22 indicates that the measurement tool and process is certainly capable of measuring these objects precisely. In the original data table, the operator performed five replicated measurements on each day, so the opportunity exists to separate repeatability error from reproducibility error in the total error represented by the pooled variance in Figure 3.22. A sound approach for this calculation is to fit a model to the data (still excluding the points from SITE = 38).

Figure 3.22 Results for Oxide Study


Figure 3.23 shows the construction of two possible models for this data.

Figure 3.23 Models of Varying Complexity to Fit the Data in Oxide study stack.jmp (after Removing Data for SITE = 38)

The model in the left window is somewhat simpler than that in the right and is the one the author prefers. NOTE: The modeling type for DATE has been changed from Continuous to Nominal. Computation of variance components does not allow continuous nested variables. The model on the right includes other two-factor interaction terms not involved in the specified nested relationship. Because the data table contains four variables, the model could include six two-factor interactions (if one eliminates the nested relationship), six three-factor interactions, and one four-factor interaction. Because DATE, SHIFT, and REP are designated random effects, all interactions involving them are also random effects by definition. The variable SITE is a fixed effect that is characteristic of the object being measured and not chargeable to the measurement process. Figure 3.24 shows the result of the analysis using REML (Restricted Maximum Likelihood), the default or recommended choice. The result in the top panel is the one the author would normally use. The residual contains all the other random effects associated with interaction terms not formally included in the model. Similarly, the result in the bottom panel has partitioned the residual to account for the three interaction terms added to the model. Compare the Total entry in each model to the pooled variance estimate found from the shortcut approach illustrated in Figure 3.22: 1.107 versus 1.160 or 1.153. As stated earlier, the shortcut method underestimates the actual total variance by an amount depending on the complexity of the experiment.


Figure 3.24 Analysis Results for Models in Figure 3.17

In Figure 3.24, the repeatability error is that variance component associated with REP[DATE, SHIFT]: 0.482 in the first panel and 0.350 in the second. Reproducibility error is the sum of the variance components of the other entries in each panel—for the first panel: 0.0997 + 0.0436 + 0.5352 = 0.6785. The value differs a bit in the second panel due to the further partitioning of the variation by the interaction terms. A further useful view of the data graphs the mean of the observations versus the observation date to provide an impression of the stability of the measurement process and tool. Figure 3.25 shows that overlay plot with a reference line added for the grand mean of the observations. This trend chart could be a precursor for a formal control chart. If this data is used to create an initial control chart, the investigator should be aware that the sample size (number of days) is relatively small. This means that any control limits derived from the data would contain considerable uncertainty. Chapters 6 and 7 discuss control charts in more detail.


Figure 3.25 Observation Mean versus Date

Summary and Lessons Learned Although this measurement study did not continue for the recommended 30 days or so, the measurement tool proved capable of making the measurement investigated. More detailed analysis of the data using random effects in a regression analysis indicated that neither DATE nor SHIFT (Operator) were major contributors to the variation. The single most important contribution was the repeatability error.

Turbidity Meter Study This example is an extension of the repeatability study on this instrument discussed in the section “Evaluating a Turbidity Meter” in Chapter 2. As originally designed, this study was to require 30 working days, but was curtailed due to operating circumstances after 23 days. On each day, the operator made five repeated measurements on each of two samples. One was Stabcal < 0.10, a standard traceable to NIST; the other was Gelcal < 10, a secondary standard. Figure 3.26 contains an excerpt of the data from Turbidity 2100P Capability Test.jmp. Two operators actually conducted the test, but they did not identify themselves in entering the data. Because the meter is highly automated and simply requires the user to insert a sample cell into it, this is probably not a major concern. However, this omission prevents the study of any operator effect.


Figure 3.26 Data from Turbidity 2100P Capability Test.jmp

Examining the Data Graphing the distributions of the data as well as preparing variability charts for each set using Date and Rep as the X-grouping variables provides initial information. Figure 3.27 shows the histograms, and Figure 3.28 shows the variability charts. Figure 3.27 suggests that outliers exist in the data for Gelcal < 10. The variability chart in Figure 3.28 confirms that data from two separate days are relatively low, compared to the bulk of the observations for this sample. It also indicates a downward trend in the values for each replicate for this sample on each day (top panel in Figure 3.28). The range of the trend is relatively small (< 0.1 units in all cases), but its presence suggests that the tool might not have been stable. In conducting the experiment, the operators always measured the five replicates on this sample first, followed by the other sample, in order to avoid excessive handling of the standards. At this stage, no compelling reason exists to remove the outlier values on the two days.


Figure 3.27 Histograms of Raw Data

Figure 3.28 Variability Charts of the Raw Data


Estimating Measurement Capability Applying the summarization technique to provide P/T values for each case produces the result in Figure 3.29. EPA regulations allow up to 2.0 NTU for water supplied to a system, so the specification range used for each set of data was 0 to 2. Calculating the variance of each sample allows comparison of the results from this method with the more rigorous analysis that follows.

Figure 3.29 Calculation of P/T for Turbidity Data

Excluding the two days of unusually low values observed for Gelcal < 10 lowers the calculated P/T for that sample to ~ 10.3, but in the absence of a true assignable cause for doing so, the analysis should stand as illustrated. To determine the partitioning of the variances between repeatability and reproducibility for both samples, one fits models to each response. The full model that applies in each case involves random effects Date and Rep[Date]. However, using the full model for the analysis leaves no degrees of freedom for Residual and confounds Rep[Date] with the Residual. Therefore, the author elected to use only the random effect Date in the model (the reproducibility) and let the residual contain the other term (the repeatability). Figure 3.30 shows the results from the analyses for both samples.


Figure 3.30 Results of Analysis Using REML

The total variance reported in each case agrees well with the values shown in Figure 3.29. For Gelcal < 10, the reproducibility error accounts for 91% of the total variation in the data. This analysis included the low values observed earlier. The slight drift observed in the data for this sample (repeatability) does not play as important a role. If the analysis excludes the low values observed, the reproducibility error drops to about 74% of the total—that is, in the absence of the unusually low values, the drift observed (repeatability) becomes relatively larger. For Stabcal < 0.10, the relative contributions of reproducibility and repeatability are very similar. The confidence interval reported for the random effect of Date in both cases reflects the sample size used (23 observations).

Summary and Lessons Learned The discussion of evaluating the turbidity meter in Chapter 2 noted that this device suffers from “inadequate measurement units” when used with low turbidity samples. The same is true here. As noted, EPA standards require turbidity values of no more than 2.0 NTU for drinking water; therefore, using a specification range of two units for computing the figure of merit was a reasonable approach for the samples. Because the low turbidity standard is the critical one, the data collected in this example for the low turbidity standard samples could become a source for a preliminary control chart for this measurement process.


A Thin Film Gauge Study Figure 3.31 presents an excerpt from Thin film gage study.jmp. Careful examination of the table indicates that two operators measured the same object at four locations over a period of about three weeks. In some cases, an operator measured the object more than once on a given day. Note also that the operators recorded not only the date, but also the time of their measurements, and that the table will require stacking before an analysis of the results can occur.

Figure 3.31 Excerpt of Thin film gage study.jmp

The original plan called for each operator to measure the standard wafer at four sites at least twice a day during the period. Examination of the data table shows that one operator managed to make more than one measurement on several days, but that the other did not achieve that objective until near the end of the experiment. The analysis could consider the repeated observations by operators on a particular day as replications, but the observations did not occur close together in time. To estimate repeatability, the replications should occur at close periods in time with minimum disturbance of the measurement process. Because the experiment includes multiple observations by different operators on different days, some instances of nested variables exist in this data table.


Adding a Variable One method for dealing with the multiple measurements by an operator on a given day is to generate a new variable in the table. Identifying the measurement by DATE, TIME, and OPER is also possible, as is the merging of the entries in the date and time columns into one column. The author believes the approach shown here is simpler and provides graphs that might be easier to interpret. To add the new variable, a count of the observations, one inserts a new column into the table and names it TRIAL. With that column highlighted, one uses the formula editor to create a count of integers from 1 to 30, as illustrated in Figure 3.32. Reordering the columns to place the new column after OPER is useful, but not absolutely necessary.

Figure 3.32 Adding a Count Column to the Data Table


Reordering the Table and Examining the Data One then uses the Stack option to create a flat file table structure suitable for analysis. In this case, one selects the columns TOP, CEN, LFT, and RGH to stack. The stacked data column becomes THICK, and the ID column is SITE. Figure 3.33 shows the new stacked data table (after having moved the TRIAL column to the position after OPER).

Figure 3.33 Stacked Data Table

The distribution of THICK suggests both high and low outlier values, but a variability chart of THICK grouped by OPER and TRIAL provides more information. Figure 3.34 shows that graph. Each operator performed the same number of measurements (15) during the study, but the data collected by OPER = CP is stable and shows no excursions in the mean value. However, the data from OPER = RSJ shows considerable variation in the mean and is likely responsible for the outlier values observed in the distribution of the data (brushing of data points would confirm that assertion). However, the standard deviation chart at the bottom of the figure shows that both operators had about the same variability among the sites being measured in each trial.


Figure 3.34 Variability Chart of Thickness Grouped by OPER and TRIAL

Notice also that OPER = CP performed all the measurements during the last several days of the study.

Estimating Measurement Capability Calculating the P/T figure of merit using the summarization techniques illustrated in previous sections provides a ready estimate of measurement capability. In this example, the only variable not chargeable to the measurement tool is SITE. Figure 3.35 shows the results.


Figure 3.35 Estimate of Measurement Capability for This Example

A value for P/T that is so much larger than 30 clearly indicates that excessive variation is present in the measurement process. Therefore, the next step is to identify the source or sources and develop a plan of action. Figure 3.34 suggests the source: at least three trials performed by OPER = RSJ produced measurements that varied considerably from the apparent overall measurement average. This study did not involve multiple parts, but suitable manipulation of the information provided in generating the variability chart can allow the use of the GageR&R function. Figure 3.36 shows how to set up the chart for this example; Figure 3.37 shows the resulting variability chart. This graphic makes it extremely clear that the measurements taken by operator RSJ contain substantially more variability than those from operator CP. Requesting the GageR&R function and specifying a tolerance range of four units produces the chart in Figure 3.37.

Figure 3.36 Setting Up a Variability Chart for GageR&R


Figure 3.37 Initial Variability Chart

In Figure 3.38, the Repeatability error is the variation within repeated measurements of the same site. Reproducibility is a measure of the variance between the means each operator found for each site; on average both operators found the same mean thickness at each site, but one had considerably more variability. The repeatability error reported here is at odds with the definition provided earlier in that the repeated measurements did not occur close together under conditions of minimum perturbation of the measurement device. In fact, this study contained no true replicates, although the operators each made an average of two measurements on each of several days.


Figure 3.38 Gage R&R report

To illustrate these results another way, one might summarize the table in Figure 3.33 again using SITE as the grouping variable and OPER as the subgroup. To the resulting table, one adds column means for each generated column to provide the grand mean and pooled variance for each operator. Finally, one generates new columns calculating the P/T figure of merit obtained by each operator. Figure 3.39 shows the results after reordering the new columns to associate them with a particular operator. Notice the relative value of the pooled variances for each operator, although each operator obtained essentially the same average value for the sites. The large pooled variance for OPER = RSJ results in an extremely large P/T value.


Figure 3.39 Calculation of P/T for Each Operator

Fitting Regression Models The graphics and calculations presented in the previous sections clearly show that one operator was imprecise in making many of the measurements in this study. That the effect of operator on the observed thickness was negligible indicates that at each measurement site, this operator found enough of both high and low thickness values to obtain the same approximate average as the other operator. The tool being investigated was the same tool described earlier in the section “A Repeatability Study Including Operators and Replicates” in Chapter 2. In that case, both operators had difficulty aligning the object correctly on the measurement stage. In this study, one of them had learned the technique very well, whereas the other was still learning. To quantify the sources of variation in a study like this and to provide a comparison with the shortcut method illustrated earlier in Figure 3.35, the author prefers to fit a regression model to the data after specifying a model carefully. Figure 3.34 definitely indicates a nested relationship between operator and trial: of the 30 trials made, 15 belong to one operator and 15 to the other. Figure 3.40 shows how to define the model and the results of running the analysis.


Figure 3.40 Defining a Regression Model and the Results of the Analysis

NOTE: The estimates of the variances for each operator from Figure 3.39 differed so much that they could cause some question about the validity of the Residual in the regression report of Figure 3.40. This experiment involves three factors: OPER, TRIAL, and SITE. TRIAL uniquely defines both DATE and TIME, so including them in the model is not necessary. Of the three factors remaining, two of them legitimately contribute to the variation observed in the measurement process—OPER and TRIAL[OPER]. These factors are random effects (sources of variation by this definition). SITE is an artifact of the object being measured; therefore, it remains a fixed effect. Not specifying TRIAL[OPER] in the model allows its contribution to become part of the residual along with other random effects not specified at this point. The analysis finds that Residual composed mainly of TRIAL[OPER] is the major contributor to variation in the process. The total random error found is 2.388 units, compared to 2.365 found as the pooled variance in Figure 3.35.


Lessons Learned This study demonstrates again that any measurement process that depends on the training or ability of an operator is likely to develop some form of operator or trial effect. In this case, one operator obviously was better trained or more careful in executing the measurements than the other. The measurement tool was quite capable of measuring this object, provided the operator using it was properly trained and precise in his or her activities. This example also presented a variety of approaches to analyzing the results. Other than needing to put a figure of merit on the process, the graphics generated defined the problem quite well. Forcing the system to allow the GageR&R approach for this analysis required some questionable assumptions, given the constraints associated with that system.

A Resistivity Study Table RS Measurement.jmp is a lengthy study dedicated to characterizing a resistivity measurement tool that reports surface resistivity of conductive coatings (ohms/square or Ω/). This study involved three shifts of operators over approximately a three-week period. The original plan called for an operator to measure a 49-site pattern on an object five times each shift without removing the object from the measurement tool. Figure 3.41 presents an excerpt of this data table.

Figure 3.41 Excerpt of RS Measurement.jmp


Examining the Data Careful examination of this data table shows an elaborate time stamp format for when an operator conducted the measurements. Sets of readings generally occurred in groups of five with apparently one exception. Had all the replications been done, then the number of rows in the table should be 155 instead of 154 as shown in Figure 3.41; therefore, a row must be missing. To account for the replications, the author added the column REP to the table and entered numbers 1 to 5 while scrolling down the new column. Figure 3.42 shows an excerpt of this result. Shift 2 performed only four replications on October 26.

Figure 3.42 Excerpt of RS Measurement.jmp with REP Column Added

Further examination of the table shows that occasionally a shift performed more than one set of measurements on a given day. In analyzing the data, a person could use the dateand-time stamp associated with each row, but the author chose to add another new column named TRIAL and fill in numbers corresponding to a change in the date-and-time stamp. Figure 3.43 shows an excerpt of these results.


Figure 3.43 Excerpt of RS Measurement with REP.jmp after Adding TRIAL Column

Notice that Shift = 1 performed two sets of measurements (TRIAL = 2, 3) on October 23. Before any analysis can begin, the user must convert this table to a flat file using the Stack option on the Tables menu. Figure 3.44 shows this conversion. The system designates the column headings that become the entries in the SITE column Character, Nominal, so the user should change them to Numeric, Nominal to make any graphs generated easier to interpret.

Figure 3.44 Excerpt of RS Measurement stack.jmp


With the table is in the proper format, a useful next step generates the distribution of values for RS (Figure 3.45). The figure suggests a number of low outlier values, so a following step should generate a variability chart of RS using SITE as the grouping variable (Figure 3.46).

Figure 3.45 Distribution of Resistivity

Figure 3.46 Variability Chart of Resistivity Grouped by SITE

Although the data shown in Figure 3.46 contains a few unusual values at some sites, the apparent outlier values show no specific pattern. The standard deviation graph that accompanies the variability chart suggests that none of the sites have particularly large variations in measurements. Although a pattern of this sort might suggest some issues with the process that deposited the film (within wafer variation), the graph shows no


indication of unusual values associated with any particular site as was the case with previous examples.

Estimating a Figure of Merit Although the data apparently contains no anomalies, the next logical step is to compute a figure of merit (P/T) for the measurement process. Of the variables in the table, only SITE is not chargeable to the measurement process, because it is an artifact of the object being measured. Therefore, one summarizes the data with SITE as a grouping variable and computes the mean and variance at each SITE. To the resulting table, one adds new columns to compute the grand mean of the measurements, the pooled variance (mean of the variance column), and the P/T figure of merit, using a tolerance of ± 10% of the grand mean. Figure 3.47 shows this result. The observed value for P/T is well within the guidelines of < 30, so no further work is absolutely necessary at this point. However, the study did contain sets of replications that allow computation of a repeatability error and separation of that error from reproducibility.

Figure 3.47 Summarizing Data and Computing P/T

Fitting a Regression Model Figure 3.48 shows the model statement for this example. Notice that SITE is a fixed effect, whereas Shift, TRIAL, and REP are all random effects. Note also the more complicated nesting for REP. To fully describe a particular replication, a person must identify the TRIAL and Shift where it occurred. Figure 3.49 shows the regression results.


Figure 3.48 Specifying a Model

Figure 3.49 Regression Results

Compare the total random error in Figure 3.49 with the pooled variance found in Figure 3.47. Of the variability in the data, < 1% of it is due to replication (repeatability). The rest is reproducibility error associated with day-to-day and shift-to-shift variability in measurements. Included in the residual are second-, third-, and even fourth-order interactions that were not included in the statements of the model.

Lessons Learned The measurement tool investigated in this study was highly automated, requiring little activity by operators other than loading the feed mechanism for the tool and selecting the correct programs. Even data logging was automated, eliminating the possibility of human error in entering data.


A Final Example Figure 3.50 presents an excerpt of the file FTIR Study.jmp. This investigation required considerable effort on the part of the two operators involved, in that the results given were not the result of any automated data collection scheme. The devices (three separate objects) are infrared sensors used in a variety of missile systems in the aerospace industry.

Figure 3.50 Excerpt of FTIR Study.jmp

The test instrument applies a voltage to the devices that results in a charted output such as that depicted in Figure 3.51. From the curve produced, the operator projected a line back to the horizontal axis and determined the intercept, recording it as E0 (the values in the columns labeled 1 to 4 in Figure 3.50). E0 represents a sensitivity threshold for the device. Each operator measured each site on each device separately four times and entered the findings.

Figure 3.51 Chart Output from FTIR Study


FTIR Study.jmp also requires generation of a flat file by using the Stack option before being suitable for use. The entries in the last four columns represent the replicate values of E0 determined by each operator. Figure 3.52 presents an excerpt of the stacked table with the names associated with the new columns. For purposes of analysis, the data type for REP became Numeric, Nominal. Because the random error routines in JMP do not allow continuous variables, DATE also became a Numeric, Nominal variable in case using it in a model became necessary later.

Figure 3.52 Excerpt of FTIR Study Stack.jmp

Looking at the Data The author subscribes absolutely to the principle that before beginning any analysis, the best approach is to look at the data. JMP provides a wealth of potential graphing capabilities, and the author usually uses at least the Distribution option to produce a histogram of the data, as well as possibly several versions of the Variability/Gage Chart option. Figure 3.53 shows the variability chart generated by specifying the X-grouping variables as OPERATOR, OBJID, and SITE, in that order. The data contains no apparent outliers, but the combination of the two charts provides some insight. Notice that the data collected by OPERATOR = JE might contain more variation than that collected by OPERATOR = GB, although on average they both obtained very nearly the same values for E0.


Figure 3.53 Variability Chart of E0

Data Structure and Model Fitting This small data table contains some of the more complex nesting of any example discussed to this point. To identify a particular replicate measurement requires specifying an operator, an object, a site measured, and the date of the measurement. The complete nesting structure for the variable REP (a contributor to measurement tool variation) is REP[OPERATOR, OBJID, SITE, DATE]. The author elected not to apply the shortcut estimation of a figure of merit for this process because of the complexities of the data structure. Fixed effects in the data—not chargeable to the measurement process—include OBJID and SITE. Using the summary approach to compute the P/T value for this study is possible, but requires using not only a grouping variable (OBJID), but also a subgrouping variable (SITE) similar to the approach presented in the first example in this chapter. The resulting summary table requires addition of columns for means and variances and separate calculations of P/T for each object type.


Figure 3.54 shows the definition of the model for this analysis. After defining E0 as the response, select the other variables and add them to the model. Then define OPERATOR and REP as random effects using the Attributes menu. The variables OBJID and SITE are fixed effects that should not be charged to the variation in the measurement process. However, a particular SITE belongs to a particular OBJID, so these variables are also nested. The replication variation is measured by REP[OPERATOR, OBJID, SITE, DATE], which is also the residual for this model, so this term is not needed in the model. Figure 3.55 shows the random effects found.

Figure 3.54 Model Definition for Regression Analysis

The paragraph immediately before Figure 3.53 suggested that the two operators found approximately the same answers in each case. The regression analysis in Figure 3.55 indicates that the results from the two operators were somewhat different after all, contributing some 60+% of the total variation observed. The total chargeable variance to the measurement process is the Total value listed in the figure: 0.0001403 If one computes a P/T figure of merit based on this variance and a ± 10% tolerance on the overall mean of all the measurements, the result is ~ 9.1.


Figure 3.55 Variance Component Estimates for E0 Measurement

Comparing the Variability Due to OPERATOR A useful secondary analysis is to compute the variance associated with the measurements made by each operator during this study. First, one computes the variance of the data grouped by OPERATOR, using OBJID and SITE as subgrouping variables. Figure 3.56 shows the setup of this summarization; Figure 3.57 displays the results.

Figure 3.56 Setting Up the Summary

Figure 3.57 Summarization Results


One then prepares a bar chart of the variances found, using the results in Figure 3.58 with OPERATOR as the X level. Figure 3.59 is the result. Notice that OPERATOR = JE collected data that was more variable than the data collected by OPERATOR = GB in all cases.

Figure 3.58 Setting Up a Bar Chart

Summary and Lessons Learned Regardless of the complexity of a measurement study, the first order of business in analyzing the data produced must be to graph it by various means to explore its content. The author has come to prefer the Variability/Gage Chart approach due to its flexibility. Although generating distributions is also valuable, and simple histograms might identify outlier values, assigning them to a source is more difficult.


Figure 3.59 Variability Due to OPERATOR

If the analyst attempts to apply the calculations available in the menu on the Variability/Gage Chart report, two things become immediately obvious. First, the Gage R&R report is limited in its capability. Choosing the option to compute variance components might not allow defining a complex model such as the one eventually analyzed here. The most powerful approach is to use the Fit Model option and define a specific model for the situation. This example illustrates a fundamental issue with measurement studies. If the measurement process requires judgments on the part of operators, then differences between operators can become extremely important unless each one has the same level of training and the same level of dedication to producing reliable data. A far better approach is to automate as much of the measurement process as possible so that sources of differences from operators and potential data entry errors are minimized.


Summary of Measurement Capability Analyses This chapter illustrates a variety of approaches for analyzing the data from measurement studies. An extremely important and critical first step graphs the data and examines it in detail. This action helps identify unusual values and potential trends in the data that might not otherwise be apparent. Clearly, in the author’s opinion, one of the most powerful graphing tools available in JMP is the Variability/Gage Chart option. Where the data structures permit, the author prefers to compute an estimate of a figure of merit once he has determined that the data is free of unusual values that have assignable causes. When that figure of merit easily meets the criteria established either for a repeatability study or for a more complex total measurement study, then no further analysis is really necessary unless the graphing of the data suggests a problem. More complex analyses become necessary when a figure of merit does not meet the established criteria. For very simple studies involving only two variables, using the Gage R&R method is extremely useful. To use this method, the analyst must assume that one of the variables represents Operator while the other represents Part, even if this is not the case. Once the analysis has been completed, the user must study the output of the routines very carefully to understand fully the results of the study. Provided the nesting structure of the data is relatively simple, then the variance component analyses available on the variability chart are also quite useful and provide information quickly regarding the contributors to variation. But where any doubt exists about the data structure, the author prefers to define a specific model for the data and run a regression analysis after declaring appropriate random and fixed effects. In defining those effects, a useful approach is to regard any artifact of the object being measured as a fixed effect, whereas any legitimate contributor to variation in the measurement process becomes a random effect. This approach is a practical one based on experience in mixed models such as the previous examples provided, but it does not exactly follow academic standards for random versus fixed definitions.

2

P a r t

Analyzing a Manufacturing Process Chapter

4

Overview of the Analysis Process

165

Chapter

5

Analysis and Interpretation of Passive Data Collections 171


C h a p t e r

4

Overview of the Analysis Process Introduction 165 How Much Data? 166 Expected Results from Passive Data Collection 167 Performing a Passive Data Collection 167 Planning the Experiment 167 Collecting the Data 168 Analyzing the Data 168 Drawing Conclusions and Reporting the Results 169

Introduction Figure 1 in the introduction to this book shows a schematic of the SEMATECH Qualification Plan. Although this schematic does not formally embrace the DMAIC provisions of Six Sigma, it contains all those elements. Previous chapters have discussed the measurement process and emphasized that unless one understands the contributions


of the measurement process to the perception of a manufacturing process, then any attempts to improve that manufacturing process further will suffer. After one characterizes and understands the measurement tools being used to monitor a particular manufacturing process, a next critical step is to understand the manufacturing process itself. In the past, identifying the location of a process (the process mean) was the focus of all investigations. In the 1980s, the Japanese engineer G. Taguchi introduced the concept that not only the location of a process is important, but that the variation in that process is also critical. Understanding the location of a process and the variation associated with that process requires that one observe that process running in an undisturbed fashion for some defined period. In other words, one must not change the settings! Not changing settings in a process for a defined period is very difficult for a dedicated process engineer. The best engineers live for the opportunity to improve something. Allowing a process that is not performing properly to run undisturbed is difficult for them. But before one can successfully improve a process, one must have prior knowledge with reasonable levels of confidence about how that process is actually behaving. Members of SEMATECH (semiconductor manufacturers) coined the phrase passive data collection or PDC for this stage of an investigation. Collecting data passively means the dedicated engineer must stand by and let a process generate enough data so that analyses can define the current operating situation. Intervention in a process to change settings to improve results is not part of a PDC. Obviously, this statement does not intend that knowledgeable people should stand by and let a functioning process self-destruct! Rather, it means that a team must collect enough information about a process to enable rational decisions about what is wrong and to suggest possible solutions to any problem. The following sections outline how much data these studies might require, what a team should expect to achieve, and how a team should plan and conduct such a study.

How Much Data? The section “Uncertainty in Estimating Means and Standard Deviations” in Chapter 1 discussed the uncertainties associated with estimating population means and standard deviations from small samples. None of those rules has changed. Because any serious Six Sigma program must concern itself not only with putting a process on target, but also with minimizing variation, sample sizes must necessarily be reasonable and sufficient for an appropriate level of confidence. In the semiconductor industry, virtually every process considered is a batch operation. Therefore, passive data collection guidelines recommend running 30 batches without changing process settings. Semiconductor processes also generally involve multiple levels of variation within a batch being processed. Therefore,

Chapter 4: Overview of the Analysis Process 167

an understanding of nested relationships is important, and the ability to use appropriate analysis of variance techniques within regression analysis becomes essential.

Expected Results from Passive Data Collection Properly executed, a passive data study and the resulting analysis yield a variety of benefits:

a preliminary estimate of process capability

an early warning of potential instability in process outputs

sources of variation within a process

suggestions for possible experiments necessary to optimize a process

Performing a Passive Data Collection As with any experiment undertaken in a manufacturing environment, a passive data collection or PDC consists of several essential activities: 1. planning the experiment 2. collecting the data 3. analyzing the data 4. drawing conclusions 5. reporting the results The following sections discuss these points in more detail.

Planning the Experiment Before starting the PDC, the team must agree on the process settings that the study will use. In addition, all those involved must understand that these settings must remain unchanged during the course of the study, because changing process inputs will add a


source of variation not contemplated by the original plan. If the team makes changes in mid-study, that study should start over. Another issue before starting the experiment is being certain that any measurement tools used have been characterized. Characterizing variation inherent in a measurement process is essential before starting a passive data study, and only measurement tools known to make minimum variation contributions should be used. A final important consideration is to be sure that the sampling plan used for collecting the data actually captures potential sources of variation in the data. Choosing the right sampling plan requires the insight of experienced engineers, technicians, and operators working on a particular process. Failing to understand possible sources of variation in a process can lead to sampling plans that might not capture all the variability in a process or that confound one source of variation with another. The first example in Chapter 5 illustrates this issue in a thermal deposition of an insulating film in semiconductor manufacture. The execution of other extremely complex examples in Chapter 5 was the result of careful discussions among team members combined with knowledge obtained in previous similar experiments. Dr. George Milliken, Professor of Statistics, Kansas State University, Manhattan, and several of his graduate students serving internships at SEMATECH were instrumental in devising clever sampling plans to capture the variability from multiple sources in a number of processes. Before their involvement, at least two projects had floundered somewhat. The lessons learned from that assistance made certain that future experiments contained the proper structures to capture sources of variation.

Collecting the Data The PDC experiment optimally should include 30 independent replications of the process. In this context, a replication usually means an independent batch. Fewer replications allow estimations of sources of variation, but the confidence intervals associated with those estimates might be too wide to be useful.

Analyzing the Data Before embarking on any complicated analysis of the data, one must take the time to examine its content for unusual values or for trends. Examples that follow in the next chapter will demonstrate thorough examination of the raw data using distributions, variability charts, and overlay graphs. The data obtained from the PDC can provide the basis for preliminary estimations of control limits and capability for a process.

Chapter 4: Overview of the Analysis Process 169

Drawing Conclusions and Reporting the Results Summaries of the analyses combined with appropriate charts and graphs can help others understand how well or how badly the particular process performs. Graphs, particularly bar charts, showing the contributions of various sources of variation in a process also help direct resources for further experimentation to optimize the process. One typical method for reporting results in the semiconductor industry is to define a variation contribution in terms of a percent nonuniformity calculated according to Equation 4.1, an adaptation of the statistic coefficient of variation.

⎛ σˆ ⎞ % Nonunif = 100⎜⎜⎜ variation ⎟⎟⎟ ⎜⎝ μˆ observation ⎟⎠ Obviously, when a process involves many sources of variation, then a report should contain a statement for each source as well as an estimate for the total.

4.1


C h a p t e r

5

Analysis and Interpretation of Passive Data Collections Introduction 172 A Thermal Deposition Process 172 Looking at the Data: Initial Analysis of Supplier-Recommended Monitor Wafers 174 Analysis of the Team-Designed Sampling Plan 176 Reporting the Results 179 Lessons Learned 180 Identifying a Problem with a New Processing Tool 181 Looking at the Data: Estimating Sources of Variation 181 An Alternative Analysis 185 Lessons Learned 188 Deposition of Epitaxial Silicon 189 Determining a Sampling Plan 189 Analyzing the Passive Data Study 191 Lessons Learned 196


A Downstream Etch Process 196 Overview of the Investigation 196 Passive Data Collections 197 Lessons Learned 207 Chemical Mechanical Planarization 208 Polishing Oxide Films 208 Polishing Tungsten Films 217 Polishing a Second Type of Oxide Film 222 Summary of Passive Data Collections 230

Introduction Chapter 4 stressed the importance of proper sampling plans to capture the true variation in a process. The complexities of some of the following examples illustrate the proper sampling plans required by some processes. The examples that follow generally originated either in the author’s work at SEMATECH or in his consulting activities for a variety of industries generally related to semiconductor manufacturing. All the data presented derives from authentic case studies; none represents data simulations.

A Thermal Deposition Process This example illustrates the characterization of a process recipe provided by the vendor of a new configuration for a furnace process. This process deposited an insulating layer of silicon nitride (Si3N4) on a large batch of silicon wafers simultaneously, and was one of the first SEMATECH programs that the author supported as a statistical resource. Understanding the variation present in the process was an important first step before conducting formal experiments to optimize the process. Previously, the furnaces in this example had been arranged horizontally. This new design arranged the furnace tube vertically in order to conserve valuable floor space within the manufacturing facility. The integrity and control of the deposition of a layer like the one deposited in these furnaces is critical to the final performance of semiconductor devices constructed on the wafers. Figure 5.1 is a simple schematic of the reactor involved in this study.

Chapter 5: Analysis and Interpretation of Passive Data Collections 173

Figure 5.1 Furnace Schematic

The reactive gases dichlorosilane (SiCl2) and ammonia (NH3) pass down the heated furnace tube over a collection of wafers; this particular furnace configuration could accommodate approximately 175 wafers. The process depletes the reactive gases as they pass through the reactor. As the reactive species decrease, so does the reaction rate and subsequent deposition of material on the substrates. To adjust for the depletion, the heating jacket on the furnace maintains an ever-increasing temperature through the use of five independently controlled temperature zones. Common practice was to place a number of “dummy” wafers at the top of the furnace and a similar number at the bottom, so the effective load allowed was approximately150 wafers in production. A quartz “boat” with numbered slots held the wafers for the reaction; an elevator mechanism at the bottom of the reactor loaded the quartz boat with wafers into the reactor. The supplier of this furnace had formulated a process based on samples placed at the bold lines in Figure 5.1. The objective of the project team was to investigate the process and to minimize the overall variability of the coating being produced while targeting a desired thickness. Minimizing this variability required estimating run-to-run, wafer-to-wafer, and within-wafer variances. Therefore, before starting a series of process optimization experiments, the engineering team elected to execute a few trials using the supplier process. For these trials, they included additional monitor wafers at the dotted lines in


Figure 5.1, as well as the monitor wafers at positions recommended by the supplier. This experiment included 10 runs, and each run contained 11 monitor wafers with thickness measured at 9 sites on each wafer. Figure 5.2 shows an excerpt of data from this experiment only from the monitor wafer locations recommended by the supplier. (This table contains a subset of Furnace all samples.jmp, which was the full table of data collected by the team.)

Figure 5.2 Excerpt of Furnace 3 samples.jmp

The team had no interest in fully characterizing the variability in this process. Instead, they wanted to establish a baseline for the variation in order to plan a series of experiments. A significant concern was that completing each run required some seven hours of process time that included loading, conditioning, reacting, cooling, and removing the coated materials. The initial trial involved only 10 replications or runs of the supplier recipe in an effort to duplicate supplier results.

Looking at the Data: Initial Analysis of SupplierRecommended Monitor Wafers Generating a variability chart for the three supplier monitor locations produces Figure 5.3. This display suppresses the raw data and presents the graphic as a box plot rather than the system default format. The line through the figure joins the cell means. The graphic indicates that not much difference exists in the average thicknesses observed at these three locations or slots.


Figure 5.3 Variability Graph Based on Supplier Sampling Plan

To determine the sources of variation in this data, one might generate an additional variability chart by specifying RUN and SLOT in that order as the X-grouping variables. On the pull-down menu for this graph, one selects Variance Components and chooses Nested for the data structure. Figure 5.4 is an excerpt of the report generated from this second graph. The Within variation corresponds to the SITE(RUN,SLOT) variation; thus SITE was not needed as an X-grouping variable.

Figure 5.4 Variance Analysis Report Based on Supplier Sampling Plan


The analysis indicates that some run-to-run variation exists (denoted by RUN), but that variation among the three slots (sampling positions denoted by SLOT[RUN] ) is not statistically significant. The bar graph at the bottom of the figure indicates that variation across the sites (the nine positions measured at each SLOT position) accounts for more than 80% of the variation in the data (denoted by Within). This analysis suggests that the team should focus most of their energy on understanding and reducing the variation among measurement sites and concern themselves with a potentially unstable process, because RUN contributes a significant amount of variation in the data. Also, based on this sampling plan and analysis, the observed nonuniformity in the thickness coating is as shown in Equation 5.1:

⎛ σˆ ⎞ (100)(13.366) % Nonunif = 100⎜⎜⎜ variation ⎟⎟⎟ = = 1.31% ⎜⎝ μˆ observation ⎟⎠ 1016.9

5.1

The nonuniformity number is well within the specifications for this process. Similarly, a capability analysis of the process using a specification range of ± 5% of the process target (1000 Å) shown in Figure 5.5 suggests a capability potential of > 3 (the section “Capability Analysis” in Chapter 6 and sections in Chapter 7 discuss capability analysis in more detail). Both of these results are extremely encouraging, but are they the true story?

Figure 5.5 Capability Analysis Based on Supplier Sampling Plan

Analysis of the Team-Designed Sampling Plan The 10 runs conducted by the engineering team contained an additional eight sample monitor wafers, as well as the three monitor wafer locations recommended by the supplier. Figure 5.6 is an excerpt of the complete data table (Furnace all samples.jmp)


generated by the experiment. An analyst must stack this table before determining the sources of variation. The result of the stack process is Furnace all samples STACK.jmp.

Figure 5.6 Complete Data Table from Initial Experiment

Generating a variability chart from the complete data table produces Figure 5.7. Notice that the figure contains data from the original three wafers from slots recommended by the supplier and the additional eight wafers from slots tested by the engineering team.

Figure 5.7 Variability Chart from Furnace all samples STACK.jmp

Collecting the additional data (a larger sample size) revealed an apparent standing wave in the film thickness produced by the recommended process. Figure 5.8 shows the results of repeating the analysis of the entire body of data, using the variance components option


on a second variability chart that specified the X grouping variables as done in Figure 5.4. NOTE: The data used for this calculation is balanced, with no missing values. In general, JMP should use EMS (Expected Mean Squares) for calculating the variance components in this case. However, the original estimate of the variance component for RUN was negative. In this situation, JMP automatically switches to REML for the estimation method.

Figure 5.8 Analysis Report Based on a More Detailed Sampling Plan

Now the significance of the variation associated with RUN has disappeared. In the analysis of the complete data set, SLOT[RUN] is now the most important contributor to the variation in the data. The estimate for SITE[RUN,SLOT] variance is obtained from the estimate for Within and appears in that row of the report. The process through 10 runs appears stable, but more important, the variation from wafer to wafer within the furnace now is the dominant source of variation. The variation of thickness across individual wafers (SITE) is somewhat smaller, but prudent experimenters should not ignore its contribution. This short study changed the focus of attention for future experimental work from concern about run-to-run stability and across wafer variation to minimizing variation among the monitor wafer locations (slots) as well as across each wafer. NOTE: In Figure 5.4, the variance component charged to thickness variation across each monitor—Within—was ~ 149 units; in Figure 5.8 the value is ~ 137. In Figure 5.4, the variance component charged to thickness variation among monitor wafer locations was ~ 6, whereas in Figure 5.8 it is ~ 369. Clearly, the improved sampling plan has identified a previously unknown problem with the recipe as provided by the supplier.


Reporting the Results In preparing an initial report of this investigation, the team produced a new table and a new bar chart. To generate this chart, they first captured the information about the variance components from the report of the analysis in a new JMP table by performing the following steps: 1. Right-clicking anywhere in the body of the report Variance Components. 2. On the menu that appears, selecting Make Into Data Table. Figure 5.9 shows the result: the left panel in the figure contains the formula used to compute the last column in the table. In that formula, the denominator is the grand average of the thickness readings (1015.97). Although these values are not dramatically different from those found using the supplier’s sampling plan, they better represent the current state of the process. Figure 5.10 is the chart of percent nonuniformity generated from this table.

Figure 5.9 Display of Variance Report Furnace all samples.jmp


Figure 5.10 Chart of Percent Nonuniformity Sources

Lessons Learned The most important information to gain from this example is that failing to sample a process adequately can produce misleading results and overlook important sources of variation. In addition, this example demonstrates the superb utility of the Variance Component calculations associated with the variability chart when the user specifies the grouping variables correctly. Finally, including a bar chart of the sources of variation in the data in a report facilitates understanding.


Identifying a Problem with a New Processing Tool Just as preliminary measurement studies called repeatability studies can identify serious shortcomings in a measurement device, preliminary process studies can identify potential problems with new equipment. In this example, a factory had received a new furnace similar to the one discussed in the previous section. Because the study conducted earlier involved a large sampling plan, the team responsible for characterizing this new tool immediately adopted that plan and proceeded to test the new machine. This study involved only five trials, with each trial containing 11 monitor wafers. On each monitor wafer, the team measured thickness at nine sites. This measurement was not an attempt to characterize fully the variation in this tool. Rather, the objective was to develop a relatively early view of how the new processing tool behaved using settings the supplier recommended for the desired deposition process. Figure 5.11 presents an excerpt of the raw data found in Furnace 2.jmp.

Figure 5.11 Excerpt of Furnace 2.jmp

Looking at the Data: Estimating Sources of Variation The first step in examining this data requires rearranging the original table using the Stack utility on the Tables menu. This step produces the table Furnace 2 STACK.jmp (Figure 5.12).


Figure 5.12 Excerpt of Furnace 2 STACK.jmp

Once the data table has the correct structure, one generates a distribution of the response (NTHK). The histogram produced suggests a tail favoring higher values as well as a few outlier values (Figure 5.13). Preparing a variability chart as illustrated in the previous section and specifying RUN, WAFER, and SITE in that order results in the chart in Figure 5.14, with the variance component analysis for the nested variables at the bottom of the figure.

Figure 5.13 Distribution of NTHK

Note that the major source of variation in the data derives from the measurement site (~88%). However, careful study of the graphic portion of Figure 5.14 suggests a source for this large variation. Note that although the actual sites are illegible in the graphic,


pairs of unusual values appear to exist across all runs for each wafer location. Brushing these suspicious points in one or more runs and examining the original data table shows that all the brushed points are from two wafer sites—6 and 7. Are the means for these two sites actually different from the means of other sites? One way to test this is to use the Fit Y by X option on the Analyze menu (Figure 5.15).

Figure 5.14 Variability Chart

Figure 5.15 Variance Components Analysis

In the dialog box that appears after the selection in Figure 5.16, one chooses NTHK as the Response and SITE as the Factor. This platform examines the modeling information for both the response and the factor. With a continuous response and a nominal or categorical factor, the system chooses a one-way analysis of variance and produces the graph in the left panel of Figure 5.17. Obviously, the two sites 6 and 7 appear to be different from the others, but is that difference statistically significant? To test the differences in means, one can select Compare Means on the pull-down menu, and then select All Pairs, Tukey HSD (right panel of Figure 5.17). This selection applies the


Tukey-Kramer HSD (“honestly significant difference”) test, which is exact if the sample sizes are the same for each group, and conservative if they are not (see the JMP Statistics and Graphics Guide for more information).

Figure 5.16 Fit Y by X

Figure 5.17 Comparing NTHK at All 9 SITES

The results of the Tukey-Kramer HSD test are shown in Figure 5.18. In this figure, the graphic at the right of the figure provides immediate information about whether the mean NTHK values at sites 6 and 7 are statistically different from the others. If circles do not overlap, they are different from each other at 95% confidence.


Figure 5.18 Tukey-Kramer HSD Results

The test indicates that the mean NTHK values from SITE = 6 and SITE = 7 are different from the results found at the other sites. To explore what impact this observation has on the sources of variation in the data, one can use a selection statement to remove these points from the data and compute the variance components again. The left panel in Figure 5.19 contains the report seen previously in Figure 5.15. The right panel in Figure 5.19 contains the analysis after excluding the data from SITE = 6 and SITE = 7.

Figure 5.19 Comparing Variance Component Results after Excluding Outlier Values

Notice the reversal of the level of variability associated with SLOT[RUN] and Within.

An Alternative Analysis The analysis just described actually blends additional terms into the error term used to determine the statistical significance of the differences among sites. An alternative approach based on fitting a model to the data provides a more valid analysis. For example, Figure 5.20 shows the model one might define for this scenario.


Figure 5.20 A More Rigorous Model to Examine the Data

The model illustrated declares that RUN is a random effect and declares that SITE, SLOT, and the interaction SITExSLOT are fixed effects. Fitting this model using all the data in the original table produces the results in Figure 5.21. The upper portion of the figure indicates that significant differences exist between SITE, SLOT, and the combination SITExSLOT. Plotting the least square means (LS MEANS PLOT, available as an option for each term in the EFFECT DETAILS section) for each variable illustrates clearly that some anomaly exists associated with SITE = 6 and SITE = 7. In fact, the lower panel of the figure suggests that the problem is worse at some monitor locations than at others.


Figure 5.21 Fitting an Alternative Model to the Data


Lessons Learned This example illustrates well the axiom that all engineers should observe: “In God we trust; all others bring data.” In this case, the manufacturing of the furnace had introduced a systematic error in the new processing tool. Further investigation of the results developed the following information: 1. The loading of the quartz boat that held the wafers oriented each of them in same way; the flat side of the wafers was on one side of the furnace in all cases. 2. The errant points were all on one side of the wafer, opposite the flat side (see the schematic of the measurement plan for each wafer in Figure 5.22). 3. Thermocouples used to control the temperature of the furnace were adjacent to the flat side of the wafers. 4. Examination of the structure of the furnace showed that the heating elements were wound more tightly on the side of the furnace where the unusual points were located than on the other side of the wafer. This anomaly produced a “hot zone” on the furnace wall at a point opposite the thermocouple sensors. Because the rate of deposition of the layer was directly proportional to the temperature, this led to thicker coatings near the hotter area.

Figure 5.22 Schematic of Measurement Locations on a Wafer


Although the supplier suffered some embarrassment from these findings, the outer shell of the furnace that contained the heater elements was replaced quickly and the problem disappeared. Although this experiment required almost two full days of valuable time as well as a delay in acceptance testing for the new equipment, careful analysis of the results saved considerable difficulty later in attempting to minimize the variation associated with this processing tool.

Deposition of Epitaxial Silicon In the process in this example, the system feeds a single wafer into the reaction chamber from a cassette of 25 wafers held in a vacuum chamber. The wafer being processed rests on a platen that spins after the wafer is in position. Reactive gases enter the chamber from a top vent, and activation of a set of intense lamps causes thermal decomposition of the reactive gases and deposition of the epitaxial silicon on the wafer surface.

Determining a Sampling Plan At the start of the investigation, several of the engineers believed that because the wafer was rotating during the deposition process, the only variation likely to exist on the wafer surface would have an axial orientation. That is, they believed that the thickness of the deposited material would be likely to vary systematically from the center of the wafer to the edges. To determine whether this was true, the first experiment coated several wafers using a nominal set of conditions for the process. Then the investigators measured a pattern of 25 points on these wafers (at the insistence of the supporting statistician, the author) to determine whether the supposed variation actually existed. The table Episilicon space.jmp contains this data, and Figure 5.23 displays an excerpt from a single representative substrate.


Figure 5.23 Sample Data from Episilicon space.jmp

Notice that this table identifies the site of the measurement on the wafer and includes X and Y coordinates for the location of that measurement site. X = 0, Y = 0 corresponds to the center of the wafer. The units of measurement are relative only. An excellent choice for examining this data is the Surface Plot option on the Graphs menu in JMP. Figure 5.24 shows how to access and initiate this platform. NOTE: This platform within JMP is extremely flexible. The figures presented below are the result of considerable modification using the control panel associated with the platform to rotate the image. They do not represent system defaults.

Figure 5.24 Accessing and Generating a Surface Plot


The left panel in Figure 5.25 shows the sampling plan as applied to the surface of the wafer. If the only variation in film thickness occurred axially, then the central panel in the figure, which shows the wafer as if one were looking at the edge, should show a predictable pattern. It does not show a pronounced pattern from center to edge, which leads to the right panel in that figure. The right panel shows that several positions on the wafer surface have elevated thicknesses, and no apparent radial pattern exists.

Figure 5.25 Visualizing the Sampling Plan

Although several explanations are possible for this pattern of film thickness, none actually settled the issue. Among the theories was the idea that the software controlling the rotation of the wafer actually stopped the rotation before the flow of gases to the chamber and the irradiation by the lamps ended. Another theory was that some form of harmonic vibration existed in the rotating platen. The facts supported none of these ideas fully, so the issue remained, and the sampling plan of 25 sites across the wafers became the standard.

Analyzing the Passive Data Study This experiment deposited epitaxial silicon on five sample wafers in each of 11 cassettes. Each cassette held 25 wafers, but the engineers used dummy wafers in 20 of the positions to control the cost of the experiment. Figure 5.26 presents an excerpt of the data table Episilicon.jmp, and Figure 5.27 is a variability chart showing the data and giving some insight into the sampling plan. Notice how the engineers chose Slot = 1 and Slot = 2, and then chose other slots throughout the 25 possibilities in each cassette. Focusing on Slot = 1 and Slot = 2 in this manner provided an opportunity to detect a first-wafer effect sometimes seen in processes of this type. That is, the first and possibly the second wafers produce different results as a reactor becomes seasoned during the processing of a cassette of wafers. Figure 5.27 does not indicate that such an effect was present.


Figure 5.26 Excerpt of Episilicon.jmp

Figure 5.27 Variability Chart of Raw Data from Episilicon.jmp

The distribution generated from this data suggested that several low outlier values were present. Brushing the outlier values and creating a subset table based on those points did not suggest a particular pattern to the low values; some occurred at Site = 25, some at Site = 16, some at Site = 10, and so on. The distribution parameters showed that these points were all within a few percent of the median as well. Therefore, they required no action. Figure 5.27 clearly suggests the nesting structure in this data: Slot[DATE, Cassette], Cassette[DATE]. Not shown in the figure are the locations of the measurements. Not including Site in an analysis assigns the variation from Site to Site to the residual


(Within in the report). Choosing to calculate the variance components using the option on the variability graph produces the results in Figure 5.28. Some variation existed across each cassette as well as within each wafer (residual). However, the minimal variation seen from day to day and from cassette to cassette was extremely good news, as it suggested that the process was stable. Dealing with the contributions of slot-to-slot variation (17.8%) and within-wafer variation (79.5%) should be the subject of designed experiments to optimize the process.

Figure 5.28 Variance Component Analysis of Episilicon.jmp

To explore further whether a significant first-wafer effect exists, one can select the Fit Y by X platform and generate the one-way analysis of variance of thickness by slot. Figure 5.29 shows this result after requesting a comparison of all means using the Tukey-Kramer HSD analysis. The figure has been modified to suppress some of the output of this test. The objective of this exercise was to detect any large first-wafer effect—that is, were the results from Slot 1 different from Slot 2, and so on?


Figure 5.29 Tukey-Kramer HSD Comparisons

Obviously, the box plots in Figure 5.29 appear quite similar. The chart at the bottom of the figure compares the means of all possible combinations of the five slots. The differences are small and the confidence interval for each difference includes 0, which means that the difference could be 0. In addition, the confidence lines at the right around the bars completely enclose each bar, which means that none of the differences are significant. One would conclude that the difference is significant only if the confidence intervals fail to include all of a given bar. The analysis just discussed ignores the nested relationship Slot[DATE, Cassette] and could be suspect. Running a more sophisticated analysis using the Fit Model option, specifying Slot as a fixed effect with its nesting, and including DATE and Cassette[DATE] (Figure 5.30) as random effects produces the result in Figure 5.31.


Figure 5.30 Model Specification for Correct Analysis

Figure 5.31 Fit Model Report

The report for the fixed effect indicates some differences among slots, depending on Cassette and DATE. Under the Effect Details portion of the report (not shown here), the investigator can generate a much more sophisticated Tukey-Kramer HSD analysis for Slot. By default, this option generates a crosstabulation of all combinations of Slot[DATE, Cassette]. A few scattered comparisons are statistically significant (confidence intervals for differences between means of slots do not include 0). The table is much too large and complicated to reproduce in this text.


Lessons Learned This example demonstrated again the importance of a proper sampling plan to capture sources of variation in a process. In addition, the results of the passive data study indicated that little or no first wafer effect existed and that the process was stable and suitable for optimization experiments.

A Downstream Etch Process Etch processes are an essential part of semiconductor manufacture. Typically, these processes occur after masking off a pattern on a wafer surface to allow relatively selective removal of unmasked material in order to define circuits on the wafer surface. Conventional etch processes expose the wafer to plasma-generated ionic species that attack the area to be selectively etched. Because the process exposes the wafer surface to the electrical discharge in a plasma and the associated high-energy particles that accompany it, damage to delicate structures on the wafer surface become a concern. This example concerns the early characterization of an etch process that eliminates exposure of wafer surfaces to plasma-induced damage. In this system, a stream of reactive gases passes through a microwave generator that dissociates elements in the gas stream into reactive species. These ions and radicals pass along a tube that makes a right-angle turn to impinge on a wafer held on a platen. The physical nature of the equipment eliminates exposure of the wafer surface to the high energy of the plasma generated. This example deals with the preliminary investigations that sought to determine the capabilities of a prototype device.

Overview of the Investigation As a largely research-oriented project, this work followed a slightly different path from what one might normally encounter in a semiconductor manufacturing facility. The original project plan included three passive data studies. In addition, an experimental segment attempted to optimize several steps in the process. Finally, a marathon study (also a passive data study) used optimized settings determined in the experimental phase to demonstrate manufacturing capability for the machine and to determine the cost of ownership (cost per wafer processed) associated with operating such a process. Before starting the project, organizations supporting the research generated multiple lots of wafers coated with various materials for etching studies. Because the project team wanted to compare passive data studies directly, some care was necessary to assure that results were directly comparable. Because different lots of wafers having the same coating might


process differently, investigators had to carefully assign the lots of raw materials to the studies to avoid confounding any differences among lots with the actual properties of the processing tool. Prior knowledge in other investigations suggested that minor differences among crystal structures of otherwise identical lots of coatings could affect not only the rate at which a coating etched, but also the variation in that etch process.

Passive Data Collections As stated in the previous section, the investigators needed to be sure that they could make direct comparisons between simple passive data collections (PDCs) before conducting experimental studies to optimize the process. Therefore, planning for the experiments had to assure that all lots of raw materials assigned to initial passive data studies were available for all such studies in order to prevent confounding of differences due to wafer lots with the variability observed in the process. At this point, Dr. George Milliken, Professor of Statistics, Kansas State University, Manhattan, and several of his graduate students played instrumental roles in assisting in the planning of the experiments. The initial passive data study (Test 1) was to include 15 cassettes loaded with a variety of wafers to fill each of the 25 slots available in each cassette. After some electrical and optical experiments on the processing tool had been completed, the team intended a second study (Test 2) containing some 10 cassettes similarly loaded. This study was to verify that nothing had changed in the process before the team committed to experiments to optimize the process. Raw materials for a third passive data study were reserved in case any modifications were necessary to the equipment before starting the experimental optimizations. Delays in other stages of the work eliminated the third passive data study in order to keep the project on schedule.

Assignment of Raw Materials The investigation involved five types of coated materials, as well as several dummy wafers to allow conditioning of the processing tool between types of coatings. This discussion considers only two of the coated materials used in the study, designated SLCS and PBL by the project team. Figure 5.32 presents an excerpt of Etchdata Setup.jmp. Creating a contingency table using the Fit Y by X option on the Analyze menu (Figure 5.33) reveals the assignments of wafers for each cassette in each of the two tests. Fitting two variables with the Nominal modeling type produces the contingency table.


Figure 5.32 Excerpt of Etchdata Setup.jmp

Figure 5.33 Generating a Contingency Table

Figure 5.34 shows the contents of the two contingency tables generated by this setup for WAFTYP = PBL, and Figure 5.35 shows the results for WAFTYP = SLCS. Notice that most lots appear at least twice (two different cassettes) in each set of runs; this prevents confounding of any effect of lot with any effect due to cassette in either of the tests.


Figure 5.34 Distribution of LOTID in CASSETTE for WAFTYP = PBL


Figure 5.35 Distribution of LOTID in CASSETTE for WAFTYP = SLCS


Data Analysis Examining the data from the two runs (Etchdata All STACK.jmp, Figure 5.36) by generating distributions separated by WAFTYP and TEST indicates no outlier values in either test for SLCS, but suggests minor outlier values for PBL in TEST = 1, and extreme outlier values for PBL in TEST = 2 (Figure 5.37).

Figure 5.36 Excerpt of Etchdata All STACK.jmp


Figure 5.37 Distributions from TEST = 1, 2 by WAFTYP

Generating a variability chart (Figure 5.38) for DLTATHK using TEST, CASSETTE, and SLOT as the X-grouping variables in that order, with WAFTYP to segregate wafer types, provides a more detailed view of the data.


Figure 5.38 Variability Graphs of Raw Data

The upper panel of Figure 5.38 shows that the PBL wafer in SLOT = 21 of CASSETTE = 16 had a very low removal rate. Similarly, one site on the SLCS wafer in SLOT = 16 of CASSETTE = 17 had relatively high removal rates. A convenient way to identify these entries is to brush them on their respective charts. Excluding those observations before formal variance component analysis is a prudent choice. The variability chart for WAFTYP = SLCS also suggests that the wafers in SLOT = 1 and SLOT = 2 often had different removal rates compared to the wafers in SLOT = 16 and SLOT = 25. Note also that removal rates differed somewhat between TEST = 1 and TEST = 2 for WAFTYP = PBL. One can perform variance component analysis of this data by using that option on the variability chart menu, provided one assigns the differences in TEST to the process of etching the PBL wafers. However, an investigation after completion of TEST = 2 found that the substrates used there had an extra treatment applied to them. Because an assignable cause exists for the differences observed, using the FIT Model option and designating TEST a fixed effect is more logical.


Figure 5.39 shows the setup for fitting a model for this example. Figure 5.40 provides the results of the REML analysis. Note that the Residual in each table is actually the variance contribution due to SITE[SLOT, CASSETTE, TEST], and that the absence of TEST in both reports indicates its designation as a fixed effect. The upper panel in Figure 5.40 reports the results for WAFTYP = PBL; the lower panel shows results for WAFTYP = SLCS.

Figure 5.39 Defining a Model for This Example

Although both wafer types show some variation due to SLOT, the SLOT variation for the WAFTYP = SLCS dominates that analysis. This dominance is in keeping with the suspected first-wafer effect (and possibly second-wafer effect) on the etch process. Partitioning the LOTID values in the manner shown in this example was valuable, because this factor contributed to the variation in both cases.


Figure 5.40 Variance Component Estimates Based on Etchdata All STACK.jmp (Filtered Data)

To visualize the first-wafer effect more clearly, one can generate a variety of effect details within the regression environment. The section “Building a Model” in this chapter shows this approach for another example. Although the approach discussed here does not take into account the nested environment among these variables, the graphics generated are informative. This simpler approach generates the one-way analysis of variance for this wafer type using the Fit Y by X platform and specifying SLOT as the X variable and DLTTHK for the Y, recognizing the potential inaccuracy in the test due to the nested relationship present. Figure 5.41 shows the results after comparing all the means as illustrated earlier.

Figure 5.41 Comparison of Slot Means for WAFTYP = SLCS


In Figure 5.41 for TEST = 1, all the pairwise differences among slot means are significantly different (p = 0.05), but the difference between slots 25 and 16 is minimal. For TEST = 2, the difference between slots 25 and 16 is not statistically significant. Because any optimization experiments planned would use a fixed etch time as these experiments did, the experimental plans needed to preclude assigning wafers to slots 1 and 2 and possibly others that follow to avoid this problem when using fixed etch times. The processing tool had an endpoint detecting mechanism built in, so low etch rates for the first several wafers would not present a problem in applying this tool to the manufacturing process.

Preparing a Report Graphic With some editing, the variance component reports from the analysis of this data can provide graphics for reports on these findings. The Make Combined Data Table option that is available when one right-clicks anywhere in a report allows the user to generate an independent JMP data table that contains all the report information when a regression analysis contains the By option. Figure 5.42 shows a table (Summary of Etch VC.jmp) prepared from the information contained in Figure 5.40, after some editing to remove unneeded items and to add a column for the standard deviation associated with each source of variation.

Figure 5.42 Summary of Etch VC.jmp

In Figure 5.42, notice that the variance components are summed to provide the total listed for each wafer type. However, the standard deviations are not summed to provide the total indicated; they each represent the square root of the variances found. The variances are additive, and the standard deviations are not. Also, the table renames the residual in each case to the term it actually represents—the variance associated with sites within each wafer type.


From this summary table, the user can generate a chart for a report that represents the relative contribution of each source of variation. Graphics usually convey information better than tables of numbers. Figure 5.43 shows a chart prepared by graphing the standard deviation contribution of each random effect source. Specifying two variables, WAFTYP and Random Effect, in that order as Categories, X, Levels when defining the chart produced the results in Figure 5.43. A combination of this graphic with the box plots displaying the variation among slots in the cassettes can help create an effective presentation.

Figure 5.43 Chart of Standard Deviations by WAFTYP

Lessons Learned The design of this passive data collection experiment was sound, but the only insurance against the operational errors that occurred is to make sure that each individual involved in the experiment fully understands the nature of a passive data experiment: Do not change the conditions being investigated. The SLCS wafer showed a definite first-wafer effect (and probably a second-wafer effect) in that etch rates (DLTATHK) for those two differed from the other two slots in each cassette. Presumably, the same first-wafer effect might have existed for other wafer types, but that was not part of this investigation. As stated earlier, any experiment involving fixed etch times and not relying on the end point detection capabilities of the processing tool must protect its results against this effect, possibly by loading dummy wafers into the offending positions in a cassette.


Chemical Mechanical Planarization Semiconductor manufacturing involves sequential steps of applying coatings to silicon wafer substrates, and then selectively removing parts of them to create the circuitry for electronic devices. Although experiments to optimize the deposition processes can find conditions that minimize variation in thickness across a wafer surface, the coatings produced are not actually smooth. They exhibit bumps and hillocks along the surface. To remove those artifacts, a new process called chemical mechanical planarization (CMP) began attracting considerable attention during the mid-1990s. The process applies a slurry containing reactive abrasive materials to a wafer surface, and then removes it using a polishing disk or belt, or a combination of the two. The process is analogous to using an orbital or belt sander to remove material, thus dressing and planarizing the surface. The process obviously contains a variety of input variables, but the first step in attempting to optimize it has to be gaining an understanding of the sources of variation involved. Typical materials treated with this process include insulating films such as silicon oxide or silicon nitride and conductive films such as aluminum, copper, and tungsten. The following examples consider several of these materials drawn from projects on which the author served as statistical resource at SEMATECH or as a private statistical consultant.

Polishing Oxide Films Silicon oxide films form insulating layers that isolate conductive layers in semiconductor devices. The smoother these layers, the better their performance in a final electrical structure. The numbers of wafers involved in any CMP study preclude the use of a single lot of coated wafers. Because each cassette treated usually involves 25 wafers, several lots of coated wafers, as well as several batches of abrasive materials, are required to study a significant number of cassettes to determine the sources and magnitudes of variation. Assigning a single lot of coated wafers to a particular cassette confounds any effect due to cassette-to-cassette variation with possible lot-to-lot variation in material being polished. Therefore, relatively complex sampling plans have to be devised to avoid any confounding of effects. In this particular example, batches of abrasive material were not large enough to allow processing of many cassettes of 25 wafers. A general rule developed in preliminary work on this project was that such a batch could usually handle about 100 wafers, or four cassettes of 25 each. Operators prepared the abrasive batches from drums containing concentrate. Because differences might exist in the performance of various abrasive batches, the experimental plan had to take care to eliminate any confounding of the


effects of differing abrasive batches with other variables in the study. Figure 5.44 presents an excerpt of the table Oxide CMP Data.jmp.

Figure 5.44 Excerpt of Oxide CMP Data.jmp

This experiment placed 25 wafers in each of six cassettes. The team measured the thickness of the film before the CMP process at nine sites on each wafer and repeated the measurement after the process. This example shows how important precise measurement tools are to the successful interpretation of process variation, because the within-wafer variation actually contained contributions from two measurement errors. Missing from the figure is the calculated amount of material removed from each site, which requires adding a column to the table and computing the difference between PRE THICK and POST THICK values.

Examining the Wafer Allotments In setting up this experiment initially, the author wrote several small programs to make random assignments of wafer lots and of wafers from those lots to particular cassettes. The structure of those programs is beyond the scope of this discussion, but their existence was a material benefit to planning the experiments and avoiding duplication of assignments. This experiment was a preliminary one and involved only six cassettes of 25 wafers. The Fit Y by X option under the Analyze menu supports the generation of contingency tables as demonstrated earlier. To facilitate this generation, one first splits the existing data table by using that option on the Tables menu. The original table had the required flat file


format for statistical analysis; splitting the table rearranges the variable SITE into the column headings of a new table. Figure 5.45 shows the setup for generating a new table from the original. The author chose PRE THICK as the column to split, with SITE providing the labels for the split column. The setup also eliminated several columns from the new table by including only CASSETTE and LOTID as columns.

Figure 5.45 Splitting Oxide CMP Data.jmp

Figure 5.46 is an excerpt of the new split table Oxide CMP Split.jmp. Splitting the table is not absolutely necessary, but it does remove the nine entries for each wafer and makes the interpretation of the contingency table somewhat easier, because one lot of wafers contains only 25 entities. The splitting process is actually the reverse of the stacking process demonstrated in earlier examples.


Figure 5.46 Excerpt of Oxide CMP Split.jmp

Generating the contingency table by modeling LOTID (Y) as a function of CASSETTE (X) produces the results in Figure 5.47. The figure shows that the six cassettes in this study consumed six lots of wafers (25 per lot) and that each cassette contained wafers from two lots. This arrangement allows one to determine the contribution due to lot and the contribution due to cassettes independently of each other.

Figure 5.47 Contingency Table for LOTID Assignments


Examining the Data Generating the column for DELTA THICK in the original data table and generating a variability chart produces the chart in Figure 5.48.

Figure 5.48 Variability Chart for DELTA THICK from Oxide CMP Data.jmp

Generation of the chart in Figure 5.48 specified ABRASIVE BATCH, CASSETTE, LOTID, and SLOT, in that order, as X-grouping variables. The figure does not suggest that the data contains any extremely unusual points, but it does suggest that an early firstwafer effect might exist in the first cassette, although that effect appears absent in later groups. However, the figure does show a problem regarding the assignments of ABRASIVE BATCH during the study. Apparently, the operators became concerned that the system might deplete the first batch at some point in CASSETTE = 4, so they substituted the second batch. This substitution presents a problem for the analysis, because otherwise a nested relationship exists between CASSETTE and ABRASIVE BATCH. The solution applied in this case was to exclude the data from the final eight slots in CASSETTE = 4 and analyze the remaining data. NOTE: Alternatively, the user could specify a more complicated model for the analysis and proceed accordingly. However, the author elected to follow the approach indicated for this example. Figure 5.49 shows one method for selecting the unwanted rows in the data table. After the system selects the rows, excluding them as a group is a single step.


Figure 5.49 Selecting Rows with Wrong ABRASIVE BATCH

NOTE: An alternative solution to the observed problem would be to designate the final slots in Cassette 4 as coming from Cassette 4A; this designation would continue the expected nested relationship. The example in the section “Polishing a Second Type of Oxide Film” in this chapter shows this solution in a similar situation. Figure 5.50 displays a new variability chart based on the filtered data. Figure 5.51 shows the variance component analysis done within the Variability Chart platform after excluding the selected points.


Figure 5.50 Variability Chart of Data after Excluding Points from CASSETTE = 4

The appearance of the results from the first few slots in CASSETTE = 1 strongly suggests an initial increase in removal rate. The line on the graph tracks the mean of each cell from SLOT to SLOT.

Figure 5.51 Variance Component Analysis after Filtering Data

The Variability Chart platform assumes that all factors are mutually nested, depending on the order given in defining the chart. This experiment saw to it that LOTID was not nested in either ABRASIVE BATCH or CASSETTE, so the analysis produced in Figure 5.50 is not absolutely correct. But does this situation make a significant difference in the interpretation of results? To determine that, the analyst must build a correct model and conduct a regression analysis.


Building a Model This analysis also excludes the data from SLOT = 17 through SLOT = 25 in CASSETTE = 4 to preserve the nested relationship CASSETTE[BATCH] just discussed. Figure 5.52 shows the setup in the Fit Model platform for this example.

Figure 5.52 Model Definition

Running the model produces Figure 5.53. The good news in the report is that variation among the lots used in the experiment was not a major source of variation in the process, and that variation between the two batches of abrasive was not excessive. Note that the input variable missing from the analysis is SITE; this contribution is in the residual value given and could be sensitive to optimization experiments. Disquieting news is that considerable variation existed between cassettes.


Figure 5.53 Variance Component Regression Results

Although the variance component values reported in Figure 5.53 differ from those in Figure 5.51, the order of each effect is very similar, and the differences are relatively small. Comments following Figure 5.50 suggested that the removal rate for the first slots polished in CASSETTE = 1 might have differed somewhat from others in that group. In the Effect Details section included with the regression report, the user can find statistical tests that compare the slots in each cassette and take into account the nesting environment. Figure 5.54 shows the path to take to perform these analyses. Figure 5.55 shows comparisons of SLOT = 1 to SLOT = 2, SLOT = 4 to SLOT = 1, and SLOT = 4 to SLOT = 2 in CASSETTE = 1.

Figure 5.54 Generating Comparisons of Effects

The concern is whether the means from the slots are the same or different; therefore, the correct line in the analysis is Prob > |t|. SLOT = 1 and SLOT = 2 are indistinguishable from each other, but they are both different from SLOT = 4. This finding supports the idea of an initial first-wafer effect in this process. This problem bears further investigation in future work.


Figure 5.55 Comparisons of Slots in CASSETTE = 1

Each variable effect has a similar display available, so the analyst can compare one cassette to another and different lots of substrates to each other as well.

Polishing Tungsten Films This example applies the CMP process to wafers having a conductive tungsten film on their surface. Figure 5.56 provides an excerpt of the raw data (Tungsten Data.jmp) for this example. The contingency table in Figure 5.57 shows how the team allocated wafers from some six lots of tungsten-coated wafers in the experiment. Some data loss occurred in handling the wafers and measuring the results, so not all 25 wafers appear for each of the six lots.


Figure 5.56 Excerpt of Tungsten Data.jmp

The process for creating the contingency table for this example is analogous to the process described for the previous example. Splitting Tungsten Data.jmp and creating Tungsten Split.jmp simplifies the appearance of the contingency table and eliminates the entries for each site.

Figure 5.57 Contingency Table Showing Lot Allocations in Tungsten Split.jmp

The plan as executed assigned two lots to each cassette to allow independent estimation of any effect due to CASSETTE and to LOTID in the study.


Examining the Raw Data Generating distributions of the values for DELTA THICK by CASSETTE suggests that a few high outliers were present in CASSETTE = 3. Similarly, the variability chart shown in Figure 5.58 suggests that some wafers from two lots in CASSETTE = 3 gave high removal rates. However, no compelling assignable cause existed for removing them, so all points remained in the analysis.

Figure 5.58 Variability Chart with Variance Component Analysis for Tungsten Data.jmp

Variance Component Analysis Selecting variance component analysis from the pull-down menu on the variability chart produces the same problem noted in the previous example. Therefore, the most reliable way to compute the variance components in this data is to fit a model to it. Figure 5.59 shows the specification of the model.


Figure 5.59 Model Specification for This Example

Figure 5.60 presents the results of the analysis. The contribution for variation due to SITE is equal to the residual variance component.

Figure 5.60 Regression Analysis Results for This Example

Compared to the previous example, the contribution due to LOTID is cause for some speculation. Examination of the Effect Details as shown previously in Figure 5.54 produces the results in Figure 5.61. Of the LOTID values used in this experiment, only LOTID = G, LOTID = H, or LOTID = K produced extreme differences in DELTA THICK compared to other LOTID values.


Figure 5.61 Detailed Comparison of LOTID Effect

Examination of the records associated with these LOTID values showed that they had been processed before and had been recycled (the original coating stripped off and a new layer deposited) in order to save wafer costs. Examination of other types of materials (Cu, Al, SiO2, and Si3N4) used in similar CMP studies found no other case where recycling the wafers had such a pronounced effect.

Lessons Learned This was the first and so far only case identified in which the practice of recycling polished wafers and redepositing films on them might lead to increased variation in the process.


Polishing a Second Type of Oxide Film The experiment in this example was a preliminary study intended to develop an initial understanding of the variation in a CMP process. This example comes from work done at a supplier site when the author was acting as a private consultant in passive data studies on CMP processes. Because of the proprietary information involved, the nature of the machine being tested and other details of the process are absent from this discussion. Figure 5.62 shows a schematic of the original experimental plan.

Figure 5.62 Experimental Plan for CMP Study

In the figure, SETUP refers to the preparation of the machine, including polishing pads, supporting films, slurries, etc. The two setups were to be the same; any difference detected between them could be due to any of the components associated with the setup procedure, but that detail was not included in this initial study. For example, the same lot of polishing pads provided pads for the operations, but individual pads within a lot could vary in their performance. Similarly, the polishing slurry used came from the same concentrated slurry lot, but any differences in mixing the slurry suspension actually used could produce performance differences. The intention was to detect any gross changes between setups and to explore individual elements within them in later experiments. As the illustration indicates, the experiment was to require six days total, with groups of three days associated with each setup procedure. Within those days, the machine was to process only two cassettes of 25 wafers each. The wafers used came from 12 individual lots of material: six dedicated to the first setup group and six dedicated to the second setup group. From the illustration, one can easily see that DAY nests within SETUP, and CASSETTE nests within DAY and SETUP. To allow separation of the effects due to differences in cassettes from any effect due to raw material lot, each cassette had wafers from two different lots. Note also that a nested relationship exists between LOTID and SETUP, as six lots were dedicated to each setup and CMP activity. The diagram in Figure 5.62 does not account for the 25 wafers used in each of the cassettes or the 52 sites measured on each wafer. Obviously, WAFER nests within CASSETTE and SITE nest within WAFER and CASSETTE, in addition to those nesting relationships already mentioned.


Examining the Raw Data Figure 5.63 shows an excerpt of the data table Second Oxide.jmp.

Figure 5.63 Excerpt of Second Oxide.jmp

Figure 5.64 shows the distribution of the DELTA THICK data. Although an analyst could brush the high and low outliers and try to determine their source, the author prefers to generate a variability chart. Creating the variability chart for the entire data set, using SETUP, DAY, CASS, and LOTID for the X-grouping variables (in that order) produces the results in Figure 5.65. In this figure, the extremely high and extremely low values are clearly visible and associated with only CASS = B on DAY = 1 and CASS = G on DAY = 4. A single additional high point also appears in CASS = F on DAY = 3. However, another problem also appears in the figure. Note that CASS = C appears in two days instead of the one originally intended. Apparently the operators set out to process three cassettes on DAY = 1 and either ran out of time or realized their mistake and stopped. This error defeats the original plan for the nesting in this experiment, so the analysis must include some form of correction.


Figure 5.64 Distribution of DELTA THICK

Figure 5.65 Variability Chart from Second Oxide.jmp


This correction can take one of at least three forms: 1. One could generate another variability chart using SETUP, CASS, and SLOT as the X-grouping variables in that order. From the new chart, one would determine the slots in CASS = 2 that were processed either on DAY = 1 or on DAY = 2. One could then write a row selection statement that captures either set and exclude it from the analysis. 2. Having identified the misprocessed slots using the variability chart just described, one could edit the original table and change the level of one of the slots. For example, those slots from CASS = C processed on DAY = 2 could be assigned CASS = C1 by editing the table and making the appropriate changes. 3. A more elegant solution would be to generate a new column, designate it Character/Nominal, and create a conditional formula to write the new assignments into the new column. Figure 5.66 shows the steps in this approach.

Figure 5.66 Creating a Formula to Rename CASS


After defining a new column (CASS MOD) with data type Character, one creates a formula. The top panel in Figure 5.66 shows the selection of the conditional If clause. Next, one adds an And element (the & symbol in the second panel comes from the conditional And choice) so that the expr box contains two empty rectangles. Then one fills each rectangle with the comparison a == b, as shown in the third panel. Finally, one fills in the clauses as shown in the bottom panel. Evaluating this formula fills the entries in the new column either with the original cassette designation or C1.

Analyzing the Results This analysis involves six variables (SETUP, DAY, CASS MOD, LOT, SLOT, Site). An analysis should allow Site to become the residual. Although the Variability Chart platform can handle a problem this complex, a better approach uses Fit Model in order to specify the model correctly. Figure 5.67 shows the setup for the model, assuming the very high and very low values for DELTA THICK have been excluded and that implementation of the new numbering system for CASS as CASS MOD has occurred (the modified table is saved as Second Oxide MOD.jmp).

Figure 5.67 Model Specification for Second Oxide MOD.jmp

Figure 5.68 shows the variance components from the regression report.


Figure 5.68 Variance Components from Regression Report

Lessons Learned Here are the most important lessons learned from this study: 1. Changing the consumables (SETUP) during the experiment had no material effect; that is, SETUP did not contribute noticeable variation to the process. 2. The 12 lots of raw material used also contributed no significant variation to the process. Each of these lots was prepared especially for this study; no recycled materials were involved. The large contribution of within-wafer variability (residual) is not surprising. Other work on similar CMP processes suggested that most of this variability was due to more rapid coating removal at the center compared to the edges of wafers, or vice versa. To determine whether that situation existed here, one could generate a summary of the original data by wafer site (excluding the extreme values), and then use the Fit Y by X platform to generate a bivariate plot of the mean of the amount removed versus Site (after changing Site to Numeric/ Continuous). Figure 5.69 shows this graph with a fitted line added.


Figure 5.69 Mean Amount Removed versus Wafer Site

In the figure, Site = 1 is at the center of the wafer, and Site = 52 is at the edge. Although the fitted line suggests a decrease in the amount removed as one moves from the center to the edge, the slope of the line is not significantly different from zero statistically. Therefore, the only supported conclusion at this point is that the amount removed varies randomly across the wafer and has no recognizable pattern. Another view of the data plots the mean amount removed versus the wafer number. To generate this graph requires several steps: 1. One splits the original table to simplify the next step. 2. One generates a new column containing a count from 1 to 300 (the number of rows in the resulting table) in 300 steps; this step provides numbers for the actual wafers. 3. One stacks the split table using the STACK option under the Tables menu, and then summarizes this table (generates the mean of amount removed) by wafer number and day. 4. One graphs the mean amount removed versus wafer number. Figure 5.70 is the result. NOTE: A somewhat less complicated method for including wafer number in a summarized table is to generate the mean DLTA_THK using SETUP, DAY, SLOT, and CASSETTE as grouping variables. This method produces a table with 300 rows (one for each wafer). Adding a column to this summary table as described previously completes the process. However, generating the column


for wafer count is not absolutely necessary in this approach. Generating the overlay plot without defining an X variable provides the same graph as shown in Figure 5.70. In Figure 5.70, the vertical lines represent approximate divisions between days. The first 150 wafers or so were processed using SETUP = 1; the remainder were processed using SETUP = 2. The most obvious feature in the figure is that the average removal rate increased each day as the number of wafers processed increased. A suggested explanation for this phenomenon is that the abrasive pad used to polish the surface of the wafer heated up during the process and increased the amount of material removed (for all wafers processed for the same amount of time). Because each day (except DAY = 1 and DAY = 2) involved two cassettes for a total of 50 wafers, the trend apparently continued from cassette to cassette on a given day, giving rise to the relatively large variance component observed (denoted by SLOT). Also apparent from the figure is the similarity in results between the three days that used different batches of consumables.

Figure 5.70 Overlay Plot of Mean Amount Removed versus Wafer


Summary of Passive Data Collections Planning is one of the most important aspects of any passive data experiment. Preferably, the planning phase involves a team of investigators and operators so that all can discuss possible sources of variation in the process and plan a sampling plan accordingly. Because the majority of processes in the semiconductor industry involve processing batches of wafers, the plan must include a means for separating any effect due to raw materials from differences due to batches. The examples in this section have shown one mechanism for avoiding that conflict. The example in the section “Identifying a Problem with a New Processing Tool” illustrated how careful attention to the data and the appropriate graphing can help identify inherent problems with a processing tool. Identifying these problems is analogous to finding problems with measurement tools by conducting separate repeatability studies before embarking on a relatively large-scale measurement study. The first example emphasized the importance of sampling a process correctly in order to capture sources of variation in that process. Failure to sample adequately can easily lead to overlooking some important source of variation. When in doubt, sample excessively. Later trials can always reduce the number of monitor wafers in a process. The combination of the Variability/Gage Chart platform with distribution graphs in JMP provides an extremely valuable tool for finding unexpected patterns in large bodies of data. When the situation permits, the Variance Component calculations associated with the Variability Chart provide extremely facile calculations of sources of variation. Where more sources of possible variation exist or the experiment requires careful specification of a model, the analyst should use the more complicated Fit Model platform, as demonstrated in these examples. JMP 6 is a powerful software package, and with the improvement of the REML engine in the Fit Model platform, it can analyze relatively large collections of data with many variables, particularly with complicated nesting structures. Finally, successful passive data studies that do not contain extremely large sources of variation and in which the process meets or exceeds specification can form the basis for developing appropriate control charts. The next chapters discuss that process.

3

P a r t

Developing Control Mechanisms Chapter

6

Overview of Control Chart Methodology

Chapter

7

Control Chart Case Studies

263

233


C h a p t e r

6

Overview of Control Chart Methodology Introduction 234 General Concepts and Basic Statistics of Control Charts 235 Types of Data 235 The Normal Distribution 236 How Many Samples? 236 Examination of Data 237 Types of Control Charts and Their Applications 237 Charts for Variables Data 237 Charts for Attributes Data 245 Special Charts for Variables Data 249 Trend Analysis 252 The Western Electric Rules 253 The Westgard Rules 254 Implementing Trend Rules 255 Capability Analysis 256 The Cp Statistic 256


The Cpk Statistic 258 The Cpm Statistic 260 Generating Capability Statistics in JMP 261 Control Charts Involving Non-Normal Data 261 Summary 261

Introduction Previous chapters have introduced the use of JMP software for understanding measurement process capability and analyzing process data to determine sources of variation. This chapter introduces concepts of Statistical Process Control (SPC) practices and philosophies, but makes no attempt at exhaustive discourse on this subject; other widely available texts cover this subject in detail. An inherent assumption in going forward in this discussion is that the measurement tools used to monitor a process and the sources of variation within a particular process have been understood. This chapter introduces the following major topics:

developing control charts and limits for process performance, including the basic statistics involved

recognizing patterns in control charts (trend analysis)

estimating process capability

Generating control charts for a process on a manufacturing floor, whether those charts monitor the measurement processes or the manufacturing process, does nothing to improve the quality of those processes. Instead, control charts provide a record of how a process is performing based on historical data and can help identify when something has changed. The control chart can provide a signal, but those responsible for that process must react to that signal in a timely fashion to determine the cause for the upset and to eliminate the cause from future work. That is, a successful quality program incorporates appropriate control chart graphics integrated with an action plan that a team actively pursues. Establishing a valid control chart for any process depends on an appropriate estimate of the stable behavior of that process: What is the correct estimate of the central tendency (mean), and what is the correct estimate of variation to apply when generating the control chart? With proper software support, applying these estimates to generate control charts becomes almost trivial. The major problems in the decision process lie in assuring that

Chapter 6: Overview of Control Chart Methodology 235

the data being used is truly representative of the stable behavior of the process, and in determining the proper estimate of variation to use in establishing the control limits. The concept of predicting future behavior of a process from historical data originated with the work of Dr. W. A. Shewhart and others at Western Electric Corporation in the 1920s. Although many of the calculations devised at that time are outdated in this age of computers, the principles defined then still apply and form the basis for understanding when a process is “behaving itself” and identifying those times it has started to “misbehave.” The following sections introduce basic concepts associated with control charts and their applications and discuss briefly and generally the support found within JMP. Some of the examples used derive directly from sample data tables furnished with JMP software. This chapter introduces various types of control charts and their applications along with means for detecting trends in data and for computing the capability of a process. Chapter 7 presents more detailed examples of each topic. For additional information on control charts, see Sall, Lehman, and Creighton (2001), Wheeler and Chambers (1986), Duncan (1986), and the Western Electric Handbook (1956). Chapters 37, 38, and 39 of the JMP Statistics and Graphics Guide are also a valuable resource.

General Concepts and Basic Statistics of Control Charts Types of Data From a control mechanism standpoint, process data falls into two broad categories: variables and attributes. Variables data can assume any value in a range of values; that is, the data is continuous in nature. The major underlying assumption in applying control charts to variables data is that a process operating undisturbed generates random observations that fall into a normal distribution with some mean and standard deviation. If that is true, then simple statistical summaries can describe its behavior. When an upset occurs, the process will generate data points that are not part of that distribution, and graphical or computational methods can detect them. Attribute data generally consists of counts of defects or failures produced by a process. This data is not continuous in the sense that the numbers of defects or failures generally are integers or proportions. When attribute data involves failures, an observation can be good or bad, so the distribution of observations follows the binomial distribution.


The Normal Distribution The simple assumption that variables data generate a normal distribution of observations provides the basis that Shewhart and others used in developing the ideas of control limits for processes. Figure 6.1 represents a unit normal distribution function with mean (μ) = 0 and standard deviation (σ) = 1, generated from Unit Normal Distribution.jmp. Variables data from a process operating undisturbed under stable conditions theoretically generates a distribution of observations like this, although the mean of the observations and standard deviation will differ from that of the unit normal distribution.

Figure 6.1 A Unit Normal Distribution

In the unit normal distribution, only ~ 0.27% of data points will lie beyond ± 3 standard deviations from the mean. This means that ~ 99.73% of the data should fall between those boundaries if the theory holds and the process suffers no outside influences. Points lying beyond those boundaries become relatively unlikely or rare events. Therefore, most SPC software, including JMP, defaults to control limits at ± 3σ. If the estimate of the standard deviation is correct or reasonable, then a stable process should provide data within those limits. In JMP, the user can specify a multiple of the sigma used to determine the control limits when a particular situation warrants such a change.

How Many Samples? The section “Uncertainty in Estimating Means and Standard Deviations” in Chapter 1 illustrated the uncertainties in estimating both means and standard deviations from sample data. Stable estimates for the mean require at least 15 independent observations, whereas similar estimates for the standard deviation require approximately 30 such


observations. Any control limits calculated with a smaller sample size than these might change considerably as sample size increases when more data becomes available. That does not mean that analysts should not estimate limits based on less than optimum samples; rather, it means that they should regard those initial estimates as approximate and be willing to modify them as more observations become available.

Examination of Data Data representative of the stable performance of a process forms the only valid basis for establishing a control mechanism for that process. Therefore, careful examination of the data should occur in order to remove any unusual values. The author commonly combines distributions of data with suitably configured variability charts in examining data. Certainly other graphics can be useful as well. Because a control chart should invoke a time element in the data used, the data collected should contain some form of date-and-time stamp. In addition, the most recently acquired data should appear last in any data table. The author has found that many automated data collection systems reverse this order, so critical examination of raw data with possible subsequent sorting is essential. Where control charts monitor the behavior of measurement tools, be sensitive to the possibility of wear on the monitor objects used; many measurement processes can inflict minor damage on the object used for the monitoring. Including data representing that damage in a control mechanism usually provides a false signal that the measurement process has changed, when the fault actually lies in the monitor being used.

Types of Control Charts and Their Applications The next sections illustrate the likely types of charts an engineer or analyst might use for a process. More detailed discussions in the next chapter generally follow the order given here. Some of the examples used come from data tables provided with JMP software. The majority of those used in the next chapter represent case histories drawn from several industries, predominantly from semiconductor manufacturing.

Charts for Variables Data JMP offers a flexible and versatile platform for generating a variety of different types of graphs, so the “one size fits all” dogma supported by too many unsophisticated texts on


control charting certainly does not apply. A central requirement for creating the right chart or charts to monitor a process is understanding the nature of the data being collected along with establishing a correct estimate of the variation of that process. Using the wrong estimate of process variation produces charts with improper control limits, making their use in improving the quality of that process impossible. JMP supports several charting options that apply to variables data: XBar, R, S, Individual Measurement, and Moving Range. An analyst might also apply specialized chart options such as the uniformly weighted moving average (UWMA), CUSUM, exponentially weighted moving average (EWMA), and Levey-Jennings chart to variables data. Ideally, any control chart application for variables data includes at least two graphs—one to monitor the location or central tendency of a process, and the other to monitor the variation in that process. Attempting to monitor a process with only one chart can lead to failure to detect a change in that process. For example, the chart for the variation in the process might detect a process change before the chart monitoring the location. If a process contains multiple sources of variation, then a proper statistical process control protocol should establish a chart for each of those sources of variation. Observations over an extended period might show that some of the charts do not provide useful information, but making that assumption in the face of limited data introduces the risk of not monitoring the correct parameters of a process.

The XBar, R, and S Charts The XBar, R, and S charts are the most common and basic control charts often discussed in simplistic SPC treatments. Too many naïve SPC practitioners attempt to make these types of control charts fit all situations. Combining the XBar chart with either the R or S charts tracks both the location and the variation in a process. This combination is valid, provided the samples used are mutually independent of each other. For example, if a continuous process is producing widgets at some rate, then sampling some number of those widgets periodically and using the information from them to produce this combination of charts is a valid approach, because the value of an individual widget probably does not depend on the widget that preceded it. To illustrate when this approach is valid, Figures 6.2 and 6.3 show excerpts of two different data tables. Figure 6.2 shows part of Generators.jmp, whereas Figure 6.3 shows RS Data Sort.jmp.


Figure 6.2 Excerpt of Generators.jmp

Figure 6.3 Excerpt of RS Data Sort.jmp

NOTE: Sorting the original data table RS Data.jmp was necessary because entries were not in ascending time order as discussed earlier. The sorting operation generated the new table by sorting on DATE TIME, LOT, and SITE. The data in Figure 6.2 came from a manufacturing process for electrical generators. At random times each day, operators pulled four individual devices from the production line and tested each for their output voltage. The data in Figure 6.3 came from a process that deposits a particular film on batches of wafers. From each lot of wafers processed, operators selected a single wafer and measured the resistivity of the coating at five sites across the wafer surface.


One way to help decide whether individual observations in a data collection are mutually independent of each other is to use common sense. In the generator case, each individual is separate in time from each of the others. In the semiconductor case, the five sites measured are all on the same object and within inches of each other. The upper panel in Figure 6.4 is a variability chart from the generator data; the lower panel shows the variability chart from the resistivity measurements.

Figure 6.4 Variability Charts Based on Data in Figures 6.2 and 6.3


The line in each chart connects the cell means of the data. Figure 6.5 shows the variance component analysis from each of these charts; the left panel contains the analysis of Generators.jmp, and the right panel contains the analysis of RS Data Sort.jmp.

Figure 6.5 Variance Component Analyses of Current Examples

For Generators.jmp, the variation among dates is negligible, but the variation among generators on a specific day (Within) essentially accounts for all the variation in the observations. For RS Data Sort.jmp, the variation among dates is very large, but the variation among the measurement sites (Within) is very small. These observations suggest that the sample generators are mutually independent of each other and that the measurement sites on the sample wafer are not. For data involving subgroups, JMP supports the combinations XBar – R and XBar – S and derives the control limits imposed on the XBar chart either from the range or the standard deviation of the samples taken. Computations of the standard deviation could become somewhat tedious in the 1920s when charts like these first appeared; therefore, an estimation of the standard deviation based on the range of the points and the sample size became standard practice. Following that process today is not necessary, given the computing power of modern software. Chapter 7 illustrates the generation of these charts in more detail. Figure 6.6 illustrates the results of generating XBar – S charts for each of these examples. The graphs in the left panels are for the generators, whereas those in the right are for the measurements on the wafers.


Figure 6.6 XBar-S Charts of Voltage from Generators.jmp and RS from RS Data Sort.jmp

The control limits for the XBar charts will differ slightly, depending on the method used to estimate the standard deviation for that chart (range of samples or standard deviation of samples). In Figure 6.6, the control limits imposed on voltage based on the variation between the individuals in each sample appear reasonable. However, the control limits generated by using the variation between sites on each wafer produced extremely narrow boundaries, resulting in almost all the points plotted being outside them. Figures 6.4 and 6.5 clearly showed that the variation of the five measurements taken within a wafer varied less than the observed means of the monitor wafers taken from each lot. Therefore, using the within-wafer variation as the basis for the control chart for the resistivity data is not a valid choice. However, given that most of the variation (in fact, nearly 100%) in the data derives from variation among samples of generators, the standard deviation of the performance from individual generators on a given day is the appropriate estimate of


variation for that example. The analyst must be sensitive to the fact that basing the estimate of variation for a process on multiple measurements of single objects can lead to serious errors in constructing control charts.

Individual Measurement and Moving Range Charts (IR) Normally the combination of individual measurement and moving range (listed as IR in JMP) charts applies to data collection scenarios that provide only one data point per group. For the individual measurement chart, the system estimates the standard deviation necessary to compute control limits as follows: 1. The system computes the range between pairs of adjacent points: between 1 and 2, between 2 and 3, etc. 2. The next step divides the average of these ranges by a conversion factor based on the number of points (span) used to determine each range. 3. The result is an estimate of the standard deviation used to determine 3σ limits for the chart. 4. The centerline of the individual measurement chart is the average of the observations. The centerline of the moving range chart is the average of the individual ranges computed above. The control limits for this chart involve an additional conversion factor applied to the average moving range dependent on the span specified for the individual range calculations. For more information, see Duncan (1986, Chapters 21 and 22) or Wheeler and Chambers (1986, Section 9.2). Figure 6.7 illustrates this combination based on the table Stress Data.jmp. In this example, operators measured the stress of a coating on a single wafer taken from each processed lot.


Figure 6.7 Individual Measurement and Moving Range Charts from Stress Data.jmp

NOTE: The data table is in reverse chronological order. For this example and some that follow, the author did not sort this table as one should do if generating actual control charts. The combination of individual measurement and moving range charts will find considerable application as situations become more complicated with batch operations involving nested variation. In the previous example taken from RS Data Sort.jmp, a useful control chart application for that process would compute the mean for RS and generate the individual measurement and moving range charts from that statistic. An additional chart would monitor the within-wafer variation using a standard deviation chart. A variety of


scenarios involving nested variables and the means to prepare control charts for each source of variation in that data are outlined in Chapter 7.

Charts for Attributes Data In general, attribute charts deal with observations of defects. For example, the observations could be the number of defective parts in a group of parts, or the observations could be the number of blemishes or defects detected on an object or objects. In the first case, the size of the group forms a boundary for the number of defects; all of the members of the group could be defective, or some fraction of the total could be defective. In this case, either the P or NP chart applies. In the second case, the possible number of defects has no upper boundary; an object or sample could contain an infinite number of blemishes or defects. For this case, the C or U chart applies. One of the problems with count data is that the observations are discrete rather than continuous. If the response variable involves counts that are very small compared to the total number of items tested, then the application of the charts described in the following sections is a reasonable approach. However, in the cases where the number of counts observed is quite large, then the discreteness of the data does not present a particular problem and those observations can be considered typical variables rather than counts. Often the control charts produced in this manner using the individual measurement chart combined with a moving range chart will provide more meaningful and useful control limits than those produced using the “legal” attribute charts. If using an attribute chart produces what appear to be absurd and irrational limits, then try generating the individual measurement and moving range charts instead. An example in Chapter 7 presents this second case.

P Charts and NP Charts The P chart plots the proportion of defective items in a sample, whereas the NP chart plots the total number of defects in a sample. Where the data collection involves dividing items into good or bad, the normal distribution does not apply, but the binomial distribution does apply. Calculation of control limits in this situation requires another approach for calculating an estimate of the standard deviation of the data, as shown in Equation 6.1.

s=

p (1 - p )

6.1

In the equation, p represents the average proportion of defective (or acceptable) items in all the samples.


NOTE: Although it makes no difference mathematically whether the observer counts defective or acceptable items, focusing attention on the defectives is usually preferable in a quality control situation. If a process produces only a relatively small number of defective items, detecting small changes in a small number is often easier than detecting small changes in a large number. The centerline of a P chart becomes the average number of defective items for a group of size n, whereas the control limits are those shown in Equation 6.2. Equation 6.2 actually provides a confidence interval for the average number of defects per group. For a P chart, the upper control limit cannot be > 1, nor can the lower control limit be < 0. SE represents the standard error of the data, and k represents the number of standard errors to use for the spread of the control limits (usually k = 3 as with other control charts).

CL = p UCL = min (CL + k (SE ) ,1) LCL = max(CL - k (SE ) , 0) SE =

6.2

s n

NOTE: As stated in the previous section, the P and NP charts work best when the number of defects is very small (0, 1, 2) compared to the number of samples observed. The table Washers.jmp (provided with the software in the directory Support Files English\Sample Data\Quality Control) contains data suitable for P or NP charts. This example contains 15 lots of 400 washers with recorded defects in each lot. The left panel in Figure 6.8 shows this data table; the right panel of Figure 6.9 contains the P chart, and the right shows the NP (total defects) chart for this data.


Figure 6.8 Display of Washers.jmp

Figure 6.9 P and NP Charts from Washers.jmp

The only real difference between the two charts is the scale of the Y-axis in each.

U Charts and C Charts The C chart plots the number of defects in an inspection unit (such as a painted panel) and requires a constant sample size. The U chart plots a ratio of the number of defects seen to the number of inspection units and can have unequal sample sizes. The counts of defects seen in either case have no limit; they are unbounded. As such, a Poisson distribution provides the models for these counts. The Poisson distribution has only one parameter associated with it—its mean number within a time period or unit area. The


standard deviation used for determining control limits is u , where u is the mean number of the counts observed per time period or unit area. The data table Shirts.jmp (left panel of Figure 6.10), provided with the software in the Quality Control folder as described in the previous example, satisfies the requirements for a C chart. A manufacturer ships shirts in boxes of ten each. An inspector records the number of defects per shirt before shipment. The response of interest to the manufacturer is the average number of defects per shirt in a box, found by dividing the total number of defects in a box by 10. The right panel in Figure 6.10 illustrates this C chart.

Figure 6.10 Excerpt of Shirts.jmp and Derived C Chart

The data table Braces.jmp (left panel of Figure 6.11) provided with the software requires a U chart. In this example, the data recorded is the number of defects found in a variable number of boxes of braces being inspected. The upper and lower bounds of the chart vary according to the number of units (boxes) inspected. The right panel in Figure 6.11 illustrates this chart.

Figure 6.11 Excerpt of Braces.jmp and Derived U Chart


Special Charts for Variables Data The charts discussed in the following sections have special applications for process monitoring in the following circumstances:

Samples take some time to generate (UWMA).

Typical control charts do not detect aberrations quickly enough (EWMA and CUSUM).

Considerable historical data about a stable process exists (Levey-Jennings).

The UWMA Chart The uniformly weighted moving average (UWMA) chart plots the moving average of a set of observations. That is, the system determines the average of observations 1 and 2, 2 and 3, 3 and 4, etc. The grand average of those averages forms the centerline of the chart. As stated previously, the UWMA chart is most effective when it takes a considerable amount of time to produce a sample. A typical scenario for it might require adding in the average of a new observation with the one immediately preceding it, and then dropping a much older observation. Except in these specific circumstances, the UWMA chart provides about the same information as the individual measurement chart. Figure 6.12 shows a UWMA chart produced by Stress Data.jmp. Notice that since the first lot does not have a previous observation to average with it, the control limits step wider for that observation.

Figure 6.12 UWMA Chart from Stress Data.jmp


Monitoring variation between lots when using this chart could require the moving range chart or some modification of it.

The EWMA Chart The exponentially weighted moving average (EWMA) chart is a more sophisticated version of the UWMA chart. In the UWMA chart, all observations have equal weight. In the EWMA chart, the most recent data have more weight, with older observations assigned some fractional weight. That is, each point plotted is the weighted average of all previous subgroup means, including the current observation. The user assigns a weight in the range 0 ≤ weight ≤ 1. Assigning weights to smaller numbers helps guard against reactions to small shifts. Figure 6.13 shows the EWMA chart based on Stress Data.jmp with the weight set to 0.5. Notice in this chart that the control limits do not stabilize until a number of observations have been considered.

Figure 6.13 EWMA Chart from Stress Data.jmp

The CUSUM Chart One of the shortcomings of a conventional Shewhart control chart is that changes in a process before the observations violate a control limit might not be detected early enough. The CUSUM chart is one approach to hastening that detection. This chart tracks the cumulative differences of observations from a target value and can signal a departure from expected behavior more quickly than a conventional control chart. JMP provides extremely flexible support for this chart so that a user can tailor it to


meet a specific requirement. The example in the left panel of Figure 6.14 shows a version of this chart for the data contained in Generators.jmp.

Figure 6.14 A CUSUM Chart Based on Generators.jmp

NOTE: In configuring this chart, the author set the target value at 325 and set the H parameter to 1 rather than the default 3. H is the vertical distance from the origin of the V-mask to either arm of that V-mask. One should specify this parameter as some multiple of the standard error of the process; lower values of H make the CUSUM chart more sensitive. The chart in Figure 6.14 tracks the cumulative sum of differences from the observed mean voltage to the target. The horizontal line represents the average of those differences. The angled line at the right of this panel is a V-mask. The user can configure the shape and character of this mask depending on the circumstances being monitored. The mask shown is a two-sided mask that will detect aberrations of the mean above or below the target value. When the line connecting the plotted points crosses this mask, an upset has occurred. The left panel in Figure 6.15 contains a copy of the original table as Generators2.jmp. To this table has been added an additional set of observations (Rows 49 through 52). The right panel shows the effect on the CUSUM chart, assuming that the user had saved the script for the original chart to the original table. Once new data enters the data table that contains a script for a control chart, the system updates that chart immediately upon executing the script.


Figure 6.15 Generators2.jmp and a Violation of the CUSUM Chart: The Effect of Adding a New Set of Observations to Generators.jmp

Note in the right panel how the average deviation from the target has changed, and that the line tracking the observations now violates the mask. The conventional XBar chart from this data does not show a violation, so this example provides an early warning of a process change. For a more detailed discussion of the CUSUM chart (and other control charts), see Duncan (1986). In addition, the JMP Statistics and Graphics Guide (Chapter 39) provides considerable detail about defining such charts.

The Levey-Jennings Chart When a long-term estimate of the standard deviation of a process is available, the LeveyJennings chart applies. This chart tracks a process mean versus an estimate of standard deviation and do not provide a similar track for variation; supposedly that quantity is known and stable. This book does not consider these charts further.

Trend Analysis When an observation is beyond a 3σ control limit, then clearly some unusual event has occurred that warrants further attention. The likelihood that a stable process would produce such a point is small—approximately 3 chances in 1000. Such a violation is a clear signal of a process upset. What is not so clear is whether some other nonrandom behavior exists in a control chart. To this end a series of rules are available that the analyst can use to examine a control


chart to see whether some unusual or unexpected pattern exists in the data being plotted. JMP supports two sets of such rules—the Western Electric Rules, invented about the same time as control charts; and the Westgard Rules. The Western Electric Rules require a constant sample size, but the Westgard Rules rely entirely on estimates of the standard deviation and do not require constant sample sizes.

The Western Electric Rules The Western Electric Rules divide the space between the control limits into zones, each designated by a multiple of the process standard deviation. These rules are a natural consequence of the work done in the 1920s and 1930s in defining control charts. Zone C lies within one standard deviation of the centerline; Zone B lies beyond Zone C and within two standard deviations of the centerline; Zone A lies beyond Zone B, but within the control limits for the process. Nelson (1984, 1985) added additional interpretations of the original rules and provides additional information on the use of these rules. Table 6.1 summarizes these rules and their meanings.

Table 6.1 Special Causes Tests (Western Electric and Nelson) Test Number

Observation

Meaning

1

A point beyond Zone A

2

Nine consecutive points on either side of the center line Six points steadily increasing or decreasing Fourteen consecutive points alternating up and down Two out of three consecutive points within one standard deviation of the center line (Zone C)

A shift in the mean, an increase in variation, or an aberration. A chart of variation can rule out an increase in variation. A shift in the mean.

3

4

5

A drift in the process.

Systematic effects due to operators, machines, or raw material lots. A shift in the process mean or an increase in variation.

(continued)


Table 6.1 (continued) Test Number 6

7

8

Observation

Meaning

Four of five consecutive points two or more standard deviations from the center line Fifteen consecutive points within one standard deviation of the center line (Zone C)

A shift in the process mean.

Eight consecutive points on either side of the centerline, but all greater than one standard deviation from the center line

Stratification, in that the observations in a single subgroup could come from various sources with different means. Also stratification of subgroups, but subgroups come from different sources with different means (similar to Test 4).

The SEMATECH Statistical Methods Group warns that overzealous application of these rules can cause operators and engineers to spend too much time looking for assignable causes when none actually exists—that is, this activity can lead to a host of false positive signals. Of this group, the clearest signals come from Test Numbers 1 through 4 in Figure 6.16. The rest can provide meaningful information, but one should use them wisely and only on mature processes. NOTE: The SEMATECH Statistical Methods group adopted a member company standard and defined a mature process as one under SPC for at least two years.

The Westgard Rules The Westgard Rules rely only on the standard deviation of the observations and do not require constant sample sizes for validity. Table 6.2 summarizes these rules.


Table 6.2 Westgard Rules Rule 12s

1

3s

2

2s

4s

4

1s

10

X

Comments This rule commonly applies to Levey-Jennings plots where one sets the control limits to two standard deviations beyond the centerline. A point falling beyond this point triggers this rule. This rule commonly applies to any plot with limits set three standard deviations from the centerline. This rule is quite similar to Western Electric Rule 1. This rule is triggered when two consecutive points are more than two standard deviations from the centerline. This rule is triggered when consecutive observations are two standard deviations above the centerline and below the centerline. This signal might be due to variations in machines, operators, raw materials, etc., similar to Western Electric Rule 4. This rule is triggered when four consecutive observations are more than one standard deviation from the centerline. This rule is triggered when 10 consecutive points are on one side of the centerline, similar to Western Electric Rule 2.

Implementing Trend Rules Each set of rules intends to identify nonrandom or changing behavior in a process. As stated earlier, overzealous application of either or both sets of rules can lead to false positives that require an engineering team to spend considerable time looking for an assignable cause that might not exist. To implement these rules, one selects the Test option on the pop-up menu associated with a control chart. The analyst can invoke either or both sets of tests on a given control chart. Some examples in Chapter 7 illustrate the application of these rules to control charts.


Capability Analysis If a process is in a state of statistical process control, estimating its capability produces a single number that states the likelihood that the process will produce material beyond a specification limit. Process capability statistics are probably the most misused and least trustworthy statistics commonly used in manufacturing environments. The specification limits used to rate the process often have little to do with the real requirements of that process; they often are arbitrary numbers assigned with little or no regard for physical requirements. Manipulating the data used to produce a capability statistic is also commonplace. If a particular capability statistic is not high enough, then the dishonest observer can manipulate the calculations to produce a desired result. Finally, management often uses small changes in capability statistics to reward or to penalize a team. Small changes in capability statistics are significant only after they survive dozens (if not hundreds) of batches.

The Cp Statistic The simplest capability statistic is Cp. This value estimates how many specification ranges fit into the natural variation of a particular process. Equation 6.3 illustrates the calculation. Cp =

where

USL - LSL 6s process

6.3

USL is the upper specification limit, LSL is the lower specification limit, and 6σ is the natural expected variation in the process.

If Cp = 1.0, then a small number of observations might fall outside the specification limits. Actually, based on the previous discussion of the normal distribution, some 0.27% (2700 parts per million) of the observations might fall outside the specification limits. To illustrate, Figure 6.16 shows an excerpt of CpCalc.jmp. All columns in that table contain computed values.


Figure 6.16 Excerpt of CpCalc.jmp

NOTE: The simulations in CpCalc.jmp assume that a stable process generates normally distributed observations with μ = 0 and σ = 1; that is, the hypothetical process generates observations that lie in a unit normal distribution. The Index column represents the location of the specification limits for the process above and below the process mean in units of σ. The Fraction Remaining column computes the fraction of the area under a unit normal distribution remaining beyond ± Index. Figure 6.17 is an overlay plot based on CpCalc.jmp that shows how differing Cp values affect the parts per million of observations that lie beyond either specification limit. Both Figures 6.16 and 6.17 indicate that the parts per million of observations that lie beyond specification limits are inversely proportional to the capability potential of the process. Increasing the capability potential of a process (ideally by reducing the variation in it) will produce marked changes in the PPM of observations exceeding specification limits. For example, increasing capability potential from 1.00 to 1.33 lowers the PPM value from ~ 2670 to ~ 63. Increasing capability potential still further to 1.5 provides a further 10-fold reduction in PPM exceeding specifications (~ 7). Six Sigma quality level corresponds to a capability potential of 2.0, or only about 0.002 PPM of observations lying outside specification limits—essentially none. Achieving this quality level also allows minor variations in the process mean without materially affecting the PPM level of the process.


Figure 6.17 Capability Potential versus PPM Outside Specifications

The Cpk Statistic As shown in Equation 6.3, Cp considers only the variation in a process and does not consider the location of the process. Figure 6.18 provides an excerpt of Mean Shift.jmp, another table with simulated data.


Figure 6.18 Excerpt of Mean Shift.jmp

The target for these simulated processes is a mean of 50 with a tolerance of ± 20 units. Figure 6.19 illustrates the distributions of the data from these two simulations with superimposed normal density curves and specifications.

Figure 6.19 Distributions of PROCESS 1 and PROCESS 2 from Mean Shift.jmp


The mean of PROCESS 1 centers nicely on the target value, but that for PROCESS 2 is displaced to a slightly higher value. Both processes have identical standard deviations— 5.084. A statistic that accounts for the centering is Cpk, calculated as shown in Equation 6.4. From the data used to generate Figure 6.19, PROCESS 1 has a Cp ~ 1.3 and a Cpk ~ 1.3 (left panel of Figure 6.20). Note that although PROCESS 2 also has Cp ~ 1.3, Cpk drops to ~ 1.0 because of the violation of the upper specification limit. USL − y 3σ process K y − LSL Cpl = 3σ process Cpk = min(Cpu, Cpl ) Cpu =

6.4

Figure 6.20 Capability Indices for PROCESS 1 and PROCESS 2 from Mean Shift.jmp

The Cpm Statistic Also reported in Figure 6.20 is Cpm. Calculations of this statistic relate closely to the Cp and Cpk statistics and provide a more rigorous measure of the centering of a process within specification limits. Equation 6.5 illustrates its calculation. As the mean of the process approaches the target value, the value of Cpm approaches those of Cpk and Cp.

Cpm =

min ( target - LSL,USL - target ) 3 s + ( y - target ) 2

2

6.5


Generating Capability Statistics in JMP JMP offers three options for generating capability statistics, two of which do not truly satisfy the requirement that a process be in a state of statistical process control before capability statistics are meaningful. These two options include selecting Capability Analysis on the pop-up menu associated with distribution graphing (as was done in this example) or choosing that option on the window where one defines a control chart and launches the platform. The third option appears on the pop-up menu associated with a control chart. If the behavior of the data in the chart does not violate control limits, then computing capability indices for the process is logical and valid. Each option provides similar results, and each requests specification limits and target values. Capability analyses associated with control charting activities are part of Chapter 7.

Control Charts Involving Non-Normal Data The discussions so far in this chapter have assumed that processes generate observations that are normally distributed or that have binomial distributions (P and NP charts) or Poisson distributions (C and U charts). Some data, particularly particle data generated in semiconductor manufacturing, does not seem to follow any particular distribution. Research funded by SEMATECH in the mid 1990s at the University of Arizona (Phoenix) and the University of Texas (Austin) invented an approach that provides reasonable control limits for particle data, particularly using the fitting of a mixture of Poisson distributions to the data. JMP does not support this distribution fitting approach in release 6.0, but potentially useful approximations for limits result from the Distribution option in graphing the data. Chapter 7, “Dealing with Non-Normal Data,” illustrates this approach based on case studies from the semiconductor industry. The approach might be general enough to apply to any data that doesn’t seem to fit a particular distribution.

Summary This chapter introduced concepts that apply to generation of appropriate control chart applications. JMP supports a wide variety of control charting options with additional support for trend and capability analysis. Proper consideration of the structure of the data as well as a proper estimation of variation in that data are essential for generating valid results. Not discussed in this chapter was the run chart. This chart simply plots the


observations in order according to some sample label. This chart can give preliminary information about process performance or can serve as a foundation for a control chart that implements defined limits, based perhaps on the distribution of the data involved. Chapter 7 considers these concepts in more detail and provides graphics showing how to launch and interpret particular platforms within JMP. The examples used generally reflect case studies, with most of them drawn from the author’s experience in semiconductor manufacturing.

C h a p t e r

7

Control Chart Case Studies Introduction 264 Measurement Tool Control 264 Some Scenarios for Measurement Tool Control Charts 266 Replicated Measurements at a Single Sample Site 267 Summary of the First Example 275 Measurements across an Object—No Replicates 275 Summary of the Second Example 279 A Measurement Study with Sample Degradation 280 Summary of Control Charting Issues for Measurement Tools 286 Scenarios for Manufacturing Process Control Charts 287 A Single Measurement on a Single Object per Run 289 An Application for XBar and S Charts? 294 A Process with Incorrect Specification Limits 302 Multiple Observations on More Than a Single Wafer in a Batch 305 Dealing with Non-Normal Data 316 The SEMATECH Approach 316


Monitoring D0 317 Cleaning Wafers (Delta Particles) 322 Defective Pixels 328 Summary of Control Chart Case Studies 332

Introduction The emphasis of this chapter is on developing proper control chart applications for scenarios derived from industrial experience. The majority of the examples discussed are necessarily from the semiconductor industry, as that was the location of the author’s most recent and detailed experience. The material notes those cases where examples are from other sources. A common thread in semiconductor data is that it often contains nested sources of variation. To create a fully effective control chart application for those situations requires recognizing the nature of the data and adapting software capabilities to meet it. As this book is written, no commercially available software addresses charting for nested variation. But the flexibility of JMP 6 allows adaptation of platforms available to meet these needs. The following sections build on the discussions in Chapter 6 by including more information about using a particular platform to generate a control chart. The first several sections discuss issues with control mechanisms for measurement tools; the final sections address these issues as they apply to manufacturing process monitoring.

Measurement Tool Control It is highly desirable for a measurement capability study to find that a measurement tool does not contribute excess variation to the observation of a process. But that initial success does not guarantee continued “good behavior” of that tool. Keeping appropriate control charts for all measurement tools is an essential part of guaranteeing that excursions in the measurement process do not affect estimates of the response variable. Simply gathering data and entering it into some control chart methodology is necessary, but not sufficient. For example, Figure 7.1 (derived from the table Ignore control.jmp)

Chapter 7: Control Chart Case Studies 265

shows an actual excursion that occurred on a particular measurement tool. Operators and engineers dutifully recorded and entered observations on the measurement tool, but no one paid any attention to the signals being generated until a process upset had occurred and had caused other problems.

Figure 7.1 An Ignored Measurement Tool Control Chart

During the period of the recorded measurement tool upset (late June to early July) a critical experiment was underway that relied on this measurement tool for data collection. The documented drift in the measurement process obscured the real effects being investigated, so the experiment was lost. When an engineer finally took the initiative and called for field service, the field service engineer found that a capacitor in a measurement tool circuit board had begun to fail. Installation of a new part and recalibration of the measurement tool produced the data starting about mid-July. The measurement tool was once again stable, but the mean value had shifted due to the change of parts. Note also that any measurement process that makes physical contact with the surface being measured has the potential for being mildly destructive to that surface. Progressive changes on the surface of a standard object being measured to provide a control mechanism for the measurement tool will likely produce an apparent drift of the measurement process with time. A logical conclusion an observer might make upon seeing the drift is that the measurement tool has changed and requires unnecessary recalibration.


Some Scenarios for Measurement Tool Control Charts The best method for monitoring a measurement tool is usually the simplest approach that matches or that contains some elements of the original measurement study, provided that study shows that no element in the original study produced unacceptable variation in the measurement process. 1. If the original measurement study involves replicated measurements of several sites on a standard by several operators, a simple approach to monitoring the measurement tool is to pick a given location on the object and measure it the same number of times on some regular basis. This approach can produce three control charts for the measurement process: an individual measurement chart of the average of the several measurements at the chosen site; a moving range chart of the average; and a standard deviation chart for the standard deviation of the replicated measurements. This set of charts provides monitoring of the location of the measurement process and two charts monitoring its variation. Of these two, the standard deviation chart monitors the repeatability of the measurement tool, whereas the moving range chart monitors the reproducibility and repeatability of the measurement tool combined. If evidence begins to accumulate that the site chosen for monitoring is becoming damaged, then one should switch to another site and continue the monitoring. The investigator can prepare the necessary initial charts with control limits from the data collected during the original study. 2. If the original measurement study involves measuring several sites on an object over a period of time with no replication of those measurements during the original study, then a useful approach is to continue the pattern of measurements in the same fashion. The data collected allows generation of an individual measurement chart for the average of the readings and a moving range chart of the differences in averages. The moving range chart again monitors a combination of the reproducibility and repeatability of the measurement process, but no means exists to separate the two elements. Monitoring schemes more complex than these depend on the individual case being considered and might or might not contribute useful additional information.


Replicated Measurements at a Single Sample Site Figure 7.2 is an excerpt of data from a measurement study (RS Measurement Study.jmp) that had multiple replications of 49 sites on a wafer by a number of operators over about a five-week period. Analysis indicated that the measurement process had a P/T = 6.8, so the measurement process was quite capable of meeting the requirements placed on it. A variance component analysis based on a variability chart grouped by DATE, SHFT, and REP in that order produced the results in Figure 7.3 (using the Variance Components option on the pull-down menu associated with the chart). The results of this analysis showed that no single source of random variation contributed large variation. The Within variance component in the analysis contains the variation among measurement sites on the surface of the object and should not be charged to the measurement process.

Figure 7.2 Excerpt of RS Measurement Study.jmp


Figure 7.3 Variability Chart and Variance Component Analysis of RS Measurement Study.jmp

Because this study contained replications, a chart for the repeatability of the measurements is possible. The simplest way to create the control charts with expected limits for this study parallels the discussion of the first scenario in the previous section. First one examines the data to see whether a given shift always performed the correct number of replications (5). One way to accomplish this is to select only those rows in the table where SITE = 1 (center of the object); Figure 7.4 shows the process for creating this subset. 1. First one creates the selection expression and executes it. 2. When the selection is in place, one chooses Subset on the Tables menu. 3. One selects a subset based on the rows selected and generates the new table.


Figure 7.4 Creating a Subset of RS Measurement Study.jmp


One then summarizes the new table (Subset of RS Measurement Study.jmp) by DATE and SHFT, generating the count of observations for RS. Figure 7.5 shows the summary table. Figure 7.6 is a chart of the measurement count versus DATE and SHFT. Examining either the chart in Figure 7.5 or the table in Figure 7.6 shows that only SHFT = 3 performed all the measurements on the days involved in the study.

Figure 7.5 Summary of Subset Table


Figure 7.6 Chart of Measurement Count at SITE = 1 by SHFT and DATE

To generate the appropriate control charts, one creates a new selection of rows choosing only those where SHFT = 3 and SITE = 1. Generating a new subset from this selection creates the table in Figure 7.7. Notice an empty row in the excerpt shown. The additional row is due to an empty duplicate row for recording a measurement; one deletes this row to leave 55 rows. From this modified table, one creates the necessary control charts to monitor this measurement tool: an individual measurements chart combined with a moving range and a standard deviation chart based on a sample size of 5. To generate the individuals and moving range charts, one selects Presummarize under Chart Type on the Control Chart launch platform. Choosing this option automatically chooses the individuals (I) and moving range (MR) charts for this example (Figure 7.8).


Figure 7.7 Excerpt of Subset2 of RS Measurement Study.jmp

Figure 7.8 Setting Up I and MR Charts for Subset2 of RS Measurement Study.jmp

Note: The individual charts (called Individual on Group Means and Individual on Group Std Devs) are actually individual measurement charts that plot either the subgroup mean or the subgroup standard deviation as the individual measurements themselves. Figure 7.9 shows the resulting I and MR charts (after some editing of the axes to improve the default display. Selecting Save Limits on the pull-down menu will save the calculated limits and the estimated standard deviation based on the group means to the original column in the data table if the user wishes (alternatively, the user can save these


limits to a separate table). This step allows updating the control chart readily as more data becomes available, without recomputing control limits unless necessary.

Figure 7.9 I and MR Charts of RS

NOTE: Saving the control limits to the original data table stores them as a property of the response column. The next step generates a standard deviation (S) chart to track the variation among the replicated measurements each day. The previously mentioned act of saving the limits to the response column saves the wrong estimate of the standard deviation for this chart.


Therefore, the user must eliminate the saved standard deviation value from the original table in order to generate correct limits for the S chart. To eliminate the saved value for sigma, select the Column Info option associated with the response column. The system lists control limits and sigma as the properties stored with that column. Remove sigma by selecting the Sigma option on the properties list before generating the new chart. This allows the JMP system to compute the correct limits for the generated S chart. Repeating the process of defining a chart, selecting the XBar option, and then selecting only the S chart from the options displayed (Figure 7.10) produces the results in Figure 7.11.

Figure 7.10 Creating an S Chart

NOTE: To match this example, the user must either eliminate the saved standard deviation value from the data table or, if the limits were originally saved to a separate table, must allow the system to compute limits based on the data. The user might also save the new limits to either the original data table or to a separate table for later use. These three charts provide the necessary information for monitoring this measurement tool. The I and MR charts monitor the general behavior and consistency of the tool, whereas the S chart specifically monitors the repeatability of the process.


Figure 7.11 S Chart from Subset2 of RS Measurement Study.jmp

Summary of the First Example The data from this measurement study showed that the measurement tool precision was suitable for its intended use and that none of the factors tested in the study produced an unacceptable level of variation in the measurements. This study had replicated measurements of each site on the wafer during each measurement trial. Therefore, as many as three control charts could apply to this system: an individuals chart of the average of the replications; a moving range chart of those averages; and a standard deviation chart of the replicated values. The simplest way to maintain a control system for this tool is to measure a single site several times periodically; the computation of initial control limits used only the data from SITE = 1 on the wafer gathered by the SHFT = 3 doing the study. The third shift was the only shift that gathered all the data requested in the study. Creating a subset of the data to avoid missing observations on particular days produced a relatively small sample (11 unique observations) for determining the control limits. Therefore, the initial control limits are likely to change once more data becomes available. This is the reason the discussion ignored the apparent violation of the S control chart upper limit in describing the example.

Measurements across an Object—No Replicates This scenario is common in a measurement study. Often a study fails to include replicated measurements at each site, so direct tracking of the repeatability of a measurement tool is not possible. Regardless of the number of sites measured across an


object, the only statistic one should chart is the average of those measurements. The variation among those measurement sites is a property of the object surface and not necessarily useful in determining measurement tool performance.

Manipulating, Examining, and Analyzing the Data Figure 7.12 contains an excerpt of the data from a measurement study (SiN measurement study.jmp) performed by four operators over about a three-week period. Each operator, independently of the others, measured nine sites on a silicon nitride wafer using an automated measurement tool (the measurement device loaded the object being measured automatically and scanned the programmed sites). Specification for the process being run was to have a thickness of 2100 ± 100 Å. Because the operators performed their measurements independently of each other, the analysis might consider the data grouped by day of the measurement. Since measurements occurred on 11 separate days during this period, this approach would produce 11 estimates of the mean thickness of the wafer. Alternatively, because the operators acted independently, the analysis might treat the data as a collection of 20 independent measurements to increase the sample size and provide more precision in establishing the initial control limits—provided the tool is capable of doing the measurement and the results are not dependent on which operator is using the tool.

Figure 7.12 Excerpt of Data from SiN measurement study.jmp

After verifying that the data table is in ascending time order, the first step in reducing this data is to convert the original file into the proper flat file structure using the Stack utility on the Tables menu (SiN measurement study STACK.jmp). A variability chart built from the stacked table grouped by DATE and OPERATOR indicates that OPERATOR = JH produced two unusual values (one high and one low) on DATE = 6/24/1994 (Figure 7.13).


Figure 7.13 Variability Chart from SiN measurement study STACK.jmp

Eliminating those two unusual values (SiN measurement study STACK filter.jmp) and applying the summarization techniques illustrated in Chapter 3 provide P/T = 13.1. This value is well below the target of 30, so combining the contributions of OPERATOR and DATE is reasonable in order to generate additional unique estimates of the object mean (the unusual values are not eliminated at this point). To combine the two columns of OPERATOR and DATE into a single ID column, one changes the DATE column type to Character and generates a new column using the Concat command under the Character options on the formula editor (Figure 7.14). Figure 7.15 provides an excerpt of this result.

Figure 7.14 Combining (Concatenating) DATE and OPERATOR


Figure 7.15 Adding an ID Column to the Stacked Table

Generating and Evaluating Initial Control Charts The process for generating the initial individuals and moving range control charts for this example is the same as that illustrated for the previous example. The left panel of Figure 7.16 shows the initial results. The analyst has the option of ignoring the violation of the control limits shown or excluding the violating point and recomputing a new control chart. The right panel in Figure 7.16 shows that approach. One can argue that initial control limits should reflect the stable behavior of a process. Following that argument, the charts in the right panel would be the ones to use to monitor the process. Note that the charts in the two panels have no large differences, although the limits of the charts in the right panel are slightly tighter than those in the left. These tighter control limits provide a more sensitive test for the measurement tool and could provide earlier warning of any upset in the measurement process.


Figure 7.16 Initial and Modified Control Charts for SiN Example

Summary of the Second Example The data from this experiment contained only single measurements performed by four operators at each of nine sites over about three weeks. Therefore, only the individuals and moving range charts apply to this case. Graphing the standard deviation of the nine measurements taken on each wafer provides no information about the behavior of the measurement tool; that value represents a property of the coating on the wafer. In this case, each operator collected observations independently of the others, even though the observations occurred on the same day. For this reason, the analysis used each set of observations in the sample to determine control limits. This same approach could have been used in the previous example, provided one limited the data to those observations that provided five replications, as some shifts did not record all five replications on a particular day.


This example also contained several outlier values. The first two outlier values would increase the P/T value, but not dramatically. The final outlier was an unusual mean value. An outlier such as that shown in the upper left panel of Figure 7.16 expands the control limits derived for the control charts and makes them less sensitive to excursions in values for the future. Removing the outlier tightens the control limits for both the individuals and the moving range charts. Interestingly enough, one operator (JH) produced all the unusual values, even though this was an automated measuring tool. Perhaps the automation does not completely insulate the results from operator errors; that point would be worth exploring further. Also in this case as in others, saving control limits to a separate outside table rather than into the original data table as discussed earlier might be a more flexible approach for future use.

A Measurement Study with Sample Degradation An earlier section noted that using a measurement tool on a sample might cause minor cumulative damage to that sample—for example, electrical measurements such as surface resistivity of a substrate. In the study in this example, each of three shifts measured the resistivity of a film on a single wafer at 49 sites. The study continued for about two weeks, with measurements occurring on almost every working day. Figure 7.17 contains an excerpt of the raw data from RS Gauge with degrade.jmp.

Figure 7.17 Excerpt of RS Gauge with degrade.jmp


Manipulating and Examining the Data Obviously, the table requires stacking, because it does not have the required flat file organization. Once the table is in the proper format (RS Gauge with degrade STACK.jmp), the next step examines the data using a combination of distributions and variability charts. An overlay plot of RS versus SITE (Figure 7.18) reveals immediately that SITE = 38 gave erratic results. This situation is similar to the one noted in previous examples and is most likely due to the presence of debris from the laser scribing process that places an identification number on each wafer. Selecting SITE = 38 and then deleting the selection excludes it from further analyses. Preparing a variability chart using DATE, SHIFT, and REP as the grouping variables in that order allows computation of the variance components in the data. Figure 7.19 shows that result.

Figure 7.18 Overlay Plot of RS versus SITE

The residual (Within) contains the contribution from individual sites on the object not charged to the measurement tool. The variances seen for each element are relatively small, so the measurement tool is likely to pass the P/T calculation.


Figure 7.19 Variability Chart and Variance Component Calculations of Measurement Study (excluding SITE 38)

Computing P/T The specification range for this example is 400 to 480 Ω/. Here is the process for summarizing and computing P/T: 1. One summarizes the table by SITE, computing the Mean(RS) and Variance(RS). 2. One determines the grand average of the thickness and the pooled variance using the Col Mean function in the formula editor. 3. One computes P/T. Figure 7.20 summarizes these steps, which were demonstrated in previous examples in Chapter 3.


Figure 7.20 Summarizing Steps to Compute P/T

Figure 7.21 shows the results. With P/T = 8.03, well below the required value of 30 (recall that with P/T < 30, the measurement error makes no material contribution to the capability of the process), the measurement tool appears capable of handling the task.

Figure 7.21 Calculation of P/T

Preparing the Data for Control Charts In this study, operators from each of three shifts measured 49 sites on a wafer with five replications over a period of about 10 days. Therefore, three control charts are possible: an individuals chart of the wafer average, a moving range chart of the wafer average, and a standard deviation chart of the replicated trials. Measuring all 49 sites on the standard wafer is not necessary in order to maintain control charts of this measurement tool. Rather, a team should pick a convenient site and choose to measure it regularly. As


before, SITE = 1 (center of the wafer) is a convenient choice, although any wafer site should be equally useful. Figure 7.22 shows an excerpt of the original data using only SITE = 1.

Figure 7.22 Excerpt of a Subset of Data Using Only SITE = 1 with Added ID Column

Notice that Figure 7.22 contains the additional column ID. The variance component analysis (Figure 7.19) showed little variation in the data due to DATE, SHIFT, and REP, so some method is necessary to identify each sample used for computing control limits. In this case, the author converted DATE and SHIFT from Numeric to Character to allow concatenation of these two columns into the ID column. The left panel in Figure 7.22 shows the results; the right panel shows the Concat expression used to generate ID. The Presummarize control chart option produced the top panel in Figure 7.23. The lower panel is an S chart generated separately from the XBar control chart environment. These three charts fully describe both reproducibility and repeatability in this measurement tool. Clearly the process changed toward the end of the measurement experiment. Close examination of the S chart shows that erratic behavior of the replicates began the last few days of October and continued through the end of the study in November. This fact suggests that the repeated measuring had damaged this wafer site. Not all sites behaved in this manner, however. Some sites remained fairly stable during the process. Because one site began to suffer damage casts doubt on the validity of the control chart limits generated.


Figure 7.23 Control Charts for RS


Dealing with Damaged Standard Wafers A well-designed measurement capability study assumes that at some point standard objects might become damaged or not usable. To plan ahead for this outcome, the investigator should select more than one (preferably three or more) standard objects at the outset that have approximately the same values. On the first day of the study, one measures all the objects as specified by the design of the experiment. Then one puts all but one of them away for safekeeping and continues the study using only one object of each nominal value being tested. If this object becomes damaged beyond use, one replaces it with one of the others in reserve. Replacing a standard might lead to a small bias in measurements, but the differences are likely to be small enough to ignore, and the variances observed should be very similar. If a backup sample does not exist, then one replaces the damaged sample with another having a similar nominal value. The most important parameter to consider in maintaining a measurement tool control system is the system variability. If the control protocol involves making multiple measurements at a single site, then the variation among those multiple measurements should be more a function of the measurement tool than the sample being measured, assuming the surface of the new sample is not damaged. One should expect a displacement in the individuals chart tracking the average of these measurements after a new sample enters the study. However, the moving range chart should not be materially affected. When enough data from the new sample has been collected (20 or so observations), one can modify the control limits of the charts to reflect the new values where necessary. When the new sample shows evidence of drift or damage, one switches to another site on the sample and continues. Measuring multiple sites on a sample for control charting purposes is not necessary and contributes to faster degradation of the standard.

Summary of Control Charting Issues for Measurement Tools The fundamental principle that supports control charts on any process is that the process is operating in a stable, undisturbed environment. When an upset occurs, an observation will fall outside the control limits, signaling the change. The previous example showed that the sample being measured apparently degraded somewhat during the initial measurement capability study. Any measurement process that requires physical contact with the sample surface is likely to cause some damage to that object. The simplistic approach of measuring several sites on a sample and then plotting the average of those sites (individuals chart) and the differences between subsequent


averages (moving range chart) is less sensitive to sample damage over time than a protocol involving multiple measurements at a single site on the sample.

Scenarios for Manufacturing Process Control Charts Processes being monitored with a set of control charts for variables data ideally require at least two charts: one to monitor the location (average) of the process, and the other to monitor the variation of that process. Some manufacturing processes, and semiconductor processes in particular, seldom generate the subgroups of independent data illustrated for the XBar, R, and S charts in the preceding chapter. Commonly, an engineer or operator measures several locations on one or more objects in a batch. These locations on the same object or objects are not statistically independent. Assuming that they are statistically independent leads to incorrect control limits and false signals that the process is producing material “out of control.” Several data-gathering scenarios are common in the semiconductor industry and can apply to other industries: 1. Measuring a single observation on a lot or batch. Typical examples of this situation are measuring the stress of a film, its refractive index (often a single point at the center of a wafer), particle counts, or defect density (a later section will deal with the unusual statistical properties of these two measurements). For this scenario (unless the distribution of observations is non-normal) one should generate an individuals chart (location) of the observations and a moving range chart (variation) of those observations. 2. Measuring multiple sites on a single object in a lot or batch. This is one of the more common data gathering scenarios in the semiconductor industry. The several sites measured on the wafer are not independent observations. The problem is not as simple as it seems, because the data contains more than one source of variability—variation within wafer and variation from batch-to-batch or lot-to-lot. The assumption that the sample size is n, where n is the number of sites measured, is likely to produce control limits that are too narrow for the XBar chart, because the within-wafer standard deviation of the sites is not the true process standard deviation and is normally smaller than other sources of variation in the


process. The section “The XBar, R, and S Charts” in Chapter 6 demonstrated this issue using data from RS Data Sort.jmp. For this scenario, one should generate a standard deviation chart with a subgroup size of n (where n is the number of sites measured on the wafer) to generate a chart for within-wafer standard deviation. Then one summarizes the observations to produce the mean value for each wafer and generates an individuals chart based on those means to track the lot-to-lot average and a moving range chart based on those means to track to the lot-to-lot variation. 3. Measuring multiple sites on more than one object in a lot or batch. Diffusion operations typically place two or more monitor wafers in each diffusion run. Each diffusion run might contain more than one lot of wafers, but the control chart must address how the diffusion operation is behaving. This is the most complicated data-gathering scenario, in that two or more sources of variation exist. The variance components are variation within the monitor wafers, variation between monitor wafers, and variation between diffusion runs. Appropriate control charts for this scenario must track not only the behavior of the average from a particular process, but also the behavior of all sources of variation. No commercial software of which the author is aware can integrate the variance component calculations necessary to display this data properly in appropriate charts. Therefore, a later section will illustrate an approximate approach for this situation that involves some preliminary calculations followed by generating a series of individuals charts and a single moving range chart. 4. Tracking defects, such as particle counts. These scenarios involve data that is not normally distributed, so conventional control chart techniques might not apply. In addition, changes in particle counts observed in wafer-cleaning operations are dependent on the number of particles initially present. This situation requires adjusting the observed particle counts to a common base before an observer can generate a useful control chart. The section “Dealing with Non-Normal Data” in this chapter illustrates this approach and suggests approaches using JMP that will generate control charts that are not dependent on normal distributions of data. Typical attribute charts such as the P, NP, C, and U charts might or might not apply to these situations. 5. Tracking cosmetic defects. These scenarios usually involve attribute control charts such as the C or U charts. Chapter 6 discusses applications of these charts using data provided with the installation of JMP.


As stated earlier, control charts are a mechanism for examining the current behavior of a process based on its undisturbed historical performance. Because estimating the correct variation in a process must precede determining the control limits for a process, the observer must collect sufficient data to allow as precise an estimate of variation as possible. Typically, this data collection requires 20 to 30 batches of a particular process. The author has found during his work in the semiconductor industry that the most useful of the control chart types are the individuals and moving range charts. In those cases involving measuring several points on a single wafer in each process batch, the standard deviation chart provides a useful record of the within-wafer variation. More complicated sampling scenarios involving more than one wafer in a process batch require variance component calculations to assure that the proper charts can be constructed. Control charts for defects, particularly changes in particle counts at cleaning operations, present additional challenges that require special considerations.

A Single Measurement on a Single Object per Run This data collection scenario is the simplest encountered in many manufacturing operations, excluding processes that are continuous flow. Because this example anticipates batch operations, the application of XBar and R or S charts does not apply. Figure 7.24 contains an excerpt of the data table Stress Data.jmp used earlier in the initial demonstration of individuals and moving range charts.

Figure 7.24 Excerpt of Stress Data.jmp

This table contains three variables: the date and time of the observation, the lot number being processed, and the observed stress. Each variable is in a separate column, and reading across any row in the table reveals all the information about that row. As noted in


Chapter 6, the data is in reverse chronological order, with the most recent data point at the top of the table, a typical result when a manufacturing operation uses automated data collection. In order to preserve the historical order of the data, one must sort the table by the DATE TIME column as illustrated in Figure 7.25.

Figure 7.25 Sorting a Table

Note also that the units for STRESS in Figure 7.24 involve a large exponential expression. JMP easily handles the units for STRESS correctly as shown in the previous discussion, but the vertical axis of the graph (Figure 6.7) includes very large negative numbers, making it difficult to interpret the graphs produced. Therefore, the author prefers to convert the measurements in a table such as this to simpler numbers by creating a new column that divides the stress measurements by the exponential part of the data entries. Figure 7.26 contains the formula used to generate the new column and an excerpt of the new table Stress Data SORT.jmp after sorting and generating the new column. Checking the distribution of the observations reveals no unusual points in the data, so proceeding to generate a control chart is a logical step.

Figure 7.26 Modifying the STRESS Column, Excerpt of Stress Data SORT.jmp


To create a control chart, one selects that option from the Graph menu. Selecting Control Chart produces a list of all charts supported by JMP (left panel of Figure 7.27). On this menu, the IR option is the proper choice for a process involving only a single observation per run. After the user makes a choice, the launch platform for that particular chart appears (right panel of Figure 7.27). To generate the moving range chart in addition to the individual measurement chart, one selects those options on the launch platform. Notice that the window for specifying the control chart allows immediate calculation of process capability. Calculating capability is truly valid only for control charts that are in a state of statistical process control, so the author advises against using this option at this point. Figure 7.28 shows the result of the specification in Figure 7.27, assuming that the user also requests the moving range chart. In generating this chart, either the DATE TIME or the LOT column can serve as the Sample Label, because the process treated only a single lot during each date/time interval. However, selecting LOT as the Sample Label produces a chart with the LOT values out of numerical order, since they were not processed in that order. Therefore, DATE TIME is the best Sample Label to use.

Figure 7.27 Selecting and Launching a Control Chart Environment


Figure 7.28 I and MR Charts for Stress Data SORT.jmp

On the pull-down menu on the individual measurement chart, the user can select any of a number of trend analysis tests to perform. The small 6 near the center of the upper portion of Figure 7.28 indicates that the chart has violated Rule 6 of the Western Electric Rules (see Tables 6.1 and 6.2). Previously the author recommended conservative use of all trend rules because they can lead to considerable effort in tracking down what might be a false positive for a trend. Of the rules listed in the tables, the SEMATECH Statistical Methods Group recommends that Rules 1 to 4 of the Western Electric Rules are the most reliable for preliminary trend analysis.


Because the process appears in control, generating the capability index for it is a logical next step. On the main pull-down menu of the charts is a Capability option (left panel in Figure 7.29). Choosing that option produces a dialog box in which the user specifies limits for the process being examined (right panel in Figure 7.29).

Figure 7.29 Accessing Capability Analysis and Defining Specification Limits

Leaving both estimates of sigma checked in this specification generates the two estimates of capability shown in Figure 7.30. The labels for the two curves explain the differences. The estimate based on overall sigma (Long Term Sigma in Figure 7.29) uses the standard deviation of the raw observations; the Specified Sigma is based on the standard deviation estimated in generating the control chart. Note also in Figure 7.30 that the system computes a confidence interval for the capability estimate, and notice the width of those intervals. A significant abuse of capability analyses is that teams will claim process improvement when a capability index increases by 0.1 or so after running a few new trials. Even with 50 samples, the confidence intervals on these indices are relatively wide, so claims like those are more than likely imaginative rather than substantive.


Figure 7.30 Estimates of Capability Dependent on Estimate of Standard Deviation

An Application for XBar and S Charts? This example illustrates a situation similar to that discussed in the section “The XBar, R, and S Charts” in Chapter 6. The data in this case is from a particular photolithography operation in the semiconductor industry. Photolithography is the part of the manufacturing process that creates the microscopic features necessary to generate the circuits on a semiconductor chip. The process exposes a light-sensitive material through a suitable mask and then develops the resulting image. Developing the exposed film removes material from the surface of the wafer either in the exposed area (negative resist) or unexposed area (positive resist). The photolithography operation is critical because the geometry of the resulting features determines the electrical functionality of the chip. In this example, the engineers measured the dimensions on the wafer at five locations to determine their critical dimension, or CD. Figure 7.31 displays an excerpt of the data table for this example, TRES.jmp.


Figure 7.31 Excerpt of TRES.jmp

Preparing to Generate Control Charts Even a cursory examination of the data table shows two problems: the data record is in reverse chronological order and the arrangement is not in the required flat file format for analysis. The first step is to sort the table according to the DATE TIME column. Before applying the Stack utility and after sorting the table, the author decided to add a new column to the table using the Count option in the formula editor. Although this step is purely optional, doing so provides an alternative axis label for control charts that does not require as much space as does the DATE TIME column. Figure 7.32 reviews the steps taken to modify the structure of this table. Figure 7.33 shows an excerpt of the result.


Figure 7.32 Sequence for Modifying TRES.jmp

In Figure 7.32, the top row sorts TRES.jmp into TRES SORT.jmp; the second row adds a new column containing a count of the rows in the sorted table; the third row stacks the sorted and modified table to provide TRES SORT STACK.jmp.


Generating the “Logical” Control Charts To the uninitiated, the data presented in Figure 7.33 seems destined for the typical XBar and S charts with a subgroup of five. Figure 7.34 shows the selection of the XBar chart platform followed by the specifications for the charts.

Figure 7.33 Excerpt of TRES SORT STACK.jmp

Figure 7.34 Setting Up the XBar and S Charts


Figure 7.35 shows the results of this specification. Either the process mean is dramatically out of control, or the control chart chosen for the mean is not the correct one. The standard deviation chart is a plot of the variation within groups of samples, and as such is valid. The author has actually overheard a conversation between a poorly trained quality “expert” in which that expert told an engineer that his process needed considerable optimization work to reduce its variability after reviewing just such a chart.

Figure 7.35 XBar and S Charts of CD

The problem is not completely with the process in this case. The problem is with the choice of charts, as illustrated in the section “The XBar, R, and S Charts” in Chapter 6. Generating a variability chart of the CD data grouped by Sample produces the results in Figure 7.36. This display uses the box plot option and connects the cell means in the data while suppressing other data displays. Compare this figure to the displays in Figure 6.4 in Chapter 6.


The appearance of Figure 7.36 more closely resembles that of the RS data than it does the Voltage data in that section. That is, the variation between samples might be somewhat larger than the variation within samples in both cases. Using the Variance Component analysis option to establish the sources of variation in this data produces the results in Figure 7.37. The analysis shows that variation between samples is approximately twice that among sites in a sample, where the Within variance component is the SITE[SAMPLE] variability. Therefore, using the variation among the sites in a sample to provide control limits for variation between samples is incorrect. NOTE: Of the variables in the TRES SORT STACK.jmp table, DATE TIME, LOT, and Sample are interchangeable. The author used Sample for the Sample Label to improve the appearance of the graphs.

Figure 7.36 Variability Chart of CD vs Sample

Figure 7.37 Variance Component Analysis of CD

Generating the Correct Control Charts The S chart generated in the previous section is the correct control chart for withinsample variation. Control limits generated by the I and MR charts are correct for the variation among sample means. Figure 7.38 shows the process for generating these charts.


Figure 7.38 Generating the I and MR Charts Using the Presummarize Option

When the variation between samples is larger than the variation between measurements, this is clear evidence that the data collection scenario fits the second scenario described in the section “Scenarios for Manufacturing Process Control Charts” in this chapter. This situation requires the use of I, MR, and S charts to fully describe the control of the process. When the reverse is true, then the use of the XBar and S charts might be acceptable. One should generate charts from both perspectives to establish which seems the more reasonable approach. Figure 7.39 shows the I and MR charts generated and includes the capability calculations for the mean of the process.


Figure 7.39 I, MR Charts and Capability Analysis of CD

After an appropriate chart has been generated, saving the limits to a separate table using that option on the pull-down menu for the chart allows easy regeneration of a similar chart when new data are available. Figure 7.40 shows the menu choice and the table saved as TRES I MR Limits.jmp for the I and MR charts in this example.

Figure 7.40 Saving Limits and TRES I MR Limits.jmp


A Process with Incorrect Specification Limits Calculation of appropriate capability indices and related Sigma Quality levels depends on determining the correct estimate of variation to use for the situation. However, specifying the wrong limits for the process specifications can also obscure true process capability. The author has witnessed several cases in which reports deliberately and dishonestly inflated specification limits to make a process look better than it was. Such methods are unethical at the core. When the author protested about one such practice, he was uninvited to be a visiting statistical resource for a manufacturing operation. However, the way one describes specifications in setting up a contract for a product can materially affect the apparent capability of the process being used. In setting up a contract for the deposition of a particular film on sapphire wafers, the manager of the group agreed to a specification that the mean thickness would be 12 ± 2μ because that represented routine performance of the process being used. He was unaware that specifications and capability calculations usually relate to individual observations and not to their means. Figure 7.41 contains an excerpt of Film Thick.jmp.

Figure 7.41 Excerpt of Film Thick.jmp

Creating a variability chart of this data by graphing THICK versus RUN and then computing the variance components (Figure 7.42) shows that almost all of the variation in this data derives from WAFSITE rather than RUN. The within-wafer variation (denoted by Within) dominates the observations. In this case, although the data collection scenario actually matches the second example given in the section “Scenarios for Manufacturing Process Control Charts” in this chapter, the reality of the data allows using an XBar/S combination of graphs instead of the I/MR set expected. Figure 7.43 shows the XBar/S charts generated for this example and includes capability calculations.


The capability calculations based on the overall sigma take into account all the variation among the observations. The calculations based on the control chart sigma use the value the system derived in computing the control limits for the charts. For the left panel, this value comes from a pooling of the differences between consecutive individual values as discussed in Chapter 6. The author prefers to use the calculations based on the overall sigma, because that unequivocally reflects the total variation in the observations being considered.

Figure 7.42 Variability Chart and Variance Component Calculations Based on Figure 7.41


Figure 7.43 XBar and S Charts with Capability Calculations Based on Film Thick.jmp

Despite the fact that the process appears to be in a state of statistical control, all of capability indices are disappointing. But recall that the opening statements said that the specifications were written with regard to the mean of the thickness values, not the individual measurement values. The specification 12 ± 2μ represents the desired 6σ range for the means. The XBar chart clearly shows this process is operating in this range. This stated specification means that the maximum allowable standard deviation for the means is 4/6 or 0.667 (for Cp = 1). To adjust the specifications for using individual values, consider Equation 7.1 (based on the Central Limit Theorem).

σy =

σy

n

7.1

Substituting the theoretical standard deviation for the means (for Cp = 1) into equation 7.1 and solving for the expected standard deviation of the individual measurements yields Equation 7.2.

σy =σy

( n ) = 0.667 ( 5 ) = 1.491

7.2


Therefore, the specification limits for individual measurements should be 12 ± 3(1.491) or USL = 16.5 and LSL = 7.5. Regenerating the capability calculations using these values and based on the overall standard deviation of the observations produces the results in Figure 7.44.

Figure 7.44 Revised Capability Calculations for Film Thick.jmp

This example could be considered meddling with specification limits to make a process look better than it is. The only justification for the manipulation just shown was that the original specifications very clearly referred to the process mean and not to individual observations. Two lessons should come from this example: 1. When writing specification limits, one should use care to describe exactly what those specifications imply. 2. When generating a control chart, a very useful first step is to compute the variance components associated with the data and determine the sources and magnitude of each.

Multiple Observations on More Than a Single Wafer in a Batch This data-collection scenario is one of the more complicated ones that exist in semiconductor manufacturing. A typical example would be a furnace operation in which operators add more than one monitor wafer to a furnace run. Figure 7.45 contains an excerpt from such a process, Furnace SPC.jmp.


Figure 7.45 Excerpt of Furnace SPC.jmp

In this process, operators placed two wafers, one near the loading port, and the other near the inlet for the reactive gases used to deposit the coating. On each of these two wafers, they measured five sites after completing the deposition process. Figure 7.46 presents a schematic of the sources of variability in the process. The total variance in such a process is a combination of the variance between runs, the pooled variances between wafers within runs, and the pooled variances between sites within wafers within runs. No current commercially available SPC package, including the platforms in JMP, comprehends this type of variance structure, so creating control charts for this scenario requires compromises that allow it to fit within JMP software’s capabilities.

Figure 7.46 Sources of Variation in Furnace SPC.jmp


The data structure contains three sources of variation:

variation between runs

variation between wafers nested in runs

variation between sites nested in wafers and runs

To monitor this scenario completely requires four control charts: 1. a chart to monitor the behavior of the means of the runs—an individuals chart based on the means. 2. a chart to monitor the variation of the means between runs—a moving range chart based on the variations between the means. 3. a chart to monitor the variation of the wafers within runs—an individuals chart tracking the wafer-to-wafer variation from run to run. The entity tracked in this case is either a variance or a standard deviation, so creating an individuals chart for this data is a compromise based on software capabilities. 4. a chart to monitor the variation of the sites within wafers within runs—an individuals chart tracking the within-wafer variation from run to run. The data plotted here is similar to that in (3) in that it is also a standard deviation, so this is a compromise as well. Electing to monitor only the first two (the easiest two) can overlook important signals if the process changes and can lead to product that does not meet specifications or has too much variability.

Manipulating the Data Examination of Furnace SPC.jmp shows that the data has been recorded in reverse chronological order. Processing this data for the generation of control charts (and for any other statistical analysis) follows generally the steps shown in the previous example (Figure 7.32). 1. One sorts the table in ascending order by DATE TIME and creates the new table Furnace SPC Sort.jmp. 2. One inserts the new column RUN into this table and generates a count from 1 to 25 using the Count option under Row in the formula editor. (This step is optional and serves to make the X axis of any control chart more readable compared to using the DATE TIME column.) This step is similar, but not identical, to that shown in Figure 7.32. Figure 7.47 shows the original form of the formula option and the information added to it before executing the command. This format inserts pairs of


duplicate values in the new column RUN. One designates the entries in the new column Numeric, Nominal.

Figure 7.47 Configuring the Count Option in the Formula Editor

3. One converts the table containing the new column to a flat file using the Stack option on the Tables menu. Figure 7.48 presents an excerpt of the result after the final use of the Stack option.

Figure 7.48 Excerpt of Furnace SPC SORT STACK.jmp after Sorting, Adding RUN Column, and Stacking

Generating a distribution of THICK followed by a variability chart using RUN and WAFERID as the grouping variables shows that the data is reasonably well-behaved, with no outlier values. The nested variance component analysis available on the variability chart (Figure 7.49) indicates that little variation arises from run to run and that the variability in the data distributes almost evenly between WAFERID and Within, which is the SITE[WAFERID] variability.


Figure 7.49 Variance Component Analysis of Furnace SPC SORT STACK.jmp

NOTE: The variability contributed by differences between runs is not actually 0. An artifact of the variance component calculations sets very small variance components to 0. Therefore, the system resorted to a REML estimation rather than EMS.

Generating the Easiest Control Charts As stated earlier, complete monitoring of this process requires four control charts—an individuals and moving range chart for the mean of each run, and individuals charts for each of the variance components due to WAFERID and SITE. The Presummarize option in the Control Chart platform makes the first two in this list the easiest to create. Figure 7.50 shows these two charts prepared as shown in the section “Replicated Measurements at a Single Sample Site” in this chapter. Choosing to run the first four tests using the Western Electric Rules identifies RUN as a violator of the upper 3σ limit. This signal means that the analyst should investigate the data to see whether some assignable cause exists for this point. If one exists, then this point does not represent normal behavior of the process. One should thus eliminate it from the data, because the objective is to establish control limits for this process. Removing this point will compress the control limits.


Figure 7.50 I and MR Charts for Run Means

Generating Control Charts for Within-Wafer and Wafer-toWafer Variance Components Ideally, the user should be able to specify a nested relationship within a data table and the software should generate the appropriate standard deviation charts for these two items. As stated earlier, the author knows of no commercial software that has this capability, including JMP. Therefore, the charts presented here are compromises based on individuals charts for these components. The first step requires generating the variance components for each RUN in the data. Doing this requires fitting a model using the By option. This generates some 25 reports and creates some extra effort. NOTE: Undoubtedly, a user knowledgeable in the JSL programming language could write an appropriate routine that would automate this process. The author is not familiar with the JSL language capabilities and so resorted to using the Make Combined Table option outlined previously.


The upper panel of Figure 7.51 shows the setup of the Fit Model platform for this exercise, and the lower panel shows the results for RUN = 1, 2, 3.

Figure 7.51 Setup and Results of Fit Model for Furnace SPC.jmp


Figure 7.52 presents an excerpt of Furnace SPC VC.jmp created by copying and pasting elements of the regression report to a new table. Note that the data transferred were variances and were converted to standard deviations for the purposes of charting. The process for creating this table involves many steps: 1. One right-clicks within a display of the results of the analysis for any RUN to reveal a menu. On this menu, one selects Make Combined Data Table. This selection generates a table the author named Furnace VC Initial.jmp. In this table, the entry for Residual is actually the within-wafer variance. 2. In this table, one deletes the columns for RUN2 (an artifact), Var Ratio, and Pct of Total. Using a Row Selection option, one selects and then removes all entries for Total. One then adds a new column called Standard Deviation and fills it with values formed by taking the square root of the Var Component column. This produces the table Furnace VC Initial Mod.jmp. An excerpt of this table appears in Figure 7.52.

Figure 7.52 Excerpt of Furnace VC Initial Mod.jmp

3. Next, one uses the Split option on the Tables menu to reorganize this table into Furnace VC Split1.jmp. The left panel in Figure 7.53 shows the setup for this conversion; the right panel shows an excerpt of the result after renaming Residual to WIW SD and WAFERID to WTW SD.


Figure 7.53 Converting Furnace VC Initial Mod.jmp Using the Split Option

4. One uses the Stack option on the Tables menu to reorganize this table into Furnace VC Stack.jmp. In the converted table, one changes the entries for RUN to Numeric/Nominal. The left panel of Figure 7.54 shows the setup for this conversion; the right panel shows an excerpt of the result.

Figure 7.54 Converting Furnace VC Split1.jmp Using the Stack Option


5. One uses the Split option a second time to generate Furnace SPC VC.jmp according to the setup in the left panel of Figure 7.55. The right panel shows that result.

Figure 7.55 Converting Furnace VC Stack.jmp Using the Second Split Option

The left panel of Figure 7.56 presents the individual measurement charts generated for each variance component. As an alternative, an analyst might generate run charts for these variables. The right panel presents run charts for each variance component. The limits on the run charts are based on the 99.5% and 0.5% quantiles found by generating the distributions of each component. The run chart limits for WIW SD are CL = 44.686, UCL = 55.155, and LCL = 35.326, so there is not a great deal of difference between the two approaches. For WTW SD, the corresponding limits are CL = 33.436, UCL = 85.519, and LCL = 0.


Figure 7.56 Individual Measurement Charts for Within-Wafer and Wafer-to-Wafer Standard Deviations

The individual measurement chart for WTW SD originally had a lower control limit (LCL) as a negative number. The author added a reference line to reflect the true lower limit for this standard deviation. Because these are individual measurement charts, lower limits for standard deviations < 0 will occur frequently, depending on the content of the data.

Summary of the Example The scenario discussed in the previous sections was somewhat more complicated than previous examples. Although this example used only two objects with multiple measurements per object per run, the techniques used here are general. Using them for


even more complicated scenarios simply requires studying the situation and adapting. If future releases of JMP can compute multiple control charts for processes involving nested variation, then the process will become somewhat simpler.

Dealing with Non-Normal Data A fundamental assumption that applies to all control charts discussed to this point (and to most comparative statistical operations) is that the data being charted or analyzed generally fits a normal distribution. The section “Charts for Attributes Data” in Chapter 6 discussed two major categories of attribute data that generally involve non-normal data. Both cases involved noncontinuous attribute data that related to measurements of defects. In one case, items in a group were either good or bad, so the statistics of the binomial distribution applied. Generally speaking, these examples concerned observations where only a small number (integer values) of items were bad. The P and NP charts apply to these situations (see “P and NP Charts” in Chapter 6). In the other case, measurements involved unbounded defects in a group of objects. The statistics of these observations generally follow a Poisson distribution, as discussed in “U Charts and C Charts” in Chapter 6. Two chart types apply in this case; the C chart applies when the sample size is constant, whereas the U chart applies with irregular sample sizes. In semiconductor manufacturing, data associated with particle counts (added or removed) and defect densities apparently follows these guidelines.

The SEMATECH Approach Given the problems associated with some forms of data, SEMATECH commissioned a series of investigations at the Arizona State University and at the University of Texas to study the distributions associated with non-normal data such as particle counts. Professor Ian Harris (formerly of the University of Texas Statistics Department, Austin) supervised the creation of software, and Don McCormack (later a statistician in the SEMATECH Statistical Methods Group) wrote the program. The result of that work makes no assumptions regarding the distribution that describes a particular set of data. With limited sample sizes (< 100 observations), the routine models the data distribution using a mixed Poisson algorithm. In cases involving > 100 observations, the research found that using nonparametric estimation techniques produced results indistinguishable from those produced by modeling approaches. That is, reasonable estimates of control limits for processes generating such data come from calculations of percentiles in an empirical distribution. SEMATECH member companies


received this program and courses detailing its use. This program is not currently available in JSL, so the examples that follow illustrate how to use existing capabilities of JMP to provide reasonable approximations for control limits in such cases.

Monitoring D0 The parameter D0 is a single number that describes how high or how low the defect density is for a particular lot at a particular manufacturing operation. The technique for monitoring D0 scans wafers with features and computes a value based on the area of the dies, the number of dies on a wafer having defects, and the total number of dies on each wafer. The value of D0 is a figure of merit for a lot or a wafer. The lower the value, the fewer defects found. This parameter has no upper boundary; its value depends on the quality of the product being examined. Figure 7.57 shows an excerpt of D0Data.jmp. The operation in this example examined two wafers per lot of 25. The data presented considers only the average value for a particular lot. In about 6 months, the process examined some 1000+ lots of 25 wafers.

Figure 7.57 Excerpt of Data from D0Data.jmp

Examining the Data Because of the large amount of data included in this example, the most reasonable approach was generating distributions of the data. The top panel in Figure 7.58 shows the original distribution generated, along with the associated statistics. Because this example contains ~ 1000 observations, using an empirical approach to establishing control limits


for this data is reasonable. Recall that 3σ limits for a conventional variables control chart assures that 99.7% of the data will lie between those boundaries. In this case, accepting the 99.5% quantile as the upper limit and 0% as the minimum provides very similar results. That is, the UCL for this process could be assigned 0.7787, with the LCL = 0.0000. However, the approach taken first was to eliminate the two extreme values, because they were well above the 99.5% quantile.

Figure 7.58 Distribution of LOTD0 Raw Data and Filtered Data

Generating a C Chart Before generating any type of control chart, the analyst should make sure that the data being graphed truly represents the undisturbed behavior of the system. The initial distribution of this data suggested that two values were unusually large, so they were removed to produce the lower panel in Figure 7.58.


Choosing to remove additional points from the data table should be based on assignable causes or an established statistical test procedure. Because the wafers were not available for the author to inspect, the next step for generating the C Chart from this data was to make sure that the Poisson distribution adequately described the data. Fitting this distribution to the data and computing goodness-of-fit statistics showed that the Poisson was indeed a good fit to this example. Figure 7.59 summarizes the findings based on the lower panel in Figure 7.58. The upper panel of Figure 7.60 shows the C Chart generated from this data after deleting the two high values. The lower panel is the same chart with the quantile limits found in the lower panel of Figure 7.58 substituted.

Figure 7.59 Testing the Fit of a Poisson Distribution

Figure 7.60 C Chart of LOTD0


Witness Wafers (Particles Added) Manufacturing conditions for semiconductor devices require clean rooms that seek to eliminate contamination of the wafers being processed by airborne materials. A typical clean room is far freer of microscopic particles than even the best operating theaters in a hospital. A witness wafer is a bare silicon wafer placed in a suitable location in such an operating area in order to monitor the particles likely to fall on a product wafer in that environment. Typically one leaves the wafer exposed for 24 hours and then measures the number of particles added during that period. Figure 7.61 shows an excerpt of Witness wafers.jmp taken from a single operating environment. Electronic particle counters scan the wafer surface initially and again after 24 hours, reporting the number of particles found per “bin” of particle sizes. The data in Figure 7.61 presents the total number of particles of all sizes added to the wafer surface per hour during a 24-hour period. From this data, engineers hope to learn what sort of routine particle counts this area might introduce on wafers awaiting processing. A control chart of this data would signal when an extraordinary event had occurred. Proper analysis of particle data requires considerable care because the particle counts observed are extremely sensitive to the techniques used in handling the monitor wafers; outlier data points are common. In addition, particle counts do not follow a normal distribution. In fact, the previously mentioned studies supported by SEMATECH indicated that they did not follow any particular distribution, although the software described earlier could fit a mixed Poisson distribution to the data when the number of samples is < 100. If the number of observations is > 100, then the software uses an empirical estimation of the control limits based on quantiles in the data. The number of particles is also an unbounded quantity, so the C chart in JMP applies as well.

Figure 7.61 Excerpt of Witness wafers.jmp


Examining the Data The upper panel in Figure 7.62 presents a distribution generated from the raw data in Witness wafers.jmp, along with the statistics of the distribution. The image in the center panel is the same distribution after deletion of the three obvious outlier values plus one additional high value from the data table. Because the sample size was somewhat greater than 100, using the empirical limits from the quantile report produced a reasonable set of control limits. The observations were unbounded defects, so fitting a Poisson distribution to the filtered distribution was a logical step. Although the Poisson distribution had poor goodness-of-fit statistics, generating a C chart from this data is an alternative approach.

Figure 7.62 Distributions from Witness wafer.jmp, Distribution Fitting


The left panel of Figure 7.63 shows a C chart based on the example using the filtered data. Because a few observations approached or violated the computed upper control limit, saving the chart limits to a separate table and then modifying them to reflect the percentile values found for the modified distribution would create the C chart based on empirical data and might be more reasonable. Alternatively, the analyst could generate a run chart and impose limits based on percentiles as reference lines on that chart; the right panel in Figure 7.63 shows that approach.

Figure 7.63 C Chart Based on Witness wafer FILTER.jmp

Cleaning Wafers (Delta Particles) In a wafer cleaning operation, one typically places one or more monitor wafers in a container of wafers being cleaned. The monitor wafers are almost never completely free of particles, but generally have no features and relatively low particle contamination (counts).


To monitor the cleaning operation, the team computes how many particles have been removed from the monitor wafers by subtracting the pre-clean count from the post-clean count. Because handling of the monitors can materially affect the number of post-clean particles found, the appearance of unusually large values for the differences is commonplace. In addition, careful examination of the values for the resulting changes in particles is extremely important. In many operations such as this, the number of particles apparently removed is not independent of the number of particles originally present, so setting up a control system for such a process introduces complications. Figure 7.64 presents an excerpt of Cleaning Op.jmp, the results from a particular cleaning operation where DELTA PCL is the change calculated as PRECOUNT minus POSTCOUNT. One monitor was present in each of the 100+ cleaning runs, but a cleaning run could contain 100 or more product wafers.

Figure 7.64 Excerpt of Cleaning Op.jmp

Examining and Manipulating the Data A distribution of the DELTA PCL column shows that the observations do not follow a normal distribution. A graph of DELTA PCL versus PRECOUNT appears in Figure 7.65.


Figure 7.65 Bivariate Fit of DELTA PCL versus PRECOUNT from Cleaning OP.jmp

NOTE: The Fit Y by X platform is more useful than the Overlay Plot in this case, because it allows fitting a line to the graph. The appearance of this graph is typical for this type of operation in that normally one finds a negative slope for the relationship between the number of particles removed and the number originally present. Convention uses a negative number to represent particles removed. Figure 7.65 clearly shows the covariant relationship between these two characteristics. Generating a successful control mechanism for this process requires neutralizing that covariance. NOTE: The author and the lead engineer for a project to optimize a new cleaning operation first encountered this problem during the analysis of a designed experiment where one of the responses was Delta Particles. Together they developed the approach described in the following sections. When other engineers would not accept the approach, the author discussed it with statistical authorities contracted to SEMATECH—Dr. Peter John, University of Texas, Austin; and Dr. George Milliken, Kansas State University, Manhattan. Both of these professors readily endorsed the approach as a useful method for neutralizing a covariant relationship.


Removing the Covariance One approach often recommended and used by some engineers is the screening of monitor wafers to make sure each has initial particle counts in some narrow range. That process is labor intensive and not extremely practical. Monitor wafers will differ among themselves, often to a considerable degree, in the number of particles originally present. In this example, this value varied from ~10 to nearly 900—nearly three orders of magnitude. Therefore, a technique was needed for determining the relationship (if any) between delta particles and precounts and adjusting the observed delta particles relative to a common starting value for monitor wafers, regardless of what the original value actually was. The first step in that process appears in Figure 7.65. The intercept of that graph suggests that running a perfectly clean wafer through that process would actually add some 24 particles. To adjust for this dependence (called covariance by statisticians) requires application of the formula in Equation 7.3:

adjusted delta particles = slope ( index - precount ) + observed delta

7.3

The value computed from this expression for each data point has the effect of adjusting out the pre-clean particle counts so the new delta particle count is independent of the number of particles originally present. The slope is the slope of a linear fit applied to a plot of delta particles versus precount. The value of the index in Equation 7.3 can be any value; typically one might use the median of existing precount data or 0 to relate the data to what might happen to a perfectly clean wafer in the process. The left panel of Figure 7.66 shows the original data table with the ADJDELTA PCLS column added according to Equation 7.3, using an index of 0 with the result rounded to the nearest integer; the right panel displays the formula used in the table to calculate the new column. Figure 7.67 plots the original DLTAPCLS and ADJDELTA PCLS versus the PRECOUNT column.

Figure 7.66 Excerpt of Cleaning Op MOD.jmp after Application of Correction Equation


Applying the Fit Y by X platform to the apparently horizontal curve in Figure 7.67 shows that the slope of the line is not statistically significant and the intercept for the data is approximately the same for modified and unmodified data. Figure 7.67 indicates that intercept at ~ 24—the same as that in Figure 7.65 (the horizontal line in Figure 7.67 is a reference line added by the author).

Figure 7.67 Overlay Plot of ADJDELTA PCLS and DELTA PCL versus PRECOUNT

Generating a distribution (upper panel of Figure 7.68) of the data in the new column reveals four unusually large values—all 100 or higher. Deleting them from the data (leaving 119 observations) and generating a new distribution produces the lower panel in Figure 7.68. Fitting a Poisson distribution to this data fails because that distribution does not allow negative values.


Figure 7.68 Distribution of ADJDLTA PCLS before and after Removing Outlier Values

Generating a Control Chart The data for ADJDLTA PCLS is still an unbounded count of defects, so the C chart still applies. Generating that chart directly (left panel of Figure 7.69) produces limits somewhat at odds with the quantile values in Figure 7.68. To impose those quantile values on a chart, one should save the limits of the existing chart to a table (Cleaning Op Limits.jmp). Then one edits that table and enters the values for the 99.5%, 50.0%, and 0.5% as the UCL, Mean, and LCL respectively. Generating the chart again and specifying to get limits from the saved table produces the right panel in Figure 7.69.


Figure 7.69 C Chart with Computed Limits and with Percentile Limits

Defective Pixels This example considers the testing of infrared (IR)-sensitive pixels in a detector array produced by an aerospace organization for the U.S. Air Force. The array, listed as 64K 16 and used in IR-seeking missiles, actually has 2 or 65536 possible pixels on its surface. Obviously, the more good pixels in each array, the better the performance of the device in controlling missile flight. Figure 7.70 shows an excerpt of Pixel test.jmp.

Figure 7.70 Excerpt of Pixel test.jmp


Because pixels must be either good or bad, the distribution of pixels is binomial—so many good, so many bad. The section “P Charts and NP Charts” in Chapter 6 discussed the P and NP charts that generally apply to this situation and noted that control mechanisms based on the number of bad objects or defects might be more easily interpreted than those based on the good objects. However, that discussion also pointed out that these charts work best when the observations are clearly simple integers rather than large numbers.

Examining and Manipulating the Data Generating a distribution of the Bad Pixels column produces Figure 7.71. Obviously, the data does not have a normal distribution and contains a number of large outlier values. Arbitrarily, the author excluded any observations with more than 2000 bad pixels by creating a row selection statement. Deleting these rows left more than 270 observations undisturbed (Pixel test FILTER.jmp).

Figure 7.71 Distribution of Bad Pixels

Generating a P Chart Choosing a P chart for this data generates the fraction defective for each observation based on the information in Pixel test FILTER.jmp. Figure 7.72 is the result.


Figure 7.72 P Chart for Bad Pixels

The very narrow control limits generated for the P chart suggest that virtually every device is out of control; either this is one of the worst processes practiced by man, or the P (or NP) chart is not appropriate for this application. In this case, the defects are relatively large numbers, not simple integers. Under these conditions, both the P and NP charts fail to generate the appropriate graph. The solution (a compromise at best) is to resort to an individuals chart for this situation.

Generating the Individuals Chart The author suggests generating a column in the table that lists fraction defective, and then generating an I chart from that data. The moving range chart is also a possibility, but might or might not apply fully in this case. The upper panel of Figure 7.73 shows the I chart.


Figure 7.73 I Chart of Fraction Defective


The system treats the data as if it is continuous, but the actual fraction defective has a lower limit of 0.000. Therefore, to impose more correct limits on this chart, one saves the limits to a table and edits that table. The next time one generates this chart, it will have the correct limits. NOTE: Dr. George Milliken, Professor of Statistics, Kansas State University, Manhattan, notes that this example presents a case of “over-dispersion.” In his critique of this material, he endorsed this approach as a reasonable approximation for this type of situation. Alternatively, one could generate a run chart and impose limits based on the quantiles in the distribution of the Fraction Defective column. The lower panel in Figure 7.73 shows this approach.

Summary of Control Chart Case Studies Failing to provide and maintain control charts for any process can become a costly error. Measurement tools can begin to drift from the results that originally characterized them and cause waste. For a control mechanism, no need exists to measure many sites multiple times. One should choose a single site and perform repeated measurements periodically. Taking this approach provides an opportunity to monitor not only the overall measurement tool performance, but also its repeatability. One should be watchful for deterioration of samples used to maintain control of a measurement tool.


The data gathered during a passive data study can provide the basis for preliminary control charts for a process. One should be aware of nested variation structures and maintain a chart for the location of the process (mean) as well as sources of variation in that process. The examples presented here used compromise solutions for charting variance components, because JMP (nor any other commercial software) does not currently handle charts for variance components. When dealing with attribute data (or variables data), one must use common sense to ensure that the data being used to generate control charts truly represent the operation of the process. One must watch for unusual values in the data being graphed and remove them before generating control charts if some form of assignable cause exists. Several examples in this chapter considered unbounded estimates of defects in a process. This count data has no boundary, and if sample sizes are constant, the C chart is appropriate. One way to verify that the C chart is appropriate is to fit a Poisson distribution function to the data and evaluate its fit. If sample sizes are large (> 100), a reasonable set of control limits is available from the quantile values displayed on the distribution platform. Research funded by SEMATECH showed that empirical results for large samples gave the same results as more sophisticated distribution-fitting approaches. The final example presented a case study in which binomial data did not provide reasonable P or NP charts because the numbers of defects were not simple integers. One should beware of this over-dispersion situation in manufacturing environments. The solutions presented here used an individuals chart for the data, with modified control limits to reflect reality. Alternatively, with large sample sizes (> 100) using empirical limits and imposing limits based on quantiles of the distribution of data will also give good results.


References Barrentine, Larry B., and Robert L. Mason. 1991. Concepts for R&R Studies. Milwaukee: ASQ Quality Press. Diamond, W. J. 1989. Practical Experiment Designs. New York: Van-Nostrand Reinhold. Duncan, A. J. 1986. Quality Control and Industrial Statistics. 5th ed. Homewood, IL: Irwin. Hollander, Myles, and Douglas A. Wolfe. 1973. Nonparametric Statistical Methods. New York: Wiley. Natrella, Mary Gibbons. 1966. Experimental Statistics, National Bureau of Standards Handbook 91. United States Department of Commerce, Washington, DC. Nelson, Lloyd S. 1984. “The Shewhart Control Chart—Tests for Special Causes,” Journal of Quality Technology 16(4): 237–239. Nelson, Lloyd S. 1985. “Interpreting Shewhart Xbar Control Charts,” Journal of Quality Technology 17(2): 114–116. Sall, John, Ann Lehman, and Lee Creighton. 2001. JMP 6 Statistics and Graphics Guide. Cary, NC: SAS Institute, Inc. Western Electric Company, Bonnie B. Small, Chairman of the Writing Committee. 1956. Statistical Quality Control Handbook. Indianapolis: AT&T Technologies (Select Code 700-444, P.O. Box 19901, Indianapolis 46219). Wheeler, Donald J., and David S. Chambers. 1986. Understanding Statistical Process Control. Knoxville, TN: SPC Press, Inc. Wheeler, Donald J., and Richard W. Lyday. 1990. Evaluating the Measurement Process. 2nd ed. Knoxville, TN: SPC Press, Inc.


Index A accuracy vs. precision in measurement 11–15 alternate hypothesis 16 analysis of manufacturing process See manufacturing process analysis attribute data, control charts for 235, 245–248 automated measurement tools 113

B bias in measurement tools, evaluating FTIR measurement tool (study) 61–63 turbidity meter evaluation (study) 36–42, 45–46 Brush tool 48

C C charts 247–248, 316, 318–319 cleaning wafers (study) 327–328 witness wafers (study) 322 calibration, reasons for 16–20 capability analysis control charts for 256–261 ellipsometer (study) 95–96 FTIR measurement tool (study) 64–66 manufacturing process, single observation 293 Capability Analysis option 261 capability potential 7–10, 176 CD (critical dimension) 294 chemical mechanical planarization 208–230 silicon oxide films 204–217, 222–229 tungsten films 217–221 CI (confidence intervals) for mean 22–24 for standard deviation 24–28 cleaning wafers (study) 322–328, 330–332

CMP See chemical mechanical planarization coefficient of variation 169 concatenating tables 82–84 confidence intervals for mean 22–24 for standard deviation 24–28 configuring a measurement tool (study) 81–82 control chart methodology 234–262 attribute data 235, 245–248 capability analysis 256–261 general concepts and basic statistics 235–237 manufacturing process control scenarios 287–316 measurement tool control scenarios 264–287 non-normal data 261, 316–332 trend analysis 252–256 types of control charts 237–252 cosmetic defect tracking 288 covariance, removing 325–327 Cp statistic 7–10, 176, 256–258 Cpk statistic 258–260 Cpm statistic 260 critical dimension (CD) 294 cumulative sum of differences (CUSUM charts) 250–252

D D0 parameter, monitoring 317 damaging objects during measurement 113 data collection See passive data collection (PDC) data entry forms for measurement trials 115–119

338 Index defect observation charts See attribute data, control charts for defect tracking 288 pixel testing (study) 328–332 degradation of sample in measurement study (example) 280–287 deposited film thickness (example) 120–127 figure of merit 123–125 Sigma Quality level 125–127 downstream etch process (study) 196–207

E ellipsometer, evaluating (study) 94–99 EMS in variance component analysis 178 entry forms for measurement trials 115–119 epitaxial silicon deposition (study) 189–196 error in measurement See measurement capability (error) etch process (study) 196–207 EWMA charts 250 excluding questionable values configuring a measurement tool (study) 81–82 measurements across object without replicates (study) 280 oxide thickness measurement tool (study) 51–53 expected mean squares in variance component analysis 178 exponentially weighted moving average charts 250

F figure-of-merit statistics 5–10, 123–125 film gauge, thin (study) 140–149 film thickness, deposited (example) 120–127 figure of merit 123–125 Sigma Quality level 125–127 films, polishing (examples) oxide films 204–217, 222–229 tungsten films 217–221

first-wafer effect 191, 193 Fit Model option See regression analysis fixed effects 30 FTIR measurement tool (study) 56–66, 155–161

G Gage R&R option ellipsometer (study) 98–99 resistance measurement tool (study) 73–75 study with operators and replicates 102–104 thin film gauge (study) 144–146 % Gage R&R statistic ellipsometer (study) 96–98 resistance measurement tool (study) 74 Gage Study option 73 goodness of fit 38

H hiding questionable values 51–53 hypothesis 16

I I and MR charts (IR option) 243–245 cleaning wafers (study) 330–332 manufacturing process, single observation 291–292 measurements across object without replicates (study) 278–279 multiple-site measurement on multiple objects (study) 309–310 multiple-site measurement on single object (study) 299–301 replicated measurements at single sample site (study) 271–274 sample degradation in measurement study (example) 284–285 incorrect specification limits 302–305

Index individual measurement and moving range charts See I and MR charts

J JMP tables for data entry 115–119

L Levey-Jennings charts 252 linearity of measurement tools 29 resistance measurement tool (study) 76–77 logging observations 114

M manufacturing process analysis 165–169 See also passive data collection (PDC) chemical mechanical planarization 208–230 control chart scenarios 287–316 downstream etch process (study) 196–207 epitaxial silicon deposition (study) 189–196 preliminary process study (example) 181–189 thermal disposition process (study) 172–180 manufacturing process control chart scenarios 287–316 cosmetic defect tracking 288 defect tracking 288 incorrect specification limits 302–305 measuring multiple sites on multiple objects 288, 305–316 measuring multiple sites on single object 287–288, 294–301 measuring single observation 287, 289–294 matched pair analysis FTIR measurement tool (study) 61–63 measurement tool without calibration standard (study) 93

339

turbidity meter evaluation (study) 43–44 mean confidence interval for 22–24 uncertainty in 21–28 measurement capability (error) 28–29, 111 See also repeatability error See also reproducibility error measurement capability studies 111–162 automated measurement tools 113 damaging objects during measurement 113 data entry forms for trials 115–119 deposited film thickness (example) 120–127 FTIR measurement tool (study) 155–161 oxide study (example) 128–135 planning measurement studies 112–120 resistance measurement tool (study) 149–154 thin film gauge (study) 140–149 turbidity meter evaluation (study) 135–139 measurement error, components of 28–29 measurement process 3–30 calibration 16–20 components of measurement error 28–29 figure-of-merit statistics 5–10 linearity of measurement tools 29 precision vs. accuracy 11–15 random vs. fixed effects 30 uncertainty in means and standard deviations 21–28 measurement studies 112–120 objectives of 112 scheduling 114–115 measurement tool control scenarios (case studies) 264–287 measurements across object without replicates 275–279, 280 replicated measurements at single sample site 267–275 study with sample degradation 280–287

340 Index measurement tools resistance (study) 66–77, 149–154 without calibration standard (study) 93 measurement tools, bias in FTIR measurement tool (study) 61–63 turbidity meter evaluation (study) 36–42, 45–46 measurement tools, configuring (study) 78–90 measurement tools, linearity of 29 resistance measurement tool (study) 76–77 measurement variation, identifying sources of 112–114 measurements across object without replicates (study) 275–279 moving range charts See I and MR charts MR charts See I and MR charts multiple-site measurement scenarios on multiple objects 288, 305–316 on single object 287–288, 294–301 mutual independence of observations 240–241

N non-normal data, control charts with 261, 316–332 % Nonunif See percent nonuniformity normal distribution 236 NP charts 245–247, 329–330 null hypothesis 16

O object standards 114, 128 objectives of measurement studies 112 observations logging 114 mutual independence of 240–241 One Sample Mean option 18 operator effects 113

outlier values, excluding configuring a measurement tool (study) 81–82 measurements across object without replicates (study) 280 oxide thickness measurement tool (study) 51–53 overlay plot to evaluate unusual values 48–51 variability charts vs. 79 oxide films, polishing (examples) 204–217, 222–229 oxide study (example) 128–135 oxide thickness measurement tool (study) 47–56

P P charts 245–247, 329–330 P/T ratio 5 capability potential vs. 7–10 deposited film thickness (example) 123–125 ellipsometer (study) 95–96 FTIR measurement tool (study) 63–64 linearity and 76 oxide study (example) 131 resistance measurement tool (study) 153 sample degradation in measurement study (example) 283–284 thin film gauge (study) 143–144 turbidity meter evaluation (study) 43–44, 46, 138–139 paired analysis See matched pair analysis passive data collection (PDC) 166 amount of data required 166 chemical mechanical planarization 208–230 downstream etch process (study) 196–207

Index epitaxial silicon deposition (study) 189–196 expected results 167 performing 167–169 planning 167–168 preliminary process study (example) 181–189 thermal disposition process (study) 172–180 PDC See passive data collection percent nonuniformity 169 thermal disposition process (study) 176 photolithography operation (example) 294–301 pixel defects (study) 328–332 planning measurement studies 112–120 PDC experiments 167–168 polishing oxide films (examples) 204–217, 222–229 tungsten films 217–221 power, test 17 power curves 18–20 precision/tolerance ratio See P/T ratio precision vs. accuracy in measurement 11–15 preliminary process study (example) 181–189 Presummarize option 271 process specification limits, incorrect (study) 302–305

Q questionable values, excluding configuring a measurement tool (study) 81–82 hiding questionable values 51–53 measurements across object without replicates (study) 280

341

oxide thickness measurement tool (study) 51–53

R R charts 238–243 random effects 30 regression analysis 161, 230 chemical mechanical planarization 215–217, 220–221, 226–227 downstream etch process (study) 203–204 ellipsometer (study) 98–99 epitaxial silicon deposition (study) 194–195 FTIR measurement tool (study) 157–159 multiple-site measurement on multiple objects (study) 310–312 oxide study (example) 132–135 resistance measurement tool (study) 153–154 study with operators and replicates 104–106 thin film gauge (study) 147–148 REML in variance component analysis 178 repeatability error 28, 111, 145 See also entries at measurement capability configuring a measurement tool (study) 78–90 ellipsometer (study) 94–99 FTIR measurement tool (study) 56–66 in total measurement 111 oxide thickness measurement tool (study) 47–56 resistance measurement tool (study) 66–77 turbidity meter evaluation (study) 33–47 with operators and replicates (study) 99–107 without a calibration tool (study) 91–94 replicated measurements at single sample site (study) 267–275

342 Index reproducibility error 29, 111–162 deposited film thickness (example) 120–127 FTIR measurement tool (study) 155–161 in total measurement 111 oxide study (example) 128–135 planning measurement studies 112–120 resistance measurement tool (study) 149–154 thin film gauge (study) 140–149 turbidity meter evaluation (study) 135–139 resistance measurement tool (study) 66–77, 149–154 risk 16–17 RR percent 6

S S charts 238–243 manufacturing process with incorrect specification limits (study) 304 photolithography operation (example) 294–301 replicated measurements at single sample site (study) 273–275 sample degradation in measurement study (example) 284–285 sample degradation in measurement study (example) 280–287 sample size requirements 18–20, 236–237 scheduling measurement trials 114–115 SEMATECH approach to non-normal data 316 SEMATECH Qualification Plan 165–166 Sigma Quality level deposited film thickness (example) 125–127 ellipsometer (study) 95–96 FTIR measurement tool (study) 64–66 signal-to-noise ratio 5, 7–10 silicon deposition, epitaxial (study) 189–196

silicon oxide films, polishing (examples) 204–217, 222–229 Six Sigma quality of measurement 6 SNR (signal-to-noise ratio) 5, 7–10 sources of variation, identifying 112–114 SPC (statistical process control) 21 special causes tests 253–254 specification limits, incorrect (study) 302–305 Stack routine 68 standard deviation confidence interval for 22–24 uncertainty in 21–28 standard deviation charts See S charts standard objects for study 114, 128 stating objective of measurement study 112 statistical process control (SPC) 21 Student's t statistic 41 suspicious values, excluding configuring a measurement tool (study) 81–82 measurements across object without replicates (study) 280 oxide thickness measurement tool (study) 51–53 system measurement error See entries at measurement capability

T table concatenation 82–84 Test Mean option 37 test power 17 thermal disposition process (study) 172–180 thickness, deposited film (example) figure of merit 123–125 Sigma Quality level 125–127 thin film gauge (study) 140–149 total measurement error 111 See also entries at measurement capability tracking defects 288 pixel testing (study) 328–332

Index trend analysis 252–256 Tukey HSD test epitaxial silicon deposition (study) 193–194 preliminary process study (example) 184–185 tungsten films, polishing 217–221 turbidity meter evaluation (study) 33–47, 135–139 two-tailed tests 41

U U charts 247–248, 316 uncertainty in means and standard deviations 21–28 uniformly weighted moving average charts 249 unit normal distribution 236 UWMA charts 249

V variability charts 162, 230 chemical mechanical planarization 212–214, 223–224 configuring a measurement tool (study) 79–82 determining mutual independence of observations 240–241 downstream etch process (study) 202–203 ellipsometer (study) 96–98 epitaxial silicon deposition (study) 191–192 FTIR measurement tool (study) 159–160 manufacturing process with incorrect specification limits (study) 303 measurements across object without replicates (study) 275–276 multiple-site measurement on single object (study) 298–299 overlay charts vs. 79 oxide thickness measurement tool (study) 53–55

343

replicated measurements at single sample site (study) 267–268 resistance measurement tool (study) 69–71 sample degradation in measurement study (example) 281–282 study with operators and replicates 102–104 thermal disposition process (study) 174–176, 177–178 thin film gauge (study) 144–145 turbidity meter evaluation (study) 136–137 Variability/Gage Chart platform 160–161, 162, 230 ellipsometer (study) 96–98 oxide thickness measurement tool (study) 53 resistance measurement tool (study) 69 variables adding to manage extra measurements 141 control charts for variables data 235, 237–245, 249–252 variance component analysis 230 chemical mechanical planarization 213–214, 219–221, 226–227 determining mutual independence of observations 240–241 downstream etch process (study) 204–205 ellipsometer (study) 96–98 EMS vs. REML for 178 epitaxial silicon deposition (study) 193 manufacturing process with incorrect specification limits (study) 303 multiple-site measurement on multiple objects (study) 308–309 multiple-site measurement on single object (study) 299

344 Index preliminary process study (example) 183–185 replicated measurements at single sample site (study) 267–268 sample degradation in measurement study (example) 281–282 thermal disposition process (study) 175–176, 177–178 Variance Components option 71 variation, identifying sources of 112–114

W wafers, cleaning (study) 322–328 Western Electric Rules for trend analysis 253–254 Westgard Rules for trend analysis 254–255 Wilcoxon Signed Rank test 39 witness wafers (study) 320–322

X XBar charts 238–243 manufacturing process with incorrect specification limits (study) 304 photolithography operation (example) 294–301

Books Available from SAS Press

Advanced Log-Linear Models Using SAS®

The Cartoon Guide to Statistics

by Daniel Zelterman

by Larry Gonick and Woollcott Smith

Analysis of Clinical Trials Using SAS®: A Practical Guide by Alex Dmitrienko, Geert Molenberghs, Walter Offen, and Christy Chuang-Stein

Analyzing Receiver Operating Characteristic Curves with SAS® by Mithat Gönen

Categorical Data Analysis Using the SAS ® System, Second Edition by Maura E. Stokes, Charles S. Davis, and Gary G. Koch

Cody’s Data Cleaning Techniques Using SAS® Software by Ron Cody

Annotate: Simply the Basics by Art Carpenter

Common Statistical Methods for Clinical Research with SAS ® Examples, Second Edition

Applied Multivariate Statistics with SAS® Software, Second Edition

by Glenn A. Walker

by Ravindra Khattree and Dayanand N. Naik

The Complete Guide to SAS ® Indexes

Applied Statistics and the SAS ® Programming Language, Fifth Edition

CRM Segmemtation and Clustering Using SAS ® Enterprise MinerTM

by Ronald P. Cody and Jeffrey K. Smith

by Randall S. Collica

by Michael A. Raithel

An Array of Challenges — Test Your SAS ® Skills

Data Management and Reporting Made Easy with SAS ® Learning Edition 2.0

by Robert Virgile

by Sunil K. Gupta

Building Web Applications with SAS/IntrNet®: A Guide to the Application Dispatcher

Data Preparation for Analytics Using SAS® by Gerhard Svolba

by Don Henderson

Carpenter’s Complete Guide to the SAS® Macro Language, Second Edition

Debugging SAS ® Programs: A Handbook of Tools and Techniques by Michele M. Burlew

by Art Carpenter

Carpenter’s Complete Guide to the SAS® REPORT Procedure

Decision Trees for Business Intelligence and Data Mining: Using SAS® Enterprise MinerTM by Barry de Ville

by Art Carpenter

support.sas.com/pubs

Efficiency: Improving the Performance of Your SAS ® Applications

Learning SAS ® in the Computer Lab, Second Edition by Rebecca J. Elliott

by Robert Virgile

The Essential Guide to SAS ® Dates and Times by Derek P. Morgan

Fixed Effects Regression Methods for Longitudinal Data Using SAS® by Paul D. Allison

Genetic Analysis of Complex Traits Using SAS ® by Arnold M. Saxton

A Handbook of Statistical Analyses Using SAS®, Second Edition by B.S. Everitt and G. Der

Health Care Data and SAS® by Marge Scerbo, Craig Dickstein, and Alan Wilson

The Little SAS ® Book: A Primer by Lora D. Delwiche and Susan J. Slaughter The Little SAS ® Book: A Primer, Second Edition by Lora D. Delwiche and Susan J. Slaughter (updated to include SAS 7 features) The Little SAS ® Book: A Primer, Third Edition by Lora D. Delwiche and Susan J. Slaughter (updated to include SAS 9.1 features) The Little SAS ® Book for Enterprise Guide® 3.0 by Susan J. Slaughter and Lora D. Delwiche The Little SAS ® Book for Enterprise Guide® 4.1 by Susan J. Slaughter and Lora D. Delwiche

by Thomas Miron

Logistic Regression Using the SAS® System: Theory and Application by Paul D. Allison

In the Know... SAS® Tips and Techniques From Around the Globe, Second Edition

Longitudinal Data and SAS®: A Programmer’s Guide by Ron Cody

The How-To Book for SAS/GRAPH ® Software

by Phil Mason

Instant ODS: Style Templates for the Output Delivery System by Bernadette Johnson Integrating Results through Meta-Analytic Review Using SAS® Software by Morgan C. Wang and Brad J. Bushman

Maps Made Easy Using SAS® by Mike Zdeb Measurement, Analysis, and Control Using JMP®: Quality Techniques for Manufacturing by Jack E. Reece Models for Discrete Date by Daniel Zelterman

Introduction to Data Mining Using SAS® Enterprise MinerTM

Multiple Comparisons and Multiple Tests Using SAS® Text and Workbook Set

by Patricia B. Cerrito

(books in this set also sold separately) by Peter H. Westfall, Randall D. Tobias, Dror Rom, Russell D. Wolfinger, and Yosef Hochberg

Learning SAS ® by Example: A Programmer’s Guide by Ron Cody


Multiple-Plot Displays: Simplified with Macros by Perry Watts

Reading External Data Files Using SAS®: Examples Handbook by Michele M. Burlew

Multivariate Data Reduction and Discrimination with SAS ® Software by Ravindra Khattree and Dayanand N. Naik

Regression and ANOVA: An Integrated Approach Using SAS ® Software by Keith E. Muller and Bethel A. Fetterman

Output Delivery System: The Basics by Lauren E. Haworth

Painless Windows: A Handbook for SAS ® Users, Third Edition by Jodie Gilmore (updated to include SAS 8 and SAS 9.1 features)

Pharmaceutical Statistics Using SAS®: A Practical Guide Edited by Alex Dmitrienko, Christy Chuang-Stein, and Ralph D’Agostino

The Power of PROC FORMAT by Jonas V. Bilenas

Predictive Modeling with SAS® Enterprise MinerTM: Practical Solutions for Business Applications

SAS ® For Dummies® by Stephen McDaniel and Chris Hemedinger

SAS ® for Forecasting Time Series, Second Edition by John C. Brocklebank and David A. Dickey

SAS ® for Linear Models, Fourth Edition by Ramon C. Littell, Walter W. Stroup, and Rudolf Freund

SAS ® for Mixed Models, Second Edition by Ramon C. Littell, George A. Milliken, Walter W. Stroup, Russell D. Wolfinger, and Oliver Schabenberger

by Kattamuri S. Sarma, Ph.D.

SAS® for Monte Carlo Studies: A Guide for Quantitative Researchers

PROC SQL: Beyond the Basics Using SAS®

˝ by Xitao Fan, Ákos Felsovályi, Stephen A. Sivo, and Sean C. Keenan

by Kirk Paul Lafler

PROC TABULATE by Example by Lauren E. Haworth

Professional SAS® Programmer’s Pocket Reference, Fifth Edition

SAS ® Functions by Example by Ron Cody

SAS® Graphics for Java: Examples Using SAS® AppDev StudioTM and the Output Delivery System

by Rick Aster

by Wendy Bohnenkamp and Jackie Iverson

Professional SAS ® Programming Shortcuts, Second Edition

SAS ® Guide to Report Writing, Second Edition

by Rick Aster

Quick Results with SAS/GRAPH ® Software by Arthur L. Carpenter and Charles E. Shipp

Quick Results with the Output Delivery System by Sunil Gupta

by Michele M. Burlew

SAS ® Macro Programming Made Easy, Second Edition by Michele M. Burlew

SAS ® Programming by Example by Ron Cody and Ray Pass


by Paul E. Spector

A Step-by-Step Approach to Using SAS ® for Univariate and Multivariate Statistics, Second Edition

SAS ® Programming in the Pharmaceutical Industry

by Norm O’Rourke, Larry Hatcher, and Edward J. Stepanski

SAS ® Programming for Researchers and Social Scientists, Second Edition

by Jack Shostak

SAS® Survival Analysis Techniques for Medical Research, Second Edition by Alan B. Cantor

by Larry Hatcher

SAS ®

System for Elementary Statistical Analysis, Second Edition

by Sandra D. Schlotzhauer and Ramon C. Littell

by Rudolf J. Freund and Ramon C. Littell

Tuning SAS ® Applications in the OS/390 and z/OS Environments, Second Edition by Michael A. Raithel

SAS ® System for Statistical Graphics, First Edition by Michael Friendly

The SAS ® Workbook and Solutions Set (books in this set also sold separately) by Ron Cody

Saving Time and Money Using SAS® by Philip R. Holland

Selecting Statistical Techniques for Social Science Data: A Guide for SAS® Users by Frank M. Andrews, Laura Klem, Patrick M. O’Malley, Willard L. Rodgers, Kathleen B. Welch, and Terrence N. Davidson

Statistical Quality Control Using the SAS ® System by Dennis W. King

Statistics Using

Survival Analysis Using SAS ®: A Practical Guide by Paul D. Allison

SAS ® System for Regression, Third Edition

SAS ®

Step-by-Step Basic Statistics Using SAS ®: Student Guide and Exercises (books in this set also sold separately)

Univariate and Multivariate General Linear Models: Theory and Applications Using SAS ® Software by Neil H. Timm and Tammy A. Mieczkowski

Using SAS ® in Financial Research by Ekkehart Boehmer, John Paul Broussard, and Juha-Pekka Kallunki

Using the SAS ® Windowing Environment: A Quick Tutorial by Larry Hatcher

Visualizing Categorical Data by Michael Friendly

Web Development with SAS® by Example, Second Edition by Frederick E. Pratter

Enterprise

Guide®

by James B. Davis

A Step-by-Step Approach to Using the SAS ® System for Factor Analysis and Structural Equation Modeling by Larry Hatcher


Your Guide to Survey Research Using the SAS ® System by Archer Gravely

JMP® Books

Elementary Statistics Using JMP® by Sandra D. Schlotzhauer

JMP ® for Basic Univariate and Multivariate Statistics: A Step-by-Step Guide by Ann Lehman, Norm O’Rourke, Larry Hatcher, and Edward J. Stepanski

JMP ® Start Statistics, Third Edition by John Sall, Ann Lehman, and Lee Creighton

Regression Using JMP ® by Rudolf J. Freund, Ramon C. Littell, and Lee Creighton


Measurement, Analysis, and Control Using Jmp: Quality Techniques for Manufacturing

Measurement and Control Basics (Resources for Measurement and Control Series)

Elementary Statistics Using JMP

Regression Using JMP

Image quality measurement using the Haar wavelet

Quality Control for Dummies

Quality Control for Dummies

Quantum Measurement and Control

Particle Control for Semiconductor Manufacturing

Quantum Measurement and Control

Fraud Analysis Techniques Using ACL

Quantum Measurement and Control

Analytical Techniques for Atmospheric Measurement

Techniques for Adaptive Control

Techniques for Adaptive Control

Techniques for Adaptive Control

Systems for Planning and Control in Manufacturing

Flavor Quality, Objective Measurement

Statistical tools for environmental quality measurement

Statistical Tools for Environmental Quality Measurement

Nonlinear analysis and synthesis techniques for aircraft control

Practical Data Analysis with JMP

Intelligent Control Systems Using Computational Intelligence Techniques

Productivity Measurement and Analysis

Control Techniques for Complex Networks

Statistical Techniques for Project Control

Quality Assurance for Environmental Analysis

Quality Assurance for Environmental Analysis

Control techniques for complex networks

Control Techniques for Complex Networks

Fiber Optic Measurement Techniques

Measurement, Analysis, and Control Using Jmp: Quality Techniques for Manufacturing

Measurement and Control Basics (Resources for Measurement and Control Series)

Elementary Statistics Using JMP

Regression Using JMP

Image quality measurement using the Haar wavelet

Quality Control for Dummies

Quality Control for Dummies

Quantum Measurement and Control

Particle Control for Semiconductor Manufacturing

Quantum Measurement and Control

Fraud Analysis Techniques Using ACL

Quantum Measurement and Control

Analytical Techniques for Atmospheric Measurement

Techniques for Adaptive Control

Techniques for Adaptive Control

Techniques for Adaptive Control

Systems for Planning and Control in Manufacturing

Flavor Quality, Objective Measurement

Statistical tools for environmental quality measurement

Statistical Tools for Environmental Quality Measurement

Nonlinear analysis and synthesis techniques for aircraft control

Practical Data Analysis with JMP

Intelligent Control Systems Using Computational Intelligence Techniques

Productivity Measurement and Analysis

Control Techniques for Complex Networks

Statistical Techniques for Project Control

Quality Assurance for Environmental Analysis

Quality Assurance for Environmental Analysis

Control techniques for complex networks

Control Techniques for Complex Networks

Fiber Optic Measurement Techniques

Recommend Documents