ANOVA Gage R&R – Part 3

October 2012

This month’s newsletter is the third in a three-part series on using the ANOVA method for a Gage R&R study. This method uses analysis of variance to analyze the results of a Gage R&R study instead of the classical Average and Range Method

Many people refer to the AIAG’s Measurement Systems Analysis Manual (www.aiag.org) when doing Gage R&R studies. AIAG stands for Automotive Industry Action Group. For years, the most used method for Gage R&R was the Average and Range Method. This method calculates the % of the total variation that is due to the Gage R&R (repeatability and reproducibility). The total variation is based on the standard deviation.

Recently more people are beginning to prefer the ANOVA method for conducting Gage R&R studies. The ANOVA method includes the interaction between the operators (or appraisers) and the parts. ANOVA breaks down the variance into four components: parts, operators, interaction between parts and operators and the repeatability error due to the measurement system (or gage) itself. The interaction seldom is often insignificant and has little, if any, impact on the results.

The ANOVA method, however, allows you to estimate the variance of the components. If you base the % of total variation on the variance, instead of the standard deviation, you can get very different results – as we will show you in this newsletter. Which is the best method to use?

In this issue:

• Sources of Variation
• Example Data
• Results Using the ANOVA Method
• Removing the Interaction Term
• Standard Deviation and the ANOVA Method
• Comparison of Average/Range Method to ANOVA Method
• Which Should I Use?
• Quick Links

The first part of this series focused on part of the ANOVA table. We took an in-depth look at how the sum of squares and degrees of freedom were determined. Many people do not understand how the calculations work and the information that is contained in the sum of squares and the degrees of freedom. In the second part of this series we completed the ANOVA table and showed how to determine the % of total variance that is due to the measurement system or what is also called the % GRR.

We will start this newsletter with a review of the sources of variation in a process. This is followed by refreshing your memory with the data set we are using and the ANOVA results from the first and second part of this newsletter series. We will then look at how AIAG uses the standard deviation to estimate of the % GRR and how this differs from using the variance to perform the estimate of the % GRR. We will then compare the ANOVA method and the average and range method.

As always, please feel free to leave a comment.

Sources of Variation

Suppose you are monitoring a process by pulling samples of the product at some regular interval and measuring one critical quality characteristic (X). Obviously, you will not always get the same result when measure for X. Why not? There are many sources of variation in the process. However, these sources can be grouped into three categories:

• variation due to the process itself
• variation due to sampling
• variation due to the measurement system

These three components of variation are related by the following:

σ_t²= σ_p²+ σ_s²+ σ_ms²

where σ_t²is the total process variance; σ_p²is the process variance; σ_s²is the sampling variance and σ_ms²is the measurement system variance. Note that the relationship is linear in terms of the variance (which is the square of the standard deviation), not the standard deviation.

For our purposes here, we will ignore the variance due to sampling (or more correctly, just include it as part of the process itself). However, for some processes, sampling variation can greatly impact the results. Thus, we will consider the total variance to be:

σ_t²= σ_p²+ σ_ms²

Remember geometry? The right triangle? The Pythagorean Theorem? The above equation can be represented by the triangle below.

Figure 1: Sources of Variation

The total standard deviation, σ_t, for a measurement is equal to the length of the hypotenuse. The process standard deviation, σ_p, is equal to the length of one side of the triangle and the measurement system standard deviation, σ_ms, is equal to the length of the remaining side.

You can easily see from this triangle what happens as the variation in the product and measurement system changes. If the product standard deviation is larger than the measurement standard deviation, it will have the larger impact on the total standard deviation. However, if the measurement standard deviation becomes too large, it will begin to have the largest impact.

Thus, the objective of improving a measurement system is to minimize the % variance due to the measurement system:

% Variance due to measurement system = 100(σ_ms²/σ_t²)

In a Gage R&R study, you can break down σ_ms²into its two components:

Repeatability is the ability of the measurement system to repeat the same measurements on the same sample under the same conditions. It represents an assessment of the ability to get the same measurement result each time.

Reproducibility is the ability of measurement system to return consistent measurements while varying the measurement conditions (different operators, different parts, etc.) It represents an assessment of the ability to reproduce the measurement of other operators.

Example Data

We are using the data from our December 2007 newsletter on the Average and Range Method for Gage R&R. In this example, there were three operators who tested five parts three times. A picture of part of the Gage R&R design is shown below.

Figure 2: Gage R&R Setup

Operator 1 will test 5 parts three times each. In the figure above, you can see that Operator 1 has tested Part 1 three times. What are the sources of variation in these three trials? It is the measurement equipment itself. The operator is the same and the part is the same. The variation in these three trials is a measure of the repeatability. It is also called the equipment variation in Gage R&R studies or the “within” variation in ANOVA studies.

Operator 1 also runs Parts 2 through 5 three times each. The variation in those results includes the variation due to the parts as well as the equipment variation. Operator 2 and 3 also test the same 5 parts three times each. The variation in all results includes the equipment variation, the part variation, the operator variation and the interaction between operators and parts. The variation in all results is the reproducibility.

The data from the December 2007 newsletter are shown in the table below.

Table 1: The Gage R&R Data

The operator is listed in first column and the part numbers in the second column. The next three columns contain the results of the three trials for that operator and part number. For example, the three trial results for Operator A and Part 1 are 3.29, 3.41 and 3.64.

Results Using the ANOVA Method

The data was analyzed using the SPC for Excel software. The ANOVA table is shown below.

Table 2: The ANOVA Results

As you can see in the table, the “operator by part” source is not significant. Its p value is 0.9964. Many software packages contain an option to remove the interaction if the p value is above a certain value – most often 0.25. In that case, the interaction is rolled into the equipment variation. We kept it in the calculations in Part 2 – though it has little impact since its mean square is so small.

The results for all the sources of variation are shown in the table below. The calculations are given in detail in Part 2 of this newsletter series. The calculations show how to estimate the variances in the table below. Then to find the contribution to % of total variance, you simply divide the estimate of the variance for the source by the total variance. You can see from the results, that the Gage R&R is responsible for 12.14% of the total variance.

Table 3: Contribution to % Total Variance Including the Interaction

The % GRR given in Table 3 will not be the same as the one you get from the Average and Range Method – as we will show below. The approach above uses the total variance to compare the each source of variation while the Average and Range Method uses the total standard deviation.

Removing the Interaction

As mentioned above, many software packages have the option to remove the interaction term if alpha is above a certain value (usually 0.25). This means that if the p value for the interaction term in the ANOVA table is greater than this value, the interaction is considered to be zero and is removed. As you can see in the ANOVA table above, the p value for the interaction term is 0.9964. If this term is removed, the contribution to the % of total variance changes slightly as seen in the table below.

Table 4: Contribution to % Total Variance Without the Interaction

If the interaction is significant, removing it will have a much greater impact on the result. You should not remove the interaction if it is significant.

Standard Deviation and the ANOVA Method

You should note that the results in the table above are based on the contribution of each source to the % of total variance. AIAG’s average and range method bases the results on standard deviation, not the variance. This is one of the major issues some folks have with the average and range method – the standard deviations are not additive – like the variances are.

However, AIAG says that “the standard deviation is easier to interpret than variance because it has the same unit of measure as the original observation” (page 196, 4th edition). In fact, AIAG includes the standard deviation in the ANOVA output. The output looks something like the following:

Table 5: % of Total Variation Based on Standard Deviation

The table above does not include the interaction term since it is insignificant. The first column is the source of the variation – such as equipment. The second column is the estimate of the variance from the ANOVA results (see Part 2 of this series for how to estimate these). These values are the same as those in table 4. The third column is the standard deviation. This is simply the square root each of the variances.

The fourth column multiplies the standard deviation by a “sigma” multiplier. In the past, 5.15 was used, but the more recent editions of the Measurement Systems Analysis manual uses 6. This gives a spread of the results for each source of variation – in this case 6 sigma. In the table above, EV = equipment variability, AV = the appraiser (or operator variability), PV = the part variation and TV = the total variation.

The fifth column is the % of total variation. This is simply each source of variation’s six sigma spread divided by the total six sigma spread. So,

GRR % of Total Variation = 1.878/5.679 = 33%

The sixth column is the % of total variance for each source from the ANOVA analysis (Table4). You can see that the results are considerably different depending on the approach. Note that AIAG refers to this column as % contribution – but it is the % of total variance.

Why the large difference when looking at the % of total variation or the % of total variance? It is because the variances are additive and the standard deviations are not.

σ_Total²= σ_Equipment²+ σ_Operators²+ σ_Parts²

σ_Total≠ σ_Equipment+ σ_Operators+ σ_Parts

What this means is that when you add up the contribution of each source to the total variance, you will get 100%. However, when you add up the contribution to each source to the total variation (based on the standard deviation), you will not get 100%. In fact, when using the standard deviation, the % of total variation never add to 100. The table below shows the results for the data we are looking at.

Table 6: Comparing % of Total Variation and % of Total Variance

 
Figure 3 shows the percentages in graphical form. There is a significant difference between using the % of total variation or the % of total variance.

Figure 3: Comparing % of Total Variation and % of Total Variance

Comparison of Average/Range Method to ANOVA Method

The average and range method described in an earlier newsletter. The output from the SPC for Excel software is shown below for the average and range method.

Table 7: GRR Based on Average and Range Method

The values for EV, AV, R&R, PV and TV are standard deviations determined using the Average and Range Method. You can see that the standard deviations and % of total variation given in this table are similar to the results from the ANOVA that are based on using the standard deviation. This is shown in Table 8 below. Thus, there is really no difference between the Average and Range Method and the ANOVA method based on AIAG’s approach of using the standard deviation.

Table 8: Comparison of Average and Range Method with the ANOVA Method

So, the results from the average and range method are very similar to the ANOVA method. The major question that should be asked is should I be basing the results on the standard deviation or on the variance?

Which Should I Use?

The question is not if you should use the ANOVA method or the Average and Range method for a Gage R&R study. Use whatever the customer wants you to. You will get pretty much the same results. The real question is: Do I base the results on the % of total variation (based on the standard deviation) or on the % of the total variance? I think it should be % of total variance. Variances are additive. The standard deviation is not. Our November 2009 newsletter took a look at how to evaluate a measurement system using control charts and the % of total variance due to the measurement system. We compared the results to the Discrimination Ratio discussed by Dr. Don Wheeler. We found that the % of total variance approach matched well. We defined a good measurement system as the following:

A good measurement system should not be responsible for more than 10% of the total variance.

If you have the option, go with the % of total variance.