Have you ever had to support your position that your measurement system is well suited for its intended purpose? This addition to our SPC knowledge database takes a look at how you can do this when a part is altered or destroyed during testing. Our SPC knowledge database has additional articles on how to perform other types of Gage R&R studies.
Any Gage R&R study is really an experiment to determine the various sources of variation. How you set up the experiment determines what sources of variation you can analyze. Last month we took a look at the differences in how a classical Gage R&R study and a destructive Gage R&R study are set up. In the classical Gage R&R study, a part is not altered or destroyed – you can re-test the same part multiple times. With a destructive Gage R&R study, the part is destroyed or altered during testing and you cannot measure it multiple times. This month’s addition to our SPC knowledge base examines how you analyze a destructive Gage R&R study.
In this issue:
- Destructive Gage R&R Analysis Setup
- Example Data
- Results of Destructive Gage R&R
- ANOVA Table
- Contribution to Total Variance
- Contribution to Total Standard Deviation, Specifications and Process Standard Deviation
- Quick Links
Destructive Gage R&R Analysis Setup
We will start with a quick review of how a destructive Gage R&R experiment is set up. Since the part or sample is altered or destroyed during testing, it cannot be retested. For example, in heat treating of steel tubes, a tensile test is often done to measure tensile strength. The sample is destroyed during testing so you cannot have the same or different operators test that sample again.
Without the ability to retest, how can you estimate the Gage R&R? There are two critical items here. First, you have to be able to assume that a batch of material is so close to the same that you can reasonably assume that the parts in the batch are the “same” part. This means that the batch is homogeneous. In the perfect world, if you took any sample from that batch, the test result would be the same. Second is the design setup. If all the operators can measure parts from each batch, then you can use the traditional method of running a Gage R&R – the crossed design. However, if each operator cannot measure parts from each batch (e.g., not enough parts from each batch to do this), then a nested Gage R&R must be used.
Figure 1 shows how a destructive Gage R&R is laid out. In this example, there are only two parts from each batch. This is not enough for every operator to run parts from each batch since the part is destroyed during testing. Operator 1 runs two parts from batch 1 and two parts from batch 2. Operator 2 runs two parts from batch 3 and two parts from batch 4. The batches are different. Since each batch is unique to a single operator, this is called a nested Gage R&R.
Figure 1: Destructive (Nested) Gage R&R
You are involved in heat treating of parts and want to perform a Gage R&R analysis on the hardness tester. To measure hardness, a piece of the product is cut, prepared and tested. That piece is altered, so it cannot be retested. The parts are produced in small batches. You are confident that the parts within a batch are homogeneous. You want to include three operators in the Gage R&R study. You would like each operator to test two parts per batch. But there are not always enough parts for each operator to test parts from each batch. You will have to use a nested design. You decide to use 15 batches and take 2 parts from each batch. Operator 1 will measure the two parts for batches 1 to 5; operator 2 will measure 2 parts from batches 6 -10; and operator 3 will measure 2 parts from batches 11 – 15. The resulting from the Gage R&R study are shown in Table 1.
Table 1: Nested Gage R&R Raw Data
Results of Destructive Gage R&R
We will use analysis of variance (ANOVA) to analyze the results of our destructive Gage R&R study. This analysis method was described in detail on our three part series on ANOVA Gage R&R. You can review these newsletters for more information on the calculations. We will show the results here.
The data were analyzed using the SPC for Excel software package. Remember there are three things you can compare the Gage R&R results to:
- Total variation in the parts in the study
- Process standard deviation
Which of these you use depends on how you use the measurement system. If you are using the measurement for process control or SPC, then you use the first or second method above. If you are using the measurement system for inspection only, you would use the specification approach. We will see the results for all three here.
To use the process standard deviation, you need an estimate of that standard deviation. It can come from a control chart kept in production or from calculating the standard deviation from a large amount of production data (be wary of special causes). Suppose you have done that and your process standard deviation is 2.5. The specifications for hardness are 30 to 38.
The ANOVA table for these data is shown in Table 2.
Table 2: ANOVA for Hardness Nested Gage R&R
The first column is the source of variability. Remember that a Gage R&R study is a study of variation. There are four sources of variability in this ANOVA approach: the operator, the batch, the repeatability, and the total. Note that the batches are nested within operators. This is sometimes denoted as “Batch (Operator).”
The second column is the degrees of freedom associated with the source of variation. The degrees of freedom integer is simply the number of values of a statistic that are free to vary. For example, suppose you have a sample that contains n observations. We use the sample to estimate something – usually an average. When we want to estimate something, it costs us one degree of freedom. So, if we have n observations and want to estimate the average, then we have n – 1 degrees of freedom left.
The third column is the sum of squares (SS) associated with the source of variation. The sum of squares is a measure of variation. It measures the squared deviations around an average.
The fourth column is the mean square associated with the source of variation. The mean square is the estimate of the variance for that source of variability based on the amount of data we have (the degrees of freedom). So, the mean square is the sum of squares divided by the degrees of freedom. The mean square is the value that we will use to estimate different variances.
The fifth column is the F value. This is the statistic that is calculated to determine if the source of variability is statistically significant. It is the ratio of two variances (or mean squares in this case). The sixth column contains the p value. The p column is the column we want to examine first. If the p value is less than 0.05, it means that the source of variation has a significant impact on the results. In this example, the “batch” has significant impact on the results, while “operator” does not. This is what you want – it means that the measurement system can distinguish between the parts used in the study. Gage R&R studies attempt to quantify this by determine the % Gage R&R value.
Contribution to Total Variance
The best way to examine results is by looking at each source’s contribution to the total variance. This approach uses the variation of the parts uses in the study. Table 3 shows the results for this example.
Table 3: Contribution to Total Variance
|Source||Variance Component||% Contribution|
|Total Gage R&R||0.0807||5.02%|
The first column is the source of variation. The second column is the variance component for that source. Note that the variance for the repeatability is the same as the mean square for repeatability in the ANOVA table. The other mean squares in the ANOVA are used to estimate the variances for the part-to-part variation and total variation.
The last column in Table 3 is the % contribution to the total variation. This column is determined by dividing the source’s variance by the total variance. For example:
% contribution due to Total Gage R&R = 0.0807/1.607 = 5.02%
A rule of thumb often used to determine if a measurement system is acceptable is as follows:
- % Gage R&R ≤ 10%: measurement systems is acceptable
- 10% < % Gage R&R < 30%: measurement system may or may not be acceptable depending on its use and the customer
- %Gage R&R ≥ 30%: measurement system needs improvement
Note that the variance component column is additive, i.e., the total variation is sum of the individual sources of variation.
So, based on these results, the hardness tester is responsible for about 5% of the total variance. This test method appears to be very good.
Contribution to Total Standard Deviation, Specifications and Process Standard Deviation
There are three other ways to look at the results. You can look at the contribution each source has to the total standard deviation from the study, to the specifications, and the process standard deviation. These results are shown in Table 4.
Table 4: Other Contributions
|Source||Stand. Dev.||6*SD||% Contribution||% Tolerance||% Process|
|Total Gage R&R||0.284||1.704||22.40%||21.30%||11.36%|
Again, the first column is the source of variation. The second column in the standard deviation. This is simply the square root of the variances given in Table 3. Note that this column is not additive – the total standard deviation does not equal the sum of the standard deviation of the individual sources. This is why the last two columns in the table do not sum to 100%.
The column “6*SD” is six times the standard deviation of the source of variation. This is the “spread” that is used to “judge” the standard deviation against. It is based on the fact that most of the data for a normal distribution is within +/- 3 standard deviations of the averages – or is contained in a spread of 6 standard deviations.
The “% Contribution” column is determined by dividing the 6*SD spread for the source of variation by the value of 6*SD for the total variation. Thus for total Gage R&R:
% contribution for total Gage R&R = 1.704/7.607 = 22.4%
This means that the “spread” of the Gage R&R takes up 22.4% of the total spread. Note that the % total Gage R&R % contribution is 22.4% – compared to 5.02% when looking at the variances. This result implies that the test method may need some work.
You can also compare the standard deviation to the specification range. The % tolerance column does this and is the 6*SD spread for the source of variation divided by the tolerance range (38 – 30) = 8. For example,
% of total tolerance due to Gage R&R = 1.704/8 = 21.3%.
This means that the Gage R&R spread takes up 21.3 % of the tolerance spread.
The column labelled “% Process” is determined by dividing the 6*SD for the individual source of variation by the process spread, which is 6 times the process standard deviation. In this example, we estimated that the process standard deviation was 2.5. So,
% of total process spread due to Gage R&R = 1.704/(6*2.5) = 11.36%
This means that the Gage R&R spread takes up about 11% of the total process spread.
The chart in Figure 1 summarizes the results. You can see that the results vary depending on what you are comparing the results against – the variation in the parts (using either the variance of standard deviation approach), the tolerance, or a known process standard deviation.
Figure 1: Variance Components for Destructive Gage R&R
This is one reason that Gage R&R is not crystal clear at times. The results depend on how you analyze the results. But in the end, whether or not a measurement system is acceptable depends on you and your customer.
This month’s addition to our SPC knowledge base has looked at how you analyze a destructive (nested) Gage R&R experiment. In this type of experiment, the part is destroyed or altered and cannot be re-measured.
Thanks so much for reading our publication. We hope you find it informative and useful. Happy charting and may the data always support your position.
Dr. Bill McNeese
BPI Consulting, LLC