The Importance of the Operator by Part Control Chart in Gage R&R Studies

August 2023

Gage R&R studies are widely used to determine how “good” a measurement process is.  The emphasis is usually on the % of the total variance that is due to the measurement process.  The analysis sometimes includes the construction of the Operator by Part control chart.  It always should. This control chart gives you a lot of information – all in visual form.  In fact, this control chart could tell you that you need to fix some issues and repeat the study.  But it also gives you an initial visual look into how “good” your measurement process is.

This publication takes a closer look at the Operator by Part Control Chart in Gage R&R studies.  In this publication:

• Two Key Decisions in Setting up a Basic Gage R&R Study
• Example Data
• Operator-Part Range Chart
• Operator-Part  Chart
• Determining the % Variance due to the Measurement Process
• Past SPC Knowledge Base Articles on Measurement Systems Analysis
• Summary
• Quick Links

Please feel free to leave a comment at the end of this publication.  You can download a pdf copy of this publication at this link.

Two Key Decisions in Setting up the Basic Gage R&R Study

There are two key decisions you must make in setting up a Gage R&R study.  One is to determine the number of parts, the number of operators and the number of trials to use.  The number of parts should be greater than or equal to 5. The number operators and the number of trials must be greater than or equal to 2.   If possible, include all the operators who run the measurement process in the study.

The next key decision is deciding how to obtain the parts to include in the study.  Part of the analysis involves comparing the measurement process variance to the total variance.  One method to estimate the total variance is to use the parts in the study. This assumes that the parts represent the variation in the process.  This is quite often very difficult to do in practice.  Obviously choosing 10 parts in a row is not going to be representative of the variation in the process.

A better method of determining the process variance is to make a control chart from several months of process data. Then use the range chart to estimate the process standard deviation.  This is often called the historical sigma.  You choose a time period that is large enough so the vast majority of sources of variation have the opportunity to occur.

After these key decisions are made, you can run the study like any other Gage R&R study – in random order, operators not knowing any previous results, etc.

Example Data

Suppose we decide to run a Gage R&R with 10 parts, 3 operators, and 3 trials. That is a total of 90 samples. The three operators are Chris, John and Mary.  Each operator runs 10 parts, 3 times each.  The data are shown in Table1.  The data are from the Automotive Industry Action Group (AIAG)  Measurement Systems Analysis manual.

Table 1: Basic Gage R&R Study Data

The question is:

How does the Gage R&R study use this data to find insights into the measurement process?

It starts with a range chart.  We will start with the operator by part range chart.

Operator-Part Range Chart

Control charts study various sources of variation.  One thing we would like to know about our measurement process is how much variation there is in it.  If one person measures the same part over and over, the difference in the results is a measure of the measurement process variation.

How can the data in Table 1 be subgrouped so that we can see that variation in the measurement process?  Look at the first column in Table 1. You will see 3 results for Chris.  The 3 results represent the value Chris received when running Part 1 in the measurement process.  He ran Part 1 three times.

So, what is the reason for the difference in the three results: 0.29, 0.41 and 0.64?  It is the same operator and the same part.  The only reason for the difference is the measurement process.  A measure of the measurement process variation is then the range of these three samples: |0.64 – 0.29| = 0.35.  Is this a good measure of the measurement process variation?  Probably not, it is just one subgroup.  But there is a total of 30 such subgroups in Table 1.  We can calculate the range in each one and construct a range chart.  The range chart is shown in Figure 1.

Figure 1: Operator-Part Range Chart

The range for each of the subgroups is plotted.  The average and the control limits are then calculated and added to the control chart as shown above.  The calculations are shown below.

R̅ = SR_i/k = 10.25/30 = 0.342

LCLr = D₃R̅ = None

UCLr = D₄R̅ = 0.88

where k is the number of subgroups, D₃ and D₄ are control chart constants that depend on subgroup size, R̅ is the average range, LCLr is the lower control, and UCLr is the upper control limit.  D₃ does not exist for a subgroup size of 3, so there is no lower control limit on the range chart.  D₄ is 2.574 for a subgroup size of 3.

The R chart is analyzed.  If all the points are within the control limits and there are no patterns (like 8 in a row above or below the average), the process is said to be consistent and predictable.  It is in statistical control.  You can predict what will happen in the future.  It also means that you have a good value for the average range.  Remember, the average range is a measure of the measurement process variation.

If there are out of control points or patterns present, then the process is said to be out of control.  It is not consistent and predictable.  The average range may not be a good measure of the measurement process variation.

Look at Figure 1.  What do you see?  Remember the R chart checks for consistency between operators.  It shows the results for the repeated measurements for each operator for each part.  Here is the conclusion:

• There is 1 out of control point on the R chart; the ranges are not consistent.
• The reason for the out of control point should be corrected and the study repeated.

You may not need to repeat the study all the time.  Sometimes you might just rerun the operator-part that was out of control.  If there are a lot of out of control points on the R chart, you need to investigate why, correct the problem and rerun the study.   Or you may just ignore the out of control point if there is just one. For this publication, we will assume that the one out of  control point does not significantly impact the results.

One more item in Figure 1: the operator Chris appears to have less variation than the other two operators in running the test.  Finding out why would be a good idea at this point and seeing if what he does can be incorporated by the other two operators.

For now, we will assume that R̅ = 0.342 is a good measure of the measurement process variation.  We can now determine the measurement process error, which is defined as the following:

s_pe = R̅ /d₂ = 0.342/1.693 = 0.2018

where s_pe is the measurement process error (test-retest error), represented as standard deviation and d₂ is a control chart that depends on subgroup size (1.693 for a subgroup size of 3).  We will use this value below in the calculations of the % of total variance due to the measurement process.

Operator-Part  Chart                                   

The R chart used the range from the subgroups.  Now the Operator-Part  chart is going to use the subgroup (part) average. Go back to Table 1. The three results that Chris obtained in running Part 1 are 0.29, 0.41 and 0.64.  The average of these results is 0.45.  Each subgroup average is plotted on the   chart.  The overall average is added along the control limits.  The chart is then interpreted – in a different manner than normal.  Figure 2 is the Operator-Part X chart.

Figure 2: Operator-Part  Chart

The first thing you notice about Figure 2 is that there are lots of out of control points.  This is one situation where that is good – where you want out of control points – the more the better.  Let’s see why this occurs.  The equations for the average and control limits are given below.

X̿ = S/k = 0.0433/30 = 0.001

LCL = X̿  – A₂R̅ = 0.001 – 1.023(0.342) = -0.348

UCL = X̿  + A₂R̅ = 0.001 + 1.023(0.342) = 0.351

where A₂ is a control chart constant that depend on subgroup size (1.023 for a subgroup size of 3), and LCL and UCL are the upper and lower control limits on the X chart.

Remember what R̅ is? It is a measure of the measurement process variation.  You hope that the measurement process variation is small compared to the process variance.  This leads to tight control limits and the abundance of out of control points.

The range between the LCL and UCL on the X chart. in Figure 2 represents the measurements that are masked by measurement error.  You can’t really tell the difference between the points between the control limits.  The range between the minimum part average and maximum part average on the X chart represent the process variation.  In Figure 2, this is much larger than the measurement process variation defined by the control limits.  Figure 3 shows this concept.

Figure 3: Variation Masked by Measurement Process Error

To summarize the information in Figure 2:

• The X chart shows the average value for each operator for each part.
• The control limits on the X chart are based on the average range.
• The average range is representative of measurement error.
• The X chartcontrol limits represent the variation obscured by measurement error.
• The relative utility of the measurement process increases:
  • The more out of control points there on are on the X chart.
  • The further the out of control points are away from the control limits.
• 22 out of 30 points are out of control on the X chart.

So, it looks like the measurement process is pretty good – it can tell the difference between samples as seen by all the out of control points in Figure 2.

If most or all of the points on the  chart are in control, then this means that the measurement process can not tell the difference between the points and is not very good.

Of course, many of us want to have a metric that tells us how good the measurement process is. We show how to do that below.

Determining the % Variance due to the Measurement System

The basic equation describing the relationship between the total variance, the product variance and the measurement process variance is given below.

σ_x²= σ_p²+σ_e²

where σ_x² = total variance of the product measurements, σ_p²= the variance of the product, and σ_e² = the variance of the measurement process.  The ratio of the measurement process variance to the total variance is given by:

Ratio of measurement process variance to the total variance = σ_e²/ σ_x²

To determine the % of total variance due to the measurement process, we need an estimate of the total variance.  As mentioned above, you can take several months of process data, construct a control chart and use the range chart to estimate the total variance.   You can also use the parts in the study to determine the total variance.  We will do that here.

The variance of the measurement process is composed of two parts: repeatability and reproducibility.  The repeatability is given by s_pe².     We calculated s_pe above from the range chart. The reproducibility (s_o²) can be calculated using the following:

s_o² = s_o² – (o/(nop))s_pe²

where s_o² = variance of the operator averages, o = number of operators, n = number of trials and p = number of parts.  The operator averages are:

• Chris: 0.1903
• John: 0.0683
• Mary: -0.2543

The variance of the operator averages is 0.05277.  The reproducibility is then given by:

s_o² = s_o² – (o/(nop))s_pe² = 0.0527-(3/90)(0.2018²) = 0.0513

The Gage R&R is then:

Gage R&R = s_o² + s_pe² = 0.0513 + 0.0407 = 0.092

The product variance given by:

s_p² = s_p² – (p/(nop))s_pe²

where s_p² = variance of part averages.

The part averages are given by:

• Part 1: 0.1689
• Part 2: -0.8511
• Part 3: 1.0989
• Part 4: 0.3667
• Part 5: -1.0644
• Part 6: -0.1856
• Part 7: 0.4544
• Part 8: -0.3422
• Part 9: 1.9400
• Part 10: -1.5711

The part variance is 1.091.  The product variance is then:

s_p² = s_p² – (p/(n o p))s_pe² = 1.091 – (10/90)(0.2018²) = 1.086

The total variance is then:

σ_x²= σ_p² + σ_e² = 1.086 + 0.092 = 1.178

And finally, the % of total variance due to the measurement process is:

% of total variance due to the measurement process = σ_e²/ σ_x² = 0.092/1.178 = 7.8%

The measurement process causes 7.8% of the total variance.  Based on this number alone, it is a good measurement process.  But you already had a pretty good idea that the measurement process was good from Figure 2.   Most of the points were out of control meaning that the measurement process can tell the difference between samples.

Past SPC Knowledge Base Articles on Measurement Systems Analysis

Ever since January 2004, we have been publishing monthly articles on a variety of SPC and statistical techniques.  We have 34 articles on the Measurement System Analysis. These are listed below.  You can access these articles at this SPC Knowledge Base link.

• Operational Definitions/Measurement System Analysis (March 2004 )
• Monitoring Test Methods Using SPC (May 2006 )
• Variable Measurement Systems – Part 1: Stability (September 2007 )
• Variable Measurement Systems – Part 2: Bias (October 2007 )
• Variable Measurement Systems – Part 3: Linearity (November 2007 )
• Variable Measurement Systems – Part 4: Gage R&R (December 2007 )
• Measurement Systems – Is Yours Any Good? (November 2009 )
• Attribute Gage R&R Studies: Comparing Appraisers (May 2010 )
• Attribute Gage R&R Studies – Part 2 (June 2010 )
• Levey-Jennings Charts (August 2010 )
• ANOVA Gage R&R – Part 1 (August 2012 )
• ANOVA Gage R&R – Part 2 (September 2012 )
• ANOVA Gage R&R – Part 3 (October 2012 )
• Gage R&R for Non-Destructive and Destructive Test Methods (September 2013 )
• Destructive Gage R&R Analysis (October 2013 )
• Evaluating the Measurement Process – Part 1 (December 2014 )
• Evaluating the Measurement Process – Part 2 (January 2015 )
• Three Methods to Analyze Gage R&R Studies (February 2015 )
• Monitoring Destructive Test Methods (November 2015 )
• Probable Error and Your Measurement System (April 2016 )
• Specifications and Measurement Error (May 2016 )
• SPC and the Lab (November 2017 )
• The Impact of Measurement Error on Control Charts (January 2018 )
• Acceptance Criteria for Measurement Systems Analysis (MSA) (April 2018 )
• Operational Definition of a Consistent Measurement System (May 2018 )
• The Short EMP Study (June 2019 )
• Evaluating the Measurement Process (EMP) Overview (July 2019 )
• Five Common Mistakes with Gage R&R Studies (November 2019 )
• Operator-Part Interaction in Gage R&R Studies (December 2019 )
• A Tale of Two Precision to Tolerance Ratios (March 2020 )
• How Does Measurement Variation Impact Control Chart Signals? (November 2020 )
• The Calculations Behind a Gage Linearity Study (February 2021 )
• Individuals Control Charts and Levey-Jennings Charts (December 2021 )
• How to Monitor Measurement Systems Over Time (May 2023 )

Summary

This publication showed how the operator-part control chart in Gage R&R studies gives you insights into how good your measurement process is.  The range chart is used to ensure that the operator’s repeatability is consistent (in control). The average range from that chart is a measure of the measurement process error.  The X chart plots each operators subgroup averages.  With this type of chart, you want out of control points – the more the better.  The control limits on the X chart show the range of values that are masked by the measurement error.  This publication also showed how to calculate the %  of the total variance caused by the measurement process.