The Calculations Behind a Gage Linearity Study
(Note: all the previous publications in the measurement systems analysis category are listed on the right-hand side. Select "Return to Categories" to go to the page with all publications sorted by category. Select this link for information on the SPC for Excel software.)
If you use your measurement system over a broad range of values, you may have to be concerned about the bias and linearity of the measurement system. The calculations behind a gage linearity study are fairly straightforward. Software packages, like SPC for Excel, handle these calculations easily. This publication shows how the calculations are done to help you further understand the output from a gage linearity study.
In this issue:
- Definitions of Bias and Linearity
- Example Data Set
- Calculate the Bias
- Calculate the Average Bias per Part
- Calculate the Best Fit Line Between Reference Values and Biases
- Determine if the Slope is Significant
- Calculate the Confidence Interval Around the Best-Fit Line
- Create the Linearity Chart
- Quick Links
You may download a pdf version of this publication at this link. Please feel free to leave a comment at the end of the publication.
Definitions of Bias and Linearity
We will use the Measurement Systems Analysis manual from the Automotive Industry Action Group (AIAG) to define bias and linearity. From the fourth edition of the manual:
Bias: The difference between the observed average of measurements (trials under repeatability conditions) and a reference value; historically referred to as accuracy. Bias is evaluated and expressed at a single point within the operating range of the measurement system.
Linearity: The difference in bias errors over the expected operating range of a measurement system. In other terms, linearity expresses the correlation of multiple and independent bias errors over the operating range.
Bias is how well your measurements compare to a reference value while linearity is assessing how accurate the measurements are over the operating range of the measurement system.
There have been two earlier SPC Knowledge Base articles on bias and linearity from 2007.
This SPC Knowledge Base article updates some of the prior information with an increased focus on the calculations.
Example Data Set
A new measurement system will be used to monitor production for some quality characteristic. Five parts are selected from the process that represent the range of the quality characteristic in production. The reference values of each part are determined. Then one operator measures each part ten times. The data are shown in Table 1.
Table 1: Example Data
We will set g = the number of parts and m = the number of replications. We will use "i" to denote the part number and "j" to denote the replication. We will walk through the calculations beginning with bias.
Calculate the Bias
The first step is to calculate the bias for each individual reading. The bias for an individual reading is given by:
Bij = xij – RVi
where Bij is the bias for the jth replication for part i, xij is the measurement value for the jth replication for part i, and RVi is the reference value for part i.
The reference value for part 1 is given by RV1, the reference value for part 2 is given by RV2, and so on.
The bias for the first point in Table 1 is then the difference between the measurement value for the first replication (7.7) and the reference value for part 1 (7).
B11 = x11 – RV1 = 7.7 - 7 = 0.7
Table 2 shows all the bias calculations.
Table 2: Bias Calculations
Calculate the Average Bias per Part
The next step is to calculate the average bias for each part. The average bias for part i is given by:
where the summation is over the number of replications for part i. The averages for all 5 parts are also shown in Table 2.
With these calculations, you can now start the linearity chart. At this point, you can plot the individual bias results and the average bias for each part against the reference values. The resulting chart is shown in Figure 1.
Figure 1: Start of Linearity Chart
This chart shows that there is evidence of linearity in the measurement system. The bias is changing as the reference value increases.
Calculate the Best Fit Line Between Reference Values and Biases
The next step is to add the best-fit line between the reference values and the individual bias. So, you want to generate a regression line for the following data in Table 3:
Table 3: Regression Data
The reference values represent the x values and the average biases represent the y values in regression lingo. The equation being fitted then is:
y = bo + b1x
B = bo + b1(RV)
where b0 = y intercept and b1 = slope. You can use Excel's built in regression to determine the values of b0 and b1. You can also use the equations below to calculate the values.
The following values can be used in the equations above:
- ΣRV*B) =- 77
- ΣRV = 550
- ΣB = -2.2
- Σ(RV2) = 6450
- g = 5
- m = 10
- Average Bias = -0.044
- Average RV = 11
The values of bo and b1 are then:
bo = 1.408
b1 = -0.132
Determine if the Slope is Significant
If there is no linearity in the bias results, i.e., the bias is the same at all reference values, you would expect b1 to be zero. However, because of variation, you will rarely get b1 = 0. The question you have to answer is:
Is the slope (b1) significantly different from 0?
You can do this in a number of ways. One is to calculate the t-statistic and determine the p-value associated with that t-statistic.
The t-statistic is given by:
where s is the standard deviation, which is given by:
The only new term in the equation for s above is Σ(B2) which is 10.14. Putting the values into the above equation gives:
You can calculate the t-statistic. The sum of the RV – average RV for all measurements is 400 so the t-statistic is given by:
The question you ask now is what is the probability you would get this t-statistic if the slope (linearity) is really 0? To find that, we can make use of the TDIST function in Excel. If you enter the following function TINV(|10.43|, 48, 2), it will give you the probability of getting a t value of 10.43 with 48 degrees of freedom when the slope is really zero. If this probability is less than 0.05, you will conclude the probability is small, so the slope must not be zero. In this example, the probability is very small (6.2E-14) so we conclude that the slope is not zero and the bias changes over the operating range of the measurement system.
Calculate the Confidence Interval Around the Best-Fit Line
The calculations now turn to the confidence limits around the regression line. The equations for calculating the lower and upper confidence bands are given below (for 95% confidence):
where RV0 is a given reference value, e.g., 7. You will need to select several reference values over the range to create the line. All the variables in the above equations are defined given above. Using RV0 = 7, gives the following results:
Lower Confidence Bound = 0.3594
Upper Confidence Bound = 0.60863
Create the Linearity Chart
You can now add three lines to the linearity chart: the best-fit line and the two confidence bounds. This is shown in Figure 2
Figure 2: Linearity Chart
A fourth line where bias = 0 has also been added. The linearity chart presents clear picture of what is happening over the operating range for the measurement system. It is clear that the bias is changing. It is decreasing as the reference value goes up.
This publication showed the major calculations involved in a gage linearity study. The purpose of the study is to determine if the bias changes over the operating range of the measurement system. You can download an example of the linearity gage output from SPC for Excel at this link. This output gives more information, for example, which bias values are significantly different than zero.
Thanks so much for reading our publication. We hope you find it informative and useful. Happy charting and may the data always support your position.
Dr. Bill McNeese
BPI Consulting, LLC
Connect with Us
Measurement Systems Analysis
- << Return to Categories
- A Tale of Two Precision to Tolerance Ratios
- ANOVA Gage R&R - Part 1
- ANOVA Gage R&R - Part 2
- ANOVA Gage R&R - Part 3
- Acceptance Criteria for Measurement Systems Analysis (MSA)
- Attribute Gage R&R Studies - Part 2
- Attribute Gage R&R Studies: Comparing Appraisers
- Destructive Gage R&R Analysis
- Evaluating the Measurement Process (EMP) Overview
- Evaluating the Measurement Process - Part 1
- Evaluating the Measurement Process - Part 2
- Five Common Mistakes with Gage R&R Studies
- Gage R&R for Non-Destructive and Destructive Test Methods
- How Does Measurement Variation Impact Control Chart Signals?
- Individuals Control Charts and Levey- Jennings Charts
- Levey-Jennings Charts
- Measurement Systems - Is Yours Any Good?
- Monitoring Destructive Test Methods
- Monitoring Test Methods Using SPC
- Operational Definition of a Consistent Measurement System
- Operational Definitions/Measurement System Analysis
- Operator-Part Interaction in Gage R&R Studies
- Probable Error and Your Measurement System
- SPC and the Lab
- Specifications and Measurement Error
- The Calculations Behind a Gage Linearity Study
- The Impact of Measurement Error on Control Charts
- The Short EMP Study
- Three Methods to Analyze Gage R&R Studies
- Variable Measurement Systems - Part 1: Stability
- Variable Measurement Systems - Part 2: Bias
- Variable Measurement Systems - Part 3: Linearity
- Variable Measurement Systems - Part 4: Gage R&R
SPC Knowledge Base
Click here to see what our customers say about SPC for Excel!
SPC Around the World
SPC for Excel is used in over 66 countries internationally. Click here for a list of those countries.