Specifications and Measurement Error

May 2016

(Note: all the previous publications in the measurement systems analysis category are listed on the right-hand side. Select “Publications” to go to the SPC Knowledge Base homepage. Selectthis linkfor information on the SPC for Excel software.)

Specifications. Is your product in spec? Is your product out of spec? And how do you know?

Customers have specifications and they want product that meets their specifications. Of course, you sample and test the product to ensure it meets those customer specifications. This is one way measurement systems are used – to determine if the product is within specifications.

Suppose one customer’s specification range for your product is 95 to 105. Recent test results have been great: 98.7, 102.4, 100.3, 99.7. Everything is meeting specifications. Now, you get a result of 95. Right on the lower specification limit. What do you do? Accept the sample result as being within specification? Well, it does meet the specification limit. Why not accept it? A little worried about doing that perhaps? So you retest the sample. The sample result comes back 95.7. Definitely, within specifications!

Many organizations do this – compare the test result directly to the customer specifications. The problem is that this approach does not take into account the uncertainty associated with the test method. This month’s publication takes a look at a method for creating manufacturing specifications by adjusting customer specifications to account for the measurement uncertainty – so that when the product is within these manufacturing specifications you can be “sure” that it is within the customer’s specification. Thanks to variation, there is a probability associated with the “sure.”

In this issue:

• Specifications and Measurement
• Setting Manufacturing Specifications Process
  • Determine Measurement System Error
  • Determine the Measurement Increment
  • Determine the Watershed Specification Limits
  • Determine the Manufacturing Specifications
• Impact on Internal Process Capability
• Workbook to Determine Manufacturing Specifications
• Summary
• Quick Links

We also created a workbook that you can download for determining manufacturing specifications using the process laid out in this publication. Please feel free to leave a comment at the end of this publication. You can download a pdf file of this publication from our e-reader at this link.

Specifications and Measurement

The website businessdictionary.com has the following definition for a specification:

“Exact statement of the particular needs to be satisfied, or essential characteristics that a customer requires (in a good, material, method, process, service, system, or work) and which a vendor must deliver. Specifications are written usually in a manner that enables both parties (and/or an independent certifier) to measure the degree of conformance.”

The definition above essentially says that all is going great as long as the product is within specifications. The customer is happy – as are we. Things are not going so great when the product is out of specifications – the customer is not happy, nor are we. This is shown in Figure 1, where LSL = lower specification limit and USL = upper specification limit.

Figure 1: The Specification View of the World

Take a look back at the definition of specifications. See the part about “to measure”? We have to be able to measure whether or not the product (or whatever is being provided) meets the specifications. Specifications are set – for example, 95 to 105. Hard numbers. And, far too often, we assume that the test result we receive is a hard number also – the true value. So, we make direct comparisons between the test result and the specifications.

Take a look at Figure 2. The circles with numbers represent test results from our process. Let’s see how we respond for each result.

Figure 2: Specifications and Measurements

• Result 1: Everyone is happy. The test result is well within specifications.
• Result 2: We are still happy. The test result is not centered; it is a little closer to the USL, but it is well within specifications.
• Result 3: We are still happy. Again, the test result is not centered; it is a little closer to the LSL, but within those specifications.
• Result 4: We are not happy. The result is above the USL. We have to do something – either rework or scrap.
• Result 5: We are not happy again. This time the result is below the lower specification limit. Rework or scrap again most likely
• Result 6: We are happy. Maybe a little nervous though. But the result is within specifications. It is close, but it is within specifications.
• Result 7: We are not happy. The test result is just below the LSL. But maybe there is hope. Retest the sample to see if it comes back within specifications! Or ship it anyway because we know the customer will take it.

Take a look at samples 6 and 7. Is there really a difference between those two results? Probably not – but we react differently. We don’t even think about retesting sample 6 most of the time because it is within specifications. But we do think about retesting sample 7 because it is just out of specifications. We just acknowledged that the measurement system has variation – but we ignore that when it is convenient.

What is really needed is a set of manufacturing specifications that take into account measurement variation – the fact that you don’t get the same result each time you test the same part. Dr. Donald Wheeler outlines a method of adjusting the customer specifications to account for this measurement uncertainty in this book: EMP III Evaluating the Measurement Process & Using Imperfect Data, www.spcpress.com. This methodology is described below.

Setting Manufacturing Specifications Process

You have your customer specifications. You want to be sure that everything you ship to the customer is within those specifications. To do that, you have to take into account measurement error. The measurement error is used to create manufacturing specifications. These manufacturing specifications are, of course, tighter than the customer specifications. The tighter the manufacturing specifications, the more likely you are to be in conformance with the customer specifications. The process for determining manufacturing specifications is given in the steps below.

• Determine the measurement system error
• Determine your measurement increment (resolution)
• Determine the Watershed Specification Limits
• Determine the manufacturing specifications based on the probability you can live with

Each step is described below.

Determine Measurement System Error

One way to define measurement system is to use the Probable Error (PE). The Probable Error is defined as the following:

PE = 0.675σ_ms

where σ_msis the precision of the test method. If you continually re-measure the same part, half of the repeated measurements should fall between the average and ± one PE.

Last month’s publication described how to determine the Probable Error for a measurement system. The example below demonstrates how to find the PE.

Based on the equation above, we need to find σ_msto find the Probable Error. This is easily found by using repeated measurements of the same sample – and using a moving range chart. Suppose one operator measures the same sample 20 times. The results are given in Table 1 along with the moving range (mR) values.

Table 1: Repeated Measurements of Same Sample

The results in Table 1 were analyzed using a moving range chart as shown below.

Figure 3: Moving Range Chart for Repeated Measurements of the Same Sample

The moving range chart plots the moving range between consecutive points. There are two purposes to this control chart. One is to determine if the moving range between consecutive points is consistent over time. As long as there are no out of control points, this moving range is consistent (in statistical control). This is true for the chart in Figure 3. If you would like more information on how the moving range is constructed, please see our individuals control chart publication.

The other purpose is to estimate the standard deviation of the measurement system (the precision). You can do this as long as the moving range control chart is in statistical control. The standard deviation for the measurement system is given by:

σ_ms=R/1.128

The average range from Figure 3 is 0.35, so σ_msis given by:

σ_ms=R/1.128 =0.35/1.128=0.31

The Probable Error is then given by

PE = 0.675σ_ms = 0.675(0.31) = 0.21

Thus, 50% of the retested results will lie within ± 1 PE of the part’s average.

Determine the Measurement Increment

The measurement increment is simply the measurement resolution. What are you measuring the results to? In Table 1, the measurements are to 0.1. This is the measurement increment. Last month’s publication used the PE to determine if your measurement increment is satisfactory. For our purposes here, we will assume that the measurement increment is satisfactory.

Determine the Watershed Specification Limits

Suppose your customer specifications for your process are LSL = 95 to USL = 105. Your measurement increment is 0.1 as shown in Table 1. A test result of 95 is in spec, but a test result of 94.9 is not in spec. The Lower Watershed Specification Limit (LWSL) is defined as being halfway between the LSL and one measurement increment below LSL:

LWSL = LSL – Measurement Increment/2 = 95 – 0.1/2 = 94.95.

The Upper Watershed Specification Limit (UWSL) is defined similarly:

UWSL = USL + Measurement Increment/2 = 105 + 0.1/2 = 105.05

These Watershed Specifications are essentially adjusting the customer specifications to account for the measurement increment of our test method. We will use the Watershed Specification Limits to set the manufacturing specifications.

Determine the Manufacturing Specifications

We used the measurement increment to define the Watershed Specification Limits. Now we will use Probable Error to define our manufacturing specifications.

Suppose we decide to set our manufacturing specifications equal to the Watershed Specification Limits. Now, suppose that a test result falls within one PE of the Watershed Specification Limits as shown in Figure 4.

Figure 4: 64% Manufacturing Specifications

Dr. Wheeler shows in his book that there is at least a 64% chance that the part is within specifications. Dr. Wheeler phrases it this way: “If you use your Watershed Specification Limits as your Manufacturing Specifications, you will have at least a 64% chance that the measured item is in spec when the measurement falls within the Manufacturing Specifications.

In our example with customer specifications of 95 to 105 and a probable error of 0.21, the 64% manufacturing specifications are given by the Watershed Specification Limits of 94.95 to 105.05. If a test result is within these values, you have at least a 64% of the part meeting the specifications.

To increase this % chance of meeting specifications, we have to tighten the manufacturing specifications further. We can do this in multiples of the Probable Error. We can decide to tighten the specifications by one PE on each side. This generates 85% manufacturing specifications. Table 2 shows how to set the manufacturing specifications to obtain up to 99.9% manufacturing specifications.

Table 2: Manufacturing Specifications

Figure 5 shows the relationship between these manufacturing specs, the Watershed Specification Limits and the Probable Error.

Figure 5: Manufacturing Specifications

Of course, tightening manufacturing specifications involve a trade-off between “being sure” that you are meeting customer specifications and making more “off spec” product. You will have to decide where this trade-off makes sense for you.

Impact on Internal Process Capability

Your process capability is based on your customer specifications. But, if you decide to use manufacturing specifications – which are tighter than your customer specifications – you may want to track your internal process capability.

Let’s stay with example where our customer specifications are LSL = 95 and USL = 105. Suppose we know that our process average is 100.5 and our process sigma is 2.0. We can calculate the process capability as following:

Cpk = Minimum (Cpu, Cpl)

Cpu = (USL-X)/3σ =(105-100.5)/(3(2))=0.75

Cpu = (X-LSL)3σ = (100.5-95)/(3(2))=0.92

where

• Cpu is the process capability based on the lower specification
• Cpl is the process capability based on the upper specification
• X is the overall process average
• σ is the process sigma

Since Cpu is less than 1, there is material out of spec on the high side. Likewise, for Cpl. The % of out of specifications can be determined. In Excel, you can use the NORMSDIST(Z) function to find the % out of specification where Z = 3Cpu or Z = 3Cpl.

% above USL: 1.22%
% below LSL: 0.30%

The total out of spec is 1.52%. Now suppose we decide to tighten our internal specifications so that we have 96% manufacturing specifications. From Table 2, the manufacturing specifications are given by:

Lower Manufacturing Specification (LMSL) = LWSL + 2(PE) = 95.95 + 2(0.21) = 95.37

Upper Manufacturing Specification (UMSL) = UWSL – 2(PE) = 105.05 – 2(0.21) = 104.63

You can now calculate the internal process capability as follows:

Cpu = (UMSL-X)/3σ =(104.63-100.5)/(3(2))=0.6

Cpu = (X-LMSL)3σ = (100.5-95.37)/(3(2))=0.85

Now you calculate the % out of spec for the manufacturing specifications.

% above USL: 1.95%

% below LSL: 0.52%

The total out of spec is now 2.47%. So, tightening to 96% manufacturing specification limits increased the out of spec from 1.52% to 2.47%.

Please note that is just one example. What happens in your process depends on the process average and sigma, the measurement increment, and the Probable Error. It may that you don’t increase out of specification material that much or you may increase it a lot more. You have to do the calculations to determine the impact.

Workbook to Determine Manufacturing Specifications

We have created an Excel workbook to help you perform these calculations. You can download it at this link. The workbook requires you to enter the following information:

• LSL
• Nominal
• USL
• Process Sigma
• Process Average
• Measurement Increment
• Measurement System Standard Deviation

It will then calculate the various manufacturing specifications and determine the impact on process capability and % out of spec.

Using the values in this publication, the workbook creates the following tables:

Table 3: Workbook Output for Manufacturing Specifications

Table 4: Workbook Output for Impact on Process Capability

Table 3 lists the various options for the manufacturing specifications. Table 4 shows the impact on process capability and % out of specifications. The workbook also generates a figure that shows how the spec limits change, as shown in Figure 6.

Figure 6: Changing Specifications

We hope you find the workbook useful.

Summary

This publication has examined how to set manufacturing specifications to help ensure that product meets the customer’s specifications. The process involves determining the measurement system variation (as defined by the Probable Error), determining the measurement increment, determining the Watershed Specification Limits, and then setting the manufacturing specifications. The impact on process capability was also examined.