Automotive Core Tools

MSA for Manufacturing Excellence

01 Introduction
 Course Contents

Applications of measurement data Types of variation in MS

Quality of measurement data Measurement Uncertainty
What is MSA ? Ideal & Good MS
What is Measurement system ? SWIPE Model
Why MSA ? Preparation for study
Effects of MS variation on various Decisions on Broad Guidelines for each study
Product/Process Overview of Attribute studies
Total variation & Measurement system variation-
Relationship
02 MSA
Applications of measurement data

• Adjust Manufacturing process

• Establish relationship between 2 or more
variables-Regression analysis
• Sort parts/Assemblies-Pass/Fail
A measurement system is a process

Input Operation Output

Output = a decision on a product, process, or service

– via a number assigned to a characteristic of a product, process, or service
The first step in assessing a system is to understand this process and determine if it
will satisfy our requirements
• From this definition, it follows that a measurement process may be viewed as a
manufacturing process that produces decisions via numbers (data) for its output.

• Viewing a measurement system this way is useful because it allows us to bring to

bear all the concepts, philosophy and tools that have already demonstrated their
usefulness in the area of statistical process control.
 OK/Not ok, Control/not in control/
correlation/regression, Root cause on
Quality of
DECISION warranty, Incentive/performance pay, Decision
Whether house keeping needs
improvement?
Depends on

DATA 10.55mm or No-Go gauge passing, Quality of

values plotted on control chart, R-
value, DOE, KRA score, 5S score (%)
Data
Depends on

Instruments, People, Gauges, Quality of

MEASUEMENT Equipment, Environment, audit MS
SYSTEM Check/Validate

MSA
 Quality of Measurement data

• Statistical properties of multiple

measurements obtained from a
measurement system operating under
stable conditions
• Close or far away?
• Bias-Location
• Variance-Spread
 What is MSA?

• Measurement System analysis

• Systematic analytic study of
measurement systems
• Mandatory as per IATF 16949 for all
Measurement systems identified in
control plan.
IATF 16949:2016 Requirements
Exercise
• Identify at least 3 Measurement systems from control plan
• IATF 16949 requires MSA study only for Important characteristics?
True/False?
• It is mandatory to conduct MSA studies as per AIAG Manual? True/False?
 Calibration MSA
• Conducted in Controlled • Conducted in actual
environment working condition
• Conducted using • Conducted using actual
masters products/parts
• Conducted by actual users
• Conducted by qualified / (operators/inspectors)
trained people
• Checks accuracy and
• Checks accuracy only precision

Can calibration replace MSA?

Exercise
• Explain at least 3 reasons why MSA is necessary although Equipment is
calibrated
 INPUT ACTIVITY OUTPUT
Control Plans, List of
instruments, new product Preparation / Updating of MSA plan

MSA Manual, Competency Identification of statistical studies to

on concept of MS and MSA Plan
meet the requirement of the
Statistical studies
measurement and decide the frequency

MSA Plan, WI & MSA

Study Report
manual Perform MSA studies

YES Prepare summary report

Is it & submit to customer if
acceptable? required

NO
Analyze the data using graphical tools/brain
storming & find out causes for variation

Initiate & implement corrective action

 What is Measurement System

Collection of instruments or gauges,

standards, Operations, methods,
fixtures, software, personnel,
environment & assumptions used to
quantify a unit of measure or fix
assessment.
 Sources of Variation (SWIPE)

• Standard
• Work piece
• Instrument
• Person/Procedure
• Environment
Exercise
• MSA includes-
1. Measuring equipment
2. Person doing measurements
3. Product to be checked
4. Only 1 & 2 above
 Observed Process Variation

Actual Process Variation Measurement Variation

Long-term
Process Variation
Short-term
Process Variation
Variation
w/i sample
Variation due
to instrument
Variation due
to appraisers
Possible Sources
of Variation
Repeatability Bias Stability Linearity Reproducibility

We look at “repeatability” and “reproducibility” as these are the

usually primary contributors to measurement error.

20
 Exercise

1. If measurement system variation increases what is impact on observed variation?

2. If Observed variation is more then what we should do first a)Reduce process variation
b)Reduce measurement system variation
3. What is difference between Actual process variation & Observed process variation?
 Effect on Product Decisions

• Type I error (Producer Risk or false alarm)

Good parts will sometimes be called bad

• Type II error (Consumer Risk)

Bad Parts called as good

• Error Rate: Type I + Type II

Exercise
• Why Type 1 error is called producer Risk?
• Why Type 2 error is called as consumer Risk?
• When you try to reduce Type I error is there a possibility that Type II error
may increase or Vice a Versa ?
Effect on Product
Decisions
Effect on Product Decisions

TWO Choices
• Improve Production process
:No parts in Areas II
• Improve measurement
system: Reduce size of area II
 Effect on Process Decisions

• Calling common cause, a special

cause
• Calling a special cause, a common
cause
• Observed CP/CPK could be different
than actual CP/CPK
Impact of MS Variation on CP/Cpk

• Cp=USL-LSL/6 Sigma (Observed)

• Sigma Sq Observed = Sigma Sq Actual + Sigma Sq MSA
Exercise
• Where does Type I error & Type II errors Occur – a)Near USL b)Near LSL
• Can MSA & SPC help to reduce Type I & II error?
• How does MSA impact SPC?
 Discrimination

• Smallest readable unit, measurement resolution

• Inherent property fixed by design
• Smallest scale unit of measure
• 10 to 1 thumb rule
Exercise
• What should be least count of weighting balance for checking weight of
part. Tolerance on weight is + 1 Gm for one product & + 2 Gm for another
product
1. 1 Gm
2. 0.5 Gm
3. 0.2 Gm
4. 0.1 Gm
 Effective Resolution

• Sensitivity of measurement system

to process variation for a particular
application.
• Smallest input that results in a
output signal of measurement
 Reference Value

• Accepted value of an artifact

• Used as the surrogate for True value
 True Value

• Actual Value of Artifact

• Unknown & unknowable
Exercise
• You have a master weight who’s calibration certificate shows that the value
of master in 10 gm. Can we say it’s actual value of Master ?
• If you use an very accurate measuring equipment then you can find actual
value of a master/product. True/False?
+

Inaccurate and Imprecise

+

Accurate and Imprecise

+

Precise but inaccurate

+

Accurate and Precise

38
 What’s Important ?

1.Accuracy
2.Precision
3.Both

21 March 2024
 Types of variation
Location(Accuracy) Width(Precision)
Bias Repeatability

Stability Reproducibility

Linearity GRR
Measurement System Errors - Location Variation
 How to Conduct Bias Study?

• Take a Master of known reference value

• Measure the master 15 times
• In Actual work environment
• With One Inspector who does normally does measurement
• Master value is not known to Inspector
 How to Interpret Bias Study

• Bias Study is acceptable if zero lies between upper & lower control limits
calculated by software
• In other words, One control limit should be positive & other negative
Exercise
• Can we conduct Bias study with out having master?
• Case study 1-Is Bias acceptable?
• Case study 2-Is Bias acceptable?
• Case study 3-Is Bias acceptable?
 Some of the reasons for unacceptable Bias?

• Ambient Temperature
• Zero setting
• Calibration
• Maintenance of Instrument
• Method of measurement
• Assumptions
• Wrong Master used
 How to correct Bias?
• Define Measurement method
• Maintain temperature
• Zero setting before use
• Increase calibration frequency
• Check & correct Calibration error
• Carryout Maintenance of Instrument
• Use correct master with correct value
 Measurement System Errors - Location Variation

Linearity
•The change in bias over the normal
operating range
•The correlation of multiple and
independent bias errors over the
operating range
•A systematic error component of the
measurement system

Size n Size m
 How to conduct Linearity Study?

• Take 5 masters of Known Reference Value covering entire operating range of

Measurement system
• Check each Master in normal working environment 15 times
• With single operator
• With Same Instrument
• Master Values are not known to Inspectors
 How to Interpret Linearity Study?

• The Linearity Study is acceptable if Bias =0 line lies in confidence limits of

best fit line
• In other words Bias=0 line should not cross confidence limits
Exercise
• Can we conduct Linearity study with out having master?
• Case study 1-Is Linearity acceptable?
• Case study 2-Is Linearity acceptable?
 Some of the reasons for unacceptable Linearity?

• Ambient Temperature
• Zero setting
• Calibration
• Maintenance of Instrument
• Method of measurement
• Assumptions
• Wrong Master used
 How to correct Linearity?

• Define Measurement method

• Maintain temperature
• Zero setting before use
• Increase calibration frequency
• Check & correct Calibration error
• Carryout Maintenance of Instrument
• Use correct master with correct value
 Measurement System Errors - Location Variation

Stability
• The change in bias over time
• A stable measurement process is
in statistical control with respect to
location
• Alias: Drift
 How to conduct Stability Study?

• Take one master /Product

• Take 5 readings at different times
• With same Instrument
• With Same Inspector
• Master Value is not known to Inspector
 How to Interpret Stability Study?

• Look at R Chart & see any points outside control limits

• Look at any trends
• Look at X chart & see any points outside control limits
• Look at any Trends
• If no point is outside control limit & there is no trend then Stability is
acceptable
What are
Trends?
Exercise
• Can we conduct Stability study with out having master?
• Case study 1-Is Stability acceptable?
• Case study 2-Is Stability acceptable?
 Some Reasons for Instability

• Environmental Conditions
• Instrument Design
• Maintenance of Instrument
• Calibration frequency
• Wear & Tear of Instrument/Master
 How to correct Instability?

• Maintain Environmental conditions

• Better Instrument Design
• Frequent Maintenance of Instrument
• Changing wear Parts
• Calibrating masters frequently
Measurement System Errors - Width variation-Repeatability

Repeatability

60
 Measurement System Errors - Width variation

Reproducibility
•Variation in the average of the
measurements made by different
appraisers using the same gage
when measuring a characteristic on
one part. Commonly referred to as
A.V. – Appraiser Variation
•Between-system (conditions)
variation

61
 Measurement System Errors - Width variation

GRR or Gage R&R

Gage repeatability and

reproducibility: the combined
estimate of measurement
system repeatability and
reproducibility.

62
 How to Conduct R & R Study?

• Take 10 products covering entire process range

• Products are numbered
• Select 3 Inspectors who normally does measurements
• Products are offered in random order to inspector. Sr No
can not be seen by Inspectors
• Each Inspector checks each product 3 times
 MEASUREMENT SYSTEM ANALYSIS (VARIABLE GAUGE STUDY)
Part Number : Gauge Name : Report :
Part Name : Gauge No. : Date :
Characteristics: Gauge Type : Page :
Specification : Performed By : Unit :
GAUGE REPEATABILITY AND REPRODUCIBILITY DATA SHEET
PART
APPRAISER Trial AVERAGE
1 2 3 4 5 6 7 8 9 10
1
A 2
3
GRR Study separates
Average
Range
Xa =
Ra =
part variation,
B
1
2 Equipment variation,
Appraiser variation
3
Average Xb =
Range Rb =

BUT HOW?
1
C 2
3
Average Xc =
Range Rc =
Xp =
Part Average (Xp)
Rp =

Average of Range = [ Ra + Rb + Rc / No.of Appraiser ] => R =

Difference of Average = [ Max. X - Min. X ] => X DIFF =

UCLr= R  D4
*D4 = 3.27 for 2 trials and 2.58 for 3 trials . UCLr repres ents the lim it of individual R’s. Circle those that are beyond this limit.
Identify the cause and correct. Repeat these readings using the s ame appraiser and unit as originally used or dis card valu
 GRR Study
separates part
variation,
GRR Data Equipment
Collection variation,
Sheet Appraiser
variation
BUT HOW?
 GAUGE REPEATABILITY AND REPRODUCIBILITY REPORT
From Data
Sheet R = Xdiff = Rp=
Measurement Unit Analysis % Total Variation (TV)
Repeatability = Equipment Variation (EV) % Repeatability

EV = R X K1 % EV = 100 [ EV / TV ]
Trails K1
= =
2
3
0.8862
0.5908 You can Calculate :
Reproducibility = Appraiser Variation (AV) % Reproducibility

% EV, AV, R&R, PV based on

AV 2 2 % AV 100 [ AV / TV ]
= [(X DIFF X K2) -(EV /nr)] =

GRR Calculation
= = = %
n = Number of Parts Appraisers 2 3

Sheet
r = Number of Trials K2 0.7071 0.5231
1. TV (from study)
Repeatability & Reproducibility (R & R) % Repeatability & Reproducibility
2 2

OR
GR & R = (EV + AV ) %R & R = 100 [ R&R / TV ]
= = %

Parts K3
Part Variation (PV)
PV = Rp X K3
2 0.7071
% Part Variation
% PV = 100 [ PV / TV ]
2. Tolerance
3 0.5231
4 0.4467
= = %
5 0.4030
6 0.3742
Total Variation
7 0.3534 Number of distinct Data categories (d)
8 0.3375
TV = GRR2 + PV2 ndc = 1.41 [ PV / R&R ]
9 0.3249
= 10 0.3146
= Data Categories

Results :
Study Performed By : Verified By :
 You can Calculate :
% EV, AV, R&R, PV based on
GRR Calculation
Sheet 1. TV (from study)
OR
2. Tolerance
 How to Interpret R & R Study-Tolerance Method?

• Used when MS is used only for pass/Fail not SPC

• GRR % =GRR/(Tol/6)*100
• R & R should be less than 10 %
• R & R between 10 to 30 % conditionally acceptable
• R & R above 30 % not acceptable
 How to Interpret R & R -TV Method?

• Used when MS is used for SPC

• GRR % = (GRR/TV)*100
• R & R should be less than 10 %
• R & R between 10 to 30 % conditionally acceptable
• R & R above 30 % not acceptable
• ndc should be greater than 5
• R Chart-No point in should be outside control Limits
• X Chart -Minimum 50 % points should be outside control limits
Range Chart Example
Purpose of range chart is identify whether measurement process is under control (free from special cause)
Condition Interpretation Action
One or more point of any There was a special cause Remove these readings and take same
appraiser out of UCL R while taking reading reading again from same part & appraiser
and recalculate

More than one point of only His method is different from Remove these readings. Train appraiser on
R- chart
one appraiser out of UCL others method of measurement & take readings interpretation
again

One or more than one point of Measurement System is Check why measurement is so sensitive stop
all appraisers out of UCL sensitive to appraisers skill further studies before taking action on
sensitivity

In one part all appraisers points Part is deformed or Damaged Remove all the reading for particular part and
are out of UCL R recalculate or replace the part with new part
& take readings
Average Chart
Example
 Average Chart Conclusion

• Used to determine
• Consistency between appraisers
• Adequacy to detect part variation
• Adequacy of resolution
• Adequacy of sample

• In this study, 22 out of 30 points are outside the control limit

• Since this is more than half of the points, the conclusion is that the
measurement system is adequate to detect part-to-part variations.
 X Chart Interpretation
X- Bar chart is important only if measured data is used for process control

Condition Interpretation Action

Less than 50 % of readings are Measurement system is not Improve discrimination of
out of control limits adequate enough to capture the measurement system
process variation or
or Select parts representing
Parts does not represent expected entire process variation
process variation
Repeatability > Reproducibility

When repeatability is large compared to reproducibility

• Instrument needs maintenance
• Redesign gage for more rigidity
• Improve clamping or location of gauging
• Excessive within – part variation

Identify the right cause & solution

75
Reproducibility > Repeatability

When reproducibility is large compared to repeatability

• Appraisers needs training on better way of using the gauge
• Needs better operational definition
• Incremental divisions on instrument are not readable
• Need fixture to provide consistency in gauge use.

Identify the right cause & solution

76
Exercise
Interpret Case study 1

Interpret Case study 2

21 March 2024
 How selection of samples impact R & R by TV method?

• If samples selected does not cover entire process range then PV- Part variation is
less
• Due this TV is less as TV Sq=Pv sq + GRR Sq
• As TV is less % GRR is more as % GRR=(GRR/TV)*100
 How Capable process impacts GRR by TV Method?

• If process is highly capable then Part variation is low.

• Due to this TV is less.
• As TV is % GRR looks higher as % GRR= GRR/TV*100
• So if process is highly capable with high Cp Values then GRR may not be good
• Team need to check if any further improvement is possible
 If process is capable do we need to conduct MSA Study

• If the process is stable & Capable then it can be assumed that MS is acceptable
• This is because MS variation is part of total variation
• When process is capable it means total variation is less than tolerance
• When total variation is less MS variation which is subset of same must be smaller
 Measurement Uncertainty

• True measurement=observed measurement (result) + - U

• U=Kuc where k=2 for 95 % confidence level
• Uc includes all significant components of variation in
measurement system
• Uc sq=sigma sq performance+ sigma sq other
Exercise
• Look at attached calibration reports & find out effective error considering
measurement uncertainty
Attribute Measurement system study
 Attribute Measurement

An attribute gage:
• compares each part to a specific set of limits and accepts the part
if the limits are satisfied
• Cannot indicate how good or how bad a part is, only whether the
part is accepted or rejected (pass/fail)
 How to Conduct Attribute Study?
1. Take 50 parts representing entire process variation, having bad, marginally bad, good &
marginally good parts
2. Mark number 1 to 50
3. Select 3 appraisers
4. Ask them to check parts in a random manner and record the decisions, 1 as OK and 0 as Not ok
5. Get the reference value for all the 50 parts using layout inspection/ variable instrument and decide
its status as 0 or 1
6. Compare each trial of each inspector with the another inspector for their decision
7. Complete the cross tabulation table
8. Calculate Kappa for
• A Vs B, A vs. C , B Vs C
• A vs. Ref, B Vs Ref., C vs. Ref.
Part A-1 A -2 A -3 B -1 B-2 B- 3 C-1 C-2 C-3 Reference
1 1 1 1 1 1 1 1 1 1 1
2 1 1 1 1 1 1 1 1 1 1
3 0 0 0 0 0 0 0 0 0 0
4
5
6
0
0
1
0
0
1
0
0
0
0
0
1
0
0
1
0
0
0
0
0
1
0
0
0
0
0
0
0
0
1
ATTRIBUTE GAGE
CALCULATIONS FOR
7 1 1 1 1 1 1 1 0 1 1
8 1 1 1 1 1 1 1 1 1 1
9 0 0 0 0 0 0 0 0 0 0

“COUNTS”
10 1 1 1 1 1 1 1 1 1 1
11 1 1 1 1 1 1 1 1 1 1
12 0 0 0 0 0 0 0 1 0 0
13 1 1 1 1 1 1 1 1 1 1
14 1 1 0 1 1 1 1 0 0 1
15 1 1 1 1 1 1 1 1 1 1
16 1 1 1 1 1 1 1 1 1 1

There are 34 times where

17 1 1 1 1 1 1 1 1 1 1
18 1 1 1 1 1 1 1 1 1 1
19 1 1 1 1 1 1 1 1 1 1
20 1 1 1 1 1 1 1 1 1 1

A-1 = 1 and B-1 = 1

21 1 1 0 1 0 1 0 1 0 1
22 0 0 1 0 1 0 1 1 0 0
23 1 1 1 1 1 1 1 1 1 1

(that is, of the 50 parts

24 1 1 1 1 1 1 1 1 1 1
25 0 0 0 0 0 0 0 0 0 0
26 0 1 0 0 0 0 0 0 1 0
27 1 1 1 1 1 1 1 1 1 1
28
29
30
1
1
0
1
1
0
1
1
0
1
1
0
1
1
0
1
1
1
1
1
0
1
1
0
1
1
0
1
1
0
checked there were 34
matches by A and B on their
31 1 1 1 1 1 1 1 1 1 1
32 1 1 1 1 1 1 1 1 1 1
33 1 1 1 1 1 1 1 1 1 1
34 0 0 1 0 0 1 0 1 1 0
35
36
37
38
1
1
0
1
1
1
0
1
1
0
0
1
1
1
0
1
1
1
0
1
1
1
0
1
1
1
0
1
1
0
0
1
1
1
0
1
1
1
0
1
FIRST check)
39 0 0 0 0 0 0 0 0 0 0
40 1 1 1 1 1 1 1 1 1 1
41 1 1 1 1 1 1 1 1 1 1
42 0 0 0 0 0 0 0 0 0 0
43 1 0 1 1 1 1 1 1 0 1
44 1 1 1 1 1 1 1 1 1 1
45 0 0 0 0 0 0 0 0 0 0
46 1 1 1 1 1 1 1 1 1 1
47 1 1 1 1 1 1 1 1 1 1
48 0 0 0 0 0 0 0 0 0 0
49 1 1 1 1 1 1 1 1 1 1
50 0 0 0 0 0 0 0 0 0 0
 Kappa Summary
Between appraisers :

kappa A B C
A - 0.86 0.78
B 0.86 - 0.79
C 0.78 0.79 -
With reference :

A B C
kappa 0.88 0.92 0.77
 Number of correct decisions = 45+97/150 = 94.6%
Effectiveness =
Total opportunities for a decision

Miss Rate1: Calling a “BAD” part GOOD.

= 3/48 = 6.3%
1 Bad as Good- Miss- Consumer’s risk
2 Good as bad – False Alarm- Producer’s risk

False Alarm Rate2: Calling a

“GOOD” part BAD.
= 5/102 = 4.9%
Kappa > 0.75 indicates good agreement – Rule of thumb

Effectiveness Decision

More than 90 % Acceptable for the appraiser

More than 80% Marginally acceptable for the

appraiser

Less than 80 % Unacceptable for the appraiser –

Need improvement

Miss Rate Max 2%

(Consumer’s Risk)

False Alarm rate Max 5%

(Producer’s Risk)
 Exercise

• Interpret Case Study 1

• Interpret Case Study 2
• Interpret Case Study 3

21 March 2024
91