5 Sensitivity Analysis and HRQOL

Pharmacoeconomics & Health Outcomes
Sensitivity Analysis
&
Health Related Quality of Life
Leon E. Cosler, R.Ph., Ph.D.

Associate Professor of Pharmacoeconomics
Albany College of Pharmacy
Road Map (sensitivity analysis)
• What is it?
• How is it used?
• Why is used?
• What types are common?
• “Where’s he getting this?”

- Sensitivity Analysis: 76-77;105-107;162; 371-372;384
- HRQOL: Text Chapter 7
What is it?
A process of varying assumptions and

variables over “plausible” ranges to test the
“robustness” of results and conclusions
Example:
When you change the cure rate of an

antibiotic, does it change your conclusions?
Why do this?
1. For effect of assumptions on conclusions
2. Identifies your critical assumptions
3. Trying to predict the future effects
4. Now a key indicator of a quality study

Sensitivity Analysis
What variables are often tested ?
• Any key assumption that affects the

outcome
• Examples
- Rx effectiveness rates
- Adverse Event Profiles
- Resources used & their costs
- Discount rates
Sources of Uncertainty
1. Variability in the data
2. Generalizability to real world practice
3. Extrapolation of results
4. Choice of Analytical method(s)

Conducting a Sensitivity Analysis
Assume you have the following :
- Rx adverse event profile range: (1% to 5%)
- “realistic” discount rates ( 3% to 5%)
- Cost of one alternative (e.g. surgery) varies

from $ 100,000 to $150,000
• How many different combinations do you have?

Types of Sensitivity Analyses
1. “Simple”
- modify key assumptions across a reasonable range
- if conclusion changes; the model is “sensitive”
2. Threshold
- sort of a “breakeven point”
3. Analysis of extremes
- “best case” & “worst case” scenarios
4. Probabilistic (Monte Carlo)

- where values in ranges selected at random
Limitations of Sensitivity Analysis
• Are the ranges tested realistic?
• If results are “sensitive” - so what?

- You can make any variable “sensitive”
• Simple vs multi-way sensitivity
• SA is not being used enough in literature

- not at all – or -
- not sufficiently
Where do we get realistic estimates?
• Clinical trials
• Individual RCTs or meta-analyses
• Health care claims data

• Hospital(s) / Managed Care Plans
• Medicare / Medicaid
• Nationally public data sets

• NCHS
• Epidemiology is fundamental
Pharmacoeconomics & Health Outcomes
Health Related Quality of Life

Psychometrics: (The Science of Surveys)
Leon E. Cosler, R.Ph., Ph.D.

Associate Professor of Pharmacoeconomics
Albany College of Pharmacy
Road Map: Measuring Health Status
1. General vs Dx-specific instruments

• Advantages & disadvantages
2. Psychometrics
- Important properties of surveys:
• Reliability
• Validity
• Precision
• Sensitivity / specificity
Health Status & Quality of Life
• Health Status includes:

• functional status
• morbidity (disability)
• well-being (mental / social / role functioning)
• Quality of life is an opinion or assessment

• based on Pt. perceptions and judgements
• influenced with their level of satisfaction
“Patient Outcomes” are not “Clinical Outcomes”
Condition / Disease Clinical Measure Patient Outcomes Measure
Osteoporosis Bone Mineral Density Pain

Fractures
Quality of Life
Cancer No. / Size of Metastases Pain

Quality of Life
Hypertension BP Quality of Life

Patient Satisfaction
Diabetes Serum Glucose Vision Loss

HgB A1c Physical functioning
Quality of Life
Asthma FEV1 Symptom Scores

PEFR Quality of Life
General vs Disease Specific Surveys
• “General” health survey(s)
- Creation of population “normal” values

• comparisons of sub-populations
- These measure several health categories

• functional status (physical / mental)
• social / role functioning
• overall assessment of health
General vs Disease Specific Surveys
• “Global” or “Generic” survey
Medical Outcomes Study Short-Form(s):
SF-36; SF-12
Sickness Impact Profile (SIP)
Nottingham Health Profile (NHP)
General Health Questionnaire (GHQ-28)
Psychological General Well-Being Index (PGWBI)
EQ-5D (EuroQol)
• SF-36
measures
health in 8
dimensions:
www.sf-36.org
http://www.sf-36.org/demos/SF-36v2.html
General Surveys
• Disadvantages or limitations
- lengthy & time consuming
- lower response rates
- complex scoring (need a computer or $$$)
- may not address concerns for specific Dxs.
Disease Specific Surveys
• Early Dx specific forms weren’t surveys

- measured functional limitations
- Ex:
• Karnofsky Performance Status Scale
• American Rheumatism Association
Functional Classification
• NY Heart Association classification
- Originally designed by clinicians
Disease Specific Surveys
• Advantages:
- more sensitive to disease specific effects
- fewer questions
- more relevant & more patient focused
• Disadvantages:
- not comparable to larger groups
- may miss “unexpected” effects
Disease Specific Surveys: Examples
Disease / Condition Instrument
Alzheimers Alzheimer's Disease Assessment Scale

The Dementia Rating Scale
Arthritis American Rheumatism Association (ARA) classification

Arthritis Impact Measuremnt Scales (AIMS)
Health Assessment Questionnaire (HAQ)
Asthma Asthma Bother Profile

Asthma Quality of Life Questionnaire (Various)
Asthma TyPE Specification
Childhood Asthma Questionnaire
Cancer European Organization for Research & Treatment of Cancer

Quality of Life Questionnaire (EORTC-QLQ30)
Functional Assessment of Cancer Therapy (FACT) for general cancer
Karnofsky Performance Scale
Cardiovascular Diseases Angina TyPE Specification

Duke Activity Status Index
Seattle Angine Questionnaire
Pain Low Back Pain TyPE Specification

McGill Pain Questionnaire
Visual Analog Pain Rating Scale
http://www.proqolid.org/
Check literature in two areas
1. Development and testing of survey itself

• sociology / psych / educational literature
2. Applied research using the selected survey
• health / medical sources
Components of a well-written report describing the use of a HRQOL instrument

1. Statement of the research questions and hypotheses
2. Description of the measuring instruments included in the study
3. Description of the subjects, sampling procedures, sampling frame
4. Description of the conditions under which the data were collected
5. Discussion of the statistical analysis
6. Presentation and discussion of the results, statement of conclusions, and
description of the limitations
Psychometrics
Important properties of surveys:
Several Important properties of surveys:
1. Reliability
2. Validity
3. Responsiveness or “Precision”
4. Sensitivity / Specificity
1. Reliability...
• This test is reliable...
Time 1 or first 1/2 Time 2 or second 1/2

Reliability...
• This test is NOT reliable…
Time 1 or first 1/2 Time 2 or second 1/2
• Notice neither one hits the target !

Important properties of surveys:
Reliability: is this a ‘true’ score ?
- Statistics used to measure reliability:
Method Question Answered
Longitudinal
Pearson Product-moment Are rankings the same over time?
Confidence Intervals (95%) Is the variation greater than expected?
Intraclass Correlation Coefficient How much variation is due to error?
Cross-Sectional
Alternate Forms How similar are score from similar surveys?
- “Chronbach’s alpha”
Internal Consistency
(alpha) Are responses to items in a scale equivalent?
- prefer values > 0.8 or > 0.9

2. Validity
• Are you measuring what you think you’re
measuring?
Valid Not Valid

Validity (several different types)
A. Content validity
• does survey contain appropriate questions ?
B. Criterion validity
• do scores survey compare to other “gold
standard” instrument?
C. Construct validity
• checks the theory behind the test
• Relates to other measures in plausible ways
Validity (several different types)
D. Factorial validity
- do items within a dimension relate?
- (factor or cluster analysis)
Reliability: must be established first

- a valid instrument will be reliable
- a reliable instrument may / may not be valid
Precision / Responsiveness
• if “the target” changes,
will you be able to detect the changes?
• This is not precise...

Precision
• if “the target” changes, will you be able to
detect the changes?
• This is a precise measurement

Sensitivity:
Reality (objective measurement)
"Present" "Absent"
"Present" A B
Survey says:
"Absent" C D
The ability to detect a change when it exists

=A/(A+C)
Specificity:
Reality (objective measurement)
"Present" "Absent"
"Present" A B
Survey says:
"Absent" C D
The ability to detect “no change” when

there really is no change
=D/(B+D)
Psychometric Theory: They really work
Mortality by Levels of Physical Health

from SF-36
Percent Mortality 5 years
25.0% 21.5%
20.0%
15.1%
15.0%
10.0%
6.2%
4.7%
5.0% 1.8%
0.0%
8-24 25-34 35-44 45-54 > 55
That’s all for today… !

5 Sensitivity Analysis and HRQOL

Uploaded by

Copyright:

Available Formats

5 Sensitivity Analysis and HRQOL

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

5 Sensitivity Analysis and HRQOL

Uploaded by

Copyright:

Available Formats

Pharmacoeconomics & Health Outcomes

Leon E. Cosler, R.Ph., Ph.D.

• “Where’s he getting this?”

A process of varying assumptions and

When you change the cure rate of an

1. For effect of assumptions on conclusions

2. Identifies your critical assumptions

3. Trying to predict the future effects

4. Now a key indicator of a quality study

What variables are often tested ?

• Any key assumption that affects the

1. Variability in the data

2. Generalizability to real world practice

4. Choice of Analytical method(s)

- Rx adverse event profile range: (1% to 5%)

- “realistic” discount rates ( 3% to 5%)

- Cost of one alternative (e.g. surgery) varies

• How many different combinations do you have?

4. Probabilistic (Monte Carlo)

• If results are “sensitive” - so what?

• Simple vs multi-way sensitivity

• SA is not being used enough in literature

• Health care claims data

• Nationally public data sets

Health Related Quality of Life

Leon E. Cosler, R.Ph., Ph.D.

1. General vs Dx-specific instruments

• Health Status includes:

• Quality of life is an opinion or assessment

Osteoporosis Bone Mineral Density Pain

Cancer No. / Size of Metastases Pain

Hypertension BP Quality of Life

Diabetes Serum Glucose Vision Loss

Asthma FEV1 Symptom Scores

• “General” health survey(s)

- Creation of population “normal” values

- These measure several health categories

Sickness Impact Profile (SIP)

Nottingham Health Profile (NHP)

General Health Questionnaire (GHQ-28)

Psychological General Well-Being Index (PGWBI)

• Early Dx specific forms weren’t surveys

Alzheimers Alzheimer's Disease Assessment Scale

Arthritis American Rheumatism Association (ARA) classification

Asthma Asthma Bother Profile

Cancer European Organization for Research & Treatment of Cancer

Cardiovascular Diseases Angina TyPE Specification

Pain Low Back Pain TyPE Specification

1. Development and testing of survey itself

Components of a well-written report describing the use of a HRQOL instrument

Time 1 or first 1/2 Time 2 or second 1/2

Time 1 or first 1/2 Time 2 or second 1/2

• Notice neither one hits the target !

- prefer values > 0.8 or > 0.9

Valid Not Valid

Reliability: must be established first

• This is not precise...

• This is a precise measurement