SURVEY RESEARCH, META-ANALYSIS, PRACTICE GUIDELINES, AND PHA6129LAB

HEALTH OUTCOMES RESEARCH

Course Facilitator: Ms. Marri Jmelou M. Roldan, RPh, MSc. Pharmacy Informatics
First Semester, Third Shifting
S.Y. 2020-2021

SURVEY RESEARCH TYPES OF DATA COLLECTED

SURVEY RESEARCH 1. Incidence

∙ try to determine the occurrence of events without
∙ uses information obtained from a selected members drawing any relationships between variables
of the population, analyzes the responses, and then
∙ Examples:
draws conclusions applicable back to the same
∙ morbidity or mortality data, manpower data
population (larger population)
2. Attitudinal
∙ can be considered as either:
∙ try to compare this dependent variable with
∙ Descriptive
independent variables such as age, education, or
∙ such as opinions and attitudes
salary
∙ Explanatory in Nature
∙ Examples:
∙ such as explaining a cause and effect
∙ job satisfaction surveys
∙ validity of the results depends on the quality of the
3. Knowledge
study’s internal rigor (methodology)
∙ attempt to document a person’s knowledge or
∙ used to:
understanding about a specific topic
∙ study the incidence, distribution, and
∙ Example:
relationships of sociologic and psychological
∙ physician’s knowledge of retail prices of
variables
medications or pharmacist’s knowledge of
∙ collect information from a sample and generalize state pharmacy laws
the findings to a larger, target population
4. Behavior Measurements
∙ identify, assess, and compare respondents’ ideas,
∙ documents what a person actually does in a
feelings, plans, beliefs, and demographics
particular situation rather than asking him or her
∙ determine how health care programs should be
in a survey, which may reflect an attitude, rather
implemented by utilizing the opinions of experts
than the actual observed behavior
with experience in a particular area
∙ study effectiveness of a program by surveying DATA COLLECTION
individuals who have used its services
∙ understand attitudes and behaviors of patients or ∙ can be performed using questionnaires, examination
members of the profession the program affects of historical records, telephone interviews, face-to-
∙ behavior, experience of both patient and face interviews, web-based questionnaires, focus
doctor groups, or panel interviews
∙ Questionnaire Forms
TYPES ∙ paper or electronic format via email or Internet-
based program
∙ key characteristic of survey sample is they represent
∙ in some cases, both paper and electronic format
the population of interest
are used to provide different options for the
participant to choose preference
1. Descriptive Surveys
∙ Web-Based Questionnaires
∙ attempt to identify psychosocial variables such as
∙ due to inadequate sample frame and flaws in data
attitudes, opinions, knowledge, and behaviors in collection
a population
∙ ex. people without telephones or those with
unlisted number would be excluded from the
2. Explanatory Surveys
sample frame of names from a telephone
∙ attempt to explain causal relationships between
directory
variables
∙ Examples: CHARACTERISTICS OF WELL-CONDUCTED SURVEY
∙ Dependent (knowledge and behavior)
∙ Independent (age, sex, education) ✓ objective and carefully planned
✓ data are quantifiable
✓ subjects surveyed are representative of the target
population

Department of Pharmacy, University of Santo Tomas Bugayong & Alcazar (3APH) | Page 1 of 6
 Survey Research, Meta-Analysis, Practice Guidelines, and Health Outcomes Research

ERRORS AFFECTING PRECISION AND ACCURACY ∙ Cronbach alpha

∙ interpreted in the same fashion that
1. Coverage Error coefficient of reliability are interpreted
∙ bias in a statistic that occurs when the target ∙ ex. A value of zero indicated no consistency
population you want to survey does not coincide between responses while increasing
with the sample population that is actually consistency is seen as you approach 0.8-1. 0.8
surveyed is the minimum valid coefficient score
∙ can compromise the ability to generalize study ∙ method section should be heavily scrutinized
results ∙ for most, a description of study methodology with
enough detail to replicate the study should also be
2. Sampling Error provided
∙ refers to the difference between the estimate
derived from a sample survey and the true value 2. Sample Size
that would result if a census of the entire target ∙ methods section should report sample size, along
population were taken under the same conditions with a description of how it was determined
∙ occurs when the researcher surveys only a subset ∙ sample size computation provides the minimum
(sample) of all possible subjects within the number of individuals that needs to be sampled
population of interest from the population
∙ use random sampling procedure and large sample ∙ appropriate sample size will give sufficient
size to minimize this error statistical power to demonstrate a difference
∙ sample error is computed by: between two groups
∙ mean plus standard error of the mean (SEM)
3. Sample Frame
3. Measurement Error ∙ describes the population from where actuals
∙ occurs when the collection of data is influenced by samples will be drawn from for the survey
the interviewer or when the survey item itself is respondents
unclear from the respondent’s point of view ∙ sample is comprehensive if all members of a
∙ face to face interview may influence the response population had a chance to be chosen and no one
of the respondents was systematically excluded
∙ survey instrument itself may be ambiguous or ∙ additionally, the readers should evaluate the
open to interpretation so bias can be introduced comprehensiveness and the probability of
into a certain survey by the cover letter or selection and efficiency of sample frame
sponsoring body either may lead to respondents
to one desired answer rather than the true 4. Sample Strategy and Response Rates
response ∙ methods section should supply the reader with
enough information to ensure that nonresponse
4. Nonresponse Error error was assessed and measures were taken to
∙ occurs when a significant number of subjects in a control the possible errors
sample do not respond to the survey ∙ authors should relate as much information about
∙ nonresponse bias can result in surveys, where the non-responders as possible i.e., reasons for non-
answers of respondents differ from the potential response or non-participations
answers of those who did not answer ∙ repeated attempts to obtain completed attempts
∙ strive for response rates in the 80% to 90% range from questionnaire from initial non-respondents
will yield higher response rate and accurate
EVALUATION OF SURVEY
results than if no follow-ups are preformed
1. Factual Data and Internal Validity
∙ there must be attempts to assess validity and 5. Survey Instrument Reliability
reliability of the survey and efforts made to ∙ assess the reliability i.e., can the results of the
validate factual data survey be repeated by another investigator and
∙ asking more than one question about a concept present the results of reliability estimates
can increase the internal validity of a survey ∙ statistical test that measures correlation between
∙ correlation analysis such as Cronbach alpha or items should be used as a reliability estimate
similar statistical test that measures correlation ∙ pretest or pilot test is an assessment of a
between items should be calculated and the questionnaire made before full-scale
coefficient factor(s) reported in the article implementation to identify and correct problems

Department of Pharmacy, University of Santo Tomas Bugayong & Alcazar (3APH) | Page 2 of 6
 Survey Research, Meta-Analysis, Practice Guidelines, and Health Outcomes Research

such as faulty questions, flawed response options, ∙ only articles published in the author’s
or interviewer training deficiencies primary language are used
∙ ex. German author
REVIEW ARTICLES AND META-ANALYSIS
3. Selection Bias
REVIEW ARTICLES
∙ to reduce selection bias, authors must be blinded to:
∙ classified as tertiary literature and may be subject to 1. Names of the Study Authors (to avoid political or
author biases or inaccuracies in the literature search personal issues)
∙ Meta-Analyses ∙ 2 or more authors should critic it in
∙ despite being classified as a review article is consideration and all evaluators concur on
considered as a primary literature because it which studies will be included in the
generates new data and findings from previous systematic review
researches 2. Institution of Publication
3. Results of the Studies
TYPES ∙ for the initial choice of study inclusion, only the
methods section should be reviewed
∙ tertiary except meta-analysis although used as
secondary because they can lead readers to primary Meta-Analysis (Quantitative Review)
references
1. Narrative or Non-systematic Review (Qualitative ∙ widely used to provide supporting evidence for clinical
review) decision making or in areas of conflicting evidences by
2. Systematic Review (Qualitative review) combining results of previous researchers and re-
3. Meta-Analysis (Quantitative review) analyzing them as a whole new data set
∙ widely used in providing support in clinical
Narrative or Non-Systematic Review (Qualitative Review) decisions
∙ analytical technique to combine data from multiple
∙ summary of research that lacks a description of
studies
systematic methods (e.g., system of literature search
and inclusion) Types
∙ ex. pharmacy therapeutics committee
∙ may pertain to one specific clinical question or disease 1. Pairwise Meta-Analysis
state, or to topics related to pharmacy administration ∙ this is the traditional method of synthesizing the
results of different trials to obtain an overall
Systematic Review (Qualitative Review) estimate of the treatment effect of one
intervention relative to the control
∙ subjects for this review are scientific investigations
with predefined methods and original studies
2. Network Meta-Analysis
∙ summary of results of primary studies where the
∙ aka multiple treatment comparison or the mixed
results are not statistically combined
treatment meta-analysis
∙ defined as a research summary conducted in an
∙ network of randomized controlled trials is
organized and reproducible manner that addresses a
developed where all these trials have one
clinical question
intervention in common
∙ should be developed in a responsible manner with the
∙ simultaneously compares and ranks a network of
highest level of integrity
treatments in which the sub-sets have been
compared in individual studies
∙ can commonly be affected by these biases:
1. Publication Bias Uses
∙ publication selection for systematic review
∙ when published studies are given more 1. supporting or refuting lesser quality evidence
consideration because they are positive in 2. overcoming reduced statistical power of small studies
nature than unpublished studies 3. assessing occurrence of rare events
∙ Drawback of Unpublished Studies 4. providing guidance with limited/conflicting data
∙ have likely not undergone a peer-review 5. displaying sample sizes and treatment effects
and revision process; errors and unclearly graphically
stated conclusions may be present 6. assessing heterogeneity between studies and
2. Language Bias publication bias
7. evaluating the natural history of disease

Department of Pharmacy, University of Santo Tomas Bugayong & Alcazar (3APH) | Page 3 of 6
 Survey Research, Meta-Analysis, Practice Guidelines, and Health Outcomes Research

8. improving estimates of effect size ∙ common statistical tests used to combine and
9. answering new questions not posed at the start of analyze the data:
individual trials ∙ Mantel-Haenszel test for categorical data
∙ Inverse Variance test for continuous data
Issues
Forest Plot
1. Publication Bias
∙ studies that show lack of efficacy are less likely to ∙ common method to present the results of a meta-
be located than those that demonstrate beneficial analysis
effects of a drug
∙ identified by funnel plots
∙ Funnel Plots
∙ scatterplot of treatment effect versus sample
size of the studies included in the meta-
analysis
∙ form of selection bias – wherein it is based on the
magnitude or direction or statistical significant of
the result
∙ most of the documented occurrence, many
researchers are more likely to publish studies that
demonstrate positive effects of drugs therefore it
is most likely to be located

Sensitivity Analysis

∙ should be done on any article that seemed to not be

as similar as the others combined
∙ once identified, analyses are repeated without each of
those suspicious articles
∙ to determine if this changes the overall results or
conclusions
∙ to confirm the accuracy of the results produced by the
2. Presence of Divergences
meta-analysis
∙ Heterogeneity
∙ performed once the data have been extracted from
∙ differences in the way the studies being
the chosen studies and when the analysis is complete
included in the meta-analysis were
∙ helps to determine if results of the meta-analysis are
conducted)
robust
∙ Publication Bias
∙ tendency of investigators to preferentially Meta-Analysis (Quantitative Review) (CON’T.)
submit studies with positive results for
publication) ∙ results should be interpreted with caution
∙ conclusions of meta-analysis depend on the quality
Areas for Evaluation and similarity of the studies included
∙ be careful with oversimplification, it may lead to
1. clearly defined the clinical question
inappropriate conclusions
2. hypothesis must be stated
3. detailed protocol developed process PRACTICE GUIDELINES
4. provide details of literature searches
5. criteria for inclusion must be determined PRACTICE GUIDELINES
6. studies should be similar enough (also referred to as
∙ documents used by clinicians that provide guides in
homogeneity) to allow pooling
clinical decision-making and criteria regarding
7. randomization techniques, compliance, blinding,
diagnosis, management, and treatment in specific
appropriate dosing and length of studies, and ITT
areas of healthcare
(intention to treat) analyses must be clearly described
∙ documents are commonly compiled evidences from
Analysis and Results Presentation clinical researchers and professional expertise of
clinicians from different fields

Department of Pharmacy, University of Santo Tomas Bugayong & Alcazar (3APH) | Page 4 of 6
 Survey Research, Meta-Analysis, Practice Guidelines, and Health Outcomes Research

∙ provides information regarding therapeutic options ∙ important because this is where we see
and most appropriate choices for a specific disease the overall impact of the treatment or
and patient intervention towards the patient’s life
∙ research has shown that within 2 years of and lifestyle
development, a practice guideline may become
outdated, and must be updated ∙ Quality-Of-Life Measures
∙ Clinical Practice Guidelines ∙ focuses on the patient’s perception of well-being
∙ statements that include recommendations ∙ most important outcome in disease specific
intended to optimize patient care that are states
informed by a systematic review of evidence and ∙ multidimensional concept focusing on impact of a
an assessment of the benefits and harms of disease and treatment relative to the well-being
alternative care options (Institute of Medicine) of a patient
∙ subjective evaluation of both positive and
Types
negative characteristic of an individual life
∙ health-related quality-of-life (HR-QOL)
1. Evidence-Based
∙ value assigned to quality and quantity of life
∙ utilize a rigorous systematic process involving
that is affected by many different variables
review and critical evaluation of the medical
including: disease, injury, treatment, or
literature to develop final recommendations
health and insurance policies
2. Formal Consensus-Based
∙ used to assist in decision making regarding
∙ utilize experience of experts in their practice area
interventions such as procedures and
to draw conclusions and develop
pharmacotherapy of patients
recommendations
∙ useful for those instances where the evidence Two Types of HR-QOL Measurements
does not exist, is not complete, or not conclusive
enough to allow the development of a final 1. Health Status Assessment
recommendation ∙ self-assessment that measures multiple aspects of
3. Mixture of EBM and Consensus-Based a patient’s perceived wellbeing
∙ uses evidence to construct the guideline and ∙ assessment is primarily designed to either
supplements those steps without evidence with compare groups of patients receiving different
experience of the experts in that field treatments or effect of a treatment for a single
∙ useful especially in areas of practice where no group over time
established evidence or proof is available to be ∙ questions used ranges from perceived impact of
the standard of practice disease and treatments to disease frequency and
severity
Practice guidelines are created primarily for: ∙ most often used in clinical trials comparing
treatment regimen
✓ facilitating clinical decision making
2. Patient Preference Assessment
✓ improving the quality of health care
∙ reflect an individual’s decision-making process at
✓ providing consistent treatment across environments
a time when the eventual outcome is unknown
✓ decreasing costs
∙ measures the patient’s trade-off between quality
✓ diminishing professional liability
and quantity of life
✓ identifying individualized alternative treatment
Quality-Adjusted Life Years (QALYs) Gained from an
HEALTH OUTCOMES RESEARCH
Intervention
∙ Health Outcomes Research
∙ literatures directed into studying the impact of
treatments or intervention to the daily lives of
patients, it encompasses literatures for:
∙ pharmacoeconomic outcomes
∙ therapeutic outcomes
∙ non-therapeutic outcomes (such as number
of visits to the emergency room and number
of hospital admissions)
∙ quality-of-life (QOL) outcomes

Department of Pharmacy, University of Santo Tomas Bugayong & Alcazar (3APH) | Page 5 of 6
 Survey Research, Meta-Analysis, Practice Guidelines, and Health Outcomes Research

∙ An understanding of strengths and limitations

inherent with each design is essential to determine
the overall quality of the evidence produced. Those
trial designs with a high level of quality provide the
most reliable evidence and that translates into the
strongest recommendation/clinical decision.

Department of Pharmacy, University of Santo Tomas Bugayong & Alcazar (3APH) | Page 6 of 6