Experimental

design
EVALUATION
TSHEGOFATSO MOSEDAME -
201200772
OFENTSE LEKGATHO-
201403673
 This design is used to determine if a program or intervention is more
effective than the current process.
WHAT IS
EXPERIMENTAL
It involves a comparison between respondents that received intervention
or treatment and a control group that did not receive any treatment.
DESIGN
EVALUATION
It is viewed by many as a premier indicator of causation.
 THEORIST OF THE MODEL
The design of experiments was invented by Ronald A. Fisher in the 1920s
and 1930s. He was a geneticist, mathematician, and statistician.
In evaluation, it was proposed by Donald Campbell. The central theme of
Donald’s work was to describe, explain and improve how humans learn
about the real world, he was particularly interested in methodology factors
that bias the collection, analysis, and interpretation of social data.
 Types of experimental Model

TRUE EXPERIMENTAL EVALUATION QUASI EXPERIMENTAL EVALUATION

• Random assignment is used to make • Groups are not based on random assignment.
groups.
TRUE EXPERIMENTAL
-It can include pre- and post-testing
Or comparison across people who received an intervention and those who didn’t.
WHY DO WE RANDOMLY ASSIGN
Randomly assigning participants allows evaluators to assume that any pre-existing differences
among participants are evenly distributed between those receiving intervention and those in the
control group.
-Individual differences are assumed to be balanced.
Theorist reasoning
• Experiments allow us to deliberately vary
something so as to discover what happens to
something else later (it discovers the effect of
presumed cause).
• He believed that the acquisition of knowledge was
a process of generating and testing falsifiable
hypotheses and retaining those that solve the
problem. He believed that Experimental design was
the key to evaluating these solutions.
 How the model works

• Participants are selected randomly for both intervention and

control.
• It assigns subjects to treatment and control.
• Measures changes in both groups to determine if an
intervention produced desired effects.
 Study that used Experimental model

Article: Camerini L, Schulz PJ, Effects of Functional Interactivity on Patients’

Knowledge,Empowerment, and Health Outcomes: An Experimental Model-
DrivenEvaluation of a Web-Based Intervention J Med Internet Res
2012;14(4):e105
Objectives:
• To test whether health knowledge and empowerment mediate a
The possible relationship between the availability of interactive features
on an eHealth application and individuals’ health outcomes.
 Methodology

Data collection tolData collection tool

• The questionnaires
• FMS outcomes are generally measured with the Fibromyalgia Impact Questionnaire (FIQ)
• The FIQ includes items covering all the disease-specific domains, accounting for functional
disability, job ability, pain intensity, sleep function, stiffness, anxiety, depression, and the
overall sense of wellbeing.
Evaluation Model
• This study used a pretest–posttest experimental design.
 Application of the model in the study
• True experiments have four elements: manipulation, control, random assignment, and
random selection.
• Random Selection: The reference population in this study consisted of patients with
fibromyalgia syndrome (FMS)
• Manipulation: To investigate the effect of functional interactivity, three different versions
of the ONESELF application, each implementing different enabling functions was created.
• The application enabled asynchronous and synchronous interactions with health
professionals and lay people.
• The system also included static informative sections in a virtual library that provided users
with relevant information on the disease first aid and frequently asked questions sections
provided brief and practical information on syndrome management.
 Application Continues

• A virtual gymnasium provided patients with tailored multimedia contents on several physical exercises
that constitute the wider part of the nonpharmacological treatment of FMS.

• Later, a section with testimonies, where patients could post their stories and read stories of other
people with the same health condition, enhanced the dimension of social support.

• We also designed and implemented synchronous interaction via a chat room and asynchronous
interaction in an online forum. Patients used these tools to communicate with the physicians and
among themselves.
 Application continues
• Randomization
• Patients were randomly assigned to one of the three versions and blinded to the others, using a computer utility
that assigned them to a randomly selected experimental condition until the conditions were equally filled
• Control
• Patients in group 1 (n = 55) were given a static version of ONESELF, including only the library, the virtual
gymnasium, the testimonials, and the generic sections such as the first aid, the frequently asked questions, and
other common contents (eg, contacts, legal notices, and ownership disclosure). No interactive enabling tools
such as the Web forum or the chat room were present in this version. This group was considered as the
reference or control group.
• Patients in group 2 (n = 55) were given an interactive-only version of ONESELF, including the Web forum, the
chat room, and the generic sections. Static sections were not implemented in this version.
• Patients in group 3 (n = 55) were given the full version of ONESELF, including both static and interactive
components.
 Measurements of Variables
• Knowledge
Knowledge was assessed with 10 multiple-choice questions adapted from the Mayo Clinic website relating to
FMS symptoms, etiology, treatments, and management. Each answer was coded 1 when correct and 0 when
incorrect.
• The final measure of knowledge was obtained by a mean score calculation of the 10 items, with a theoretical
range from 0 (no correct responses) to 1 (all correct responses).
• Empowerment
This measure reflects the multidimensionality of the construct of empowerment, which is a combination of
meaning, competence, self-determination, and impact. Each one of the subdimensions is treated as a latent
construct with three observed indicators. Each indicator was measured on a 7-point Likert scale.
• Health outcome
Health Outcomes were measured with the FIQ in its Italian version. The FIQ is a validated questionnaire that
consists of 20 indicators to assess patients’ disability to carry out everyday activities, patients’ intensity of pain,
and the interference of FMS with patients’ sleep and emotional state. The FIQ provides a single score ranging
from 0 to 100, where a higher score indicates a greater impact of FMS on the patient. For this reason, it should
be considered a measure of negative health outcomes.
 Analysis and Results
• Analyses
• The main analyses were conducted using structural equation modelling techniques. SPSS AMOS 18 (IBM
Corporation, Somers, NY, USA) was used for the analyses. Specifically, we examined the effect of
interactivity on knowledge, meaning, competence, self-determination, and impact.
• Results
• The main finding was that functional interactivity had no impact on empowerment dimensions, nor direct
observable effects on knowledge. However, knowledge positively affected health outcomes (b = –.12, P
= .02), as did the empowerment dimensions of meaning (b = –.49, P < .001) and impact (b = –.25, P
< .001).
• In summary, the presence of interactive elements in our eHealth intervention did not affect knowledge, did
not affect patient empowerment in the expected direction (but reduced the empowerment dimension of the
meaning), and did not improve the health outcome of perceived fibromyalgia impact. In contrast to other
studies, ours did not find beneficial effects of functional interactivity. However, knowledge and two
dimensions of empowerment (meaning and self-determination) did affect health outcomes.
 References
• Camerini, L., & Schulz, P. J. (2012). Effects of functional interactivity on patients’ knowledge, empowerment, and health
outcomes: An experimental model-driven evaluation of a web-based intervention. Journal of medical Internet research, 14(4),
e1953.
• Craig, S. B., & Hannum, K. M. (2007). Experimental and quasi-experimental evaluations. The handbook of leadership
development evaluation, 19-47.
• Campbell, D. T., & Stanley, J. (1996). Experimental and Quasi-experimental. Design for Research. Ran McNally, Chicago, III.
• Rogers, J., & Revesz, A. (2019). Experimental and quasi-experimental designs. In The Routledge handbook of research methods
in applied linguistics (pp. 133-143): Routledge.
• Kotz, D., Newport, C., Gray, R. S., Liu, J., Yuan, Y., & Elliott, C. (2004). Experimental evaluation of wireless simulation
assumptions. Paper presented at the Proceedings of the 7th ACM international symposium on Modeling, analysis and
simulation of wireless and mobile systems.
• https://www.scribbr.com/methodology/experimental-design/