Introduction To Statistics With GraphPad Prism Slides
Introduction To Statistics With GraphPad Prism Slides
Introduction To Statistics With GraphPad Prism Slides
Prism 8
Anne Segonds-Pichon
v2019-03
Outline of the course
• In a nutshell: the bigger the experiment (big sample size), the bigger the power (more likely
to pick up a difference).
• Main output of a power analysis:
• Estimation of an appropriate sample size
• Too big: waste of resources,
• Too small: may miss the effect (p>0.05)+ waste of resources,
• Grants: justification of sample size,
• Publications: reviewers ask for power calculation evidence,
• Home office: the 3 Rs: Replacement, Reduction and Refinement.
Experimental design
Think stats!!
• Translate the hypothesis into statistical questions:
• What type of data?
• What statistical test ?
• What sample size?
• Very important: Difference between technical and biological replicates.
Technical Biological
n=1 n=3
Power analysis
• the larger the effect size, the smaller the experiment will need to be to detect it.
• How to determine it?
• Substantive knowledge, previous research, pilot study …
T Distribution
0.95
0.025 0.025
t(14)
t=-2.1448 t=2.1448
• Cohen (1988):
• For most researchers: Type I errors are four times
more serious than Type II errors: 0.05 * 4 = 0.2
• Compromise: 2 groups comparisons: 90% = +30% sample size, 95% = +60%
5 The sample size: the bigger the better?
• It takes huge samples to detect tiny differences but tiny samples to detect huge differences.
• Is the question:
• Is the there a difference?
• Is it bigger than or smaller than?
Experimental design
Choice of a Statistical test
Experiment(s)
Data exploration
• Free packages:
• R
• G*Power and InVivoStat
• Russ Lenth's power and sample-size page:
• http://www.divms.uiowa.edu/~rlenth/Power/
Statistical test
• In a chi-square test, the observed frequencies for two or more groups are compared with
expected frequencies by chance.
Total
% within Did they
Count
dance? 16.7%
32
83.3%
36
100.0%
68 Direct counts approach:
% within Did they dance? 47.1% 52.9% 100.0%
Dog Did they Yes Count 23 24 47
dance?
No
% within Did they
Count
dance? 48.9%
9
51.1%
10
100.0%
19
Expected frequency=(row total)*(column total)/grand total
Total
% within Did they
Count
dance? 47.4%
32
52.6%
34
100.0%
66
= 32*32/68 = 15.1
% within Did they dance? 48.5% 51.5% 100.0%
Probability approach:
Did they dance? * Type of Training * Animal Crosstabulation
Total
Expected Count
Count
9.2
32
9.8
34
19.0
66
Chi2 = (26-15.1)2/15.1 + (6-16.9)2/16.9 + (6-16.9)2 /16.9 + (30-19.1)2/19.1 = 28.4
Expected Count 32.0 34.0 66.0
Counts
10 10
0 0
Food Affection Food Affection
Cat Dog
100 100
• In our example: 80 80
cats are more likely to line dance if they are given food as
Percentage
Percentage
60 60 Dance No
Dance Yes
reward than affection (p<0.0001) whereas dogs don’t mind 40 40
(p>0.99).
20 20
0 0
Food Affection Food Affection
Quantitative data
Quantitative data
• They take numerical values (units of measurement)
) = (-1.6)+(-0.6)+(0.4)+(1.4) = 0
No errors !
• Positive and negative: they cancel each other out.
Sum of Squared errors (SS)
• To avoid the problem of the direction of the errors: we square them
• Instead of sum of errors: sum of squared errors (SS):
• The standard deviation is a measure of how well the mean represents the data.
Standard deviation
• The SD quantifies how much the values vary from one another: scatter or spread
• The SD does not change predictably as you acquire more data.
• The SEM quantifies how accurately you know the true mean of the population.
• Why? Because it takes into account: SD + sample size
Sample means
Big samples (n=30)
‘Infinite’ number of samples
Samples means =
Sample means
SD and SEM
The SD quantifies the scatter of the data. The SEM quantifies the distribution
of the sample means.
SD or SEM ?
• If you are using an in vitro system with no biological variability, the scatter is
about experimental imprecision (no biological meaning).
• Report the SEM to show how well you have determined the mean.
Confidence interval
• Range of values that we can be 95% confident contains the true mean of the population.
- So limits of 95% CI: [Mean - 1.96 SEM; Mean + 1.96 SEM] (SEM = SD/√N)
4) Independence
• Data from different subjects are independent
• Values corresponding to one subject do not influence the values corresponding to another subject.
• Important in repeated measures experiments
Analysis of Quantitative Data
Statistical test
15
Dependent variable
Dependent variable
12
14
11 13
~ 2 x SE: p~0.05 ~ 4.5 x SE: p~0.01
10 12
11
9
10
8 9
A B A B
SE gap ~ 2 n>=10
SE gap ~ 1 n>=10
12.0
11.5
Dependent variable
11.5
Dependent variable
11.0
11.0
~ 1 x SE: p~0.05 ~ 2 x SE: p~0.01
10.5
10.5
10.0 10.0
9.5 9.5
A B A B
CI overlap ~ 1 n=3 CI overlap ~ 0.5 n=3
14
Dependent variable
Dependent variable
12 15
10 ~ 1 x CI: p~0.05
~ 0.5 x CI: p~0.01
8 10
A B
A B
CI overlap ~ 0.5 n>=10
CI overlap ~ 0 n>=10
12
12
Dependent variable
Dependent variable
11 11
~ 0.5 x CI: p~0.05
~ 0 x CI: p~0.01
10 10
9 9
A B A B
Student’s t-test
• 3 types:
• Independent t-test
• compares means for two independent groups of cases.
• Paired t-test
• looks at the difference between two variables for a single group:
• the second ‘sample’ of values comes from the same subjects (mouse, petri dish …).
• One-Sample t-test
• tests whether the mean of a single variable differs from a specified constant (often 0)
Example: coyotes.xlsx
• Sample size
• Data exploration
• Check the assumptions for parametric test
• Statistical analysis: Independent t-test
Power analysis
• Example case:
No data from a pilot study but we have found some information in the
literature.
In a study run in similar conditions as in the one we intend to run, male coyotes
were found to measure: 92cm+/- 7cm (SD).
We expect a 5% difference between genders.
• smallest biologically meaningful difference
G*Power
Independent t-test
Example case:
H0 H1
Power Analysis
For a range of sample sizes:
Data exploration plotting data
Coyote
110
Maximum
100
60
Male Female
Assumptions for parametric tests
Histogram of Coyote (Bin size 2)
10
Females
8 Males
6
Counts
0 70 72 74 76 78 80 82 84 86 88 90 92 94 96 98100102104106 70 72 74 76 78 80 82 84 86 88 90 92 94 96 98100102104106
Normality
Counts
100
90
Length (cm)
80
70
60
Females Males
Independent t-test: results
Homogeneity in variance
• It takes huge samples to detect tiny differences but tiny samples to detect huge
differences.
A group of rhesus monkeys (n=15) performs a task involving memory after having received
a placebo. Their performance is graded on a scale from 0 to 100. They are then asked to
perform the same task after having received a dopamine depleting agent.
Normality
Another example of t-test:
working memory.xlsx
Paired t-test: Results
working memory.xlsx
0
-2
Difference in performance
-4
-6
-8
-10
-12
-14
-16
-18
Comparison of more than 2 means
• Running multiple tests on the same data increases the familywise error rate.
• What is the familywise error rate?
– The error rate across tests conducted on the same experimental data.
• So the probability of making at least one Type I Error is 1-0.857 = 0.143 or 14.3%
• The probability has increased from 5% to 14.3%
• Comparisons between 5 groups instead of 3, the familywise error rate is 40% (=1-(0.95)n)
Familywise error rate
• Solution to the increase of familywise error rate: correction for multiple comparisons
– Post-hoc tests
• Many different ways to correct for multiple comparisons:
– Different statisticians have designed corrections addressing different issues
• e.g. unbalanced design, heterogeneity of variance, liberal vs conservative
Pooled SEM
• If the variance amongst sample means is greater than the error/random variance, then
F>1
– In an ANOVA, we test whether F is significantly higher than 1 or not.
Analysis of variance
Source of variation Sum of Squares df Mean Square F p-value
0
2
4
6
8
0
2
4
6
8
10
A
A
B
B
C
C
D
D
E
E
Parametric tests assumptions
10
Protein expression
1
0.1
A B C D E
0.5
Protein expression
Log Protein
1 0.0
-0.5
0.1 -1.0
A B C D E A B C D E
Parametric tests assumptions
Analysis of variance: Post hoc tests
• The ANOVA is an “omnibus” test: it tells you that there is (or not) a difference
between your means but not exactly which means are significantly different
from which other ones.
– These post hoc tests should only be used when the ANOVA finds a significant
effect.
Analysis of variancec
Analysis of variance: results
Homogeneity of variance
F=0.6727/0.08278=8.13
Correlation
• A correlation coefficient is an index number that measures:
– The magnitude and the direction of the relation between 2 variables
– It is designed to range in value between -1 and +1
Correlation
• Most widely-used correlation coefficient:
– Pearson product-moment correlation coefficient “r”
• The 2 variables do not have to be measured in the same units but they have to be proportional
(meaning linearly related)
– Coefficient of determination:
• r is the correlation between X and Y
• r2 is the coefficient of determination:
– It gives you the proportion of variance in Y that can be explained by X, in
percentage.
Correlation
Example: roe deer.xlsx
• Is there a relationship between parasite burden and body mass in roe deer?
30
Male
Female
25
Body Mass
20
15
10
1.0 1.5 2.0 2.5 3.0 3.5
Parasites burden
Correlation
Example: roe deer.xlsx
Stimulation: Inhibition:
Y=Bottom + (Top-Bottom)/(1+10^((LogEC50-X)*HillSlope)) Y=Bottom + (Top-Bottom)/(1+10^((X-LogIC50)))
Curve fitting
Example: Inhibition data.xlsx 500
No inhibitor
400 Inhibitor
300
200
100
0
-10 -8 -6 -4 -2
1- Choose a Model:
not necessary to normalise
should choose it when values defining 0 and 100 are precise
variable slope better if plenty of data points (variable slope or 4 parameters)
2- Choose a Method: outliers, fitting method, weighting method and replicates
200
100
0
-10 -8 -6 -4 -2
-100 log(Agonist], M
9- Diagnostics:
check for normality (weights) and outliers (but keep them in the analysis)
check Replicates test
residual plots
Curve fitting
Example: Inhibition data.xlsx
Non- normalized data 4 parameters Non- normalized data 3 parameters
500
500
450
450
400
400
350
350
300
300
250
Response
250
Response
EC50 200 EC50
200
150
150 No inhibitor
No inhibitor
100
100 Inhibitor
Inhibitor
50
50
0
0
-9.5 -9.0 -8.5 -8.0 -7.5 -7.0 -6.5 -6.0 -5.5 -5.0 -4.5 -4.0 -3.5 -3.0
-9.5 -9.0 -8.5 -8.0 -7.5 -7.0 -6.5 -6.0 -5.5 -5.0 -4.5 -4.0 -3.5 -3.0
-50 log(Agonist)
-50 log(Agonist)
-100
-100
60
60
EC50 50
50
No inhibitor 40
40
Inhibitor 30
30 No inhibitor
20 20
Inhibitor
10 10
0 0
-10.0 -9.5 -9.0 -8.5 -8.0 -7.5 -7.0 -6.5 -6.0 -5.5 -5.0 -4.5 -4.0 -3.5 -3.0 -10.0 -9.5 -9.0 -8.5 -8.0 -7.5 -7.0 -6.5 -6.0 -5.5 -5.0 -4.5 -4.0 -3.5 -3.0
log(Agonist) log(Agonist)
Curve fitting
Example: Inhibition data.xlsx
500
Non- normalized data 4 parameters No inhibitor Inhibitor
450
350
300
Response
SD replicates 22.71 25.52 200 EC50
450
400
Replicates test for lack of fit 350
Response
Discrepancy (F) 2.982 1.438 200 EC50
50
Inhibitor -7.159 -6.017
0
-9.5 -9.0 -8.5 -8.0 -7.5 -7.0 -6.5 -6.0 -5.5 -5.0 -4.5 -4.0 -3.5 -3.0
-50 log(Agonist)
-100
100
Response (%)
60
No inhibitor
P value 0.0125 0.2618 40
Inhibitor
Evidence of inadequate model? Yes No
30
20
-7.017 -5.943
10
0
-10.0 -9.5 -9.0 -8.5 -8.0 -7.5 -7.0 -6.5 -6.0 -5.5 -5.0 -4.5 -4.0 -3.5 -3.0
log(Agonist)
100
90
50
20
No inhibitor
Inhibitor
Evidence of inadequate model? Yes No 10
-7.031 -5.956
0
-10.0 -9.5 -9.0 -8.5 -8.0 -7.5 -7.0 -6.5 -6.0 -5.5 -5.0 -4.5 -4.0 -3.5 -3.0
log(Agonist)
My email address if you need some help with GraphPad:
anne.segonds-pichon@babraham.ac.uk
https://www.bioinformatics.babraham.ac.uk/training.html