DPBS 1203

Business and Economic Statistics

Lecture 5.1
Week 5 topics
 Concept of an estimator
 Properties of estimators
– Sampling distributions
 Confidence intervals
 Statistical confidence
 Estimating the population proportion: Point and interval
estimation
 Introduction to hypothesis testing
Estimation
 Inferential statistics:
– Extracting information about population parameters on the basis of
sample statistics
 “Past data indicates that 60% of passengers choose beef over
chicken”
 What does that sample proportion tell us about the population
proportion?
 In practical situations like this, parameters for the population are
unknown
 Using sample statistics to try to GUESS what the parameters are is
usually the only practical alternative
Process of inferential statistics

Calculate 𝑝𝑝̂
to estimate 𝑝𝑝
Population Sample
𝑝𝑝 𝑝𝑝̂
(parameter ) (statistic )

Select a
random sample
The process of inference, in words
 Parameters describe key features of populations
 In practical situations, parameters are unknown
 Instead, a sample is drawn from the population to provide basic
data
 These data are used to calculate various sample statistics
 These sample statistics are used as estimators for population
parameters
Estimation…
 Estimators
– Consider a generic parameter θ that characterizes the pdf,
f(x), of a random variable X.
– Suppose X1, X2, …, Xn is a sample of size n drawn from f(x)
– A statistic is any function of data in the sample
– An estimator is a statistic whose purpose is to estimate a
parameter or some function thereof
– A point estimator is simply a formula (rule) for combining
sample information to produce a single number to estimate θ
Estimation ...
 Estimators are random variables because they are
functions of random variables, X1, X2, …, Xn
 Examples of point estimators
– The sample proportion is a point estimator for the
population proportion
– The sample mean is a point estimator for the population
mean
– The sample variance is a point estimator for the population
variance
– Why does it make no sense to expect an estimator to
always produce an estimate equal to the parameter of
interest?
Properties of estimators
 The sample mean is a ‘natural’ choice of estimator for the population mean
– But there may be other (better?) estimators
– Recall our discussion of using s2 as an estimator for σ2 (“why do we
divide by n-1?”)
 Desirable properties of estimators
– Unbiasedness: If we constructed it for each of many hypothetical
samples of the same size, will the estimator deliver the correct value
(i.e. the value of the parameter) on average?
– Consistency: As the sample size gets larger, does the probability that
the estimator deviates from the parameter by more than a ‘small’
amount become smaller?
– Relative efficiency: If there are two competing estimators of a
parameter, does the sampling distribution of one have less expected
dispersion than that of the other?
Properties of estimators …
Suppose θˆ is an estimator of θ . Then :
θˆ is an unbiased estimator of θ if E(θˆ) = θ

θˆ is a consistent estimator of θ if, for any small ε ,

lim
P(| θˆ − θ |< ε ) = 1
n→∞

~
If θˆ and some other estimator θ are both unbiased, then
~
θˆ is relatively efficient if var(θˆ) < var(θ )
Properties of estimators…
Recall our definition of the sample variance as
n n

∑(X i − X) 2
∑(X i − X )2
s2 = i =1
rather than σˆ 2 = i =1
n −1 n

E ( s 2 ) = σ 2 and thus s 2 is an unbiased estimator of σ 2

but
 n −1 2  n −1 2
E (σˆ 2 ) = E  s = σ ≠ σ 2 and thus σˆ 2 is a biased estimator of σ 2 .
 n  n
However, for large n the bias is small, and σˆ 2 is a consistent estimator of σ 2 .
Properties of estimators (Optional)
Why is 𝐸𝐸 𝑠𝑠 2 = 𝜎𝜎 2 and thus 𝑠𝑠 2 is an unbiased estimator of 𝜎𝜎 2 ? Remember that:

𝑉𝑉𝑉𝑉𝑉𝑉 𝑋𝑋 = 𝜎𝜎 2 = 𝐸𝐸 𝑋𝑋 2 − 𝜇𝜇2

𝑎𝑎𝑎𝑎𝑎𝑎
𝜎𝜎 2
𝑉𝑉𝑉𝑉𝑉𝑉 𝑥𝑥̅ = = 𝐸𝐸 𝑋𝑋� 2 − 𝜇𝜇2
𝑛𝑛

Thus we have:
2 ∑𝑛𝑛 ̄ 2
𝑖𝑖=1(𝑋𝑋𝑖𝑖 − 𝑋𝑋) ∑𝑛𝑛 2 ̄ �2
𝑖𝑖=1(𝑋𝑋𝑖𝑖 −2𝑋𝑋𝑖𝑖 𝑋𝑋+𝑋𝑋 )
𝐸𝐸(𝑠𝑠 ) = 𝐸𝐸 →E
𝑛𝑛−1 𝑛𝑛−1
1
𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 𝑜𝑜𝑜𝑜𝑜𝑜 𝑡𝑡𝑡𝑡𝑡 𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 𝑜𝑜𝑜𝑜 𝑤𝑤𝑤𝑤 ℎ𝑎𝑎𝑎𝑎𝑎𝑎:
𝑛𝑛 − 1
𝑛𝑛
1
E �(𝑋𝑋𝑖𝑖2 − 2𝑋𝑋𝑖𝑖 𝑋𝑋̄ + 𝑋𝑋� 2 )
𝑛𝑛 − 1
𝑖𝑖=1
Properties of estimators (Optional)
𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁𝑁 𝑡𝑡𝑡𝑡𝑡𝑡𝑡, 𝑤𝑤𝑤𝑤 ℎ𝑎𝑎𝑎𝑎𝑎𝑎:
𝑛𝑛 𝑛𝑛 𝑛𝑛 𝑛𝑛
1 1
E �(𝑋𝑋𝑖𝑖2 − 2𝑋𝑋𝑖𝑖 𝑋𝑋̄ + 𝑋𝑋� 2 ) → E �(𝑋𝑋𝑖𝑖2 ) − 2𝑋𝑋� � 𝑋𝑋𝑖𝑖 + �(𝑋𝑋� 2 )
𝑛𝑛 − 1 𝑛𝑛 − 1
𝑖𝑖=1 𝑖𝑖=1 𝑖𝑖=1 𝑖𝑖=1

𝑛𝑛

𝐴𝐴𝐴𝐴𝐴𝐴𝐴𝐴 𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛 𝑡𝑡𝑡𝑡𝑡𝑡𝑡 𝑋𝑋� 𝑖𝑖𝑖𝑖 𝑎𝑎 𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 𝑎𝑎𝑎𝑎𝑎𝑎 � 𝑋𝑋𝑖𝑖 = 𝑛𝑛𝑋𝑋� 𝑤𝑤𝑤𝑤 ℎ𝑎𝑎𝑎𝑎𝑎𝑎:
𝑖𝑖=1
𝑛𝑛 𝑛𝑛
1 1
� 𝑋𝑋� + 𝑛𝑛𝑋𝑋� 2
E �(𝑋𝑋𝑖𝑖2 ) − 2𝑋𝑋𝑛𝑛 → E �(𝑋𝑋𝑖𝑖2 ) − 𝑛𝑛𝑋𝑋� 2
𝑛𝑛 − 1 𝑛𝑛 − 1
𝑖𝑖=1 𝑖𝑖=1

This can be re-written as

𝑛𝑛
1
� 𝐸𝐸(𝑋𝑋𝑖𝑖2 ) − 𝑛𝑛𝐸𝐸(𝑋𝑋� 2 )
𝑛𝑛 − 1
𝑖𝑖=1
Properties of estimators (Optional)
Remember that:

𝑉𝑉𝑉𝑉𝑉𝑉 𝑋𝑋 = 𝜎𝜎 2 = 𝐸𝐸 𝑋𝑋 2 − 𝜇𝜇2

𝑎𝑎𝑎𝑎𝑎𝑎
𝜎𝜎 2
𝑉𝑉𝑉𝑉𝑉𝑉 𝑋𝑋� = = 𝐸𝐸 𝑋𝑋� 2 − 𝜇𝜇2
𝑛𝑛

Thus we have:
𝑛𝑛 𝑛𝑛
1 1 𝜎𝜎 2
2 � 2
� 𝐸𝐸(𝑋𝑋𝑖𝑖 ) − 𝑛𝑛𝐸𝐸(𝑋𝑋 ) → 2 2
� 𝜎𝜎 + 𝜇𝜇 − 𝑛𝑛 + 𝜇𝜇2
𝑛𝑛 − 1 𝑛𝑛 − 1 𝑛𝑛
𝑖𝑖=1 𝑖𝑖=1
Properties of estimators (Optional)
Simplifying we have:
1 2 2
𝜎𝜎 2 1 𝑛𝑛 − 1
𝑛𝑛(𝜎𝜎 + 𝜇𝜇 ) − 𝑛𝑛 + 𝜇𝜇2 → 𝑛𝑛𝜎𝜎 2 − 𝜎𝜎 2 → 𝜎𝜎 2 = 𝜎𝜎 2
𝑛𝑛 − 1 𝑛𝑛 𝑛𝑛 − 1 𝑛𝑛 − 1
Thus it is an unbiased estimator
Estimating the population proportion
 Recall our binomial random variable X:
– “Number of successes in n trials”
– We know that E(X)=np and Var(X)= np(1-p)
 Now consider the random variable X/n
– Imagine that we observe a particular value of this variable, x/n,
by actually doing n Bernoulli trials and counting the number of
successes we see, and dividing it with n.
– This is simply the sample proportion of successes (𝑝𝑝)̂
 This quantity has a sampling distribution! (It is an estimator!)
The sampling distribution of sample proportion
 Any one observed sample proportion is calculated from only one
possible sample that we could have taken.
 To learn more about the variability of the sample proportion, we
have to imagine how the sample proportion would vary across all
possible samples of the same size
 One way to get a sense of that is to simulate lots of samples of
the same size from a population we have pre-set to have a
particular population proportion (recall: that population proportion
is the target parameter)
The sampling distribution of sample proportion
 To give you a bit of context consider this:
 20% of the population are university graduates.
 If we go to George Street and choose 1,000 people as a sample
and calculate the proportion of university graduates in that
sample.
 And we keep repeating this over and over again lets say 10,000
times.
 We can record the proportions each time on a histogram. This
histogram is what we call the sampling distribution.
 In this case, we can create a histogram of 10,000 sample
proportions, each for a random sample of size 1000, using
p = 0.2 as the true proportion
A simulation of the sampling distribution of 𝒑𝒑
�
The sampling distribution of sample proportion…
 We have shown:
– X/n, when observed across many samples of size n and then put into a
histogram, shows us the sampling distribution of the sample proportion of
successes (𝑝𝑝)̂
– This estimator for the population proportion, p, is an unbiased estimator of
p: E(𝑝𝑝)̂ = p
– It also has variance of p(1-p)/n because:
 Var (X/n) = (1/n)2Var (X)
= (1/n2) × np (1-p) Recall:
E(X)=np
=np(1-p)/n2
Var(X)= np(1-p)
=p(1-p)/n
 How does this help us?

19
Estimating the population proportion…
 Recall that for large n, X is approximately normal (we showed how to
approximate the Binomial with the Normal)
 Thus, if our sampled values are independent, and our n is large enough,
we can conveniently conclude that:
X  p (1 − p ) 
If pˆ = then pˆ ~ N  p,  and
n  n 
pˆ − p
Z= ~ N (0,1)
p (1 − p ) / n

 This looks great in theory, but the whole point of trying to use a sample to
guess p is that WE DON’T KNOW P! Yet it appears in the formula
above!
 So what do we do?
But we don’t know p!
X  p (1 − p ) 
If pˆ = then pˆ ~ N  p,  and
n  n 
pˆ − p
Z= ~ N (0,1)
p (1 − p ) / n

 What DO we know? 𝑝𝑝. � And so we are reduced to calculating not the standard
deviation but the standard error of the sample proportion:
pˆ (1 − pˆ )
n
 As it turns out, this implies that in small samples, 𝑝𝑝� no longer follows a normal
distribution. (More on that next week.)
 We can still use the standard error to construct a confidence interval around our
point estimate of p (that is, 𝑝𝑝).
�