Exact Inference

Single Linear Hypothesis
Multiple Linear Hypothesis

Confidence Intervals and Regions
Hypothesis Testing and Confidence Intervals under

Normality
Walter Sosa-Escudero
Econ 507. Econometric Analysis. Spring 2009
December 11, 2008
Hypothesis Testing and Confidence Intervals under Normality

Introduction
Classical linear model:
1
Linearity: Y = X + u.
Strict exogeneity: E(u|X) = 0
No Multicollinearity: (X) = K.
No heteroskedasticity/ serial correlation: V (u|X) = 2 X.
0 X)1 X 0 Y
OLS Estimator: = (XP
2
2
Estimator of : S = 2i=1 e2i /(n K).
Now we are interested in testing hypothesis about the unknown

vector of coefficients , or in constructing confidence intervals
We do not have enough information to perform this task!

Why? Consider the following example.

We are interested in testing H0 : j = 0 (significance).
j is not observed. But we can produce an estimator j
Classical approach: if H0 is correct, and j is good estimator
of j , then j ' 0. So we should reject H0 is j is sufficiently
different from 0.
What do we mean by j being sufficiently different from 0 ?
j is by construction a random variable, so it has to be a
probabilistic statement.
We need the distribution of j .
Can we get it based on the model as it is so far?

Assumption 5: normality. u|X is normally distributed (it is a

vector, so this involves the multivariate normal).
Note that this together with the classical assumptions imply
u|X N (0, 2 In )
Remember that = + (X 0 X)1 X 0 u.Then
| X N , 2 (X 0 X)1

Hypothesis about single coefficients

H0 : j = j0 vs. HA : j 6= j0
Let ais denote de (i, s) element of (X 0 X)1 .
Then, when H0 is true j j0 N (0, 2 ajj ) so
j j0
N (0, 1)
zj p
2 ajj
Note that 2 ajj = V (j ).
Special case: j0 = 0 significance hypothesis.
The distribution of zk does not depend on X.
If 2 is observed, reject if zj lies outside an acceptance region.

The problem is that 2 is not observed. Define:

j j0
tj p
S 2 ajj
which is zk with 2 replaced by its unbiased estimator S 2 .
Result: Under assumptions 1 to 5 and when H0 holds,

tj t(n K).

Proof: First, two preliminary results:

If X N (0, 1) and Y 2 (m) and independent, then
X
p
tm
Y /m
If X N (0, Im ), then X 0 AX 2 ((A)).
Note that:
j j0
zj
zj
tj = p
=p
=q
e0 e/(nK)
S 2 ajj
S 2 / 2
2
We have already shown zj N (0, 1) so if we can show

e0 e/ 2 2 (n K) and independence, we are done.

2 numerator: Recall e = M u, so e0 e = u0 M M 0 u = u0 M u
since M is idempotent. Also, we have assumed
u|X N (0, 2 In ), then by our previous result
u0 M u/ 2 2 (n K) since (M ) = n K.
Independence: = + (X 0 X)1 X 0 u and e = M u are linear
functions of u, they are jointly normal given X. Now
e) = E[( )e0 ]
Cov(,
= E[(X 0 X)1 X 0 uu0 M ]
= (X 0 X)1 X 0 E(uu|X)M
= 2 (X 0 X)1 X 0 M = 0
using LIE and the spherical disturbances assumption.
Since zj depends solely on and the numerator only on e, then

the result follows.

Hypothesis about linear combinations of .
H0 : c0 r = 0 vs. HA : c0 r 6= 0, c <K , r <.

WLOG, supose K = 3 so,
Yi = 1 X1i + 2 X2i + 3 X3i + ui
i = 1, . . . , n
Consider the following hypotheses:

a) H0 : 2 = 3 , or H0 : 2 3 = 0. In this case
c = (0, 1, 1) and r = 0.
b) H0 : 2 + 3 = 1, so now c = (0, 1, 1) and r = 1.

To derive an appropiate test statistic note:

c0 r N (0, 2 c0 (X 0 X)c) N (0, 1)
So
c0 r
N (0, 1)
z=p
2 c0 (X 0 X)c)
And again, by the same argument as before, a feasible version is

c0 r
t= p
t(n K)
S 2 c0 (X 0 X)1 c)

As a simple exercise, the appropriate statistics for the cases

considered before are
a) c0 r = 2 3 and
d 1 , 2 ), so
2 c0 (X 0 X)1 c = V (2 ) + V (2 ) 2Cov(
t=
2 3
d 1 , 2 )
V (2 ) + V (2 ) 2Cov(
b) c0 r = 2 + 3 1, and
d 1 , 2 ), so
2 c0 (X 0 X)1 c = V (2 ) + V (2 ) + 2Cov(
t=
2 + 3 1
d 1 , 2 )
V (2 ) + V (2 ) + 2Cov(


H0 : R r = 0, R is a q K matrix with (R) = q, and r <q
Example. In our previous case consider the multiple hypothesis
H0 : 2 = 0 : 3 = 0
These are actually two joint hypothesis about the coefficient vector
. In this case

0 1 0
0
R=
r=
0 0 1
0
with q = 2. r is the number of restrictions.
What is the full row rankrequirement, (R) = q, asking for?

Consider the following test statistic:

1
(R r)0 / q
(R r)0 R(X 0 X)1 R0
F =
S2
Result: under all assumptions and when H0 is true,
F F (q, n K).
= 2 R(X 0 X)1 R0 . Then,
Intuition: Note that V (R)
2
0
1
= S R(X X) R0 , so
V (R)
1
F = (R r)0 V (R|X)
(R r)0 / q
F is actually checking how large R r is.

Proof: Again, let us start with two results

If X 2 (m) and Y 2 (n) and independent, then
(X/m)
F (m, n)
X/n
If X is an m random vector X N (0, ) with invertible,
then X 0 1 X 2 (m)
Reexpress the F statistic as follows:

1
(R r)0 R(X 0 X)1 R0
(R r) / q
W/r
=
F =
2
S
Q/(n K)

1
with W (R r)0 2 R(X 0 X)1 R0
(R r) and
0
2
Q = e e/ .
Now it is obvious what we have to do...

We already know e0 e/ 2 2 (n K).

Now, under H0 : R r = 0 and our assumptions
R r | X N (0, 2 R(X 0 X)1 R0 )
It is easy to check that R(X 0 X)1 R0 is invertible (why? do it!),
hence
W | X 2 (r)
Now the desired result follows since W is a function of and Q a
function of e, which are independent.

An Alternative Formulation: Restricted Least Squares

Consider H0 : R r = 0, and the following estimator that arises
from the optimization problem
s.t. RR = r
R = argmin SSR()
R is the restricted least squares estimator. Lets adopt the

following notation
eR = Y X 0 R , SSRR = e0R eR
SSRU = e0 e

Result:
F =
(SSRR SSRU ) / q
SSRU /(n K)
Intuition
SSRR SSRU 0. Why?
(SSRR SSRU ) is the loss in explanatory power by imposing
H0 as a restriction.
When H0 is true, the restriction should not matter so the loss
should be small.
Proof: see homework


Let t/2 be the 1 /2 quantile of a Students t RV with (n K)
df. By construction
j j
< t/2 = 1
P r t/2 < q
V (j )

q
q
P r j t/2 V (j ) < j < j + t/2 V (j ) = 1
Then
j t/2
V (j )
is an 1 confidence interval for j . Note that it contains all the

values of j for which H0 : j = 0 is accepted.

Exact Inference

Uploaded by

Copyright:

Available Formats

Exact Inference

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Exact Inference

Uploaded by

Copyright:

Available Formats

Single Linear Hypothesis

Multiple Linear Hypothesis

Hypothesis Testing and Confidence Intervals under

December 11, 2008

Hypothesis Testing and Confidence Intervals under Normality

Single Linear Hypothesis

Strict exogeneity: E(u|X) = 0

No heteroskedasticity/ serial correlation: V (u|X) = 2 X.

Now we are interested in testing hypothesis about the unknown

Hypothesis Testing and Confidence Intervals under Normality

Single Linear Hypothesis

Why? Consider the following example.

Hypothesis Testing and Confidence Intervals under Normality

Single Linear Hypothesis

Assumption 5: normality. u|X is normally distributed (it is a

Hypothesis Testing and Confidence Intervals under Normality

Single Linear Hypothesis

Hypothesis about single coefficients

Hypothesis Testing and Confidence Intervals under Normality

Single Linear Hypothesis

The problem is that 2 is not observed. Define:

Result: Under assumptions 1 to 5 and when H0 holds,

Hypothesis Testing and Confidence Intervals under Normality

Single Linear Hypothesis

Proof: First, two preliminary results:

We have already shown zj N (0, 1) so if we can show

Hypothesis Testing and Confidence Intervals under Normality

Single Linear Hypothesis

Since zj depends solely on and the numerator only on e, then

Hypothesis Testing and Confidence Intervals under Normality

Single Linear Hypothesis

Hypothesis about linear combinations of .

H0 : c0 r = 0 vs. HA : c0 r 6= 0, c <K , r <.

Consider the following hypotheses:

Hypothesis Testing and Confidence Intervals under Normality

Single Linear Hypothesis

To derive an appropiate test statistic note:

And again, by the same argument as before, a feasible version is

Hypothesis Testing and Confidence Intervals under Normality

Single Linear Hypothesis

As a simple exercise, the appropriate statistics for the cases

Hypothesis Testing and Confidence Intervals under Normality

Single Linear Hypothesis

Multiple Linear Hypothesis

Hypothesis Testing and Confidence Intervals under Normality

Single Linear Hypothesis

Consider the following test statistic:

F is actually checking how large R r is.

Hypothesis Testing and Confidence Intervals under Normality

Single Linear Hypothesis

Proof: Again, let us start with two results

Hypothesis Testing and Confidence Intervals under Normality

Single Linear Hypothesis

We already know e0 e/ 2 2 (n K).

Hypothesis Testing and Confidence Intervals under Normality

Single Linear Hypothesis

An Alternative Formulation: Restricted Least Squares

R is the restricted least squares estimator. Lets adopt the

Hypothesis Testing and Confidence Intervals under Normality