RS – EC2 - Lecture 15

Lecture 15
Forecasting

Forecasting
• A shock is often used to describe an unexpected change in a variable
or in the value of the error terms at a particular time period.

• A shock is defined as the difference between expected (a forecast)

and what actually happened.

• One of the most important objectives in time series analysis is to

forecast its future values. It is the primary objective of ARIMA
modeling:

• Two types of forecasts.

- In sample (prediction): The expected value of the RV (in-sample),
given the estimates of the parameters.
- Out of sample (forecasting): The value of a future RV that is not
observed by the sample.

1
RS – EC2 - Lecture 15

Forecasting – Basic Concepts

• Any forecasts needs an information set, IT. This includes data,
models and/or assumptions available at time T. The forecasts will be
conditional on IT.

• The variable to forecast YT+l is a RV. It can be fully characterized by

a pdf.

• In general, it is difficult to get the pdf for the forecast. In practice,

we get a point estimate (the forecast) and a C.I.

• Notation:
- Forecast for T+l made at T: 𝑌 ,𝑌 | ,𝑌 𝑙 .

- T+l forecast error: 𝑒 𝑌 𝑌

- Mean squared error (MSE): 𝑀𝑆𝐸 𝑒 𝐸𝑌 𝑌 ]2

Forecasting – Basic Concepts

• To get a point estimate, YˆT   , we need a cost function to judge
various alternatives. This cost function is call loss function. Since we are
working with forecast, we work with a expected loss function.

• A popular loss functions is the MSE, which is quadratic and

symmetric. We can use asymmetric functions, for example, functions
that penalize positive errors more than negative errors.

• If we use the MSE as the cost function, we look for 𝑌 , which

minimizes it. That is
min E[eT2   ]  E[(YT   YˆT  ) 2 ]  E[YT   2YT YˆT   YˆT  ]
2 2

Then, f.o.c. implies:

E[2YT   2YˆT  ]  0  E[YT  ]  YˆT  .

2
RS – EC2 - Lecture 15

Forecasting – Basic Concepts

• The optimal point forecast under MSE is the (conditional) mean:

𝑌 𝐸𝑌 |𝐼

• Different loss functions lead to different optimal forecast. For

example, for the MAE, the optimal point forecast is the median.

• The computation of E[YT+l|IT] depends on the distribution of {εt}.

If {εt} ~ WN, then E[εT+l|IT] = 0, which greatly simplifies
computations, especially in the linear model.

• Then, for ARMA(p, q) stationary process (with a Wold

representation), the minimum MSE linear forecast (best linear
predictor) of YT+l, conditioning on IT is:
𝑌 θ Ψε Ψ ε ⋯

Forecasting Steps for ARMA Models

• Process:
- ARIMA model Yt  Yt 1  at


- Estimation ˆ (Estimate of  )
(Evaluation in-sample) 
Yˆt  ˆYt 1 (Predictio n)

- Forecast
(Evaluation out-of-sample) Yˆ t 1  ˆYˆ (Forecast)
t

3
RS – EC2 - Lecture 15

Forecasting From ARMA Models

• We observe the time series: IT = {Y1, Y2,…,YT}.
- At time T, we want to forecast: YT+1, YT+2,…,YT+l.
- T: The forecast origin.
- l: Forecast horizon
- 𝑌 𝑙 : l-step ahead forecast = Forecasted value YT+l

• Use the conditional expectation of YT+l, given the observed sample.

Yˆ    E Y
T Y , Y , , Y 
T  T T 1 1

Example: One-step ahead forecast: 𝑌 𝐸𝑌 |𝑌 , 𝑌 ,…,𝑌

• Forecast accuracy to be measured by MSE

 conditional expectation, best forecast. 7

Forecasting From ARMA Models

• The stationary ARMA model for Yt is
 p  B Y t   0   q  B a t
or
Yt   0  1Yt 1     pYt  p   t  1 t 1     q  t  q

• Assume that we have data Y1, Y2, ... , YT . We want to forecast YT+l.
Then,
YT   0  1YT 1   pYT  p  T   1T 1  qT q

• Considering the Wold representation:

 q B 
YT     0   B  t   0  t
 p B 
  0  T    1T   1  2 T    2     T  

4
RS – EC2 - Lecture 15

Forecasting From ARMA Models

• Taking the expectation of YT+l, we have

YˆT    E YT   YT , YT 1 ,  , Y1 
  T   1T 1  

 0, j0
where E  T  j YT ,  , Y1   
 T  j , j  0
• Then, we define the forecast error:
e T    Y T    YˆT  
 1
  T     1 T    1       1 T  1  
i0
i T i

• The expectation of the forecast error: E e T    0

Forecasting From ARMA Models

• The expectation of the forecast error: EeT   0
 the forecast in unbiased.

• The variance of the forecast error:

  1   1
Var eT    Var 
  i  T    i    2
  i2
 i 0  i0

• Example 1: One-step ahead forecast (l = 1).

YT 1   0   T 1  1 T   2  T 1  
YˆT 1   0  1 T   2  T 1  
e 1  Y  Yˆ 1  
T T 1 T T 1

Var eT 1   2

5
RS – EC2 - Lecture 15

Forecasting From ARMA Models

• Example 2: One-step ahead forecast (l = 2).
Y T  2   0   T  2   1 T  1   2  T  
YˆT 2    0   2  T  
eT 2   YT  2  YˆT 2    T  2  1 T  1
Var eT 2    2
1   
1
2

• Note: lim YˆT      0


lim Var eT     0  


• As we forecast into the future, the forecasts are not very interesting
(unconditional forecasts!). That is why ARMA (or ARIMA)
forecasting is useful only for short-term forecasting.

Forecasting From ARMA Model: C.I.

• A 100(1- )% prediction interval for YT+l (l-steps ahead) is

YˆT    z  / 2 Var eT  

 1
YˆT    z  / 2  
i0
i
2

• Example: 95% C.I. for the 2 step-ahead forecast

YˆT 2   1 . 96  1   12

• When computing prediction intervals from data, we substitute

estimates for parameters, giving approximate prediction intervals.
 1

i are RV, MSE[εT+l]=MSE[eT+l]=   i

2 2
Note: Since ̂ ’s
i0

6
RS – EC2 - Lecture 15

Forecasting From ARMA Model: Updating

• Suppose we have T observations at time t=T. We have a good
ARMA model for Yt. We obtain the forecast for YT+1, YT+2, etc.

• At t = T + 1, we observe YT+1. Now, we update our forecasts using

the original value of YT+1 and the forecasted value of it.
 1
• The forecast error is: eT    YT    YˆT     
i 0
i T   i

We can also write this as

eT 1   1  YT 1  1  YˆT 1   1
 
 
i0
 i  T 1    1  i  
i0
i T   i

 1
 
i0
i T   i     T  eT       T

Forecasting From ARMA Model: Updating

• Then,
YT    YˆT 1   1  YT   YˆT     T
Yˆ    Yˆ   1   
T T 1  T


YˆT    YˆT 1   1   YT  YˆT 1 1 

YˆT 1    YˆT   1   YT 1  YˆT 1 
• Example: l = 1, T = 100.
Yˆ101 1  Yˆ100 2   1 Y101  Yˆ100 1

7
RS – EC2 - Lecture 15

Forecasting From ARMA Model: Transformations

• If we use variance stabilizing transformation, after the forecasting,
we need to convert the forecasts for the original series.

• For example, if we use log-transformation, then,

  
E Y T   Y1 ,  , Y T  exp E ln Y T    ln Y1 ,  , ln Y T 
 2 
• If X ~ N(, 2), then, E exp  X   exp     .
 2 

• The MSE forecast for the original series is:

 1 
exp  Zˆ n    Var e n   where Z n    ln Yn   
 2 
  E Z n   Z1 , , Z n   2  Var Z n   Z1 , , Z n 

Forecasting From ARMA Model: Remarks

• In general, we need a large T. Better estimates and it is possible to
check for model stability and check forecasting ability of model by
withholding data.

• Seasonal patterns also need large T. Usually, you need 4 to 5 seasons

to get reasonable estimates.

• Parsimonious models are very important. Easier to compute and

interpret models and forecasts. Forecasts are less sensitive to
deviations between parameters and estimates.

8
RS – EC2 - Lecture 15

Forecasting From Simple Models: ES

• Industrial companies, with a lot of inputs and outputs, want quick
and inexpensive forecasts. Easy to fully automate.

• Exponential Smoothing Models (ES) fulfill these requirements.

• In general, these models are limited and not optimal, especially

compared with Box-Jenkins methods.

• Goal of these models: Suppress the short-run fluctuation by

smoothing the series. For this purpose, a weighted average of all
previous values works well.

• There are many ES models. We will go over the Simple Exponential

Smoothing (SES) and Holt-Winter’s Exponential Smoothing (HW
ES).

Forecasting From Simple Models: ES

• Observed time series: Y1, Y2, …, YT

• The equation for the model is S t   Yt 1  1   S t 1

where
- : the smoothing parameter, 0    1
- Yt: the value of the observation at time t
- St: the value of the smoothed observation at time t.

• The equation can also be written as

S t  S t 1   Yt 1  S t 1   S t 1   forecast erro r 

• Then, the forecast is: St 1  Yt  1 St  St  Yt  St 

That is, a simple updating equation.

9
RS – EC2 - Lecture 15

Forecasting From Simple Models: ES

• Q: Why Exponential?
For the observed time series Y1,Y2,…,Yn, Yn+1 can be expressed as a
weighted sum of previous observations.

Yˆt 1  c 0Yt  c1Yt 1  c 2Yt  2  

where ci’s are the weights.

• Giving more weights to the recent observations, we can use the

geometric weights (decreasing by a constant ratio for every unit
increase in lag):
 ci   1   i ; i  0,1,...; 0    1.

• Then, Yˆt 1  1   0 Yt  1   1Yt 1  1   2 Yt  2  

Yˆt 1  Yt  1   Yˆt 1 1  St 1  Yt  St 19

Forecasting From Simple Models: Selecting 

• Choose  between 0 and 1.
- If  = 1, it becomes a naive model; if  is close to 1, more weights
are put on recent values. The model fully utilizes forecast errors.
- If  is close to 0, distant values are given weights comparable to
recent values. Set  ≈ 0 when there are big random variations in Yt.
-  is often selected as to minimize the MSE.

• In empirical work, 0.05    0.3 are used ( ≈ 1 is used rarely).

Numerical Minimization Process:
- Take different  values ranging between 0 and 1.
- Calculate 1-step-ahead forecast errors for each .
- Calculate MSE for each case.
n
Choose  which has the min MSE: et  Yt  S t  min  et2 20
t 1

10
RS – EC2 - Lecture 15

Forecasting From Simple Models: Example

Time Yt St+1 (=0.10) (YtSt)2

1 5 - -
2 7 (0.1)5+(0.9)5=5 4
3 6 (0.1)7+(0.9)5=5.2 0.64
4 3 (0.1)6+(0.9)5.2=5.28 5.1984
5 4 (0.1)3+(0.9)5.28=5.052 1.107
TOTAL 10.945

SSE
MSE   2 . 74
n 1
• Calculate this for =0.2, 0.3,…, 0.9, 1 and compare the MSEs.
Choose  with minimum MSE 21

Forecasting From Simple Models: Remarks

• Some computer programs automatically chooses the optimal 

using the search method or non-linear optimization techniques.

• Initial Value Problem

– Set S2 to Y1 is one method of initialization.
– Also, take the average of, say first 4 or 5 observations. Use this
average as an initial value.

• This model ignores trends or seasonalities. Not very realistic. But,

deterministic components, Dt, can be easily incorporated. The model
that incorporates both features is called Holt-Winter’s ES.

22

11
RS – EC2 - Lecture 15

Forecasting From Simple Models: HW ES

• Now, we introduce trend (Tt) and seasonality (It) factors. Both can
can be included as additively or multiplicatively factors.
• Details
- We use multiplicative seasonalities –i.e., Yt/It– and additive trend.
- The forecast, St, is adjusted by the deterministic trend: St + Tt.
- The trend, Tt, is a weighted average of Tt-1 and the change in St.
- The seasonality is also a weighted average of It-S and the Yt/St
• Then, the model has three equations:
Y
S t   t  1  1    S t  1  T t  1 
Its
T t   S t  S t  1   1   T t  1
Yt
It    1   I t  s
St 23

Forecasting From Simple Models: HW ES

• We think of (Yt /St) as capturing seasonal effects.

s = # of periods in the seasonal cycles
(s = 4, for quarterly data)

• We have only three parameters :

 = smoothing parameter
 = trend coefficient
 = seasonality coefficient

• Q: How do we determine these 3 parameters?

- Ad-hoc method: α , and  can be chosen as value between 0.02< ,
, <0.2
- Minimization of the MSE, as in SES. 24

12
RS – EC2 - Lecture 15

Forecasting From Simple Models: HW ES

• h-step ahead forecast Yˆt h   S t  hT t I t  h  s

Note: Seasonal factor is multiplied in the h-step ahead forecast

• Initial values for algorithm

- We need at least one complete season of data to determine the
initial estimates of It-s.
- Initial values:
s
1.S 0   Yt / s
t 1

1 Y Y Y Y Y  Ys 
2.T0   s 1 1  s  2 2    s  s 
s s s s 
or T0     Yt / s     Yt / s  / s
s 2s

 t 1  t  s 1  25

Forecasting From Simple Models: HW ES

• Algorithm to compute initial values for seasonal component Is.

Assume we have T observation and quarterly seasonality (s=4):
(1) Compute the averages of each of T years.
4
At   Yt ,i / 4, t  1,2, ,6 ( yearly averages)
i 1

(2) Divide the observations by the appropriate yearly mean: Yt,i/At.

(3) Is is formed by computing the average Yt,i/At per year:

T
I s   Yt , s / At 4 s  1,2,3,4
i 1

26

13
RS – EC2 - Lecture 15

Forecasting From Simple Models: HW ES

• Remarks
- Note that, if a computer program selects = 0= , this does not a
lack of trend or seasonality. It implies a constant (deterministic)
component.
- In this case, an ARIMA model with deterministic trend may be a
more appropriate model.
- For HW ES, a seasonal weight near one implies that a non-seasonal
model may be more appropriate.
- We modeled seasonalities as multplicative:
=> Multiplicative seasonality: Forecastt = St * It-s.
- But, seasonal components can also be additive. For example, during
the month of December sales at a particular store increase by $X
every year. In this case, we just $X to the December forecast.
=> Additive seasonality: Forecastt = St + It-s.
27

ES Models – Different Types

1. No trend and additive 2. Additive seasonal variability with

seasonal variability (1,0) an additive trend (1,1)

3. Multiplicative seasonal variability 4. Multiplicative seasonal variability

with an additive trend (2,1) with a multiplicative trend (2,2)

14
RS – EC2 - Lecture 15

ES Models – Different Types

5. Dampened trend with additive 6. Multiplicative seasonal variability

seasonal variability (1,1) and dampened trend (2,2)

• Select the type of model to fit based on the presence of

- Trend – additive or multiplicative, dampened or not
- Seasonal variability – additive or multiplicative

Evaluation of forecasts
• Summary of loss functions of out-of-sample forecast accuracy:
T m T m
1 1
Mean Error =  ( yˆi  yi ) 
m i T 1 ei
m i T 1
T m T m
1 1
Mean Absolute Error (MAE) =  | yˆi  yi |
m i T 1 
| ei |
m i T 1
T m T m
1 1
Mean Squared Error (MSE) =  ( yˆi  yi )2 
m i T 1 
ei 2
m i T 1

T m T m
1 1
Root Mean Square Error (RMSE)=  ( yˆi  yi )2 
m iT 1
ei 2
m iT 1 
T m
1
 ei 2
m iT 1
U
Theil’s U-stat = 1
T

T y
i 1
i
2

15
RS – EC2 - Lecture 15

Evaluation of forecasts – DM Test

• To determine if one model predicts better than another, we define
the loss differential between two forecasts:
dt = g(etM1) – g(etM2).
where g(.) is the forecasting loss function. M1 and M2 are two
competing sets of forecasts –could be from models or something else.

• We only need {etM1} & {etM2}, not the structure of M1 or M2. In

this sense, this approach is “model-free.”

• Typical (symmetric) loss functions: g(et) = et2 & g(et) =|et|.

• But other g(.)’s can be used: g(et) = exp(λet2 ) – λet2 (λ>0).

Evaluation of forecasts – DM Test

• Then, we test the null hypotheses of equal predictive accuracy:
H0: E[dt] = 0 vs.
H1: E[dt] = μ ≠ 0.

- Diebold and Mariano (1995) assume {etM1} & {etM2} is covariance

stationarity and other regularity conditions (finite Var[dt ],
independence of forecasts after l periods) needed to apply CLT. Then,
d  1 T m

Var [ d ] / T

d
N ( 0 ,1), d 
m
d
i  T 1
i

• Then, the DM test is a simple z-test:

d
DM  
d
N ( 0 ,1)
ˆ
V ar [ d ] / T

16
RS – EC2 - Lecture 15

Evaluation of forecasts – DM Test

where Vˆar[d ] is a consistent estimator of the variance, usually based
on sample autocovariances of dt:
l
Vˆar [ d ]   ( 0 )  2   ( j )
jk
• Assume the l-step ahead forecast errors have zero autocorrelations at
order l. Harvey et al. (1998) propose a small-sample modification of
the DM test:
DM* = DM/{[T + 1 – 2l + l (l – 1)/T]/T}1/2 ~tT-1.

• If ARCH is suspected, replace l with [0.5 √(T)]+l. ([.]=integer part).

•Note: If {etM1} & {etM2} are perfectly correlated, the numerator and
denominator of the DM test are both converging to 0 as T → ∞.
 Avoid DM test when this situation is suspected (say, two
nested models.) Though, in small samples, it is OK.

Evaluation of forecasts – DM Test

Example: Code in R
dm.test <- function (e1, e2, h = 1, power = 2) {
d <- c(abs(e1))^power - c(abs(e2))^power
d.cov <- acf(d, na.action = na.omit, lag.max = h - 1, type = "covariance", plot = FALSE)$acf[, , 1]
d.var <- sum(c(d.cov[1], 2 * d.cov[-1]))/length(d)
dv <- d.var#max(1e-8,d.var)
if(dv > 0)
STATISTIC <- mean(d, na.rm = TRUE) / sqrt(dv)
else if(h==1)
stop("Variance of DM statistic is zero")
else
{
warning("Variance is negative, using horizon h=1")
return(dm.test(e1,e2,alternative,h=1,power))
}
n <- length(d)
k <- ((n + 1 - 2*h + (h/n) * (h-1))/n)^(1/2)
STATISTIC <- STATISTIC * k
names(STATISTIC) <- "DM"
}

17
RS – EC2 - Lecture 15

Evaluation of forecasts – DM Test: Remarks

• The DM tests is routinely used. Its “model-free” approach has
appeal. There are model-dependent tests, see West (1996), Clark and
McCracken (2001), and, more recent, Clark and McCracken (2011),
with more complicated asymptotic distributions.

• The loss function does not need to be symmetric (like MSE).

• The DM test is based on the notion of unconditional –i.e., on

average over the whole sample- expected loss.

• Following Morgan, Granger and Newbold (1977), the DM statistic

can be calculated by regression of dt, on an intercept, using NW SE.
But, we can also condition on variables that may explain dt. We move
from an unconditional to a conditional expected loss perspective.

Evaluation of forecasts – Conditional Test

18
RS – EC2 - Lecture 15

Combination of Forecasts
• Idea – from Bates & Granger (Operations Research Quarterly, 1969):
- We have different forecasts from R models:
YˆTM 1  , YˆTM 2
 ,.... YˆTMR  
• Q: Why not combine them?

YˆTComb     M 1 YˆTM 1     M 2 YˆTM 2

   ....   MR YˆTMR  

• Very common practice in economics, finance and politics, reported

by the press as “consensus forecast.” Usually, as a simple average.

• Q: Advantage? Lower forecast variance. Diversification argument.

Intuition: Individual forecasts are each based on partial information

sets (say, private information) or models. 37

Combination of Forecasts – Optimal Weights

• The variance of the forecasts is:
R R R
Var [YˆTComb  ]   (
j 1
Mj )
2
Var [YˆTMj  ]  2  
j 1 i  j 1
Mj  Mi Co var[YˆTMj  YˆTMi  ]

Note: Ideally, we would like to have negatively correlated forecasts.

• Assuming unbiased forecasts and uncorrelated errors,

R
Var [YˆTComb  ]   (
j 1
Mj ) 2  2j
R
Example: Simple average: ωj=1/R. Then, Var [YˆT
Comb
 ]  1 / R 2 
j 1
2
j.

• We can derived optimal weights –i,e., ωj’s that minimize the variance
of the forecast. Under the uncorrelated assumption:
R
 Mj *   j 2 
j 1
 j 2

19
RS – EC2 - Lecture 15

Combination of Forecasts – Optimal Weights

• Under the uncorrelated assumption:
R
 Mj *   j 2 
j 1
2
j

• The ωj*’s are inversely proportional to their variances.

• In general, forecasts are biased and correlated. The correlations will

appear in the above formula for the optimal weights. For the two
forecasts case:
 Mj *  (12  12 ) (12   22  212 )  (12  1  2 ) (12   22  21  2 )

• We do not observe the forecast variances and covariances, nor the

biases. We need a history of forecasts to estimate the optimal weights.

Combination of Forecasts: Regression Weights

• In general, forecasts are biased and correlated. The correlations will
appear in the above formula for the optimal weights. Ideally, we
would like to have negatively correlated forecasts.

• Granger and Ramanathan(1984) used a regression method to

combine forecasts.
- Regress the actual value on the forecasts. The estimated
coefficients are the weights.
y T     1 YˆTM 1     2 YˆTM 2
   ....   R YˆTMR     T  

• Should use a constrained regression

– Omit the constant
– Enforce non-negative coefficients.
40
– Constrain coefficients to sum to one

20
RS – EC2 - Lecture 15

Combination of Forecasts: Regression Weights

• Remarks:
- To get weights, we do not include a constant. Here, we are assuming
unbiased forecasts. If the forecasts are biased, we include a constant.
- To account for potential correlation of errors, Coulson and Robbins
(1993) suggests allowing for ARMA residuals or include yT+l-1 in the
regression.
- Time varying weights are also possible.

• Should weights matter? Two views:

- Simple averages outperform more complicated combination
techniques --Stock and Watson (1999) and Fildes and Ord (2002).
- Sampling variability may affect weight estimates to the extent that
the combination has a larger MSFE --Harvey and Newbold (2005).
- Bayesian techniques, using priors, may help in the latter situation.
41

Combination of Forecasts: Bayesian Weights

• In our discussion of model selection, we mentioned that the
BIC is consistent. That means, the probability that a model is true,
given the data is proportional to BIC:
P(Mj |data) α exp(-BICj/2).

• Based on this, we use the BIC of different models to derive weights.

This is a simplified form of Bayesian model averaging (BMA).

• Easy calculation of weights. Let BIC* be the smallest BIC among

the R models considered. Define ΔBICMj = BICMj ‐ BIC*.
Then,
 Mj *  exp(   BIC Mj / 2 )
R
 Mj   Mj * / 
j 1
Mj *
42

21
RS – EC2 - Lecture 15

Combination of Forecasts: Bayesian Weights

• Steps:
(1) Compute BIC for the R different models.
(2) Find best-fitting BIC*.
(3) Compute ΔBIC & exp(‐ΔBIC/2).
(4) Add up all values and re-normalize.

• BMA puts the most weight on the model with the smallest BIC.

• Some authors have suggested replacing BIC with AIC in the

weight formula –i.e., ωj α exp(-AICj/2).
- There is no clear theory for this formula. It is simple and works
well in practice.
- This method is called weighted AIC (WAIC). 43

Combination of Forecasts: Bayesian Weights

• Q: Does it make a difference the criteria used? Two situations:
(1) The selection criterion (AIC, BIC) are close for competing models.
Then, it is difficult to select one over the other.
- WAIC and BMA will produce similar weights.
(2) The selection criterion are different.
- WAIC and BMA will produce different weights.
- They will give zero weight if the difference is large, say, above 10.

Q: Which one to use?

- Not clear. WAIC works well in practice.

General finding: Simple averaging works well, but it is not optimal. A

combination beats the lowest criteria used. 44

22
RS – EC2 - Lecture 15

Forecasting: Final Comments

• Since Bates and Granger (1969) and Granger and Ramanathan
(1984), combination weights have generally been chosen to minimize
a symmetric, squared-error loss function.

• But, asymmetric loss functions can also be used. Elliot and

Timmermann (2004) allow for general loss functions (and
distributions). They find that the optimal weights depend on higher
order moments, such a skewness.

• It is also possible to forecast quantiles and combine them. Testing

of quantile forecasts can be based on the general approach of G&W
(2006). Giacomini and Komunjer (2005) present an application.

45

23