Let us consider the linear model again
yt=xt𝛽+𝜀t, t=1,2, . . . ,T, (5.1) or, in matrix notation,
y=X𝛽+𝜀. (5.2)
In Chapters 2 and 4 we saw that the OLS estimator b is unbiased for 𝛽 if it can be assumed that 𝜀 is mean zero and conditional mean independent of X, that is, if E{𝜀|X} =0 (assumption (A10) from Chapter 4). This says that knowing any of the explanatory variables is uninformative about the expected value of any of the error terms.
Independence ofXand𝜀withE{𝜀} =0 (assumptions (A1) and (A2) from Section 2.3) implies thatE{𝜀|X} =0 but is stronger, as it does not allow the variance of𝜀to depend uponXeither.
In many cases, the assumption that 𝜀is conditionally mean independent ofX is too strong. To illustrate this, let us start with a motivating example. The efficient market hypothesis (under constant expected returns) implies that the returns on any asset are unpredictable from any publicly available information. Under the so-called weak form of the efficient market hypothesis, asset returns cannot be predicted from their own past (see Fama, 1991). This hypothesis can be tested statistically using a regression model and testing whether lagged returns explain current returns. That is, in the model
yt=𝛽1+𝛽2yt−1+𝛽3yt−2+𝜀t, (5.3) where yt denotes the return in period t, the null hypothesis of weak form efficiency implies that 𝛽2=𝛽3 =0. Because the explanatory variables are lagged dependent variables (which are a function of lagged error terms), the assumptionE{𝜀|X} =0 is inappropriate. Nevertheless, we can make weaker assumptions under which the OLS estimator is consistent for𝛽= (𝛽1, 𝛽2, 𝛽3).
In the notation of the more general model (5.1), consider the following set of assumptions:
xtand𝜀tare independent(for eacht) (A8)
𝜀t∼IID(0, 𝜎2), (A11)
where the notation in (A11) is shorthand for saying that the error terms𝜀tare independent and identically distributed (i.i.d.) with mean zero and variance𝜎2. Under some additional regularity conditions,2 the OLS estimatorbis consistent for𝛽 and asymptotically nor- mally distributed (CAN) with covariance matrix𝜎2Σ−1xx, with
Σxx=plim
T→∞
1 T
∑T t=1
xtxt.
2We shall not present any proofs or derivations here. The interested reader is referred to more advanced text- books, like Hamilton (1994, Chapter 8). The most important ‘regularity condition’ is thatΣxxis finite and invertible (compare assumption (A6) from Section 2.6).
k k
A REVIEW OF THE PROPERTIES OF THE OLS ESTIMATOR 141
Formally it holds that √
T(b−𝛽)→N(0, 𝜎2Σ−1xx), (5.4) which corresponds to (2.74) from Chapter 2. In small samples, it thus holds approxi- mately that
b∼a N (
𝛽, 𝜎2 ( T
∑
t=1
xtxt )−1)
. (5.5)
This distributional result for the OLS estimator is the same as that obtained under the Gauss–Markov assumptions (A1)–(A4), combined with normality of the error terms in (A5), albeit that (5.5) only holds approximately by virtue of the asymptotic result in (5.4). This means thatall standard tests in the linear model(t-tests,F-tests and Wald tests) are valid by approximation, provided assumptions (A8) and (A11) are satisfied.
For the asymptotic distribution in (5.4) to be valid we have to assume thatxt and𝜀tare independent (for each t). This means that xs is allowed to depend upon 𝜀t as long as s≠t. The inclusion of a lagged dependent variable as in (5.3) is the most important example of such a situation. The current result shows that, as long as the error terms are independently and identically distributed, the presence of a lagged dependent variable in xt only affects the small sample properties of the OLS estimator but not the asymptotic distribution. Under assumptions (A6), (A8) and (A11), the OLS estimator is consistent, asymptotically normally distributed (CAN) and asymptotically efficient.
Assumption (A11) excludes autocorrelation and heteroskedasticity in 𝜀t. In the example above, autocorrelation can be excluded as it is a violation of market efficiency (returns should be unpredictable). The homoskedasticity assumption is more problem- atic. Heteroskedasticity may arise when the error term is more likely to take on extreme values for particular values of one or more of the regressors. In this case the variance of 𝜀tdepends uponxt. Similarly, shocks in financial time series are usually clustered over time, that is, big shocks are likely to be followed by big shocks, in either direction. An example of this is that, in periods of financial turbulence, it is hard to predict whether stock prices will go up or down, but it is clear that there is much more uncertainty in the market than in other periods. In this case, the variance of𝜀tdepends upon historical innovations𝜀t−1, 𝜀t−2, . . .. Such cases are referred to as conditional heteroskedasticity, or sometimes just as ARCH or GARCH, which are particular specifications to model this phenomenon.3
When assumption (A11) is dropped, it can no longer be claimed that𝜎2Σ−1xx is the appro- priate covariance matrix, nor that (5.5) holds by approximation. This means that routinely computed standard errors are incorrect. In general, however, consistency and asymptotic normality ofbare not affected. Moreover, asymptotically valid inferences can be made if we estimate the covariance matrix in a different way. Let us relax assumptions (A8) and (A11) to
E{xt𝜀t} =0 for eacht (A7)
𝜀tare serially uncorrelated with expectation zero. (A12)
3ARCH is short for AutoRegressive Conditional Heteroskedasticity, and GARCH is a Generalized form of that. We shall discuss this in more detail in Chapter 8.
k k Assumption (A7) imposes that xt is uncorrelated4 with 𝜀t, while (A12) allows for
heteroskedasticity in the error term, but excludes autocorrelation. Under some additional regularity conditions, it can be shown that the OLS estimatorbis consistent for𝛽and asymptotically normal according to
√T(b−𝛽)→N(0,Σ−1xxΣΣ−1xx), (5.6) where
Σ≡plim 1 T
∑T t=1
𝜀2txtxt.
In this case, the asymptotic covariance matrix can be estimated following the method of White (see Subsection 4.3.4), and
V̂{b} = ( T
∑
t=1
xtxt )−1∑T
t=1
e2txtxt ( T
∑
t=1
xtxt )−1
, (5.7)
whereetdenotes the OLS residual, is a consistent estimator for the true covariance matrix of the OLS estimator under assumptions (A6), (A7) and (A12). Consequently, all standard tests for the linear model are asymptotically valid in the presence of heteroskedasticity of unknown form if the test statistics are adjusted by replacing the standard estimate for the OLS covariance matrix with the heteroskedasticity-consistent estimate from (5.7).
Suppose one is interested in predictability of long-horizon returns, for example over a horizon of several years. In principle, tests of long-term predictability can be carried out along the same lines as short-term predictability tests. However, for horizons of 5 years, say, this would imply that only a limited number of 5-year returns can be analysed, even if the sample period covers several decades. Therefore, tests of predictability of long- horizon returns have typically tried to make more efficient use of the available information by using overlapping samples (compare Subsection 4.11.3); see Fama and French (1988) for an application. In this case, 5-year returns are computed over all periods of five con- secutive years. Ignoring second-order effects, the return over 5 years is simply the sum of five annual returns, so that the return over 1990–1994 partly overlaps with, for example, the returns over 1991–1995 and 1992–1996. Denoting the return in yeartasyt, the 5-year return over the yearsttot+4 is given byYt=∑4
j=0yt+j. To test the predictability of these 5-year returns, suppose we estimate a model that explainsYtfrom its value in the previous 5-year period(Yt−5)using data for every year, that is,
Yt=𝛿5+𝜃5Yt−5+𝜀t, t=1, . . . ,Tyears. (5.8) AllT annualobservations in the sample on5-yearreturns are regressed on a constant and the 5-year returnlagged 5 years. In this model the error term exhibitsautocorrelation because of the overlapping samples problem. In order to explain this issue, assume that the following model holds for annual returns
yt =𝛿1+𝜃1yt−1+ut, (5.9) where ut exhibits no autocorrelation. Under the null hypothesis that𝜃1 =0, it can be shown that 𝛿5=5𝛿1 and 𝜃5 =0, while 𝜀t=∑4
j=0ut+j. Consequently, the covariance
4Note thatE{xtzt} =cov{xt,zt}if eitherxtorzthas a zero mean (see Appendix B).
k k
CASES WHERE THE OLS ESTIMATOR CANNOT BE SAVED 143
between 𝜀t and 𝜀t−j is nonzero as long as j<5. From Chapter 4 we know that the presence of autocorrelation invalidates routinely computed standard errors, including those based on the heteroskedasticity-consistent covariance matrix in (5.7). However, if we can still assume thatE{𝜀t}=0, the regressors are contemporaneously uncorrelated with the error terms (condition (A7)), and the autocorrelation is zero after H periods, it can be shown that all results based on assumptions (A7) and (A12) hold true if the covariance matrix of the OLS estimator is estimated by the Newey–West (1987) estimator presented in Subsection 4.10.2. Then,
V̂∗{b} = ( T
∑
t=1
xtxt )−1
TS∗ ( T
∑
t=1
xtxt )−1
, (5.10)
where
S∗= 1 T
∑T t=1
e2txtxt+ 1 T
H−1∑
j=1
wj
∑T s=j+1
eses−j(xsxs−j +xs−jxs) (5.11) with wj=1−j∕H. Note that in the above example H equals 5. As a consequence, the standard tests from the linear model are asymptotically valid in the presence of heteroskedasticity and autocorrelation (up to a finite number of lags) if we replace the standard covariance matrix estimate with the heteroskedasticity- and autocorrelation- consistent estimate from (5.10).