π Lecture 20 β Autocorrelation
π Overview: This lecture introduces the concept of autocorrelation (serial correlation), a violation of the Gauss-Markov assumption that error terms are independent. It explains what autocorrelation is, its common causes in time series and cross-sectional data, and the serious consequences it has for Ordinary Least Squares (OLS) estimation, particularly on standard errors and hypothesis testing.
ποΈ Topics Covered
The lecture defines autocorrelation and the autocorrelation coefficient (rho), introduces first-order serial correlation, and then lists nine possible causes including missing variables, inertia, incorrect functional form, and the cobweb phenomenon. It explains spatial autocorrelation as a special case and concludes with a detailed list of consequences such as biased standard errors, overestimated R-squared, and inefficient but unbiased estimators.
π Lecture Summary
What is Autocorrelation / Serial Correlation?
The Gauss-Markov assumptions for BLUE (Best Linear Unbiased Estimators) require that errors are normally distributed with zero mean and constant variance, and that errors are independent. Independence means one error term is not correlated with any other error term. A violation of this independence assumption is called serial correlation.
With serial correlation, error terms are correlated with one another. The error term of different time periods (usually adjacent) or different cross-sectional observations are correlated. This means if we know something about the error term of one observation, we know something about the error term of another observation. Errors associated in one time period carry over to future time periods (e.g., in lagged models). Serial correlation is usually associated with time series data, so we use the subscript t instead of i.
Error terms may be correlated more with nearby observations compared to distant observations: Ο(eβ, eββα΅’) > Ο(eβ, eβββ±Ό) where i < j. Here, Ο (rho) is the autocorrelation coefficient. In cases where serial correlation exists (in time or space), Ο(eβ, eββα΅’) β 0 where t β 0 in time series data, and Ο(eα΅’, eβ±Ό) β 0 where i β j in cross-sectional data.
First Order Serial Correlation
The autocorrelation coefficient Ο ranges from -1 β€ Ο β€ 1. One specific type is first order (linear) serial correlation, where the error in one time period is correlated with the error in the previous time period. For first order serial correlation: Ο(eβ, eβββ) β 0, where -1 β€ Ο β€ 1.
In dynamic models: eβ = Οeβββ + uβ, where uβ is called white noise and is independently and identically distributed (i.i.d.) with zero mean and constant variance.
π‘ Why this matters: The sign (positive or negative) and magnitude of rho tell us how errors are related over time.
Possible Causes of Autocorrelation
-
Missing Variables: The error term may include all variables not in the regression equation. A change in any unobserved variable in one time period may impact errors in different time periods, causing errors to follow patterns of unobserved variables. Example: Y depends on Xββ and Xββ, and Xββ is not included; or sales depend on unmodeled seasonal changes.
-
Inertia or Sluggishness: The speed of change in variables depends on time. Business cycles (GDP, prices) are an example. The speed of price change may depend on how far it is from equilibrium. This is common in time series data.
-
Incorrect Functional Form: Autocorrelation may occur if linear models are specified when non-linear models are required. Linear-in-variables or simple models used when a log form is needed is called a model specification error.
-
Cobweb Phenomenon / Reaction with Lag: When the dependent variable acts with a lag, autocorrelation may be observed. Examples: QΛ’β = a + bPβββ + eβ; Cβ = ayβββ + byβββ + eβ. Overproduction in one year may lead to underproduction the next year.
-
Lagged Relationship: The dependent variable may depend on its own previous value. Examples: Stock prices; Consumption Cβ = a + bCβββ + eβ.
-
Ratchet Effect: The tendency of people to be influenced by the previous (high, low, or best) level of a variable can cause this effect. Consumption changes quickly upward (when income rises) but does not come down easily if income declines, causing over- or underestimation.
-
Data Manipulation: Data manipulated by: averaging or smoothing, converting quarterly data to annual, converting monthly to quarterly, finding mid-points when faced with ranges. Sometimes this is needed due to measurement error (e.g., in monthly data), but a byproduct is autocorrelation.
-
Systematic Measurement Error: Measurement error in one time period may be carried forward. Errors in inventories, stock of capital, or asset value can all cause autocorrelation. Time series is a noisy proxy.
-
Variable Behavior: Electricity consumption at different hours of the day; similar temperature patterns may cause patterns in errors.
Spatial Autocorrelation
Spatial autocorrelation is a special cause in cross-sectional data. It follows the "First Law of Geography" by Waldo Tobler: "Everything is related to everything else but near things are more related than distant things."
Spatial autocorrelation is a correlation of a variable with itself through space, due to the systematic pattern of spatial distribution of a variable. Nearby areas are often more alike (positive spatial autocorrelation). Values in the sample do not remain independent; the occurrence of one event in an area makes it more likely in other areas. Areas with higher concentration of events have more impact on results (spatially clustered observations). Example: Due to high crime in Karachi, more police and rangers are deployed. The Karachi operation decreases crime in Karachi. Crime rate in nearby cities increases, even though they did not decrease police, etc.
Consequences of Autocorrelation
- Estimators remain unbiased and consistent.
- Estimators are no longer efficient.
- Standard Error of Estimate / variance of error is likely to be underestimated, resulting in an overestimated R-squared.
- Variances of estimators are biased.
- Forecasts are unbiased but inefficient (with larger variances).
- In case of positive autocorrelation, standard errors (of coefficients) are too small, resulting in overestimation of t-statistic (making variables appear more significant than they are).
- In case of negative autocorrelation, standard errors (of coefficients) are too large, resulting in underestimation of t-statistic (making significant variables appear insignificant).
π‘ Why this matters: The most dangerous consequence is on hypothesis testing β positive autocorrelation makes t-statistics artificially high, leading to false conclusions that variables are significant when they are not.
β Key Takeaways
Autocorrelation violates the Gauss-Markov assumption of independent errors, meaning error terms are correlated over time (or space). The first-order autoregressive model eβ = Οeβββ + uβ is the most common form, where rho measures the strength and direction of the correlation. Nine distinct causes include inertia, missing variables, incorrect functional form, and data manipulation. While OLS estimators remain unbiased and consistent under autocorrelation, they are no longer efficient, and most critically, standard errors are biased β underestimated in positive AC (inflating t-stats) and overestimated in negative AC (deflating t-stats). This makes standard hypothesis testing unreliable.
π§ Quick Revision Questions
- What is the key Gauss-Markov assumption violated by autocorrelation?
- Write the equation for a first-order autoregressive error process and define each term.
- If the Durbin-Watson statistic suggests positive autocorrelation, what is the likely effect on the estimated t-statistics?
- Explain the difference between "inertia" and "cobweb phenomenon" as causes of autocorrelation.
- What is spatial autocorrelation, and how does it differ from time-series autocorrelation?
π Lecture 21 β Detection of Autocorrelation
π Overview: This lecture covers methods for detecting autocorrelation (serial correlation) in regression residuals. It explains both graphical and formal statistical tests, including the AR(1) test, Durbin-Watson d test, Durbin h test, and LM test, which are essential for validating regression assumptions in time series data.
ποΈ Topics Covered
The lecture covers detection of autocorrelation through graphic methods and formal tests. Formal tests include AR(1) test with strictly exogenous regressors, AR(1) test without strictly exogenous regressors, Durbin Watson d Test with its assumptions and decision rules, Durbin h Test for models with lagged dependent variables, and the Simple LM Test. Each method is demonstrated with examples using Microsoft Excel.
π Lecture Summary
Graphic method
Looking at the graph, we may see patterns or trends in residuals with respect to time or with respect to previous values of the errors. When errors are plotted against time, a visible pattern indicates errors are not random. Graphs showing residuals plotted against past values of errors can also reveal patterns and trends, which is an indication of serial correlation.
Two types of patterns are shown: negative autocorrelation (residuals oscillate) and positive autocorrelation (residuals trend in same direction).
Detecting Autocorrelation by formal tests
AR(1) test with strictly exogenous regressors
This tests first order autocorrelation with exogenous regressors (regressors uncorrelated with errors).
Procedure:
- Run the regression (Y) on (X_1, X_2, ... X_K) and obtain residual (e_t)
- Run the regression (e_t) on (e_{t-1}) for (N = 2) to (n): (e_t = \rho e_{t-1} + u_t)
- Apply t-test (individual variable significance test) to test the hypothesis (H_0: \rho = 0)
π Definition β AR(1) test with strictly exogenous regressors: A formal test for first-order autocorrelation where the explanatory variables are assumed to be uncorrelated with the error term.
π Formula: (e_t = \rho e_{t-1} + u_t) β Residual at time t depends on previous residual times rho plus random error.
π Example:
Step 1: Run regression Y on X and obtain residuals.
Step 2: Run regression (e_t = a + \rho e_{t-1} + u_t)
Results using =linest(E52:E65,F52:F65,true,true):
- (\rho = 0.6494361), standard error = 0.2328061
- t-value = 0.6494/0.2328 = 2.789600395
- Critical t = T.INV.2T(0.05,13) = 2.160368656 Since calculated t (2.7896) > tabulated t (2.1604), reject (H_0) and conclude coefficient (\rho) is significant β there is a problem of autocorrelation.
π‘ Why this matters: This simple test quickly checks if residuals depend on their own past values, indicating autocorrelation.
AR(1) test without strictly exogenous regressors
This tests first order autocorrelation without exogenous regressors (where explanatory variables may be correlated to residuals).
Procedure:
- Run the regression (Y) on (X_1, X_2, ... X_K) and obtain residual (e_t)
- Run the regression (e_t) on (X_1, X_2, ... X_K) and (e_{t-1}) for (N = 2) to (n): (e_t = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \cdots + \beta_K X_K + \rho e_{t-1} + u_t)
- Apply t-test to test (H_0: \rho = 0)
π Example: Step 1: Run regression Y on X and obtain residuals. Step 2: Run regression (e_t = a + bX + \rho e_{t-1} + u_t) (rearrange columns to put (e_{t-1}) and X together). Results:
- (\rho = 0.645754025), standard error = 0.247399143
- t-value = 0.64575/0.247399 = 2.610170824
- Critical t = T.INV.2T(0.05,11) = 2.20098516 Since calculated t (2.6102) > tabulated t (2.2010), reject (H_0) β there is autocorrelation.
Durbin Watson d Test
This tests first order autocorrelation with exogenous regressors. It is not good when regressors are not exogenous or when the model contains a lagged dependent variable.
Assumptions:
- Regression model has an intercept
- Errors are generated by first order autoregressive scheme: (e_t = \rho e_{t-1} + u_t)
- Regression model does not have lagged dependent variable as regressor
- There are no missing observations
π Definition β Durbin Watson d statistic: A test statistic for detecting first-order autocorrelation in regression residuals.
π Formula: [ d = \frac{\sum_{t=2}^n (e_t - e_{t-1})^2}{\sum_{t=1}^n e_t^2} ] For large samples: (d = 2(1 - \hat{\rho})), where (\hat{\rho} = \frac{\sum e_t e_{t-1}}{\sum e_t^2})
Since (-1 \leq \rho \leq 1), d ranges from 0 to 4:
- If (\rho = 0), (d = 2) (no autocorrelation)
- If (\rho = -1) (negative), (d = 4)
- If (\rho = 1) (positive), (d = 0)
Decision Rules:
- Accept (H_0) if (d_U < d < 4 - d_U) (no autocorrelation)
- Reject (H_0) if (d < d_L) (positive autocorrelation) or if (d > 4 - d_L) (negative autocorrelation)
- Inconclusive if (d_L < d < d_U) or (4-d_U < d < 4-d_L)
General Rules:
- d lies between 0 and 4. As d is closer to 2, chances of autocorrelation decrease
- If (d < 2), may indicate positive autocorrelation; if (d > 2), may indicate negative autocorrelation
π Example: Data from AC.xlsx, regression Y on X, obtain residuals.
- (\sum e_t^2 = 451.6195) (SUMSQ(E51:E65))
- (\sum (e_t - e_{t-1})^2 = 307.333) (SUMXMY2(E52:E65,F52:F65))
- (d = 307.333 / 451.6195 = 0.680513)
- (d_L = 1.08) (from table, n=15, K=2)
- (d_U = 1.36) (from table, n=15, K=2) Since (d < d_L), reject (H_0) and conclude positive autocorrelation exists.
Durbin h Test
Procedure:
- Run the regression: (Y_t = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \cdots + \beta_K X_K + \gamma Y_{t-1} + u_t)
- Compute Durbin h:
π Formula: [ h = \hat{\rho} \sqrt{\frac{N}{1 - N \cdot Var(\gamma)}} ] where (\hat{\rho} = 1 - \frac{d}{2}) (provided sample is large)
Note: h is normally distributed with unit variance so conclusions may be formed by looking at the normal distribution table.
Problem: Cannot be computed if (N \cdot Var(\gamma) > 1).
Simple LM Test: Testing for AR(1) model
Procedure:
- Run the regression: (Y_t = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \cdots + \beta_K X_K + e_t)
- Obtain the residuals (e_t)
- Regress (e_t) on all explanatory variables and (e_{t-1})
- Compute (LM = (n - 1) R^2) (because we lose one observation so n-1)
- Test using Chi-square (0.05, 1) distribution
π Definition β LM Test (Lagrange Multiplier Test): A test for autocorrelation similar to AR(1) test without exogenous regressors, but uses LM instead of F-test.
β Key Takeaways
Students must remember that autocorrelation can be detected graphically by observing patterns in residuals plotted against time or lagged residuals. The Durbin-Watson d statistic ranges from 0 to 4, with d=2 indicating no autocorrelation, d close to 0 indicating positive autocorrelation, and d close to 4 indicating negative autocorrelation, but the test has inconclusive regions defined by (d_L) and (d_U) from tables. The AR(1) tests differ based on whether regressors are strictly exogenous; when regressors may be correlated with errors, additional explanatory variables must be included in the auxiliary regression. The Durbin h test is specifically designed for models containing lagged dependent variables. The LM test provides an alternative approach using Chi-square distribution. In all examples presented, the calculated test statistics exceeded critical values, leading to rejection of the null hypothesis of no autocorrelation.
π§ Quick Revision Questions
- What is the range of the Durbin-Watson d statistic, and what do values of 0, 2, and 4 indicate about autocorrelation?
- How does the AR(1) test with strictly exogenous regressors differ from the AR(1) test without strictly exogenous regressors in terms of the auxiliary regression?
- Under what condition can the Durbin h test not be computed, and what distribution is used to evaluate the test statistic?
- In the Durbin-Watson test example with n=15 and K=2, why was the null hypothesis rejected when d=0.68 and dL=1.08?
- What is the formula for the LM test statistic and what distribution is used to test significance?
π Lecture 22 β Treating Autocorrelation
π Overview: This lecture addresses the consequences of ignoring autocorrelation in regression models and provides remedial measures to correct it. The focus is on Generalized Least Squares (GLS) methods when the autocorrelation parameter Ο is known or unknown, including the Prais-Winsten transformation, Cochrane-Orcutt method, and Hildreth-Lu search procedure.
ποΈ Topics Covered
The lecture begins with the consequences of ignoring autocorrelation, including biased standard errors and inflated RΒ². It then covers remedial measures starting with Generalized Differencing when Ο is known, including special cases where Ο = +1 and Ο = β1. The Prais-Winsten transformation is introduced for small samples. Methods for when Ο is unknown follow: using the Durbin-Watson d statistic, the Cochrane-Orcutt iterative method with a detailed example, and the Hildreth-Lu search procedure. Each method includes worked examples and comparisons.
π Lecture Summary
Consequences of Ignoring Autocorrelation
If autocorrelation is present but ignored, several problems arise. Coefficients remain unbiased and consistent but are not efficient (not BLUE), even in large samples β similar to heteroskedasticity. Standard error estimates become inappropriate, leading to wrong inferences. The t-statistic is overestimated, making regression coefficients appear significant when they are not. RΒ² is inflated and residual variance is underestimated for positively correlated residuals (if X grows over time). Forecasts remain unbiased but have large variances.
Remedial Measures for Autocorrelation
Two situations exist: when Ο is known β use GLS (Generalized Least Squares); when Ο is not known β estimate Ο first, then apply GLS.
Generalized Differencing / Generalized Least Square
Consider a two-variable model:
Yβ = Ξ²β + Ξ²βXβ + eβ
Assume the error term follows an AR(1) scheme:
eβ = Ο eβββ + vβ
where β1 < Ο < 1, and vβ satisfies OLS assumptions. Ο can be known or estimated.
Procedure to transform the model:
- Write the regression with a one-period lag: Yβββ = Ξ²β + Ξ²βXβββ + eβββ
- Multiply by Ο: ΟYβββ = ΟΞ²β + ΟΞ²βXβββ + Οeβββ
- Subtract from the original equation: Yβ β ΟYβββ = Ξ²β(1 β Ο) + Ξ²β(Xβ β ΟXβββ) + (eβ β Οeβββ)
- This gives the transformed model:
Yβ* = Ξ²β* + Ξ²βXβ* + vβ
Where: π Yβ* = Yβ β ΟYβββ π Ξ²β* = Ξ²β(1 β Ο) π Xβ* = Xβ β ΟXβββ
π Example with Ο = 0.6: A dataset with 15 time periods (Y, X) is transformed using Ο=0.6 to create Y* and X*. OLS on the transformed model yields: Yβ* = 17.995 + 2.1554 Xβ*. The slope 2.1554 is Ξ²β directly. Since Ξ²β* = Ξ²β(1 β Ο), we compute Ξ²β = 17.995 / (1 β 0.6) = 44.9877.
Generalized Differencing: Special Cases
Special Case: Ο = +1 Substituting Ο = 1 into the differencing procedure gives:
βYβ = Ξ²ββXβ + βeβ
Where βYβ = Yβ β Yβββ and βXβ = Xβ β Xβββ. This equation is estimated without an intercept.
Special Case: Ο = β1 Substituting Ο = β1 gives:
Yβ* = 2Ξ²β + Ξ²βXβ* + vβ
Where Yβ* = Yβ + Yβββ and Xβ* = Xβ β Xβββ.
π‘ Why this matters: These special cases show that first-differencing (Ο=1) eliminates the intercept entirely, which can dramatically change model interpretation.
Prais-Winsten Transformation
Differencing causes loss of one observation (the first). In small samples, we can estimate the first observation as: π Yβ* = β(1 β ΟΒ²) Γ Yβ π Xβ* = β(1 β ΟΒ²) Γ Xβ
This is not necessary in large samples. The example dataset with 15 observations uses this transformation where relevant.
Removing Autocorrelation when Ο is Unknown
Method 1: Using Durbin-Watson d statistic The relationship between Ο and d is: π d β 2(1 β Ο) β Ο β 1 β d/2
Since 0 β€ d β€ 4, we get β1 β€ Ο β€ 1. For small samples, Theil-Nagar suggest: π ΟΜ = [NΒ²(1 β d/2) + KΒ²] / (NΒ² β KΒ²)
Where N is sample size and K is number of coefficients.
π Example: N = 14, K = 2, d = 0.6805. Using Theil-Nagar: ΟΜ = [14Β²(1 β 0.6805/2) + 2Β²] / [14Β² β 2Β²] = 0.667. This Ο is then used in the generalized difference model.
Method 2: Cochrane-Orcutt Method This iterative method uses residuals to estimate Ο. Steps:
- Round 1: Apply OLS, find residuals eβ, compute ΟΜ = Ξ£(eβeβββ) / Ξ£(eβΒ²)
- Round 2: Perform GLS using ΟΜ, obtain new residuals, compute new ΟΜ
- Round 3+: Continue until ΟΜ from two successive rounds is approximately equal
π Example: Data with 15 periods:
- Round 1: OLS gives intercept 39.27556, slope 2.715066. ΟΜ = 240.49056 / 406.83312 = 0.591128
- Round 2: Transform Y and X, OLS gives intercept 18.33661, slope 2.166349. ΟΜ = 108.2871 / 208.098 = 0.520366
- Round 3: Intercept 19.55304, slope 2.794671. Iteration continues until convergence.
Method 3: Hildreth-Lu Search Procedure Step 1: Choose a grid of possible Ο values between β1 and +1 (e.g., 0, 0.1, 0.2, β¦, 0.9, 1). Step 2: For each Ο value, estimate the generalized difference model and find the sum of squared residuals (SS). Step 3: The equation with minimum SS is the best.
If first grid gives 0.7, refine with a finer grid around 0.7 (e.g., 0.66, 0.67, β¦, 0.74) and repeat.
Problems: Time-consuming; grid must be chosen carefully to find global minimum SS, not just local minimum.
Comparison: Hildreth-Lu is more computer-time-intensive than Cochrane-Orcutt. Cochrane-Orcutt may converge to a local minimum and miss the global minimum SS.
β Key Takeaways
Ignoring autocorrelation leads to unbiased but inefficient coefficients, overestimated t-statistics, inflated RΒ², and wrong inferences. The primary remedial measure is Generalized Differencing (GLS), which transforms variables using Ο to produce a model with uncorrelated errors. When Ο is unknown, it can be estimated using the Durbin-Watson d statistic (via Theil-Nagar for small samples), the Cochrane-Orcutt iterative method (using residuals to refine Ο), or the Hildreth-Lu search procedure (grid search for minimum SS). For Ο=+1, the model reduces to first differences without an intercept; for Ο=β1, the intercept doubles. The Prais-Winsten transformation preserves the first observation in small samples.
π§ Quick Revision Questions
- What are four consequences of ignoring autocorrelation in a regression model?
- How does Generalized Differencing transform the model Yβ = Ξ²β + Ξ²βXβ + eβ under AR(1) errors?
- What is the transformed equation when Ο = +1, and why is there no intercept?
- How does the Cochrane-Orcutt method estimate Ο in each round, and when does the iteration stop?
- In the Hildreth-Lu search procedure, what criterion determines the best Ο value, and what is a key limitation of this method?