ECO601 — Midterm Summary (Lectures 1–22)

📘 Lecture 01 — Introducing Econometrics & types of data

📖 Overview: This lecture introduces the field of Business Econometrics, its definition, methodology, and importance for students of business and economics. It also covers the different types of data structures used in econometric analysis, providing foundational knowledge for the entire course.

🗂️ Topics Covered

The lecture covers the traditional and modern definitions of econometrics, the reasons for studying it, the step-by-step methodology of business econometrics, and the four main types of data structures: cross-sectional, time series, pooled cross sections, and panel/longitudinal data. It also lists data sources and common file types.

📝 Lecture Summary

What is Econometrics or Business Econometrics?

Traditional Perception: Econometrics is traditionally seen as the branch of economics concerned with using mathematical methods (especially statistics) to describe economic systems. It is a set of quantitative techniques for making economic decisions and testing hypotheses using real-world data (e.g., "Is the value of the US Dollar correlated to Oil Prices?"). The Economist's Dictionary defines it as setting up mathematical models of economic relationships, testing hypotheses, and estimating parameters to measure the influence of independent variables. Econometrics is the intersection of economics, mathematics, and statistics, adding empirical content to economic theory.

🔑 Definition — Econometrics (Traditional): The branch of economics concerned with using mathematical and statistical methods in describing, analyzing, estimating, and forecasting economic relationships.

Modern View: Econometrics is no longer limited to testing economic theory; it is now used in Finance, Marketing, Management, and Sociology. The advent of modern computers and software (like Stata) has enabled the estimation of more complex models, making computer programming an essential component. Econometrics now encompasses both quantitative and qualitative research.

🔑 Definition — Econometrics (Modern): Using the tools of mathematics, statistics, and computer sciences, Econometrics analyses quantitative or qualitative phenomena (from Economics or other disciplines), based on evolution and development of theory, by recording observations based on sampling, related by appropriate methods of inference.

The lecture provides a flow chart showing how Theory (from Economics, Management, etc.) and Mathematical/Statistical Tools (calculus, regression) are combined using Computer Software (Excel, Stata, SPSS) to form Econometrics.

Why should you study Econometrics?

Econometrics provides research tools for your subject. It offers empirical evidence for theoretical statements; without it, theories may have no value. The key phrase is: "Data never speaks for themselves; Econometrics makes Data speak." The process goes from Idea to Forecasting: An idea becomes a theory, then a functional form showing variable relationships, model specification with mathematical equations, data collection, and finally estimation, testing, and forecasting.

The Methodology of Business Econometrics

The methodology follows these steps:

• Creation of a statement of theory or hypothesis
• Collection of Data
• Model Specification
• Model Estimation
• Performing Diagnostic Tests
• Testing the Hypothesis
• Prediction or Forecasting

For example, knowing that quantity demanded depends on price, income, and prices of substitutes, we collect data, specify a model based on demand theory, estimate it using econometric techniques, perform diagnostic tests to check for problems, test hypotheses about estimator validity, and ultimately predict or forecast outcomes.

Structure of Data

Cross-Sectional Data: A sample of entities (individuals, firms, countries) at a given point in time.

🔑 Definition — Cross-Sectional Data: Sample of entities at a given point in time. 📌 Example: Monthly income of Ali (75,000), Faisal (42,000), Iqbal (33,000), Noreen (65,000) in 2014.

Time Series Data: Observations over time for a single entity.

🔑 Definition — Time Series Data: Observations over time. 📌 Example: Average monthly income of a person from 2010 (35,000) to 2014 (55,000).

💡 Why this matters: Time series data often follow a time trend, requiring special attention in analysis. Time series econometrics is now a separate subfield.

Pooled Data / Pooled Cross Sections: Combined cross sections from different years, where the entities sampled can change each year.

🔑 Definition — Pooled Data: Combined Cross Sections from different years. 📌 Example: Income of Ali and Iqbal in 2011, Salma and Kumail in 2012, Sultan and Lubna in 2013. Note that individuals change in different years.

Panel / Longitudinal Data: A time series for each cross section unit, where the same cross-sectional units are followed over time.

🔑 Definition — Panel Data: Time Series of each Cross Section, same cross sectional units are followed over time. 📌 Example: Exchange rate data for Indonesia, Pakistan, and Sri Lanka from 2008 to 2010. Note that individual entities (countries) do not change over time.

⭐ Key Takeaways

Econometrics is the intersection of economics, mathematics, and statistics, using these tools (now including computer science) to give empirical content to theories. Its modern definition encompasses both quantitative and qualitative analysis across many disciplines. The core methodology involves moving from a theory to data collection, model specification and estimation, diagnostic testing, hypothesis testing, and forecasting. The four fundamental data structures are cross-sectional (entities at one time), time series (one entity over time), pooled cross sections (different entities over time), and panel data (same entities over time).

🧠 Quick Revision Questions

1. What is the modern definition of Econometrics?
2. What are the key steps in the methodology of Business Econometrics?
3. What is the difference between cross-sectional data and time series data?
4. How does pooled data differ from panel (longitudinal) data?
5. State two reasons why a business student should study Econometrics.

📘 Lecture 02 — The Summation Notation

📖 Overview: This lecture introduces the summation operator (∑), which is fundamental for expressing and manipulating sums in econometrics. It covers basic summation notation, properties of the summation operator, common pitfalls, and applications including double summation and linear functions. Understanding summation algebra is critical for later topics in regression analysis and model estimation.

🗂️ Topics Covered

The lecture begins with the concept and notation of summation, then moves through five practice exercises to build familiarity. It then systematically presents three core properties of the summation operator, followed by a warning about invalid operations. The latter half covers an application proving useful expressions for sums of deviations, introduces double (nested) summation, and concludes with linear functions and their econometric applications including demand functions and various functional forms.

📝 Lecture Summary

The Summation Notation

The summation operator is heavily used in econometrics. This operator is used to show that we are summing up something e.g. an expression. The Greek letter ∑ (sigma) is used to indicate summation or addition. Usually ∑ is followed by an expression. Summation Notation is an effective and comprehensive way to describe a sum of terms.

For example, the sum of a column of variable X is given as: Sum of X = X₁ + X₂ + X₃ + X₄ + X₅ = $\sum_{i=1}^{5} X_i$

Where i is a subscript and changes from 1 to 5. In general we write summation of X as $\sum_{i=1}^{n} X_i$, where n is a finite number.

Another example: Consider the expression $\frac{2}{3} + \frac{3}{4} + \frac{4}{5} + \frac{5}{6} + \frac{6}{7}$. Let k = 2, then the expression can be written as $\sum_{k=2}^{6} \frac{k}{k+1}$. This is called expanding the summation expression.

Practice Question 2.1: Try expanding $\sum_{i=1}^{5} \frac{(i+1)^2}{i}$ and finding the value.

Practice Question 2.2: Try expanding $\sum_{j=1}^{3} \frac{(2j+1)^2}{10j^2}$ and finding the value.

Practice Question 2.3: Try expanding and finding the value of $\sum_{i=1}^{5} X^2$, where X assumes the values 5, 6, 7, 8, and 9.

Practice Question 2.4: Try to write the following in summation notation: 1 + 4 + 9 + 16 + 25 + 36 + 49 + 64 + 81 + 100.

Practice Question 2.5: Try to write the following in summation notation: $2 + \frac{3}{4} + \frac{4}{9} + \frac{5}{16} + \frac{6}{25} + \frac{7}{36}$.

Properties of the Summation Operator

Property 1 $\sum_{i=1}^{n} a_i = na$

For example, if a = 2 for all observations, then $\sum_{i=1}^{5} a_i = 2 + 2 + 2 + 2 + 2 = 10$. In fact it is five times 2 = 5 multiplied by 2 = na = 10.

IMPORTANT: We usually do not write subscript 'i' with a constant. Note that 'a' is a constant and all values of it are identical. When ∑ is multiplied by a constant we can write 'n' instead of ∑.

🔑 Definition — Property 1 (Sum of a Constant): The sum of a constant a taken n times equals n times that constant.

Property 2 $\sum_{i=1}^{n} kX_i = k \sum_{i=1}^{n} X_i$

Let k = 5. In a table with X values 1, 2, 3, 4, 5, the sum of 5X is 5 + 10 + 15 + 20 + 25 = 75 = $\sum_{i=1}^{5} 5X_i$. This can also be computed as 5 x 15 = 75 = $5 \sum_{i=1}^{5} X_i$.

🔑 Definition — Property 2 (Factoring Out a Constant): A constant value can be factored out of the summation operator.

📌 Example: Let k=5, X={1,2,3,4,5}. $\sum_{i=1}^{5} 5X_i$ = 75. Also, $5 \sum_{i=1}^{5} X_i$ = 5 × 15 = 75. Therefore both methods yield the same result.

Property 3 $\sum_{i=1}^{n} (X_i + Y_i) = \sum_{i=1}^{n} X_i + \sum_{i=1}^{n} Y_i$

For example, with X={1,2,3,4,5} and Y={5,12,18,22,27}, $\sum_{i=1}^{5} (X_i + Y_i)$ = 6 + 14 + 21 + 26 + 32 = 99. Which can also be computed as $\sum_{i=1}^{5} X_i + \sum_{i=1}^{5} Y_i$ = 15 + 84 = 99.

🔑 Definition — Property 3 (Sum of Sums): The sum of a sum equals the sum of the individual sums.

Extension: Combining Property 2 & 3 $\sum_{i=1}^{n} (aX_i + bY_i) = a \sum_{i=1}^{n} X_i + b \sum_{i=1}^{n} Y_i$

Also: $\sum_{i=1}^{n} (aX_i + b) = \sum_{i=1}^{n} (aX_i) + \sum_{i=1}^{n} b = a \sum_{i=1}^{n} X_i + nb$

What Can NOT be Done in the Summation Notation?

The summation algebra is not just identical to normal algebra. Some things that may seem obvious in normal algebra may not apply to summation algebra. Remember that the following expressions are NOT equal:

• $\sum_{i=1}^{n} (X_i / Y_i) \neq \sum_{i=1}^{n} X_i \div \sum_{i=1}^{n} Y_i$
• $\sum_{i=1}^{n} (X_i Y_i) \neq \sum_{i=1}^{n} X_i \cdot \sum_{i=1}^{n} Y_i$
• $\sum_{i=1}^{n} X_i^2 \neq (\sum_{i=1}^{n} X_i)^2$

💡 Why this matters: These common mistakes can lead to completely wrong calculations in econometric analysis.

Practice Question 2.6: Construct a table to prove the first and second inequality discussed above.

Application of Summation Algebra

Different forms of $\sum (X - \bar{X})(Y - \bar{Y})$

Subscripts ('i') are omitted for simplicity. The expression can be derived as:

$\sum (X - \bar{X})(Y - \bar{Y}) = \sum XY - \frac{(\sum X)(\sum Y)}{n}$

Also equals: $\sum XY - n\bar{X}\bar{Y}$

The full derivation shows: $\sum (X - \bar{X})(Y - \bar{Y}) = \sum XY - \bar{X}\sum Y - \bar{Y}\sum X + n\bar{X}\bar{Y}$ Leading to: $\sum XY - \frac{\sum X \sum Y}{n}$

Different forms of $\sum (X - \bar{X})^2$

$\sum (X - \bar{X})^2 = \sum X^2 - \frac{(\sum X)^2}{n}$

The full derivation: $\sum (X - \bar{X})^2 = \sum [X^2 + \bar{X}^2 - 2X\bar{X}]$ $= \sum X^2 + n\bar{X}^2 - 2\bar{X}\sum X$ $= \sum X^2 - \frac{(\sum X)^2}{n}$

Double Summation

Double Summation or nested summation also can be used.

📌 Example: $\sum_{i=1}^{3} \sum_{j=1}^{2} X_{ij} = X_{11} + X_{12} + X_{21} + X_{22} + X_{31} + X_{32}$

📌 Example: $\sum_{i=1}^{3} \sum_{j=1}^{2} X_i Y_j = X_1Y_1 + X_1Y_2 + X_2Y_1 + X_2Y_2 + X_3Y_1 + X_3Y_2$

Linear Functions

Most of you would be familiar with straight lines or linear functions. A variable may be a linear function of another if its plot produces a straight line. A linear function may be written as:

Y = a + bX

Where:

• a = intercept (the point where the line intersects the y-axis)
• b = slope, rate of change, derivative

As Y = a + bX:

• ∆Y = b∆X
• b = ∆Y/∆X = marginal effect

Linear function: A function whose graph forms a straight line OR for which the rate of change 'b' is constant. Linear functions can be with or without intercept. Assuming linear relationship makes the models easy to solve.

📌 Example: Given a table with X={1,2,3,4,5} and Y={7,9,11,13,15}:

• b = ∆Y/∆X = (9-7)/(2-1) = 2/1 = 2
• a = Y - bX = 7 - 2(1) = 5
• Therefore the equation is Y = 5 + 2X

Slope: b = ∆Y/∆X = 2/1 = 2 (with ∆Y=2 and ∆X=1)

Intercept: a = 5

Simple Examples of Linear Functions

Linear Demand Functions The Demand Function: $Q_d = f(P, Y, P_s, P_c, A)$

Where:

• $Q_d$ = Quantity Demanded
• P = Price
• Y = Income
• $P_s$ = Price of Substitute
• $P_c$ = Price of Complementary Good
• A = Advertisement Expenditure

Expression in terms of linear equation: $Q_d = a + bP + cY + dP_s + eP_c + fA$

Simple Demand Function: $Q_d = a + bP$, Ceteris Paribus

We estimate the parameters 'a' and 'b' from data. What do we expect? The sign of 'b' is negative for 'normal' goods, sign of b is positive for 'Giffen' goods.

Practice Question 2.7: Assume $Q_d = 50 - 2P$, Ceteris Paribus. Activity: Assume values of P (price) to be 1, 2, 3, 4, and 5. Compute $Q_d$ and plot the 'Demand Curve'.

Simple Examples of Using Linear Equations

Sometimes we can 'linearize' equations.

1. Simple linear regression (linear in variable functional form): $Y = \beta_0 + \beta_1 X$

   • Marginal effect = $\beta_1$
   • Elasticity = $\epsilon = \beta_1 (X/Y)$

2. Double log functional form: $lnY = \beta_0 + \beta_1 lnX$

   • Can be written as $Y^* = \beta_0 + \beta_1 X^$ where $Y^ = lnY$, $X^* = lnX$
   • Marginal effect = $\beta_2 (Y/X)$
   • Elasticity = $\epsilon = \beta_1$

3. Linear-Log functional form: $Y = \beta_0 + \beta_1 lnX$

   • Can be written as $Y = \beta_0 + \beta_1 X^$ where $X^ = lnX$
   • Marginal effect = $\beta_1 / X$
   • Elasticity = $\epsilon = \beta_1 / Y$

4. Log-Linear functional form: $lnY = \beta_0 + \beta_1 X$

   • Can be written as $Y^* = \beta_0 + \beta_1 X$ where $Y^* = lnY$
   • Marginal effect = $\beta_1 Y$
   • Elasticity = $\epsilon = \beta_1 X$

5. Cobb-Douglas Production Function: $Y = AL^\alpha K^\beta$

   • Taking log on both sides: $lnY = lnA + \alpha lnL + \beta lnK$
   • Can be written as $Y^* = a + \alpha L^* + \beta K^$ where $L^ = lnL$, $K^* = lnK$, $Y^* = lnY$
   • The equation is not linear but we can estimate it by transformation

⭐ Key Takeaways

The summation operator (∑) is essential for compactly representing sums in econometrics, with three critical properties: summing a constant yields n times that constant, constants can be factored out of summations, and the sum of a sum is the sum of individual sums. A common pitfall is assuming summation behaves like normal algebra — specifically, the sum of products does NOT equal the product of sums, and the sum of squares does NOT equal the square of the sum. The derived formulas for $\sum (X - \bar{X})(Y - \bar{Y})$ and $\sum (X - \bar{X})^2$ are foundational for computing variances and covariances in regression analysis. Finally, linear functions Y = a + bX provide the framework for understanding marginal effects (slope b) and can be extended to non-linear forms through transformation (e.g., log-linear, double-log, and Cobb-Douglas functions).

🧠 Quick Revision Questions

1. What is the value of $\sum_{i=1}^{n} k$ where k is a constant? Explain using Property 1.

2. Why is $\sum (X_i Y_i)$ NOT equal to $(\sum X_i)(\sum Y_i)$? Provide a simple numerical example to illustrate.

3. Derive the expression $\sum (X - \bar{X})^2$ in terms of $\sum X^2$ and $(\sum X)^2$.

4. In a linear function Y = a + bX, what does 'b' represent, and how is it calculated from two data points?

5. How can the non-linear Cobb-Douglas production function $Y = AL^\alpha K^\beta$ be transformed into a form that can be estimated using linear regression techniques?

📘 Lecture 03 — Quadratic Function

📖 Overview: This lecture introduces quadratic functions and their properties, then explores their practical applications in econometrics through inverted-U and U-shaped relationships. It also covers exponential and logarithmic functions, and provides a foundational introduction to simple derivatives and their rules, which are essential for calculating marginal functions and elasticities in economic analysis.

🗂️ Topics Covered

The lecture begins with the definition and properties of quadratic functions, including the role of coefficients and the shape of the parabola. It then presents several real-world econometric examples of inverted-U relationships (liquidity and profitability, competition and innovation, Kuznets curve, Calmfors–Driffill hypothesis) and U-shaped relationships (economic development and fertility, marginal and average cost curves). The second half introduces exponential and logarithmic functions, followed by a comprehensive section on simple derivatives, covering rules like the power rule, constant function rule, sum-difference rule, product rule, quotient rule, and chain rule, with applications to finding marginal functions and elasticity.

📝 Lecture Summary

Quadratic Function

A quadratic function is a function of the form ( f(x) = Y = aX^2 + bX + c ) where (a \neq 0). (a), (b), and (c) are called coefficients. Its graph forms a parabola, which has either a maxima or a minima. A line called the axis of symmetry divides the graph into two equal parts.

• Form: (Y = aX^2 + bX + c)
• When ‘a’ is positive, the graph concaves downward (U-shaped).
• When ‘a’ is negative, the graph concaves upward (inverted-U shaped).
• When ‘c’ is positive, the graph moves up; when ‘c’ is negative, the graph moves down.

🔑 Definition — Axis of Symmetry: A vertical line that divides the parabola into two mirror-image halves. For the function (Y = X^2) (where a=1, b=0, c=0), the axis of symmetry is x = 0.

📌 Examples:

1. (Y = 2X^2 + 3X + 10)
2. (Y = 3X^2 - 5X + 5)
3. (Y = 10X^2 + 2X)
4. (Y = 5X^2)

Quadratic Function in Econometrics

Quadratic functions are useful for modeling relationships that are not linear but have an initial increasing then decreasing (or vice-versa) pattern.

Inverted U relationships (a is negative)

• Liquidity and Profitability: Initially, an increase in liquidity (e.g., current ratio from below 1 to the 1-2 range) can increase profit because the firm can meet its obligations. However, excess liquidity (current ratio above 2) means funds are not invested productively, so further increases in liquidity negatively affect profit.
• Competition and Innovation: Initially, increased competition encourages innovation. But too much competition drives prices down to a break-even level, leaving firms with only normal profit and no incentive to innovate.
• Kuznets Curve: The hypothesis that as a country's income per capita increases, income inequality first rises naturally, then falls after a certain average income is attained.
• Calmfors–Driffill Hypothesis: A macroeconomic theory stating a non-linear (inverted-U) relationship between the degree of collective bargaining (measured by trade union size) and the level of unemployment. As union size increases from zero, unemployment rises, but then falls as unions wield their monopoly power.

U shaped quadratic relationships (a is positive)

• Economic Development and Fertility: Initially, as economic development occurs, fertility declines. However, with further development, countries may offer incentives for childbearing, reducing the cost, so fertility rates may start rising again.
• Marginal Cost and Average Cost Curves: In cost theory, both marginal cost (MC) and average cost (AC) curves are U-shaped. They decline with increased production initially, but after a certain point (minimum point), they start rising as production increases further. The MC curve intersects the AC curve at its minimum point.

💡 Why this matters: These examples show how a simple quadratic mathematical function can model complex, non-linear economic behaviors found in firm management, macroeconomics, and development economics.

Exponential & Logarithmic Functions

Exponential functions are functions where a constant base ‘a’ is raised to a variable exponent ‘x’.

• General Form: (Y = a^x) where (a > 0) and (a \neq 1).
• ‘a’ is the base and ‘x’ is the exponent.
• The base can be any value, including the Euler’s constant (e = 2.71828).
• Natural Logarithm: If (Y = e^x), then (log_e Y = \ln Y = x).

📐 Formula — Logarithmic Form: If (Y = a^x), then (\log_a Y = x).

📐 Formula — Continuous Compound Interest: (C = Pe^{rt}), where C is the compounded balance after t years, P is the principal amount, r is the annual rate of interest, and t is the number of years. This models exponential growth.

📐 Example — Logarithmic Equation for Elasticity: Equations of the type (\ln Y = \beta_0 + \beta_1 \ln X) directly provide the elasticity coefficient (\beta_1).

📐 Formula — Exponential Growth Example: (P(t) = 2e^{3t}).

Simple Derivative

The derivative is the instantaneous rate of change of a dependent variable due to a very small change in the independent variable. For a function (Y = f(x)):

• The average rate of change is (\frac{\Delta f}{\Delta x}).
• The derivative, (f'(x)) or (\frac{dy}{dx}), is the slope of the tangent line at a point.
• For normal comprehension, derivative, slope of a function, and marginal function (like MC as the derivative of TC) can be thought of as identical.

🔑 Definition — Derivative: ( \frac{dy}{dx} = y' = f'(x) = \lim_{\Delta x \to 0} \frac{f(x+\Delta x) - f(x)}{\Delta x} )

Important Notes on Notation:

• (\frac{dy}{dx}) is a single symbol for the derivative and does not mean 'dy' divided by 'dx'.

The Power Rule If (y = a x^n), then (\frac{dy}{dx} = a n x^{n-1}).

📌 Example 1: (y = 10x^3) (\frac{dy}{dx} = 10(3)x^{3-1} = 30x^2)

📌 Example 2: (y = 5x^2) (\frac{dy}{dx} = 5(2)x^{2-1} = 10x)

📌 Example 3: (y = \frac{10}{x} = 10x^{-2}) (\frac{dy}{dx} = 10(-2)x^{-2-1} = -20x^{-3} = \frac{-20}{x^3})

The Constant Function Rule If (y = k) where (k) is a constant, then (\frac{dy}{dx} = 0). This is because a constant does not change.

📌 Example: (y = 10), then (y' = 0).

Derivative of (y = x) If (y = x = 1 \cdot x^1), using the power rule: (\frac{dy}{dx} = 1(1)x^{1-1} = 1x^0 = 1).

The Sum-Difference Rule If (y = f(x) \pm g(x)), then (\frac{dy}{dx} = f'(x) \pm g'(x)).

📌 Example 1: (y = 10x^3 + 5x^2) (\frac{dy}{dx} = 10(3)x^{2} + 5(2)x^{1} = 30x^2 + 10x)

📌 Example 2: (y = 2x^3 - 3x^2 - 10x + 5) (\frac{dy}{dx} = 6x^2 - 6x - 10)

The Product Rule If (y = f(x) \cdot g(x)), then (\frac{dy}{dx} = g(x) \cdot f'(x) + f(x) \cdot g'(x)). The derivative of the product of two functions is equal to the second function times the derivative of the first plus the first function times the derivative of the second.

📌 Example: (y = (10 - x)(5 + x)), where (f(x) = 10-x) and (g(x) = 5+x). (\frac{dy}{dx} = (5+x)(-1) + (10-x)(1) = -5 - x + 10 - x = 5 - 2x)

The Quotient Rule If (y = \frac{f(x)}{g(x)}), then (\frac{dy}{dx} = \frac{g(x) \cdot f'(x) - f(x) \cdot g'(x)}{[g(x)]^2}).

📌 Example: (y = \frac{10-x}{5+x}). (\frac{dy}{dx} = \frac{(5+x)(-1) - (10-x)(1)}{(5+x)^2} = \frac{-5 - x - 10 + x}{(5+x)^2} = \frac{-15}{(5+x)^2})

The Chain Rule: functions involving different variables If (y = f(g(x))) where (z = g(x)), then (\frac{dy}{dx} = \frac{dy}{dz} \cdot \frac{dz}{dx}).

📌 Example: (y = (5x^2 + 2x + 10)^3). Let (z = 5x^2 + 2x + 10), then (\frac{dz}{dx} = 10x + 2). (y = z^3), then (\frac{dy}{dz} = 3z^2). Using the chain rule: (\frac{dy}{dx} = (3z^2)(10x+2) = 3(5x^2+2x+10)^2(10x+2)).

Some Application of Simple Derivatives

The derivative is the same as the marginal function or the rate of change.

• Finding Marginal Cost (MC): If Total Cost (TC) is (C = \frac{1}{3}Q^3 - 2Q^2 + 120Q + 1000), then MC is its derivative. (MC = \frac{dC}{dQ} = Q^2 - 4Q + 120)

• Applying the Chain Rule (Marginal Revenue Product): If Total Revenue (R) is a function of Quantity (Q), and Quantity is a function of Labor (L), then the marginal revenue product of labor is: (\frac{dR}{dL} = \frac{dR}{dQ} \cdot \frac{dQ}{dL} = MR \cdot MP_{PL} = MRP_L).

• Finding Price Elasticity of Demand: Given a demand function (Q_d = 100 - 2P). The derivative is (\frac{dQ}{dP} = -2). If Price (P=10), then (Q_d = 80). Formula: Price Elasticity of Demand (E_p = \frac{dQ}{dP} \cdot \frac{P}{Q} = (-2) \cdot \frac{10}{80} = -0.25). This means a one percent change in price leads to a 0.25 percent change in quantity demanded in the opposite direction.

Higher Order Derivatives

Derivatives of a derivative are called higher order derivatives.

• The second order derivative ((y'') or (\frac{d^2y}{dx^2})) shows the rate of change of the rate of change.
• Third order derivative ((y''')) is the derivative of the second derivative.

📌 Example: (y = 10x^3) (y' = 30x^2) (y'' = \frac{d}{dx}(30x^2) = 60x) (y''' = \frac{d}{dx}(60x) = 60)

⭐ Key Takeaways

1. A quadratic function (Y = aX^2 + bX + c) forms a parabola; a negative 'a' gives an inverted-U shape, and a positive 'a' gives a U-shape. This is central to modeling non-linear economic phenomena like the Kuznets curve and cost curves.
2. The derivative is the instantaneous rate of change and is synonymous with the slope of a function and the marginal function (e.g., MC from TC). This concept is foundational for optimization in economics.
3. The power rule (( \frac{d}{dx}(ax^n) = a n x^{n-1} )) is the most fundamental rule of differentiation and forms the basis for the constant and sum-difference rules.
4. Specialized rules like the product rule, quotient rule, and chain rule are essential for differentiating more complex functions, such as revenue functions or composite cost functions.
5. The derivative is a powerful tool for calculating elasticity, a key economic concept. For a linear demand function (Q = a - bP), elasticity is (E = \frac{dQ}{dP} \cdot \frac{P}{Q}).

🧠 Quick Revision Questions

1. What are the three coefficients in a quadratic function (Y = aX^2 + bX + c), and what does the sign of 'a' determine about the graph's shape?
2. Describe the "inverted-U" relationship between liquidity and profitability as explained in the lecture.
3. State the Power Rule for differentiation. Then, find the derivative of the function (y = 12x^4).
4. A firm's total cost function is (C = 2Q^2 + 50Q + 500). Using the sum-difference rule, find the marginal cost function.
5. If a demand function is (Q_d = 50 - 5P) and the price is 5, calculate the price elasticity of demand. Interpret the value.

📘 Lecture 04 — Multivariate Functions

📖 Overview: This lecture introduces multivariate functions — functions of more than one independent variable — and explains how to compute partial derivatives and second-order derivatives. It then extends optimization (maxima/minima) from single-variable to multivariate contexts, covering both first-order and second-order conditions, which are fundamental for econometric modeling and finding optimal solutions in business problems.

🗂️ Topics Covered

The lecture begins with the definition and examples of multivariate functions, then explains partial derivatives and their notation. It provides detailed examples of first-order partial differentiation, followed by second-order direct and cross partial derivatives. The concept of symmetry of cross partial derivatives (Young's theorem) is demonstrated. Finally, the lecture introduces simple optimization for single-variable functions, showing how to find maxima and minima using first and second derivative tests.

📝 Lecture Summary

Multivariate Functions

Functions of more than one variable are called multivariate functions. In econometrics, dependent variables typically depend on several independent variables. For example, Quantity Demanded is a function of Price, Income, Prices of other goods, and other variables: 푄ᵈ = 푓(푃, 퐼, 푃ₒ, 푂). Similarly, Profitability depends on liquidity, capital structure, government regulations, prices of raw material, etc.: 휋 = 푓(퐿푄, 퐶푆, 퐺푅, 푃푅).

💡 Why this matters: Most real-world economic and business relationships involve multiple influencing factors, so understanding multivariate functions is essential for building realistic models.

Partial Derivatives

Partial Derivatives measure the rate of change of the dependent variable with respect to a change in one independent variable, while all other independent variables are held constant. The mathematical symbol 휕 ("partial" or "del") denotes partial derivatives. The expression 휕푧/휕푥 is read as "partial derivative of z with respect to x." Alternative notations include 푍ₓ or 푍₁. For second-order derivatives: 휕²푧/휕푥² or 푍ₓₓ, 푍ₓᵧ, 푍₁₁, 푍₁₂.

🔑 Definition — Partial Derivative: The derivative of a multivariate function with respect to one variable, treating all other variables as constants.

Partial Differentiation: Method

To partially differentiate a function: (1) There are as many first-order partial derivatives as there are independent variables. (2) When differentiating with respect to one variable, treat all other variables as constants. (3) All usual differentiation rules apply. (4) Higher-order derivatives are of two types: Direct Partial Derivatives (differentiate twice with respect to the same variable) and Cross Partial Derivatives (differentiate with respect to one variable, then another). Cross partial derivatives are always equal (symmetry of second derivatives or equality of mixed partials).

Partial Differentiation: Examples

Example 1: 푍 = 푓(푥, 푦) = 2푥² + 3푦² + 5푥푦 + 20

Differentiating w.r.t. x (treat y as constant): 휕푧/휕푥 = 4푥 + 0 + 5푦(1) + 0 = 4푥 + 5푦

Here, 3푦² is a constant (derivative 0), and 5푦 is a constant coefficient multiplied by derivative of x (which is 1).

Differentiating w.r.t. y (treat x as constant): 휕푧/휕푦 = 0 + 6푦 + 5푥(1) + 0 = 6푦 + 5푥 = 5푥 + 6푦

Here, 2푥² is a constant, and 5푥 is a constant coefficient multiplied by derivative of y (which is 1).

📌 Example: For 푍 = 2푥² + 3푦² + 5푥푦 + 20, at point (푥=2, 푦=1): 휕푧/휕푥 = 4(2) + 5(1) = 8 + 5 = 13 휕푧/휕푦 = 5(2) + 6(1) = 10 + 6 = 16

Example 2: 푍 = 푓(푥, 푦) = 2푥²푦² + 5푥³푦⁴

휕푧/휕푥 = 푍ₓ = 2푦²·(2푥) + 5푦⁴·(3푥²) = 4푥푦² + 15푥²푦⁴

휕푧/휕푦 = 푍ᵧ = 2푥²·(2푦) + 5푥³·(4푦³) = 4푥²푦 + 20푥³푦³

📌 Example: For 푍 = 2푥²푦² + 5푥³푦⁴, at point (푥=1, 푦=2): 휕푧/휕푥 = 4(1)(2²) + 15(1²)(2⁴) = 4(1)(4) + 15(1)(16) = 16 + 240 = 256 휕푧/휕푦 = 4(1²)(2) + 20(1³)(2³) = 4(1)(2) + 20(1)(8) = 8 + 160 = 168

Second Order Direct Partial Derivatives

Using Example 1: 푍 = 2푥² + 3푦² + 5푥푦 + 20

First, 푍ₓ = 4푥 + 5푦. Differentiating again w.r.t. x: 푍ₓₓ = 4(1) + 0 = 4

First, 푍ᵧ = 5푥 + 6푦. Differentiating again w.r.t. y: 푍ᵧᵧ = 0 + 6(1) = 6

These are called Second Order DIRECT Partial Derivatives.

Using Example 2: 푍 = 2푥²푦² + 5푥³푦⁴

푍ₓ = 4푥푦² + 15푥²푦⁴. Differentiating again w.r.t. x: 푍ₓₓ = 4푦² + 30푥푦⁴

푍ᵧ = 4푥²푦 + 20푥³푦³. Differentiating again w.r.t. y: 푍ᵧᵧ = 4푥² + 60푥³푦²

Second Order Cross Partial Derivatives

Using Example 1: Starting with 푍ₓ = 4푥 + 5푦, differentiate w.r.t. y: 푍ₓᵧ = 0 + 5(1) = 5

Starting with 푍ᵧ = 5푥 + 6푦, differentiate w.r.t. x: 푍ᵧₓ = 5(1) + 0 = 5

Note: 푍ₓᵧ = 푍ᵧₓ = 5

Using Example 2: Starting with 푍ₓ = 4푥푦² + 15푥²푦⁴, differentiate w.r.t. y: 푍ₓᵧ = 8푥푦 + 60푥²푦³

Starting with 푍ᵧ = 4푥²푦 + 20푥³푦³, differentiate w.r.t. x: 푍ᵧₓ = 8푥푦 + 60푥²푦³

Again, 푍ₓᵧ = 푍ᵧₓ = 8푥푦 + 60푥²푦³

📐 Formula — Equality of Mixed Partials: If 푍 = 푓(푥, 푦) has continuous second-order partial derivatives, then 푍ₓᵧ = 푍ᵧₓ.

Example with Chain Rule and Summation Algebra

Let 푍 = Σ(푦 − 푎 − 푏푥)². Here 'a' and 'b' act as unknowns (variables).

Differentiating w.r.t. 'a': 푍ₐ = 2Σ(푦 − 푎 − 푏푥)(0 − 1 − 0) = −2Σ(푦 − 푎 − 푏푥)

Using summation rules: 푍ₐ = −2(Σ푦 − 푛푎 − 푏Σ푥)

Differentiating w.r.t. 'b': 푍ᵦ = 2Σ(푦 − 푎 − 푏푥)(0 − 0 − 푥(1)) = −2Σ푥(푦 − 푎 − 푏푥)

Simple Optimization: Maxima and Minima

To optimize a function of one variable: (1) First Order Condition: Set the first derivative equal to zero — at minima or maxima, the slope (derivative) is zero. (2) Second Order Condition: The second derivative is positive for a minimum and negative for a maximum.

🔑 Definition — First Order Condition (FOC) for optimization: 푓'(푥) = 0 🔑 Definition — Second Order Condition (SOC) for optimization: 푓''(푥) > 0 for minimization; 푓''(푥) < 0 for maximization.

📌 Example: If 푦 = 40푥 − 2푥²

FOC: 푑푦/푑푥 = 40 − 4푥 = 0 → 4푥 = 40 → 푥 = 10

SOC: 푑²푦/푑푥² = −4 < 0, so the function is maximized at 푥 = 10.

Maximum value: 푦ₘₐₓ = 40(10) − 2(10)² = 400 − 200 = 200

📌 Example: Profit function π = 100Q − 120 − 2Q²

FOC: π' = 100 − 4Q = 0 → 4Q = 100 → Q* = 25

SOC: π'' = 푑(100 − 4Q)/푑Q = −4 < 0, so profit is maximized at Q = 25.

Maximum profit: πₘₐₓ = 100(25) − 120 − 2(25)² = 2500 − 120 − 1250 = 1130

⭐ Key Takeaways

The most critical concepts from this lecture are: (1) Partial derivatives measure the rate of change of a multivariate function with respect to one variable while holding all others constant — treat other variables as constants during differentiation. (2) There are two types of second-order partial derivatives: direct (same variable twice) and cross (different variables); importantly, cross partial derivatives are always equal (푍ₓᵧ = 푍ᵧₓ). (3) For optimization of single-variable functions, the first derivative must equal zero (FOC), and the second derivative determines whether it's a maximum (negative) or minimum (positive). (4) Chain rule and summation algebra apply to partial derivatives, as shown in the Σ(푦 − 푎 − 푏푥)² example. These tools are foundational for econometric estimation (e.g., Ordinary Least Squares) and business optimization problems.

🧠 Quick Revision Questions

1. Explain the key difference between an ordinary derivative and a partial derivative. How do you treat other variables when taking a partial derivative?

2. For the function 푍 = 3푥³푦² + 2푥푦³ + 10, find both first-order partial derivatives (푍ₓ and 푍ᵧ) and all four second-order partial derivatives (푍ₓₓ, 푍ᵧᵧ, 푍ₓᵧ, 푍ᵧₓ). Verify the equality of mixed partials.

3. If 푍 = Σ(푦 − 푎 − 푏푥)², derive the expressions for 푍ₐ and 푍ᵦ. Show the steps using chain rule and summation algebra.

4. Given the profit function π = 80Q − 100 − 3Q², use the first-order and second-order conditions to determine the output level Q that maximizes profit, and calculate the maximum profit value.

5. A company's cost function is 퐶(푄) = 500 + 20푄 + 0.5푄² and revenue function is 푅(푄) = 100푄 − 푄². Write the profit function π(푄), find the profit-maximizing output, and verify the second-order condition.

📘 Lecture 5 — Multivariate Optimization

📖 Overview: This lecture extends optimization from single-variable to multi-variable functions, teaching how to find local minima, maxima, and saddle points for functions with two independent variables. It then transitions into a comprehensive review of probability theory, covering fundamental concepts and important probability distributions essential for econometric analysis.

🗂️ Topics Covered

The lecture begins with multivariate optimization, explaining local minimum and maximum conditions through first and second order derivatives, followed by saddle points and the third condition using the Hessian determinant to rule out inflection points. Two worked examples demonstrate maximization and minimization of economic functions. The second half provides a thorough review of probability, covering random experiments, sample spaces, events, random variables, probability distributions, and concludes with the four major continuous probability distributions: Normal, t, Chi-square, and F-distributions.

📝 Lecture Summary

Local Minimum

Consider the following diagram: Point 'O' is a local minimum FROM ALL DIRECTIONS. At point 'O', the derivative of z w.r.t. x OR w.r.t. y both are zero — the slope of the tangents parallel to x-axis and y-axis at point 'O' are both zero, i.e. Zₓ = 0 AND Zᵧ = 0. This is the First Order Condition for a local minimum.

🔑 Definition — Local Minimum: A point where the function value is lower than all neighboring points in every direction. 📐 Formula: First Order Condition: Zₓ = 0 AND Zᵧ = 0 → At the optimum point, both partial derivatives equal zero. 📌 Example: If z = x² + y², then Zₓ = 2x = 0 gives x = 0, and Zᵧ = 2y = 0 gives y = 0, so (0,0) is a candidate for local minimum.

Local Minimum: Second Order Condition

Now consider point 'O' again. When we move the tangents parallel to x-axis or y-axis, there is a positive change in the derivative (derivative of the derivative is positive): Zₓₓ > 0 AND Zᵧᵧ > 0. This gives the Second Order Condition for minimization — both second order derivatives must be positive.

🔑 Definition — Second Order Condition (Minimum): Both second-order partial derivatives must be positive at the critical point, confirming the function curves upward in all principal directions. 📐 Formula: Zₓₓ > 0 AND Zᵧᵧ > 0 → The function is convex in both x and y directions.

Local Maximum

Consider the following diagram: Point 'a' is a local maximum FROM ALL DIRECTIONS. At point 'a', derivative of z w.r.t. x OR w.r.t. y both are zero — the slope of the tangents parallel to x-axis or y-axis at point 'a' are both zero, i.e. Zₓ = 0 AND Zᵧ = 0. This is the First Order Condition for a local maximum.

🔑 Definition — Local Maximum: A point where the function value is higher than all neighboring points in every direction. 📐 Formula: First Order Condition: Zₓ = 0 AND Zᵧ = 0 → Same first-order condition as minimum, but second-order condition differs.

Local Maximum: Second Order Condition

Now consider point 'a' again. When we move the tangents parallel to x-axis or y-axis, there is a negative change in the derivative (derivative of the derivative is negative): Zₓₓ < 0 AND Zᵧᵧ < 0. This gives the Second Order Condition for maximization — both second order derivatives must be negative.

📐 Formula: Zₓₓ < 0 AND Zᵧᵧ < 0 → The function is concave in both x and y directions.

Saddle Point: Second order derivatives have different signs

Consider point 'O' in the following diagram: A tangent at this point has a zero slope (first derivative is zero, so the first condition is met). If we shift the tangent in the direction of the x-axis, the slope of the tangent increases, so this is a local minima form one direction (x-axis): Zₓₓ > 0. But if we shift the tangent at point 'O' in the direction of the y-axis, its slope will decrease: Zᵧᵧ < 0. This is a saddle point.

🔑 Definition — Saddle Point: A critical point where the first-order condition is met, but the second-order partial derivatives have opposite signs, making it a minimum in one direction and a maximum in another. 📐 Condition: Zₓₓ > 0 AND Zᵧᵧ < 0 (or vice versa) → different signs for the two second-order partials. 💡 Why this matters: Saddle points can mislead optimization algorithms — they satisfy the first-order condition but are not true optima.

A third condition: ruling out point of inflection

When evaluated at the critical point(s), the product of the second order partials must exceed the product of the cross partials. This condition rules out critical points that are neither points of maximum or minimum, but are points of inflection. A point of inflection is where certain conditions of optima are met, but the function is not actually a maximum or minimum.

📐 Formula: Zₓₓ · Zᵧᵧ > (Zₓᵧ)²

This can be written as a determinant: |Zₓₓ Zₓᵧ| |Zᵧₓ Zᵧᵧ| > 0

We call this a Hessian determinant or simply Hessian, which shows that Zₓₓ · Zᵧᵧ - (Zₓᵧ)² > 0, or Zₓₓ · Zᵧᵧ > (Zₓᵧ)².

🔑 Definition — Hessian Determinant: A determinant of second-order partial derivatives that tests whether a critical point is a true maximum, minimum, or inflection point.

Example: Maximization

Consider the following profit function where x and y are the levels of output: π = 80x − 2x² − xy − 3y² + 100y

Step 1: First Order Condition πₓ = 80 − 4x − y = 0 πᵧ = −x − 6y + 100 = 0

Solving simultaneously gives: x̄ = 16.52 and ȳ = 13.91

Step 2: Second Order Condition πₓₓ = −4 < 0 πᵧᵧ = −6 < 0 Both negative, confirming profit is maximized from the principal directions at the critical points.

Step 3: Third Condition (Hessian) πₓₓ · πᵧᵧ = (−4)(−6) = 24 AND (πₓᵧ)² = (−1)² = 1 Hence 24 > 1, so πₓₓ · πᵧᵧ > (πₓᵧ)²

The profit function is maximized from all directions at the critical point. Maximum profit can be found by substituting the critical points into the profit function.

Example: Minimization

Consider the following marginal cost function where x and y are the level of output: MC = 5x² − 8x − 2xy − 6y + 4y² + 100y

Step 1: First Order Condition MCₓ = 10x − 8 − 2y = 0 MCᵧ = −2x − 6 + 8y = 0

Solving simultaneously gives: x̄ = 1 and ȳ = 1

Step 2: Second Order Condition MCₓₓ = 10 > 0 MCᵧᵧ = 8 > 0 Both positive, confirming MC is minimized from the principal directions at the critical points.

Step 3: Third Condition (Hessian) MCₓₓ · MCᵧᵧ = (10)(8) = 80 AND (MCₓᵧ)² = (−2)² = 4 Hence 80 > 4, so MCₓₓ · MCᵧᵧ > (MCₓᵧ)²

The function is minimized from all directions at the critical point. Minimum MC can be found by substituting the critical points into the MC function.

Review of Probability

Probability: This is only a 'Review'

Random Experiment

Any process of observation or measurement that has more than one possible outcome and we are not certain about which outcome will materialize.

🔑 Definition — Random Experiment: An experiment where all possible outcomes are known, but the specific outcome of any single trial cannot be predicted with certainty. 📌 Examples: Tossing a coin, throwing a pair of dice, drawing a card from a deck of cards.

Sample Space/Population

The set of all possible outcomes of an experiment.

🔑 Definition — Sample Space: The complete set of all possible outcomes of a random experiment. 📌 Example: When you toss a coin, S = {H, T} 📌 Example: When you toss two coins, S = {HH, HT, TH, TT}

Sample Point

Each member of the sample space is a sample point.

Event

An event is a particular collection of outcomes — a subset of the sample space. 📌 Example: Event 'A' is occurrence of one head and one tail in the experiment of tossing two coins: A = {HT, TH}

Mutually Exclusive Events

Occurrence of one event prevents the occurrence of the other event at the same time. 📌 Example: When we toss two coins, occurrence of two heads means the other three outcomes cannot occur at the same time. 📌 Example: When we toss a single coin, occurrence of a head means that the tail did not occur and cannot occur at the same time.

Equally Likely Events

If one event is as likely to occur as the other. 📌 Example: Head and tail have the same possibility or chance of occurring.

Collectively Exhaustive Events

If they exhaust all possible outcomes of an experiment. 📌 Example: Event is the sample space. A = Occurrence of a head or tail while tossing a single coin.

Stochastic or Random Variable

A variable whose value is determined by the outcome of an experiment.

🔑 Definition — Random Variable: A variable that takes numerical values determined by the outcome of a random experiment. 📌 Example: Let X = Number of heads in an experiment of tossing two coins, then X can have values of 0, 1, or 2.

A random variable can be discrete (can take only whole numbers and finite values) or continuous (can take any values within an interval, either whole numbers or fractions, e.g., height of an individual).

Classical Definition of Probability

Probability of an event 'A' = P(A) = Number of favorable outcomes / Number of total outcomes

📌 Example: Total number of outcomes in tossing two coins is 4 {HT, HH, TH, TT}. Probability of getting exactly one head = 2/4 = 0.5

Probability Distribution

The possible values that a random variable can take with the number of occurrences (frequency) of those values.

📌 Example: Probability Distribution of discrete random variable: Let X = Number of heads in an experiment of tossing two coins. X can have values of 0, 1, or 2.

Probability Mass Function or simply Probability Function

f(X = xᵢ) = P(X = xᵢ) for i = 1, 2, … and 0 if X ≠ xᵢ

Properties: • 0 ≤ f(xᵢ) ≤ 1 • Σ f(xᵢ) = 1 for all x

Probability Density Function (PDF)

Probability Distribution of a continuous random variable, e.g., X = height of person measured in inches.

• X is a continuous random variable • Probability of continuous random variable is always computed for a range, not for a single value • PDF is: P(x₁ < X < x₂) = ∫ f(x) dx from x₁ to x₂ This calculates the probability as the area under a curve between a range (x₁ to x₂).

Cumulative Distribution Function (CDF)

F(X) = P[X ≤ x]

Important Probability Distributions

Some important probability distributions are Normal Distribution, t distribution, Chi-square distribution, and the F-distribution.

Normal Distribution

It is the most important probability distribution for a continuous random variable. It has a bell-shaped curve (highest point at mean value) where X ~ N(μₓ, σ²), −∞ < X < ∞.

Change in μ shifts the curve to right or left, while change in σ increases or decreases the spread of the curve.

The function: f(x) = (1/σ√(2π)) · e^(-½[(x−μ)/σ]²)

Mathematical constants: π = 3.14159, e = 2.71828

Properties (Empirical Rule): • 68% of data lies within μ ± 1σ • 95% of data lies within μ ± 2σ • 99.7% of data lies within μ ± 3σ

Standard Normal Distribution

If Z = (X − μ̄)/σ, then μ = 0 and σ = 1. The distribution of Z is a 'Standard' normal distribution: Z ~ N(0,1)

p(Z) = (1/√(2π)) · e^(-½[Z]²)

The probabilities or areas under the standard normal curve are already calculated and available in the shape of tables (the Z-table).

Student's t-distribution

The t-distribution is a probability distribution of a continuous random variable when the sample size is small and the population variance is not known. Its curve is symmetric and bell-shaped but flatter than the normal distribution. The mean is zero but the variance is larger (heavier tails) than the variance of the standard normal distribution (which is unity). It has only one parameter: the degree of freedom. As the degree of freedom (or the number of observations) increases, the distribution approaches the normal distribution.

Chi-Square (χ²) distribution

The square of a standard normal variable is distributed as a Chi-square probability distribution with one degree of freedom. Z² = χ²(1)

If we need the sampling distribution of the sample variance, we have the Chi-square distribution.

Properties: • Chi-square takes only positive values (zero to infinity) • It is skewed (depending on the d.f.) unlike the normal distribution • As the d.f. increases, the distribution approaches the normal distribution • Its mean is k (=d.f.) and variance is 2k (variance is twice the mean)

💡 Why this matters: Normal distribution and t-distributions are related to means, but the Chi-square and F-distributions are related to variances.

F-distribution

This is a variance ratio distribution — the ratio of sample variances of two independent samples. This is also equal to ratios of two Chi-squares. It has two parameters k₁ and k₂ (degrees of freedom in both samples, i.e., numerator and denominator of F = S₁²/S₂²).

Uses: • Testing equality of variances • Tests in Regression models like Goodness of fit test

Properties: • Skewed to the right between zero and infinity • Approaches normal distribution as d.f. increases

⭐ Key Takeaways

For multivariate optimization, always follow three steps: first-order condition (both partial derivatives = 0), second-order condition (both second partials positive for minima, negative for maxima), and the third condition using the Hessian determinant (product of second partials must exceed square of cross-partial) to rule out inflection points. For probability, understand that a random variable can be discrete (probability mass function) or continuous (probability density function with area under curve), and the four key distributions — Normal (bell-shaped, for means with known variance), t-distribution (for means with unknown variance, flatter tails), Chi-square (for variance, skewed, positive-only), and F-distribution (ratio of variances) — each have specific properties, parameters, and applications in econometric testing.

🧠 Quick Revision Questions

1. What are the three conditions needed to confirm a true maximum for a function of two variables?
2. How does a saddle point differ from a local minimum or maximum in terms of second-order partial derivatives?
3. What is the Hessian determinant, and what does it test for in multivariate optimization?
4. What is the difference between a probability mass function (PMF) and a probability density function (PDF)?
5. For each of the four major distributions (Normal, t, Chi-square, F), state one specific use in econometrics and one key property.

📘 Lecture 06 — The Simple Regression Model

📖 Overview: This lecture introduces the fundamental concept of regression analysis, a statistical method to model the relationship between a dependent variable and one or more independent variables. It explains how to estimate this relationship using a simple linear model and the Ordinary Least Squares (OLS) technique, which is the foundation for understanding more complex econometric models. This lecture is crucial as it provides the core tools for quantifying how changes in one variable affect another, a key skill for business analysis.

🗂️ Topics Covered

This lecture covers the basic concept of regression, distinguishing it from correlation. It introduces the simple regression model, the use of scatter diagrams to identify linear relationships, and the process of estimating the model's parameters (intercept and slope) using the Ordinary Least Squares (OLS) method. The lecture also derives the "normal equations," provides a step-by-step example with calculations, and discusses the interpretation of the results, including trend values, residuals, and the key assumptions and properties (BLUE) of the OLS estimators.

📝 Lecture Summary

The Basic Concept

Regression is a statistical measure to determine the strength of the relationship between a dependent variable (explained variable, response variable, or regressand) and a list of independent variables (explanatory variables, predictor variables, or regressors). It is a process of estimating the relationship among variables, focusing on how and how much the dependent variable changes in response to changes in the independent variable(s). This differs from correlation, as the goal is to understand the causal or predictive nature of the relationship. For example, in a demand function, we want to know how quantity demanded changes due to changes in price. The dependent variable is denoted by Y and the regressor(s) by X.

Since the relationship is not exact (e.g., the same X can yield different Y values), regression analyzes how Y changes, on average, in response to changes in X. Before performing regression, we must have an idea about the nature of the functional relationship (e.g., linear, quadratic, exponential). This insight can be gained by examining a scatter diagram.

Scatter Diagram

A scatter diagram shows pairs of actual observations, typically plotting the dependent variable (Y) against an explanatory variable (X) to observe a pattern. For instance, the diagram below shows that expenditure on food is a direct (increasing) function of income levels. The plotted points do not lie exactly on a straight line but are scattered around a hypothetical straight line, suggesting a positive linear relationship.

Similarly, the second diagram shows that annual sales seem to be inversely related to the price of the commodity, as the dots of observations are scattered around a hypothetical straight line that is negatively sloped.

Remember that straight lines are shown by equations of the type Y = a + bX where a is the y-intercept (the point where the line intersects the Y-axis) and b is the slope of the line (the change in Y due to a one-unit change in X, or ΔY/ΔX). In simple regression, we try to estimate the best values of a and b using an appropriate technique, such as Ordinary Least Squares (OLS) .

Simple Regression Line by OLS

The relationship is captured by writing a simple regression equation: Y = a + bX + e, or Y = β₀ + β₁X + e.

• Residual (e) : Note that we have added e, which is called an error term or residual. We add this because the actual values do not exactly lie on a straight line but are scattered around it. To account for this difference, we capture it in the residual e. When we estimate the parameters ‘a’ and ‘b’, they do not provide exact estimates of the value of the dependent variable. The difference is called the error term or residual.

The equation with subscripts is: Yᵢ = β₀ + β₁Xᵢ + eᵢ. The subscript i indicates that the variable may have multiple observations. β₀ and β₁ are written instead of a and b to follow the tradition of regression analysis.

Regression Explained

The Population Regression Equation is an assumed equation that may have possibly been estimated from a population. We use samples to get the values of the parameters β₀ and β₁ as all the population may not be available or observed. The equation is Yᵢ = β₀ + β₁Xᵢ + eᵢ, where Yᵢ is the dependent variable, β₀ and β₁ are parameters, and X is the independent variable.

Regression Equation Estimation

Given Yᵢ = β₀ + β₁Xᵢ + eᵢ as the population regression equation, let a and b be the estimated values of β₀ and β₁, respectively. We estimate a and b from a sample. The estimated value of Y based on the estimated regression equation is Ŷ = a + bX, where Ŷ is the estimated value or ‘Trend Value’. Then, eᵢ = Y – Ŷ.

It is good to have low values of errors (residuals). Negative and positive errors cancel each other out, and we want to ‘magnify’ larger errors. Therefore, we focus on the ‘Square of Errors’ and try to minimize their sum. In least square estimation, we minimize the ‘Sum of Squared Residuals’ or ‘Sum of Square of Errors’.

We try to estimate the parameters a and b for which we have the minimum possible ‘Sum of Square of Residuals’. Finding the values of ‘a’ and ‘b’ in the regression equation is a minimization problem: Min Σ e².

Remember that:

• e = Y – Ŷ
• Ŷ = a + bX
• For ‘Optimization’ we take the first derivative and set it equal to zero.
• In this minimization problem, ‘a’ and ‘b’ are considered the unknowns.

The above minimization becomes: Min Σ e² = Σ (Y – Ŷ)² = Σ (Y – a – bX)². Let Z denote this expression, so we need to minimize Min Z = Σ (Y – a – bX)².

Deriving the Normal Equations

To find a and b that minimize Z, we use calculus. We partially differentiate Z with respect to ‘a’ and then with respect to ‘b’, and set each derivative to zero.

Partially differentiating Z w.r.t. ‘a’ and setting equal to zero: ∂Z/∂a = Σ 2(Y – a – bX)(-1) = 0 This simplifies to: Σ (Y – a – bX) = 0 Applying summation algebra: Σ Y – na – b Σ X = 0 This gives the first normal equation: Σ Y = na + b Σ X

Partially differentiating Z w.r.t. ‘b’ and setting equal to zero: ∂Z/∂b = Σ 2(Y – a – bX)(-X) = 0 This simplifies to: Σ (Y – a – bX)(X) = 0 Applying summation algebra: Σ XY – a Σ X – b Σ X² = 0 This gives the second normal equation: Σ XY = a Σ X + b Σ X²

In summary, to estimate a linear regression line Yᵢ = β₀ + β₁Xᵢ + eᵢ, where ‘a’ is a sample estimate of β₀ and ‘b’ is a sample estimate of β₁, we minimized the Sum of Squared Residuals, Min Z = Σ (Y – a – bX)². As a result, we got two normal equations. To find the values of ‘a’ and ‘b’, we need some observations of X and Y. We can solve these two equations simultaneously.

Finding the Values of Parameters Directly

Instead of solving two normal equations, we can derive direct expressions for a and b. Starting from the normal equations:

1. Σ Y = na + b Σ X
2. Σ XY = a Σ X + b Σ X²

From the first normal equation, dividing by n gives: ΣY/n = a + b(ΣX/n), which yields a = (ΣY/n) – b(ΣX/n) or a = Ȳ – bX̄, where Ȳ and X̄ are the means of Y and X.

Substituting this value of ‘a’ into the second normal equation and solving for b gives the formula for the slope coefficient: b = (n Σ XY – Σ X Σ Y) / (n Σ X² – (Σ X)²)

And a = Ȳ – bX̄

Example

Consider the example where X = Income (in thousand rupees) and Y = Expenditure on food items (thousand rupees).

The normal equations are: Σ Y = na + b Σ X and Σ XY = a Σ X + b Σ X². Substituting values gives: 145 = 5a + 175b and 5280 = 175a + 6375b.

Solving these simultaneously gives a = 0.3 and b = 0.82. The regression line can be written as Ŷ = 0.3 + 0.82 X.

Using the direct formula for b: b = (5 * 5280 – 175 * 145) / (5 * 6375 – 175²) b = (26400 – 25375) / (31875 – 30625) b = 1025 / 1250 = 0.82

The value of a is: a = Ȳ – bX̄ = (145/5) – 0.82 * (175/5) = 29 – 0.82 * 35 = 29 – 28.7 = 0.3

Interpretation: The value of a (0.3) is the Y-intercept. The value of b (0.82) is the slope of the line. This means that a one unit (1000 rupees) change in X (income) brings a 0.82 unit change in Y (expenditure on food), on the average. An increase in income by 1000 rupees may increase expenditure on food items by 820 rupees.

Trend Values and Errors

We can substitute the values of X in the estimated regression equation to find Trend Values (Ŷ).

The first trend value is computed as Ŷ = 0.3 + 0.82(25) = 20.8. If you change the values of a and b and compute new squares of errors, the new sum would be larger than 11.9 (the sum of least squares).

The Error Term

We assume that errors are normally distributed with zero mean and constant variance: e ~ N(0, σ²). As noticed during estimation, Σ eᵢ = 0 and you can verify that Σ eᵢXᵢ = 0. Also, as we got the regression equation by minimization process, Σ e² is minimum.

Nature of the Error Term:

• The error term may represent the influence of variables NOT included in the model (Missing Variables).
• Even if all variables are included, there will remain randomness in the error as human behavior is not perfectly predictable.
• The error term (e) may represent ‘Measurement Error’ when data is collected through rounding or observation in ranges.

Assumptions of OLS Estimators (Gauss-Markov assumptions)

1. Linear in Parameters: The model is linear in its parameters.
2. Random Sampling: A random sample of n observations is used.
3. Sample variation in explanatory variable (Xᵢ): The Xᵢ values are not all the same.
4. Zero conditional mean: The error e has an expected value of 0, given any values of the explanatory variable. E(e|X) = 0.
5. Homoskedasticity: The error has the same variance given any value of the explanatory variable. Var(e|X) = σ².

BLUE: Best Linear Unbiased Estimators

Under the Gauss-Markov Assumptions, the OLS estimators are Best, Linear, and Unbiased.

• Linear: The model is linear in parameters (e.g., Y = a + bX is linear, Y = a + b²X is not, but Y = a + bX + cX² is fine). ‘Linear’ means the slope coefficient can be expressed as a linear function of Y.
• Unbiased: A parameter is unbiased if the average value of the estimator in repeated samples is equal to the true population parameter. E(bⱼ) = βⱼ.
• Best / Efficient: A parameter is best if its variance is less than any other estimator of the parameter. Var(b) ≤ Var(b̃), where b̃ is any other unbiased estimator of β₁.

🔑 Definition — Slope Coefficient (b): The slope coefficient (b) represents the estimated change in the dependent variable (Y) for a one-unit change in the independent variable (X), on average. 📐 Formula: b = (n Σ XY – Σ X Σ Y) / (n Σ X² – (Σ X)²) → This formula gives the sample estimate of the population slope parameter (β₁). 📌 Example: In the income-expenditure example, b = 0.82. This means that for every additional 1000 rupees in income, the expected expenditure on food increases by 820 rupees.

⭐ Key Takeaways

The core concept of this lecture is the simple linear regression model, which is used to quantify the relationship between a dependent and an independent variable. The OLS method is the standard technique for estimating the parameters (slope and intercept) of this model by minimizing the sum of squared residuals. You must be able to derive and solve the two normal equations (Σ Y = na + b Σ X and Σ XY = a Σ X + b Σ X²) to find the intercept (a) and slope (b). Finally, it is critical to understand the BLUE property, which states that under the Gauss-Markov assumptions, the OLS estimators are the Best (minimum variance), Linear, and Unbiased estimators of the regression parameters, making them the preferred choice.

🧠 Quick Revision Questions

1. What is the key difference between regression and correlation?
2. State the two 'normal equations' used to estimate the parameters (a and b) of a simple linear regression model.
3. A regression analysis yields b = 0. Using all other data, would this be a useful model? Why or why not?
4. What does the "zero conditional mean" assumption (E(e|X) = 0) imply for OLS estimation?
5. Under the Gauss-Markov assumptions, what does the "Best" property in "BLUE" refer to?

📘 Lecture 07 — Estimation and Testing in Regression Analysis

📖 Overview: This lecture focuses on interpreting regression coefficients, measuring prediction accuracy, and testing the statistical significance of regression results. It explains standard error calculations, hypothesis testing for slope coefficients, and goodness-of-fit measures using R-squared and F-tests, with practical examples and Microsoft Excel applications.

🗂️ Topics Covered

The lecture covers interpretation of intercept (β₀) and slope (β₁) coefficients, calculation of standard error of estimate and standard error of 'b', the t-test for significance of the slope coefficient, the coefficient of determination (R²) for explanatory power, the F-test for overall model fit, and practical implementation using Excel formulas including LINEST and TREND functions.

📝 Lecture Summary

Example

Consider data where X = Income in thousand rupees and Y = expenditure on food items (thousand rupees):

Calculations:

b = [n∑XY − (∑X)(∑Y)] / [n∑X² − (∑X)²] b = [5(5280) − (175)(145)] / [5(6375) − (175)²] = 0.82

a = Ȳ − bX̄ = (145/5) − 0.82(175/5) = 0.3

Interpretation of Regression Coefficients

The Intercept 'a' (usually also denoted by β₀) This is the y-intercept of the straight line and indicates the value of Y when X is zero; usually the base or the initial value. Examples: Autonomous Consumption in the Keynesian Consumption function, Autonomous Investment in the Investment function.

The Slope Coefficient 'b' (Usually denoted also by β₁) In linear regression lines, it shows the average unit change in the dependent variable due to one unit change in the independent variable. In a linear equation, it can also be called the derivative of the dependent variable with respect to the independent variable. The slope coefficient b is an unbiased estimate of the population regression coefficient β₁.

Standard Error of Estimate/Standard Error of Regression

The standard error of the estimate is a measure of the accuracy of predictions. It is the standard deviation of errors and defined as:

🔑 Definition — σ̂ₑ = √[∑eᵢ² / (N − k)]

Where N = Number of observations, k = number of restrictions imposed which is equal to number of parameters, N − k = degrees of freedom.

Standard Error of 'b' (for regression with one independent variable)

se(b) = σ̂ₑ / √[∑(X − X̄)²] Or equivalently: se(b) = σ̂ₑ / √[∑X² − (∑X)²/N]

In the example: σ̂ₑ = √(11.9 / (5−2)) = √(11.9/3) = 1.9916

se(b) = 1.9916 / √[6375 − (175)²/5] = 1.9916 / 15.81139 = 0.12596

Testing for the significance of slope coefficient: Individual variable significance test

The procedure is as follows:

• H₀: b = 0
• H₁: b ≠ 0
• α = 0.05 (or 0.01)
• Level of significance: Probability of type I error (rejecting a true hypothesis)

🔑 Test statistic: t = b / se(b)

Region of Rejection: |t| > t_α/2, n−k

As this is a two-tailed test, search for the value of t in the table of t-distribution corresponding to α/2 and n−k.

In this example: t = 0.82 / 0.12596 = 6.51

Looking into the table of t-distribution: t_0.025,3 = 3.182

As |t| > t_α/2, n−k: 6.51 > 3.182

So we reject H₀ and conclude that 'b' is significant and the variable X has a significant impact on the variable Y.

Reading the value of t-distribution in Microsoft Excel (2007 or later): Type in any cell and press ENTER: = T.INV.2T(0.05,3) The cell will display the value 3.182. This formula has two parameters: the first is the level of significance and the second is the d.f. (N − K).

💡 Why this matters: The t-test tells us whether the independent variable actually has a statistically significant effect on the dependent variable, or whether the relationship we observe could be due to random chance.

The Coefficient of Determination: Explanatory power of the model

The proportion of variation in Y that is explained by X:

R² = Explained Variation / Total Variation = ∑(Ŷ − Ȳ)² / ∑(Y − Ȳ)²

Since Total Variation = Explained Variation + Unexplained Variation:

R² = 1 − [Unexplained Variation / Total Variation] = 1 − [∑e² / ∑(Y − Ȳ)²]

Which can also be transformed as:

R² = 1 − [N∑e²] / [N∑Y² − (∑Y)²]

In the current example: R² = 1 − [5(11.9)] / [5(4385) − (145)²] = 0.934

Interpretation: The explanatory power of the model is 93.4%. Or: With the simple regression model, the variations in X can explain 93.4% of variation in Y.

Note: The coefficient of determination is equal to the square of the correlation coefficient only in case of the simple regression line with one independent variable.

The Goodness of Fit Test: Using the F-distribution

The procedure is as follows:

• H₀: The Fit is not good
• H₁: The Fit is good
• α = 0.05 (or 0.01)
• Test statistic: F = [R² / (1−R²)] × [(N−k) / (k−1)]

Region of Rejection: F > F_α, k−1, N−k

In the current example: F = [0.934 / (1−0.934)] × (3/1) = 42.4

Looking into the table of F-distribution: F_0.05, 1,3 = 10.13

As F > F_α, k−1, N−k: 42.4 > 10.13

So we reject H₀ and conclude that The Fit is Good.

Reading the value of F-distribution in Microsoft Excel (2007 or later): Type in any cell and press ENTER: = F.INV.RT(0.05,1,3) The cell will display the value 10.13. This formula has three parameters: the level of significance, k−1, and N−k.

Using basic Microsoft Excel Formulas: intercept and slope

To estimate Y = a + bX + e, find the values of parameter 'a' (intercept) and parameter 'b' (slope) in Microsoft Excel:

• Value of 'a': = intercept (Cell range of Values of Y, Cell range of Values of X)
• Value of 'b': = slope (Cell range of Values of Y, Cell range of Values of X)

The cells will display values 0.3 and 0.82.

Using basic Microsoft Excel Formulas: the LINEST formula

Syntax: LINEST(known_y's, [known_x's], [const], [stats])

Steps:

1. Select cells D3 to E7 (2 columns and five rows to display results)
2. Type = LINEST(B3:B7, C3:C7, TRUE, TRUE)
3. Press and hold Ctrl + SHIFT and press ENTER (Ctrl + SHIFT + ENTER)

Results are displayed as:

• value of 'b' | value of 'a'
• standard Error of 'b' | standard error of 'a'
• R-squared | standard error of estimate
• F-statistic | N−k
• Regression Sum of Squares | Residual SS = Sum of square of errors

💡 Why this matters: The LINEST function provides a complete regression output in a single array, giving you all key statistics (coefficients, standard errors, R², F-statistic) needed for econometric analysis without manual calculations.

Understanding the TREND formula

You may compute all the trend values without calculating the regression coefficients by using this formula. Note the $ signs; they are inserted around the letter so that the reference does not change when you extend the formula to other cells by dragging. Drag the formula to get other trend values.

⭐ Key Takeaways

The slope coefficient (b=0.82) indicates that for every thousand rupee increase in income, food expenditure increases by 0.82 thousand rupees. The standard error of the estimate (σ̂ₑ=1.9916) measures the average prediction error. The t-test (t=6.51 > critical value 3.182) confirms that income has a statistically significant effect on food expenditure. The R² value of 0.934 indicates that income explains 93.4% of the variation in food expenditure, and the F-test (F=42.4 > critical value 10.13) confirms the overall model fit is good.

🧠 Quick Revision Questions

1. How do you calculate the standard error of the estimate (σ̂ₑ), and what does it measure?
2. What are the null and alternative hypotheses when testing the significance of the slope coefficient b?
3. If R² = 0.934, what does this value tell you about the relationship between X and Y?
4. How is the F-statistic calculated from R², and what is its rejection rule?
5. How would you use the LINEST function in Excel to obtain regression statistics for a simple linear regression?

📘 Lecture 08 — Examples Using Microsoft Excel

📖 Overview: This lecture demonstrates the step-by-step procedure for performing Ordinary Least Squares (OLS) regression using Microsoft Excel formulas. It covers the manual calculation of all key regression statistics — coefficients, standard errors, t-statistics, R-squared, and F-statistic — and verifies these results using Excel’s LINEST function. The lecture also introduces unit of measurement changes and transformation of non-linear models for OLS estimation.

🗂️ Topics Covered

This lecture covers the manual calculation of regression parameters ‘a’ and ‘b’ using Excel formulas, followed by computation of the standard error of estimate and standard error of b. It then explains how to test individual variable significance using t-statistics and assess goodness of fit using the F-distribution and R-squared. The LINEST Excel formula is introduced to verify all results. The lecture concludes with change of unit of measurement, non-linear model transformations (including Cobb-Douglas and various functional forms), interpretation of coefficients across models, and the concept of outliers in regression.

📝 Lecture Summary

Model Exam Question

The lecture presents a model exam question requiring the student to calculate regression parameters ‘a’ and ‘b’ for the model Y = a + b X + e using learned formulas, compute standard error of estimate and standard error of b, calculate t-values for variable significance, compute the coefficient of determination, calculate the F-statistic for goodness of fit, and verify all results using the LINEST Excel formula. Students are instructed to interpret results at each step and use the equation editor for formulas.

Calculating Regression Coefficients ‘a’ and ‘b’

The data is entered in Excel with P as X and Q as Y. The slope coefficient b is calculated using the formula: 📐 Formula: b = (n ΣXY − ΣX ΣY) / (n ΣX² − (ΣX)²) → This formula computes the slope of the regression line by dividing the covariance of X and Y by the variance of X. Using the given data: b = (10 * C14 − B14 * A14) / (10 * D14 − (B14)²) = −0.84737

The intercept a is calculated using: 📐 Formula: a = Ȳ − b X̄ → The intercept is the mean of Y minus the slope times the mean of X. Using the data: a = (A14/10) − K6 * (B14/10) = 340.8812

Verification is done using Excel’s SLOPE and INTERCEPT functions: =SLOPE(A4:A13,B4:B13) for b =INTERCEPT(A4:A13,B4:B13) for a

The regression equation is: Ŷ = 340.8812 − 0.84737 X

🔑 Definition — Interpretation of ‘b’ (the slope coefficient): For every one unit change in X, there may be, on the average, 0.84737 unit change in Y in the inverse direction.

Standard Error of Estimate and Standard Error of b

The standard error of estimate (σ̂ₑ) measures the typical distance between observed values and the regression line. 📐 Formula: σ̂ₑ = √(Σeᵢ² / (N − k)) = 10.63916 (Where N = number of observations, k = number of parameters estimated)

The standard error of b (se(b)) measures the precision of the slope estimate. 📐 Formula: se(b) = σ̂ₑ / √(ΣX² − (ΣX)²/N) = 0.08668

Individual Variable Significance Test

This test determines whether the independent variable X has a statistically significant effect on Y.

Hypotheses: H₀: b = 0 (X has no effect on Y) H₁: b ≠ 0 (X has a significant effect on Y) α = 0.05

Test statistic: 📐 Formula: t = b / se(b) = −0.84737 / 0.08668 = −9.77614

Region of Rejection: |t| > t_(α/2, n−k) t_(0.025, 8) = 2.306

✅ Conclusion: Since the absolute value of t-statistic (9.77614) is greater than 2.306, we reject H₀ and conclude that ‘b’ is significant.

The Goodness of Fit Test: Using the F-distribution

This test evaluates whether the overall regression model provides a good fit to the data.

Hypotheses: H₀: The fit is not good H₁: The fit is good α = 0.05

Coefficient of Determination (R²): 📐 Formula: R² = 1 − (Σe² / (ΣY² − (ΣY)²/N)) = 0.9228 🔑 Definition — R-squared: The proportion of the total variation in Y that is explained by the regression model.

F-statistic: 📐 Formula: F = (R² / (k−1)) / ((1−R²) / (N−k)) = (0.9228 / 1) / ((1−0.9228) / 8) = 95.573

Region of Rejection: F > F_(α, k−1, N−k) = F > F_(0.05, 1, 8) = 5.3177

✅ Conclusion: Since 95.573 > 5.3177, we reject H₀ and conclude that the FIT IS GOOD.

Verifying Results with the LINEST Formula

Syntax: LINEST(known_y's, [known_x's], [const], [stats])

Procedure:

1. Select cells D18:D22
2. Type: =LINEST(A4:A13,B4:B13,TRUE,TRUE)
3. Press Ctrl + SHIFT + ENTER (array formula)

Output layout:

All values displayed by LINEST are identical to the manual calculations. The t-statistic can be calculated by dividing the value of ‘b’ by the cell containing the standard error of b.

Change of Unit of Measurement

Sometimes units are changed for larger or smaller values to make calculations easier and coefficients more meaningful.

• Examples: Population in millions, Prices in thousand rupees

🔑 Definition — Linear transformation: Dividing or multiplying all values of a variable by a constant factor. This preserves linear relationships between variables; therefore, correlation between x and y does not change.

Transformation of Models and Use of OLS

Some models are not linear but can be transformed to use OLS estimation.

🔑 Definition — Nonlinear transformation: A transformation that changes (increases or decreases) linear relationships between variables. Correlation between variables changes.

• Examples: Taking natural logs, squares, square roots, or reciprocals of a variable

Cobb-Douglas Production Function Example: Original: Q = A Lᵅ Kᵝ Reduced to one variable: Q = A Kᵝ Taking log on both sides: ln Q = ln A + β ln K Let Y = ln Q, α = ln A, and X = ln K, then: Y = α + βX (estimable by OLS)

Summary of Functional Forms:

Interpretation of Different Functional Forms:

💡 Why this matters: For interpretation, we assume that Gauss-Markov assumptions hold and parameters are significant. The marginal effect is the partial derivative of Y with respect to X, and elasticity may be computed at mean values of X and Y.

Outliers in Regression

🔑 Definition — Outliers: Points that diverge a lot from the data in general and may affect the slope of the regression equation or the predictive power of the model.

Types of Outliers:

1. Extreme X values
2. Extreme Y values
3. Extreme X and Y values
4. Distant point with normal X or Y value

🔑 Definition — Influential point: If the removal of an outlier changes the slope or coefficient of determination substantially, it may be called an influential point.

Two diagrams in the lecture illustrate outliers as dots that appear far from the main cluster of data points.

⭐ Key Takeaways

The OLS regression parameters must be computed systematically: first the slope ‘b’ using the covariance-variance ratio formula, then the intercept ‘a’ using the means of X and Y. The standard error of b and the t-statistic are essential for testing whether the independent variable is statistically significant. Goodness of fit is evaluated through R-squared (proportion of explained variation) and the F-statistic, which compares explained to unexplained variance; a significant F confirms the model fits well. The LINEST function in Excel provides a convenient verification tool, returning all key regression statistics in a compact array. Finally, understanding functional form transformations (log-log, level-log, log-level) is critical because they change how coefficients are interpreted — particularly the difference between marginal effects and elasticities — and being aware of outliers is essential for assessing the robustness of regression results.

🧠 Quick Revision Questions

1. What formula is used to calculate the slope coefficient ‘b’ in OLS regression, and how is it computed using Excel cell references?
2. How is the standard error of b calculated, and what does it measure about the regression coefficient?
3. In the individual variable significance test, what is the test statistic formula and decision rule using the t-distribution?
4. How is the F-statistic derived from R-squared, and what conclusion does an F-statistic of 95.573 with critical value 5.3177 lead to?
5. For a log-log model where ln Y = a + b ln X, how would you interpret the coefficient ‘b’, and how does this differ from the interpretation of ‘b’ in a simple linear model?

📘 Lecture 09 — Multiple Regressions

📖 Overview: This lecture introduces multiple regression analysis, which extends simple regression to include two or more independent variables to explain variation in a dependent variable. It covers the mathematical formulation, estimation using Ordinary Least Squares (OLS), derivation of parameters in deviation form, and the manual procedure for solving multiple regression equations, along with diagnostic tests for significance.

🗂️ Topics Covered

The lecture begins with the general idea of multiple regression and examples. It then presents the multiple linear regression equation for two independent variables and the concept of fitting a line in 3-dimensional space. The least square estimation method is detailed, including the derivation of normal equations. The deviation form is introduced to simplify calculations, and formulas for estimating coefficients β₁, β₂, and β₀ are derived. A manual procedure is outlined, along with a method to convert summations from normal to deviation form. An example is solved manually and verified using the LINEST function in Excel. Finally, diagnostic tests including t-tests for individual significance and the F-test for overall model significance are discussed.

📝 Lecture Summary

General Idea

Multiple Regression considers the relation between one dependent variable and two or more independent variables. This allows us to examine the impact of one independent variable on the dependent variable while holding other independent variables constant. For example, earnings may depend on both education and experience, or quantity demanded may depend on price, income, and prices of substitutes.

Multiple Regression Equation

The multiple regression equation with two independent variables is written as: 푌ᵢ = β₀ + β₁푋₁ᵢ + β₂푋₂ᵢ + eᵢ While a simple regression fits a line in 2-dimensional space, a multiple regression with two independent variables fits a plane in 3-dimensional space.

In multiple regression, we can:

• Use several variables simultaneously to explain variation in a continuous dependent variable.
• Isolate the unique effect of one variable on the dependent variable while accounting for other variables.
• Write a mathematical equation that shows the overall effects of several variables together and the unique effect of each.

Estimating a Multiple Regression Line: Least Square Estimation

We use the Ordinary Least Squares (OLS) method to estimate the parameters β₀, β₁, and β₂ by minimizing the sum of squared residuals. The minimization problem is: Min ∑ eᵢ² = ∑(Y − Ŷ)² = ∑(Y − β₀ − β₁X₁ − β₂X₂)²

Taking first derivatives with respect to β₀, β₁, and β₂ and setting them to zero yields the three normal equations: ∑ Y = nβ₀ + β₁ ∑ X₁ + β₂ ∑ X₂ ∑ X₁Y = β₀ ∑ X₁ + β₁ ∑ X₁² + β₂ ∑ X₁X₂ ∑ X₂Y = β₀ ∑ X₂ + β₁ ∑ X₁X₂ + β₂ ∑ X₂²

The Deviation Form in Regression

To simplify calculations, we can use the deviation form, where variables are expressed as deviations from their means. Small letters indicate a variable in deviation form. x₁ᵢ = X₁ᵢ − X̄₁ x₂ᵢ = X₂ᵢ − X̄₂ yᵢ = Yᵢ − Ȳ

The regression equation in deviation form becomes: y = β₁x₁ + β₂x₂

Minimizing the sum of squared residuals in deviation form results in two normal equations: ∑ x₁y = β₁ ∑ x₁² + β₂ ∑ x₁x₂ ∑ x₂y = β₁ ∑ x₁x₂ + β₂ ∑ x₂²

🔑 Definition — Deviation Form: A method of expressing variables as the difference between each observation and its arithmetic mean, which simplifies regression calculations.

Deriving Expression for β Coefficients

Solving the two normal equations in deviation form provides formulas for β₁ and β₂.

📐 Formula: β₁ = [∑ x₁y ∑ x₂² − ∑ x₂y ∑ x₁x₂] / [∑ x₁² ∑ x₂² − (∑ x₁x₂)²]

📐 Formula: β₂ = [∑ x₂y ∑ x₁² − ∑ x₁y ∑ x₁x₂] / [∑ x₁² ∑ x₂² − (∑ x₁x₂)²]

📐 Formula: β₀ = Ȳ − β₁X̄₁ − β₂X̄₂

Manual Procedure to Solve for Multiple Regression

The manual procedure uses the formulas in deviation form. The necessary summations in deviation form can be calculated directly from the original data using conversion formulas.

📐 Formula: ∑ x₁y = ∑ X₁Y − (∑ X₁ ∑ Y) / n 📐 Formula: ∑ x₂y = ∑ X₂Y − (∑ X₂ ∑ Y) / n 📐 Formula: ∑ x₁x₂ = ∑ X₁X₂ − (∑ X₁ ∑ X₂) / n 📐 Formula: ∑ x₁² = ∑ X₁² − (∑ X₁)² / n 📐 Formula: ∑ x₂² = ∑ X₂² − (∑ X₂)² / n

📌 Example: Using the provided dataset, the following deviation form summations were calculated: ∑ x₁y = 5513.5, ∑ x₂y = -1115, ∑ x₁x₂ = -1665, ∑ x₁² = 6780.1, ∑ x₂² = 330. Substituting into the formulas gives β₁ = -0.0372 and β₂ = -5.1713. Then β₀ = Ȳ − β₁X̄₁ − β₂X̄₂ = 264.67. The estimated regression line is: Ŷ = 264.67 − 0.0372X₁ − 5.1713X₂.

💡 Why this matters: The manual calculation demonstrates the direct application of OLS formulas, building a deeper understanding before relying on software. The example shows how to interpret the coefficients: holding X₁ constant, a one-unit increase in X₂ is associated with a decrease of 5.1713 units in Y.

Using LINEST in Multiple Regression

The LINEST function in Excel can be used for multiple regression by selecting an array with 5 rows and as many columns as parameters (3 in this case). The function is entered as an array formula (Ctrl + Shift + Enter). The output provides coefficients, standard errors, R-squared, F-statistic, and the regression and residual sums of squares.

Simple vs Multiple Regression

A key difference is degrees of freedom: with n observations and k independent variables, degrees of freedom = n − k. In the example with 10 observations and 2 independent variables, the degree of freedom is 7. The coefficient of determination (R²) in multiple regression is not simply the square of a single correlation coefficient.

Diagnostic Tests for Regression Analysis

We perform t-tests for individual slope coefficients and an F-test for overall model significance.

The t-statistic for a coefficient βⱼ is: t = βⱼ / se(βⱼ)

📌 Example: For β₁, t = -0.0372 / 0.02892 = -1.287. The critical value t₀.₀₂₅,₇ = 2.36. Since |t| < t_critical, β₁ is not significant, meaning X₁ does not have a significant impact on Y. 📌 Example: For β₂, t = -5.1713 / 0.1311 = -39.45. Since |t| > t_critical, β₂ is significant, meaning X₂ has a significant impact on Y.

The F-statistic tests the overall significance of the regression. Its formula in terms of R² is: F = [R² / (k − 1)] / [(1 − R²) / (n − k)]

📌 Example: R² = 0.9979, so F = 1667.53. The critical value F₀.₀₅,₂,₇ = 4.74. Since F > F_critical, the overall model fit is good.

⭐ Key Takeaways

Multiple regression extends simple regression to analyze the unique effect of multiple independent variables on a dependent variable. The OLS method minimizes the sum of squared residuals, leading to normal equations that can be solved using deviation forms to derive formulas for β coefficients. The manual estimation procedure, though time-consuming, is essential for understanding the mechanics behind the analysis. The t-test is used to assess the significance of individual independent variables, while the F-test evaluates the overall model fit, both of which are critical for validating the regression results.

🧠 Quick Revision Questions

1. What is the main difference between simple and multiple regression?
2. Why is the deviation form used in estimating multiple regression parameters?
3. How many normal equations are obtained when minimizing the sum of squared residuals for a multiple regression with two independent variables?
4. What does it mean if a t-test for a coefficient βⱼ shows that |t| > t_critical?
5. What does the F-test in multiple regression assess?

📘 Lecture 10 — Linear Regression Estimation with K-independent variables

📖 Overview: This lecture extends simple linear regression to include multiple (K) independent variables, explaining how estimation and interpretation change as more regressors are added. It provides a practical, step-by-step guide to using Microsoft Excel's Analysis ToolPak for multiple regression analysis, including installation, data setup, and interpretation of outputs. The lecture emphasizes both the computational convenience and the limitations of Excel for econometric work.

🗂️ Topics Covered

The lecture begins by defining the linear regression model with K independent variables and the normal equations derived from minimizing sum of squared residuals. It then discusses how adding independent variables changes key statistics like degrees of freedom, standard errors, and R-squared. A major portion is dedicated to installing, setting up, and using the Data Analysis ToolPak in various Excel versions, followed by a detailed walkthrough of performing regression with this add-in. The lecture concludes with interpreting regression output (coefficients, p-values, F-statistic, adjusted R-squared), practicing on real World Bank data for Pakistan, and outlining common problems in multiple regression such as multicollinearity and heteroskedasticity.

📝 Lecture Summary

Linear Regression Model with K Independent Variables

The general multiple linear regression model is defined as: Yᵢ = β₀ + β₁X₁ᵢ + β₂X₂ᵢ + β₃X₃ᵢ + ... + βₖXₖᵢ + eᵢ, where 1 < K < M and M is a finite number. Minimizing the sum of squared residuals yields K+1 normal equations that must be solved simultaneously, typically using matrix algebra in software like Microsoft Excel (via LINEST or Data Analysis Add-in).

🔑 Definition — Degrees of Freedom: The degrees of freedom for a multiple regression is calculated as (N - K), where N is the number of observations and K is the number of independent variables. This decreases as more variables are added.

Adding independent variables changes several aspects of regression: degrees of freedom (N-k) decreases; potential problems like multicollinearity may arise; the formula for standard errors of coefficients changes; a larger number of observations may be needed; the coefficient of determination (R²) increases artificially; and regression coefficient values may change when a new variable is added.

Analysis ToolPak: DATA ANALYSIS Add-In in Microsoft Excel

The Analysis ToolPak is a Microsoft Excel add-in that uses the LINEST function to perform regressions and produce basic regression statistics in tabular form. It has a limit of maximum 16 independent variables. The tool typically comes with Excel but must be added (installed) from within the program—usually without needing a CD unless it is not already installed.

Installation steps vary by Excel version:

• Excel 2003 and earlier: Tools Menu → Add-ins
• Excel 2007: Office Button → Excel Options → Add-ins → Manage Excel Add-ins → GO
• Excel 2010 (demonstrated in lecture): FILE → Options → Add-Ins → Click on Analysis ToolPak (not VBA) → GO → Check box → OK → If prompted, click YES to install

Once installed, verify by checking if "Data Analysis" appears under the DATA ribbon.

Regression with The Analysis ToolPak

To perform regression, first enter data (10 observations each for Y and X variables) in an Excel sheet. Navigate to the DATA ribbon, click Data Analysis (rightmost item), scroll to Regression, and click OK. In the regression dialog box, set Input Y Range by selecting the Y values column, set Input X Range by selecting all X variable columns (they must be adjoining), check New Worksheet, and click OK. Results display in a new worksheet; resize columns for better readability.

📌 Example: Using sample data with Y and three X variables (X₁, X₂, X₃), the regression output table shows coefficients, standard errors, t-statistics, and p-values for each variable.

Understanding the Regression Results

The regression output table contains three main parts. The third part displays coefficients, standard errors, t-statistics, and p-values. P-values show the level of significance at which, based on the t-statistic, the null hypothesis (H₀: coefficient is not significant) can be rejected. In the example, the coefficient of X₂ has a p-value of 0.00001003 (below 1%), making it statistically significant at the 1% level. The intercept also shows significance with p-value below 1%, while other coefficients are not significant.

🔑 Definition — Adjusted R²: Adjusted R² = R² − [(1 − R²) × (k − 1) / (N − k)]. It adjusts the coefficient of determination for degrees of freedom, providing a more accurate measure of goodness-of-fit when multiple independent variables are present, as R² is overstated when more variables are added.

Multiple correlation equals the square root of R² and is computable from partial correlation coefficients. The Coefficient of Determination (R²) represents the explanatory power of the model—in the example, it is 0.99 or 99%. The F-statistic and its significance (p-value ≈ 0.0000000178, essentially zero) indicate that the model is a good fit at the 1% level of significance.

The p-value equals Pr{|t| > t-Stat} where t follows a t-distribution with (n − k) degrees of freedom and t-Stat is the computed t-statistic. It represents the level of significance at which the calculated t-statistic becomes larger than the table value.

💡 Why this matters: Understanding p-values and F-statistics allows researchers to make objective decisions about which variables and models are statistically meaningful, rather than relying on arbitrary cutoffs.

LIMITATIONS of Working with Data Analysis Toolpak

Excel's Analysis ToolPak has several important limitations: it restricts independent variables to maximum 16; requires all independent variables to be in adjoining columns; its standard errors, t-statistics, and p-values assume errors are independent with constant variance; and it does not provide alternative models like robust standard errors. More powerful software such as STATA, EVIEWS, or SPSS may be needed for advanced econometric analysis.

Practicing Regression Estimation

The lecture provides a practical exercise using real economic data from the World Bank's World Development Indicators (http://data.worldbank.org/data-catalog/world-development-indicators). Use the Data Bank resource to select Pakistan as country; variables: GDP (constant 2005 US$), Labor force (total), Gross fixed capital formation (constant 2005 US$), and Exports of goods and services (constant 2005 US$); and time period: 1991 to 2012. Download in Excel format.

After downloading, reshape data by copying all data and using Paste Special → Transpose (Ctrl+V then check TRANSPOSE). Then use the Data Analysis ToolPak to run a regression with GDP as dependent variable and Labor Force, Stock of Capital (using GFCF as proxy), and Exports as independent variables. All independent variables must be in adjacent columns.

Possible Problems

Multiple regression models can suffer from various issues: the relationship may not be linear (wrong specification); important variables may be omitted; indicators may be inappropriate; there may be outliers (e.g., abnormal years); errors may not be normally distributed; and there may be econometric problems like multicollinearity, heteroskedasticity, or autocorrelation (to be discussed in later lectures).

⭐ Key Takeaways

The multiple linear regression model extends simple regression by incorporating K independent variables, which changes key statistical properties: degrees of freedom decrease, R² artificially inflates, and coefficients can shift when new variables are added. The Analysis ToolPak in Excel provides a practical, step-by-step method for estimating these models, from installation and data setup to interpreting output tables with coefficients, p-values, and F-statistics. However, Excel's limitations (max 16 variables, no robust errors) mean that serious econometric work may require specialized software. The adjusted R² is a more reliable measure of goodness-of-fit than R² when comparing models with different numbers of predictors.

🧠 Quick Revision Questions

1. What are the key statistical changes that occur when additional independent variables are added to a linear regression model?
2. How do you install and access the Analysis ToolPak in Microsoft Excel 2010?
3. What does the p-value represent in the regression output, and how is it used to determine statistical significance?
4. Write the formula for Adjusted R² and explain why it is preferred over R² in multiple regression.
5. List three limitations of using Excel's Data Analysis ToolPak for regression analysis.

📘 Lecture 11 — Transformation for Regression

📖 Overview: This lecture explains that "linear" in regression refers to linearity in parameters, not necessarily in variables. It covers various non-linear functional forms including quadratic, cubic, logarithmic, and exponential relationships, as well as dynamic models with lagged variables and interaction terms, which allow for more realistic modeling of economic relationships.

🗂️ Topics Covered

The lecture covers the concept that linearity is often a poor approximation of reality, and explores alternative functional forms including exponential and logarithmic functions, linear-log, log-linear, and log-log forms, polynomial functions of higher order, dynamic models with lagged independent variables, and interaction terms. Examples include marginal cost curves, total cost curves, diminishing returns, and the impact of investment changes over time.

📝 Lecture Summary

Linear in the ‘Linear Regression’ means that the model is ‘linear in parameters’ and not necessarily linear in variables. Linearity is a poor approximation of truth.

Linear regression models assume linearity in parameters, but relationships between variables themselves can be non-linear. Common non-linear relationships include quadratic (U-shaped curves like marginal cost), cubic (S-shaped curves like total cost with inflection points), logarithmic (diminishing returns), and exponential (growth/decay) forms. The scatter plot of data can help identify which type of relationship exists between variables.

💡 Why this matters: Economic relationships are rarely perfectly linear, so knowing how to model non-linear patterns is essential for accurate econometric analysis.

Other Functional Forms

The scatter plot may indicate the type of relation. Different functional forms include:

• Linear: Simple straight-line relationship
• Exponential: Curved relationship showing rapid growth or decay
• Logarithmic: Curved relationship showing diminishing growth

Exponential and Logarithmic Function

The exponential functional form is (y = b^x) where (b > 0). It can be estimated as (\ln y = x \ln b). This transforms to (y^* = b^* x) where (b^* = \ln b) and (y^* = \ln y). This becomes a regression equation without an intercept (though (y = ab^x) would include an intercept). When estimated, we need to compute the inverse log to get the original parameter (b).

🔑 Definition — Logarithmic function: Inversely related to exponential functions. From the exponential case, by definition of logarithm to the base (a), the logarithmic function can be written as (x = \log_a y).

📌 Example: Uses include modeling growth or decay like economic growth models, investment increasing by a constant percentage each time period, sales of a company increasing at a constant percentage each year, and models of the spread of an epidemic.

Logs of variables

Linear-log, log-linear and log-log forms are commonly used:

• Linear-log form: (y = a + b \ln x + e)
• Log-linear or semi-log model: (\ln y = a + bx + e)
• Log-log form: Refer to the Cobb-Douglas production function discussed earlier

The graph of (y) against (x) is curved, but the graph of (y) against (\ln x) is a straight line.

📌 Example: Short-run production — (Y) experiences diminishing marginal returns with respect to increases in (X). The log transformation bends a concave curve into a straight line.

🔑 Definition — Log-linear model: A model where the dependent variable is in logarithmic form while the independent variable remains in its original units.

Polynomial functions of higher order

(y = \beta_0 + \beta_1 x + \beta_2 x^2 + \beta_3 x^3 + \beta_4 x^4 + ... + \beta_k x^k + e)

The model is linear in parameters but not in variables. The squared term captures curvature in the relationship.

📌 Example — Total Cost (TC) curve: (TC = \beta_0 + \beta_1 x + \beta_2 x^2 + \beta_3 x^3) where (\beta_3 \neq 0) and (\beta_2 < 0). This cubic form captures the S-shaped total cost curve.

📌 Example — Marginal Cost (MC) curve: Quadratic form (y = \beta_0 + \beta_1 x + \beta_2 x^2) where (\beta_0, \beta_2 > 0) and (\beta_1 < 0). This U-shaped curve reflects diminishing then increasing marginal costs.

Dynamic Models: Models with lagged independent Variable

Dynamic models capture relationships where variables have effects that persist over time.

🔑 Definition — Lagged variable: A variable from a previous time period used as an independent variable in the current period's model.

📌 Example 1 — Multiplier effect: A change in investment in period (t-2) impacts output in periods (t-2), (t-1), and (t): (\Delta Q = \beta_0 + \beta_1 \Delta I_t + \beta_2 \Delta I_{t-1} + \beta_3 \Delta I_{t-2} + e)

📌 Example 2 — Agricultural supply: Supply of an agricultural crop depends on the price from the previous time period, as farmers sow based on last year's observed price: (Q^s = a + b P_{t-1} + e)

📌 Example 3 — Autoregressive model: A variable may depend on its own previous value: (y_t = a + b y_{t-1} + e)

What is Interaction term?

An interaction term captures the situation where the impact of one independent variable on the dependent variable depends on another independent variable.

🔑 Definition — Interaction term: A new variable computed as the product of two independent variables, represented as ((X_1 * X_2)).

📌 Example 1 — Education: Marks in econometrics ((M)) depend on IQ level ((I)) and hard work ((H) = time spent studying). The impact of hard work on marks may depend on IQ level (more intelligent students may need less hard work). This is captured by: (M = a + bI + cH + d(I * H) + e)

The derivative of (M) with respect to (H) is ((c + dI)), which depends on the value of (I).

📌 Example 2 — Health: Consider the impact of hypertension ((B) = blood pressure) and diabetes ((S) = average blood sugar) on heart health ((H)). Both are risk factors, but having both simultaneously may have a greater impact than the sum of individual effects: (H = a + bB + cS + d(B * S) + e)

The coefficient of the interaction term (d) captures the additional impact of having BOTH risk factors simultaneously.

⭐ Key Takeaways

The most critical concept is that "linear" in linear regression refers to linearity in parameters, not variables, allowing for flexible modeling of non-linear relationships through transformations. Students must remember the four main functional forms (linear-log, log-linear, log-log, and polynomial) and when to use each based on scatter plot patterns. Dynamic models with lagged variables capture time-dependent effects like multiplier impacts and agricultural supply responses. Interaction terms model how the effect of one variable depends on another, such as how IQ moderates the impact of study time on exam performance. Proper interpretation of coefficients changes depending on the functional form used, and exponential models require inverse log transformation to recover original parameters.

🧠 Quick Revision Questions

1. What does "linear" mean in the context of linear regression models?
2. How does the exponential functional form (y = b^x) get transformed for estimation using OLS?
3. What is the difference between linear-log, log-linear, and log-log functional forms?
4. Why would a cubic polynomial be used to model a total cost curve rather than a quadratic?
5. What does the coefficient of an interaction term ((I * H)) represent in the model (M = a + bI + cH + d(I * H) + e)?

📘 Lecture 12 — Regression on standardized variables

📖 Overview: This lecture covers the concept of standardizing variables in regression analysis to eliminate the influence of measurement units, allowing for direct comparison of coefficient magnitudes. It also discusses the theoretical foundation of regression, summary statistics, correlation matrices, graphs, and best practices for presenting regression results in research papers.

🗂️ Topics Covered

The lecture begins with the standardization of variables and interpretation of beta coefficients (standardized coefficients), along with their advantages and disadvantages. It then covers the theory behind regression, summary statistics in Excel, correlation matrices, graphs before regression, how to present regression results in research papers, and strategies for keeping or dropping variables. An example of presented regression results is provided, followed by a review of regression-related functions in Microsoft Excel and a summary of steps before regression.

📝 Lecture Summary

Regression on standardized variables

The units in which the regressand and regressor are expressed affect the interpretation of coefficients. Standardizing variables solves this problem. A variable is standardized by subtracting its mean and dividing by the standard deviation.

🔑 Definition — Standardized variable: (X_i^* = \frac{X_i - \bar{X}}{S_X}), (Y_i^* = \frac{Y_i - \bar{Y}}{S_Y})

The standardized variables have zero means and unit variances: (\bar{X}^* = \bar{Y}^* = 0) and (Var(X_i^) = Var(Y_i^) = 1). It does not matter in what unit the original variables are expressed.

Running the regression (Y_i^* = a + bX_i^* + e) gives standardized coefficients, often called BETA coefficients. The intercept term will always be zero because (\bar{X}^* = \bar{Y}^* = 0) (since (a = \bar{Y}^* - b\bar{X}^*)).

📌 Interpretation: If the (standardized) regressor increases by one standard deviation, on average, the (standardized) regressand increases by β standard deviation units.

Advantages: Standard coefficients ignore the independent variable's scale of units, making comparisons easy.

Disadvantages: Such standardization can be misleading. The meaning of a standard deviation may vary markedly between non-normal distributions (e.g., skewed distributions).

Theory behind regression

Regression makes sense only if there is a sound theory behind it. Ensure to include all necessary predictor variables depending on the problem statement, theory, and previous knowledge.

Some variables may measure the same things—either keep one of them or combine them. Also consider possible interactions between variables.

💡 Why this matters: Including irrelevant or redundant variables can distort regression results; theoretical grounding protects against this.

Summary Statistics

In Excel, after entering data, click Data Analysis in the Data Ribbon, select Descriptive Statistics, and click OK. Click in the Input Range, select your data including variable names by dragging your mouse. Check the option Labels in First Row, check Summary Statistics, and click OK. The output appears in a new worksheet.

Correlation Matrix

In Excel, after entering data, click Data Analysis in the Data Ribbon, select Correlation, and click OK. Click in the Input Range, select your data including variable names by dragging your mouse. Check the option Labels in First Row and click OK. The resulting matrix of correlations helps understand the relations between variables.

Graphs

Graphs before regression include Scatter Diagrams, Scatter Diagrams with line plots, Histograms, and Pie Charts. In the Insert Ribbon, click on the required chart, provide the required information, and get the chart.

How to present Regression Results in research papers

Every paper uses a slightly different strategy. Standard information to report in a regression table includes:

• Dependent variable
• Explanatory variables
• Number of observations, sample period, data labels, etc.
• Estimates of intercept and other coefficients
• Standard errors of estimate
• Significance of variables/coefficients
• R-squared and other required statistics

It is common to present more than one regression result. Variables can be added or dropped and regressions performed.

Keeping or Dropping Variables

Regression results by adding or including variables vary markedly between non-normal distributions (e.g., skewed). A good strategy:

• Keep significant predictors
• Keep insignificant predictors with the expected sign
• Drop insignificant predictors with unexpected sign
• For significant predictors with unexpected sign, keep after review and including/excluding other variables

Examples of presented results

The lecture shows a sample regression table with two models. For example, in both Model 1 and Model 2, the dependent variable is Car Price. Millage coefficients are 855.25* and 852.55*, Weight coefficients are 1256.9** and 1246.1**, Foreign coefficients are 950.5* and 855.25*, and Price of Oil appears only in Model 2 as -132.25. Number of observations is 2500 for both, R-Squared is 0.65 and 0.69 respectively. *, ** significance at 1% and 5% respectively.

Regression related function in Microsoft Excel: A Brief review

The available functions include:

1. The LINEST (up to 50 independent variables)
2. Data Analysis ToolPak (up to 16 independent variables)
3. RSQ
4. Basic formulas: INTERCEPT, SLOPE, TREND

Steps before Regression

• Theory
• Graphs
• Summary statistics
• Model specification
• Regression
• Estimation
• Change of scale if required
• Different functional forms
• Interaction
• Post Regression: Change of scale if required, Different functional forms, Interaction

⭐ Key Takeaways

Standardization removes units of measurement so coefficients represent changes in standard deviation units, enabling direct comparison of variable importance—but this is only valid for normally distributed data. Regression must be grounded in economic theory, not just data mining. Summary statistics, correlation matrices, and graphs are essential pre-regression diagnostics in Excel. When presenting results, always report coefficients, standard errors, significance levels, R-squared, and sample size. A disciplined strategy for variable selection is to keep significant predictors and those with expected signs, while dropping insignificant predictors with unexpected signs.

🧠 Quick Revision Questions

1. How do you standardize a variable, and what are the properties of standardized variables?
2. How is a standardized (beta) coefficient interpreted in a regression model?
3. What is the main advantage and the main disadvantage of using standardized coefficients?
4. List the standard information that must be reported in a regression results table.
5. What is the recommended strategy for keeping or dropping variables based on their significance and sign?

📘 Lecture 13 — Qualitative Independent Variable / Dummy Variables

📖 Overview: This lecture introduces dummy variables — binary or polytomous variables used to encode qualitative information (e.g., gender, race) into regression models. It explains why ignoring such variables can cause bias or inefficiency, how to include them, interpret their coefficients, and handle interactions.

🗂️ Topics Covered

This lecture covers the definition and types of dummy variables, the consequences of ignoring qualitative variables (correlated or uncorrelated with other regressors), binary dummy variable regression with interpretation using the wage example, polytomous (multi-category) dummy variables with the dummy variable trap and reference category, and interaction effects between dummy variables and continuous predictors, including interaction with multiple categories.

📝 Lecture Summary

Qualitative Independent Variable / Dummy Variables

This section defines dummy variables (also called categorical, indicator, binary, or dichotomous variables) as variables that encode qualitative information into numerical form, most commonly as binary (0 or 1). They divide data into mutually exclusive categories. Examples include gender (0/1), yes/no, control vs. treatment group, race, and educational categories. The assignment of 0 and 1 is arbitrary — it does not matter which status gets which value.

🔑 Definition — Dummy Variable: A variable that encodes a qualitative characteristic, typically taking the value 0 or 1 to indicate the absence or presence of a category.

What happens if we ignore Gender?

If we ignore a qualitative variable like gender, the consequences depend on its relationship with other independent variables:

• Scenario 1 (Education and Gender not correlated): The slope estimates for other variables remain correct, but the model will have larger errors (higher standard errors).
• Scenario 2 (Education and Gender correlated): The slope estimates become biased and errors become larger, violating the Gauss-Markov theorem.

Possible solutions include: (1) running separate regressions for each gender, but this does not allow testing gender differences; (2) using dummy variables to capture the influence of gender in a single regression.

Example of binary dummy variable

Consider hourly wages (W) as a function of years of education (E), years of experience (X), and gender (G), where G = 1 for Female and G = 0 for Male. The regression model is: $$W = \beta_0 + \beta_1 E + \beta_2 X + \beta_3 G + e$$

Using the data file dummy.xlsx, the estimated regression is: $$\hat{W} = 77.76 + 8.29E + 28.52X - 26.6G$$

📌 Interpretation: The coefficient -26.6 means that female respondents' wages are, on average, 26.6 rupees less than males, holding education and experience constant. 📌 Example calculation: For a female with E=14, X=5: $$\hat{W} = 77.76 + 8.29(14) + 28.52(5) - 26.6(1) = 309.82$$ For a male with identical characteristics: $$\hat{W} = 77.76 + 8.29(14) + 28.52(5) - 26.6(0) = 336.42$$ The difference (336.42 − 309.82 = 26.6) is the average gender wage gap for individuals with the same education and experience.

Exam Model Question

Using the LINEST function from dummy.xlsx, the regression output for $W = \beta_0 + \beta_1E + \beta_2X + \beta_3G + e$ yields:

📌 Interpretation: The coefficient of the dummy variable (G) is -26.869, meaning females earn on average 26.87 rupees less than the reference (males). To test significance, calculate t-statistic = coefficient/SE: $$t = \frac{-26.869}{12.594} \approx -2.13$$ Compare with critical t-value (e.g., at 5% significance with 31 df ≈ 2.04). Since |−2.13| > 2.04, the dummy coefficient is statistically significant at the 5% level.

Polytomous qualitative variables

Polytomous variables are qualitative variables with more than two categories (e.g., race: White, Black, Asian). For k categories, we need k-1 binary dummy variables to avoid perfect multicollinearity (the dummy variable trap). The excluded category becomes the reference category.

For race with three categories:

• $D_1 = 1$ if Black, 0 otherwise
• $D_2 = 1$ if Asian, 0 otherwise
• $D_3 = 1$ if White, 0 otherwise (reference — not included in regression)

The regression model becomes: $$W = \beta_0 + \beta_1E + \beta_2X + \beta_3G + \beta_4D_1 + \beta_5D_2 + e$$

📌 Interpretation by race:

• White (reference): $W = \beta_0 + \beta_1E + \beta_2X + \beta_3G + e$
• Asian: $W = \beta_0 + \beta_1E + \beta_2X + \beta_3G + \beta_5 + e$
• Black: $W = \beta_0 + \beta_1E + \beta_2X + \beta_3G + \beta_4 + e$

From the estimated output (using dummy.xlsx with 35 observations, R² = 0.85, F = 33.83):

*, ** denote significance at 1% and 5% respectively. Black individuals earn on average 45.17 fewer rupees than White individuals, and Asian individuals earn 35.43 fewer rupees than White individuals.

Interaction with Dummy Variables

Interaction terms are used when the effect of one independent variable depends on the value of another independent variable. This is different from correlation (which is about variables being related to each other) — interaction is about dependence of effects. Interaction terms are created by multiplying the dummy variable with the continuous variable. This does not cause perfect collinearity because the product is a nonlinear function.

Example with gender and education interaction: $$W = \beta_0 + \beta_1E + \beta_2X + \beta_3G + \beta_4(G \times E) + e$$

📌 For Men (G=0): $W = \beta_0 + \beta_1E + \beta_2X + e$ 📌 For Women (G=1): $W = (\beta_0 + \beta_3) + (\beta_1 + \beta_4)E + \beta_2X + e$

The interaction coefficient $\beta_4$ captures how the effect of education on wages differs between men and women.

For multiple categorical variables (e.g., gender and race), include interaction terms with each dummy: $$W = \beta_0 + \beta_1E + \beta_2X + \beta_3G + \beta_4D_1 + \beta_5D_2 + \beta_6(G \times D_1) + \beta_7(G \times D_2) + e$$

📌 All possible group equations:

• White Men: $W = \beta_0 + \beta_1E + \beta_2X + e$
• Black Men: $W = (\beta_0 + \beta_4) + \beta_1E + \beta_2X + e$
• Asian Men: $W = (\beta_0 + \beta_5) + \beta_1E + \beta_2X + e$
• White Women: $W = (\beta_0 + \beta_3) + \beta_1E + \beta_2X + e$
• Black Women: $W = (\beta_0 + \beta_3 + \beta_4 + \beta_6) + \beta_1E + \beta_2X + e$
• Asian Women: $W = (\beta_0 + \beta_3 + \beta_5 + \beta_7) + \beta_1E + \beta_2X + e$

💡 Why this matters: Interaction terms allow the model to capture differential effects — for example, whether the gender wage gap is different for different races, or whether the return to education varies by gender.

⭐ Key Takeaways

1. Dummy variables encode qualitative data as binary (0/1) and are used to include categorical predictors in regression. For a variable with k categories, include k−1 dummies to avoid perfect multicollinearity (the dummy variable trap), with the omitted category serving as the reference group.
2. Ignoring a relevant qualitative variable that is correlated with other regressors causes biased and inconsistent slope estimates; if uncorrelated, estimates remain unbiased but standard errors increase.
3. The coefficient on a dummy variable measures the average difference in the dependent variable between that category and the reference category, holding other variables constant — this can be tested for statistical significance using t-tests.
4. Interaction terms (products of a dummy with a continuous variable) allow the slope of the continuous variable to differ across categories, capturing heterogeneous effects without causing perfect multicollinearity.
5. For polytomous variables with interaction across multiple categorical variables (e.g., race and gender), include all main effects plus all cross-product interaction terms — the resulting regression yields distinct intercepts for each subgroup combination.

🧠 Quick Revision Questions

1. Why do we include only k−1 dummy variables for a qualitative variable with k categories? What happens if we include all k?
2. In the wage regression $\hat{W} = 77.76 + 8.29E + 28.52X - 26.6G$, interpret the coefficient −26.6. Is it the gender gap for all individuals or only for those with identical E and X?
3. How would you test whether the coefficient on a dummy variable (e.g., gender) is statistically significant at the 5% level, given its estimate and standard error?
4. Explain the difference between correlation between independent variables and interaction between them. Give an example where interaction is needed but correlation is irrelevant.
5. Write the regression equation for a model with education (E), experience (X), gender (G), race (with dummies D1 for Black, D2 for Asian, ref=White), and all gender×race interaction terms. Then write the specific equation for a Black female.

📘 Lecture 14 — Transforming Variables in Regression

📖 Overview: This lecture introduces the use of logarithms in regression analysis to handle skewed distributions and interpret percentage relationships. It also provides a comprehensive overview of missing data mechanisms, including their probability distributions, and various methods for dealing with missing data through deletion and imputation techniques.

🗂️ Topics Covered

The lecture begins with the reasons for using logarithms in regression, including handling positively skewed variables and linearizing relationships. It then presents an empirical example using wage data with tenure and race. The remainder of the lecture focuses on missing data: why data may be missing, the probability of missing data across groups, missing data mechanisms (MCAR, MAR, MNAR), and methods for dealing with missing data including listwise deletion, pairwise deletion, single imputation, and model-based imputation.

📝 Lecture Summary

Using logs in regression analysis

Logarithms are used in regression when a variable has a positively skewed distribution, when residuals are skewed, when changes in variables are related in percentage terms, or to linearize a relationship. Taking the natural log of a variable can make its distribution more symmetric.

Logs in regression

The lecture presents regression results using the nlsw88.dta dataset, modeling wages with tenure and a dummy variable for race.

🔑 Definition — Log-level model: A regression where the dependent variable is in logarithmic form but the independent variables are in their original units. 📐 Formula: ln(wage) = β0 + β1(tenure) + β2(race) + ε → A one-unit increase in tenure is associated with a 100*β1 percent change in wage. 📌 Example: From the table, the coefficient on tenure in the ln(wage) model is 0.0317. This means one additional year of tenure is associated with a 3.17% increase in wage (holding race constant). The R-squared increases from 0.039 to 0.1043 when using ln(wage) instead of wage, indicating a better fit.

Treatment of Missing Data

Data may be missing for various reasons including attrition (death, dropouts, migration), lack of legal availability, non-response or refusal by respondents, conditional questions in surveys, and data collection or encoding issues.

Probability of Missing Data

Certain groups are more likely to have missing values. Examples include businessmen compared to salaried individuals, rural areas compared to urban areas in developing countries, less educated individuals compared to educated ones, and developing countries compared to developed countries. Some variables are more prone to missingness, such as income (especially for high-income groups), area-specific variables, and variables related to taboos.

Missing Data Mechanism: probability distributions

1. MCAR: Missing Completely at Random The probability that Y is missing is independent of both Y and X. Pr(Y is missing | X, Y) = Pr(Y is missing)

2. MAR: Missing at Random The probability that Y is missing depends on X but not on Y itself. Pr(Y is missing | X, Y) = Pr(Y is missing | X) 📌 Example: The probability of missing income depends on occupation, but within each occupation, the probability is not dependent on income level.

3. Ignorable MAR This is essentially MAR where the parameters governing the missing data are distinct from the parameters being estimated. The missing data mechanism is said to be ignorable.

4. MNAR: Missing Not at Random The probability that Y is missing depends on a variable that is itself missing. This is the most problematic case. 💡 Why this matters: With MNAR, the data contains no information about the missing values, and results become very sensitive to the choice of model. Special techniques like Heckman regression are needed.

Dealing with missing data: Deletion Methods

1. Listwise deletion (complete case analysis) Only rows with no missing values for any variable are analyzed. Software like Stata may do this automatically. 📌 Example: If any variable in a row is missing, the entire row is excluded from the analysis.

• Advantages: Simple to implement.
• Disadvantages: Reduces statistical power (sample size decreases).

2. Pairwise deletion (available case analysis) Missing values are handled on an analysis-to-analysis basis, using all available data for each specific calculation. 📌 Example: Different observations may be used for calculating a correlation than for a regression.

• Advantages: Uses all possible information for each case.
• Disadvantages: The sample differs each time, making comparisons across models difficult.

Imputation

1. Single Imputation

a) Substitute Mean / Median / Mode

• Method: Replace missing values with the mean, median, or mode of the observed values.
• Problems: Weaker covariance and reduced variability; standard errors are artificially lowered.

b) Dummy variable control

• Method: Create an indicator (dummy) variable for rows with missing values, fill in missing values using single imputation, then include the dummy in the regression.
• Problems: Estimates are biased and this method lacks a strong theoretical foundation.

c) Regression Imputation

• Method: Use regression to predict and substitute the missing values.
• Problems: Overestimates model fit and produces weaker variance.

Model Based Imputation

1. Maximum Likelihood

• Method: Identify the set of parameters that produces the maximum log likelihood — the value most likely to have resulted in the observed data.
• Uses full estimation and provides unbiased estimates.
• Problem: Standard errors are biased downwards.

2. Multiple Imputation

• Method: Data is filled in by applying specific regressions. This experiment is repeated n times, generating separate datasets. Regression is performed on each dataset, and then the results are pooled.
• Advantages: Good variability is preserved.
• Disadvantages: Errors are possible when specifying the regression models used for imputation.

⭐ Key Takeaways

The most critical concepts from this lecture are: first, know when and why to use log transformations in regression, particularly for skewed distributions and percentage-change interpretations. Second, understand the three main missing data mechanisms (MCAR, MAR, MNAR) and be able to distinguish them based on whether missingness depends on Y or X. Third, recognize that listwise deletion reduces power while pairwise deletion uses changing samples. Fourth, know the difference between single imputation methods (mean substitution, dummy control, regression) and model-based methods (maximum likelihood, multiple imputation), including the key problems with each. Finally, be aware that MNAR is the most serious missing data problem and requires specialized estimation.

🧠 Quick Revision Questions

1. What are four reasons for using logarithms in regression analysis?
2. In the log-level model where ln(wage) is regressed on tenure, how is the coefficient on tenure interpreted?
3. What distinguishes MAR from MCAR in missing data mechanisms?
4. What is the main disadvantage of listwise deletion when dealing with missing data?
5. What is the key advantage of multiple imputation over single imputation methods?

📘 Lecture 15 — Multicollinearity

📖 Overview: This lecture explores the concept of multicollinearity in regression analysis, distinguishing between perfect and imperfect forms. It explains the sources, theoretical and practical consequences, and methods for detecting this common data problem, which is crucial for accurate model interpretation and valid inference.

🗂️ Topics Covered

The lecture begins by defining perfect and near/imperfect multicollinearity, illustrating perfect multicollinearity with a mathematical example showing parameter non-identifiability. It then presents an empirical example contrasting perfect and imperfect multicollinearity using Excel output. Sources of multicollinearity are listed, followed by theoretical consequences (unbiasedness, data deficiency) and practical consequences (large standard errors, low t-ratios, high R²). The lecture concludes with detection methods: symptoms, correlation matrix, and Variance Inflation Factor (VIF)/Tolerance, including a decision table for severity.

📝 Lecture Summary

Perfect Multicollinearity: Definition

Perfect Multicollinearity is defined as an exact or perfect linear relationship between a pair or more of the explanatory variables. In a K-variable linear regression model Y = β₀X₀ + β₁X₁ + β₂X₂ + β₃X₃ + ... + βₖXₖ + eᵢ (where X₀ = 1 for all observations to allow the intercept term), an exact linear relationship exists if λ₀X₀ + λ₁X₁ + λ₂X₂ + ... + λₖXₖ = 0, where λᵢ cannot all be zero at the same time.

🔑 Definition — Perfect Multicollinearity: An exact linear relationship between two or more explanatory variables, where one variable can be expressed as a perfect linear combination of the others.

Near or Imperfect Multicollinearity: Definition

In practice, we rarely observe perfect multicollinearity but rather a degree of multicollinearity. For the model Y = β₀X₀ + β₁X₁ + β₂X₂ + β₃X₃ + ... + βₖXₖ + eᵢ, a near or imperfect linear relationship exists if λ₀X₀ + λ₁X₁ + λ₂X₂ + ... + λₖXₖ + vᵢ = 0, where vᵢ is a stochastic error term.

🔑 Definition — Near/Imperfect Multicollinearity: A situation where explanatory variables are highly, but not perfectly, correlated, meaning one variable is approximately a linear combination of others plus a random disturbance.

Understanding Perfect Multicollinearity

For perfect multicollinearity, if λ₀X₀ + λ₁X₁ + λ₂X₂ + ... + λₖXₖ = 0 and all λᵢ except λ₁ and λ₂ are zero, then X₁ = (λ₂/λ₁)X₂ (e.g., X₁ = 5X₂). Both variables can be expressed as linear combinations of each other. Substituting into the regression: Y = β₀ + β₁X₁ + β₂X₂ + eᵢ becomes Y = β₀ + β₁(5X₂) + β₂X₂ + eᵢ = β₀ + (5β₁ + β₂)X₂ + eᵢ = β₀ + βₘX₂ + eᵢ. This is a simple regression where Y depends only on X₂. Hence, we cannot estimate all the parameters.

📐 Formula: X₁ = (λ₂/λ₁)X₂ → One explanatory variable is a perfect multiple of another. 📌 Example: With X₁ = 5X₂ and the model Y = β₀ + β₁X₁ + β₂X₂ + eᵢ, substituting gives Y = β₀ + (5β₁ + β₂)X₂ + eᵢ = β₀ + βₘX₂ + eᵢ. Only the combined coefficient βₘ can be estimated, not β₁ or β₂ individually. Software may drop one variable (e.g., X₁) and show a zero coefficient, effectively estimating a simple regression of Y on X₂.

💡 Why this matters: Perfect multicollinearity makes it impossible to estimate unique coefficients for the correlated variables, rendering the model unidentified.

An example dataset is provided with 10 observations for Y, X₁, and X₂ where X₁ = 5X₂. The Excel LINEST output shows coefficients for X₁ (0) and X₂ (0.8826), an intercept (-31.5), R² (0.974), standard errors, and t-statistics (17.17 for X₂). The zero coefficient for X₁ confirms it was dropped due to perfect collinearity.

The data is then slightly changed to create imperfect multicollinearity (e.g., Y: 10, X₁: 10, X₂: 52). The new LINEST output estimates all parameters: X₁ coefficient = 1.49 (se=0.86, t=1.74), X₂ coefficient = -2.93 (se=4.24, t=-0.69), intercept = -33.31 (se=4.85, t=-6.86), R² = 0.98. Note that standard errors are large, t-statistics are low, coefficients may not seem significant, but the F-test shows a good fit.

Sources of Multicollinearity

1. The data collection method: Sampling over a limited range or subgroups of a population can create correlation that only exists in that subgroup.
2. Natural Constraints on the model or population: For example, if we regress GDP on exports and imports, high imports normally imply high exports as well; in a sample survey, both income and status of house are included as explanatory variables; model specification, such as adding polynomial terms, especially when X has a small range.
3. Over-determined model: A large number of explanatory variables with very low degrees of freedom.
4. Common Time Series Trend: The explanatory variables may share a common trend over time.

Theoretical consequences of Multicollinearity

1. OLS estimators remain unbiased: There is no violation of the BLUE property; estimators are still unbiased and best linear unbiased.
2. Interpretation of coefficients is not independent: A simple ceteris paribus interpretation does not seem to be valid.
3. Multicollinearity is a data deficiency problem: Sometimes it is difficult to increase the sample size due to cost and time constraints.
4. Multicollinearity is a sample phenomenon: This is especially true with non-experimental data that occurs naturally, like GDP and its determinants.
5. We need larger samples: Larger samples are required than without multicollinearity.

Practical consequences of Multicollinearity

• Large variances and standard errors of coefficients: This occurs when the correlation between pairs of explanatory variables is high, leading to a fall in the precision of estimators.
• Wider confidence intervals: This is a direct consequence of larger standard errors.
• Insignificant t-ratios: As a consequence of larger standard errors, the null hypothesis H₀: βᵢ = 0 tends to be accepted.
• High R² but low t: This seems to be a contradictory result.
• Estimators and standard errors are very sensitive to changes in data: Model estimates are unstable.
• Wrong signs of coefficients: The estimated signs may not align with economic and finance theory.
• Difficulty in assessing the individual contribution of regressors: Due to correlated regressors, it is hard to isolate each variable's unique contribution.

Detecting Multicollinearity

Remember that multicollinearity is a question of degree, not just presence; it is a sample phenomenon; and it is a data deficiency problem. There are three detection methods:

1. Look at the symptoms/indicators: High R² but low t-statistics, large standard errors, and individual variables not being significant even though the model seems to be a good fit.
2. Look at the Correlation Matrix: High pairwise correlations between the explanatory variables (e.g., greater than 0.8). A sample correlation matrix is provided for data containing Y, X₁, X₂, and X₃.
3. Calculate VIF or Tolerance:
   • Method: Run auxiliary regressions where each explanatory variable is regressed on all other explanatory variables, and find the coefficient of determination (R²).
   • You can test the significance of R² using an F-test or goodness-of-fit test.
   • Estimate Tolerance or VIF.

📐 Formulas:

• Tolerance = 1 - R²
• VIF = 1 / (1 - R²) → Variance Inflation Factor measures how much the variance of the coefficient is inflated due to collinearity. 🔑 Key Definitions:
• Auxiliary Regression: A regression of one explanatory variable on all other explanatory variables to calculate R² for VIF.
• Tolerance: 1 - R² from the auxiliary regression; low values indicate high multicollinearity.
• VIF (Variance Inflation Factor): 1 / Tolerance; a measure of how much the variance of an estimated coefficient is inflated due to collinearity.

Decision Table for Degree of Multicollinearity

Important Points to note:

• VIF or Tolerance for a variable is calculated by the R² of the auxiliary regression of that variable regressed on all other explanatory variables.
• One single variable is not responsible for multicollinearity, so we may estimate a number of auxiliary regressions equal to one less than the number of explanatory variables.

⭐ Key Takeaways

Multicollinearity is a data problem of degree, not just presence, where OLS estimators remain unbiased but lose precision, leading to large standard errors, low t-ratios, and potentially unstable coefficient estimates. Perfect multicollinearity is a crippling exact linear relationship that prevents unique parameter estimation, whereas imperfect multicollinearity is common and manifests when a model has high R² but insignificant individual t-tests. Detection relies on three methods: observing symptoms (high R², low t), examining pairwise correlations in a matrix, and using auxiliary regressions to compute VIF values, with a VIF above 10 indicating a problematic level of multicollinearity.

🧠 Quick Revision Questions

1. What is the key difference between perfect and imperfect (near) multicollinearity?
2. Why can OLS estimators remain unbiased under multicollinearity, yet their standard errors become large?
3. List and explain the three main methods for detecting multicollinearity in a regression model.
4. How is the Variance Inflation Factor (VIF) calculated, and what does a VIF value of 10 or more indicate?
5. According to the lecture, what is the primary practical consequence of high multicollinearity on the t-ratios of the estimated coefficients?

📘 Lecture 16 — Multicollinearity: Remedial Measures

📖 Overview: This lecture presents practical solutions for dealing with multicollinearity in regression models. It covers several remedial measures ranging from doing nothing to dropping variables or transforming them, illustrated with detailed business and economics examples using real datasets. Understanding these remedies is crucial for obtaining reliable coefficient estimates when predictors are highly correlated.

🗂️ Topics Covered

The lecture covers seven remedial measures for multicollinearity: doing nothing, dropping one variable, transforming variables, getting additional data, combining cross-section and time series, using panel data, and advanced methods like ridge regression. It then presents two detailed business examples: first using data on quantity demanded with prices and income, and second using Pakistani GDP data with population, investment, and trade variables. Each example demonstrates the diagnostic process using VIF, F-statistics, and correlation matrices, followed by application of remedial measures.

📝 Lecture Summary

1. Do nothing

If multicollinearity is mild or if the purpose is only forecasting, you may choose to do nothing. If data deficiency is the problem and we have no choice over data, it is better to try to increase the data by extending the sample if possible. Multicollinearity is not a problem if theory permits us to estimate the missing coefficient — e.g., in Cobb-Douglas production function, if we assume constant returns to scale, either of alpha or beta can be estimated if one is estimated by regression.

2. Drop one of the variables

Drop the variable that is less significant or drop the one with larger VIF. But this may lead to wrong model specification or may go against theoretical considerations (e.g., dropping price of substitute in demand function). An example of dropping variables is of import and export in GDP equation.

3. Transform the variable

Combine the variables — e.g., just add exports and imports to get a new variable labeled as openness. Another option is to convert the variables (import = f(GNP, CPI) can divide by CPI to get real imports = f(real GNP)); but error term may become heteroskedastic. Another way is to use first difference form (loss of one observation):

[ Y_t - Y_{t-1} = \beta_2(X_{2t} - X_{2,t-1}) + \beta_3(X_{3t} - X_{3,t-1}) + v_t ]

This may not be appropriate in cross sectional data — it has no sense.

Other options include: 4. Get additional data and increase the sample size 5. Combine cross section and time series (pool) 6. Use of panel data 7. Use ridge regression, factor analysis, etc.

Multicollinearity: Examples from Business and Economics

Consider the data given in MC.xlsx containing quantity demanded, prices, monthly income in thousands, and prices of two different substitutes. First example has 50 observations. Let us consider a small example with 10 observations:

Auxiliary Regressions produced using Microsoft Excel:

🔑 Definition — Auxiliary Regression: A regression of one explanatory variable on the other explanatory variables to detect multicollinearity — a high R² indicates multicollinearity exists.

Regression: X₁ on X₂ & X₃

• First two variables show MC
• Look at F
• Look at T/VIF

[ X_1 = 4.977 + 0.188X_2 - 0.176X_3 ] R² = 0.998 F = 1848.5 Tolerance = 0.002 VIF = 500

Regression: X₂ on X₁ & X₃ [ X_2 = -22.11 + 5.24X_1 + 0.794X_3 ] R² = 0.9978 F = 1608.6 Tolerance = 0.0022 VIF = 454.5455

Regression: X₃ on X₁ & X₂ [ X_3 = 28.97 - 2.03X_1 + 0.3296X_2 ] R² = 0.89 F = 28.43 Tolerance = 0.11 VIF = 9.090909

💡 Why this matters: VIF values above 10 (or tolerance below 0.1) indicate serious multicollinearity — X₁ and X₂ show extremely high VIF values of 500 and 454.5.

Multicollinearity: Example-1 from Business and Economics

Using MC.xlsx, first look at the correlation matrix:

Model 1 (N=10) R Square = 0.9106, F = 12.73**

Model 2 (N=50) R Square = 0.978, F = 508.41*

Note: In the second model VIF has decreased but still greater than 10.

Model 3 (N=50) — Dropping Substitute-2: R Square = 0.978, F = 687.98*

Multicollinearity: Example-2 from Business and Economics

Data on GDP, Population, GFCF, Exports, Imports (2000-2012 Pakistan) downloaded from WDI.

Model with all variables: Dependent Variable: GDP R-square = 0.9927, F = 271.4

Result: No significance, high VIF, wrong signs

After dropping imports: Dependent Variable: GDP R-square = 0.9897, F = 289.17

New Results show: better significance, low VIF, signs correct now

Another option — Transform variables: Generate a new variable, volume of trade as a proxy for openness, defined as sum of imports and exports.

Dependent Variable: GDP R-square = 0.7947, F = 19.36

Results: openness is significant, low VIF indicates no multicollinearity, signs of coefficients are correct now — however we may need other variables.

⭐ Key Takeaways

The most critical things to remember are: first, multicollinearity can be addressed through seven possible remedial measures — doing nothing (if mild or for forecasting), dropping variables, transforming them, increasing sample size, pooling data, using panel data, or advanced methods like ridge regression. Second, auxiliary regressions produce high R² values and VIF exceeding 10 indicate serious multicollinearity — in the first example, the price of substitute variables had VIF values of 33 and 22.75, dropping one variable to VIF 13.6 (still above 10). Third, increasing sample size from 10 to 50 observations reduced VIF from 33.04 to 13.74 for Substitute-1. Fourth, in the Pakistan GDP example, dropping imports (VIF=14.36) improved all other variables' significance and signs. Fifth, transforming variables by creating an openness measure (exports + imports) successfully eliminated multicollinearity but reduced R-square from 0.99 to 0.79.

🧠 Quick Revision Questions

1. When is it appropriate to "do nothing" about multicollinearity?
2. What is the formula for first differencing to transform variables, and what is its limitation for cross-sectional data?
3. In Example-1, what happened to the VIF values when sample size increased from 10 to 50 observations?
4. In the Pakistan GDP example, what was the VIF for imports before dropping it, and what happened to the significance of exports after dropping imports?
5. How does creating an "openness" variable (sum of exports and imports) affect the regression results compared to including both separately?

📘 Lecture 18 — Detection of Heteroskedasticity: Formal Tests

📖 Overview: This lecture covers formal statistical tests for detecting heteroskedasticity in regression models. It explains multiple testing procedures including the Goldfeld-Quandt test, Park LM test, Harvey-Godfrey test, Glejser test, Breusch-Pagan test, and the White test, detailing their steps, assumptions, and limitations.

🗂️ Topics Covered

The lecture begins with the Goldfeld-Quandt test for heteroskedasticity detection using ordered data and subsample regressions, followed by its example and drawbacks. It then covers the Park LM test using logarithmic auxiliary regressions, the Harvey-Godfrey test using level variables, the Glejser test with multiple functional forms, and the Breusch-Pagan test with squared residuals. The lecture concludes with the White test, which includes squared terms and cross-products, and discusses the sensitivity of LM tests to normality violations.

📝 Lecture Summary

The Goldfeld-Quandt Test: Stephen Goldfeld, Richard Quandt

The following steps are performed for this test.

Step 1. Arrange the data from small to large values of the explanatory variable Xⱼ (the one we suspect responsible for HSK).

Step 2. Omit the middle C observations (C = roughly 20% observations).

Step 3. Run two separate regressions, one for small values of Xⱼ and one for large values of Xⱼ, omitting C middle observations and record the residual sum of squares RSS for each regression: RSS₁ for large values of Xⱼ and RSS₂ for small values of Xⱼ.

Step 4. Calculate the ratio: F = RSS₁ / RSS₂

Degrees of Freedom = (N−C)/2 − K both in the numerator and the denominator, where N = total number of observations, C is the number of omitted observations, and K is the Number of explanatory variables + one.

Step 5. Test if F > F_α, (N−C)/2−K, (N−C)/2−K to reject homoskedasticity (H₀).

🔑 Definition — Goldfeld-Quandt Test: A formal test for heteroskedasticity that compares the residual variance from two separate regressions on ordered data subsamples. 📐 Formula: F = RSS_high / RSS_low → Tests if variance is larger in one subsample 📌 Example: H₀: Homoskedasticity, H₁: Heteroskedasticity, α = 0.05 or 0.01. Test statistic F = RSS_high / RSS_low. If you think var(e) is increasing function of X. Region of Rejection: F > F_α, (N−C)/2−K, (N−C)/2−K. RSS = Residual Sum of Squares, N = Number of observations, C = central observations excluded, K = Number of parameters estimated = Number of explanatory variables plus one.

💡 Why this matters: The Goldfeld-Quandt test is intuitive but limited to single-variable heteroskedasticity detection.

The Goldfeld-Quandt Test: Drawbacks

It cannot handle situations where several variables jointly cause Heteroskedasticity. There is no fix rule to know how many middle observations should be excluded. It is also possible that the difference in variance of errors may be observed in subsamples with different number of observations. The middle C observations are lost. It accounts only for linear relationship of independent variable and the variance of errors.

The Park LM Test

The following steps are performed in this test.

Step 1. Run the required regression e.g. Y = β₀ + β₁X₁ + β₂X₂ + ... + βₖXₖ + e, obtain residuals e, compute ln e².

Step 2. Run the auxiliary regression: ln e² = α₀ + α₁lnX₁ + α₂lnX₂ + ... + αₖlnXₖ + u

Step 3. Compute LM = N·R² (N and R² are from the auxiliary regression).

Step 4. If LM > χ²_ₖ₋₁ then reject Null hypothesis and conclude that there is significant evidence of Heteroskedasticity.

🔑 Definition — Park LM Test: A test for heteroskedasticity using logarithmic auxiliary regression of squared residuals on log-transformed independent variables.

Harvey-Godfrey Test

The following steps are performed in this test.

Step 1. Run the required regression e.g. Y = β₀ + β₁X₁ + β₂X₂ + ... + βₖXₖ + e, obtain residuals e, compute ln e².

Step 2. Run the auxiliary regression [assume σ² = exp(α₀ + αₖXₖ)]: ln e² = α₀ + α₁X₁ + α₂X₂ + ... + αₖXₖ + u

Step 3. Compute LM = N·R² (N and R² are from the auxiliary regression).

Step 4. If LM > χ²_ₖ₋₁ then reject Null hypothesis and conclude that there is significant evidence of Heteroskedasticity.

🔑 Definition — Harvey-Godfrey Test: A test for heteroskedasticity using logarithmic auxiliary regression on the original (untransformed) independent variables.

Glejser Test

The following steps are performed in this test.

Step 1. Run the required regression e.g. Y = β₀ + β₁X₁ + e, obtain residuals e, compute ln e².

Step 2. Run the auxiliary regressions [assume σ² = exp(α₀ + αₖXₖ)]: |eᵢ| = α₀ + α₁X₁ + uᵢ |eᵢ| = α₀ + α₁√X₁ + uᵢ |eᵢ| = α₀ + α₁(1/X₁) + uᵢ

Step 3. Compute LM = N·R² (N and R² are from the auxiliary regression).

Step 4. If LM > χ²_ₖ₋₁ then reject Null hypothesis and conclude that there is significant evidence of Heteroskedasticity.

🔑 Definition — Glejser Test: A test for heteroskedasticity that uses the absolute value of residuals and tests multiple functional forms of the relationship with independent variables.

Breusch-Pagan Test

The following steps are performed in this test.

Step 1. Run the required regression e.g. Y = β₀ + β₁X₁ + β₂X₂ + ... + βₖXₖ + e, obtain residuals e, compute ln e².

Step 2. Run the auxiliary regression [assume σ² = exp(α₀ + αₖXₖ)]: e² = α₀ + α₁X₁ + α₂X₂ + ... + αₖXₖ + u

Step 3. Compute LM = N·R² (N and R² are from the auxiliary regression).

Step 4. If LM > χ²_ₖ₋₁ then reject Null hypothesis and conclude that there is significant evidence of Heteroskedasticity.

OR test F-statistic for the above regression in step 2 (Goodness of fit).

🔑 Definition — Breusch-Pagan Test: A test for heteroskedasticity that regresses squared residuals on the original independent variables in linear form.

Problems with Breusch-Pagan and others

Specification of model for variance dependence is needed e.g. Breusch Pagan assume linear relation. If the errors are not normally distributed, then these tests may not be valid. Breusch Pagan Test has been shown to be sensitive to any violation of the normality assumption. Three other popular LM tests: the Glejser test; the Harvey-Godfrey test, and the Park test, are also sensitive to such violations.

The White Test: Most popular

The following steps are performed in this test.

Step 1. Run the required regression e.g. Y = β₀ + β₁X₁ + β₂X₂ + e, obtain residuals e, compute ê².

Step 2. Run the auxiliary regression: e² = α₀ + α₁X₁ + α₂X₂ + α₃X₁² + α₄X₂² + α₅X₁X₂ + u (Include all square and product terms)

OR Run the regression: e² = γ₀ + γ₁Ŷ + γ₂Ŷ² + v

Step 3. Compute LM = N·R² (N and R² are from the auxiliary regression) OR Compute the F-Statistic.

Step 4. If LM > χ²_ₖ₋₁ then reject Null hypothesis and conclude that there is significant evidence of Heteroskedasticity OR test F-statistic for the above regression in step 2 (Goodness of fit).

🔑 Definition — White Test: The most popular test for heteroskedasticity that includes squared terms and cross-products of independent variables in the auxiliary regression, without requiring a specific functional form for the variance.

⭐ Key Takeaways

The Goldfeld-Quandt test requires ordering data by the suspected variable and comparing RSS from two subsamples, but it cannot handle multiple heteroskedasticity sources and loses middle observations. The Park, Harvey-Godfrey, Glejser, and Breusch-Pagan tests are LM tests using auxiliary regressions with N·R² statistic, but they require specifying the variance model and are sensitive to non-normality of errors. The White test is the most popular because it does not require specifying a particular functional form for heteroskedasticity and includes squared and cross-product terms in the auxiliary regression. All LM tests compare the test statistic to a chi-square distribution with (K−1) degrees of freedom. The Breusch-Pagan test assumes a linear relationship between squared residuals and independent variables, making it vulnerable to misspecification.

🧠 Quick Revision Questions

1. What are the five steps of the Goldfeld-Quandt test and how is the F-statistic calculated?
2. What is the main drawback of the Goldfeld-Quandt test regarding multiple explanatory variables?
3. How does the White test differ from other LM tests in terms of the auxiliary regression specification?
4. Why are the Breusch-Pagan, Glejser, Harvey-Godfrey, and Park tests sensitive to violations of normality?
5. What is the critical value and degrees of freedom used for all LM-based heteroskedasticity tests?

📘 Lecture 19 — Examples of detecting Heteroskedasticity

📖 Overview: This lecture demonstrates how to apply various formal tests for detecting heteroskedasticity in regression models using real data examples. It also covers methods for handling heteroskedasticity when detected, including model transformation, White standard errors, and Weighted Least Squares (WLS). Understanding these techniques is crucial for ensuring valid statistical inference in econometric analysis.

🗂️ Topics Covered

The lecture begins with the general procedure for all heteroskedasticity tests including null and alternative hypotheses, significance levels, test statistics (LM and F), and rejection regions. It then works through detailed examples of five different tests: the Goldfeld-Quandt test (revision), the Park LM test, the Harvey-Godfrey test, the Glesjer test, and the Breusch Pagan test, followed by the White test with two independent variables. The second half covers three methods for handling heteroskedasticity: changing model specification, using Huber/White standard errors, and applying GLS/WLS with known and unknown error variance, including transformation examples and a worked example.

📝 Lecture Summary

General Procedure for all the tests

The general procedure for testing heteroskedasticity begins with setting up the null hypothesis (H_0): Homoskedasticity (constant error variance) against the alternative hypothesis (H_1): Heteroskedasticity (non-constant error variance), typically using a significance level (\alpha = 0.05) or (\alpha = 0.01). The test statistic can be either the Lagrange Multiplier (LM) statistic calculated as (LM = N \cdot R^2), or the F-statistic calculated as (F = \frac{R^2/(1-R^2)}{(N-K)/(K-1)}). For the Goldfeld-Quandt test, the F-statistic is calculated as (F = \frac{RSS_{high}}{RSS_{low}}). The region of rejection for the LM test is when (LM > \chi^2_{\alpha, K-1}), and for the F-test when (F > F_{\alpha, K-1, N-K}) (or for Goldfeld-Quandt, (F > F_{\alpha, \frac{N-C}{2}-K, \frac{N-C}{2}-K})). All tests follow the same decision rule: if the test statistic exceeds the critical value, reject the null hypothesis and conclude that heteroskedasticity is present.

🔑 Definition — Homoskedasticity: The assumption that the variance of the error terms is constant across all observations, i.e., (Var(e_i) = \sigma^2) for all (i).

🔑 Definition — Heteroskedasticity: A violation of the classical linear regression assumption where the variance of the error terms is not constant across observations.

📐 Formula: LM Test Statistic: (LM = N \cdot R^2) → The sample size multiplied by the R-squared from the auxiliary regression. 📐 Formula: F-Test Statistic: (F = \frac{R^2/(1-R^2)}{(N-K)/(K-1)}) → The ratio of explained to unexplained variation, adjusted for degrees of freedom.

The Goldfeld - Quandt Test: Example Revision

The Goldfeld-Quandt test is revisited as a revision example. This test involves ordering observations by the variable suspected to cause heteroskedasticity, omitting some central observations (C), and then running separate regressions on the low-variance and high-variance subsamples. The test statistic is the ratio of the Residual Sum of Squares (RSS) from the high-variance group to the low-variance group: (F = \frac{RSS_{high}}{RSS_{low}}). If this ratio exceeds the critical F-value, heteroskedasticity is present.

Example: The Park LM Test (ln(square of errors) and ln(income))

The Park LM test uses the natural logarithm of the squared errors as the dependent variable in an auxiliary regression. First, run the original regression of Saving on Income. Then generate the squared errors and take their natural logarithm, and also take the natural logarithm of the income variable. Run the auxiliary regression: (ln(e^2) = \alpha_0 + \alpha_1 ln(Income) + u). From the auxiliary regression, (R^2 = 0.0634). The LM statistic is (LM = N \cdot R^2 = 13(0.0634) = 0.8243). The critical Chi-square value at 5% with 1 degree of freedom is (CHISQ.INV.RT(0.05,1) = 3.84146). Since (LM (0.8243) < \chi^2_{0.05,1} (3.8415)), we cannot reject (H_0) and conclude that errors are not heteroskedastic. The F-test confirms this: (F = \frac{R^2/(1-R^2)}{(N-K)/(K-1)} = 0.7446), and (F_{0.05,1,11} = 4.8443), so (F < F_{critical}), again we cannot reject homoskedasticity.

📌 Example: Park LM Test Data — For 13 observations of Savings and Income, the squared errors range from 1,761.62 to 1,418,726.13, and the ln(e²) values range from 7.4740 to 14.1653. The auxiliary regression of ln(e²) on ln(Income) yields R² = 0.0634, indicating that only 6.34% of the variation in ln(e²) is explained by ln(Income). Both the LM test (0.8243 < 3.8415) and F-test (0.7446 < 4.8443) fail to reject homoskedasticity.

Example: Harvey-Godfrey Test for Heteroskedasticity

The Harvey-Godfrey test differs from the Park test in that it uses the original Income variable (not logged) as the explanatory variable in the auxiliary regression. After running the original regression of Savings on Income, find the errors and then (ln(square \ of \ errors)). Run the auxiliary regression: (ln(e^2) = \alpha_0 + \alpha_1 Income + u). From this auxiliary regression, (R^2 = 0.087076). The LM statistic is (LM = N \cdot R^2 = 13(0.087076) = 1.13199). The critical Chi-square value is 3.84146. Since (LM (1.132) < \chi^2_{0.05,1} (3.8415)), we cannot reject (H_0) and conclude that errors are not heteroskedastic using the LM test. Interestingly, the F-test gives (F = 1.0492) and (F_{critical} = 4.8443), so (F < F_{critical}), also indicating no heteroskedasticity. In this example, both tests agree that heteroskedasticity is not present, though the lecture text notes that sometimes the LM and F-tests can give conflicting conclusions.

📌 Example: Harvey-Godfrey Test — The auxiliary regression of ln(squared errors) on Income (not logged) gives R² = 0.0871. LM = 13 × 0.0871 = 1.132, which is less than the critical value of 3.8415, so we cannot reject the null hypothesis of homoskedasticity. The F-statistic (1.049) is also less than the critical value (4.844), confirming the conclusion.

Example: Glesjer's Test for Heteroskedasticity

The Glesjer test is more flexible because it allows for different functional forms of the relationship between the absolute errors and the independent variable. After running the original regression of Savings on Income, find the errors, then create variables (1/X) and (\sqrt{X}). Run three different auxiliary regressions:

• Regression 1: (|e_i| = \alpha_0 + \alpha_1 X + u_i) → R² = 0.11666, LM = 1.51657
• Regression 2: (|e_i| = \alpha_0 + \alpha_1 \sqrt{X} + u_i) → R² = 0.10218, LM = 1.3284
• Regression 3: (|e_i| = \alpha_0 + \alpha_1 (1/X) + u_i) → R² = 0.04528, LM = 0.5887

For all three regressions, the LM statistic is less than the critical value of 3.84146, so we have strong evidence not to reject (H_0) in all three cases. This means that regardless of which functional form we assume for the relationship between errors and X, we conclude that errors are not heteroskedastic.

Example: Breusch Pagan Test for Heteroskedasticity

The Breusch Pagan test uses the square of the errors (not logged) as the dependent variable in the auxiliary regression on the original independent variable(s). After running the original regression of Savings on Income, find the errors, then square them. Run the auxiliary regression: (e^2 = \alpha_0 + \alpha_1 Income + u). From this auxiliary regression, (R^2 = 0.143182). The LM statistic is (LM = N \cdot R^2 = 13(0.143182) = 1.86136). The critical Chi-square value is 3.84146. Since (LM (1.861) < \chi^2_{0.05,1} (3.8415)), we cannot reject (H_0) and conclude that errors are not heteroskedastic. The F-statistic is (F = 1.8382) with a critical value of (F_{0.05,1,11} = 4.8443). Since (F < F_{critical}), the F-test also indicates no heteroskedasticity.

Example: White Test for Heteroskedasticity - version 1

The White test is the most general test because it uses the squared errors as the dependent variable and regresses them on the original independent variables, their squares, and their cross-products. This test requires at least two independent variables. First, run a regression of Savings on Income and interest rate, find the errors, then square them. Then run the auxiliary regression: (e^2 = \alpha_0 + \alpha_1 X_1 + \alpha_2 X_2 + \alpha_3 X_1 \cdot X_2 + \alpha_4 X_1^2 + \alpha_5 X_2^2 + u). From this auxiliary regression, (R^2 = 0.5021). The LM statistic is (LM = N \cdot R^2 = 13(0.5021) = 6.5273). The critical Chi-square value with 5 degrees of freedom is (CHISQ.INV.RT(0.05,5) = 11.0705). Since (LM (6.5273) < \chi^2_{0.05,5} (11.0705)), we cannot reject (H_0) and conclude that errors are not heteroskedastic. The F-test gives (F = 0.4706) with critical value (F_{0.05,4,7} = 4.1203), also indicating no heteroskedasticity.

💡 Why this matters: The White test is particularly powerful because it can detect heteroskedasticity that depends on the squares and interactions of variables, not just their levels. However, it uses many degrees of freedom, which can be a problem with small samples.

Handling Heteroskedasticity

When heteroskedasticity is detected, three main methods can be used to address it:

Method 1: Change the model specification / transform variables. The relationship between variables may not be linear, so consider transforming the model. Some important variable may be missing, so find and use them. If there are subgroup differences, use dummy variables. If possible, use panel data techniques.

Method 2: Use Huber / White Standard Errors. This type of standard error is also called Heteroskedasticity consistent standard errors. With heteroskedasticity, the standard errors are incorrect, so t-statistics would be incorrect (another formula is required for SE). White's idea: Use simple OLS and correct the standard errors. For a simple regression line, the White estimator of error variance is:

• (White \ Variance(\hat{\beta}_1) = \frac{\sum (X_i - \bar{X})^2 \hat{\sigma}_i^2}{(\sum (X_i - \bar{X})^2)^2})
• Estimate (\hat{\sigma}_i^2 = e_i^2) (using the squared residual for each observation as the estimate of its variance)

Method 3: GLS / WLS (Generalized/Weighted Least Squares). This method has two cases: one where the variance of errors is known, and another where the variance of errors is not known.

🔑 Definition — GLS/WLS (variance of errors known): Consider the model (Y_i = \beta_0 + \beta_1 X_i + e_i). Suppose we know (\sigma_i), i.e., the standard deviation of the error for each observation. Divide all variables by this error standard deviation: (\frac{Y_i}{\sigma_i} = \frac{\beta_0}{\sigma_i} + \frac{\beta_1 X_i}{\sigma_i} + \frac{e_i}{\sigma_i}). The new model's variance is: (Var(\frac{e_i}{\sigma_i}) = \frac{1}{\sigma_i^2} Var(e_i) = \frac{\sigma_i^2}{\sigma_i^2} = 1). This transformed model has constant variance (homoskedastic) errors.

🔑 Definition — GLS/WLS (variance of errors NOT known): Make assumptions about the variance structure. Always transform the model by dividing by the standard deviation of errors based on the assumed relationship between variance and X.

🔑 Definition — What does WLS do? OLS minimizes the sum of squared errors and gives equal weight (importance) to all observations. WLS (Weighted Least Squares) gives observations with larger error variance less weight. WLS minimizes a weighted sum of square of errors, e.g., minimizes (\sum \frac{e_i^2}{Var(e_i)}).

📐 Formula: Transformation when (Var(e_i) = \sigma^2 X_i) (variance increases linearly with X): (\frac{Y_i}{\sqrt{X_i}} = \frac{\beta_0}{\sqrt{X_i}} + \beta_1 \frac{X_i}{\sqrt{X_i}} + \frac{e_i}{\sqrt{X_i}})

📐 Formula: Transformation when (Var(e_i) = \sigma^2 X_i^2) (variance is related to square of X): (\frac{Y_i}{X_i} = \frac{\beta_0}{X_i} + \beta_1 + \frac{e_i}{X_i}) Note: In this transformed model, the intercept becomes (\beta_1) and the slope becomes (\beta_0).

Example: Handling Heteroskedasticity

Consider the model (Y_i = \beta_0 + \beta_1 X_i + e_i) with 20 observations. The Breusch Pagan test gives (R^2 = 0.22837), (LM = N \cdot R^2 = 4.5673), and (F = 5.3271). Both F and LM indicate the presence of heteroskedasticity. The White test gives (R^2 = 0.34954), (LM = 6.9907), and (F = 4.5676). Both F and LM indicate the presence of heteroskedasticity in a non-linear fashion.

Assume the relationship (Var(e_i) = \sigma^2 X_i^2). Transform the model: (\frac{Y_i}{X_i} = \frac{\beta_0}{X_i} + \beta_1 + \frac{e_i}{X_i}). This can be rewritten as: (\frac{Y_i}{X_i} = \beta_1 + \beta_0 \frac{1}{X_i} + \frac{e_i}{X_i}). Note that the intercept is (\beta_1) (original slope) and the slope is (\beta_0) (original intercept). Estimating this transformed model gives: (\frac{Y_i}{X_i} = 11.51 + 25.61 (\frac{1}{X_i})).

Testing the new transformed regression for heteroskedasticity: using the White test on the transformed model, (R^2 = 0.03892), (LM = 0.7783), and (F = 0.3442). The critical values are (\chi^2_{0.05,2} = 5.9915) and (F_{0.05,2,17} = 3.5915). Since both test statistics are far below their critical values, we conclude that heteroskedasticity does not exist in the transformed model.

📌 Example: The original model (Y = \beta_0 + \beta_1 X + e) showed heteroskedasticity in both the Breusch Pagan test (LM = 4.57 > 3.84?) and White test (LM = 6.99 > 5.99?). After assuming (Var(e_i) = \sigma^2 X_i^2) and transforming by dividing by (X_i), the new model (\frac{Y_i}{X_i} = \beta_1 + \beta_0 \frac{1}{X_i} + \frac{e_i}{X_i}) yields White test results of LM = 0.78 (far below 5.99) and F = 0.34 (far below 3.59), indicating successful removal of heteroskedasticity.

⭐ Key Takeaways

This lecture demonstrates the practical application of five different heteroskedasticity detection tests (Park, Harvey-Godfrey, Glesjer, Breusch Pagan, and White) using the same data, showing that different tests can give different conclusions. The general procedure for all tests involves setting up (H_0): Homoskedasticity, calculating either the LM statistic ((N \cdot R^2)) or F-statistic from an auxiliary regression, and comparing to the appropriate critical value. When heteroskedasticity is detected, three remedies are available: respectifying the model, using White (heteroskedasticity-consistent) standard errors, or applying Weighted Least Squares by transforming the model based on assumptions about how the error variance relates to the independent variables. The key practical skill is the transformation example where (Var(e_i) = \sigma^2 X_i^2) leads to dividing all variables by (X_i), which successfully removes heteroskedasticity as confirmed by subsequent testing.

🧠 Quick Revision Questions

1. What is the general formula for the LM test statistic in all heteroskedasticity tests, and how is it used to decide whether to reject the null hypothesis?
2. How does the Park LM test differ from the Harvey-Godfrey test in terms of the auxiliary regression specification?
3. In the Glesjer test, three different auxiliary regressions were run. What were the three functional forms tested, and why is this flexibility useful?
4. When using the White test with two independent variables, what additional terms are included in the auxiliary regression beyond the original variables?
5. In the handling heteroskedasticity example, after transforming the model by dividing by (X_i) (assuming (Var(e_i) = \sigma^2 X_i^2)), what happens to the interpretation of the intercept and slope coefficients in the transformed regression?

📘 Lecture 20 — Autocorrelation

📖 Overview: This lecture introduces the concept of autocorrelation (serial correlation), a violation of the Gauss-Markov assumption that error terms are independent. It explains what autocorrelation is, its common causes in time series and cross-sectional data, and the serious consequences it has for Ordinary Least Squares (OLS) estimation, particularly on standard errors and hypothesis testing.

🗂️ Topics Covered

The lecture defines autocorrelation and the autocorrelation coefficient (rho), introduces first-order serial correlation, and then lists nine possible causes including missing variables, inertia, incorrect functional form, and the cobweb phenomenon. It explains spatial autocorrelation as a special case and concludes with a detailed list of consequences such as biased standard errors, overestimated R-squared, and inefficient but unbiased estimators.

📝 Lecture Summary

What is Autocorrelation / Serial Correlation?

The Gauss-Markov assumptions for BLUE (Best Linear Unbiased Estimators) require that errors are normally distributed with zero mean and constant variance, and that errors are independent. Independence means one error term is not correlated with any other error term. A violation of this independence assumption is called serial correlation.

With serial correlation, error terms are correlated with one another. The error term of different time periods (usually adjacent) or different cross-sectional observations are correlated. This means if we know something about the error term of one observation, we know something about the error term of another observation. Errors associated in one time period carry over to future time periods (e.g., in lagged models). Serial correlation is usually associated with time series data, so we use the subscript t instead of i.

Error terms may be correlated more with nearby observations compared to distant observations: ρ(eₜ, eₜ₋ᵢ) > ρ(eₜ, eₜ₋ⱼ) where i < j. Here, ρ (rho) is the autocorrelation coefficient. In cases where serial correlation exists (in time or space), ρ(eₜ, eₜ₋ᵢ) ≠ 0 where t ≠ 0 in time series data, and ρ(eᵢ, eⱼ) ≠ 0 where i ≠ j in cross-sectional data.

First Order Serial Correlation

The autocorrelation coefficient ρ ranges from -1 ≤ ρ ≤ 1. One specific type is first order (linear) serial correlation, where the error in one time period is correlated with the error in the previous time period. For first order serial correlation: ρ(eₜ, eₜ₋₁) ≠ 0, where -1 ≤ ρ ≤ 1.

In dynamic models: eₜ = ρeₜ₋₁ + uₜ, where uₜ is called white noise and is independently and identically distributed (i.i.d.) with zero mean and constant variance.

💡 Why this matters: The sign (positive or negative) and magnitude of rho tell us how errors are related over time.

Possible Causes of Autocorrelation

1. Missing Variables: The error term may include all variables not in the regression equation. A change in any unobserved variable in one time period may impact errors in different time periods, causing errors to follow patterns of unobserved variables. Example: Y depends on Xₜ₁ and Xₜ₂, and Xₜ₂ is not included; or sales depend on unmodeled seasonal changes.

2. Inertia or Sluggishness: The speed of change in variables depends on time. Business cycles (GDP, prices) are an example. The speed of price change may depend on how far it is from equilibrium. This is common in time series data.

3. Incorrect Functional Form: Autocorrelation may occur if linear models are specified when non-linear models are required. Linear-in-variables or simple models used when a log form is needed is called a model specification error.

4. Cobweb Phenomenon / Reaction with Lag: When the dependent variable acts with a lag, autocorrelation may be observed. Examples: Qˢₜ = a + bPₜ₋₁ + eₜ; Cₜ = ayₜ₋₁ + byₜ₋₂ + eₜ. Overproduction in one year may lead to underproduction the next year.

5. Lagged Relationship: The dependent variable may depend on its own previous value. Examples: Stock prices; Consumption Cₜ = a + bCₜ₋₁ + eₜ.

6. Ratchet Effect: The tendency of people to be influenced by the previous (high, low, or best) level of a variable can cause this effect. Consumption changes quickly upward (when income rises) but does not come down easily if income declines, causing over- or underestimation.

7. Data Manipulation: Data manipulated by: averaging or smoothing, converting quarterly data to annual, converting monthly to quarterly, finding mid-points when faced with ranges. Sometimes this is needed due to measurement error (e.g., in monthly data), but a byproduct is autocorrelation.

8. Systematic Measurement Error: Measurement error in one time period may be carried forward. Errors in inventories, stock of capital, or asset value can all cause autocorrelation. Time series is a noisy proxy.

9. Variable Behavior: Electricity consumption at different hours of the day; similar temperature patterns may cause patterns in errors.

Spatial Autocorrelation

Spatial autocorrelation is a special cause in cross-sectional data. It follows the "First Law of Geography" by Waldo Tobler: "Everything is related to everything else but near things are more related than distant things."

Spatial autocorrelation is a correlation of a variable with itself through space, due to the systematic pattern of spatial distribution of a variable. Nearby areas are often more alike (positive spatial autocorrelation). Values in the sample do not remain independent; the occurrence of one event in an area makes it more likely in other areas. Areas with higher concentration of events have more impact on results (spatially clustered observations). Example: Due to high crime in Karachi, more police and rangers are deployed. The Karachi operation decreases crime in Karachi. Crime rate in nearby cities increases, even though they did not decrease police, etc.

Consequences of Autocorrelation

1. Estimators remain unbiased and consistent.
2. Estimators are no longer efficient.
3. Standard Error of Estimate / variance of error is likely to be underestimated, resulting in an overestimated R-squared.
4. Variances of estimators are biased.
5. Forecasts are unbiased but inefficient (with larger variances).
6. In case of positive autocorrelation, standard errors (of coefficients) are too small, resulting in overestimation of t-statistic (making variables appear more significant than they are).
7. In case of negative autocorrelation, standard errors (of coefficients) are too large, resulting in underestimation of t-statistic (making significant variables appear insignificant).

💡 Why this matters: The most dangerous consequence is on hypothesis testing — positive autocorrelation makes t-statistics artificially high, leading to false conclusions that variables are significant when they are not.

⭐ Key Takeaways

Autocorrelation violates the Gauss-Markov assumption of independent errors, meaning error terms are correlated over time (or space). The first-order autoregressive model eₜ = ρeₜ₋₁ + uₜ is the most common form, where rho measures the strength and direction of the correlation. Nine distinct causes include inertia, missing variables, incorrect functional form, and data manipulation. While OLS estimators remain unbiased and consistent under autocorrelation, they are no longer efficient, and most critically, standard errors are biased — underestimated in positive AC (inflating t-stats) and overestimated in negative AC (deflating t-stats). This makes standard hypothesis testing unreliable.

🧠 Quick Revision Questions

1. What is the key Gauss-Markov assumption violated by autocorrelation?
2. Write the equation for a first-order autoregressive error process and define each term.
3. If the Durbin-Watson statistic suggests positive autocorrelation, what is the likely effect on the estimated t-statistics?
4. Explain the difference between "inertia" and "cobweb phenomenon" as causes of autocorrelation.
5. What is spatial autocorrelation, and how does it differ from time-series autocorrelation?

📘 Lecture 21 — Detection of Autocorrelation

📖 Overview: This lecture covers methods for detecting autocorrelation (serial correlation) in regression residuals. It explains both graphical and formal statistical tests, including the AR(1) test, Durbin-Watson d test, Durbin h test, and LM test, which are essential for validating regression assumptions in time series data.

🗂️ Topics Covered

The lecture covers detection of autocorrelation through graphic methods and formal tests. Formal tests include AR(1) test with strictly exogenous regressors, AR(1) test without strictly exogenous regressors, Durbin Watson d Test with its assumptions and decision rules, Durbin h Test for models with lagged dependent variables, and the Simple LM Test. Each method is demonstrated with examples using Microsoft Excel.

📝 Lecture Summary

Graphic method

Looking at the graph, we may see patterns or trends in residuals with respect to time or with respect to previous values of the errors. When errors are plotted against time, a visible pattern indicates errors are not random. Graphs showing residuals plotted against past values of errors can also reveal patterns and trends, which is an indication of serial correlation.

Two types of patterns are shown: negative autocorrelation (residuals oscillate) and positive autocorrelation (residuals trend in same direction).

Detecting Autocorrelation by formal tests

AR(1) test with strictly exogenous regressors

This tests first order autocorrelation with exogenous regressors (regressors uncorrelated with errors).

Procedure:

1. Run the regression (Y) on (X_1, X_2, ... X_K) and obtain residual (e_t)
2. Run the regression (e_t) on (e_{t-1}) for (N = 2) to (n): (e_t = \rho e_{t-1} + u_t)
3. Apply t-test (individual variable significance test) to test the hypothesis (H_0: \rho = 0)

🔑 Definition — AR(1) test with strictly exogenous regressors: A formal test for first-order autocorrelation where the explanatory variables are assumed to be uncorrelated with the error term.

📐 Formula: (e_t = \rho e_{t-1} + u_t) → Residual at time t depends on previous residual times rho plus random error.

📌 Example: Step 1: Run regression Y on X and obtain residuals. Step 2: Run regression (e_t = a + \rho e_{t-1} + u_t) Results using =linest(E52:E65,F52:F65,true,true):

• (\rho = 0.6494361), standard error = 0.2328061
• t-value = 0.6494/0.2328 = 2.789600395
• Critical t = T.INV.2T(0.05,13) = 2.160368656 Since calculated t (2.7896) > tabulated t (2.1604), reject (H_0) and conclude coefficient (\rho) is significant → there is a problem of autocorrelation.

💡 Why this matters: This simple test quickly checks if residuals depend on their own past values, indicating autocorrelation.

AR(1) test without strictly exogenous regressors

This tests first order autocorrelation without exogenous regressors (where explanatory variables may be correlated to residuals).

Procedure:

1. Run the regression (Y) on (X_1, X_2, ... X_K) and obtain residual (e_t)
2. Run the regression (e_t) on (X_1, X_2, ... X_K) and (e_{t-1}) for (N = 2) to (n): (e_t = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \cdots + \beta_K X_K + \rho e_{t-1} + u_t)
3. Apply t-test to test (H_0: \rho = 0)

📌 Example: Step 1: Run regression Y on X and obtain residuals. Step 2: Run regression (e_t = a + bX + \rho e_{t-1} + u_t) (rearrange columns to put (e_{t-1}) and X together). Results:

• (\rho = 0.645754025), standard error = 0.247399143
• t-value = 0.64575/0.247399 = 2.610170824
• Critical t = T.INV.2T(0.05,11) = 2.20098516 Since calculated t (2.6102) > tabulated t (2.2010), reject (H_0) → there is autocorrelation.

Durbin Watson d Test

This tests first order autocorrelation with exogenous regressors. It is not good when regressors are not exogenous or when the model contains a lagged dependent variable.

Assumptions:

• Regression model has an intercept
• Errors are generated by first order autoregressive scheme: (e_t = \rho e_{t-1} + u_t)
• Regression model does not have lagged dependent variable as regressor
• There are no missing observations

🔑 Definition — Durbin Watson d statistic: A test statistic for detecting first-order autocorrelation in regression residuals.

📐 Formula: [ d = \frac{\sum_{t=2}^n (e_t - e_{t-1})^2}{\sum_{t=1}^n e_t^2} ] For large samples: (d = 2(1 - \hat{\rho})), where (\hat{\rho} = \frac{\sum e_t e_{t-1}}{\sum e_t^2})

Since (-1 \leq \rho \leq 1), d ranges from 0 to 4:

• If (\rho = 0), (d = 2) (no autocorrelation)
• If (\rho = -1) (negative), (d = 4)
• If (\rho = 1) (positive), (d = 0)

Decision Rules:

• Accept (H_0) if (d_U < d < 4 - d_U) (no autocorrelation)
• Reject (H_0) if (d < d_L) (positive autocorrelation) or if (d > 4 - d_L) (negative autocorrelation)
• Inconclusive if (d_L < d < d_U) or (4-d_U < d < 4-d_L)

General Rules:

• d lies between 0 and 4. As d is closer to 2, chances of autocorrelation decrease
• If (d < 2), may indicate positive autocorrelation; if (d > 2), may indicate negative autocorrelation

📌 Example: Data from AC.xlsx, regression Y on X, obtain residuals.

• (\sum e_t^2 = 451.6195) (SUMSQ(E51:E65))
• (\sum (e_t - e_{t-1})^2 = 307.333) (SUMXMY2(E52:E65,F52:F65))
• (d = 307.333 / 451.6195 = 0.680513)
• (d_L = 1.08) (from table, n=15, K=2)
• (d_U = 1.36) (from table, n=15, K=2) Since (d < d_L), reject (H_0) and conclude positive autocorrelation exists.

Durbin h Test

Procedure:

1. Run the regression: (Y_t = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \cdots + \beta_K X_K + \gamma Y_{t-1} + u_t)
2. Compute Durbin h:

📐 Formula: [ h = \hat{\rho} \sqrt{\frac{N}{1 - N \cdot Var(\gamma)}} ] where (\hat{\rho} = 1 - \frac{d}{2}) (provided sample is large)

Note: h is normally distributed with unit variance so conclusions may be formed by looking at the normal distribution table.

Problem: Cannot be computed if (N \cdot Var(\gamma) > 1).

Simple LM Test: Testing for AR(1) model

Procedure:

1. Run the regression: (Y_t = \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \cdots + \beta_K X_K + e_t)
2. Obtain the residuals (e_t)
3. Regress (e_t) on all explanatory variables and (e_{t-1})
4. Compute (LM = (n - 1) R^2) (because we lose one observation so n-1)
5. Test using Chi-square (0.05, 1) distribution

🔑 Definition — LM Test (Lagrange Multiplier Test): A test for autocorrelation similar to AR(1) test without exogenous regressors, but uses LM instead of F-test.

⭐ Key Takeaways

Students must remember that autocorrelation can be detected graphically by observing patterns in residuals plotted against time or lagged residuals. The Durbin-Watson d statistic ranges from 0 to 4, with d=2 indicating no autocorrelation, d close to 0 indicating positive autocorrelation, and d close to 4 indicating negative autocorrelation, but the test has inconclusive regions defined by (d_L) and (d_U) from tables. The AR(1) tests differ based on whether regressors are strictly exogenous; when regressors may be correlated with errors, additional explanatory variables must be included in the auxiliary regression. The Durbin h test is specifically designed for models containing lagged dependent variables. The LM test provides an alternative approach using Chi-square distribution. In all examples presented, the calculated test statistics exceeded critical values, leading to rejection of the null hypothesis of no autocorrelation.

🧠 Quick Revision Questions

1. What is the range of the Durbin-Watson d statistic, and what do values of 0, 2, and 4 indicate about autocorrelation?
2. How does the AR(1) test with strictly exogenous regressors differ from the AR(1) test without strictly exogenous regressors in terms of the auxiliary regression?
3. Under what condition can the Durbin h test not be computed, and what distribution is used to evaluate the test statistic?
4. In the Durbin-Watson test example with n=15 and K=2, why was the null hypothesis rejected when d=0.68 and dL=1.08?
5. What is the formula for the LM test statistic and what distribution is used to test significance?

📘 Lecture 22 — Treating Autocorrelation

📖 Overview: This lecture addresses the consequences of ignoring autocorrelation in regression models and provides remedial measures to correct it. The focus is on Generalized Least Squares (GLS) methods when the autocorrelation parameter ρ is known or unknown, including the Prais-Winsten transformation, Cochrane-Orcutt method, and Hildreth-Lu search procedure.

🗂️ Topics Covered

The lecture begins with the consequences of ignoring autocorrelation, including biased standard errors and inflated R². It then covers remedial measures starting with Generalized Differencing when ρ is known, including special cases where ρ = +1 and ρ = −1. The Prais-Winsten transformation is introduced for small samples. Methods for when ρ is unknown follow: using the Durbin-Watson d statistic, the Cochrane-Orcutt iterative method with a detailed example, and the Hildreth-Lu search procedure. Each method includes worked examples and comparisons.

📝 Lecture Summary

Consequences of Ignoring Autocorrelation

If autocorrelation is present but ignored, several problems arise. Coefficients remain unbiased and consistent but are not efficient (not BLUE), even in large samples — similar to heteroskedasticity. Standard error estimates become inappropriate, leading to wrong inferences. The t-statistic is overestimated, making regression coefficients appear significant when they are not. R² is inflated and residual variance is underestimated for positively correlated residuals (if X grows over time). Forecasts remain unbiased but have large variances.

Remedial Measures for Autocorrelation

Two situations exist: when ρ is known — use GLS (Generalized Least Squares); when ρ is not known — estimate ρ first, then apply GLS.

Generalized Differencing / Generalized Least Square

Consider a two-variable model:

Yₜ = β₀ + β₁Xₜ + eₜ

Assume the error term follows an AR(1) scheme:

eₜ = ρ eₜ₋₁ + vₜ

where −1 < ρ < 1, and vₜ satisfies OLS assumptions. ρ can be known or estimated.

Procedure to transform the model:

1. Write the regression with a one-period lag: Yₜ₋₁ = β₀ + β₁Xₜ₋₁ + eₜ₋₁
2. Multiply by ρ: ρYₜ₋₁ = ρβ₀ + ρβ₁Xₜ₋₁ + ρeₜ₋₁
3. Subtract from the original equation: Yₜ − ρYₜ₋₁ = β₀(1 − ρ) + β₁(Xₜ − ρXₜ₋₁) + (eₜ − ρeₜ₋₁)
4. This gives the transformed model:

Yₜ* = β₀* + β₁Xₜ* + vₜ

Where: 🔑 Yₜ* = Yₜ − ρYₜ₋₁ 🔑 β₀* = β₀(1 − ρ) 🔑 Xₜ* = Xₜ − ρXₜ₋₁

📌 Example with ρ = 0.6: A dataset with 15 time periods (Y, X) is transformed using ρ=0.6 to create Y* and X*. OLS on the transformed model yields: Yₜ* = 17.995 + 2.1554 Xₜ*. The slope 2.1554 is β₁ directly. Since β₀* = β₀(1 − ρ), we compute β₀ = 17.995 / (1 − 0.6) = 44.9877.

Generalized Differencing: Special Cases

Special Case: ρ = +1 Substituting ρ = 1 into the differencing procedure gives:

∆Yₜ = β₁∆Xₜ + ∆eₜ

Where ∆Yₜ = Yₜ − Yₜ₋₁ and ∆Xₜ = Xₜ − Xₜ₋₁. This equation is estimated without an intercept.

Special Case: ρ = −1 Substituting ρ = −1 gives:

Yₜ* = 2β₀ + β₁Xₜ* + vₜ

Where Yₜ* = Yₜ + Yₜ₋₁ and Xₜ* = Xₜ − Xₜ₋₁.

💡 Why this matters: These special cases show that first-differencing (ρ=1) eliminates the intercept entirely, which can dramatically change model interpretation.

Prais-Winsten Transformation

Differencing causes loss of one observation (the first). In small samples, we can estimate the first observation as: 🔑 Y₁* = √(1 − ρ²) × Y₁ 🔑 X₁* = √(1 − ρ²) × X₁

This is not necessary in large samples. The example dataset with 15 observations uses this transformation where relevant.

Removing Autocorrelation when ρ is Unknown

Method 1: Using Durbin-Watson d statistic The relationship between ρ and d is: 📐 d ≈ 2(1 − ρ) → ρ ≈ 1 − d/2

Since 0 ≤ d ≤ 4, we get −1 ≤ ρ ≤ 1. For small samples, Theil-Nagar suggest: 📐 ρ̂ = [N²(1 − d/2) + K²] / (N² − K²)

Where N is sample size and K is number of coefficients.

📌 Example: N = 14, K = 2, d = 0.6805. Using Theil-Nagar: ρ̂ = [14²(1 − 0.6805/2) + 2²] / [14² − 2²] = 0.667. This ρ is then used in the generalized difference model.

Method 2: Cochrane-Orcutt Method This iterative method uses residuals to estimate ρ. Steps:

1. Round 1: Apply OLS, find residuals eₜ, compute ρ̂ = Σ(eₜeₜ₋₁) / Σ(eₜ²)
2. Round 2: Perform GLS using ρ̂, obtain new residuals, compute new ρ̂
3. Round 3+: Continue until ρ̂ from two successive rounds is approximately equal

📌 Example: Data with 15 periods:

• Round 1: OLS gives intercept 39.27556, slope 2.715066. ρ̂ = 240.49056 / 406.83312 = 0.591128
• Round 2: Transform Y and X, OLS gives intercept 18.33661, slope 2.166349. ρ̂ = 108.2871 / 208.098 = 0.520366
• Round 3: Intercept 19.55304, slope 2.794671. Iteration continues until convergence.

Method 3: Hildreth-Lu Search Procedure Step 1: Choose a grid of possible ρ values between −1 and +1 (e.g., 0, 0.1, 0.2, …, 0.9, 1). Step 2: For each ρ value, estimate the generalized difference model and find the sum of squared residuals (SS). Step 3: The equation with minimum SS is the best.

If first grid gives 0.7, refine with a finer grid around 0.7 (e.g., 0.66, 0.67, …, 0.74) and repeat.

Problems: Time-consuming; grid must be chosen carefully to find global minimum SS, not just local minimum.

Comparison: Hildreth-Lu is more computer-time-intensive than Cochrane-Orcutt. Cochrane-Orcutt may converge to a local minimum and miss the global minimum SS.

⭐ Key Takeaways

Ignoring autocorrelation leads to unbiased but inefficient coefficients, overestimated t-statistics, inflated R², and wrong inferences. The primary remedial measure is Generalized Differencing (GLS), which transforms variables using ρ to produce a model with uncorrelated errors. When ρ is unknown, it can be estimated using the Durbin-Watson d statistic (via Theil-Nagar for small samples), the Cochrane-Orcutt iterative method (using residuals to refine ρ), or the Hildreth-Lu search procedure (grid search for minimum SS). For ρ=+1, the model reduces to first differences without an intercept; for ρ=−1, the intercept doubles. The Prais-Winsten transformation preserves the first observation in small samples.

🧠 Quick Revision Questions

1. What are four consequences of ignoring autocorrelation in a regression model?
2. How does Generalized Differencing transform the model Yₜ = β₀ + β₁Xₜ + eₜ under AR(1) errors?
3. What is the transformed equation when ρ = +1, and why is there no intercept?
4. How does the Cochrane-Orcutt method estimate ρ in each round, and when does the iteration stop?
5. In the Hildreth-Lu search procedure, what criterion determines the best ρ value, and what is a key limitation of this method?