📘 Lecture 8 — Secant Method
📖 Overview: This lecture covers the Secant Method, a modification of the Newton-Raphson method for finding roots of equations. It explains the derivation, geometrical interpretation, convergence properties, and provides worked examples. The lecture also introduces Muller’s Method, which uses quadratic interpolation to approximate roots, including complex ones.
🗂️ Topics Covered
The lecture begins with the Secant Method, its derivation by replacing the derivative in Newton-Raphson with a difference ratio, and its geometrical interpretation as drawing secants. It then discusses the method's convergence rate (super-linear, order ~1.618) and provides a detailed step-by-step example for finding the root of a cubic equation. A second example for a transcendental equation is also worked out. Finally, the lecture introduces Muller’s Method, which approximates the function f(x)=0 by a second-degree polynomial (quadratic) passing through three points, allowing it to find both real and complex roots, and derives the formula for the next approximation.
📝 Lecture Summary
Secant Method
The Secant Method is a modified form of the Newton-Raphson method. In the Newton-Raphson method, the derivative f'(xₙ) is replaced by a difference ratio:
f'(xₙ) = [f(xₙ) – f(xₙ₋₁)] / (xₙ – xₙ₋₁)
Where xₙ and xₙ₋₁ are two approximations of the root. Substituting this into the Newton-Raphson formula gives the secant formula:
xₙ₊₁ = xₙ – [f(xₙ)(xₙ – xₙ₋₁)] / [f(xₙ) – f(xₙ₋₁)] = [xₙ₋₁ f(xₙ) – xₙ f(xₙ₋₁)] / [f(xₙ) – f(xₙ₋₁)]
provided f(xₙ) ≠ f(xₙ₋₁). This method requires two starting values, x₀ and x₁, and the function values at both these points are calculated. The next point on the curve is obtained by using the derived formula. This procedure is continued until the root is found with the required accuracy.
📌 Example: Do three iterations of secant method to find the root of f(x) = x³ – 3x + 1 = 0, taking x₀ = 1, x₁ = 0.5.
- n = 1: f(x₀) = f(1) = 1³ – 3(1) + 1 = -1; f(x₁) = f(0.5) = 0.5³ – 3(0.5) + 1 = -0.375. x₂ = [x₀ f(x₁) – x₁ f(x₀)] / [f(x₁) – f(x₀)] = [(1)(-0.375) – (0.5)(-1)] / [-0.375 – (-1)] = 0.125 / 0.625 = 0.2
- n = 2: f(x₂) = f(0.2) = 0.2³ – 3(0.2) + 1 = 0.408. x₃ = [x₁ f(x₂) – x₂ f(x₁)] / [f(x₂) – f(x₁)] = [(0.5)(0.408) – 0.2(-0.375)] / [0.408 – (-0.375)] = 0.279 / 0.783 = 0.3563
- n = 3: f(x₃) = f(0.3563) = 0.3563³ – 3(0.3563) + 1 = -0.02367. x₄ = [x₂ f(x₃) – x₃ f(x₂)] / [f(x₃) – f(x₂)] = [(0.2)(-0.02367) – 0.3563(0.408)] / [-0.02367 – 0.408] = -0.1498 / -0.43167 = 0.3471 x₅ = 0.3473, f(x₅) = -0.0000096, and x₅ - x₄ = 0.0004. Though X5 is not the true root, it is a good approximation and convergence is faster than bisection.
💡 Why this matters: The Secant method converges faster than linear (Bisection) and slower than Newton’s quadratic rate, but it avoids the need to compute the derivative analytically.
Geometrical Interpretation
Geometrically, the secant method corresponds to drawing secants rather than tangents to obtain various approximations to root α. To obtain x₂, we find the intersection between the secant through the points (x₀, f(x₀)) and (x₁, f(x₁)) and the x-axis.
🔑 Definition — Secant Method Geometry: The next approximation x₂ is obtained as the point of intersection of y = 0 and the chord passing through the points (x₀, f(x₀)) and (x₁, f(x₁)).
The equation of the chord is: y – f(x₀) = [f(x₁) – f(x₀)] / (x₁ – x₀) * (x – x₀)
Putting y = 0 gives: x = x₂ = [x₀ f(x₁) – x₁ f(x₀)] / [f(x₁) – f(x₀)]
The method converges rather quickly, though there is a possibility of divergence if the two roots lie on the same side of the curve. The order of convergence of the secant method is (1 + √5)/2 ≈ 1.618. This shows that this method has an order of convergence slightly inferior to that of Newton-Raphson method. In this method, f(x) is not required to change signs between the estimates.
💡 Why this matters: The secant method does not require the function to change sign between successive approximations, which is a requirement in the Regula-Falsi method.
Convergence of Secant Method
The sequence is generated by the rule: xₙ₊₁ = [xₙ₋₁ f(xₙ) – xₙ f(xₙ₋₁)] / [f(xₙ) – f(xₙ₋₁)]
Starting with x₀ and x₁, the sequence {x₀, x₁, ...} will converge to ‘a’, where f(a) = 0.
The Secant method converges faster than linear and slower than Newton’s quadratic convergence.
Muller’s Method
In Muller’s method, f(x) = 0 is approximated by a second-degree polynomial; that is, by a quadratic equation that fits through three points in the vicinity of a root. The roots of this quadratic equation are then approximated to the roots of the equation f(x) = 0. This method is iterative in nature and does not require the evaluation of derivatives as in Newton-Raphson method. This method can also be used to determine both real and complex roots of f(x) = 0.
Suppose xᵢ₋₂, xᵢ₋₁, xᵢ be any three distinct approximations to a root of f(x) = 0, with function values f(xᵢ₋₂) = fᵢ₋₂, f(xᵢ₋₁) = fᵢ₋₁, f(xᵢ) = fᵢ. A general polynomial of second degree is given by f(x) = ax² + bx + c. The coefficients a, b, c are determined by the condition that the polynomial passes through the three points.
We define: hᵢ = xᵢ – xᵢ₋₁ hᵢ₋₁ = xᵢ₋₁ – xᵢ₋₂ λᵢ = hᵢ / hᵢ₋₁ δᵢ = 1 + λᵢ
Using these substitutions, a simplified equation is obtained. To compute λ, set f = 0, which leads to a quadratic equation in λ. To avoid loss of accuracy, a specific formula is used: λ = -2 fᵢ δᵢ / { gᵢ ± [gᵢ² – 4 fᵢ δᵢ λᵢ (fᵢ₋₂ λᵢ – fᵢ₋₁ δᵢ + fᵢ) ]¹/² }
Here, the positive sign must be chosen so that the denominator becomes largest in magnitude. A better approximation to the root is then obtained by using: xᵢ₊₁ = xᵢ + hᵢ λ
⭐ Key Takeaways
The Secant Method is a root-finding technique derived by replacing the derivative in Newton-Raphson with a finite difference ratio, requiring two initial guesses but no derivative evaluation. Its order of convergence is approximately 1.618, which is faster than linear methods like Bisection but slower than Newton's quadratic rate; it does not require the function to change sign between successive approximations. A full step-by-step example for the cubic equation x³ – 3x + 1 = 0 demonstrates the iterative formula xₙ₊₁ = [xₙ₋₁ f(xₙ) – xₙ f(xₙ₋₁)] / [f(xₙ) – f(xₙ₋₁)]. Muller’s Method is an alternative that approximates the function by a quadratic through three points, allowing it to find complex roots without derivative evaluation. The key difference from Regula-Falsi is that the secant method does not check whether the root lies between successive iterates, which can lead to divergence if points are on the same side of the curve.
🧠 Quick Revision Questions
- Derive the formula for the Secant Method starting from the Newton-Raphson formula.
- What is the order of convergence of the Secant Method, and how does it compare to the Bisection and Newton-Raphson methods?
- Perform two iterations of the Secant Method to approximate a root of f(x) = x³ – 2x – 5 = 0, using initial guesses x₀ = 2 and x₁ = 3.
- What is the key difference between the Secant Method and the Regula-Falsi method in terms of the condition on the root's location between successive approximations?
- Explain the general principle of Muller’s Method and state one key advantage it has over the Secant Method.
📘 Lecture 9 — Muller’s Method
📖 Overview: This lecture covers two advanced methods for finding roots of equations. Muller's method generalizes the secant method using three starting points and quadratic interpolation to converge faster than the secant method. Graeffe's Root Squaring Method is introduced as a powerful technique for finding all roots of a polynomial simultaneously by repeatedly squaring the polynomial and taking appropriate roots.
🗂️ Topics Covered
Muller's method is presented as an iterative method requiring three starting points, constructing a parabola through them, and using the quadratic formula for the next approximation. A complete two-iteration example is worked through for solving x³ − 3x + 1 = 0. Graeffe's Root Squaring Method is introduced for finding all roots of polynomial equations by repeatedly squaring the polynomial, transforming roots to their powers, and then extracting roots. Multiple examples demonstrate the method including cubic and quartic equations. The lecture concludes with a derivation of the Newton-Raphson extended formula (Chebyshev's formula of third order).
📝 Lecture Summary
Muller’s Method
Muller's method is a generalization of the secant method. It does not require the derivative of the function. It is an iterative method that requires three starting points x₀, x₁, and x₂. A parabola is constructed that passes through the three points; then the quadratic formula is used to find a root of the quadratic for the next approximation. Near a simple root, Muller's method converges faster than the secant method and almost as fast as Newton's method. The method can find real or complex zeros.
💡 Why this matters: Unlike Newton's method, Muller's method does not require derivatives, making it useful when derivatives are difficult or impossible to compute.
Example: Do two iterations of Muller's method to solve x³ − 3x + 1 = 0 starting with x₀ = 0.5, x₁ = 0, x₂ = 1
Solution:
Iteration 1:
- f(x₀) = f(0.5) = (0.5)³ − 3(0.5) + 1 = −0.375
- f(x₁) = f(1) = (1)³ − 3(1) + 1 = −1
- f(x₂) = f(0) = 0 − 3(0) + 1 = +1
- c = f₀ = −0.375
- h₁ = x₁ − x₀ = 0.5 − 0.5 = 0
- h₂ = x₀ − x₂ = 0.5 − 0 = 0.5
Wait — correction from lecture:
- h₁ = x₁ − x₀ = 0.5 (Note: x₁ = 1, x₀ = 0.5, so h₁ = 0.5)
- h₂ = x₀ − x₂ = 0.5 − 0 = 0.5
a = [h₂f₁ − (h₁ + h₂)f₀ + h₁f₂] / [h₁h₂(h₁ + h₂)] = [(0.5)(−1) − (−0.375)(0.5 + 0.5) + (0.5)(1)] / [(0.5)(0.5)(1)] = [−0.5 − (−0.375)(1) + 0.5] / [0.25] = [−0.5 + 0.375 + 0.5] / 0.25 = 0.375 / 0.25 = 1.5
b = (f₁ − f₀ − ah₁²) / h₁ = (−1 − (−0.375) − (1.5)(0.5)²) / 0.5 = (−1 + 0.375 − 0.375) / 0.5 = (−1) / 0.5 = −2
x₃ = x₀ − [2c / (b − √(b² − 4ac))] = 0.5 − [2(−0.375) / (−2 − √(4 − 4(1.5)(−0.375)))] = 0.5 − [−0.75 / (−2 − √(4 + 2.25))] = 0.5 − [−0.75 / (−2 − √6.25)] = 0.5 − [−0.75 / (−2 − 2.5)] = 0.5 − [−0.75 / (−4.5)] = 0.5 − 0.16667 = 0.33333
Iteration 2: Take x₂ = 0, x₀ = 0.33333, x₁ = 0.5
- h₁ = x₁ − x₀ = 0.5 − 0.33333 = 0.16667
- h₂ = x₀ − x₂ = 0.33333 − 0 = 0.33333
- c = f₀ = f(0.33333) = (0.33333)³ − 3(0.33333) + 1 = 0.037046
- f₁ = (0.5)³ − 3(0.5) + 1 = −0.375
- f₂ = 0³ − 3(0) + 1 = 1
a = [(0.33333)(−0.375) − (0.16667 + 0.33333)(0.037046) + (0.16667)(1)] / [(0.16667)(0.33333)(0.5)] = [−0.125 + 0.018523 + 0.16667] / 0.027778 = 0.023148 / 0.027778 = 0.8333
b = (−0.375 − 0.037046 − (0.8333)(0.16667)²) / 0.16667 = −2.6
x₃ = 0.33333 − [2(0.037046) / (−2.6 − √(6.76 − 4(0.8333)(0.037046)))] = 0.33333 − [0.074092 / (−2.6 − √(6.76 − 0.1235))] = 0.33333 − [0.074092 / (−2.6 − √6.6365)] = 0.33333 − [0.074092 / (−2.6 − 2.5761)] = 0.33333 − [0.074092 / (−5.1761)] = 0.33333 + 0.01431 = 0.34764
For third iteration, take x₂ = 0.333333, x₀ = 0.3475, x₁ = 0.5
Graeffe’s Root Squaring Method
Graeffe's Root Squaring Method is particularly attractive for finding all the roots of a polynomial equation. Consider a polynomial of third degree: f(x) = a₀ + a₁x + a₂x² + a₃x³
f(x)f(−x) = a₃²x⁶ − (a₂² − 2a₁a₃)x⁴ + (a₁² − 2a₀a₂)x² − a₀²
With substitution t = x²: f(x)f(−x) = a₃²t³ − (a₂² − 2a₁a₃)t² + (a₁² − 2a₀a₂)t − a₀²
The roots of this equation are squares or 2ⁱ (i = 1) powers of the original roots. After each squaring, the coefficients become large and overflow is possible as i increases. After squaring 'i' times, we estimate the roots by evaluating the 2ⁱ-th root of aᵢ₋₁/aᵢ.
🔑 Definition — Graeffe's Root Squaring Method: A method for finding all roots of a polynomial by repeatedly squaring the polynomial to separate roots by magnitude, then extracting appropriate roots to recover the original roots.
The method fails when the roots of the given polynomial are repeated. It does not require prior information about approximate values of the roots but is applicable to polynomials only.
Procedure: For polynomial f(x) = xⁿ + a₁xⁿ⁻¹ + a₂xⁿ⁻² + ... + aₙ₋₁x + aₙ, separate even and odd powers, square both sides, put x² = y to get new equation yⁿ + b₁yⁿ⁻¹ + ... + bₙ = 0 where b₁ = −a₁² + 2a₂, b₂ = a₂² − 2a₁a₃ + 2a₄, ..., bₙ = (−1)ⁿaₙ².
If p₁, p₂, ..., pₙ are roots of original equation, then roots of the squared equation are p₁², p₂², ..., pₙ².
Example: Using Graeffe root squaring method, find all the roots of x³ − 6x² + 11x − 6 = 0
Solution: For i = 1: x³ − (36 − 22)x² + (121 − 72)x − 36 = x³ − 14x² + 49x − 36 For i = 2: x³ − (196 − 98)x² + (2401 − 1008)x − 1296 = x³ − 98x² + 1393x − 1296 For i = 3: x³ − (9604 − 2786)x² + (1940449 − 254016)x − 1679616 = x³ − 6818x² + 16864333x − 1679616
Root estimates from polynomial (i=1): ⁴√(36/49) = 0.85714, ⁴√(49/14) = 1.8708, ⁴√(14/1) = 3.7417
From polynomial (i=2): ⁸√(1296/1393) = 0.9821, ⁸√(1393/98) = 1.9417, ⁸√(98/1) = 3.1464
From polynomial (i=3): ¹⁶√(1679616/16864333) = 0.99949, ¹⁶√(16864333/6818) = 1.99143, ¹⁶√(6818/1) = 3.0144
Exact roots: 1, 2, 3
Example: Apply Graeffe's root squaring method to solve x³ − 8x² + 17x − 10 = 0
Three sign changes are observed, so f(x) may have three positive roots.
After repeated squaring and substitution, the final equation is u³ − 378882u² + 264082u − 10⁸ = 0
p₁ = (378882)^(1/8) = 4.9809593 ≈ 5 p₂ = (264082/378882)^(1/8) = 0.9558821 ≈ 1 p₃ = (10⁸/264082)^(1/8) = 2.1003064 ≈ 2
Check: f(5) = f(2) = f(1) = 0, confirming all are roots.
Example: Find all roots of x⁴ − 3x + 1 = 0
Two sign changes → two positive real roots. No sign change in f(−x) → two complex roots.
After repeated squaring and substitution, final equation: u⁴ − 4u³ + 645u² − 5917u + 1 = 0
p₁ = 4^(1/8) = 1.189207 (real) p₄ = (1/5971)^(1/8) = 0.3376659 (real)
For complex roots ρe^(±iφ) = ξ ± iη: ρ = (5917/654)^(1/4) = 1.5780749 ξ₂ = −½(p₁ + p₄) = −0.7634365 η₂ = √(ρ² − ξ²) = √(1.9074851) = 1.3811173
The four roots: 1.1892071, 0.3376659, −0.7634365 ± 1.3811173i
Revision Example: Newton-Raphson Extended Formula
Obtain the Newton-Raphson extended formula for finding the root of f(x) = 0:
x₁ = x₀ − f(x₀)/f'(x₀) − ½[f(x₀)]²f''(x₀)/[f'(x₀)]³
Solution: Expanding f(x) by Taylor's series and retaining up to second-order term: 0 = f(x) = f(x₀) + (x − x₀)f'(x₀) + (x − x₀)²f''(x₀)/2
f(x₁) = f(x₀) + (x₁ − x₀)f'(x₀) + (x₁ − x₀)²f''(x₀)/2 = 0
This gives the Newton-Raphson extended formula: x₁ = x₀ − f(x₀)/f'(x₀) − ½[f(x₀)]²f''(x₀)/[f'(x₀)]³
🔑 Definition — Chebyshev's formula of third order: An extended Newton-Raphson formula that includes second derivative information, providing cubic convergence.
⭐ Key Takeaways
Muller's method uses three points to construct a parabola and requires no derivatives, converging faster than secant method but almost as fast as Newton's method near simple roots. Graeffe's Root Squaring Method finds all roots of a polynomial simultaneously by repeatedly squaring the polynomial, separating roots by magnitude, then extracting the 2ⁱ-th root of coefficient ratios. The method handles real and complex roots but fails for repeated roots. The Newton-Raphson extended formula (Chebyshev's formula) improves convergence by incorporating second derivative information, giving cubic instead of quadratic convergence. Graeffe's method is especially useful because it requires no initial approximations, though it only works on polynomials and coefficients may overflow.
🧠 Quick Revision Questions
- How many starting points does Muller's method require, and what geometric object is constructed through them?
- In Graeffe's Root Squaring Method, what relationship exists between the roots of the transformed equation and the original equation?
- What is the convergence rate of Muller's method near a simple root, and how does it compare to Newton's method and the secant method?
- For the equation x³ − 6x² + 11x − 6 = 0, the Graeffe method with i=3 gives root approximations of 0.99949, 1.99143, and 3.0144. What are the exact roots, and what happens to the approximations as i increases?
- Write down the Newton-Raphson extended formula (Chebyshev's formula of third order) and explain what makes it different from the standard Newton-Raphson formula.
📘 Lecture 10 — Solution of Linear System of Equations and Matrix Inversion
📖 Overview: This lecture introduces numerical methods for solving systems of linear equations. It covers direct methods (Gaussian elimination with partial pivoting) for obtaining exact solutions after a finite number of steps, explaining the process of reducing a system to upper triangular form and using back substitution. The importance of pivoting to avoid zero divisors and ensure numerical stability is emphasized through detailed examples.
🗂️ Topics Covered
The lecture begins by classifying numerical methods into direct and iterative categories. It then focuses on the Gaussian elimination method, explaining the two stages: reduction to upper triangular form using elementary row transformations, and back substitution. The concept of pivoting, including partial and full pivoting, is introduced to handle cases where pivot elements become zero. Several fully worked examples demonstrate the application of Gaussian elimination with and without partial pivoting to solve 3×3 and 4×4 systems.
📝 Lecture Summary
Solution of Linear System of Equations and Matrix Inversion
A system of n linear equations in n unknowns has a unique solution if the determinant of the coefficient matrix A is non-zero (|A| ≠ 0). If |A| = 0, either no solution or infinite solutions exist. Numerical methods are classified into direct methods (where the solution is obtained after performing all steps) and iterative methods (where an initial approximation is improved step-by-step). Direct methods include elimination methods (Gaussian elimination, Gauss-Jordan) and decomposition methods (Crout’s/Cholesky’s reduction). Iterative methods include Jacobi, Gauss-Seidel, and relaxation methods.
💡 Why this matters: The choice between direct and iterative methods depends on the size and structure of the system. Direct methods are preferred for smaller, dense systems, while iterative methods are more efficient for large, sparse systems.
Gaussian Elimination Method
This method has two stages. Stage I: The given system is reduced to an equivalent upper triangular form using elementary row transformations. Stage II: The upper triangular system is solved using back substitution.
🔑 Definition — Upper triangular form: A matrix where all entries below the main diagonal are zero (cᵢⱼ = 0 for i > j). 📐 Process: Starting with the system Ax = b, the first equation is used to eliminate x₁ from all subsequent equations. This process is repeated for x₂, x₃, etc., until the system takes the form: c₁₁x₁ + c₁₂x₂ + … + c₁ₙxₙ = d₁ c₂₂x₂ + … + c₂ₙxₙ = d₂ ... cₙₙxₙ = dₙ
Then, backward substitution proceeds: xₙ = dₙ/cₙₙ, then substitute to find xₙ₋₁, and so on. 📌 Example: Solve 2x + 3y - z = 5, 4x + 4y - 3z = 3, -2x + 3y - z = 1. Stage I: Divide first eqn by 2, eliminate x from eqn 2 and 3 → x + 1.5y - 0.5z = 2.5, -2y - z = -7, 6y - 2z = 6. Divide 2nd eqn by -2, eliminate y from eqn 3 → x + 1.5y - 0.5z = 2.5, y + 0.5z = 3.5, -5z = -15. Stage II: From eqn 3, z = 3. From eqn 2, y = 3.5 - 0.5(3) = 2. From eqn 1, x = 2.5 - 1.5(2) + 0.5(3) = 1. Solution: x = 1, y = 2, z = 3.
Partial and Full Pivoting
The Gaussian elimination method fails if any pivot element becomes zero. Pivoting is the process of reordering equations (and occasionally variables) to avoid zero pivots.
🔑 Definition — Partial pivoting: If the pivot in the i-th column is zero, search all rows below (j > i) for the numerically largest element in that column, then interchange the i-th row with the j-th row. 🔑 Definition — Full (complete) pivoting: Both rows and columns are interchanged so that the largest element in the entire remaining submatrix becomes the pivot. This is more complicated and less commonly used for hand computation.
📌 Example (Partial pivoting necessity): Solve the system 10⁻⁵x₁ + x₂ = 1, x₁ + x₂ = 2. Without pivoting (using first eqn to eliminate x₁): The multiplier is 10⁵, leading to severe round-off error and an incorrect solution x₁ = 0, x₂ = 1. With partial pivoting: Swap rows so the largest element (1) in column 1 is the pivot → x₁ + x₂ = 2, 10⁻⁵x₁ + x₂ = 1. Now elimination gives the correct solution x₁ = 1, x₂ = 1.
📌 Example: Solve x + y + z = 7, 3x + 3y + 4z = 24, 2x + y + 3z = 16 using Gaussian elimination with partial pivoting. Matrix form: [1 1 1 | 7], [3 3 4 | 24], [2 1 3 | 16]. Pivot check: a₁₁ = 1 (non-zero), but column 1 has a larger element 3 in row 2. Swap R₁ and R₂ → [3 3 4 | 24], [1 1 1 | 7], [2 1 3 | 16]. Stage I: Divide R₁ by 3 → [1 1 4/3 | 8], [1 1 1 | 7], [2 1 3 | 16]. Eliminate x from R₂ (R₂ - R₁) and R₃ (R₃ - 2R₁) → [1 1 4/3 | 8], [0 0 -1/3 | -1], [0 -1 1/3 | 0]. Swap R₂ and R₃ to get non-zero pivot → [1 1 4/3 | 8], [0 -1 1/3 | 0], [0 0 -1/3 | -1]. Stage II: From eqn 3, z = 3. From eqn 2, -y + 1 = 0 → y = 1. From eqn 1, x + 1 + 4 = 8 → x = 3.
📌 Example: Solve 0x₁ + 4x₂ + 2x₃ + 8x₄ = 24, 4x₁ + 10x₂ + 5x₃ + 4x₄ = 32, 4x₁ + 5x₂ + 6.5x₃ + 2x₄ = 26, 9x₁ + 4x₂ + 4x₃ + 0x₄ = 21 using Gaussian elimination with partial pivoting. Matrix: [0 4 2 8 | 24], [4 10 5 4 | 32], [4 5 6.5 2 | 26], [9 4 4 0 | 21]. Pivot a₁₁ = 0, so swap R₁ with R₄ (largest in column 1) → [9 4 4 0 | 21], [4 10 5 4 | 32], [4 5 6.5 2 | 26], [0 4 2 8 | 24]. Stage I: Divide R₁ by 9, eliminate x₁ from R₂, R₃, R₄. Continue elimination, choosing new pivots in each column sequentially (8.2222, 3.4594, 5.999). Final upper triangular system yields x₁ = 1.0, x₂ = 1.0, x₃ = 2.0, x₄ = 2.0.
📌 Example: Solve 3x + y - z = 3, 2x - 8y + z = -5, x - 2y + 9z = 8. Augmented matrix: [3 1 -1 | 3], [2 -8 1 | -5], [1 -2 9 | 8]. Stage I: R₂ ← R₂ - (2/3)R₁, R₃ ← R₃ - (1/3)R₁ → [3 1 -1 | 3], [0 -26/3 5/3 | -7], [0 -7/3 28/3 | 7]. Pivot in column 2 is -26/3 (non-zero). R₃ ← R₃ - (7/26)R₂ → [3 1 -1 | 3], [0 -26/3 5/3 | -7], [0 0 693/78 | 231/26]. Stage II: z = 1. y = 1. x = 1. Solution: x = y = z = 1.
📌 Example: Solve 3.15x - 1.96y + 3.85z = 12.95, 2.13x + 5.12y - 2.89z = -8.61, 5.92x + 3.05y + 2.15z = 6.88. Augmented matrix: [3.15 -1.96 3.85 | 12.95], [2.13 5.12 -2.89 | -8.61], [5.92 3.05 2.15 | 6.88]. R₂ ← R₂ - (2.13/3.15)R₁, R₃ ← R₃ - (5.92/3.15)R₁ → [3.15 -1.96 3.85 | 12.95], [0 6.4453 -5.4933 | -17.3667], [0 6.7335 -5.0855 | -17.4578]. Pivot = 6.4453. R₃ ← R₃ - (6.7335/6.4453)R₂ → [3.15 -1.96 3.85 | 12.95], [0 6.4453 -5.4933 | -17.3667], [0 0 0.6534 | 0.6853]. Back substitution: z = 0.6853/0.6534 = 1.0488215, y = (-17.3667 + 5.4933(1.0488215))/6.4453 = -1.8005692, x = (12.95 + 1.96(-1.8005692) - 3.85(1.0488215))/3.15 = 1.708864.
📌 Example: Solve x₁ + x₂ + x₃ + x₄ = 2, x₁ + x₂ + 3x₃ - 2x₄ = -6, 2x₁ + 3x₂ - x₃ + 2x₄ = 7, x₁ + 2x₂ + x₃ - x₄ = -2. Augmented matrix: [1 1 1 1 | 2], [1 1 3 -2 | -6], [2 3 -1 2 | 7], [1 2 1 -1 | -2]. Stage I: R₂ ← R₂ - R₁, R₃ ← R₃ - 2R₁, R₄ ← R₄ - R₁ → [1 1 1 1 | 2], [0 0 2 -3 | -8], [0 1 -3 0 | 3], [0 1 0 -2 | -4]. Pivot a₂₂ = 0, so swap R₂ and R₃ → [1 1 1 1 | 2], [0 1 -3 0 | 3], [0 0 2 -3 | -8], [0 1 0 -2 | -4]. R₄ ← R₄ - R₂ → [1 1 1 1 | 2], [0 1 -3 0 | 3], [0 0 2 -3 | -8], [0 0 3 -2 | -7]. Pivot = 2. R₄ ← R₄ - (3/2)R₃ → [1 1 1 1 | 2], [0 1 -3 0 | 3], [0 0 2 -3 | -8], [0 0 0 5/2 | 5]. Back substitution: x₄ = 2, x₃ = (-8 + 3(2))/2 = -1, x₂ = 3 + 3(-1) = 0, x₁ = 2 - 0 - (-1) - 2 = 1. Solution: x₁ = 1, x₂ = 0, x₃ = -1, x₄ = 2.
⭐ Key Takeaways
The Gaussian elimination method is a foundational direct method for solving linear systems, operating in two distinct phases: forward elimination to create an upper triangular matrix, followed by backward substitution. Pivoting, especially partial pivoting, is crucial to avoid zero pivots and mitigate round-off errors; it involves swapping rows to place the numerically largest element in the current column as the pivot. The method is applied systematically by first forming the augmented matrix, then performing elementary row operations with carefully computed multipliers. When a pivot is zero, the current row must be interchanged with a row below it that contains a non-zero (preferably the largest) element in that column. The solution is then extracted sequentially from the bottom up, substituting known values into each preceding equation.
🧠 Quick Revision Questions
- What are the two main stages of the Gaussian elimination method?
- Why is partial pivoting necessary, and what is the rule for choosing which row to swap with?
- In the system x + y + z = 6, 2x + 3y + z = 10, 3x + 2y + 2z = 11, what is the first step (with pivoting) when using Gaussian elimination?
- After forward elimination, how is the value of the last unknown found?
- What distinguishes a direct method from an iterative method for solving linear systems?
📘 Lecture 11 — Solution of Linear System of Equations and Matrix Inversion Gauss–Jordon Elimination Method
📖 Overview: This lecture covers two powerful direct methods for solving systems of linear equations. The Gauss-Jordan elimination method extends Gaussian elimination to produce a diagonal matrix, directly yielding the solution without back substitution. Crout’s reduction method (also known as LU decomposition) factors the coefficient matrix into lower and upper triangular matrices, enabling efficient solution through forward and backward substitution. These techniques are fundamental for numerical analysis and computer programming.
🗂️ Topics Covered
The lecture begins with the Gauss-Jordan elimination method, which reduces a system to an equivalent diagonal form by making elements above and below the diagonal simultaneously zero. Several worked examples demonstrate the procedure for 3x3 and 4x4 systems with partial pivoting. The second half introduces Crout’s reduction method, which decomposes the coefficient matrix [A] into a lower-triangular matrix [L] and an upper-triangular matrix [U] with 1’s on the main diagonal. Detailed step-by-step examples show how to compute the [L] and [U] matrices, then solve for unknowns using forward substitution (Ly = B) and backward substitution (Ux = y).
📝 Lecture Summary
Gauss–Jordon Elimination Method
This method is a variation of Gaussian elimination. In this method, the elements above and below the diagonal are simultaneously made zero. That is, a given system is reduced to an equivalent diagonal form using elementary transformations. Then the solution of the resulting diagonal system is obtained. Sometimes, we normalize the pivot row with respect to the pivot element before elimination. Partial pivoting is also used whenever the pivot element becomes zero.
💡 Why this matters: Unlike Gaussian elimination, Gauss-Jordan eliminates the need for back substitution — the solution appears directly as the last column of the augmented matrix.
Example
Solve the system of equations using Gauss-Jordan elimination method:
x + 2y + z = 8
2x + 3y + 4z = 20
4x + 3y + 2z = 16
Solution
In matrix notation, the given system can be written as:
[1 2 1] [x] [8 ]
[2 3 4] [y] = [20]
[4 3 2] [z] [16]
After row operations (-2)R1+R2 and (-4)R1+R3:
[1 2 1] [x] [8 ]
[0 -1 2] [y] = [4 ]
[0 -5 -2] [z] [-16]
Eliminate y from the first and third rows using the second row:
[1 0 5] [x] [16]
[0 -1 2] [y] = [4 ]
[0 0 -12][z] [-36]
Normalize the third row (divide by -12):
[1 0 5] [x] [16]
[0 -1 2] [y] = [4 ]
[0 0 1] [z] [3 ]
Eliminate z from the first and second rows:
[1 0 0] [x] [1 ]
[0 -1 0] [y] = [-2]
[0 0 1] [z] [3 ]
The solution is x = 1, y = 2, z = 3.
Example
Solve using Gauss-Jordan method:
10x + y + z = 12
2x + 10y + z = 13
x + y + 5z = 7
The augmented matrix is:
[10 1 1 | 12]
[2 10 1 | 13]
[1 1 5 | 7 ]
After multiple row operations (R1 - 9R3, then R2 - 2R1 and R3 - R1, then R2 - 3R3, then R1 + 8R2 and R3 - 9R2, then dividing R3 by -473, then R1 - 420R3 and R2 - 58R3), the system reduces to reduced echelon form giving x = y = z = 1.
Example
Solve using Gauss-Jordan method:
10x₁ + x₂ + x₃ = 12
x₁ + 10x₂ - x₃ = 10
x₁ - 2x₂ + 10x₃ = 9
The solution gives x₁ = x₂ = x₃ = 1.
Example
Solve using Gauss-Jordan method:
x + 2y + z - w = -2
2x + 3y - z + 2w = 7
x + y + 3z - 2w = -6
x + y + z + w = 2
Through systematic row operations, the final reduced matrix gives:
x = 1, y = 0, z = -1, w = 2.
Crout’s Reduction Method
Here the coefficient matrix [A] of the system of equations is decomposed into the product of two matrices [L] and [U], where [L] is a lower-triangular matrix and [U] is an upper-triangular matrix with 1’s on its main diagonal.
For a general 3x3 matrix:
[l₁₁ 0 0 ] [1 u₁₂ u₁₃] [a₁₁ a₁₂ a₁₃]
[l₂₁ l₂₂ 0 ] [0 1 u₂₃] = [a₂₁ a₂₂ a₂₃]
[l₃₁ l₃₂ l₃₃] [0 0 1 ] [a₃₁ a₃₂ a₃₃]
The sequence of steps:
- Multiply all rows of [L] by the first column of [U]: l₁₁ = a₁₁, l₂₁ = a₂₁, l₃₁ = a₃₁ (first column of [L] = first column of [A]).
- Multiply the first row of [L] by the second and third columns of [U]: u₁₂ = a₁₂/l₁₁, u₁₃ = a₁₃/l₁₁.
- Multiply the 2nd and 3rd rows of [L] by the 2nd column of [U]: l₂₂ = a₂₂ - l₂₁u₁₂, l₃₂ = a₃₂ - l₃₁u₁₂.
- Multiply the 2nd row of [L] by the 3rd column of [U]: u₂₃ = (a₂₃ - l₂₁u₁₃)/l₂₂.
- Multiply the 3rd row of [L] by the 3rd column of [U]: l₃₃ = a₃₃ - l₃₁u₁₃ - l₃₂u₂₃.
This algorithm is generalized to any linear system of order n.
To solve A = (B): Let [A] = [L][U], then [L]U = (B). Substituting U = (Z), we get L = (B). Having computed Z (forward substitution), compute X from U = (Z) (back substitution).
This method is also known as the Cholesky reduction method. It is very popular from a computer programming point of view, since the storage space reserved for matrix [A] can be used to store the elements of [L] and [U] at the end of computation. This method fails if any aᵢᵢ = 0.
💡 Why this matters: LU decomposition is particularly efficient when solving multiple systems with the same coefficient matrix but different right-hand sides — the decomposition is done once.
Example
Solve using Crout’s reduction method:
5x₁ - 2x₂ + x₃ = 4
7x₁ + x₂ - 5x₃ = 8
3x₁ + 7x₂ + 4x₃ = 10
Solution
Let [L][U] = [A]:
[l₁₁ 0 0 ] [1 u₁₂ u₁₃] [5 -2 1 ]
[l₂₁ l₂₂ 0 ] [0 1 u₂₃] = [7 1 -5]
[l₃₁ l₃₂ l₃₃] [0 0 1 ] [3 7 4 ]
Step I: l₁₁ = 5, l₂₁ = 7, l₃₁ = 3
Step II: u₁₂ = -2/5, u₁₃ = 1/5
Step III: l₂₂ = 1 - 7(-2/5) = 19/5, l₃₂ = 7 - 3(-2/5) = 41/5
Step IV: u₂₃ = (-5 - 7(1/5))/(19/5) = -32/19
Step V: l₃₃ = 4 - 3(1/5) - (41/5)(-32/19) = 327/19
System becomes [L][U][X] = (B):
[5 0 0 ] [1 -2/5 1/5 ] [x₁] [4 ]
[7 19/5 0 ] [0 1 -32/19] [x₂] = [8 ]
[3 41/5 327/19] [0 0 1 ] [x₃] [10]
5z₁ = 4 → z₁ = 4/5
7z₁ + (19/5)z₂ = 8 → z₂ = 12/19
3z₁ + (41/5)z₂ + (327/19)z₃ = 10 → z₃ = 46/327
Back substitution from U = (Z):
x₃ = 46/327
x₂ - (32/19)x₃ = 12/19 → x₂ = 284/327
x₁ - (2/5)x₂ + (1/5)x₃ = 4/5 → x₁ = 366/327
The solution is x₁ = 366/327, x₂ = 284/327, x₃ = 46/327.
Example
Solve using Crout’s method:
2x - 3y + 10z = 3
-x + 4y + 2z = 20
5x + 2y + z = -12
Here [L] has 1’s on the diagonal (different convention):
[1 0 0] [u₁₁ u₁₂ u₁₃] [2 -3 10]
[l₂₁ 1 0] [0 u₂₂ u₂₃] = [-1 4 2 ]
[l₃₁ l₃₂ 1] [0 0 u₃₃] [5 2 1 ]
Computing: u₁₁=2, u₁₂=-3, u₁₃=10, l₂₁=-1/2, l₃₁=5/2, u₂₂=5/2, u₂₃=7, l₃₂=19/5, u₃₃=-253/5
Let UX = Y, then LY = B:
y₁ = 3
(-1/2)y₁ + y₂ = 20 → y₂ = 43/2
(5/2)y₁ + (19/5)y₂ + y₃ = -12 → y₃ = -506/5
From UX = Y:
2x - 3y + 10z = 3
(5/2)y + 7z = 43/2
(-253/5)z = -506/5
By back substitution: x = -4, y = 3, z = 2.
Example
Solve using Crout’s method:
x + 3y + 8z = 4
x + 4y + 3z = -2
x + 3y + 4z = 1
Computing: u₁₁=1, u₁₂=3, u₁₃=8, l₂₁=1, l₃₁=1, u₂₂=1, u₂₃=-5, l₃₂=0, u₃₃=-4
From LY = B: y₁=4, y₂=-2, y₃=-3
From UX = Y: x+3y+8z=4, y-5z=-2, -4z=-3
Solution: z = 3/4, y = 7/4, x = 29/4.
⭐ Key Takeaways
The Gauss-Jordan elimination method reduces a system directly to a diagonal form by making elements both above and below the diagonal zero simultaneously, eliminating the need for back substitution — this is its key advantage over Gaussian elimination. Crout’s reduction method (LU decomposition) factors [A] into [L] (lower triangular) and [U] (upper triangular with 1’s on the diagonal), then solves through forward substitution ([L]Z = B) followed by back substitution ([U]X = Z), making it highly efficient for computer implementation where the same matrix is used with multiple right-hand sides. In Crout’s method, the first column of [L] equals the first column of [A], and the first row of [U] (except the first element which is 1) is obtained by dividing by l₁₁ — failure occurs if any diagonal element becomes zero.
🧠 Quick Revision Questions
- What is the fundamental difference between Gauss-Jordan elimination and Gaussian elimination in terms of the final form of the matrix?
- In Crout’s reduction method, what are the values on the main diagonal of the [U] matrix?
- For the system represented by [L]U = (B), what two substitution steps are performed and in what order?
- What condition causes Crout’s reduction method to fail?
- In the Gauss-Jordan method for the system {x + 2y + z = 8, 2x + 3y + 4z = 20, 4x + 3y + 2z = 16}, what was the purpose of normalizing the third row (dividing by -12) before eliminating z from the first and second rows?
📘 Lecture 12 — Solution of Linear System of Equations and Matrix Inversion: Jacobi’s Method
📖 Overview: This lecture introduces Jacobi’s Method, an iterative technique for solving linear systems of equations, particularly advantageous when the coefficient matrix is sparse. It covers the method’s derivation, the crucial condition of diagonal dominance for convergence, and demonstrates the iterative process through multiple worked examples.
🗂️ Topics Covered
The lecture begins by introducing Jacobi’s Method as an iterative approach for solving linear systems, especially sparse ones. It then states the requirement for the coefficient matrix to be strictly diagonally dominant. The iterative formulas are derived from the original system by solving for each variable. Two detailed examples are presented, showing the step-by-step calculation of successive approximations from an initial guess of zero. A third example demonstrates how to rearrange a non-diagonally-dominant system to meet the convergence condition before applying the method.
📝 Lecture Summary
Solution of Linear System of Equations and Matrix Inversion
This section introduces Jacobi’s Method as an iterative method, where an initial approximate solution is assumed and progressively improved. This is in contrast to direct methods like Gaussian elimination.
💡 Why this matters: Iterative methods are highly efficient for sparse matrices (matrices with many zero elements), which are common in fields like solving partial differential equations. They require less computer memory than direct methods for such systems.
Jacobi’s Method
The method begins by assuming the coefficient matrix [A] is strictly diagonally dominant. This means that for every row, the absolute value of the diagonal element is greater than the sum of the absolute values of all other elements in that row.
|a_ii| > Σ |a_ij|forj ≠ i
It is also assumed that no diagonal element is zero. If it is, the equations should be rearranged.
The system of n equations is then rewritten to isolate each variable on the left-hand side:
x_i = (b_i - Σ a_ij * x_j) / a_iiforj ≠ i
Given an initial approximation vector (x₁⁽¹⁾, x₂⁽¹⁾, …, xₙ⁽¹⁾)ᵀ, the second approximation is computed by substituting the first into the right-hand side. This process yields the general formula for the (r+1)th approximation:
x_i^(r+1) = (b_i - Σ a_ij * x_j^(r)) / a_iiforj ≠ i
This is known as the method of simultaneous displacements because all variables from the r-th iteration are used to compute all new (r+1)-th values in one step, without updating them during the iteration.
🔑 Definition — Diagonal Dominance: A matrix condition where the diagonal element in each row is larger in magnitude than the sum of the magnitudes of all other elements in that row. It is a sufficient condition for the convergence of Jacobi’s method.
📐 Formula: x_i^(r+1) = (b_i - Σ a_ij * x_j^(r)) / a_ii for j ≠ i → The new value of variable x_i is calculated by dividing the constant b_i minus the sum of all off-diagonal terms (from row i, each multiplied by their current iteration r value) by the diagonal element a_ii.
Example 1
Problem: Solve the system using Jacobi’s iterative method for five iterations.
83x + 11y - 4z = 957x + 52y + 13z = 1043x + 8y + 29z = 71
Solution: First, the equations are rewritten.
x = (95/83) - (11/83)y + (4/83)z→x = 1.1446 - 0.1325y + 0.0482zy = (104/52) - (7/52)x - (13/52)z→y = 2.0 - 0.1346x - 0.25zz = (71/29) - (3/29)x - (8/29)y→z = 2.4483 - 0.1035x - 0.2759y
The initial approximation is (x, y, z) = (0, 0, 0).
Iteration 1: Substitute the initial guess into the right-hand side.
x = 1.1446, y = 2.0, z = 2.4483 → (1.1446, 2.0, 2.4483)
Iteration 2: Substitute the values from Iteration 1.
x = 1.1446 - 0.1325(2.0) + 0.0482(2.4483) = 0.9976 y = 2.0 - 0.1346(1.1446) - 0.25(2.4483) = 1.2339 z = 2.4483 - 0.1035(1.1446) - 0.2759(2.0) = 1.7424 → (0.9976, 1.2339, 1.7424)
Further iterations yield the sequence:
| r | x | y | z |
|---|---|---|---|
| 1 | 1.1446 | 2.0000 | 2.4483 |
| 2 | 0.9976 | 1.2339 | 1.7424 |
| 3 | 1.0651 | 1.4301 | 2.0046 |
| 4 | 1.0517 | 1.3555 | 1.9435 |
| 5 | 1.0587 | 1.3726 | 1.9655 |
The values appear to be converging towards the exact solution.
Example 2
Problem: Solve the system using Jacobi’s method (four iterations).
8x - 3y + 2z = 204x + 11y - z = 336x + 3y + 12z = 35
Solution: The system is diagonally dominant. The equations are rewritten.
x = (1/8)[20 + 3y - 2z]y = (1/11)[33 - 4x + z]z = (1/12)[35 - 6x - 3y]
Starting from (0, 0, 0):
Iteration 1: x₁ = 2.5, y₁ = 3, z₁ = 2.916667
Iteration 2: x₂ = 2.895833, y₂ = 2.3560606, z₂ = 0.9166666
Iteration 3: x₃ = 3.1543561, y₃ = 2.030303, z₃ = 0.8797348
Iteration 4: x₄ = 3.0419299, y₄ = 1.9329373, z₄ = 0.8319128
Example 3
Problem: Solve the system using Jacobi’s method (four iterations).
3x + 4y + 15z = 54.8x + 12y + 3z = 39.6610x + y - 2z = 7.74
Solution: The original system is not diagonally dominant. It is rearranged to meet the condition.
10x + y - 2z = 7.74x + 12y + 3z = 39.663x + 4y + 15z = 54.8
Now, the system is diagonally dominant. The equations are rewritten.
x = (1/10)[7.74 - y + 2z]y = (1/12)[39.66 - x - 3z]z = (1/15)[54.8 - 3x - 4y]
Starting from (0, 0, 0):
Iteration 1: x₁ = 0.774, y₁ = 3.305, z₁ = 3.6533333
Iteration 2: x₂ = 1.3908333, y₂ = 2.3271667, z₂ = 3.1949778
Iteration 3: x₃ = 1.1802789, y₃ = 2.3903528, z₃ = 2.7545889
Iteration 4: x₄ = 1.0781704, y₄ = 2.51779962, z₄ = 2.7798501
⭐ Key Takeaways
- Jacobi’s method is an iterative technique that solves linear systems by repeatedly substituting the entire previous approximation into a set of reformulated equations, making it a method of simultaneous displacements.
- A sufficient condition for the convergence of the Jacobi method is that the coefficient matrix is strictly diagonally dominant. If a system is not diagonally dominant, it must be rearranged before the method can be reliably applied.
- The method is particularly advantageous for sparse matrices (where many elements are zero), as it is more memory-efficient than direct methods like Gaussian elimination.
- The process begins with an initial guess (often the zero vector) and generates a sequence of approximations that, if convergent, approach the true solution to the system.
🧠 Quick Revision Questions
- What is the key mathematical condition that is sufficient for the Jacobi iterative method to converge?
- Explain why Jacobi’s method is called the "method of simultaneous displacements."
- Given the system
10x - y + z = 12and2x + 10y - z = 11and-x + y + 10z = 10, write the equations in the form suitable for Jacobi's method. - For the system in question 3, perform the first iteration starting from
(0,0,0). What is the resulting approximate solution vector? - If a system of linear equations is not diagonally dominant, what must be done before applying Jacobi’s method, and why?
📘 Lecture 13 — Solution of Linear System of Equations and Matrix Inversion
📖 Overview: This lecture introduces the Gauss-Seidel Iteration Method for solving systems of linear equations. It explains how this method differs from Jacobi's method by using updated values immediately, making it faster and more efficient for large systems.
🗂️ Topics Covered
The lecture covers the Gauss-Seidel iteration method formulation, its comparison with Jacobi's method, the general compact form of the procedure, and multiple solved examples demonstrating the iterative process for various systems of equations including 3×3 and 4×4 systems.
📝 Lecture Summary
Solution of Linear System of Equations and Matrix Inversion – Gauss–Seidel Iteration Method
The Gauss-Seidel Iteration Method is an iterative technique for solving linear systems of the form a₁₁x₁ + a₁₂x₂ + … + a₁ₙxₙ = b₁, a₂₁x₁ + a₂₂x₂ + … + a₂ₙxₙ = b₂, … , aₙ₁x₁ + aₙ₂x₂ + … + aₙₙxₙ = bₙ.
In Jacobi’s method, the (r+1)th approximation uses only values from the rth iteration. No element of the new approximation replaces the old one during computation. In contrast, in the Gauss-Seidel method, the corresponding elements of x^(r+1) replace those of x^(r) as soon as they become available. Hence, it is called the method of successive displacements.
🔑 Definition — Gauss-Seidel Iteration: For a system of n linear equations, the (r+1)th approximation is computed using the most recently updated values. For the first equation, we use x₂^(r), x₃^(r), …, xₙ^(r). For the second equation, we use x₁^(r+1), x₃^(r), …, xₙ^(r). For the third, we use x₁^(r+1), x₂^(r+1), x₄^(r), …, and so on.
📐 Formula: For each variable xᵢ at iteration (r+1): xᵢ^(r+1) = (bᵢ - Σⱼ₌₁^(i-1) aᵢⱼ xⱼ^(r+1) - Σⱼ₌ᵢ₊₁ⁿ aᵢⱼ xⱼ^(r)) / aᵢᵢ
This means: To find each new value, subtract the contributions of all previously updated variables and the remaining old variables from the constant term, then divide by the diagonal coefficient.
💡 Why this matters: The immediate replacement of values in Gauss-Seidel typically leads to faster convergence than Jacobi's method, requiring fewer iterations to reach the same accuracy.
Example — System of 4 Equations
Solve the system using Gauss-Seidel method with first five iterations: 4x₁ - x₂ - x₃ = 2 -x₁ + 4x₂ - x₄ = 2 -x₁ + 4x₃ - x₄ = 1 -x₂ - x₃ + 4x₄ = 1
📌 Example: The system is rewritten as: x₁ = 0.5 + 0.25x₂ + 0.25x₃ x₂ = 0.5 + 0.25x₁ + 0.25x₄ x₃ = 0.25 + 0.25x₁ + 0.25x₄ x₄ = 0.25 + 0.25x₂ + 0.25x₃
Starting with x₂ = x₃ = x₄ = 0 in the first equation: x₁^(1) = 0.5 Using x₁^(1) = 0.5 and x₃ = x₄ = 0: x₂^(1) = 0.5 + (0.25)(0.5) + 0 = 0.625 Using x₁^(1) = 0.5, x₂^(1) = 0.625, and x₄ = 0: x₃^(1) = 0.25 + (0.25)(0.5) + 0 = 0.375 Using x₂^(1) = 0.625 and x₃^(1) = 0.375: x₄^(1) = 0.25 + (0.25)(0.625) + (0.25)(0.375) = 0.5
Continuing iterations: Iteration 1: x₁=0.5, x₂=0.625, x₃=0.375, x₄=0.5 Iteration 2: x₁=0.75, x₂=0.8125, x₃=0.5625, x₄=0.59375 Iteration 3: x₁=0.84375, x₂=0.85938, x₃=0.60938, x₄=0.61719 Iteration 4: x₁=0.86719, x₂=0.87110, x₃=0.62110, x₄=0.62305 Iteration 5: x₁=0.87305, x₂=0.87402, x₃=0.62402, x₄=0.62451
Example — 3×3 System with Diagonal Dominance
Solve by Gauss-Seidel iterative method (four iterations): 8x - 3y + 2z = 20 4x + 11y - z = 33 6x + 3y + 12z = 35
📌 Example: The system is diagonally dominant (each diagonal coefficient is greater than the sum of absolute values of off-diagonal coefficients in its row). Rewrite: x = (1/8)[20 + 3y - 2z] y = (1/11)[33 - 4x + z] z = (1/12)[35 - 6x - 3y]
Starting with x₀ = y₀ = z₀ = 0: Iteration 1: x₁ = (1/8)[20] = 2.5, y₁ = (1/11)[33 - 4(2.5) + 0] = 2.0909091, z₁ = (1/12)[35 - 6(2.5) - 3(2.0909091)] = 1.1439394 Iteration 2: x₂ = (1/8)[20 + 3(2.0909091) - 2(1.1439394)] = 2.9981061, y₂ = (1/11)[33 - 4(2.9981061) + 1.1439394] = 2.0137741, z₂ = (1/12)[35 - 6(2.9981061) - 3(2.0137741)] = 0.9141701 Iteration 3: x₃ = 3.0266228, y₃ = 1.9825163, z₃ = 0.9077262 Iteration 4: x₄ = 3.0165121, y₄ = 1.9856071, z₄ = 0.8319128
Example — System Requiring Equation Reordering
Solve using Gauss-Seidel iteration method: 28x + 4y - z = 32 x + 3y + 10z = 24 2x + 17y + 4z = 35
📌 Example: The original system is not diagonally dominant. The equations must be rearranged to make the system diagonally dominant: 28x + 4y - z = 32 (keep as is) 2x + 17y + 4z = 35 (move to second position) x + 3y + 10z = 24 (move to third position)
Now rewrite: x = (1/28)[32 - 4y + z] y = (1/17)[35 - 2x - 4z] z = (1/10)[24 - x - 3y]
Starting with y = z = 0: x₁ = 32/28 = 1.1428571 Using x₁ = 1.1428571, z = 0: y₁ = (1/17)[35 - 2(1.1428571)] = 1.9243697 Using x₁ = 1.1428571, y₁ = 1.9243697: z₁ = (1/10)[24 - 1.1428571 - 3(1.9243697)] = 1.7084034
Continuing iterations: Iteration 2: x₂ = 0.9289615, y₂ = 1.5475567, z₂ = 1.8408368 Iteration 3: x₃ = 0.9875932, y₃ = 1.5090274, z₃ = 1.8485325 Iteration 4: x₄ = 0.9933008, y₄ = 1.5070158, z₄ = 1.8485652
Example — 4×4 System
Solve using Gauss-Seidel iteration method (four iterations): 10x - 2y - z - w = 3 -2x + 10y - z - w = 15 -x - y + 10z - 2w = 27 -x - y - 2z + 10w = -9
📌 Example: The system is diagonally dominant. Rewrite: x = (1/10)[3 + 2y + z + w] y = (1/10)[15 + 2x + z + w] z = (1/10)[27 + x + y + 2w] w = (1/10)[-9 + x + y + 2z]
Starting with y = z = w = 0: x₁ = 0.3 Using x₁ = 0.3: y₁ = (1/10)[15 + 2(0.3)] = 1.56 Using x₁ = 0.3, y₁ = 1.56: z₁ = (1/10)[27 + 0.3 + 1.56] = 2.886 Using x₁ = 0.3, y₁ = 1.56, z₁ = 2.886: w₁ = (1/10)[-9 + 0.3 + 1.56 + 2(2.886)] = -0.1368
Continuing iterations: Iteration 2: x₂ = 0.88692, y₂ = 1.952304, z₂ = 2.9565624, w₂ = -0.0247651 Iteration 3: x₃ = 0.9836405, y₃ = 1.9899087, z₃ = 2.9924019, w₃ = -0.0041647 Iteration 4: x₄ = 0.9968054, y₄ = 1.9981848, z₄ = 2.9986661, w₄ = -0.0007677
When to Stop the Iterative Process
We stop the iterative process when we achieve the required accuracy. For example, if asked to find values accurate to six decimal places, and consecutive values are 1.919326355, 1.919327145, 1.919327128, the accuracy up to seven decimal places is achieved, so we stop. The stopping criterion depends on the desired precision specified in the problem.
⭐ Key Takeaways
The Gauss-Seidel iteration method is an improvement over Jacobi's method because it uses the most recently updated values immediately, leading to faster convergence. The system must be diagonally dominant for convergence, and if it isn't, the equations must be rearranged to achieve diagonal dominance. The general formula for each variable uses the sum of products of coefficients with previously updated values and remaining old values. The iterative process continues until the desired accuracy is reached, which is determined by comparing consecutive approximations. Gauss-Seidel is particularly effective for large sparse systems where direct methods are computationally expensive.
🧠 Quick Revision Questions
- What is the key difference between Jacobi's method and Gauss-Seidel method in terms when updated values are used?
- What condition must a system of equations satisfy for the Gauss-Seidel method to converge?
- For a 4×4 system using Gauss-Seidel, which variable values are used to compute x₂^(r+1)?
- In the example 10x - 2y - z - w = 3, what is the iterative formula for z?
- How do you determine when to stop the Gauss-Seidel iterative process?