Numerical Methods: free practice, theory and problems

1. Solving equations

What is it about?

Many practical engineering equations, such as $f(x) = 0$ arising from pipe friction, beam stresses or equilibrium points in a process, have no simple formula for the solution. Numerical methods find an approximate root by guessing, checking and gradually improving the guess. The goal is always the same: a number $x^*$ where $f(x^*) \approx 0$, within a given tolerance.

Concepts and formulas

• Root: a value $x^*$ where $f(x^*) = 0$.
• Bisection method: requires an interval $[a,b]$ where $f(a)$ and $f(b)$ have opposite signs. The midpoint $c=(a+b)/2$ is tested, and the half with the sign change is kept. Number of steps for tolerance $\varepsilon$: $n \geq \log_2\!\big((b-a)/\varepsilon\big)$.
• Newton's method: $x_{n+1} = x_n - \dfrac{f(x_n)}{f'(x_n)}$. Requires the derivative and a starting point near the root. Convergence is quadratic (the number of correct digits roughly doubles each step) when it works, but the method can diverge with a poor start or if $f'(x_n)\approx 0$.
• Secant method: like Newton, but the derivative is replaced by a difference quotient: $x_{n+1} = x_n - f(x_n)\dfrac{x_n-x_{n-1}}{f(x_n)-f(x_{n-1})}$. Needs two starting points, not the derivative.
• Fixed-point iteration: rewrite $f(x)=0$ as $x=g(x)$ and iterate $x_{n+1}=g(x_n)$. Converges if $|g'(x)|<1$ near the root.
• Order of convergence: how quickly the error $e_n=x_n-x^*$ shrinks. Linear (bisection, fixed point): the error scales by a roughly constant factor each step. Quadratic (Newton): $e_{n+1}\approx C e_n^2$.

How to solve the problems

1. Check what is given: an interval (bisection), a function and one starting point with the derivative (Newton), two starting points (secant), or a $g(x)$ (fixed point).
2. Evaluate $f$ (and $f'$ if needed) at the required points.
3. Use the correct update formula and find the next value.
4. Repeat until the change is smaller than the tolerance, or the desired number of steps is reached.

Example

Find a root of $f(x) = x^2 - 3$ with Newton's method, starting from $x_0 = 2$.

1. $f(x) = x^2-3$ and $f'(x)=2x$.
2. Step 1: $x_1 = x_0 - \dfrac{f(x_0)}{f'(x_0)} = 2 - \dfrac{4-3}{4} = 2 - 0.25 = 1.75$.
3. Step 2: $f(1.75)=1.75^2-3=0.0625$ and $f'(1.75)=3.5$, so $x_2 = 1.75 - 0.0625/3.5 \approx 1.7321$.

The answer quickly approaches $\sqrt3 \approx 1.7321$, typical for quadratic convergence.

Common mistakes

• Starting bisection on an interval where $f(a)$ and $f(b)$ have the same sign.
• Forgetting to check whether $f'(x_n)$ is close to zero in Newton's method (the method can then jump far off track).
• Confusing the speed of convergence (how fast) with whether the method converges at all.
• Believing that more decimal places in the intermediate work automatically give a more accurate answer than the method can actually deliver.

Concepts in this part

Practise solving equations in the app →

2. Interpolation and integration

What is it about?

Often you only have a few known points – from a table, a sensor or a calculation – and need a value between them, or the area under a curve you cannot integrate by hand. Interpolation builds a function that passes exactly through the known points, while numerical integration approximates a definite integral by summing the area of simple shapes (rectangles, trapezoids, parabolas).

Concepts and formulas

• Linear interpolation between $(x_0,y_0)$ and $(x_1,y_1)$: $y = y_0 + \dfrac{y_1-y_0}{x_1-x_0}(x-x_0)$.
• Lagrange interpolation: for $n+1$ points there is a unique polynomial of degree at most $n$ passing through all of them. It is built as a sum of basis polynomials $L_i(x)$, one per point, where $L_i(x_i)=1$ and $L_i(x_j)=0$ for $j\neq i$.
• Runge's phenomenon: high-degree interpolation at evenly spaced points can produce large, unrealistic oscillations near the ends. This is avoided with a lower degree, Chebyshev points, or splines (piecewise polynomials, smooth at the joints).
• Midpoint rule: the simplest quadrature rule, $\int_a^b f(x)\,dx \approx (b-a)f\big(\tfrac{a+b}2\big)$.
• Trapezoidal rule: $\int_a^b f(x)\,dx \approx (b-a)\dfrac{f(a)+f(b)}2$. Composite, with $n$ equal intervals ($h=(b-a)/n$): $\dfrac{h}{2}\big(f_0+2f_1+\dots+2f_{n-1}+f_n\big)$. The error is $O(h^2)$.
• Simpson's rule: uses parabolas over pairs of intervals, exact for polynomials up to degree 3. Composite (requires an even number of intervals): $\dfrac{h}{3}\big(f_0+4f_1+2f_2+4f_3+\dots+f_n\big)$. The error is $O(h^4)$.

How to solve the problems

1. Check what is asked for: an in-between value (interpolation) or an area/integral (quadrature).
2. For interpolation: set up the right formula (linear for 2 points, Lagrange/quadratic for 3).
3. For integration: find $h$, evaluate the function at every point, and use the correct weighted sum.
4. Check whether the method is exact for that type of function (e.g. Simpson is exact for a cubic polynomial).

Example

Interpolate quadratically through $(0,0)$, $(1,1)$ and $(2,4)$ (i.e. $y=x^2$). Find $y$ at $x=1.5$ with Lagrange's formula.

At $x=1.5$: $L_0=-0.125$, $L_1=0.75$ and $L_2=0.375$. Then $y = 0\cdot L_0 + 1\cdot L_1 + 4\cdot L_2 = 0.75+1.5 = 2.25$, exactly like $1.5^2$.

Common mistakes

• Using Simpson's rule with an odd number of intervals (the method requires an even number).
• Believing the trapezoidal rule is exact for a curved function, not just for a straight line.
• Confusing the number of points with the number of intervals ($n+1$ points give $n$ intervals).
• Extrapolating (going outside the data points) and assuming the result is as reliable as interpolation.

Concepts in this part

Practise interpolation and integration in the app →

3. Differential equations and stability

What is it about?

Many physical processes (cooling, electrical circuits, chemical reactions, mechanical vibrations) are described by differential equations with no simple formula for the solution. Numerical solvers build the solution step by step: given $y$ at time $t$, they compute an approximation of $y$ at $t+h$. As an engineer you use this to simulate systems over time, for example the temperature in an engine or the current in a circuit.

Concepts and formulas

• Euler's explicit method: $y_{n+1} = y_n + h f(t_n, y_n)$. Order 1 (global error $O(h)$). Simple, but can require a very small $h$.
• Heun's method (improved Euler): a predictor–corrector method of order 2. Predictor: $y_p = y_n + h f(t_n,y_n)$. Corrector: $y_{n+1} = y_n + \dfrac{h}{2}\big(f(t_n,y_n)+f(t_{n+1},y_p)\big)$ – i.e. the average of the slope at the start and end of the step.
• Backward (implicit) Euler: $y_{n+1} = y_n + h f(t_{n+1}, y_{n+1})$. $y_{n+1}$ appears on both sides, so an equation must be solved. More expensive per step, but stable even for large $h$ – important for stiff problems.
• Classic Runge–Kutta (RK4): order 4, uses four evaluations of $f$ per step: $k_1=f(t_n,y_n)$, $k_2=f(t_n+\tfrac h2,y_n+\tfrac h2k_1)$, $k_3=f(t_n+\tfrac h2,y_n+\tfrac h2k_2)$, $k_4=f(t_n+h,y_n+hk_3)$, and $y_{n+1}=y_n+\dfrac h6(k_1+2k_2+2k_3+k_4)$.
• Stiff problem: very different time scales in the same system. Explicit methods then need an unrealistically small $h$ to stay stable, even though the accuracy requirement itself does not demand it.
• Stability: for $y'=\lambda y$, explicit Euler is stable only when $|1+h\lambda|\leq 1$. Implicit methods are stable for much larger $h$.

How to solve the problems

1. Identify $f(t,y)$, the initial value and the step size $h$.
2. Choose the right method based on what the problem asks for (Euler, Heun, RK4 or implicit).
3. Compute one step at a time, using the result of the previous step as the starting point for the next.
4. For stability: form $|1+h\lambda|$ (explicit Euler) and compare it with 1.

Example

Solve $y'=-2y$, $y(0)=5$, with one step of Heun's method and $h=0.5$.

1. $f(t,y)=-2y$, so $f(t_0,y_0)=-2\cdot5=-10$.
2. Predictor: $y_p = 5 + 0.5\cdot(-10) = 0$.
3. Corrector: $f(t_1,y_p)=-2\cdot 0=0$, so $y_1 = 5 + \dfrac{0.5}{2}(-10+0) = 5-2.5=2.5$.

For comparison, explicit Euler gives $y_1=5+0.5\cdot(-10)=0$ – Heun's method is more accurate because it uses the average of two slopes.

Common mistakes

• Using $y_n$ instead of the predictor $y_p$ when computing the corrector step in Heun's method.
• Believing implicit Euler is "wrong" because $y_{n+1}$ appears on both sides – that is exactly the point, and for a linear problem it can be solved algebraically.
• Using too large an $h$ for a stiff problem with an explicit method, and getting a result that "explodes".
• Confusing the number of function evaluations (RK4 uses four per step) with the number of time steps.

Concepts in this part

Practise differential equations and stability in the app →