Linear Algebra and Differential Equations: free practice, theory and problems

1. Matrices

What is it about?

A matrix is a table of numbers, but its most important interpretation is as a linear map: $A$ takes a vector $\vec x$ and returns a new vector $A\vec x$. A truss, an electric circuit or a finite element model eventually becomes a large matrix system. The determinant tells you whether the map can be reversed, and the inverse matrix reverses it.

Key concepts and formulas

• An $m\times n$ matrix has $m$ rows and $n$ columns. $AB$ is defined when the number of columns of $A$ equals the number of rows of $B$: $(m\times n)(n\times p)$ gives $m\times p$.
• The element $(AB)_{ij}$ is row $i$ of $A$ dotted with column $j$ of $B$. In general $AB \neq BA$.
• Transpose: $(A^T)_{ij} = A_{ji}$, and $(AB)^T = B^TA^T$.
• Determinant: $\det\begin{pmatrix}a&b\\c&d\end{pmatrix} = ad - bc$. For $3\times 3$, use cofactor expansion along a row or column.
• Rules: $\det(AB) = \det A\cdot\det B$, $\det A^T = \det A$ and $\det(kA) = k^n\det A$ for an $n\times n$ matrix. Swapping two rows changes the sign.
• $|\det A|$ is the factor by which area (2D) or volume (3D) is scaled.
• Inverse: $AA^{-1} = A^{-1}A = I$. It exists only when $\det A \neq 0$. For $2\times 2$: swap the diagonal elements, change the sign of the others and divide by the determinant.
• Vectors: $\vec u\cdot\vec v = |\vec u||\vec v|\cos\theta$ and $|\vec u| = \sqrt{\vec u\cdot\vec u}$. The cross product $\vec u\times\vec v$ is perpendicular to both, and its length is the area of the parallelogram they span.

How to solve the problems

1. Write down the dimensions first, so you can see whether the product is defined and how large it is.
2. Compute products element by element: row times column.
3. $3\times 3$ determinant: expand along the row or column with the most zeros. For triangular matrices the determinant is the product of the diagonal.
4. Inverse: check that $\det A \neq 0$ first, and verify the answer with $AA^{-1} = I$.
5. Small systems $A\vec x = \vec b$ can be solved with $\vec x = A^{-1}\vec b$. Large systems are solved by elimination.

Example

Solve $A\vec x = \vec b$ with $A = \begin{pmatrix}3&1\\5&2\end{pmatrix}$ and $\vec b = (4, 7)$.

1. $\det A = 3\cdot 2 - 1\cdot 5 = 1 \neq 0$, so $A$ is invertible.
2. $A^{-1} = \frac11\begin{pmatrix}2&-1\\-5&3\end{pmatrix}$.
3. $\vec x = A^{-1}\vec b = (2\cdot 4 - 1\cdot 7,\ -5\cdot 4 + 3\cdot 7) = (1, 1)$.
4. Check: $3 + 1 = 4$ and $5 + 2 = 7$. Correct.

Common mistakes

• Multiplying element by element instead of row times column.
• Changing the order: $AB$ and $BA$ are usually different, and $(AB)^{-1} = B^{-1}A^{-1}$.
• Believing that $\det(A + B) = \det A + \det B$. It is not true.
• Forgetting that $\det(2A) = 2^n\det A$, not $2\det A$.
• Mixing up the dot product (a number) and the cross product (a vector).

Concepts in this part

Practise matrices in the app →

2. Linear systems and eigenvalues

What is it about?

Many engineering problems end up as a system of linear equations $A\vec x = \vec b$: currents in a circuit, member forces in a truss, temperatures at a grid of points. Gaussian elimination solves such systems systematically and tells you whether there is one solution, none or infinitely many.

Eigenvalues are about the special directions in which a matrix only stretches or shrinks: $A\vec v = \lambda\vec v$. They determine natural frequencies of vibrating systems, stability of control systems and the long-term behavior of models that are updated step by step.

Key concepts and formulas

• Row operations (swapping two rows, multiplying a row by a nonzero number, adding a multiple of one row to another) do not change the solution set.
• Echelon form: each pivot lies to the right of the pivot in the row above. The rank $r$ is the number of pivots.
• Three cases for $n$ unknowns: a row $[0\ \cdots\ 0 \mid c]$ with $c \neq 0$ gives no solution. Otherwise $r = n$ gives exactly one solution, and $r < n$ gives infinitely many with $n - r$ free variables.
• Eigenvalues: solve $\det(A - \lambda I) = 0$. For $2\times 2$ this is $\lambda^2 - (\operatorname{tr}A)\lambda + \det A = 0$.
• Eigenvectors: solve $(A - \lambda I)\vec v = \vec 0$ for each eigenvalue.
• The sum of the eigenvalues is the trace, and their product is the determinant.
• Diagonalization: $A = PDP^{-1}$ with the eigenvectors as the columns of $P$ and the eigenvalues on the diagonal of $D$. Then $A^k = PD^kP^{-1}$.

How to solve the problems

1. Write the augmented matrix $[A \mid \vec b]$ and eliminate downward to echelon form.
2. Read off the result: a contradiction, free variables or a unique solution. Back-substitute from the bottom row.
3. Eigenvalues: set up the characteristic equation and solve it.
4. For each eigenvalue: find a nonzero vector with $(A - \lambda I)\vec v = \vec 0$. The rows are then linearly dependent, so one of them is enough for a $2\times 2$ matrix.
5. Check with the trace and determinant, and by computing $A\vec v$.

Example

Find the eigenvalues and eigenvectors of $A = \begin{pmatrix}5&2\\2&2\end{pmatrix}$.

1. $\operatorname{tr}A = 7$ and $\det A = 10 - 4 = 6$, so $\lambda^2 - 7\lambda + 6 = (\lambda - 1)(\lambda - 6) = 0$.
2. $\lambda = 6$: $A - 6I = \begin{pmatrix}-1&2\\2&-4\end{pmatrix}$. The first row gives $-v_1 + 2v_2 = 0$, so $\vec v = (2, 1)$.
3. $\lambda = 1$: $A - I = \begin{pmatrix}4&2\\2&1\end{pmatrix}$ gives $2v_1 + v_2 = 0$, so $\vec v = (1, -2)$.
4. Check: $A(2, 1) = (12, 6) = 6\cdot(2, 1)$. The eigenvectors are perpendicular, as expected for a symmetric matrix.

Common mistakes

• Making an arithmetic error during elimination and not checking by substitution.
• Believing that a zero row always means infinitely many solutions. If it says $0 = c$ with $c \neq 0$, there is no solution.
• Computing $\det A - \lambda$ instead of $\det(A - \lambda I)$.
• Giving the zero vector as an eigenvector.
• Forgetting that eigenvectors are only determined up to a factor: $(2, 1)$ and $(4, 2)$ are equally valid.

Concepts in this part

Practise linear systems and eigenvalues in the app →

3. Differential equations

What is it about?

A differential equation relates an unknown function to its own derivatives. It describes how something changes: a capacitor being charged, a cup of coffee cooling down, a mass oscillating on a spring. Solving the equation means finding the function. The initial conditions pick out the one solution that fits the situation.

Key concepts and formulas

• Order: the highest derivative that appears. An equation of order $n$ needs $n$ initial conditions.
• Separable: $y' = g(x)h(y)$ is solved with $\int\frac{dy}{h(y)} = \int g(x)\,dx$.
• First-order linear: $y' + p(x)y = q(x)$. Multiply the whole equation by the integrating factor $\mu = e^{\int p\,dx}$. The left-hand side then becomes $(\mu y)'$.
• $y' = -k(y - y_\infty)$ has the solution $y = y_\infty + (y_0 - y_\infty)e^{-kt}$. It describes cooling and charging.
• Second order with constant coefficients: $y'' + py' + qy = 0$. Try $y = e^{rx}$, which gives the characteristic equation $r^2 + pr + q = 0$.
• Two distinct real roots: $y = C_1e^{r_1x} + C_2e^{r_2x}$.
• Double root: $y = (C_1 + C_2x)e^{rx}$.
• Complex roots $r = \alpha \pm i\beta$: $y = e^{\alpha x}(C_1\cos\beta x + C_2\sin\beta x)$.
• Nonhomogeneous equation: $y = y_h + y_p$, where $y_p$ is one particular solution. Guess a $y_p$ of the same type as the right-hand side.
• Mass–spring–damper $my'' + cy' + ky = 0$: $c^2 < 4mk$ gives damped oscillations, $c^2 > 4mk$ gives no oscillation (overdamping).

How to solve the problems

1. Determine the type: separable, first-order linear, or linear with constant coefficients.
2. Find the general solution, with constants.
3. If the equation is nonhomogeneous, find $y_p$ and add it to $y_h$.
4. Apply the initial conditions last, to the complete solution $y_h + y_p$.
5. Check by substituting the solution into the equation.

Example

Solve $y'' + 3y' + 2y = 4$ with $y(0) = 0$ and $y'(0) = 0$.

1. Homogeneous part: $r^2 + 3r + 2 = (r + 1)(r + 2) = 0$ gives $y_h = C_1e^{-x} + C_2e^{-2x}$.
2. The right-hand side is constant, so try $y_p = K$: $2K = 4$ gives $K = 2$.
3. $y = C_1e^{-x} + C_2e^{-2x} + 2$. The initial conditions give $C_1 + C_2 + 2 = 0$ and $-C_1 - 2C_2 = 0$.
4. This gives $C_2 = 2$ and $C_1 = -4$, so $y = 2 - 4e^{-x} + 2e^{-2x}$.

The solution starts at 0 and approaches the equilibrium $y = 2$ without oscillating, because the roots are real and negative.

Common mistakes

• Applying the initial conditions to $y_h$ before $y_p$ has been added.
• Writing $C_1e^{rx} + C_2e^{rx}$ for a double root. The extra factor $x$ is missing.
• Mixing up $\alpha$ and $\beta$ for complex roots: the real part gives the damping, the imaginary part gives the angular frequency.
• Forgetting to multiply the right-hand side by the integrating factor.

Concepts in this part

Practise differential equations in the app →