Matrices as Linear Transformations

Every $m\times n$ matrix $A$ defines a function $T:\mathbb{R}^n\to\mathbb{R}^m$ by

$T(\mathbf{x}) = A\mathbf{x}.$

This is a linear map: $T(\mathbf{u}+\mathbf{v})=T(\mathbf{u})+T(\mathbf{v})$ and $T(c\mathbf{u})=cT(\mathbf{u})$. Linearity is not an assumption — it falls straight out of matrix multiplication.

Geometrically, $A$ takes every vector in $\mathbb{R}^n$ and lands it somewhere in $\mathbb{R}^m$. The entire input space gets transformed at once:

• A $2\times 2$ matrix rotates, reflects, stretches, or shears the plane.
• A $3\times 2$ matrix embeds a plane into 3-D space.
• A $1\times 3$ matrix collapses 3-D space onto a line.

The columns of $A$ are the key: $A\mathbf{x} = x_1\mathbf{a}_1 + x_2\mathbf{a}_2 + \cdots + x_n\mathbf{a}_n$ — just a linear combination of the columns of $A$, weighted by the entries of $\mathbf{x}$.

If $A$ transforms $\mathbb{R}^n\to\mathbb{R}^m$ and $B$ transforms $\mathbb{R}^p\to\mathbb{R}^n$, then applying $B$ first then $A$ gives a single transformation $\mathbb{R}^p\to\mathbb{R}^m$ encoded by the product $AB$:

$(AB)\mathbf{x} = A(B\mathbf{x}).$

This is why matrix multiplication is defined the way it is — it isn't an arbitrary rule, it's composition of functions.

Concrete example. Let $R$ be a 90° counter-clockwise rotation: $R = \begin{bmatrix}0&-1\\1&0\end{bmatrix}$ and $S$ a uniform scale by $2$: $S=2I$. Then $RS$ first scales then rotates, but in 2-D it happens that $RS = SR$ — scaling and rotation commute. In general $AB\ne BA$: composition of linear maps is not commutative. Order matters.

Two subspaces defined by $A:\mathbb{R}^n\to\mathbb{R}^m$ reveal everything about its behavior.

Column space $\mathcal{C}(A)$ — the set of all outputs: $\mathcal{C}(A) = \{A\mathbf{x} : \mathbf{x}\in\mathbb{R}^n\} = \operatorname{span}(\mathbf{a}_1,\dots,\mathbf{a}_n)\subseteq\mathbb{R}^m.$ $A\mathbf{x}=\mathbf{b}$ is solvable iff $\mathbf{b}\in\mathcal{C}(A)$.

Null space $\mathcal{N}(A)$ — the set of inputs that map to zero: $\mathcal{N}(A) = \{\mathbf{x}\in\mathbb{R}^n : A\mathbf{x}=\mathbf{0}\}\subseteq\mathbb{R}^n.$ $\mathcal{N}(A)$ measures redundancy in $A$'s columns: a nonzero null vector means two different inputs produce the same output (the map is not injective). If $\hat{\mathbf{x}}$ is any particular solution to $A\mathbf{x}=\mathbf{b}$, then the full solution set is the affine subspace $\hat{\mathbf{x}}+\mathcal{N}(A)$.

Gilbert Strang's four fundamental subspaces of an $m\times n$ matrix $A$:

The key orthogonality relations: $\mathcal{C}(A^\top)\perp\mathcal{N}(A)$ (they partition $\mathbb{R}^n$) and $\mathcal{C}(A)\perp\mathcal{N}(A^\top)$ (they partition $\mathbb{R}^m$). Every input vector splits into a row-space component (which $A$ maps faithfully) and a null-space component (which $A$ kills). These four spaces are the complete anatomy of what any matrix does.

The rank of $A$ is the dimension of its column space:

$\operatorname{rank}(A) = \dim\mathcal{C}(A) = \dim\mathcal{C}(A^\top).$

(Row rank equals column rank — always, for any matrix. That's non-obvious but true.)

The rank-nullity theorem is linear algebra's conservation law:

$\operatorname{rank}(A) + \operatorname{nullity}(A) = n$

where $\operatorname{nullity}(A) = \dim\mathcal{N}(A)$ and $n$ is the number of columns. Inputs are split between the row space (dimension = rank, faithfully mapped) and the null space (dimension = nullity, sent to zero). The rank tells you how much of $\mathbb{R}^m$ you can reach; the nullity tells you how much of $\mathbb{R}^n$ you lose.

A square $n\times n$ matrix is full rank ($\operatorname{rank}=n$) iff its null space is $\{\mathbf{0}\}$ — the only way to invert a map is to lose nothing.

For a square $n\times n$ matrix $A$, the determinant $\det(A)$ is the signed factor by which $A$ scales $n$-dimensional volumes. Feed in a unit cube; out comes a parallelepiped with volume $|\det(A)|$.

For $2\times 2$: $\det\begin{bmatrix}a&b\\c&d\end{bmatrix} = ad-bc$ — the signed area of the parallelogram formed by the columns.

Key facts:

• $\det(A)=0$ iff $A$ collapses space to a lower dimension (rank-deficient, singular).
• $\det(AB)=\det(A)\det(B)$ — determinants compose under multiplication.
• $\det(A^{-1})=1/\det(A)$ — inverting a map inverts the scaling factor.
• Sign encodes orientation: $\det>0$ preserves handedness; $\det<0$ flips it (like a mirror reflection).

Do not use the determinant to check invertibility for floating-point matrices — a determinant of $10^{-15}$ isn't zero, but it indicates a nearly-singular matrix that behaves poorly numerically. Use np.linalg.matrix_rank or condition numbers instead.

How an $m\times n$ matrix $A$ partitions its domain $\mathbb{R}^n$ and codomain $\mathbb{R}^m$:

A square $n\times n$ matrix $A$ is invertible (nonsingular) iff any of the following equivalent conditions hold:

• $\det(A)\ne 0$
• $\operatorname{rank}(A)=n$ (full rank)
• $\mathcal{N}(A)=\{\mathbf{0}\}$ (trivial null space)
• The columns of $A$ form a basis of $\mathbb{R}^n$
• $A\mathbf{x}=\mathbf{b}$ has a unique solution for every $\mathbf{b}$

All five conditions are the same condition, five disguises. The inverse satisfies $AA^{-1}=A^{-1}A=I$ and $(AB)^{-1}=B^{-1}A^{-1}$ — inversion reverses composition.

For rectangular $m\times n$ matrices, exact inverses don't exist, but left inverses ($n>m$) and right inverses ($m>n$) do under full-rank conditions. The pseudoinverse (Lesson 4) handles the general case.

Put it all together in code:

import numpy as np

A = np.array([[1, 2, 3],
              [4, 5, 6],
              [7, 8, 9]], dtype=float)

print("rank =", np.linalg.matrix_rank(A))   # 2 — not full rank!
print("det  =", np.linalg.det(A))            # ~0 (singular)

# Find the null space vector (Ax = 0)
_, _, Vt = np.linalg.svd(A)
null_vec = Vt[-1]                            # last right singular vector
print("A @ null_vec =", np.round(A @ null_vec, 10))  # [0, 0, 0]

# Invertible matrix
B = np.array([[1, 0, 2], [0, 1, 0], [3, 0, 1]], dtype=float)
print("rank(B) =", np.linalg.matrix_rank(B))  # 3
print("B_inv @ B =", np.round(np.linalg.inv(B) @ B, 10))  # Identity

The SVD approach to the null space is not an accident — it's the numerically correct way. The last right singular vector of $A$ corresponds to its smallest singular value, pointing maximally "into" the null space.

Any matrix $A$ performs exactly four things to space, visible in its four fundamental subspaces:

1. Maps faithfully the row-space component of each input — stretching along singular directions (singular values, revealed in Lesson 4).
2. Kills the null-space component — irreversibly loses information.
3. Reaches exactly the column space in the output — not all of $\mathbb{R}^m$ if rank $< m$.
4. Misses the left null space entirely — no input ever lands there.

Full rank ($m=n=\operatorname{rank}$) means steps 2 and 4 are empty: nothing is lost, everything is reached, the map is invertible. Drop rank by one and you collapse an entire direction to zero — the determinant records this as a zero.

Next lesson: what if $A$ has special vectors that only get scaled (not rotated) by the transformation? Those are eigenvectors, and they unlock the deepest structure inside a matrix.