SVD: Eigendecomposition for Any Matrix

WHY IT EXISTS: Eigendecomposition is limited. It only works for a square normal matrix with an orthonormal eigenbasis. Many matrices are not square or normal, so eigendecomposition does not apply. SVD exists to provide a factorization that works for any real or complex matrix. It gives the same geometric clarity, breaking a transformation into simpler parts, without requiring the matrix to be square or normal.

THE MENTAL MODEL: Imagine SVD as a three-stage machine. Every matrix, no matter its shape or contents, performs a linear transformation that can be viewed as rotation, then scaling, then rotation. First, the input space rotates. Then the axes scale. Then the result rotates again into the output position. This rotation-scaling-rotation story is the complete geometric essence of the matrix.

HOW IT WORKS: SVD factors any matrix into three components. The first component is a rotation. The second component is a scaling. The third component is another rotation. This sequence generalizes eigendecomposition, which is restricted to square normal matrices with orthonormal eigenbases. Because SVD applies to any matrix, it extends the reach of decomposition to cases where eigendecomposition is undefined. The decomposition is also related to the polar decomposition, which provides another way to split a matrix into simpler geometric operations.

WHEN TO USE IT: Use SVD when you need to analyze or decompose a matrix that is not square or not normal. Since it handles any real or complex matrix, it is the tool to reach for when eigendecomposition is unavailable. It exposes the scaling behavior and rotational structure hidden inside the linear map.

WHEN NOT TO USE IT: If your matrix is square and normal with an orthonormal eigenbasis, eigendecomposition may be sufficient. SVD is more general, but do not use it when the simpler decomposition already solves your problem. Do not treat SVD as identical to eigendecomposition; they are related, but they produce different factorizations.

ONE CANONICAL EXAMPLE: Consider a matrix that is not square, mapping from one vector space to another. Eigendecomposition cannot apply because it is restricted to square normal matrices. SVD handles this by factoring the transformation into a rotation in the domain, a scaling step, and a rotation in the codomain. This shows how SVD generalizes spectral decomposition to any matrix, including those that eigendecomposition cannot factor at all.