Eigenvectors and Eigenvalues: The Unchanging Directions of a Transformation

WHY IT EXISTS Linear transformations like rotations and shears can be complex. To understand them, we need to find their 'skeleton'—the fundamental axes that remain stable. Eigenvectors and eigenvalues provide this simplified view, distilling a complex matrix operation into its most essential directional components and their scaling factors.

THE MENTAL MODEL Imagine a transformation being applied to every vector in a space. Most vectors get knocked off their original line. The eigenvectors are the special ones that stay on their line, only getting longer or shorter. The eigenvalue tells you exactly how much longer (eigenvalue > 1), shorter (0 < eigenvalue < 1), or if it flips direction (eigenvalue < 0). An eigenvector with an eigenvalue of 1 is unchanged by the transformation.

HOW IT WORKS Mathematically, for a square matrix A (the transformation), a non-zero vector v is an eigenvector if applying A to v results in a scaled version of v. This is written as Av = λv. Here, v is the eigenvector and λ (lambda) is its corresponding scalar eigenvalue. To find them, you solve the characteristic equation det(A - λI) = 0 for the eigenvalues λ. Then, for each eigenvalue, you solve the system (A - λI)v = 0 to find the corresponding eigenvectors v.

WHEN TO USE IT Eigen-analysis is critical in many fields. First, in data science for Principal Component Analysis (PCA), where eigenvectors of the covariance matrix become the principal components that capture the most variance. Second, in search algorithms like Google's PageRank, which finds the principal eigenvector of the web's link matrix to rank pages. Third, in physics for analyzing vibrations and quantum states.

WHEN NOT TO USE IT Eigen-analysis is defined for square matrices, as they represent transformations from a vector space to itself; it's not directly applicable to non-square matrices. Also, not all transformations have real eigenvectors. A 90-degree rotation in a 2D plane changes the direction of every single vector, so it has no real eigenvectors, only complex ones.

ONE CANONICAL EXAMPLE Consider a scaling matrix A = [[2, 0], [0, 3]]. This transformation doubles the x-component and triples the y-component of any vector. The vector v1 = [1, 0] is an eigenvector because A*v1 = [2, 0], which is exactly 2 * v1. Its eigenvalue is 2. The vector v2 = [0, 1] is also an eigenvector because A*v2 = [0, 3], which is 3 * v2. Its eigenvalue is 3. These vectors define the axes of the transformation.