Eigen decomposition is a way of breaking a square matrix into simpler pieces: its eigenvalues and eigenvectors. The core idea is that certain matrices can be expressed as the product of three matrices, revealing the fundamental directions and scaling factors that define how the original matrix transforms space. It’s one of the most important factorizations in linear algebra, with direct applications in data science, physics, and engineering.
The Core Formula
If you have a square n × n matrix A with n linearly independent eigenvectors, you can factor it as A = QΛQ⁻¹. Here, Q is a matrix whose columns are the eigenvectors of A, and Λ (capital lambda) is a diagonal matrix whose diagonal entries are the corresponding eigenvalues. Q⁻¹ is simply the inverse of Q.
This formula comes straight from the definition of eigenvectors. For any eigenvector v of A, the equation Av = λv holds, meaning that multiplying A by v just scales v by the factor λ. When you stack all n of those equations together, you get AQ = QΛ. Multiply both sides on the right by Q⁻¹, and the decomposition falls out: A = QΛQ⁻¹.
What this really says is that A’s behavior can be completely described by the directions it stretches (the eigenvectors) and how much it stretches along each direction (the eigenvalues). The matrix Q translates into eigenvector coordinates, Λ does the simple scaling, and Q⁻¹ translates back.
What Eigenvectors and Eigenvalues Mean Geometrically
Think of a matrix as a machine that transforms vectors. Most vectors get both rotated and stretched when a matrix acts on them. But eigenvectors are special: they only get stretched (or flipped), never rotated off their original line. The eigenvalue tells you the stretch factor. An eigenvalue of 3 means the eigenvector gets tripled in length. An eigenvalue of -1 means it gets flipped to point the opposite direction without changing length.
In two dimensions, you can visualize this as a linear transformation of the plane. Most arrows in the plane get pushed to new orientations, but the eigenvectors stay pinned to their original lines. The eigenvalue decomposition reveals that any such transformation is really just stretching along a few special directions, with everything else being a consequence of those stretches.
When Eigen Decomposition Is Possible
Not every square matrix can be eigen-decomposed. The key requirement is that the matrix must have n linearly independent eigenvectors, where n is the size of the matrix. A matrix meeting this condition is called “diagonalizable.” If eigenvectors are not independent (they collapse into fewer distinct directions than needed), the decomposition breaks down because Q won’t have an inverse.
The formal test involves comparing two quantities for each eigenvalue: its algebraic multiplicity (how many times it appears as a root of the characteristic equation) and its geometric multiplicity (how many independent eigenvectors it actually produces). A matrix is diagonalizable if and only if these two numbers match for every eigenvalue. In practice, most matrices you encounter in applied work are diagonalizable, but it’s worth knowing the edge cases exist.
Symmetric Matrices Are the Ideal Case
Real symmetric matrices (where the matrix equals its own transpose) have especially clean properties. Their eigenvalues are always real numbers, never complex. Their eigenvectors corresponding to different eigenvalues are automatically perpendicular to each other. And every symmetric matrix is guaranteed to have a full set of orthonormal eigenvectors, meaning the decomposition always exists and Q is an orthogonal matrix.
This matters because when Q is orthogonal, Q⁻¹ is just Q transposed, which is computationally cheap. Covariance matrices in statistics are always symmetric, which is one reason eigen decomposition works so reliably in data analysis applications.
How to Calculate It Step by Step
The process starts with finding eigenvalues by solving the characteristic equation: det(A − λI) = 0. You subtract λ from each diagonal entry of A, compute the determinant of the resulting matrix, and set it equal to zero. This gives you a polynomial in λ whose roots are the eigenvalues.
For a concrete example, take the 2 × 2 matrix A with rows [4, 3] and [2, -1]. The characteristic equation becomes (4 − λ)(−1 − λ) − (3)(2) = 0, which simplifies to λ² − 3λ − 10 = 0. Factoring gives (λ + 2)(λ − 5) = 0, so the eigenvalues are λ = 5 and λ = −2.
Next, for each eigenvalue, you plug it back into the equation Av = λv and solve for the eigenvector v. For λ = 5, you solve the system 4v₁ + 3v₂ = 5v₁ and 2v₁ − v₂ = 5v₂, which yields the eigenvector (3, 1). For λ = −2, the same process gives (−1, 2). Place these as columns of Q, put the eigenvalues on the diagonal of Λ, and you have the decomposition.
For anything larger than a 3 × 3 matrix, solving by hand becomes impractical. Software libraries handle this using iterative numerical algorithms that are far more efficient than directly solving the characteristic polynomial.
Principal Component Analysis
One of the most common real-world uses of eigen decomposition is principal component analysis (PCA), a technique for reducing the number of variables in a dataset while keeping as much information as possible. PCA works by eigen-decomposing the covariance matrix of the data.
The covariance matrix captures how all the variables in a dataset relate to each other. Its eigenvectors point in the directions of greatest variance in the data, and the eigenvalues tell you how much variance lies along each direction. The eigenvector with the largest eigenvalue is the “first principal component,” the direction along which the data is most spread out. The second principal component is the direction of greatest remaining variance, perpendicular to the first, and so on.
Because covariance matrices are symmetric, all eigenvalues are real and all eigenvectors are orthogonal, which guarantees that the principal components are uncorrelated with each other. If the first few eigenvalues are much larger than the rest, you can drop the low-variance directions and represent your data with far fewer dimensions while losing very little information. This is how PCA compresses datasets with hundreds of variables down to a manageable number.
Google’s PageRank Algorithm
The original Google search engine ranked web pages using an eigen decomposition concept. The web was modeled as a massive matrix, where each entry represented the probability of following a link from one page to another. The PageRank of every page was determined by finding the eigenvector corresponding to the eigenvalue 1 of this matrix.
That eigenvector, normalized so its entries sum to 1, gave each page a score reflecting its relative importance in the network. Pages linked to by many other important pages ended up with high eigenvector entries. The entire ranking system boiled down to finding a single dominant eigenvector of a very large, very sparse matrix.
Why the Decomposition Is Useful
Beyond specific applications like PCA and PageRank, the decomposition makes certain matrix operations dramatically simpler. Raising a matrix to a power, for instance, normally requires repeated multiplication. But if you have A = QΛQ⁻¹, then A raised to the kth power is just QΛᵏQ⁻¹. Since Λ is diagonal, raising it to a power means raising each diagonal entry individually, which is trivial.
This property is why eigen decomposition shows up in solving systems of differential equations, analyzing the long-term behavior of dynamical systems, and computing matrix exponentials in physics. It transforms problems involving complicated matrix algebra into problems involving simple scalar arithmetic along each eigenvector direction.
The main limitation is that the decomposition only works for square matrices, and only when those matrices have a full set of independent eigenvectors. For non-square matrices or cases where eigen decomposition fails, related factorizations like singular value decomposition (SVD) fill the gap.