SVDlow-rank approximationEckart–Young

The few directions that matter

A grayscale image is just a matrix of numbers. The singular value decomposition rewrites that matrix as a stack of simple rank-1 layers, sorted from most important to least. Keep only the top few and you get the mathematically best possible approximation at that size. Drag k and watch the picture rebuild itself, one direction at a time.

Reconstruction · rank k

Original · full rank

1 of ? layers kept

–

compression

–

reconstruction error

–

energy retained

The singular-value spectrum

σ₁ … σₙ (log scale)

kept (top k) discarded tail bar height = importance of that layer

Each layer is an outer product σ_i u_i v_i^T — a single vertical pattern times a single horizontal pattern, scaled by its singular value σ_i. The rank-k image is just their sum:

Aₖ = σ₁u₁v₁ᵀ + σ₂u₂v₂ᵀ + … + σₖuₖvₖᵀ = Uₖ Σₖ Vₖᵀ

Eckart–Young (1936): among all matrices of rank k, this truncation minimizes the error. And the leftover error is exactly the energy you threw away — the sum of the squared discarded singular values, ‖A − Aₖ‖² = σ²_k+1 + … + σ²_r. Smooth gradients and big shapes live in the first handful of layers; sharp edges, the checkerboard, and the noise patch need the long tail.

Self-contained: the SVD was computed offline (one-sided Jacobi) on this 160×160 synthetic image and the top singular triplets baked in. The browser only re-sums UₖΣₖVₖ^T. Verified: full-rank reconstruction matches the original to ~1e-11, error decreases monotonically in k, and the Frobenius error squared equals the discarded singular energy to ~1e-14.