svd_intro.md currently derives principal components three ways — as an application of the SVD (Application: Principal Components Analysis (PCA)), through the explicit Relationship of PCA to SVD, and again in PCA with Eigenvalues and Eigenvectors. What none of them gives is the variational characterisation: that the principal components are the successive maximisers of the Rayleigh quotient, which is what the Courant–Fischer min-max theorem states.
That framing answers a question the current derivations leave open — not just what the principal components are, but why they are optimal, and in what precise sense the k-th component is the best remaining direction.
This was requested in QuantEcon/meta#28, Tom Sargent's brief on eigenvalues and eigenfunctions, which asked to "use principal components analysis as an application of the min-max theorem". That issue has been closed against the material that did ship; this is one of the residuals it did not cover, spun out so it is not lost.
Suggested shape: a section after PCA with Eigenvalues and Eigenvectors stating the Rayleigh quotient, giving the Courant–Fischer characterisation for a symmetric matrix, and connecting it back to the components already computed earlier in the lecture — ideally reusing the same worked example so the reader sees the two routes agree.
The original brief also links the Wikipedia min-max theorem article as a starting reference.
svd_intro.mdcurrently derives principal components three ways — as an application of the SVD (Application: Principal Components Analysis (PCA)), through the explicitRelationship of PCA to SVD, and again inPCA with Eigenvalues and Eigenvectors. What none of them gives is the variational characterisation: that the principal components are the successive maximisers of the Rayleigh quotient, which is what the Courant–Fischer min-max theorem states.That framing answers a question the current derivations leave open — not just what the principal components are, but why they are optimal, and in what precise sense the k-th component is the best remaining direction.
This was requested in QuantEcon/meta#28, Tom Sargent's brief on eigenvalues and eigenfunctions, which asked to "use principal components analysis as an application of the min-max theorem". That issue has been closed against the material that did ship; this is one of the residuals it did not cover, spun out so it is not lost.
Suggested shape: a section after
PCA with Eigenvalues and Eigenvectorsstating the Rayleigh quotient, giving the Courant–Fischer characterisation for a symmetric matrix, and connecting it back to the components already computed earlier in the lecture — ideally reusing the same worked example so the reader sees the two routes agree.The original brief also links the Wikipedia min-max theorem article as a starting reference.