PCA study Exam Questions and Answers
with Verified Solutions | Latest Updated 2026
Two Definitions of PCA Definition 1 (traditional): PCs are linear
combinations that sequentially maximize
variance and are uncorrelated.
Definition 2 (alternative): PCs are
defined jointly to maximize total variance
explained (or minimize residual variance)
in regression of original variables on PCs.
The second definition is needed because
after rotation, PCs may no longer be
ordered or uncorrelated, so the first
definition breaks down.
Judge the number of PCs to Retain the number of PCs before the
retain from scree plot "elbow," where eigenvalues shift from
steep decline to flat.
Mechanism of parallel analysis Simulate data with independent variables,
and how to use it to judge the compute eigenvalues, and retain PCs
number of PCs. whose eigenvalues exceed simulated
ones.
Find the total variance, the Total variance = sum of eigenvalues
proportion of variance Proportion explained = eigenvalue / total
explained from eigenvalues. variance
, Why we analyze a correlation Because PCA is not scale-invariant and
matrix in PCA variables with large variance would
dominate.
Given eigenvalues and - Weight matrix: W=URDR−1/2W = U_R
eigenvectors, find weight D_R^{-1/2}W=URDR−1/2
matrix, loading matrix before - Loading matrix: Λ=URDR1/2\Lambda =
and after standardizing the U_R D_R^{1/2}Λ=URDR1/2
PCs. - After standardizing PCs, loadings
become correlations.
Find communalities and - Communality = row sum of squared
different proportions of loadings
variances from an unrotated - Variance explained by PC = column sum
or orthogonally rotated of squared loadings
loading matrix.
Difference between the - Orthogonal: PCs uncorrelated, loadings
orthogonal and oblique remain correlations, variance
rotations. What change and interpretation preserved
what do not change for each - Oblique: PCs correlated, loadings no
type. longer correlations, variance
interpretation changes
Rotation criteria: whether they - Varimax / Quartimax → simplicity
are simplicity or complexity criteria → maximized
criteria; whether they are - Oblimin / CF → complexity criteria →
minimized or maximized; minimized
important features of each of
them. No need to remember
the formulae.
with Verified Solutions | Latest Updated 2026
Two Definitions of PCA Definition 1 (traditional): PCs are linear
combinations that sequentially maximize
variance and are uncorrelated.
Definition 2 (alternative): PCs are
defined jointly to maximize total variance
explained (or minimize residual variance)
in regression of original variables on PCs.
The second definition is needed because
after rotation, PCs may no longer be
ordered or uncorrelated, so the first
definition breaks down.
Judge the number of PCs to Retain the number of PCs before the
retain from scree plot "elbow," where eigenvalues shift from
steep decline to flat.
Mechanism of parallel analysis Simulate data with independent variables,
and how to use it to judge the compute eigenvalues, and retain PCs
number of PCs. whose eigenvalues exceed simulated
ones.
Find the total variance, the Total variance = sum of eigenvalues
proportion of variance Proportion explained = eigenvalue / total
explained from eigenvalues. variance
, Why we analyze a correlation Because PCA is not scale-invariant and
matrix in PCA variables with large variance would
dominate.
Given eigenvalues and - Weight matrix: W=URDR−1/2W = U_R
eigenvectors, find weight D_R^{-1/2}W=URDR−1/2
matrix, loading matrix before - Loading matrix: Λ=URDR1/2\Lambda =
and after standardizing the U_R D_R^{1/2}Λ=URDR1/2
PCs. - After standardizing PCs, loadings
become correlations.
Find communalities and - Communality = row sum of squared
different proportions of loadings
variances from an unrotated - Variance explained by PC = column sum
or orthogonally rotated of squared loadings
loading matrix.
Difference between the - Orthogonal: PCs uncorrelated, loadings
orthogonal and oblique remain correlations, variance
rotations. What change and interpretation preserved
what do not change for each - Oblique: PCs correlated, loadings no
type. longer correlations, variance
interpretation changes
Rotation criteria: whether they - Varimax / Quartimax → simplicity
are simplicity or complexity criteria → maximized
criteria; whether they are - Oblimin / CF → complexity criteria →
minimized or maximized; minimized
important features of each of
them. No need to remember
the formulae.
