Here is the step-by-step calculation of the standard, mean-centered PCA for this dataset.
1. Calculate the Feature Means
First, we find the exact mean of the $x$ and $y$ columns from the 6 data points in the image:
2. Compute the Centered Covariance Matrix ($\Sigma_{centered}$)
Instead of subtracting the mean from every single point manually, we can use the variance shift formula $\sum (x-\mu)^2 = \sum x^2 - n\mu^2$.
Using the uncentered sums calculated previously ($\sum x^2 = 63.25$, $\sum y^2 = 63.25$, $\sum xy = -48.75$):
$\Sigma_{xx}$: $63.25 - 6(2.083)^2 = 63.25 - 26.041 \approx \mathbf{37.208}$
$\Sigma_{yy}$: $63.25 - 6(-0.583)^2 = 63.25 - 2.041 \approx \mathbf{61.208}$
$\Sigma_{xy}$: $-48.75 - 6(2.083)(-0.583) = -48.75 - (-7.291) \approx \mathbf{-41.458}$
This gives us the true, mean-centered scatter matrix:
$$\Sigma_{centered} = \begin{bmatrix} 37.208 & -41.458 \\ -41.458 & 61.208 \end{bmatrix}$$
3. Find the Eigenvalues ($\lambda$)
We solve the characteristic equation $\det(\Sigma_{centered} - \lambda I) = 0$:
$$(37.208 - \lambda)(61.208 - \lambda) - (-41.458)^2 = 0$$
$$\lambda^2 - 98.416\lambda + (2277.427 - 1718.766) = 0$$
$$\lambda^2 - 98.416\lambda + 558.661 = 0$$
Applying the quadratic formula yields the new variances (eigenvalues) for the principal components:
4. Find the Eigenvectors ($v_1, v_2$)
To find the first principal component $v_1$, substitute $\lambda_1 = 92.368$ into $(\Sigma_{centered} - \lambda I)v = 0$:
$$\begin{bmatrix} 37.208 - 92.368 & -41.458 \\ -41.458 & 61.208 - 92.368 \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix}$$
$$-55.16x - 41.458y = 0 \implies y \approx -1.3305x$$
Normalizing the vector $[1, -1.3305]^T$ to unit length (dividing by its magnitude $\sqrt{1^2 + (-1.3305)^2} \approx 1.664$) yields:
$v_1 = \begin{bmatrix} 0.601 \\ -0.799 \end{bmatrix}$
To find the second principal component $v_2$, substitute $\lambda_2 = 6.048$:
$$\begin{bmatrix} 37.208 - 6.048 & -41.458 \\ -41.458 & 61.208 - 6.048 \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix}$$
$$31.16x - 41.458y = 0 \implies y \approx 0.7516x$$
Normalizing the vector $[1, 0.7516]^T$ to unit length (dividing by its magnitude $\approx 1.251$) yields:
$v_2 = \begin{bmatrix} 0.799 \\ 0.601 \end{bmatrix}$
By properly mean-centering the data, the eigenvectors $[0.601, -0.799]^T$ and $[0.799, 0.601]^T$ are slightly rotated from the uncentered eigenvectors of $[\frac{1}{\sqrt{2}}, -\frac{1}{\sqrt{2}}]^T$ ($\approx [0.707, -0.707]^T$). This shift occurs because the true center of mass $(2.083, -0.583)$ is not at the origin, meaning the axis of maximum variance has to tilt slightly to pass directly through the physical center of the cluster rather than forcing its way through $(0,0)$.
