How to whiten the matrix in PCA

Question

I am working with Python and I have implemented PCA using this tutorial .

Everything works fine, I got the covariance, I made a successful transformation, brought it to the original dimensions, not problems.

But how can I bleach? I tried to divide the eigenvectors into eigenvalues:

S, V = numpy.linalg.eig(cov) V = V / S[:, numpy.newaxis] 

and used V to transform the data, but this led to strange data values. Can someone please clarify this?

Answer 1

Here's a multi-valued implementation of some Matlab code for whitening matrices that I got from here .

 import numpy as np def whiten(X,fudge=1E-18): # the matrix X should be observations-by-components # get the covariance matrix Xcov = np.dot(XT,X) # eigenvalue decomposition of the covariance matrix d, V = np.linalg.eigh(Xcov) # a fudge factor can be used so that eigenvectors associated with # small eigenvalues do not get overamplified. D = np.diag(1. / np.sqrt(d+fudge)) # whitening matrix W = np.dot(np.dot(V, D), VT) # multiply by the whitening matrix X_white = np.dot(X, W) return X_white, W 

You can also whiten the matrix using SVD:

 def svd_whiten(X): U, s, Vt = np.linalg.svd(X, full_matrices=False) # U and Vt are the singular matrices, and s contains the singular values. # Since the rows of both U and Vt are orthonormal vectors, then U * Vt # will be white X_white = np.dot(U, Vt) return X_white 

The second method is a bit slower, but probably more numerically stable.

Answer 2

If you use the python scikit-learn library for this, you can just set the built-in parameter

 from sklearn.decomposition import PCA pca = PCA(whiten=True) pca.fit(X) 

check the documentation .

Answer 3

I think you need to transpose V and take the square root of S. So the formula

matrix_to_multiply_with_data = transpose (v) * s ^ (- 1/2)