Computing the SVD¶

In [1]:
#keep
import numpy as np
import numpy.linalg as la

In [2]:
#keep
np.random.seed(15)
n = 5
A = np.random.randn(n, n)


Now compute the eigenvalues and eigenvectors of $A^TA$ as eigvals and eigvecs using la.eig or la.eigh (symmetric):

In [3]:
eigvals, eigvecs = la.eigh(A.T.dot(A))

In [4]:
#keep
eigvals

Out[4]:
array([ 0.08637178,  0.457892  ,  2.04177547,  2.34383161,  8.37000184])

Eigenvalues are real and positive. Coincidence?

In [5]:
#keep
eigvecs.shape

Out[5]:
(5, 5)

Check that those are in fact eigenvectors and eigenvalues:

In [6]:
B = A.T.dot(A)
B - eigvecs.dot(np.diag(eigvals)).dot(la.inv(eigvecs))

Out[6]:
array([[ -8.88178420e-16,   4.44089210e-16,   4.44089210e-16,
-4.44089210e-16,   6.66133815e-16],
[ -1.77635684e-15,  -3.10862447e-15,  -2.22044605e-15,
-2.22044605e-15,  -1.94289029e-16],
[ -2.22044605e-15,  -1.77635684e-15,  -8.88178420e-16,
-1.11022302e-15,   3.33066907e-16],
[ -1.33226763e-15,  -8.88178420e-16,  -4.44089210e-16,
-4.44089210e-16,   2.22044605e-16],
[ -4.44089210e-16,  -8.04911693e-16,  -6.66133815e-16,
-4.44089210e-16,   1.11022302e-16]])

eigvecs are orthonormal! (Why?)

Check:

In [7]:
eigvecs.T.dot(eigvecs) - np.eye(n)

Out[7]:
array([[  6.66133815e-16,   1.83230524e-16,  -6.80556436e-17,
-1.00482385e-16,  -3.40005801e-16],
[  1.83230524e-16,   1.11022302e-15,  -1.41968548e-16,
2.53432919e-16,   4.67507977e-16],
[ -6.80556436e-17,  -1.41968548e-16,   4.44089210e-16,
-2.25729902e-16,   1.04083409e-17],
[ -1.00482385e-16,   2.53432919e-16,  -2.25729902e-16,
4.44089210e-16,   2.35922393e-16],
[ -3.40005801e-16,   4.67507977e-16,   1.04083409e-17,
2.35922393e-16,  -1.11022302e-16]])

Now piece together the SVD:

In [19]:
Sigma = np.diag(np.sqrt(eigvals))

In [20]:
V = eigvecs

In [38]:
U = A.dot(V).dot(la.inv(Sigma))


Check orthogonality of U:

In [39]:
U.dot(U.T) - np.eye(n)

Out[39]:
array([[ -2.10942375e-15,  -1.08246745e-15,  -8.46545056e-16,
8.32667268e-17,   8.25728375e-16],
[ -1.08246745e-15,   4.44089210e-16,  -1.38777878e-16,
-5.55111512e-17,  -1.38777878e-16],
[ -8.46545056e-16,  -1.38777878e-16,   2.22044605e-16,
9.85322934e-16,  -3.95516953e-16],
[  8.32667268e-17,  -5.55111512e-17,   9.85322934e-16,
2.22044605e-16,   4.99600361e-16],
[  8.25728375e-16,  -1.38777878e-16,  -3.95516953e-16,
4.99600361e-16,  -2.22044605e-16]])