Learnings for the day

There are three ways to compute pseudo-inverse of a matrix $A$
- If $A$ has independent columns, then $A^{+} = (A^{T} A)^{- 1} A^{T}$ and so $A^{+} A = I$
- If $A$ has independent rows, then $A^{+} = A^{T} (A^{T} A)^{- 1}$ and so $A A^{+} = I$
- A diagonal matrix $Σ$ is inverted wherever possible - otherwise $Σ^{+}$ has zeros
Typically one comes across the solution to $A x = b$ presented as $x = A^{+} b =$ . However using psuedo inverse is not the right way to solve computationally. Why ?
- Every matrix has a pseudo-inverse. The pseudo-inverse contains $1 / σ_{k}$ for all non zero elements of $Σ$ . However it is meant to contain 0’s for all zeros present in $Σ$ . To know when a number is exactly 0 is an extremely rigid requirement
Generalized SVD - Two matrices are factorized simultaneously
- $A$ and $B$ can be factored in to $A = U_{A} Σ_{A} Z$ and $B = U_{B} Σ_{B} Z$
- $U_{A}$ and $U_{B}$ are orthogonal matrices
- $Σ_{A}$ and $Σ_{B}$ are positive diagonal matrices
- $Z$ is an invertible matrix