$latex \textsc{Left Inverse - Full column rank} $
If you have a matrix A that has full column rank, then $latex {A^T\; A}&fg=000000$ also is a full rank matrix.

$latex \displaystyle A^{-1}_{\text{left} } = (A^T \; A)^{-1} A^T &fg=000000$

is called the left inverse because multiplying by $latex {A}&fg=000000$ give $latex {I}&fg=000000$. This is crucial for least squares. In the case of full column rank, there is a left inverse for A. left inverse is not right inverse if the matrix is not full rank matrix. A rectangular matrix cannot have two sided inverse.

left inverse is not right inverse if the matrix is not full rank matrix. A rectangular matrix cannot have two sided inverse.

$latex \textsc{Right Inverse - Full row rank} $
If you have a matrix A that has full row rank, then $latex {A \; A^T}&fg=000000$ also is a full rank matrix. There are infinitely many solutions to $latex {Ax = b }&fg=000000$

$latex \displaystyle A^{-1}_{\text{right} } = A\; ( A \; A^T )^{-1} &fg=000000$

is called the right inverse

Basic recap - There are four spaces. Row space, Column Space, Null Space, Left Null Space. Given any matrix $latex {m \times n }&fg=000000$ with a rank $latex {r}&fg=000000$

  • Dimension of column space is $latex {r}&fg=000000$
  • Dimension of null space is $latex {m-r}&fg=000000$
  • Dimension of row space is $latex {r}&fg=000000$
  • Dimension of null space is $latex {n-r}&fg=000000$

Statisticians love generalized inverses as they are always worried that they might have repeated experiments, repeated features in which case the matrix is not full row rank or full column rank. Hence the presence of generalized inverse is the best that they can hope for. The generalized inverse operates between row space and column space only ignoring the null space and left null space . The pseudo inverse is denoted by $latex {A^+}&fg=000000$