Q : What has Matrix Decomposition got to do with Regression Model ?

A : For getting the parameters, it is always better to do a QR decomposition of X and then get the whole thing working.

http://www.stat.wisc.edu/~larget/math496/qr.html