Linear Regression with Python numpy

Question

I am trying to make a simple linear regression function, but continue to run into

numpy.linalg.linalg.LinAlgError: singular matrix error

Existing function (with debugging prints):

def makeLLS(inputData, targetData): print "In makeLLS:" print " Shape inputData:",inputData.shape print " Shape targetData:",targetData.shape term1 = np.dot(inputData.T, inputData) term2 = np.dot(inputData.T, targetData) print " Shape term1:",term1.shape print " Shape term2:",term2.shape #print term1 #print term2 result = np.linalg.solve(term1, term2) return result 

Exit to the console with my test data:

 In makeLLS: Shape trainInput1: (773, 10) Shape trainTargetData: (773, 1) Shape term1: (10, 10) Shape term2: (10, 1) 

Then these are the errors on the linalg.solve line. This is a linear regression function of the textbook, and I cannot understand why it fails.

What is the error of the singular matrix?

Answer 1

As explained in another answer by linalg.solve , a full rank matrix is ​​expected. This is because he is trying to solve a matrix equation, not a linear regression, which should work for all ranks.

There are several linear regression methods. The simplest one I would suggest is the standard least squares method. Just use numpy.linalg.lstsq . Documentation including an example is here .

Answer 2

A singular matrix is ​​a unit for which the determinant is zero. This means that your matrix has rows that are not linearly independent. For example, if one of the lines is not linearly independent of the others, then it can be constructed by a linear combination of other lines. I will use the numpy linalg.solve example for demonstration. Here is a sample document:

 >>> import numpy as np >>> a = np.array([[3,1], [1,2]]) >>> b = np.array([9,8]) >>> x = np.linalg.solve(a, b) >>> x array([ 2., 3.]) 

Now I will modify a to make it singular.

 >>> a = np.array([[2,4], [1,2]]) >>> x = np.linalg.solve(a, b) ... LinAlgError: Singular matrix 

This is a very obvious example, because the first line only doubles the second line, but hopefully you get the point.