solving for 5 variables using 6 linear equation using numpy

Question

I'm trying to solve following system of equations=

 -14a +   b +            e = 0 
   2a - 14b +        d     = 0 
          b  -14c  +2d     = 0 
                  -15d + e = 0 
          +  2c       -14e = 0 
    a +   b + c  +  d +  e = 1 

I appended required zeroes to matrices formed from above equations . I used numpy.linalg.solve function. I always get this error:: numpy.linalg.linalg.LinAlgError: Singular matrix. I know that I have created a singular matrix by making one row elements zero.

My matrices & code ::

a= np.array([
    [-14, 1, 0, 0, 1, 0],
    [2, -14, 0, 1, 0, 0],
    [0, 1, -14, 2, 0, 0],
    [0, 0, 0, -15, 1, 0],
    [0, 0, 2, 0, -14, 0],
    [1, 1, 1, 1, 1, 0]
])

b=np.array(   [0, 0, 0, 0, 0, 1]      )

x = np.linalg.solve(a, b)

Is there another way to solve this ?

Using np.linalg.lstsq returns ::

(array([ 0.00674535,  0.00713199,  0.00709352,  0.00582019,  0.006766  ,  0.
]), array([], dtype=float64), 5, array([ 15.88397122,  15.68586038,  14.59368088,  13.14182044,
    12.12312981,   0.        ]))

How am I supposed to get my solutions from above array ??.. None of the no. in above array is the solution..

Answer 1

You are correct in your calling sequence, though I would pull out the last column of A:

A = np.array([
        [-14, 1, 0, 0, 1],
        [2, -14, 0, 1, 0],
        [0, 1, -14, 2, 0],
        [0, 0, 0, -15, 1],
        [0, 0, 2, 0, -14],
        [1, 1, 1, 1, 1  ]])
b = np.array([0, 0, 0, 0, 0, 1])

sol = np.linalg.lstsq(A, b)

As everyone else has mentioned, your system is overdetermined. This means that any fit is likely to be a bad one. Indeed, np.linalg.lstsq returns the residuals:

residuals : {(), (1,), (K,)} ndarray Sums of residuals; squared Euclidean 2-norm for each column in b - a*x. If the rank of a is < N or > M, this is an empty array. If b is 1-dimensional, this is a (1,) shape array. Otherwise the shape is (K,).

Which in this case is:

print sol[1]
>>> array([0.96644295])

This indicates that the fit is very poor (and there is no approximate linear solution here). We can see that by again checking:

print (b - np.dot(A, sol[0])).sum()
>>> 1.36912751678

Which would be zero in the NxN case.

Answer 2

The second argument to np.linalg.solve should be a 1-d array, not a row vector:

>>> np.linalg.solve(a, b.ravel())
Traceback (most recent call last):
  File "<ipython-input-13-81809fe2e837>", line 1, in <module>
    np.linalg.solve(a, b.ravel())
  File "/usr/lib/python2.7/dist-packages/numpy/linalg/linalg.py", line 328, in solve
    raise LinAlgError('Singular matrix')
LinAlgError: Singular matrix

(It still doesn't work, but that's because a is singular, np.linalg.det(a) == 0.0. Better try np.linalg.lstsq.)

Answer 3

Oh my god !!!!. The problem is with the last equation. If you add up the first 5 equation's

-14a +   b  +                e  = 0 
  2a - 14b  +          d        = 0 
     +   b  -  14c  +  2d       = 0 
                    - 15d +  e  = 0 
            +  2c         - 14e = 0

You get ..

 a + b + c + d + e = 0

So, the last equation in the given system is incorrect. i.e.

 a + b + c + d + e = 1

The initial situation error:: numpy.linalg.linalg.LinAlgError: Singular matrix is due to the fact that you need the system matrix a as in code:

 x = np.linalg.solve(a, b)

should be either a square and a non-singular matrix.

 >> help(np.linalg.solve)
 solve(a, b)
Solve a linear matrix equation, or system of linear scalar equations.

Computes the "exact" solution, `x`, of the well-determined, i.e., full
rank, linear matrix equation `ax = b`.

Parameters
----------
a : (..., M, M) array_like
    Coefficient matrix.
b : {(..., M,), (..., M, K)}, array_like
    Ordinate or "dependent variable" values.

Returns
-------
x : {(..., M,), (..., M, K)} ndarray
    Solution to the system a x = b.  Returned shape is identical to `b`.

Raises
------
LinAlgError
    If `a` is singular or not square.

Including the 6 equations for solving 5-variables results in a non-square system matrix (5x6).

But all-in together the system of equation's is itself incorrect.

The trivial solution to the problem is then:

[0.0, 0.0, 0.0, 0.0, 0.0]