Python error "cannot perform reduce with flexible type" when finding maximum

1
votes

I'm trying to find the maximal value of a list of lists of numbers.

Using Python 2.7 IDLE, I tried this:

import numpy
vectors = [[1, 2, 3], [4,5,6]]
numpyFiles = numpy.array(vectors)
maxRawFreq = numpyFiles.max()

It works, and maxRawFreq = 6

I tried using a very similar code, with a much bigger list, and using Pydev (Eclipse), but I get the following error:

cannot perform reduce with flexible type

What does it mean? (other SO questions about this error gave too specific solutions...).

My code:

import numpy
with open(inputFile) as f:
  vectors = f.readlines()

vectorLength=len(vectors[0])#number of columns (word vector length)

numpyFiles = numpy.array(vectors)

#both these line gave me the same error:
#maxRawFreq = numpyFiles.max()
maxRawFreq = numpy.max(numpyFiles)

My inputFile contains numbers, like so:

-1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
-1, 0, 0, 3, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 4, 0, 0,
+1, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 6,
2
pythonnumpymax
You are reading in strings, not numbers. You need to cast your input accordingly. - Burhan Khalid
Why are you not using numpy.loadtxt or numpy.genfromtxt to load the data from the file? file.readlines simply returns a list of lines from the file. - Ashwini Chaudhary

2 Answers

3
votes

The problem is here:

with open(inputFile) as f:
  vectors = f.readlines()

if your file looks like this:

a, b, c, d
1, 1, 3, 4
2, 3, 5, 6
...

your vectors will look like this: ['a, b, c, d\n', '1, 1, 3, 4\n', '2, 3, 5, 6\n', ...] and then you need to cast these strings to numeric values.

try to read csv (or what is your input file is) in proper way

2
votes
In [81]: data=numpy.genfromtxt(inputFile, delimiter=',')[:, :-1]

In [82]: data
Out[82]: 
array([[-1.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  1.,
         0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],
       [-1.,  0.,  0.,  3.,  0.,  0.,  1.,  0.,  0.,  0.,  0.,  0.,  1.,
         0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  4.,  0.,  0.],
       [ 1.,  0.,  2.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,
         0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  6.]])

If you want to parse it yourself anyway:

In [89]: with open(inputFile) as f:
    ...:     d=[map(float, l.strip(',\n').split(',')) for l in f]
    ...:     print d
    ...:     
[[-1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [-1.0, 0.0, 0.0, 3.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 4.0, 0.0, 0.0], [1.0, 0.0, 2.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 6.0]]