cannot perform reduce with flexible type

Question

I have this dataset:

           Game1    Game2   Game3   Game4     Game5

Player1       2        6        5       2        2

Player2       6        4        1       8        4

Player3       8        3        2       1        5

Player4       4        9        4       7        9

I want to calcultate the sum of the 5 games for every player.

This is my code :

import csv
f=open('Games','rb')
f=csv.reader(f,delimiter=';')
lst=list(f)
lst
import numpy as np
myarray = np.asarray(lst)
x=myarray[1,1:] #First player
y=np.sum(x)

I had the error "cannot perform reduce with flexible type". Im really very new to python and I need your help.

Thank you

You should show the actual csv file, that one that contains the ; delimiter. Otherwise you leave us guessing as to how the dataset was written. x is probably an array of strings, since nothing in your code converts strings to numbers. — hpaulj

NaN NaN · Accepted Answer · 2017-04-17T00:23:57

You can still use a structured array as long as you familiarize yourself with the dtypes. Since your data set is extremely small, the following may serve as an example of using numpy in conjunction with list comprehensions when your dtype is uniform but named

dt = [('Game1', '<i4'), ('Game2', '<i4'), ('Game3', '<i4'),
      ('Game4', '<i4'), ('Game5', '<i4')]
a = np.array([(2, 6, 5, 2, 2),
              (6, 4, 1, 8, 4),
              (8, 3, 2, 1, 5),
              (4, 9, 4, 7, 9)], dtype= dt)

nms = a.dtype.names
by_col = [(i, a[i].sum()) for i in nms if a[i].dtype.kind in ('i', 'f')]
by_col
[('Game1', 20), ('Game2', 22), ('Game3', 12), ('Game4', 18), ('Game5', 20)]

by_row = [("player {}".format(i), sum(a[i])) for i in range(a.shape[0])]
by_row
[('player 0', 17), ('player 1', 23), ('player 2', 19), ('player 3', 33)]

In this example, it would be a real pain to get each sum individually for each column name. That is where the ... a[i] for i in nms bit is useful since the list of names was retrieved by nms = a.dtype.names. Since you are doing a 'sum' then you want to restrict the summation to only integer and float types, hence the a[i].dtype.kind portion.

Summing by row is just as easy but you will notice that I didn't use this syntax but a slightly different one to avoid the error message

a[0].sum()  # massive failure
....snip out huge error stuff...
TypeError: cannot perform reduce with flexible type
# whereas, this works....
sum(a[0])   # use list/tuple summation

Perhaps 'flexible' data types don't live up to their name. So you can still work with structured and recarrays if that is the way that your data comes in. You can become adept at simply reformatting your data by slicing and altering dtypes to suit your purpose. For example, since your data type are all the same and you don't have a monstrous dataset, then you can use many methods to convert to a simple structured array.

b = np.array([list(a[i]) for i in range(a.shape[0])])
b
array([[2, 6, 5, 2, 2],
       [6, 4, 1, 8, 4],
       [8, 3, 2, 1, 5],
       [4, 9, 4, 7, 9]])

b.sum(axis=0)
array([20, 22, 12, 18, 20])

b.sum(axis=1)
array([17, 23, 19, 33])

So you have many options when dealing with structured arrays and depending on whether you need to work in pure python, numpy, pandas or a hybrid, then you should familiarize yourself with all the options.

ADDENDUM

As a shortcut, I failed to mention taking 'views' of arrays that are structured in nature, but have the same dtype. In the above case, a simple way to produce the requirements for simple array calculations by row or column are as follows... a copy of the array was made, but not necessary

b = a.view(np.int32).reshape(len(a), -1)
b
array([[2, 6, 5, 2, 2],
       [6, 4, 1, 8, 4],
       [8, 3, 2, 1, 5],
       [4, 9, 4, 7, 9]])
b.dtype
dtype('int32')

b.sum(axis=0)
array([20, 22, 12, 18, 20])

b.sum(axis=1)
array([17, 23, 19, 33])

cannot perform reduce with flexible type

4 Answers