numpy array specify a default

Question

I have a python list of lists to convert to a numpy array. I have defined the dtype for the numpy array. Some array values might be None or ‘’. An error is issued for those, if the numpy array respective dtype value is float or int. Is there a way to say numpy to assign 1 (or field specified value) for a particular dtype if the value is None or ‘’.

e.g: following code gives me the error

ValueError: could not convert string to float.

    import re
    import numpy as np
    dt = np.dtype([ ('a', np.int32,),
          ('b', np.float32),
          ('c', np.int32,),
          ('d', np.float32),
          ('e', np.int32,),
      ]) 
     npar = np.array(('667000', '0', '0', '', ''), dt)

The expected output for npar is: (assigned 0.0 for d, 1 for e as default values)

    (667000, 0.0, 0, 0.0, 1)

I have large multidimensional arrays to be converted. So performance is some thing important to be considered.

Let a link to a sample of your data. Pandas seems to be the best way to solve your problem. — B. M.
sample data is an array of few thousands of rows as shown above. and the other statements are same. I want to convert the 2d array at once. — py_works

Pruthvi Raj Pruthvi Raj · Accepted Answer · 2015-06-23T18:05:13

This might work:

One liner:

s = ('667000', '0', '0', '', '')
npar = np.array(tuple([0 if dt.names[x]== 'd' else 1 if dt.names[x]=='e' else s[x] for x in range(0,len(s))]),dt)

Or:

import numpy as np
dt = np.dtype([ ('a', np.int32,),
          ('b', np.float32),
          ('c', np.int32,),
          ('d', np.float32),
          ('e', np.int32,),
])
s = ('667000', '0', '0', '', '')
t = np.array(s)
if not t[4]:
    t[4] = 1
t[t==''] = 0
npar = np.array(tuple(t),dt)

numpy array specify a default

2 Answers