Tensorflow fails to interpret what is stored in numpy ndarray

0
votes

Hello So I have a pandas df with urls I then download/load cache then store that into the df. The problem arises because pandas stores the numpy arrays as ndarrays so their shape is lost. Is there any way to tell tensorflow the shape of the stored arrays?

def NN(self):
    #Trains on validation then commence batch prediction
    data = self.category_validation.agg({'URL':self.process_image,'label':self.le.fit_transform}).dropna()

    print(data['URL'].values[0])
    print(data['URL'].values[0].shape)
    print(data['URL'].values.shape)
    exit(1)


    #One of Keras' best templates
    self.nn = model(...)

    #Compile the model
    self.nn.compile(...)

    #Fit the first instance of the data
    self.nn.fit(data['URL'].values,data['label'].values)

tf.Tensor(..., shape=(299, 299, 3), dtype=float32) (299, 299, 3) (490,)

ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type tensorflow.python.framework.ops.EagerTensor).

    def process_image(self,url):
        #Read image from filepath and reshape it to the appropriate shape for model 
        path = "path/"+self.clean_url(url)

        #Checks if files exists, if not it tries to download if that doesn't work
        if os.path.exists(path):
            image = tf.keras.preprocessing.image.load_img(path,target_size=(299,299))
            image = tf.keras.preprocessing.image.img_to_array(image)
        elif self.get_image(url) == 0:
            return float('nan')
        else:
            image = tf.keras.preprocessing.image.load_img(path,target_size=(299,299))
            image = tf.keras.preprocessing.image.img_to_array(image)
            
        return image/255
2
pythonpandasnumpytensorflow

2 Answers

1
votes

Would you care to change

self.nn.fit(data['URL'].values,data['label'].values)

to

self.nn.fit(data['URL'].to_numpy(),data['label'].to_numpy())

0
votes

Ok, the df.apply/agg function assumes an ambiguous shape for the numpy array. Ergo, I needed to manually use a list and a for loop to iterate over the values and put them into the tmp list to then be converted into a numpy array to only then be able to convert them into a tensor. What a pain.

l = []

for image in df['URL'].values:
    l.append(image)
x_train = np.array(l)
...