UnknownError: Failed to get convolution algorithm when running LSTM using Tensorflow-gpu

Question

I'm trying to run a RNN using tensorflow-gpu since with CPU it took me around 13 minutes to treat 1 epoch (8000 pictues in train/ 2000 in test). However, I tried several times the installation of tensorflow-gpu on my Windows 10 but could not manage to run it successfully. I have the following configuration:

GPU 1: NVIDIA GeForce GTX 1650 GPU 0: Intel(R) UHG Graphics 630 CPU: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz

I have the following installed in my virtualenv Tensorflow in pip: pip packages

I followed the following tutorials:

Also since this is a popular issue, i tried different solutions suggested in other topics like:

compatibility between tensorflow-gpu / CUDA / Cudnn (i tried with different combinations CUDA 9.0/ ..)
I downgraded my tensoflow-gpu to 1.8.0 but did not worked

Currently, i have tensorflow-gpu 2.00, CUDA 10.0, Cudnn 7.4.1.5, this seems compatible as what is defined (https://www.tensorflow.org/install/source_windows) Visual studio is installed and the latest drivers fir my Intel Card.

But i also found that my GeForce GTX 1650 was not mentioned in the NVIDIA gpu enabled for CUDA (https://developer.nvidia.com/cuda-gpus) don't know whether this could be the issue?

The errormessage i get is: UnknownError: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above. [Op:Conv2D]

from tensorflow.keras import backend
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Conv2D
from tensorflow.keras.layers import MaxPooling2D
from tensorflow.keras.layers import Flatten
from tensorflow.keras.layers import Dense

classifier = Sequential()

classifier.add(Conv2D(32, (3, 3), input_shape = (64, 64, 3), activation = 'relu'))

classifier.add(MaxPooling2D(pool_size = (2, 2)))

classifier.add(Conv2D(32, (3, 3), activation = 'relu'))
classifier.add(MaxPooling2D(pool_size = (2, 2)))

classifier.add(Flatten())

classifier.add(Dense(units = 128, activation = 'relu'))
classifier.add(Dense(units = 1, activation = 'sigmoid'))

classifier.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])

from tensorflow.keras.preprocessing.image import ImageDataGenerator

train_datagen = ImageDataGenerator(rescale = 1./255,
                                   shear_range = 0.2,
                                   zoom_range = 0.2,
                                   horizontal_flip = True)

test_datagen = ImageDataGenerator(rescale = 1./255)

training_set = train_datagen.flow_from_directory('dataset/training_set',
                                                 target_size = (64, 64),
                                                 batch_size = 32,
                                                 class_mode = 'binary')

test_set = test_datagen.flow_from_directory('dataset/test_set',
                                            target_size = (64, 64),
                                            batch_size = 32,
                                            class_mode = 'binary')

classifier.fit_generator(training_set,
                         steps_per_epoch = 8000,
                         epochs = 25,
                         validation_data = test_set,
                         validation_steps = 2000)

Daniel Möller Daniel Möller · Accepted Answer · 2019-12-23T14:10:32

Are your images all RGB? I got this error because of a dimension error between the model and the data.

If your images are grayscale, they will have 1 channel: (64,64,1) and will not be compatible with your model's input shape (64,64,3).
If you images are RGBA, they will have 4 channels, also being incompatible.

Take some time to inspect the results of your generator:

print('training data shapes:')
for i in range(len(training_set)):
    print(training_set[i][0].shape)

print('\ntest data shapes:')
for i in range(len(test_set)):
    print(test_set[i][0].shape)

Take also some time to plot a few of the images from the generators with their classes.

UnknownError: Failed to get convolution algorithm when running LSTM using Tensorflow-gpu

1 Answers