Training works but prediction produces constant values (cnn with pytorch)

0
votes

I have a model trying to predict the class of image: cat or dog. I receive 95% accuracy in training. However when I try to predict a single image, I am stuck with almost constant output every time I run the model. There are some non-constant values, but they mostly look like catastrophic failure.

I read similar topics from forums but that hasn't helped, as it appears there is no particular solution for this problem...

I have tried the following:

  • Changing epochs 5 to 15,20,30...
  • Changing lr = 0.001 to 0.01, 0.0001...
  • I implemented with both dropout regularization model and batch normalization model...
  • I changed testing pictures...
  • Changing last activation layer from softmax to torch.sigmoid...
  • Reducing batch size from 100 to 30, 75...
  • Trying with a batch, which results with normal acc, loss and predictions.
  • My dataset is scaled which is mentioned in forums as solution.
  • My optim is Adam which is mentioned in forums as solution.
  • Loading dataset with torch.data.DataLoader...
  • Sampling randomly...
  • I saved and load the model, in case there are problems with that. However, I already checked that state_dict's are different...
  • I re-prepared data which resulted the constant value to change otherwise (dog to cat), somehow? Idk if that's a coincidence though.

Infos:

Dataset : https://download.microsoft.com/download/3/E/1/3E1C3F21-ECDB-4869-8368-6DEBA77B919F/kagglecatsanddogs_3367a.zip

Here is all my code with predictions in Jupyter Notebook, feel free to investigate. I am really tired of this solution. Any help is highly appreciated!

https://github.com/yusuftengriverdi/neural_networks/blob/master/CNN_Last.ipynb

Similar topics around the web:

https://discuss.pytorch.org/t/rnn-predicting-a-constant-output/40397/5

https://discuss.pytorch.org/t/cnn-does-not-predict-properly-does-not-converge-as-expected/43567

https://discuss.pytorch.org/t/making-a-prediction-with-a-trained-model/2193

https://datascience.stackexchange.com/questions/46779/predict-gives-the-same-output-value-for-every-image-keras

https://github.com/keras-team/keras/issues/6447

PyTorch model prediction fail for single item

Having trouble with CNN prediction

1
pythontensorflowdeep-learningpytorchconv-neural-network

1 Answers

1
votes

If something works in training but fails during prediction, the most likely cause is you're not preprocessing the data the same way.

I had a look at the notebook (huge amount of code, in future please condense this to just the relevant parts here). At a glance - this is your prediction code which doesn't work as expected:

img = cv2.resize(img, (IMG_SIZE, IMG_SIZE))
plt.imshow(img, cmap='gray')
x = torch.Tensor([i for i in img]).view(-1, 50, 50)
y= torch.Tensor([0,1]).to(device)
test_x = x.view(-1, 1, 50, 50)
test_x = test_x.to(device)
net.eval()
#with torch.no_grad():
yhat.append(net(test_x))

But during training you're using a dataloader

testloader = DataLoader(v_dataset, batch_size = BATCH_SIZE, sampler= test_sampler)
...

test_dt = next(iter(testloader))
X, y = test_dt[0].view(-1, 1, 50, 50), test_dt[1]
val_acc, val_loss = fwd_pass(X.view(-1, 1, 50, 50).to(device), y.to(device))

which works (since your test/validation accuracy goes up to a good level).

Figure out what the dataloader code path does which the other code path doesn't do, and you'll have the solution. Eg, load the same image in both ways and compare - same dimensions? data average / standard deviation the same? etc

For a shortcut - just use a dataloader to make predictions as well. P.S. Yes, it is okay to create a dataloader for just one image.