Large datasets and Cuda memory Issue

0
votes

I was processing a large dataset and ran into this error: "RuntimeError: CUDA out of memory. Tried to allocate 1.35 GiB (GPU 0; 8.00 GiB total capacity; 3.45 GiB already allocated; 1.20 GiB free; 4.79 GiB reserved in total by PyTorch).

Any thought on how to solve this?

2
pytorch

2 Answers

3
votes

I met the same problem before. It's not a bug, you just ran out of memory on your GPU.

  1. One way to solve it is to reduce the batch size until your code will run without this error.
  2. if it not works, better to understand your model. A single 8GiB GPU may not handle a large and deep model. You should consider changing a GPU with larger memory and find a lab to help you (Google Colab can help)
  3. if you are just doing evaluate, force a tensor to be run CPU would be fine
  4. Try model compression algorithm
1
votes

If you are using full batch gradient descent (or similar), use mini batch instead with smaller batch size and reflect the same in dataloaders.