pytorch code sudden fails on colab with NVIDIA driver on your system is too old

Question

I had some code which worked on colab (gpu runtime) just a short while ago. Suddenly I am getting

The NVIDIA driver on your system is too old (found version 10010).

nvcc shows Cuda compilation tools, release 10.1, V10.1.243

I tried torch versions 1.5.1, then 1.13.0. Both keep getting this error.

There is a discussion showing other people having doubts. with no clear resolution. https://github.com/pytorch/pytorch/issues/27738

Anyone having the same problem?

Answer 1

The light-the-torch package is designed to solve exactly this type of issue. Try this:

!pip install light-the-torch
!ltt install torch torchvision

Answer 2

I think this might be to do with the fact that Google Colab randomly connects you to a GPU when you start a runtime. Some might have different drivers installed, which could result in that error to display only part of the time, as you've experienced.

You can see the current version of CUDA by running !nvidia-smi in Colab. You can then simply install a version of PyTorch that is compatible with this version of CUDA. The PyTorch website can generate a pip command for your language/environment/CUDA version, and there is also a list of previous versions and their corresponding commands if you have a CUDA version that the current version doesn't support.

This is what I got working with a CUDA version of 10.1:

!pip install torch==1.7.1+cu101 torchvision==0.8.2+cu101 -f https://download.pytorch.org/whl/torch_stable.html

Answer 3

Late reply, but maybe it will help others to ease their problems.

Pytorch has a previous versions page that has both PyTorch and Torchvision installation commands recommended for older versions. For your case, I used !pip install torch==1.6.0+cu101 torchvision==0.7.0+cu101 -f https://download.pytorch.org/whl/torch_stable.html and worked just fine in Google Colab, taking into consideration the NVidia driver version as well.

Answer 4

I was also getting the same error. This one fixed the issue for me:

pip install torch==1.4.0+cu100 torchvision==0.5.0+cu100 -f https://download.pytorch.org/whl/torch_stable.html

Answer 5

I just had the same issue on google colab. Following this github issue I downgraded to torch==1.4.0 using !pip.

Since I had several other requirements, I am reading them from a local file like this:

with open('attribute_hallucination/editing_tool/requirements.txt', 'w') as f:
  f.write("cupy-cuda101==7.4.0\ncycler==0.10.0\nfastrlock==0.4\nfuture==0.18.2\nimageio==2.8.0\njoblib==0.14.1\nkiwisolver==1.2.0\nmatplotlib==3.2.1\nnumpy==1.18.4\nopencv-python==4.2.0.34\nPillow==7.1.2\npynvrtc==9.2\npyparsing==2.4.7\npython-dateutil==2.8.1\nscikit-learn==0.22.2.post1\nscipy==1.2.0\nsix==1.14.0\nsklearn==0.0\ntorch==1.4.0\ntorchvision==0.6.0\ntqdm==4.46.0")
!pip install -r attribute_hallucination/editing_tool/requirements.txt