Fix torch.cuda.is_available() Returns False

You have an NVIDIA GPU, you installed PyTorch, and yet:

>>> import torch
>>> torch.cuda.is_available()
False

There are half a dozen reasons this happens, but they are not equally likely. The overwhelming majority of the time, the GPU is fine and the driver is fine — you just installed the wrong PyTorch. One command tells you whether that is your problem, and it takes two seconds.

This guide starts with that command, then works through the rest in the order you should actually check them. It also clears up the single most common piece of bad advice online: that you need to install the CUDA Toolkit. You almost certainly do not.

Quick Answer

Run print(torch.version.cuda). If it prints None, you installed a CPU-only wheel — reinstall PyTorch from the correct CUDA index. If it prints a version (e.g. 12.4), the wheel is fine and the problem is the driver (check nvidia-smi), an environment variable (CUDA_VISIBLE_DEVICES), or container GPU passthrough. PyTorch bundles its own CUDA runtime, so you never need the system CUDA toolkit — only a recent enough driver.

TL;DR

• First test: torch.version.cuda. None → CPU-only wheel (the #1 cause).
• Fix that: uninstall, then reinstall from the CUDA index (e.g. cu124).
• Myth: you do not need the CUDA Toolkit. PyTorch bundles the runtime.
• If the wheel is fine: nvidia-smi (driver), then CUDA_VISIBLE_DEVICES.
• Docker: needs --gpus all + NVIDIA Container Toolkit.
• WSL2: needs the Windows driver, not a Linux one.
• Still False with everything set: the driver is too old for the wheel's CUDA.

The Two-Second Diagnosis

Before anything else, ask PyTorch what it was built with:

import torch
print(torch.__version__)      # e.g. 2.4.0+cpu  or  2.4.0+cu124
print(torch.version.cuda)     # None  → CPU build ;  '12.4' → CUDA build

• +cpu in the version, or torch.version.cuda == None → CPU-only wheel. No amount of driver fixing will help; you must reinstall PyTorch.
• +cu124 and a CUDA version string → the build is fine; skip to the driver and environment checks.

This one distinction routes you to the right fix and saves hours of chasing drivers that were never the problem.

The Diagnostic Ladder

Work top to bottom — each rung rules out a whole category:

1. CPU-only wheel? torch.version.cuda is None → reinstall the CUDA build.
2. Driver working? nvidia-smi fails → install/repair the driver, reboot.
3. GPU hidden? CUDA_VISIBLE_DEVICES empty or -1 → unset it.
4. Container/WSL? Missing --gpus all or a Linux driver in WSL → fix passthrough.
5. Everything passes but still False? The driver is too old for the wheel's CUDA runtime → upgrade the driver.

The Mental Model That Prevents Most Mistakes

Here is the idea that most articles get wrong. When you install a GPU PyTorch wheel, it already contains the CUDA runtime and cuDNN it needs. Those libraries ship inside the wheel. The only thing PyTorch needs from your system is a compatible NVIDIA driver.

The system CUDA Toolkit (the thing that installs nvcc) is a separate product for compiling CUDA code. PyTorch does not use it at runtime. Installing it will not fix is_available(), and having the "wrong" toolkit version installed does not break PyTorch either — the wheel ignores it.

Step-by-step Solution

Fix 1: Reinstall the correct CUDA wheel (the #1 fix)

If torch.version.cuda is None, replace the CPU wheel. Uninstall first so pip doesn't keep the cached CPU build:

pip uninstall -y torch torchvision torchaudio
 
# Install the CUDA build — pick the index that matches a CUDA your driver supports
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124

• --index-url .../cu124 → pulls the CUDA 12.4 wheels (with the runtime bundled).
• Use the official PyTorch selector to get the exact command for your OS, package manager, and CUDA version.
• The bundled runtime means you do not separately install CUDA 12.4 on the system.

Verify immediately:

import torch
print(torch.version.cuda)          # '12.4'
print(torch.cuda.is_available())   # True
print(torch.cuda.get_device_name(0))

Fix 2: Repair the NVIDIA driver

If the wheel is a CUDA build but is_available() is still False, check the driver:

nvidia-smi

After a fresh driver install, reboot — the kernel module must load cleanly.

Fix 3: Unhide the GPU

An environment variable can hide every GPU from PyTorch even when the driver and wheel are perfect:

echo $CUDA_VISIBLE_DEVICES        # empty is fine; "" or "-1" hides all GPUs

import os
# If it was set to "" or "-1" somewhere (a script, a shell profile, a scheduler):
os.environ.pop("CUDA_VISIBLE_DEVICES", None)   # or set it before importing torch

Set it before import torch, or unset it entirely. A value of "" or -1 means "no visible devices," which is a legitimate but easily-forgotten way to get False.

Fix 4: Confirm you're in the right environment

The classic trap: you installed the CUDA wheel in one environment and are running Python in another. Make them match:

which python                       # is this the venv/conda env you installed into?
python -c "import torch, sys; print(sys.executable, torch.__file__, torch.version.cuda)"

If sys.executable points at a different interpreter than the one you pip installed into, activate the correct environment and reinstall there.

Platform-Specific Notes

Docker

A CUDA wheel and a healthy host driver are not enough — the container needs GPU access:

# On the host: install the NVIDIA Container Toolkit, then:
docker run --gpus all --rm nvidia/cuda:12.4.0-base-ubuntu22.04 nvidia-smi

• --gpus all → exposes the host GPUs to the container. Without it, the container sees none.
• Confirm nvidia-smi works inside the container before blaming PyTorch.
• In Kubernetes, this is the nvidia.com/gpu resource request plus the device plugin.

WSL2 (Windows)

The counter-intuitive rule: install the driver on Windows, not in Linux.

• Install the current NVIDIA Windows driver (it includes WSL CUDA support).
• Do not install a Linux display driver inside the WSL distribution — it breaks the passthrough.
• Then nvidia-smi works inside WSL2 and PyTorch sees the GPU.

Cloud instances

If nvidia-smi reports no device at all, verify you actually launched a GPU instance type (e.g. an A10/A100/T4 shape). A CPU instance will never expose a GPU no matter how PyTorch is installed. On some images the driver is present but needs a reboot after first boot.

Verification Steps

1. torch.version.cuda prints a version, not None.
2. torch.cuda.is_available() returns True.
3. torch.cuda.get_device_name(0) prints your GPU.
4. A tiny tensor actually lands on the GPU:

import torch
x = torch.rand(3).cuda()
print(x.device)          # cuda:0

5. nvidia-smi shows your Python process using GPU memory while that tensor lives.

Prevention

• Install PyTorch from the CUDA index URL, never a bare pip install torch.
• Pin the wheel with its build tag (+cu124) in requirements so CI can't drift to CPU.
• Keep the NVIDIA driver current; don't install the CUDA Toolkit to "fix" GPU issues.
• Use one virtual environment per project and verify sys.executable.
• In Docker, standardise on --gpus all and test nvidia-smi in the image build.
• Add a startup assertion: assert torch.cuda.is_available() in GPU jobs so failures are loud.

Troubleshooting Matrix

Related Guides

• Resolve CUDA error: device-side assert triggered
• Fix bitsandbytes CUDA Setup failed
• Fix "Token indices sequence length is longer than the specified maximum"
• Resolve OSError: Can't load tokenizer
• Setting up a reproducible GPU training environment

External References

• PyTorch — Get Started (install selector)
• PyTorch — CUDA semantics
• NVIDIA — CUDA on WSL user guide
• NVIDIA Container Toolkit docs

FAQs

Is torch.cuda.is_available() slow the first time? The first call initialises the CUDA context and can take a moment. That is normal and unrelated to it returning False — a False result is immediate.

Does the CUDA version in the wheel need to match my driver exactly? No. It needs a driver that is new enough for that CUDA runtime, thanks to NVIDIA's forward-compatibility. A newer driver runs older CUDA wheels fine; a driver that is too old for the wheel's CUDA is what fails.

Can I have both CPU and GPU PyTorch installed? Not usefully in the same environment — the last install wins. Use separate environments if you need both, and check torch.version.cuda in each.

Why does pip install torch give me the CPU build? On some platforms the default PyPI wheel is CPU-only. You have to pass the CUDA --index-url to get the GPU build. This is the single most common way people end up here.

is_available() was True yesterday and False today — what changed? Usually a driver update pending a reboot, a changed CUDA_VISIBLE_DEVICES in a new shell, or an environment/requirements.txt change that pulled the CPU wheel. Check torch.version.cuda and nvidia-smi first.