Fix CUDA and cuDNN Library Load Errors in TensorFlow: A Diagnostic Guide

TensorFlow's GPU errors all look alike — some shared object could not be loaded — but they come from a small number of distinct causes. This guide gives you the order to check them in, then links to a detailed walkthrough for each specific error.

First: is it actually an error?

Not every red line is fatal. TensorFlow logs missing optional libraries at warning level and carries on:

Could not load dynamic library 'libnvinfer_plugin.so.6'

TensorRT is an optional inference optimiser. If your model still trains and tf.config.list_physical_devices('GPU') lists your GPU, nothing is broken. Confirm GPU availability before you change anything:

import tensorflow as tf
print(tf.config.list_physical_devices('GPU'))

If that returns a non-empty list, your GPU stack works and the message is noise. See the full explanation of the libnvinfer warning.

Second: is the library present, and can the loader see it?

This one command separates the two most commonly confused causes:

ldconfig -p | grep libcudart

• Prints a path — the library exists. Your problem is a version mismatch or an environment that overrides the loader path.
• Prints nothing — search the filesystem before assuming it is missing:

find / -name 'libcudart.so*' 2>/dev/null

If the file turns up but ldconfig did not list it, the directory simply is not on the loader path. Add it:

export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH

Append that line to ~/.bashrc to make it persistent. This is the fix for the classic libcublas.so.9.0 ImportError.

Third: does the version match your TensorFlow build?

The number in the filename is an ABI version and must match exactly. libcudart.so.11.0 is not satisfied by CUDA 12 — installing a newer CUDA does not fix it, and often causes it.

Check what you have against what TensorFlow wants:

nvcc --version          # CUDA toolkit version
nvidia-smi              # driver version and GPU memory
python -c "import tensorflow as tf; print(tf.__version__)"

Then either install the CUDA version your TensorFlow build expects, or install a TensorFlow build compiled against your CUDA. Walkthrough: fixing the libcudart.so.11.0 error.

Fourth: is conda isolating or shadowing your CUDA?

Conda installs CUDA inside the environment prefix, not system-wide. Two failure modes follow: TensorFlow cannot find the environment's libraries, or a system CUDA install shadows them.

Keep everything in one environment and do not mix with a system install:

conda install -c conda-forge cudatoolkit cudnn

Details: TensorFlow not detecting CUDA installed through conda.

Runtime failures: CUDA_ERROR_LAUNCH_FAILED and CUDNN_STATUS_INTERNAL_ERROR

These appear after the libraries load, so they are not path problems. The usual cause is GPU memory exhaustion — TensorFlow grabs all GPU memory by default, leaving cuDNN unable to allocate its workspace. Enable memory growth first:

import tensorflow as tf
 
for gpu in tf.config.list_physical_devices('GPU'):
    tf.config.experimental.set_memory_growth(gpu, True)

If the failure persists, verify your cuDNN version matches your TensorFlow build. Full guide: troubleshooting CUDA_ERROR_LAUNCH_FAILED and CUDNN_STATUS_INTERNAL_ERROR.

The diagnostic checklist

Work top to bottom and stop at the first hit:

1. Does tf.config.list_physical_devices('GPU') list your GPU? If yes, treat load messages as warnings.
2. Does ldconfig -p | grep <library> print a path? If no, find it; if found, add its directory to LD_LIBRARY_PATH.
3. Does the filename's version match what nvcc --version reports? If not, align TensorFlow and CUDA versions.
4. Are you in a conda environment? Install cudatoolkit and cudnn inside it and avoid mixing with system CUDA.
5. Do the libraries load but training crash? Enable GPU memory growth, then check cuDNN versions.

Most reported "broken CUDA installs" are step 2 or step 3 — a path or a version, not a missing package.