Local Ai
Unable to get GPU Passthrough working - Docker
Setup as follows: Proxmox -> Debian -> Docker -> Ollama. Other containers work. Compose file contains gpu device. Does it need nvidia runtime or any other options? If someone could provide an example
Setup as follows: Proxmox -> Debian -> Docker -> Ollama. Other containers work. Compose file contains gpu device. Does it need nvidia runtime or any other options? If someone could provide an example of a working compose file it would be appreciated. The container recognises the GPU, as confirmed with docker exec -it ollama nvidia-smi However, cannot load models, as confirmed with docker exec -it ollama ollama ps, showing 100% CPU. GPU is pretty old, GTX 970, so this could be the issue? Any ideas appreciated. submitted by /u/JamieEC [link] [comments]
Related
- Ollama with Docker OpenWebUi using CUDA/GPU Support container - Can't get it working
- Is it possible to use 2 GPUs with ollama (AMD + Nvidia)?
- Unable to run Ollama on AMD gpu
Source: r/ollama | 2026-07-27