fantastic: latest llama.cpp server webui can now run commands for tools into rootless sandboxed containers

while having much fun testing LLMs Houdini-like attitudes and abilities to evade, excalate and escape from carefully reciprocally arranged security enhancing sandboxing VMs, containers, namespaces and jails, I've discovered a new revolutionary option to let llama-server (build 10423) to launch guest containers (running for ex. bare bones linux distros like Alpine) as sandboxes for securely (rootless) running shell commands for its tools (enabled with the "--tools" option) directly from the llama-server webui. The new (experimental) option is called "--tools-runtime" and the initial argument to it that I suggest you to try is "podman:alpine" (or "docker:alpine"). The server will also download the indicated container image and then instantiate it. Of course podman (preferred by myself) or docker have to be installed beforehand on the host. Enjoy! P.S. almost forgot! The --tools-runtime option can also run commands, invoked by the server native tools, on a remote host, by accepting as argument the additional target "ssh:user@host" where remote access to that account has been previously authorized submitted by /u/DevelopmentBorn3978 [link] [comments]

Source: r/LocalLLaMA | 2026-08-14