How do you deal with long-context sessions after restarting llama.cpp?

I run local models on a 128GB Strix Halo and restart llama.cpp fairly often while testing builds, backends and model parameters. The annoying part is long-running agent sessions. Hermes/OpenCode sessions can easily reach 50k-100k context, and after every restart the same context has to be prefetched again, which can take several minutes. I know llama-server already has slot save/restore APIs, but personally I don't think every agent or client should need custom integration for this. Ideally the inference server should handle it automatically. Something like: use conversation -> cache KV/prefill state to disk-> restart llama.cpp/reboot -> same conversation returns -> automatically restore the longest valid cached prefix. Old caches could simply be evicted by size/LRU. DS4 seems to implement something close to this: https://github.com/antirez/ds4 I understand doing this generically for every architecture llama.cpp supports may be difficult. But would it make sense to support it first for a few popular models/architectures, for example Qwen3.8 27B? For slower-prefill hardware like Strix Halo, avoiding repeated 50k-100k prefills would make a very noticeable difference in actual daily use. Has anyone experimented with making this transparent on the server side, rather than requiring clients to manage save/restore themselves? English isn't my first language, so I typed this out and then translated it. submitted by /u/Dazzling_Equipment_9 [link] [comments]

Source: r/LocalLLaMA | 2026-08-20