Model Releases
I built a local server that speaks the Ollama API - sessions survive crashes, and your chat doesn't wait behind your agents
Disclosure first: I'm one of the maintainers, this is my project. If you've pointed Open WebUI or a script at a local server, you know the failure modes: the server restarts and every conversation is
Disclosure first: I'm one of the maintainers, this is my project. If you've pointed Open WebUI or a script at a local server, you know the failure modes: the server restarts and every conversation is gone; one long agent prefill and your chat sits in the queue; and if you ever bind past loopback, you're one port-forward away from being someone's free inference box. superfluid is a daemon we built for exactly that machine: it fronts llama.cpp and MLX with one server that speaks OpenAI, Anthropic and Ollama-compatible APIs, so existing clients keep working. What's different underneath: - every committed token is in a write-ahead log, kill the server mid-reply, restart it, and sessions resume; - QoS classes with preemption: your request runs in the interactive class and jumps the agents' queue (chat behind 8 agents: first token in ~1.5 s vs minutes on the stock servers, same Mac, same model); - binds to 127.0.0.1 by default, refuses cross-origin browser requests and DNS-rebinding Host headers, and asks for a key before anything wider. It's not a replacement for Ollama — it uses llama.cpp's own builds and installs them on demand. Apple silicon + Linux. Repo + benchmarks: link. submitted by /u/nightowlsleeping [link] [comments]
Related
- I built an open-source RAG chatbot starter that runs fully locally with Ollama (FastAPI + ChromaDB)
- Released a Windows Agent Server for Reins/Ollama with Self-Healing Execution Loop (Standalone .EXE included)
- I built an open-source LLM Gateway to route and fallback between local Ollama models and cloud APIs
Source: r/ollama | 2026-10-08