I built an open-source LLM Gateway to route and fallback between local Ollama models and cloud APIs

Hey r/ollama 👋 If you run Ollama locally alongside cloud endpoints for agent workflows, Cursor/Windsurf, or custom scripts, managing API switching, failover logic, and context limits can get messy fast. I’ve been building Atom (an open-source multi-agent platform), and I built the LLM Gateway module specifically to serve as a unified local proxy layer designed around local model execution. Core Features for Local Workflows: Strict Local Isolation (ATOM_LOCAL_ONLY=true): Toggle a single flag to block all outgoing cloud API traffic and route 100% of your requests exclusively to your local Ollama instances or Llama.cpp servers. Automated Cloud Failovers: Define explicit fallback rules. Route primary calls to your local qwen2.5-coder or llama3.1 Ollama models, and fall back to DeepSeek or Anthropic only if local context limits or VRAM constraints fail. Unified OpenAI-Compatible Interface: Normalizes request/response formats and stream chunks across Ollama endpoints, Cloud LLMs (OpenAI, Anthropic, DeepSeek, Gemini), and third-party local hosts. BYOK & Local Key Encryption: Centralized local key management with zero external telemetry—your configuration and API keys stay encrypted on your own machine. I documented the proxy pipeline, model translation layer, and failover logic in the technical architecture docs for anyone interested in how it's structured under the hood. (Note: Repo and architecture doc links are in the comments to keep this post clean for filters!) Curious to hear how others here are proxying their local Ollama endpoints or setting up local-to-cloud fallback pipelines in your setups! submitted by /u/rush86999 [link] [comments]

Source: r/ollama | 2026-08-02