Model Releases
Built a VRAM scheduler (vramble) to automate model swapping on my local AI workstation
I set up a dedicated workstation for local AI experimentation with various models and tools. But every single one of them wants all the available VRAM. To fix this, I built a request dispatching layer
I set up a dedicated workstation for local AI experimentation with various models and tools. But every single one of them wants all the available VRAM. To fix this, I built a request dispatching layer called vramble. It automatically loads and unloads models and manages the queue using a priority system. Now I can run different workflows without manually managing models (or wasting Claude tokens to orchestrate the process). Any contribution is welcome, it works for me but is a new build so can't guarantee that will work directly on any machine Side note: Qwen 27B at 200+ t/s is performing way better than I expected. opeMIT license, available on GitHub: github.com/ValerioDolci/vramble submitted by /u/FeydRowan [link] [comments]
Related
- Qwen3.8-Flash-Next 177B NVFP4(119GiB): SSD streaming at 9-10 tok/s on one 16 GB RTX 5060 Ti + 32 GB RAM
- Memoria 1.0.0 — a local, model-agnostic memory system for LLMs
- You can offload most of Qwen3.8-Flash-Next's KV cache to RAM with little decode slowdown
- 16 GB VRAM purgatory discussion thread
Source: r/LocalLLaMA | 2026-10-13