Model Releases
LLM Inference Dashboard
Working on a resource dashboard, rich logs, lightweight 64mb cap, all local, scales on network API endpoints via collector, supports multiple engines (llama, strata, custom cuda engines, unsloth, LMS)
Working on a resource dashboard, rich logs, lightweight 64mb cap, all local, scales on network API endpoints via collector, supports multiple engines (llama, strata, custom cuda engines, unsloth, LMS) Not public yet, but curious if anyone would be interested? It’s better logging and metrics then the default endpoint api provides. If you’re like me, you don’t just use 1 engine submitted by /u/Distinct-Pie2389 [link] [comments]
Related
- How well do multiple GPUs scale for LLM inference? (Trying to understand the basics)
- Qwen3.8 flash next ISTA-DASLab GGUF 50t/s TG and 1500t/s PP with 12GB VRAM and 64GB RAM Laptop on 'Strata' engine
- The Rise of Overfit Inference Engines
Source: r/LocalLLaMA | 2026-10-04