Model Releases
I built a desktop benchmark for local Ollama models — TTFT, tok/s, VRAM, with a HuggingFace GGUF catalogue
Built this because I got tired of guessing whether a model would fit in my 6GB VRAM before downloading it, and because ollama.com's library doesn't let you sort by size. What it does: - Benchmarks Oll
Built this because I got tired of guessing whether a model would fit in my 6GB VRAM before downloading it, and because ollama.com's library doesn't let you sort by size. What it does: - Benchmarks Ollama models across 7 categories (coding, chat, reasoning/math, JSON output, long context, VLM, and a manual "did it refuse/comply" classification for edgier prompts) - Measures TTFT, tokens/s, and VRAM peak (NVML) per prompt - Browses/searches GGUF repos on HuggingFace directly, shows real per-quant download size (including vision projectors) before you pull anything, and can hand a chosen variant straight to Ollama - A VRAM forecast before you run a test, using the model's real architecture metadata (not just param count) for the KV cache math Some things I verified rather than assumed, because I got burned by early "looks done" claims from the coding agents I used: - Ollama silently drops non-snake_case option keys — a camelCase bug would look successful while doing nothing. There's a regression test for it now. - Measured the GPU/CPU split trade-off on my own 1060: full offload ~37 tok/s, forcing partial offload via num_gpu ~6.2 tok/s. Same model, same prompt. - The VRAM forecast was checked against real /api/ps readings on four models — off by 1-11% depending on the case, documented in the README instead of just claimed. - The "does this parse as Python/JSON" check never executes anything — it's pure parsing, and it's honest about what it doesn't claim (valid syntax ≠ correct code). Built with heavy use of AI coding agents (mostly DeepSeek, some MiMo) under my own planning/spec, and audited/tested by hand and by Claude along the way — I'm not going to pretend otherwise, but every claim above has a test or a measurement behind it, not just an agent saying "done." Windows only for now (Tauri + Rust + React), MIT licensed, unsigned installer (SmartScreen will complain, source is all there if you don't trust that). GitHub: https://github.com/AnonBOTpl/LLM-Benchmark Happy to answer questions or take issues — this started as a personal itch-scratcher so there's probably stuff a wider audience will hit that I haven't. submitted by /u/SprayPuzzleheaded533 [link] [comments]
Related
- I built a thing that benchmarks Ollama models on your own hardware and ranks them
- VeriLoop E2 is now open on Hugging Face — curious what Ollama/GGUF quantization would make the most sense for this 27B model
- Your model fits in VRAM. Your context doesn't. The KV cache math nobody shows you.
- Benchmark Your Local LLMs in 3 Commands
Source: r/ollama | 2026-09-24