Model Releases
Agens Volundr 32B v1: we fixed what you flagged in the Preview. 0/50 loops, session continuation, and an MIT engine that runs it with 262K context (measured to 239K) within a 16 GB budget
Hi r/LocalLLaMA. I'm on the team at Blockway (disclosure: this is our model). Two weeks ago we posted the Volundr 32B Preview here, and you were blunt about what was wrong with it. This is v1, and mos
Hi r/LocalLLaMA. I'm on the team at Blockway (disclosure: this is our model). Two weeks ago we posted the Volundr 32B Preview here, and you were blunt about what was wrong with it. This is v1, and most of the work went into what you flagged. WHAT v1 FIXES FROM THE PREVIEW "It loops." Repetition loops on SWE-bench tasks: 36/50 → 0/50. "It thinks forever." Reasoning steps used to get cut at 32K. Now the median step is 120 tokens and the longest is 12K. "No prefix caching, it re-reads the whole history every turn." SGLang build: v1 adds session continuation, so follow-up turns reuse the model's state instead of re-prefilling. Follow-up turns are 2.9× faster on our agent replay (1.6 s median with the drafter), with greedy-equivalent outputs (not byte-identical: a cached prefix is prefilled in different pieces). GGUF: uses llama-server's stock --ctx-checkpoints prompt cache, with no custom server code. Our fork only adds the model architecture. Agent results: SWE-bench Verified (50-task subset) 22 → 27/50, τ²-bench 89 → 92/120, median SWE task time 124 → 82 min. "MAKE IT RUN ON CONSUMER GPUS" → AGENS ENGINE (MIT) We're open-sourcing Agens Engine under the MIT licence. It's an inference engine built for this hybrid architecture: 54 of 72 layers keep a fixed-size state instead of a KV cache, and the engine is designed around that. Same GPU, same v1 INT4 weights and drafter, single stream: Agens Engine decodes 1.4–1.6× faster than our own SGLang build (up to 1.8× on code with thinking). Chat 58 → 86, code 79 → 113, agent at 8K context 91 → 121, JSON 125 → 199 tok/s. This is one run per side and compares against our SGLang build only, not stock SGLang, vLLM or llama.cpp, and it is single-stream only. New build: Agens-Volundr-32B-v1-Engine-16GB. It's about 2.85 bits/weight (mixed 2/3/4-bit), gives 262K context (we measured up to 239K) within a 16 GB memory budget, and runs only on Agens Engine. A 12 GB build is on the roadmap. Honest caveats: Speed within the 16 GB budget, single conversation, v1 drafter on: short context: 90 (chat) / 118 (code) / 107 (thinking) / 112 (tool) / 139 (agent, 8K) tok/s after a 219K-token document: 62 / 86 / 77 / 69 / 75 tok/s; 151 s to first token for the 219K prompt a conversation ran to 239K tokens inside the budget weights download is 25 GB These were measured on a 48 GB card under the engine's hard 16 GiB cap (peak 15.5 GiB), not on a real 16 GB card. On a GPU that also drives a display, use a shorter context. Real cards, especially laptop GPUs, may be slower. Post your numbers and we'll add them. At this bit-width, quality drops. GPQA is 73.7 vs 80.3 at bf16 (best of 4 runs each). Coding holds up: HumanEval is 73.2% (1 run) vs 73.8% for INT4. Top-1 agreement with bf16 is 90.4%, KL 0.117. This build is for running locally, not for leaderboard scores. Needs Linux, an NVIDIA driver with CUDA 12.9, the NVIDIA container toolkit, and 32 GB+ host RAM for 262K.docker run --gpus device=0 --ipc=host -p 30000:30000 -v /path/to/Agens-Volundr-32B-v1-Engine-16GB:/model:ro -v /path/to/Agens-Volundr-32B-v1-DFlash2:/drafter:ro ghcr.io/blockwayz/agens-engine:1.0.0 --profile 16gb --model /model --drafter /drafter STRENGTHS (same harness and settings as our comparison model) Competitive coding: LiveCodeBench v6 67.9 (best of 2 runs), +8.4 over Qwen3.8-27B Competition maths: AIME 2025 80.0 (best of 2 runs), +6.7 over Qwen3.8-27B Long context stays cheap: only 18 of 72 layers keep a KV cache. INT4 (24.6 GB) runs on one 48 GB GPU with a single 131K-token input; 262K configured window (validated to 107K). Decode speed stays flat from 1K to 128K. DFlash2 drafter, ships today: JSON / structured output up to 3.0× (125 tok/s INT4 on one 48 GB GPU; 76 tok/s bf16 on a pair) Code 1.9–2.2× Thinking 1.3–1.5× FULL TABLE (Preview → v1; dips included. Where we ran a benchmark more than once, we report the best of N runs. Every run is listed in the model card footnote.) Benchmark Preview v1 SWE-bench Verified (50) 22/50 27/50 τ²-bench 89/120 92/120 LiveCodeBench v6 63.4 67.9 AIME 2025 76.7 80.0 MMLU-Pro 79.9 80.0 MATH-500 98.2 96.8 HumanEval 81.7 78.7 GPQA Diamond 84.3 80.3 CMMLU 86.5 85.9 IFEval 87.6 87.2 IFBench 66.3 64.3 NIAH (to 107K) 100% 100% A few scores went down (GPQA, HumanEval, IFBench). The HumanEval and IFBench dips are within one standard error of the runs; GPQA's isn't. That's the trade-off from tuning for agent behaviour and shorter reasoning. Harness settings are in the footnote under the results table: huggingface.co/Blockway/Agens-Volundr-32B-v1#results-same-harness-for-every-row Thanks to mmssix on Hugging Face for the independent benchmark write-up in our HF discussions. A lot of the above started there. KNOWN LIMITATIONS 32B, not a frontier model, and we don't claim it is. SGLang and llama.cpp still need our builds. Next we're working to upstream the architecture into llama.cpp mainline, and we'll post here when the first PR is open. Stock vLLM can't load it. No Metal/Mac in the llama.cpp path yet. Agens Engine 1.0 is text only. LINKS Collection (all five repos): https://huggingface.co/collections/Blockway/agens-volundr-32b-v1-6ad4817aa6e143550bec1122 Weights: huggingface.co/Blockway/Agens-Volundr-32B-v1 (-INT4, -GGUF, -DFlash2, -Engine-16GB) Agens Engine (MIT): github.com/BlockWayz/agens-engine · ghcr.io/blockwayz/agens-engine:1.0.0 SGLang: ghcr.io/blockwayz/agens-sglang:v1-sm89 (48 GB-class GPUs) / v1-sm90 (141 GB-class GPUs) llama.cpp fork: github.com/BlockWayz/llama.cpp (branch volundr) We'll be in the comments for the next couple of days. "Why doesn't X work yet" questions are welcome. https://preview.redd.it/4d8or6olv6wh1.png?width=2360&format=png&auto=webp&s=a83cbce77656ec8d9f28d9a3f9a5bfb8b96ce989 https://preview.redd.it/koyrx6olv6wh1.png?width=2360&format=png&auto=webp&s=6817372b70f3d4c7230fed90593c0a7015cbcc18 submitted by /u/ComfortableKindly507 [link] [comments]
Related
Source: r/LocalLLaMA | 2026-10-18