Benchmarking qwen3.8:27b by having it port a 34k line C game to one HTML file. The agent mattered more than I expected.

My hobby project skill-issue is an arena FPS in a single C file, about 34k lines, everything procedural. My test for local setups is to have an agent port it to a single file HTML version with three.js. The C original is the reference, so it is easy to see how much came through. Ran the same task through three agents on the same model, plus a cloud run for scale: agent model HTML LOC time my rating claude-code opus5 1474 21 min good pi-agent qwen3.8:27b 1439 100 min low codehamr qwen3.8:27b 1120 90 min low opencode qwen3.8:27b 1126 102 min very low One run each, they take about an evening, so treat this as rough. LOC is lines of code in the HTML file that came out, so it is volume only. The rating is just me comparing the HTML port to the C original by eye. The failure mode was the same in all three. It writes plausible code and then wrecks parts of the file it was never pointed at, which looks a lot like a model seeing less than you think you gave it. Which is why the spread between the three local rows is the useful bit here. Same model, same machine, same prompt, and the lean agents produced the port I could actually open in a browser. Tool schemas and prompt boilerplate get spent before the model reads a line of my code, and on a 27B running locally that is most of the window. On this hardware the harness turned out to be a bigger lever than the weights. C reference: https://github.com/codehamr/skill-issue The agent I am building: https://github.com/codehamr/codehamr submitted by /u/codehamr [link] [comments]

Source: r/ollama | 2026-08-22

Related