I’m testing whether a local 9B/14B coding agent can recover from its own failed code changes — work in progress

I’m building a personal local coding-agent project called Cody, using local 9B/14B models through Ollama. This is still a work in progress. It has not passed its full graduation process yet, and the latest development run stopped because the available development credits ran out. My goal is not simply to generate code. I’m experimenting with whether a relatively small local model can participate in a development loop that includes: codebase exploration multi-file modifications planning and implementation automated testing failure diagnosis automatic rollback re-planning after failure consulting a second AI called Chappy, while checking its advice against measured results development memory / reuse of previous work recovery / Last Known Good mechanisms resource monitoring long-running development tasks The eventual graduation goal is for Cody + Chappy to complete real development tasks without relying on Codex as the normal development path. During the current development and testing process, Cody has encountered real failures rather than only passing prepared examples. For example, one local 14B attempt generated code with a syntax/indentation error. The candidate failed compilation, the isolated candidate was rolled back, and Cody was given diagnostic information such as the exception type, line number, and relevant source context. Cody then generated another fix itself and passed the corresponding compilation and individual tests. In another case, a candidate got through existing tests but an added UI interaction test found a real regression: changing a voice selection did not correctly update its description. I did not count that candidate as successful. It was kept isolated and the failure was fed back into the development loop. I also found that simply giving the model more retries was not a good solution. A lot of the work has instead been about improving the infrastructure around the model: making sure the correct failure information reaches it, preventing repeated history-reading loops, preserving failed attempts as useful diagnostic context, validating planning dependencies, and allowing safe read-only inspection of local library/API documentation when the model needs information it does not know. The graduation suite currently has 1,222 mandatory tests fixed in its test inventory. That number does not mean I’m claiming all 1,222 have already passed the final graduation run. Separate tests have also exercised recovery from intentional crashes/hangs, UI behavior, rollback behavior, advice verification, and isolated candidate execution. When new regressions have been found, I’ve been treating them as failures rather than weakening the tests or increasing retry limits just to get a pass. One thing I’m particularly interested in is this loop: fail → diagnose → rollback → rethink → consult if needed → generate a different fix → test again The project is trying to make that loop useful without requiring a human to manually guide every individual step. Again, this is not a finished autonomous system, and I’m not claiming AGI or fully autonomous self-improvement. The current run actually stopped before the final graduation result because the development credits available to the external development environment ran out. The isolated candidate and test/recovery state were being preserved rather than promoting an unverified candidate to the main application. Once the infrastructure is finished and the graduation process is actually passed, I’d like to disable the normal Codex path and give Cody completely unseen development tasks. If I get that far, I’d also be interested in letting people here suggest adversarial or unknown coding tasks for Cody to attempt. I’d love feedback from people working with local models: what would you test to determine whether this kind of agent is genuinely useful rather than just overfitting to its own test environment? submitted by /u/Setsuna-Akatsuki [link] [comments]

Source: r/ollama | 2026-09-21