I fine-tuned Qwen3.5-4B on ~3k browser trajectories and improved benchmark results from 22% to 62%

Smaller models still struggle with browser tasks that require open-ended research or deep reasoning. But many practical browser-automation workflows are much more procedural: extracting structured data, downloading or uploading documents, creating or submitting forms, etc. These tasks can usually be defined precisely and repeated, which made me wonder whether a small model could perform them reasonably well when paired with the right browser harness. To test this, I fine-tuned Qwen3.5-4B on roughly 3,000 Browser Agent trajectories generated by GPT-5.6 Luna, which is a strong baseline for browsing tasks. The results: On BrowseWebApp Bench, which tests more complex but well-defined web-app workflows rather than web research, accuracy improved from 22% to 63%. On the more balanced BU Bench V1, accuracy still improved from 15% to 53%. On BU Bench V1, the 4B SFT model also outperformed the reported 45% results for DeepSeek V4 Pro and Kimi K2.6 using the open-source Browser Use harness. https://preview.redd.it/z86uw1zlu3mh1.png?width=1472&format=png&auto=webp&s=f16db66bb7d3b0cc7b99aa86696cdb0c1b31253f For SOTA performance, stronger models still need to be paired with either Browser Agent or a harness such as BrowserCode. I chose Browser Agent for the fine-tuning work because BrowserCode is built around a coding harness and produces substantially longer trajectories, making training and trajectory generation more expensive. Links: Model: Qwen3.5-4B Browser Agent SFT FP8 Harness: Browser Agent Benchmark: BrowseWebApp Bench BU Bench V1: Browser Use benchmark submitted by /u/dnx2200 [link] [comments]

Related

• I benchmarked Qwen3.8 27B on browser tasks. It's on par with GPT 5.6 Luna (xhigh)
• EigentSearch-Q+: Enhancing Deep Research Agents with Structured Reasoning Tools
• Deepagents https://github.com/langchain-ai/deepagents

Source: r/ollama | 2026-08-28