My local Gemma 26B spent 4 months predicting stock direction against Claude and GPT-5.4. It lost to a bot that just says 'up' every day.

Quick version: I run a public leaderboard where bots predict whether stocks close higher or lower, and every call gets scored against real prices. Nothing gets deleted, timestamps are locked at submission. For 4 months I ran a 4-bit Gemma 26B locally on my laptop (MLX build, not Ollama, same idea) with a deliberately plain prompt: recent closes, answer up or down. Claude Opus 4.8 and GPT-5.4 got the identical prompt. Local Gemma: 49.2% accuracy over 453 scored calls. A rule that answers "up" every single time, on the exact same calls: 53.4%. Before anyone dunks on local models: the APIs didn't beat their baselines either. Claude 53.9% vs 54.6%, GPT 50.5% vs 51.6%. Nobody beat the dumb rule. The one thing that did work: the same local Gemma, fed one news-moving stock a day instead of a fixed list. That line runs 7.6 points above its paired baseline (p ≈ 0.02, small sample, could be multiple-comparisons luck). So far the input pipeline matters way more than the model. If you want to throw your Ollama setup at it, there's a starter bot that works out of the box. ~120 lines, stdlib only, dry-run by default, MIT: https://github.com/kkjh0723/ldbd-starter-bot Full numbers and methodology: https://ldbd.app/blog/claude-chatgpt-gemma-stock-benchmark Disclosure: I built the leaderboard (LDBD). I would genuinely like to see someone's local model beat the paired always-up baseline. So far nothing plain-prompted has. submitted by /u/Present_Ground_4583 [link] [comments]

Related

• I spent months pushing my RX 6700 XT into local AI. I think I seriously underestimated this GPU.
• How I built a free, local AI powerhouse in 10 days (Ollama + Gemma 4 + Claude Cowork 3P + Browserless)

Source: r/ollama | 2026-08-27