Search: “benchmark”

✈ DeepSeek-V4.1-Flash-1017 is on Ollama! US-based hosting with zero data retention. It's strong on agentic tasks while being cheaper per task. Try it on Ollama Cloud: ollama run deepseek-v4.1-flash:10

Great application of small language models from Salesforce and colleagues. AI-scientist pipelines are well known to struggle to propose novel and diverse research ideas. This team trains an 8B reasoni

Must-read paper integrating decision models like Jev into harnesses. They distilled an open 4B decision model, mostly from Jev's answers on the agent's own states, that beats Jev inside that agent and

Guanzhi's deep dive thread: https://x.com/guanzhi_wang/status/2111127089428701366?s=46 ASPIRE is now open-source! Since our initial release, we've made exciting progress: 🤖 Real-world robot manipulati

In October 2026 Georgi Gerganov posted that he ran **llama.cpp v0.7.0 on a 128‑GB‑RAM Mac (Spark)**, serving the Qwen3.8‑Flash‑Next GGUF model in IQ4_NL precision. The command used was `llama serve -h

Pay attention to this new wave of VLMs. Glimpse is the first model from @ElorianAI, the brains that led Google's Gemini and Apple's MM-1. Quick test: 10 game boards from an overhead camera. Same promp

🆕 show: HIGH TASTE TESTERS We convene top Al power users to compare what actually works-and what falls apart once you start using it.@davis7, @benhylak, @dexhorthy, @allisonology, @altryne and @swyx g

We are open-sourcing V1.0 of our agentic robotics framework, ASPIRE, with full real robot skill discovery and self improvement capabilities! On the day Astra arrived, our lab switched the backend mode

Argon has a similar performance profile on ARC-AGI-3 as Astra. Google is back at the frontier. Gemini 4 Argon from @Google on ARC-AGI (Verified): - ARC-AGI-3: 97.5%, 4.8K (provider adapter harness), 4

New work from Meta AI on agents that manage their own search. (bookmark it) A ReAct agent keeps every failed attempt in one context, making useful evidence harder to find. To improve on this, they pro

We want to support the mathematics community in turning progress in AI into deeper human understanding and new discoveries. To support that goal, we are contributing 5 million to a new Simons Foundati

Banger paper from Stanford, Together AI, and colleagues. Why is this an interesting paper? Today's LLM agents must write out every reasoning step as text before they can search, run code, or take acti

New open source app built with Together AI! Gloss turns your saved articles, PDFs, and X bookmarks into a beautiful personal library, with lightning-fast AI answers powered by DeepSeek V4.1 Flash. Try

A worthwhile (and unnerving) reminder from Anthropic that we should be paying attention to AI ability thresholds, not just steady climbs In this case, the threshold is the capability needed for a self

After much testing and talking to actual users across all three proactive AI agents: Muse - best one for your parents, heavy IG/FB users, people who like or need easy setups, people who don’t want to

Banger paper from Google DeepMind on co-evolving an agent's harness and its weights. (bookmark it) If you already log your harness changes and results, those logs can become training data for the mode

Brilliant paper on agentic continual learning from Meta and MIT. (bookmark it) Agent memory harnesses usually only write forward. A lesson goes into the bank and is never checked against whether it he

Learn to build custom harnesses, folks! Here is an impressive paper showing the impact of a good custom harness on long-context work. GLM-5.3-Flash goes from 4.4% to 61.1% exact match on OfficeQA Pro

🚀 Introducing MiniCPM-V 4.7, our new open-source MLLM series! Advancing the Performance–Efficiency Pareto Frontier. It brings strong multimodal capabilities to iOS and Android devices with extreme eff

Must-read paper on the effectiveness of multi-agent systems and persistent memory. We have recently seen many viral examples of multi-agent systems scaled via persistent memory. This work takes a clos

The Bitter Lesson for harness design and formal math! Instead of hand-designing how an AI should prove (decompose, plan, assign agents), what if we just give a frontier model good Lean feedback and le

A new work suggesting that division of labor among self-improving AI agents can help them avoid reinforcing their own errors. I found this work interesting from a metascience perspective. Science is a

Introducing Multi-Agent Self-Supervision (MASS): Recursive self-improvement through collective intelligence Blog: https://pub.sakana.ai/mass/ As AI systems tackle increasingly open-ended research prob

New paper from the team at Sakana AI: Recursive Self-Improvement through Multi-Agent Self-Supervision (MASS) How do we keep improving AI when the tasks become too complex for humans to reliably evalua

'a corollary is that information about product quality will become more important and impactful' This feels like the next evolution of SEO/AEO that every product will try to game. Even if agents do th

AI solved open mathematical problems. But the biggest surprise came afterwards. A team at Google DeepMind has used AI to resolve 9 open Erdős problems, including questions that had remained unsolved f

Finetuned Qwen3.5-4B into a decision model like Jev using @UnslothAI. - 4-bit QLoRA, bf16, single A100, 15 min. - Train/val loss flatten after 2 epochs so most of the learning happens fast. - Trained

Recommended paper from Sakana AI on recursive self-improvement. They propose an interesting way to scale recursive self-improvement through multi-agent self-supervision. In this line of research, self

Build your own harness, folks. Reading papers like this makes me realize how underexplored harness engineering really is. The authors find that on whole-repository migration, GPT-5.6 Sol goes from 6.5

arXiv:2602.24230v2 Announce Type: replace-cross Abstract: In probabilistic classification, calibration error (CE) measures the average divergence of predicted probabilities f(X) from P(Y|f(X)), the tr

AI agents stopped being tools. They're becoming workers. I sat down with @EMostaque to explore how fast intelligence is getting cheaper, what it means for jobs and robotics, and why energy and compute

an open model just beat Jev outright on clinical decisions 🔥 pplx-decider v1.1 got 643 of 669 right vs 628 for Jev, at 42% less per decision and about the same speed. and you can download the weights

OpenDocRouter is basically OpenRouter for document OCR You can compare and use all your favorite OCR models under a single API / billing interface. They range from some of the lightweight/OSS models (

Banger paper from NVIDIA. One exciting trend I am seeing is building verifiable environments for your agents and training the harness alongside the model. Co-evolving the harness and the model is a bi

Claude Haiku 5.5 is now available on OpenDocRouter as one of the cheapest models for document parsing. It is roughly $1.2 / 1k pages and does quite well on tables and text reading order compared to ot

Introducing MCP tools for @dair_ai. Connect the MCP to your Codex/Claude/Grok Bot to discover and explore the top AI papers. We carefully curate the papers, so you can expect mostly bangers. The index

Mistral AI introduced **Mistral Large 4** in October 2026 as a general‑purpose agentic model capable of gathering information and producing final deliverables across complex workflows. The launch high

The Windows launch event today has mentioned 'local AI' 41 times (so far) 🤯 Media It is very cool to see llama.cpp on the big stage in todays Windows event! The software and hardware stacks are finall

AI is driving the biggest infrastructure build-out in history, with @Google alone forecasting $200B of capex in 2026E. What are the real constraints on a system at this scale? Amin Vahdat is the chief

Mistral 4 is pretty good at lawyer bench and cyber security bench, but give it's European heritage, one can only assume it's also good at regulation bench Meet Mistral Large 4, aka Le Chonk. • 1T para

Mistral can pelican now! https://tools.simonwillison.net/markdown-svg-renderer?url=https%3A%2F%2Fgist.github.com%2Fsimonw%2F0db8b8196e57711b352f6c5857cf5b35#response-1 Meet Mistral Large 4, aka Le Cho

Super excited about this and I know weights are coming (https://huggingface.co/mistralai/Mistral-Large-4.0-1T05-A52B) but just to be fair, you can't be the best open-weight model if you're not open-we

That one took some groundwork. Trained and served on our own compute, and RL shows no sign of saturation Meet Mistral Large 4, aka Le Chonk. • 1T parameters, natively multimodal. 49B active. It is the

Today, we are launching a preview of our new model, Mistral Large 4 (ML4), aka le Chonk 🐈. ML4 is a 1T-parameter model with 49B active parameters, trained natively with multimodal capabilities. It is

Banger paper from Microsoft and colleagues. If you run agents that search a large document collection, this one is worth your time. (bookmark it) They introduce CorpusMap, which resolves recurring ent

Learn to optimize your own harness, folks. You can squeeze much more performance and value from a harness. This work claims that a self-modified harness matched Codex for 4.03. Specifically, a coding