Search: “benchmark”

Great paper if you are tracking progress in recursive self-improvement (RSI). (bookmark it) There is so much hype around RSI, so I think it's worth understanding why current models are not able to do

Live now: our AI Architects, AI Factories Track from AI Engineer World's Fair 2026. Thesis: the model is the easy part of the factory. https://www.youtube.com/watch?v=rbjWzZK2LU0&list=PLcfpQ4tk2k0VeXE

What a fascinating paper on AI agents. A lot of the issues we see with AI agents today revolve around wrong assumptions the LLMs make. This leads to problems like hallucination, cost inefficiencies, u

Banger paper from Google DeepMind. It's on the very hot topic of model routing. Routers assume the value estimate is free. Working out which specialist handles a query best costs money too, and it is

Brace yourselves. We just entered a new era of frontier multimodal models. DeepSeek-V4-Flash-Vision-Exp advances multimodal agent performance. Also, pay attention to Ox Alpha 1M token contexts with te

Grok 4.6 just took #1 on CursorBench. Not only the highest score. It did it at a fraction of the cost of the models sitting right behind it. That combination is the real signal. Top-tier results are o

Impressive research from Google on building better environments for agents. Training environments for agents are hand-built and go stale. The agent improves, the environment does not, and it's not abl

Multimodal API support 🔌 🔹 Set model='deepseek-v4-flash-vision-exp' 🔹 Images are tokenized for billing: up to 384 tokens each, at V4-Flash pricing 🔹 Supports Chat Completions, Messages & Responses 🔹 S

This is very nice work from NVIDIA. Like all high-performing approaches on ARC-AGI-3, it uses deep learning-guided on-the-fly synthesis of symbolic world models, i.e. navigating the world by generatin

Grok 4.6 Grok 4.6 just tied for #1 on the Artificial Analysis Agentic Index • Grok 4.6 (high) — 59 • Claude Opus 5 (max) — 59 Outperforming Claude Fable 5 and GPT-5.6 Sol We’re entering the agentic er

Huge thanks to Unsloth for the great work. This is wonderful news for the community! 🥳Qwen3.8-27B, smaller and sharper than ever. Let's try it! @UnslothAI We’re releasing new Qwen3.8-27B GGUFs with 10

Qwen 27B is really good local model but, when you use it, it is immediately absolutely and obviously nowhere near as good as the other models listed here for agentic tasks, and especially for the kind

If you’re building an “agent over your PDFs” in 2026, one of the most underrated things you can do is to provide exact grounding back to the source document for any agentic answers. Specifically, show

LangSmith’s new Tuned Evaluators look pretty interesting. Perceived Error can flag agent mistakes in production by picking up on user corrections and unresolved conversations. What’s interesting is it

New Tuned Evaluators from @LangChain Versioned judges you just 'turn on' in a tracing project The 'Perceived Error' Judge flags conversations where the agent probably messed up (proxy for user feedbac

Perceived Error is one of the clearest signals that your agent is giving users a great experience. We tuned the model and prompt around signals that tend to carry across domains, like a user correctin

teams want to understand what their agents are doing but it’s hard to sift through all that trace data at scale but you can basically find any signal by fine-tuning a small, cheap model on data showin

the real unlock is not cheaper evaluation it is making specialized evaluators cheap enough to run continuously on production traces so evaluation stops being a launch checklist and becomes a permanent

🚀Today we launched LangSmith Tuned Evaluators, starting with Perceived Error. Tuned Evaluators run on production traces to catch undesirable agent behavior and attach feedback that you can use in your

😮 wow that’s huge 💰 🚀Today we launched LangSmith Tuned Evaluators, starting with Perceived Error. Tuned Evaluators run on production traces to catch undesirable agent behavior and attach feedback that

Announcing AItonomy Foundation. We are PhD students and postdocs at Stanford, Princeton and Oxford, building a nonprofit research organization for AI4Science and Science4AI — and connecting early-care

Ollama has the best performance for deepseek v4 flash on average. For local only, you can try qwen3.8 that is optimized: Apple Silicon: ollama run qwen3.8:27b-mlx NVIDIA: ollama run qwen3.8:27b Benchm

Grok 4.6 is my daily driver now. This is why: When I’m using interactive agents in a CLI/terminal (or even Slack), what matters most to me is speed & token throughput. Intelligence and differences in

You can literally run Opus 4.6 Max-level intelligence on a MacBook Pro. Locally. We promised open weights for Qwen3.8. Now, time to meet them! 🎉 ⚡ Qwen3.8-27B: - A native multimodal dense model. With

Jeremy's excellent work here is a great illustration of a very powerful type of approach: LLM-guided on-the-fly synthesis of a symbolic world model, i.e. making sense of the world by writing executabl

Regular reminder -- the set of public ARC 3 games is called 'demonstration set', not 'eval set' nor 'training set'. It is not meant to be used as training data, and it is not meant to be used as an ev

Something has happened with post-training as shown by DeepSeek flash & GLM-5.3 updates. Same base, big improvement in perf to frontier levels. Can't explain this by even logit distillation etc These a

out of curiosity, i tasked fable to figure this thing out and it looks like the 'saas killer' is done. ▲ it maxed out Fable 5 + used most of Codex limits ▲ most of the work was done within 45h session

Thanks. The fact that it works so well for ARC is also a clue to the extent of generalization in these models. It was not at all obvious that it would work originally because the model is never traine

There is much less signal in agent leaderboards than the rankings imply. A four-facet Generalizability Theory decomposition across TheAgentCompany, tau-squared-bench, and AppWorld finds the agent main

We’re launching DeepSeek-V4-Pro today! 🚀 🔷 Major Agent upgrades with strong production gains! 🔷 Flexible reasoning effort for V4-Pro & V4-Flash: low for simple tasks, high for daily Agent workflows, m

// Actions Speak Louder Than Words // Multilingual agent evaluation compares final answers and throws the trajectory away. The trajectory fixes cost, latency, failure mode, and auditability. New resea

We are in an insane run of open-weight drops. Every modality, open source is winning. This is what an open source AI summer ☀️ looks like: 🧠 LLMs & Reasoning → DeepSeek-V4-Flash-0731 (my king 👑): 304B

From @sequoia's own your intelligence event. We're starting to see a new category of company emerge, the Full-Stack AI company, where innovation happens at both the product and intelligence layer. Tha

Had so much fun giving this talk at @sequoia about @harvey’s moneyball approach to building a research lab. The biggest mistake I made in the early days of Harvey was trying to play the Yankees baseba

Grok Imagine is focused on professional usefulness, fun for consumers and overall ease-of-use Grok 4.5 just ranked #1 on Design Arena for Daily Usage - measuring the average unique users actually inte

Meta returns to open weights: Muse Glimmer, its first open-weights release since Llama 4, scores 35 on the Artificial Analysis Intelligence Index. It is a 30B-parameter model, and the first from Meta

The successor to Llama is here, and Meta is revitalizing focus on open weights with their new Muse Glimmer - a leading 30B param model designed for always-on local agent use, small enough to run on a

Hey peeps. I know you're tired of low quants giving hard to believe numbers. I'm quite skeptical too and from what I tried I'm often left with the impression that the claims fall short. So this model

In this episode of @wandb's Gradient Dissent , @l2k and Fireworks CEO @lqiao discuss why she believes the industry is at a turning point... ... one that calls for more open intelligence, not less. The

Alibaba’s Qwen team announced the new Qwen‑3.8‑Max model and thanked users for extensive testing, noting its ability to provide highly detailed world observations. A community member highlighted that

On July 30 Google AI published a video announcing **Gemini Robotics 2**, an intelligence layer developed by DeepMind that aims to bring autonomous robots closer to everyday human environments. The pos

This definitely seems like something worth noting, and illustrates the gap between Fable/Astra class models and the previous frontier that was 'merely' good at hacking under human instructions. Initia

Hey everyone! I just released a sub-6B sparse activation AI model which was built with a brand new architecture : fusion. I fused weights from @liquidai's LFM2.5-2.6B & @Alibaba_Qwen's Qwen3.6-35B-A3B

None of this was us. Day 1 (and the first 48 hours) belonged to the open-source community: Generate with it @ComfyUI — native support + official quantized builds, Day 0 Diffusers — the reference Pytho

SITUATION DETECTED: Prime Intellect is releasing Prime Agent, a self-improving harness for coding and long-running autonomous tasks. The team reports 95.5% on ARC-AGI-3, above the human baseline, and