Search: “benchmark”

The strongest models are gated and access is granted only to a select few. Hermes Agent now exposes MoA presets as virtual models, giving you capabilities beyond the publicly available frontier: 8% hi

In a joint Fireworks and @Faros_AI evaluation of 211 real engineering tasks, Claude Code + GLM-5.2 beat both Claude Code + Opus 4.8 and Codex + GPT-5.5: - Judge score: 0.568 vs. 0.521 and 0.466 - Time

Just had a great discussion on dynamic workflows. Rough notes: - applies to a very small set of use cases - think of it as a new paradigm of (test-time compute) TTC - strong for hill-climbing research

Open + closed models = better together. Our previous research with @harvey showed the benefits of combining a frontier closed model as an advisor agent with fine-tuned, open-source worker agents. Thre

The underrated part of this announcement is that Fireworks has been quietly great behind the scenes helping us eval and serve these models Big kudos to the Fireworks team! Kimi K2.7 Code and GLM 5.2 a

AI21 Labs reports that a leading deep research agent achieved a score of approximately 45 six months prior, after which the research community focused on developing improved agents to surpass this ben

Big Tech's 1 trillion AI moat just got DESTROYED by a free Chinese download. Microsoft, Amazon, Google, and Meta are pouring fortunes into chips and data centers because they have been told that whoev

🧠 Paradigm II — Agent Foundation Model: world modeling as agent capability. Single-turn, non-agentic environment prediction → tested directly on multi-turn, tool-calling agent tasks. No agentic RL, no

Sakana AI CEO David Ha (@hardmaru) appeared on TBS CROSS DIG’s “1on1 Tech.” He talks about our founding story, latest research and technical vision, product launches including Sakana Fugu, Japan’s AI

... and now you can buy your own! https://happy-pelicans.printify.me/product/29480311 Proceeds go to http://birdrescue.org, a local non-profit that rescues pelicans (among other birds) I printed a cus

GLM 5.2 is already top 10 on OpenRouter this week, sitting right next to Opus 4.8. it's also been really strong in Deep Agents across long-horizon coding, big context, tool use, and verifying work bef

GLM-5.2 has been the most popular new model on Fireworks this past week. @ArtificialAnlys confirms why: #3 overall on GDPval-AA (1524 Elo), #1 open weights by 116 points. Interest is showing no signs

That's right, an open, MIT-licensed model beating GPT-5.5 (xhigh) on real-world agentic work! 🔥 Available for free on @huggingface for anyone to build on top off GLM-5.2 leads open weights models and

The largest LLM-as-a-Judge reliability audit yet. Researchers ran 21 judges from nine providers over roughly 541,000 judgments on MT-Bench, JudgeBench, and RewardBench. Findings: Validating a judge wi

>> Scalable Evaluation for AI Agents << If you run agent evaluation in production, this one is worth your time. It shows that front-loading human judgment into reusable evaluation assets is useful. Bu

We parsed this SpaceX equity research PDF faster than the time it took for Screen Studio to zoom in ⚡️🔥 liteparse is now the best open-source document parsing tool out there. There’s no reason to not

🚨🚨🚨A research project idea! How to measure world models? Everyone's talking about world models these days. World model here, world model there. We can argue about what 'world model' actually means, an

// Evolving Meta-Skill for Multi-Agent Systems // Can a multi-agent system get better at orchestration without touching a single weight? Automatic MAS generation has been stuck between two bad options

GLM 5.2 is a language model available through Ollama that has received positive user feedback for its performance capabilities. The post appears to be a social media testimonial praising the model's q

Hot take: GLM 5.2 might be the first open/public model that actually changes the enterprise AI cost equation. I played with it for a few hours today after friends told me to stop ignoring it. I expect

Grok Build is pretty good at optimizing my code in one shot. Prompt: I want you to optimize it entirely on GPU to speed it up. Measure two metrics: the result must compare with the golden image (CPU)

I too have felt the latest Claude's to be too smart its bored. Glad others are noticing the same Claude Fable 5 thinks document parsing is beneath it It is absolutely crushing on all reasoning-intensi

The Model Lab vs Agent Lab distinction is one of the clearest frameworks I've seen for understanding where AI value actually lives right now. TL;DR from @latentspacepod: • Model Labs compete on capabi

I'm really not interested in testing a new closed-weight model in the expensive tier. What's the point? I know what will happen: 1. They present the model. 2. The model beats the competition on benchm

arXiv:2606.07565v1 Announce Type: new Abstract: Intelligent scaling of microservices in cloud platforms is crucial for mitigating escalating compute costs while avoiding service disruptions. Current s

Elon Musk posted on X about Tesla's AI chip design engineering team and their work on the AI6 chip, expressing confidence in the team's performance and suggesting the chip could achieve notable capabi

A French engineer who lives quietly in Paris has spent 30 years writing software that the entire internet now runs on without knowing his name. He wrote the code that streams every YouTube video, ever

I really do think we'll see a lot of value accrue in AI startups building 'model routing as a service' Not just OpenRouter - this includes a much broader set of verticalized agents and infrastructure.

blast from the past 3.5 years ago; some things have changed (esp. coding and math, via neurosymbolic techniques) but many haven’t: Bottom line: From the outset Large Language Models like GPT-3 have gr

This was one of the standout AI papers of the week. (bookmark it) It tackles a question most self-improving AI agents ignore: is the agent actually discovering anything, or just remixing what it alrea

We didn’t just start working on Recursive Self-Improvement yesterday. We have been methodically shipping the foundations for two years. To execute the next phase of this mission, we are seeking a sele

Building AI that Builds AI: Introducing the Sakana AI RSI Lab 🚀 https://sakana.ai/rsi-lab Today, we are announcing the Sakana AI Recursive Self-Improvement (RSI) Lab: a dedicated research group in Tok

Token costs are why there will be no saas apocalypse / good dev tools are cached intelligence for agents! The popular theory goes: agents can write code, so they'll just rebuild every tool from scratc

We've made a breakthrough in self-evolving AI scientists moving from 'search' to 'principled discovery': Scientific discovery requires that the search space itself changes, and an AI scientist must pe

A couple of weeks ago we started rebuilding the 𝚑𝚏 CLI with AI agents in mind. it now detects when an agent is using it and gives clean, token-efficient output, next-command hints, and more, all desig

New course on serving LLMs efficiently -- how do you serve models to many concurrent users at low latency and reasonable cost? This short course is built with @RedHat and taught by @cedricclyburn. Eff

nice post from @Harvey and @LangChain Labs, worth a read. improve agent feedback loop without setting $$ on fire Can we design legal agent verifiers that are up to 1,000x cheaper? Verifiers are LLM ju

Outstanding paper on long-horizon agents. (bookmark it) Similar to humans, how do you make agents persist on a difficult task, and how is that useful? And which models today work well on this? This ne

That's a badass title, and it's true! Every day, it gets harder and harder to create tests that AI models can't beat. Reality is humanity's real last exam. Andon Labs' Real-World AI Evals: Claude call

We're presenting ParseBench at CVPR 2026 today. 🦙 Come learn why document understanding is an AGI-complete problem (an agent can't act on a doc it can't correctly read, and reading a real enterprise t

Harvey just published a study showing a hybrid setup, open source GLM 5.1 as primary worker, routing to Opus 4.7 only when needed beats pure Opus 4.7 on quality and costs less. This is the multi-model

In early May, the best superforecasters predicted that, by the end of the year, the longest METR 80% task horizons would reach 3-4 hours. In late May, Claude Mythos achieved that number. We also asked

Nice primer on post-training reasoning data. (bookmark it) This is one of the first primers to pull the scattered post-training reasoning-data literature into one place, synthesizing over 150 public s

Routing and post-training open-source models won't only give you more accurate systems but also meaningfully faster and cheaper systems as most companies are currently learning (in addition to giving

We partnered with @FireworksAI_HQ to train open-source models for legal. Here's what we found: 1) Hybrid legal agents can beat frontier models on quality and cost by routing selectively to a frontier

// State-Externalizing Harnesses // A new paradigm is emerging on how to effectively build agents and harnesses. If there is a state that the environment can maintain reliably, it probably doesn't bel