Search: “benchmark”

some good self improvement research here The first experimental evidence of recursive self-improvement (RSI). Autoresearching the autoresearch agent for eight days. The result beats the harness we han

Big unlock for open-source AI inference: Hugging Face Transformers models can now run in vLLM at native speed, often matching or beating hand-written implementations. Until now, every new architecture

congrats on the launch! As the task horizons on agents gets longer, we need more work evaluating and training models to be better in long-running, open-ended, evolving real-world environments. Today w

New model for AMD Strix Halo users: My 198B Step 3.7 Flash release was a big hit, but this one may be even better: 298B-parameter Hy3, now running on a new 2-bit FPX codebook designed to map efficient

Today we present Morpheus, a persistent enterprise simulation platform designed to make Continual Learning a reality. Morpheus is the world’s first real world Reinforcement Learning environment. Every

playing with local AI models and doing some research and came across a fun extrapolation: 'consumer grade graphics cards will be running Fable-equivalent models by 2029' the argument: 1) been said tha

Grok 4.5 is an AI language model developed by xAI, featuring improvements in reasoning, coding, and multimodal capabilities compared to earlier versions. The model demonstrates enhanced performance ac

You can no longer just drop in the smartest model into a harness and expect it to behave similarly to any other model of similar ability levels. This is starting to look a lot more like hiring for a r

My feed has been hijacked this week by frontier model influencers who say they've had 5.6 Sol and Fable for 'months'. I think this tells an inaccurate story the field. From the outside it makes AI pro

Since I am needling every model maker tonight about minor but important issues, one more: Grok 4.5 has no model card. Companies that are trying to compete near the frontier should be releasing model c

Socialists imagine a class struggle. In their made-up fantasy the CEO is in competition with low level workers, the wealthy entrepreneur is stealing from the underpaid nurse. In reality, workers do no

spent a ton of time working on this!! so happy to see the results come out. very impressive model from @NVIDIAAI that we fit our harness to so we could get better performance! We tuned the harness for

The model has an advisor tool that natively escalates to a stronger model when needed. This model is hosted in the U.S. by Perplexity on Nvidia B200 GPUs. We will improve the model in research preview

Perplexity conducted a cost efficiency comparison of language models, using GLM 5.2 as the baseline metric for cost per task. Results showed GLM 5.2 with an advisor achieved 2.1x cost efficiency on th

Added Grok 4.5 to the Harbor Town Playable gallery. Here is it's 'build me a procedurally generated 3D simulation showing the evolution of a harbor town from 3000 BCE to 3000 AD, it should look beauti

Box Agent uses @LangChain’s Deep Agents harness to bring specialized agents into the enterprise content platform.w @NVIDIA + LangChain’s work with Nemotron 3 Ultra reinforces where AI is headed: open,

Busy couple days ahead! Today, @nvidia Nemotron 3 Ultra support for @LangChain Deep Agents at a fraction of the cost -w/ Quick Start Prompt Tomorrow is Wikimania🔥 @hwchase17 is chatting with with @Bra

Come help us scale @harvey’s model training team. If you’re interested in bringing frontier agent research into the Harvey product and working with: - @baseten to scale up RL to 80M+ token virtual dat

dcode <> nemotron 3 ultra <> OpenShell 🔥 Never been easier to own your agent IP! Introducing the NemoClaw Deep Agents Blueprint, a reference architecture for building open agent systems developed with

Enterprises don’t just need agents that perform. They need agents they can shape, govern and improve as their business evolves. That’s the idea behind our work with LangChain. Read more: https://nvda.

Excited to team up with partners across the AI infrastructure + enterprise ecosystem. @EY_US @baseten @FireworksAI_HQ @nebiusai @CrusoeAI @DeepInfra @togethercompute Introducing the NemoClaw Deep Agen

Highly-recommended read. Aligns with what I see in my own harness: > Pi harness got the same success rate as harnesses from the LLM vendors with Opus and GPT, but at 2x less cost > GLM 5.2 was a major

I cannot put into words how stoked I am on this It's not an understatement to say that this blueprint could represent the future of enterprise inference Introducing the NemoClaw Deep Agents Blueprint,

“Model orchestration is in many ways the natural outgrowth of agentic engineering” Huge thanks to @AndrewYNg and @DeepLearningAI for the deep dive into Sakana AI’s Fugu and Fugu-Ultra. The article hig

Our internal assessment is that Grok 4.5 is roughly comparable to Opus 4.7, but much faster. The combination of capability, faster speed and lower cost is what makes it competitive. We are closing the

Own the frontier: 'In our evals, Nemotron 3 Ultra with a tuned LangChain Deep Agents harness provides advanced agent performance at a much lower inference cost. The main takeaway is that agent perform

Rate of improvement is accelerating. Users should notice a meaningful improvement in the usefulness of the Grok Build harness with our V9 foundation model (aka Grok 4.5) every week. Grok 4.5. Pareto d

Really excited to partner with @nvidia on the NemoClaw Deep Agents Blueprint Deep Agents is a fully open source agent harness that we are tuning to make perform incredibly well with open models Introd

We need AI model selection to be MUCH easier ASAP. I want proactive flags from my AI systems suggesting models. I want my AI harness to say 'hey allie, my girl, you keep asking for bar recommendations

Banger compression paper from NVIDIA. (bookmark it) Bigger MoE models keep winning on quality, but serving them at interactive latency is still hard. NVIDIA compresses the hybrid MoE Nemotron-3-Super

If you care about AI, and want a nuanced view, you should watch this. I promise it will be worth your time. An excellent video from the @WorldSciFest featuring @GaryMarcus , in which he discusses—amon

Harrison Chase argues that evaluations (evals) are the most intellectually demanding aspect of agent engineering, distinguishing them from other engineering tasks that may be more routine or formulaic

Cool open-source release. Physical AI is the next frontier, so things are shifting from thinking to taking action. LingBot-Vision looks like a strong visual foundation model and shows progress in cons

How do you trace one number in a 200 page ESG report back to the exact page it came from? We dug into that with the @llama_index team behind LiteParse. We tested five ways to retrieve evidence across

Less than 2 years later, with Fable: 'simulate an encounter between a mind flayer and a drow warrior of equal CR. Set up initial stats then simulate each move, including initiative, rolling dice as ne

私もICMLに行きます！もし行く方がいらっしゃったらぜひ現地でお会いしましょう😀 Sakana AI is heading to #ICML2026 in Seoul (July 6–11)! 🐟🇰🇷 Our team will present 11 papers spanning multi-agent coordination, sparse and efficient LLMs, test-

introducing tinyrouter i reverse engineered the routing architecture behind Skana AI's Fugu and built replication for open frontier models. it's a tiny ~10K parameter LLM router that learns which mode

Sakana AI is heading to #ICML2026 in Seoul (July 6–11)! 🐟🇰🇷 Our team will present 11 papers spanning multi-agent coordination, sparse and efficient LLMs, test-time scaling, long-term memory, and agent

arXiv:2511.02496v2 Announce Type: replace Abstract: We study latent geometry as an explicit component of representation quality in data-scarce learning. For an encoder (phi), we define (Q_{eta,gamma}(

Highly-recommended read from MIT on the part of RL with verifiable rewards that everyone keeps hitting. RLVR only optimizes what you can objectively score, so style, structure, and diversity quietly c

We analyzed GLM 5.2 vs Sonnet 5 for software engineering tasks using DeepSWE. GLM 5.2 gets you ~80% of Sonnet 5's capability at ~20% of the price. More insights in the thread! Deepdive: Sonnet 5 and G

Continual learning is probably the biggest barrier to explosive AI adoption (& may have big implications for recursive self-improvement as well) As long as you deal with amnesiac models that require h

ARC-AGI-3 is built different, it has dumbfounded almost all regular attempts so far because it's so much harder than anything that came before. It has no rules, it's agentic and has no explicit goals,

Cool new paper from NVIDIA. Looks like agentic coding is moving into hardware design. HORIZON treats hardware design as repository-level code evolution. A Markdown harness becomes a project pack with

Featuring the man of the moment @dylan522p @shaunmmaguire in today's Training Data episode. Nobody is a more trusted industry insider to the biggest infrastructure build-out in history. The story of h

𝗚𝗟𝗠-𝟱.𝟮 (the latest open weights model) is having an Enterprise moment, and it is not an exaggeration.🚀 🔥 We have been impressed by how strongly GLM-5.2 is pushing long-horizon performance .. not just