Search: “reasoning”

great write up, evals are hard! here are 2 broad buckets we use to evaluate agents: 1. Measure the State of the World 2. Agent as a Judge on the Trajectory 1. Measure the state of the environment befo

Some things to note: 1) AI is getting very good at math and science. 2) Two years ago LLMs could not do basic math consistently 3) According to @polynoamial this cost less than $2000 in current API co

“Stochastic parrots” is not my term (it’s @emilymbender’s). But a lot of people today commenting on it are confused. To some extent (though I don’t think it’s a perfect metaphor, and have said that be

The “AGI-is-near” community keeps committing the same logical fallacy over and over; I have seen it at least half a dozen times today alone. Every time there’s an advance, I see the same error. Here’s

Your design, your model. Create with your pick of the world's leading models, including Claude, GPT-5, Gemini, Kimi, and GLM. Compare outputs across model families and keep the result that nails it. O

Can AI agents conduct open-ended AI research? Most evaluations of agents conducting AI research focus on narrow, verifiable tasks. But AI research is often open ended. Researchers pick hypotheses, dec

BREAKING: Grok 4.5 just claimed the top spot on the new HighWalk Benchmark. The independent test measures how well AI models update real technical specifications from 46 Laravel commits — heavy on cod

Model + harness. We have barely begun to understand the best ways to do harness engineering. A huge amount of untapped potential even without models getting better (but models are getting better) Turn

We hope these experiments serve as a reminder that evals rarely measure models in isolation—they also measure a bundle of less visible choices about API settings, harness design, and prompting. If you

Alibaba has released Qwen Audio 3.0 Realtime, with the Plus variant debuting as the new #1 model on the Artificial Analysis Speech to Speech Index at 84.1%, ahead of GPT-Realtime-2.1 High at 79.1% Rel

New research from Meta and CMU. This one is on agentic context management for long horizon tasks. (bookmark it) Production agents accumulate context every turn. The usual fix compresses on a token thr

Sam Altman: “Concentration of power with AI is a terrifying thing.” 'A lot of the talk about safety concerns is well-founded, and then a lot of it is about people that just really, even if it's slight

Happy to have @FireworksAI_HQ as our day0 launch partner and bring Kimi K3 to more developers. With Fireworks, you can now deploy and fine-tune the 2.8T Kimi K3 model with just a few clicks! Kimi K3 i

Kimi K3 is live on Fireworks. Day 0, inference and training. US-hosted, and zero data retention. This is the first frontier open model in the 3 trillion parameter class. It sports 1M context, native v

Kimi K3 is now available for inference & training in @FireworksAI_HQ. Crazy how easy they make it to tune frontier open models like K3 using LoRA adapters. Best time to figure out how to own your inte

Looking forward to chatting with @hugobowne today (July 27) at 4 pm PT on the Vanishing Gradient livestream on YouTube. Will cover open source, the newest LLMs & trends, agent frameworks, and whatever

New research from NVIDIA. They just dropped a PyTorch-native training framework for agentic RL. (bookmark it) Paper summary: Molt is a PyTorch-native agentic RL framework with an unusual design target

Nice little insights on doing autoresearch with coding agents. Hand a coding agent a dataset, an eval script, one editable file, and no supervision. That's autoresearch and it tries to optimize the nu

Very cool paper from Microsoft. The idea is to train agents on replayed teacher trajectories instead of live environment rollouts. On-policy distillation for agentic tasks is expensive because every u

We are releasing PerceptionBench, a benchmark that isolates visual perception and evaluates it as a set of atomic capabilities - discovered from how today's models fail, rather than defined in advance

BREAKING: A Redditor just discovered that shared Claude conversations have been showing up in public search results. The post has 4K upvotes and hundreds of comments, so this is spreading fast. Here's

Simply untrue. The Economist asked Musk about a wide range of things, including AI and robotics. Watch it here for yourself: https://www.economist.com/insider/the-insider/an-interview-with-elon-musk?u

Diffusion LLMs can now handle real agentic work. LLaDA 2.2 is the first large-scale diffusion LLM built to operate as a real agent, planning, calling tools, and self-correcting across long multi-turn

Very happy to support this on behalf of Google. We have long benefited from open source, are big contributors to open source and in fact have consistently made open weights models with Gemma available

Am I supposed to interpret that it's better than Fable 5 from the benchmarks? Or it's ~close but cheaper? Introducing Claude Opus 5. It's a thoughtful and proactive model that comes close to the front

Announcing Fugu-Ultra v1.1 🐡 We’ve been thrilled by the reception to the Fugu model family. Thanks to everyone who tried it, shared feedback, and trusted Fugu with real work. Today, we’re releasing Fu

For my first post, I’m sharing a letter @NVIDIA signed on why open models matter. AI will transform every industry, power every company, and be built by every country. Open models strengthen safety an

Ling-3.0-flash, the new MoE model from @AntLingAGI, is now free in Nous Portal for the next week! At 124B parameters and 5.1B active, it's quick to run and built for agent workloads: coding, search, r

If I had a nickel for every snide but underinformed person who failed to distinguish between pure LLMs and neurosymbolic AI I would be a billionaire. @GaryMarcus Hold on Gary, I thought these LLMs wer

Nice to see someone around here actually understand the technical details. (The vast majority of the attacks on me come from people who don’t.) And the probabilistic next-token critics were correct ab

We had guessed reward seeking might increase over the course of capabilities-focused RL training, but had no way of measuring it until now. We’re continuing to collaborate with Apollo Research to impr

NVIDIA AI has released Cosmos 3 Edge, a 4‑billion‑parameter open‑source world model designed for on‑device inference. It enables robots to learn and act, autonomous vehicles to interpret road scenes a

Backpropagation by hand ✍️ ~ 11 steps walkthrough below Backpropagation is the algorithm that actually trains a neural network, and it is where most people stop following along. It is not calculus you

Excited for our first general model Inkling -- open weights, 975B, natively multimodal (text, image, audio). Available on Tinker, HuggingFace and partners. It is yours to personalize and use openly. I

Given GB300 I would estimate this is was trained on about 1e25 flops (same as DeepSeek v4) over 1.6m hours (1 month on 2k chips/28 racks) Cost 10-20m ($6-12/hour/chip) The lite version likely 4x less,

Never too late for any company to get in on open models. Inkling is a MoE transformer with 975B (41B active). Native multimodal support. Context lengths: 64K and 256K. Will be testing and providing th

Something I have been thinking about: in the past, the best engineers I knew spent a lot of time automating their work in various ways. Better vim/emacs automations, writing lint rules to catch repeat

The best base model for training isn't about someone else's benchmarks. Excited to offer @thinkymachines' first open-weights model: Inkling. 975B MoE, multimodal, with Apache 2.0. A great foundation f

Thinky with a ~1T param, 41B active, apache-2 model Benchmarks are a clear step up from Nemotron Ultra (55B active), new best American model, and omni input. A bit behind GLM 5.2 on agentic benchies,

Bonsai 27B is a big step for local AI: 27B-class multimodal capability in a phone-class footprint. Congrats to @PrismML on the launch. Try Ternary Bonsai 27B on Together AI: https://www.together.ai/mo

Highly-recommended overview of metacognition in LLMs. (bookmark it) Interesting behaviors in LLMs like confidence calibration, self-verification, knowing when to stop, and knowing what you do not know

Huge if true! We are talking about a 27B multimodal model that runs locally on a phone. That's wild! Bonsai 27B reaches up to 163 tok/s in 1-bit and 134 tok/s in Ternary on an NVIDIA GeForce RTX 5090.

ICYMI: @LangChain Deep Agents on @nvidia Nemotron 3 Ultra. frontier open-model agents at ~10x lower cost than closed. Run on Fireworks, then post-train it into specialized intelligence you own. https:

Today, we’re announcing Bonsai 27B: the first 27B-class model to run on a phone. Bonsai 27B is the new multimodal flagship of the Bonsai family. Based on Qwen3.6 27B, it brings a new capability tier t

Ethan Mollick notes that image input represents the primary advanced capability of current AI models, and that tool‑use can effectively replace outputs from non‑omni models. He observes that multimoda

Absolutely fascinating work by @SakanaAILabs reproducing @kenneth0stanley Picbreeder in a non-interactive, VLM-agentic way. I've had years to reflect on Kenneth Stanley's ideas as originally communica