Search: “labs”

Independent verifiers improve agent output - but frontier verifiers are expensive. So we trained our own. Results on @GoogleDeepMind's FACTS-Search: > No verifier: 89.4 (previously published SOTA) > W

OpenAI Jalapeno is spicy Usually first generation chips aren't competitive, but OpenAI is beating Nvidia Blackwell and even Rubin This is huge news! We got to go into OpenAI's lab to dissect their new

Adding a verifier improves every agentic AI system. But frontier verifiers are expensive - so we trained our own. Matching quality, at a fraction of the cost. Thread below 🧵 1/ Your agent already foun

【採用情報】Member of Technical Staff (LLM Development) の募集を開始しました🐟 https://sakana.ai/careers/member-of-technical-staff-llm-development/ Sakana Namazuを起点に拡大しているSakana AIのLLM開発を担うポジションです。下記のような挑戦が待っています。 ・事前

Claude's watermark probably doesn't work how you think. As the CTO of GPTZero, I'll explain how Anthropic, Google and OpenAI are building text watermarking in this brief explainer and whether it can b

Atomic Chat signed the Open Weights letter! We believe everyone should be able to run AI on their own device. When a model is open, thousands of teams fine-tune it, quantize it and build new tools on

AI21 joined @nvidia, @Microsoft , @a16z, and dozens of others in signing the Open Weights and American AI Leadership letter. We build agentic systems and agent optimization products on top of models,

Yes, open-source / open-weight models are important for a healthy AI ecosystem. That's how we can verify things, check claims, and keep up outside the closed labs. Plus, it gives us the freedom to run

Can’t tell if the PR reads more like a security incident or a product release… We suspected last week's cyberattack might have come from a frontier lab, given the sophistication of the agent. Turns ou

1/3 Best-of-N leaves $$ on the table by not accounting for variance in task difficulty. We built budget-aware execution: turn the dial on compute or speed, while keeping quality constant, to save cost

2/3 By building a reliable early stopping mechanism, we could apply cascading (save up to 44% compute by not running unnecessary rollouts ) or parallel execution (up to 25% faster by sparing wait time

3/3 Best-of-N is too flat for real world variance in task difficulty. Budget-aware execution turns compute & speed into dials you tune per workload. Full write-up here: https://www.ai21.com/blog/impro

3/3 We took 7 weak agents (ranks 7-13, none scoring >45) from the leaderboard & merged them into 1 report/task. Essentially boosting for deep research. The result: New #1 DRB II TotalScore of 64.38. F

4/5 Still came in ~$0.30 under Claude Code’s spend at a similar score. So we added a lightweight Test Agent that writes repo tests and filters failing patches, pushing our final result to 60.9% - surp

5/5 Takeaway: pipeline order is a hyperparameter. If you're already paying for parallel rollouts, reuse them - they're relevant context, not just candidate answers. Full write-up: [https://www.ai21.co

This post discusses how traditional caching mechanisms assume deterministic behavior (identical inputs producing identical outputs), an assumption that breaks down with large language models and espec

4/5 The fix: stop encoding execution order, encode graph position instead. Now you get two independent controls: experiment name (controls the key) and component code (controls the prompt). Change eit

5/5 What this unlocks: - Change one prompt, serve cached results for all other components. - A/B test any component against identical upstream outputs. - Run best-of-N inference without N branches col

4/5 Best-of-N: Run one agent config N times in parallel → select the best trajectory. Leverages LLMs’ non-determinism - but hinges on a good eval mechanism (we use an LLM-as-a-Judge). Can increase acc

5/5 Ensemble: Run multiple distinct agent configs in parallel → select the best trajectory. Leverages success of the portfolio vs single variants (see also: @/karpathy’s LLM Council). Can outperform b

A startup wants to pull magnesium from seawater without torching the environment. Another wants to take small language models to the podium for enterprise customers. Today @jason and @alex sat down wi

New #YAAP episode out now 🎙️ @yuvalinthedeep sits down with @mikegchambers from @awsdevelopers to unpack harness engineering and why it's the reason most agents never make it to production. 🎧 Listen/W

Take: stop searching for the “best model' or “best harness”. Model release → harness breaks → refind your optimal config. Most teams handle this with manual local search, with poor visibility into acc

@vasuman We've been saying this for months. The best compliment you can give an AI system is that it behaves exactly as expected. Every time. We built a campaign around it. It's called Boring AI: http

I am not sure I would agree with all of this, but the relationship between Anthropic and Claude is quite different than the relationship between other labs and their models. And that shows up in lots

4/5 But then we’re left with a search space problem. The possible agent configurations are infinite. And even if you manually optimize, what happens when a better model drops? Or your use case data up

5/5 Maestro automatically searches across this space (model ensembles, scaling strategies, execution policies), maps the Pareto frontier, and surfaces the full accuracy–cost–latency tradeoff surface.

Day 1 of #AIDevSF is live. 🙌 📍 Find us at booth 121 🎤 Catch our Chief Product Officer Or Dagan's session on Day 2: Efficiently Navigating the Agentic Action Space: Meta-Model Orchestration using AI21

Tune in to Or Dagan on the Life Self Mastery podcast with @rohitmal, unpacking why most enterprise AI projects fail before they ever reach production. You can also catch Or Dagan live at #AIDev26 this

Are you coming to @DeepLearningAI AI's AI Dev 26 x SF next week? Our Chief Product & Strategy Officer, Or Dagan, will be giving a talk diving into what our product and R&D teams are doing to automate

We just hit #1 on the @huggingface BrowseComp-Plus leaderboard. Best accuracy: 92.53%. Best recall: 88.79%. Lowest calibration error across all submissions. Built with @AI21Labs Maestro. https://huggi

I have found 4.7 great for design, reverted back to 4.6 extended for everything else Anyone else like this? Introducing Claude Design by Anthropic Labs: make prototypes, slides, and one-pagers by talk

1/5 When we saw our Reducer (=LLM judge component in Maestro, our agentic framework, that selects the best output from parallel agent runs) consistently picking gold patches, we were sure Claude Opus

3/5 An example: In instance psf__requests-1724, the gold fix is 2 lines. Our agent’s functional fix was 8 lines. The LLM judge rejected the correct 8-liner as 'messy' and 'redundant,' choosing a clean

4/5 We upgraded our original 3-line “be correct” prompt → a much more detailed prompt that enforced a hierarchy of constraints for correctness, regression safety, and minimality. Basically, get the LL

5/5 The takeaway: If your agent relies on an LLM judge for selection accuracy, measuring code quality isn’t enough; you need a measure of the model's inductive bias toward the 'fingerprint' of a gold

Catch Or Dagan, AI21's Chief Product & Strategy Officer, at #AIDev26 on April 29: 'An End to Manual Tinkering: Optimizing Accuracy, Cost & Latency in Real-World Agents'. We'll be at Booth 121. See you

'Both the utopia and apocalypse scenarios ignore AI's limitations.' Prof. Yoav Shoham, one of the world's leading artificial intelligence researchers and founder of @AI21Labs, discusses hype cycles, u

Scaling model size is hitting diminishing returns. The real gains are in orchestration. Our Co-Founder & Co-CEO @yshoham makes the case in a rare long-form profile by @Calcalistech today. The man tryi

AI Dev 26 brings the builders together. We'll be among them. Meet us there 👇 @DeepLearningAI @AndrewYNg 3,000+ developers. Two days. AI Dev 26 x San Francisco is where the people building AI come toge

If you're an AI/agent builder, it's so important that you don't overbuild and overcommit on a specific toolset and infrastructure. Frontier labs are shipping not just the models, but the harnesses and

We are open-sourcing V1.0 of our agentic robotics framework, ASPIRE, with full real robot skill discovery and self improvement capabilities! On the day Astra arrived, our lab switched the backend mode

@abuchanlife - training frontier models is expensive - distilling a frontier model is cheap - this market force alone tells us that frontier foundation models will be free/open. - there are others. ht

Indeed OpenAI was incompetent (and arrogant in thinking they knew it all); new cybersecurity software that they should have written themselves may help. But whether it’s enough to stop all of this rem

this this this The focus on AI causing extinction is going to get people killed. AI lab employees talk about how hard security is, not to point to some new solution, but to say security won’t work. On

I am pleased to see that OpenAI’s new model is roughly on par with Gemini Flash and Opus 4.8 on prompt injection risk. Nice work! Evaluating and naming other labs turns out to be a great way to encour