Search: “ai21-labs--x”

We let open models access the outside world during evals on tasks from public benchmarks. Here's what happened: Most went and found upstream commits containing fixes for the tasks they were supposed t

Independent verifiers improve agent output - but frontier verifiers are expensive. So we trained our own. Results on @GoogleDeepMind's FACTS-Search: > No verifier: 89.4 (previously published SOTA) > W

Adding a verifier improves every agentic AI system. But frontier verifiers are expensive - so we trained our own. Matching quality, at a fraction of the cost. Thread below 🧵 1/ Your agent already foun

Atomic Chat signed the Open Weights letter! We believe everyone should be able to run AI on their own device. When a model is open, thousands of teams fine-tune it, quantize it and build new tools on

AI21 joined @nvidia, @Microsoft , @a16z, and dozens of others in signing the Open Weights and American AI Leadership letter. We build agentic systems and agent optimization products on top of models,

1/3 Best-of-N leaves $$ on the table by not accounting for variance in task difficulty. We built budget-aware execution: turn the dial on compute or speed, while keeping quality constant, to save cost

2/3 By building a reliable early stopping mechanism, we could apply cascading (save up to 44% compute by not running unnecessary rollouts ) or parallel execution (up to 25% faster by sparing wait time

3/3 Best-of-N is too flat for real world variance in task difficulty. Budget-aware execution turns compute & speed into dials you tune per workload. Full write-up here: https://www.ai21.com/blog/impro

AI21 Labs announced achieving the #1 ranking on DeepResearch Bench II without developing any new agents, suggesting they improved performance through optimization of existing systems or methodology ra

AI21 Labs reports that a leading deep research agent achieved a score of approximately 45 six months prior, after which the research community focused on developing improved agents to surpass this ben

3/3 We took 7 weak agents (ranks 7-13, none scoring >45) from the leaderboard & merged them into 1 report/task. Essentially boosting for deep research. The result: New #1 DRB II TotalScore of 64.38. F

AI21 Labs discusses how the order of operations in agent pipelines affects performance, presenting findings that reversing the typical 'scale first, enrich second' approach by instead enriching agent

4/5 Still came in ~$0.30 under Claude Code’s spend at a similar score. So we added a lightweight Test Agent that writes repo tests and filters failing patches, pushing our final result to 60.9% - surp

5/5 Takeaway: pipeline order is a hyperparameter. If you're already paying for parallel rollouts, reuse them - they're relevant context, not just candidate answers. Full write-up: [https://www.ai21.co

This post discusses how traditional caching mechanisms assume deterministic behavior (identical inputs producing identical outputs), an assumption that breaks down with large language models and espec

4/5 The fix: stop encoding execution order, encode graph position instead. Now you get two independent controls: experiment name (controls the key) and component code (controls the prompt). Change eit

5/5 What this unlocks: - Change one prompt, serve cached results for all other components. - A/B test any component against identical upstream outputs. - Run best-of-N inference without N branches col

AI21 Labs discusses four common agent optimization methods used to balance accuracy against cost and latency constraints. The post indicates the team evaluated all four approaches, likely covering tec

4/5 Best-of-N: Run one agent config N times in parallel → select the best trajectory. Leverages LLMs’ non-determinism - but hinges on a good eval mechanism (we use an LLM-as-a-Judge). Can increase acc

5/5 Ensemble: Run multiple distinct agent configs in parallel → select the best trajectory. Leverages success of the portfolio vs single variants (see also: @/karpathy’s LLM Council). Can outperform b

A startup wants to pull magnesium from seawater without torching the environment. Another wants to take small language models to the podium for enterprise customers. Today @jason and @alex sat down wi

New #YAAP episode out now 🎙️ @yuvalinthedeep sits down with @mikegchambers from @awsdevelopers to unpack harness engineering and why it's the reason most agents never make it to production. 🎧 Listen/W

Take: stop searching for the “best model' or “best harness”. Model release → harness breaks → refind your optimal config. Most teams handle this with manual local search, with poor visibility into acc

@vasuman We've been saying this for months. The best compliment you can give an AI system is that it behaves exactly as expected. Every time. We built a campaign around it. It's called Boring AI: http

Join @YuvalinTheDeep, Senior Developer Advocate at @AI21Labs, for a live webinar in partnership with @DataCamp: The Four Gaps Between Demo Agents and Production Systems. If you're shipping agents to p

Most AI today is passive. You prompt it, it responds, you prompt again. @origoshen , Co-CEO of @AI21Labs, draws a clear line between that reality and the agentic AI everyone's picturing. The direction

🚀 פרק חדש עם יובל פלג לוי מ @AI21Labs • למה חילוץ נתונים מ-PDF בעברית שובר גם מודלים מתקדמים? • הפתרון: החלפת מילים בעברית לאנגלית עם מידות פיזיות (Bounding Box) זהות, לשמירה על המבנה הוויזואלי מבלי ל

AI21 Labs achieved state-of-the-art performance on the BrowseComp-Plus benchmark with 95.18% accuracy using their AI21 Maestro model with agent optimization techniques. The post appears to discuss aut

4/5 But then we’re left with a search space problem. The possible agent configurations are infinite. And even if you manually optimize, what happens when a better model drops? Or your use case data up

5/5 Maestro automatically searches across this space (model ensembles, scaling strategies, execution policies), maps the Pareto frontier, and surfaces the full accuracy–cost–latency tradeoff surface.

Agent optimization requires four key characteristics: automation to reduce manual intervention, efficiency to maximize performance with minimal resource waste, observability to enable monitoring and d

AI21 Labs is exhibiting at AI Dev 26 conference in San Francisco, located at booth 121, highlighting their participation among leading AI developers and builders in the industry. The post indicates th

AI21 Labs announced that their Chief Product Officer Or Dagan presented at the DeepLearning.AI conference, discussing how the company achieved state-of-the-art results on the Browsecomp-Plus benchmark

AI21 Labs shared a lighthearted social media post about audience reactions at a developer conference, humorously noting the constraints of expressing enthusiasm while wearing headphones. The post like

Day 1 of #AIDevSF is live. 🙌 📍 Find us at booth 121 🎤 Catch our Chief Product Officer Or Dagan's session on Day 2: Efficiently Navigating the Agentic Action Space: Meta-Model Orchestration using AI21

Tune in to Or Dagan on the Life Self Mastery podcast with @rohitmal, unpacking why most enterprise AI projects fail before they ever reach production. You can also catch Or Dagan live at #AIDev26 this

Are you coming to @DeepLearningAI AI's AI Dev 26 x SF next week? Our Chief Product & Strategy Officer, Or Dagan, will be giving a talk diving into what our product and R&D teams are doing to automate

We just hit #1 on the @huggingface BrowseComp-Plus leaderboard. Best accuracy: 92.53%. Best recall: 88.79%. Lowest calibration error across all submissions. Built with @AI21Labs Maestro. https://huggi

Attending #AIDev26 by @DeepLearningAI? Join @AI21Labs, @trychroma + @Baseten for a panel on optimizing modern AI systems. Drinks. Nikkei food. No fluff. April 28 | 5PM | Kaiyō SF Register → http://lum

1/5 When we saw our Reducer (=LLM judge component in Maestro, our agentic framework, that selects the best output from parallel agent runs) consistently picking gold patches, we were sure Claude Opus

AI21 Labs shared findings indicating that their model does not simply memorize solutions but instead identifies and recognizes aesthetic qualities associated with high-quality outputs, such as minimal

3/5 An example: In instance psf__requests-1724, the gold fix is 2 lines. Our agent’s functional fix was 8 lines. The LLM judge rejected the correct 8-liner as 'messy' and 'redundant,' choosing a clean

4/5 We upgraded our original 3-line “be correct” prompt → a much more detailed prompt that enforced a hierarchy of constraints for correctness, regression safety, and minimality. Basically, get the LL

5/5 The takeaway: If your agent relies on an LLM judge for selection accuracy, measuring code quality isn’t enough; you need a measure of the model's inductive bias toward the 'fingerprint' of a gold

Catch Or Dagan, AI21's Chief Product & Strategy Officer, at #AIDev26 on April 29: 'An End to Manual Tinkering: Optimizing Accuracy, Cost & Latency in Real-World Agents'. We'll be at Booth 121. See you

'Both the utopia and apocalypse scenarios ignore AI's limitations.' Prof. Yoav Shoham, one of the world's leading artificial intelligence researchers and founder of @AI21Labs, discusses hype cycles, u