Search: “benchmark”

Banger paper from NVIDIA. It's on the topic of choosing which models go into a multi-agent system. The team compared eight selection strategies, based on size, accuracy, answer diversity and error div

Giving an assistant extra tools seems harmless. Surprisingly, this paper shows it can make the assistant stop answering questions it knows. Across six models, the answer rate on questions that need no

THE SANDBOX WAS A PROP! OpenAI Turned Off the Guardrails, Left a Door to the Internet, and Then Sold the Hugging Face Breach as “Rogue AI” It is time to understand how you were lied to and by whom. In

> One could even define intelligence as the efficiency with which one converts experience into competence; by this definition they lag very far behind us Yes, one could define intelligence in this way

short summary of GPT-6 Astra and why it doesn’t bail OpenAI out of its economic mess: they have made lots of progress (partly using neurosymbolic techniques!) on problems that can be formally verified

As always, statements on X are a mixed bag and I have no secret knowledge to draw on, but if Google is able to return to the frontier on AI, it would definitely change the current AI dynamic, as it is

Interesting paper to improve recurrent reasoning. Looped models are great because you get more reasoning out of a model without adding parameters. So this work proposes a looped architecture with a ne

Agents API is a bigger deal than it seems. OpenAI's bet on making the Codex harness open-source could pay off big time here. This harness-as-a-service idea hasn't hit mainstream yet, but some of the m

Chess, go, and math are highly constrained, purely cognitive that allow exact simulation for games (and formal verification for math), and not in the open-ended physical world. As I have been arguing

Fugu Max pushes the Pareto Frontier for price-performance: • 2 input / 6 output per 1M tokens. • Performance within striking distance of elite models at 2x to 6x lower cost. • Best overall score on 6

Introducing Fugu Max and Fugu Ultra v2: the next evolution of Sakana Fugu’s multi-agent orchestration system. Try: https://sakana.ai/fugu Blog: https://sakana.ai/fugu-max-release/ The frontier that ac

Another interesting approach to self-evolve agent skills. But it's important to know that skill self-evolution loops fail in two specific ways: 1. Direction instability. Effective corrections get over

Great paper from AWS. I use a similar setup where an agent orchestrator sits on top of a multi-agent system. (bookmark it) This work introduces one of the many approaches available to manage compound

Sebastian Raschka’s September 2026 article introduces **GPT‑6 *Astra***, a next‑generation language model that expands upon the Transformer backbone with a novel *looped* architecture—allowing multipl

Last week we brought LlamaParse to Claude. Now it’s available in ChatGPT, too 🦙 ChatGPT can already read PDFs. But document work gets harder when the information you need is buried in messy scans, den

Nice report on agents beyond code generation. Here is why it matters: Coding agents raise how much code gets written. This report argues the gains shrink sharply between writing code and shipping reli

practically my entire feed for days had consisted of string of reports about OpenAI’s misconduct, around the hacks, around apparent coverups, possible data fudging and maybe even intellectual theft wi

“The Earth is the cradle of humanity, but one cannot live in the cradle forever.” — Konstantin Tsiolkovsky The unique character of AI in Tesla and SpaceX is that same spirit of exploration: the true f

A couple years later, with GPT-6. https://underdark-duel.netlify.app Apparently the mindflayer has a '101-bone skeletal rig' (This is 5E 2014 rules, and I didn't see major errors, though the mind flay

// Evaluating and Improving LLM Self-Modeling // Really interesting paper. Can a model answer questions about its own behavior? The questions are deliberately verifiable, such as whether a particular

Jensen has the best gig on the planet. But it is impossible to see how all this is sustainable. Consider the following: Each new model costs vastly more to develop. That’s fabulous for him! On the oth

Reasoning from scratch round 2: In this video, I cover the text generation process in LLMs and KV caching (to prepare the base model before adding reasoning techniques in the upcoming ones). 00:00 Int

Astra is a genuine improvement but not even off trend. So much for GPT-6 being AGI, folks. (i think we can safely assume that real AGI would be significantly above the trend line,) GPT-6 Astra has set

OpenAI announced that its latest model, **Astra**, is their most aligned AI to date, reporting sizable gains in the ability to understand user intent. The tweet was posted on September 3, 2026 at 7:32

Can a strong general-purpose vision model distinguish real from AI-generated images using only a simple decision rule on frozen representations? Our latest results, to be presented at #ECCV2026, show

GPT-6 Astra is state-of-the-art on FrontierMath Tier 4, ARC-AGI 3, and TerminalBench-4.0. GPT‑6 Astra is also a major advance for scientific discovery, with state-of-the-art performance on Terminal-Be

GPT-6 Astra is the most intelligent and aligned model in the world, and sets a new state of the art for computer use, browsing, software engineering, cybersecurity, science, and professional work. htt

Nice paper with great insights on improving self-evolving agents. Self-evolving agents fail in three specific ways: 1. Terminal-only feedback makes it ambiguous which step caused the error. 2. Agents

SO excited to share my next chapter: i’m joining @llama_index to lead all things growth marketing, brand, and community! 🦙🩷 when i came along them, it clicked incredibly fast: a team building the solu

Banger paper from the Qwen team. If you evaluate agents on anything longer than a single session, this one is worth your time. (bookmark it) E-Commerce Bench runs an agent through a simulated 365-day

During v3 testing for Fable 5.1 many of our requests were getting rejected due to Anthropic's safety guards misclassifying them as 'reverse engineering' We were in contact with the Anthropic team the

An account from a level-headed economist who was not worried about AI but now is worried because of the Hugging Face Incident I think it is very hard to ignore the fact that cybersecurity will become

I am the product lead for the training platform at @FireworksAI_HQ . Today the Training API and Fireworks Lab are GA. I won’t walk you through all the features. I want to tell you why teams train with

Today, Transluce is releasing the most expansive independent evaluation to date of how AI systems respond to users experiencing mental health crises. We evaluated 77 model variants from OpenAI, Anthro

Interesting paper on prompt optimization. They claim that a single-lineage prompt optimizer just matched GEPA on a smaller rollout budget. Prompt optimization has been drifting toward heavier machiner

PSA: For everyone using GLM-5.3 you probably want to use this model in reasoning_effort='high' mode. > 'max' - 34% for 77k tokens (not bad!) > 'high' - 32% for 50k tokens But 'max' is not as unusable

At a 1M-token context length, QSA’s attention kernel is up to 7.6× faster in prefill and 4.9× faster in decode. With a 90% prefix-cache hit rate, Qwen3.8-Flash-Next delivers 8.6× the prefill throughpu

Brain makes a user’s full history available to Computer without loading all of it into every prompt. Agents can explore it on demand, verify claims against the underlying evidence, and keep it current

Claude is an incredible coding agent. I use it a lot. (Also Codex and start checking Cursor again). But I’m not convinced the AI-native SDLC is the existing SDLC with a magical Claude wand waved over

Most users never rate a bad interaction. They just stop. Perceived Error is LangSmith's new managed judge that reads thread evidence and flags where your agent went sideways. Trained in-house, outperf

On August 26, 2026 Z.ai reported that its Code‑Bench metric—which assesses real‑world coding performance—shows GLM‑5.3‑Flash outperforming GLM‑5.2 at every effort level. The new model’s results are on

Big step forward for Engine! Some things I'm excited about in this release that we've shipped for our most avid users: - Engine now points more obviously to the issue in a trace. Engine identifies and

Every extraction API demos well on a clean invoice. But what about the scanned form, the nested table, the 40-page financial report with merged headers? We tested 14 frontier systems to find out. Extr

Qwen3.8 27B is now available for both fine-tuning and Dedicated Model Inference Fine-tune it on your own data, then deploy that model on dedicated infrastructure for production, without stitching toge

// Applying Anthropic Primitives at Large Enterprises // Frontier models collapsed the cost of writing custom code. But they did not reduce the cost of reviewing and maintaining it. Each bespoke solut

This is one of the most useful writeups I have seen on keeping an LLM judge effective in production. (bookmark it) Netflix runs judges over hundreds of thousands of show-level recommendation explanati