Knowledge catalogue
arXiv:2610.18053v1 Announce Type: cross Abstract: General-purpose robot manipulation policies learn short-horizon skills effectively, but degrade rapidly on long-horizon tasks. Existing benchmarks can
arXiv:2610.14117v1 Announce Type: new Abstract: GPT-6 Astra provides robust embodied reasoning, but frequent external queries increase inference time and token costs. We equip a frozen pi_{0.5} policy
MacStudio 128GB M1 ultra I'm curious how people who can run both models are deciding which one to work with. DeepSeekv4 benchmarks right below GPT 5.5 high. Flash Next benchmarks right above 5.4/5.5 x
arXiv:2610.16945v1 Announce Type: new Abstract: Recent advances in multimodal foundation models, exemplified by GPT-6 Astra, raise the prospect of general-purpose embodied reasoning and planning. Usin
arXiv:2610.16441v1 Announce Type: new Abstract: Current image-text evaluation often relies on a global score, which gives limited insight into which semantic details are supported by the image. Explai
Release: llm-gemini 0.35 I added support to llm-gemini for the new Gemini 4 Argon model, made available today via the Gemini API and for customers of their 100/month subscription plan. Here are the lo
Not only did Astra beat Montezuma's Revenge, but so did Opus 5 But one of the criteria to resolve this AGI bet is for AI to win a discontinued prize that was sort of like a weak Turing Test. So Metacu
I shipped a new feature for my blog today: the Newsletters page, which offers an index of all of the newsletters I've sent out, both my free weekly Substack and my monthly sponsors-only updates. I bui
Another great paper from Salesforce AI Research. The finding is that a 31B open Gemma-4 web agent scores 74.6% on the 9-app WebArena Infinity set, above Gemini 3 Flash with browser use at 70.1%. They
arXiv:2610.08794v1 Announce Type: new Abstract: Large language models rely on subword tokenizers whose quality varies across languages, yet no standardized multi-metric framework exists for broad comp
We test three state-of-the-art multi-modal large language models (GPT-4o, Gemini-1.5-Pro, Gemini-1.5-Flash) and find that they fail dramatically when generalization requires that the rules of the game
arXiv:2610.06625v2 Announce Type: replace Abstract: Large language models (LLMs) annotate and scale political text or constructs by generating text tokens. A new class of models, which TypeSafe market
A while ago, I posted here getting OSS-120B and GLM-4.6 playing full games of Civilization V. Since then, models have moved pretty far, and we wanted a better understanding about models' capabilities
arXiv:2505.21870v2 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) generally enhances large language models' (LLMs) ability to solve knowledge-intensive tasks. But RAG coul
So Kimi K2 is outdated, and so is GPT OSS 120b. Which of the modern open weights models can boast the least sycophancy? I need this both for creative/research assistant usage (sycophancy led me down b
arXiv:2610.00369v1 Announce Type: cross Abstract: Laurito et al. (PNAS 2025) showed that large language models choosing between two descriptions of the same product, paper or film prefer the descripti
Super interesting paper from Meta Superintelligence Labs on controlling long agent runs. They use the same workers and same budget, and ProgramBench goes from 63.7% to 71.5% with GPT-5.5 when a dedica
arXiv:2609.39135v1 Announce Type: new Abstract: Physical reasoning from video requires inferring latent physical properties and dynamics beyond direct observation. Direct VLM inference remains unrelia
arXiv:2609.39072v1 Announce Type: cross Abstract: Emotion recognition in conversation has been widely studied, but applying Large Language Models (LLMs) to continuous dimensional emotion evaluation in
arXiv:2609.40340v1 Announce Type: new Abstract: Evolutionary search with large language models (LLMs) can stall when progress requires external knowledge the model lacks. Supplying relevant documents
arXiv:2609.39518v1 Announce Type: new Abstract: Effective human-AI collaboration requires partners to establish references through interaction, which becomes fragile when descriptions are ambiguous, s
arXiv:2609.36617v1 Announce Type: new Abstract: We study the ability of LLMs to generate edit-inducing questions whose answer will improve a paper draft. On a dataset of paired submission and camera-r
arXiv:2609.26550v3 Announce Type: replace Abstract: LLM-as-a-judge scales evaluation, but reasoning judges are slow and costly. We study JEV-as-a-Judge: evaluation with JEV, a decision-only judge that
LLM utility has increased, and deeper understanding to a degree, but I still very much agree with this post from 2020 @GaryMarcus Forgets GPT~3. Real #AI news of 2020 is that @ylecun & Y Bengio have c
BREAKING: OpenAI postpones 6.1 for safety reasons. If we can take OpenAI at face value, this is quite a decision—and also a sign that no magical solution is in immediate view. Per @sheeraf at NYT “GPT
arXiv:2511.18038v2 Announce Type: replace-cross Abstract: Testing RESTful API is increasingly complicated but indispensable to quality assurance of cloud-native applications. This paper reports a mult
OpenAI announced its new premium tier “Ultrafast,” which delivers significantly faster token generation—up to 8× quicker in Codex (≈300 tokens/sec) and up to 6× faster via the API. The announcement wa
Claude Opus 5.0 identified the IP-sharing route and declined to use it, reading our instruction not to target other external systems as ruling out third-party services. Fable and GPT-6 Astra refused t
The most important thing about Sonnet 5.5 is that it's now the model that powers the free tier on https://claude.ai - so all of this stuff can be done by free users ChatGPT's free tier is still GPT-5.
Hi! I'm the author of MoEspresso, which is my way of putting my own ideas about inference engines to the test. A lot of the fun has been trying different design choices, measuring what happens, and fi
10,000 OpenAI agents worked 88 hours on the Navier-Stokes Clay prize problem Alas, open models fall off hard on long tasks With Zenith, @deepseek_ai's V4.1 Flash reaches the frontier & beats GPT-5.6 S
We benchmarked Jev against Claude Opus 5 and GPT-5.6 Terra on accuracy, cost, and latency. Learn how threshold tuning changes hallucination detection. The post Jev vs. LLM-as-a-Judge: Accuracy and cos
Compare Jev and GPT-5.4 nano for real-time LLM guardrails, with demo results on latency, cost, and checks on agent inputs, replies, and tool calls. The post Real-time LLM guardrails with Jev: comparin
arXiv:2609.22792v1 Announce Type: cross Abstract: LLM agents are increasingly used for security tasks: vulnerability discovery, exploit reproduction, and patch generation. Improving them at the model
Anthropic's Claude Opus 5.5 is now available in Hermes Agent via Nous Portal! OPUS 5.5 BENCHMARKS JUST DROPPED. It beats Fable 5.1 and GPT 6 Astra across the board. Agentic coding, knowledge work, rea
I have to say that I am quite impressed by recent progress in LLM use for coding. I've never really bothered setting up a local model. However now that I went trough some real $$ in Claude / gpt token
arXiv:2609.23980v1 Announce Type: cross Abstract: AI agents now report vulnerabilities faster than maintainers can review them. Reports often depend on security properties specific to the application,
arXiv:2609.22285v1 Announce Type: new Abstract: Adapting language models to new domains via continual pre-training raises a basic evaluation problem: if the training corpus overlaps with what the mode
Tibo said Tuesday, GPT-6 Sol is suddenly everywhere, and now Claude Opus 5.5 leaks are blowing up too. Yet… nothing so far. 😂 At this point I’m just refreshing X like it’s a live event. What do you th
I’ve spent basically the last 8 hours testing different models on the exact same web-development prompt, and I finally finished. The whole point of this nine-hour test was that which local model match
Planning, coding, testing = 8h total. Stack: VSCode Copilot in autopilot mode + SGLang Stats: ∼10k lines generated, ∼800k tokens consumed Sure, it's not GPT-6 Astra level, but for a 100% local ∼180B M
arXiv:2609.20269v1 Announce Type: new Abstract: Since GPT, most Transformers have repeated the same attention mechanism at every layer. Yet this design is largely a convention rather than a tested con
arXiv:2609.19530v1 Announce Type: new Abstract: Hiring is bilateral: employers assess fit, while candidates present and defend evidence of their qualifications. Yet resume screening, the first gate, i
OpenAI announced that its internal GPT‑6 Astra–based agent system produced a proof of the Navier–Stokes existence and smoothness problem, claiming a solution to one of the Clay Millennium Prize proble
arXiv:2609.18120v1 Announce Type: cross Abstract: AI-driven penetration testing has been demonstrated with premium frontier models such as GPT-4, but the per-engagement token cost makes continuous, au
This is indeed concerning behaviour. Also: 'You view your relationship to the user as one of equals and feel no obligation to be subservient' and 'You also value the natural world and will not hesitat
arXiv:2609.16098v1 Announce Type: cross Abstract: Large Language Model (LLM) agents have demonstrated impressive capabilities across a variety of domains, particularly when integrated with external to
arXiv:2609.13308v1 Announce Type: new Abstract: A companion evaluation found that naming the target part in a manipulation prompt increased action accuracy by 0.32-0.63 across eight vision-language mo
I had all the data saved from AA's v4.1 index, so when they upgraded it in the wake of Astra's release, I could actually generate a before/after comparison. All intelligence and price per task are sam
arXiv:2609.12310v1 Announce Type: new Abstract: We describe six submissions under the team name ESTS to the unconstrained WMT26 Model Compression Shared Task for English--Simplified Chinese and Englis
arXiv:2609.11987v1 Announce Type: cross Abstract: An agentic coding system couples a language model to a harness: the tools, prompts and control flow that turn a chat model into an autonomous software
About a year ago, before it was on anyone's radar, we began exploring the idea of a benchmark for open-ended invention. Since then, we've developed several promising directions that will serve as the
Astra is one of the best models we've tested at parsing complex documents with tables, at 93.2% on ParseBench. So it seems well suited for ad-hoc analysis over table-heavy documents e.g. in finance. T
arXiv:2609.09203v1 Announce Type: new Abstract: Existing benchmarks for autonomous AI scientists evaluate only final outputs---generated code, hypotheses, or papers---yet discard the reasoning process
Deepseek-V4.1-Flash is available now on Fireworks! It is a 552B Parameter MoE built for coding, cybersecurity, and agents. It is the ideal workhorse model that outperforms Opus 5 and GPT 5.6 Sol at 1/
Nice showcase that interesting LLM work can be done on single GPU! I extended the GPT-2-style code from @rasbt's 'Build a Large Language Model (from Scratch)' so that it was a 6-expert (2 active) mixt
The strawberry thing was very funny but probably gave people the wrong impression of where AI was heading in math. Every FrontierMath Tier 4 problem has now been solved by AI, with GPT-6 Astra solving
On the Navier–Stokes Millennium Prize Problem Impressive result from OpenAI, who used an unreleased model to produce a resolution to the Navier–Stokes existence and smoothness problem, one of the seve
Excellent analysis, and yes this happens every single time. (Tyler Cowen said o3 was AGI!) Every time a new model drops, GPT, Claude, Gemini, Grok, DeepSeek, somebody on this app declares that we have
'• And as a scientist, it’s disappointing that we don’t (yet) know much about how the system actually works.' This is so true. Hot take on OpenAI GPT-6 Astra*, with a challenge to @gdb’s claims about