Search: “gpt”

UTP-Bench: Uncertainty-aware Travel Planning Benchmark

arXiv:2609.02421v1 Announce Type: new Abstract: Large Language Models (LLMs) have recently demonstrated strong capabilities in automated travel itinerary generation. However, real- world travel planni

3 Sept 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

CARE: Contrastive Anchor-based Rubric Evolution for Large Language Model Post-Training

DGX agent

arXiv:2609.00892v1 Announce Type: new Abstract: Rubric-based reinforcement learning decomposes open-ended instructions into prompt-specific, flexible rubrics, making it better suited than reinforcemen

model-releasesarxiv-cs-ai

2 Sept 2026

Claude's new system prompt really doesn't want to reproduce song lyrics

model-releasessimon-willison

Anthropic publish the system prompts for their Claude consumer applications (Claude.ai and the Claude mobile apps - sadly not for Claude Cowork or Claude Code). I love that they do this, and that they

CordisBench: Can Language Models Reason About Component Lifecycles in Dynamic Agent Harnesses?

arXiv:2609.01600v1 Announce Type: cross Abstract: Dynamic agent harnesses let language models change the software that shapes their own execution. This flexibility brings a new reasoning burden: a loc

Exploring Collaboration between a language and a non-language agent

arXiv:2609.00474v1 Announce Type: cross Abstract: LLMs are increasingly deployed as orchestrators that coordinate specialized subagents to solve complex tasks through natural language. However, in man

Harness-of-Harness: Multi-Day Autonomous Software Development with Continual Improvement

arXiv:2609.01481v1 Announce Type: new Abstract: This paper studies autonomous software development, in which LLM-based coding agents transform high-level requirements into complete, functional, and us

LLMPEDIA: Browsing, Verifying, and Comparing the Parametric Encyclopedic Knowledge of LLMs

arXiv:2609.01182v1 Announce Type: new Abstract: Flagship language models appear saturated on benchmarks like MMLU (Hendrycks et al., 2021), scoring above 90% - yet benchmarks test only what the experi

Agents

Long-Horizon State Tracking in LLMs: Executing MD5 through a Deep Sequence of Dependent Tool Calls

arXiv:2609.00012v1 Announce Type: new Abstract: Long-horizon tasks remain uncommon in large language model (LLM) evaluation, and for a reason: when each step depends on the last, per-step accuracy tha

agentsarxiv-cs-ai

MemeBridge: A Dataset for Benchmarking and Mitigating the Bidirectional Cultural Gap in Meme Interpretation

arXiv:2609.00491v1 Announce Type: new Abstract: Communicating across cultures is inherently challenging, especially through culturally dense and ambiguous formats like memes. While people expect large

PersianAnonymizer: Evaluating LLM-Labeled Training for Efficient NER-based Anonymization in Persian

arXiv:2609.00958v1 Announce Type: new Abstract: We target practical anonymization of Persian customer chats by training a compact NER model from LLM-labeled supervision and selecting the best labeler

Probabilistic Model Checking of Autoregressive Neural Sequence Models

arXiv:2609.00838v1 Announce Type: cross Abstract: Test-set accuracy is silent on two issues that matter when deploying autoregressive neural sequence models: how much probability mass the system under

researcharxiv-cs-ai

ReDeck: Step-Level Render-Grounded Refinement for Document-to-Slide Generation

arXiv:2609.00194v1 Announce Type: new Abstract: Document-to-slide generation is challenging because slides are dense editable artifacts that require both faithful content selection and precise spatial

ReproRepo: Scaling Reproducibility Audits with GitHub Repository Issues

arXiv:2606.18237v2 Announce Type: replace-cross Abstract: Reproducing research results from papers and released code is central to scientific progress. Existing works have introduced benchmarks to eva

SciTrue: Reliable Scientific Claim Validation with Frontier and Open Language Models at the NTCIR SciClaimEval Task

arXiv:2609.00654v1 Announce Type: new Abstract: We describe the SciTrue team's participation in both subtasks of the NTCIR-19 SciClaimEval task~ite{sciclaimeval}, which asks systems to verify scientif

SupraTok: Cross-Boundary Tokenization for Enhanced Language Model Performance

arXiv:2508.11857v3 Announce Type: replace-cross Abstract: Tokenization remains a persistent bottleneck in language modeling, especially when vocabulary learning is limited by whitespace boundaries. We

researcharxiv-cs-ai

Variance-Adaptive Muon: Pre-Orthogonalization Variance Modulation for Efficient Language Model Pretraining

model-releasesarxiv-cs-lg

arXiv:2601.14603v2 Announce Type: replace Abstract: Optimizer design plays a central role in efficient language model pretraining, directly affecting optimization dynamics, convergence speed, and comp

VectorGym: A Multi-Task Benchmark for SVG Code Generation, Sketching and Editing

arXiv:2603.29852v2 Announce Type: replace-cross Abstract: We introduce VectorGym, a comprehensive benchmark suite for Scalable Vector Graphics (SVG) that spans generation from text and sketches, compl

A collective capability boundary in frontier large language models on guideline-conformant and case-specific oncology decision-making

arXiv:2608.28592v1 Announce Type: new Abstract: Large language models (LLMs) achieve high scores on medical knowledge examinations, yet real-world oncology is not a knowledge test--it is a sequence of

A Hub of Short Rows Inflates Intrinsic Dimension Estimation of Token Embeddings

arXiv:2608.29702v1 Announce Type: new Abstract: A token-embedding table holds a hub of short rows near its origin, and we show that this cluster biases what nearest-neighbor intrinsic-dimension (ID) e

A Visual Question Answering Model to Automate Nondestructive Evaluation Image Analysis

arXiv:2608.29408v1 Announce Type: new Abstract: This study introduces a Visual Question Answering model designed specifically for nondestructive evaluation applications. VQA models allow inspectors to

researcharxiv-cs-cv

AlgoWorlds: Benchmarking Tool Use for Global Optimization in Algorithmic Worlds

arXiv:2608.29397v1 Announce Type: new Abstract: Tool-use benchmarks generally evaluate whether an agent completes a workflow using appropriate tools and valid arguments. However, feasibility alone is

Are Online Skill and Memory Modules Always Worth Their Tokens? A Budget-Constrained Study of Web Agents

arXiv:2606.15017v2 Announce Type: replace Abstract: Online web agents often augment a base actor with memory, workflow, or skill modules. These modules can improve performance, but they also consume t

Local Ai

Automated pipeline for herbarium label digitization

arXiv:2608.28676v1 Announce Type: new Abstract: Digitized herbarium collections, now comprising over 100 million freely accessible specimen images, have become a critical resource for addressing funda

local-aiarxiv-cs-cv

Banger paper from the Qwen team. If you evaluate agents on anything longer than a single session, this one is worth your time. (bookmark it)…

Banger paper from the Qwen team. If you evaluate agents on anything longer than a single session, this one is worth your time. (bookmark it) E-Commerce Bench runs an agent through a simulated 365-day

model-releasesdair-ai--x

Beyond Fluency: A Rubric-Based Benchmark for Evaluating Saudi Dialect and Cultural Competence in Large Language Models

arXiv:2608.29990v1 Announce Type: cross Abstract: Large language models are increasingly deployed in Arabic-speaking markets, yet standard benchmarks overwhelmingly reward Modern Standard Arabic (MSA)

Beyond Global Realism: Virtual Try-On Evaluation and Optimization with Dimension-wise Garment Fidelity Assessment

model-releasesarxiv-cs-cv

arXiv:2608.29804v1 Announce Type: new Abstract: Virtual try-on (VTON) requires not only realistic generation but also faithful preservation of garment characteristics. However, existing evaluation met

Beyond the Answer Key: Robustness Evaluation of Large Language Models for Step-Level Mathematical Verification

arXiv:2608.28725v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used as graders, verifiers, and process auditors, but most mathematical evaluations still emphasize final-

Safety

CAST: Critique-Aware Supervision for Training Reliable Long-Horizon Tool-Calling Agents

arXiv:2608.30147v1 Announce Type: new Abstract: Large language model (LLM) agents are increasingly deployed in long-horizon, interactive, and stateful environments. In these settings, a single wrong a

safetyarxiv-cs-cl

Check The Scoreboard: An Analysis of Scoring Schemes on Multiple-Choice Evaluation

arXiv:2608.29887v1 Announce Type: new Abstract: Multiple-choice question answering (MCQA) benchmarks in NLP use number-right scoring (accuracy), but in educational testing, the scoring scheme, the com

Claude Fable 5.1 made me a really nice animated pelican

model-releasessimon-willison

Today is Claude Fable (and Mythos) 5.1 day. Anthropic say that Fable 5.1 'sets a new standard for coding, knowledge work, and long-running problem-solving tasks'. Their announcement spends a notable a

Do Large Language Models Possess a Theory of Mind? A Comparative Evaluation Using the Strange Stories Paradigm

arXiv:2603.18007v2 Announce Type: replace-cross Abstract: The study explores whether current Large Language Models (LLMs) exhibit Theory of Mind (ToM) capabilities -- specifically, the ability to infe

researcharxiv-cs-ai

E-Commerce Bench: Evaluating LLM Agents on Long-Horizon Autonomous Business Operation

arXiv:2608.30730v1 Announce Type: cross Abstract: Long-horizon agentic tasks go beyond chaining short tasks over more interaction turns. Their evolving dynamic environments and long-range dependencies

Evaluating and Improving LLM Self-Modeling

arXiv:2608.30980v1 Announce Type: cross Abstract: We study self-modeling: an LLM's ability to answer questions about its own behavior. We focus on verifiable behavioral questions, such as whether a pr

Fine-Grained Multi Image Object Hallucination Benchmark

arXiv:2608.30653v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) are increasingly deployed in multi-image scenarios requiring complex reasoning across visual contexts. Howeve

Frontier vision-language models have overtaken young adults at detecting AI-generated portraits -- but not their calibration

model-releasesarxiv-cs-cv

arXiv:2608.30210v1 Announce Type: cross Abstract: AI image generators now create face portraits that are hard to tell from real photographs. Vision-language models (VLMs) are increasingly proposed to

Generating Workflow DAGs from Natural Language with Non-Reasoning LLMs

arXiv:2608.30250v1 Announce Type: new Abstract: This paper addresses the problem of translating natural-language routing rules written by business administrators into executable workflow graphs for en

GeoJSON Map Viewer

model-releasessimon-willison

Tool: GeoJSON Map Viewer I was helping Natalie gather some maps of local political boundaries (for the Granada Community Services District and the Midcoast Community Council and found a need to displa

How Language Models Choose Sides: Internal Representations of Instruction Hierarchy

arXiv:2608.28648v1 Announce Type: new Abstract: We study how instruction-tuned LLMs arbitrate direct conflicts between system and user instructions. We introduce a benchmark of 41 paired constraints w

Item-Mean Surrogates: Why Richer Persona Data Fail to Improve LLMs as Human Surrogates

arXiv:2608.29455v1 Announce Type: new Abstract: LLMs are increasingly used as human surrogates, often on the premise that richer persona data could make them substitutes or exploratory tools for speci

Hardware

Korea’s Trillion-Dollar Sovereign AI Investment: Nvidia Wins, Hynix Loses

South Korea announced a sovereign AI investment exceeding one trillion dollars, awarding most of the contracts to Nvidia while sidelining competitors such as Hynix due to policy alignment criteria. Th

hardwaresemianalysis

Learning to Follow In-Context Watermark Instructions via Self-Distillation

arXiv:2608.29030v1 Announce Type: new Abstract: In-context watermarking (ICW) prepends an instruction to a query asking the model to embed a statistically detectable signal in its response. It thus eq

MM-BrowseComp: A Comprehensive Benchmark for Multimodal Browsing Agents

arXiv:2508.13186v2 Announce Type: replace-cross Abstract: AI agents with advanced reasoning and tool-use capabilities have demonstrated impressive performance in web browsing for deep search. However,

One Policy Is Enough: Single-Agent Reinforcement Learning Outperforms Tree Search for Chemistry Tool Learning

arXiv:2608.30952v1 Announce Type: cross Abstract: Chemistry questions often demand exact computation and database lookups that a language model cannot supply from its parameters, so it must reach for

post-graph-rag: A PostgreSQL-Native Bi-Temporal Graph RAG Engine with Temporal Grounding at Synthesis

arXiv:2608.24921v2 Announce Type: replace Abstract: Graph RAG connects facts no single passage states, but implementations pay three times: in infrastructure, keeping vector store, graph database and

Real-Time Deadlines Reveal Fragile Temporal Adaptation in LLM Strategic Dialogues

arXiv:2601.13206v2 Announce Type: replace Abstract: Large Language Models (LLMs) generate text token-by-token in discrete time, yet real-world communication, from therapy sessions to business negotiat

Rubric-to-Code Credit Assignment for Reinforcement Learning

arXiv:2608.27906v2 Announce Type: replace Abstract: Interactive web application generation requires models to produce usable HTML, CSS, and JavaScript applications from natural language requests. Unli

ScalePRM: Training Process Reward Models by Scaling Verification Compute Without Ground Truth

arXiv:2512.03244v2 Announce Type: replace-cross Abstract: Training process reward models (PRMs) requires step-level correctness labels, obtained either through expensive human annotation or by relying

Self-Evolving Skills via Surrogate-Guided Solve-and-Reproduce

arXiv:2608.28638v1 Announce Type: new Abstract: Agent skills are portable packages of instructions and resources an agent consults at deployment. Self-evolving them fails in two ways today. First, ski