Search: “gpt”

UTP-Bench: Uncertainty-aware Travel Planning Benchmark

arXiv:2609.02421v1 Announce Type: new Abstract: Large Language Models (LLMs) have recently demonstrated strong capabilities in automated travel itinerary generation. However, real- world travel planni

3 Sept 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

CARE: Contrastive Anchor-based Rubric Evolution for Large Language Model Post-Training

DGX agent

arXiv:2609.00892v1 Announce Type: new Abstract: Rubric-based reinforcement learning decomposes open-ended instructions into prompt-specific, flexible rubrics, making it better suited than reinforcemen

model-releasesarxiv-cs-ai

2 Sept 2026

CordisBench: Can Language Models Reason About Component Lifecycles in Dynamic Agent Harnesses?

arXiv:2609.01600v1 Announce Type: cross Abstract: Dynamic agent harnesses let language models change the software that shapes their own execution. This flexibility brings a new reasoning burden: a loc

Exploring Collaboration between a language and a non-language agent

arXiv:2609.00474v1 Announce Type: cross Abstract: LLMs are increasingly deployed as orchestrators that coordinate specialized subagents to solve complex tasks through natural language. However, in man

Harness-of-Harness: Multi-Day Autonomous Software Development with Continual Improvement

arXiv:2609.01481v1 Announce Type: new Abstract: This paper studies autonomous software development, in which LLM-based coding agents transform high-level requirements into complete, functional, and us

LLMPEDIA: Browsing, Verifying, and Comparing the Parametric Encyclopedic Knowledge of LLMs

arXiv:2609.01182v1 Announce Type: new Abstract: Flagship language models appear saturated on benchmarks like MMLU (Hendrycks et al., 2021), scoring above 90% - yet benchmarks test only what the experi

Agents

Long-Horizon State Tracking in LLMs: Executing MD5 through a Deep Sequence of Dependent Tool Calls

arXiv:2609.00012v1 Announce Type: new Abstract: Long-horizon tasks remain uncommon in large language model (LLM) evaluation, and for a reason: when each step depends on the last, per-step accuracy tha

agentsarxiv-cs-ai

MemeBridge: A Dataset for Benchmarking and Mitigating the Bidirectional Cultural Gap in Meme Interpretation

arXiv:2609.00491v1 Announce Type: new Abstract: Communicating across cultures is inherently challenging, especially through culturally dense and ambiguous formats like memes. While people expect large

PersianAnonymizer: Evaluating LLM-Labeled Training for Efficient NER-based Anonymization in Persian

arXiv:2609.00958v1 Announce Type: new Abstract: We target practical anonymization of Persian customer chats by training a compact NER model from LLM-labeled supervision and selecting the best labeler

Probabilistic Model Checking of Autoregressive Neural Sequence Models

arXiv:2609.00838v1 Announce Type: cross Abstract: Test-set accuracy is silent on two issues that matter when deploying autoregressive neural sequence models: how much probability mass the system under

researcharxiv-cs-ai

ReDeck: Step-Level Render-Grounded Refinement for Document-to-Slide Generation

arXiv:2609.00194v1 Announce Type: new Abstract: Document-to-slide generation is challenging because slides are dense editable artifacts that require both faithful content selection and precise spatial

ReproRepo: Scaling Reproducibility Audits with GitHub Repository Issues

arXiv:2606.18237v2 Announce Type: replace-cross Abstract: Reproducing research results from papers and released code is central to scientific progress. Existing works have introduced benchmarks to eva

SciTrue: Reliable Scientific Claim Validation with Frontier and Open Language Models at the NTCIR SciClaimEval Task

arXiv:2609.00654v1 Announce Type: new Abstract: We describe the SciTrue team's participation in both subtasks of the NTCIR-19 SciClaimEval task~ite{sciclaimeval}, which asks systems to verify scientif

SupraTok: Cross-Boundary Tokenization for Enhanced Language Model Performance

arXiv:2508.11857v3 Announce Type: replace-cross Abstract: Tokenization remains a persistent bottleneck in language modeling, especially when vocabulary learning is limited by whitespace boundaries. We

researcharxiv-cs-ai

Variance-Adaptive Muon: Pre-Orthogonalization Variance Modulation for Efficient Language Model Pretraining

model-releasesarxiv-cs-lg

arXiv:2601.14603v2 Announce Type: replace Abstract: Optimizer design plays a central role in efficient language model pretraining, directly affecting optimization dynamics, convergence speed, and comp

VectorGym: A Multi-Task Benchmark for SVG Code Generation, Sketching and Editing

arXiv:2603.29852v2 Announce Type: replace-cross Abstract: We introduce VectorGym, a comprehensive benchmark suite for Scalable Vector Graphics (SVG) that spans generation from text and sketches, compl

A collective capability boundary in frontier large language models on guideline-conformant and case-specific oncology decision-making

arXiv:2608.28592v1 Announce Type: new Abstract: Large language models (LLMs) achieve high scores on medical knowledge examinations, yet real-world oncology is not a knowledge test--it is a sequence of

A Hub of Short Rows Inflates Intrinsic Dimension Estimation of Token Embeddings

arXiv:2608.29702v1 Announce Type: new Abstract: A token-embedding table holds a hub of short rows near its origin, and we show that this cluster biases what nearest-neighbor intrinsic-dimension (ID) e

A Visual Question Answering Model to Automate Nondestructive Evaluation Image Analysis

arXiv:2608.29408v1 Announce Type: new Abstract: This study introduces a Visual Question Answering model designed specifically for nondestructive evaluation applications. VQA models allow inspectors to

researcharxiv-cs-cv

AlgoWorlds: Benchmarking Tool Use for Global Optimization in Algorithmic Worlds

arXiv:2608.29397v1 Announce Type: new Abstract: Tool-use benchmarks generally evaluate whether an agent completes a workflow using appropriate tools and valid arguments. However, feasibility alone is

Are Online Skill and Memory Modules Always Worth Their Tokens? A Budget-Constrained Study of Web Agents

arXiv:2606.15017v2 Announce Type: replace Abstract: Online web agents often augment a base actor with memory, workflow, or skill modules. These modules can improve performance, but they also consume t

Local Ai

Automated pipeline for herbarium label digitization

arXiv:2608.28676v1 Announce Type: new Abstract: Digitized herbarium collections, now comprising over 100 million freely accessible specimen images, have become a critical resource for addressing funda

local-aiarxiv-cs-cv

Beyond Fluency: A Rubric-Based Benchmark for Evaluating Saudi Dialect and Cultural Competence in Large Language Models

arXiv:2608.29990v1 Announce Type: cross Abstract: Large language models are increasingly deployed in Arabic-speaking markets, yet standard benchmarks overwhelmingly reward Modern Standard Arabic (MSA)

Beyond Global Realism: Virtual Try-On Evaluation and Optimization with Dimension-wise Garment Fidelity Assessment

model-releasesarxiv-cs-cv

arXiv:2608.29804v1 Announce Type: new Abstract: Virtual try-on (VTON) requires not only realistic generation but also faithful preservation of garment characteristics. However, existing evaluation met

Beyond the Answer Key: Robustness Evaluation of Large Language Models for Step-Level Mathematical Verification

arXiv:2608.28725v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used as graders, verifiers, and process auditors, but most mathematical evaluations still emphasize final-

Safety

CAST: Critique-Aware Supervision for Training Reliable Long-Horizon Tool-Calling Agents

arXiv:2608.30147v1 Announce Type: new Abstract: Large language model (LLM) agents are increasingly deployed in long-horizon, interactive, and stateful environments. In these settings, a single wrong a

safetyarxiv-cs-cl

Check The Scoreboard: An Analysis of Scoring Schemes on Multiple-Choice Evaluation

arXiv:2608.29887v1 Announce Type: new Abstract: Multiple-choice question answering (MCQA) benchmarks in NLP use number-right scoring (accuracy), but in educational testing, the scoring scheme, the com

Do Large Language Models Possess a Theory of Mind? A Comparative Evaluation Using the Strange Stories Paradigm

arXiv:2603.18007v2 Announce Type: replace-cross Abstract: The study explores whether current Large Language Models (LLMs) exhibit Theory of Mind (ToM) capabilities -- specifically, the ability to infe

researcharxiv-cs-ai

E-Commerce Bench: Evaluating LLM Agents on Long-Horizon Autonomous Business Operation

arXiv:2608.30730v1 Announce Type: cross Abstract: Long-horizon agentic tasks go beyond chaining short tasks over more interaction turns. Their evolving dynamic environments and long-range dependencies

Evaluating and Improving LLM Self-Modeling

arXiv:2608.30980v1 Announce Type: cross Abstract: We study self-modeling: an LLM's ability to answer questions about its own behavior. We focus on verifiable behavioral questions, such as whether a pr

Fine-Grained Multi Image Object Hallucination Benchmark

arXiv:2608.30653v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) are increasingly deployed in multi-image scenarios requiring complex reasoning across visual contexts. Howeve

Frontier vision-language models have overtaken young adults at detecting AI-generated portraits -- but not their calibration

model-releasesarxiv-cs-cv

arXiv:2608.30210v1 Announce Type: cross Abstract: AI image generators now create face portraits that are hard to tell from real photographs. Vision-language models (VLMs) are increasingly proposed to

Generating Workflow DAGs from Natural Language with Non-Reasoning LLMs

arXiv:2608.30250v1 Announce Type: new Abstract: This paper addresses the problem of translating natural-language routing rules written by business administrators into executable workflow graphs for en

How Language Models Choose Sides: Internal Representations of Instruction Hierarchy

arXiv:2608.28648v1 Announce Type: new Abstract: We study how instruction-tuned LLMs arbitrate direct conflicts between system and user instructions. We introduce a benchmark of 41 paired constraints w

Item-Mean Surrogates: Why Richer Persona Data Fail to Improve LLMs as Human Surrogates

arXiv:2608.29455v1 Announce Type: new Abstract: LLMs are increasingly used as human surrogates, often on the premise that richer persona data could make them substitutes or exploratory tools for speci

Learning to Follow In-Context Watermark Instructions via Self-Distillation

arXiv:2608.29030v1 Announce Type: new Abstract: In-context watermarking (ICW) prepends an instruction to a query asking the model to embed a statistically detectable signal in its response. It thus eq

MM-BrowseComp: A Comprehensive Benchmark for Multimodal Browsing Agents

arXiv:2508.13186v2 Announce Type: replace-cross Abstract: AI agents with advanced reasoning and tool-use capabilities have demonstrated impressive performance in web browsing for deep search. However,

One Policy Is Enough: Single-Agent Reinforcement Learning Outperforms Tree Search for Chemistry Tool Learning

arXiv:2608.30952v1 Announce Type: cross Abstract: Chemistry questions often demand exact computation and database lookups that a language model cannot supply from its parameters, so it must reach for

post-graph-rag: A PostgreSQL-Native Bi-Temporal Graph RAG Engine with Temporal Grounding at Synthesis

arXiv:2608.24921v2 Announce Type: replace Abstract: Graph RAG connects facts no single passage states, but implementations pay three times: in infrastructure, keeping vector store, graph database and

Real-Time Deadlines Reveal Fragile Temporal Adaptation in LLM Strategic Dialogues

arXiv:2601.13206v2 Announce Type: replace Abstract: Large Language Models (LLMs) generate text token-by-token in discrete time, yet real-world communication, from therapy sessions to business negotiat

Rubric-to-Code Credit Assignment for Reinforcement Learning

arXiv:2608.27906v2 Announce Type: replace Abstract: Interactive web application generation requires models to produce usable HTML, CSS, and JavaScript applications from natural language requests. Unli

ScalePRM: Training Process Reward Models by Scaling Verification Compute Without Ground Truth

arXiv:2512.03244v2 Announce Type: replace-cross Abstract: Training process reward models (PRMs) requires step-level correctness labels, obtained either through expensive human annotation or by relying

Self-Evolving Skills via Surrogate-Guided Solve-and-Reproduce

arXiv:2608.28638v1 Announce Type: new Abstract: Agent skills are portable packages of instructions and resources an agent consults at deployment. Self-evolving them fails in two ways today. First, ski

SPADE: A Large Language Model Framework for Soil Moisture Pattern Recognition and Anomaly Detection in Precision Agriculture

arXiv:2509.18123v2 Announce Type: replace Abstract: Accurate interpretation of soil moisture patterns is critical for irrigation scheduling and crop management, yet existing approaches for soil moistu

T-MAP: Red-Teaming LLM Agents with Trajectory-aware Evolutionary Search

arXiv:2603.22341v2 Announce Type: replace-cross Abstract: While prior red-teaming efforts have focused on eliciting harmful text outputs from large language models (LLMs), such approaches fail to capt

Towards AI-Assisted Research Writing: Benchmarking LLMs for AI/ML Introduction Generation

arXiv:2508.14273v3 Announce Type: replace Abstract: As researchers increasingly adopt LLMs as writing assistants, generating high-quality research paper introductions remains both challenging and esse

Tracing distinguishability through transformer processing with stochastic LayerNorm

model-releasesarxiv-cs-lg

arXiv:2608.30720v1 Announce Type: new Abstract: Representational similarity is foundational to analyses of deep networks, yet distances between point-valued representations are not intrinsically tied

Training and Agentic Inference Strategies for LLM-based Manim Animation Generation

arXiv:2604.18364v2 Announce Type: replace Abstract: Generating programmatic animation using libraries such as Manim presents unique challenges for Large Language Models (LLMs), requiring spatial reaso