CompanyAnthropic8 recent entries12 Oct 2026SWE-Duel: Towards a Self-Scaling Arena for Coding AgentsarXiv:2610.12569v1 Announce Type: new Abstract: Static coding benchmarks are fixed at release. Over time their tasks leak into training data, and stronger agents eventually saturate them. We argue tha→12 Oct 2026Solving VeriContest with a Lean-Backed Rust VerifierarXiv:2610.03994v2 Announce Type: replace-cross Abstract: VeriContest is a benchmark of 1007 competitive-programming problems in Rust, each with a Verus specification, a judge-accepted solution, and a
CompanyOpenAI8 recent entries5 Oct 2026Evaluating VQA in Vision Language Models using Cooperative PrinciplesarXiv:2610.02878v1 Announce Type: new Abstract: We evaluate the performance of Vision Language Models in Visual Question Answering (VQA) when questions violate Grice's maxims. To do this, we use VLMs →5 Oct 2026Automating the Application of HCI Principles: Skills for On-Demand UI Construction, the Human-AI Space to Think, and the Future of HCIarXiv:2610.02369v1 Announce Type: cross Abstract: Human-computer interaction (HCI) is in the middle of a transition: large language models can now generate functional user interfaces (UIs) on demand f→6 Oct 2026Errors of LLM-Assisted Literature Retrieval in Environmental Science: A Comparison Study of Abstract versus Full-text Based PromptsarXiv:2610.05690v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used for literature search and synthesis. However, it is unclear whether they retrieve accurate bibliogr→7 Oct 2026Rethinking Visual Provenance: Detection and Watermarking Across Direct Visual Generation and LLM-Driven Code RenderingarXiv:2610.08137v1 Announce Type: cross Abstract: AI systems create images and videos with image/video generation models or by writing code and graphics descriptions that are then rendered. These rout→7 Oct 2026MiniScope: Authorizing Agents with Least-Privilege PermissionsarXiv:2512.11147v2 Announce Type: replace-cross Abstract: AI agents are increasingly granted autonomous access to sensitive user data and third-party services, making effective permission management a→8 Oct 2026Arctic Questions, Missing Answers: A Dataset and Benchmark for LLM Abstention in Arctic SciencearXiv:2610.09446v1 Announce Type: new Abstract: Large language models (LLMs) should abstain from scientific multiple-choice questions when no option is valid, but frequent abstention alone does not de→9 Oct 2026Why LLM Agents Favor Their Group: Stakes, Observed Norms, and ReputationarXiv:2610.11008v1 Announce Type: cross Abstract: Language-model agents favor their own group because they have watched their members favor each other. The group label alone does little once the decis→9 Oct 2026Arbiter: Detecting Interference in LLM Agent System PromptsarXiv:2603.08993v2 Announce Type: replace-cross Abstract: System prompts for LLM-based coding agents are software artifacts that govern agent behavior, yet lack the testing infrastructure applied to c
CompanyGoogle8 recent entries8 Oct 2026Arctic Questions, Missing Answers: A Dataset and Benchmark for LLM Abstention in Arctic SciencearXiv:2610.09446v1 Announce Type: new Abstract: Large language models (LLMs) should abstain from scientific multiple-choice questions when no option is valid, but frequent abstention alone does not de→8 Oct 2026A Positive Case for Faithfulness: LLM Self-Explanations Help Predict Model BehaviorarXiv:2602.02639v2 Announce Type: replace-cross Abstract: LLM self-explanations are often presented as a promising tool for AI oversight, yet their faithfulness to the model's true reasoning process i→9 Oct 2026WorldBench: Evaluating LLMs on Three.js Voxel World GenerationarXiv:2610.10622v1 Announce Type: cross Abstract: Large language models can now write complete, interactive 3D worlds as code, but grading those worlds automatically is unreliable. Existing judges tak→9 Oct 2026GroundSight at GroundLM 2026 Shared Tasks: GoldenViewVQAarXiv:2610.11402v1 Announce Type: new Abstract: GoldenViewVQA requires models to jointly answer driving-scene questions and identify the camera view containing the supporting visual evidence, making p→9 Oct 2026Constitutional Gating and Deterministic Recovery for Multi-Agent LLM Negotiation: Ablations Against a Stateful Adversarial GatekeeperarXiv:2610.11542v1 Announce Type: cross Abstract: Multi-agent LLM systems negotiating with a stateful counterpart waste model calls in three ways: polite loops that never meet the counterpart's hidden→9 Oct 2026Arbiter: Detecting Interference in LLM Agent System PromptsarXiv:2603.08993v2 Announce Type: replace-cross Abstract: System prompts for LLM-based coding agents are software artifacts that govern agent behavior, yet lack the testing infrastructure applied to c→12 Oct 2026People Making Conversation: Evaluating How Naturally Speech Models ConversearXiv:2610.12802v1 Announce Type: new Abstract: A conversational speech model has to decide, at every pause, whether to speak. Existing evaluations test boundary detection, policy adherence, or aggreg→12 Oct 2026MathVis: Can AI Reason through Drawing? A Benchmark for Mathematical Figure GenerationarXiv:2610.12954v1 Announce Type: new Abstract: This paper introduces MathVis, a benchmark designed to evaluate LLMs' ability to visualize frontier mathematical problems and perform diagram-assisted r
CompanyMeta8 recent entries30 Sept 2026Thinking Before Thinking: Scaling Agentic Inference Through Meta-ReasoningarXiv:2609.38147v1 Announce Type: new Abstract: As agents take on longer and more complex problems, controlling the execution becomes a task in its own right. Each step in the run brings new control c→1 Oct 2026The Invisible Language Tax: Token Premiums of French and Regional Languages in 2026 LLM Tokenizers, and a French-Optimized PrototypearXiv:2609.39001v1 Announce Type: cross Abstract: LLM services are billed per token and context windows are measured in tokens, yet the number of tokens needed for the same content varies across langu→1 Oct 2026RankEvolve: A Reliable Multi-Agent Auto-Research Harness for Evolving Ranking ModelsarXiv:2609.39551v1 Announce Type: new Abstract: Auto-research agents, LLM systems that propose, implement, train, and evaluate model changes across iterations, promise to automate applied ML's experim→2 Oct 2026VERITYGATE: A Four-Gate Schema-Level Faithfulness Framework and Paired Benchmark for Grounded LLM Narrations over Structured EvidencearXiv:2610.00833v1 Announce Type: cross Abstract: Fluent LLM explanations may not follow the evidence from a structured system. We present VERITYGATE, a four-gate checker for declared evidence IDs, en→5 Oct 2026When Terminal-Agent Training Stalls: Demystifying Data Generation and Verification ChallengearXiv:2610.02405v1 Announce Type: new Abstract: Using a frontier model like Claude Opus as a meta-agent to generate terminal tasks and verifiers for RL training is increasingly common. Yet a runnable →7 Oct 2026PlotPick: AI-powered batch extraction of numerical data from scientific figuresarXiv:2605.06021v2 Announce Type: replace Abstract: Systematic reviews and meta-analyses often need numerical data reported only in figures, and extracting them with interactive digitisers usually req→9 Oct 2026When Interfaces Speak: Data-Aware Generative UI Harness for Active InteractionarXiv:2610.11123v1 Announce Type: new Abstract: Most human-agent interaction today remains text-based. Natural language can impose cognitive overload, ambiguity, information chaos, and slow input for →9 Oct 2026Can LLMs Fix It Without Code? Toward Automated Verification of No-Code Bug FixesarXiv:2610.11963v1 Announce Type: cross Abstract: A no-code fix resolves an invalid bug report by directing the user to change a setting, update to a version where the problem is already fixed, or adj
CompanyMistral8 recent entries26 Aug 2026Can large language models assist choice modelling? Insights into prompting strategies and current models' capabilitiesarXiv:2507.21790v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are becoming widely used to support various workflows across different disciplines, yet their potential in discre→4 Sept 2026Measuring Harmfulness of Computer-Using AgentsarXiv:2508.00935v3 Announce Type: replace-cross Abstract: Computer-using agents (CUAs), which can autonomously control computers to perform multi-step actions, might pose significant safety risks if m→4 Sept 2026ESPO: Error-Structured Prompt Optimization via Diagnose, Diversify, and StabilizearXiv:2609.04197v1 Announce Type: cross Abstract: Evolutionary prompt optimizers such as GEPA suffer from prompt bloat: each iteration appends rules and caveats, producing prompts up to 3imes longer y→11 Sept 2026Trust Me, I'm Your Developer: Self-Issued Authentication in Large Language ModelsarXiv:2609.03247v1 Announce Type: cross Abstract: Large language model (LLM) security has largely focused on role-playing jailbreaks, with less attention to what happens when a user asks an LLM to ver→11 Sept 2026OpenDiscoveryTrace: Process Traces for Evaluating AI Scientist WorkflowsarXiv:2609.09203v1 Announce Type: new Abstract: Existing benchmarks for autonomous AI scientists evaluate only final outputs---generated code, hypotheses, or papers---yet discard the reasoning process→15 Sept 2026Issue Bias in Generative AI Writing Assistance: Political Issues and LLMs in the Swedish 2026 ElectionarXiv:2609.15207v1 Announce Type: new Abstract: Generative AI writing assistants and the Large Language Models (LLMs) that power them are increasingly part of how voters gather information before elec→22 Sept 2026Auditing Political Alignment in LLM Assistants: Engagement, Stance, and User IdentityarXiv:2609.23039v1 Announce Type: new Abstract: LLM-based AI systems answer political questions for hundreds of millions of people. Current audits measure what they say to an average user, but their b→1 Oct 2026The Invisible Language Tax: Token Premiums of French and Regional Languages in 2026 LLM Tokenizers, and a French-Optimized PrototypearXiv:2609.39001v1 Announce Type: cross Abstract: LLM services are billed per token and context windows are measured in tokens, yet the number of tokens needed for the same content varies across langu
CompanyxAI8 recent entries22 Sept 2026Auditing Political Alignment in LLM Assistants: Engagement, Stance, and User IdentityarXiv:2609.23039v1 Announce Type: new Abstract: LLM-based AI systems answer political questions for hundreds of millions of people. Current audits measure what they say to an average user, but their b→25 Sept 2026LLM Agents Can Easily Tamper With Their Own TracesarXiv:2609.30266v1 Announce Type: cross Abstract: Asynchronous monitoring, incident investigations, and compliance audits primarily rely on agent traces to reconstruct what happened. These analyses as→30 Sept 2026Auditable Long-Term Memory: A Deterministic Retrieval Chain Measured at 479/475 of 500 on LongMemEval-SarXiv:2609.38021v1 Announce Type: cross Abstract: We evaluate an auditable long-term memory system on LongMemEval-S. Its retrieval chain uses hybrid candidate retrieval, cross-encoder reranking, cover→1 Oct 2026DrivingBench: Can Vision-Language Models Drive a Toyota Corolla?arXiv:2609.38948v1 Announce Type: cross Abstract: Frontier models excel at many digital benchmarks, yet their ability to drive a real car, an everyday human skill, remains largely untested. We present→5 Oct 2026Dissecting model behavior through agent trajectoriesarXiv:2606.17454v3 Announce Type: replace Abstract: AI agent performance is not just a modeling problem, it is fundamentally a systems problem. The advanced capabilities of models are realized through→6 Oct 2026Errors of LLM-Assisted Literature Retrieval in Environmental Science: A Comparison Study of Abstract versus Full-text Based PromptsarXiv:2610.05690v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used for literature search and synthesis. However, it is unclear whether they retrieve accurate bibliogr→9 Oct 2026WorldBench: Evaluating LLMs on Three.js Voxel World GenerationarXiv:2610.10622v1 Announce Type: cross Abstract: Large language models can now write complete, interactive 3D worlds as code, but grading those worlds automatically is unreliable. Existing judges tak→12 Oct 2026People Making Conversation: Evaluating How Naturally Speech Models ConversearXiv:2610.12802v1 Announce Type: new Abstract: A conversational speech model has to decide, at every pause, whether to speak. Existing evaluations test boundary detection, policy adherence, or aggreg
CompanyDeepSeek8 recent entries30 Sept 2026Who Warmed the Archives? LLMs Overestimate Historical WarmtharXiv:2609.37499v1 Announce Type: new Abstract: Historical archives are an under-used source for extending the instrumental climate record backward in time, and LLMs offer a way to extract the indices→1 Oct 2026The Invisible Language Tax: Token Premiums of French and Regional Languages in 2026 LLM Tokenizers, and a French-Optimized PrototypearXiv:2609.39001v1 Announce Type: cross Abstract: LLM services are billed per token and context windows are measured in tokens, yet the number of tokens needed for the same content varies across langu→2 Oct 2026Finding the Right Fit: Model-Harness Interactions across Agent TasksarXiv:2610.00917v1 Announce Type: new Abstract: Choosing an agent system means choosing both a language model and the harness through which it acts. We ask whether a strong model, harness, or pairing →5 Oct 2026EvoRiskBench: An Evolving Benchmark for Runtime Security Risks in Workspace AgentsarXiv:2610.03153v1 Announce Type: cross Abstract: Workspace agents combine large language models with execution harnesses to perform stateful, multi-step tasks that access or modify external resources→6 Oct 2026Errors of LLM-Assisted Literature Retrieval in Environmental Science: A Comparison Study of Abstract versus Full-text Based PromptsarXiv:2610.05690v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used for literature search and synthesis. However, it is unclear whether they retrieve accurate bibliogr→9 Oct 2026StoreBench: A Live-Commerce Environment for Evaluating and Training Autonomous Operator AgentsarXiv:2610.10942v1 Announce Type: new Abstract: Reinforcement learning environments are now a primary lever for improving large language model (LLM) capabilities in post-training, yet most agentic ben→9 Oct 2026Mine Odyssey: Benchmarking Spatial Agentic Intelligence in the WildarXiv:2610.11328v1 Announce Type: new Abstract: Advances in foundation models are driving efforts to introduce agents to assist people in the physical world. Such agents require agentic spatial intell→12 Oct 2026Not All Correct Fixes Run Alike: An Empirical Study of Patch Performance in Repository-Level Issue ResolutionarXiv:2610.13563v1 Announce Type: cross Abstract: Repository-level issue-resolution benchmarks such as SWE-bench evaluate submitted patches primarily based on whether they pass the correctness tests,
CompanyNVIDIA7 recent entries22 Apr 2026CASS: Nvidia to AMD Transpilation with Data, Models, and BenchmarkarXiv:2505.16968v4 Announce Type: replace-cross Abstract: Cross-architecture GPU code transpilation is essential for unlocking low-level hardware portability, yet no scalable solution exists. We intro→24 Apr 2026Kernel-Smith: A Unified Recipe for Evolutionary Kernel OptimizationarXiv:2603.28342v2 Announce Type: replace Abstract: We present Kernel-Smith, a framework for high-performance GPU kernel and operator generation that combines a stable evaluation-driven evolutionary a→6 Jun 2026Evaluation of LLMs for Mathematical Formalization in LeanarXiv:2606.05632v1 Announce Type: new Abstract: Within the past few years, the ability of Large Language Models (LLMs) to generate formal mathematical proofs has improved drastically. We provide a com→10 Jul 2026Compete Then Collaborate: Frontier AI Teachers Build a Verifiable Curriculum to Improve a Coding Student Beyond ImitationarXiv:2607.08255v1 Announce Type: new Abstract: Large language models increasingly serve as teachers generating training data for smaller students. Prior multi-teacher knowledge distillation methods m→16 Jul 2026Inference Economics of Enterprise Coding Agents: A Case Study of Cloud vs. On-Premise LLMsarXiv:2607.13080v1 Announce Type: cross Abstract: Autonomous coding agents force engineering organizations to choose between API-based frontier models -- strong reasoning at high token cost -- and on-→1 Oct 2026Pretext: Defeating Malicious Skill Detection Frameworks for AI AgentsarXiv:2609.39607v1 Announce Type: cross Abstract: Skills extend an agent's capabilities by injecting instructions and information into the context, and are widely used by agents such as OpenClaw and C→5 Oct 2026D2K-Bench: Can LLM Agents Turn Expert Designs into Efficient GPU Kernels?arXiv:2610.03226v1 Announce Type: cross Abstract: GPU kernels generated by large language model (LLM) agents can remain less efficient than expert implementations, but runtime alone does not reveal ho