Search: “benchmark”

Adversarial Closed-Loop Curriculum for Evolving Role-Playing Agents

arXiv:2609.28609v1 Announce Type: new Abstract: Role-playing agents based on large language models have been widely applied in areas such as personalized assistance and social simulation. Recent RL me

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

Broadening Uncertainty Estimation for Audio Question Answering Across Methods, Formats, and Inputs

DGX agent

arXiv:2609.28879v1 Announce Type: cross Abstract: Audio-language models can produce confident answers unsupported by the audio, motivating uncertainty estimates that identify unreliable responses. We

researcharxiv-cs-ai

25 Sept 2026

ENDOPROMPT: Victim-Side Pseudo-References for Utility Degradation

arXiv:2609.29948v1 Announce Type: new Abstract: Prompt injection can degrade benign task performance without eliciting harmful content. Yet many attack objectives depend on task labels or predefined t

FluidRain: Incompressible Rain Flow as an Attention Bias for Loop-in-Loop Video Deraining

arXiv:2609.29006v1 Announce Type: new Abstract: Existing video deraining methods typically exploit neighboring frames through either explicit alignment or implicit spatiotemporal aggregation. Explicit

safetyarxiv-cs-cv

Agents

HEXIS: Compiling Skills into Extended Finite State Machines

arXiv:2609.30123v1 Announce Type: new Abstract: Agent skills provide reusable knowledge and instructions, yet agents must repeatedly infer how to apply them and which operation should follow. This cou

agentsarxiv-cs-ai

LeafTrackNet: A Deep Learning Framework for Robust Leaf Tracking in Top-Down Plant Phenotyping

model-releasesarxiv-cs-cv

arXiv:2512.13130v2 Announce Type: replace Abstract: High-resolution phenotyping at the level of individual leaves offers fine-grained insights into plant development and stress responses. However, the

Neural Transport Nested Sampling

arXiv:2609.29413v1 Announce Type: new Abstract: Sampling from Boltzmann distributions of molecular systems is an inference problem that has seen significant recent developments fuelled by advances in

Local Ai

PPTBench: Can Coding Agents Reconstruct the Visual World through Structured, Editable Slides

arXiv:2609.29718v1 Announce Type: cross Abstract: Coding agents are beginning to act in the visual world. They now build webpages, GUIs, games, 3D scenes, diagrams, and documents. Success in such visu

local-aiarxiv-cs-ai

Research

PROVE: Proof-guided Regime-aware Operator Verification for Hallucination Detection in Medical Visual Question Answering

arXiv:2609.29604v1 Announce Type: new Abstract: In medical visual question answering (VQA), hallucinations of vision-language models (VLMs) may lead to confident but incorrect responses, raising the r

researcharxiv-cs-cv

Requirement-Bound Verified Commissioning: A Frozen Four-Billion-Parameter Local Model as a Candidate Generator under an External Acceptance Layer with Verification and Release Authority

arXiv:2609.30219v1 Announce Type: cross Abstract: An acceptance protocol is developed for sensor-coordinate and polarity binding in mechatronic commissioning. Candidate generation is separated from re

Revalidation Beats Stateful Routing for Scientific Surrogates Under Distribution Shift

arXiv:2609.29715v1 Announce Type: cross Abstract: Surrogate models are often chosen during development and then left in place as new measurements arrive. That practice becomes risky when noise, input

Safe Skill Retirement for Physical Agents

arXiv:2609.29543v1 Announce Type: new Abstract: Agent skills bundle procedural guidance with execution conditions governing authority, user consent, and live environment state. When model capabilities

StepCOPS: Closed-Testing Lower-Tail Certificates for Language-Model Policy Selection

arXiv:2609.29549v1 Announce Type: new Abstract: Post-training pipelines must select one language-model policy from many checkpoints, prompts, and decoding rules. Mean evaluator scores can conceal rare

To Trust or Not to Trust: Retrieval-Augmented Fact Checking in Speech

arXiv:2609.30227v1 Announce Type: cross Abstract: Online misinformation increasingly appears in spoken formats such as news clips, podcasts, interviews, political speeches, and social media videos, cr

A comparative assessment of global building and settlement datasets across geographic and settlement contexts

model-releasesarxiv-cs-cv

arXiv:2609.28154v1 Announce Type: new Abstract: Global building and settlement datasets increasingly support population mapping, exposure assessment, urban monitoring, and other analyses of the built

Agent-Editing World Model: Rethinking World Modeling for LLM Agents

arXiv:2609.28416v1 Announce Type: cross Abstract: Recent advances in large language models (LLMs) have enabled agents to tackle long-horizon tasks across diverse environments. To further improve agent

ASCIIBench: Evaluating Language-Model-Based Understanding of Visually-Oriented Text

arXiv:2512.04125v2 Announce Type: replace Abstract: Large language models (LLMs) have demonstrated several emergent behaviors with scale, including reasoning and fluency in long-form text generation.

Automated Extraction of Records of Processing Activities (RoPA) Using Hybrid RAG and Locally Deployed Large Language Models

arXiv:2609.27359v1 Announce Type: new Abstract: Vietnam's Personal Data Protection Law (Law No. 91/2025/QH15) and Decree No. 356/2025/ND-CP, effective January 1, 2026, require organizations to establi

BranchDrive: A Branch-Structured Dataset for Action-Conditioned Driving Prediction

model-releasesarxiv-cs-ro

arXiv:2609.27275v1 Announce Type: new Abstract: Most autonomous-driving datasets record only the action executed by a behavior policy and the single future that followed, providing limited supervision

Counterfactual Constraint-Conditioned On-Policy Distillation for Multi-Constraint Instruction Following

arXiv:2609.27421v1 Announce Type: new Abstract: Multi-constraint instruction following requires a model to respond to a query under many simultaneously active constraints. Even strong instruction-tune

DeltaS: Reading the Gated Linear Attention State for KV Cache Eviction in Streaming Video

arXiv:2609.27470v1 Announce Type: cross Abstract: Recent video-language models increasingly adopt hybrid architectures that interleave linear and full attention layers for efficient long-context proce

Research

EidosDoc: Implicit Structure Encoding for Cost-Effective Semi-Structured Document QA

arXiv:2609.27784v1 Announce Type: cross Abstract: Semi-structured documents are ubiquitous in scientific reports, financial statements, and technical manuals. Question answering over such documents re

researcharxiv-cs-ai

EmbodiedSWE: Coding Agents for Long Horizon Dexterous Robotics

model-releasesarxiv-cs-ro

arXiv:2609.27308v1 Announce Type: new Abstract: We study coding agents for long-horizon, dexterous robotics and ask whether their solutions can provide scalable supervision for learning general robot

FedIncome: Federated Learning for Income Estimation in Digital Lending Under Data Sovereignty Constraints

arXiv:2609.27654v1 Announce Type: cross Abstract: Verified income is often unavailable in digital loan applications, forcing lenders to rely on reported income and potentially leading to over-lending,

Fine-Tuning LLMs for Translation: General Forgetting Mitigation Does Not Preserve MT-Specific Instruction Following

arXiv:2609.28395v1 Announce Type: new Abstract: Fine-tuning large language models on parallel data improves translation quality but can cause catastrophic forgetting. Mitigation methods are generally

Local Ai

Giving Credit Where It's Due: Redundancy-Aware Learning for Efficient Reasoning

arXiv:2609.27156v1 Announce Type: new Abstract: Large reasoning models can produce correct yet unnecessarily long reasoning traces. Existing methods improve reasoning efficiency with trajectory-level

local-aiarxiv-cs-cl

Learning Where to Look: A Shared Relative-Alignment Module for Time-Series Forecasting and PPG-to-Vital-Sign Reconstruction

arXiv:2609.27473v1 Announce Type: new Abstract: PPG-to-vital-sign reconstruction turns a wrist-worn photoplethysmogram into clinical waveforms such as the ECG. Long-horizon multivariate time-series fo

safetyarxiv-cs-lg

Lost in Speech: Trilingual Spoken Hallucination Detection Across Audio and Transcripts

arXiv:2608.24707v2 Announce Type: replace Abstract: While text-based hallucination detection is well studied, reference-free detection of factual alterations in speech remains underexplored, especiall

Mizar: A 159M-Parameter Audio-Language Model for Audio Understanding

arXiv:2609.28344v1 Announce Type: cross Abstract: Audio-language models (ALMs) integrate acoustic perception with the knowledge encoded in language models, enabling contextual understanding of auditor

Reachable Global Optimization in AI Systems: How Global Is Global?

arXiv:2609.27855v1 Announce Type: new Abstract: AI systems increasingly claim to optimize prompts, policies, architectures, plans, tool-use trajectories, reasoning traces, and test-time computation. T

Realize What Matters: Principled Context Representation for Large-Scale Reasoning

arXiv:2609.27173v1 Announce Type: new Abstract: Solving complex tasks in domains such as science, medicine, law, and finance often requires assembling interdependent information scattered across vast,

Remote Surfaces at Your Fingertips: Electrovibration-Based Tactile Feedback for Robot Teleoperation via Touchscreen Interfaces

arXiv:2609.27938v1 Announce Type: new Abstract: Enabling operators to perceive and interact with remote environments naturally is a fundamental challenge in robotic teleoperation. This is especially c

safetyarxiv-cs-ro

Applications

SatUnreal: A High-Precision Synthetic Dataset for Satellite Stereo Matching via Unreal Engine

arXiv:2609.27442v1 Announce Type: new Abstract: 3D reconstruction from satellite imagery is essential for large-scale topographic analysis, yet the lack of high-fidelity training datasets with accurat

applicationsarxiv-cs-cv

Self-Evolving Multimedia Verification through Memory Consolidation of Contestation Experiences

arXiv:2609.27175v1 Announce Type: cross Abstract: Multimedia verification requires not only accurate decisions but also traceable evidence, reliable human correction, and safe reuse of prior experienc

The Tasteful Agent: Measuring and Improving Taste in Long-Horizon Tasks

arXiv:2609.25804v2 Announce Type: replace Abstract: LLM agents increasingly work on long-horizon tasks, and the decisions they make along the way, such as which hypothesis to test or which implementat

Validation and Simulation Catch Different Errors: Four Levels of Evaluation for LLM-Generated Circuits

arXiv:2609.26830v1 Announce Type: cross Abstract: Simulation success is not equivalent to structural correctness for LLM-generated circuits. We define and measure four evaluation levels -- schema vali

What Confidence Routing Is Actually Doing: Auditing Routing, Calibration, and Commitment in Multi-Agent Deliberation

arXiv:2609.27822v1 Announce Type: cross Abstract: A common multi-agent design asks agents to report confidence and lets the highest-scoring agent speak next, implicitly using one scalar both to route

When Context Misleads: In-context Learning with Jurisdiction in Large Language Models

arXiv:2609.27603v1 Announce Type: cross Abstract: In-Context Learning (ICL) has become a cornerstone of modern LLM deployment. However, existing ICL post-training methods have a critical blind spot: t

AgentBetta: Verification-Driven Adaptive Configuration of an AI Nano-Agent through Selective Expansion and Verified Contraction

arXiv:2609.23512v1 Announce Type: new Abstract: Large language model agents are typically deployed with predefined configurations, although the required model capability, context, tools, permissions,

APEXA: Execution-Integrity Enforcement for Multi-Agent LLM Automation of Synchrotron Data Reduction

arXiv:2609.24165v1 Announce Type: new Abstract: Synchrotron data reduction, detector calibration followed by azimuthal integration of terabyte-scale diffraction series, is a multi-step, expert-bound b

Beyond Endpoint Performance: Process-Level Evaluation of Self-Evolving Agents

arXiv:2609.24663v1 Announce Type: new Abstract: Self-evolving agents convert interaction feedback into persistent artifacts, such as memories or skills, which in turn guide subsequent decisions. As th

Calibration as a First-Class Criterion in LLM Evaluation

arXiv:2609.26489v1 Announce Type: new Abstract: Calibration of language models -- the alignment between expressed or implicit confidence and empirical correctness -- is a well-studied subfield within

safetyarxiv-cs-cl

Calibration Count Reuse: Validity Does Not Determine Efficiency

arXiv:2609.25138v1 Announce Type: cross Abstract: Calibration count reuse raises separate validity and efficiency questions. We give a validity criterion for general count-dependent nonconformity scor

CraftBench-UE: Deterministic Evaluation for Coding Agents in Unreal Engine

arXiv:2609.23142v1 Announce Type: new Abstract: Building gameplay features in a game engine requires more than code, as code that compiles and runs does not necessarily implement the requested gamepla

Local Ai

Entropy Can Flow, or It Can Guide. Be Entropy. LEDFlow: Introducing Entropy-guided Generation Order into Uniform Discrete Flow

arXiv:2609.25131v1 Announce Type: new Abstract: Uniform discrete flow permits repeated updates at every generation position. While continued revision supports correction of wrong tokens, it also expos

local-aiarxiv-cs-lg

Exposing Blind Spots in Deep Imbalanced Regression Evaluation

arXiv:2609.25152v1 Announce Type: new Abstract: Deep Imbalanced Regression (DIR) addresses a common failure mode of regression models: target distributions are highly non-uniform, causing models to pe

From Inference Engine to Inference Control Plane: Connecting vLLM, llm-d, and the Evolution of Efficient Distributed LLM Serving

arXiv:2609.23130v1 Announce Type: new Abstract: Large language model (LLM) inference is evolving from an engine-local optimization problem into a distributed control problem involving reusable state,

Latent Commonality Expectation-Maximisation for Box-supervised Tree Crown Instance Segmentation

model-releasesarxiv-cs-cv

arXiv:2609.26549v1 Announce Type: new Abstract: Individual tree crown segmentation from aerial imagery underpins tree-level carbon accounting, biodiversity, and restoration monitoring at landscape sca