Search: “benchmark”

Caption-Mediated Perceived-Safety Estimation for Pedestrian Routing

arXiv:2609.38479v1 Announce Type: cross Abstract: This paper presents an explainable approach to pedestrian routing, in which perceived safety is estimated from street-level imagery through an explici

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

EviRover: Reinforcing Agentic Perception Beyond a Glance

DGX agent

arXiv:2609.40230v1 Announce Type: cross Abstract: Visual perception is conventionally formulated as a one-shot prediction from a single glance at the image, under the assumption that the image content

model-releasesarxiv-cs-ai

1 Oct 2026

Agents

Experimental Experience Modeling for Autonomous Research

arXiv:2609.39392v1 Announce Type: new Abstract: Autonomous research agents can generate hypotheses and conduct experiments, but experimentation remains a major source of computational cost. A fundamen

agentsarxiv-cs-ai

Marginal Response Surface Elicitation for Zero-Label Tabular Learning

arXiv:2609.39639v1 Announce Type: new Abstract: Tabular learning uses structured data to predict target outcomes. Traditionally, this process has relied on labeled data. However, large language models

RAIM: Robust Aggregation of Inexpensive Models for Hallucination Detection

arXiv:2609.39229v1 Announce Type: cross Abstract: Automatic evaluation of faithfulness increasingly relies on a large language model acting as a judge, yet the most reliable judges are proprietary fro

Self-Spec Verifiable Code Generation

arXiv:2609.39568v1 Announce Type: cross Abstract: Large language models (LLMs) may generate unreliable code on corner cases missed by testing, while formal verification can provide machine-checkable g

When a Kindergartener Solves Calculus: Measuring Capability Leakage in Role-Prompted Reasoning Models

arXiv:2609.39846v1 Announce Type: cross Abstract: We investigate the problem of role-capability leakage (RCL), in which a role-prompted reasoning model generates convincing in-role text while continui

WorldAuditBench: Interactive 3D World Auditing with Multimodal Agents

arXiv:2609.40325v1 Announce Type: new Abstract: As interactive 3D worlds are increasingly used to study intelligent behavior, it becomes important to develop efficient pipelines for identifying anomal

LoLBench: Evaluating Coding Agents with Long-Horizon Proposals on Large Software Systems

arXiv:2609.37143v1 Announce Type: cross Abstract: Modern coding agents can deliver increasingly large repository-level changes, and recent benchmarks reflect this by emphasizing long-horizon tasks wit

30 Sept 2026

Teaching LLMs to Generate Challenging MILP Instances via Solver Feedback

arXiv:2609.37356v1 Announce Type: new Abstract: Generating optimization instances that are both feasible and computationally challenging is crucial for benchmarking solvers and training learning-based

30 Sept 2026

LLM Unlearning Evaluation with TRIAGE

model-releasesarxiv-cs-lg

arXiv:2609.32103v1 Announce Type: new Abstract: Large language models can memorize private or harmful information, motivating machine unlearning methods that remove targeted knowledge while preserving

29 Sept 2026

SCLATE: a Substrate for Continual-Learning Agent Training and Evaluation

model-releasesarxiv-cs-lg

arXiv:2609.32391v1 Announce Type: cross Abstract: Continual-learning agents are systems of models, harnesses, and memory operating over long multi-session horizons. Evaluating and training them requir

29 Sept 2026

SynDORBench: Evaluating LVLM Perceptual Robustness Under Physically Constrained Visibility Conditions

arXiv:2609.31823v1 Announce Type: new Abstract: Large vision-language models (LVLMs) have demonstrated remarkable performance on multimodal reasoning benchmarks, yet their perceptual reliability under

29 Sept 2026

BeatGraph: Self-Supervised Heartbeat Graphs for Infant ECG Representations from the Home Environment

model-releasesarxiv-cs-lg

arXiv:2609.31546v1 Announce Type: new Abstract: Electrocardiogram (ECG) foundation models typically tokenize the signal into fixed-length patches that ignore cardiac structure, so a patch may split a

Evaluation is All You Need: Strategic Overclaiming of LLM Reasoning Capabilities Through Evaluation Design

arXiv:2506.04734v3 Announce Type: replace Abstract: Reasoning models represented by the Deepseek-R1-Distill series have been widely adopted by the open-source community due to their strong performance

SkillFlow: Scalable and Efficient Agent Skill Retrieval System

arXiv:2504.06188v3 Announce Type: replace Abstract: AI agents can extend their capabilities at inference time by loading reusable skills into context, yet equipping an agent with too many skills, part

Safety

UniAR: A Unified Framework for Autism Recognition Enhanced by Multi-View Prompt Learning

arXiv:2609.31298v1 Announce Type: cross Abstract: Autism Spectrum Disorder (ASD) is a complex neurodevelopmental disorder for which early and accurate diagnosis is critical to improving long-term deve

safetyarxiv-cs-ai

Refusing Everything Looks Safe: Restoring the Benign Arm to Encoded-Prompt Evaluation

arXiv:2609.26176v2 Announce Type: replace-cross Abstract: Encoded-prompt attacks are evaluated almost entirely on their harmful arm: a benchmark sends obfuscated harmful requests and reports how often

25 Sept 2026

SWE-Prometheus: Measuring Engineering Governance Improvements in Real-World Repositories

arXiv:2609.29465v1 Announce Type: new Abstract: Large language model based coding agents have made substantial progress on repository-level software engineering tasks. Existing repository benchmarks,

25 Sept 2026

Ajar: Measuring Open Privilege in Agent Defenses

arXiv:2609.26900v1 Announce Type: cross Abstract: A language model agent acts through the tools it is given. The data it reads while working on a task can redirect what it does with those tools. A gro

Evaluating Open-Weight LLMs for Turkish Domain Documents Under Retrieval and Hardware Constraints

arXiv:2609.28007v1 Announce Type: new Abstract: Most Turkish-capable large language models (LLMs) are evaluated using general-purpose benchmarks rather than long, structurally complex domain documents

Learn How to Act from Your Own Interactions: On-Policy Self-Distillation for GUI Agents

arXiv:2609.27307v1 Announce Type: new Abstract: Graphical User Interface (GUI) agents enable the fulfillment of complex user instructions through multi-turn interactions with software environments, re

Math Reasoning in LLMs is Organized by Approach, Not Topic

arXiv:2609.27041v1 Announce Type: new Abstract: Mathematical reasoning benchmarks are typically organized by topic, but language models may organize their internal computation by reusable reasoning ap

PASTABench: Proactive Assessment of Sequential Trajectories for Agent Safety

arXiv:2609.28197v1 Announce Type: new Abstract: As Large Language Models (LLMs) evolve into autonomous agents that alter real-world states, ensuring operational safety across multi-step workflows has

Prompt, Probe, Train, or Annotate? Single-camera sports video understanding in amateur settings

arXiv:2609.28049v1 Announce Type: cross Abstract: Video understanding is usually benchmarked on curated, single-actor, or professionally filmed clips, and a strong score there is routinely read as evi

State-Grounded Conditioning: Wrapping User-Facing LLM Agents Where Direction Depends on Live State

arXiv:2609.27606v1 Announce Type: new Abstract: We introduce State-Grounded Conditioning (SGC), a design principle for user-facing LLM agents that must condition on live user state (game state, sessio

Using Vision Language Foundation Models to Generate Plant Simulation Configurations via In-Context Learning

arXiv:2603.08930v2 Announce Type: replace-cross Abstract: This paper introduces a benchmark for evaluating whether vision-language models (VLMs) can generate plant simulation configurations from image

BigO(Bench): Can LLMs Generate Code with Controlled Time and Space Complexity?

arXiv:2503.15242v3 Announce Type: replace Abstract: We introduce BigO(Bench), a novel coding benchmark designed to evaluate the capabilities of generative language models in understanding and generati

23 Sept 2026

CivBench: Progress-Based Evaluation for LLMs' Strategic Decision-Making in Civilization V

arXiv:2604.07733v2 Announce Type: replace Abstract: Evaluating strategic decision-making in LLM-based agents requires generative, competitive, and longitudinal environments, yet few benchmarks provide

23 Sept 2026

Applications

MultiViewDx: Evidence-Linked Multi-View Clinical Diagnosis

arXiv:2410.14948v2 Announce Type: replace Abstract: Medical multimodal large language models (MLLMs) can perform well on existing medical visual question answering (MedVQA) benchmarks, but their train

applicationsarxiv-cs-cl

23 Sept 2026

RRSI: Regularized Recursive Self-Improvement of Agent Harnesses

arXiv:2609.24972v1 Announce Type: cross Abstract: An LLM agent's capability is largely magnified by its harness, namely the prompts, control flow, tooling, memory, and context management surrounding t

22 Sept 2026

KoNeoBench: A Curated Evaluation Dataset for LLM Understanding of Korean Neologisms

arXiv:2609.19916v1 Announce Type: cross Abstract: Large language models (LLMs) are typically evaluated on static benchmarks, even though natural language constantly evolves through newly emerging word

18 Sept 2026

PersonalAI 2.0: Enhancing knowledge graph traversal/retrieval with planning mechanism for Personalized LLM Agents

arXiv:2605.13481v2 Announce Type: replace Abstract: We introduce PersonalAI 2.0 (PAI-2), a novel framework designed to enhance LLM-based systems through integration of external knowledge graphs (KGs).

18 Sept 2026

Decodable but Misrouted: Sparse Features Uncover a Readout Gap in Vision-Language Models for Harmful Meme Detection

arXiv:2609.18860v1 Announce Type: cross Abstract: When a large vision-language model misclassifies a harmful meme, the failure may reflect missing internal evidence or an inability to route represente

Do Social Patterns Hold in Synthetic Data? Analyzing Cyberbullying Dynamics in LLM-Generated and Authentic Dialogues

arXiv:2609.17549v1 Announce Type: new Abstract: Cyberbullying (CB) is a complex social phenomenon characterized by repeated aggression, power imbalance, and multi-party interaction. Although large lan

I code or AI code: A comparative evaluation of AI-rated scores in classroom observations

arXiv:2609.18274v1 Announce Type: cross Abstract: Classroom observations are widely recognized as a key tool for establishing benchmarks of education quality and guiding pedagogical improvement, yet t

Multi-View Mixture-of-Experts with Vision-Language Reranking for Cross-View Object Geo-Localization

arXiv:2609.18139v1 Announce Type: new Abstract: Cross-view object geo-localization (CVOGL) locates a target in satellite imagery using drone or street-view queries. Existing methods train separate det

ShotFinder: Imagination-Driven Open-Domain Video Shot Retrieval via Web Search

arXiv:2601.23232v4 Announce Type: replace-cross Abstract: In recent years, large language models (LLMs) have made rapid progress in information retrieval, yet existing research has mainly focused on t

Extracting ontology-compliant knowledge from scientific text describing irradiated materials using large language models

arXiv:2609.17291v1 Announce Type: new Abstract: The quest for new materials increasingly relies on predictive models and comprehensive simulations that span scales from atomic to macroscopic levels. H

16 Sept 2026

What Do Hallucinations Reveal About Multimodal Reasoning? Diagnosing Visual Grounding Failures via Contrastive Decoding Probes

arXiv:2609.16646v1 Announce Type: new Abstract: When strong multimodal models are widely available, progress requires new scientific methodologies beyond benchmark scores---using models as instruments

16 Sept 2026

World Model Science: Self-Organized Criticality, Weak Chaos, and Metastable Belief Dynamics in Long-Horizon LLM Agents

arXiv:2609.17419v1 Announce Type: new Abstract: Long-horizon LLM agents must maintain task state across extended sequences of observations, actions, tool calls, and intermediate beliefs. We study thes

16 Sept 2026

Beyond Accuracy: Robustness, Cost, and Governance Trade-offs for Vision-Language Models in Templated Document Extraction

arXiv:2609.15706v1 Announce Type: new Abstract: Vision-language models (VLMs) are increasingly used to extract structured fields from business documents, yet most evaluations report accuracy on clean

FaithfulBench: Does AI Counsel Uphold or Undermine the User's Professed Faith?

arXiv:2609.13634v1 Announce Type: cross Abstract: Do AI assistants help believers reason about moral dilemmas consistently with their faith? We present FaithfulBench, the first benchmark to score AI c

Graph-Transformer Fraud Detection with Self-Supervised Pretraining and Conformal Risk Control

arXiv:2609.14234v1 Announce Type: cross Abstract: Financial fraud in corporate transaction networks has grown more coordinated and harder to detect with rule-based engines and with classical learning

IWC-Bench: Evaluating Web Application Generation from a Software Testing Perspective

arXiv:2609.15387v1 Announce Type: cross Abstract: Human evaluation provides a direct measure of the quality of LLM-generated web applications. However, fitting human judgments through automated evalua

ModularRSI: Modular and Generalizable Recursive Harness Self-Improvement

arXiv:2609.14857v1 Announce Type: new Abstract: Recent work extends recursive self-improvement (RSI) to agent harnesses for long-horizon coding and terminal tasks, enabling agents to improve execution

PARSE: Provenance-Aware Retrieval Sanitization for Professional Domain LLM Agents

arXiv:2606.17467v3 Announce Type: replace-cross Abstract: Prompt injection defenses evaluated on synthetic benchmarks do not generalize to real enterprise documents, which are longer, denser, and inte

Patient-Level, Leakage-Aware Deep Learning for Cross-Center Periapical Radiograph Classification

arXiv:2609.14703v1 Announce Type: new Abstract: Dental caries and endodontic disease are among the most common health conditions worldwide, and intraoral periapical radiographs are central to their de