Search: “benchmark”

What Improves Multimodal Misinformation Detection? Answers from a Large-Scale Empirical Study

arXiv:2609.30402v1 Announce Type: cross Abstract: Multimodal misinformation is increasingly crafted to look convincing by pairing a textual claim with an image that appears to 'prove' it. Yet in pract

28 Sept 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

CounterRoute: Self-Routed Reasoning via Hierarchical Counterfactual Credit Assignment

DGX agent

arXiv:2609.29109v1 Announce Type: new Abstract: Reasoning-capable language models often produce long chains of thought when direct answers suffice, wasting inference compute. Many dual-mode models lea

safetyarxiv-cs-ai

25 Sept 2026

Design and Evaluation of LLM Chaining-Based Task Planning for General Purpose Service Robots

arXiv:2609.29043v1 Announce Type: cross Abstract: General Purpose Service Robot (GPSR) tasks, as defined in the RoboCup@Home benchmark, require robots to interpret diverse natural language commands an

On the Effectiveness of Kernel-Level Evidence for Agent Security

arXiv:2609.28915v1 Announce Type: cross Abstract: LLM agents are deployed into infrastructure that grants them broad host authority, yet existing agent-security benchmarks and defenses operate almost

Polite but Misaligned: Evaluating LLM Politeness Judgments Against Human Pragmatic Norms

arXiv:2609.29001v1 Announce Type: new Abstract: Despite strong performance on standard benchmarks, it remains unclear whether large language models (LLMs) evaluate social pragmatics in ways that align

safetyarxiv-cs-cl

Sharp Limits for Honest Uncertainty in Hard-Budget Repeated Evaluation

arXiv:2609.29140v1 Announce Type: new Abstract: Repeated evaluation can estimate a benchmark score accurately while still requiring replication to certify narrow uncertainty. We characterize that requ

TW3Cast: A Frozen Router of Lightly Fine-Tuned Foundation Models for Time-Series Forecasting on GIFT-Eval, Selected Entirely on the Training Split

arXiv:2609.28506v1 Announce Type: new Abstract: TW3Cast is a time-series forecasting system that reaches position 3 of 130 entries on the GIFT-Eval benchmark by mean MASE rank, as of 2026-09-14. The t

BiCFlow-MER: Orchestrating Discriminative and Generative Multimodal Emotion Recognition via Conditional Transport

arXiv:2609.27615v1 Announce Type: new Abstract: In multimodal emotion recognition (MER), human affective states are inferred by integrating complementary cues from multiple modalities. In audio-text M

COPE: Continual Personalization of LLMs under Sparse User Feedback via User Embeddings and Self-Evaluation

arXiv:2609.26853v1 Announce Type: cross Abstract: While Large Language Models (LLMs) have achieved remarkable results across various benchmarks, their alignment with normative values often results in

safetyarxiv-cs-ai

Fed-ReMasker: Federated Tabular Imputation under Feature-Level Missingness

arXiv:2609.28105v1 Announce Type: cross Abstract: Multi-center clinical studies and biomedical research collaborations increasingly seek to utilize data across centers to build models that generalize

Invisible in Space, Visible in Time: Motion Vision CAPTCHA against GUI Agents

arXiv:2609.27461v1 Announce Type: new Abstract: Most existing visual CAPTCHAs remain spatially solvable: the required information is exposed by static appearance, local structure, and interface state.

Live Assistant: Learning Whether, When, and Whom to Assist in Real-World Live Social Streams

arXiv:2609.27303v1 Announce Type: new Abstract: Livestreams are long-lasting interactive environments where audiovisual content, viewer activity, host behavior, and platform signals evolve together, c

MolDesignBench: Evaluating LLM-based Agent for Scenario-grounded Molecular Design

arXiv:2609.27349v1 Announce Type: new Abstract: Real-world molecular design remains challenging for large language model (LLM)-based agents. It requires them to interpret design contexts, satisfy mult

Pro-Bench: Prompt-Robust Open-Vocabulary Visual Grounding Across Real-World Heterogeneous Environments

arXiv:2609.27076v1 Announce Type: new Abstract: Open-vocabulary visual grounding enables robots to localise task-relevant entities from natural-language queries without dependence on predefined percep

Provably Complete Generalized Planning with LLMs

arXiv:2609.27105v1 Announce Type: new Abstract: Generalized planning aims to compute a plan that solves all instances of a planning domain. Recent work has used LLMs to automatically generate and debu

Schrodinger's Code Repository: Have LLMs Learned SWE-bench or Memorized It?

arXiv:2609.27891v1 Announce Type: cross Abstract: Repository-level coding benchmarks have become the standard for evaluating coding agents, yet they inherently suffer from data leakage because they ar

Silent Failures in Agent-Tool Interaction: An Audit of ToolUniverse

arXiv:2609.26836v1 Announce Type: new Abstract: Agentic AI systems are increasingly adopting automated pipelines that integrate multiple tools. While prior research and benchmarks have studied about t

Spec2COBOLRot: An Agentic-AI Degradation Loop for Realistic COBOL Corpus Generation

arXiv:2609.26835v1 Announce Type: cross Abstract: COBOL remains widely deployed, yet representative corpora reflecting real production code are rarely available, limiting rigorous benchmarking of mode

Visual Tripwires: Anticipating Failure in Deep Vision Systems

arXiv:2609.28099v1 Announce Type: new Abstract: Deep vision systems remain vulnerable to corruption, occlusion, and distribution shift despite strong benchmark performance. Existing reliability method

VLM2GeoVec: Toward Universal Multimodal Embeddings for Remote Sensing

arXiv:2512.11490v2 Announce Type: replace Abstract: Satellite imagery differs from natural images in viewpoint, resolution, scale variation, and the prevalence of small objects -- demanding both regio

When Labels Are Scarce: An Oscillatory State Space Model for Vibration Diagnosis

arXiv:2609.27411v1 Announce Type: new Abstract: Machine fault diagnosis from vibration requires learning from scarce labelled fault recordings while meeting the computational constraints of edge devic

Beyond Static Charts: Can Language and Vision Language Models Generate Interactive Data Visualization Interfaces?

arXiv:2609.26208v1 Announce Type: new Abstract: Data visualization is central to analytical reasoning, but real-world analysis increasingly requires language-driven interactive interfaces rather than

Certified Against Which Oracle? Execution Labels Set the Reported Risk of Conformal Abstention for Text-to-SQL

arXiv:2609.25938v1 Announce Type: cross Abstract: A conformal abstention certificate for text-to-SQL is only as truthful as the correctness labels it is calibrated on. The uncertainty pipelines that r

Deep Reinforcement Learning on Item-Compatibility Graphs for One-Dimensional Bin Packing

arXiv:2609.25397v1 Announce Type: new Abstract: The one-dimensional bin packing problem (1D-BPP) is a classical NP-hard combinatorial optimization problem with applications ranging from logistics and

FrontierMath Erdos

arXiv:2609.25050v1 Announce Type: new Abstract: We introduce FrontierMath Erdos (FME), a benchmark of 68 Erdos problems that are open as of August 2026. To solve a task in FME, AI systems must resolve

Modality-Gated Deep Adapters: Adding a Modality to a Frozen Embedding Model with Exact Preservation

arXiv:2609.26182v1 Announce Type: new Abstract: Multimodal embedding models are deployed at scale: retrieval indices, benchmark results, and behavioral audits all depend on the base model's exact outp

Real-World Perception for Autonomous Driving in Adverse Weather: Enhancing Standard Detectors via Foundation-Guided Auto-Annotation

arXiv:2609.25515v1 Announce Type: new Abstract: Standard deployment-ready object detectors for autonomous vehicles degrade in adverse weather and lighting conditions without being trained on extensive

The Endless Exam: Mathematical Constructions from Today's Models toward Superintelligence

arXiv:2609.24555v1 Announce Type: new Abstract: We introduce the Endless Exam, a benchmark for measuring mathematical progress from today's models toward artificial superintelligence through fourteen

Variational objectives for amortized Bayesian inference in inverse problems: The role of posterior conditioning

arXiv:2609.25145v1 Announce Type: cross Abstract: Variational autoencoders (VAEs) offer an efficient approach to amortized Bayesian inference for inverse problems, but posterior accuracy can depend st

WorkWorlds: An Infrastructure for Evaluating AI Agents on Workplace Tasks

arXiv:2609.23806v1 Announce Type: new Abstract: Many knowledge-work benchmarks are constructed around individual tasks, with the context needed for each task selected together with or after the task h

CE^4L: Continual Ego, Exo, and Ego-Exo Learning

arXiv:2609.23492v1 Announce Type: new Abstract: Perception for embodied agents is video-based, often multi-view (ego, exo, or both), and inherently continual, with simultaneous task and viewpoint shif

GRACE: Grounded Adversarial Reasoning over Canadian Law

arXiv:2609.23726v1 Announce Type: new Abstract: Large language models have shown strong performance across a range of legal tasks, but existing benchmarks rarely evaluate the ability to take and defen

safetyarxiv-cs-cl

IMPLICIT-Bench: Measuring Implicit Bias in Text-to-Image Models under Neutral Prompts

arXiv:2609.24228v1 Announce Type: new Abstract: Text-to-image (T2I) models are typically evaluated for bias using slot-based templates such as ``a photo of a [profession]''. Such templates probe only

Is Imagination Derived from Hallucination? A Cross-Taxonomy Evaluation of Imagination and Hallucination in Large Language Models

arXiv:2609.22152v1 Announce Type: new Abstract: Imagination performs as a high-level function of large language models (LLMs) which determines the potential of how an LLM creates unseen or creative co

Learning and Control Beyond Linearity: Towards a Non-asymptotic Theory for Bilinear Systems

arXiv:2609.22338v1 Announce Type: cross Abstract: This tutorial provides a unified view of the emerging area of bilinear learning and control. Using linear systems as a benchmark, it explains what fun

Research

Matched-Input Estimates Differ in Sign Across Architectures: Auditing EEG Foundation Models on Motor Imagery

arXiv:2609.23924v1 Announce Type: new Abstract: Pretrained EEG foundation models are increasingly proposed as general-purpose encoders for brain-computer interfaces, yet recent benchmarks disagree abo

researcharxiv-cs-lg

Monitorable Chart Reasoning Agents via Verifiable Process Rewards

arXiv:2609.24071v1 Announce Type: new Abstract: Chart reasoning agents are increasingly used to extract actionable insights in critical domains, achieving state-of-the-art performance on multiple benc

Multicentre Bi-atrial Segmentation from LGE-MRI for Atrial Fibrillation with a 2D and 3D Framework

arXiv:2609.22398v1 Announce Type: cross Abstract: Accurate delineation of bi-atrial structures from late gadolinium enhancement MRI (LGE-MRI) is an important prerequisite for structural analysis and f

ReSTI: A Source-Grounded Audit and Repair of STI-Bench

arXiv:2609.24727v1 Announce Type: new Abstract: Spatial--temporal benchmarks are valid only when their questions, source annotations, and answer options identify the same physical quantity. We audit S

safetyarxiv-cs-cv

XYEval: Agents say yes to bad advice

arXiv:2609.23939v1 Announce Type: new Abstract: Effective communication between users and AI agents is essential for human-AI collaboration. The XY problem is a well-known communication pitfall where

A Scene Language Model for Open-Vocabulary Scene Mapping

arXiv:2609.21400v1 Announce Type: new Abstract: Open-vocabulary 3D scene mapping aims to build a persistent representation of the objects in an environment. Existing systems typically rely on engineer

EnSol: an environment-aware graph neural network for molecular solubility prediction

arXiv:2609.21151v1 Announce Type: cross Abstract: Molecular solubility directly affects key aspects of molecular development such as reaction feasibility, formulation performance, separation efficienc

Not All Irregularity Is Equal: Causally Isolating a Rare Failure Mode in Japanese Morphological Inflection

arXiv:2609.21179v1 Announce Type: new Abstract: Neural morphological generation systems often achieve high aggregate accuracy on benchmark datasets, yet such performance can conceal systematic errors

SWE-Proof: Can Language Models Resolve Real-World Issues with Machine-Checked Proofs?

arXiv:2609.21190v1 Announce Type: cross Abstract: Ensuring the correctness of LLM-generated code is a core challenge for modern software engineering. Benchmarks for agentic code generation check corre

VISPATH: Visual-Intent-Guided Path Reasoning for Multimodal Knowledge Graph Question Answering

arXiv:2609.20843v1 Announce Type: new Abstract: Knowledge graph question answering (KGQA) enables models to answer natural-language questions through structured graph reasoning and has achieved substa

A Unified Evaluation Framework for Trustworthy Large Language Models, Agentic AI, and Multimodal Systems

arXiv:2609.19524v1 Announce Type: new Abstract: Benchmark scores alone provide an incomplete basis for assessing the trustworthiness of modern artificial intelligence systems. Large language models (L

18 Sept 2026

Chain-of-Thought Entropy as a Reliability Signal: A Preregistered Reproduction

arXiv:2609.19606v1 Announce Type: new Abstract: This empirical study is an independent reproduction of the dissociation Zhao reported in 2026. The shape of a large language model's chain-of-thought en

18 Sept 2026

Comparing Commercial Depth Sensor Accuracy for Medical Applications

arXiv:2606.13028v2 Announce Type: replace-cross Abstract: Depth estimation has numerous medical and surgical applications. We benchmark four depth sensors on a porcine bone specimen, a porcine belly s