Search: “benchmark”

High Performance, Low Reliability: Uncertainty Benchmarking for Tabular Foundation Models

arXiv:2605.28554v1 Announce Type: new Abstract: Recent Tabular Foundation Models (TFMs) have demonstrated state-of-the-art predictive performance, often surpassing Gradient-Boosted Decision Trees (GBD

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

HRBench: Benchmarking and Understanding Thinking-Mode Switch Strategies in Hybrid-Reasoning LLMs

DGX agent

arXiv:2605.28398v1 Announce Type: new Abstract: Hybrid-reasoning large language models (LLMs) expose explicit controls over reasoning effort, allowing users or systems to trade off answer quality agai

researcharxiv-cs-ai

28 May 2026

MIRA: A Bilingual Benchmark for Medical Information Response Audit

arXiv:2605.28025v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used to provide public-facing health information, yet existing safety evaluations overlook whether respons

SuiChat-CN: Benchmarking Contextual Suicide Risk Assessment in Chinese Group Chats

arXiv:2605.27911v1 Announce Type: new Abstract: Suicide is a critical global public health challenge, causing approximately 720,000 deaths each year and calling for timely, effective prevention strate

What-If World: A Causal Benchmark for General World Models in Embodied Scenarios

arXiv:2605.27589v1 Announce Type: new Abstract: Video generation models are increasingly used as world simulators for tasks like driving and robotic manipulation. What matters in these settings is not

Anchor: Mitigating Artifact Drift in Agent Benchmark Generation

arXiv:2605.26321v1 Announce Type: new Abstract: AI agents are beginning to complete valuable, long-horizon business operations tasks, but training and evaluation environments for enterprise work still

Benchmarking Convolutional, Transformer, Hybrid, and Vision Language Models for Multi Disease Retinal Screening

arXiv:2605.26283v1 Announce Type: new Abstract: Modern deep learning offers powerful tools for automated retinal screening, but it remains unclear how different visual model families compare in realis

Cesarean Scar Defect Segmentation in Transvaginal Ultrasound Images: a Dataset and Benchmark

arXiv:2605.26774v1 Announce Type: new Abstract: Cesarean Scar Defect (CSD) is one of the most prevalent complications following cesarean delivery. Transvaginal ultrasonography is widely used for prima

ENPMR-Bench: Benchmarking Proactive Memory Retrieval for Emotional Support Agents

arXiv:2605.27240v1 Announce Type: new Abstract: Memory-augmented language agents are increasingly deployed in affective applications such as emotional support, where understanding and responding to us

OmniToM: Benchmarking Theory of Mind in LLMs via Explicit Belief Modeling

arXiv:2605.26322v1 Announce Type: new Abstract: Theory of Mind (ToM), the ability to infer others' knowledge, intentions, and emotions, is commonly evaluated in large language models (LLMs) using end-

PersLitEval: Fine-grained Benchmark and Evaluation of LLMs on Persian Literature Questions

arXiv:2605.27015v1 Announce Type: new Abstract: Despite impressive multilingual capabilities, large language models (LLMs) remain poorly evaluated on literary knowledge in non-English languages. We in

Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History

arXiv:2602.17003v2 Announce Type: replace-cross Abstract: Large language models have advanced web agents, yet current agents lack personalization capabilities. Since users rarely specify every detail

PRBench: A Standardized Probabilistic Robustness Benchmark

arXiv:2511.01724v3 Announce Type: replace Abstract: Deep learning models are notoriously vulnerable to imperceptible perturbations. Most existing research centers on adversarial robustness (AR), which

Receipt Replay OOD: A Small Benchmark for Screen Replay Detection Under Domain Shift

arXiv:2605.26855v1 Announce Type: new Abstract: Public datasets such as DLC-2021, SynID, and KID34K have significantly contributed to research on presentation attack detection for identity documents,

A Matched Spectral Benchmark of Quantum Inspired Feature Maps

arXiv:2605.24324v1 Announce Type: cross Abstract: Quantum machine learning is often motivated by the idea that quantum systems can expose useful high-dimensional structure that is difficult to access

AgentHijack: Benchmarking Computer Use Agent Robustness to Common Environment Corruptions

arXiv:2605.25707v1 Announce Type: new Abstract: Autonomous computer use agents that powered by multimodal large language models (MLLMs) are emerging as capable assistants for completing complex digita

Benchmarking and Improving Monitors for Out-Of-Distribution Alignment Failure in LLMs

arXiv:2605.21602v2 Announce Type: replace Abstract: Many safety and alignment failures of large language models (LLMs) occur due to out-of-distribution (OOD) situations: unusual prompt or response pat

ContextEcho: A Benchmark for Persona Drift in Long Agentic-Coding Sessions

arXiv:2605.24279v1 Announce Type: new Abstract: A frontier language model's acknowledged 'helpful programming assistant' persona does not survive long agentic-coding sessions in the deployment regime

FoodMonitor: Benchmarking MLLMs for Explainable Compliance Analysis

arXiv:2605.24503v1 Announce Type: cross Abstract: As AI-powered compliance monitoring becomes increasingly important in public governance and industrial safety, the ability to provide verifiable evide

LLM-as-a-Reviewer: Benchmarking Their Ability, Divergence, and Prompt Injection Resistance as Paper Reviewers

arXiv:2605.25415v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used in academic peer review, yet their reliability, alignment with human judgment, and robustness to adve

Personalize-then-Store: Benchmarking and Learning Personalized Memory for Long-horizon Agents

arXiv:2605.25535v1 Announce Type: new Abstract: Existing large language model (LLM) based memory systems apply universal, static policies that overlook a fundamental reality: the contexts that are wor

Rethinking Continual Anomaly Detection on the Edge: Benchmarking Under Realistic Industrial Conditions

arXiv:2605.24251v1 Announce Type: new Abstract: Continual anomaly detection (CAD) addresses the need for industrial inspection systems to adapt to evolving production conditions, yet existing methods

SkillEvolBench: Benchmarking the Evolution from Episodic Experience to Procedural Skills

arXiv:2605.24117v1 Announce Type: new Abstract: Large language model (LLM) agents accumulate rich episodic trajectories while solving real-world tasks, but it remains unclear whether such experience c

StreamProfileBench: A Benchmark for Fine-Grained User Profile Inference in Real-World Streaming Scenarios

arXiv:2605.25758v1 Announce Type: new Abstract: Large Language Models (LLMs) have reshaped user profiling, yet current evaluations mainly focus on static data snapshots. This paradigm overlooks the re

The Age of Curiosity Meets the Age of AI: Benchmarking Child Safety in Large Language Models

arXiv:2605.25510v1 Announce Type: new Abstract: Children increasingly have access to Large Language Models (LLMs), which may expose them to responses that are developmentally inappropriate or require

The LSCD Benchmark: a Testbed for Diachronic Word Meaning Tasks

arXiv:2404.00176v3 Announce Type: replace Abstract: Lexical Semantic Change Detection (LSCD) is a complex, lemma-level task, which is usually operationalized based on two subsequently applied usage-le

TimeSpot: Benchmarking Geo-Temporal Understanding in Vision-Language Models in Real-World Settings

arXiv:2603.06687v2 Announce Type: replace-cross Abstract: Geo-temporal understanding, the ability to infer location, time, and contextual properties from visual input alone, underpins applications suc

ViroBench: Benchmarking Nucleotide Foundation Models on Viral Genomics Tasks

arXiv:2605.25388v1 Announce Type: new Abstract: Nucleotide sequences constitute the fundamental genetic basis of biological systems, rendering viral genomic analysis critical for biomedical advancemen

Agents

EvalVerse: Pipeline-Aware and Expert-Calibrated Benchmarking for Professional Cinematic Video Generation

arXiv:2605.23271v1 Announce Type: cross Abstract: The rapid evolution of generative video foundation models has propelled the field toward professional-grade cinematic synthesis. To achieve such deman

agentsarxiv-cs-ai

25 May 2026

StereoGenBench: A Synthetic Multi-Camera Benchmark for Stereo Generation under Controlled Baseline Regimes

arXiv:2605.23237v1 Announce Type: new Abstract: Stereo image and video generation, stereo geometry estimation, and condition-controlled view synthesis require paired data in which the variables that d

25 May 2026

Tutorials

Benchmarking Machine Learning Architectures for Antimicrobial Stewardship in Pediatric ICUs

arXiv:2605.22611v1 Announce Type: new Abstract: Antimicrobial stewardship (AMS) is critical in pediatric intensive care units (PICUs), where diagnostic uncertainty often drives broad-spectrum antibiot

tutorialsarxiv-cs-lg

Measuring Cross-Modal Synergy: A Benchmark for VLM Explainability

arXiv:2605.22168v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) map complex visual inputs to semantic spaces, but interpreting the cross-modal reasoning of VLMs currently relies on pos

MoralityGym: A Benchmark for Evaluating Hierarchical Moral Alignment in Sequential Decision-Making Agents

arXiv:2602.13372v2 Announce Type: replace-cross Abstract: Evaluating moral alignment in agents navigating conflicting, hierarchically structured human norms is a critical challenge at the intersection

Stabilising Explainability Fragility in Cybersecurity AI: The Impact and Mitigation of Multicollinearity in Public Benchmark Datasets

arXiv:2605.22529v1 Announce Type: new Abstract: This paper investigates a unexplored yet impactful vulnerability in AI explainability used in intrusion detection (IDS): multicollinearity-induced insta

AgroVG: A Large-Scale Multi-Source Benchmark for Agricultural Visual Grounding

arXiv:2605.22034v1 Announce Type: new Abstract: Visual grounding, the task of localizing objects described by natural-language expressions, is a foundational capability for agricultural AI systems, en

ChronoMedKG: A Temporally-Grounded Biomedical Knowledge Graph and Benchmark for Clinical Reasoning

arXiv:2605.22734v1 Announce Type: new Abstract: Biomedical knowledge graphs (KGs) treat disease associations as static facts, but temporal information is crucial for clinical reasoning, e.g., a sympto

Security Document Classification with a Fine-Tuned Local Large Language Model: Benchmark Data and an Open-Source System

arXiv:2605.20368v1 Announce Type: cross Abstract: Organizations that scan documents for sensitive information face a practical problem. Cloud services require data to be sent to external infrastructur

Agents

STRUCTSENSE: A Task-Agnostic Agentic Framework for Structured Information Extraction with Human-In-The-Loop Evaluation and Benchmarking

arXiv:2507.03674v3 Announce Type: replace Abstract: Extracting structured information from scientific literature is critical for accelerating discovery, yet Large Language Models (LLMs) often struggle

agentsarxiv-cs-cl

ArchSIBench: Benchmarking the Architectural Spatial Intelligence of Vision-Language Models

arXiv:2605.20837v1 Announce Type: new Abstract: Architectural spatial intelligence, the ability to recognize and infer architectural space, is fundamental to tasks such as robot navigation, embodied i

Benchmarking Empirical and Learning-Based Approaches for Feedforward Steering Control in Autonomous Racing

model-releasesarxiv-cs-ro

arXiv:2605.21111v1 Announce Type: new Abstract: Feedforward steering control is a key component of hierarchical control architectures for autonomous racing. The goal is to reduce steering corrections

Fine-grained Claim-level RAG Benchmark for Law

arXiv:2605.21071v1 Announce Type: new Abstract: The rapid progress of large language models (LLMs) is shifting semantic search toward a question-answering paradigm, where users ask questions and LLMs

GraphRAG on Consumer Hardware: Benchmarking Local LLMs for Healthcare EHR Schema Retrieval

arXiv:2605.20815v1 Announce Type: new Abstract: Graph-based Retrieval Augmented Generation (GraphRAG) extends retrieval-augmented generation to support structured reasoning over complex corpora, but i

HalluCXR: Benchmarking and Mitigating Hallucinations in Medical Vision-Language Models for Chest Radiograph Interpretation

arXiv:2605.20469v1 Announce Type: new Abstract: Vision-language models (VLMs) are increasingly used for medical image interpretation, yet they frequently hallucinate, generating clinically plausible b

Research

Lowering the Barrier to IREX Participation: Open-Source Algorithms, Toolkit, and Benchmarking for Iris Recognition

arXiv:2605.20735v1 Announce Type: new Abstract: This paper proposes two new open-source iris recognition algorithms, providing both Python and IREX-compliant C++ implementations to be submitted to the

researcharxiv-cs-cv

ShadeBench: A Benchmark Dataset for Building Shade Simulation in Sustainable Society

arXiv:2605.20510v1 Announce Type: new Abstract: Urban heat exposure is becoming an increasingly critical challenge due to the intensifying urban heat island effect. Fine-grained shade patterns, especi

Benchmarking and Evolving Reason-Reflect-Rectify for Reflective Visual Generation

arXiv:2605.19639v1 Announce Type: new Abstract: Text-to-Image (T2I) models and Unified Multimodal Models (UMMs) have achieved remarkable progress in visual generation. However, their reliance on a sin

20 May 2026

BuildArena: A Physics-Aligned Interactive Benchmark of LLMs for Engineering Construction

arXiv:2510.16559v5 Announce Type: replace Abstract: Engineering construction automation aims to transform natural language specifications into physically viable structures, requiring complex integrate

20 May 2026

CogScale: Scalable Benchmark for Sequence Processing

arXiv:2605.19758v1 Announce Type: new Abstract: The ability to maintain and manipulate information over time is a fundamental aspect of living beings and Artificial Intelligence. While modern models h