Search: “benchmark”

ReFACT: A Benchmark for Scientific Confabulation Detection with Positional Error Annotations

arXiv:2509.25868v3 Announce Type: replace Abstract: The mechanisms underlying scientific confabulation in Large Language Models (LLMs) remain poorly understood. We introduce ReFACT (Reddit False And C

24 Apr 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

The First Challenge on Remote Sensing Infrared Image Super-Resolution at NTIRE 2026: Benchmark Results and Method Overview

DGX agent

arXiv:2604.21312v1 Announce Type: cross Abstract: This paper presents the NTIRE 2026 Remote Sensing Infrared Image Super-Resolution (x4) Challenge, one of the associated challenges of NTIRE 2026. The

model-releasesarxiv-cs-ai

24 Apr 2026

Benchmarking ResNet for Short-Term Hypoglycemia Classification with DiaData

arXiv:2511.02849v2 Announce Type: replace-cross Abstract: Individualized therapy is driven forward by medical data analysis, which provides insight into the patient's context. In particular, for Type

Safety

LLM Agents Predict Social Media Reactions but Do Not Outperform Text Classifiers: Benchmarking Simulation Accuracy Using 120K+ Personas of 1511 Humans

arXiv:2604.19787v1 Announce Type: cross Abstract: Social media platforms mediate how billions form opinions and engage with public discourse. As autonomous AI agents increasingly participate in these

safetyarxiv-cs-ai

Research

On the Quantization Robustness of Diffusion Language Models in Coding Benchmarks

arXiv:2604.20079v1 Announce Type: cross Abstract: Auto-regressive Large Language Models (LLMs) achieve strong performance on coding tasks, but incur high memory and inference costs. Diffusion-based la

researcharxiv-cs-cl

RareSpot+: A Benchmark, Model, and Active Learning Framework for Small and Rare Wildlife in Aerial Imagery

arXiv:2604.20000v1 Announce Type: new Abstract: Automated wildlife monitoring from aerial imagery is vital for conservation but remains limited by two persistent challenges: the difficulty of detectin

SpeechParaling-Bench: A Comprehensive Benchmark for Paralinguistic-Aware Speech Generation

arXiv:2604.20842v1 Announce Type: cross Abstract: Paralinguistic cues are essential for natural human-computer interaction, yet their evaluation in Large Audio-Language Models (LALMs) remains limited

SurgCoT: Advancing Spatiotemporal Reasoning in Surgical Videos through a Chain-of-Thought Benchmark

arXiv:2604.20319v1 Announce Type: new Abstract: Fine-grained spatiotemporal reasoning on surgical videos is critical, yet the capabilities of Multi-modal Large Language Models (MLLMs) in this domain r

Towards Secure Logging: Characterizing and Benchmarking Logging Code Security Issues with LLMs

arXiv:2604.20211v1 Announce Type: cross Abstract: Logging code plays an important role in software systems by recording key events and behaviors, which are essential for debugging and monitoring. Howe

Safety

Benchmarking Misuse Mitigation Against Covert Adversaries

arXiv:2506.06414v2 Announce Type: replace-cross Abstract: Existing language model safety evaluations focus on overt attacks and low-stakes tasks. In reality, an attacker can easily subvert existing sa

safetyarxiv-cs-ai

Research

Benchmarking Vision Foundation Models for Domain-Generalizable Face Anti-Spoofing

arXiv:2604.19196v1 Announce Type: new Abstract: Face Anti-Spoofing (FAS) remains challenging due to the requirement for robust domain generalization across unseen environments. While recent trends lev

researcharxiv-cs-cv

CASS: Nvidia to AMD Transpilation with Data, Models, and Benchmark

arXiv:2505.16968v4 Announce Type: replace-cross Abstract: Cross-architecture GPU code transpilation is essential for unlocking low-level hardware portability, yet no scalable solution exists. We intro

Chat2Workflow: A Benchmark for Generating Executable Visual Workflows with Natural Language

arXiv:2604.19667v1 Announce Type: cross Abstract: At present, executable visual workflows have emerged as a mainstream paradigm in real-world industrial deployments, offering strong reliability and co

CrossPan: A Comprehensive Benchmark for Cross-Sequence Pancreas MRI Segmentation and Generalization

arXiv:2604.18797v1 Announce Type: new Abstract: Automatic pancreas segmentation is fundamental to abdominal MRI analysis, yet deep learning models trained on one MRI sequence often fail catastrophical

Applications

From Top-1 to Top-K: A Reproducibility Study and Benchmarking of Counterfactual Explanations for Recommender Systems

arXiv:2604.19663v1 Announce Type: cross Abstract: Counterfactual explanations (CEs) provide an intuitive way to understand recommender systems by identifying minimal modifications to user-item interac

applicationsarxiv-cs-lg

SMART-Ship: A Comprehensive Synchronized Multi-modal Aligned Remote Sensing Targets Dataset and Benchmark for Berthed Ships Analysis

arXiv:2508.02384v2 Announce Type: replace Abstract: Given the limitations of satellite orbits and imaging conditions, multi-modal remote sensing (RS) data is crucial in enabling long-term earth observ

A Benchmark Study of Segmentation Models and Adaptation Strategies for Landslide Detection from Satellite Imagery

arXiv:2604.16663v1 Announce Type: new Abstract: Landslide detection from high resolution satellite imagery is a critical task for disaster response and risk assessment, yet the relative effectiveness

Benchmarking System Dynamics AI Assistants: Cloud Versus Local LLMs on CLD Extraction and Discussion

arXiv:2604.18566v1 Announce Type: cross Abstract: We present a systematic evaluation of large language model families -- spanning both proprietary cloud APIs and locally-hosted open-source models -- o

Beyond Word Boundaries: A Hebrew Coreference Benchmark and an Evaluation Protocol for Morphologically Complex Text

arXiv:2604.17108v1 Announce Type: new Abstract: Coreference Resolution (CR) is a fundamental NLP task critical for long-form tasks as information extraction, summarization, and many business applicati

EgoSound: Benchmarking Sound Understanding in Egocentric Videos

arXiv:2602.14122v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have recently achieved remarkable progress in vision-language understanding. Yet, human perception is inher

LexRel: Benchmarking Legal Relation Extraction for Chinese Civil Cases

arXiv:2512.12643v2 Announce Type: replace Abstract: Legal relations serve as an important analytical framework for dispute resolution in civil cases. However, legal relations in Chinese civil cases re

Same Claim, Different Judgment: Benchmarking Scenario-Induced Bias in Multilingual Financial Misinformation Detection

arXiv:2601.05403v2 Announce Type: replace Abstract: Large language models (LLMs) have been widely applied across various domains of finance. Since their training data are largely derived from human-au

WeatherArchive-Bench: Benchmarking Retrieval-Augmented Reasoning for Historical Weather Archives

arXiv:2510.05336v2 Announce Type: replace Abstract: Historical archives on weather events are collections of enduring primary source records that offer rich, untapped narratives of how societies have

Beyond MCQ: An Open-Ended Arabic Cultural QA Benchmark with Dialect Variants

arXiv:2510.24328v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are increasingly used to answer everyday questions, yet their performance on culturally grounded and dialectal co

COMPASS: Benchmarking Constrained Optimization in LLM Agents

arXiv:2510.07043v2 Announce Type: replace Abstract: Human decision-making often involves constrained optimization. As LLM agents are deployed to assist with real-world tasks like travel planning, shop

ECG-Lens: Benchmarking ML & DL Models on PTB-XL Dataset

arXiv:2604.15822v1 Announce Type: cross Abstract: Automated classification of electrocardiogram (ECG) signals is a useful tool for diagnosing and monitoring cardiovascular diseases. This study compare

From Zero to Detail: A Progressive Spectral Decoupling Paradigm for UHD Image Restoration with New Benchmark

arXiv:2604.15654v1 Announce Type: new Abstract: Ultra-high-definition (UHD) image restoration poses unique challenges due to the high spatial resolution, diverse content, and fine-grained structures p

MemEvoBench: Benchmarking Memory MisEvolution in LLM Agents

arXiv:2604.15774v1 Announce Type: new Abstract: Equipping Large Language Models (LLMs) with persistent memory enhances interaction continuity and personalization but introduces new safety risks. Speci

RoleConflictBench: A Benchmark of Role Conflict Scenarios for Evaluating LLMs' Contextual Sensitivity

arXiv:2509.25897v2 Announce Type: replace-cross Abstract: People often encounter role conflicts -- social dilemmas where the expectations of multiple roles clash and cannot be simultaneously fulfilled

TwoHamsters: Benchmarking Multi-Concept Compositional Unsafety in Text-to-Image Models

arXiv:2604.15967v1 Announce Type: cross Abstract: Despite the remarkable synthesis capabilities of text-to-image (T2I) models, safeguarding them against content violations remains a persistent challen

AD4AD: Benchmarking Visual Anomaly Detection Models for Safer Autonomous Driving

arXiv:2604.15291v1 Announce Type: new Abstract: The reliability of a machine vision system for autonomous driving depends heavily on its training data distribution. When a vehicle encounters significa

Benchmarking Linguistic Adaptation in Comparable-Sized LLMs: A Study of Llama-3.1-8B, Mistral-7B-v0.1, and Qwen3-8B on Romanized Nepali

arXiv:2604.14171v1 Announce Type: new Abstract: Romanized Nepali, the Nepali language written in the Latin alphabet, is the dominant medium for informal digital communication in Nepal, yet it remains

Benchmarking Optimizers for MLPs in Tabular Deep Learning

arXiv:2604.15297v1 Announce Type: new Abstract: MLP is a heavily used backbone in modern deep learning (DL) architectures for supervised learning on tabular data, and AdamW is the go-to optimizer used

CoPA: Benchmarking Personalized Question Answering with Data-Informed Cognitive Factors

arXiv:2604.14773v1 Announce Type: new Abstract: While LLMs have demonstrated remarkable potential in Question Answering (QA), evaluating personalization remains a critical bottleneck. Existing paradig

DeEscalWild: A Real-World Benchmark for Automated De-Escalation Training with SLMs

arXiv:2604.13075v1 Announce Type: new Abstract: Effective de-escalation is critical for law enforcement safety and community trust, yet traditional training methods lack scalability and realism. While

HINTBench: Horizon-agent Intrinsic Non-attack Trajectory Benchmark

arXiv:2604.13954v1 Announce Type: new Abstract: Existing agent-safety evaluation has focused mainly on externally induced risks. Yet agents may still enter unsafe trajectories under benign conditions.

LaoBench: A Large-Scale Multidimensional Lao Benchmark for Large Language Models

arXiv:2511.11334v3 Announce Type: replace Abstract: The rapid advancement of large language models (LLMs) has not been matched by their evaluation in low-resource languages, especially Southeast Asian

LongCoT: Benchmarking Long-Horizon Chain-of-Thought Reasoning

arXiv:2604.14140v1 Announce Type: new Abstract: As language models are increasingly deployed for complex autonomous tasks, their ability to reason accurately over longer horizons becomes critical. An

MERRIN: A Benchmark for Multimodal Evidence Retrieval and Reasoning in Noisy Web Environments

arXiv:2604.13418v1 Announce Type: new Abstract: Motivated by the underspecified, multi-hop nature of search queries and the multimodal, heterogeneous, and often conflicting nature of real-world web re

Synthetic Tabular Generators Fail to Preserve Behavioral Fraud Patterns: A Benchmark on Temporal, Velocity, and Multi-Account Signals

arXiv:2604.13125v1 Announce Type: new Abstract: We introduce behavioral fidelity -- a third evaluation dimension for synthetic tabular data that measures whether generated data preserves the temporal,

Benchmarking Foundation Models with Retrieval-Augmented Generation in Olympic-Level Physics Problem Solving

arXiv:2510.00919v3 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) with foundation models has achieved strong performance across diverse tasks, but their capacity for exper

CodeSpecBench: Benchmarking LLMs for Executable Behavioral Specification Generation

arXiv:2604.12268v1 Announce Type: cross Abstract: Large language models (LLMs) can generate code from natural language, but the extent to which they capture intended program behavior remains unclear.

CompliBench: Benchmarking LLM Judges for Compliance Violation Detection in Dialogue Systems

arXiv:2604.12312v1 Announce Type: new Abstract: As Large Language Models (LLMs) are increasingly deployed as task-oriented agents in enterprise environments, ensuring their strict adherence to complex

Research

DeepTest Tool Competition 2026: Benchmarking an LLM-Based Automotive Assistant

arXiv:2604.12615v1 Announce Type: new Abstract: This report summarizes the results of the first edition of the Large Language Model (LLM) Testing competition, held as part of the DeepTest workshop at

researcharxiv-cs-ai

League of LLMs: A Benchmark-Free Paradigm for Mutual Evaluation of Large Language Models

arXiv:2507.22359v4 Announce Type: replace Abstract: Although large language models (LLMs) have shown exceptional capabilities across a wide range of tasks, reliable evaluation remains a critical chall

AffordSim: A Scalable Data Generator and Benchmark for Affordance-Aware Robotic Manipulation

arXiv:2604.11674v1 Announce Type: cross Abstract: Simulation-based data generation has become a dominant paradigm for training robotic manipulation policies, yet existing platforms do not incorporate

14 Apr 2026

BareBones: Benchmarking Zero-Shot Geometric Comprehension in VLMs

arXiv:2604.10528v1 Announce Type: new Abstract: While Vision-Language Models (VLMs) demonstrate remarkable zero-shot recognition capabilities across a diverse spectrum of multimodal tasks, it yet rema

14 Apr 2026

Exploring Knowledge Conflicts for Faithful LLM Reasoning: Benchmark and Method

arXiv:2604.11209v1 Announce Type: cross Abstract: Large language models (LLMs) have achieved remarkable success across a wide range of applications especially when augmented by external knowledge thro