Search: “benchmark”

AmbiModBench: Benchmarking Gene Perturbation Prediction Beyond Shared Responses

arXiv:2609.32527v1 Announce Type: cross Abstract: Predicting cellular responses to genetic perturbations helps prioritize experiments in single-cell genomics, where exhaustive measurement is infeasibl

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

AnesTRACE: Benchmarking Intraoperative Anesthesia from Multimodal Perception to Multi-step Decision-Making

DGX agent

arXiv:2609.32740v1 Announce Type: new Abstract: Intraoperative anesthesia requires systems to interpret evolving multimodal evidence, recommend timely management, and revise decisions as patient state

safetyarxiv-cs-cv

29 Sept 2026

ARCH-B: Architectural Representation, Comprehension and Hierarchy Benchmark

arXiv:2609.34047v1 Announce Type: new Abstract: Multimodal models increasingly interpret visual environments, but their ability to recognize the same building across photographs, floor plans, elevatio

Beyond Geometry: Benchmarking and Consistency Reasoning for 3D Logical Anomaly Detection

arXiv:2609.34143v1 Announce Type: new Abstract: Existing 3D industrial anomaly detection mainly targets local geometric deviations. In contrast, many industrial anomalies violate object-level design o

ConflictVLA-Bench: Benchmarking Behavioral Responses of Vision-Language-Action Models to Premise Conflicts

arXiv:2609.31792v1 Announce Type: cross Abstract: While Vision-Language-Action (VLA) models perform strongly on manipulation tasks, their responses to invalid task premises remain underexplored. Exist

CoSE-E: A Benchmark for Code-switched Speech Evaluation in Enterprise Settings

arXiv:2609.35645v1 Announce Type: cross Abstract: Code-switching (CS), a seamless alternation between languages within a single utterance, remains a critical challenge in automatic speech recognition

Cross-Dataset Generalization of Bangladeshi Rice Leaf Disease Classifiers: Benchmark, Diagnosis, and Mitigation

arXiv:2609.31709v1 Announce Type: new Abstract: Cross-dataset transfer in rice leaf disease classification remains a significant challenge, with models trained on one image collection performing subst

Deep Reinforcement Learning for Equity Trading: Benchmarking Actor-Critic Methods with Forward Retraining

arXiv:2609.31870v1 Announce Type: new Abstract: Consistently profitable trading is difficult because equity markets are noisy, non-stationary, and only partially predictable from historical data. We b

eval-unlearn: Benchmarking unlearning in Text-to-Image Diffusion Models

arXiv:2609.35269v1 Announce Type: cross Abstract: The rising number of concept unlearning techniques for text-to-image (T2I) diffusion models has produced a fragmented evaluation landscape. Methods ar

Generative AI-Based Data Augmentation for Oral Lesion Classification: The PhotoMOCI Dataset and Benchmark

arXiv:2609.35226v1 Announce Type: new Abstract: Early detection of oral cancer via photographic imaging presents a promising avenue for large-scale oral cavity screening. However, the development of r

Jev in Medicine: A Benchmark Evaluation. Preliminary Results

arXiv:2609.34024v1 Announce Type: cross Abstract: Jev is a non-generative 'System One' model that assigns probabilities to predefined answer options and cannot answer outside them. Its accuracy and ca

Large Language Models for Automated Cross-Domain Machine Learning Task Type Identification: A Benchmark Dataset and Evaluation

arXiv:2609.35335v1 Announce Type: new Abstract: Machine learning task type identification is essential for constructing valid ML pipelines, yet in practice it is typically specified manually. We inves

MemTransfer: Benchmarking Memory Beyond Matched Experience in Embodied Decision-Making

arXiv:2609.32313v1 Announce Type: new Abstract: Memory lets an embodied agent reuse past experience, yet retaining useful information does not ensure that the agent can apply it when conditions change

NetInjectBench: Benchmarking Indirect Prompt Injection in Tool-Using Large Language Model Agents for Network Operations

arXiv:2607.10490v2 Announce Type: replace-cross Abstract: Tool-using large language model (LLM) agents are attractive for network operations, but tickets, alerts, logs, runbooks, and ChatOps messages

One-Step Is Optimal: Unconditional Rectified Flows are Noise2Noise Denoisers, and Multi-Step Integration Provably Hurts---A Benchmark and Task-Based Detectability Study on Low-Dose CT

arXiv:2609.31670v1 Announce Type: new Abstract: Iterative and generative denoisers are increasingly used under the assumption that multi-step refinement outperforms a single regression pass. We show t

OPIS: An Input-Grounded Benchmark for Multi-Object Memory in Video World Models

arXiv:2609.35052v1 Announce Type: new Abstract: Video world models must preserve the visual state of the world over time, but existing evaluation protocols often rely on generated histories, video ref

ORAV: Benchmarking Audio-Video Generation from Multimodal Contexts

arXiv:2609.34843v1 Announce Type: new Abstract: Audio-video generation using heterogeneous multimodal references has emerged as a new challenge, requiring both compositional control over generation an

SV2V-RSim: A Comprehensive Benchmark for Self-Selective V2V Cooperative Perception with Near-Realistic Data

arXiv:2609.32863v1 Announce Type: new Abstract: Vehicle-to-Vehicle (V2V) cooperative perception enhances autonomous driving by enabling vehicles to share information beyond their direct line of sight.

THEIA: A Multimodal Dataset and Benchmark for Vision-Language Analysis of Layout

arXiv:2609.35035v1 Announce Type: new Abstract: The integration of artificial intelligence into computer-aided design frameworks has sparked a shift in the design of analog integrated circuits (ICs),

Traceable Human-to-Humanoid Sign Language Benchmarking

arXiv:2609.33354v1 Announce Type: new Abstract: Sign data collection is costly, and teleoperation scales poorly, motivating reuse of large video corpora. Humanoid signing requires converting video-der

When Does Geometric View Synthesis Help Wine Label Retrieval? A Public One-Shot Benchmark Across Self-Supervised and Vision-Language Backbones

arXiv:2609.33359v1 Announce Type: new Abstract: Geometric view synthesis can expand a single wine-label photograph into a training set, but its value with pretrained image encoders is unclear. We stud

Research

Which Self-Improvements Should We Trust? Reliable Self-Improvement When Agents Reuse Their Benchmarks

arXiv:2609.33180v1 Announce Type: cross Abstract: As recursive self-improvement (RSI) rapidly advances, reliable evaluation becomes critical for guiding adaptive search. RSI typically relies on finite

researcharxiv-cs-lg

A Benchmark and Diagnostic Study of Epistemic Admission in Shared Agent Memory

arXiv:2609.30813v1 Announce Type: new Abstract: Evaluating claim admission in shared agent memory is challenging because repeated claims may be mistaken for independent evidence. An agent may copy or

28 Sept 2026

Artificial Societies Benchmark: A Validation Framework for Synthetic Research

arXiv:2609.30030v1 Announce Type: new Abstract: A synthetic survey can reproduce the average answer while misrepresenting how people differ, how their answers relate to one another, or how they respon

Benchmarking Arabic--Russian Machine Translation: A Comparison of Fine-tuned NMT and Few-shot LLMs under Rich Morphology and Low Lexical Overlap

arXiv:2609.29559v1 Announce Type: new Abstract: Arabic-Russian machine translation (MT) remains under-explored due to the rich morphology of Arabic and low lexical overlap between the two languages. W

Benchmarking Argumentative Behaviour of LLMs: A Study of Defences Against Character Attacks

arXiv:2609.28673v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly deployed as argumentative agents in persuasive dialogues, necessitating rigorous evaluation of their debat

Clinical Intent Extraction: A FHIR-Aligned Representation and the CIRCA Benchmark

arXiv:2609.29479v1 Announce Type: new Abstract: Prospective clinical actions, the follow-ups, orders, referrals, and instructions that deter-mine what happens to a patient next, are annotated today in

Language Specificity vs. Domain Diversity: Benchmarking Transformers for Bangla Medical NER

arXiv:2609.29101v1 Announce Type: new Abstract: Medical Named Entity Recognition (NER) for low-resource languages remains a challenging task due to high linguistic variability and a scarcity of domain

Benchmarking Active Spot Selection for Cost-Efficient Spatial Transcriptomics

arXiv:2609.27208v1 Announce Type: new Abstract: Spatial transcriptomics (ST) measures gene expression in tissue context, but dense capture grids can be costly and may repeatedly sample morphologically

Benchmarking Hyperspectral Foundation Models for Hyperspectral Unmixing

arXiv:2609.28283v1 Announce Type: new Abstract: Several foundation models dedicated to hyperspectral images have recently been made available. These models are trained on large unlabeled datasets and

Can LLMs Catch a Rigged Backtest? A Clean-Control Calibration Benchmark

arXiv:2609.28090v1 Announce Type: cross Abstract: Backtest auditing is a calibration problem: high flaw recall is not useful when the model falsely flags matched clean strategies. We build a 96-item p

Safety

EmbodiedMemory-Bench: Benchmarking Embodied Memory for Long-Horizon Embodied Tasks

arXiv:2609.28236v1 Announce Type: new Abstract: Long-horizon embodied interaction requires agents to retain and continually update information about the environment as they observe, act, and encounter

safetyarxiv-cs-cv

EvEMTBench: An Open Benchmark for Machine Learning in Power System Protection

arXiv:2609.28149v1 Announce Type: cross Abstract: Studies of machine-learning-based power system protection are difficult to compare because task definitions, measurement access, data partitions, metr

Google launches two benchmark-topping speech generation models

model-releasessiliconangle

Google LLC today made two new text-to-speech models, Gemini 3.8 Flash TTS and Gemini 3.8 Flash-Lite TTS, available through its cloud platform. The algorithms have highly similar application programmin

MLPerf Training Introduces Its First LLM Post-Training Benchmark

MLPerf Training v6.1 adds an agentic reinforcement-learning workload that measures how quickly systems can teach a 397-billion-parameter language model to repair real software projects. The post MLPer

model-releasesmlcommons

MultiVENT-Raw: A Benchmark for Retrieval and Reasoning over Raw Videos

arXiv:2609.28437v1 Announce Type: new Abstract: Online information is increasingly consumed in video format. Much of this comes in the form of *raw video*: continuous footage taken on a cell phone, wi

SafeTutors: Benchmarking Pedagogical Safety in AI Tutoring Systems

arXiv:2603.17373v2 Announce Type: replace Abstract: Large language models are rapidly being deployed as AI tutors, yet current evaluation paradigms assess problem-solving accuracy and generic safety i

WAInjectBench: Benchmarking Prompt Injection Detections for Web Agents

arXiv:2510.01354v2 Announce Type: replace-cross Abstract: Multiple prompt injection attacks have been proposed against web agents. At the same time, various methods have been developed to detect gener

DUMA-Bench: A Dual-Control Multi-Agent Benchmark for Evaluating LLM Agent Security

arXiv:2609.24662v1 Announce Type: new Abstract: LLM-based agents increasingly operate in environments where they interact with users, tools, and external systems. Yet most security evaluations assume

Label-Efficient Learning for Ground-Based Sky-Image Classification: A Benchmark of Transfer Learning, Active Learning, and Pseudo-Labeling on GCD

arXiv:2609.26631v1 Announce Type: new Abstract: Accurate ground-based cloud classification is important for atmospheric monitoring, solar-energy forecasting, aviation weather assessment, and climate o

MatcherCompass: A Deployment-Aware Benchmark to Guide Image Matcher Selection in the Wild

arXiv:2609.25688v1 Announce Type: new Abstract: Field robots operating across time of day and sensing modalities require accurate image correspondences within onboard time and resource budgets. Howeve

Passes Alone, Fails Together: Benchmarking Semantic Coordination in Parallel LLM-Agent Development

arXiv:2609.25396v1 Announce Type: new Abstract: Parallel coding agents can produce patches that work alone but fail when merged. This happens when one agent changes an interface or rule that another a

Peerify: Benchmarking Peer-Review Claim Verification

arXiv:2609.25046v1 Announce Type: new Abstract: Peer review plays a central role in scholarly publishing, yet verifying whether reviewer claims are supported by manuscript evidence remains a largely m

Trains but Doesn't Learn: A Post-Training Delivery Benchmark for LLM Agents as Forward-Deployed Engineers

arXiv:2609.25237v1 Announce Type: cross Abstract: Post-training is becoming a service (PTaaS): a customer hands an operator data and a goal, and a forward-deployed engineer (FDE) returns a fine-tuned,

What is the Difference Between Me and You? Benchmarking the Quality Gap Between Human-Written and AI-Generated Code

arXiv:2609.12708v2 Announce Type: replace-cross Abstract: AI coding assistants are becoming co-authors of production software, yet their evaluation centers on functional correctness, leaving open whet

WidgetVA: A Widget-Centric Framework and Benchmark for Agentic Visual Analytics

arXiv:2609.24094v1 Announce Type: cross Abstract: Visual analytics (VA) enables sensemaking through interactive visualization, but effective analysis often requires experts to translate high-level int

An Empirical Survey and Benchmark of Learned Distance Indexes for Road Networks

arXiv:2602.04068v2 Announce Type: replace Abstract: The calculation of shortest-path distances in road networks is a core operation in navigation systems, location-based services, and spatial analytic

22 Sept 2026

ARCGym: Benchmarking Deep Reinforcement Learning in Autonomous Robotic Colonoscopy

arXiv:2609.22803v1 Announce Type: new Abstract: Simulations for learning-based autonomous colonoscopic navigation focus mainly on fully actuated capsule robots, failing to capture the contact-rich nav