Search: “benchmark”

VidOmni-Bench: A Benchmark for Fine-Grained Video Understanding via Spatio-Temporal Event Verification across Complexity and Duration

arXiv:2609.21521v1 Announce Type: cross Abstract: While Video Large Language Models (Video-LLMs) have recently demonstrated strong performance, reliably evaluating their fine-grained video understandi

21 Sept 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

BioPhys-Bridge: A Benchmark for Interdisciplinary Scientific Reasoning in Physics-Grounded Biological Research

DGX agent

arXiv:2609.19180v1 Announce Type: new Abstract: Language models face unique challenges in analyzing interdisciplinary scientific research literature. In biophysics research, faithful answers require g

model-releasesarxiv-cs-ai

18 Sept 2026

BuildBench: Benchmarking LLM Agents on Compiling Real-World Open-Source Software

arXiv:2509.25248v2 Announce Type: replace-cross Abstract: Automatically compiling open-source software (OSS) projects is a vital, labor-intensive, and complex task, which makes it a good challenge for

18 Sept 2026

Benchmarking Visual-Inertial Odometry in Subterranean Environments Under Sensor Degradation, Miscalibration, and Dynamic Occlusion

arXiv:2609.18628v1 Announce Type: new Abstract: Visual-inertial odometry (VIO) is a core capability for autonomous operation in GPS-denied subterranean environments, yet its reliability can degrade sh

17 Sept 2026

iMINDBench: iEEG Multi-Institution Neural Decoding Benchmark

arXiv:2609.18104v1 Announce Type: new Abstract: Intracranial electroencephalography (iEEG) is widely used to record electrical activity directly from electrodes inside the human brain, making it an at

17 Sept 2026

BRAVE-6D: Benchmark for Robotic Active Vision in 6DOF Pose Estimation

arXiv:2609.17106v1 Announce Type: new Abstract: Detecting and grasping small objects remains a significant challenge in robotics. Active vision, where the robot moves closer to the object, is an intui

16 Sept 2026

What Breaks Local Watermarks? A Robustness Benchmark for Local Invisible Image Watermarking

arXiv:2609.16832v1 Announce Type: cross Abstract: Local image watermarking embeds an invisible signal into selected image regions rather than spreading it across the entire image, enabling payload rec

16 Sept 2026

Bangla Sentence Function Classification: Corpus Development, Model Benchmarking, and Interpretability

arXiv:2609.13869v1 Announce Type: cross Abstract: Automatic sentence function identification is important for many downstream natural language processing (NLP) applications such as dialogue systems, t

Benchmarking Intra-Patient 3D Deformable Multimodal Image Registration

arXiv:2609.15669v1 Announce Type: cross Abstract: Multimodal image registration is a key component of many clinical workflows, yet it remains challenging because corresponding anatomical structures of

Benchmarking Optimizers to Solve Inverse Problems with Differentiable Physics Simulators

arXiv:2609.13819v1 Announce Type: new Abstract: Solving inverse problems with differentiable physics simulators holds the potential to revolutionize scientific discovery and engineering design, as it

EMCStereo: Attention-Enhanced Stereo Matching for Thin-Structure Depth Estimation with a Synthetic Tree-Branch Benchmark

arXiv:2609.13233v1 Announce Type: new Abstract: Thin structures such as tree branches are among the hardest cases for stereo matching: a branch is only a few pixels wide, the background is cluttered,

Research

GP-VMimesSMA: Benchmarking General-Purpose Vision Models and Specialized Architectures for 2D Medical Image Segmentation

arXiv:2603.13044v2 Announce Type: replace-cross Abstract: Medical image segmentation (MIS) is a fundamental component of computer-assisted diagnosis and clinical decision support. Over the past decade

researcharxiv-cs-ai

K-Bench: a clinically calibrated benchmark for evaluating large language models in high-risk mental health conversations

arXiv:2609.15855v1 Announce Type: cross Abstract: % !TEX root = ../main.tex People increasingly use large language models (LLMs) for mental health support, yet their safety in evolving, high-risk conv

SomBench: Benchmark Dataset for Advancing Machine Learning in Lunar Science

arXiv:2609.13277v1 Announce Type: new Abstract: Lunar orbital missions, such as Lunar Reconnaissance Orbiter, Kaguya/SELENE, Gravity Recovery and Interior Laboratory, and Lunar Prospector, among other

The MODA General Attribute Suite: A Four-Track Evaluation Benchmark for Fashion Attribute Extraction

arXiv:2609.13279v1 Announce Type: new Abstract: Fashion attribute extraction is evaluated inconsistently: results are reported as single aggregate numbers across image types that pose different proble

Toward Secure Code Generation: Bridging Correctness and Security via Task-Adaptive Vulnerability Modeling and Execution-Based Benchmarking

arXiv:2407.02395v3 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used for program synthesis, yet they often generate code that is functionally plausible but inse

Statistical Uncertainty Quantification for Aggregate Performance Metrics in Machine Learning Benchmarks

arXiv:2501.04234v2 Announce Type: replace-cross Abstract: Modern artificial intelligence is supported by machine learning models (e.g., foundation models) that are pretrained on a massive data corpus

14 Sept 2026

Agents

UrduFactCheck: An Agentic Fact-Checking Framework for Urdu with Evidence Boosting and Benchmarking

arXiv:2505.15063v3 Announce Type: replace Abstract: The rapid adoption of Large Language Models (LLMs) has raised important concerns about the factual reliability of their outputs, particularly in low

agentsarxiv-cs-cl

14 Sept 2026

Applications

Beyond Benchmarks: Using VLMs to Reveal Systematic Classification Failures Under Real World Conditions

arXiv:2609.11126v1 Announce Type: new Abstract: Verification and validation (V&V) of classification models is crucial to enable a wide range of sensor processing applications. Currently, the V&V proce

applicationsarxiv-cs-cv

11 Sept 2026

CMNIE: An Information Extraction Benchmark for Chinese Military News

arXiv:2609.10722v1 Announce Type: new Abstract: Structured extraction from Chinese military news supports intelligence analysis, decision-making, and knowledge base construction. However, existing res

11 Sept 2026

ReactHuman: A Physics-Grounded Benchmark for Human-Like Reactive Decision-Making in Embodied Multimodal LLMs

arXiv:2609.10895v1 Announce Type: new Abstract: Reacting to sudden physical hazards (catching a slipping plate, dodging a falling knife) is both a meaningful test of embodied intelligence and a hard r

11 Sept 2026

Beyond Cross-Lingual Transfer: Benchmarking Propagation Boundaries in Multilingual LLM Unlearning

arXiv:2609.05976v1 Announce Type: cross Abstract: Large Language Model (LLM) unlearning aims to suppress target knowledge while preserving general capabilities. In multilingual settings, unlearning mu

10 Sept 2026

LogiScope-VQA: Benchmarking Vision-Language Models for Logistics Hazard Identification in Industrial Scenarios

arXiv:2609.09790v1 Announce Type: cross Abstract: Large Multimodal Models (LMMs) large-scale deployment in industrial warehouse settings specifically necessitates that models exhibit human-expert-leve

10 Sept 2026

YallaMorph: A Benchmark for Evaluating Arabic Morphological Generation in Large Language Models

arXiv:2609.10153v1 Announce Type: new Abstract: Arabic morphology remains challenging for large language models, since fluent generation does not guarantee accurate morphosyntactic control. Existing A

10 Sept 2026

AgentDrift: A Step-Labeled Benchmark of Injection-Hijacked LLM Agent Trajectories

arXiv:2609.06972v1 Announce Type: cross Abstract: LLM agents complete tasks by issuing sequences of tool calls, and every observation they read is a channel through which an indirect prompt injection

EvoNav-Bench: Benchmarking Lifelong Navigation in Evolving Environments

arXiv:2609.08292v1 Announce Type: new Abstract: Lifelong navigation (LN) requires an embodied agent to solve a sequence of navigation subtasks in the same environment. Since solving each subtask from

HealthLoopQA: A Context-Aware Question Answering Benchmark for Interpreting Wearable Monitoring Data in Diabetes Care

arXiv:2609.06976v1 Announce Type: new Abstract: As medical wearables become integrated into daily chronic disease care, effectively interpreting longitudinal monitoring data is essential for patients

LatentMD: Benchmarking Markdown Boundary Failures in LLM-Generated Text

arXiv:2609.06993v1 Announce Type: cross Abstract: Large language models (LLMs) increasingly generate Markdown that is consumed by renderers, agents, code extractors, and structured downstream pipeline

MEMOBench: A Process Level Memory Benchmark for Robotic Manipulation

arXiv:2609.07047v1 Announce Type: new Abstract: Robotic manipulation often requires acting on information that is no longer visible, yet Vision-Language-Action policies are usually evaluated when the

TabBench-Bio: A Living Benchmark for Machine Learning on High-Dimensional Biomedical Tables

arXiv:2609.07441v1 Announce Type: new Abstract: Biomedical tables often combine thousands of measured variables with only tens or hundreds of labelled samples, a regime that is poorly represented in g

AxQM: A Textbook-Scale Benchmark for Formal Proof Synthesis in a Library of Finite-Dimensional Quantum Mechanics

arXiv:2609.05157v1 Announce Type: cross Abstract: Formalizing mathematics in a proof assistant, where a machine checks every definition, statement and proof, has set a new standard of rigor. Large lan

Corporate-Family Resolution Is Not a String-Matching Problem: A Public Benchmark Stratified by Name Visibility

arXiv:2609.04269v1 Announce Type: cross Abstract: Deciding whether two supplier records belong to the same corporate family is a prerequisite for spend consolidation, credit exposure aggregation and s

MMTClinic: Multimodal, Multilingual Time Series Question Answering and Reasoning Benchmark for Clinical Domain

arXiv:2609.04842v1 Announce Type: cross Abstract: Time-series data in clinical settings is crucial for capturing dynamic changes in a patient's health over time, enabling timely diagnosis, personalize

MultihopSpatial: Multi-hop Compositional Spatial Reasoning Benchmark for Vision-Language Model

arXiv:2603.18892v2 Announce Type: replace-cross Abstract: Spatial reasoning is foundational for Vision-Language Models (VLMs), particularly when deployed as Vision-Language-Action (VLA) agents in phys

NavArena: Automated Construction of Goal-Oriented Navigation Benchmarks from 3D Gaussian Splatting Reconstructions

arXiv:2609.04602v1 Announce Type: new Abstract: Fixed 3D Gaussian Splatting (3DGS) reconstructions provide realistic novel views but lack the traversability constraints, valid goals, and closed-loop p

SciDocBench: A Workflow-Centered Benchmark and Data Pipeline for Scientific Document Understanding

arXiv:2609.05141v1 Announce Type: new Abstract: Scientific papers require models to reason jointly over text, equations, figures, tables, code, and datasets while preserving the provenance of supporti

TruthInsightBench: An Evidence-Grounded Benchmark for Automated Evaluation of Open-Ended Scientific Discovery Agents

arXiv:2609.05079v1 Announce Type: new Abstract: Autonomous coding agents are increasingly proposed as AI-scientist systems that conduct analyses and write research reports, but executing a prescribed

FrameBench:A Language Understanding Benchmark Based on Frame Semantics

arXiv:2609.03370v1 Announce Type: new Abstract: In frame semantics, sentence comprehension is assumed to proceed by relating lexical meaning to background knowledge called semantic frames, thereby ena

4 Sept 2026

ThousandWorlds: A benchmark for climate emulation of potentially habitable exoplanets

arXiv:2606.18338v2 Announce Type: replace Abstract: The search for life beyond Earth will depend on detecting faint signatures in the atmospheres of potentially habitable exoplanets. Interpreting thos

4 Sept 2026

Updated my benchmark with a new vLLM based recipe for Qwen 3.8 Flash Next : now up to 98/100 (instead of 91 previously)

model-releasesr-localllama

I was using: weights https://huggingface.co/RadixArk/Qwen3.8-Flash-Next-NVFP4 with the optimized SGLANG (patched) from https://old.reddit.com/r/BlackwellPerformance/comments/1w04xb7/qwen38_flashnext_o

4 Sept 2026

A Physics-Consistent Benchmark for Contact-Rich Human-Robot Interaction in Assistive Care

arXiv:2609.02402v1 Announce Type: new Abstract: Conventional task-level evaluation asks whether a robot policy completes a specified action, but can miss failures that emerge only during physical huma

GeoStore: Finding Small Storefronts in Large Scenes -- A Fine-Grained POI Localization Benchmark with Global-to-Local Asymmetric Matching

arXiv:2609.02012v1 Announce Type: new Abstract: Point-of-interest (POI) localization -- matching a user's close-up storefront photograph against large-scale geo-tagged street-view imagery -- underpins

Sim2Signal: Sim-to-Real Benchmarks for Traffic Signal Control

arXiv:2609.01676v1 Announce Type: new Abstract: Reinforcement learning achieves strong traffic signal control performance in simulation, yet policies trained in simulators often fail once deployed in

T2LSC-Bench: Benchmarking Localized Semantic Control in Text-to-Image Generation

arXiv:2609.02255v1 Announce Type: new Abstract: Recent text-to-image models have become increasingly capable of rendering explicit text, but reliable localized text control requires more than generati

My Minimax H3 Workflow Benchmark Data -

model-releasesr-stablediffusion

Alright, I posted that I had my agent test a bunch of different workflows for over 12 hours and got the 'Bro just wasted 12 hours of credits'. It was obvious the proof should come from the visual data

2 Sept 2026

A Degradation-Tolerance Benchmark for Camera-Only End-to-End Driving

arXiv:2608.29005v1 Announce Type: new Abstract: Camera-only end-to-end (E2E) driving models are nearing deployment, where the camera stream is degraded by blur, noise, low light, weather, frame loss,

1 Sept 2026

CORAL: A Benchmark for Structure-aware and Brain-wide Neuron Reconstruction in Light Microscopy

arXiv:2608.30768v1 Announce Type: new Abstract: Automatic neuron reconstruction from light microscopy images is a central problem in computational neuroanatomy. While recent methods have achieved enco

1 Sept 2026

LIBERO-Para: A Diagnostic Benchmark and Metrics for Paraphrase Robustness in VLA Models

arXiv:2603.28301v2 Announce Type: replace Abstract: Vision-Language-Action (VLA) models achieve strong performance in robotic manipulation by leveraging pre-trained vision-language backbones. However,