Search: “benchmark”

StarEmbed: Benchmarking Time Series Foundation Models on Astronomical Observations of Variable Stars

arXiv:2510.06200v4 Announce Type: replace-cross Abstract: Current time series foundation model (TSFM) training corpora largely omit data with certain complexities like irregular temporal sampling. Ast

14 Aug 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

StrAD: A Streaming Method and Benchmark for Audio Description Generation for Long-form Videos

DGX agent

arXiv:2608.12549v1 Announce Type: new Abstract: Visual content is the dominant medium of communication, yet without audio descriptions (ADs), it remains inaccessible to blind and low-vision people. AD

model-releasesarxiv-cs-cv

14 Aug 2026

CT-DeltaBench: A Benchmark for Longitudinal 3D Medical Imaging Difference Reporting with Vision-Language Models

arXiv:2608.11534v1 Announce Type: new Abstract: In medical imaging, the clinical value of Computed Tomography (CT) lies not only in depicting current disease status, but crucially in enabling longitud

Diagram-MMU: A Multi-Modal Benchmark for Scientific Diagrams

arXiv:2608.12262v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) have been growing the capability for scientific writing and collaboration. For example, OpenAI Prism is a fre

HandEdit: A Unified Benchmark for Egocentric Human-to-Robot Dexterous Hand Image Editing

arXiv:2608.12122v1 Announce Type: cross Abstract: Robotic manipulation with dexterous hands is a cornerstone of Embodied AI, yet its progress is stifled by the high cost of collecting embodiment-aware

JieZi: A Large-Scale Expert-Audited Dataset and Benchmark for Ancient Chinese Character Exegesis

arXiv:2608.11741v1 Announce Type: cross Abstract: The scholarly exegesis of ancient Chinese characters demands integrating visual observation, linguistic analysis, and historical context. However, exi

E^3mo-Bench: A Scalable Benchmark for Multimodal Evoked and Expressed Emotion Understanding via Bayesian Pairwise Alignment

arXiv:2608.10796v1 Announce Type: new Abstract: Understanding both expressed and evoked emotions is critical for multimodal large language models (MLLMs) to achieve comprehensive affect-aware interact

12 Aug 2026

Locally Deployable Small Language Models for Emergency Department Decision Support: A Systematic Benchmark of Fine-Tuning Strategies

arXiv:2608.10273v1 Announce Type: cross Abstract: Deploying large language models (LLMs) for decision support in emergency departments (EDs) faces two major challenges: privacy risks of transmitting p

12 Aug 2026

ActBench: Self-Evolving Benchmark of Behavioral Safety in Cowork Agents

arXiv:2608.09476v1 Announce Type: cross Abstract: Cowork agents may complete benign tasks while disclosing protected data, manipulating unauthorized state, invocate unauthorized API. We define behavio

Benchmarking In-context Experiential Learning Through Repeated Product Recommendations

model-releasesarxiv-cs-lg

arXiv:2511.22130v2 Announce Type: replace Abstract: To navigate ever-shifting real-world environments, agents must grapple with incomplete knowledge and adapt their strategies through experience. Howe

CAP: A Scalable Benchmark for Evaluating Cross-Site Browser Agents with Complex Actions and Perception

arXiv:2608.08392v1 Announce Type: new Abstract: Large language models are increasingly deployed as autonomous agents that interact with the web through browsers. While recent progress has been driven

Counterfactual Benchmarking and Training for Factuality Consistency and Order-Robust Grounded Reasoning in LLMs over Heterogeneous Knowledge

arXiv:2608.07838v1 Announce Type: new Abstract: Large language models (LLMs) have increasingly supported response generation grounded in user-provided knowledge spanning heterogeneous structures. Howe

ELBench: A Multi-Dimensional Benchmark for Education-Facing Large Language Models

arXiv:2608.09548v1 Announce Type: cross Abstract: Large language models are increasingly deployed in education as tutors, teaching assistants, and content generators. These roles place demands that or

EnergyBridge: Benchmarking Household Energy Management, User Participation, and Grid Flexibility

arXiv:2608.08691v1 Announce Type: new Abstract: Residential virtual power plants (VPPs) can provide grid flexibility by shifting household demand, but physical flexibility becomes dependable capacity

FitAQA: A Benchmark of Fitness Action Quality Assessment for Multimodal Large Language Models

arXiv:2608.08736v1 Announce Type: new Abstract: Fitness Action Quality Assessment (AQA) is important for intelligent sports training, yet the capabilities of Multimodal Large Language Models (MLLMs) i

From Diagnosis to Correction: Benchmarking and Improving Real-World Table Parsing

arXiv:2608.09842v1 Announce Type: new Abstract: Recent document parsers achieve table TEDS scores above 93 on OmniDocBench v1.6, yet community feedback and our audit reveal persistent failures on comp

How Much Does It Cost to Answer My Question? Benchmarking Cloud VLM-based VQA Systems

arXiv:2608.07861v1 Announce Type: new Abstract: Vision-language models (VLMs) are becoming a practical backend for mobile visual question answering (VQA) systems, enabling smartphones and smart glasse

JUMP-lite: Compact, reproducible benchmarking of cell representations

arXiv:2608.07632v1 Announce Type: cross Abstract: Image-based profiling captures rich phenotypic signatures for drug discovery and functional genomics. Large public datasets like JUMP Cell Painting no

LexKairos: Benchmarking Legal Temporal Capabilities in LLMs

arXiv:2608.09106v1 Announce Type: new Abstract: Large language models (LLMs) have demonstrated strong performance across a wide range of legal tasks. In legal practice, time is a critical concept that

LIBAD: A Multimodal Anomaly Detection Benchmark for Li-Ion Battery Electrode Manufacturing

arXiv:2608.07958v1 Announce Type: new Abstract: Multimodal industrial anomaly detection has largely focused on discrete products using strongly correlated RGB and 3D observations, leaving continuous p

MasDrift: Benchmarking Authorization Preservation Across Multi-Agent Architectures

arXiv:2608.07556v1 Announce Type: cross Abstract: Multi-agent systems (MAS) decompose long-horizon tasks across supervisors and subagents, but delegated goals do not necessarily carry their original a

MiraMind: Benchmarking Reliable Mental Health Reasoning beyond Answer Accuracy

arXiv:2512.09636v3 Announce Type: replace Abstract: Mental-health reasoning with large language models (LLMs) is an evidence-constrained judgment problem: models must transform limited, subjective, an

MonitorBench: A Comprehensive Benchmark for Chain-of-Thought Monitorability in Large Language Models

arXiv:2603.28590v3 Announce Type: replace Abstract: Large language models (LLMs) can generate chains of thought (CoTs) that are not always causally responsible for their final outputs. When such a mis

NBA_Streaming: A Large-Scale Benchmark for Fine-Grained Basketball Commentary Generation in Continuous Streams

arXiv:2608.09200v1 Announce Type: new Abstract: Live basketball commentary generation requires determining when an event is sufficiently observable and describing it before subsequent events unfold. H

NL2SHACL-Bench: A Benchmark Suite for Natural Language to SHACL Translation

arXiv:2608.07530v1 Announce Type: new Abstract: SHACL is a core technology for validating the conformance of RDF knowledge graphs (KGs). Yet, authoring SHACL shapes requires technical expertise that m

QuArch: A Benchmark for Evaluating LLM Reasoning in Computer Architecture

arXiv:2510.22087v3 Announce Type: replace-cross Abstract: The field of computer architecture, which bridges high-level software abstractions and low-level hardware implementations, remains absent from

SurakshaEval: An Indic Safety Benchmark for Multilingual LLMs

arXiv:2608.07862v1 Announce Type: new Abstract: Existing safety evaluation datasets for large language models (LLMs) predominantly focus on English and Western contexts, often overlooking the linguist

SurgWMBench: A Vision-Based Benchmark for World-Modeling Surgical Instrument Motion Planning

model-releasesarxiv-cs-ro

arXiv:2608.08070v1 Announce Type: new Abstract: Reliable surgical planning requires models that move beyond recognizing the current surgical step or imitating expert demonstrations, and instead antici

SurveyReview: A Reviewer-Aligned Benchmark for Survey Evaluators

arXiv:2608.07641v1 Announce Type: new Abstract: The rapid advancement of large language models has transformed survey writing from a months-long manual effort into an automated process. As generation

TeXFix-Bench: An Empirically Grounded Multi-Format Benchmark for LLM-Based Document Source Repair

arXiv:2608.07617v1 Announce Type: new Abstract: Scientific and technical writing depends on markup sources that must compile: LaTeX, Typst, and Markdown pipelines fail on missing delimiters, mismatche

WuYuEval: A Multi-Level Benchmark for Large Language Models in Solid Waste Management

arXiv:2608.07529v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as technical assistants, but their competence in solid waste management (SWM) remains difficult to

Explanation Stability of Test-Time Adaptation in Computational Pathology: A Large-Scale Benchmark

arXiv:2608.07062v1 Announce Type: new Abstract: Test-time adaptation (TTA) has become a practical way to adapt deployed models to unlabeled target data, a setting that is especially relevant in comput

10 Aug 2026

FinRank: An Evidence-Grounded Benchmark for Financial Question Answering and Retrieval over SEC Filings

arXiv:2608.07400v1 Announce Type: new Abstract: Financial question answering is typically evaluated by answer correctness, yet in SEC filings a plausible and even numerically correct answer can be gro

10 Aug 2026

UAV3DCrop: Benchmarking 3D Reconstruction in Repeated Multi-Angle UAV Crop Surveys

arXiv:2608.06404v1 Announce Type: new Abstract: Accurate 3D crop monitoring underpins data-driven precision agriculture by enabling field-scale analysis of plant structure, growth dynamics, and manage

10 Aug 2026

An Axiomatic Benchmark for Evaluation of Scientific Novelty Metrics

arXiv:2604.15145v2 Announce Type: replace Abstract: The rigorous evaluation of the novelty of a scientific paper is, even for human scientists, a challenging task. With the increasing interest in AI s

Benchmarking and Enhancing LLMs for Rule-Intensive Review of National Standard Documents

arXiv:2608.06312v1 Announce Type: new Abstract: Large language models (LLMs) increasingly support complex professional tasks, yet their capabilities in rule-intensive document review remain insufficie

GAUGE: A Measurement-Grounded Benchmark for Physical Fidelity in Simulation Engines and Video World Models

arXiv:2608.05948v1 Announce Type: new Abstract: Physics engines facilitate large-scale training and evaluation for embodied intelligence, while generative video world models are emerging as implicit s

IPV-Bench: Benchmarking Image Protection Methods under Diverse Image-to-Video Generation Scenarios

arXiv:2603.26154v2 Announce Type: replace Abstract: Image-to-video (I2V) generation models can be misused to animate a single image into a convincing fake video, motivating perturbation-based image pr

M^3R-Bench: A Unified Benchmark for Evidence-Grounded Multimodal Metaphor Understanding

arXiv:2608.05817v1 Announce Type: new Abstract: Metaphor enables the understanding of abstract concepts through cross-domain mappings while conveying affective attitudes. In multimodal scenarios, visu

Matching Matters: A Fair Quality-Efficiency Benchmark for Command-Line Agents

arXiv:2606.21140v2 Announce Type: replace-cross Abstract: Rapid advances in large language models have improved the task-solving capabilities of command-line-interface (CLI)-based agents, whose CLIs d

NavTrust: Benchmarking Trustworthiness for Embodied Navigation

arXiv:2603.19229v2 Announce Type: replace-cross Abstract: There are two major categories of embodied navigation: Vision-Language Navigation (VLN), where agents navigate by following natural language i

SkillTV-Bench: Benchmarking How Well Judges Perform on Skill-Augmented Agentic Execution

arXiv:2608.05573v1 Announce Type: new Abstract: LLM agents increasingly execute long-horizon tasks through tool use and environment interaction, shifting evaluation from final-response scoring to veri

Active-SWE: Benchmarking Coding Agents for Proactive Bug Fixing without Issue Reports

arXiv:2608.04682v1 Announce Type: cross Abstract: Coding agents powered by large language models (LLMs) are increasingly adopted in software engineering (SWE) scenarios, capable of fixing a specific b

RepoProbe: Benchmarking Architecture-Aware Repository Comprehension with Checklists

arXiv:2608.04783v1 Announce Type: cross Abstract: The integration of Large Language Models (LLMs) into software engineering has shifted the focus from function-level generation to repository-scale ass

RESPClinBench: Benchmarking Multimodal Clinical Decision-Making and Longitudinal Disease Management in Respiratory Specialty Care

arXiv:2608.04514v1 Announce Type: new Abstract: Background: Respiratory specialty care requires multimodal interpretation, longitudinal risk assessment, guideline-concordant intervention, and whole-co

SciCode-Verified: How Benchmark Defects Underestimated the Scientific-Coding Ability of Language Models

arXiv:2608.04975v1 Announce Type: cross Abstract: SciCode is the standard measure of the scientific-coding ability of language models: research-level problems that demand both frontier scientific theo

Text2GraphQuery-Bench: A Text to Graph Query Benchmark

arXiv:2602.11745v2 Announce Type: replace Abstract: Graph models are fundamental to data analysis in domains rich with complex relationships. Unlike SQL, which benefits from a rel- atively unified sta

TIDE: A Physically Diverse 3D Turbulence Benchmark Dataset for Advancing Scientific Machine Learning

model-releasesarxiv-cs-lg

arXiv:2608.04222v1 Announce Type: cross Abstract: Turbulence is a central testbed for machine learning on physical dynamics because its governing laws are known exactly. However, most existing studies