Search: “benchmark”

RMBench: Memory-Dependent Robotic Manipulation Benchmark with Insights into Policy Design

model-releasesarxiv-cs-ro

arXiv:2603.01229v3 Announce Type: replace Abstract: Robotic manipulation policies have made rapid progress in recent years, yet most existing approaches give limited consideration to memory capabiliti

31 Jul 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

Benchmarking ConvLSTM for One-Day-Ahead IMDAA Rainfall-Field Prediction across Four Indian Cities

DGX agent

arXiv:2607.26581v1 Announce Type: new Abstract: Convolutional long short-term memory networks (ConvLSTMs) are widely used for precipitation forecasting, but most evidence for their performance comes f

researcharxiv-cs-lg

30 Jul 2026

Cost-Sensitive Conformal Prediction and Human-in-the-Loop Abstention for Imbalanced High-Stakes Decision Support: A Multi-Domain Benchmark

arXiv:2607.27143v1 Announce Type: new Abstract: High-stakes decision systems in credit scoring, fraud detection, healthcare, and industrial safety require reliable uncertainty quantification under sev

30 Jul 2026

Foundation Models for Face Presentation Attack Detection: A Unified Linear-Probing Benchmark

arXiv:2607.26993v1 Announce Type: new Abstract: Face presentation attack detection (PAD) remains challenging under cross-dataset evaluation, where domain shift degrades models trained on a single data

30 Jul 2026

VideoNorms: Benchmarking Cultural Awareness of Video Language Models

arXiv:2510.08543v2 Announce Type: replace-cross Abstract: As Video Large Language Models (VideoLLMs) are deployed globally, it is important to assess their ability to reason across cultural contexts.

researcharxiv-cs-cl

30 Jul 2026

AI Security Leaderboard: benchmarking model robustness [P]

model-releasesr-machinelearning

We developed a leaderboard ranking frontier model security. There's no shortage of model capability rankings, but we didn't find anything comparable for model security. Yet security is becoming increa

Benchmarking Deep Learning Models for Raman Spectroscopy Across Open-Source Datasets

arXiv:2601.16107v2 Announce Type: replace Abstract: Deep learning classifiers for Raman spectroscopy are increasingly reported to outperform classical chemometric approaches. However, their evaluation

researcharxiv-cs-lg

CD-RMOT-Bench: Benchmarking the Cross-Domain Referring Multi-Object Tracking

arXiv:2607.25239v1 Announce Type: new Abstract: Referring multi-object tracking (RMOT) extends tracking from category-driven perception to language-guided understanding by grounding object trajectorie

Data Quality Profiling at Scale with Progressive Sampling: A Benchmark for Data-Centric AI Pipelines

arXiv:2607.25356v1 Announce Type: cross Abstract: Data quality profiling -- computing missing-value rates, duplicate fractions, outlier densities, and functional-dependency violations -- is foundation

ObliCity: A Benchmark and Baseline for Roof-to-Ground Projection Displacement Correction

arXiv:2607.25210v1 Announce Type: new Abstract: Oblique-view urban remote sensing imagery inevitably exhibits geometric projection displacements between building roofs and footprints, leading to signi

When Does Deep Representation Learning Help Single-Cell Clustering? A Sensitivity-Aware Diagnostic Benchmark for Biomedical AI Pipelines

arXiv:2607.25288v1 Announce Type: new Abstract: Single-cell ribonucleic acid sequencing (scRNA-seq) is a foundational technology for precision-medicine workflows that contribute to United Nations Sust

BioProBench: A Corpus and Benchmark for Biological Protocol Reasoning in Autonomous Science

arXiv:2505.07889v4 Announce Type: replace Abstract: The realization of autonomous scientific experimentation is currently limited by LLMs' struggle to grasp the strict procedural logic and accuracy re

LIBMoE: A Library for comprehensive benchmarking Mixture of Experts in Large Language Models

arXiv:2411.00918v5 Announce Type: replace-cross Abstract: Mixture of experts (MoE) architectures have become a cornerstone for scaling up and are a key component in most large language models such as

LU-500: A Logo Benchmark for Concept Unlearning

arXiv:2607.24101v1 Announce Type: cross Abstract: Concept unlearning is increasingly used to limit the reproduction of protected or unsafe visual concepts in text-to-image models. Existing evaluations

Variational Quantum Conditional Boltzmann Machines for Time-Series Forecasting: Architectures, Symmetric Hyperparameter Evaluation, and a Nonlinear Benchmark

arXiv:2607.24065v1 Announce Type: cross Abstract: In this study, we developed and evaluated four conditional energy-based forecasting architectures: a classical Gaussian-Bernoulli CRBM, a hybrid quant

A Comparative Benchmark of Federated Learning Strategies for Mortality Prediction on Heterogeneous and Imbalanced Clinical Data

arXiv:2509.10517v3 Announce Type: replace Abstract: Machine learning can predict in-hospital mortality, but data privacy and the statistical heterogeneity of clinical data hamper its use. Federated Le

27 Jul 2026

Local Ai

From Seasonality to Semantics: Benchmarking a Hybrid Probabilistic Forecasting System for Roadblocks in Bolivia

arXiv:2607.21785v1 Announce Type: cross Abstract: Roadblocks in Bolivia are a social conflict phenomenon with devastating economic impacts, estimated at losses equivalent to 4% of the national Gross D

local-aiarxiv-cs-cl

27 Jul 2026

Operator-Informed Gaussian Processes for Complex Helmholtz Wavefields: From Synthetic Benchmarks to In Vivo Brain Elastography

arXiv:2607.14193v2 Announce Type: replace-cross Abstract: The Helmholtz equation governs time-harmonic wave propagation, and in dissipative media a complex modulus renders its squared wavenumber kappa

27 Jul 2026

Ollama Cloud Quota Benchmark

Recently I bought an Ollama Cloud sub and accidently spent my whole 5h quota upon using DeepSeek V4 Pro... but why? isnt it supposed to be a cheap model? Youd think there would be a correlation betwee

model-releasesr-ollama

25 Jul 2026

Benchmarking the Personalization Capabilities of Large Language Models

arXiv:2607.20471v1 Announce Type: new Abstract: Personalization, the act of varying a message to induce action from a specific receiver while keeping sender, channel, and time fixed, has a long tradit

GlucoTune: A Unified Framework for Blood Glucose Preprocessing, Forecasting, and Benchmarking in Diabetes

arXiv:2607.21117v1 Announce Type: cross Abstract: Preprocessing blood glucose time-series data is a critical yet often overlooked step in developing data-driven methods for diabetes management, partic

researcharxiv-cs-ai

IssueTrojanBench: Benchmarking AI Coding Agents Against Malicious Issue Requests

arXiv:2607.20759v1 Announce Type: cross Abstract: AI coding agents powered by LLMs are increasingly integrated into real-world software development, where they generate, edit, and execute code with au

LegalCiteTrust: Benchmarking Citation Trustworthiness in Chinese Long-Form Legal Research Reports

arXiv:2607.20872v1 Announce Type: new Abstract: Long-form legal research reports increasingly rely on LLMs and agentic research systems, but their reliability depends not only on answering the task, b

One More Turn, Less Regret: A Regret-Based Multi-Turn Benchmark for LLMs' Clarification Policies

arXiv:2607.21143v1 Announce Type: cross Abstract: Ambiguous user requests make clarification a sequential decision problem for conversational LLM assistants: they must decide whether to ask, what to a

People are using Minecraft farms as AI agent benchmarks

Someone modelled sugarcane farming as an integer program. See, sugarcane only grows next to water. Water costs one tile and can feed at most four cane tiles. The layout therefore becomes a coverage pr

model-releasesr-chatgpt

Local Ai

Spent two weeks on a kernel that benchmarked 29x faster. End to end it's maybe 6-10%, and it's not even wired in yet.

I've been building a C99 inference engine from scratch (no Python, no BLAS, just gcc and make) that runs BitNet's ternary models on CPU. A few weeks ago I got obsessed with the matmul kernel - wrote a

local-air-localllama

Tencent WorkBuddy Bench: A Multi-Domain Coding-Agent Benchmark with Contamination-Resistant Task Construction

arXiv:2607.20911v1 Announce Type: new Abstract: We introduce Tencent WorkBuddy Bench, a multi-domain evaluation suite for coding agents; this report documents its construction methodology, scoring pro

TOUR: A Trajectory-Level Unlearning Benchmark for Offline Reinforcement Learning

arXiv:2607.21111v1 Announce Type: cross Abstract: Offline Reinforcement Learning (RL) agents are trained on fixed behavioral trajectories, which makes trajectory-level deletion important when selected

Unlearning Under Imbalance: Benchmarking Fairness in Multimodal LLM Unlearning

arXiv:2607.21300v1 Announce Type: cross Abstract: Machine unlearning has emerged as a tool for removing personal data from trained models to comply with recent AI regulations. To evaluate unlearning e

Alipay-PIBench: A Realistic Payment Integration Benchmark for Coding Agents

arXiv:2607.14573v3 Announce Type: replace Abstract: Payment integration is a demanding repository-level software task: agents must select a suitable product, implement coordinated client-server flows,

Benchmarking Confidential GPU Inference on NVIDIA H100 under Intel TDX

arXiv:2607.19353v1 Announce Type: new Abstract: Confidential computing is becoming a practical deployment requirement for AI inference workloads that process sensitive inputs or protect proprietary mo

Benchmarking Deep Learning Approaches for AEC Engineering Drawing Layout Detection and Information Extraction

arXiv:2607.18997v1 Announce Type: new Abstract: Information Extraction (IE) from Architecture, Engineering, and Construction (AEC) drawings remains hindered by manual inefficiency, while Layout Detect

researcharxiv-cs-cv

Cross-Modal UAV Object Tracking: State-Aware Representation Learning and A Unified Benchmark

arXiv:2607.18768v1 Announce Type: new Abstract: Unmanned Aerial Vehicle (UAV) object tracking has emerged as a popular research field with broad practical applications. Modern UAVs are increasingly eq

KineBench: Benchmarking Embodied World Models via IDM-Free Kinematic Grounding

model-releasesarxiv-cs-ro

arXiv:2607.19876v1 Announce Type: new Abstract: Evaluating the physical consistency of embodied world models(EWMs) is a critical open challenge. While closed-loop evaluation via simulator rollouts off

Read or Ignore? A Unified Benchmark for Typographic-Attack Robustness and Text Recognition in Vision-Language Models

arXiv:2512.11899v2 Announce Type: replace Abstract: Large vision-language models (LVLMs) are vulnerable to typographic attacks, where misleading text inserted into an image can override visual underst

Trend strength predicts when generative foundation models win: a power-controlled benchmark, a mechanism, and an actionable selection rule

arXiv:2607.19383v1 Announce Type: cross Abstract: Pretrained generative foundation models cast forecasting as conditional generation from a learned predictive distribution and forecast unseen series z

TerraLogic: A Benchmark for Hierarchical Geospatial Reasoning in Earth Observation

arXiv:2607.12497v1 Announce Type: new Abstract: Beyond perception, reasoning is essential in remote sensing for advanced interpretation, inference, and decision-making. Recent advances in large langua

15 Jul 2026

Classical versus Deep Mirror-Symmetry Scoring: A Benchmark of Thirteen Methods

arXiv:2607.08379v1 Announce Type: new Abstract: Quantifying how mirror-symmetric an image is about a given axis (symmetry scoring) underpins applications from visual aesthetics to medical imaging, yet

Continual Test-Time Adaptation in Computer Vision: Methods, Benchmarks, and Future Directions

arXiv:2607.08164v1 Announce Type: new Abstract: Deep neural nets achieve remarkable performance when training and test data share the same distribution, but this assumption frequently breaks in real-w

Do You Need a Frontier Model as a Citation Verifier? Benchmarking Rubric LLMs for Deep-Research Source Attribution

arXiv:2607.08700v1 Announce Type: new Abstract: Reinforcement learning increasingly relies on an LLM judge to score each rubric criterion, and that judge acts as the reward model during training. Befo

Dual-Correlation Hypergraph Network for Unaligned RGBT Video Object Detection and A Large-scale Benchmark

arXiv:2607.08191v1 Announce Type: new Abstract: RGB-Thermal (RGBT) Video Object Detection (VOD) has gained significant traction due to its ability to overcome the limitations of conventional RGB-based

Mixture of Enhanced-View Experts for Multi-Query Vehicle ReID and A Large-Scale Benchmark

arXiv:2607.08085v1 Announce Type: new Abstract: Multi-query vehicle ReID aims to leverage complementary information from diverse views for robust feature learning. However, current methods suffer from

SolarChain-Eval: A Physics-Constrained Benchmark for Trustworthy Economic Agents in Decentralized Energy Markets

arXiv:2607.08681v1 Announce Type: new Abstract: As agentic AI systems are increasingly applied to cyber-physical environments, their evaluation requires assessment of both task performance and trustwo

Where to Intervene? Benchmarking Fairness-Aware Learning on Differentially Private Synthetic Tabular Data

arXiv:2607.07471v1 Announce Type: cross Abstract: Machine learning models are increasingly deployed in high-stakes domains, raising concerns about both privacy and fairness. Differential Privacy (DP)

9 Jul 2026

BaFCo: A Document Understanding Benchmark for Complex Bangla Form Comprehension

arXiv:2607.05614v1 Announce Type: cross Abstract: Document comprehension is a challenging yet impactful task for Multimodal Large Language Models, especially as these systems see growing adoption in r

Industry

Benchmarking Coding Agents on Databricks’ Multi-Million Line Codebase

This Databricks blog post evaluates the performance and capabilities of coding agents when applied to real-world scenarios involving their own multi-million line codebase, likely assessing metrics suc

industrydatabricks

Benchmarking the Robustness of Autonomous Driving to Environmental Illusions: A Lane Perception Perspective

arXiv:2607.05783v1 Announce Type: new Abstract: Environmental illusions (eg., shadows, reflections, and tire marks) are naturally existing yet overlooked phenomena in real-world driving environments.

FootsiesGym: A Fighting Game Benchmark for Two-Player Zero-Sum Imperfect-Information Games

arXiv:2607.06514v1 Announce Type: new Abstract: We present FootsiesGym, an open-source environment for learning in a non-trivial two-player, zero-sum, imperfect-information game. Built on HiFight's mi