Search: “benchmark”

Ace! Motion Planning of Professional-Level Table Tennis Serves with a Robot Arm

model-releasesarxiv-cs-ro

arXiv:2607.06989v1 Announce Type: new Abstract: Table tennis, a dynamic, compact, and popular sport, has received significant attention as a robotics benchmark over the last decades. Most of the resea

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

AI Chatbot Suicide Risk Detection and Response: Human Validation Study of the Open-Source VERA-MH Safety Evaluation

DGX agent

arXiv:2602.05088v4 Announce Type: replace Abstract: Millions of people now use generative AI chatbots for psychological support. Despite their promise, the most pressing question in AI for mental heal

model-releasesarxiv-cs-ai

9 Jul 2026

From Text to Parameters: Predicting Item Parameters from Embedding Regularization with Reliability and Design Ceilings

arXiv:2607.07141v1 Announce Type: new Abstract: Newly developed items must ordinarily be field tested before their psychometric properties are known, creating a cold start problem for item calibration

Agents

Measuring Intelligence Beyond Human Scale

arXiv:2607.07040v1 Announce Type: new Abstract: How can we measure intelligence beyond human capability? Human-authored benchmarks saturate, and above human capability, examiners may not know which ta

agentsarxiv-cs-ai

Optimization Geometrodynamics: A Framework for Dynamic Geometric Optimization

arXiv:2607.06723v1 Announce Type: cross Abstract: Most gradient-based optimization methods move parameters through a fixed background geometry, even when their internal states implicitly define changi

Applications

SoccerNet 2026 Challenges Results

arXiv:2607.07320v1 Announce Type: new Abstract: The SoccerNet 2026 Challenges constitute the sixth annual edition of the SoccerNet open benchmarking effort, dedicated to advancing computer vision rese

applicationsarxiv-cs-cv

Beyond the Leaderboard: A Synthesis of Tool-Use, Planning, and Reasoning Failures in Large Language Model Agents

arXiv:2607.05775v1 Announce Type: new Abstract: Large language model (LLM) agents are increasingly evaluated on their ability to use tools, plan multi-step tasks, coordinate with other agents, and ope

Agents

EvalLoop: A Methodology for Evaluation-Driven Iterative Improvement of Business AI Systems

arXiv:2607.05638v1 Announce Type: cross Abstract: Teams deploying large language models in business contexts need evaluation systems, yet most treat evaluation as static model selection: run benchmark

agentsarxiv-cs-ai

Foundation Models for Automatic CAD Generation

arXiv:2607.05573v1 Announce Type: new Abstract: Recent advances in Large Language Models (LLMs) and Vision-Language Models (VLMs) enable the automatic generation of parametric 3D designs from natural-

Latency-Constrained Hardware-Aware Quantum Error Correction Co-Design with Adaptive Confidence-Gated Neural Decoding for the Rotated Surface Code

arXiv:2607.05814v1 Announce Type: cross Abstract: Real-time decoding is a major bottleneck in scaling quantum error correction (QEC) from noisy intermediate-scale quantum (NISQ) devices to fault-toler

Spider 2.0-AIFunc: Extending Real-World Text-to-SQL to AI-Native SQL Workflows

arXiv:2607.06229v1 Announce Type: cross Abstract: Major cloud data platforms now expose large language model capabilities as native SQL functions, enabling analysts to perform classification, filterin

An Intervention-Based Framework for Shortcut Diagnosis in Spoofing Countermeasures

arXiv:2607.03150v1 Announce Type: cross Abstract: While deepfake audio detection systems achieve high performance in controlled benchmarks, their reliability often diminishes in the wild. Prior work s

researcharxiv-cs-lg

Do Medical Vision Language Models Actually See? A Counterfactual Grounding Framework and Hard-Negative Contrastive Training for Visually-Reliant Medical VLMs

arXiv:2607.03647v1 Announce Type: new Abstract: Large vision language models (VLMs) report strong accuracy on medical question-answering, yet it remains unclear whether they reason from visual evidenc

Evaluating and Understanding Model Editing for Medical Vision Language Models

arXiv:2607.05310v1 Announce Type: new Abstract: Model editing promises a fast, targeted way to correct post-deployment mistakes in medical vision-language models (VLMs) without costly retraining. Howe

Evaluating LLM Uncertainty in Long-Form Generation Using Deterministic Ground Truth

arXiv:2607.03870v1 Announce Type: new Abstract: As LLMs generate increasingly long outputs, effective uncertainty estimation must identify errors at fine-grained levels rather than discard entire resp

From General Actions to Domain-Specific Monitoring: Prior-Adaptive Transfer for Skeleton-Based Action Recognition

arXiv:2607.03327v1 Announce Type: new Abstract: Skeleton-based action recognition models have recently shown strong performance on large-scale benchmarks with general actions. However, directly transf

Generative Inverse Design with Abstention via Diagonal Flow Matching

arXiv:2603.15925v2 Announce Type: replace Abstract: Inverse design aims to find design parameters x achieving target performance y^*. Generative approaches learn bidirectional mappings between designs

LiNO: Lifting based multiresolution neural operator

arXiv:2607.02715v1 Announce Type: new Abstract: Recently, neural operators have shown promising outcomes for learning solution operators of differential equations directly from data. This framework le

Local Ai

LoMa: Local Feature Matching Revisited

arXiv:2604.04931v2 Announce Type: replace Abstract: Local feature matching has long been a fundamental component of 3D vision systems such as Structure-from-Motion (SfM), yet progress has lagged behin

local-aiarxiv-cs-cv

Obey, Diverge, Collapse: Blind Obedience to Incorrect Instructions Drives Code LLMs to Irrecoverable Code Semantic Collapse

arXiv:2607.04537v1 Announce Type: cross Abstract: Code language models are now trusted collaborators in production workflows for debugging, refactoring, and iterative repair, and every benchmark that

RES-DARE: Failure-Aware Expert Adaptation and Rollback-Safe Self-Repair for Intrusion Detection

arXiv:2607.02687v1 Announce Type: cross Abstract: Intrusion detection systems are often trained under static benchmark conditions, although deployed network environments are affected by traffic drift,

SafeGuard: A Multi-Agent Perception-Reasoning Framework for Social-Risk AI-Generated Video Detection

arXiv:2607.03069v1 Announce Type: new Abstract: As video generation paradigms evolve from localized manipulation to full-scene synthesis, AI-generated video detection becomes increasingly challenging,

SovereignNegotiation-Bench: Evaluating User-Owned Personal Agents In Delegated Bargaining Under Privacy, Consent, Evidence, And Institutional Pressure

arXiv:2607.02814v1 Announce Type: cross Abstract: Personal agents will increasingly negotiate on behalf of users: splitting costs with other personal agents, appealing platform decisions, escalating s

SovereignPA-Bench: Evaluating User-Owned Personal Agents under Evolving Intent, Platform Mediation, and Consent Constraints

arXiv:2607.05363v1 Announce Type: new Abstract: Personal agents are becoming persistent user-owned intermediaries: they remember preferences, filter platform-mediated information, use tools, and negot

Agents

The 'I Don't Know' Filter: Enhancing Agentic Reliability in Function Calling

arXiv:2607.04034v1 Announce Type: cross Abstract: The language models that underpin agents have seen a rapid rise in performance on function calling benchmarks. However, the metrics used in the traini

agentsarxiv-cs-ai

The P^3 Dataset: Pixels, Points and Polygons for Multimodal Building Vectorization

arXiv:2505.15379v2 Announce Type: replace Abstract: We present the P^3 dataset, a large-scale multimodal benchmark for building vectorization, constructed from aerial LiDAR point clouds, high-resoluti

TSP with Predictions: Heatmap to Tour with Provable Guarantees

arXiv:2607.03791v1 Announce Type: cross Abstract: The Traveling Salesperson Problem (TSP) has long served as a benchmark for evaluating the strength of optimization techniques in the classical theory

VERITAS: Towards a General-Purpose Replication Tool for Scientific Research

arXiv:2607.02931v1 Announce Type: new Abstract: AI tools are accelerating scientific publication while the systems that review it struggle to keep up, and independent verification of published researc

VideoSearcher: Empowering Video Deep Research with Multi-Tool Agentic Reasoning via Reinforcement Learning

arXiv:2607.02927v1 Announce Type: cross Abstract: Video understanding is moving beyond closed-context perception toward open-world evidence exploration, a paradigm formalized as Video Deep Research (V

When Do Foundation Models Pay Off? A Break-Even Analysis of Pretrained Time Series Forecasters

arXiv:2607.04919v1 Announce Type: new Abstract: Deploying a time series foundation model requires GPU infrastructure, engineering overhead, and carries no guarantee of improvement over XGBoost. We pro

When Simpler Is Better: Evaluating Translation Pipelines for Medieval Latin Manuscripts

arXiv:2607.03836v1 Announce Type: cross Abstract: Despite remarkable progress in machine translation, Vision Language Models (VLMs) struggle on historical manuscripts, a domain that stresses core Natu

HaloGuard 1.0: An Open Weights Constitutional Classifier for Multilingual AI Safety

arXiv:2607.02079v1 Announce Type: new Abstract: We present HaloGuard 1.0, an open-weights implementation of the constitutional-classifier paradigm for input safety. It achieves state-of-the-art perfor

3 Jul 2026

ALEE: Any-Language Evaluation of Embeddings via English-Centric Minimal Pairs

arXiv:2607.00171v1 Announce Type: new Abstract: Text embeddings are standard for semantic similarity tasks, yet their evaluation remains an open challenge. Current benchmarks are static, cover only a

researcharxiv-cs-cl

Beyond Document Grounding: Span-Level Hallucination Detection over Code, Tool Output, and Documents

arXiv:2607.00895v1 Announce Type: new Abstract: Hallucination detection for retrieval-augmented generation (RAG) is usually evaluated on natural-language document evidence. However, grounded generatio

Can Agents Generalize to the Open World? Unveiling the Fragility of Static Training in Tool Use

arXiv:2607.01084v1 Announce Type: new Abstract: While Large Language Model (LLM) agents demonstrate proficiency in static benchmarks, their deployment in real-world scenarios is hindered by the dynami

FusionFactory: Fusing LLM Capabilities with Multi-LLM Log Data

arXiv:2507.10540v3 Announce Type: replace Abstract: The rapid advancement of large language models (LLMs) has created a diverse landscape of models, each excelling at different tasks. This diversity d

GAIA: Geometry-Adaptive Operator Learning for Forward and Inverse Problems

arXiv:2607.01128v1 Announce Type: new Abstract: Operator learning for partial differential equations (PDEs) on arbitrary geometries builds fast neural surrogates for large-scale simulation. Although r

Interpretable vs Learned Encoders for High-Cardinality Fraud Detection

arXiv:2607.00477v1 Announce Type: new Abstract: A total of seven categorical encoding methods were tested on the IEEE-CIS fraud benchmark dataset (590,540 records, 3.5% positives, 8 high-cardinality c

Rethinking Visual Privacy: A Compositional Privacy Risk Framework for Severity Assessment with VLMs

arXiv:2603.21573v2 Announce Type: replace Abstract: Existing visual privacy benchmarks largely treat privacy as a binary property, labeling images as private or non-private based on visible sensitive

researcharxiv-cs-cv

TerraBench: Can Agents Reason Over Heterogeneous Earth-System Data?

arXiv:2606.13148v2 Announce Type: replace Abstract: Climate and environmental decision-making increasingly requires reasoning across heterogeneous inputs, including gridded physical data, satellite im

Testing Frontier Large Language Models' Physics Literacy in Parallel Physical Worlds

arXiv:2607.00276v1 Announce Type: cross Abstract: Current large-language-model (LLM) physics benchmarks are usually scored by answer accuracy, which cannot distinguish genuine reasoning from recall of

Auditing Generalization in AI-Generated Video Detection: A Six-Control Protocol and the VidAudit Toolkit

arXiv:2606.31004v1 Announce Type: new Abstract: AI-generated video detection benchmarks such as GenVidBench and AIGVDBench are the de facto leaderboards, yet most evaluation protocols leave uncontroll

researcharxiv-cs-cv

Building an ASR Solution for Training and Assessing Children's Reading

arXiv:2606.31508v1 Announce Type: new Abstract: Automatic speech recognition for children's reading remains underdeveloped for most African languages, including Bambara, despite its potential value fo

CDR-Bench: Evaluating Faithful Execution of Compositional, Order-Sensitive Data Refinement Recipes

arXiv:2606.31435v1 Announce Type: new Abstract: Data refinement involves executing multi-step recipes over evolving text states, where both composition and execution order of processing operators dete

Safety

CLExEval: A Human-in-the-Loop Framework for Qualitative Evaluation of LLM Clinical Reasoning

arXiv:2606.31608v1 Announce Type: new Abstract: Large Language Models (LLMs) achieve strong results on many medical benchmarks, but their clinical reasoning remains difficult to evaluate reliably. A c

safetyarxiv-cs-cl

DetPO: In-Context Learning with Multi-Modal LLMs for Few-Shot Object Detection

arXiv:2603.23455v2 Announce Type: replace Abstract: Multi-Modal LLMs (MLLMs) demonstrate strong visual grounding capabilities on popular object detection benchmarks like OdinW-13 and RefCOCO. However,

researcharxiv-cs-cv

Modeling Cell-Cycle-Aware Single-Cell Drug Perturbation Responses

arXiv:2606.30695v1 Announce Type: cross Abstract: Single-cell drug perturbation models should predict not only transcriptional response magnitude, but also whether a treatment alters the proliferative

RESOLVE: A Multi-Resolution and Multi-Modal Dataset for Roadside Cooperative Perception

arXiv:2606.31895v1 Announce Type: new Abstract: LiDAR has increasingly been integrated into traffic cameras to expand coverage and mitigate occlusion in roadside cooperative perception. However, how u