Search: “benchmark”

Unmasking the Illusion of Embodied Reasoning in Vision-Language-Action Models

model-releasesarxiv-cs-ro

arXiv:2604.18000v1 Announce Type: new Abstract: Recent Vision-Language-Action (VLA) models report impressive success rates on standard robotic benchmarks, fueling optimism about general-purpose physic

21 Apr 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Who is the richest club in the championship? Detecting and Rewriting Underspecified Questions Improve QA Performance

DGX agent

arXiv:2602.11938v5 Announce Type: replace Abstract: Large language models (LLMs) perform well on well-posed questions, yet standard question-answering (QA) benchmarks remain far from solved. We argue

model-releasesarxiv-cs-cl

21 Apr 2026

Research

Comparison Study: Glacier Calving Front Delineation in Synthetic Aperture Radar Images With Deep Learning

arXiv:2501.05281v2 Announce Type: replace Abstract: Continuous monitoring of glacier calving fronts is essential for sea level rise projections. This study benchmarks Deep Learning systems for front d

researcharxiv-cs-cv

Agents

Discover and Prove: An Open-source Agentic Framework for Hard Mode Automated Theorem Proving in Lean 4

arXiv:2604.15839v1 Announce Type: new Abstract: Most ATP benchmarks embed the final answer within the formal statement -- a convention we call 'Easy Mode' -- a design that simplifies the task relative

agentsarxiv-cs-ai

DiZiNER: Disagreement-guided Instruction Refinement via Pilot Annotation Simulation for Zero-shot Named Entity Recognition

arXiv:2604.15866v1 Announce Type: cross Abstract: Large language models (LLMs) have advanced information extraction (IE) by enabling zero-shot and few-shot named entity recognition (NER), yet their ge

EvoTest: Evolutionary Test-Time Learning for Self-Improving Agentic Systems

arXiv:2510.13220v2 Announce Type: replace Abstract: A fundamental limitation of current AI agents is their inability to learn complex skills on the fly at test time, often behaving like 'clever but cl

'Excuse me, may I say something...' CoLabScience, A Proactive AI Assistant for Biomedical Discovery and LLM-Expert Collaborations

arXiv:2604.15588v1 Announce Type: cross Abstract: The integration of Large Language Models (LLMs) into scientific workflows presents exciting opportunities to accelerate biomedical discovery. However,

MEDLEY-BENCH: Scale Buys Evaluation but Not Control in AI Metacognition

arXiv:2604.16009v1 Announce Type: new Abstract: Metacognition, the ability to monitor and regulate one's own reasoning, remains under-evaluated in AI benchmarking. We introduce MEDLEY-BENCH, a benchma

Research

GUI-Perturbed: Domain Randomization Reveals Systematic Brittleness in GUI Grounding Models

arXiv:2604.14262v1 Announce Type: new Abstract: GUI grounding models report over 85% accuracy on standard benchmarks, yet drop 27-56 percentage points when instructions require spatial reasoning rathe

researcharxiv-cs-lg

17 Apr 2026

Safety

Pushing the Boundaries of Multiple Choice Evaluation to One Hundred Options

arXiv:2604.14634v1 Announce Type: new Abstract: Multiple choice evaluation is widely used for benchmarking large language models, yet near ceiling accuracy in low option settings can be sustained by s

safetyarxiv-cs-cl

17 Apr 2026

Rethinking Patient Education as Multi-turn Multi-modal Interaction

arXiv:2604.14656v1 Announce Type: cross Abstract: Most medical multimodal benchmarks focus on static tasks such as image question answering, report generation, and plain-language rewriting. Patient ed

17 Apr 2026

From Feelings to Metrics: Understanding and Formalizing How Users Vibe-Test LLMs

arXiv:2604.14137v1 Announce Type: new Abstract: Evaluating LLMs is challenging, as benchmark scores often fail to capture models' real-world usefulness. Instead, users often rely on ``vibe-testing'':

16 Apr 2026

Scaling Test-Time Compute to Achieve IOI Gold Medal with Open-Weight Models

arXiv:2510.14232v2 Announce Type: replace-cross Abstract: Competitive programming has become a rigorous benchmark for evaluating the reasoning and problem-solving capabilities of large language models

16 Apr 2026

WorkRB: A Community-Driven Evaluation Framework for AI in the Work Domain

arXiv:2604.13055v1 Announce Type: new Abstract: Today's evolving labor markets rely increasingly on recommender systems for hiring, talent management, and workforce analytics, with natural language pr

16 Apr 2026

Beyond Scores: Diagnostic LLM Evaluation via Fine-Grained Abilities

arXiv:2604.12191v1 Announce Type: new Abstract: Current evaluations of large language models aggregate performance across diverse tasks into single scores. This obscures fine-grained ability variation

15 Apr 2026

Modality-Agnostic Prompt Learning for Multi-Modal Camouflaged Object Detection

arXiv:2604.12380v1 Announce Type: new Abstract: Camouflaged Object Detection (COD) aims to segment objects that blend seamlessly into complex backgrounds, with growing interest in exploiting additiona

15 Apr 2026

BankerToolBench: Evaluating AI Agents in End-to-End Investment Banking Workflows

arXiv:2604.11304v1 Announce Type: new Abstract: Existing AI benchmarks lack the fidelity to assess economically meaningful progress on professional workflows. To evaluate frontier AI agents in a high-

ChatCLIDS: Simulating Persuasive AI Dialogues to Promote Closed-Loop Insulin Adoption in Type 1 Diabetes Care

arXiv:2509.00891v3 Announce Type: replace Abstract: Real-world adoption of closed-loop insulin delivery systems (CLIDS) in type 1 diabetes remains low, driven not by technical failure, but by diverse

DecepGPT: Schema-Driven Deception Detection with Multicultural Datasets and Robust Multimodal Learning

arXiv:2603.23916v2 Announce Type: replace-cross Abstract: Multimodal deception detection aims to identify deceptive behavior by analyzing audiovisual cues for forensics and security. In these high-sta

EgoFun3D: Modeling Interactive Objects from Egocentric Videos using Function Templates

arXiv:2604.11038v1 Announce Type: new Abstract: We present EgoFun3D, a coordinated task formulation, dataset, and benchmark for modeling interactive 3D objects from egocentric videos. Interactive obje

HiPRAG: Hierarchical Process Rewards for Efficient Agentic Retrieval Augmented Generation

arXiv:2510.07794v2 Announce Type: replace-cross Abstract: Agentic RAG is a powerful technique for incorporating external information that LLMs lack, enabling better problem solving and question answer

Learning World Models for Interactive Video Generation

arXiv:2505.21996v3 Announce Type: replace-cross Abstract: Foundational world models must be both interactive and preserve spatiotemporal coherence for effective future planning with action choices. Ho

LLMs for Text-Based Exploration and Navigation Under Partial Observability

arXiv:2604.09604v1 Announce Type: new Abstract: Exploration and goal-directed navigation in unknown layouts are central to inspection, logistics, and search-and-rescue. We ask whether large language m

What Users Leave Unsaid: Under-Specified Queries Limit Vision-Language Models

arXiv:2601.06165v2 Announce Type: replace-cross Abstract: Current vision-language benchmarks predominantly feature well-structured questions with clear, explicit prompts. However, real user queries ar

Safety

Do LLMs Follow Their Own Rules? A Reflexive Audit of Self-Stated Safety Policies

arXiv:2604.09189v1 Announce Type: cross Abstract: LLMs internalize safety policies through RLHF, yet these policies are never formally specified and remain difficult to inspect. Existing benchmarks ev

safetyarxiv-cs-ai

Do We Really Need to Approach the Entire Pareto Front in Many-Objective Bayesian Optimisation?

arXiv:2604.09417v1 Announce Type: new Abstract: Many-objective optimisation, a subset of multi-objective optimisation, involves optimisation problems with more than three objectives. As the number of

DRBENCHER: Can Your Agent Identify the Entity, Retrieve Its Properties and Do the Math?

arXiv:2604.09251v1 Announce Type: new Abstract: Deep research agents increasingly interleave web browsing with multi-step computation, yet existing benchmarks evaluate these capabilities in isolation,

From Paper to Program: Accelerating Quantum Many-Body Algorithm Development via a Multi-Stage LLM-Assisted Workflow

arXiv:2604.04089v2 Announce Type: replace-cross Abstract: Large language models (LLMs) can generate code rapidly but remain unreliable for scientific algorithms whose correctness depends on structural

Hidden in Plain Sight: Visual-to-Symbolic Analytical Solution Inference from Field Visualizations

arXiv:2604.08863v1 Announce Type: new Abstract: Recovering analytical solutions of physical fields from visual observations is a fundamental yet underexplored capability for AI-assisted scientific rea

HTNav: A Hybrid Navigation Framework with Tiered Structure for Urban Aerial Vision-and-Language Navigation

arXiv:2604.08883v1 Announce Type: cross Abstract: Inspired by the general Vision-and-Language Navigation (VLN) task, aerial VLN has attracted widespread attention, owing to its significant practical v

Memory-efficient Continual Learning with Prototypical Exemplar Condensation

arXiv:2603.13804v2 Announce Type: replace-cross Abstract: Rehearsal-based continual learning (CL) mitigates catastrophic forgetting by maintaining a subset of samples from previous tasks for replay. E

Temperature-Dependent Performance of Prompting Strategies in Extended Reasoning Large Language Models

arXiv:2604.08563v1 Announce Type: cross Abstract: Extended reasoning models represent a transformative shift in Large Language Model (LLM) capabilities by enabling explicit test-time computation for c

TiAb Review Plugin: A Browser-Based Tool for AI-Assisted Title and Abstract Screening

arXiv:2604.08602v1 Announce Type: cross Abstract: Background: Server-based screening tools impose subscription costs, while open-source alternatives require coding skills. Objectives: We developed a b

A Systematic Study of Retrieval Pipeline Design for Retrieval-Augmented Medical Question Answering

arXiv:2604.07274v1 Announce Type: cross Abstract: Large language models (LLMs) have demonstrated strong capabilities in medical question answering; however, purely parametric models often suffer from

CAMO: A Class-Aware Minority-Optimized Ensemble for Robust Language Model Evaluation on Imbalanced Data

arXiv:2604.07583v1 Announce Type: new Abstract: Real-world categorization is severely hampered by class imbalance because traditional ensembles favor majority classes, which lowers minority performanc

Consistency-Guided Decoding with Proof-Driven Disambiguation for Three-Way Logical Question Answering

arXiv:2604.06196v1 Announce Type: cross Abstract: Three-way logical question answering (QA) assigns True/False/Unknown to a hypothesis H given a premise set S. While modern large language models

DISSECT: Diagnosing Where Vision Ends and Language Priors Begin in Scientific VLMs

arXiv:2604.06250v1 Announce Type: cross Abstract: When asked to describe a molecular diagram, a Vision-Language Model correctly identifies ``a benzene ring with an -OH group.'' When asked to reason ab

Lost in Cultural Translation: Do LLMs Struggle with Math Across Cultural Contexts?

arXiv:2503.18018v2 Announce Type: replace Abstract: We demonstrate that large language models' (LLMs) mathematical reasoning is culturally sensitive: testing 14 models from Anthropic, OpenAI, Google,

SearchAD: Large-Scale Rare Image Retrieval Dataset for Autonomous Driving

arXiv:2604.08008v1 Announce Type: new Abstract: Retrieving rare and safety-critical driving scenarios from large-scale datasets is essential for building robust autonomous driving (AD) systems. As dat

TeamLLM: A Human-Like Team-Oriented Collaboration Framework for Multi-Step Contextualized Tasks

arXiv:2604.06765v1 Announce Type: cross Abstract: Recently, multi-Large Language Model (LLM) frameworks have been proposed to solve contextualized tasks. However, these frameworks do not explicitly em

TEMPER: Testing Emotional Perturbation in Quantitative Reasoning

arXiv:2604.07801v1 Announce Type: new Abstract: Large language models are trained and evaluated on quantitative reasoning tasks written in clean, emotionally neutral language. However, real-world quer

Tutorials

Understanding Task Transfer in Vision-Language Models

arXiv:2511.18787v2 Announce Type: replace Abstract: Vision-Language Models (VLMs) perform well on multimodal benchmarks but lag behind humans and specialized models on visual perception tasks like dep

tutorialsarxiv-cs-cv

Visually-grounded Humanoid Agents

arXiv:2604.08509v1 Announce Type: new Abstract: Digital human generation has been studied for decades and supports a wide range of real-world applications. However, most existing systems are passively

Weakly Supervised Distillation of Hallucination Signals into Transformer Representations

arXiv:2604.06277v1 Announce Type: new Abstract: Existing hallucination detection methods for large language models (LLMs) rely on external verification at inference time, requiring gold answers, retri

Distributionally Robust Regularization of Sparse Integer Programming Trained Learning Models

arXiv:2311.01994v2 Announce Type: replace-cross Abstract: Building explainable machine learning models is crucial for human users to be able to interpret the proposed statistical relationship obtained

FlashAttention with Statistics: Low-Overhead Attention Features for Compact Hallucination Detectors

arXiv:2610.21483v1 Announce Type: new Abstract: Large language models can produce fluent answers that the supplied context does not support. A compact frozen proxy model can detect such answers by rea

Lethe: Link Inference Attacks For Evaluation of Edge Unlearning Methods

model-releasesarxiv-cs-lg

arXiv:2610.21869v1 Announce Type: new Abstract: Graph Unlearning (GU) aims to remove the influence of specific training data from a Graph Neural Network without retraining anew. While most methods add

MeddeX: A Retrieval-Augmented LLaMA-3 System with Gemini Vision for Evidence-Backed Medical Report Analysis

arXiv:2610.20632v1 Announce Type: new Abstract: Modern healthcare generates increasingly complex medical reports that challenge clinical professionals in terms of time and interpretive burden. Manual