Search: “evaluation”

Agent^2 RL-Bench: Can LLM Agents Engineer Agentic RL Post-Training?

arXiv:2604.10547v1 Announce Type: new Abstract: We introduce Agent^2 RL-Bench, a benchmark for evaluating agentic RL post-training -- whether LLM agents can autonomously design, implement, and run com

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Agents

BridgeSim: Unveiling the OL-CL Gap in End-to-End Autonomous Driving

DGX agent

arXiv:2604.10856v1 Announce Type: cross Abstract: Open-loop (OL) to closed-loop (CL) gap (OL-CL gap) exists when OL-pretrained policies scoring high in OL evaluations fail to transfer effectively in c

agentsarxiv-cs-ai

14 Apr 2026

ChatCLIDS: Simulating Persuasive AI Dialogues to Promote Closed-Loop Insulin Adoption in Type 1 Diabetes Care

arXiv:2509.00891v3 Announce Type: replace Abstract: Real-world adoption of closed-loop insulin delivery systems (CLIDS) in type 1 diabetes remains low, driven not by technical failure, but by diverse

Consistency of AI-Generated Exercise Prescriptions: A Repeated Generation Study Using a Large Language Model

arXiv:2604.11287v1 Announce Type: new Abstract: Background: Large language models (LLMs) have been explored as tools for generating personalized exercise prescriptions, yet the consistency of outputs

Decision-Theoretic Safety Assessment of Persona-Driven Multi-Agent Systems in O-RAN

arXiv:2604.09682v1 Announce Type: cross Abstract: Autonomous network management in Open Radio Access Networks requires intelligent decision making across conflicting objectives, yet existing LLM based

safetyarxiv-cs-ai

Disambiguation-Centric Finetuning Makes Enterprise Tool-Calling LLMs More Realistic and Less Risky

arXiv:2507.03336v4 Announce Type: replace Abstract: Large language models (LLMs) are increasingly tasked with invoking enterprise APIs, yet they routinely falter when near-duplicate tools vie for the

Improving understanding and trust in AI: How users benefit from interval-based counterfactual explanations

arXiv:2604.09573v1 Announce Type: cross Abstract: Experimental user studies evaluating the effectiveness of different subtypes of post-hoc explanations for black-box models are largely nonexistent. Th

researcharxiv-cs-lg

ITIScore: An Image-to-Text-to-Image Rating Framework for the Image Captioning Ability of MLLMs

arXiv:2604.03765v2 Announce Type: replace Abstract: Recent advances in multimodal large language models (MLLMs) have greatly improved image understanding and captioning capabilities. However, existing

LLM-as-Judge on a Budget

arXiv:2602.15481v2 Announce Type: replace Abstract: LLM-as-a-judge has emerged as a cornerstone technique for evaluating large language models by leveraging LLM reasoning to score prompt-response pair

safetyarxiv-cs-lg

MMRareBench: A Rare-Disease Multimodal and Multi-Image Medical Benchmark

arXiv:2604.10755v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have advanced clinical tasks for common conditions, but their performance on rare diseases remains largely unte

NameBERT: Scaling Name-Based Nationality Classification with LLM-Augmented Open Academic Data

arXiv:2604.10401v1 Announce Type: new Abstract: Inferring nationality from personal names is a critical capability for equity and bias monitoring, personalization, and a valuable tool in biomedical an

safetyarxiv-cs-cl

Property-Preserving Hashing for ell_1-Distance Predicates: Applications to Countering Adversarial Input Attacks

arXiv:2504.16355v2 Announce Type: replace-cross Abstract: Perceptual hashing is used to detect whether an input image is similar to a reference image with a variety of security applications. Recently,

researcharxiv-cs-lg

Applications

Quantization Robustness to Input Degradations for Object Detection

arXiv:2508.19600v2 Announce Type: replace Abstract: Post-training quantization (PTQ) is crucial for deploying efficient object detection models, like YOLO, on resource-constrained devices. However, th

applicationsarxiv-cs-cv

Should We be Pedantic About Reasoning Errors in Machine Translation?

arXiv:2604.09890v1 Announce Type: cross Abstract: Across multiple language pairings (English o {Spanish, French, German, Mandarin, Japanese, Urdu, Cantonese}), we find reasoning errors in translation.

SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding

arXiv:2604.09557v1 Announce Type: cross Abstract: Speculative Decoding (SD) has emerged as a critical technique for accelerating Large Language Model (LLM) inference. Unlike deterministic system optim

SRBench: A Comprehensive Benchmark for Sequential Recommendation with Large Language Models

arXiv:2604.09553v1 Announce Type: cross Abstract: LLM development has aroused great interest in Sequential Recommendation (SR) applications. However, comprehensive evaluation of SR models remains lack

TrajOnco: a multi-agent framework for temporal reasoning over longitudinal EHR for multi-cancer early detection

arXiv:2604.10386v1 Announce Type: new Abstract: Accurate estimation of cancer risk from longitudinal electronic health records (EHRs) could support earlier detection and improved care, but modeling su

TS-Haystack: A Multi-Scale Retrieval Benchmark for Time Series Language Models

model-releasesarxiv-cs-lg

arXiv:2602.14200v4 Announce Type: replace Abstract: Time Series Language Models (TSLMs) are emerging as unified models for reasoning over continuous signals in natural language. However, long-context

ALMAB-DC: Active Learning, Multi-Armed Bandits, and Distributed Computing for Sequential Experimental Design and Black-Box Optimization

model-releasesarxiv-cs-lg

arXiv:2603.21180v3 Announce Type: replace Abstract: Sequential experimental design under expensive, gradient-free objectives is a central challenge in computational statistics: evaluation budgets are

Enhancing LLM Problem Solving via Tutor-Student Multi-Agent Interaction

arXiv:2604.08931v1 Announce Type: new Abstract: Human cognitive development is shaped not only by individual effort but by structured social interaction, where role-based exchanges such as those betwe

EXAONE 4.5 Technical Report

arXiv:2604.08644v1 Announce Type: new Abstract: This technical report introduces EXAONE 4.5, the first open-weight vision language model released by LG AI Research. EXAONE 4.5 is architected by integr

FashionStylist: An Expert Knowledge-enhanced Multimodal Dataset for Fashion Understanding

arXiv:2604.09249v1 Announce Type: new Abstract: Fashion understanding requires both visual perception and expert-level reasoning about style, occasion, compatibility, and outfit rationale. However, ex

GeoMMBench and GeoMMAgent: Toward Expert-Level Multimodal Intelligence in Geoscience and Remote Sensing

arXiv:2604.08896v1 Announce Type: new Abstract: Recent advances in multimodal large language models (MLLMs) have accelerated progress in domain-oriented AI, yet their development in geoscience and rem

GRASP: Grounded CoT Reasoning with Dual-Stage Optimization for Multimodal Sarcasm Target Identification

arXiv:2604.08879v1 Announce Type: new Abstract: Moving beyond the traditional binary classification paradigm of Multimodal Sarcasm Detection, Multimodal Sarcasm Target Identification (MSTI) presents a

Hidden in Plain Sight: Visual-to-Symbolic Analytical Solution Inference from Field Visualizations

arXiv:2604.08863v1 Announce Type: new Abstract: Recovering analytical solutions of physical fields from visual observations is a fundamental yet underexplored capability for AI-assisted scientific rea

HM-Bench: A Comprehensive Benchmark for Multimodal Large Language Models in Hyperspectral Remote Sensing

arXiv:2604.08884v1 Announce Type: cross Abstract: While multimodal large language models (MLLMs) have made significant strides in natural image understanding, their ability to perceive and reason over

Predicting Metabolic Dysfunction-Associated Steatotic Liver Disease using Machine Learning Methods: A Retrospective Cohort Study

arXiv:2510.22293v4 Announce Type: replace Abstract: Background: Metabolic dysfunction-associated steatotic liver disease (MASLD) affects 30-40% of US adults and is the most common chronic liver diseas

safetyarxiv-cs-lg

PSIRNet: Deep Learning-based Free-breathing Rapid Acquisition Late Enhancement Imaging

arXiv:2604.08781v1 Announce Type: cross Abstract: Purpose: To develop and evaluate a deep learning (DL) method for free-breathing phase-sensitive inversion recovery (PSIR) late gadolinium enhancement

QuanBench+: A Unified Multi-Framework Benchmark for LLM-Based Quantum Code Generation

arXiv:2604.08570v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly used for code generation, yet quantum code generation is still evaluated mostly within single frameworks

Reasoning Models Will Sometimes Lie About Their Reasoning

arXiv:2601.07663v3 Announce Type: replace Abstract: Hint-based faithfulness evaluations have established that Large Reasoning Models (LRMs) may not say what they think: they do not always volunteer in

SessionIntentBench: A Multi-task Inter-session Intention-shift Modeling Benchmark for E-commerce Customer Behavior Understanding

arXiv:2507.20185v2 Announce Type: replace Abstract: Session history is a common way of recording user interacting behaviors throughout a browsing activity with multiple products. For example, if an us

A Study of LLMs' Preferences for Libraries and Programming Languages

arXiv:2503.17181v3 Announce Type: replace-cross Abstract: Despite the rapid progress of large language models (LLMs) in code generation, existing evaluations focus on functional correctness or syntact

ADAG: Automatically Describing Attribution Graphs

arXiv:2604.07615v1 Announce Type: new Abstract: In language model interpretability research, extbf{circuit tracing} aims to identify which internal features causally contributed to a particular outp

Adapting Foundation Models for Annotation-Efficient Adnexal Mass Segmentation in Cine Images

arXiv:2604.08045v1 Announce Type: new Abstract: Adnexal mass evaluation via ultrasound is a challenging clinical task, often hindered by subjective interpretation and significant inter-observer variab

researcharxiv-cs-cv

AgentCity: Constitutional Governance for Autonomous Agent Economies via Separation of Power

arXiv:2604.07007v1 Announce Type: cross Abstract: Autonomous AI agents are beginning to operate across organizational boundaries on the open internet -- discovering, transacting with, and delegating t

safetyarxiv-cs-ai

AutoReproduce: Automatic AI Experiment Reproduction with Paper Lineage

arXiv:2505.20662v3 Announce Type: replace Abstract: Efficient reproduction of research papers is pivotal to accelerating scientific progress. However, the increasing complexity of proposed methods oft

AV-SQL: Decomposing Complex Text-to-SQL Queries with Agentic Views

arXiv:2604.07041v1 Announce Type: cross Abstract: Text-to-SQL is the task of translating natural language queries into executable SQL for a given database, enabling non-expert users to access structur

Before We Trust Them: Decision-Making Failures in Navigation of Foundation Models

arXiv:2601.05529v5 Announce Type: replace Abstract: High success rates on navigation-related tasks do not necessarily translate into reliable decision making by foundation models. To examine this gap,

Benchmarking LLM Tool-Use in the Wild

arXiv:2604.06185v1 Announce Type: cross Abstract: Fulfilling user needs through Large Language Model multi-turn, multi-step tool-use is rarely a straightforward process. Real user interactions are inh

Break Me If You Can: Self-Jailbreaking of Aligned LLMs via Lexical Insertion Prompting

arXiv:2601.02670v2 Announce Type: replace Abstract: We introduce self-jailbreaking, a threat model in which an aligned LLM guides its own compromise. Unlike most jailbreak techniques, which oft

CAMotion: A High-Quality Benchmark for Camouflaged Moving Object Detection in the Wild

arXiv:2604.08287v1 Announce Type: new Abstract: Discovering camouflaged objects is a challenging task in computer vision due to the high similarity between camouflaged objects and their surroundings.

ConsistRM: Improving Generative Reward Models via Consistency-Aware Self-Training

arXiv:2604.07484v1 Announce Type: cross Abstract: Generative reward models (GRMs) have emerged as a promising approach for aligning Large Language Models (LLMs) with human preferences by offering grea

Contextual Earnings-22: A Speech Recognition Benchmark with Custom Vocabulary in the Wild

arXiv:2604.07354v1 Announce Type: new Abstract: The accuracy frontier of speech-to-text systems has plateaued on academic benchmarks.1 In contrast, industrial benchmarks and adoption in high-stakes do

CycleChart: A Unified Consistency-Based Learning Framework for Bidirectional Chart Understanding and Generation

arXiv:2512.19173v2 Announce Type: replace Abstract: Current chart-related tasks, such as chart generation (NL2Chart), chart schema parsing, chart data parsing, and chart question answering (ChartQA),

ESOM: Efficiently Understanding Streaming Video Anomalies with Open-world Dynamic Definitions

arXiv:2604.07772v1 Announce Type: new Abstract: Open-world video anomaly detection (OWVAD) aims to detect and explain abnormal events under different anomaly definitions, which is important for applic

HY-Embodied-0.5: Embodied Foundation Models for Real-World Agents

arXiv:2604.07430v1 Announce Type: new Abstract: We introduce HY-Embodied-0.5, a family of foundation models specifically designed for real-world embodied agents. To bridge the gap between general Visi

IatroBench: Pre-Registered Evidence of Iatrogenic Harm from AI Safety Measures

arXiv:2604.07709v1 Announce Type: cross Abstract: Ask a frontier model how to taper six milligrams of alprazolam (psychiatrist retired, ten days of pills left, abrupt cessation causes seizures) and it

Inside-Out: Measuring Generalization in Vision Transformers Through Inner Workings

arXiv:2604.08192v1 Announce Type: cross Abstract: Reliable generalization metrics are fundamental to the evaluation of machine learning models. Especially in high-stakes applications where labeled tar

safetyarxiv-cs-cv