Search: “evaluation”

CareGuardAI: Context-Aware Multi-Agent Guardrails for Clinical Safety & Hallucination Mitigation in Patient-Facing LLMs

arXiv:2604.26959v1 Announce Type: cross Abstract: Integrating large language models (LLMs) into patient-facing healthcare systems offers significant potential to improve access to medical information.

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Claw-Eval-Live: A Live Agent Benchmark for Evolving Real-World Workflows

DGX agent

arXiv:2604.28139v1 Announce Type: cross Abstract: LLM agents are expected to complete end-to-end units of work across software tools, business services, and local workspaces. Yet many agent benchmarks

model-releasesarxiv-cs-ai

1 May 2026

ClimateVID -- Social Media Videos Analysis and Challenges Involved

arXiv:2604.27968v1 Announce Type: new Abstract: The pervasive growth of digital content, specifically short videos on social media platforms, has significantly altered how topics are discussed and und

researcharxiv-cs-cv

Decoding Scientific Experimental Images: The SPUR Benchmark for Perception, Understanding, and Reasoning

model-releasesarxiv-cs-cv

arXiv:2604.27604v1 Announce Type: new Abstract: We introduce SPUR, a comprehensive benchmark for scientific experimental image perception, understanding, and reasoning, comprising 4,264 question-answe

Detecting Clinical Discrepancies in Health Coaching Agents: A Dual-Stream Memory and Reconciliation Architecture

arXiv:2604.27045v1 Announce Type: cross Abstract: As Large Language Model (LLM) agents transition from single-session tools to persistent systems managing longitudinal healthcare journeys, their memor

FinChain: A Symbolic Benchmark for Verifiable Chain-of-Thought Financial Reasoning

arXiv:2506.02515v4 Announce Type: replace-cross Abstract: Multi-step symbolic reasoning is essential for robust financial analysis; yet, current benchmarks largely overlook this capability. Existing d

Flattery in Motion: Benchmarking and Analyzing Sycophancy in Video-LLMs

arXiv:2506.07180v3 Announce Type: replace-cross Abstract: As video large language models (Video-LLMs) become increasingly integrated into real-world applications that demand grounded multimodal reason

Function-based Parametric Co-Design Optimization of Dexterous Hands

model-releasesarxiv-cs-ro

arXiv:2604.27557v1 Announce Type: new Abstract: Despite advances in dexterous hand manipulation, robotic hand design is still largely decoupled from task-driven evaluation and control, limiting system

Learning to Spend: Model Predictive Control for Budgeting under Non-Stationary Returns

arXiv:2604.27186v1 Announce Type: cross Abstract: We study finite-horizon budget allocation as a closed-loop economic control problem and evaluate receding-horizon Model Predictive Control (MPC) relat

LLMs Capture Emotion Labels, Not Emotion Uncertainty: Distributional Analysis and Calibration of Human--LLM Judgment Gaps

arXiv:2604.27345v1 Announce Type: new Abstract: Human annotators frequently disagree on emotion labels, yet most evaluations of Large Language Model (LLM) emotion annotation collapse these judgments i

researcharxiv-cs-cl

Mitigating Selection Bias in Large Language Models via Permutation-Aware GRPO

arXiv:2603.21016v2 Announce Type: replace-cross Abstract: Large language models (LLMs) used for multiple-choice and pairwise evaluation tasks often exhibit selection bias due to non-semantic factors l

PhotIQA: A photoacoustic image data set with image quality ratings

arXiv:2507.03478v2 Announce Type: replace-cross Abstract: Image quality assessment (IQA) is crucial in the evaluation stage of novel algorithms operating on images, including traditional and machine l

researcharxiv-cs-cv

Political Bias Audits of LLMs Capture Sycophancy to the Inferred Auditor

arXiv:2604.27633v1 Announce Type: new Abstract: Large language models (LLMs) are commonly evaluated for political bias based on their responses to fixed questionnaires, which typically place frontier

REBENCH: A Procedural, Fair-by-Construction Benchmark for LLMs on Stripped-Binary Types and Names (Extended Version)

arXiv:2604.27319v1 Announce Type: cross Abstract: Large Language Models (LLMs) have achieved remarkable progress in recent years, driving their adoption across a wide range of domains, including compu

RuC: HDL-Agnostic Rule Completion Benchmark Generation

arXiv:2604.27780v1 Announce Type: cross Abstract: Large Language Models (LLMs) have rapidly improved in performance across code-related tasks, making their integration into Register Transfer Level (RT

Sampler-Robust Optimization under Generative Models

arXiv:2604.27447v1 Announce Type: cross Abstract: Modern stochastic optimization pipelines increasingly rely on learned generative models to represent uncertainty, while downstream decisions are evalu

Skills-Coach: A Self-Evolving Skill Optimizer via Training-Free GRPO

arXiv:2604.27488v1 Announce Type: new Abstract: We introduce Skills-Coach, a novel automated framework designed to significantly enhance the self-evolution of skills within Large Language Model (LLM)-

SpecVQA: A Benchmark for Spectral Understanding and Visual Question Answering in Scientific Images

arXiv:2604.28039v1 Announce Type: new Abstract: Spectra are a prevalent yet highly information-dense form of scientific imagery, presenting substantial challenges to multimodal large language models (

WaferSAGE: Large Language Model-Powered Wafer Defect Analysis via Synthetic Data Generation and Rubric-Guided Reinforcement Learning

arXiv:2604.27629v1 Announce Type: new Abstract: We present WaferSAGE, a framework for wafer defect visual question answering using small vision-language models. To address data scarcity in semiconduct

When Roles Fail: Epistemic Constraints on Advocate Role Fidelity in LLM-Based Political Statement Analysis

arXiv:2604.27228v1 Announce Type: new Abstract: Democratic discourse analysis systems increasingly rely on multi-agent LLM pipelines in which distinct evaluator models are assigned adversarial roles t

ATLAS: An Annotation Tool for Long-horizon Robotic Action Segmentation

arXiv:2604.26637v1 Announce Type: cross Abstract: Annotating long-horizon robotic demonstrations with precise temporal action boundaries is crucial for training and evaluating action segmentation and

CurEvo: Curriculum-Guided Self-Evolution for Video Understanding

model-releasesarxiv-cs-cv

arXiv:2604.26707v1 Announce Type: new Abstract: Recent advances in self-evolution video understanding frameworks have demonstrated the potential of autonomous learning without human annotations. Howev

Electricity price forecasting across Norway's five bidding zones in the post-crisis era

arXiv:2604.26634v1 Announce Type: new Abstract: Norway's electricity market is heavily dominated by hydropower, but the 2021--2022 energy crisis and stronger integration with Continental Europe have f

From Prompt Risk to Response Risk: Paired Analysis of Safety Behavior of Large Language Model

arXiv:2604.26052v1 Announce Type: new Abstract: Safety evaluations of large language models (LLMs) typically report binary outcomes such as attack success rate, refusal rate, or harmful/not-harmful re

safetyarxiv-cs-cl

Information Extraction from Electricity Invoices with General-Purpose Large Language Models

arXiv:2604.25927v1 Announce Type: new Abstract: Information extraction from semi-structured business documents remains a critical challenge for enterprise management. This study evaluates the capabili

Preserving Disagreement: Architectural Heterogeneity and Coherence Validation in Multi-Agent Policy Simulation

arXiv:2604.26561v1 Announce Type: cross Abstract: Multi-agent deliberation systems using large language models (LLMs) are increasingly proposed for policy simulation, yet they suffer from artificial c

Rethinking the Harmonic Loss via Non-Euclidean Distance Layers

arXiv:2603.10225v3 Announce Type: replace-cross Abstract: Cross-entropy loss has long been the standard choice for training deep neural networks, yet it suffers from interpretability limitations, unbo

Local Ai

SAND: Spatially Adaptive Network Depth for Fast Sampling of Neural Implicit Surfaces

arXiv:2604.25936v1 Announce Type: cross Abstract: Implicit neural representations are powerful for geometric modeling, but their practical use is often limited by the high computational cost of networ

local-aiarxiv-cs-cv

SongBench: A Fine-Grained Multi-Aspect Benchmark for Song Quality Assessment

arXiv:2604.25937v1 Announce Type: cross Abstract: Recent advancements in Text-to-Song generation have enabled realistic musical content production, yet existing evaluation benchmarks lack the professi

The Fools are Certain; the Wise are Doubtful: Exploring LLM Confidence in Code Completion

arXiv:2508.16131v2 Announce Type: replace-cross Abstract: Code completion entails the task of providing missing tokens given a surrounding context. It can boost developer productivity while providing

Uncertainty-Aware Reward Discounting for Mitigating Reward Hacking

arXiv:2604.26360v1 Announce Type: cross Abstract: Reinforcement learning (RL) systems typically optimize scalar reward functions that assume precise and reliable evaluation of outcomes. However, real-

When Annotators Disagree, Topology Explains: Mapper, a Topological Tool for Exploring Text Embedding Geometry and Ambiguity

arXiv:2510.17548v2 Announce Type: replace Abstract: Language models are often evaluated with scalar metrics like accuracy, but such measures fail to capture how models internally represent ambiguity,

safetyarxiv-cs-cl

Agents

Agentic AI for Remote Sensing: Technical Challenges and Research Directions

arXiv:2604.24919v1 Announce Type: new Abstract: Earth Observation (EO) is moving beyond static prediction toward multi-step analytical workflows that require coordinated reasoning over data, tools, an

agentsarxiv-cs-cv

Agents

AIDOVECL: AI-generated Dataset of Outpainted Vehicles for Eye-level Classification and Localization

arXiv:2410.24116v3 Announce Type: replace Abstract: Image labeling is a critical bottleneck in the development of computer vision technologies, often constraining machine learning performance due to t

agentsarxiv-cs-cv

Below-Chance Blindness: Prompted Underperformance in Small LLMs Produces Positional Bias Rather than Answer Avoidance

arXiv:2604.25249v1 Announce Type: new Abstract: Detecting sandbagging--the deliberate underperformance on capability evaluations--is an open problem in AI safety. We tested whether symptom validity te

Carbon-Taxed Transformers: A Green Compression Pipeline for Overgrown Language Models

arXiv:2604.25903v1 Announce Type: cross Abstract: The accelerating adoption of Large Language Models (LLMs) in software engineering (SE) has brought with it a silent crisis: unsustainable computationa

safetyarxiv-cs-lg

Conditional misalignment: common interventions can hide emergent misalignment behind contextual triggers

arXiv:2604.25891v1 Announce Type: new Abstract: Finetuning a language model can lead to emergent misalignment (EM) [Betley et al., 2025b]. Models trained on a narrow distribution of misaligned behavio

safetyarxiv-cs-lg

CRAFT: Grounded Multi-Agent Coordination Under Partial Information

arXiv:2603.25268v2 Announce Type: replace Abstract: We introduce CRAFT, a multi-agent benchmark for evaluating pragmatic communication in large language models under strict partial information. In thi

Cutscene Agent: An LLM Agent Framework for Automated 3D Cutscene Generation

arXiv:2604.25318v1 Announce Type: cross Abstract: Cutscenes are carefully choreographed cinematic sequences embedded in video games and interactive media, serving as the primary vehicle for narrative

Digitizing Nepal's Written Heritage: A Comprehensive HTR Pipeline for Old Nepali Manuscripts

arXiv:2512.17111v2 Announce Type: replace Abstract: This paper presents the first end-to-end pipeline for Handwritten Text Recognition (HTR) for Old Nepali, a historically significant but low-resource

researcharxiv-cs-lg

EOS-Bench: A Comprehensive Benchmark for Earth Observation Satellite Scheduling

model-releasesarxiv-cs-ro

arXiv:2604.25782v1 Announce Type: cross Abstract: Earth observation satellite imaging scheduling is a challenging NP-hard combinatorial optimisation problem central to space mission operations. While

Frontier Coding Agents Can Now Implement an AlphaZero Self-Play Machine Learning Pipeline For Connect Four That Performs Comparably to an External Solver

arXiv:2604.25067v1 Announce Type: cross Abstract: Forecasting when AI systems will become capable of meaningfully accelerating AI research is a central challenge for AI safety. Existing benchmarks mea

Knowledge Distillation Must Account for What It Loses

arXiv:2604.25110v1 Announce Type: new Abstract: This position paper argues that knowledge distillation must account for what it loses: student models should be judged not only by retained task scores,

safetyarxiv-cs-lg

Local Ai

Luminol-AIDetect: Fast Zero-shot Machine-Generated Text Detection based on Perplexity under Text Shuffling

arXiv:2604.25860v1 Announce Type: new Abstract: Machine-generated text (MGT) detection requires identifying structurally invariant signals across generation models, rather than relying on model-specif

local-aiarxiv-cs-cl

MICo-150K: A Comprehensive Dataset Advancing Multi-Image Composition

model-releasesarxiv-cs-cv

arXiv:2512.07348v2 Announce Type: replace Abstract: In controllable image generation, synthesizing coherent and consistent images from multiple reference inputs, i.e., Multi-Image Composition (MICo),

NUBO: A Transparent Python Package for Bayesian Optimization

arXiv:2305.06709v4 Announce Type: replace Abstract: NUBO, short for Newcastle University Bayesian Optimisation, is a Bayesian optimization framework for the optimization of expensive-to-evaluate black

ProDrive: Proactive Planning for Autonomous Driving via Ego-Environment Co-Evolution

arXiv:2604.25329v1 Announce Type: new Abstract: End-to-end autonomous driving planners typically generate trajectories from current observations alone. However, real-world driving is highly dynamic, a

safetyarxiv-cs-ro

PSP: An Interpretable Per-Dimension Accent Benchmark for Indic Text-to-Speech

arXiv:2604.25476v1 Announce Type: cross Abstract: Standard text-to-speech (TTS) evaluation measures intelligibility (WER, CER) and overall naturalness (MOS, UTMOS) but does not quantify accent. A synt