Search: “evaluation”

The impact of postediting on AI generative translation in Yemeni context: Translating literary prose by ChatGPT

arXiv:2604.16704v1 Announce Type: new Abstract: This study examines the role of artificial intelligence in translation, focusing on ChatGPT, specifically ChatGPT-4, and the extent to which human poste

researcharxiv-cs-cl

21 Apr 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

ADAPT: Benchmarking Commonsense Planning under Unspecified Affordance Constraints

DGX agent

arXiv:2604.14902v1 Announce Type: cross Abstract: Intelligent embodied agents should not simply follow instructions, as real-world environments often involve unexpected conditions and exceptions. Howe

model-releasesarxiv-cs-cl

17 Apr 2026

CaptionQA: Is Your Caption as Useful as the Image Itself?

arXiv:2511.21025v2 Announce Type: replace Abstract: Image captions serve as efficient surrogates for visual content in multimodal systems such as retrieval, recommendation, and multi-step agentic infe

CoPA: Benchmarking Personalized Question Answering with Data-Informed Cognitive Factors

arXiv:2604.14773v1 Announce Type: new Abstract: While LLMs have demonstrated remarkable potential in Question Answering (QA), evaluating personalization remains a critical bottleneck. Existing paradig

MEBench: A Novel Benchmark for Understanding Mutual Exclusivity Bias in Vision-Language Models

arXiv:2505.20122v2 Announce Type: replace Abstract: This paper introduces MEBench, a novel benchmark for evaluating mutual exclusivity (ME) bias, a cognitive phenomenon observed in children during wor

ProRe: A Proactive Reward System for GUI Agents via Reasoner-Actor Collaboration

arXiv:2509.21823v2 Announce Type: replace Abstract: Reward is critical to the evaluation and training of large language models (LLMs). However, existing rule-based or model-based reward methods strugg

ReasonScaffold: A Scaffolded Reasoning-based Annotation Protocol for Human-AI Co-Annotation

arXiv:2603.21094v3 Announce Type: replace Abstract: Human annotation is central to NLP evaluation, yet subjective tasks often exhibit substantial variability across annotators. While large language mo

researcharxiv-cs-cl

RL-STPA: Adapting System-Theoretic Hazard Analysis for Safety-Critical Reinforcement Learning

arXiv:2604.15201v1 Announce Type: new Abstract: As reinforcement learning (RL) deployments expand into safety-critical domains, existing evaluation methods fail to systematically identify hazards aris

safetyarxiv-cs-lg

C2: Scalable Rubric-Augmented Reward Modeling from Binary Preferences

arXiv:2604.13618v1 Announce Type: new Abstract: Rubric-augmented verification guides reward models with explicit evaluation criteria, yielding more reliable judgments than single-model verification. H

researcharxiv-cs-cl

Chain of Uncertain Rewards with Large Language Models for Reinforcement Learning

arXiv:2604.13504v1 Announce Type: cross Abstract: Designing effective reward functions is a cornerstone of reinforcement learning (RL), yet it remains a challenging and labor-intensive process due to

Hybrid Retrieval for COVID-19 Literature: Comparing Rank Fusion and Projection Fusion with Diversity Reranking

arXiv:2604.13728v1 Announce Type: cross Abstract: We present a hybrid retrieval system for COVID-19 scientific literature, evaluated on the TREC-COVID benchmark (171,332 papers, 50 expert queries). Th

Synthetic Tabular Generators Fail to Preserve Behavioral Fraud Patterns: A Benchmark on Temporal, Velocity, and Multi-Account Signals

model-releasesarxiv-cs-lg

arXiv:2604.13125v1 Announce Type: new Abstract: We introduce behavioral fidelity -- a third evaluation dimension for synthetic tabular data that measures whether generated data preserves the temporal,

A Sanity Check on Composed Image Retrieval

arXiv:2604.12904v1 Announce Type: new Abstract: Composed Image Retrieval (CIR) aims to retrieve a target image based on a query composed of a reference image, and a relative caption that specifies the

Benchmarking Deflection and Hallucination in Large Vision-Language Models

arXiv:2604.12033v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) increasingly rely on retrieval to answer knowledge-intensive multimodal questions. Existing benchmarks overlook c

Beyond Perception Errors: Semantic Fixation in Large Vision-Language Models

arXiv:2604.12119v1 Announce Type: new Abstract: Large vision-language models (VLMs) often rely on familiar semantic priors, but existing evaluations do not cleanly separate perception failures from ru

Does Visual Token Pruning Improve Calibration? An Empirical Study on Confidence in MLLMs

arXiv:2604.12035v1 Announce Type: new Abstract: Visual token pruning is a widely used strategy for efficient inference in multimodal large language models (MLLMs), but existing work mainly evaluates i

researcharxiv-cs-cv

GlotOCR Bench: OCR Models Still Struggle Beyond a Handful of Unicode Scripts

arXiv:2604.12978v1 Announce Type: new Abstract: Optical character recognition (OCR) has advanced rapidly with the rise of vision-language models, yet evaluation has remained concentrated on a small cl

MAML-KT: Addressing Cold Start Problem in Knowledge Tracing for New Students via Few-Shot Model-Agnostic Meta Learning

arXiv:2603.00137v2 Announce Type: replace-cross Abstract: Knowledge tracing (KT) models are commonly evaluated by training on early interactions from all students and testing on later responses. While

researcharxiv-cs-ai

A Metamorphic Testing Perspective on Knowledge Distillation for Language Models of Code: Does the Student Deeply Mimic the Teacher?

arXiv:2511.05476v4 Announce Type: replace-cross Abstract: Transformer-based language models of code have achieved state-of-the-art performance across a wide range of software analytics tasks, but thei

researcharxiv-cs-lg

Beyond Compliance: A Resistance-Informed Motivation Reasoning Framework for Challenging Psychological Client Simulation

arXiv:2604.10507v1 Announce Type: new Abstract: Psychological client simulators have emerged as a scalable solution for training and evaluating counselor trainees and psychological LLMs. Yet existing

BlasBench: An Open Benchmark for Irish Speech Recognition

arXiv:2604.10736v1 Announce Type: new Abstract: No open Irish-specific benchmark compares end-user ASR systems under a shared Irish-aware evaluation protocol. To solve this, we release BlasBench, an o

Both Ends Count! Just How Good are LLM Agents at 'Text-to-Big SQL'?

arXiv:2602.21480v4 Announce Type: replace-cross Abstract: Text-to-SQL and Big Data are both extensively benchmarked fields, yet there is limited research that evaluates them jointly. In the real world

COMPOSITE-Stem

arXiv:2604.09836v1 Announce Type: new Abstract: AI agents hold growing promise for accelerating scientific discovery; yet, a lack of frontier evaluations hinders adoption into real workflows. Expert-w

Edu-MMBias: A Three-Tier Multimodal Benchmark for Auditing Social Bias in Vision-Language Models under Educational Contexts

arXiv:2604.10200v1 Announce Type: new Abstract: As Vision-Language Models (VLMs) become integral to educational decision-making, ensuring their fairness is paramount. However, current text-centric eva

EmbodiedGovBench: A Benchmark for Governance, Recovery, and Upgrade Safety in Embodied Agent Systems

arXiv:2604.11174v1 Announce Type: cross Abstract: Recent progress in embodied AI has produced a growing ecosystem of robot policies, foundation models, and modular runtimes. However, current evaluatio

GLEaN: A Text-to-image Bias Detection Approach for Public Comprehension

arXiv:2604.09923v1 Announce Type: new Abstract: Text-to-image (T2I) models, and their encoded biases, increasingly shape the visual media the public encounters. While researchers have produced a rich

Hodoscope: Unsupervised Monitoring for AI Misbehaviors

arXiv:2604.11072v1 Announce Type: new Abstract: Existing approaches to monitoring AI agents rely on supervised evaluation: human-written rules or LLM-based judges that check for known failure modes. H

Local Ai

Limited Perfect Monotonical Surrogates constructed using low-cost recursive linkage discovery with guaranteed output

arXiv:2604.11524v1 Announce Type: new Abstract: Surrogates provide a cheap solution evaluation and offer significant leverage for optimizing computationally expensive problems. Usually, surrogates onl

local-aiarxiv-cs-ai

M2-Verify: A Large-Scale Multidomain Benchmark for Checking Multimodal Claim Consistency

arXiv:2604.01306v2 Announce Type: replace Abstract: Evaluating scientific arguments requires assessing the strict consistency between a claim and its underlying multimodal evidence. However, existing

MERMAID: Memory-Enhanced Retrieval and Reasoning with Multi-Agent Iterative Knowledge Grounding for Veracity Assessment

arXiv:2601.22361v2 Announce Type: replace-cross Abstract: Assessing the veracity of online content has become increasingly critical. Large language models (LLMs) have recently enabled substantial prog

Optimization-Guided Diffusion for Interactive Scene Generation

arXiv:2512.07661v3 Announce Type: replace Abstract: Realistic and diverse multi-agent driving scenes are crucial for evaluating autonomous vehicles, but safety-critical events which are essential for

safetyarxiv-cs-cv

QFS-Composer: Query-focused summarization pipeline for less resourced languages

arXiv:2604.10687v1 Announce Type: new Abstract: Large language models (LLMs) demonstrate strong performance in text summarization, yet their effectiveness drops significantly across languages with res

safetyarxiv-cs-cl

Retrieval-Augmented Large Language Models for Evidence-Informed Guidance on Cannabidiol Use in Older Adults

arXiv:2604.09548v1 Announce Type: cross Abstract: Older adults commonly experience chronic conditions such as pain and sleep disturbances and may consider cannabidiol for symptom management. Safe use

Switch-JustDance: Benchmarking Whole Body Motion Tracking Controllers Using a Commercial Console Game

arXiv:2511.17925v3 Announce Type: replace-cross Abstract: Recent advances in whole-body robot control have enabled humanoid and legged robots to perform increasingly agile and coordinated motions. How

Trust Your Memory: Verifiable Control of Smart Homes through Reinforcement Learning with Multi-dimensional Rewards

arXiv:2604.10110v1 Announce Type: new Abstract: Large Language Models (LLMs) have become a key foundation for enabling personalized smart home experiences. While existing studies have explored how sma

VidAudio-Bench: Benchmarking V2A and VT2A Generation across Four Audio Categories

arXiv:2604.10542v1 Announce Type: cross Abstract: Video-to-Audio (V2A) generation is essential for immersive multimedia experiences, yet its evaluation remains underexplored. Existing benchmarks typic

WebForge: Breaking the Realism-Reproducibility-Scalability Trilemma in Browser Agent Benchmark

arXiv:2604.10988v1 Announce Type: new Abstract: Existing browser agent benchmarks face a fundamental trilemma: real-website benchmarks lack reproducibility due to content drift, controlled environment

AssemLM: Spatial Reasoning Multimodal Large Language Models for Robotic Assembly

model-releasesarxiv-cs-ro

arXiv:2604.08983v1 Announce Type: new Abstract: Spatial reasoning is a fundamental capability for embodied intelligence, especially for fine-grained manipulation tasks such as robotic assembly. While

DRBENCHER: Can Your Agent Identify the Entity, Retrieve Its Properties and Do the Math?

arXiv:2604.09251v1 Announce Type: new Abstract: Deep research agents increasingly interleave web browsing with multi-step computation, yet existing benchmarks evaluate these capabilities in isolation,

Facet-Level Tracing of Evidence Uncertainty and Hallucination in RAG

arXiv:2604.09174v1 Announce Type: new Abstract: Retrieval-Augmented Generation (RAG) aims to reduce hallucination by grounding answers in retrieved evidence, yet hallucinated answers remain common eve

Temporal Dropout Risk in Learning Analytics: A Harmonized Survival Benchmark Across Dynamic and Early-Window Representations

arXiv:2604.08870v1 Announce Type: cross Abstract: Student dropout is a persistent concern in Learning Analytics, yet comparative studies frequently evaluate predictive models under heterogeneous proto

Through Their Eyes: Fixation-aligned Tuning for Personalized User Emulation

arXiv:2604.09368v1 Announce Type: cross Abstract: Large language model (LLM) agents are increasingly deployed as scalable user simulators for recommender system evaluation. Yet existing simulators per

safetyarxiv-cs-cv

AgentOpt v0.1 Technical Report: Client-Side Optimization for LLM-Based Agent

arXiv:2604.06296v1 Announce Type: cross Abstract: AI agents are increasingly deployed in real-world applications, including systems such as Manus, OpenClaw, and coding agents. Existing research has pr

AgriPath: A Systematic Exploration of Architectural Trade-offs for Crop Disease Classification

model-releasesarxiv-cs-lg

arXiv:2603.13354v3 Announce Type: replace-cross Abstract: Reliable crop disease detection requires models that perform consistently across diverse acquisition conditions, yet existing evaluations ofte

An empirical study of LoRA-based fine-tuning of large language models for automated test case generation

arXiv:2604.06946v1 Announce Type: cross Abstract: Automated test case generation from natural language requirements remains a challenging problem in software engineering due to the ambiguity of requir

ASBench: Image Anomalies Synthesis Benchmark for Anomaly Detection

arXiv:2510.07927v2 Announce Type: replace Abstract: Anomaly detection plays a pivotal role in manufacturing quality control, yet its application is constrained by limited abnormal samples and high man

ClawBench: Can AI Agents Complete Everyday Online Tasks?

arXiv:2604.08523v1 Announce Type: new Abstract: AI agents may be able to automate your inbox, but can they automate other routine aspects of your life? Everyday online tasks offer a realistic yet unso

Contrastive Decoding Mitigates Score Range Bias in LLM-as-a-Judge

arXiv:2510.18196v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are commonly used as evaluators in various applications, but the reliability of the outcomes remains a challenge.