Search: “evaluation”

HalalBench: A Multilingual OCR Benchmark for Food Packaging Ingredient Extraction

arXiv:2604.22754v1 Announce Type: cross Abstract: No standardized benchmark exists for evaluating OCR on food packaging, despite its critical role in automated halal food verification. Existing benchm

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

In-Sync: Adaptation of Speech Aware Large Language Models for ASR with Word Level Timestamp Predictions

DGX agent

arXiv:2604.22817v1 Announce Type: cross Abstract: Recent advances in speech-aware language models have coupled strong acoustic encoders with large language models, enabling systems that move beyond tr

safetyarxiv-cs-cl

28 Apr 2026

JudgeSense: A Benchmark for Prompt Sensitivity in LLM-as-a-Judge Systems

arXiv:2604.23478v1 Announce Type: new Abstract: Large language models are increasingly deployed as automated judges for evaluating other models, yet the stability of their verdicts under semantically

Large Language Model based Interactive Decision-Making for Autonomous Driving

arXiv:2604.23513v1 Announce Type: new Abstract: In high-conflict mixed-traffic scenarios involving human-driven and autonomous vehicles, most existing autonomous driving systems default to overly cons

safetyarxiv-cs-ro

LLM4SCREENLIT: Recommendations on Assessing the Performance of Large Language Models for Screening Literature in Systematic Reviews

arXiv:2511.12635v2 Announce Type: replace-cross Abstract: Context: Large language models (LLMs) are increasingly used to screen literature for systematic reviews (SRs), but the standard confusion-matr

Point Cloud Registration for Fusion between SPECT MPI and CTA Images

arXiv:2604.24524v1 Announce Type: new Abstract: Clinical fusion of Single Photon Emission Computed Tomography Myocardial Perfusion Imaging (SPECT MPI) and Computed Tomography Angiography (CTA) remains

researcharxiv-cs-cv

Probing CLIP's Comprehension of 360-Degree Textual and Visual Semantics

arXiv:2604.24642v1 Announce Type: new Abstract: The dream of instantly creating rich 360-degree panoramic worlds from text is rapidly becoming a reality, yet a crucial gap exists in our ability to rel

safetyarxiv-cs-cv

Self-Supervised Learning for Android Malware Detection on a Time-Stamped Dataset

arXiv:2604.23025v1 Announce Type: cross Abstract: Android malware detectors built with machine learning often suffer from temporal bias: models are trained and evaluated without respecting apps' actua

safetyarxiv-cs-lg

SwissGov-RSD: A Human-annotated, Cross-lingual Benchmark for Token-level Recognition of Semantic Differences Between Related Documents

arXiv:2512.07538v3 Announce Type: replace Abstract: Recognizing semantic differences across documents is crucial for text generation evaluation and content alignment, especially in cross-lingual setti

The Alignment Target Problem: Divergent Moral Judgments of Humans, AI Systems, and Their Designers

arXiv:2604.24155v1 Announce Type: cross Abstract: The quest to align machine behavior with human values raises fundamental questions about the moral frameworks that should govern AI decision-making. M

The Collapse of Heterogeneity in Silicon Philosophers

arXiv:2604.23575v1 Announce Type: cross Abstract: Silicon samples are increasingly used as a low-cost substitute for human panels and have been shown to reproduce aggregate human opinion with high fid

safetyarxiv-cs-cl

The Price of Agreement: Measuring LLM Sycophancy in Agentic Financial Applications

arXiv:2604.24668v1 Announce Type: new Abstract: Given the increased use of LLMs in financial systems today, it becomes important to evaluate the safety and robustness of such systems. One failure mode

Zero-shot Large Language Models for Automatic Readability Assessment

arXiv:2604.24470v1 Announce Type: new Abstract: Unsupervised automatic readability assessment (ARA) methods have important practical and research applications (e.g., ensuring medical or educational ma

researcharxiv-cs-cl

A Kinematic Analysis of Palm Degrees of Freedom for Enhancing Thumb Opposability in Robotic Hands

arXiv:2604.22283v1 Announce Type: new Abstract: This study investigates the kinematic role of palm degrees of freedom (DoF) in enhancing thumb opposability in a five-finger robotic hand. A hand model

researcharxiv-cs-ro

An Artifact-based Agent Framework for Adaptive and Reproducible Medical Image Processing

arXiv:2604.21936v1 Announce Type: new Abstract: Medical imaging research is increasingly shifting from controlled benchmark evaluation toward real-world clinical deployment. In such settings, applying

Assessing the impact of dimensionality reduction on clustering performance -- a systematic study

arXiv:2604.22099v1 Announce Type: new Abstract: Dimensionality reduction is a critical preprocessing step for clustering high-dimensional data, yet comprehensive evaluation of its impact across divers

Can Large Language Models Adequately Perform Symbolic Reasoning Over Time Series?

arXiv:2508.03963v4 Announce Type: replace Abstract: Uncovering hidden symbolic laws from time series data, as an aspiration dating back to Kepler's discovery of planetary motion, remains a core challe

Don't try to build a self-improving AI agent without evals. You are just wasting time and compute. An agent can't improve from traces it can…

Don't try to build a self-improving AI agent without evals. You are just wasting time and compute. An agent can't improve from traces it can't evaluate. This is why it's exciting to see @FutureAGI_ go

safetydair-ai--x

Local Ai

How Supply Chain Dependencies Complicate Bias Measurement and Accountability Attribution in AI Hiring Applications

arXiv:2604.22679v1 Announce Type: cross Abstract: The increasing adoption of AI systems in hiring has raised concerns about algorithmic bias and accountability, prompting regulatory responses includin

local-aiarxiv-cs-ai

LLMs as Assessors: Right for the Right Reason?

arXiv:2601.08919v2 Announce Type: replace-cross Abstract: A good deal of recent research has focused on how Large Language Models (LLMs) may be used as judges in place of humans to evaluate the qualit

Performance Anomaly Detection in Athletics: A Benchmarking System with Visual Analytics

arXiv:2604.21953v1 Announce Type: new Abstract: Anti-doping programs rely on biological testing to detect performance-enhancing drugs, but such testing costs over $800 per sample and is limited by sho

260 things we announced at Google Cloud Next '26 – a recap

model-releasesgoogle-cloud-ai

Google Cloud Next ‘26 took place this week in Las Vegas, and the energy was incredible as we welcomed over 32,000 leaders, developers, and partners to explore the Agentic Era with us. Across three key

A Metamorphic Testing Approach to Diagnosing Memorization in LLM-Based Program Repair

arXiv:2604.21579v1 Announce Type: cross Abstract: LLM-based automated program repair (APR) techniques have shown promising results in reducing debugging costs. However, prior results can be affected b

Can MLLMs 'Read' What is Missing?

arXiv:2604.21277v1 Announce Type: new Abstract: We introduce MMTR-Bench, a benchmark designed to evaluate the intrinsic ability of Multimodal Large Language Models (MLLMs) to reconstruct masked text d

Co-Evolving LLM Decision and Skill Bank Agents for Long-Horizon Tasks

arXiv:2604.20987v1 Announce Type: new Abstract: Long horizon interactive environments are a testbed for evaluating agents skill usage abilities. These environments demand multi step reasoning, the cha

From If-Statements to ML Pipelines: Revisiting Bias in Code-Generation

arXiv:2604.21716v1 Announce Type: new Abstract: Prior work evaluates code generation bias primarily through simple conditional statements, which represent only a narrow slice of real-world programming

safetyarxiv-cs-cl

Hi-WM: Human-in-the-World-Model for Scalable Robot Post-Training

arXiv:2604.21741v1 Announce Type: new Abstract: Post-training is essential for turning pretrained generalist robot policies into reliable task-specific controllers, but existing human-in-the-loop pipe

safetyarxiv-cs-ro

How VLAs (Really) Work In Open-World Environments

arXiv:2604.21192v1 Announce Type: cross Abstract: Vision-language-action models (VLAs) have been extensively used in robotics applications, achieving great success in various manipulation problems. Mo

Improving Performance in Classification Tasks with LCEN and the Weighted Focal Differentiable MCC Loss

arXiv:2604.21252v1 Announce Type: new Abstract: The LASSO-Clip-EN (LCEN) algorithm was previously introduced for nonlinear, interpretable feature selection and machine learning. However, its design an

Applications

Integrated packing, placement, scheduling, and routing of personalized production: a pharmaceutical Industry 4.0 use-case with a planar transport system

arXiv:2604.21029v1 Announce Type: cross Abstract: The recent emergence of planar transport systems necessitates re-evaluation of Flexible Manufacturing Systems (FMS) to address the simultaneous schedu

applicationsarxiv-cs-ai

Speculative Actions: A Lossless Framework for Faster Agentic Systems

arXiv:2510.04371v2 Announce Type: replace Abstract: AI agents are increasingly deployed in complex, interactive environments, yet their runtime remains a major bottleneck for training, evaluation, and

The Effect of Idea Elaboration on the Automatic Assessment of Idea Originality

arXiv:2604.20569v1 Announce Type: cross Abstract: Automatic systems are increasingly used to assess the originality of responses in creative tasks. They offer a potential solution to key limitations o

The Last Harness You'll Ever Build

arXiv:2604.21003v1 Announce Type: new Abstract: AI agents are increasingly deployed on complex, domain-specific workflows -- navigating enterprise web applications that require dozens of clicks and fo

TraceScope: Interactive URL Triage via Decoupled Checklist Adjudication

arXiv:2604.21840v1 Announce Type: cross Abstract: Modern phishing campaigns increasingly evade snapshot-based URL classifiers using interaction gates (e.g., checkbox/slider challenges), delayed conten

VidHal: Benchmarking Temporal Hallucinations in Vision LLMs

model-releasesarxiv-cs-cv

arXiv:2411.16771v3 Announce Type: replace Abstract: Vision Large Language Models (VLLMs) are widely acknowledged to be prone to hallucinations. Existing research addressing this problem has primarily

Why are all LLMs Obsessed with Japanese Culture? On the Hidden Cultural and Regional Biases of LLMs

arXiv:2604.21751v1 Announce Type: cross Abstract: LLMs have been showing limitations when it comes to cultural coverage and competence, and in some cases show regional biases such as amplifying Wester

Accelerating PayPal's Commerce Agent with Speculative Decoding: An Empirical Study on EAGLE3 with Fine-Tuned Nemotron Models

arXiv:2604.19767v1 Announce Type: cross Abstract: We evaluate speculative decoding with EAGLE3 as an inference-time optimization for PayPal's Commerce Agent, powered by a fine-tuned llama3.1-nemotron-

AstaBench: Rigorous Benchmarking of AI Agents with a Scientific Research Suite

arXiv:2510.21652v2 Announce Type: replace Abstract: AI agents hold the potential to revolutionize scientific productivity by automating literature reviews, replicating experiments, analyzing data, and

Can 'AI' Be a Doctor? A Study of Empathy, Readability, and Alignment in Clinical LLMs

arXiv:2604.20791v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly deployed in healthcare, yet their communicative alignment with clinical standards remains insufficiently

Applications

CHASM: Unveiling Covert Advertisements on Chinese Social Media

arXiv:2604.20511v1 Announce Type: cross Abstract: Current benchmarks for evaluating large language models (LLMs) in social media moderation completely overlook a serious threat: covert advertisements,

applicationsarxiv-cs-ai

Efficient Multi-Cohort Inference for Long-Term Effects and Lifetime Value in A/B Testing with User Learning

arXiv:2604.20777v1 Announce Type: new Abstract: In streaming platforms churn is extremely costly, yet A/B tests are typically evaluated using outcomes observed within a limited experimental horizon. E

EvoForest: A Novel Machine-Learning Paradigm via Open-Ended Evolution of Computational Graphs

arXiv:2604.19761v1 Announce Type: new Abstract: Modern machine learning is still largely organized around a single recipe: choose a parameterized model family and optimize its weights. Although highly

Forage V2: Knowledge Evolution and Transfer in Autonomous Agent Organizations

arXiv:2604.19837v1 Announce Type: new Abstract: Autonomous agents operating in open-world tasks -- where the completion boundary is not given in advance -- face denominator blindness: they systematica

HiPO: Hierarchical Preference Optimization for Adaptive Reasoning in LLMs

arXiv:2604.20140v1 Announce Type: new Abstract: Direct Preference Optimization (DPO) is an effective framework for aligning large language models with human preferences, but it struggles with complex

MOA: Multi-Objective Alignment for Role-Playing Agents

arXiv:2512.09756v2 Announce Type: replace Abstract: Role-playing agents (RPAs) require balancing multiple objectives, such as instruction following, persona consistency, and stylistic fidelity, which

safetyarxiv-cs-cl

Phase 1 Implementation of LLM-generated Discharge Summaries showing high Adoption in a Dutch Academic Hospital

arXiv:2604.19774v1 Announce Type: cross Abstract: Writing discharge summaries to transfer medical information is an important but time-consuming process that can be assisted by Large Language Models (

researcharxiv-cs-ai

Same Content, Different Answers: Cross-Modal Inconsistency in MLLMs

arXiv:2512.08923v2 Announce Type: replace Abstract: We introduce two new benchmarks REST and REST+ (Render-Equivalence Stress Tests) to enable systematic evaluation of cross-modal inconsistency in mul

researcharxiv-cs-ai

Stabilising Generative Models of Attitude Change

arXiv:2604.19791v1 Announce Type: new Abstract: Attitude change - the process by which individuals revise their evaluative stances - has been explained by a set of influential but competing verbal the

researcharxiv-cs-ai