Search: “evaluation”

arXiv:2604.22679v1 Announce Type: cross Abstract: The increasing adoption of AI systems in hiring has raised concerns about algorithmic bias and accountability, prompting regulatory responses includin

local-aiarxiv-cs-ai

27 Apr 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

LLMs as Assessors: Right for the Right Reason?

DGX agent

arXiv:2601.08919v2 Announce Type: replace-cross Abstract: A good deal of recent research has focused on how Large Language Models (LLMs) may be used as judges in place of humans to evaluate the qualit

model-releasesarxiv-cs-cl

27 Apr 2026

Performance Anomaly Detection in Athletics: A Benchmarking System with Visual Analytics

arXiv:2604.21953v1 Announce Type: new Abstract: Anti-doping programs rely on biological testing to detect performance-enhancing drugs, but such testing costs over $800 per sample and is limited by sho

27 Apr 2026

A Metamorphic Testing Approach to Diagnosing Memorization in LLM-Based Program Repair

arXiv:2604.21579v1 Announce Type: cross Abstract: LLM-based automated program repair (APR) techniques have shown promising results in reducing debugging costs. However, prior results can be affected b

Can MLLMs 'Read' What is Missing?

arXiv:2604.21277v1 Announce Type: new Abstract: We introduce MMTR-Bench, a benchmark designed to evaluate the intrinsic ability of Multimodal Large Language Models (MLLMs) to reconstruct masked text d

Co-Evolving LLM Decision and Skill Bank Agents for Long-Horizon Tasks

arXiv:2604.20987v1 Announce Type: new Abstract: Long horizon interactive environments are a testbed for evaluating agents skill usage abilities. These environments demand multi step reasoning, the cha

From If-Statements to ML Pipelines: Revisiting Bias in Code-Generation

arXiv:2604.21716v1 Announce Type: new Abstract: Prior work evaluates code generation bias primarily through simple conditional statements, which represent only a narrow slice of real-world programming

Hi-WM: Human-in-the-World-Model for Scalable Robot Post-Training

arXiv:2604.21741v1 Announce Type: new Abstract: Post-training is essential for turning pretrained generalist robot policies into reliable task-specific controllers, but existing human-in-the-loop pipe

safetyarxiv-cs-ro

How VLAs (Really) Work In Open-World Environments

arXiv:2604.21192v1 Announce Type: cross Abstract: Vision-language-action models (VLAs) have been extensively used in robotics applications, achieving great success in various manipulation problems. Mo

Improving Performance in Classification Tasks with LCEN and the Weighted Focal Differentiable MCC Loss

arXiv:2604.21252v1 Announce Type: new Abstract: The LASSO-Clip-EN (LCEN) algorithm was previously introduced for nonlinear, interpretable feature selection and machine learning. However, its design an

Applications

Integrated packing, placement, scheduling, and routing of personalized production: a pharmaceutical Industry 4.0 use-case with a planar transport system

arXiv:2604.21029v1 Announce Type: cross Abstract: The recent emergence of planar transport systems necessitates re-evaluation of Flexible Manufacturing Systems (FMS) to address the simultaneous schedu

applicationsarxiv-cs-ai

Speculative Actions: A Lossless Framework for Faster Agentic Systems

arXiv:2510.04371v2 Announce Type: replace Abstract: AI agents are increasingly deployed in complex, interactive environments, yet their runtime remains a major bottleneck for training, evaluation, and

The Effect of Idea Elaboration on the Automatic Assessment of Idea Originality

arXiv:2604.20569v1 Announce Type: cross Abstract: Automatic systems are increasingly used to assess the originality of responses in creative tasks. They offer a potential solution to key limitations o

The Last Harness You'll Ever Build

arXiv:2604.21003v1 Announce Type: new Abstract: AI agents are increasingly deployed on complex, domain-specific workflows -- navigating enterprise web applications that require dozens of clicks and fo

TraceScope: Interactive URL Triage via Decoupled Checklist Adjudication

arXiv:2604.21840v1 Announce Type: cross Abstract: Modern phishing campaigns increasingly evade snapshot-based URL classifiers using interaction gates (e.g., checkbox/slider challenges), delayed conten

VidHal: Benchmarking Temporal Hallucinations in Vision LLMs

model-releasesarxiv-cs-cv

arXiv:2411.16771v3 Announce Type: replace Abstract: Vision Large Language Models (VLLMs) are widely acknowledged to be prone to hallucinations. Existing research addressing this problem has primarily

Why are all LLMs Obsessed with Japanese Culture? On the Hidden Cultural and Regional Biases of LLMs

arXiv:2604.21751v1 Announce Type: cross Abstract: LLMs have been showing limitations when it comes to cultural coverage and competence, and in some cases show regional biases such as amplifying Wester

Accelerating PayPal's Commerce Agent with Speculative Decoding: An Empirical Study on EAGLE3 with Fine-Tuned Nemotron Models

arXiv:2604.19767v1 Announce Type: cross Abstract: We evaluate speculative decoding with EAGLE3 as an inference-time optimization for PayPal's Commerce Agent, powered by a fine-tuned llama3.1-nemotron-

AstaBench: Rigorous Benchmarking of AI Agents with a Scientific Research Suite

arXiv:2510.21652v2 Announce Type: replace Abstract: AI agents hold the potential to revolutionize scientific productivity by automating literature reviews, replicating experiments, analyzing data, and

Can 'AI' Be a Doctor? A Study of Empathy, Readability, and Alignment in Clinical LLMs

arXiv:2604.20791v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly deployed in healthcare, yet their communicative alignment with clinical standards remains insufficiently

Applications

CHASM: Unveiling Covert Advertisements on Chinese Social Media

arXiv:2604.20511v1 Announce Type: cross Abstract: Current benchmarks for evaluating large language models (LLMs) in social media moderation completely overlook a serious threat: covert advertisements,

applicationsarxiv-cs-ai

Efficient Multi-Cohort Inference for Long-Term Effects and Lifetime Value in A/B Testing with User Learning

arXiv:2604.20777v1 Announce Type: new Abstract: In streaming platforms churn is extremely costly, yet A/B tests are typically evaluated using outcomes observed within a limited experimental horizon. E

EvoForest: A Novel Machine-Learning Paradigm via Open-Ended Evolution of Computational Graphs

arXiv:2604.19761v1 Announce Type: new Abstract: Modern machine learning is still largely organized around a single recipe: choose a parameterized model family and optimize its weights. Although highly

Forage V2: Knowledge Evolution and Transfer in Autonomous Agent Organizations

arXiv:2604.19837v1 Announce Type: new Abstract: Autonomous agents operating in open-world tasks -- where the completion boundary is not given in advance -- face denominator blindness: they systematica

HiPO: Hierarchical Preference Optimization for Adaptive Reasoning in LLMs

arXiv:2604.20140v1 Announce Type: new Abstract: Direct Preference Optimization (DPO) is an effective framework for aligning large language models with human preferences, but it struggles with complex

MOA: Multi-Objective Alignment for Role-Playing Agents

arXiv:2512.09756v2 Announce Type: replace Abstract: Role-playing agents (RPAs) require balancing multiple objectives, such as instruction following, persona consistency, and stylistic fidelity, which

Phase 1 Implementation of LLM-generated Discharge Summaries showing high Adoption in a Dutch Academic Hospital

arXiv:2604.19774v1 Announce Type: cross Abstract: Writing discharge summaries to transfer medical information is an important but time-consuming process that can be assisted by Large Language Models (

Same Content, Different Answers: Cross-Modal Inconsistency in MLLMs

arXiv:2512.08923v2 Announce Type: replace Abstract: We introduce two new benchmarks REST and REST+ (Render-Equivalence Stress Tests) to enable systematic evaluation of cross-modal inconsistency in mul

Stabilising Generative Models of Attitude Change

arXiv:2604.19791v1 Announce Type: new Abstract: Attitude change - the process by which individuals revise their evaluative stances - has been explained by a set of influential but competing verbal the

Tokenised Flow Matching for Hierarchical Simulation Based Inference

arXiv:2604.20723v1 Announce Type: cross Abstract: The cost of simulator evaluations is a key practical bottleneck for Simulation Based Inference (SBI). In hierarchical settings with shared global para

Towards Certified Malware Detection: Provable Guarantees Against Evasion Attacks

arXiv:2604.20495v1 Announce Type: cross Abstract: Machine learning-based static malware detectors remain vulnerable to adversarial evasion techniques, such as metamorphic engine mutations. To address

Benchmarking Physics-Informed Neural Networks and Boundary Elements Methods for Wave Scattering

arXiv:2509.12483v4 Announce Type: replace Abstract: This study compares the Boundary Element Method (BEM) and Physics-Informed Neural Networks (PINNs) for solving the two-dimensional Helmholtz equatio

Beyond Itinerary Planning-A Real-World Benchmark for Multi-Turn and Tool-Using Travel Tasks

arXiv:2512.22673v3 Announce Type: replace Abstract: Travel planning is a natural real-world task to test large language models' (LLMs) planning and tool-use abilities. Although prior work has studied

Beyond One Output: Visualizing and Comparing Distributions of Language Model Generations

arXiv:2604.18724v1 Announce Type: new Abstract: Users typically interact with and evaluate language models via single outputs, but each output is just one sample from a broad distribution of possible

Capturing Classic Authorial Style in Long-Form Story Generation with GRPO Fine-Tuning

arXiv:2512.05747v3 Announce Type: replace Abstract: Evaluating and optimising authorial style in long-form story generation remains challenging because style is often assessed with ad hoc prompting an

Fairness Audits of Institutional Risk Models in Deployed ML Pipelines

arXiv:2604.19468v1 Announce Type: cross Abstract: Fairness audits of institutional risk models are critical for understanding how deployed machine learning pipelines allocate resources. Drawing on mul

From Experience to Skill: Multi-Agent Generative Engine Optimization via Reusable Strategy Learning

arXiv:2604.19516v1 Announce Type: new Abstract: Generative engines (GEs) are reshaping information access by replacing ranked links with citation-grounded answers, yet current Generative Engine Optimi

Mind2Drive: Predicting Driver Intentions from EEG in Real-world On-Road Driving

arXiv:2604.19368v1 Announce Type: new Abstract: Predicting driver intention from neurophysiological signals offers a promising pathway for enhancing proactive safety in advanced driver assistance syst

safetyarxiv-cs-cv

Mitigating Judgment Preference Bias in Large Language Models through Group-Based Polling

arXiv:2510.08145v2 Announce Type: replace Abstract: Large Language Models (LLMs) as automatic evaluators, commonly referred to as LLM-as-a-Judge, have also attracted growing attention. This approach p

ReflectMT: Internalizing Reflection for Efficient and High-Quality Machine Translation

arXiv:2604.19144v1 Announce Type: new Abstract: Recent years have witnessed growing interest in applying Large Reasoning Models (LRMs) to Machine Translation (MT). Existing approaches predominantly ad

RIFT: A RubrIc Failure Mode Taxonomy and Automated Diagnostics

arXiv:2604.01375v2 Announce Type: replace Abstract: Rubric-based evaluation is widely used in LLM benchmarks and training pipelines for open-ended, less verifiable tasks. While prior work has demonstr

When and What to Ask: AskBench and Rubric-Guided RLVR for LLM Clarification

arXiv:2602.11199v2 Announce Type: replace Abstract: Large language models (LLMs) often respond even when prompts omit critical details or include misleading information, leading to hallucinations or r

When Safety Fails Before the Answer: Benchmarking Harmful Behavior Detection in Reasoning Chains

arXiv:2604.19001v1 Announce Type: new Abstract: Large reasoning models (LRMs) produce complex, multi-step reasoning traces, yet safety evaluation remains focused on final outputs, overlooking how harm

Advancing MAPF Toward the Real World: A Scalable Multi-Agent Realistic Testbed (SMART)

arXiv:2503.04798v3 Announce Type: replace Abstract: We present Scalable Multi-Agent Realistic Testbed (SMART), a realistic and efficient software tool for evaluating Multi-Agent Path Finding (MAPF) al

agentsarxiv-cs-ro

Local Ai

Aligning Language Models with Real-time Knowledge Editing

arXiv:2508.01302v3 Announce Type: replace Abstract: Knowledge editing aims to modify outdated knowledge in language models efficiently while retaining their original capabilities. Mainstream datasets

local-aiarxiv-cs-cl

Audio-DeepThinker: Progressive Reasoning-Aware Reinforcement Learning for High-Quality Chain-of-Thought Emergence in Audio Language Models

arXiv:2604.18187v1 Announce Type: cross Abstract: Large Audio-Language Models (LALMs) have made significant progress in audio understanding, yet they primarily operate as perception-and-answer systems

Auditing Support Strategies in LLMs through Grounded Multi-Turn Social Simulation

arXiv:2604.17079v1 Announce Type: new Abstract: When users seek social support from chatbots, they disclose their situation gradually, yet most evaluations of supportive LLMs rely on single-turn, full

Benchmarking Real-Time Question Answering via Executable Code Workflows

arXiv:2604.16349v1 Announce Type: cross Abstract: Retrieving real-time information is a fundamental capability for search-integrated agents in real-world applications. However, existing benchmarks are