Search: “evaluation”

Capturing Classic Authorial Style in Long-Form Story Generation with GRPO Fine-Tuning

arXiv:2512.05747v3 Announce Type: replace Abstract: Evaluating and optimising authorial style in long-form story generation remains challenging because style is often assessed with ad hoc prompting an

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

Fairness Audits of Institutional Risk Models in Deployed ML Pipelines

DGX agent

arXiv:2604.19468v1 Announce Type: cross Abstract: Fairness audits of institutional risk models are critical for understanding how deployed machine learning pipelines allocate resources. Drawing on mul

safetyarxiv-cs-ai

22 Apr 2026

From Experience to Skill: Multi-Agent Generative Engine Optimization via Reusable Strategy Learning

arXiv:2604.19516v1 Announce Type: new Abstract: Generative engines (GEs) are reshaping information access by replacing ranked links with citation-grounded answers, yet current Generative Engine Optimi

Mind2Drive: Predicting Driver Intentions from EEG in Real-world On-Road Driving

arXiv:2604.19368v1 Announce Type: new Abstract: Predicting driver intention from neurophysiological signals offers a promising pathway for enhancing proactive safety in advanced driver assistance syst

safetyarxiv-cs-cv

Mitigating Judgment Preference Bias in Large Language Models through Group-Based Polling

arXiv:2510.08145v2 Announce Type: replace Abstract: Large Language Models (LLMs) as automatic evaluators, commonly referred to as LLM-as-a-Judge, have also attracted growing attention. This approach p

ReflectMT: Internalizing Reflection for Efficient and High-Quality Machine Translation

arXiv:2604.19144v1 Announce Type: new Abstract: Recent years have witnessed growing interest in applying Large Reasoning Models (LRMs) to Machine Translation (MT). Existing approaches predominantly ad

RIFT: A RubrIc Failure Mode Taxonomy and Automated Diagnostics

arXiv:2604.01375v2 Announce Type: replace Abstract: Rubric-based evaluation is widely used in LLM benchmarks and training pipelines for open-ended, less verifiable tasks. While prior work has demonstr

researcharxiv-cs-ai

When and What to Ask: AskBench and Rubric-Guided RLVR for LLM Clarification

arXiv:2602.11199v2 Announce Type: replace Abstract: Large language models (LLMs) often respond even when prompts omit critical details or include misleading information, leading to hallucinations or r

When Safety Fails Before the Answer: Benchmarking Harmful Behavior Detection in Reasoning Chains

arXiv:2604.19001v1 Announce Type: new Abstract: Large reasoning models (LRMs) produce complex, multi-step reasoning traces, yet safety evaluation remains focused on final outputs, overlooking how harm

Agents

Advancing MAPF Toward the Real World: A Scalable Multi-Agent Realistic Testbed (SMART)

arXiv:2503.04798v3 Announce Type: replace Abstract: We present Scalable Multi-Agent Realistic Testbed (SMART), a realistic and efficient software tool for evaluating Multi-Agent Path Finding (MAPF) al

agentsarxiv-cs-ro

Local Ai

Aligning Language Models with Real-time Knowledge Editing

arXiv:2508.01302v3 Announce Type: replace Abstract: Knowledge editing aims to modify outdated knowledge in language models efficiently while retaining their original capabilities. Mainstream datasets

local-aiarxiv-cs-cl

Audio-DeepThinker: Progressive Reasoning-Aware Reinforcement Learning for High-Quality Chain-of-Thought Emergence in Audio Language Models

arXiv:2604.18187v1 Announce Type: cross Abstract: Large Audio-Language Models (LALMs) have made significant progress in audio understanding, yet they primarily operate as perception-and-answer systems

Auditing Support Strategies in LLMs through Grounded Multi-Turn Social Simulation

arXiv:2604.17079v1 Announce Type: new Abstract: When users seek social support from chatbots, they disclose their situation gradually, yet most evaluations of supportive LLMs rely on single-turn, full

Benchmarking Real-Time Question Answering via Executable Code Workflows

arXiv:2604.16349v1 Announce Type: cross Abstract: Retrieving real-time information is a fundamental capability for search-integrated agents in real-world applications. However, existing benchmarks are

BhashaSutra: A Task-Centric Unified Survey of Indian NLP Datasets, Corpora, and Resources

arXiv:2604.18423v1 Announce Type: new Abstract: India's linguistic landscape, spanning 22 scheduled languages and hundreds of marginalized dialects, has driven rapid growth in NLP datasets, benchmarks

BIASEDTALES-ML: A Multilingual Dataset for Analyzing Narrative Attribute Distributions in LLM-Generated Stories

arXiv:2604.17008v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly used to generate narrative content, including children's stories, which play an important role in social a

E3VS-Bench: A Benchmark for Viewpoint-Dependent Active Perception in 3D Gaussian Splatting Scenes

model-releasesarxiv-cs-cv

arXiv:2604.17969v1 Announce Type: new Abstract: Visual search in 3D environments requires embodied agents to actively explore their surroundings and acquire task-relevant evidence. However, existing v

How Robustly do LLMs Understand Execution Semantics?

model-releasesarxiv-cs-lg

arXiv:2604.16320v1 Announce Type: cross Abstract: LLMs demonstrate remarkable reasoning capabilities, yet whether they utilize internal world models or rely on sophisticated pattern matching remains o

IceBreaker for Conversational Agents: Breaking the First-Message Barrier with Personalized Starters

arXiv:2604.18375v1 Announce Type: new Abstract: Conversational agents, such as ChatGPT and Doubao, have become essential daily assistants for billions of users. To further enhance engagement, these sy

LiveFact: A Dynamic, Time-Aware Benchmark for LLM-Driven Fake News Detection

arXiv:2604.04815v2 Announce Type: replace Abstract: The rapid development of Large Language Models (LLMs) has transformed fake news detection and fact-checking tasks from simple classification to comp

MedProbeBench: Systematic Benchmarking at Deep Evidence Integration for Expert-level Medical Guideline

model-releasesarxiv-cs-cv

arXiv:2604.18418v1 Announce Type: new Abstract: Recent advances in deep research systems enable large language models to retrieve, synthesize, and reason over large-scale external knowledge. In medici

Model in Distress: Sentiment Analysis on French Synthetic Social Media

arXiv:2604.18226v1 Announce Type: new Abstract: Automated analysis of customer feedback on social media is hindered by three challenges: the high cost of annotated training data, the scarcity of evalu

PRISM: Probing Reasoning, Instruction, and Source Memory in LLM Hallucinations

arXiv:2604.16909v1 Announce Type: new Abstract: As large language models (LLMs) evolve from conversational assistants into agents capable of handling complex tasks, they are increasingly deployed in h

Tools

QIMMA قِمّة ⛰: A Quality-First Arabic LLM Leaderboard

QIMMA (قِمّة) is a quality-focused leaderboard for evaluating Arabic Large Language Models, developed by TIIUAE. The leaderboard benchmarks Arabic LLMs across various performance metrics and tasks to

toolshugging-face

QU-NLP at QIAS 2026: Multi-Stage QLoRA Fine-Tuning for Arabic Islamic Inheritance Reasoning

arXiv:2604.16396v1 Announce Type: new Abstract: Islamic inheritance law (ilm al-mawar{i}th) presents a challenging domain for evaluating large language models' structured reasoning capabilities, requi

Reference-state System Reliability method for scalable uncertainty quantification of coherent systems

arXiv:2604.17066v1 Announce Type: new Abstract: Coherent systems are representative of many practical applications, ranging from infrastructure networks to supply chains. Probabilistic evaluation of s

researcharxiv-cs-lg

Revisiting a Pain in the Neck: A Semantic Reasoning Benchmark for Language Models

arXiv:2604.16593v1 Announce Type: new Abstract: We present SemanticQA, an evaluation suite designed to assess language models (LMs) in semantic phrase processing tasks. The benchmark consolidates exis

SafeVLA: Towards Safety Alignment of Vision-Language-Action Model via Constrained Learning

model-releasesarxiv-cs-ro

arXiv:2503.03480v4 Announce Type: replace Abstract: Vision-language-action models (VLAs) show potential as generalist robot policies. However, these models pose extreme safety challenges during real-w

TextTIGER: Text-based Intelligent Generation with Entity Prompt Refinement for Text-to-Image Generation

arXiv:2504.18269v2 Announce Type: replace Abstract: When generating images from prompts that include specific entities, the model must retain as much entity-specific knowledge as possible. However, th

When Choices Become Risks: Safety Failures of Large Language Models under Multiple-Choice Constraints

arXiv:2604.16916v1 Announce Type: new Abstract: Safety alignment in large language models (LLMs) is primarily evaluated under open-ended generation, where models can mitigate risk by refusing to respo

When Helpers Become Hazards: A Benchmark for Analyzing Multimodal LLM-Powered Safety in Daily Life

arXiv:2601.04043v2 Announce Type: replace Abstract: As Multimodal Large Language Models (MLLMs) become an indispensable assistant in human life, the unsafe content generated by MLLMs poses a danger to

When Misinformation Speaks and Converses: Rethinking Fact-Checking in Audio Platforms

arXiv:2604.16767v1 Announce Type: new Abstract: Audio platforms have evolved beyond entertainment. They have become central to public discourse, from podcasts and radio to WhatsApp voice notes and liv

Tutorials

A methodology to rank importance of frequencies and channels in electromyography data with Decision Tree classifiers

arXiv:2604.15353v1 Announce Type: cross Abstract: This study presents a methodology for identifying the most informative frequencies and channels in electromyography (EMG) data to evaluate muscle reco

tutorialsarxiv-cs-lg

CIG: Measuring Conversational Information Gain in Deliberative Dialogues with Semantic Memory Dynamics

arXiv:2604.15647v1 Announce Type: new Abstract: Measuring the quality of public deliberation requires evaluating not only civility or argument structure, but also the informational progress of a conve

DiZiNER: Disagreement-guided Instruction Refinement via Pilot Annotation Simulation for Zero-shot Named Entity Recognition

arXiv:2604.15866v1 Announce Type: cross Abstract: Large language models (LLMs) have advanced information extraction (IE) by enabling zero-shot and few-shot named entity recognition (NER), yet their ge

GTA-2: Benchmarking General Tool Agents from Atomic Tool-Use to Open-Ended Workflows

arXiv:2604.15715v1 Announce Type: cross Abstract: The development of general-purpose agents requires a shift from executing simple instructions to completing complex, real-world productivity workflows

Stargazer: A Scalable Model-Fitting Benchmark Environment for AI Agents under Astrophysical Constraints

model-releasesarxiv-cs-lg

arXiv:2604.15664v1 Announce Type: new Abstract: The rise of autonomous AI agents suggests that dynamic benchmark environments with built-in feedback on scientifically grounded tasks are needed to eval

Applications

Tabular foundation models for in-context prediction of molecular properties

arXiv:2604.16123v1 Announce Type: new Abstract: Accurate molecular property prediction is central to drug discovery, catalysis, and process design, yet real-world applications are often limited by sma

applicationsarxiv-cs-lg

VEFX-Bench: A Holistic Benchmark for Generic Video Editing and Visual Effects

arXiv:2604.16272v1 Announce Type: cross Abstract: As AI-assisted video creation becomes increasingly practical, instruction-guided video editing has become essential for refining generated or captured

Source tweet is here: https://x.com/jerryjliu0/status/2044902620746363016?s=20

model-releasesjerry-liu--x

Source tweet is here: https://x.com/jerryjliu0/status/2044902620746363016?s=20 We comprehensively benchmarked Opus 4.7 on document understanding. We evaluated it through ParseBench - our comprehensive

19 Apr 2026

Applications

An Underexplored Frontier: Large Language Models for Rare Disease Patient Education and Communication -- A scoping review

arXiv:2604.14179v1 Announce Type: new Abstract: Rare diseases affect over 300 million people worldwide and are characterized by complex care pathways, limited clinical expertise, and substantial unmet

applicationsarxiv-cs-cl

Local Ai

Atropos: Improving Cost-Benefit Trade-off of LLM-based Agents under Self-Consistency with Early Termination and Model Hotswap

arXiv:2604.15075v1 Announce Type: cross Abstract: Open-weight Small Language Models(SLMs) can provide faster local inference at lower financial cost, but may not achieve the same performance level as

local-aiarxiv-cs-lg

Applications

De-Anonymization at Scale via Tournament-Style Attribution

arXiv:2601.12407v2 Announce Type: replace-cross Abstract: As LLMs rapidly advance and enter real-world use, their privacy implications are increasingly important. We study an authorship de-anonymizati

applicationsarxiv-cs-cl

Diagnosing LLM Judge Reliability: Conformal Prediction Sets and Transitivity Violations

arXiv:2604.15302v1 Announce Type: cross Abstract: LLM-as-judge frameworks are increasingly used for automatic NLG evaluation, yet their per-instance reliability remains poorly understood. We present a

Internal Knowledge Without External Expression: Probing the Generalization Boundary of a Classical Chinese Language Model

arXiv:2604.14180v1 Announce Type: new Abstract: We train a 318M-parameter Transformer language model from scratch on a curated corpus of 1.56 billion tokens of pure Classical Chinese, with zero Englis

Learning to Think Like a Cartoon Captionist: Incongruity-Resolution Supervision for Multimodal Humor Understanding

arXiv:2604.15210v1 Announce Type: cross Abstract: Humor is one of the few cognitive tasks where getting the reasoning right matters as much as getting the answer right. While recent work evaluates hum

Meituan Merchant Business Diagnosis via Policy-Guided Dual-Process User Simulation

arXiv:2604.15190v1 Announce Type: cross Abstract: Simulating group-level user behavior enables scalable counterfactual evaluation of merchant strategies without costly online experiments. However, bui

Mitigating LLM biases toward spurious social contexts using direct preference optimization

arXiv:2604.02585v2 Announce Type: replace-cross Abstract: LLMs are increasingly used for high-stakes decision-making, yet their sensitivity to spurious contextual information can introduce harmful bia