Search: “evaluation”

Tokenised Flow Matching for Hierarchical Simulation Based Inference

arXiv:2604.20723v1 Announce Type: cross Abstract: The cost of simulator evaluations is a key practical bottleneck for Simulation Based Inference (SBI). In hierarchical settings with shared global para

23 Apr 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Towards Certified Malware Detection: Provable Guarantees Against Evasion Attacks

DGX agent

arXiv:2604.20495v1 Announce Type: cross Abstract: Machine learning-based static malware detectors remain vulnerable to adversarial evasion techniques, such as metamorphic engine mutations. To address

researcharxiv-cs-lg

23 Apr 2026

Benchmarking Physics-Informed Neural Networks and Boundary Elements Methods for Wave Scattering

arXiv:2509.12483v4 Announce Type: replace Abstract: This study compares the Boundary Element Method (BEM) and Physics-Informed Neural Networks (PINNs) for solving the two-dimensional Helmholtz equatio

researcharxiv-cs-lg

Beyond Itinerary Planning-A Real-World Benchmark for Multi-Turn and Tool-Using Travel Tasks

arXiv:2512.22673v3 Announce Type: replace Abstract: Travel planning is a natural real-world task to test large language models' (LLMs) planning and tool-use abilities. Although prior work has studied

Beyond One Output: Visualizing and Comparing Distributions of Language Model Generations

arXiv:2604.18724v1 Announce Type: new Abstract: Users typically interact with and evaluate language models via single outputs, but each output is just one sample from a broad distribution of possible

researcharxiv-cs-ai

Capturing Classic Authorial Style in Long-Form Story Generation with GRPO Fine-Tuning

arXiv:2512.05747v3 Announce Type: replace Abstract: Evaluating and optimising authorial style in long-form story generation remains challenging because style is often assessed with ad hoc prompting an

Fairness Audits of Institutional Risk Models in Deployed ML Pipelines

arXiv:2604.19468v1 Announce Type: cross Abstract: Fairness audits of institutional risk models are critical for understanding how deployed machine learning pipelines allocate resources. Drawing on mul

safetyarxiv-cs-ai

From Experience to Skill: Multi-Agent Generative Engine Optimization via Reusable Strategy Learning

arXiv:2604.19516v1 Announce Type: new Abstract: Generative engines (GEs) are reshaping information access by replacing ranked links with citation-grounded answers, yet current Generative Engine Optimi

Mind2Drive: Predicting Driver Intentions from EEG in Real-world On-Road Driving

arXiv:2604.19368v1 Announce Type: new Abstract: Predicting driver intention from neurophysiological signals offers a promising pathway for enhancing proactive safety in advanced driver assistance syst

safetyarxiv-cs-cv

Mitigating Judgment Preference Bias in Large Language Models through Group-Based Polling

arXiv:2510.08145v2 Announce Type: replace Abstract: Large Language Models (LLMs) as automatic evaluators, commonly referred to as LLM-as-a-Judge, have also attracted growing attention. This approach p

ReflectMT: Internalizing Reflection for Efficient and High-Quality Machine Translation

arXiv:2604.19144v1 Announce Type: new Abstract: Recent years have witnessed growing interest in applying Large Reasoning Models (LRMs) to Machine Translation (MT). Existing approaches predominantly ad

RIFT: A RubrIc Failure Mode Taxonomy and Automated Diagnostics

arXiv:2604.01375v2 Announce Type: replace Abstract: Rubric-based evaluation is widely used in LLM benchmarks and training pipelines for open-ended, less verifiable tasks. While prior work has demonstr

researcharxiv-cs-ai

When and What to Ask: AskBench and Rubric-Guided RLVR for LLM Clarification

arXiv:2602.11199v2 Announce Type: replace Abstract: Large language models (LLMs) often respond even when prompts omit critical details or include misleading information, leading to hallucinations or r

When Safety Fails Before the Answer: Benchmarking Harmful Behavior Detection in Reasoning Chains

arXiv:2604.19001v1 Announce Type: new Abstract: Large reasoning models (LRMs) produce complex, multi-step reasoning traces, yet safety evaluation remains focused on final outputs, overlooking how harm

Agents

Advancing MAPF Toward the Real World: A Scalable Multi-Agent Realistic Testbed (SMART)

arXiv:2503.04798v3 Announce Type: replace Abstract: We present Scalable Multi-Agent Realistic Testbed (SMART), a realistic and efficient software tool for evaluating Multi-Agent Path Finding (MAPF) al

agentsarxiv-cs-ro

Local Ai

Aligning Language Models with Real-time Knowledge Editing

arXiv:2508.01302v3 Announce Type: replace Abstract: Knowledge editing aims to modify outdated knowledge in language models efficiently while retaining their original capabilities. Mainstream datasets

local-aiarxiv-cs-cl

Audio-DeepThinker: Progressive Reasoning-Aware Reinforcement Learning for High-Quality Chain-of-Thought Emergence in Audio Language Models

arXiv:2604.18187v1 Announce Type: cross Abstract: Large Audio-Language Models (LALMs) have made significant progress in audio understanding, yet they primarily operate as perception-and-answer systems

Auditing Support Strategies in LLMs through Grounded Multi-Turn Social Simulation

arXiv:2604.17079v1 Announce Type: new Abstract: When users seek social support from chatbots, they disclose their situation gradually, yet most evaluations of supportive LLMs rely on single-turn, full

Benchmarking Real-Time Question Answering via Executable Code Workflows

arXiv:2604.16349v1 Announce Type: cross Abstract: Retrieving real-time information is a fundamental capability for search-integrated agents in real-world applications. However, existing benchmarks are

BhashaSutra: A Task-Centric Unified Survey of Indian NLP Datasets, Corpora, and Resources

arXiv:2604.18423v1 Announce Type: new Abstract: India's linguistic landscape, spanning 22 scheduled languages and hundreds of marginalized dialects, has driven rapid growth in NLP datasets, benchmarks

BIASEDTALES-ML: A Multilingual Dataset for Analyzing Narrative Attribute Distributions in LLM-Generated Stories

arXiv:2604.17008v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly used to generate narrative content, including children's stories, which play an important role in social a

E3VS-Bench: A Benchmark for Viewpoint-Dependent Active Perception in 3D Gaussian Splatting Scenes

model-releasesarxiv-cs-cv

arXiv:2604.17969v1 Announce Type: new Abstract: Visual search in 3D environments requires embodied agents to actively explore their surroundings and acquire task-relevant evidence. However, existing v

How Robustly do LLMs Understand Execution Semantics?

model-releasesarxiv-cs-lg

arXiv:2604.16320v1 Announce Type: cross Abstract: LLMs demonstrate remarkable reasoning capabilities, yet whether they utilize internal world models or rely on sophisticated pattern matching remains o

IceBreaker for Conversational Agents: Breaking the First-Message Barrier with Personalized Starters

arXiv:2604.18375v1 Announce Type: new Abstract: Conversational agents, such as ChatGPT and Doubao, have become essential daily assistants for billions of users. To further enhance engagement, these sy

LiveFact: A Dynamic, Time-Aware Benchmark for LLM-Driven Fake News Detection

arXiv:2604.04815v2 Announce Type: replace Abstract: The rapid development of Large Language Models (LLMs) has transformed fake news detection and fact-checking tasks from simple classification to comp

MedProbeBench: Systematic Benchmarking at Deep Evidence Integration for Expert-level Medical Guideline

model-releasesarxiv-cs-cv

arXiv:2604.18418v1 Announce Type: new Abstract: Recent advances in deep research systems enable large language models to retrieve, synthesize, and reason over large-scale external knowledge. In medici

Model in Distress: Sentiment Analysis on French Synthetic Social Media

arXiv:2604.18226v1 Announce Type: new Abstract: Automated analysis of customer feedback on social media is hindered by three challenges: the high cost of annotated training data, the scarcity of evalu

PRISM: Probing Reasoning, Instruction, and Source Memory in LLM Hallucinations

arXiv:2604.16909v1 Announce Type: new Abstract: As large language models (LLMs) evolve from conversational assistants into agents capable of handling complex tasks, they are increasingly deployed in h

Tools

QIMMA قِمّة ⛰: A Quality-First Arabic LLM Leaderboard

QIMMA (قِمّة) is a quality-focused leaderboard for evaluating Arabic Large Language Models, developed by TIIUAE. The leaderboard benchmarks Arabic LLMs across various performance metrics and tasks to

toolshugging-face

QU-NLP at QIAS 2026: Multi-Stage QLoRA Fine-Tuning for Arabic Islamic Inheritance Reasoning

arXiv:2604.16396v1 Announce Type: new Abstract: Islamic inheritance law (ilm al-mawar{i}th) presents a challenging domain for evaluating large language models' structured reasoning capabilities, requi

Reference-state System Reliability method for scalable uncertainty quantification of coherent systems

arXiv:2604.17066v1 Announce Type: new Abstract: Coherent systems are representative of many practical applications, ranging from infrastructure networks to supply chains. Probabilistic evaluation of s

researcharxiv-cs-lg

Revisiting a Pain in the Neck: A Semantic Reasoning Benchmark for Language Models

arXiv:2604.16593v1 Announce Type: new Abstract: We present SemanticQA, an evaluation suite designed to assess language models (LMs) in semantic phrase processing tasks. The benchmark consolidates exis

SafeVLA: Towards Safety Alignment of Vision-Language-Action Model via Constrained Learning

model-releasesarxiv-cs-ro

arXiv:2503.03480v4 Announce Type: replace Abstract: Vision-language-action models (VLAs) show potential as generalist robot policies. However, these models pose extreme safety challenges during real-w

TextTIGER: Text-based Intelligent Generation with Entity Prompt Refinement for Text-to-Image Generation

arXiv:2504.18269v2 Announce Type: replace Abstract: When generating images from prompts that include specific entities, the model must retain as much entity-specific knowledge as possible. However, th

When Choices Become Risks: Safety Failures of Large Language Models under Multiple-Choice Constraints

arXiv:2604.16916v1 Announce Type: new Abstract: Safety alignment in large language models (LLMs) is primarily evaluated under open-ended generation, where models can mitigate risk by refusing to respo

When Helpers Become Hazards: A Benchmark for Analyzing Multimodal LLM-Powered Safety in Daily Life

arXiv:2601.04043v2 Announce Type: replace Abstract: As Multimodal Large Language Models (MLLMs) become an indispensable assistant in human life, the unsafe content generated by MLLMs poses a danger to

When Misinformation Speaks and Converses: Rethinking Fact-Checking in Audio Platforms

arXiv:2604.16767v1 Announce Type: new Abstract: Audio platforms have evolved beyond entertainment. They have become central to public discourse, from podcasts and radio to WhatsApp voice notes and liv

Tutorials

A methodology to rank importance of frequencies and channels in electromyography data with Decision Tree classifiers

arXiv:2604.15353v1 Announce Type: cross Abstract: This study presents a methodology for identifying the most informative frequencies and channels in electromyography (EMG) data to evaluate muscle reco

tutorialsarxiv-cs-lg

CIG: Measuring Conversational Information Gain in Deliberative Dialogues with Semantic Memory Dynamics

arXiv:2604.15647v1 Announce Type: new Abstract: Measuring the quality of public deliberation requires evaluating not only civility or argument structure, but also the informational progress of a conve

DiZiNER: Disagreement-guided Instruction Refinement via Pilot Annotation Simulation for Zero-shot Named Entity Recognition

arXiv:2604.15866v1 Announce Type: cross Abstract: Large language models (LLMs) have advanced information extraction (IE) by enabling zero-shot and few-shot named entity recognition (NER), yet their ge

GTA-2: Benchmarking General Tool Agents from Atomic Tool-Use to Open-Ended Workflows

arXiv:2604.15715v1 Announce Type: cross Abstract: The development of general-purpose agents requires a shift from executing simple instructions to completing complex, real-world productivity workflows

Stargazer: A Scalable Model-Fitting Benchmark Environment for AI Agents under Astrophysical Constraints

model-releasesarxiv-cs-lg

arXiv:2604.15664v1 Announce Type: new Abstract: The rise of autonomous AI agents suggests that dynamic benchmark environments with built-in feedback on scientifically grounded tasks are needed to eval

Applications

Tabular foundation models for in-context prediction of molecular properties

arXiv:2604.16123v1 Announce Type: new Abstract: Accurate molecular property prediction is central to drug discovery, catalysis, and process design, yet real-world applications are often limited by sma

applicationsarxiv-cs-lg

VEFX-Bench: A Holistic Benchmark for Generic Video Editing and Visual Effects

arXiv:2604.16272v1 Announce Type: cross Abstract: As AI-assisted video creation becomes increasingly practical, instruction-guided video editing has become essential for refining generated or captured

Source tweet is here: https://x.com/jerryjliu0/status/2044902620746363016?s=20

model-releasesjerry-liu--x

Source tweet is here: https://x.com/jerryjliu0/status/2044902620746363016?s=20 We comprehensively benchmarked Opus 4.7 on document understanding. We evaluated it through ParseBench - our comprehensive

19 Apr 2026

Applications

An Underexplored Frontier: Large Language Models for Rare Disease Patient Education and Communication -- A scoping review

arXiv:2604.14179v1 Announce Type: new Abstract: Rare diseases affect over 300 million people worldwide and are characterized by complex care pathways, limited clinical expertise, and substantial unmet

applicationsarxiv-cs-cl

17 Apr 2026

Local Ai

Atropos: Improving Cost-Benefit Trade-off of LLM-based Agents under Self-Consistency with Early Termination and Model Hotswap

arXiv:2604.15075v1 Announce Type: cross Abstract: Open-weight Small Language Models(SLMs) can provide faster local inference at lower financial cost, but may not achieve the same performance level as

local-aiarxiv-cs-lg

17 Apr 2026

Applications

De-Anonymization at Scale via Tournament-Style Attribution