Search: “evaluation”

Culture-Aware Humorous Captioning: Multimodal Humor Generation across Cultural Contexts

arXiv:2604.18091v1 Announce Type: new Abstract: Recent multimodal large language models have shown promising ability in generating humorous captions for images, yet they still lack stable control over

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

DSH-Bench: A Difficulty- and Scenario-Aware Benchmark with Hierarchical Subject Taxonomy for Subject-Driven Text-to-Image Generation

DGX agent

arXiv:2603.08090v2 Announce Type: replace Abstract: Significant progress has been achieved in subject-driven text-to-image (T2I) generation, which aims to synthesize new images depicting target subjec

model-releasesarxiv-cs-cv

21 Apr 2026

Fairness Constraints in High-Dimensional Generalized Linear Models

arXiv:2604.16610v1 Announce Type: cross Abstract: Machine learning models often inherit biases from historical data, raising critical concerns about fairness and accountability. Conventional fairness

safetyarxiv-cs-lg

'Faithful to What?' On the Limits of Fidelity-Based Explanations

arXiv:2506.12176v5 Announce Type: replace Abstract: In explainable AI, surrogate models are commonly evaluated by their fidelity to a neural network's predictions. Fidelity, however, measures alignmen

safetyarxiv-cs-lg

Hard to Be Heard: Phoneme-Level ASR Analysis of Phonologically Complex, Low-Resource Endangered Languages

arXiv:2604.18204v1 Announce Type: new Abstract: We present a phoneme-level analysis of automatic speech recognition (ASR) for two low-resourced and phonologically complex East Caucasian languages, Arc

HiRAS: A Hierarchical Multi-Agent Framework for Paper-to-Code Generation and Execution

arXiv:2604.17745v1 Announce Type: new Abstract: Recent advances in large language models have highlighted their potential to automate computational research, particularly reproducing experimental resu

HORIZON: A Benchmark for In-the-wild User Behaviour Modeling

arXiv:2604.17259v1 Announce Type: cross Abstract: User behavior in the real world is diverse, cross-domain, and spans long time horizons. Existing user modeling benchmarks however remain narrow, focus

Human-Centered Supervision for Sentiment Analysis in Telugu: A Systematic Inquiry Beyond Accuracy

arXiv:2508.01486v3 Announce Type: replace Abstract: Sentiment analysis for low-resource languages remains challenging in an era where interpretability, human alignment, and fairness are increasingly n

IDOBE: Infectious Disease Outbreak forecasting Benchmark Ecosystem

arXiv:2604.18521v1 Announce Type: new Abstract: Epidemic forecasting has become an integral part of real-time infectious disease outbreak response. While collaborative ensembles composed of statistica

Is SAM3 ready for pathology segmentation?

arXiv:2604.18225v1 Announce Type: new Abstract: Is Segment Anything Model 3 (SAM3) capable in segmenting Any Pathology Images? Digital pathology segmentation spans tissue-level and nuclei-level scales

researcharxiv-cs-cv

Live Avatar: Streaming Real-time Audio-Driven Avatar Generation with Infinite Length

arXiv:2512.04677v5 Announce Type: replace Abstract: Audio-driven avatar interaction demands real-time, streaming, and infinite-length generation -- capabilities fundamentally at odds with the sequenti

MedPRMBench: A Fine-grained Benchmark for Process Reward Models in Medical Reasoning

arXiv:2604.17282v1 Announce Type: new Abstract: Process-Level Reward Models (PRMs) are essential for guiding complex reasoning in large language models, yet existing PRM benchmarks cover only general

MetaLint: Easy-to-Hard Generalization for Code Linting

arXiv:2507.11687v4 Announce Type: replace-cross Abstract: Large language models excel at code generation but struggle with code linting, particularly in generalizing to unseen or evolving best practic

Mix and Match: Context Pairing for Scalable Topic-Controlled Educational Summarisation

arXiv:2604.18087v1 Announce Type: new Abstract: Topic-controlled summarisation enables users to generate summaries focused on specific aspects of source documents. This paper investigates a data augme

Agents

MODEST: Multi-Optics Depth-of-Field Stereo Dataset

arXiv:2511.20853v3 Announce Type: replace Abstract: Reliable depth estimation under real optical conditions remains a core challenge for camera vision in systems such as autonomous robotics and augmen

agentsarxiv-cs-cv

No One Fits All: From Fixed Prompting to Learned Routing in Multilingual LLMs

arXiv:2604.16937v1 Announce Type: new Abstract: Translation-based prompting is widely used in multilingual LLMs, yet its effectiveness varies across languages and tasks. We evaluate prompting strategi

P-Check: Advancing Personalized Reward Model via Learning to Generate Dynamic Checklist

arXiv:2601.02986v2 Announce Type: replace Abstract: Recent approaches in personalized reward modeling have primarily focused on leveraging user interaction history to align model judgments with indivi

PFDelta: A Benchmark Dataset for Power Flow under Load, Generation, and Topology Variations

arXiv:2510.22048v3 Announce Type: replace Abstract: Power flow (PF) calculations are the backbone of real-time grid operations, across workflows such as contingency analysis (where repeated PF evaluat

Precise Debugging Benchmark: Is Your Model Debugging or Regenerating?

arXiv:2604.17338v1 Announce Type: cross Abstract: Unlike code completion, debugging requires localizing faults and applying targeted edits. We observe that frontier LLMs often regenerate correct but o

Predicting LLM Compression Degradation from Spectral Statistics

arXiv:2604.18085v1 Announce Type: new Abstract: Matrix-level low-rank compression is a promising way to reduce the cost of large language models, but running compression and evaluating the resulting m

Pref-GRPO: Pairwise Preference Reward-based GRPO for Stable Text-to-Image Reinforcement Learning

arXiv:2508.20751v2 Announce Type: replace Abstract: Recent advancements highlight the importance of GRPO-based reinforcement learning methods and benchmarking in enhancing text-to-image (T2I) generati

Process Reward Models Meet Planning: Generating Precise and Scalable Datasets for Step-Level Rewards

arXiv:2604.17957v1 Announce Type: new Abstract: Process Reward Models (PRMs) have emerged as a powerful tool for providing step-level feedback when evaluating the reasoning of Large Language Models (L

Progressive Online Video Understanding with Evidence-Aligned Timing and Transparent Decisions

arXiv:2604.18459v1 Announce Type: new Abstract: Visual agents operating in the wild must respond to queries precisely when sufficient evidence first appears in a video stream, a critical capability th

SpeechMedAssist: Efficiently and Effectively Adapting Speech Language Models for Medical Consultation

arXiv:2601.04638v2 Announce Type: replace Abstract: Medical consultations are intrinsically speech-centric. However, most prior works focus on long-text-based interactions, which are cumbersome and pa

SPENCE: A Syntactic Probe for Detecting Contamination in NL2SQL Benchmarks

arXiv:2604.17771v1 Announce Type: new Abstract: Large language models (LLMs) have achieved strong performance on natural language to SQL (NL2SQL) benchmarks, yet their reported accuracy may be inflate

Synthia: Scalable Grounded Persona Generation from Social Media Data

arXiv:2507.14922v2 Announce Type: replace Abstract: Persona-driven simulations are increasingly used in computational social science, yet their validity critically depends on the fidelity of the under

Systematic Capability Benchmarking of Frontier Large Language Models for Offensive Cyber Tasks

arXiv:2604.17159v1 Announce Type: cross Abstract: We present, to our knowledge, the most comprehensive cross-model evaluation of LLM agents on offensive cybersecurity tasks, benchmarking 10 frontier m

TeleEmbedBench: A Multi-Corpus Embedding Benchmark for RAG in Telecommunications

arXiv:2604.17778v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly deployed in the telecommunications domain for critical tasks, relying heavily on Retrieval-Augmented Gener

Thermal-GEMs: Generalized Models for Building Thermal Dynamics

arXiv:2604.16443v1 Announce Type: cross Abstract: Data-driven models for building thermal dynamics are a scalable approach for enabling energy-efficient operation through fault detection & diagnosis o

applicationsarxiv-cs-lg

Training-inference input alignment outweighs framework choice in longitudinal retinal image prediction

arXiv:2604.16955v1 Announce Type: new Abstract: Quantitative prediction of future retinal appearance from longitudinal imaging would support clinical decisions in progressive macular disease that curr

safetyarxiv-cs-cv

Trustworthy deep domain adaptation for wearable photoplethysmography signal analysis with decision-theoretic uncertainty quantification

arXiv:2604.17480v1 Announce Type: new Abstract: In principle, deep generative models can be used to perform domain adaptation; i.e. align the input feature representations of test data with that of a

applicationsarxiv-cs-lg

Using large language models for embodied planning introduces systematic safety risks

arXiv:2604.18463v1 Announce Type: cross Abstract: Large language models are increasingly used as planners for robotic systems, yet how safely they plan remains an open question. To evaluate safe plann

WeatherArchive-Bench: Benchmarking Retrieval-Augmented Reasoning for Historical Weather Archives

arXiv:2510.05336v2 Announce Type: replace Abstract: Historical archives on weather events are collections of enduring primary source records that offer rich, untapped narratives of how societies have

ASMR-Bench: Auditing for Sabotage in ML Research

arXiv:2604.16286v1 Announce Type: new Abstract: As AI systems are increasingly used to conduct research autonomously, misaligned systems could introduce subtle flaws that produce misleading results wh

AutoNFS: Automatic Neural Feature Selection

arXiv:2503.13304v2 Announce Type: replace Abstract: Feature selection (FS) is a fundamental challenge in machine learning, particularly for high-dimensional tabular data, where interpretability and co

applicationsarxiv-cs-lg

BAGEL: Benchmarking Animal Knowledge Expertise in Language Models

arXiv:2604.16241v1 Announce Type: cross Abstract: Large language models have shown strong performance on broad-domain knowledge and reasoning benchmarks, but it remains unclear how well language model

ConlangCrafter: Constructing Languages with a Multi-Hop LLM Pipeline

arXiv:2508.06094v4 Announce Type: replace Abstract: Constructed languages (conlangs) such as Esperanto and Quenya have played diverse roles in art, philosophy, and international communication. Meanwhi

Cut Your Losses! Learning to Prune Paths Early for Efficient Parallel Reasoning

arXiv:2604.16029v1 Announce Type: new Abstract: Parallel reasoning enhances Large Reasoning Models (LRMs) but incurs prohibitive costs due to futile paths caused by early errors. To mitigate this, pat

applicationsarxiv-cs-cl

CXR-LT 2026 Challenge: Multi-Center Long-Tailed and Zero Shot Chest X-ray Classification

arXiv:2604.15555v1 Announce Type: new Abstract: Chest X-ray (CXR) interpretation is hindered by the long-tailed distribution of pathologies and the open-world nature of clinical environments. Existing

Agents

DeepER-Med: Advancing Deep Evidence-Based Research in Medicine Through Agentic AI

arXiv:2604.15456v1 Announce Type: new Abstract: Trustworthiness and transparency are essential for the clinical adoption of artificial intelligence (AI) in healthcare and biomedical research. Recent d

agentsarxiv-cs-ai

Differential privacy representation geometry for medical image analysis

arXiv:2603.01098v2 Announce Type: replace-cross Abstract: Differential privacy (DP)'s effect in medical imaging is typically evaluated only through end-to-end performance, leaving the mechanism of pri

researcharxiv-cs-ai

Dynamic Tool Dependency Retrieval for Lightweight Function Calling

arXiv:2512.17052v4 Announce Type: replace Abstract: Function calling agents powered by Large Language Models (LLMs) select external tools to automate complex tasks. On-device agents typically use a re

Facial-Expression-Aware Prompting for Empathetic LLM Tutoring

arXiv:2604.15336v1 Announce Type: cross Abstract: Large language models (LLMs) enable increasingly capable tutoring-style conversational agents, yet effective tutoring requires sensitivity to learners

FETAL-GAUGE: A Benchmark for Assessing Vision-Language Models in Fetal Ultrasound

arXiv:2512.22278v2 Announce Type: replace Abstract: The growing demand for prenatal ultrasound imaging has intensified a global shortage of trained sonographers, creating barriers to essential fetal h

HarmfulSkillBench: How Do Harmful Skills Weaponize Your Agents?

arXiv:2604.15415v1 Announce Type: cross Abstract: Large language models (LLMs) have evolved into autonomous agents that rely on open skill ecosystems (e.g., ClawHub and Skills.Rest), hosting numerous

IPQA: A Benchmark for Core Intent Identification in Personalized Question Answering

arXiv:2510.23536v2 Announce Type: replace Abstract: Intent identification serves as the foundation for generating appropriate responses in personalized question answering (PQA). However, existing benc

KWBench: Measuring Unprompted Problem Recognition in Knowledge Work

arXiv:2604.15760v1 Announce Type: new Abstract: We introduce the first version of KWBench (Knowledge Work Bench), a benchmark for unprompted problem recognition in large language models: can an LLM id

Machine learning approaches to uncover the neural mechanisms of motivated behaviour: from ADHD to individual differences in effort and reward sensitivity

arXiv:2604.15363v1 Announce Type: cross Abstract: Motivated behaviour relies on the brain's capacity to evaluate effort and reward. Dysregulation within these processes contributes to a spectrum of co

researcharxiv-cs-lg