Search: “evaluation”

CrowdMath: A Dataset of Crowdsourced Mathematical Research Discussions

arXiv:2606.06526v1 Announce Type: new Abstract: Large language models have made substantial progress on mathematical reasoning, but existing benchmarks typically evaluate well-specified problems with

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

DEFINED: A Data-Efficient Computational Framework for Fine-Grained Creativity Assessment in Debate Scenarios

DGX agent

arXiv:2606.07226v1 Announce Type: cross Abstract: Human creativity has emerged as a critical competency in the era of large language models. Assessing creativity in complex, open-ended environments is

safetyarxiv-cs-ai

8 Jun 2026

Front-to-Attractors: Modifying the Front-to-Front Heuristic in Bidirectional Search

arXiv:2606.07047v1 Announce Type: new Abstract: Heuristics play a central role in the performance of bidirectional search algorithms, which commonly rely on two main classes. Front-to-end (F2E) heuris

Local Ai

How Far Can Chord-Symbol Time-Series Adaptation Carry Genre Identity? Capabilities and Boundaries in Multi-Genre Chord-Symbol Modeling

arXiv:2606.07334v1 Announce Type: cross Abstract: Harmony is a compact symbolic layer where mathematical pitch relations, acoustic consonance, and musical convention meet. This report treats chord-sym

local-aiarxiv-cs-lg

Re-Centering Humans in LLM Personalization

arXiv:2606.06614v1 Announce Type: cross Abstract: Despite growing interest, most evaluations of large language models' (LLMs') personalization abilities have relied on synthetic data. It remains uncle

VALUEFLOW: Toward Pluralistic and Steerable Value-based Alignment in Large Language Models

arXiv:2602.03160v2 Announce Type: replace Abstract: Aligning Large Language Models (LLMs) with the diverse spectrum of human values remains a central challenge: preference-based methods often fail to

CTIConnect: A Benchmark for Retrieval-Augmented LLMs over Heterogeneous Cyber Threat Intelligence

arXiv:2510.11974v2 Announce Type: replace-cross Abstract: Cyber Threat Intelligence (CTI) is foundational to modern cybersecurity, enabling organizations to proactively defend against evolving threats

DPBench: Structural Determinants of Multi-Agent LLM Coordination Under Simultaneous Resource Contention

arXiv:2602.13255v2 Announce Type: replace Abstract: We present DPBench, a benchmark for evaluating coordination in multi-agent systems built from large language models. Existing benchmarks measure tas

Exploring LLMs for South Asian Music Understanding and Generation

arXiv:2606.05522v1 Announce Type: cross Abstract: Recent advancements in Large Language Models (LLMs) have shown promising results in music understanding and generation tasks. However, existing works

From Risk Classification to Action Plan Remediation: A Guardrail Feedback Driven Framework for LLM Agents

arXiv:2606.05805v1 Announce Type: new Abstract: LLM-based guardrails typically safeguard agents by evaluating proposed actions or inputs before execution, producing safety signals such as binary allow

Safety Paradox: How Enhanced Safety Awareness Leaves LLMs Vulnerable to Posterior Attack

arXiv:2606.05614v1 Announce Type: new Abstract: Large language models (LLMs) are rigorously aligned to refuse harmful requests, a process that inherently cultivates a latent capacity to evaluate and r

When Tools Fail: Benchmarking Dynamic Replanning and Anomaly Recovery in LLM Agents

arXiv:2606.05806v1 Announce Type: new Abstract: Existing benchmarks evaluate Tool-Integrated Reasoning (TIR) in LLMs on idealized ''happy paths'', largely overlooking real-world tool failures. We intr

A Model of Multi-turn Human Persuadability Using Probabilistic Belief Tracing

arXiv:2606.05330v1 Announce Type: new Abstract: Large language models can shift human beliefs across high-stakes domains, but most persuasion studies rely on pre/post belief change. These endpoint mea

researcharxiv-cs-cl

Beyond Alignment: Value Diversity as a Collective Property in Multicultural Agent Systems

arXiv:2606.05985v1 Announce Type: new Abstract: Multicultural multi-agent systems are increasingly deployed in globally diverse settings, where different agents are grounded in different cultural back

safetyarxiv-cs-cl

Deep Learning-assisted AMD Staging based on OCT and OCT Angiography

arXiv:2606.05379v1 Announce Type: new Abstract: To develop and evaluate deep learning models for automated grading of age-related macular degeneration (AMD) severity using optical coherence tomography

researcharxiv-cs-cv

RedditPersona: A Modular Framework for Community-Conditioned LLM Adaptation from Reddit

model-releasesarxiv-cs-cl

arXiv:2606.06027v1 Announce Type: cross Abstract: Community-conditioned language model adaptation requires choices about data collection, community definition, and evaluation that are currently made i

Tamaththul3D: High-Fidelity 3D Saudi Sign Language Avatars from Monocular Video

arXiv:2605.05367v2 Announce Type: replace Abstract: Existing 3D sign language avatar reconstruction methods are developed and evaluated exclusively on Western sign languages, and no 3D parametric anno

researcharxiv-cs-cv

TensorBench: Benchmarking Coding Agents on a Compiler-Based Tensor Framework

model-releasesarxiv-cs-cl

arXiv:2606.05570v1 Announce Type: new Abstract: Repository-level coding benchmarks face a trade-off between task difficulty and evaluation reliability: tasks that challenge frontier models often invol

A Study of the Scale Invariant Signal to Distortion Ratio in Speech Separation with Noisy References

arXiv:2508.14623v2 Announce Type: replace-cross Abstract: This paper examines the implications of using the Scale-Invariant Signal-to-Distortion Ratio (SI-SDR) as both evaluation and training objectiv

An Open-Source Two-Stage Computer Vision Pipeline for Fine-Grained Vehicle Classification using Vision Transformers

arXiv:2606.05149v1 Announce Type: new Abstract: Vehicle body type is a significant determinant of cyclist injury severity in overtaking crashes, yet automated tools for classifying vehicles into injur

DetectZoo: A Unified Toolkit for AI-Generated Content Detection Across Text, Audio, and Image Modalities

arXiv:2606.04205v1 Announce Type: cross Abstract: The growing popularity and capacity of generative models have eroded the distinction between human and machine-generated content, motivating a growing

Tools

EVA-Bench Data 2.0: 3 Domains, 121 Tools, 213 Scenarios

EVA-Bench Data 2.0 is an expanded benchmark dataset containing tools and scenarios across 3 domains, featuring 121 tools and 213 test scenarios for evaluating AI agent performance. This dataset enable

toolshugging-face

FoeGlass: Simple In-Context Learning Is Enough for Red Teaming Audio Deepfake Detectors

arXiv:2606.05101v1 Announce Type: cross Abstract: Audio deepfake detection (ADD) models are critical for countering the malicious use of text-to-speech (TTS) models. Evaluating and strengthening ADD m

researcharxiv-cs-lg

Inclusion-of-Thoughts: Mitigating Preference Instability via Purifying the Decision Space

arXiv:2604.04944v2 Announce Type: replace-cross Abstract: Multiple-choice questions (MCQs) are widely used to evaluate large language models (LLMs). However, LLMs remain vulnerable to the presence of

Neural Radiated-Noise Fields for Unmanned Underwater Vehicle Noise Spectrum Prediction in Three-Dimensional Scenes

arXiv:2606.04008v1 Announce Type: cross Abstract: Radiated noise in unmanned underwater vehicles (UUVs) is an important indicator for characterizing acoustic signatures and evaluating platform perform

Tools

Reality: The Final Eval — Lukas Petersson and Axel Backlund of Andon Labs

Lukas Petersson and Axel Backlund of Andon Labs discuss their work on evaluation methods and benchmarking for AI systems, likely covering approaches to assessing AI capabilities and safety in real-wor

toolslatent-space

Testing Neural Networks via Bayesian-Guided Exploration of Decision Landscapes

arXiv:2606.04314v1 Announce Type: new Abstract: As neural networks are increasingly deployed in safety-critical domains, testing is essential to evaluate and improve their reliability. Existing testin

safetyarxiv-cs-lg

Unifying Information-Theoretic and Pair-Counting Clustering Similarity

arXiv:2511.03000v2 Announce Type: replace-cross Abstract: Comparing clusterings is central to evaluating unsupervised models, yet the many existing similarity measures can produce widely divergent, so

researcharxiv-cs-lg

When Clients Stop Following: A Cognitive Conceptualization Diagram-driven Framework for Strategic Counseling

arXiv:2606.04389v1 Announce Type: new Abstract: Large Language Models (LLMs) show promise in psychological counseling, yet existing benchmarks rely heavily on highly cooperative simulated clients. We

researcharxiv-cs-cl

Any2Poster: Any-Source Poster Generation Across Modalities and Domains

arXiv:2606.02915v1 Announce Type: new Abstract: Visual posters are a compact medium for communicating dense information, yet progress on automatic poster generation remains difficult to measure becaus

Assessing Region-Level EEG Contributions to Cognitive Workload Prediction

arXiv:2606.02598v1 Announce Type: new Abstract: Accurate and generalizable estimation of cognitive workload from electroencephalography (EEG) is critical for human-centered and safety-critical systems

safetyarxiv-cs-lg

BigFinanceBench: A Workflow-Grounded Benchmark for Financial-Research Agents

arXiv:2606.03829v1 Announce Type: new Abstract: Financial-research answers are decision-relevant only when another analyst can audit how they were produced: which source was chosen, which period and a

Agents

CodeHacker: Automated Test Case Generation for Detecting Vulnerabilities in Competitive Programming Solutions

arXiv:2602.20213v2 Announce Type: replace-cross Abstract: The evaluation of Large Language Models (LLMs) for code generation relies heavily on the quality and robustness of test cases. However, existi

agentsarxiv-cs-ai

CoMPAS3D: A Dataset and Benchmark for Interactive Motion

arXiv:2507.19684v2 Announce Type: replace-cross Abstract: Socially interactive humanoid robots must engage with humans through their bodies, adapting in real time to a partner's movement, intent, and

Distribution-Calibrated Inference Time Compute for Thinking LLM-as-a-Judge

arXiv:2512.03019v2 Announce Type: replace-cross Abstract: Thinking Large Language Models (LLMs) used as judges for pairwise preferences remain noisy at the single-sample level, and common aggregation

Effect of Demographic Bias on Skin Lesion Classification

arXiv:2606.03214v1 Announce Type: new Abstract: In this study, we evaluate the performance of skin lesion classification using ResNet-based convolutional models, focusing on the impact of demographic

Enginuity: A Dataset and Benchmark for Vision-Language Understanding of Engineering Diagrams

arXiv:2606.03410v1 Announce Type: new Abstract: Engineering diagrams pose a distinct challenge for vision-language models: unlike natural images or general documents, they encode information through d

Fairness Definitions and Metrics in Deep Reinforcement Learning for Drug Discovery in Healthcare: A Rapid Evidence Review

arXiv:2606.02902v1 Announce Type: cross Abstract: Deep reinforcement learning (DRL) is increasingly applied to de novo molecular design, but choices in data, rewards, and evaluation can yield uneven p

safetyarxiv-cs-lg

From Script to Semantics: Prompting Strategies for African NLI

model-releasesarxiv-cs-cl

arXiv:2606.03304v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly evaluated in multilingual settings, yet their inference behavior in low-resource African languages remains

Tutorials

Grounding Functional Similarity by Invariance-Aware Model Stitching

arXiv:2505.20142v2 Announce Type: replace Abstract: In deep learning, functional similarity evaluation quantifies the extent to which independently trained models learn similar input--output relations

tutorialsarxiv-cs-lg

NVIDIA OmniDreams: Real-Time Generative World Model for Closed-Loop Autonomous Vehicle Simulation

arXiv:2606.03159v1 Announce Type: cross Abstract: As autonomous vehicle capabilities advance, the safe evaluation of driving policies in long-tail scenarios remains a critical bottleneck. In closed-lo

Towards Characterizing Scientific Image Utility and Upgradability

arXiv:2606.03401v1 Announce Type: new Abstract: Scientific images function as critical evidence in research communication, yet their integrity faces unprecedented threats from AI-generated content tha

Applications

Training-Free Multi-Concept LoRA Composition with Prompt-Aware Weighting

arXiv:2606.03792v1 Announce Type: new Abstract: Low-Rank Adaptation (LoRA) successfully enables personalization in text-to-image generation by adapting pre-trained diffusion models to specific visual

applicationsarxiv-cs-cv

Whose Name Comes Up? II: Benchmarking and Intervention-Based Auditing of LLM-Based Scholar Recommendation

arXiv:2602.08873v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are now used for academic expert recommendation. Existing audits typically evaluate such recommendations in isola

Applications

WISE-HAR: A Generalizable Ensemble Deep Learning Framework for WiFi-Based Human Activity Recognition

arXiv:2606.02974v1 Announce Type: new Abstract: Human Activity Recognition (HAR) using WiFi signals has emerged as a transformative technology for smart homes, healthcare monitoring, security systems,

applicationsarxiv-cs-ai

3DCodeBench: Benchmarking Agentic Procedural 3D Modeling Via Code

arXiv:2606.01057v1 Announce Type: cross Abstract: Procedural 3D modeling through code is emerging as a versatile paradigm, offering deterministic, engine-ready, and precisely editable assets that neur

2 Jun 2026

Accuracy, Stability, and Repeated-Run Reliability of Large Language Models on Deterministic Programming Tasks

arXiv:2606.00920v1 Announce Type: cross Abstract: Run-level pass rate overstates retry-free coverage by up to 17.8 percentage points -- and the gap is largest precisely for mid-performing systems. We

2 Jun 2026

Are LLMs Ready for Neural-integrated Mechanistic Modeling? A Benchmark and Agentic Framework

arXiv:2602.18008v2 Announce Type: replace-cross Abstract: Large language models (LLMs) have shown promise in constructing mechanistic models from data. However, existing evaluations largely focus on s