Search: “evaluation”

From Risk Classification to Action Plan Remediation: A Guardrail Feedback Driven Framework for LLM Agents

arXiv:2606.05805v1 Announce Type: new Abstract: LLM-based guardrails typically safeguard agents by evaluating proposed actions or inputs before execution, producing safety signals such as binary allow

safetyarxiv-cs-ai

6 Jun 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Safety Paradox: How Enhanced Safety Awareness Leaves LLMs Vulnerable to Posterior Attack

DGX agent

arXiv:2606.05614v1 Announce Type: new Abstract: Large language models (LLMs) are rigorously aligned to refuse harmful requests, a process that inherently cultivates a latent capacity to evaluate and r

model-releasesarxiv-cs-ai

6 Jun 2026

When Tools Fail: Benchmarking Dynamic Replanning and Anomaly Recovery in LLM Agents

arXiv:2606.05806v1 Announce Type: new Abstract: Existing benchmarks evaluate Tool-Integrated Reasoning (TIR) in LLMs on idealized ''happy paths'', largely overlooking real-world tool failures. We intr

6 Jun 2026

A Model of Multi-turn Human Persuadability Using Probabilistic Belief Tracing

arXiv:2606.05330v1 Announce Type: new Abstract: Large language models can shift human beliefs across high-stakes domains, but most persuasion studies rely on pre/post belief change. These endpoint mea

researcharxiv-cs-cl

Beyond Alignment: Value Diversity as a Collective Property in Multicultural Agent Systems

arXiv:2606.05985v1 Announce Type: new Abstract: Multicultural multi-agent systems are increasingly deployed in globally diverse settings, where different agents are grounded in different cultural back

safetyarxiv-cs-cl

Deep Learning-assisted AMD Staging based on OCT and OCT Angiography

arXiv:2606.05379v1 Announce Type: new Abstract: To develop and evaluate deep learning models for automated grading of age-related macular degeneration (AMD) severity using optical coherence tomography

researcharxiv-cs-cv

RedditPersona: A Modular Framework for Community-Conditioned LLM Adaptation from Reddit

arXiv:2606.06027v1 Announce Type: cross Abstract: Community-conditioned language model adaptation requires choices about data collection, community definition, and evaluation that are currently made i

Tamaththul3D: High-Fidelity 3D Saudi Sign Language Avatars from Monocular Video

arXiv:2605.05367v2 Announce Type: replace Abstract: Existing 3D sign language avatar reconstruction methods are developed and evaluated exclusively on Western sign languages, and no 3D parametric anno

researcharxiv-cs-cv

TensorBench: Benchmarking Coding Agents on a Compiler-Based Tensor Framework

arXiv:2606.05570v1 Announce Type: new Abstract: Repository-level coding benchmarks face a trade-off between task difficulty and evaluation reliability: tasks that challenge frontier models often invol

A Study of the Scale Invariant Signal to Distortion Ratio in Speech Separation with Noisy References

arXiv:2508.14623v2 Announce Type: replace-cross Abstract: This paper examines the implications of using the Scale-Invariant Signal-to-Distortion Ratio (SI-SDR) as both evaluation and training objectiv

An Open-Source Two-Stage Computer Vision Pipeline for Fine-Grained Vehicle Classification using Vision Transformers

arXiv:2606.05149v1 Announce Type: new Abstract: Vehicle body type is a significant determinant of cyclist injury severity in overtaking crashes, yet automated tools for classifying vehicles into injur

DetectZoo: A Unified Toolkit for AI-Generated Content Detection Across Text, Audio, and Image Modalities

arXiv:2606.04205v1 Announce Type: cross Abstract: The growing popularity and capacity of generative models have eroded the distinction between human and machine-generated content, motivating a growing

FoeGlass: Simple In-Context Learning Is Enough for Red Teaming Audio Deepfake Detectors

arXiv:2606.05101v1 Announce Type: cross Abstract: Audio deepfake detection (ADD) models are critical for countering the malicious use of text-to-speech (TTS) models. Evaluating and strengthening ADD m

researcharxiv-cs-lg

Inclusion-of-Thoughts: Mitigating Preference Instability via Purifying the Decision Space

arXiv:2604.04944v2 Announce Type: replace-cross Abstract: Multiple-choice questions (MCQs) are widely used to evaluate large language models (LLMs). However, LLMs remain vulnerable to the presence of

researcharxiv-cs-ai

Neural Radiated-Noise Fields for Unmanned Underwater Vehicle Noise Spectrum Prediction in Three-Dimensional Scenes

arXiv:2606.04008v1 Announce Type: cross Abstract: Radiated noise in unmanned underwater vehicles (UUVs) is an important indicator for characterizing acoustic signatures and evaluating platform perform

researcharxiv-cs-ai

Testing Neural Networks via Bayesian-Guided Exploration of Decision Landscapes

arXiv:2606.04314v1 Announce Type: new Abstract: As neural networks are increasingly deployed in safety-critical domains, testing is essential to evaluate and improve their reliability. Existing testin

safetyarxiv-cs-lg

Unifying Information-Theoretic and Pair-Counting Clustering Similarity

arXiv:2511.03000v2 Announce Type: replace-cross Abstract: Comparing clusterings is central to evaluating unsupervised models, yet the many existing similarity measures can produce widely divergent, so

researcharxiv-cs-lg

When Clients Stop Following: A Cognitive Conceptualization Diagram-driven Framework for Strategic Counseling

arXiv:2606.04389v1 Announce Type: new Abstract: Large Language Models (LLMs) show promise in psychological counseling, yet existing benchmarks rely heavily on highly cooperative simulated clients. We

researcharxiv-cs-cl

Any2Poster: Any-Source Poster Generation Across Modalities and Domains

arXiv:2606.02915v1 Announce Type: new Abstract: Visual posters are a compact medium for communicating dense information, yet progress on automatic poster generation remains difficult to measure becaus

Assessing Region-Level EEG Contributions to Cognitive Workload Prediction

arXiv:2606.02598v1 Announce Type: new Abstract: Accurate and generalizable estimation of cognitive workload from electroencephalography (EEG) is critical for human-centered and safety-critical systems

safetyarxiv-cs-lg

BigFinanceBench: A Workflow-Grounded Benchmark for Financial-Research Agents

arXiv:2606.03829v1 Announce Type: new Abstract: Financial-research answers are decision-relevant only when another analyst can audit how they were produced: which source was chosen, which period and a

Agents

CodeHacker: Automated Test Case Generation for Detecting Vulnerabilities in Competitive Programming Solutions

arXiv:2602.20213v2 Announce Type: replace-cross Abstract: The evaluation of Large Language Models (LLMs) for code generation relies heavily on the quality and robustness of test cases. However, existi

agentsarxiv-cs-ai

CoMPAS3D: A Dataset and Benchmark for Interactive Motion

arXiv:2507.19684v2 Announce Type: replace-cross Abstract: Socially interactive humanoid robots must engage with humans through their bodies, adapting in real time to a partner's movement, intent, and

Distribution-Calibrated Inference Time Compute for Thinking LLM-as-a-Judge

arXiv:2512.03019v2 Announce Type: replace-cross Abstract: Thinking Large Language Models (LLMs) used as judges for pairwise preferences remain noisy at the single-sample level, and common aggregation

researcharxiv-cs-ai

Effect of Demographic Bias on Skin Lesion Classification

arXiv:2606.03214v1 Announce Type: new Abstract: In this study, we evaluate the performance of skin lesion classification using ResNet-based convolutional models, focusing on the impact of demographic

safetyarxiv-cs-ai

Enginuity: A Dataset and Benchmark for Vision-Language Understanding of Engineering Diagrams

arXiv:2606.03410v1 Announce Type: new Abstract: Engineering diagrams pose a distinct challenge for vision-language models: unlike natural images or general documents, they encode information through d

Fairness Definitions and Metrics in Deep Reinforcement Learning for Drug Discovery in Healthcare: A Rapid Evidence Review

arXiv:2606.02902v1 Announce Type: cross Abstract: Deep reinforcement learning (DRL) is increasingly applied to de novo molecular design, but choices in data, rewards, and evaluation can yield uneven p

safetyarxiv-cs-lg

From Script to Semantics: Prompting Strategies for African NLI

arXiv:2606.03304v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly evaluated in multilingual settings, yet their inference behavior in low-resource African languages remains

Tutorials

Grounding Functional Similarity by Invariance-Aware Model Stitching

arXiv:2505.20142v2 Announce Type: replace Abstract: In deep learning, functional similarity evaluation quantifies the extent to which independently trained models learn similar input--output relations

tutorialsarxiv-cs-lg

NVIDIA OmniDreams: Real-Time Generative World Model for Closed-Loop Autonomous Vehicle Simulation

arXiv:2606.03159v1 Announce Type: cross Abstract: As autonomous vehicle capabilities advance, the safe evaluation of driving policies in long-tail scenarios remains a critical bottleneck. In closed-lo

safetyarxiv-cs-ai

Towards Characterizing Scientific Image Utility and Upgradability

arXiv:2606.03401v1 Announce Type: new Abstract: Scientific images function as critical evidence in research communication, yet their integrity faces unprecedented threats from AI-generated content tha

Applications

Training-Free Multi-Concept LoRA Composition with Prompt-Aware Weighting

arXiv:2606.03792v1 Announce Type: new Abstract: Low-Rank Adaptation (LoRA) successfully enables personalization in text-to-image generation by adapting pre-trained diffusion models to specific visual

applicationsarxiv-cs-cv

Whose Name Comes Up? II: Benchmarking and Intervention-Based Auditing of LLM-Based Scholar Recommendation

arXiv:2602.08873v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are now used for academic expert recommendation. Existing audits typically evaluate such recommendations in isola

Applications

WISE-HAR: A Generalizable Ensemble Deep Learning Framework for WiFi-Based Human Activity Recognition

arXiv:2606.02974v1 Announce Type: new Abstract: Human Activity Recognition (HAR) using WiFi signals has emerged as a transformative technology for smart homes, healthcare monitoring, security systems,

applicationsarxiv-cs-ai

3DCodeBench: Benchmarking Agentic Procedural 3D Modeling Via Code

arXiv:2606.01057v1 Announce Type: cross Abstract: Procedural 3D modeling through code is emerging as a versatile paradigm, offering deterministic, engine-ready, and precisely editable assets that neur

Accuracy, Stability, and Repeated-Run Reliability of Large Language Models on Deterministic Programming Tasks

arXiv:2606.00920v1 Announce Type: cross Abstract: Run-level pass rate overstates retry-free coverage by up to 17.8 percentage points -- and the gap is largest precisely for mid-performing systems. We

Are LLMs Ready for Neural-integrated Mechanistic Modeling? A Benchmark and Agentic Framework

arXiv:2602.18008v2 Announce Type: replace-cross Abstract: Large language models (LLMs) have shown promise in constructing mechanistic models from data. However, existing evaluations largely focus on s

Local Ai

ArrythML: An Autoencoder-Based TinyML Approach for On-Device Arrhythmia Detection on Resource-Constrained Embedded Systems

arXiv:2606.02256v1 Announce Type: new Abstract: Our work presents a method for ECG segmentation and arrhythmia detection using Tiny Machine Learning (TinyML) models for real-time, on-device inference

local-aiarxiv-cs-lg

au_0-WM: A Unified Video-Action World Model for Robotic Manipulation

arXiv:2606.01027v1 Announce Type: new Abstract: Robotic manipulation requires models that generate executable actions while anticipating and evaluating their future consequences before physical execut

safetyarxiv-cs-ro

Benchmarking Multimodal LLMs on Code Generation for Complex Interactive Webpages

arXiv:2606.00154v1 Announce Type: cross Abstract: Recent advancements in multimodal large language models (MLLMs) have achieved remarkable progress in multimodal reasoning and code generation, catalyz

Benchmarking Recursive-Collapse Warning Claims Under Matched False-Positive Control

model-releasesarxiv-cs-lg

arXiv:2606.00329v1 Announce Type: cross Abstract: Recursive systems can enter collapse-like regimes -- self-reinforcing amplification, persistent recursion, and narrowing diversity that mask accelerat

Child-directed speech facilitates production, not comprehension, in BabyLMs

arXiv:2606.01045v1 Announce Type: new Abstract: Recent studies suggest that child-directed speech is not conducive to language learning in BabyLMs. However, current evaluations focus predominantly on

Claudini: Autoresearch Discovers State-of-the-Art Adversarial Attack Algorithms for LLMs

arXiv:2603.24511v2 Announce Type: replace-cross Abstract: We show that AI agents are capable of discovering novel algorithms for adversarial attacks against LLMs, advancing the state of the art on whi

Consistent and Distinctive: LLM Benchmark Efficiency via Maximum Independent Set Prompt Selection on Similarity Graphs

arXiv:2606.01400v1 Announce Type: cross Abstract: Evaluating large language models (LLMs) across comprehensive benchmarks is expensive and time-consuming. We propose a graph-based prompt selection fra

CV-Arena: An Open Benchmark for Instructional Computer Vision Problem Solving with Human-AI Collaborative Preferences

arXiv:2606.00931v1 Announce Type: cross Abstract: Instruction-guided image editing is becoming a general interface for visual work, yet existing benchmarks still focus largely on narrow appearance edi

DetailMaster: Can Your Text-to-Image Model Handle Long Prompts?

arXiv:2505.16915v3 Announce Type: replace-cross Abstract: While recent Text-to-Image (T2I) models show impressive capabilities in synthesizing images from brief descriptions, they struggle with the lo

Early Prediction of Liver Cirrhosis Up to Two Years in Advance: A Machine Learning Study Benchmarking Against the FIB-4 and APRI Scores

model-releasesarxiv-cs-lg

arXiv:2601.00175v2 Announce Type: replace Abstract: Objective: Develop and evaluate machine learning (ML) models for predicting incident liver cirrhosis (LC) one and two years prior to diagnosis using

FeynmanBench: Benchmarking Multimodal LLMs on Diagrammatic Physics Reasoning

arXiv:2604.03893v2 Announce Type: replace Abstract: Current multimodal benchmarks for scientific reasoning primarily evaluate local information extraction -- models recognize symbols and values and th