Search: “evaluation”

Toward a Theory of Value in AI Alignment

arXiv:2608.10327v1 Announce Type: new Abstract: Can AI systems be aligned to human values? The popularization of large language models (LLMs) and multi-modal foundation models has seen a rise in harms

12 Aug 2026

VIDS-Seg: Towards Reliable Uncertainty Quantification in Pediatric Cardiac Ultrasound Segmentation

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

DGX agent

arXiv:2608.10903v1 Announce Type: new Abstract: Reliable clinical deployment of machine learning requires models that know when they are likely to fail, particularly for subgroups underrepresented in

safetyarxiv-cs-cv

12 Aug 2026

Applications

Accurate but Natural? Diagnosing Grammatical and Idiomatic Gaps in Japanese EFL Writing

arXiv:2608.09289v1 Announce Type: new Abstract: Second language writing research distinguishes grammatical accuracy from native-like idiomaticity, yet automated writing evaluation often conflates thes

applicationsarxiv-cs-cl

Action- and Language-Conditioned Video Assessment for Embodied Control

arXiv:2608.08273v1 Announce Type: cross Abstract: Vision-based embodied agents executing multi-step natural language instructions require feedback mechanisms that assess task progress over complete tr

safetyarxiv-cs-cv

Beyond Aggregate Calibration: Decomposing Income-Conditional Recall Disparities in Automated Credit Default Prediction

arXiv:2608.08202v1 Announce Type: new Abstract: Data-centric curation pipelines frequently rely on model confidence scores to flag and filter noisy or mislabeled training instances. Evaluating this fi

safetyarxiv-cs-lg

Beyond Pixels: Benchmarking and Reward-Based Assessing Framework for Visual Spatial Aesthetics

arXiv:2512.05098v2 Announce Type: replace-cross Abstract: In recent years, Image Quality Assessment (IQA) for AI-generated images (AIGI) has advanced rapidly; however, existing methods primarily targe

Tutorials

Beyond the Node: Clade-level Selection for Efficient MCTS in Automatic Heuristic Design

arXiv:2602.00549v2 Announce Type: replace Abstract: While Monte Carlo Tree Search (MCTS) shows promise in Large Language Model (LLM) based Automatic Heuristic Design (AHD), it suffers from a critical

tutorialsarxiv-cs-lg

Business Arena: Benchmarking LLM Agents in a Realistic Marketplace

arXiv:2608.08621v1 Announce Type: new Abstract: Running a business is a challenging form of intelligent work. Operators must infer opportunities from partial signals, commit capital under uncertainty,

Compiling and Benchmarking Task-State Horizons for Embodied Agents

model-releasesarxiv-cs-ro

arXiv:2608.08036v1 Announce Type: new Abstract: Frontier agentic models are increasingly deployed as high-level planners for long-horizon embodied tasks. Existing robotic benchmarks have advanced long

CORDA: A Benchmark for Hierarchical Harm-Centric Moral Reasoning in Large Language Models

arXiv:2608.08061v1 Announce Type: new Abstract: The key question in moral judgement is not simply whether someone chooses the 'right' answer, but how they decide what matters most when moral principle

CresOWLve: Benchmarking Creative Problem-Solving Over Real-World Knowledge

arXiv:2604.03374v2 Announce Type: replace-cross Abstract: Creative problem-solving requires combining multiple cognitive abilities, including logical reasoning, lateral thinking, analogy-making, and c

CyberAGENTS: Structured Autonomy for Agentic Gamified Learning in Cybersecurity

arXiv:2608.07965v1 Announce Type: new Abstract: Gamification is especially effective in learning domains requiring active problem-solving and iterative skill-building, such as cybersecurity education.

ELBench: A Multi-Dimensional Benchmark for Education-Facing Large Language Models

arXiv:2608.09548v1 Announce Type: cross Abstract: Large language models are increasingly deployed in education as tutors, teaching assistants, and content generators. These roles place demands that or

Estimating Uncertainty in Galaxy Morphology Classification

arXiv:2608.08398v1 Announce Type: new Abstract: Astronomers classify galaxy morphology to investigate cosmic evolution. While deep foundation models are increasingly utilized in Galaxy Morphology Clas

Flow-by-Flow:Content-Judgment Bypass for Governing AI Output in High-Loss Domains

arXiv:2608.07474v1 Announce Type: new Abstract: Prior work showed that human-in-the-loop oversight becomes structurally untenable in high-loss domains when AI output velocity V exceeds human cognitive

From Rebound to Remedy: Understanding and Mitigating Reward Hacking via Representation Engineering

arXiv:2604.01476v2 Announce Type: replace-cross Abstract: Reinforcement learning for LLMs is vulnerable to reward hacking, where models exploit shortcuts to maximize reward without solving the intende

safetyarxiv-cs-cl

Geometry Beats Estimated Depth: RGB-Only Multi-Camera 3D Tracking under Sim2Real

arXiv:2608.07579v1 Announce Type: cross Abstract: The AI City Challenge 2026 Track 1 evaluates multi-camera 3D perception in large indoor warehouses under a synthetic-to-real (Sim2Real) setting; depth

IDRAAK: From Multi-Agent NLP to Few-Shot Prompting for Semantic Drift Detection in Technical Requirements

arXiv:2608.08801v1 Announce Type: new Abstract: Translating technical requirements across languages can introduce semantic drift, altering numerical constraints, polarities, modalities, or other speci

agentsarxiv-cs-cl

Large Multimodal Agents for Intelligent Transportation Systems: Architectures, Evidence, and Deployment Challenges

arXiv:2608.08184v1 Announce Type: new Abstract: Large multimodal agents (LMAs) are increasingly proposed for intelligent transportation systems (ITS), but existing studies often conflate multimodality

LazyHMC: Hamiltonian Monte Carlo Simulation for Lazy, Infinite Dimensional Probabilistic Programs

model-releasesarxiv-cs-lg

arXiv:2608.08588v1 Announce Type: cross Abstract: Hamiltonian Monte Carlo (HMC) is a successful generic inference method in probabilistic programming, but in its ordinary formulation it needs gradient

LoRA-based Adaptation Alone Is Not Enough: Understanding the Limits of Foundation Models for Face Presentation Attack Detection

arXiv:2608.09633v1 Announce Type: new Abstract: Face presentation attack detection (PAD) aims to reliably detect a wide range of presentation attacks. While PAD methods achieve strong performance with

researcharxiv-cs-cv

MedCalc-R1: Knowledge-Guided Reward Framework for Medical Mathematical Reasoning

arXiv:2608.08623v1 Announce Type: new Abstract: In Reinforcement Learning with Verifiable Rewards (RLVR) frameworks for mathematical reasoning tasks, floating-point results are typically evaluated usi

Multi-agent discovery of practical quantum LDPC codes

arXiv:2608.08996v1 Announce Type: cross Abstract: Quantum low-density parity-check (qLDPC) codes can encode multiple logical qubits using sparse parity checks, yet searching for useful finite-length i

Penalizing Length: Uncovering Systematic Bias in Quality Estimation Metrics

arXiv:2510.22028v4 Announce Type: replace Abstract: Quality Estimation (QE) metrics are vital in machine translation for reference-free evaluation and increasingly serve as selection criteria in data

safetyarxiv-cs-cl

Privacy-Preserving Data Drift Detection and Recovery for Large-Scale LLM Applications via Proxy Representations

arXiv:2608.08245v1 Announce Type: cross Abstract: LLM applications deployed at scale face a fundamental challenge: privacy constraints prevent direct inspection of user interactions, making it difficu

ReMIND: Orchestrating Modular Large Language Models for Controllable Serendipity A REM-Inspired System Design for Emergent Creative Ideation

arXiv:2601.07121v3 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly used not only for problem solving but also for creative ideation; however, generating ideas that

RISE-RL: Rubric-Informed Selective Exploration for Open-Ended Reinforcement Learning

arXiv:2608.09123v1 Announce Type: new Abstract: Aligning Large Language Models (LLMs) for open-ended tasks is challenging because responses must satisfy multidimensional criteria without following a s

RobustDefect-LLM: Explainable and Robustness-Aware Industrial Surface Defect Classification with Decision Support and AI-Assisted Reporting

arXiv:2608.08589v1 Announce Type: new Abstract: This paper presents RobustDefect-LLM, an industrial surface-defect inspection framework integrating deep-learning classification, operator-facing visual

safetyarxiv-cs-cv

Satellite Trajectory Optimization via Proximal Policy Optimization for Space Debris Avoidance

arXiv:2608.09628v1 Announce Type: new Abstract: Collision avoidance systems are commonly used to avoid fragmentation events occurring in Low-Earth Orbit (LEO) and Geosynchronous Equatorial Orbit (GEO)

safetyarxiv-cs-lg

The Collaboration Gap: Exploration and Benchmarking of Open-World Agentic Cooperation

arXiv:2511.02687v2 Announce Type: replace Abstract: The trajectory of AI development suggests that we will increasingly rely on agent-based systems powered by language models, composed of independentl

Twin Rollouts: Noise-Coupled Counterfactual Branching in Interactive Video World Models

arXiv:2608.08982v1 Announce Type: new Abstract: Interactive video world models generate rollouts autoregressively under an action stream, yet they are trained and evaluated almost exclusively on factu

researcharxiv-cs-lg

ADIAS: Automated Design of Interactive Agentic Systems

arXiv:2608.06410v1 Announce Type: new Abstract: Automated agent design improves agent harnesses through iterative revision, evaluation, and feedback summarization. Existing methods are largely candida

Agentic Planning for Symbolic Execution

arXiv:2608.06397v1 Announce Type: cross Abstract: Symbolic execution seeks to explore feasible program paths, yet a practical run may exhaust its resources while much program behaviour remains unreach

CASA: Classification Augmented with Safety Attention for Robust Multimodal Alignment

arXiv:2604.00310v2 Announce Type: replace-cross Abstract: Multimodal large-language models (MLLMs) often experience degraded safety alignment when harmful queries exploit cross-modal interactions. Mod

CoBa: Cost-Effective Test-Time Scaling via Compute-Balanced Routing

arXiv:2608.07424v1 Announce Type: new Abstract: Test-time scaling is often implemented by spending more compute along one axis: sampling more solutions, extending a chain of thought, or applying a str

Confidence Estimation for Financial Vision-Language Models in Chart and Document Understanding

arXiv:2608.06532v1 Announce Type: new Abstract: LVLMs are increasingly used to read financial charts, tables, and documents, where a single misread figure can move a decision and the most authoritativ

safetyarxiv-cs-cl

From Test-Time Scaling to Reusable Memory: Measuring Crystallization in Text-to-SQL

arXiv:2608.07213v1 Announce Type: new Abstract: Test-time scaling can correct difficult text-to-SQL queries, but the extra computation is normally discarded after each answer. Systems increasingly ret

researcharxiv-cs-cl

Generative Embedding Benchmark: How Much Information Survives in a Dense Embedding?

model-releasesarxiv-cs-cv

arXiv:2608.06972v1 Announce Type: new Abstract: Embeddings have emerged as a standard representational interface linking foundation models with downstream systems. Most embedding benchmarks assess rep

How Molecular Generative Models Organize Molecular Identity

arXiv:2608.06956v1 Announce Type: new Abstract: Generative models for matter are often evaluated as samplers over output representations, and their latent spaces are commonly used as proxies for navig

researcharxiv-cs-lg

Joint Optimization of Reasoning and Dual-Memory for Self-Learning Diagnostic Agent

arXiv:2604.07269v2 Announce Type: replace Abstract: Clinical expertise improves not only by acquiring medical knowledge, but by accumulating experience that yields reusable diagnostic patterns. Recent

agentsarxiv-cs-cl

Long-Horizon Agent Trajectory Attribution: A Unified Benchmark and Fine-Grained Annotation Framework

arXiv:2608.06909v1 Announce Type: new Abstract: Large language model (LLM) agents increasingly operate through long-horizon trajectories involving user instructions, tool use, external observations, a

Provable Training Data Identification for Large Language Models

arXiv:2510.09717v3 Announce Type: replace-cross Abstract: Identifying training data of large-scale models is critical for copyright litigation, privacy auditing, and ensuring fair evaluation. However,

RegionDet: A Benchmark for Region Detection Beyond Object Instances

model-releasesarxiv-cs-cv

arXiv:2608.06850v1 Announce Type: new Abstract: Object detection is a fundamental task in computer vision and has achieved remarkable progress on standard benchmarks by localizing discrete and well-bo

Stockmark-Nemotron-3-Nano-Omni-JapanDocReader: Structured Document Parsing via Capability Injection and Forgetting Control

model-releasesarxiv-cs-cl

arXiv:2608.06758v1 Announce Type: new Abstract: We present Stockmark-Nemotron-3-Nano-Omni-JapanDocReader, a Japanese document understanding model built from Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16

TradeVerse: A Longitudinal Benchmark of Political Negotiation in International Trade

arXiv:2608.06549v1 Announce Type: cross Abstract: LLMs are increasingly being applied to tasks involving institutional and political texts, but existing benchmarks evaluate them on isolated documents

WebGrader: Training LLMs for Web Development with Self-Evolving Programmatic Grader

arXiv:2608.06474v1 Announce Type: new Abstract: Large language models increasingly generate complete websites from natural-language descriptions, and reinforcement learning has become a central approa

Benchmarking and Enhancing LLMs for Rule-Intensive Review of National Standard Documents

model-releasesarxiv-cs-cl

arXiv:2608.06312v1 Announce Type: new Abstract: Large language models (LLMs) increasingly support complex professional tasks, yet their capabilities in rule-intensive document review remain insufficie

7 Aug 2026

Local Ai

Beyond Feature Importance: A Comparative Analysis of Pattern Detection Methods in Cluster Interpretation