Search: “evaluation”

Data Leakage in Automotive Perception: Practitioners' Insights

arXiv:2604.06899v1 Announce Type: cross Abstract: Data leakage is the inadvertent transfer of information between training and evaluation datasets that poses a subtle, yet critical, risk to the reliab

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

DosimeTron: Automating Personalized Monte Carlo Radiation Dosimetry in PET/CT with Agentic AI

DGX agent

arXiv:2604.06280v1 Announce Type: cross Abstract: Purpose: To develop and evaluate DosimeTron, an agentic AI system for automated patient-specific MC internal radiation dosimetry in PET/CT examination

model-releasesarxiv-cs-ai

10 Apr 2026

FinTruthQA: A Benchmark for AI-Driven Financial Disclosure Quality Assessment in Investor -- Firm Interactions

arXiv:2406.12009v5 Announce Type: replace Abstract: Accurate and transparent financial information disclosure is essential for market efficiency, investor decision-making, and corporate governance. Ch

Making MLLMs Blind: Adversarial Smuggling Attacks in MLLM Content Moderation

model-releasesarxiv-cs-cv

arXiv:2604.06950v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) are increasingly being deployed as automated content moderators. Within this landscape, we uncover a critic

PhyAVBench: A Challenging Audio Physics-Sensitivity Benchmark for Physically Grounded Text-to-Audio-Video Generation

arXiv:2512.23994v3 Announce Type: replace-cross Abstract: Text-to-audio-video (T2AV) generation is central to applications such as filmmaking and world modeling. However, current models often fail to

PokeGym: A Visually-Driven Long-Horizon Benchmark for Vision-Language Models

model-releasesarxiv-cs-cv

arXiv:2604.08340v1 Announce Type: new Abstract: While Vision-Language Models (VLMs) have achieved remarkable progress in static visual understanding, their deployment in complex 3D embodied environmen

Say Something Else: Rethinking Contextual Privacy as Information Sufficiency

arXiv:2604.06409v1 Announce Type: cross Abstract: LLM agents increasingly draft messages on behalf of users, yet users routinely overshare sensitive information and disagree on what counts as private.

researcharxiv-cs-ai

Spatio-Temporal Grounding of Large Language Models from Perception Streams

model-releasesarxiv-cs-ro

arXiv:2604.07592v1 Announce Type: new Abstract: Embodied-AI agents must reason about how objects move and interact in 3-D space over time, yet existing smaller frontier Large Language Models (LLMs) st

TeamLLM: A Human-Like Team-Oriented Collaboration Framework for Multi-Step Contextualized Tasks

arXiv:2604.06765v1 Announce Type: cross Abstract: Recently, multi-Large Language Model (LLM) frameworks have been proposed to solve contextualized tasks. However, these frameworks do not explicitly em

TEMPER: Testing Emotional Perturbation in Quantitative Reasoning

arXiv:2604.07801v1 Announce Type: new Abstract: Large language models are trained and evaluated on quantitative reasoning tasks written in clean, emotionally neutral language. However, real-world quer

3DCodeVerse: Exploring 3D Worlds as Code

arXiv:2610.22341v1 Announce Type: cross Abstract: We introduce 3DCodeVerse, a research ecosystem that enables coding agents to create editable and verifiable 3D content at scale. 3DCodeVerse comprises

Tutorials

Active Visual Perception: Opportunities and Challenges

arXiv:2512.03687v2 Announce Type: replace Abstract: Active visual perception enables an intelligent system to improve perception by controlling how visual evidence is acquired. However, differences in

tutorialsarxiv-cs-cv

APRiL: An Open-ended Benchmark for Chinese Social Awareness in Large Language Models

arXiv:2610.20785v1 Announce Type: new Abstract: Large language models are increasingly used in social interactions, where appropriate responses require recognizing implicit cues, inferring mental stat

Atomic Proximal Policy Optimization for Electric Robo-Taxi Dispatch and Charger Allocation

arXiv:2502.13392v3 Announce Type: replace Abstract: Pioneering companies such as Waymo have deployed electric robo-taxi services in several U.S. cities. Operating such fleets requires coordinating rid

Beyond Context Length: Characterising the Difficulty of Long-Context Reasoning

arXiv:2610.22067v1 Announce Type: new Abstract: Long-context language models support ever longer context windows, and evaluations typically treat context length as the main indicator of difficulty. Ho

ChaosBench-Logic: A Benchmark for Logical and Symbolic Reasoning on Chaotic Dynamical Systems

arXiv:2601.01982v2 Announce Type: replace Abstract: Large language models (LLMs) excel at natural language tasks but remain brittle in domains requiring precise logical and symbolic reasoning. Chaotic

CIM-Embed: Learning Instruction-Conditioned Multimodal Embeddings for Clustering

arXiv:2610.21805v1 Announce Type: new Abstract: Multimodal objects can require different partitions under different instructions, as when products in an e-commerce catalog are grouped by color, materi

EMO: A Closed-Loop Test-Time Computation Framework via Alternating Context Optimization and Logits Steering

arXiv:2610.22160v1 Announce Type: cross Abstract: Test-time context optimization adapts frozen language models through textual feedback, but ordinary conditional generation provides no separate contro

Applications

Grounded Human-Attributed Description and Activity Recognition in Videos

arXiv:2610.22295v1 Announce Type: new Abstract: Understanding characteristics and activities of individuals in complex multi-person environments is crucial for real-world applications. However, existi

applicationsarxiv-cs-cv

Improving Environment-Agnostic Coding Agents: The Role of Scaffolding

arXiv:2610.22289v1 Announce Type: new Abstract: Stronger language models may increasingly construct useful procedures for themselves, reducing the advantage of developer-supplied scaffolding. We inves

In the Guise of Correction: How Framing and Cultural Adaptation Erode LLM Safety on Real-World Criminal Cases

arXiv:2610.21789v1 Announce Type: cross Abstract: Existing safety benchmarks rely largely on synthetic prompts with loosely grounded taxonomies and offer little focused analysis of severe crime catego

Noise-Space Reward Alignment for Few-Step Generative Models

arXiv:2610.21125v1 Announce Type: new Abstract: Few-step generative models produce high-quality samples in one or a few evaluations. However, steering them toward desired properties remains difficult,

Online Quantile Omniprediction for Proper Losses

arXiv:2610.21342v1 Announce Type: new Abstract: The task of constructing a predictor that is simultaneously accurate under multiple evaluation metrics has been popularized recently under the name omni

researcharxiv-cs-lg

Orbia: Do Generated Videos Form a Consistent 3D World?

model-releasesarxiv-cs-cv

arXiv:2610.20693v1 Announce Type: new Abstract: Video generators serve as interactive worlds for exploration and simulation, yet realistic and visually appealing frames do not necessarily depict a con

Physics-Aware Distillation of a Grid Foundation Model for Real-Time Edge Inference

model-releasesarxiv-cs-lg

arXiv:2610.21833v1 Announce Type: new Abstract: Grid foundation models (GridFMs) are graph transformers pretrained on a masked power-flow objective, and evaluate power flow orders of magnitude faster

PiCo: Physics-Informed Concept Probing for Geo-Foundation Models

arXiv:2610.20973v1 Announce Type: new Abstract: Geo-foundation models (GFMs) are typically evaluated by downstream task predictive performance, revealing little about which physical features they are

researcharxiv-cs-cv

Plans Are Not Executions: Adapting Frozen World-Model Planners to Controller-Induced Execution Shift

arXiv:2610.22041v1 Announce Type: new Abstract: World-model planners evaluate candidate trajectories by predicting their future consequences, implicitly relying on the plan-to-motion relationship lear

researcharxiv-cs-ro

Qwen-Audio-Jev: A Unified Interface for Audio Decisions

arXiv:2610.21614v1 Announce Type: cross Abstract: Large audio language models (LALMs) have made substantial progress in audio understanding. However, in high-throughput applications requiring frequent

Semantic Alignment of EEG and Text: A Contrastive Learning Framework for Decoding Imagined Speech

arXiv:2610.20766v1 Announce Type: new Abstract: Imagined speech decoding from EEG offers a promising route toward naturalistic brain-computer interfaces. Traditional closed-set classifiers treat seman

Tutorials

Sensor Diagnosis for Fuel and Emissions Monitoring in Heavy-Duty Diesel Vehicles

arXiv:2610.21188v1 Announce Type: new Abstract: Faulty sensor readings can distort estimates of fuel use and carbon dioxide emissions from diesel engines. We evaluate two procedures that use relations

tutorialsarxiv-cs-lg

SSR: Structural Signal Retriever for Scalable Agent Discovery in LLM-Orchestrated Multi-Agent Systems

arXiv:2610.21113v1 Announce Type: cross Abstract: Large-scale multi-agent ecosystems require mechanisms for discovering suitable agents without exposing entire registries to a large language model (LL

What do AI chatbots do after users decline therapeutic redirection for suicide thoughts?

arXiv:2610.21253v1 Announce Type: cross Abstract: Safety evaluations of artificial intelligence (AI) chatbots focus on therapeutic redirection, statements urging users who disclose suicide thoughts to

safetyarxiv-cs-ai

When Does Muon Actually Win? A Paired Data-, Time-, and Tuning-Budget Benchmark

arXiv:2610.21555v1 Announce Type: cross Abstract: Muon's gains in data efficiency motivate evaluating its performance under explicit data, time, and tuning budgets. On a single A100 GPU, we compare a

Agents

Agent Sociodrama: Social-Dynamics-Guided Persona Recasting for LLM Agent Teams

arXiv:2610.15788v1 Announce Type: new Abstract: Large language model (LLM) agents are increasingly deployed as teams, yet they fail in characteristically social ways: information holders stay silent,

agentsarxiv-cs-ai

Agentic Architect: An Agentic AI Framework for Architecture Design Exploration and Optimization

arXiv:2604.25083v2 Announce Type: replace Abstract: Computer Architecture research has traditionally required the exploration of vast, highly complex design spaces through manual human effort. Large L

Agents

AutoMARS: Automatic and Scalable Multi-Agent Large Language Model Framework for Supply Chain Risk Assessment

arXiv:2610.17206v1 Announce Type: new Abstract: We propose AutoMARS (Automatic Multi-Agent Risk Assessment), an agentic framework for automatic and scalable comparative risk assessment in supply chain

agentsarxiv-cs-ai

Bayesian Learning to Select Fair Classifiers

arXiv:2610.20514v1 Announce Type: new Abstract: Evaluations of classifiers (AI or human or agents) often assume a batch setting in which every agent produces labels on every example in a test set, whi

Big-Big-SycoBench: A Benchmark to Detect Sycophancy Through Model Responses to Valid vs. Invalid Arguments

arXiv:2610.15634v1 Announce Type: new Abstract: Sycophancy in large language models risks amplifying unsupported claims or beliefs across the knowledge ecosystem as these systems increasingly contribu

Can Agents Forget When Asked? A Benchmark for Memory Unlearning in LLM Agents

arXiv:2610.18052v1 Announce Type: new Abstract: Long-term memory has become a core component of large language model (LLM) agents. As agents accumulate personal information over long-term use, users i

Can You Trust Your Preference Models? Stackelberg Games for Alignment

arXiv:2610.15008v1 Announce Type: new Abstract: Language models are frequently aligned to human preferences via imperfect evaluators whose errors can be exploited during optimization. We train policie

safetyarxiv-cs-ai

CASI: Credit Assignment via Set Intervention for Agentic Retrieval

arXiv:2610.20293v1 Announce Type: cross Abstract: Large language models increasingly generate multiple retrieval actions jointly, while downstream systems often provide only one reward for the resulti

CWE-bench: Refocusing Agentic Security from Exploitation to Defense

arXiv:2610.17020v1 Announce Type: cross Abstract: Cybersecurity evaluations of AI agents have focused largely on offense, yet a working exploit does not show that an agent can make software safe. An a

Do Factuality Methods Make LLMs More Factual? Reproducibility and Fair Comparison

arXiv:2610.18415v1 Announce Type: cross Abstract: Large Language Models (LLMs) are aligned to human preferences, after pre-training and supervised fine-tuning steps. However, standard RL alignment ste

Agents

Dungeon and Visibility World Generator

arXiv:2610.19523v1 Announce Type: new Abstract: Robotic exploration, coverage, and multi-agent deployment algorithms require reproducible environments with controllable geometry and visibility. This p

agentsarxiv-cs-ro

EvoWeb: A Benchmark for Coding Agents on Evolving Web Applications

arXiv:2610.16402v1 Announce Type: new Abstract: Web application development often proceeds through successive requests that extend an existing implementation. Yet web-generation benchmarks primarily e

From Human Corrections to Learned Segmentations: A Critical Mathematical Review of Information, Learning, and Human Effort

arXiv:2610.20396v1 Announce Type: new Abstract: When does a human correction become useful learning, and what establishes better segmentation at matched human effort? This critical narrative review fo

How Do LLMs Understand Legal Philosophy? Diagnosing Legal-Philosophical Reasoning in LLMs

arXiv:2610.19427v1 Announce Type: new Abstract: Legal reasoning is a fundamental capability for large language models (LLMs) to complete legal AI tasks such as legal judgment prediction. In legal prac

Judges Follow the Reference: The Derivation Gap in Verifying Tool-Call Arguments

arXiv:2610.18497v1 Announce Type: new Abstract: Tool-call judges evaluate argument values proposed by LLM agents. We find that they accept a correct value less often when their input holds no referenc