Search: “evaluation”

GradeTrap: Authority Cues in Images Shift VLM Judgments Despite Explicit Instructions to Ignore Them

arXiv:2609.06058v1 Announce Type: new Abstract: As vision-language models (VLMs) become increasingly capable and are deployed in consequential real-world settings, they must evaluate evidence independ

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

HealthLoopQA: A Context-Aware Question Answering Benchmark for Interpreting Wearable Monitoring Data in Diabetes Care

DGX agent

arXiv:2609.06976v1 Announce Type: new Abstract: As medical wearables become integrated into daily chronic disease care, effectively interpreting longitudinal monitoring data is essential for patients

model-releasesarxiv-cs-lg

9 Sept 2026

Is Single-View Mesh Reconstruction Ready for Robotics?

arXiv:2505.17966v3 Announce Type: replace-cross Abstract: This paper evaluates single-view mesh reconstruction models for their potential in enabling instant digital twin creation for real-time planni

researcharxiv-cs-cv

MLIP Detective: Active Failure Mode Discovery Beyond Benchmark Scores for Machine-Learning Interatomic Potentials

arXiv:2609.08399v1 Announce Type: new Abstract: Universal machine-learning interatomic potentials (u-MLIPs) aim to generalize across diverse configurations. Benchmarks enable reproducible evaluation b

NutriBench-Kitchen: Benchmarking Embodied AI for Nutrition Management

arXiv:2609.07135v1 Announce Type: new Abstract: An embodied kitchen assistant must do more than recognize food in isolated frames. It must track ingredient states over time and integrate visual observ

Agents

Online Surrogate Repair: Decoupling High-Fidelity Feedback from Search Length in Closed-Loop Discovery

arXiv:2609.07655v1 Announce Type: new Abstract: Closed-loop AI scientists can generate candidate designs at low marginal computational cost, whereas reliable feedback may require wet-lab synthesis, ch

agentsarxiv-cs-lg

Hardware

Proper Dataset Valuation by Pointwise Mutual Information

arXiv:2405.18253v4 Announce Type: replace Abstract: Data plays a central role in advancements in modern artificial intelligence, with high-quality data emerging as a key driver of model performance. T

hardwarearxiv-cs-lg

Agents

Report of the 8th LSVOS Challenge: Complex and Multimodal Video Object Segmentation

arXiv:2609.06078v1 Announce Type: new Abstract: This report summarizes the 8th Large-scale Video Object Segmentation (LSVOS) Challenge, held in conjunction with ECCV 2026. The challenge evaluates vide

agentsarxiv-cs-cv

SAFIRE: Safety-Critical Benchmark for Fine-grained Fire and Smoke Understanding in Multimodal LLMs

arXiv:2609.07823v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) show strong progress on vision-language tasks, yet their reliability in safety-critical settings remains undere

Applications

Sparse Incident-Cluster Learning for 12-hour Port Flood Pre-warning in Digital-Twin Analytics

arXiv:2609.06109v1 Announce Type: new Abstract: Port flood digital twins require analytics that warn operators before disruption, but official warning incidents are often few and adjacent observations

applicationsarxiv-cs-lg

Sustained Performance and Energy Accounting for Nonlinear Forecasting Across Classical and Simulated Quantum Models

arXiv:2510.25183v2 Announce Type: replace-cross Abstract: Energy-efficient AI should be evaluated across the full application pipeline, not only by lowest error or shortest training time. We study thi

Tempora-Fusion: Time-Lock Puzzle with Efficient Verifiable Homomorphic Linear Combination

arXiv:2406.15070v3 Announce Type: replace-cross Abstract: We present Tempora-Fusion, the first homomorphic TLP scheme with efficient public verification of both individual puzzle solutions and homomor

A Deep Generative Model for Synthesizing Labeled Wireless Signals

arXiv:2609.05396v1 Announce Type: new Abstract: Wireless signals with position-related labels are pivotal for both performance evaluation and model training in the realm of wireless sensing. However,

Hardware

A Sim-to-Real Study of Surface-Code Decoder Benchmarking

arXiv:2609.04557v1 Announce Type: cross Abstract: Quantum error-correction decoders are typically benchmarked against synthetic circuit-level noise, under the assumption that a decoder's ranking under

hardwarearxiv-cs-lg

Disentangling Attention in Deep Operator Learning: A Controlled Study of Data-Driven and Physics-Informed Architectures

arXiv:2609.04407v1 Announce Type: new Abstract: Deep neural operators learn mappings between input functions and complete PDE solution fields, enabling forward evaluations of new problem instances ord

EVOHARNESSBENCH: Can Your Agents Keep Pace with an Evolving Harness?

model-releasesarxiv-cs-cl

arXiv:2609.04280v1 Announce Type: cross Abstract: Modern LLM-based agents operate through a harness of tools, reusable skills, and specialist agents that shapes what they observe and what they can do.

Knowing What Not to Answer: Selective Non-Compliance in Vision-Language Models

arXiv:2609.04720v1 Announce Type: cross Abstract: Vision-language models (VLMs) are expected to respond helpfully to appropriate requests while withholding compliance with requests that are incorrect,

Knowing When Not to Answer: Pseudo-Ensembles for Abstention in Music Audio-Language Models

arXiv:2609.04362v1 Announce Type: cross Abstract: Music audio-language models are evaluated almost entirely by accuracy on multiple-choice questions. This protocol forces the model to commit to an opt

researcharxiv-cs-cl

Measuring AI Accountability Through Argumentation Analysis: Can Model Reasoning Withstand Scrutiny?

arXiv:2609.05088v1 Announce Type: new Abstract: AI oversight methods rely on ground truth for validation, but what constitutes appropriate AI behavior is contested. This leaves evaluation of moral rea

safetyarxiv-cs-ai

Measuring the Novelty of Biomedical Papers Using the Latent Distances between Knowledge Units

arXiv:2609.05175v1 Announce Type: cross Abstract: Measuring the novelty of scientific papers is a central concern in research evaluation and scientometrics. From a recombination perspective, prior stu

safetyarxiv-cs-cl

PetQA: Benchmarking Veterinary Knowledge and Clinical Reasoning

arXiv:2609.04598v1 Announce Type: cross Abstract: We introduce PetQA, a Korean long-form question-answering (QA) benchmark for evaluating veterinary knowledge and clinical reasoning in large language

SciDocBench: A Workflow-Centered Benchmark and Data Pipeline for Scientific Document Understanding

arXiv:2609.05141v1 Announce Type: new Abstract: Scientific papers require models to reason jointly over text, equations, figures, tables, code, and datasets while preserving the provenance of supporti

Single-Query Black-Box Calibration Auditing via Logit Bias

arXiv:2609.05125v1 Announce Type: new Abstract: Evaluating the calibration of Large Language Models (LLMs) is critical for their safe deployment as zero-shot classifiers. Yet, commercial API providers

TreeFI: Value-Aware Statistical Fault Injection for Deep Neural Networks

arXiv:2609.04912v1 Announce Type: cross Abstract: Reliability evaluation of deep neural networks under hardware faults commonly relies on fault injection, but exhaustive campaigns are intractable for

researcharxiv-cs-ai

Applications

Wireless Foundation Models: State-of-the-Art and Open Challenges

arXiv:2609.04707v1 Announce Type: cross Abstract: Wireless foundation models (WFMs) have emerged as a promising approach for learning reusable representations from large-scale wireless data and adapti

applicationsarxiv-cs-ai

A comparative study on the accuracy & repeatability of mobile robotic platforms for the delivery of precision NDE measurement

arXiv:2609.03794v1 Announce Type: new Abstract: Mobile robotic platforms offer a flexible alternative to fixed manipulators for non-destructive evaluation (NDE) of large aerospace structures, but thei

safetyarxiv-cs-ro

Beyond BLEU: A Case for Redefining Sign Language Translation Benchmarks

arXiv:2609.03734v1 Announce Type: cross Abstract: BLEU-4 is the standard metric for evaluating sign language translation (SLT), but spoken-language metrics may not adequately reflect sign language pro

researcharxiv-cs-ai

Dual-Form ASR: Semantics-Aware Inverse Text Normalization for Chinese Speech Recognition

arXiv:2609.02901v1 Announce Type: new Abstract: Modern automatic speech recognition (ASR) scenarios require both spoken-form transcripts for faithful transcription and readable written-form transcript

researcharxiv-cs-cl

Environment Evolution for Terminal Agents

arXiv:2609.04128v1 Announce Type: new Abstract: Scaling interactive and verifiable environments is critical for training terminal agents. As frontier models become more capable, environments synthesiz

FailBench: How Reliable are VLMs at Judging Robot Task Success?

arXiv:2609.03611v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) are increasingly used to evaluate robot manipulation outcomes, but existing benchmarks offer limited evidence of cross-d

Improving Clinical Target Volume Segmentation Accuracy using Anatomical Priors and Active Learning for the AGITG TOPGEAR Clinical Trial

arXiv:2609.03186v1 Announce Type: cross Abstract: Training deep learning-based medical image segmentation models is challenging with limited curated datasets. For AGITG TOPGEAR, a gastric cancer trial

researcharxiv-cs-cv

Interface-Induced Trajectory Censoring

arXiv:2609.03966v1 Announce Type: new Abstract: Agent evaluations report a tool-call rate read off the serving stack. That number can be zero while the model is emitting well-formed calls: the interfa

Applications

LLM4CKD: Large Language Models for Early Stage Chronic Kidney Disease Screening

arXiv:2609.04013v1 Announce Type: new Abstract: Early screening of chronic kidney disease (CKD) is critical for timely intervention, yet most machine learning (ML) and deep learning (DL) approaches re

applicationsarxiv-cs-ai

LongCounsel-8: A Benchmark Suite for Longitudinal Depression Tracking from Multi-Session Counseling Dialogues

arXiv:2609.03507v1 Announce Type: cross Abstract: Tracking depression from multi-session counseling dialogues requires estimating both current symptom severity and how it changes across sessions. Yet

ProAct: Harnessing Streaming Motion Generation and Agentic Reasoning for Real-Time Embodied Social Interaction

arXiv:2602.14048v2 Announce Type: replace-cross Abstract: Real-time embodied social interaction places two equally demanding requirements on an agent: continuously generating fluent multimodal interac

ScoreMix: Synthetic Data Generation by Score Composition in Diffusion Models Improves Recognition

arXiv:2506.10226v3 Announce Type: replace-cross Abstract: Synthetic data generation is increasingly used in machine learning for training and data augmentation. Yet, current strategies often rely on e

safetyarxiv-cs-ai

SHELF: A Synthetic Harness for Multi-Task Bibliographic Benchmarking

arXiv:2609.03047v1 Announce Type: cross Abstract: Libraries and archives manage large collections with limited staff and computing budgets, yet common benchmarks do not systematically test their bibli

Tree species mapping in Denmark: A comparison of spectral-temporal features with geospatial foundation model embeddings

arXiv:2609.03480v1 Announce Type: cross Abstract: We map tree species across Denmark using National Forest Inventory plots and EO data, while evaluating the potential of foundation models for large-sc

User Perceptions vs. Proxy LLM Judges: Privacy and Helpfulness in LLM Responses to Privacy-Sensitive Scenarios

arXiv:2510.20721v4 Announce Type: replace-cross Abstract: Large language models (LLMs) are rapidly being adopted for tasks like drafting emails, summarizing meetings, and answering health questions. I

safetyarxiv-cs-ai

When Users Don't Ask: Benchmarking Context-Driven Memory Retrieval in Conversational Agents

arXiv:2609.03467v1 Announce Type: cross Abstract: Large language models (LLMs) are increas- ingly deployed as long-horizon conversational agents, motivating growing interest in mem- ory systems. Howev

Hardware

A Computational Comparison of Fourier Spectral Differentiation and Spatial Automatic Differentiation in Periodic Physics-Informed Neural Networks

arXiv:2609.02110v1 Announce Type: new Abstract: Physics-informed neural networks (PINNs) commonly evaluate the spatial derivatives appearing in partial differential equation residuals using automatic

hardwarearxiv-cs-lg

Auditory Illusion Benchmark for Large Audio Language Models

arXiv:2609.02277v1 Announce Type: cross Abstract: Perceptual illusions have long served as crucial probes into human cognition, revealing biases and limitations of perception. In the auditory domain,

Belief-Calibrated Optimization: An Explicit World Model for Agentic Optimization

arXiv:2609.01861v1 Announce Type: new Abstract: The performance of an LLM agent depends on the scaffold around a frozen model. A common way to improve that scaffold is to use a coding agent as an opti

Benchmarking Language Models for Statistical Problem Formulation

arXiv:2609.01982v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used as assistants for statistical and data science work, yet existing evaluations largely assume the anal

DiscoSign: Discourse-Aware Text to Sign Language Gloss Translation

arXiv:2609.02796v1 Announce Type: new Abstract: Sign language processing systems have traditionally operated at the sentence level, ignoring critical discourse phenomena fundamental to sign language c

researcharxiv-cs-cl

Disease Burden over Skin Tone: Decomposing the Dermatology-AI Generalization Gap

arXiv:2609.02111v1 Announce Type: cross Abstract: Dermatology artificial intelligence (AI) models are predominantly trained on light-skinned, cancer-focused image collections, yet they are increasingl

researcharxiv-cs-ai

DocHop: Benchmarking Out-of-domain Multi-hop Reasoning in Information-Dense Documents

arXiv:2609.02059v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have achieved strong performance on structured visual understanding tasks such as chart and document question a

Does Playing it Safe Count as Faithfulness? Reassessing LVLM Hallucination Mitigation Methods

arXiv:2609.01888v1 Announce Type: new Abstract: Recent inference-time hallucination mitigation methods for large vision-language models (LVLMs) report strong gains on hallucination benchmarks. However