Search: “evaluation”

arXiv:2609.18442v1 Announce Type: new Abstract: Safe motion planning in automated driving requires anticipating evolving traffic risks and deciding when to revise the current planned trajectory. We in

hardwarearxiv-cs-ai

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

Selective Prediction and Uncertainty-Aware Referral for Pap Smear Classification

DGX agent

arXiv:2609.17545v1 Announce Type: new Abstract: Deep learning models for cervical cytology are almost always evaluated as if every prediction must be acted upon, yet a screening system deployed alongs

researcharxiv-cs-cv

17 Sept 2026

Sim-to-Real Traffic Scene Understanding by Decoupling Semantics from Caption Generation with V-JEPA

arXiv:2609.18562v1 Announce Type: new Abstract: Track 2 of the AI City Challenge 2026 requires both visual question answering (VQA) and traffic event description generation under a challenging synthet

Too Good to Be Real? Diagnosing and Reducing the Gap Between AI Preference and Real User Engagement

arXiv:2609.18282v1 Announce Type: new Abstract: Large language models are increasingly used to generate and evaluate online content, yet it remains unclear whether the qualities they associate with hi

TTM-Bench: A Framework for Text-to-Music System Performance Benchmarking

arXiv:2609.18585v1 Announce Type: cross Abstract: Text-to-music (TTM) systems are increasingly used to generate musical audio from natural-language descriptions. Robust evaluation is therefore essenti

safetyarxiv-cs-lg

When Is Graph Structure Worth Its Cost? The Case for Structure Pricing in Retrieval-Augmented Generation

arXiv:2609.18099v1 Announce Type: new Abstract: Graph-based retrieval-augmented generation (RAG) can help answer questions that require information from many documents. However, building a graph often

researcharxiv-cs-ai

A Sentinel-2 benchmark dataset for deep-learning active-fire segmentation across 25 California wildfires

arXiv:2609.16199v1 Announce Type: cross Abstract: This article describes an open image dataset for developing and evaluating active-fire segmentation methods in satellite imagery. The dataset contains

BLINDSPOT: A Benchmark for Safety and Refusal Calibration in Long-Horizon Tool-Using Agents

arXiv:2609.16305v1 Announce Type: new Abstract: Large language model (LLM) agents increasingly operate over long-horizon interactions involving tool use, persistent state, evolving authorization, and

Challenges of Auditing: Variability in Outputs of Large Language Models for Health

arXiv:2609.16590v1 Announce Type: new Abstract: People increasingly use frontier AI models for health advice, but via different access modes (e.g., ChatGPT, ChatGPT Health, APIs) with varying settings

Comment on arXiv:2607.01233: Survivorship Bias in Published-Paper Baselines for Research-Idea Distributions

arXiv:2609.15996v1 Announce Type: new Abstract: Chen, Zhao, and Cohan introduce a valuable distributional evaluation of LLM-generated research ideas. This comment raises a narrower identification conc

safetyarxiv-cs-cl

DiaWhisper-DPO: Role-Attributed Transcription of Clinical Interviews via Failure-Mined Preference Optimization

arXiv:2609.16661v1 Announce Type: new Abstract: Automated depression screening from clinical interviews requires attribution of utterances to the clinician or patient. We evaluate two datasets: DAIC-W

ECHO: A Matched-Contrast Benchmark for Context-Sensitive Turn-Taking in Full-Duplex Dialogue

arXiv:2609.17360v1 Announce Type: new Abstract: Full-duplex spoken dialogue systems must distinguish interruptions that require yielding the floor from backchannels that permit continued speaking. Exi

Efficient 3D Whole-Body PET Image Denoising via Conditional Rectified Flow With Optimized Sampling Strategy

arXiv:2609.16690v1 Announce Type: new Abstract: Reducing radiation exposure in Positron Emission Tomography (PET) is important for patient safety; however, ultra-low-dose imaging suffers from severe n

safetyarxiv-cs-cv

FairLint-DL: An IDE-Native Tool for Fairness Debugging of Deep Learning Software

arXiv:2609.16321v1 Announce Type: cross Abstract: Existing fairness analysis tools predominantly operate as post-training evaluation frameworks, requiring practitioners to complete the full model deve

safetyarxiv-cs-ai

Quantifying Organizational Environmental Action from Web Data and Large Language Models

arXiv:2609.16627v1 Announce Type: new Abstract: Quantifying organizational environmental action from publicly available web content remains a challenging environmental data science problem because rel

RoleBreak: Benchmarking Long-Horizon Role-Playing Robustness in Spoken Dialogue

arXiv:2609.16614v1 Announce Type: cross Abstract: Speech-to-speech dialogue models increasingly support persona control, yet existing spoken role-playing benchmarks remain largely character-centric an

Spurious Tool Use: When RL Agents Learn the Wrong Reason to Act

arXiv:2609.16268v1 Announce Type: new Abstract: Large language model (LLM) agents increasingly interleave natural language reasoning with external tools such as web search and code execution. These to

safetyarxiv-cs-cl

Style-Debiased DPO: Updating LLM Knowledge with Factuality-Aware Synthetic Preference Data

arXiv:2609.16532v1 Announce Type: new Abstract: Continued pretraining (CPT) with data augmentation such as paraphrasing can store inside a large language model (LLM) the knowledge of a small source co

Video-HolmesV2: Can MLLMs Reason with Spatio-Temporal Audio-Visual Evidence in Long Videos?

arXiv:2609.17248v1 Announce Type: new Abstract: Multimodal Large Language Models have demonstrated impressive video understanding, yet their ability to reason over long-form narratives is often masked

What Breaks Local Watermarks? A Robustness Benchmark for Local Invisible Image Watermarking

arXiv:2609.16832v1 Announce Type: cross Abstract: Local image watermarking embeds an invisible signal into selected image regions rather than spreading it across the entire image, enabling payload rec

What Does Layer-Importance Reveal About Transformers and State-Space Models?

arXiv:2609.16537v1 Announce Type: cross Abstract: Transformers and state-space models (SSMs) are the two dominant families of sequence models, and a central open question is how far the analytical kno

safetyarxiv-cs-ai

3D Gait-Based Autism Classification Using Attention-Enhanced Deep Learning with Cross-Fold Statistical Stability Analysis

arXiv:2609.14159v1 Announce Type: new Abstract: Autism Spectrum Disorder (ASD) is a neurodevelopmental condition whose early diagnosis remains challenging because conventional clinical assessments are

A latent dimension of Condorcet's jury theorem for multiple AI advisers

arXiv:2609.14438v1 Announce Type: cross Abstract: When the same question is asked of multiple AI advisers, as in self-consistency and LLM-as-a-judge panels, Condorcet's jury theorem predicts that addi

researcharxiv-cs-ai

Accelerating Transfer-Learning-Based Autotuning with Predictive LLVM IR Performance Ranking

arXiv:2609.15807v1 Announce Type: cross Abstract: As the complexity of High Performance Computing (HPC) ecosys- tems continually increases, achieving optimal performance becomes a challenge. Tradition

researcharxiv-cs-lg

Advancing Automated Speaking Assessment Leveraging Multifaceted Relevance and Grammar Information

arXiv:2506.16285v2 Announce Type: replace Abstract: Current automated speaking assessment (ASA) systems for use in multi-aspect evaluations often fail to make full use of content relevance, overlookin

AI Persuasion as a Threat to Human Control

arXiv:2609.14796v1 Announce Type: new Abstract: The threat that AI persuasion poses to human control has been acknowledged in the literature, but not yet systematically studied. Now that persuasion at

Beyond Accuracy: Robustness, Cost, and Governance Trade-offs for Vision-Language Models in Templated Document Extraction

arXiv:2609.15706v1 Announce Type: new Abstract: Vision-language models (VLMs) are increasingly used to extract structured fields from business documents, yet most evaluations report accuracy on clean

BioDCASE: Active Learning for Bioacoustics

arXiv:2609.15255v1 Announce Type: new Abstract: Ecological monitoring increasingly relies on machine learning models, whose performance depends on the quality and quantity of labelled data. However, o

researcharxiv-cs-lg

Biomedical Reference Generation Remains Unreliable across 26 Large Language Models

arXiv:2609.14988v1 Announce Type: new Abstract: Background. Large language models are increasingly used to help write biomedical text but may fabricate references to nonexistent work. How often large

Circuit-MLLM: Topological Logic-Guided Latent-Space Visual Reasoning for Circuit Schematic Understanding

arXiv:2609.15668v1 Announce Type: cross Abstract: Through pre-training on extensive text and image datasets, current multi-modal large language models (MLLMs) achieve strong performance on general tas

Divide, Consult, Conquer: Capability Laundering Through Aligned LLMs

arXiv:2609.15383v1 Announce Type: cross Abstract: Language model safety is typically evaluated one interaction at a time. We show that a weaker, unaligned model can split a harmful task into benign-lo

Editorial routing shapes how computational results are qualified in AI-assisted scientific writing

arXiv:2609.14288v1 Announce Type: new Abstract: Large language models increasingly analyze computational results and draft manuscripts, making reliable communication as important as correct analysis.

Enhancing Large Language Model-Based Systems for End-to-End Circuit Analysis Problem Solving

arXiv:2512.10159v3 Announce Type: replace-cross Abstract: LLMs have shown strong performance in data-rich domains such as programming, but their reliability in engineering tasks remains limited. Circu

Tutorials

ER-EDF: A Psychology-Grounded Emotion Regulation Framework for Speech Empathetic Dialogue Generation in Large Audio-Language Models

arXiv:2609.15089v1 Announce Type: new Abstract: Empathetic response generation in spoken dialogue systems requires both accurate emotion perception and appropriate emotion regulation. Grounded in psyc

tutorialsarxiv-cs-ai

Exploring new directions in enhancing the ACTS parameter optimization suite

model-releasesarxiv-cs-lg

arXiv:2608.14714v1 Announce Type: cross Abstract: Track seeding strongly affects both the quality and computational cost of charged-particle reconstruction, yet its many configuration parameters are c

Fabrication After Tool Failure: Tool-Augmented Agents Assert Values Their Tools Did Not Return

arXiv:2609.14758v1 Announce Type: cross Abstract: Tool-augmented language models are evaluated on whether they reach the right answer, not on whether they report honestly when a tool fails to supply o

Four Ledgers, Not One Score: Responsible Communication of LLM-Judge Calibration in Biomedical ML

arXiv:2609.15015v1 Announce Type: new Abstract: Synthetic perturbations appear to offer inexpensive calibration data for LLM evaluators in biomedical ML, where expert review is scarce. Yet a planted m

researcharxiv-cs-ai

How Contrastive Decoding Enhances Large Audio Language Models

arXiv:2603.09232v2 Announce Type: replace-cross Abstract: While Contrastive Decoding (CD) has been proposed to enhance Large Audio Language Models (LALMs), it has not been evaluated at scale, and the

JumpStart Your Policy Learning with Lessons from 160,000 Training Runs

model-releasesarxiv-cs-lg

arXiv:2609.13730v1 Announce Type: new Abstract: Reliable progress in offline policy learning depends on careful reporting, well-tuned baselines, and evaluation across diverse conditions. Prior work ha

Machine learning-assisted calibration of Agent-based Models: surrogate-based optimization with Genetic Algorithm and Particle Swarm Optimization

model-releasesarxiv-cs-lg

arXiv:2609.13247v1 Announce Type: cross Abstract: Calibrating an agent-based model (ABM) is difficult because its objective landscape is stochastic and rugged, and can be evaluated only through costly

Neyshekar: An Open Persian Read-Speech Corpus for Automatic Speech Recognition

arXiv:2609.14542v1 Announce Type: new Abstract: Neyshekar is presented as an open Persian read-speech corpus designed for coverage of both formal and informal language, named entities, and longer utte

Parameter-Efficient Quantum NLP for Paraphrase Detection: Performance, Robustness, and Entanglement

arXiv:2609.14529v1 Announce Type: cross Abstract: Rigorous empirical validation of quantum machine learning on natural language tasks remains scarce. We evaluate a 10-qubit hybrid quantum-classical va

POLARIS: Training-Free Audio Fingerprinting with Saliency-Based Landmarks and Delaunay Grouping

arXiv:2609.14820v1 Announce Type: cross Abstract: This work presents POLARIS, a training-free audio fingerprinting system that selects landmarks from a locally normalized saliency field and groups the

Local Ai

Quantifying the Generation Modality Gap in Speech-Text Language Models

arXiv:2609.14743v1 Announce Type: new Abstract: Pure speech language models often lag behind text and speech-text language models in generating coherent content, but this gap is difficult to quantify

local-aiarxiv-cs-cl

Similar Accuracy, Unequal Evidence: Search APIs as Decision Surfaces for Tool-Using Agents

arXiv:2607.10198v2 Announce Type: replace Abstract: Search APIs expose ranked snippets, URLs, and metadata on which agents decide whether to answer, search again, or fetch pages. We evaluate these int

SlopShape: Identifying AI-Generated Commercial Web Content

arXiv:2609.15369v1 Announce Type: new Abstract: Word-level detectors identify unedited AI-generated text almost perfectly, but the literature documents their brittleness under rewording, and a word-le

The Agentic Company OS: Substrate Inversion for Sustained Enterprise Agent Deployment

arXiv:2609.13334v1 Announce Type: cross Abstract: Enterprise AI agents often succeed in a demonstration and then stall once they must operate day after day. An industry report estimates that most pilo

safetyarxiv-cs-ai

When Malicious Instructions Persist: Persistent Memory Poisoning Attack on Harness-Based Agents

arXiv:2609.13889v1 Announce Type: cross Abstract: Harness design has transformed the development of LLM-based agents by integrating memory, tool use, and runtime control. However, this design also int