Search: “evaluation”

arXiv:2609.13057v1 Announce Type: new Abstract: Deep neural networks are expressive enough to contain worst-case targets that can be evaluated in polynomial time but cannot be learned in polynomial ti

tutorialsarxiv-cs-lg

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Kraken: LLM-based Speech-to-Speech Translation via Low-bitrate VQ and Dual-path Source Conditioning

DGX agent

arXiv:2609.13045v1 Announce Type: new Abstract: Speech-to-speech translation (S2ST) has advanced significantly with speech LLMs, offering the potential for joint optimization and preserving non-lingui

researcharxiv-cs-cl

14 Sept 2026

Learning Sparse Latent Predictive Foundation Model for Multimodal Neuroimaging

arXiv:2606.14957v4 Announce Type: replace Abstract: Brain MRIs are routinely acquired as multiple complementary sequences with unique contrast weighting, including T1-weighed imaging (T1w) anatomic an

Not All Speech Is Intent: Adaptive Self-Correcting Inference Layer for Post-ASR False Wake-Up

arXiv:2609.12469v1 Announce Type: new Abstract: False wake-up activations remain a persistent challenge in conversational AI. Speech phonetically similar to a device's wake word can produce a syntacti

Patient-Reported Survey Data Improve Prediction of Opioid Use Disorder

arXiv:2609.12224v1 Announce Type: new Abstract: Electronic health records (EHRs) may incompletely capture patient-reported factors associated with opioid use disorder (OUD). We evaluated whether surve

researcharxiv-cs-lg

Statistical Uncertainty Quantification for Aggregate Performance Metrics in Machine Learning Benchmarks

arXiv:2501.04234v2 Announce Type: replace-cross Abstract: Modern artificial intelligence is supported by machine learning models (e.g., foundation models) that are pretrained on a massive data corpus

SteerDuplex: Steerable Duplex Speech Dialogue Models

arXiv:2609.12623v1 Announce Type: cross Abstract: Full-duplex spoken dialogue models support low-latency turn taking, interruption handling, and backchanneling, yet a key capability remains underexplo

Mr.LHDR: A Benchmark for Multimodal Real-World Long-Horizon Deep Research Agents

arXiv:2609.11318v1 Announce Type: new Abstract: Deep research agents are increasingly capable of web search, tool use, multimodal evidence analysis, and information synthesis. However, existing benchm

12 Sept 2026

Prompt Revision as a Source of Cultural Bias in Text-to-Image Systems

arXiv:2609.11532v1 Announce Type: new Abstract: Commercial text-to-image systems silently revise user prompts before generating images, a step users typically cannot disable or even see. Yet, existing

12 Sept 2026

Sci-MMR: Benchmarking Multi-Step Evidence-Grounded Scientific Reasoning in Multimodal Agents

arXiv:2609.11243v1 Announce Type: new Abstract: Autonomous research agents are increasingly expected to search the literature, analyze experimental evidence, and generate scientific hypotheses. These

12 Sept 2026

Local Ai

AgentHijack: Visual Patch Attacks on Multimodal Computer-Use Agents

arXiv:2609.09212v1 Announce Type: cross Abstract: This paper presents an end-to-end evaluation framework for image-triggered command injection against computer-use agents (CUAs). The goal is to test w

local-aiarxiv-cs-ai

Safety

Are We Really Doing Few-Shot Learning? A Critical Examination of Pre-Training Assumptions

arXiv:2609.10851v1 Announce Type: new Abstract: Few-shot learning is commonly evaluated under protocols that pre-train a model on a large auxiliary set whose classes are disjoint from the target episo

safetyarxiv-cs-cv

Agents

Black-Box Red Teaming of Agentic AI: A Taxonomy-Driven Framework for Automated Risk Discovery

arXiv:2609.09647v1 Announce Type: new Abstract: Agentic systems are rapidly moving to production, where they read untrusted inputs, call tools with real permissions, and act autonomously, expanding th

agentsarxiv-cs-ai

Can AI Agents Detect and Repair Artifact Drift in Network Experiments?

arXiv:2609.09849v1 Announce Type: cross Abstract: In recent years, AI agents have evolved into capable assistants that carry out multi-step tasks in digital environments. The network systems community

Certifying Lower Bounds for Risk-Sensitive Reinforcement Learning under Adversarial State Perturbations

arXiv:2609.10866v1 Announce Type: new Abstract: Reinforcement learning (RL) agents deployed in real-world environments are often vulnerable to adversarial perturbations in state observations, creating

Applications

Combining Synthetic and Real Data for Low-Resource Historical OCR: A Manchu Case Study

arXiv:2609.11495v1 Announce Type: new Abstract: Manchu, now critically endangered, was one of the principal languages of the Qing empire (1636-1912), and its extensive archival record is increasingly

applicationsarxiv-cs-lg

Hardware

Composable CXL Memory as a Kubernetes-Native Shared Memory for LLM Serving

arXiv:2609.10790v1 Announce Type: cross Abstract: We present a Kubernetes Dynamic Resource Allocation (DRA) driver that makes composable CXL memory a schedulable cluster resource, and evaluate the res

hardwarearxiv-cs-lg

Diversity of EML-type operators

arXiv:2609.11210v1 Announce Type: cross Abstract: The discovery of the EML operator, sufficient to evaluate the standard explicit purely transcendental elementary functions, has led to considerable in

researcharxiv-cs-lg

From Repetition to Recognition: Inductive Discovery of Disinformation Narratives

arXiv:2609.11128v1 Announce Type: new Abstract: In disinformation datasets, narratives are often understood as recurring interpretive patterns that group texts under narrative labels. Recent work form

researcharxiv-cs-cl

JarvisGUI: Towards Cross-Device GUI Agents with Dynamic Task Composition

arXiv:2609.10451v1 Announce Type: new Abstract: Real-world GUI usage frequently involves workflows that span multiple devices and platforms, requiring the transfer of intermediate results, maintenance

ProMediConv: Benchmarking Proactive Conversational Agents in Legal Dispute Mediation

arXiv:2609.11101v1 Announce Type: new Abstract: Dispute mediation is essential for maintaining social harmony and resilience, yet developing skilled mediators is costly and time-consuming. Existing LL

Recognizing Is Not Reversing: A Controlled Inversion Test of Fact-Preserving News Framing

arXiv:2609.11769v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used to analyze and rewrite news, yet current framing studies mainly evaluate generation, detection, or wh

RelayS2S: A Dual-Path Speculative Generation for Real-Time Dialogue

arXiv:2603.23346v2 Announce Type: replace Abstract: Real-time spoken dialogue systems face a fundamental tension between latency and response quality. End-to-end speech-to-speech (S2S) models respond

SWRouter: Similarity-Contractive Window Routing for Multi-Turn Large Language Model Conversations

arXiv:2609.11414v1 Announce Type: new Abstract: Large language models exhibit complementary strengths, motivating routing methods that dispatch each query to the most suitable model. Although existing

researcharxiv-cs-cl

Towards Reliable Medical LLMs: Benchmarking and Enhancing Confidence Estimation of Large Language Models in Medical Consultation

arXiv:2601.15645v2 Announce Type: replace Abstract: Large-scale language models (LLMs) often offer clinical judgments based on incomplete information, increasing the risk of misdiagnosis. Existing stu

5-Dialects-BN: Unmasking the Impact of Transliteration on Bangla Dialectal LLMs

arXiv:2609.09964v1 Announce Type: new Abstract: Large Language Models (LLMs) have achieved remarkable progress across natural language processing (NLP) tasks, yet their capabilities degrade sharply fo

CityPlanner: A Sandbox Agent for Executable Urban Planning

arXiv:2609.09578v1 Announce Type: cross Abstract: Urban planning is a real-world spatial optimization problem that requires selecting feasible actions from large candidate spaces under practical objec

Development and Validation of a Physics-Guided Machine Learning Extrapolation Framework Using a Classical Transient Diffusion Benchmark

arXiv:2609.09912v1 Announce Type: new Abstract: Machine learning models used in engineering are typically trained within limited operating ranges, yet reliable predictions are often required beyond th

EmoMed: An Emotionally-Aware Agent for Multimodal Medical Support with Real-Time Information Retrieval

arXiv:2609.07194v1 Announce Type: new Abstract: We present EmoMed - a multimodal medical consultation agent that adapts its responses based on users' emotional states while maintaining clinical accura

Federated Learning for Cross-Modality Medical Image Segmentation via Augmentation-Driven Generalization

arXiv:2602.20773v2 Announce Type: replace Abstract: Purpose: Developing generalizable medical image segmentation models is challenging because imaging data are distributed across institutions and diff

In RAG We Trust? Measuring Robustness of Retrieval-Augmented Generation Under Document Poisoning

arXiv:2609.09243v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) grounds a language model in retrieved documents, which reduces hallucination but creates a new attack surface: if

Applications

It Is Not My Code Anymore

arXiv:2609.09022v1 Announce Type: cross Abstract: AI-assisted programming raises distinct questions about who produces code, who feels ownership of it, and who is responsible when it fails. This resea

applicationsarxiv-cs-ai

Measuring LLM Sycophancy under Sustained Multi-Turn Pressure

arXiv:2609.09090v1 Announce Type: cross Abstract: Large language models (LLMs) may abandon correct positions when users push back, exhibiting a failure mode known as sycophancy. Existing evaluations t

NOPE-HYPE: A Structured Simulation Workflow for Robust Speech-to-Text Across Diverse Acoustic Environments

arXiv:2609.10058v1 Announce Type: cross Abstract: Robust speech-to-text translation systems should perform reliably across diverse acoustic conditions, yet practical pipelines lack controllable tools

researcharxiv-cs-cl

Open Tabular Insight Extraction: Where Do We Stand, and Where Should We Go?

arXiv:2609.07629v1 Announce Type: cross Abstract: Democratizing access to the knowledge held in large corpora of tables such as data lakes is emerging as a central research challenge. Research in this

researcharxiv-cs-ai

Pretraining and Distillation Matter More Than Architecture Family for Label-Free Single-Cell Classification

model-releasesarxiv-cs-cv

arXiv:2609.09863v1 Announce Type: new Abstract: Choosing a deep learning architecture for label-free single-cell classification remains an open question, with microscopy benchmarks reporting conflicti

ProcArena: A Multi-Scenario Benchmark for LLMs on Direct and Interactive PL/SQL Development from Natural Language

arXiv:2609.06527v1 Announce Type: cross Abstract: Large language models (LLMs) have shown strong potential for translating natural-language (NL) requirements into PL/SQL programs, attracting increasin

SRD-GUARD: A Defense Framework of LLMs via Semantic Rewriting and Joint Multi-Model Scoring for Latent Intent Exposure

arXiv:2609.06540v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly deployed in safety-critical applications, yet jailbreak attacks can conceal harmful intent through role-

The Unreliable Progress Bar: Can LLM Agents Reliably Report Task Progress Throughout Execution?

arXiv:2609.08589v1 Announce Type: cross Abstract: Recent large language models can emit task-progress signals that agent frameworks use to decide whether a task should continue or stop, yet whether a

TTGBench: Benchmarking Topological Evolution and Semantic Drift in Text-attributed Temporal Graphs

arXiv:2609.08226v1 Announce Type: new Abstract: Temporal graph learning models the evolution of dynamic systems, where both structural interactions and semantic states change over time. However, exist

What Makes Adversarial Examples Transfer Across Deepfake Detectors?

arXiv:2609.10002v1 Announce Type: new Abstract: Deepfake detectors remain vulnerable to transfer-based black-box attacks, in which adversarial examples are generated on a source surrogate model and tr

Safety

XAI-Refine: An Automated Explanation-Knowledge Loop for Brain-Age Prediction

arXiv:2609.09388v1 Announce Type: new Abstract: Brain-age prediction models are commonly evaluated by predictive accuracy, yet accurate predictions alone do not establish that a model relies on reprod

safetyarxiv-cs-lg

A Comparative Study of GAN-Based Deep Learning Models for Pneumonia Detection in Chest X-Rays

arXiv:2609.06276v1 Announce Type: new Abstract: This study evaluates pneumonia classification in chest X-rays using VGG19, MobileNetV2, ResNet50, and a custom CNN, and explores Generative Adversarial

Answer-Distribution Trajectories: A Stochastic-Dynamics View of LLM Reasoning

arXiv:2609.09030v1 Announce Type: cross Abstract: Chain-of-thought reasoning provides a structured computation between a model's input and final answer. Yet it is often evaluated through endpoint accu

researcharxiv-cs-lg

CausalBN-Bench: A Comprehensive Benchmark for Causal Learning Capability of LLMs

arXiv:2404.06349v3 Announce Type: replace Abstract: The ability to understand causality significantly impacts the competence of large language models (LLMs) in output explanation and counterfactual re

CrowdTraj: A Benchmark for Dense Crowd Trajectory Prediction in Realistic Crowded Environments

model-releasesarxiv-cs-cv

arXiv:2609.07685v1 Announce Type: new Abstract: In real-world applications, pedestrian trajectory prediction models rely on inputs from detection and tracking systems. Prior trajectory prediction benc

Safety

Diagnosing LLM Reranker Behavior Under Fixed Evidence Pools

arXiv:2602.18613v2 Announce Type: replace Abstract: Standard reranking evaluations study how a reranker orders candidates returned by an upstream retriever. This setup couples ranking behavior with re

safetyarxiv-cs-lg

DriveMotion: A Large-Scale Multi-Source Benchmark for Driver Motion Sequence Modeling and Forecasting

model-releasesarxiv-cs-cv

arXiv:2609.08117v1 Announce Type: new Abstract: Driver motion can provide cues to ongoing behavior, attention, and near-term driving intent. However, most existing driver-centric datasets focus on rec