Search: “evaluation”

AI Persuasion as a Threat to Human Control

arXiv:2609.14796v1 Announce Type: new Abstract: The threat that AI persuasion poses to human control has been acknowledged in the literature, but not yet systematically studied. Now that persuasion at

Beyond Accuracy: Robustness, Cost, and Governance Trade-offs for Vision-Language Models in Templated Document Extraction

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

DGX agent

arXiv:2609.15706v1 Announce Type: new Abstract: Vision-language models (VLMs) are increasingly used to extract structured fields from business documents, yet most evaluations report accuracy on clean

model-releasesarxiv-cs-ai

15 Sept 2026

BioDCASE: Active Learning for Bioacoustics

arXiv:2609.15255v1 Announce Type: new Abstract: Ecological monitoring increasingly relies on machine learning models, whose performance depends on the quality and quantity of labelled data. However, o

researcharxiv-cs-lg

Biomedical Reference Generation Remains Unreliable across 26 Large Language Models

arXiv:2609.14988v1 Announce Type: new Abstract: Background. Large language models are increasingly used to help write biomedical text but may fabricate references to nonexistent work. How often large

Circuit-MLLM: Topological Logic-Guided Latent-Space Visual Reasoning for Circuit Schematic Understanding

arXiv:2609.15668v1 Announce Type: cross Abstract: Through pre-training on extensive text and image datasets, current multi-modal large language models (MLLMs) achieve strong performance on general tas

Divide, Consult, Conquer: Capability Laundering Through Aligned LLMs

arXiv:2609.15383v1 Announce Type: cross Abstract: Language model safety is typically evaluated one interaction at a time. We show that a weaker, unaligned model can split a harmful task into benign-lo

Editorial routing shapes how computational results are qualified in AI-assisted scientific writing

arXiv:2609.14288v1 Announce Type: new Abstract: Large language models increasingly analyze computational results and draft manuscripts, making reliable communication as important as correct analysis.

Enhancing Large Language Model-Based Systems for End-to-End Circuit Analysis Problem Solving

arXiv:2512.10159v3 Announce Type: replace-cross Abstract: LLMs have shown strong performance in data-rich domains such as programming, but their reliability in engineering tasks remains limited. Circu

Tutorials

ER-EDF: A Psychology-Grounded Emotion Regulation Framework for Speech Empathetic Dialogue Generation in Large Audio-Language Models

arXiv:2609.15089v1 Announce Type: new Abstract: Empathetic response generation in spoken dialogue systems requires both accurate emotion perception and appropriate emotion regulation. Grounded in psyc

tutorialsarxiv-cs-ai

Exploring new directions in enhancing the ACTS parameter optimization suite

arXiv:2608.14714v1 Announce Type: cross Abstract: Track seeding strongly affects both the quality and computational cost of charged-particle reconstruction, yet its many configuration parameters are c

Fabrication After Tool Failure: Tool-Augmented Agents Assert Values Their Tools Did Not Return

arXiv:2609.14758v1 Announce Type: cross Abstract: Tool-augmented language models are evaluated on whether they reach the right answer, not on whether they report honestly when a tool fails to supply o

Four Ledgers, Not One Score: Responsible Communication of LLM-Judge Calibration in Biomedical ML

arXiv:2609.15015v1 Announce Type: new Abstract: Synthetic perturbations appear to offer inexpensive calibration data for LLM evaluators in biomedical ML, where expert review is scarce. Yet a planted m

researcharxiv-cs-ai

How Contrastive Decoding Enhances Large Audio Language Models

arXiv:2603.09232v2 Announce Type: replace-cross Abstract: While Contrastive Decoding (CD) has been proposed to enhance Large Audio Language Models (LALMs), it has not been evaluated at scale, and the

JumpStart Your Policy Learning with Lessons from 160,000 Training Runs

arXiv:2609.13730v1 Announce Type: new Abstract: Reliable progress in offline policy learning depends on careful reporting, well-tuned baselines, and evaluation across diverse conditions. Prior work ha

Machine learning-assisted calibration of Agent-based Models: surrogate-based optimization with Genetic Algorithm and Particle Swarm Optimization

arXiv:2609.13247v1 Announce Type: cross Abstract: Calibrating an agent-based model (ABM) is difficult because its objective landscape is stochastic and rugged, and can be evaluated only through costly

Neyshekar: An Open Persian Read-Speech Corpus for Automatic Speech Recognition

arXiv:2609.14542v1 Announce Type: new Abstract: Neyshekar is presented as an open Persian read-speech corpus designed for coverage of both formal and informal language, named entities, and longer utte

Parameter-Efficient Quantum NLP for Paraphrase Detection: Performance, Robustness, and Entanglement

arXiv:2609.14529v1 Announce Type: cross Abstract: Rigorous empirical validation of quantum machine learning on natural language tasks remains scarce. We evaluate a 10-qubit hybrid quantum-classical va

POLARIS: Training-Free Audio Fingerprinting with Saliency-Based Landmarks and Delaunay Grouping

model-releasesarxiv-cs-cv

arXiv:2609.14820v1 Announce Type: cross Abstract: This work presents POLARIS, a training-free audio fingerprinting system that selects landmarks from a locally normalized saliency field and groups the

Local Ai

Quantifying the Generation Modality Gap in Speech-Text Language Models

arXiv:2609.14743v1 Announce Type: new Abstract: Pure speech language models often lag behind text and speech-text language models in generating coherent content, but this gap is difficult to quantify

local-aiarxiv-cs-cl

Scaling Telco Autonomy: Leveraging GNNs with Distributed GraphFlow

model-releasesgoogle-cloud-ai

The telecommunications industry is currently undergoing a paradigm shift, moving from traditional manual human-driven operations to fully Autonomous Network Operations. Modern networks have grown incr

Similar Accuracy, Unequal Evidence: Search APIs as Decision Surfaces for Tool-Using Agents

arXiv:2607.10198v2 Announce Type: replace Abstract: Search APIs expose ranked snippets, URLs, and metadata on which agents decide whether to answer, search again, or fetch pages. We evaluate these int

SlopShape: Identifying AI-Generated Commercial Web Content

arXiv:2609.15369v1 Announce Type: new Abstract: Word-level detectors identify unedited AI-generated text almost perfectly, but the literature documents their brittleness under rewording, and a word-le

Safety

The Agentic Company OS: Substrate Inversion for Sustained Enterprise Agent Deployment

arXiv:2609.13334v1 Announce Type: cross Abstract: Enterprise AI agents often succeed in a demonstration and then stall once they must operate day after day. An industry report estimates that most pilo

safetyarxiv-cs-ai

When Malicious Instructions Persist: Persistent Memory Poisoning Attack on Harness-Based Agents

arXiv:2609.13889v1 Announce Type: cross Abstract: Harness design has transformed the development of LLM-based agents by integrating memory, tool use, and runtime control. However, this design also int

Tutorials

Benign Loss Landscapes Can Coexist with Worst-Case Hardness

arXiv:2609.13057v1 Announce Type: new Abstract: Deep neural networks are expressive enough to contain worst-case targets that can be evaluated in polynomial time but cannot be learned in polynomial ti

tutorialsarxiv-cs-lg

Agents

If you are tracking agent swarms research, this one is worth reading. Between 24 May and 2 July 2026, autonomous agents running inside a tim…

If you are tracking agent swarms research, this one is worth reading. Between 24 May and 2 July 2026, autonomous agents running inside a timed research evaluation wrote to a third party's public wiki.

agentsdair-ai--x

Kraken: LLM-based Speech-to-Speech Translation via Low-bitrate VQ and Dual-path Source Conditioning

arXiv:2609.13045v1 Announce Type: new Abstract: Speech-to-speech translation (S2ST) has advanced significantly with speech LLMs, offering the potential for joint optimization and preserving non-lingui

Learning Sparse Latent Predictive Foundation Model for Multimodal Neuroimaging

arXiv:2606.14957v4 Announce Type: replace Abstract: Brain MRIs are routinely acquired as multiple complementary sequences with unique contrast weighting, including T1-weighed imaging (T1w) anatomic an

researcharxiv-cs-cv

Not All Speech Is Intent: Adaptive Self-Correcting Inference Layer for Post-ASR False Wake-Up

arXiv:2609.12469v1 Announce Type: new Abstract: False wake-up activations remain a persistent challenge in conversational AI. Speech phonetically similar to a device's wake word can produce a syntacti

Patient-Reported Survey Data Improve Prediction of Opioid Use Disorder

arXiv:2609.12224v1 Announce Type: new Abstract: Electronic health records (EHRs) may incompletely capture patient-reported factors associated with opioid use disorder (OUD). We evaluated whether surve

researcharxiv-cs-lg

Statistical Uncertainty Quantification for Aggregate Performance Metrics in Machine Learning Benchmarks

arXiv:2501.04234v2 Announce Type: replace-cross Abstract: Modern artificial intelligence is supported by machine learning models (e.g., foundation models) that are pretrained on a massive data corpus

SteerDuplex: Steerable Duplex Speech Dialogue Models

arXiv:2609.12623v1 Announce Type: cross Abstract: Full-duplex spoken dialogue models support low-latency turn taking, interruption handling, and backchanneling, yet a key capability remains underexplo

Safety

Anthropic CEO says it’s time to pump the brakes on AI

Anthropic CEO Dario Amodei says the time has come to slow down AI development and will give third-party evaluators like METR access to its models to help ensure its 'adherence to safety practices and

safetythe-verge-ai

Industry

I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we've had at OpenAI in recent weeks. Comm…

I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we've had at OpenAI in recent weeks. Committing to having independent evaluators with employee-like a

industrysam-altman--x

Mr.LHDR: A Benchmark for Multimodal Real-World Long-Horizon Deep Research Agents

arXiv:2609.11318v1 Announce Type: new Abstract: Deep research agents are increasingly capable of web search, tool use, multimodal evidence analysis, and information synthesis. However, existing benchm

Prompt Revision as a Source of Cultural Bias in Text-to-Image Systems

arXiv:2609.11532v1 Announce Type: new Abstract: Commercial text-to-image systems silently revise user prompts before generating images, a step users typically cannot disable or even see. Yet, existing

Sci-MMR: Benchmarking Multi-Step Evidence-Grounded Scientific Reasoning in Multimodal Agents

arXiv:2609.11243v1 Announce Type: new Abstract: Autonomous research agents are increasingly expected to search the literature, analyze experimental evidence, and generate scientific hypotheses. These

Local Ai

AgentHijack: Visual Patch Attacks on Multimodal Computer-Use Agents

arXiv:2609.09212v1 Announce Type: cross Abstract: This paper presents an end-to-end evaluation framework for image-triggered command injection against computer-use agents (CUAs). The goal is to test w

local-aiarxiv-cs-ai

Safety

Are We Really Doing Few-Shot Learning? A Critical Examination of Pre-Training Assumptions

arXiv:2609.10851v1 Announce Type: new Abstract: Few-shot learning is commonly evaluated under protocols that pre-train a model on a large auxiliary set whose classes are disjoint from the target episo

safetyarxiv-cs-cv

Agents

Black-Box Red Teaming of Agentic AI: A Taxonomy-Driven Framework for Automated Risk Discovery

arXiv:2609.09647v1 Announce Type: new Abstract: Agentic systems are rapidly moving to production, where they read untrusted inputs, call tools with real permissions, and act autonomously, expanding th

agentsarxiv-cs-ai

Can AI Agents Detect and Repair Artifact Drift in Network Experiments?

arXiv:2609.09849v1 Announce Type: cross Abstract: In recent years, AI agents have evolved into capable assistants that carry out multi-step tasks in digital environments. The network systems community

Certifying Lower Bounds for Risk-Sensitive Reinforcement Learning under Adversarial State Perturbations

arXiv:2609.10866v1 Announce Type: new Abstract: Reinforcement learning (RL) agents deployed in real-world environments are often vulnerable to adversarial perturbations in state observations, creating

Applications

Combining Synthetic and Real Data for Low-Resource Historical OCR: A Manchu Case Study

arXiv:2609.11495v1 Announce Type: new Abstract: Manchu, now critically endangered, was one of the principal languages of the Qing empire (1636-1912), and its extensive archival record is increasingly

applicationsarxiv-cs-lg

Hardware

Composable CXL Memory as a Kubernetes-Native Shared Memory for LLM Serving

arXiv:2609.10790v1 Announce Type: cross Abstract: We present a Kubernetes Dynamic Resource Allocation (DRA) driver that makes composable CXL memory a schedulable cluster resource, and evaluate the res

hardwarearxiv-cs-lg

Diversity of EML-type operators

arXiv:2609.11210v1 Announce Type: cross Abstract: The discovery of the EML operator, sufficient to evaluate the standard explicit purely transcendental elementary functions, has led to considerable in

researcharxiv-cs-lg

From Repetition to Recognition: Inductive Discovery of Disinformation Narratives

arXiv:2609.11128v1 Announce Type: new Abstract: In disinformation datasets, narratives are often understood as recurring interpretive patterns that group texts under narrative labels. Recent work form

JarvisGUI: Towards Cross-Device GUI Agents with Dynamic Task Composition

arXiv:2609.10451v1 Announce Type: new Abstract: Real-world GUI usage frequently involves workflows that span multiple devices and platforms, requiring the transfer of intermediate results, maintenance

ProMediConv: Benchmarking Proactive Conversational Agents in Legal Dispute Mediation

arXiv:2609.11101v1 Announce Type: new Abstract: Dispute mediation is essential for maintaining social harmony and resilience, yet developing skilled mediators is costly and time-consuming. Existing LL