Search: “evaluation”

Agentic System as Compressor: Quantifying System Intelligence in Bits

arXiv:2606.25960v1 Announce Type: new Abstract: Large language models are turning from isolated predictors into agentic systems: they call tools, retrieve evidence, obey environment constraints, use v

agentsarxiv-cs-ai

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

AI-Assisted Computational Reproducibility on the FABRIC Testbed

DGX agent

arXiv:2606.25879v1 Announce Type: cross Abstract: Computational reproducibility remains difficult despite being central to scientific research. In this paper, we show how the international FABRIC test

researcharxiv-cs-ai

25 Jun 2026

AMVICC: A Novel Benchmark for Cross-Modal Failure Mode Profiling for VLMs and IGMs

arXiv:2601.17037v2 Announce Type: replace Abstract: We investigate visual reasoning limitations of both multimodal large language models (MLLMs) and image generation models (IGMs) by creating a novel

Are Tabular Foundation Models Robust to Realistic Query Distribution Shifts in Microbiome Data?

arXiv:2606.24995v1 Announce Type: new Abstract: Tabular foundation models (TFMs) achieve strong performance on microbiome abundance data, yet their robustness under realistic distribution shift remain

ASSCG: Just-Right Gating over Chattering for Fast-Slow LLM Planning in Autonomous Driving

arXiv:2606.25509v1 Announce Type: cross Abstract: Large language models (LLMs) can improve autonomous driving planning but are costly to query online, and existing fast-slow planners often rely on han

agentsarxiv-cs-cv

Auto-Labelling-Based Domain Transfer for 3D Object Detection on a Bicycle-Mounted LiDAR Platform

arXiv:2606.25652v1 Announce Type: new Abstract: Reliable 3D perception of vulnerable road users (VRUs) such as cyclists and pedestrians is essential for their safety in urban traffic and a core requir

BCoughBench: Benchmarking Respiratory Acoustic Foundation Models Under Body-Coupled Wearable Sensor Conditions

arXiv:2606.25116v1 Announce Type: cross Abstract: Respiratory acoustic foundation models (FMs) are benchmarked exclusively on smartphone recordings, yet clinical deployment increasingly targets body-c

researcharxiv-cs-ai

Beyond Visual Forensics: Auditing Multimodal Robustness for Synthetic Medical Image Detection

arXiv:2606.25375v1 Announce Type: new Abstract: With the rapid adoption of generative AI, synthetic medical images pose growing risks, including diagnostic deception and insurance fraud. Although prio

Bias-Controlled Primal-Dual Natural Actor-Critic: Optimal Rates for Constrained Multi-Objective Average-Reward RL

arXiv:2606.25012v1 Announce Type: new Abstract: Many reinforcement learning (RL) problems in the infinite-horizon average-reward setting require optimizing multiple conflicting objectives while satisf

safetyarxiv-cs-lg

Closed-Loop Graph Algorithm Execution with Small Language Models: Step Accuracy and Rollout Reliability

arXiv:2606.24980v1 Announce Type: new Abstract: Small language models offer an efficient alternative to large-scale systems, but their ability to execute structured algorithms over multiple dependent

agentsarxiv-cs-lg

Colon-Bench: An Agentic Workflow for Scalable Dense Lesion Annotation in Full-Procedure Colonoscopy Videos

arXiv:2603.25645v2 Announce Type: replace-cross Abstract: Early screening via colonoscopy is critical for colon cancer prevention, yet developing robust AI systems for this domain is hindered by the l

Color Matters: Trigger Color Affects Success in Federated Backdoor Attacks

arXiv:2606.25858v1 Announce Type: cross Abstract: Federated learning is vulnerable to backdoor attacks in which malicious clients inject poisoned updates while preserving benign-task performance. In t

researcharxiv-cs-cv

Constraint Tax in Open-Weight LLMs: An Empirical Study of Tool Calling Suppression Under Structured Output Constraints

model-releasesarxiv-cs-cl

arXiv:2606.25605v1 Announce Type: new Abstract: Tool Calling and Structured Output are two core capabilities of modern Agent systems, yet their interaction under joint deployment conditions remains in

Cross-Attention Multimodal Learning for Predicting Response to Neoadjuvant Imatinib in Gastrointestinal Stromal Tumors: A Multicenter Retrospective Study

arXiv:2606.25579v1 Announce Type: cross Abstract: Background: Response to neoadjuvant imatinib in gastrointestinal stromal tumors (GISTs) is highly variable and cannot be reliably predicted using curr

researcharxiv-cs-cv

Diagnosing and Mitigating Compounding Failures in Agentic Persuasion via Taxonomic Strategy Retrieval

arXiv:2606.24976v1 Announce Type: cross Abstract: Foundation-model agents in multi-step, open-ended environments frequently suffer from compounding errors, where early mistakes contaminate long-horizo

agentsarxiv-cs-cl

Error-Aware TF-IDF Retrieval-Augmented Generation for ASR Error Correction

arXiv:2606.24915v1 Announce Type: new Abstract: End-to-end automatic speech recognition systems frequently hallucinate rare entities and domain-specific terms, especially in low-resource languages. Wh

researcharxiv-cs-cl

Evidential Perfusion Physics-Informed Neural Networks with Residual Uncertainty Quantification

arXiv:2603.09359v2 Announce Type: replace Abstract: Physics-informed neural networks (PINNs) have shown promise in addressing the ill-posed deconvolution problem in computed tomography perfusion (CTP)

Falcon: Functional Assembly and Language for Compositional Reasoning in X-ray

arXiv:2606.25701v1 Announce Type: new Abstract: Conventional vision-language models are largely object-centric, focusing on detecting and describing individual entities. In safety-critical X-ray bagga

FlowID : Enhancing Forensic Identification with Latent Flow-Matching Models

arXiv:2603.29591v2 Announce Type: replace Abstract: Every day, many people die under violent circumstances, whether from crimes, war, migration, or climate disasters. Medico-legal and law enforcement

FunPiQ: A New Benchmark for Pixel-Level Quality Assessment in Fundus Images

arXiv:2606.25915v1 Announce Type: new Abstract: Color fundus photography (CFP) is the most common ophthalmic imaging modality for large-scale screening. However, it is highly susceptible to degradatio

Laplace--Fisher Gate Identities for Optimal Matrix-Gated Blended Score Estimation

arXiv:2606.25169v1 Announce Type: cross Abstract: Sampling from an unnormalized target by reversing an Ornstein--Uhlenbeck diffusion requires the score of each noise-perturbed marginal. Tweedie's iden

Leaking Circuit Secrets: Gradient Leakage Attacks on Graph Neural Networks

arXiv:2606.25589v1 Announce Type: new Abstract: As graph neural networks (GNNs) become standard tools for critical tasks in circuit design and analysis, their security and privacy risks require carefu

researcharxiv-cs-lg

MacroLens: A Multi-Task Benchmark for Contextual Financial Reasoning under Macroeconomic Scenarios

arXiv:2606.24950v1 Announce Type: new Abstract: Financial decision-making is contextual: forecasting prices, valuing companies, and assessing event exposure weigh price history, accounting fundamental

Tutorials

Multifidelity-Augmented Gaussian Process Inputs for Surrogate Modeling from Scarce Data

arXiv:2603.22050v2 Announce Type: replace-cross Abstract: Supervised machine learning describes the practice of fitting a parameterized model to labeled input-output data. Supervised machine learning

tutorialsarxiv-cs-lg

Plausible but Wrong: A case study on Agentic Failures in Astrophysical Workflows

arXiv:2604.25345v2 Announce Type: replace Abstract: Agentic AI systems are increasingly being integrated into scientific workflows, yet their behavior under realistic conditions remains insufficiently

agentsarxiv-cs-ai

Project Auto-World: Towards Automated Benchmarking of Neural Relational Reasoners

arXiv:2606.24965v1 Announce Type: cross Abstract: Reasoning about relational structures remains a significant challenge for neural models, particularly when they must systematically apply learned know

RoboRouter: Training-Free Policy Routing for Robotic Manipulation

model-releasesarxiv-cs-ro

arXiv:2603.07892v4 Announce Type: replace Abstract: Research on robotic manipulation has developed a diverse set of policy paradigms, including vision-language-action (VLA) models, vision-action (VA)

Sarashina2.2-TTS: Tackling Kanji Polyphony in Japanese Speech Generation via Data Scaling and Targeted Data Synthesis

model-releasesarxiv-cs-cl

arXiv:2606.25369v1 Announce Type: cross Abstract: While large language model (LLM)-based text-to-speech (TTS) systems have achieved high-quality speech synthesis, most existing systems focus on Englis

SSMNBench: Diagnosing Image-based Cross-View Human-Object Understanding via Single-View Sufficiency and Multi-View Necessity

arXiv:2606.25634v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have shown remarkable progress in single-image perception, yet their ability to reason about complex cross-view

Stable-Shift: Biologically Structured Prediction of Transcriptional Responses to Unseen Gene Perturbations

arXiv:2606.24940v1 Announce Type: cross Abstract: Predicting transcriptional responses to genetic perturbations could reduce the experimental burden of functional genomics, but extrapolation to genes

TACO: Towards Task-Consistent Open-Vocabulary Adaptation in Video Recognition

arXiv:2606.25478v1 Announce Type: new Abstract: Adapting CLIP for open-vocabulary video recognition necessitates a delicate balance between newly acquired video knowledge and the pretrained generaliza

Local Ai

TL++: Accuracy and Privacy Preserving Traversal Learning for Distributed Intelligent Systems

arXiv:2606.25627v1 Announce Type: new Abstract: Distributed intelligent systems increasingly need to train across data silos without centralizing raw data. Federated learning keeps data local but can

local-aiarxiv-cs-lg

What Does It Mean to Break a Distillation Defense?

arXiv:2606.25059v1 Announce Type: cross Abstract: Black-box LLMs (accessible only via API) are vulnerable to distillation attacks, in which an attacker queries the model and trains a student on its ou

safetyarxiv-cs-ai

Yuvion VL: A Multimodal Foundation Model for Adversarial Content and AI Safety

arXiv:2606.25034v1 Announce Type: new Abstract: General-purpose models often struggle to reliably identify and understand real-world multimodal risks, largely due to the inherent multimodal adversaria

safetyarxiv-cs-cv

A Benchmark of State-Space Models vs. Transformers and BiLSTM-based Models for Historical Newspaper OCR

arXiv:2604.00725v2 Announce Type: replace Abstract: End-to-end OCR for historical newspapers remains challenging, as models must handle long text sequences, degraded print quality, and complex layouts

A Comparative Study of Bayesian Contextual Bandits for Real-Time Warehouse Sorter Optimization

arXiv:2606.23977v1 Announce Type: new Abstract: Efficient sorter diversion control of automated material handling systems (MHS) is critical for optimizing operational efficiency in large-scale warehou

safetyarxiv-cs-lg

A Pairwise Human-Human Interaction Detection and Recognition Framework for Mobile Service Robots

model-releasesarxiv-cs-ro

arXiv:2602.22346v2 Announce Type: replace Abstract: Autonomous mobile service robots, such as lawnmowers or cleaning robots, operating in human-populated environments need to reason about human-human

Adaptive Hebbian Memory Routing in Vision Transformers for Few-Shot Learning

arXiv:2606.24756v1 Announce Type: new Abstract: Few-shot image recognition requires models to adapt to new classes from a small labeled support set. Hebbian fast-weight memory can provide temporary as

researcharxiv-cs-cv

AI-Driven Predictive Maintenance with Environmental Context Integration for Connected Vehicles: Simulation, Benchmarking, and Field Validation

arXiv:2603.13343v3 Announce Type: replace-cross Abstract: Predictive maintenance for connected vehicles offers the potential to reduce unexpected breakdowns and improve fleet reliability, but most exi

Are Text-to-Image Models Inductivist Turkeys? A Counterfactual Benchmark for Causal Reasoning

arXiv:2606.24548v1 Announce Type: new Abstract: Text-to-image (T2I) generation models have achieved remarkable progress in producing visually realistic images from natural language prompts. Yet it rem

Assessing Distribution Shift in Human Activity Recognition for Domain Generalization

arXiv:2606.24781v1 Announce Type: new Abstract: While the field of Human Activity Recognition (HAR) continues to draw interest from researchers and advance in important ways, some key challenges remai

AutoSpec: Safety Rule Evolution for LLM Agents via Inductive Logic Programming

arXiv:2606.24245v1 Announce Type: cross Abstract: Large language model (LLM) agents increasingly automate complex tasks by integrating language models with external tools and environments. However, th

safetyarxiv-cs-ai

Benchmarking LLMs' Mathematical Reasoning with Unseen Random Variables Questions

arXiv:2501.11790v5 Announce Type: replace-cross Abstract: Recent studies have raised significant concerns regarding the reliability of current mathematics benchmarks, highlighting issues such as simpl

Beyond U-Net: A Latent-Representation-Aligned Skip-Free Backbone for Flow-Matching Speech Enhancement

arXiv:2606.24745v1 Announce Type: cross Abstract: Generative models, particularly diffusion and score-based approaches, have recently achieved strong performance in speech enhancement, but their itera

safetyarxiv-cs-ai

CAVEWOMAN: How Large Language Models Behave Under Linguistic Input and Output Compression

arXiv:2606.24083v1 Announce Type: cross Abstract: 'Talk short. Drop grammar. Save token.' This caveman style is widely promoted as a way to cut inference cost, but whether it actually saves anything d

CineCap: Structured Reasoning with Spatio-Temporal Anchors for Cinematographic Video Captioning

arXiv:2606.24636v1 Announce Type: new Abstract: Cinematographic captioning aims to describe how a video is filmed using professional film-language concepts such as camera movement, shot size, depth of

CN-NewsTTS Bench: a target-level automatic benchmark for raw-input Chinese news TTS pronunciation

model-releasesarxiv-cs-cl

arXiv:2606.24714v1 Announce Type: new Abstract: Chinese news text contains dense written forms such as scores, hyphenated model names, ranges, unit symbols, percentages, English abbreviations, and mix

Debate2Create: Robot Co-design via Multi-Agent LLM Debate

arXiv:2510.25850v3 Announce Type: replace-cross Abstract: We introduce Debate2Create (D2C), a multi-agent LLM framework that formulates robot co-design as structured, iterative debate grounded in phys

agentsarxiv-cs-lg