Search: “evaluation”

Barrier Function Conformal Safety Clearance Certification with CVaR for Driving Trajectory Selection

arXiv:2608.26533v1 Announce Type: cross Abstract: Autonomous driving motion planners generate and select candidate trajectories while accounting for interactions with surrounding agents. However, thes

safetyarxiv-cs-ro

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Beyond Classification: Task-Dependent Learnability under Privacy-Motivated Image Transformations

DGX agent

arXiv:2608.27066v1 Announce Type: cross Abstract: Privacy-Enhancing Technologies (PETs) in computer vision often rely on noise or image perturbations to protect visual data while securely processing i

model-releasesarxiv-cs-ai

28 Aug 2026

CIFQA: A Deterministic Tool-Grounded Multi-Agent LLM Framework for Financial Query Answering

arXiv:2608.26114v1 Announce Type: new Abstract: Calculation-intensive financial question answering requires exact reasoning over structured rates, temporal conditions, numerical formulas, and rule-bas

Applications

Exploring the Role of LLMs in HPC Programming: A Survey

arXiv:2608.26110v1 Announce Type: cross Abstract: Large Language Models (LLMs) are emerging as promising assistants in High-Performance Computing (HPC), where programming remains complex and expertise

applicationsarxiv-cs-ai

GSM8K-V: Can Vision Language Models Solve Grade School Math Word Problems in Visual Contexts

arXiv:2509.25160v2 Announce Type: replace-cross Abstract: Mathematical reasoning is a key capability for vision-language models (VLMs), yet current benchmarks mainly evaluate text-based or explicitly

How Unlikely Is 'Unlikely'? Assessing Verbal Probability Perception Across Large Language Models

arXiv:2608.26327v1 Announce Type: cross Abstract: Large language models increasingly produce and interpret verbal probability expressions, yet whether these expressions carry consistent meaning across

Rethinking Image Processing for the Age of AI: A Problem-First Framework for Scientific Progress

arXiv:2608.26833v1 Announce Type: new Abstract: Modern AI has greatly expanded the capabilities of image processing. However, the ready availability of powerful models, public datasets, and benchmark

Translation Asymmetry in LLMs as a Data Augmentation Factor: A Case Study for 6 Romansh Language Varieties

arXiv:2603.25489v2 Announce Type: replace Abstract: Recent strategies for low-resource machine translation rely on LLMs to generate synthetic data based on text in higher-resource languages. We revisi

VIPER: An Expert-Curated Benchmark for Vision-Language Models in Veterinary Pathology

arXiv:2608.26382v1 Announce Type: new Abstract: Pathology vision-language models are advancing rapidly, yet existing benchmarks remain focused on human tissue, particularly oncology, leaving non-human

LibriBrain100: One Hundred Hours of Broad and Deep MEG Data for Neural Speech Decoding at Scale

arXiv:2608.25204v1 Announce Type: cross Abstract: We introduce LibriBrain100, a large-scale MEG dataset for speech decoding designed from the ground up for reproducible, standardised evaluation. Libri

Query-Side Attacks on GNN-Based KGQA: Tracing Failures from Entity Linking to Answer Generation

arXiv:2608.25922v1 Announce Type: new Abstract: GNN-based Knowledge Graph Question Answering (KGQA) pipelines process queries through four discrete stages: entity linking, subgraph retrieval, GNN reas

The 'Curse of Knowledge' in LLM Query Simulation: Concept Provenance for Tracing Answer-Side Intrusion

arXiv:2608.25245v1 Announce Type: cross Abstract: LLM-generated search queries are widely used to augment IR evaluation, yet they may contain concepts that presuppose answer-side document knowledge, v

researcharxiv-cs-cl

Think-Probe-Respond: Improving Large Language Models as Judges of Research Idea Novelty

arXiv:2608.25660v1 Announce Type: new Abstract: Automated novelty judgment can accelerate scientific discovery by enabling efficient evaluation, refinement, and comparison of research ideas. While lar

safetyarxiv-cs-cl

What Do Medical Vision-Language Models Learn in Radiology? Transfer, Alignment, and Source-Proxy Leakage Under Distribution Shift

arXiv:2608.25251v1 Announce Type: new Abstract: Medical vision-language models (VLMs) can appear reliable in-domain while failing when acquisition domain, paired supervision, or evaluation protocol ch

safetyarxiv-cs-cv

Comparative Assessment of Deep Learning Architectures for Underwater Subsurface Kelp Forest Segmentation with The Kelp-o-Tron

arXiv:2608.24594v1 Announce Type: new Abstract: Submerged kelp forests are vital coastal ecosystems that support marine biodiversity and ecosystem dynamics, yet accurate underwater kelp segmentation r

researcharxiv-cs-cv

Applications

FLARE: A Systematic, Uncertainty-Aware Framework for Evidence-Based Adoption of Artificial Intelligence in Healthcare

arXiv:2608.23643v1 Announce Type: new Abstract: Artificial intelligence is increasingly being introduced into healthcare workflows, yet most evaluations emphasize model accuracy rather than whether ad

applicationsarxiv-cs-ai

Gated Activation Steering for Reducing Sycophancy & Hallucination in Medical Question Answering

arXiv:2608.23666v1 Announce Type: new Abstract: Sycophancy and hallucination are persistent failure modes of Large Language Models (LLMs) across domains. However, it becomes particularly consequential

StateTune: Transforming LLM-Assisted EDA Flow Tuning into a Stateful, Closed-Loop Process

model-releasesarxiv-cs-lg

arXiv:2608.23601v1 Announce Type: cross Abstract: EDA flow parameter tuning is critical for quality-of-results~(QoR), yet the parameter space is large, tightly coupled, and full evaluations are prohib

SyPS: Measuring Sycophancy Prompt Sensitivity in Large Language Models

arXiv:2608.23837v1 Announce Type: new Abstract: Large language models (LLMs) are known to exhibit social sycophancy, often validating or agreeing with users in socially sensitive contexts. Existing ev

TurboT2VA: Fast Large-Scale Text-to-Video-Audio Generation via Score-Regularized Consistency Distillation

arXiv:2608.24674v1 Announce Type: new Abstract: Joint text-to-video-audio generation produces synchronized visual and acoustic content, but the long sampling trajectories and heterogeneous multimodal

BanglaVeilGuard: Cross-Script Safety Benchmarking and Lightweight Guardrails for Bangla Large Language Models

arXiv:2608.21880v1 Announce Type: new Abstract: Bangla large language model (LLM) safety is difficult to evaluate with English-centric or standard-script benchmarks because Bangla users routinely writ

Evidence-State Reliability Under Controlled Degradation: Parser-Validity Divergence in a Multi-Stage LLM Pipeline

arXiv:2608.21559v1 Announce Type: new Abstract: Multi-stage LLM pipelines can remain structurally valid even when evidence available to downstream stages becomes incomplete, compressed, or conflicting

researcharxiv-cs-cl

Measuring Activation Control in Large Language Models

arXiv:2608.21664v1 Announce Type: new Abstract: Safe deployment of increasingly capable models will likely come to rely on latent-space monitoring as a complement to behavioral evaluations, especially

Register Shifts Break LLM Safety: A Bengali Benchmark with Culturally Grounded Harms

arXiv:2608.22335v1 Announce Type: cross Abstract: Bengali is the seventh-most-spoken language globally, yet LLM safety evaluation remains overwhelmingly English-centric. We introduce BanglaSafe, a ben

Rethinking Communication Metrics: How Should We Measure Meaning?

arXiv:2608.21626v1 Announce Type: new Abstract: Semantic communication shifts the objective of communication systems from accurate symbol reconstruction toward meaning preservation, task accomplishmen

researcharxiv-cs-lg

Shaping the Evolutionary Dynamics of Robot Morphology via Adaptive Control Learning

arXiv:2608.23100v1 Announce Type: cross Abstract: Robot co-design via bi-level optimization couples within-lifetime controller learning for fitness evaluation with cross-generational morphological evo

safetyarxiv-cs-ai

Software Frameworks for Explainable AI in Time Series Classification: A Systematic Review

arXiv:2608.21449v1 Announce Type: new Abstract: Time series arise in a wide range of application domains and are analyzed using machine learning in decision-critical settings. Time series classificati

Text-ADBench: Text Anomaly Detection Benchmark Based on LLM Embeddings

arXiv:2507.12295v2 Announce Type: replace-cross Abstract: Text anomaly detection is a critical task in natural language processing (NLP), with applications spanning fraud detection, misinformation ide

What Does Activation Steering Control? Attribution Across Answer Encodings and Output-Sensitive Subspaces

arXiv:2608.22985v1 Announce Type: new Abstract: Activation steering is often evaluated under the answer encoding used to construct the direction. A reported gain may reflect the intended judgment or c

researcharxiv-cs-cl

A Dataset-Centric Benchmark of Deep Learning Methods for Grape Leaf Disease Classification and Detection

arXiv:2608.20608v1 Announce Type: new Abstract: Grape leaf disease recognition is important for precision agriculture, enabling early diagnosis, timely intervention, and improved vineyard management.

24 Aug 2026

Clarify-Then-Search: A Clarification Benchmark for Deep Search with End-to-End Nugget Restoration

arXiv:2608.20357v1 Announce Type: cross Abstract: Deep search is brittle on underspecified user queries: missing constraints such as time, location, scope, or definitions can lead to retrieval drift a

24 Aug 2026

OmniAssistBench: Assistant-style Interaction Benchmark for Omni-LLMs

arXiv:2608.21360v1 Announce Type: new Abstract: Recent omni-modal large language models (Omni-LLMs) show great potential as real-time video assistants, which continuously perceive environments and gui

24 Aug 2026

A Standardized Framework for Machine Learning in Power System Protection

arXiv:2608.20181v1 Announce Type: cross Abstract: Studies of machine-learning-based power-system protection increasingly report near-perfect scores, yet the meaning of those scores depends strongly on

ArmorOCR: Grounded Adversarial Visual Perception via Observation-Transferred Self-Distillation

arXiv:2608.20122v1 Announce Type: new Abstract: Large multimodal models (LMMs) have demonstrated strong OCR recognition capabilities, yet remain vulnerable to adversarial visual text that is readable

LLMs as Acquisition Policies for Finite-Pool Materials Optimization: A Controlled Study

arXiv:2608.19790v1 Announce Type: new Abstract: Discovering materials with desirable properties often requires searching large candidate spaces while experimental or computational evaluations remain c

PersonalBench: Measuring the Authorship Gap in LLM Personalization

arXiv:2608.19746v1 Announce Type: new Abstract: Personalized text generation aims to make LLMs write in a specific individual's style, yet existing benchmarks measure task accuracy or preference align

A FEM-Based Surrogate Modelling and Optimization Framework for Physics-Constrained Electromagnetic Coil Design

model-releasesarxiv-cs-lg

arXiv:2608.18903v1 Announce Type: new Abstract: This work evaluates surrogate-assisted optimization of a seven-parameter current-excited coil--core benchmark subject to geometric, manufacturing, and s

Automated Computational Energy Minimization of ML Algorithms using Constrained Bayesian Optimization

arXiv:2407.05788v2 Announce Type: replace-cross Abstract: Bayesian optimization (BO) is an efficient framework for optimization of black-box objectives when function evaluations are costly and gradien

LabDex: A Hierarchical Benchmark for Dexterous Manipulation in Laboratories

model-releasesarxiv-cs-ro

arXiv:2608.18618v1 Announce Type: new Abstract: Autonomous laboratories hold great promise for accelerating scientific discovery. To achieve this vision, robots are supposed to dexterously manipulate

Multimodal Rapport Estimation in Real-World HRI

arXiv:2608.18401v1 Announce Type: cross Abstract: Evaluating interaction quality in real-world HRI is an important challenge. If interaction quality can be estimated reliably, the results can be used

Agents

D^2ACCI: A Dual-Loop Diagnostic Protocol for Evidence-Preserving Agent Memory

arXiv:2608.17756v1 Announce Type: new Abstract: Memory is a key capability of LLM agents. Persistent memory extends this across sessions---enabling recall, revision, and personalization. Yet its multi

agentsarxiv-cs-ai

Do LLMs Know a Good Hypothesis When They See One? Logit-Based Energy Scoring Outperforms Prompted LLM-as-Judge for Scientific Hypothesis Ranking

arXiv:2608.17270v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used for scientific hypothesis generation. However, evaluating generated hypotheses remains a challenge fo

Effects of Answer Format Variation on Gender Bias in Large Language Models

arXiv:2608.17516v1 Announce Type: new Abstract: Gender bias or other social biases in large language models (LLMs) are frequently evaluated with question answering or survey benchmarks where the LLM n

OOD Detection for EEG-based Machine Learning in High-Risk Environments

model-releasesarxiv-cs-lg

arXiv:2608.17620v1 Announce Type: new Abstract: Machine learning models for electroencephalography (EEG) analysis show great promise across a wide range of applications, but their deployment in high-r

PTXBench: Benchmark and Adapt LLMs for GPU Kernel Optimization with Architecture-specific PTX

arXiv:2608.17379v1 Announce Type: cross Abstract: We introduce PTXBench, a benchmark for evaluating and adapting large language models (LLMs) to use architecture-specific PTX for GPU kernel optimizati

Quantifying Memorization and Privacy Risks in Genomic Language Models

arXiv:2603.08913v2 Announce Type: replace Abstract: Genomic language models (GLMs) have emerged as powerful tools for learning representations of DNA sequences, enabling advances in variant prediction

researcharxiv-cs-lg

Hardware

Structured Driving-State Narratives for Small Language Model-Based GNSS Spoofing Detection

arXiv:2608.17092v1 Announce Type: cross Abstract: Autonomous vehicles (AVs) depend on reliable Global Navigation Satellite System (GNSS) positioning. However, spoofed GNSS signals can induce plausible

hardwarearxiv-cs-ai

Agentic Data Cleaning Without a Clean Reference: An Experimental Study of Capabilities and Trade-offs