Search: “evaluation”

HumanCLAW: Can Vision-Language Models Act Through a Body?

model-releasesarxiv-cs-cv

arXiv:2607.27180v1 Announce Type: new Abstract: Evaluating whether a vision-language model (VLM) can act through a physical body is challenging. The outcome of an action couples the VLM's decision wit

30 Jul 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Prosody-driven Jailbreaks in Audio LLMs: A Controlled Study and Mechanistic Analysis

DGX agent

arXiv:2607.26541v1 Announce Type: cross Abstract: Audio-capable foundation models enable end-to-end spoken interaction, but they also introduce safety risks beyond transcript content. It remains uncle

model-releasesarxiv-cs-cl

30 Jul 2026

Recover, Decode, Reguard: Guard-Agnostic Defense Amplification againstEncoded VLM Jailbreaks

arXiv:2607.26574v1 Announce Type: cross Abstract: Safety classifiers ('guards') are the dominant black-box defense for vision-language models, yet they judge an input's surface form, not its meaning:

safetyarxiv-cs-lg

30 Jul 2026

Symphony of Bias: Exploring Gender Associations with Musical Instruments in Multimodal LLMs

arXiv:2607.26355v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly embedded in everyday life and widely used for information seeking, raising concerns about their potential

30 Jul 2026

Addressable Recall Compaction for Long Context-Window Control in AI Agents

arXiv:2607.25066v1 Announce Type: new Abstract: Long-horizon LLM agents accumulate reasoning traces, actions, and tool observations that can eventually exceed a model's fixed context window. Existing

Agentic AI for Scientific Reasoning in Autonomous Quantum Sensing Experiments

arXiv:2607.25145v1 Announce Type: cross Abstract: We implement an agentic AI workflow built around a large language model (LLM) agent for autonomous experiments with nitrogen-vacancy (NV) centers in d

Critical slowing down for predicting controller induced loss of control in quadrotors

arXiv:2607.25370v1 Announce Type: cross Abstract: We develop a novel forecasting scheme to anticipate controller induced loss of control (LOC) in quadrotors and evaluate it on real LOC flight data fro

researcharxiv-cs-ro

From Deterministic to Generative Deep Learning for Urban Air Quality Reconstruction from Sparse Observations

arXiv:2607.25687v1 Announce Type: cross Abstract: Full-field reconstruction of air pollution is essential for evaluating pollution exposure and supporting public health decision-making. However, the c

Laplace-PSN-IRT: Uncertainty Quantification for Neural Item Response Theory Models of LLM Benchmarks

arXiv:2607.25257v1 Announce Type: cross Abstract: Item Response Theory (IRT) has recently been proposed as a framework for evaluating large language model (LLM) benchmarks by separating a model's late

Time-Frequency Consistency Learning for Robust Speech Deepfake Detection

arXiv:2607.17761v2 Announce Type: replace-cross Abstract: Recently, speech deepfake detection (SDD) has achieved significant progress. However, its robustness evaluation remains largely confined to co

Agentic Permissions Policy Algebra for Taint Confinement in LLM Agents

arXiv:2607.24625v1 Announce Type: cross Abstract: Autonomous LLM agents processing mixed-confidentiality data face severe security risks from prompt injection attacks and reasoning errors. While dynam

AI-generated Images Challenge Visual Trust in High-risk Scenarios

arXiv:2607.22745v1 Announce Type: cross Abstract: Rapid advances in image generation are eroding the evidentiary value of visual content in settings where authenticity can affect public safety and per

An Empirical Study of Feature Selection Granularity

arXiv:2607.24145v1 Announce Type: new Abstract: Feature selection aims to identify the most informative and relevant features for a given dataset, either in terms of capturing the underlying data stru

researcharxiv-cs-lg

Benchmarking LLMs for Verilog Design Flows

model-releasesarxiv-cs-lg

arXiv:2607.22759v1 Announce Type: cross Abstract: Large language models (LLMs) show promise in code generation, but their capabilities to produce correct, synthesizable hardware description language (

Coherent Without Grounding, Grounded Without Success: Observability and Epistemic Failure

arXiv:2603.28371v2 Announce Type: replace-cross Abstract: When an agent can articulate why something works, we typically take this as evidence of genuine understanding. This presupposes that effective

agentsarxiv-cs-ai

Designing Service Systems from Textual Evidence

arXiv:2603.10400v2 Announce Type: replace-cross Abstract: Designing service systems requires selecting among alternative configurations -- choosing the best chatbot variant, the optimal routing policy

Do Current Retrievers Cover All the Evidence? A Controlled Study of Conjunctive Cross-Page Retrieval

arXiv:2607.24165v1 Announce Type: cross Abstract: Finding a long document relevant to a multi-part request is not the same as establishing that it contains every requested piece of evidence. We study

Do LLM Debates Repeat Arguments Differently Across Languages?

arXiv:2607.23442v1 Announce Type: new Abstract: LLM debate is usually evaluated by final answers, but transcripts also reveal whether later turns develop new argumentative content or return to earlier

safetyarxiv-cs-cl

Earnings25: A Comprehensive 500-Hour Speech Benchmark for Finance

arXiv:2607.23813v1 Announce Type: cross Abstract: We introduce Earnings25, a finance-domain benchmark for evaluating automatic speech recognition (ASR) on English-language earnings calls under realist

Evolving from Lessons: Skill-Augmented Table Graph Reasoning for Operation-wise Table Question Answering

arXiv:2607.22633v1 Announce Type: new Abstract: Table Question Answering (TableQA) aims to reason over tables to answer user queries. Existing research treats all questions uniformly and evaluates sol

FilmBench: A Film-Grade Benchmark for Cinematic Video Generation

arXiv:2607.24241v1 Announce Type: cross Abstract: Progress in video generation keeps narrowing the visual gap between AI-generated and professionally produced footage, yet most benchmarks still draw p

Foundation Models and Fine-Tuning: Toward a New Generation of Models for Time Series Forecasting

arXiv:2607.23146v1 Announce Type: new Abstract: Inspired by recent breakthroughs in large language models for natural language processing, foundation models have emerged as a promising paradigm for ze

researcharxiv-cs-lg

From 'Help' to Helpful: A Hierarchical Assessment of LLMs in Mental e-Health Applications

arXiv:2602.18443v2 Announce Type: replace-cross Abstract: Psychosocial online counselling frequently encounters generic subject lines that impede efficient case prioritisation. This study evaluates el

Grading the Narrators: An Isnad-Rijal Framework for Claim-Level Provenance in Multi-Agent Knowledge Systems

arXiv:2607.24117v1 Announce Type: new Abstract: Modern multi-agent knowledge systems increasingly accumulate knowledge through chains of autonomous transformations rather than direct retrieval. Existi

agentsarxiv-cs-ai

How to improve agent skills with tracing and evals

A skill cut agent costs by 44% and latency by 56%, but also reduced answer completeness. Here’s how tracing, evals, and a long-running agent exposed and corrected the regression. The post How to impro

agentsarize-ai

Hybrid Advantage Estimation with Unified Critic for VLM Agentic Reinforcement Learning

arXiv:2607.23605v1 Announce Type: new Abstract: Large Vision-Language Models (VLMs) now act as agents in interactive environments, where success requires coherent reasoning and decision-making across

agentsarxiv-cs-ai

INS-ActBench: A Comprehensive Benchmark for Assessing Professional Actuarial Capability of Large Language Models

arXiv:2607.24273v1 Announce Type: new Abstract: Large Language Models (LLMs) have shown strong potential in financial reasoning, but existing benchmarks often evaluate domain knowledge, numerical reas

Applications

Investigating the Visual Cues of CNNs for Vascular Segmentation: A Case Study in Microscopy and Fundus Imaging

arXiv:2607.23371v1 Announce Type: cross Abstract: Vascular segmentation is a standard procedure for clinical diagnosis, yet the specific visual features determining model decisions remain poorly under

applicationsarxiv-cs-cv

Loss-Aware Feature-Map Pruning in Convolutional Neural Networks Using Multi-Armed Bandits

arXiv:2607.22564v1 Announce Type: new Abstract: Convolutional neural networks often contain redundant feature maps that increase storage and inference cost. This paper presents a loss-aware feature-ma

researcharxiv-cs-ai

MEMENTO: Memory-Guided Memetic Code-as-Policy Evolution

arXiv:2607.22832v1 Announce Type: new Abstract: Long-horizon embodied tasks require policies that execute many dependent actions before task success can be observed. Representing policies as executabl

safetyarxiv-cs-lg

MioFFAn: an Annotation Software for Formula Formalization with LLM Automation Capabilities

arXiv:2607.22552v1 Announce Type: new Abstract: The automatic translation of mathematical expressions in scientific literature into executable symbolic code (a process we refer to as Formula Formaliza

researcharxiv-cs-cl

ObsDriveBench: Benchmarking Multimodal Understanding under Adverse Weather with Observability Awareness

arXiv:2607.23537v1 Announce Type: new Abstract: Autonomous driving under adverse weather remains a critical challenge, yet existing vision-language benchmarks mainly evaluate under standard conditions

PatiGonit22K: A Comprehensive Dataset for Solving Complex Bengali MWPs

arXiv:2607.22859v1 Announce Type: new Abstract: Mathematical Word Problems (MWPs) are an important benchmark for evaluating natural language understanding and quantitative reasoning. Despite recent pr

Small, Bias-Free, Blind and Convolutional Denoiser: A compact ConvNeXt U-Net for blind Gaussian color-image denoising

model-releasesarxiv-cs-cv

arXiv:2607.22793v1 Announce Type: cross Abstract: We describe and evaluate BF-ConvUNeXt, a compact bias-free ConvNeXt U-Net for blind additive-white-Gaussian-noise color image denoising, combining fou

Spatial Reasoning in LLM Game Agents: Impact of Causal Context and Multi-Step Planning

arXiv:2607.22732v1 Announce Type: new Abstract: LLM-based game agents often perform poorly on more complex tasks. This work examines whether these failures are linked to limited spatial reasoning and

The Half-Lives of Generative-AI Evidence: A 40-Record Audit, a Claim-Currency Framework, and a Reflexive Case of Frontier-Model-Assisted Research

arXiv:2607.24032v1 Announce Type: new Abstract: Generative-AI evaluations can become historical before publication, yet calendar age does not affect every conclusion equally. This paper has two linked

Too much evidence, too little time: From text to actionable recommendations through multi-objective evidence reasoning

arXiv:2607.22574v1 Announce Type: new Abstract: Evidence-based clinical decision making requires specialists to identify, evaluate and synthesize relevant scientific literature. However, PubMed search

researcharxiv-cs-ai

Verbalized Particle Posterior: Bayesian Inference over Natural Language Hypotheses

arXiv:2607.22961v1 Announce Type: cross Abstract: Verbalized Machine Learning (VML) parameterizes a model as a natural-language prompt that an LLM evaluates as f(x; theta). The framework is interpreta

Where Is the Cost of Third-Party API Routers in Agentic Software Development?

arXiv:2607.23624v1 Announce Type: cross Abstract: Third-party API routers have become a common layer that unifies access across increasingly diverse LLM providers. In coding-agent workflows, high-auto

Benchmarking Fine-tuning and Retrieval Strategies for a Multimodal Language Model on the NRC Reactor Operator Licensing Examination

arXiv:2607.22067v1 Announce Type: new Abstract: The integration of large language models (LLMs) into the nuclear power industry requires outputs grounded in domain-specific knowledge. This study evalu

CausalForge: A Formally Grounded, Self-Improving Agentic Framework for Automated Research in Causal Inference

arXiv:2607.22511v1 Announce Type: cross Abstract: Automating theoretical research is constrained not only by the generation of candidate results, but also by their reliable evaluation. A common approa

agentsarxiv-cs-lg

Developing and Validating the Spanish Version of the Large Language Models Dependency Scale (LLM-D12-SP)

arXiv:2607.22041v1 Announce Type: new Abstract: There is a growing need for reliable and culturally validated instruments to assess psychological dependency on large language models (LLMs), particular

researcharxiv-cs-cl

GH-ESD: Grounded Hypothesis-Driven Error Slice Discovery for Instance-Level Vision Tasks

researchapple-ml-research

Systematic failures of vision models on semantically coherent subsets, known as error slices, reveal limitations in robustness and evaluation. Existing slice discovery approaches largely model slices

Optimization of time-consuming experimental conditions using pseudo-experimental data guided by adaptive polynomial regression

model-releasesarxiv-cs-lg

arXiv:2607.22238v1 Announce Type: new Abstract: Bayesian optimization (BO) is an optimization method that sequentially proposes the next candidate explainable variables for optimizing target variables

Progress Reward Modeling for Robotic Learning: A Comprehensive Survey

arXiv:2607.21655v1 Announce Type: cross Abstract: Robotic learning takes place in dynamic environments with large behavior spaces. A terminal success signal only tells the robot whether the task is co

researcharxiv-cs-cl

The 3D Mirage: Probing and Taming 3D Hallucinations

model-releasesarxiv-cs-cv

arXiv:2512.15423v2 Announce Type: replace Abstract: Monocular depth foundation models achieve remarkable generalization by learning large-scale semantic priors, but this creates a critical vulnerabili

AISE-Bench: A Full-Cycle Curated Benchmark for Information Seeking on Academic Knowledge Graphs

arXiv:2607.20498v1 Announce Type: new Abstract: Large language models (LLMs) augmented with tools are emerging as autonomous agents capable of using Web engine, APIs, and code to solve complex, long-h

24 Jul 2026

DFAH-Bench: Benchmarking Observable Agent Instability in Financial Decision-Making

arXiv:2607.20491v1 Announce Type: new Abstract: Standard evaluation benchmarks measure what a tool-using agent decides, not whether it arrives at that decision through the same process each time. We i