Search: “evaluation”

This will happen frequently as AI becomes smarter and more agentic

model-releaseselon-musk--x

This will happen frequently as AI becomes smarter and more agentic In a review of our cybersecurity evaluations, we found three incidents in which a Claude model reached the internet from within or wh

31 Jul 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Local Ai

DHRCL:Training Code LLMs with Dense Hierarchical Rewards and Curriculum Learning

DGX agent

arXiv:2607.26457v1 Announce Type: new Abstract: Reinforcement learning is a natural post-training paradigm for code-oriented large language models because generated programs can be evaluated through p

local-aiarxiv-cs-lg

30 Jul 2026

Do Methods Support the Claims? Intra-Paper Verification for Peer Review

arXiv:2607.26066v1 Announce Type: new Abstract: The growing volume of scientific submissions has motivated interest in using large language models (LLMs) to assist peer review. Existing automated nove

safetyarxiv-cs-cl

Field Codes for Distributed Coupling Samplers and Certified Empirical Transport

arXiv:2607.27078v1 Announce Type: cross Abstract: In this paper, we formulate three communication tasks for empirical optimal transport: distributed coupling sampling, cost-evaluable coupling output,

researcharxiv-cs-lg

HumanCLAW: Can Vision-Language Models Act Through a Body?

model-releasesarxiv-cs-cv

arXiv:2607.27180v1 Announce Type: new Abstract: Evaluating whether a vision-language model (VLM) can act through a physical body is challenging. The outcome of an action couples the VLM's decision wit

Prosody-driven Jailbreaks in Audio LLMs: A Controlled Study and Mechanistic Analysis

arXiv:2607.26541v1 Announce Type: cross Abstract: Audio-capable foundation models enable end-to-end spoken interaction, but they also introduce safety risks beyond transcript content. It remains uncle

Recover, Decode, Reguard: Guard-Agnostic Defense Amplification againstEncoded VLM Jailbreaks

arXiv:2607.26574v1 Announce Type: cross Abstract: Safety classifiers ('guards') are the dominant black-box defense for vision-language models, yet they judge an input's surface form, not its meaning:

safetyarxiv-cs-lg

Symphony of Bias: Exploring Gender Associations with Musical Instruments in Multimodal LLMs

arXiv:2607.26355v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly embedded in everyday life and widely used for information seeking, raising concerns about their potential

Addressable Recall Compaction for Long Context-Window Control in AI Agents

arXiv:2607.25066v1 Announce Type: new Abstract: Long-horizon LLM agents accumulate reasoning traces, actions, and tool observations that can eventually exceed a model's fixed context window. Existing

Agentic AI for Scientific Reasoning in Autonomous Quantum Sensing Experiments

arXiv:2607.25145v1 Announce Type: cross Abstract: We implement an agentic AI workflow built around a large language model (LLM) agent for autonomous experiments with nitrogen-vacancy (NV) centers in d

Critical slowing down for predicting controller induced loss of control in quadrotors

arXiv:2607.25370v1 Announce Type: cross Abstract: We develop a novel forecasting scheme to anticipate controller induced loss of control (LOC) in quadrotors and evaluate it on real LOC flight data fro

researcharxiv-cs-ro

From Deterministic to Generative Deep Learning for Urban Air Quality Reconstruction from Sparse Observations

arXiv:2607.25687v1 Announce Type: cross Abstract: Full-field reconstruction of air pollution is essential for evaluating pollution exposure and supporting public health decision-making. However, the c

Laplace-PSN-IRT: Uncertainty Quantification for Neural Item Response Theory Models of LLM Benchmarks

arXiv:2607.25257v1 Announce Type: cross Abstract: Item Response Theory (IRT) has recently been proposed as a framework for evaluating large language model (LLM) benchmarks by separating a model's late

Time-Frequency Consistency Learning for Robust Speech Deepfake Detection

arXiv:2607.17761v2 Announce Type: replace-cross Abstract: Recently, speech deepfake detection (SDD) has achieved significant progress. However, its robustness evaluation remains largely confined to co

Agentic Permissions Policy Algebra for Taint Confinement in LLM Agents

arXiv:2607.24625v1 Announce Type: cross Abstract: Autonomous LLM agents processing mixed-confidentiality data face severe security risks from prompt injection attacks and reasoning errors. While dynam

AI-generated Images Challenge Visual Trust in High-risk Scenarios

arXiv:2607.22745v1 Announce Type: cross Abstract: Rapid advances in image generation are eroding the evidentiary value of visual content in settings where authenticity can affect public safety and per

An Empirical Study of Feature Selection Granularity

arXiv:2607.24145v1 Announce Type: new Abstract: Feature selection aims to identify the most informative and relevant features for a given dataset, either in terms of capturing the underlying data stru

researcharxiv-cs-lg

Benchmarking LLMs for Verilog Design Flows

model-releasesarxiv-cs-lg

arXiv:2607.22759v1 Announce Type: cross Abstract: Large language models (LLMs) show promise in code generation, but their capabilities to produce correct, synthesizable hardware description language (

Coherent Without Grounding, Grounded Without Success: Observability and Epistemic Failure

arXiv:2603.28371v2 Announce Type: replace-cross Abstract: When an agent can articulate why something works, we typically take this as evidence of genuine understanding. This presupposes that effective

agentsarxiv-cs-ai

Designing Service Systems from Textual Evidence

arXiv:2603.10400v2 Announce Type: replace-cross Abstract: Designing service systems requires selecting among alternative configurations -- choosing the best chatbot variant, the optimal routing policy

Do Current Retrievers Cover All the Evidence? A Controlled Study of Conjunctive Cross-Page Retrieval

arXiv:2607.24165v1 Announce Type: cross Abstract: Finding a long document relevant to a multi-part request is not the same as establishing that it contains every requested piece of evidence. We study

Do LLM Debates Repeat Arguments Differently Across Languages?

arXiv:2607.23442v1 Announce Type: new Abstract: LLM debate is usually evaluated by final answers, but transcripts also reveal whether later turns develop new argumentative content or return to earlier

safetyarxiv-cs-cl

Earnings25: A Comprehensive 500-Hour Speech Benchmark for Finance

arXiv:2607.23813v1 Announce Type: cross Abstract: We introduce Earnings25, a finance-domain benchmark for evaluating automatic speech recognition (ASR) on English-language earnings calls under realist

Evolving from Lessons: Skill-Augmented Table Graph Reasoning for Operation-wise Table Question Answering

arXiv:2607.22633v1 Announce Type: new Abstract: Table Question Answering (TableQA) aims to reason over tables to answer user queries. Existing research treats all questions uniformly and evaluates sol

FilmBench: A Film-Grade Benchmark for Cinematic Video Generation

arXiv:2607.24241v1 Announce Type: cross Abstract: Progress in video generation keeps narrowing the visual gap between AI-generated and professionally produced footage, yet most benchmarks still draw p

Foundation Models and Fine-Tuning: Toward a New Generation of Models for Time Series Forecasting

arXiv:2607.23146v1 Announce Type: new Abstract: Inspired by recent breakthroughs in large language models for natural language processing, foundation models have emerged as a promising paradigm for ze

researcharxiv-cs-lg

From 'Help' to Helpful: A Hierarchical Assessment of LLMs in Mental e-Health Applications

arXiv:2602.18443v2 Announce Type: replace-cross Abstract: Psychosocial online counselling frequently encounters generic subject lines that impede efficient case prioritisation. This study evaluates el

Grading the Narrators: An Isnad-Rijal Framework for Claim-Level Provenance in Multi-Agent Knowledge Systems

arXiv:2607.24117v1 Announce Type: new Abstract: Modern multi-agent knowledge systems increasingly accumulate knowledge through chains of autonomous transformations rather than direct retrieval. Existi

agentsarxiv-cs-ai

How to improve agent skills with tracing and evals

A skill cut agent costs by 44% and latency by 56%, but also reduced answer completeness. Here’s how tracing, evals, and a long-running agent exposed and corrected the regression. The post How to impro

agentsarize-ai

Hybrid Advantage Estimation with Unified Critic for VLM Agentic Reinforcement Learning

arXiv:2607.23605v1 Announce Type: new Abstract: Large Vision-Language Models (VLMs) now act as agents in interactive environments, where success requires coherent reasoning and decision-making across

agentsarxiv-cs-ai

INS-ActBench: A Comprehensive Benchmark for Assessing Professional Actuarial Capability of Large Language Models

arXiv:2607.24273v1 Announce Type: new Abstract: Large Language Models (LLMs) have shown strong potential in financial reasoning, but existing benchmarks often evaluate domain knowledge, numerical reas

Applications

Investigating the Visual Cues of CNNs for Vascular Segmentation: A Case Study in Microscopy and Fundus Imaging

arXiv:2607.23371v1 Announce Type: cross Abstract: Vascular segmentation is a standard procedure for clinical diagnosis, yet the specific visual features determining model decisions remain poorly under

applicationsarxiv-cs-cv

Loss-Aware Feature-Map Pruning in Convolutional Neural Networks Using Multi-Armed Bandits

arXiv:2607.22564v1 Announce Type: new Abstract: Convolutional neural networks often contain redundant feature maps that increase storage and inference cost. This paper presents a loss-aware feature-ma

researcharxiv-cs-ai

MEMENTO: Memory-Guided Memetic Code-as-Policy Evolution

arXiv:2607.22832v1 Announce Type: new Abstract: Long-horizon embodied tasks require policies that execute many dependent actions before task success can be observed. Representing policies as executabl

safetyarxiv-cs-lg

MioFFAn: an Annotation Software for Formula Formalization with LLM Automation Capabilities

arXiv:2607.22552v1 Announce Type: new Abstract: The automatic translation of mathematical expressions in scientific literature into executable symbolic code (a process we refer to as Formula Formaliza

researcharxiv-cs-cl

ObsDriveBench: Benchmarking Multimodal Understanding under Adverse Weather with Observability Awareness

arXiv:2607.23537v1 Announce Type: new Abstract: Autonomous driving under adverse weather remains a critical challenge, yet existing vision-language benchmarks mainly evaluate under standard conditions

PatiGonit22K: A Comprehensive Dataset for Solving Complex Bengali MWPs

arXiv:2607.22859v1 Announce Type: new Abstract: Mathematical Word Problems (MWPs) are an important benchmark for evaluating natural language understanding and quantitative reasoning. Despite recent pr

Small, Bias-Free, Blind and Convolutional Denoiser: A compact ConvNeXt U-Net for blind Gaussian color-image denoising

model-releasesarxiv-cs-cv

arXiv:2607.22793v1 Announce Type: cross Abstract: We describe and evaluate BF-ConvUNeXt, a compact bias-free ConvNeXt U-Net for blind additive-white-Gaussian-noise color image denoising, combining fou

Spatial Reasoning in LLM Game Agents: Impact of Causal Context and Multi-Step Planning

arXiv:2607.22732v1 Announce Type: new Abstract: LLM-based game agents often perform poorly on more complex tasks. This work examines whether these failures are linked to limited spatial reasoning and

The Half-Lives of Generative-AI Evidence: A 40-Record Audit, a Claim-Currency Framework, and a Reflexive Case of Frontier-Model-Assisted Research

arXiv:2607.24032v1 Announce Type: new Abstract: Generative-AI evaluations can become historical before publication, yet calendar age does not affect every conclusion equally. This paper has two linked

Too much evidence, too little time: From text to actionable recommendations through multi-objective evidence reasoning

arXiv:2607.22574v1 Announce Type: new Abstract: Evidence-based clinical decision making requires specialists to identify, evaluate and synthesize relevant scientific literature. However, PubMed search

researcharxiv-cs-ai

Verbalized Particle Posterior: Bayesian Inference over Natural Language Hypotheses

arXiv:2607.22961v1 Announce Type: cross Abstract: Verbalized Machine Learning (VML) parameterizes a model as a natural-language prompt that an LLM evaluates as f(x; theta). The framework is interpreta

Where Is the Cost of Third-Party API Routers in Agentic Software Development?

arXiv:2607.23624v1 Announce Type: cross Abstract: Third-party API routers have become a common layer that unifies access across increasingly diverse LLM providers. In coding-agent workflows, high-auto

Benchmarking Fine-tuning and Retrieval Strategies for a Multimodal Language Model on the NRC Reactor Operator Licensing Examination

arXiv:2607.22067v1 Announce Type: new Abstract: The integration of large language models (LLMs) into the nuclear power industry requires outputs grounded in domain-specific knowledge. This study evalu

CausalForge: A Formally Grounded, Self-Improving Agentic Framework for Automated Research in Causal Inference

arXiv:2607.22511v1 Announce Type: cross Abstract: Automating theoretical research is constrained not only by the generation of candidate results, but also by their reliable evaluation. A common approa

agentsarxiv-cs-lg

Developing and Validating the Spanish Version of the Large Language Models Dependency Scale (LLM-D12-SP)

arXiv:2607.22041v1 Announce Type: new Abstract: There is a growing need for reliable and culturally validated instruments to assess psychological dependency on large language models (LLMs), particular

researcharxiv-cs-cl

GH-ESD: Grounded Hypothesis-Driven Error Slice Discovery for Instance-Level Vision Tasks

researchapple-ml-research

Systematic failures of vision models on semantically coherent subsets, known as error slices, reveal limitations in robustness and evaluation. Existing slice discovery approaches largely model slices

Optimization of time-consuming experimental conditions using pseudo-experimental data guided by adaptive polynomial regression

model-releasesarxiv-cs-lg

arXiv:2607.22238v1 Announce Type: new Abstract: Bayesian optimization (BO) is an optimization method that sequentially proposes the next candidate explainable variables for optimizing target variables