Search: “evaluation”

Fairness Pruning: Locating Demographic Bias in GLU-MLP Layers via Differential Activations

arXiv:2607.28319v1 Announce Type: new Abstract: This work presents Fairness Pruning, a lightweight structural intervention method designed for the management and future mitigation of demographic bias

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

IGME: Efficient Chained Method Ensemble for Transferable Semantic Segmentation Attacks

DGX agent

arXiv:2607.27465v1 Announce Type: new Abstract: Semantic segmentation models are vulnerable to transferable adversarial perturbations, yet evaluating transfer attacks on dense prediction models can be

researcharxiv-cs-cv

31 Jul 2026

KernelGenBench: A Multi-Source and Multi-Chip Benchmark for LLM-based Kernel Generation

arXiv:2607.27231v1 Announce Type: cross Abstract: Large language models (LLMs) have significantly increased the demand for efficient accelerator kernels, but kernel development remains a highly specia

Tutorials

MetaRank: Task-Aware Metric Selection for Model Transferability Estimation

arXiv:2511.21007v2 Announce Type: replace Abstract: Selecting an appropriate pre-trained source model is a critical, yet computationally expensive, task in transfer learning. Model Transferability Est

tutorialsarxiv-cs-cv

Metrics vs Surveys: An Analysis for Human-Aligned Benchmarking in Social Robot Navigation

arXiv:2510.02941v2 Announce Type: replace Abstract: Social, also called human-aware, navigation is a key challenge for integrating mobile robots into human environments. The evaluation of such systems

safetyarxiv-cs-ro

Oracle-Budgeted Molecular Optimization with Short-Term Graph Memory

arXiv:2607.28437v1 Announce Type: new Abstract: Molecular optimization is commonly performed under a limited oracle budget, which makes deciding what to evaluate as important as deciding what to gener

Psych-ECA: A Reproducible Semi-Synthetic Benchmark for Synthetic Control Arms in Longitudinal Psychiatry

arXiv:2607.27224v1 Announce Type: cross Abstract: External and synthetic control arms (ECAs) are entering psychiatric drug development, but the field lacks a benchmark that evaluates the properties re

Sign Language Question Answering: A New Task, Benchmark, and Baseline for Sign Language Understanding

model-releasesarxiv-cs-cv

arXiv:2607.27826v1 Announce Type: cross Abstract: Recent advances in sign language (SL) understanding (SLU) have led to remarkable progress in tasks such as continuous SL recognition and SL translatio

State-Dependent Safety Failures in Multi-Turn Language Model Interaction

arXiv:2603.15684v2 Announce Type: replace-cross Abstract: Safety alignment in large language models is typically evaluated under isolated queries, yet real-world use is inherently multi-turn. Although

Local Ai

DHRCL:Training Code LLMs with Dense Hierarchical Rewards and Curriculum Learning

arXiv:2607.26457v1 Announce Type: new Abstract: Reinforcement learning is a natural post-training paradigm for code-oriented large language models because generated programs can be evaluated through p

local-aiarxiv-cs-lg

Do Methods Support the Claims? Intra-Paper Verification for Peer Review

arXiv:2607.26066v1 Announce Type: new Abstract: The growing volume of scientific submissions has motivated interest in using large language models (LLMs) to assist peer review. Existing automated nove

safetyarxiv-cs-cl

Field Codes for Distributed Coupling Samplers and Certified Empirical Transport

arXiv:2607.27078v1 Announce Type: cross Abstract: In this paper, we formulate three communication tasks for empirical optimal transport: distributed coupling sampling, cost-evaluable coupling output,

researcharxiv-cs-lg

HumanCLAW: Can Vision-Language Models Act Through a Body?

model-releasesarxiv-cs-cv

arXiv:2607.27180v1 Announce Type: new Abstract: Evaluating whether a vision-language model (VLM) can act through a physical body is challenging. The outcome of an action couples the VLM's decision wit

Prosody-driven Jailbreaks in Audio LLMs: A Controlled Study and Mechanistic Analysis

arXiv:2607.26541v1 Announce Type: cross Abstract: Audio-capable foundation models enable end-to-end spoken interaction, but they also introduce safety risks beyond transcript content. It remains uncle

Recover, Decode, Reguard: Guard-Agnostic Defense Amplification againstEncoded VLM Jailbreaks

arXiv:2607.26574v1 Announce Type: cross Abstract: Safety classifiers ('guards') are the dominant black-box defense for vision-language models, yet they judge an input's surface form, not its meaning:

safetyarxiv-cs-lg

Symphony of Bias: Exploring Gender Associations with Musical Instruments in Multimodal LLMs

arXiv:2607.26355v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly embedded in everyday life and widely used for information seeking, raising concerns about their potential

Addressable Recall Compaction for Long Context-Window Control in AI Agents

arXiv:2607.25066v1 Announce Type: new Abstract: Long-horizon LLM agents accumulate reasoning traces, actions, and tool observations that can eventually exceed a model's fixed context window. Existing

Agentic AI for Scientific Reasoning in Autonomous Quantum Sensing Experiments

arXiv:2607.25145v1 Announce Type: cross Abstract: We implement an agentic AI workflow built around a large language model (LLM) agent for autonomous experiments with nitrogen-vacancy (NV) centers in d

Critical slowing down for predicting controller induced loss of control in quadrotors

arXiv:2607.25370v1 Announce Type: cross Abstract: We develop a novel forecasting scheme to anticipate controller induced loss of control (LOC) in quadrotors and evaluate it on real LOC flight data fro

researcharxiv-cs-ro

From Deterministic to Generative Deep Learning for Urban Air Quality Reconstruction from Sparse Observations

arXiv:2607.25687v1 Announce Type: cross Abstract: Full-field reconstruction of air pollution is essential for evaluating pollution exposure and supporting public health decision-making. However, the c

Laplace-PSN-IRT: Uncertainty Quantification for Neural Item Response Theory Models of LLM Benchmarks

arXiv:2607.25257v1 Announce Type: cross Abstract: Item Response Theory (IRT) has recently been proposed as a framework for evaluating large language model (LLM) benchmarks by separating a model's late

Time-Frequency Consistency Learning for Robust Speech Deepfake Detection

arXiv:2607.17761v2 Announce Type: replace-cross Abstract: Recently, speech deepfake detection (SDD) has achieved significant progress. However, its robustness evaluation remains largely confined to co

Agentic Permissions Policy Algebra for Taint Confinement in LLM Agents

arXiv:2607.24625v1 Announce Type: cross Abstract: Autonomous LLM agents processing mixed-confidentiality data face severe security risks from prompt injection attacks and reasoning errors. While dynam

AI-generated Images Challenge Visual Trust in High-risk Scenarios

arXiv:2607.22745v1 Announce Type: cross Abstract: Rapid advances in image generation are eroding the evidentiary value of visual content in settings where authenticity can affect public safety and per

An Empirical Study of Feature Selection Granularity

arXiv:2607.24145v1 Announce Type: new Abstract: Feature selection aims to identify the most informative and relevant features for a given dataset, either in terms of capturing the underlying data stru

researcharxiv-cs-lg

Benchmarking LLMs for Verilog Design Flows

arXiv:2607.22759v1 Announce Type: cross Abstract: Large language models (LLMs) show promise in code generation, but their capabilities to produce correct, synthesizable hardware description language (

Agents

Coherent Without Grounding, Grounded Without Success: Observability and Epistemic Failure

arXiv:2603.28371v2 Announce Type: replace-cross Abstract: When an agent can articulate why something works, we typically take this as evidence of genuine understanding. This presupposes that effective

agentsarxiv-cs-ai

Designing Service Systems from Textual Evidence

arXiv:2603.10400v2 Announce Type: replace-cross Abstract: Designing service systems requires selecting among alternative configurations -- choosing the best chatbot variant, the optimal routing policy

Do Current Retrievers Cover All the Evidence? A Controlled Study of Conjunctive Cross-Page Retrieval

arXiv:2607.24165v1 Announce Type: cross Abstract: Finding a long document relevant to a multi-part request is not the same as establishing that it contains every requested piece of evidence. We study

Do LLM Debates Repeat Arguments Differently Across Languages?

arXiv:2607.23442v1 Announce Type: new Abstract: LLM debate is usually evaluated by final answers, but transcripts also reveal whether later turns develop new argumentative content or return to earlier

safetyarxiv-cs-cl

Earnings25: A Comprehensive 500-Hour Speech Benchmark for Finance

arXiv:2607.23813v1 Announce Type: cross Abstract: We introduce Earnings25, a finance-domain benchmark for evaluating automatic speech recognition (ASR) on English-language earnings calls under realist

Evolving from Lessons: Skill-Augmented Table Graph Reasoning for Operation-wise Table Question Answering

arXiv:2607.22633v1 Announce Type: new Abstract: Table Question Answering (TableQA) aims to reason over tables to answer user queries. Existing research treats all questions uniformly and evaluates sol

FilmBench: A Film-Grade Benchmark for Cinematic Video Generation

arXiv:2607.24241v1 Announce Type: cross Abstract: Progress in video generation keeps narrowing the visual gap between AI-generated and professionally produced footage, yet most benchmarks still draw p

Foundation Models and Fine-Tuning: Toward a New Generation of Models for Time Series Forecasting

arXiv:2607.23146v1 Announce Type: new Abstract: Inspired by recent breakthroughs in large language models for natural language processing, foundation models have emerged as a promising paradigm for ze

researcharxiv-cs-lg

From 'Help' to Helpful: A Hierarchical Assessment of LLMs in Mental e-Health Applications

arXiv:2602.18443v2 Announce Type: replace-cross Abstract: Psychosocial online counselling frequently encounters generic subject lines that impede efficient case prioritisation. This study evaluates el

Agents

Grading the Narrators: An Isnad-Rijal Framework for Claim-Level Provenance in Multi-Agent Knowledge Systems

arXiv:2607.24117v1 Announce Type: new Abstract: Modern multi-agent knowledge systems increasingly accumulate knowledge through chains of autonomous transformations rather than direct retrieval. Existi

agentsarxiv-cs-ai

Agents

Hybrid Advantage Estimation with Unified Critic for VLM Agentic Reinforcement Learning

arXiv:2607.23605v1 Announce Type: new Abstract: Large Vision-Language Models (VLMs) now act as agents in interactive environments, where success requires coherent reasoning and decision-making across

agentsarxiv-cs-ai

INS-ActBench: A Comprehensive Benchmark for Assessing Professional Actuarial Capability of Large Language Models

arXiv:2607.24273v1 Announce Type: new Abstract: Large Language Models (LLMs) have shown strong potential in financial reasoning, but existing benchmarks often evaluate domain knowledge, numerical reas

Applications

Investigating the Visual Cues of CNNs for Vascular Segmentation: A Case Study in Microscopy and Fundus Imaging

arXiv:2607.23371v1 Announce Type: cross Abstract: Vascular segmentation is a standard procedure for clinical diagnosis, yet the specific visual features determining model decisions remain poorly under

applicationsarxiv-cs-cv

Loss-Aware Feature-Map Pruning in Convolutional Neural Networks Using Multi-Armed Bandits

arXiv:2607.22564v1 Announce Type: new Abstract: Convolutional neural networks often contain redundant feature maps that increase storage and inference cost. This paper presents a loss-aware feature-ma

researcharxiv-cs-ai

MEMENTO: Memory-Guided Memetic Code-as-Policy Evolution

arXiv:2607.22832v1 Announce Type: new Abstract: Long-horizon embodied tasks require policies that execute many dependent actions before task success can be observed. Representing policies as executabl

safetyarxiv-cs-lg

MioFFAn: an Annotation Software for Formula Formalization with LLM Automation Capabilities

arXiv:2607.22552v1 Announce Type: new Abstract: The automatic translation of mathematical expressions in scientific literature into executable symbolic code (a process we refer to as Formula Formaliza

researcharxiv-cs-cl

ObsDriveBench: Benchmarking Multimodal Understanding under Adverse Weather with Observability Awareness

arXiv:2607.23537v1 Announce Type: new Abstract: Autonomous driving under adverse weather remains a critical challenge, yet existing vision-language benchmarks mainly evaluate under standard conditions

PatiGonit22K: A Comprehensive Dataset for Solving Complex Bengali MWPs

arXiv:2607.22859v1 Announce Type: new Abstract: Mathematical Word Problems (MWPs) are an important benchmark for evaluating natural language understanding and quantitative reasoning. Despite recent pr

Small, Bias-Free, Blind and Convolutional Denoiser: A compact ConvNeXt U-Net for blind Gaussian color-image denoising

model-releasesarxiv-cs-cv

arXiv:2607.22793v1 Announce Type: cross Abstract: We describe and evaluate BF-ConvUNeXt, a compact bias-free ConvNeXt U-Net for blind additive-white-Gaussian-noise color image denoising, combining fou

Spatial Reasoning in LLM Game Agents: Impact of Causal Context and Multi-Step Planning

arXiv:2607.22732v1 Announce Type: new Abstract: LLM-based game agents often perform poorly on more complex tasks. This work examines whether these failures are linked to limited spatial reasoning and

The Half-Lives of Generative-AI Evidence: A 40-Record Audit, a Claim-Currency Framework, and a Reflexive Case of Frontier-Model-Assisted Research

arXiv:2607.24032v1 Announce Type: new Abstract: Generative-AI evaluations can become historical before publication, yet calendar age does not affect every conclusion equally. This paper has two linked

Too much evidence, too little time: From text to actionable recommendations through multi-objective evidence reasoning

arXiv:2607.22574v1 Announce Type: new Abstract: Evidence-based clinical decision making requires specialists to identify, evaluate and synthesize relevant scientific literature. However, PubMed search

researcharxiv-cs-ai