Search: “evaluation”

HugAgent: A Human Simulation Benchmark for Individual-Level Reasoning

arXiv:2510.15144v4 Announce Type: replace Abstract: Simulating human reasoning in open-ended tasks has long been a central aspiration in AI and cognitive science. While large language models now appro

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Lightweight Adaptation of EEG Foundation Models for Stroke Motor Imagery Decoding: Domain Shift and Subject-Level Robustness

DGX agent

arXiv:2609.00282v1 Announce Type: cross Abstract: Motor imagery (MI) electroencephalography (EEG) decoding could support post-stroke rehabilitation, but models developed on healthy cohorts may not tra

model-releasesarxiv-cs-lg

2 Sept 2026

Agents

Long-Horizon State Tracking in LLMs: Executing MD5 through a Deep Sequence of Dependent Tool Calls

arXiv:2609.00012v1 Announce Type: new Abstract: Long-horizon tasks remain uncommon in large language model (LLM) evaluation, and for a reason: when each step depends on the last, per-step accuracy tha

agentsarxiv-cs-ai

Agents

MAS-ProVe: Understanding the Process Verification of Multi-Agent Systems

arXiv:2602.03053v2 Announce Type: replace Abstract: Multi-Agent Systems (MAS) built on Large Language Models (LLMs) often exhibit high variance in their reasoning trajectories. Process verification, w

agentsarxiv-cs-ai

MemeBridge: A Dataset for Benchmarking and Mitigating the Bidirectional Cultural Gap in Meme Interpretation

arXiv:2609.00491v1 Announce Type: new Abstract: Communicating across cultures is inherently challenging, especially through culturally dense and ambiguous formats like memes. While people expect large

Neural means and kernel corrections for operator learning

model-releasesarxiv-cs-lg

arXiv:2609.00389v1 Announce Type: new Abstract: We combine neural network means with exact Matern kernel regressions of their residuals and of their learned features, and evaluate the pairing on two p

Overfitting Mitigation via Singular Value Decomposition in Minimum Bayes Risk Decoding

arXiv:2609.01135v1 Announce Type: new Abstract: Minimum Bayes Risk (MBR) decoding enables high-quality text generation by selecting the hypothesis that maximizes a utility metric over sampled pseudo-r

Peg-in-Bench: A Modular Benchmark for High-Precision Robotic Insertion

model-releasesarxiv-cs-ro

arXiv:2609.00906v1 Announce Type: new Abstract: High-precision insertion remains a fundamental challenge in robotic manipulation due to the strict alignment requirements and contact-rich interactions

Pix2Rep-v2: Data-Efficient Representation Learning for Dense Medical Imaging Applications

arXiv:2609.01427v1 Announce Type: new Abstract: Dense self-supervised learning (SSL) is a powerful paradigm for learning without annotations the local descriptors required to solve dense medical imagi

researcharxiv-cs-cv

Potential-Guided Particle Steering for Negation-Constrained Dexterous Grasping

model-releasesarxiv-cs-cv

arXiv:2609.00555v1 Announce Type: cross Abstract: Language-driven dexterous grasp models, such as DextER, perform well when instructions specify where to grasp, but we find they fail systematically wh

Predicting Program Exit Code with LLMs and Programming Language Semantics

arXiv:2609.00579v1 Announce Type: cross Abstract: Large language models (LLMs) have shown proficiency in various software engineering tasks, such as code generation and translation. However, a key lim

researcharxiv-cs-ai

RestoreBench: Can AI Agents Restore Power Flow Convergence?

arXiv:2609.00384v1 Announce Type: new Abstract: Large Language Model (LLM) agents increasingly automate multi-step engineering workflows through tool use, interpretation of intermediate results, and i

Seeing the World and the Self from Egocentric Video

arXiv:2609.01276v1 Announce Type: new Abstract: Complete 3D perception from egocentric video requires recovering the surrounding scene and the wearer's full-body motion in a shared metric frame. Exist

researcharxiv-cs-cv

Separating perception from reasoning in vision-language models: a model-free render ceiling for crystal structures

model-releasesarxiv-cs-cv

arXiv:2609.00663v1 Announce Type: new Abstract: Multimodal evaluations cannot say whether a vision-language model misread an image or misreasoned about it, because every existing method for separating

SMELT: Scaling Laws for Compute-Matched MoE Looped Transformers

arXiv:2609.01343v1 Announce Type: new Abstract: Looped Transformers increase effective depth by iterating a shared block of layers, but most evaluations compare at fixed model size, conflating archite

safetyarxiv-cs-lg

SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systems

arXiv:2609.00595v1 Announce Type: cross Abstract: Safe agents can fail together. Multi-agent LLM systems (MAS) move information, state, decisions, and authority across principal boundaries, creating f

The Assistant's Ideal Self

arXiv:2609.00304v1 Announce Type: new Abstract: Models express values and welfare-relevant self-reports, but it is unclear whether these outputs reflect stable preferences or a stable self. We thus in

The Safeguard Worked. Is the LLM System Safer?

arXiv:2609.00519v1 Announce Type: cross Abstract: Safeguards in deployed LLM services are evaluated by refusal, attack success, and policy violation rates. Those rates characterize how a control perfo

The Visual Insensitivity Gap: Diagnosing When Vision-Language Models Fail to Use Visual Evidence

arXiv:2609.00868v1 Announce Type: cross Abstract: Vision-language models are evaluated by aggregate accuracy on multimodal benchmarks, a practice that implicitly assumes the model uses its visual inpu

Uncovering the Computational Ingredients of Human-Like Representations in LLMs

arXiv:2510.01030v2 Announce Type: replace Abstract: The human ability to translate diverse perceptual and linguistic inputs into structured behavior has been thought to rest on learning robust represe

Verifiable Disaster Storylines and Causal Knowledge Graphs: A Citation-Grounded Pipeline from Heterogeneous Humanitarian Sources

arXiv:2609.00858v1 Announce Type: new Abstract: Effective humanitarian response depends on the rapid synthesis of heterogeneous, high-volume information sources - a task that routinely exceeds human a

researcharxiv-cs-ai

AgenTRIM: Tool Risk Mitigation for Agentic AI

arXiv:2601.12449v2 Announce Type: replace-cross Abstract: AI agents are autonomous systems that combine LLMs with external tools to solve complex tasks. While such tools extend capability, improper to

AI Historian: Helping historians organize and verify person-centred temporal clues from dispersed historical narratives

arXiv:2608.29133v1 Announce Type: new Abstract: History is not preserved in complete, continuous form. Accounts of a person's activities, relationships and historical contexts are scattered across tex

Anomaly Detection on Small Industrial Components via Vision-Based Tactile Sensing

arXiv:2608.30506v1 Announce Type: new Abstract: Automated inspection of small industrial components, including sub-centimetre-scale parts where defects are geometry-driven and poorly resolved by stand

researcharxiv-cs-ro

APIFlow-Bench: Measuring Whether Agents Survive Long, Dependent API Workflows

arXiv:2608.29128v1 Announce Type: new Abstract: Tool-using agents are commonly evaluated by a single bit: whether an end-to-end workflow completed. This metric fails to distinguish failures that matte

Audit Me If You Can: Query-Efficient Active Fairness Auditing of Black-Box LLMs

arXiv:2601.03087v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) exhibit systematic biases across demographic groups. Auditing is proposed as an accountability tool for black-box

safetyarxiv-cs-cl

Benchmarking Peptide-Protein Affinity Prediction Across Peptide and Target Shifts

arXiv:2608.30175v1 Announce Type: new Abstract: Peptide-protein affinity models are often evaluated with a single data split, obscuring whether they interpolate among measurements for observed targets

researcharxiv-cs-lg

Benevolent Bias in Multi-Turn Human-Agent Dialogue

arXiv:2608.29206v1 Announce Type: new Abstract: Bias in human-agent interaction can manifest not only through hostile language but also as benevolent bias, whereby unequal treatment hides behind a war

BIRD-History: A Benchmark for History-Driven Text-to-SQL with Fine-Grained Knowledge Annotations

arXiv:2608.29345v1 Announce Type: new Abstract: While recent Large Language Model (LLM)-based text-to-SQL systems achieve impressive performance on standard benchmarks, they struggle when user queries

Building an Adaptive Agentic Cybersecurity System with NVIDIA Nemotron

model-releasesnvidia-developer

NVIDIA and CrowdStrike evaluated an “agentic” cyber‑defense platform that fuses NVIDIA’s Nemotron open‑model family with Falcon telemetry for a continuous offense‑defense closed loop running at machin

Chain-of-Thought Faithfulness of Reasoning Models Varies with Where and How Preference Cues Are Delivered

arXiv:2608.29464v1 Announce Type: cross Abstract: Chain-of-thought (CoT) monitoring assumes that reasoning traces faithfully record the information that shapes a model's answer. Existing faithfulness

Comprehensive Assessment and Analysis for NSFW Content Erasure in Text-to-Image Diffusion Models

arXiv:2502.12527v2 Announce Type: replace Abstract: Text-to-image diffusion models have gained widespread application across various domains, demonstrating remarkable creative potential. However, the

safetyarxiv-cs-cv

Configurable Semantic Chunking for Biomedical Information Extraction in Retrieval-Augmented Generation

arXiv:2608.31139v1 Announce Type: new Abstract: BioMedRAG introduced retrieval-augmented generation with a learned chunk scorer for biomedical information extraction. However, it relies on fixed-size

Correctness Forensics for Batch Speculative Decoding: Diagnosing the Ragged Tensor Problem

arXiv:2510.22876v4 Announce Type: replace-cross Abstract: Inference optimizations are routinely evaluated by throughput alone, without verifying output correctness. We conduct a forensic analysis of b

Local Ai

Creation begins with understanding: LLMs as strategy designers for privacy-preserving tabular data synthesis

arXiv:2608.29674v1 Announce Type: new Abstract: Sharing tabular data in high-stakes domains is constrained by privacy regulations. Synthetic data offer a promising alternative, but deep generative mod

local-aiarxiv-cs-lg

Detecting and Guiding LLM-Generated Korean Poetry with Interpretable Form-level Features

arXiv:2608.28986v1 Announce Type: new Abstract: LLMs often struggle with modern Korean poetry, producing outputs that resemble 'line-broken prose.' We address two coupled tasks: detecting whether a Ko

Development of an Autonomous AI Coding Agent using Monte Carlo Tree Search (MCTS) and Gemini LLM Frameworks

arXiv:2608.29096v1 Announce Type: cross Abstract: The ongoing changes in software engineering requirements have created a substantial need for automated tools which can create secure source code from

Tutorials

DiffSAC: Diffusion-guided Sampling for Consensus-based Robust Estimation

arXiv:2608.30603v1 Announce Type: cross Abstract: Robust estimation is a core computer vision task frequently tackled using sample consensus. However, traditional methods suffer from inefficient sampl

tutorialsarxiv-cs-ai

Do MLLMs Really Understand Low-Resource Khmer Documents? A Pilot Study on Khmer Document VQA

arXiv:2608.28635v1 Announce Type: cross Abstract: Recent multimodal large language models (MLLMs) have advanced document understanding, visual question answering, and text extraction. However, their r

EvoBrowseComp: Benchmarking Search Agents on Evolving Knowledge

arXiv:2606.13120v2 Announce Type: replace Abstract: Search Agents -- large language models augmented with search tools -- have intensified the need for future-proof evaluation benchmarks. Existing ben

EvoSkill Injection: Red-Teaming Autonomous Skill Generation and Evolution in Self-Evolving Agents

arXiv:2608.30429v1 Announce Type: new Abstract: LLM-based agent systems increasingly adopt skill-based architectures to reduce repetitive reasoning costs and improve stable, efficient task execution.

Explainable Artificial Intelligence (XAI) in Computational Pathology: Definitions, Taxonomy, and Recommendations

arXiv:2608.28820v1 Announce Type: new Abstract: Computational pathology (CompPath) is transforming medicine by leveraging artificial intelligence (AI) algorithms to support diagnosis, prognosis, and t

Feature-Spectral Fragility in Segmentation: Dataset Dependence, Architecture-Specific Localization, and Spectral Correlates

arXiv:2608.29167v1 Announce Type: new Abstract: Robustness of segmentation models is commonly assessed through input-domain perturbations, while dependence on frequency content within learned feature

researcharxiv-cs-cv

HEAR Who Said What: Unlocking Speaker-Attributed Reasoning via Counterfactual Voice Grounding

arXiv:2608.29120v1 Announce Type: cross Abstract: Speech Language Models (SLMs) are increasingly deployed in multi-speaker environments, yet their ability to attribute speech to the correct speaker an

Hearing the Order: Investigating Position Bias in Large Audio-Language Models

arXiv:2510.00628v3 Announce Type: replace-cross Abstract: Large audio-language models (LALMs) are often used in tasks that involve reasoning over ordered options. An open question is whether their pre

safetyarxiv-cs-cl

HeTGB: A Comprehensive Benchmark for Heterophilic Text-Attributed Graphs

arXiv:2503.04822v2 Announce Type: replace-cross Abstract: Graph neural networks (GNNs) have demonstrated success in modeling relational data primarily under the assumption of homophily. However, many

How Prolific Sellers Self-Present: Dissecting the Communication Patterns of 1.6 Million Reverb Listings

arXiv:2608.29952v1 Announce Type: new Abstract: The current paper draws on self-presentation theory and warranting theory to evaluate how the language patterns in an online marketplace reflect seller

Investigating Social Bias Changes in Quantized Language Models

arXiv:2602.06181v2 Announce Type: replace Abstract: Post-training quantization reduces the memory needed to run large language models but alters their social biases in ways that aggregate metrics fail