Search: “evaluation”

Births are difficult to predict even with rich survey and full-population register data

arXiv:2609.01194v1 Announce Type: new Abstract: Major life events have proven difficult to predict. Does this reflect limits of theory, data, and algorithms, or the large role of chance? We examine on

researcharxiv-cs-lg

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

BS: Take the Hint - Interactive Multitracer PET/CT Lesion Segmentation with a Scribble-Conditioned ResEnc U-Net

DGX agent

arXiv:2609.01554v1 Announce Type: cross Abstract: Automated lesion segmentation in whole-body PET/CT is complicated by the variety of physiological tracer uptake patterns and by the differing appearan

researcharxiv-cs-ai

2 Sept 2026

CHARM: Character Hallucination for Multicultural Role Play Benchmark

arXiv:2609.01352v1 Announce Type: cross Abstract: Role-playing large language models (LLMs) are expected to adopt a character's style while also respecting that character's knowledge boundaries. Prior

ClinTraceBench: Source-Verifiable Longitudinal Clinical Reasoning over EHR-Derived Dialogues

arXiv:2609.01111v1 Announce Type: new Abstract: Clinical LLM assistants must reason over multi-visit patient trajectories, yet whether the compact history representations used to scale them---retrieva

CordisBench: Can Language Models Reason About Component Lifecycles in Dynamic Agent Harnesses?

arXiv:2609.01600v1 Announce Type: cross Abstract: Dynamic agent harnesses let language models change the software that shapes their own execution. This flexibility brings a new reasoning burden: a loc

Counterfactual Fragility Certificates: Exposing High-Confidence Brittleness under Structured Evidence Failure

arXiv:2609.00366v1 Announce Type: cross Abstract: High test accuracy and good aggregate calibration do not show whether an individual prediction is structurally supported by its evidence. In tabular d

researcharxiv-cs-ai

Applications

CoVer: Conflict-Aware Claim Verification

arXiv:2609.00508v1 Announce Type: new Abstract: Social media fact-checking has long been challenged by evidence-level and aggregation-level conflicts, where erroneous evidence mimics authoritative new

applicationsarxiv-cs-ai

Agents

EAAE: Energy-Aware Autonomous Exploration for UAVs in Unknown 3D Environments

arXiv:2603.15604v2 Announce Type: replace Abstract: Battery-powered multirotor unmanned aerial vehicles (UAVs) can rapidly map unknown environments, but mission performance is often limited by energy

agentsarxiv-cs-ro

EdiTikZ: Scientific Figure Editing from Revision Trajectories

arXiv:2609.01409v1 Announce Type: new Abstract: Vision-language models (VLMs) have shown strong performance in generating scientific figures from text or images. However, producing publication-ready f

EvoSCM: Scientific Belief Revision Through Causal Model Evolution and Experimentation

arXiv:2609.01526v1 Announce Type: new Abstract: Scientific agents must learn not only how to reason, but also what to believe. However, existing LLM agents typically express scientific hypotheses in f

Foundation models for electricity price forecasting and battery arbitrage: Can they replace market-specific forecasting models?

model-releasesarxiv-cs-lg

arXiv:2609.00089v1 Announce Type: new Abstract: Foundation models promise accurate forecasts with little or no task-specific training, but whether they can replace models designed specifically for ele

From Tool Use to Technological Agency: LoopCAT as a Local-First, Open-Source Tool for Translation Technology Education

arXiv:2609.00344v1 Announce Type: new Abstract: Translation students need to learn both how to use translation technologies and how to judge the choices those technologies make available. This article

HugAgent: A Human Simulation Benchmark for Individual-Level Reasoning

arXiv:2510.15144v4 Announce Type: replace Abstract: Simulating human reasoning in open-ended tasks has long been a central aspiration in AI and cognitive science. While large language models now appro

Lightweight Adaptation of EEG Foundation Models for Stroke Motor Imagery Decoding: Domain Shift and Subject-Level Robustness

model-releasesarxiv-cs-lg

arXiv:2609.00282v1 Announce Type: cross Abstract: Motor imagery (MI) electroencephalography (EEG) decoding could support post-stroke rehabilitation, but models developed on healthy cohorts may not tra

Agents

Long-Horizon State Tracking in LLMs: Executing MD5 through a Deep Sequence of Dependent Tool Calls

arXiv:2609.00012v1 Announce Type: new Abstract: Long-horizon tasks remain uncommon in large language model (LLM) evaluation, and for a reason: when each step depends on the last, per-step accuracy tha

agentsarxiv-cs-ai

Agents

MAS-ProVe: Understanding the Process Verification of Multi-Agent Systems

arXiv:2602.03053v2 Announce Type: replace Abstract: Multi-Agent Systems (MAS) built on Large Language Models (LLMs) often exhibit high variance in their reasoning trajectories. Process verification, w

agentsarxiv-cs-ai

MemeBridge: A Dataset for Benchmarking and Mitigating the Bidirectional Cultural Gap in Meme Interpretation

arXiv:2609.00491v1 Announce Type: new Abstract: Communicating across cultures is inherently challenging, especially through culturally dense and ambiguous formats like memes. While people expect large

Neural means and kernel corrections for operator learning

model-releasesarxiv-cs-lg

arXiv:2609.00389v1 Announce Type: new Abstract: We combine neural network means with exact Matern kernel regressions of their residuals and of their learned features, and evaluate the pairing on two p

Overfitting Mitigation via Singular Value Decomposition in Minimum Bayes Risk Decoding

arXiv:2609.01135v1 Announce Type: new Abstract: Minimum Bayes Risk (MBR) decoding enables high-quality text generation by selecting the hypothesis that maximizes a utility metric over sampled pseudo-r

researcharxiv-cs-cl

Peg-in-Bench: A Modular Benchmark for High-Precision Robotic Insertion

model-releasesarxiv-cs-ro

arXiv:2609.00906v1 Announce Type: new Abstract: High-precision insertion remains a fundamental challenge in robotic manipulation due to the strict alignment requirements and contact-rich interactions

Pix2Rep-v2: Data-Efficient Representation Learning for Dense Medical Imaging Applications

arXiv:2609.01427v1 Announce Type: new Abstract: Dense self-supervised learning (SSL) is a powerful paradigm for learning without annotations the local descriptors required to solve dense medical imagi

researcharxiv-cs-cv

Potential-Guided Particle Steering for Negation-Constrained Dexterous Grasping

model-releasesarxiv-cs-cv

arXiv:2609.00555v1 Announce Type: cross Abstract: Language-driven dexterous grasp models, such as DextER, perform well when instructions specify where to grasp, but we find they fail systematically wh

Predicting Program Exit Code with LLMs and Programming Language Semantics

arXiv:2609.00579v1 Announce Type: cross Abstract: Large language models (LLMs) have shown proficiency in various software engineering tasks, such as code generation and translation. However, a key lim

researcharxiv-cs-ai

RestoreBench: Can AI Agents Restore Power Flow Convergence?

arXiv:2609.00384v1 Announce Type: new Abstract: Large Language Model (LLM) agents increasingly automate multi-step engineering workflows through tool use, interpretation of intermediate results, and i

Seeing the World and the Self from Egocentric Video

arXiv:2609.01276v1 Announce Type: new Abstract: Complete 3D perception from egocentric video requires recovering the surrounding scene and the wearer's full-body motion in a shared metric frame. Exist

researcharxiv-cs-cv

Separating perception from reasoning in vision-language models: a model-free render ceiling for crystal structures

model-releasesarxiv-cs-cv

arXiv:2609.00663v1 Announce Type: new Abstract: Multimodal evaluations cannot say whether a vision-language model misread an image or misreasoned about it, because every existing method for separating

SMELT: Scaling Laws for Compute-Matched MoE Looped Transformers

arXiv:2609.01343v1 Announce Type: new Abstract: Looped Transformers increase effective depth by iterating a shared block of layers, but most evaluations compare at fixed model size, conflating archite

safetyarxiv-cs-lg

SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systems

arXiv:2609.00595v1 Announce Type: cross Abstract: Safe agents can fail together. Multi-agent LLM systems (MAS) move information, state, decisions, and authority across principal boundaries, creating f

The Assistant's Ideal Self

arXiv:2609.00304v1 Announce Type: new Abstract: Models express values and welfare-relevant self-reports, but it is unclear whether these outputs reflect stable preferences or a stable self. We thus in

The Safeguard Worked. Is the LLM System Safer?

arXiv:2609.00519v1 Announce Type: cross Abstract: Safeguards in deployed LLM services are evaluated by refusal, attack success, and policy violation rates. Those rates characterize how a control perfo

The Visual Insensitivity Gap: Diagnosing When Vision-Language Models Fail to Use Visual Evidence

arXiv:2609.00868v1 Announce Type: cross Abstract: Vision-language models are evaluated by aggregate accuracy on multimodal benchmarks, a practice that implicitly assumes the model uses its visual inpu

researcharxiv-cs-cl

Uncovering the Computational Ingredients of Human-Like Representations in LLMs

arXiv:2510.01030v2 Announce Type: replace Abstract: The human ability to translate diverse perceptual and linguistic inputs into structured behavior has been thought to rest on learning robust represe

Verifiable Disaster Storylines and Causal Knowledge Graphs: A Citation-Grounded Pipeline from Heterogeneous Humanitarian Sources

arXiv:2609.00858v1 Announce Type: new Abstract: Effective humanitarian response depends on the rapid synthesis of heterogeneous, high-volume information sources - a task that routinely exceeds human a

researcharxiv-cs-ai

AgenTRIM: Tool Risk Mitigation for Agentic AI

arXiv:2601.12449v2 Announce Type: replace-cross Abstract: AI agents are autonomous systems that combine LLMs with external tools to solve complex tasks. While such tools extend capability, improper to

AI Historian: Helping historians organize and verify person-centred temporal clues from dispersed historical narratives

arXiv:2608.29133v1 Announce Type: new Abstract: History is not preserved in complete, continuous form. Accounts of a person's activities, relationships and historical contexts are scattered across tex

Anomaly Detection on Small Industrial Components via Vision-Based Tactile Sensing

arXiv:2608.30506v1 Announce Type: new Abstract: Automated inspection of small industrial components, including sub-centimetre-scale parts where defects are geometry-driven and poorly resolved by stand

researcharxiv-cs-ro

APIFlow-Bench: Measuring Whether Agents Survive Long, Dependent API Workflows

arXiv:2608.29128v1 Announce Type: new Abstract: Tool-using agents are commonly evaluated by a single bit: whether an end-to-end workflow completed. This metric fails to distinguish failures that matte

Audit Me If You Can: Query-Efficient Active Fairness Auditing of Black-Box LLMs

arXiv:2601.03087v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) exhibit systematic biases across demographic groups. Auditing is proposed as an accountability tool for black-box

safetyarxiv-cs-cl

Benchmarking Peptide-Protein Affinity Prediction Across Peptide and Target Shifts

arXiv:2608.30175v1 Announce Type: new Abstract: Peptide-protein affinity models are often evaluated with a single data split, obscuring whether they interpolate among measurements for observed targets

researcharxiv-cs-lg

Benevolent Bias in Multi-Turn Human-Agent Dialogue

arXiv:2608.29206v1 Announce Type: new Abstract: Bias in human-agent interaction can manifest not only through hostile language but also as benevolent bias, whereby unequal treatment hides behind a war

BIRD-History: A Benchmark for History-Driven Text-to-SQL with Fine-Grained Knowledge Annotations

arXiv:2608.29345v1 Announce Type: new Abstract: While recent Large Language Model (LLM)-based text-to-SQL systems achieve impressive performance on standard benchmarks, they struggle when user queries

Chain-of-Thought Faithfulness of Reasoning Models Varies with Where and How Preference Cues Are Delivered

arXiv:2608.29464v1 Announce Type: cross Abstract: Chain-of-thought (CoT) monitoring assumes that reasoning traces faithfully record the information that shapes a model's answer. Existing faithfulness

Comprehensive Assessment and Analysis for NSFW Content Erasure in Text-to-Image Diffusion Models

arXiv:2502.12527v2 Announce Type: replace Abstract: Text-to-image diffusion models have gained widespread application across various domains, demonstrating remarkable creative potential. However, the

safetyarxiv-cs-cv

Configurable Semantic Chunking for Biomedical Information Extraction in Retrieval-Augmented Generation

arXiv:2608.31139v1 Announce Type: new Abstract: BioMedRAG introduced retrieval-augmented generation with a learned chunk scorer for biomedical information extraction. However, it relies on fixed-size

researcharxiv-cs-cl

Correctness Forensics for Batch Speculative Decoding: Diagnosing the Ragged Tensor Problem

arXiv:2510.22876v4 Announce Type: replace-cross Abstract: Inference optimizations are routinely evaluated by throughput alone, without verifying output correctness. We conduct a forensic analysis of b

Local Ai

Creation begins with understanding: LLMs as strategy designers for privacy-preserving tabular data synthesis

arXiv:2608.29674v1 Announce Type: new Abstract: Sharing tabular data in high-stakes domains is constrained by privacy regulations. Synthetic data offer a promising alternative, but deep generative mod

local-aiarxiv-cs-lg

Detecting and Guiding LLM-Generated Korean Poetry with Interpretable Form-level Features

arXiv:2608.28986v1 Announce Type: new Abstract: LLMs often struggle with modern Korean poetry, producing outputs that resemble 'line-broken prose.' We address two coupled tasks: detecting whether a Ko

Development of an Autonomous AI Coding Agent using Monte Carlo Tree Search (MCTS) and Gemini LLM Frameworks

arXiv:2608.29096v1 Announce Type: cross Abstract: The ongoing changes in software engineering requirements have created a substantial need for automated tools which can create secure source code from