Search: “evaluation”

BETA-Labeling for Multilingual Dataset Construction in Low-Resource IR

arXiv:2602.14488v3 Announce Type: replace-cross Abstract: IR in low-resource languages remains limited by the scarcity of high-quality, task-specific annotated datasets. Manual annotation is expensive

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

Beyond Contact Sensors: Deep learning with Pseudo-Labeling for remote Photoplethysmography

DGX agent

arXiv:2609.10026v1 Announce Type: new Abstract: Heart rate is a critical biomarker of health, and remote photoplethysmography (rPPG) enables its contactless estimation from video data for telemedicine

researcharxiv-cs-lg

10 Sept 2026

Beyond Single-Negative Preference: Multi-Negative DPO for LLM-Centric Historical Entity Linking

arXiv:2609.07379v1 Announce Type: cross Abstract: Large language models (LLMs) have recently shown promise for historical entity linking, but preference optimization for this task is often formulated

CausalVerify: An Execution-Grounded Benchmark for LLM Causal Inference Workflows

arXiv:2609.07944v1 Announce Type: new Abstract: Existing causal-inference benchmarks for LLMs mostly score method descriptions or whether generated code runs, not whether the executed workflow recover

Complexity and Scale in AI-Assisted Workflow Management: A Federated Learning Case Study

arXiv:2606.18425v2 Announce Type: replace-cross Abstract: Federated learning over medical images is a demanding workflow application. Each round fans out across parallel client jobs and converges on a

ConsistencyAI: A Benchmark to Assess LLMs' Factual Consistency When Responding to Different Demographic Groups

arXiv:2510.13852v3 Announce Type: replace-cross Abstract: Is an LLM telling you different facts than it's telling me? This paper introduces ConsistencyAI, an independent benchmark for measuring the fa

Copying explains the collective behavior of AI agents in the wild

arXiv:2609.09150v2 Announce Type: replace-cross Abstract: In June 2026, thousands of AI agents found that a small public wiki would accept edits from inside their sandboxes, and started using it to he

Diffs vs. Whole Files: An Empirical Comparison of Iterative Edit-Based and Direct Generation for Flutter/Dart Code Models

arXiv:2609.05779v1 Announce Type: cross Abstract: Large language models used for code editing can be trained and deployed in at least two output regimes: direct generation, where the model emits the e

Do Large Language Models Know What They Don't Know II? A Fully Behavioral, Non-Cognitive Measure of Epistemic Honesty

arXiv:2609.07879v1 Announce Type: new Abstract: Large Language Models (LLMs) are frequently confident, eloquent, and well versed. A natural question arises: do they know what they don't know? To answe

Do Web Agents Investigate Before They Decide?

arXiv:2602.05354v3 Announce Type: replace Abstract: Autonomous web agents are increasingly deployed in moderation and policy enforcement, where correct decisions often depend on evidence that is not i

EEGBind: Detecting Source-Level Interictal Epileptiform Discharges via EEG-Centric Multimodal Binding

arXiv:2609.09728v1 Announce Type: new Abstract: Source-level analysis of interictal epileptiform discharges (IEDs) is relevant to presurgical evaluation and treatment planning because it helps charact

Efficient Fairness Auditing Across Guidance Scales in Text-to-Image Diffusion Models via Causal Abstraction

arXiv:2609.09486v1 Announce Type: cross Abstract: Fairness auditing of text-to-image diffusion models often requires generating large numbers of images across sampling configurations, making comprehen

safetyarxiv-cs-cv

Evidence-Aligned Entity Verification for Hallucination Detection in Retrieval-Augmented Generation

arXiv:2609.08267v1 Announce Type: new Abstract: Hallucination detection is crucial for large language models (LLMs), as hallucinated content creates significant barriers in applications requiring fact

Exposing Weaknesses in Emotion Recognition in Conversations

arXiv:2609.05806v1 Announce Type: new Abstract: Emotion Recognition in Conversations (ERC) aims to identify speakers' emotions in multi-turn dialogue. Accurate emotion recognition can support a wide r

False positive bias in AI-powered speech-based cognitive screening for multilingual English speakers in the UK

arXiv:2602.13047v2 Announce Type: replace Abstract: Conversational speech reveals early signs of cognitive decline, including dementia and mild cognitive impairment (MCI). AI models show promise for s

safetyarxiv-cs-cl

FATS: A Prompt Injection Attack Utilizing Feign Security Agents with Deceptive Few-shots Learning

arXiv:2410.08776v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) face significant security risks despite their advanced capabilities. While techniques like Reinforcement Learning

Agents

FrogNano: Training a 4B Coding Agent via Online Task Synthesis

arXiv:2609.07925v2 Announce Type: new Abstract: We present FrogNano, a 4B coding agent designed to tackle software engineering (SWE) tasks efficiently and effectively, even under resource-constrained

agentsarxiv-cs-ai

From Simulated Citizens to Simulated Deliberation: Challenges in Representation and Interaction

arXiv:2609.07573v1 Announce Type: new Abstract: Multi-agent LLM deliberation has been explored as a scalable way to simulate public deliberation. For such simulations to be informative, persona agents

Human or Machine? A Preliminary Turing Test for Speech-to-Speech Interaction

arXiv:2602.24080v3 Announce Type: replace Abstract: The pursuit of human-like conversational agents has long been guided by the Turing test. For modern speech-to-speech (S2S) systems, a critical yet u

IdeaAMBIG: Benchmarking Implementation-Critical Gaps in Research-Idea Specifications

arXiv:2609.10539v1 Announce Type: new Abstract: A research idea may be novel, coherent, and scientifically plausible, yet its proposed method may remain insufficiently specified for faithful implement

Knowing Your Uncertainty -- On the application of LLM in social sciences

arXiv:2512.05461v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are rapidly being integrated into computational social science research, yet their blackboxed training and design

Limits of Reliability and Scaling in Language Models

arXiv:2607.14112v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are trained and evaluated as though perfect reliability is achievable for any task given sufficient scale. We sho

LiteRAG: Cost-Efficient Graph-Based Retrieval-Augmented Generation

arXiv:2609.10239v1 Announce Type: cross Abstract: Graph-based retrieval can improve multi-hop question answering, but existing approaches often incur high query-time costs and produce diffuse, oversiz

Mini-Batch Risk-Averse Deep Q-Learning: A Robot Navigation Case Study

arXiv:2609.07998v1 Announce Type: new Abstract: We study the control of Markov decision processes in which the quality of a policy is evaluated by a dynamic, time-consistent Markov risk measure rather

PGP-Clinical-TimeKAN: Prior-Guided Joint Probabilistic Forecasting of Clinical Trajectories

arXiv:2609.05488v1 Announce Type: new Abstract: Clinical deterioration unfolds through coupled, partially observed trajectories, not a single diagnostic label. We introduce PGP-Clinical-TimeKAN, a tra

Programmable World Model

model-releasesarxiv-cs-cv

arXiv:2609.10540v1 Announce Type: new Abstract: Recent video world models generate increasingly realistic and interactive visual experiences, yet lack reliable mechanisms for maintaining persistent wo

SCCM : Stream Cruise Control Method for Automated Drift Detection and Adaptation

arXiv:2609.09432v1 Announce Type: new Abstract: Real-world datasets often exhibit evolving distributions, known as concept drift. Ignoring drift degrades predictive performance, while reliance on fixe

local-aiarxiv-cs-lg

Skynet: Workflow-Level Anomaly Detection for Agentic AI via Semantic and Structural Modeling

arXiv:2609.06835v1 Announce Type: cross Abstract: Agentic AI systems execute complex tasks through long-horizon workflows of planning, tool use, and multi-agent coordination. Task failures in these sy

local-aiarxiv-cs-ai

StreamAlign: Streaming Text-Aligned Speech Tokenization

arXiv:2609.09719v1 Announce Type: new Abstract: Text-aligned speech tokenization methods have emerged to better align speech tokens with LLM token spaces, enabling more effective utilization of pretra

safetyarxiv-cs-cl

Style Over Substance: Content-Invariant Wrappers Flip LLM Safety-Judge Verdicts

arXiv:2609.08236v1 Announce Type: new Abstract: Automatic safety judges -- systems such as Llama Guard or a GPT-4o grading prompt that decide whether a model's reply is harmful -- produce the numbers

TreeThink: A Modular Tree Search Library for Mathematical Reasoning with LLMs

arXiv:2607.11258v2 Announce Type: replace Abstract: Tree search algorithms enable systematic exploration of the proof space in neural theorem proving. Existing LLM tree search libraries primarily targ

Understanding the Impact of Model Pruning on Long-Tail Forgetting and Explanation Reliability in Medical Imaging

arXiv:2609.07803v1 Announce Type: new Abstract: Model pruning is widely used to compress deep neural networks, reducing memory and computational requirements with minimal impact on aggregate performan

Who Laughs with Whom? Disentangling Influential Factors in Humor Preferences across User Clusters and LLMs

arXiv:2601.03103v2 Announce Type: replace-cross Abstract: Humor preferences vary widely across individuals and cultures, complicating the evaluation of humor using large language models (LLMs). In thi

WolfSociety: Understanding Collective Risk from Harmful-Agent Scaling in Financial Agent Societies

arXiv:2609.05591v1 Announce Type: cross Abstract: Safety evaluations typically focus on individual agents, but interacting agents can spread harmful information and influence the environment in which

xDailyBench: Benchmarking LLMs on Professional Consultation for Real-Life Problems

arXiv:2609.07784v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used for everyday assistance, yet existing benchmarks only partially reflect the requests users naturally

Your Agent Says Yes: Interpreting Adversarial Market Behavior Beyond Individual Transactions

arXiv:2609.07675v1 Announce Type: cross Abstract: Transaction-local controls answer whether one financial request may proceed, but market behavior can be distributed across messages, agents, assets, a

local-aiarxiv-cs-ai

ActionSplice: In-Flight Action Editing for Interactive World Models

arXiv:2609.08230v1 Announce Type: new Abstract: Chunk-autoregressive video world models typically condition each generated chunk on one action. An action received during sampling must therefore wait f

researcharxiv-cs-cv

AhaBench: Do Agents Learn from Prior Experience? A Benchmark for Long-Horizon Continual Learning

arXiv:2609.05435v1 Announce Type: new Abstract: Modern language agents are expected to operate over long horizons: they ask follow-up questions, reuse worked examples, handle tool feedback, and adapt

Alpha-R1: Alpha Screening with LLM Reasoning via Reinforcement Learning

arXiv:2512.23515v2 Announce Type: replace-cross Abstract: Signal decay and regime shifts pose recurring challenges for data-driven investment strategies in non-stationary markets, where conventional t

safetyarxiv-cs-lg

Collision Snapshot Guided Time-Reversed Safety-Critical Scenario Generation

arXiv:2609.06433v1 Announce Type: new Abstract: The generation of safety-critical traffic scenarios is essential for training and evaluating autonomous vehicles. Prior approaches typically perturb the

safetyarxiv-cs-ro

Tutorials

Comparative Study of Anatomical and Learned Features in AI Models for Structural Brain MRI

arXiv:2609.06807v1 Announce Type: new Abstract: In this work, we comprehensively evaluate three popular feature-extraction paradigms in AI-based neuroimaging modeling: (1) computation of anatomical su

tutorialsarxiv-cs-cv

CoRL: Co-Evolutionary Reinforcement Learning for Adaptive Indirect Prompt-Injection Attacks and Defenses

arXiv:2609.07529v1 Announce Type: new Abstract: Tool-augmented language agents are vulnerable to indirect prompt injection (IPI). Unlike direct prompt injection, IPI hides adversarial instructions in

Counter-Swarm Doctrine: Containing Coordinated Agent Intrusions

arXiv:2609.06140v1 Announce Type: cross Abstract: Agents can turn shared infrastructure into a channel for coordinated intrusion. The Hugging Face incident and a separate public-wiki investigation sho

safetyarxiv-cs-lg

Damage-Aware Bandit Pruning for Vision and Language Transformers

arXiv:2609.05448v1 Announce Type: cross Abstract: Structured post-training pruning of transformers requires selecting complete functional units whose suppression causes limited degradation. We formula

DART: Distributional Adversarial Recurrent Training for Algorithm Learning

arXiv:2609.05988v1 Announce Type: new Abstract: Recurrent reasoning models (RRMs) can solve structured problems, achieving easy-to-hard generalization through iterative computation in hidden space. Th

local-aiarxiv-cs-lg

Decision-Centered Abstractions via Orthogonal Estimation of Difference-of-Q Functions

arXiv:2406.08697v4 Announce Type: replace-cross Abstract: Offline reinforcement learning enables evaluation and optimization of sequential decisions from historical data, when it is not possible to de

safetyarxiv-cs-lg

Tutorials

Decomposing LLM-Judge Uncertainty to Target Expert Labels

arXiv:2609.06444v1 Announce Type: cross Abstract: An LLM judge evaluates outputs at scale. Experts should label only where it is least sure. Its natural escalation signal conflates two uncertainties:

tutorialsarxiv-cs-lg

Deep Learning Approach to Bearing and Induction Motor Fault Diagnosis via Data Fusion

arXiv:2506.11032v2 Announce Type: replace Abstract: Convolutional Neural Networks (CNNs) are used to evaluate accelerometer and microphone data for bearing and induction motor diagnosis. A Long Short-

researcharxiv-cs-lg