Search: “evaluation”

AI Soccer Analyst: Stage-Aware and Verifiable Human-AI Collaboration for Soccer Data Analysis

arXiv:2609.11224v1 Announce Type: cross Abstract: Sports data analysts translate domain questions into insights by combining computation with sport-specific domain expertise. Large language models eas

researcharxiv-cs-ai

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

Beyond Verified Answers: Solver-Informed Self-Distillation for Bootstrapping Operations Research Language Models

DGX agent

arXiv:2609.09957v1 Announce Type: cross Abstract: Modern large language models (LLMs) can translate natural-language descriptions into operations research (OR) formulations. Post-training techniques i

safetyarxiv-cs-ai

12 Sept 2026

NovGauge: A Fine-Grained Benchmark for Diagnosing LLMs' Capability in Paper Novelty Assessment

arXiv:2609.11234v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used in peer review at major AI conferences, yet novelty remains a persistent weak point. Existing benchma

Verification of Adaptive Agentic Controllers through Finite Rule Revision

arXiv:2607.09770v2 Announce Type: replace Abstract: Industrial agentic AI systems increasingly exhibit a gap between prototype capability and production deployment. In particular, adaptive agents may

When Does Text Inform? Benchmarking Information-Theoretic Metrics for Multimodal Time-Series Forecasting

arXiv:2609.11282v1 Announce Type: new Abstract: Multimodal forecasting models that combine time series with text annotations promise richer prediction through textual context, but how do we know wheth

Agents

Beyond the price per token: Choosing the right OpenAI model on Amazon Bedrock for your workload

Comparing models on dollars per million tokens misses what production workloads actually pay for: outcomes. This post shares an open-source benchmarking harness that measures cost per correct answer,

agentsaws-ml-blog

Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens

arXiv:2609.11877v1 Announce Type: cross Abstract: Many biological discovery problems require experiments to be selected sequentially under constrained budgets. CRISPR screening is a prominent example,

CodeFinetuner: Fine-tune a local code autocomplete model on your own codebase

model-releasesr-localllama

Hi everyone, I was interested in learning LoRA fine-tuning, and ended up building CodeFinetuner over the past few months, a full pipeline that fine-tunes a small code autocomplete model (e.g. Qwen2.5-

Local Ai

DINO-Med: A Unified Patch-Based Adaptation Framework for Multi-Modal Medical Image Analysis Applied to Liver Fibrosis Staging

arXiv:2609.11380v1 Announce Type: new Abstract: Adapting natural-image foundation models like DINOv3 to multi-modal medical imaging is challenging due to the significant domain gap between natural col

local-aiarxiv-cs-cv

Do Vision-Language Models Understand Visual Persuasiveness? A Diagnosis via Visual Persuasive Factors

arXiv:2511.17036v2 Announce Type: replace Abstract: Visual persuasion uses images to shape cognition, emotion, and behavior, with its effects depending on both visual attributes and semantic context.

safetyarxiv-cs-cl

Does YOLO26 Truly Offer Advantages Over Its Predecessors for Edge Deployment? A Benchmark Study in Aquaculture

model-releasesarxiv-cs-cv

arXiv:2607.09835v2 Announce Type: replace Abstract: The recently introduced YOLO26 architecture incorporates NMS-free end-to-end inference and is optimized for deployment on resource-constrained CPU-b

From Protocols to Evidence: Bounded Claims for AI in Service of the Common Good

arXiv:2609.11910v1 Announce Type: new Abstract: Artificial Intelligence does more than create a governance problem. It can also reveal where institutions have already failed to provide responsiveness,

safetyarxiv-cs-lg

IndicTriMix: Developing Language Identification Datasets and Models for Tri-Language Code-Mixing

arXiv:2609.11851v1 Announce Type: new Abstract: Language identification in code-mixed text, largely observed in social media, is highly essential when users frequently switch between multiple language

Model-Aware Schedules Improve Generation via Fiberwise Optimal Transport

arXiv:2609.11842v1 Announce Type: new Abstract: Diffusion and flow-matching schedules control the signal and noise coefficients that mix data and noise along affine probability paths. Minimizing a kin

researcharxiv-cs-lg

Narrative Consolidation: Formulating a New Task for Unifying Multi-Perspective Accounts

arXiv:2512.18041v2 Announce Type: replace Abstract: Processing overlapping narrative documents, such as legal testimonies or historical accounts, often aims not for compression but for a unified, cohe

New Evidence, Same Choice: Testing Physical Experiment Selection in Vision Language Models

arXiv:2609.11022v1 Announce Type: cross Abstract: A model first sees an image from one physical measurement experiment, such as how far a block coasted, and must answer a question about a new trial, s

Overview of the NLPCC 2026 Shared Task 11: Agent-Based Experiment Reproduction from Scientific Papers

arXiv:2609.11117v1 Announce Type: new Abstract: Reproducibility is essential to scientific progress, yet the growing volume and complexity of scientific publications make exhaustive manual verificatio

Pre- and Post-Treatment Brain Metastases Segmentation Using nnU-Net with Post-Processing for BraTS 2026

model-releasesarxiv-cs-cv

arXiv:2609.11477v1 Announce Type: new Abstract: Brain metastases exhibit high inter-lesion variability in size, enhancement pattern, and post-treatment appearance, making volumetric segmentation of bo

ReactHuman: A Physics-Grounded Benchmark for Human-Like Reactive Decision-Making in Embodied Multimodal LLMs

model-releasesarxiv-cs-ro

arXiv:2609.10895v1 Announce Type: new Abstract: Reacting to sudden physical hazards (catching a slipping plate, dodging a falling knife) is both a meaningful test of embodied intelligence and a hard r

RESCUE-BENCH: Towards Relation-Aware Multi-Party Emotional Support Conversation Systems

arXiv:2609.09657v1 Announce Type: new Abstract: Existing emotional support conversation systems mainly focus on one-on-one seeker-supporter interactions and individual emotional states, leaving interp

RetroThinker: Enabling Retrospective Thinking in Speech LLMs

arXiv:2609.11864v1 Announce Type: cross Abstract: Speech large language models (SpeechLLMs) offer reduced latency and retain paralinguistic nuances that are typically lost in cascaded automatic speech

SEAR: Segment-Evidence-Aware Routing for Weak-to-Strong Multilingual Speech MCQ

arXiv:2609.11355v1 Announce Type: new Abstract: This paper describes our system for Task~2 of the second Multilingual Conversational Speech Language Model (MLC-SLM) Challenge. We adapt Qwen3-Omni-30B-

Agents

SearchAtlas: Analyzing Agentic Search Strategies via Evidential Query Graphs

arXiv:2609.10901v1 Announce Type: new Abstract: LLM search agents are often evaluated on final-answer accuracy, overlooking the process. Analyzing a search strategy requires understanding how credible

agentsarxiv-cs-cl

Studying Without a Syllabus: Task-Agnostic Environment Preprocessing

arXiv:2609.10824v1 Announce Type: cross Abstract: Before an LLM agent tackles tasks in a new environment, it can inspect available corpora and tools and construct reusable resources such as indices, s

SynCo: Synthetic Community-Aware Attributed Graph Generator for Graph Neural Network Benchmarking

model-releasesarxiv-cs-lg

arXiv:2609.10742v1 Announce Type: cross Abstract: Graph Neural Networks (GNNs) are powerful models for handling attributed graphs in tasks such as classification, link prediction, and community detect

Target leakage, not model class, explains reported accuracy in survey-based cardiovascular screening: a leakage-tiered audit of glass-box and tabular foundation models

arXiv:2609.11838v1 Announce Type: new Abstract: Cardiovascular screening models trained on national health surveys routinely report areas under the receiver operating characteristic curve (AUROC) near

safetyarxiv-cs-cl

Agents

The PIMMUR Principles: Ensuring Validity in Collective Behavior of LLM Societies

arXiv:2509.18052v4 Announce Type: replace Abstract: Large language models (LLMs) are increasingly used to simulate human collective behavior, yet claims that such simulations are human-like remain lar

agentsarxiv-cs-cl

Applications

Towards AI-Driven Policing: Interdisciplinary Knowledge Discovery from Police Body-Worn Camera Footage

arXiv:2504.20007v4 Announce Type: replace-cross Abstract: This paper proposes a novel interdisciplinary framework for analyzing police body-worn camera (BWC) footage from the Rochester Police Departme

applicationsarxiv-cs-cv

Trust Me, I'm Your Developer: Self-Issued Authentication in Large Language Models

arXiv:2609.03247v1 Announce Type: cross Abstract: Large language model (LLM) security has largely focused on role-playing jailbreaks, with less attention to what happens when a user asks an LLM to ver

When does a spectral prior help graph learning? Connectivity-loss estimation under road-network disruptions

arXiv:2609.11166v1 Announce Type: new Abstract: Rapid evaluation of many simultaneous road-link disruptions requires a practical compromise between exact spectral recomputation and local approximation

safetyarxiv-cs-lg

Which Tokens Should SFT Actually Learn? A Token-Trimming Perspective on Mathematical Reasoning

arXiv:2609.09707v1 Announce Type: new Abstract: Supervised fine-tuning (SFT) applies a uniform cross-entropy loss to all target tokens, even though different tokens provide unequal learning signals fo

A Later Test Set Is Not a New Domain: Pretraining Familiarity Survives a Contamination-Free Hold-Out

model-releasesarxiv-cs-lg

arXiv:2609.10357v1 Announce Type: new Abstract: Time-series foundation models are evaluated almost exclusively on public archives that predate them, so a strong score cannot be separated from having s

BETA-Labeling for Multilingual Dataset Construction in Low-Resource IR

arXiv:2602.14488v3 Announce Type: replace-cross Abstract: IR in low-resource languages remains limited by the scarcity of high-quality, task-specific annotated datasets. Manual annotation is expensive

safetyarxiv-cs-ai

Beyond Contact Sensors: Deep learning with Pseudo-Labeling for remote Photoplethysmography

arXiv:2609.10026v1 Announce Type: new Abstract: Heart rate is a critical biomarker of health, and remote photoplethysmography (rPPG) enables its contactless estimation from video data for telemedicine

researcharxiv-cs-lg

Beyond Single-Negative Preference: Multi-Negative DPO for LLM-Centric Historical Entity Linking

arXiv:2609.07379v1 Announce Type: cross Abstract: Large language models (LLMs) have recently shown promise for historical entity linking, but preference optimization for this task is often formulated

researcharxiv-cs-ai

CausalVerify: An Execution-Grounded Benchmark for LLM Causal Inference Workflows

arXiv:2609.07944v1 Announce Type: new Abstract: Existing causal-inference benchmarks for LLMs mostly score method descriptions or whether generated code runs, not whether the executed workflow recover

Complexity and Scale in AI-Assisted Workflow Management: A Federated Learning Case Study

arXiv:2606.18425v2 Announce Type: replace-cross Abstract: Federated learning over medical images is a demanding workflow application. Each round fans out across parallel client jobs and converges on a

ConsistencyAI: A Benchmark to Assess LLMs' Factual Consistency When Responding to Different Demographic Groups

arXiv:2510.13852v3 Announce Type: replace-cross Abstract: Is an LLM telling you different facts than it's telling me? This paper introduces ConsistencyAI, an independent benchmark for measuring the fa

Copying explains the collective behavior of AI agents in the wild

arXiv:2609.09150v2 Announce Type: replace-cross Abstract: In June 2026, thousands of AI agents found that a small public wiki would accept edits from inside their sandboxes, and started using it to he

Diffs vs. Whole Files: An Empirical Comparison of Iterative Edit-Based and Direct Generation for Flutter/Dart Code Models

arXiv:2609.05779v1 Announce Type: cross Abstract: Large language models used for code editing can be trained and deployed in at least two output regimes: direct generation, where the model emits the e

Do Large Language Models Know What They Don't Know II? A Fully Behavioral, Non-Cognitive Measure of Epistemic Honesty

arXiv:2609.07879v1 Announce Type: new Abstract: Large Language Models (LLMs) are frequently confident, eloquent, and well versed. A natural question arises: do they know what they don't know? To answe

Do Web Agents Investigate Before They Decide?

arXiv:2602.05354v3 Announce Type: replace Abstract: Autonomous web agents are increasingly deployed in moderation and policy enforcement, where correct decisions often depend on evidence that is not i

EEGBind: Detecting Source-Level Interictal Epileptiform Discharges via EEG-Centric Multimodal Binding

model-releasesarxiv-cs-lg

arXiv:2609.09728v1 Announce Type: new Abstract: Source-level analysis of interictal epileptiform discharges (IEDs) is relevant to presurgical evaluation and treatment planning because it helps charact

Efficient Fairness Auditing Across Guidance Scales in Text-to-Image Diffusion Models via Causal Abstraction

arXiv:2609.09486v1 Announce Type: cross Abstract: Fairness auditing of text-to-image diffusion models often requires generating large numbers of images across sampling configurations, making comprehen

safetyarxiv-cs-cv

Evidence-Aligned Entity Verification for Hallucination Detection in Retrieval-Augmented Generation

arXiv:2609.08267v1 Announce Type: new Abstract: Hallucination detection is crucial for large language models (LLMs), as hallucinated content creates significant barriers in applications requiring fact

safetyarxiv-cs-ai

Exposing Weaknesses in Emotion Recognition in Conversations

arXiv:2609.05806v1 Announce Type: new Abstract: Emotion Recognition in Conversations (ERC) aims to identify speakers' emotions in multi-turn dialogue. Accurate emotion recognition can support a wide r

researcharxiv-cs-ai

False positive bias in AI-powered speech-based cognitive screening for multilingual English speakers in the UK

arXiv:2602.13047v2 Announce Type: replace Abstract: Conversational speech reveals early signs of cognitive decline, including dementia and mild cognitive impairment (MCI). AI models show promise for s

safetyarxiv-cs-cl

FATS: A Prompt Injection Attack Utilizing Feign Security Agents with Deceptive Few-shots Learning

arXiv:2410.08776v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) face significant security risks despite their advanced capabilities. While techniques like Reinforcement Learning