Search: “evaluation”

arXiv:2609.12871v1 Announce Type: cross Abstract: Datasets are a crucial element in the development of perception algorithms. They relate sensor measurement data to annotated reference information and

agentsarxiv-cs-cv

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Can LLMs in Draft-Verify-Revise Pipelines Resolve Deictic Ambiguity?

DGX agent

arXiv:2609.12162v1 Announce Type: cross Abstract: Draft-verify-revise is a common LLM orchestration pattern for scaling inference-time compute. One LLM drafts, a second critiques the draft and provide

model-releasesarxiv-cs-cl

14 Sept 2026

DementiaCare-Bench: A Modality-Validated Video Benchmark

model-releasesarxiv-cs-cv

arXiv:2609.12929v1 Announce Type: new Abstract: Dementia affects an estimated 57 million people worldwide, and for most families the hardest part of care is not memory loss but the behavioral and psyc

EAR: Entity-Aware Partitioning Approach for Retrieval-Augmented Generation Development

arXiv:2609.12268v1 Announce Type: new Abstract: Retrieval-augmented generation (RAG) can improve knowledge-intensive question answering, but the first design choice is easy to overlook: how should the

From Bench-to-Bedside: A Review of Clinical Trials in Drug Discovery and Development

arXiv:2412.09378v4 Announce Type: replace-cross Abstract: Clinical trials bridge basic research and clinical application, serving as essential steps in drug development. This review examines clinical

Hardware

HoliBench: A Cross-Platform Benchmarking and Deployment Toolkit for Foundation Models in CPS-IoT Applications

arXiv:2609.12412v1 Announce Type: cross Abstract: Foundation models, including large language models, vision-language models, and time-series foundation models, are increasingly deployed on embedded a

hardwarearxiv-cs-lg

Information Specialization and Constrained Synthesis in Multi-Agent LLM Forecasting: A Prospective Live-Study of the 2026 FIFA World Cup

arXiv:2609.12495v1 Announce Type: cross Abstract: Large language models are being organized into multi-agent systems with specialized roles, but whether such specialization produces distinct forecasts

LatentVerse: A Framework for Understanding Shared and Modality-Specific Information in Multimodal Latent Representations

arXiv:2609.12364v1 Announce Type: new Abstract: Latent embeddings have become a central data abstraction in modern machine learning, especially in biomedicine, where foundation models are increasingly

Learning the Geometry of Collider Events with Metric-Aware Deep Sets

arXiv:2609.12024v1 Announce Type: cross Abstract: Optimal transport gives structured data a geometry, but exact evaluation is costly in large pairwise analyses that exploit relationships among distanc

Measuring Pragmatic Influence in Large Language Model Instructions

arXiv:2602.21223v2 Announce Type: replace Abstract: It is not only what we ask large language models (LLMs) to do that matters, but also how we ask them. Phrases like ``This is urgent'' or ``As your s

researcharxiv-cs-cl

Meddies-PII: A Multilingual Framework for Personally Identifiable Information Extraction in Clinical De-identification

arXiv:2609.12544v1 Announce Type: new Abstract: Clinical de-identification relies on accurately identifying personally identifiable information (PII). However, manually annotated datasets are costly t

MedSNIP: Building and Benchmarking Snippet-Level Granularity for Medical Fact Verification

arXiv:2609.12884v1 Announce Type: new Abstract: A medical claim's correctness often depends not on the claim alone, but on the clinical structure around it. A claim may require a lab reference range,

Optimizing for the decision not the prediction: an exploration of Smooth Net Benefit as a training objective

model-releasesarxiv-cs-lg

arXiv:2609.12752v1 Announce Type: new Abstract: Objective Prediction models are commonly trained using objectives such as Bernoulli negative log-likelihood (NLL), although downstream clinical decision

ORQA: An Occupation-Realistic Question and Answer Framework for LLM Professional Knowledge

arXiv:2609.12366v1 Announce Type: new Abstract: We present ORQA, a method for testing occupation-level knowledge in large language models. Prior methods either map abstract LLM skills to occupations v

PACIFIC: Can LLMs Discern the Psychometric Traits Influencing Your Preferences? Personality-Driven Preference Alignment in LLMs

arXiv:2602.07181v4 Announce Type: replace Abstract: User preferences are increasingly used to personalize Large Language Model (LLM) responses, yet reliably leveraging preference signals remains under

Parameter-Efficient Retrievers for Polish and European Languages

arXiv:2609.12913v1 Announce Type: new Abstract: Dense retrieval systems increasingly rely on multi-billion-parameter language models, whose memory and computational requirements make large-scale index

Predicting Collision Cross Sections with GRACE: Geometric Residual Adduct Conditioning via Early-fusion

arXiv:2609.12223v1 Announce Type: new Abstract: Collision cross section (CCS), derived from ion mobility mass spectrometry, is a common descriptor for molecular annotation. Prediction is challenging f

When Connected Does Not Mean Similar: Charting the Homophily Boundary of SNAP-KG for Streaming Entity Integration

arXiv:2609.12356v1 Announce Type: new Abstract: SNAP-KG is a framework for assigning newly arriving entities to semantic communities in a growing knowledge graph (KG) using only their raw features, wi

AI Soccer Analyst: Stage-Aware and Verifiable Human-AI Collaboration for Soccer Data Analysis

arXiv:2609.11224v1 Announce Type: cross Abstract: Sports data analysts translate domain questions into insights by combining computation with sport-specific domain expertise. Large language models eas

researcharxiv-cs-ai

Beyond Verified Answers: Solver-Informed Self-Distillation for Bootstrapping Operations Research Language Models

arXiv:2609.09957v1 Announce Type: cross Abstract: Modern large language models (LLMs) can translate natural-language descriptions into operations research (OR) formulations. Post-training techniques i

safetyarxiv-cs-ai

NovGauge: A Fine-Grained Benchmark for Diagnosing LLMs' Capability in Paper Novelty Assessment

arXiv:2609.11234v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used in peer review at major AI conferences, yet novelty remains a persistent weak point. Existing benchma

Verification of Adaptive Agentic Controllers through Finite Rule Revision

arXiv:2607.09770v2 Announce Type: replace Abstract: Industrial agentic AI systems increasingly exhibit a gap between prototype capability and production deployment. In particular, adaptive agents may

When Does Text Inform? Benchmarking Information-Theoretic Metrics for Multimodal Time-Series Forecasting

arXiv:2609.11282v1 Announce Type: new Abstract: Multimodal forecasting models that combine time series with text annotations promise richer prediction through textual context, but how do we know wheth

Biology-in-the-loop: Amortized Adaptive Hit Discovery in CRISPR Screens

arXiv:2609.11877v1 Announce Type: cross Abstract: Many biological discovery problems require experiments to be selected sequentially under constrained budgets. CRISPR screening is a prominent example,

Local Ai

DINO-Med: A Unified Patch-Based Adaptation Framework for Multi-Modal Medical Image Analysis Applied to Liver Fibrosis Staging

arXiv:2609.11380v1 Announce Type: new Abstract: Adapting natural-image foundation models like DINOv3 to multi-modal medical imaging is challenging due to the significant domain gap between natural col

local-aiarxiv-cs-cv

Do Vision-Language Models Understand Visual Persuasiveness? A Diagnosis via Visual Persuasive Factors

arXiv:2511.17036v2 Announce Type: replace Abstract: Visual persuasion uses images to shape cognition, emotion, and behavior, with its effects depending on both visual attributes and semantic context.

Does YOLO26 Truly Offer Advantages Over Its Predecessors for Edge Deployment? A Benchmark Study in Aquaculture

model-releasesarxiv-cs-cv

arXiv:2607.09835v2 Announce Type: replace Abstract: The recently introduced YOLO26 architecture incorporates NMS-free end-to-end inference and is optimized for deployment on resource-constrained CPU-b

From Protocols to Evidence: Bounded Claims for AI in Service of the Common Good

arXiv:2609.11910v1 Announce Type: new Abstract: Artificial Intelligence does more than create a governance problem. It can also reveal where institutions have already failed to provide responsiveness,

safetyarxiv-cs-lg

IndicTriMix: Developing Language Identification Datasets and Models for Tri-Language Code-Mixing

arXiv:2609.11851v1 Announce Type: new Abstract: Language identification in code-mixed text, largely observed in social media, is highly essential when users frequently switch between multiple language

Model-Aware Schedules Improve Generation via Fiberwise Optimal Transport

arXiv:2609.11842v1 Announce Type: new Abstract: Diffusion and flow-matching schedules control the signal and noise coefficients that mix data and noise along affine probability paths. Minimizing a kin

Narrative Consolidation: Formulating a New Task for Unifying Multi-Perspective Accounts

arXiv:2512.18041v2 Announce Type: replace Abstract: Processing overlapping narrative documents, such as legal testimonies or historical accounts, often aims not for compression but for a unified, cohe

New Evidence, Same Choice: Testing Physical Experiment Selection in Vision Language Models

arXiv:2609.11022v1 Announce Type: cross Abstract: A model first sees an image from one physical measurement experiment, such as how far a block coasted, and must answer a question about a new trial, s

Overview of the NLPCC 2026 Shared Task 11: Agent-Based Experiment Reproduction from Scientific Papers

arXiv:2609.11117v1 Announce Type: new Abstract: Reproducibility is essential to scientific progress, yet the growing volume and complexity of scientific publications make exhaustive manual verificatio

Pre- and Post-Treatment Brain Metastases Segmentation Using nnU-Net with Post-Processing for BraTS 2026

model-releasesarxiv-cs-cv

arXiv:2609.11477v1 Announce Type: new Abstract: Brain metastases exhibit high inter-lesion variability in size, enhancement pattern, and post-treatment appearance, making volumetric segmentation of bo

ReactHuman: A Physics-Grounded Benchmark for Human-Like Reactive Decision-Making in Embodied Multimodal LLMs

model-releasesarxiv-cs-ro

arXiv:2609.10895v1 Announce Type: new Abstract: Reacting to sudden physical hazards (catching a slipping plate, dodging a falling knife) is both a meaningful test of embodied intelligence and a hard r

RESCUE-BENCH: Towards Relation-Aware Multi-Party Emotional Support Conversation Systems

arXiv:2609.09657v1 Announce Type: new Abstract: Existing emotional support conversation systems mainly focus on one-on-one seeker-supporter interactions and individual emotional states, leaving interp

RetroThinker: Enabling Retrospective Thinking in Speech LLMs

arXiv:2609.11864v1 Announce Type: cross Abstract: Speech large language models (SpeechLLMs) offer reduced latency and retain paralinguistic nuances that are typically lost in cascaded automatic speech

SEAR: Segment-Evidence-Aware Routing for Weak-to-Strong Multilingual Speech MCQ

arXiv:2609.11355v1 Announce Type: new Abstract: This paper describes our system for Task~2 of the second Multilingual Conversational Speech Language Model (MLC-SLM) Challenge. We adapt Qwen3-Omni-30B-

Agents

SearchAtlas: Analyzing Agentic Search Strategies via Evidential Query Graphs

arXiv:2609.10901v1 Announce Type: new Abstract: LLM search agents are often evaluated on final-answer accuracy, overlooking the process. Analyzing a search strategy requires understanding how credible

agentsarxiv-cs-cl

Studying Without a Syllabus: Task-Agnostic Environment Preprocessing

arXiv:2609.10824v1 Announce Type: cross Abstract: Before an LLM agent tackles tasks in a new environment, it can inspect available corpora and tools and construct reusable resources such as indices, s

SynCo: Synthetic Community-Aware Attributed Graph Generator for Graph Neural Network Benchmarking

model-releasesarxiv-cs-lg

arXiv:2609.10742v1 Announce Type: cross Abstract: Graph Neural Networks (GNNs) are powerful models for handling attributed graphs in tasks such as classification, link prediction, and community detect

Target leakage, not model class, explains reported accuracy in survey-based cardiovascular screening: a leakage-tiered audit of glass-box and tabular foundation models

arXiv:2609.11838v1 Announce Type: new Abstract: Cardiovascular screening models trained on national health surveys routinely report areas under the receiver operating characteristic curve (AUROC) near

Agents

The PIMMUR Principles: Ensuring Validity in Collective Behavior of LLM Societies

arXiv:2509.18052v4 Announce Type: replace Abstract: Large language models (LLMs) are increasingly used to simulate human collective behavior, yet claims that such simulations are human-like remain lar

agentsarxiv-cs-cl

Applications

Towards AI-Driven Policing: Interdisciplinary Knowledge Discovery from Police Body-Worn Camera Footage

arXiv:2504.20007v4 Announce Type: replace-cross Abstract: This paper proposes a novel interdisciplinary framework for analyzing police body-worn camera (BWC) footage from the Rochester Police Departme

applicationsarxiv-cs-cv

Trust Me, I'm Your Developer: Self-Issued Authentication in Large Language Models

arXiv:2609.03247v1 Announce Type: cross Abstract: Large language model (LLM) security has largely focused on role-playing jailbreaks, with less attention to what happens when a user asks an LLM to ver

When does a spectral prior help graph learning? Connectivity-loss estimation under road-network disruptions

arXiv:2609.11166v1 Announce Type: new Abstract: Rapid evaluation of many simultaneous road-link disruptions requires a practical compromise between exact spectral recomputation and local approximation

safetyarxiv-cs-lg

Which Tokens Should SFT Actually Learn? A Token-Trimming Perspective on Mathematical Reasoning

arXiv:2609.09707v1 Announce Type: new Abstract: Supervised fine-tuning (SFT) applies a uniform cross-entropy loss to all target tokens, even though different tokens provide unequal learning signals fo

A Later Test Set Is Not a New Domain: Pretraining Familiarity Survives a Contamination-Free Hold-Out