Search: “benchmark”

DreamSat-Bench: Development and Initial Testing of a Testbed for AI-Based Pose Estimation from 3D Reconstruction

model-releasesarxiv-cs-ro

arXiv:2609.14183v1 Announce Type: new Abstract: This paper presents the development and initial testing of DreamSat-Bench, a modular rendezvous and proximity operation testbed designed to benchmark AI

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Enhancing Large Language Model-Based Systems for End-to-End Circuit Analysis Problem Solving

DGX agent

arXiv:2512.10159v3 Announce Type: replace-cross Abstract: LLMs have shown strong performance in data-rich domains such as programming, but their reliability in engineering tasks remains limited. Circu

model-releasesarxiv-cs-ai

15 Sept 2026

Inside VLM Chart Reading: Tracing Value Reading from Vertical Bar Charts Across Space and Depth

arXiv:2609.13745v1 Announce Type: new Abstract: Vision--language models (VLMs) can answer chart questions accurately, but output accuracy does not show how they combine the evidence needed to recover

Realtime-Venus: A full-duplex interaction system with asynchronous delegation

arXiv:2609.13814v1 Announce Type: new Abstract: Natural interaction in digital and physical environments requires continuous perception and timely responses. Spoken dialogue relies on acoustic and lin

Root-Cause Attribution Is a Search Problem: Continual Search for Long-Horizon Agent Failures

arXiv:2609.13463v1 Announce Type: new Abstract: The increasing deployment of AI agents in long-horizon tasks yields massive execution logs. Diagnosing failures within these records is crucial for reli

Rubrics as an Attack Surface: Stealthy Preference Drift in LLM Judges

arXiv:2602.13576v2 Announce Type: replace-cross Abstract: Evaluation and alignment pipelines for large language models increasingly rely on LLM-based judges, whose behavior is guided by natural-langua

Self-Evolving Memory for Generative Recommendation

arXiv:2609.15598v1 Announce Type: cross Abstract: Generative recommendation has emerged as a promising end-to-end paradigm for personalized recommendation. However, user preferences continuously evolv

Research

Sharing standardized image-derived data in computational pathology using DICOM

arXiv:2609.14530v1 Announce Type: cross Abstract: Development and evaluation of computational pathology methods require access to large and diverse datasets. Over the past decade, various initiatives

researcharxiv-cs-ai

Stellar Colosseum: A Many-Agent Harness for Long-Horizon Research in Mathematics and Theoretical Computer Science

arXiv:2609.15983v1 Announce Type: new Abstract: Language models can produce plausible short proofs, but may still be unreliable on long-horizon research problems, where progress depends on a sequence

Sweet Talkers: How Query Formulation Shapes Sycophancy in Romantic Relationship Advice

arXiv:2609.13841v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used for emotional support and relationship advice, where a model's tendency to preserve a user's face can

The Router Within: Eliciting Native Skill Routing from a Frozen LLM

arXiv:2609.15982v1 Announce Type: cross Abstract: Skills extend an LLM agent beyond its parametric knowledge, and the gain they promise rests on picking the right one. Deployed harnesses route by prel

Towards a Mechanistic Understanding of Propositional Logical Reasoning in Large Language Models

arXiv:2601.04260v2 Announce Type: replace Abstract: Understanding how Large Language Models (LLMs) perform logical reasoning internally remains a fundamental challenge. While prior mechanistic studies

TwinICL: Diagnosing Multimodal In-Context Learning through Paired Counterfactuals

arXiv:2609.15028v1 Announce Type: new Abstract: In-context learning (ICL) enables models to infer tasks from demonstrations, but existing benchmarks generally lack matched text and image versions need

Uncertainty-Aware Calibrated Clinical Text Classification with Large Language Models

arXiv:2509.19375v2 Announce Type: replace-cross Abstract: Large language models are increasingly used for clinical text classification, where overconfident misclassifications can directly affect patie

V-ICAL Bench: Evaluating Video In-Context Learning for Multimodal Agents in Interactive Environments

arXiv:2609.15683v1 Announce Type: new Abstract: While In-Context Learning (ICL) enables models to adapt from exemplars without parameter updates, multimodal ICL remains largely underexplored, particul

Verifiable by Construction: Claim-Level Evaluation of Verbatim Citation in Clinical Question Answering

arXiv:2609.15964v1 Announce Type: new Abstract: Large language models (LLMs) have been widely adopted for clinical question answering (QA). Current systems can attach citations to their answers, but t

When Consistency Does Not Mean Reliability: Evaluating Local LLM Judges Against Human Ratings

arXiv:2609.13824v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used to evaluate the responses of other language models. This approach, known as LLM-as-a-Judge, is faster

Physics as the label for measuring and correcting materials reasoning in multimodal models

arXiv:2609.12181v1 Announce Type: new Abstract: Vision-language and language models increasingly interpret materials data, yet benchmarks report that they hallucinate invalid properties and violate ph

14 Sept 2026

Semantically Aligned Gradient-Driven Context-Preserving Image Editing

arXiv:2609.12691v1 Announce Type: new Abstract: Instruction-guided image editing has a training-time blind spot. Generative editors are never required to semantically verify whether their outputs actu

14 Sept 2026

COBRA-Skills: Contextual Bandit-Guided Evolution for Agent Skill Optimization

arXiv:2609.11682v1 Announce Type: new Abstract: Large language model (LLM) agents can benefit from reusable skills distilled from prior task experience, yet existing skill optimization methods often r

12 Sept 2026

Making Alternative Data Work: Context-Augmented LLMs for Financial Forecasting

arXiv:2609.11607v1 Announce Type: new Abstract: When forecasting a firm's future financial performance, alternative data - data collected from non-traditional sources such as consumer transactions, we

12 Sept 2026

AgentAudit: An Open, Extensible Framework for Full-Lifecycle Trust Evaluation of AI Agents

arXiv:2609.09875v1 Announce Type: new Abstract: Existing evaluation frameworks mostly assess only one part of AI agents, such as task completion (AgentBench) or security robustness (AgentDojo, ASB), r

Local Ai

Beyond Word Error Rate: A Switch Aware Evaluation of ASR and Audio Language Models on English Yoruba Code-Switched Speech

arXiv:2609.11786v1 Announce Type: new Abstract: Automatic speech recognition (ASR) systems and audio language models (audio LMs) now report low error rates on monolingual benchmarks, but their behavio

local-aiarxiv-cs-cl

Enabling Knowledge Graph Understanding at Scale with the EXplore Your Graphs ENgine (EXYGEN)

model-releasesarxiv-cs-lg

arXiv:2609.11569v1 Announce Type: cross Abstract: We present EXYGEN (EXplore Your Graphs ENgine), a framework for knowledge graph (KG) understanding that enables conversational access to KGs at scale.

LLM-Anchored Paralinguistic Enrichment for Alzheimer's Disease Detection

arXiv:2609.10896v1 Announce Type: new Abstract: Speech-based automatic detection of Alzheimer's disease (AD) provides a non-invasive and scalable approach to early cognitive screening. AD affects both

R4Tun: LLM-guided adaptive segmental tunnel lining segmentation in point clouds

arXiv:2609.11360v1 Announce Type: new Abstract: Automated inspection of segmental tunnel linings requires adaptive segmentation from 3D point clouds, yet expert-tuned pipelines often degrade when tunn

RAG-Safety-Bench: Reliable Evaluation of Retrieval-Augmented LLM Safety

arXiv:2609.11758v1 Announce Type: new Abstract: Allowing large language models (LLMs) to retrieve information from a set of trusted documents can increase reliability and reduce hallucination. However

Recognizing Is Not Reversing: A Controlled Inversion Test of Fact-Preserving News Framing

arXiv:2609.11769v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used to analyze and rewrite news, yet current framing studies mainly evaluate generation, detection, or wh

Reinforcement learning for Quantum Tiq-Taq-Toe

arXiv:2411.06429v2 Announce Type: replace Abstract: Quantum Tiq-Taq-Toe is a well-known benchmark and playground for both quantum computing and machine learning. Despite its popularity, no reinforceme

Same Day, Same Story; One Day Ahead, a Different Signal: The Dual Validity of Financial Sentiment

arXiv:2609.11144v1 Announce Type: cross Abstract: Financial NLP has a standard workflow: validate a sentiment tool against human labels, then trust it to extract market signal. This assumes the two ev

5-Dialects-BN: Unmasking the Impact of Transliteration on Bangla Dialectal LLMs

arXiv:2609.09964v1 Announce Type: new Abstract: Large Language Models (LLMs) have achieved remarkable progress across natural language processing (NLP) tasks, yet their capabilities degrade sharply fo

AtomCite: Verification and Correction of Supplied Page-Level Citations in Multi-Page Documents

arXiv:2609.05802v1 Announce Type: cross Abstract: Large language models answering questions over multi-page documents are expected to cite the supporting pages, yet supplied citations are sometimes in

Beyond 'AI Helps Humans': Decision-Targeted Evaluation Design for Human-Agent Teams in the Agentic Era

arXiv:2609.05527v1 Announce Type: new Abstract: Wherever a coding agent works under engineer supervision, or a clinical model assists a radiologist, the deployment question is whether to keep the huma

Dynamics of meaning: Towards the Evaluation of Diachronic Semantic Change in Sinhala

arXiv:2609.08609v2 Announce Type: replace Abstract: Tracking semantic change in low-resource languages across extensive historical timelines presents significant challenges due to data scarcity and th

Evaluating the Scalability and Adversarial Generalization of GRPO-Trained NLI Models

arXiv:2504.18376v3 Announce Type: replace-cross Abstract: Natural Language Inference (NLI) is a central task in natural language understanding with applications in fact-checking, question answering, a

Research

FreqBLiMP: Frequency-Controlled Minimal Pairs Reveal Robustness and Fragility of LLMs Under Lexical Rarity

arXiv:2609.07153v1 Announce Type: cross Abstract: Minimal-pair benchmarks such as BLiMP evaluate linguistic knowledge by testing whether language models (LMs) prefer acceptable sentences over minimall

researcharxiv-cs-ai

ObGynLongBench: Revealing the Evidence-to-EHR Gap in Longitudinal EHR Decision-Making

arXiv:2609.07601v1 Announce Type: cross Abstract: The application of large language models (LLMs) to personalized medical assistants has garnered growing interest. However, existing medical benchmarks

Safety

Quantization Amplifies Determinism, Not Bias: Scale-Dependent Behavioral Effects of Serving-Time Weight Compression

arXiv:2609.07901v1 Announce Type: new Abstract: Weight quantization largely determines the economics of serving open-weight LLMs. Its costs are usually assessed with capability benchmarks, on which 4-

safetyarxiv-cs-ai

SCIRIGOR:Evaluating Open-Ended Scientific Analysis Beyond Final Scores

arXiv:2609.06192v1 Announce Type: new Abstract: Scientific coding agents produce interdependent code, results, figures, and claims, yet evaluating final outputs alone does not establish whether their

Applications

Solving versus Verifying: Catching Contradictions in Tax Reasoning Systems

arXiv:2609.05928v1 Announce Type: cross Abstract: Large language models now compute correct tax liabilities on over 90% of well-formed cases in statutory benchmarks, which makes them candidates for th

applicationsarxiv-cs-ai

The human-authorship halo: attribution bias in literary style evaluation by humans and AI

arXiv:2510.08831v2 Announce Type: replace Abstract: As AI writing tools become widespread, we need to understand how both humans and machines evaluate literary style, a domain where objective standard

TreeThink: A Modular Tree Search Library for Mathematical Reasoning with LLMs

arXiv:2607.11258v2 Announce Type: replace Abstract: Tree search algorithms enable systematic exploration of the proof space in neural theorem proving. Existing LLM tree search libraries primarily targ

Typed Federated Artifacts for the Agentic Web:Sharing Tool-Routing Knowledge Across Frozen,Heterogeneous LLM Agents

arXiv:2609.06815v1 Announce Type: cross Abstract: An open, networked web will allow agents to run frozen models from multiple vendors, keep their history private, and teach each other which tool to ca

We're Cooked! - Probing LLM Political Alignment Via Conflict-Framed Recipe Translation

arXiv:2609.07568v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly deployed for translation tasks, yet their implicit political positioning in such contexts remains unders

A Multimodal Label Forecasting Method for Aperiodic Visuo-Motor Time Series

model-releasesarxiv-cs-ro

arXiv:2609.07930v1 Announce Type: new Abstract: Deep learning models have been increasingly applied to Time Series Forecasting (TSF) in recent years. Transformer-based and MLP-based models have both b

BanglaMemeX: Advancing Cultural Metaphoric Image Interpretation in Bangla with a Multimodal Explainable Dataset

arXiv:2609.08029v1 Announce Type: cross Abstract: Vision Language Models have achieved strong performance on multimodal benchmarks, yet their ability to reason about culturally grounded and metaphor-r

CALIPER: Clean Scenes Cannot Rank Physical Inference in Pretrained Visual Representations

arXiv:2609.08250v1 Announce Type: cross Abstract: How far a pushed object slides depends on its mass and friction, which no single image reveals. Pretrained visual encoders are increasingly used as th

Charts Are Beyond Pixels: Probing for Layer-Wise Chart Understanding and Editing

arXiv:2609.08657v1 Announce Type: new Abstract: Charts are structured visual compositions whose elements have distinct functional roles, semantic correspondences, and visibility relations. This struct