Search: “benchmark”

From Synthetic Priors to Model Behavior: Structural Coverage in Tabular Foundation Models

arXiv:2609.06912v1 Announce Type: new Abstract: Tabular foundation models (TFMs) are commonly pretrained on large collections of procedurally generated synthetic tasks, yet it remains unclear how well

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Multi-Grid Post-Training for Long-Form Multi-Shot Video Generation

DGX agent

arXiv:2609.06373v1 Announce Type: new Abstract: Generating long-form multi-shot videos requires coherent within-shot motion and visually consistent narratives across shots. Existing video generators f

model-releasesarxiv-cs-cv

9 Sept 2026

Segment Any Motion with Radar: Robust Multimodal Moving-Object Segmentation and Tracking

arXiv:2609.08346v1 Announce Type: new Abstract: Moving-object perception must decide which image regions correspond to real motion and keep every instance identified over time. Methods that read motio

Social Intuition vs. Machine Reasoning: Anticipating Human-Robot Interaction from multiple modalities

arXiv:2609.07394v1 Announce Type: new Abstract: Anticipating whether a person will interact from one's own perspective is a highly intuitive task for humans, that relies on a combination of cues. We i

STSG-VQA: Evidence-Grounded Temporal Question Answering from Surgical Spatio-Temporal Scene Graphs

arXiv:2609.08543v1 Announce Type: new Abstract: Despite recent advances in surgical vision-language models (VLMs), temporal reasoning remains limited because existing supervision is largely frame-cent

SynthGait-19K: A Physically Grounded Synthetic Video Dataset for Gait Parameter Estimation

arXiv:2609.08108v1 Announce Type: new Abstract: Accurate estimation of clinically meaningful gait parameters from monocular video is important for scalable mobility assessment, yet progress is limited

Vision-Encoder Behavioral Fingerprints of Image-to-Image Generative Models: A Training-Paradigm-Driven Taxonomy of Six Commercial APIs

arXiv:2606.14787v2 Announce Type: replace Abstract: We study six production image-to-image AI systems (gpt-image-1, Gemini 2.5 Flash Image, Flux Kontext, SDXL img2img, SD3 img2img, and Qwen Image Edit

AVENUE: Audio-Video EditiNg Understanding and Evaluation

arXiv:2609.04253v1 Announce Type: cross Abstract: Audio-video (AV) editing aims to modify audio and video content according to a target prompt. Unlike single-modality editing, AV editing requires mode

ICM-Bench: Person-Level Identity Reasoning in Multimodal Agents with Long-Term Memory

arXiv:2609.04438v1 Announce Type: new Abstract: Long-horizon multimodal agents should remember not only what happened but also who participated. This capability depends on linking recurring faces, voi

Phase Transition Frequency as a Training Time Predictor of Test Accuracy in ResNets

arXiv:2609.05194v1 Announce Type: cross Abstract: The number of discrete class-separability jumps observed during ResNet finetuning is examined empirically as a predictor of final test accuracy. Acros

Physics-Aware Random Walk Fingerprints for Scalable Power Grid Graph Classification

arXiv:2609.04943v1 Announce Type: new Abstract: Recent benchmarks such as PowerGraph provide large collections of power-grid graphs for cascading-failure classification. Graph neural networks (GNNs) a

Beyond Majority Vote: Multi-Perspective Adjudication for Medical Hallucination Detection

arXiv:2609.03953v1 Announce Type: new Abstract: Understanding the frequency of factual errors in chatbot-generated text and evaluating systems that detect these errors is critical for determining chat

CORE: Improving Compositional Reasoning in MLLM Embedding via Reranker Distillation

arXiv:2609.04083v1 Announce Type: cross Abstract: MLLM-based embedding models remain limited in compositional retrieval, often failing to distinguish scenes containing the same concepts but different

FailBench: How Reliable are VLMs at Judging Robot Task Success?

arXiv:2609.03611v1 Announce Type: cross Abstract: Vision-Language Models (VLMs) are increasingly used to evaluate robot manipulation outcomes, but existing benchmarks offer limited evidence of cross-d

IDSPACE: A Novel Document Generator for Reliable Evaluation of Digital Identity Verification Systems [Extended Technical Report]

arXiv:2609.03052v1 Announce Type: new Abstract: As services move online, trust institutions such as banks, lenders, and governments must verify the identity of remote users. Fraud detection tools are

Mind the Gap: Robustness Risks in PII Detection Systems

arXiv:2609.03464v1 Announce Type: new Abstract: Personally Identifiable Information (PII) detection is a foundational component of data protection infrastructure where missed entities constitute direc

MINERVA: How Small Can a Manipulation Policy Be and Still Solve LIBERO?

model-releasesarxiv-cs-ro

arXiv:2609.03715v1 Announce Type: new Abstract: Vision-language-action (VLA) models with billions of parameters now dominate the LIBERO manipulation benchmark, but the model capacity actually required

PatchBench: Evaluating AI Agents for Vulnerability Patching

arXiv:2609.04075v1 Announce Type: cross Abstract: AI agents have recently demonstrated strong performance in automated vulnerability patching. However, existing evaluations often validate a patch only

Rent-a-RAG: Embedding-Space Watermarks for Auditing Third-Party RAG

arXiv:2609.03749v1 Announce Type: cross Abstract: Third-party retrieval-augmented generation (RAG) marketplaces create a new auditing problem: data providers may license corpora to a RAG operator, yet

Restricted Eigenvalues Beyond Gaussian Width: Threshold Occupancy under Heavy Tails

arXiv:2609.03504v1 Announce Type: new Abstract: Restricted eigenvalue (RE) bounds govern stable recovery by norm-regularized estimators. For isotropic sub-Gaussian measurements, the benchmark sample s

SVG-Score: Human-Aligned Evaluation of Text-to-SVG Generation

arXiv:2609.03806v1 Announce Type: new Abstract: Scalable Vector Graphics (SVG) generation is attracting increasing attention as generative models improve in expressiveness and controllability. Progres

Agents That Model Agents: Five Principles Toward a Theory of Mind for 6G Networks

arXiv:2609.01779v1 Announce Type: cross Abstract: Future 6G networks will rely on Large Language Model (LLM) agents to manage the Radio Access Network (RAN). However, current architectures assume inte

C^{3}T: Counterfactual Causal Reasoning for Sentiment Shifts in Social-Media Conversation Trees

arXiv:2609.02131v1 Announce Type: cross Abstract: Sentiment in social-media threads does not only vary across posts; it shifts as users react to claims, corrections, evidence, and hostility within a b

CoViT: Instance-Correspondence Contrastive Learning for Vision Transformer

arXiv:2609.01787v1 Announce Type: new Abstract: Vision Transformers (ViT) excel in semantic understanding but fail to discriminate between object instances (e.g., identical embeddings for two dogs), l

Evaluating the Evaluator: Summarization Metrics and LLM-Judges beyond English

arXiv:2503.17039v3 Announce Type: replace-cross Abstract: Automatic text summarization relies on automatic evaluation to quickly determine the quality of summarization models via automatic metrics and

ExecRetrieval: Measuring the Functional-Correctness Gap in Code-Embedding Retrieval

arXiv:2609.01865v1 Announce Type: cross Abstract: Embedding-based code retrieval is a core component of coding agents and retrieval-augmented code generation, where retrieving correct code matters mor

Post-Training Language Models for Gold-Medal Performance in Coding Competitions

arXiv:2609.02849v1 Announce Type: cross Abstract: Competitive programming has become a key test of large language model reasoning, with international competitions such as IOI and ICPC representing its

AutoXRD: Autonomous LLM Agents and Comprehensive Evaluation for Powder Diffraction Analysis

arXiv:2609.00070v1 Announce Type: cross Abstract: Powder X-ray diffraction (XRD) is central to materials characterization, yet reliable end-to-end automation remains challenging. An XRD agent must int

Local Ai

Calibration is the Bottleneck: An Action-Class Diagnostic of Multi-Turn Tool-Calling

arXiv:2609.00949v1 Announce Type: cross Abstract: Multi-turn tool calling is a core evaluation scenario for large language model (LLM) agents. On public tool-calling benchmarks, open-weight models now

local-aiarxiv-cs-ai

Can Scene Text Recognition Read Rare Compositions?

arXiv:2609.00816v1 Announce Type: new Abstract: Scene text recognition is reported as 89--97% accurate on the six standard benchmarks, and the problem is widely treated as saturated. We present an alt

ClinTraceBench: Source-Verifiable Longitudinal Clinical Reasoning over EHR-Derived Dialogues

arXiv:2609.01111v1 Announce Type: new Abstract: Clinical LLM assistants must reason over multi-visit patient trajectories, yet whether the compact history representations used to scale them---retrieva

DAGGER: Distractor-Aware Graph Generation for Executable Reasoning in Math Problems

arXiv:2601.06853v3 Announce Type: replace Abstract: Chain-of-Thought (CoT) prompting is widely adopted for mathematical problem solving, including in low-resource languages, yet its behavior under irr

Evaluating Style-Personalized Text Generation: Challenges and Directions

arXiv:2508.06374v4 Announce Type: replace Abstract: With the surge of large language models (LLMs) and their ability to produce customized output, style-personalized text generation--'write like me'--

Fleets Need a Context Plane: Rethinking Cooperative Perception for Autonomous Drones

model-releasesarxiv-cs-ro

arXiv:2609.00659v1 Announce Type: new Abstract: Cooperative perception allows a drone fleet to combine observations from multiple viewpoints. However, existing systems typically fix their feature-shar

Good Memory Has ECC: Evaluating the Memory of Vision-Language Models Beyond Accuracy

arXiv:2609.00103v1 Announce Type: cross Abstract: Memory is widely viewed as an important unsolved problem for LLMs and VLMs, and current benchmarks typically evaluate it by testing accuracy over long

Incremental Risk Assessment of Progressive Elder Financial Scams via Instruction-Tuned Small Language Models

arXiv:2609.00005v1 Announce Type: new Abstract: Financial scams targeting older adults increasingly occur through text and voice channels such as email, SMS, and phone calls, unfolding over multiple c

Learning to Remember: End-to-End Training of Memory Agents for Long-Context Reasoning

arXiv:2602.18493v2 Announce Type: replace-cross Abstract: Long-context LLMs and Retrieval-Augmented Generation defer state tracking and evidence consolidation to query time, which is brittle when fact

Research

The Visual Insensitivity Gap: Diagnosing When Vision-Language Models Fail to Use Visual Evidence

arXiv:2609.00868v1 Announce Type: cross Abstract: Vision-language models are evaluated by aggregate accuracy on multimodal benchmarks, a practice that implicitly assumes the model uses its visual inpu

researcharxiv-cs-cl

AdaPath: Query-Adaptive Path-Finding via Path-Bank for Multi-Hop Implicit Biomedical KGQA

arXiv:2608.30556v1 Announce Type: new Abstract: Path-finding over knowledge graphs has become an effective way to ground LLM reasoning on multi-hop questions. However, biomedical QA introduces two dis

Arkios: An Open Bilingual English-Nepali Language Model Trained From Scratch, with a Devanagari-Aware Tokenizer

arXiv:2608.30092v1 Announce Type: cross Abstract: We present Arkios, a 1.04B-parameter dense transformer pretrained from scratch on 150B tokens of bilingual English-Nepali text, using a custom single-

Beyond Correctness: Validity-Oriented Evaluation of Biomedical LLM Judges

arXiv:2608.29127v1 Announce Type: new Abstract: We propose a scalable, validity-oriented pipeline for evaluating biomedical LLM judges when high-quality human judgments are scarce. First, we augment e

Beyond Global Realism: Virtual Try-On Evaluation and Optimization with Dimension-wise Garment Fidelity Assessment

arXiv:2608.29804v1 Announce Type: new Abstract: Virtual try-on (VTON) requires not only realistic generation but also faithful preservation of garment characteristics. However, existing evaluation met

Can Agentic Trading Systems Pay for Their Own Intelligence?

arXiv:2607.10286v2 Announce Type: replace Abstract: Large language model (LLM) agents are increasingly used in trading systems, where model reasoning, tool use, and continual decisions incur costs tha

Can LLMs Take the Pulse of the Economy? A Real-Time Evaluation of LLM Nowcasts on Macroeconomic Indicators

arXiv:2608.30110v1 Announce Type: cross Abstract: Nowcasting headline macroeconomic indicators, i.e., estimating an indicator's value for the current reference period before its official release, is c

Confidence-Aware Ensemble and Long-Word Refinement for Artistic Text Recognition

arXiv:2608.29970v1 Announce Type: new Abstract: Artistic Text Recognition (ATR) remains challenging because word images often combine decorative fonts, curved layouts, object-like characters, clutter,

Defending Wearable VLMs Against Private Attribute Inference

arXiv:2608.28691v1 Announce Type: cross Abstract: Wearable VLM pipelines promise continuous multimodal assistance from egocentric visual capture: a user asks a task-driven question about the surroundi

En-ViMedNER: An English-Vietnamese Parallel Biomedical Corpus with UMLS Semantic Type Annotations

arXiv:2608.29890v1 Announce Type: new Abstract: Biomedical Named Entity Recognition (NER) is fundamental to healthcare AI applications, including clinical decision support and medical information extr

Evaluating and Mitigating Anti-LGBTQ Biases in German and Multilingual Language Models

arXiv:2608.30884v1 Announce Type: cross Abstract: While gender and racial biases in language models have been widely studied, anti-LGBTQ biases remain underexplored, particularly beyond English. Exist