Search: “benchmark”

Memory as Weights: Internalizing Long-Term History for Streaming Videos

arXiv:2610.22070v1 Announce Type: cross Abstract: Streaming vision-language models (VLMs) must continuously integrate information from the past while remaining responsive to current observations and u

16 Oct 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

VELD-QA: A Large-Scale Dataset for Visual Evidence Grounding and Question Answering in Long Documents

DGX agent

arXiv:2610.20964v1 Announce Type: cross Abstract: Existing long-document QA benchmarks rarely combine large-scale human-authored questions with detailed visual evidence annotations, limiting diagnosis

model-releasesarxiv-cs-cl

16 Oct 2026

VisualTrust: Video Reasoning under On-Screen Claim Framing

arXiv:2610.21159v1 Announce Type: new Abstract: On-screen text can describe what a video shows, or it can frame what the video is supposed to mean. Current benchmarks treat both alike, testing whether

16 Oct 2026

WorkflowPerturb: Calibrated Stress Tests for Evaluating Multi-Agent Workflow Metrics

arXiv:2602.17990v4 Announce Type: replace Abstract: Multi-agent LLM systems that generate structured workflows from natural-language requests are now deployed in production across cloud automation, De

16 Oct 2026

ACED-Bench: Evaluating How LLM Agents Acquire and Act on Causal Evidence

arXiv:2610.14381v1 Announce Type: new Abstract: Large language models are increasingly used as scientific assistants, yet we still lack reliable ways to evaluate whether they can investigate a system

Anuvani-Diarization: Zero Shot Speaker Diarization and Annotation Conventions Across 22 Languages

arXiv:2610.20327v1 Announce Type: new Abstract: India's 22 scheduled languages are spoken by over a billion people in conversations that mix languages, often within one turn, overlap freely, and take

Are We Really Measuring Forgetting? Pitfalls in Evaluating Machine Unlearning for Multimodal LLMs

arXiv:2610.19613v1 Announce Type: new Abstract: As machine unlearning for multimodal large language models (MLLMs) rapidly advances, rigorous evaluation of its effectiveness becomes increasingly impor

Agents

Choosing LLMs for Strategic Agent Interactions

arXiv:2610.18354v1 Announce Type: new Abstract: Game-theoretic benchmarks claim that large language model (LLM) strategic decision-making varies across game types, interaction structures, and capabili

agentsarxiv-cs-ai

ChronoCulture: Evaluating Cultural Awareness Across Time in Large Language Models

arXiv:2610.20472v1 Announce Type: new Abstract: Cultural attitudes and social norms evolve over time, yet most cultural benchmarks for large language models (LLMs) treat culture as a static attribute

Decomposability-Guided Cooperative Coevolution for Large-Scale Itinerary Planning

arXiv:2506.06121v3 Announce Type: replace Abstract: Large-scale itinerary planning is a variant of the traveling salesman problem, aiming to determine an optimal path that maximizes the collected poin

DivAlgo: A systematic evaluation of algorithmic diversity for code generation models

arXiv:2610.19157v1 Announce Type: cross Abstract: Code generation models are actively being used in professional and educational settings. However, the competence of large language models (LLMs) to ge

Do Factuality Methods Make LLMs More Factual? Reproducibility and Fair Comparison

arXiv:2610.18415v1 Announce Type: cross Abstract: Large Language Models (LLMs) are aligned to human preferences, after pre-training and supervised fine-tuning steps. However, standard RL alignment ste

Does the Linguistic Register of Thinking Matter? Wenyan as a Compact Reasoning Register for Chinese LLMs

arXiv:2610.19741v1 Announce Type: new Abstract: Visible reasoning traces of Chinese reasoning-oriented LLMs are typically written in modern vernacular Chinese, which may be verbose under limited decod

Evaluating LLMs as Agentic Workflow Executors in Real-World Customer Support Skills

arXiv:2610.14346v2 Announce Type: replace Abstract: Enterprise customer-support agents increasingly rely on Large Language Models (LLMs) to execute bounded, tool-using workflows. We study this problem

GesBench: When and Where Do Multimodal Language Models Perceive Social Meaning?

arXiv:2610.15129v1 Announce Type: new Abstract: Human social perception is incremental: we infer intent before an utterance is complete, revise our interpretation as new cues arrive, and attend to spe

How Do LLMs Understand Legal Philosophy? Diagnosing Legal-Philosophical Reasoning in LLMs

arXiv:2610.19427v1 Announce Type: new Abstract: Legal reasoning is a fundamental capability for large language models (LLMs) to complete legal AI tasks such as legal judgment prediction. In legal prac

Jev-ToM: Evaluating Theory of Mind in Typed Decision Models

arXiv:2610.18610v1 Announce Type: cross Abstract: Dialogue and collaborative assistants repeatedly judge what a partner knows, believes, or intends. Typed decision models offer a potential alternative

Kandinsky S2V 1.0: Audio-Driven Avatar Video Generation

arXiv:2610.19290v1 Announce Type: new Abstract: We present Kandinsky S2V 1.0, an open family of speech-to-video models for controllable audio-driven human animation, comprising Kandinsky S2V 1.0 Lite

MIRAGE: Metric Inference and Reliability-Aware Gaussian Estimation from Feed-Forward Vision

arXiv:2610.12568v2 Announce Type: replace Abstract: Feed-forward 3D priors enable high-quality reconstruction from image streams but often lack reliable metric scale and suffer from long-term trajecto

PXRD-Bench: How Close Are We to Determining Crystal Structures from Powder Diffraction?

arXiv:2610.16480v1 Announce Type: new Abstract: Determining crystal structures from powder X-ray diffraction (PXRD) would enable routine characterization of newly synthesized materials, including thos

Tutorials

ReferFlow: Beyond Final-Target Localization for Complex Referring Expressions

arXiv:2610.20352v1 Announce Type: new Abstract: Multimodal Large Language Models have made significant progress in referring expression comprehension (REC), yet most REC benchmarks still focus on fina

tutorialsarxiv-cs-cv

RoTA-Bench: Evaluating 3D Spatial Reasoning through Interactive Projection Alignment

arXiv:2610.17665v2 Announce Type: new Abstract: Can vision-language models turn 2D observations into 3D understanding that supports precise action? This requires inferring object geometry, anticipatin

SAGE: Pushing 2-Bit Vector-Quantized LLMs toward Real-World Usability

arXiv:2610.14745v1 Announce Type: new Abstract: Extreme low-bit weight-only quantization is an effective approach to reducing the deployment costs of large language models. Below 3 bits, scalar quanti

Security in LLM-Assisted Coding: A Systematic Literature Review

arXiv:2610.15482v1 Announce Type: cross Abstract: Large Language Models accelerate software development but introduce severe security vulnerabilities. This systematic review investigates security risk

SPARC: Reliable Spatial Annotations from Robot Demonstrations at Scale

arXiv:2606.13497v2 Announce Type: replace-cross Abstract: This work introduces Spatial Annotations from Robot Demonstrations with Reliability Calibration (SPARC), a risk-aware framework that automatic

The Gap Is in the Geometry: A Convergent Meta-Analysis of Multi-Facet Aerial and Remote Sensing Video Understanding

arXiv:2610.19102v1 Announce Type: new Abstract: Aerial and remote sensing video is increasingly evaluated like web video, by posing multiple-choice questions to multimodal large language models (MLLMs

When 1+1 Backfires: Rethinking Relevance-Diversity Balancing for Sample-Aware Visual Token Reduction

arXiv:2610.16615v1 Announce Type: new Abstract: Visual token compression has emerged as a critical technique for reducing the inference cost of Vision-Language Models. Existing methods select tokens b

Conditional Multi-Event Temporal Grounding in Long-Form Video

arXiv:2606.15320v2 Announce Type: replace Abstract: Multimodal large language models have made rapid progress in video temporal grounding, yet real-world applications routinely require localizing ever

From Rainfall to Road-Network Functionality: Transferable Graph Forecasting for Flood-Disrupted Transportation Networks

arXiv:2610.14622v1 Announce Type: new Abstract: Extreme rainfall can rapidly degrade road-network functionality (RNF), disrupting emergency mobility, accessibility, evacuation, logistics, and infrastr

Game2Real-Bench: Do Toy Games Reveal and Transfer to Real-World Visual Reasoning?

arXiv:2610.18834v1 Announce Type: new Abstract: Visual reasoning is an important capability for vision-language models (VLMs) to understand visual evidence, spatial structure, constraints, and state c

Grables: When Graph Structure Helps Tabular Prediction

arXiv:2602.03945v2 Announce Type: replace Abstract: Simple row-wise predictors that score each row independently remain competitive with far more expressive architectures on tabular benchmarks, which

Research

Image Diffusion Models Are Good 3D Human Motion Generators

arXiv:2610.14254v1 Announce Type: new Abstract: Text-to-motion generation typically relies on dataset-specific skeletal representations, with models trained separately for benchmarks with different jo

researcharxiv-cs-cv

IND-Eval: Aligning Evaluation and Simulation for Industrial Time Series Foundation Models

arXiv:2610.16652v1 Announce Type: new Abstract: We introduce IND-Eval, a public benchmark that organizes industrial time series across domains into source-specific forecasting tasks. Relative to estab

Applications

Is Complex Query Answering Really Complex?

arXiv:2410.12537v4 Announce Type: replace Abstract: Complex query answering (CQA) on knowledge graphs (KGs) is gaining momentum as a challenging reasoning task. In this paper, we show that the current

applicationsarxiv-cs-lg

ManipShield: A Unified Framework for Image Manipulation Detection, Localization and Explanation

arXiv:2511.14259v4 Announce Type: replace Abstract: With the rapid advancement of generative models, powerful image editing methods now enable diverse and highly realistic image manipulations that far

Robust Logical Anomaly Detection under Capture Shifts via Synthetic Anomalies

arXiv:2610.18295v1 Announce Type: new Abstract: Detectors of logical anomalies, such as missing, extra or wrongly combined parts, usually score a test image by its distance from normal (good) images.

SceneForesee: Learning Scene Evolution for Spatiotemporal Reasoning

arXiv:2610.14439v1 Announce Type: new Abstract: Large multimodal models (LMMs) need to understand how entities move and interact as a scene develops. This requires following the same entities over tim

ScenePercept: Scenario-Conditioned Perceptual Assessment of Real-World User-Generated Content Images

arXiv:2610.17661v1 Announce Type: new Abstract: The rapid development of mobile imaging and social platforms has led to explosive growth in user-generated content images, making their systematic and c

scHFBiMamba: a bidirectional Mamba autoencoder for cardiac disease-state classification from single-nucleus RNA-seq data

arXiv:2610.15117v1 Announce Type: new Abstract: To evaluate whether a compact bidirectional state-space autoencoder improves joint classification of species, cell type, and disease state from single-n

Sensitivity or Criterion? Measuring and Moving When Language Models Act

arXiv:2610.15415v1 Announce Type: cross Abstract: Before an agent acts, it must decide whether action is warranted. A single accuracy cannot distinguish failure to recognize a warranted action from re

Text-Vision Co-Instructed Image Editing

arXiv:2606.16767v2 Announce Type: replace Abstract: Existing image editing methods can be generally categorized into textual instruction-based and visual prompt-based ones. Textual instructions are se

XForecast: The KDD 2026 Finance Day Challenge on News-Grounded Financial Forecasting

arXiv:2610.16131v1 Announce Type: new Abstract: Financial forecasting has traditionally relied on numerical time-series (TS) signals, while textual information such as news remains largely underutiliz

Language Models Have Interpretable Narrative Mechanisms

arXiv:2610.12989v1 Announce Type: new Abstract: We investigate whether language models contain internal components that contribute to representing narrativity, the degree to which a text exhibits narr

MetaJoin: Adaptive Join Selections for Heterogeneous Tables

arXiv:2512.02444v2 Announce Type: replace-cross Abstract: Joining heterogeneous tables requires different forms of correspondence, including exact agreement, syntactic transformation, semantic similar

ReasoningShield: Safety Detection over Reasoning Traces of Large Reasoning Models

arXiv:2505.17244v3 Announce Type: replace-cross Abstract: Large Reasoning Models (LRMs) leverage explicit reasoning traces, known as Chain-of-Thought (CoT) reasoning, to decompose complex problems int

REFLECT: Role-Aware Detection of Sentiment, Emotion, Politeness, and Formality

arXiv:2610.13797v1 Announce Type: new Abstract: Detecting communicative meaning in dialogue requires modeling not only lexical content but also the participant-role structures through which sentiment,

Travel Back To Move Forward: Retrospective Exploration and Evolving for Long Horizon Agents

arXiv:2610.13058v1 Announce Type: new Abstract: Large language model agents increasingly operate in long-horizon environments where early mistakes can trigger cascading failures and trap agents in dea

An Explainable Header-Centric Framework for Large-Scale Semantic Table Interpretation and Data Quality Assessment

arXiv:2610.10541v1 Announce Type: new Abstract: Knowledge Graph (KG) quality depends not only on downstream graph validation, but also on the quality of tabular metadata used before integration. In me