Search: “benchmark”

Observe Before You Alert: Adaptive Driver Alerting with Vision-Language Models

arXiv:2609.08130v1 Announce Type: new Abstract: Driver alerting from dashcam video requires sequential decision-making under partial observability: a system must decide not only whether a scene is ris

9 Sept 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

TRAIL: Trajectory-Aware Visual Place Recognition against Unordered Databases

DGX agent

arXiv:2609.07373v1 Announce Type: new Abstract: Modern Visual Place Recognition (VPR) methods excel on standard benchmarks yet remain brittle in feature-poor environments. By treating each query image

model-releasesarxiv-cs-cv

9 Sept 2026

What Are You Doing? A Closer Look at Controllable Human Video Generation

arXiv:2503.04666v2 Announce Type: replace Abstract: High-quality benchmarks are crucial for driving progress in machine learning research. However, despite the growing interest in video generation, th

9 Sept 2026

Higher acceptance length, slower prose: Ling’s n=1/2/3 MTP test on one Spark

model-releasesr-localllama

The missing control is visible in sudoingX’s Ling-3.0-flash benchmark graphics. The earlier table leaves Ling’s no-speculation baseline as “not measured.” The later code/prose graphic fills it in: abo

LIBERO-RECOVER: Beyond Task Success Towards Failure Recovery in Robotic Manipulation Models

model-releasesarxiv-cs-ro

arXiv:2609.05178v1 Announce Type: new Abstract: Vision-Language-Action (VLA) or World Action (WAM) models have recently demonstrated remarkable performance in robotic manipulation. On LIBERO, SOTA met

Molecular Deja Vu: Digit-Level Retrieval of Published Values in Frontier Language Models

arXiv:2609.05381v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly evaluated on molecular property benchmarks, but accuracy cannot distinguish a model that predicts a proper

VISTA: Dense Multi-Label Classroom Coding with Vision-Language Models

arXiv:2609.04550v1 Announce Type: new Abstract: Video-language benchmarks are usually constructed by the dataset authors without published reliability statistics, leaving the noise floor of the constr

CulturalMenuBench: Probing the Knowledge-Application Gap in Multimodal Culinary Reasoning

arXiv:2609.03526v1 Announce Type: new Abstract: Multimodal language models achieve near-ceiling scores on food recognition benchmarks, yet it remains unclear whether this success reflects genuine cult

Evaluating Large Language Models on Urdu Idioms

arXiv:2510.17460v2 Announce Type: replace Abstract: Idioms remain a persistent challenge in natural language processing due to their figurative and culturally grounded meanings, which distinguish them

IndicSafeEval: Safety Robustness of Large Language Models under Multilingual Persuasive Jailbreak Attacks

arXiv:2609.03781v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used in multilingual settings, yet their safety is still evaluated primarily in English. This limits our

ToolDF: Tool-Integrated Reasoning for Mixed-Authenticity Audio Deepfake Detection

arXiv:2609.03620v1 Announce Type: cross Abstract: Audio deepfake detection is commonly formulated as clip-level binary classification of single-domain audio. However, real-world manipulated audio can

AdaMem: Learning What to Remember with Adaptive Memory Policies for Personalized Agents

arXiv:2606.21144v2 Announce Type: replace-cross Abstract: Long-term memory systems allow LLM agents to preserve information beyond a single context window, but most systems focus on storing and retrie

Does Playing it Safe Count as Faithfulness? Reassessing LVLM Hallucination Mitigation Methods

arXiv:2609.01888v1 Announce Type: new Abstract: Recent inference-time hallucination mitigation methods for large vision-language models (LVLMs) report strong gains on hallucination benchmarks. However

GPT-6 Astra is coming to Devin. On FrontierCode 1.1, Astra performs within 0.4 points of Fable 5 at a 64% lower cost. It also sets a new SOT…

model-releasescognition-ai--x

GPT-6 Astra is coming to Devin. On FrontierCode 1.1, Astra performs within 0.4 points of Fable 5 at a 64% lower cost. It also sets a new SOTA on our internal testing benchmark, generating more compreh

Safety

READY or Not: Reliable Enterprise Agent Deployment

arXiv:2609.02095v1 Announce Type: new Abstract: An AI agent can perform well on benchmarks and still be unsuitable for deployment. Existing AI-agent benchmarks measure whether an agent can complete re

safetyarxiv-cs-ai

Discrete-Time MDP Modeling for Multi-Item Capacitated Lot Sizing with Stochastic Demand Timing

arXiv:2609.00004v1 Announce Type: new Abstract: This paper studies a finite-horizon multi-item capacitated lot-sizing problem in which demand quantities are deterministic, while demand-arrival periods

ReproRepo: Scaling Reproducibility Audits with GitHub Repository Issues

arXiv:2606.18237v2 Announce Type: replace-cross Abstract: Reproducing research results from papers and released code is central to scientific progress. Existing works have introduced benchmarks to eva

Staged Linguistic Seeding: Grounded Query Expansion for Verified-Unit QA in AI Contact Centers

arXiv:2609.00844v1 Announce Type: new Abstract: Customer-service QA in an AI contact center (AICC) runs under deployment constraints that benchmark QA misses: tight voice-hotline latency and a high co

VIBE-Bench: Evaluating Personalized Large Language Models When Profiles Don't Mean Preferences

arXiv:2609.00921v1 Announce Type: new Abstract: Personalized Large Language Models (PLLMs) aim to tailor responses to individual users, where a central challenge is preference reasoning: inferring que

Beyond Semantic Accuracy: Consequence-Aware Evaluation for Safety-Critical Language Understanding

arXiv:2608.24621v2 Announce Type: replace Abstract: Can language models be trusted in safety- critical operations? In such settings, strong per- formance on semantic metrics does not guaran- tee opera

ChessQueries: Toward Better Chess Board Recognition

arXiv:2608.30762v1 Announce Type: new Abstract: Chess board recognition is the task of mapping the image of a chess board to the information of which piece is on which square. So far this task has two

Driving on Memory

arXiv:2608.31029v1 Announce Type: new Abstract: End-to-end autonomous driving models plan future trajectories from raw sensor input. While earlier driving benchmarks often measured deviation from the

Evaluating the Semantic Specificity of Representation Steering in Language Models

arXiv:2608.29431v1 Announce Type: new Abstract: Localized Representation Steering (LRS) is widely used to correct reasoning pathologies in large language models. However, standard benchmark evaluation

FORESIGHT-9: Prospective and Process-Aware Evaluation of Adaptive Trading Agents

arXiv:2608.29372v1 Announce Type: new Abstract: Retrospective backtests provide a limited test of adaptive trading agents: they cannot rule out historical contamination, expose sensitivity to a single

Looking Again: Measuring Sycophancy in the Reasoning Chains of Multimodal Models Under Pressure

arXiv:2608.28623v1 Announce Type: cross Abstract: Large multimodal reasoning models (LMRMs) are getting increasingly capable, primarily through generating explicit chain-of-thought reasoning before an

Multi-Agent Self-Improving Reinforcement Learning for Video Reasoning

arXiv:2608.28675v1 Announce Type: cross Abstract: Video reasoning tasks such as grounded video question answering and temporal grounding require selecting temporal evidence that supports the query. In

N_0-Foundation: Towards the Age of Tactile Intelligence

arXiv:2608.29601v1 Announce Type: cross Abstract: We present N_0-Foundation, a paradigm for tactile-enabled embodied manipulation, which integrates tactile sensing hardware, large-scale multimodal dat

PaperBanana-Interact: Scientific Diagram Refinement with Multi-Turn Human Feedback

arXiv:2608.30241v1 Announce Type: new Abstract: Recent efforts have aimed to automate scientific diagram generation from paper content (Lin et al., 2026; Zhu et al., 2026a). However, fully satisfying

Reactivating Test-Time Scaling for Plane Geometry Problem Solving

arXiv:2608.30156v1 Announce Type: new Abstract: Plane geometry problem (PGP) solving has become a critical benchmark for multimodal reasoning because it requires accurate visual perception and precise

Real-Time Deadlines Reveal Fragile Temporal Adaptation in LLM Strategic Dialogues

arXiv:2601.13206v2 Announce Type: replace Abstract: Large Language Models (LLMs) generate text token-by-token in discrete time, yet real-world communication, from therapy sessions to business negotiat

SegWave: Wavelet-Driven Segmentation of Tampered Regions

arXiv:2608.30714v1 Announce Type: new Abstract: Verifying image authenticity is increasingly difficult, posing serious risks across journalism, law enforcement, and political domains. Most existing fo

Who Flips? Self- and Cross-Model Counterarguments Reveal Answer Instability in LLMs

arXiv:2606.16011v2 Announce Type: replace Abstract: Standard accuracy benchmarks evaluate whether large language models (LLMs) reach correct answers. However, they do not test whether models maintain

Denoising-Aware Temporal Point Cloud Completion for 3D Crop Architecture Recovery and Phenotypic Trait Extraction

arXiv:2608.28343v1 Announce Type: new Abstract: High-throughput phenotyping depends on accurate 3D reconstruction of plants across growth stages, yet the development and evaluation of temporal complet

31 Aug 2026

Safety

Medical Imaging AI Competitions Lack Fairness

arXiv:2512.17581v2 Announce Type: replace Abstract: Benchmarking competitions are central to the development of artificial intelligence (AI) in medical imaging, defining performance standards and shap

safetyarxiv-cs-cv

31 Aug 2026

// Your LLM judge disagrees with the experts // LLM Judges can be tricky to build. Here is an interesting showcasing why: There propose a re…

// Your LLM judge disagrees with the experts // LLM Judges can be tricky to build. Here is an interesting showcasing why: There propose a reference-full benchmark of hundreds of complete human-to-huma

model-releasesdair-ai--x

30 Aug 2026

ADeptS-Bench: Measuring the Trustworthiness of Computer Use Agents Across Devices

arXiv:2608.26204v1 Announce Type: cross Abstract: Computer Use Agents (CUAs) are increasingly deployed to navigate mobile and desktop applications on behalf of users, yet no benchmark comprehensively

Aphanta: Diagnosing Task-Aligned Image-Edited Intermediates for Multimodal Reasoning

arXiv:2608.26993v1 Announce Type: new Abstract: Explicit visual intermediates can help multimodal large language models (MLLMs) externalize spatial evidence and updated visual states, but their utilit

Legal Rule Induction: Towards Generalizable Principle Discovery from Analogous Judicial Precedents

arXiv:2505.14104v2 Announce Type: replace Abstract: Legal rules encompass not only codified statutes but also implicit adjudicatory principles derived from precedents that contain discretionary norms,

R2M-Bench: Evaluating Revisit Memory via Relative Consistency in Interactive Video World Models

arXiv:2608.27328v1 Announce Type: new Abstract: High similarity between first-visit and return frames does not necessarily show that a video world model remembered the scene; the intervening rollout m

b10649

model-releasesllama-cpp-releases

spec: Add benchmark-only synthetic speculative acceptance options (#27711) Add benchmark-only synthetic speculative acceptance to llama-server and llama-cli Address review comments Address review comm

27 Aug 2026

SeVeR: Selective Visual Exposure and Retrieval for 3D Medical Image Question Answering

arXiv:2608.25630v1 Announce Type: new Abstract: Volumetric medical VQA requires reasoning over long and redundant 3D visual token sequences, especially in multi-sequence MRI where complementary modali

27 Aug 2026

When Does Frequency Decomposition Benefit Physics-Informed Neural Networks? A Preliminary Ablation Study

model-releasesarxiv-cs-lg

arXiv:2608.24940v1 Announce Type: new Abstract: Partial differential equations (PDEs) often have high-frequency and multi-scale features that neural networks struggle to approximate. Physics-Informed

27 Aug 2026

Cross-Generation Optimization of YOLOv26, YOLOv11, and YOLOv8 for Fine-Grained Small-Object Detection and Instance Segmentation in Complex Orchards

arXiv:2608.23636v1 Announce Type: new Abstract: Small-object detection and instance segmentation remain challenging in orchard environments because of green-on-green similarity, occlusion, and limited

26 Aug 2026

'Ghaib in Translation' aka Unseen Harm: Measuring Cross-Script Safety Inconsistency with 'Missed-in-Urdu' Scores in LLM Hate Speech Detection

arXiv:2608.24191v1 Announce Type: cross Abstract: Urdu, the world's tenth most spoken language with 246 million speakers, remains almost entirely absent from mainstream LLM safety evaluation and nine

26 Aug 2026

GUI-Primitives: Diagnosing Spatial Reasoning Failures in Vision-Language GUI Grounding

arXiv:2608.21832v1 Announce Type: new Abstract: Computer-use agents ground natural-language instructions in screenshots to locate interface elements, yet existing benchmarks do not isolate whether mod

25 Aug 2026

When Does AI for PDEs Yield Scientific Evidence?

arXiv:2608.22504v1 Announce Type: new Abstract: Existing AI-for-PDE benchmarks primarily assess models in terms of predictive or approximation accuracy. In physics research, however, AI outputs often

25 Aug 2026

AgentMercury: Your Agent Can Synthesize Verifiable Environments for Business Scenarios at scale

arXiv:2608.20634v1 Announce Type: cross Abstract: Agents learn to act through interaction with environments, yet the environments used for training are often manually constructed or synthesized around

24 Aug 2026

Identify, Locate, Link: End-to-End Key-Value Extraction from Document Images

arXiv:2608.20868v1 Announce Type: cross Abstract: Document processing pipelines traditionally cascade optical character recognition (OCR) engines with downstream models for structured information extr