Search: “benchmark”

Game2Real-Bench: Do Toy Games Reveal and Transfer to Real-World Visual Reasoning?

arXiv:2610.18834v1 Announce Type: new Abstract: Visual reasoning is an important capability for vision-language models (VLMs) to understand visual evidence, spatial structure, constraints, and state c

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Grables: When Graph Structure Helps Tabular Prediction

DGX agent

arXiv:2602.03945v2 Announce Type: replace Abstract: Simple row-wise predictors that score each row independently remain competitive with far more expressive architectures on tabular benchmarks, which

model-releasesarxiv-cs-lg

14 Oct 2026

Research

Image Diffusion Models Are Good 3D Human Motion Generators

arXiv:2610.14254v1 Announce Type: new Abstract: Text-to-motion generation typically relies on dataset-specific skeletal representations, with models trained separately for benchmarks with different jo

researcharxiv-cs-cv

IND-Eval: Aligning Evaluation and Simulation for Industrial Time Series Foundation Models

arXiv:2610.16652v1 Announce Type: new Abstract: We introduce IND-Eval, a public benchmark that organizes industrial time series across domains into source-specific forecasting tasks. Relative to estab

Applications

Is Complex Query Answering Really Complex?

arXiv:2410.12537v4 Announce Type: replace Abstract: Complex query answering (CQA) on knowledge graphs (KGs) is gaining momentum as a challenging reasoning task. In this paper, we show that the current

applicationsarxiv-cs-lg

ManipShield: A Unified Framework for Image Manipulation Detection, Localization and Explanation

arXiv:2511.14259v4 Announce Type: replace Abstract: With the rapid advancement of generative models, powerful image editing methods now enable diverse and highly realistic image manipulations that far

Robust Logical Anomaly Detection under Capture Shifts via Synthetic Anomalies

arXiv:2610.18295v1 Announce Type: new Abstract: Detectors of logical anomalies, such as missing, extra or wrongly combined parts, usually score a test image by its distance from normal (good) images.

SceneForesee: Learning Scene Evolution for Spatiotemporal Reasoning

arXiv:2610.14439v1 Announce Type: new Abstract: Large multimodal models (LMMs) need to understand how entities move and interact as a scene develops. This requires following the same entities over tim

ScenePercept: Scenario-Conditioned Perceptual Assessment of Real-World User-Generated Content Images

arXiv:2610.17661v1 Announce Type: new Abstract: The rapid development of mobile imaging and social platforms has led to explosive growth in user-generated content images, making their systematic and c

scHFBiMamba: a bidirectional Mamba autoencoder for cardiac disease-state classification from single-nucleus RNA-seq data

arXiv:2610.15117v1 Announce Type: new Abstract: To evaluate whether a compact bidirectional state-space autoencoder improves joint classification of species, cell type, and disease state from single-n

Sensitivity or Criterion? Measuring and Moving When Language Models Act

arXiv:2610.15415v1 Announce Type: cross Abstract: Before an agent acts, it must decide whether action is warranted. A single accuracy cannot distinguish failure to recognize a warranted action from re

Text-Vision Co-Instructed Image Editing

arXiv:2606.16767v2 Announce Type: replace Abstract: Existing image editing methods can be generally categorized into textual instruction-based and visual prompt-based ones. Textual instructions are se

Agents

There are more document parsing models than ever, and they’re getting harder to tell apart. That’s why we built OpenDocRouter. Every model w…

There are more document parsing models than ever, and they’re getting harder to tell apart. That’s why we built OpenDocRouter. Every model we serve is benchmarked on ParseBench, with latency and cost

agentsjerry-liu--x

XForecast: The KDD 2026 Finance Day Challenge on News-Grounded Financial Forecasting

arXiv:2610.16131v1 Announce Type: new Abstract: Financial forecasting has traditionally relied on numerical time-series (TS) signals, while textual information such as news remains largely underutiliz

Chatgpt Astra has been gutted. That's what the people living in the walls tell me.

It's confirmed. OpenAI has been secretly replacing Astra with increasingly shit models since release. My evidence is overwhelming. Three Reddit comments, a benchmark that fluctuated 4%, and a guy who

model-releasesr-chatgpt

13 Oct 2026

DigitalOcean Inference Router Enhancements: Performance Tuned Routing, No Rules to Write

model-releasesdigitalocean

Every few weeks, a new model steps up and claims best-in-class at something. The launch chart makes the case, the link makes the rounds, and your team is back to the question nobody has time for: shou

13 Oct 2026

GPT-4 was the undisputed best model in the world for a full year and we all debated whether anyone could catch up or if OpenAI had some secr…

model-releasesethan-mollick--x

GPT-4 was the undisputed best model in the world for a full year and we all debated whether anyone could catch up or if OpenAI had some secret formula that nobody else knew about. Now there are are 19

13 Oct 2026

Language Models Have Interpretable Narrative Mechanisms

arXiv:2610.12989v1 Announce Type: new Abstract: We investigate whether language models contain internal components that contribute to representing narrativity, the degree to which a text exhibits narr

MetaJoin: Adaptive Join Selections for Heterogeneous Tables

arXiv:2512.02444v2 Announce Type: replace-cross Abstract: Joining heterogeneous tables requires different forms of correspondence, including exact agreement, syntactic transformation, semantic similar

ReasoningShield: Safety Detection over Reasoning Traces of Large Reasoning Models

arXiv:2505.17244v3 Announce Type: replace-cross Abstract: Large Reasoning Models (LRMs) leverage explicit reasoning traces, known as Chain-of-Thought (CoT) reasoning, to decompose complex problems int

REFLECT: Role-Aware Detection of Sentiment, Emotion, Politeness, and Formality

arXiv:2610.13797v1 Announce Type: new Abstract: Detecting communicative meaning in dialogue requires modeling not only lexical content but also the participant-role structures through which sentiment,

Travel Back To Move Forward: Retrospective Exploration and Evolving for Long Horizon Agents

arXiv:2610.13058v1 Announce Type: new Abstract: Large language model agents increasingly operate in long-horizon environments where early mistakes can trigger cascading failures and trap agents in dea

An Explainable Header-Centric Framework for Large-Scale Semantic Table Interpretation and Data Quality Assessment

arXiv:2610.10541v1 Announce Type: new Abstract: Knowledge Graph (KG) quality depends not only on downstream graph validation, but also on the quality of tabular metadata used before integration. In me

BrickBench: Evaluating Agentic Brick Design

arXiv:2610.12452v1 Announce Type: new Abstract: We propose BrickBench, a benchmark for agentic text-conditioned LEGO-set design. Given a prompt, an agent is tasked with producing an assembly that not

Obscuring Data Contamination Through Translation: Evidence from Arabic Corpora

arXiv:2601.14994v2 Announce Type: replace-cross Abstract: Data contamination can invalidate benchmark evaluation when a model benefits from memorized evaluation content rather than genuine generalizat

Reasoning-Informed Visual Editing

arXiv:2610.12343v1 Announce Type: new Abstract: Large Multi-modality Models (LMMs) have made significant progress in visual understanding and generation, but still face challenges in visual editing, p

SWE-Journey: Towards More Realistic Evaluation of Coding Assistants through Long-Horizon, Multi-Turn Interaction

arXiv:2610.11559v1 Announce Type: cross Abstract: Coding assistants such as Claude Code and Codex have become a major application of LLM agents, yet existing benchmarks remain far from real-world use,

WOVEN: Weaving Visual World Modeling into Multimodal LLMs

arXiv:2610.12417v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) struggle with spatial, embodied, physical, and temporal reasoning. We hypothesize that these failures reflect

Conditional Accuracy Profiles: Diagnosing LLM Judges across Deployment Conditions

arXiv:2610.09229v1 Announce Type: new Abstract: LLM-as-judge is now a standard tool for scalable evaluation, but judge performance is still often summarized by a single accuracy number. This aggregate

LightOn OCR-3 is now available on OpenDocRouter, priced at 0.28 / 1m input and 1.40 / 1m output (it is ~$3.19 / 1k pages on ParseBench). W…

model-releasesjerry-liu--x

LightOn OCR-3 is now available on OpenDocRouter, priced at 0.28 / 1m input and 1.40 / 1m output (it is ~$3.19 / 1k pages on ParseBench). We benchmarked it on ParseBench. It is on the pareto frontier f

LLM4Impact: Integrating Heterogeneous Information for Scientific Impact Prediction

arXiv:2610.10138v1 Announce Type: new Abstract: Predicting the future impact of a newly published paper is challenging because it must be inferred from heterogeneous evidence available at publication

Quad-State Safety Evaluation of Open-Weight Large Language Models on Non-Canonical Inputs

arXiv:2610.09033v1 Announce Type: new Abstract: Standard safety evaluations of large language models assess harmful requests written in canonical plain text, while models in real-world deployment rout

VideoZeroBench: Probing the Limits of Video MLLMs with Spatio-Temporal Evidence Verification

arXiv:2604.01569v2 Announce Type: replace Abstract: Video multimodal large language models achieve strong results on existing benchmarks, but answer accuracy alone does not establish whether they can

2x CMP 170HX 64GB: GLM-5.3-Flash at 384K context / ~90 tok/s (EXL3, HBM-first setup) + Qwen3.8 comparison

model-releasesr-localllama

I've been tinkering with GLM-5.3-Flash on two 64GB CMP 170HX cards for a while, and the setup is finally stable enough that I figured I'd share it. I also compared it against the Qwen3.8-Flash-Next se

BoT-Feedback: Grounding Multimodal Reasoning in Biomechanical Evidence for Explainable Human Action Feedback

arXiv:2610.06972v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in visual understanding and multimodal reasoning, yet they remain fun

CroissantMiner: Automated Extraction and Validation of Croissant Metadata for ML Datasets

arXiv:2610.07132v1 Announce Type: cross Abstract: Croissant has emerged as a standard for machine-readable dataset metadata, yet populating its fields remains labor-intensive and requires careful read

Harness Engineering for Software Engineering via Modular Executable Dev-Primitives

arXiv:2610.07832v1 Announce Type: cross Abstract: Large language models (LLMs) equipped with terminal access have demonstrated strong capabilities in automating software engineering tasks. However, ex

Image Bitstream Fine-grained Understanding for Privacy-Friendly AIoT

arXiv:2610.08414v1 Announce Type: new Abstract: Image Bitstream Fine-grained Understanding (IBFU) aims to directly perform fine-grained classification and semantic description generation from encoded

Measuring climate backlash in Twitter and Reddit archives: Lexical definitions, recorded responses and participant turnover

arXiv:2610.07634v1 Announce Type: new Abstract: Social media archives are often used to study resistance to climate action, but words, response counters and observed participants do not measure the sa

Medical Image Alignment Assessment as a Test of Generalist Visual Reasoning in Frontier Multimodal Models

arXiv:2610.06896v1 Announce Type: new Abstract: Frontier multimodal large language models (MLLMs) are increasingly positioned as general purpose visual reasoners as part of the quest for artificial ge

MemMux: Runtime Verification and Honest Resource Attribution for Fleets of Parallel Coding Agents

arXiv:2610.07257v1 Announce Type: new Abstract: Developers increasingly run a fleet of coding agents side by side on one workstation. The tools they reach for, terminal multiplexers like tmux and a ne

Same-Number Citation Swaps: Stress-Testing Jev as a Financial Evidence Judge

arXiv:2610.08675v1 Announce Type: new Abstract: Financial reports repeat values across periods, metrics and accounting lines, allowing an LLM-generated calculation to be numerically correct while citi

Seeing Through the Displaced Frame: Privileged Noise Distillation for Vision-Force Precision Assembly

model-releasesarxiv-cs-ro

arXiv:2610.07745v1 Announce Type: new Abstract: Pose error in precision assembly can corrupt not only what a robot observes but also the coordinate frame in which it acts. On the FORGE benchmark, the

Too Categorical to be Human: Emotion Concepts in LLMs and Humans

arXiv:2508.05880v3 Announce Type: replace-cross Abstract: Understanding human emotions is central to user-facing AI applications, safety alignment, and the simulation of human behavior. As emotional s

Why is ik_llama.cpp said to be faster than Mainline? On my hybrid multi-GPU rig, Mainline easily beats it

model-releasesr-localllama

I constantly see recommendations saying that ik_llama.cpp (ikawrakow's fork) is the undisputed king of hybrid CPU/GPU offloading and MoE performance. However, every time I benchmark it against mainlin

WorldSolver: Can LLM Agents Simulate the Physical Dynamics via Solver Generation?

arXiv:2610.08720v1 Announce Type: new Abstract: LLM-based agents are increasingly advancing scientific and engineering problem solving, with physics simulation emerging as a challenging yet practical

zkLLMPoT: Efficient Zero Knowledge Proof of Training for Large Language Models

arXiv:2610.08258v1 Announce Type: new Abstract: Auditing the claimed outcomes of large language model (LLM) training is challenging when model weights and training data are private, while cryptographi

BrainTRACE: Tracing Longitudinal, Multimodal, and Volumetric Evidence in Brain MRI Clinical Reasoning

arXiv:2610.06571v1 Announce Type: new Abstract: Brain MRI interpretation is a longitudinal clinical reasoning problem: radiologists compare serial studies, integrate information across MRI sequences,