Search: “benchmark”

Evaluating Escalation Signals for LLM Routing: Targets, Controls, and Five Ways to Fool Yourself

arXiv:2610.07354v1 Announce Type: new Abstract: Deciding when to escalate a query from a small language model to a larger one requires a cheap signal that predicts, before the large model is called, w

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Even Small Reasoners Should Quote Their Sources: Introducing the Pleias-RAG Model Family

DGX agent

arXiv:2504.18225v2 Announce Type: replace Abstract: We introduce a new generation of small reasoning models for RAG, search, and source summarization. Pleias-RAG-350m and Pleias-RAG-1B are mid-trained

model-releasesarxiv-cs-cl

7 Oct 2026

Evidence-Bound Reasoning: Neuro-Semantic Verification of Biomedical AI in Glioblastoma Radiogenomics

model-releasesarxiv-cs-cl

arXiv:2610.08660v1 Announce Type: new Abstract: Background: Biomedical AI can generate plausible explanations without reliably verifying whether each statement is supported by patient-specific evidenc

Geometric Coherence via Weighted Matching for 3D Heterogeneous Multi-Agent Reach-Avoid Games

model-releasesarxiv-cs-ro

arXiv:2610.06882v1 Announce Type: new Abstract: We study assignment quality in 3D heterogeneous multi-agent reach-avoid games and identify a recurring failure mode of cardinality-only matching in geom

Interpretable Hypergraph Learning via Neural Additive Models

arXiv:2610.07458v1 Announce Type: new Abstract: Hypergraphs offer a natural framework for modeling networked data, where dependencies among entities are governed by higher-order interactions. While hy

Looking for developer-friendly inference providers who give you enough API credits to experiment [D]

model-releasesr-machinelearning

I’m hitting rate limits on Together AI. For context, I’ve been working on an agentic repository indexing and benchmark generation tool, and I’m running multiple agents in parallel across models like L

MedZERO: Self-Evolving Agents for Open-Ended Medical Reasoning Through Controlled Knowledge Accumulation

arXiv:2610.08327v1 Announce Type: new Abstract: Large language models (LLMs) have shown promise in medical question answering and clinical reasoning, yet their improvement remains constrained by stati

researcharxiv-cs-ai

Multimodal Knowledge Distillation for Gastric Adenocarcinoma Classification from Whole-Slide Images

arXiv:2610.07913v1 Announce Type: cross Abstract: Gastric adenocarcinoma (GA) is a leading cause of cancer-related mortality worldwide, and accurate histopathological subtype classification from whole

PERSIST: Who-What-When Memory Across Sessions for Full-Duplex Spoken Dialogue

arXiv:2610.07725v1 Announce Type: new Abstract: Modern voice assistants may be shared by multiple users and should be able to answer questions about earlier conversations such as 'When did I originall

Personal-Agent Mediated Recommendation with Cross-Platform User History

arXiv:2610.07588v1 Announce Type: new Abstract: Modern recommendation is shifting from platform-centric personalization toward user-governed personalization, where a personal LLM agent can act on the

Real-Time Generation of Game Video Commentary with Multimodal LLMs: Pause-Aware Decoding Approaches

arXiv:2603.02655v2 Announce Type: replace-cross Abstract: Real-time video commentary generation provides textual descriptions of ongoing events in videos. It supports accessibility and engagement in d

ReGraph: A Computational Account of Emergent Generalization in the 'what' and 'where' Dual Visual Streams

arXiv:2610.07962v1 Announce Type: cross Abstract: Where generalization capacity--the ability to extract context-invariant relational structures--first emerges remains a central question in AI and neur

Single 3090 Qwen 27B user, considering buying 128GB of RAM because of the hype

My current setup: - single 3090 running turboderp/Qwen3.8-27B-exl3:SC_5.00bpw_H6_V6 - ~150k context, ~60t/s, unknown prefill because I didn't benchmark it (but it is ok) - Intel 12400 CPU with 32GB DD

Uncertainty in Representation Learning on Knowledge Graphs

arXiv:2610.06974v1 Announce Type: new Abstract: Knowledge graph embedding (KGE) methods represent entities and predicates in continuous vector spaces to infer missing knowledge. Despite strong benchma

Verifying Coordination in Parallel Coding Agents: NP-Bench and a Scheduling Planner

arXiv:2610.07261v1 Announce Type: new Abstract: A team of coding agents can look fine agent by agent yet fail as a team: each passes its own tests while the merged result is broken, and single-agent e

We show how you can use the River API to fine-tune open weight models to beat GPT-6 Astra and Claude Opus 5.5 on tasks you care about, with …

model-releasesclem-delangue--x

We show how you can use the River API to fine-tune open weight models to beat GPT-6 Astra and Claude Opus 5.5 on tasks you care about, with less than 1% of the cost! A few years from now most of the L

What the Elevation Map Cannot See: Semantic-Aware Locomotion and Execution-Aware Navigation for Humanoid Robot

model-releasesarxiv-cs-ro

arXiv:2610.07396v1 Announce Type: new Abstract: Navigation for humanoid robots is critical, yet large-scale evaluation on physical hardware is often impractical due to cost and safety concerns, making

All against the machine: the Solo score for rating skill in variable environments

arXiv:2610.04523v1 Announce Type: cross Abstract: We propose a distribution-free metric to rate individual skill in ``player-versus-environment'' settings, where participants face heterogeneous tasks

Assembling Insights for Agentic Machine Learning Engineering Systems

arXiv:2610.04927v1 Announce Type: cross Abstract: Agentic machine learning engineering (MLE) is an emerging AI4SE application for complex ML tasks and a step toward recursive self-improvement of AI sy

BossouChimpanzee: Long-term Chimpanzee Video Dataset

arXiv:2610.05293v1 Announce Type: new Abstract: We describe the BossouChimpanzee video dataset, a unique long-term visual record of wild chimpanzees at an outdoor laboratory for field experiments in B

Causal Lag Structure Discovery in Confounded Time Series via Orthogonalized Adaptive Estimation

arXiv:2610.05618v1 Announce Type: cross Abstract: Finding which variables cause which others in multivariate time series, and at what lags, is central to science and policy, yet existing methods force

Code2Games: Enabling Coding Agents for Gaming World Generation

arXiv:2610.05033v1 Announce Type: new Abstract: Generating a high-quality gaming world from a natural-language game intent requires joint reasoning about scene structure, spatial layout, gameplay obje

CodeForge-MA: Execution-Verified Multi-Agent Learning with Language-Conditioned LoRA for Multilingual Code Generation

arXiv:2610.05481v1 Announce Type: cross Abstract: Large language models for code generation often fail on execution, multilingual coverage, and contamination control, especially under frozen backbone

CoHyFuse: Condition-wise Hypergraph Fusion with Global Connectome in Task-fMRI

arXiv:2610.05913v1 Announce Type: new Abstract: Task-fMRI connectomes reveal state-dependent neural reconfigurations, yet conventional methods marginalize these signals by aggregating distinct conditi

DelegationBench: Measuring When AI Agents Should Ask Before Acting

arXiv:2610.05532v1 Announce Type: cross Abstract: AI agents that send emails, edit files, and make purchases must decide when to act on their own and when to check with the user first. This decision i

Disentangling Task Difficulty from Run-Level Failure in Agent Failure Prediction

arXiv:2610.05572v1 Announce Type: new Abstract: Predicting whether an LLM agent will fail has emerged as a promising direction for supporting intervention during execution. Recent approaches report st

FADE: Frame-Aware Diffusion-Transformer-based Multi-Concept Erasure for Video Unlearning

arXiv:2610.03980v1 Announce Type: new Abstract: Text-to-video (T2V) diffusion models can reproduce copyrighted, violent, or explicit content, which motivates concept erasure: removing designated conce

Homogeneous Semantic Alignment and Hierarchical Expert Routing for Radiology Report Generation

arXiv:2610.04499v1 Announce Type: new Abstract: Radiology report generation (RRG) aims to convert medical images into diagnostic texts to assist in clinical decision-making and alleviate the workload

Lossy Compression of PDE Training Inputs: Field Reconstruction Error Does Not Order the Cost to a Trained Operator

arXiv:2610.06095v1 Announce Type: new Abstract: Operator-learning benchmarks are stored at full precision and have grown to terabyte scale. Rate-distortion theory says how many bits the stored field n

researcharxiv-cs-lg

Meet Mistral Large 4, aka Le Chonk. • 1T parameters, natively multimodal. 49B active. It is the best open weights model from US or Europe on…

model-releasesclem-delangue--x

Meet Mistral Large 4, aka Le Chonk. • 1T parameters, natively multimodal. 49B active. It is the best open weights model from US or Europe on aggregated benchmarks. • State-of-the-art on critical workl

Hardware

MetaKernelBench: Measuring GPU Kernel Knowledge Transfer Beyond Code

arXiv:2610.05014v1 Announce Type: new Abstract: Recent GPU kernel optimization agents retain what they learn in knowledge bases or as distilled skills. Kernel benchmarks score each attempt's implement

hardwarearxiv-cs-lg

MiMo V2.6 Pro almost matched GPT-6 Astra at figuring out what a user actually wants

Introducing InferBench: A benchmark testing how well frontier LLMs infer a user's priorities from their instructions. We tested 12 LLMs across 20 scenarios with 2.8k conversations to see which models

Mistral Large 4

model-releasessimon-willison

My comment on Mistral Large 4 — Hacker News. wren6991: The benchmark is saturated. Frontier models are tested with an armadillo in fishnet tights jaywalking on Mars. OK well I couldn't resist this one

One Figure, Every Canvas: Editable Flowchart Relayout via Agentic Pipeline

arXiv:2610.06852v1 Announce Type: new Abstract: Pipeline figures in ML papers must be repurposed across many canvases, including paper columns, 16:9 slides, portrait posters, 1:1 social teasers, 9:16

OVAL: Output-Aware Local Page Bases for KV Cache Retrieval

arXiv:2610.06686v1 Announce Type: new Abstract: Long context inference with large language models becomes increasingly expensive as attention must operate over an ever growing KV cache. Page sparse at

pplx-decider-v1.1-27b, our updated open weights multimodal decision model, is now available. It scores the highest on the new @huggingface D…

model-releasesperplexity--x

pplx-decider-v1.1-27b, our updated open weights multimodal decision model, is now available. It scores the highest on the new @huggingface Decision Index 0.3 benchmark. pplx-decider-v1.1-27b costs hal

Protocol-Sensitive Evaluation of Log Anomaly Detection: Component Costs and Target-Access Sensitivity on HDFS and BGL

arXiv:2610.05807v1 Announce Type: cross Abstract: Protocol choices can change the conclusions drawn from log anomaly detection benchmarks even when detector settings are fixed. We present a joint empi

researcharxiv-cs-lg

Qwen 3.8 27b just feels… ok?

I’ve seen posts here raving about how good Qwen 3.8 27b is. The benchmarks look incredible, and all the online discourse seems to deem it the best local model. I have 32GB VRAM and run Unsloth’s Q6 ve

RefAtomNet++: Advancing Referring Atomic Video Action Recognition using Multi-Trajectory Semantic Retrieval

arXiv:2510.16444v2 Announce Type: replace Abstract: Who is being described, where are they, and what are they doing? Referring Atomic Video Action Recognition (RAVAR) answers these questions jointly b

Rethinking Tabular Foundation Models On Data Streams

arXiv:2610.05352v1 Announce Type: new Abstract: Tabular foundation models (TFMs) outperform established machine learning models on tabular benchmarks through in-context learning. Building on this succ

researcharxiv-cs-lg

SepRQ : Self-Supervised Speech Mixture Representation Learning via Mask-Free, Multi-Scale Source Separation

arXiv:2610.04690v1 Announce Type: cross Abstract: Self-supervised learning (SSL) is standard for speech representation learning, but mainstream models are designed around single-speaker audio, limitin

Sparse-View 4D Gaussian Splatting via Spatiotemporal Priors and Generative Assistance

arXiv:2610.04606v1 Announce Type: new Abstract: We present a 4D Gaussian Splatting framework for the Sparse-View Track of the SIGGRAPH Asia 2026 Volumetric Video Challenge, which requires dynamic scen

Video Encoders Built on Image Representations

arXiv:2610.06616v1 Announce Type: new Abstract: The design of a video encoder determines when frames begin to interact and which frame-specific visual evidence remains accessible to the language model

VideoResearchAgent: Grounded Task Synthesis and Sim-to-Real RL for Open-Web Video Research

arXiv:2610.04911v1 Announce Type: cross Abstract: Existing deep research agents are designed primarily for text- and image-based web sources, while video reasoning systems typically assume that releva

We’re so back.

model-releaseslm-studio--x

We’re so back. Meet Mistral Large 4, aka Le Chonk. • 1T parameters, natively multimodal. 49B active. It is the best open weights model from US or Europe on aggregated benchmarks. • State-of-the-art on

What Does a Harness Repair? A Preregistered Study of Visibility, Baseline Adequacy and Evaluation Defects

arXiv:2610.05533v1 Announce Type: new Abstract: Harness search keeps a change to the prompts, reasoning switches, token budgets or parsers around a frozen model if the change raises a score. Such a ga

What Does It Cost to Simulate a Quantum Sentence Classifier? An Energy and Compute Perspective on Near-Term QNLP

arXiv:2610.06176v1 Announce Type: cross Abstract: Near-term quantum natural language processing (QNLP) experiments often run on classical simulators, so simulator cost is part of the field's practical

Why 38% of AI Agent container escapes didn't need kernel 0-days: Analysis of 109 empirical incidents (Open Dataset + Defense Harness)

Over the past several months, we conducted an empirical post-mortem investigation into 109 autonomous AI agent security incidents (cataloged with 193 falsification criteria across tool-use and multi-a