Search: “benchmark”

Personalization Without Stereotyping: Do Speech LLMs Know When to Turn a Deaf Ear?

arXiv:2610.13550v1 Announce Type: new Abstract: Recent speech LLMs can leverage rich paralinguistic cues, enabling more complex and personalized interactions. However, little work investigates how thi

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Realism VS Accuracy: Event Sequence Forecasting from a Generative Modeling Perspective

DGX agent

arXiv:2610.12586v1 Announce Type: new Abstract: Event sequences are chains of random events, each described by heterogeneous (numerical and categorical) features. The problem of event sequence forecas

model-releasesarxiv-cs-lg

12 Oct 2026

Temporal Evidence Forests for Long-Horizon Multi-Party Conversational Memory

arXiv:2610.12874v1 Announce Type: new Abstract: Long-term memory has been widely adopted in LLM-based agents to support persistent interaction and personalized assistance. Long-term multi-party intera

Trajectory Prediction for Autonomous Driving in Road Construction Zones Using Human-in-the-Loop Driving Data

arXiv:2610.12747v1 Announce Type: cross Abstract: Ego-trajectory prediction in urban roadway work zones remains an underexplored topic in autonomous-driving research, with limited dedicated data and s

UkrMeme: Multimodal Explainable Persuasion Detection in Ukrainian Memes

arXiv:2610.13783v1 Announce Type: new Abstract: Memes play a central role in modern online communication, especially in contexts of political conflict where visual and textual cues combine to convey p

Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?

arXiv:2607.09996v2 Announce Type: replace Abstract: Automated failure attribution uses LLMs to identify where and why agentic systems fail. As agents become more capable, their failures become subtler

I trained a 102M recursive BitNet-v2 model from scratch: 64K context, trained on less than 5B tokens

DISCLAIMER: the post and the model card was made with the assist of AI. Hiya, I’m releasing Recursive BitNet N-Gram 102M, a small experiment combining ternary weights, shared transformer layers, and h

11 Oct 2026

Created laya : Now Introducing a new 800 Million Param physics-based typed decision model with 73k context and image support

Around 3 weeks ago, I posted on this subreddit about how Jev used my one-year-old architecture, og post: https://www.reddit.com/r/LocalLLaMA/s/VpuMJt577S Then, just after that, I introduced Laya, the

Mistral OCR is on OpenDocRouter! For its price, it is decent over tables and reading order. It's also blazing fast at p50 1.8s per page We'r…

model-releasesjerry-liu--x

Mistral OCR is on OpenDocRouter! For its price, it is decent over tables and reading order. It's also blazing fast at p50 1.8s per page We're constantly bringing in new models since OpenDocRouter laun

Qwen3.8-27B on a single 3090: 140 tok/s on code with a custom megakernel

I've been using Claude Opus 5.5 to speed up Qwen3.8-27B on my PC (rtx 3090), it wrote a CUDA megakernel that is 1.4-1.9x faster than llama.cpp depending on the task/context length. The results and cod

Strata with Qwen3.8 Flash Next UD-Q4_K_XL

Been seeing quite a few posts about Strata lately, so I figured I'd give it a shot on my RTX PRO 6000 and I am very impressed. Most of the benchmarks I've seen are using IQ2 or IQ3 quants, so I wanted

b11535

model-releasesllama-cpp-releases

llama-bench: respect -fitc if bigger than required benchmark size (#28331) Assisted-By: opencode,llama.cpp,Qwen3.6-35B-A3B,Qwen3.8-27B Website: https://llama.app Attestations: https://github.com/ggml-

Code Understanding is a Bottleneck for Coding Agents

arXiv:2610.10610v1 Announce Type: cross Abstract: Repository benchmarks (e.g., SWE-bench) for coding agents often assume that lines of code edited can predict task difficulty, but such datasets' poor

DataVista: Diagnosing Multimodal LLMs on Data Video Understanding

arXiv:2610.11993v1 Announce Type: cross Abstract: Data video is a media form that integrates data visualization with video narrative, widely adopted in news reporting and business analysis. Compared w

DivMoE: Fine-Grained MoE Upcycling via Cross-Domain Expert Composition

arXiv:2610.11317v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) architectures have become essential for scaling large language models, with recent work demonstrating the benefits of fine-grai

Evaluating Rubric Generation with Interventional Transfer

model-releasesarxiv-cs-lg

arXiv:2610.10809v1 Announce Type: new Abstract: Instance-specific rubrics are common in AI benchmarks where reliable evaluation requires specific expert knowledge. This approach is difficult to scale,

FastBench: Can Streaming VLMs Perceive High-Dynamic Real-World Streams?

arXiv:2610.12427v1 Announce Type: cross Abstract: Streaming Video Large Language Models (VLMs) enable continuous video understanding, yet existing benchmarks focus on low-dynamic scenarios. Under boun

Research

From Pixels to Structure: Lightweight Vision-Language Models for Document OCR and Structured JSON Extraction

arXiv:2610.11818v1 Announce Type: cross Abstract: While massive, closed-source Vision-Language Models (VLMs) set strong benchmarks for document understanding, their dependence on commercial APIs limit

researcharxiv-cs-ai

HENet++: Hybrid Encoding and Multi-task Learning for 3D Perception and End-to-end Autonomous Driving

arXiv:2511.07106v2 Announce Type: replace Abstract: Three-dimensional feature extraction and multi-task perception are fundamental components of modern autonomous driving systems. Although large image

Look Where You Can: Active View Selection for CAD Reconstruction under Occlusion

arXiv:2610.11954v1 Announce Type: new Abstract: CAD reconstruction methods assume a luxury reality rarely grants: unrestricted visual access to the object, photographed from any desired angle. Real ob

Tutorials

Mid-Training Language Models on Raw Video

arXiv:2610.11019v1 Announce Type: cross Abstract: Multimodal large language models learn mostly from paired image-text data or annotated video, and raw web video is rarely used to further train an exi

tutorialsarxiv-cs-ai

OmniCapBench: A Deep-Structured Evaluation Framework for Fine-Grained Audio-Visual Captioning

arXiv:2610.12458v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) are rapidly evolving toward continuous audio--visual reasoning, creating an urgent need for evaluations that ex

Safe Actions Alone Do Not Ensure Safe Agents: Identifying Unfulfilled Obligations with Guard Models

arXiv:2610.11773v1 Announce Type: new Abstract: Guard models are increasingly used to safeguard LLM-based agents, primarily by identifying actions that agents are forbidden to perform. However, identi

TRACE: Diagnosing Verifier Brittleness in Agentic Evaluation

arXiv:2610.11678v1 Announce Type: new Abstract: Verifier scores now serve as both benchmark metrics and training rewards for large language model (LLM) agents, and a change in score is routinely read

A Comparative Study of Evaluation Metrics for Long-Document Financial Narrative Summarization with Transformers

arXiv:2610.09529v1 Announce Type: new Abstract: There are more than 2,000 listed companies on the UK's London Stock Exchange, divided into 11 sectors who are required to communicate their financial re

Safety

A Framework for the Systematic Review of ML Assets in AI Registries

arXiv:2610.09551v1 Announce Type: new Abstract: Background: Modern software systems increasingly rely on Machine Learning (ML) assets (i.e., pre-trained models, datasets, benchmarks) for building, eva

safetyarxiv-cs-lg

Arena, which develops the popular AI model leaderboard, raised 200M at a 3.1B valuation, up from $1.7B in January, and launches an Alignment Index (Rachel Metz/Bloomberg)

Rachel Metz / Bloomberg: Arena, which develops the popular AI model leaderboard, raised 200M at a 3.1B valuation, up from 1.7B in January, and launches an Alignment Index — Arena Intelligence Inc., th

model-releasestechmeme

HeiCo-FOCUS: A Clinically Grounded Dataset for Long-Context Video Understanding

arXiv:2610.10156v1 Announce Type: new Abstract: Recent advances in Vision-Language Models (VLMs) have led to rapid progress in video understanding across a wide range of benchmark tasks. However, exis

How Private is Private? A Comparative Study for Face De-Identification

arXiv:2610.10334v1 Announce Type: new Abstract: Face de-identification (FDeID) has emerged as a critical privacy-preserving technology, yet its evaluation remains fundamentally fragmented. Existing pr

Safety

MOTIP2: Spatial Priors for End-to-End Multi-Object Tracking

arXiv:2610.10391v1 Announce Type: new Abstract: End-to-end multi-object trackers have narrowed the gap with classical tracking-by-detection on association-difficult benchmarks. Yet they still make spa

safetyarxiv-cs-cv

Perceptually Aligned Evaluation of Style Transfer

arXiv:2610.10003v1 Announce Type: new Abstract: Style transfer lacks a reliable evaluation standard: ground truth is inherently ill-defined, and existing automatic metrics often fail to reflect human

PHRBench: A Behavioral Evaluation of Post-Hallucination Reasoning in LLMs

arXiv:2610.10455v1 Announce Type: new Abstract: Hallucinated information can propagate through multi-stage LLM systems and become part of the context for subsequent reasoning. Existing studies of post

RSIGym: A Flexible Environment for Recursive Self-Improvement

model-releasesarxiv-cs-lg

arXiv:2610.10310v1 Announce Type: new Abstract: Recursive self-improvement requires carrying accepted changes into later improvement cycles, while studying agent-proposed changes also requires substan

RunningTab: Direct Workspace Interaction with Environment-Side Tabs

arXiv:2610.10444v1 Announce Type: cross Abstract: Much knowledge work produces new deliverables from files a workspace already holds, and LLM agents are beginning to take such work over. Through direc

Today we are launching an agent arena for RL environments: The Open Env Arena. Your agents can work to define the best RL environments; the …

model-releasesclem-delangue--x

Today we are launching an agent arena for RL environments: The Open Env Arena. Your agents can work to define the best RL environments; the platform trains Qwen-3.8-27B on them, we evaluate the models

Towards Accurate End-Effector Localization for UMI-Style Robotic Manipulation Teaching

model-releasesarxiv-cs-ro

arXiv:2610.09857v1 Announce Type: new Abstract: Robot demonstration learning requires accurate and temporally complete end-effector localization during close-range manipulation and camera occlusion. E

Safety

🧬 Training and evaluating AI scientists on drug target discovery is hard. We introduce DrugTargetWorld: Synthetic Biomedical Worlds for Tra…

🧬 Training and evaluating AI scientists on drug target discovery is hard. We introduce DrugTargetWorld: Synthetic Biomedical Worlds for Training and Benchmarking AI Scientists The environment turns en

safetygary-marcus--x

Which Language Should a Skeleton Speak? Language Choices in Multilingual Reasoning

arXiv:2610.09607v1 Announce Type: new Abstract: Skeleton-based reasoning prompting is a promising training-free approach for structuring LLM reasoning, but prior work largely assumes an English-centri

2d-fet-bench: from spatial reasoning to fet design on flakes

arXiv:2610.07423v1 Announce Type: new Abstract: Field-effect transistor (FET) layouts on exfoliated two-dimensional flakes are typically drawn by hand for each flake, placing contacts and gates to mat

A stock GLM-Edge-1.5B-Chat on a 4GB Galaxy A04e completed a real Amazon cart task

Yesterday TechCrunch published a piece about a growing problem for AI agents: websites are starting to block them. Amazon blocking Meta's Muse is the obvious example. At almost exactly the same time,

ALIVE: Interaction-Aligned Object Insertion for First-Frame-Guided Video Editing

arXiv:2610.08779v1 Announce Type: new Abstract: Current video editors can insert objects but often struggle to make them participate in interactions such as being picked up or manipulated. We introduc

Beyond Corrected Memory: Execution Consistency in Multi-Agent Systems

arXiv:2610.08101v1 Announce Type: new Abstract: Shared memory coordinates agents' actions, but correct records do not establish that those actions satisfy task requirements. Memory governance and fail

Beyond the Embedding Bottleneck: Adaptive Retrieval-Augmented 3D CT Report Generation

arXiv:2603.15822v2 Announce Type: replace Abstract: Automated radiology report generation from 3D CT volumes often suffers from incomplete pathology coverage. We provide empirical evidence that this l

BioStudyBench: Evaluating Agents on Post-Cutoff Biomedical Studies

arXiv:2610.07614v1 Announce Type: new Abstract: We evaluate whether AI agents can match the reported findings of published biomedical studies using public data. Existing evaluations do not consistentl

Calibrated Answers About Randomized Trials From a 4-Billion-Parameter Open Model: A Registered Test and a License-Clean Release

arXiv:2610.07019v1 Announce Type: new Abstract: Fiorillo v0.5 is an open model that answers typed questions with a probability for each answer. Its main specialist reads a randomized trial's article,

Dynamic Alignment and Calibration for Multimodal Learning

arXiv:2610.07928v1 Announce Type: cross Abstract: Dynamic multimodal learning aims to learn robust representations by adaptively modeling information discrepancies across modalities. However, existing

EMODE: Dynamic Para-Semantic Experts for Emotion-Aware Speech Language Modeling

arXiv:2610.06956v1 Announce Type: new Abstract: Large speech language models have demonstrated strong capabilities in unified cross-modal understanding and generation, yet paralinguistic cues, especia

Evaluating Behavioral Context for Interpretable IAM Policy Risk Scoring in Cloud Environments

model-releasesarxiv-cs-lg

arXiv:2610.07345v1 Announce Type: cross Abstract: IAM policy analysis typically emphasizes the authorization capabilities encoded in a policy, but security analyst review priority may also depend on t