Search: “benchmark”

Listening to the Wise Few: Query-Key Alignment Unlocks Latent Correct Answers in Large Language Models

arXiv:2410.02343v2 Announce Type: replace Abstract: Large language models (LLMs) routinely fail to output the correct option in multiple-choice question answering (MCQA) while encoding the answer inte

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Lowest Span Confidence: Zero-Shot Hallucination Detection from a Single LLM Response

DGX agent

arXiv:2601.19918v2 Announce Type: replace Abstract: Hallucinations in Large Language Models (LLMs), i.e., plausible but non-factual generations, pose a significant challenge to reliable deployment in

model-releasesarxiv-cs-cl

1 Oct 2026

MASCRDM: Multi-Agent System for Compliance Risk Detection and Mitigation in Training Process of Large Language Models

arXiv:2609.39107v1 Announce Type: new Abstract: Large Language Models (LLMs) have been applied in various fields. However, ensuring compliance and safety of LLMs, such as avoiding discrimination and b

MatLoom: Layered Text-to-Material Generation in a Compact Program Space

arXiv:2609.40322v1 Announce Type: cross Abstract: Material generation should produce not only an appearance, but also the rules that construct it. We introduce MatLoom, a compact, layer-oriented langu

MedKIT: Evaluating Knowledge Integration and Generalization in Large Language Models

arXiv:2609.38543v1 Announce Type: new Abstract: Constantly evolving real-world knowledge necessitates models to be updated continuously. Especially in medicine, as clinical evidence changes over time,

NMIXX: Domain-Adapted Neural Embeddings for Cross-Lingual eXploration of Finance

arXiv:2507.09601v3 Announce Type: replace-cross Abstract: Financial text embeddings must distinguish changes in event status, perspective, and obligations even when passages share similar wording. NMI

PhantomEnvironments: Training LLM Agents in Fictional Worlds

arXiv:2609.40221v1 Announce Type: cross Abstract: Training LLM agents with reinforcement learning (RL) is bottlenecked by environments, which must provide verifiable rewards, support long-horizon inte

Removing Timing Shortcuts Improves Non-Invasive Brain-to-Text

arXiv:2609.40359v1 Announce Type: new Abstract: We find that major reported improvements in decoding words from non-invasive brain recordings are largely reproducible without any brain data. In the in

Riemannian Flow Models with Reinforcement Learning for Molecular Crystal Structure Prediction

arXiv:2609.39773v1 Announce Type: new Abstract: Crystal structure governs material properties, making crystal structure prediction (CSP) a fundamental problem in materials science. Generative models a

RoPE at the End of Its Rope? Theory, Diagnosis, and Mitigation of Long-Context Failures

arXiv:2609.39929v1 Announce Type: cross Abstract: Long-context failures of RoPE-based language models can arise from RoPE's intrinsic tradeoff between maintaining stable token preferences and distingu

RSI-Master: Structuring Experiments to Guide Autonomous Model Improvement

arXiv:2609.35561v2 Announce Type: replace Abstract: Recursive self-improvement (RSI) seeks to enable AI systems to participate in improving their own capabilities. A concrete pathway is autonomous mod

Self-Repulsive Sampling for Diffusion Language Models

arXiv:2609.39560v1 Announce Type: new Abstract: Sampling several responses and voting over their answers can improve a language model's accuracy, but repeated answers limit the benefit of additional s

Sequential Bayesian Evaluation of Large Language Model Behavior

arXiv:2511.10661v2 Announce Type: replace Abstract: It is increasingly important to evaluate the characteristics of systems based on large language models (LLMs). Evaluations in this context often rel

Should I stay or should I show? Learning to selectively disclose information

arXiv:2609.39818v1 Announce Type: new Abstract: In many high-stakes settings, human decision-makers can acquire support information before making a decision. However, acquiring information is costly,

SkillSeek: Revisiting Agent Skill Retrieval at Marketplace Scale

arXiv:2609.38822v1 Announce Type: cross Abstract: Anthropic's Agent Skills package reusable procedural know-how for an LLM agent into SKILL.md directories, and open-source aggregations have grown past

STARS: From Spatiotemporal Dynamics to Social Representations in Human-Robot Interaction

arXiv:2609.40245v1 Announce Type: cross Abstract: Robot navigation in dynamic, human-centered environments requires socially-compliant decisions grounded in robust scene understanding. Recent Vision-L

TomasuLLM: Out-of-Order Speculative Execution for LLM Agents

arXiv:2609.38201v1 Announce Type: new Abstract: Long-running tools can dominate coding-agent latency: compilers, test suites, and repository commands take seconds to minutes while the agent idles. Thi

Trust Is Not a Score: Runtime Assurance Contracts for High-Risk AI Agents

arXiv:2609.39717v1 Announce Type: new Abstract: Benchmarks, audits, and agent protocols describe performance, permissions, and repair, but not how observed evidence should change an agent's authority

safetyarxiv-cs-ai

VAmoS Part Deux: Harder, More Realistic Voice-Agent Simulation

arXiv:2609.38512v1 Announce Type: new Abstract: Voice agents in production must handle several requests, background speech, and customers who lose patience. We introduce VAmoS Energy, a benchmark that

Who Said What, and Will It Be Remembered? Evaluating Persistent Speaker Attribution Across Meetings

arXiv:2609.39344v1 Announce Type: cross Abstract: Speech transcripts used as long-term memory must preserve both words and stable speaker identities. Existing meeting-transcription metrics either igno

AgentBug-Smith: Automatically Reproducing Real-World Harness Bugs in Agentic Systems

arXiv:2609.37864v1 Announce Type: cross Abstract: Agent harness bugs exhibit unique characteristics and remain challenging for state-of-the-art software agents to repair. Progress in this area is furt

Agents write application code, but you still get paged at 2am when it breaks. We wanted to know whether AI could handle that part of the job…

Agents write application code, but you still get paged at 2am when it breaks. We wanted to know whether AI could handle that part of the job too. Introducing Incident Arena: a benchmark that puts codi

model-releasesswyx--x

DeepForestVisionV2: Ecology-Driven Taxonomy Expansion for Camera-Trap Monitoring in African Tropical Forests

arXiv:2606.20223v2 Announce Type: replace Abstract: Camera-trap monitoring in African tropical forests increasingly extends beyond closed-canopy interiors to riverbanks, clearings, and park edges. Amo

From Dead Code and Static Requirements to Working Engines: Software Revival with Coding Agents

arXiv:2609.36161v1 Announce Type: cross Abstract: Can coding agents restore software that no longer runs while preserving its underlying methods, and reconstruct industrial software engines from open

From Dissonance to Orchestration: Teacher Intervention in On-Policy Distillation

arXiv:2609.37510v1 Announce Type: new Abstract: On-policy distillation (OPD) trains a student on its own reasoning trajectories using feedback from a stronger teacher. Teacher interventions can improv

Agents

GLM 5.3 Flash is now available for training on the Serverless Training API - accessible to all, with both vision and text support. This mode…

GLM 5.3 Flash is now available for training on the Serverless Training API - accessible to all, with both vision and text support. This model performs great on our benchmarks for agentic coding, docum

agentsfireworks-ai--x

Google’s new frontier AI model Gemini 4 Argon goes to cybersecurity defenders first

model-releasessiliconangle

Google LLC today began rolling out Gemini 4 Argon, a frontier artificial intelligence model that beats rival models from Anthropic PBC and OpenAI Group PBC on most of Google’s benchmarks, to vetted cy

HERO: Histology Encoder for Robust Representation in Oncology

arXiv:2609.35943v1 Announce Type: new Abstract: Foundation models trained on large pathology image corpora now provide strong, transferable representations for computational pathology. Over the past f

Large-scale factor analysis shows machine intelligence is only partially interpretable

arXiv:2609.36515v1 Announce Type: cross Abstract: A common assumption in language model development is that cognitive abilities are organized around a general, domain-free intelligence factor, like fl

safetyarxiv-cs-ai

Replication Failure and Trivial Baselines in Road-Level Crash Prediction

arXiv:2609.35917v1 Announce Type: new Abstract: Graph neural networks are increasingly applied to road-level crash prediction, but the stability of their reported gains has received little scrutiny. W

SafeLLM4SE: Statistical Evaluation and Reporting for LLM-based Software Engineering Systems

arXiv:2609.37294v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used for software engineering tasks, yet their stochastic behavior challenges the validity, reproducibil

TabFM-Auto: Self-Evolving Pipelines for Tabular Foundation Models

arXiv:2609.37989v1 Announce Type: new Abstract: Tabular foundation models achieve strong zero-shot accuracy on structured data by pretraining on synthetic tables, but they ignore the column names, tas

Transolver-sigma: Joint Spectral-Physical Subspace Modeling for Neural PDE Solving

arXiv:2609.37279v1 Announce Type: new Abstract: Neural solvers offer efficient surrogates for numerical simulation of partial differential equations (PDEs). For time-dependent problems, strong one-ste

VehicleArena: A Realistic Urban Environment for Multi-Agent Driving

arXiv:2609.35916v1 Announce Type: cross Abstract: Real-world embodied agents often pursue independent objectives within a shared physical environment, where their actions can alter the conditions face

VIF-Bench: Evaluating Visual Instruction Following in Multi-Reference Image Generation

arXiv:2609.37709v1 Announce Type: cross Abstract: Recent multimodal image generation models can take multiple images and textual instructions as input, enabling reference-based generation guided not o

A Free Knob: Decoupling Calibration and Predictive Skill in Threshold-Based Evaluation

arXiv:2609.33457v1 Announce Type: new Abstract: Many dense-prediction benchmarks evaluate rare events by pooling prediction and target over spatial blocks, thresholding each, and scoring the contingen

Before Answering: Evidence Sufficiency under Size-Matched Memory Construction

arXiv:2609.32269v1 Announce Type: cross Abstract: Agents that answer questions from compressed or retrieved memory must recognize when the evidence a query needs is no longer in memory. Benchmarks for

ConvCue: Complementary Visual Inductive Biases for Vision-Language Models

arXiv:2609.34196v1 Announce Type: new Abstract: Modern vision-language models (VLMs) achieve strong performance across a broad range of multimodal tasks, yet still struggle with visual questions that

FIDAL: Diversity-Aware Federated Active Learning Under Real-World Distribution Shifts

arXiv:2609.31637v1 Announce Type: cross Abstract: Federated learning enables collaborative model training across institutions without centralizing data, yet high annotation costs, domain shifts, and c

Handwritten Text Recognition Lives in the High-Pixel Variance Subspace

arXiv:2609.35473v1 Announce Type: new Abstract: In self-supervised pretraining for Handwritten Text Recognition (HTR), pixel reconstruction methods outperform contrastive methods, unlike in natural-im

InfoEdit: Probing Global Layout Reasoning in Infographic Editing

arXiv:2609.33286v1 Announce Type: new Abstract: Multimodal foundation models edit natural photographs at production quality, yet the same models struggle with structured visual content such as infogra

Learning High-Risk High-Precision Motion Control

arXiv:2609.34851v1 Announce Type: new Abstract: Deep reinforcement learning (DRL) algorithms for movement control are typically evaluated and benchmarked on sequential decision tasks where imprecise a

safetyarxiv-cs-lg

Learning Steadily: Accumulating Relative Point Margin Scores for Face Image Quality Assessment

arXiv:2609.31662v1 Announce Type: new Abstract: Face Image Quality Assessment determines the suitability of captured face images for automated face recognition (FR), a critical capability for reliable

Learning to Act under Visual Interruptions with Vision-Language-Action Models

model-releasesarxiv-cs-ro

arXiv:2609.35003v1 Announce Type: new Abstract: Vision-language-action (VLA) models have demonstrated strong capabilities in robotic manipulation, but they are typically developed and evaluated with a

Prioritizing Repeated LLM Evaluation for Hidden Failure Discovery

arXiv:2609.32547v1 Announce Type: new Abstract: Large language models are commonly evaluated by generating a small number of stochastic responses for each prompt in a benchmark. Because inference budg

Quoting Anthropic Frontier Red Team

model-releasessimon-willison

We evaluate several models on 100 tasks from the [internal Binary Exploitation benchmark] (selected at random), and find that GLM-5.3 develops full control flow hijacks in 4% of the trials; Claude Myt

RAGWarrant: Evidence-Preserving Governance for RAG Policy Promotion Under Quality, Cost, Latency, and Risk Constraints

arXiv:2609.34179v1 Announce Type: cross Abstract: Retrieval-augmented generation systems are extensively instrumented with metrics, benchmarks, traces, and automated judges, but these tools do not dec

safetyarxiv-cs-lg

Rate-Adaptive One-Step Diffusion Compression for AIGC Images

arXiv:2609.31795v1 Announce Type: new Abstract: We describe our entry to the LoViF 2026 AIGC Image Compression Challenge, a benchmark for ultra-low-bitrate coding of AI-generated images under a strict