Search: “benchmark”

Grok Voice Think Fast 1.0 is officially the most well-rounded agentic voice AI on the market right now It now ranks #1 in the latest τ-Voice agentic performance benchmarks in real-world tests on Artif

agentselon-musk--x

14 May 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

LeanSearch v2: Global Premise Retrieval for Lean 4 Theorem Proving

DGX agent

arXiv:2605.13137v1 Announce Type: cross Abstract: Proving theorems in Lean 4 often requires identifying a scattered set of library lemmas whose joint use enables a concise proof -- a task we call glob

model-releasesarxiv-cs-ai

14 May 2026

MMCL-Bench: Multimodal Context Learning from Visual Rules, Procedures, and Evidence

arXiv:2605.12703v1 Announce Type: cross Abstract: We introduce MMCL-Bench, a benchmark for multimodal context learning: learning task-local rules, procedures, and empirical patterns from visual or mix

14 May 2026

RTLC -- Research, Teach-to-Learn, Critique: A three-stage prompting paradigm inspired by the Feynman Learning Technique that lifts LLM-as-judge accuracy on JudgeBench with no fine-tuning

arXiv:2605.13695v1 Announce Type: cross Abstract: LLM-as-a-judge is now the default measurement instrument for open-ended generation, but on the public JudgeBench benchmark even strong instruction-tun

14 May 2026

A Proof-of-Concept Simulation-Driven Digital Twin Framework for Decision-Aware Diabetes Modeling

arXiv:2605.11247v1 Announce Type: new Abstract: This paper presents a proof-of-concept digital twin framework for simulation-driven diabetes modeling using benchmark clinical data, synthetic temporal

AgentDisCo: Towards Disentanglement and Collaboration in Open-ended Deep Research Agents

arXiv:2605.11732v1 Announce Type: cross Abstract: In this paper, we present AgentDisCo, a novel Disentangled and Collaborative agentic architecture that formulates deep research as an adversarial opti

AutoMonitor-Bench: Evaluating the Reliability of LLM-Based Misbehavior Monitor

arXiv:2601.05752v3 Announce Type: replace Abstract: We introduce AutoMonitor-Bench, the first benchmark designed to systematically evaluate the reliability of LLM-based misbehavior monitors across div

我們開源了這顆星球🌎上速度最快的低成本 bm25 引擎。

model-releasesemad-mostaque--x

我們開源了這顆星球🌎上速度最快的低成本 bm25 引擎。 so we built psql_bm25s. exact BM25 retrieval. native Postgres access method. ~23x faster than pg_search on the standard benchmark. retrieval stops being a budget item. the

Checkup2Action: A Multimodal Clinical Check-up Report Dataset for Patient-Oriented Action Card Generation

arXiv:2605.11533v1 Announce Type: new Abstract: Clinical check-up reports are multimodal documents that combine page layouts, tables, numerical biomarkers, abnormality flags, imaging findings, and dom

Evaluating the Pre-Consultation Ability of LLMs using Diagnostic Guidelines

arXiv:2601.03627v3 Announce Type: replace Abstract: We introduce EPAG, a benchmark dataset and framework designed for Evaluating the Pre-consultation Ability of LLMs using diagnostic Guidelines. LLMs

HE-SNR: Uncovering Latent Logic via Entropy for Guiding Mid-Training on SWE-bench

arXiv:2601.20255v2 Announce Type: replace-cross Abstract: SWE-bench has emerged as the premier benchmark for evaluating Large Language Models on complex software engineering tasks. While these capabil

PRISM: : Planning and Reasoning with Intent in Simulated Embodied Environments

model-releasesarxiv-cs-ro

arXiv:2605.11534v1 Announce Type: new Abstract: When an LLM-based embodied agent fails at a household task, the culprit could be misidentified objects, forgotten sub-goals, or poor action sequencing -

Robust Promptable Video Object Segmentation

model-releasesarxiv-cs-cv

arXiv:2605.12006v1 Announce Type: new Abstract: The performance of promptable video object segmentation (PVOS) models substantially degrades under input corruptions, which prevents PVOS deployment in

STRUM: A Spectral Transcription and Rhythm Understanding Model for End-to-End Generation of Playable Rhythm-Game Charts

arXiv:2605.12135v1 Announce Type: cross Abstract: We present STRUM (Spectral Transcription and Rhythm Understanding Model), an audio-to-chart pipeline that converts raw recordings into playable Clone

Research

Targeted Tests for LLM Reasoning: An Audit-Constrained Protocol

arXiv:2605.11599v1 Announce Type: new Abstract: Fixed reasoning benchmarks evaluate canonical prompts, but semantically valid changes in presentation can still change model behavior. Studies of prompt

researcharxiv-cs-lg

Agentic MIP Research: Accelerated Constraint Handler Generation

arXiv:2605.09186v1 Announce Type: new Abstract: Mixed-integer programming (MIP) research is both mathematically sophisticated and engineering-intensive: testing an algorithmic hypothesis within a bran

AgentPSO: Evolving Agent Reasoning Skill via Multi-agent Particle Swarm Optimization

arXiv:2605.08704v1 Announce Type: new Abstract: Multi-agent reasoning has shown promise for improving the problem-solving ability of large language models by allowing multiple agents to explore divers

Applications

Beyond Accuracy: Evaluating Posterior Fidelity of Diffusion Inverse Solvers

arXiv:2602.04189v2 Announce Type: replace Abstract: Uncertainty evaluation is critical in scientific and engineering inverse problems. However, existing benchmarks on Diffusion Inverse Solvers (DIS) p

applicationsarxiv-cs-lg

Can Deep Research Agents Retrieve and Organize? Evaluating the Synthesis Gap with Expert Taxonomies

arXiv:2601.12369v3 Announce Type: replace Abstract: Deep Research Agents increasingly automate survey generation, yet whether they match human experts at retrieving essential papers and organizing the

Computer Use at the Edge of the Statistical Precipice

arXiv:2605.08261v1 Announce Type: cross Abstract: Evaluating Computer Use Agents (CUAs) on interactive environments is fraught with methodological pitfalls that the field has yet to systematically add

CREATE: Testing LLMs for Associative Creativity

arXiv:2603.09970v2 Announce Type: replace Abstract: A key component of creativity is associative reasoning: the ability to draw novel yet meaningful connections between concepts. We introduce CREATE,

Tutorials

Developing a foundation model for high-resolution remote sensing data of the Netherlands

arXiv:2605.10184v1 Announce Type: cross Abstract: We develop a foundation model using 1.2m high resolution satellite images of the Netherlands. By combining a Convolutional Neural Network and a Vision

tutorialsarxiv-cs-ai

EdgeFlowerTune: Evaluating Federated LLM Fine-Tuning Under Realistic Edge System Constraints

arXiv:2605.08636v1 Announce Type: new Abstract: Federated fine-tuning offers a promising paradigm for adapting large language models (LLMs) on edge devices by leveraging the rich, diverse, and continu

Explicit Reasoning Makes Better Judges: A Systematic Study on Accuracy, Efficiency, and Robustness

arXiv:2509.13332v2 Announce Type: replace Abstract: As Large Language Models (LLMs) are increasingly adopted as automated judges in benchmarking and reward modeling, ensuring their reliability, effici

Exploring the AI Obedience: Why is Generating a Pure Color Image Harder than CyberPunk?

arXiv:2603.00166v2 Announce Type: replace-cross Abstract: Recent advances in generative AI have shown human-level performance in complex content creation. However, we identify a 'Paradox of Simplicity

Applications

From Controlled to the Wild: Evaluation of Pentesting Agents for the Real-World

arXiv:2605.10834v1 Announce Type: new Abstract: AI pentesting agents are increasingly credible as offensive security systems, but current benchmarks still provide limited guidance on which will perfor

applicationsarxiv-cs-ai

GONE: Structural Knowledge Unlearning via Neighborhood-Expanded Distribution Shaping

arXiv:2603.12275v1 Announce Type: cross Abstract: Unlearning knowledge is a pressing and challenging task in Large Language Models (LLMs) because of their unprecedented capability to memorize and dige

Agents

Grok Voice is #1!

Grok Voice is #1! Announcing agentic performance benchmarking for Speech to Speech models on Artificial Analysis. We use 𝜏-Voice to measure tool calling and customer interaction voice agent capabiliti

agentselon-musk--x

Human-Inspired Memory Architecture for LLM Agents

arXiv:2605.08538v1 Announce Type: new Abstract: Current LLM agents lack principled mechanisms for managing persistent memory across long interaction horizons. We present a biologically-grounded memory

Human-LLM Dialogue Improves Diagnostic Accuracy in Emergency Care

arXiv:2605.08533v1 Announce Type: new Abstract: Clinical decision-making in emergency medicine demands rapid, accurate diagnoses under uncertainty. Despite benchmark progress, evidence for LLMs as int

Improving TMS EEG Signal Quality for Closed-Loop Neuro Stimulation via Source-Domain Denoising

arXiv:2605.08184v1 Announce Type: cross Abstract: This research addresses a validated TMS EEG cleaning pipeline and a corresponding benchmark dataset. It evaluates two widely used artifact removal pip

Safety

Investigating Anisotropy in Visual Grounding under Controlled Counterfactual Perturbations

arXiv:2605.09090v1 Announce Type: cross Abstract: Visual Grounding benchmarks assume that the object described by a referring expression is always present in the image, and grounding models are theref

safetyarxiv-cs-ai

LegalCiteBench: Evaluating Citation Reliability in Legal Language Models

arXiv:2605.10186v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly integrated into legal drafting and research workflows, where incorrect citations or fabricated precedent

LEVI: Stronger Search Architectures Can Substitute for Larger LLMs in Evolutionary Search

arXiv:2605.09764v1 Announce Type: cross Abstract: LLM-guided evolutionary methods such as AlphaEvolve have proven effective in domains like math, systems research, and algorithmic discovery, but their

Med-StepBench: A Hierarchical Reasoning Framework for Evaluating Hallucinations in Medical Vision-Language Models

model-releasesarxiv-cs-cv

arXiv:2605.10002v1 Announce Type: new Abstract: Large vision-language models (VLMs) demonstrate strong performance in medical image understanding, but frequently generate clinically plausible yet inco

Meet physics-intern🧑‍🎓, our agentic framework for theoretical physics. It takes Gemini 3.1 Pro from 17.7% to 31.4% on CritPt, a new SOTA o…

model-releasesclem-delangue--x

Meet physics-intern🧑‍🎓, our agentic framework for theoretical physics. It takes Gemini 3.1 Pro from 17.7% to 31.4% on CritPt, a new SOTA on one of the hardest benchmarks for LLMs. Theoretical physics

MolRGen: A Training and Evaluation Setting for De Novo Molecular Generation with Reasonning Models

arXiv:2603.18256v2 Announce Type: replace-cross Abstract: Recent reasoning-based large language models have shown strong performance on tasks with verifiable outcomes, but their use in de novo molecul

MonitoringBench: Semi-Automated Red-Teaming for Agent Monitoring

arXiv:2605.09684v1 Announce Type: cross Abstract: We introduce a red-teaming methodology that exposes harder-to-catch attacks for coding-agent monitors, suggesting that current practices may under-eli

MOTOR-Bench: A Real-world Dataset and Multi-agent Framework for Zero-shot Human Mental State Understanding

model-releasesarxiv-cs-cv

arXiv:2605.09703v1 Announce Type: new Abstract: Understanding human mental states from natural behavior is crucial for intelligent systems in the real world. However, most current research focuses on

Safety

Multi-layer attentive probing improves transfer of audio representations for bioacoustics

arXiv:2605.10494v1 Announce Type: cross Abstract: Probing heads map the representations learned from audio by a machine learning model to downstream task labels and are a key component in evaluating r

safetyarxiv-cs-ai

Safety

MVB-Grasp: Minimum-Volume-Box Filtering of Diffusion-based Grasps for Frontal Manipulation

arXiv:2605.09672v1 Announce Type: new Abstract: State-of-the-art 6-DoF grasp generators excel on tabletop benchmarks with overhead cameras but struggle in frontal grasping scenarios on low-cost manipu

safetyarxiv-cs-ro

Optimised Support Vector Regression for California Housing Price Prediction: The Critical Role of Feature Engineering and Hyperparameter Tuning

arXiv:2605.08660v1 Announce Type: new Abstract: In the recent literature, Support Vector Regression (SVR) has been cited as one of the weakest performers on the California Housing benchmark dataset, w

Repeated-Token Counting Reveals a Dissociation Between Representations and Outputs

arXiv:2605.09239v1 Announce Type: new Abstract: Large language models fail at counting repeated tokens despite strong performance on broader reasoning benchmarks. These failures are commonly attribute

SEMASIA: A Large-Scale Dataset of Semantically Structured Latent Representations

arXiv:2605.09485v1 Announce Type: new Abstract: Latent representations learned by neural networks often exhibit semantic structure, where concept similarity is reflected by geometric proximity in embe

Local Ai

StereoTales: A Multilingual Framework for Open-Ended Stereotype Discovery in LLMs

arXiv:2605.10442v1 Announce Type: cross Abstract: Multilingual studies of social bias in open-ended LLM generation remain limited: most existing benchmarks are English-centric, template-based, or rest

local-aiarxiv-cs-ai

TIDES: Implicit Time-Awareness in Selective State Space Models

arXiv:2605.09742v1 Announce Type: cross Abstract: Selective state space models (SSMs), such as Mamba, achieve strong per-token expressivity by making the time discretization step Tilde{Delta} a learne

Try Grok Voice

model-releaseselon-musk--x

Try Grok Voice Grok Voice Think Fast 1.0 ranks #1 on the Artificial Analysis τ-Voice benchmark for real-world agentic customer service resolution Absolutely outperforming GPT-Realtime-2 (High) and Gem

Research

Useful for Exploration, Risky for Precision: Evaluating AI Tools in Academic Research

arXiv:2605.10125v1 Announce Type: new Abstract: Artificial intelligence (AI) tools are being incorporated into scientific research workflows with the potential to enhance efficiency in tasks such as d

researcharxiv-cs-ai