Search: “benchmark”

JudgeMeNot: Personalizing Large Language Models to Emulate Judicial Reasoning in Hebrew

arXiv:2604.18041v1 Announce Type: new Abstract: Despite significant advances in large language models, personalizing them for individual decision-makers remains an open problem. Here, we introduce a s

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Kimi K2.6 has captured #1 on the open-weight Vals Index, and is #7 overall.

DGX agent

Kimi K2.6, a language model developed by Moonshot AI, has achieved the top ranking on the open-weight category of the Vals Index benchmark, while placing 7th overall across all model categories. The V

model-releaseskimi-moonshot--x

21 Apr 2026

Live Avatar: Streaming Real-time Audio-Driven Avatar Generation with Infinite Length

arXiv:2512.04677v5 Announce Type: replace Abstract: Audio-driven avatar interaction demands real-time, streaming, and infinite-length generation -- capabilities fundamentally at odds with the sequenti

MeasHalu: Mitigation of Scientific Measurement Hallucinations for Large Language Models with Enhanced Reasoning

arXiv:2604.16929v1 Announce Type: new Abstract: The accurate extraction of scientific measurements from literature is a critical yet challenging task in AI4Science, enabling large-scale analysis and i

Applications

Omni-Embed-Audio: Leveraging Multimodal LLMs for Robust Audio-Text Retrieval

arXiv:2604.18360v1 Announce Type: cross Abstract: Audio-text retrieval systems based on Contrastive Language-Audio Pretraining (CLAP) achieve strong performance on traditional benchmarks; however, the

applicationsarxiv-cs-cl

OptunaHub: A Platform for Black-Box Optimization

model-releasesarxiv-cs-lg

arXiv:2510.02798v2 Announce Type: replace Abstract: Black-box optimization (BBO) underpins advances in domains such as AutoML and Materials Informatics, yet implementations of algorithms and benchmark

PDDL-Mind: Large Language Models are Capable on Belief Reasoning with Reliable State Tracking

arXiv:2604.17819v1 Announce Type: new Abstract: Large language models (LLMs) perform substantially below human level on existing theory-of-mind (ToM) benchmarks, even when augmented with chain-of-thou

ReTraceQA: Evaluating Reasoning Traces of Small Language Models in Commonsense Question Answering

arXiv:2510.09351v2 Announce Type: replace Abstract: While Small Language Models (SLMs) have demonstrated promising performance on an increasingly wide array of commonsense reasoning benchmarks, curren

SpeechMedAssist: Efficiently and Effectively Adapting Speech Language Models for Medical Consultation

arXiv:2601.04638v2 Announce Type: replace Abstract: Medical consultations are intrinsically speech-centric. However, most prior works focus on long-text-based interactions, which are cumbersome and pa

ARC-AGI-3: A New Challenge for Frontier Agentic Intelligence

arXiv:2603.24621v2 Announce Type: replace Abstract: We introduce ARC-AGI-3, an interactive benchmark for studying agentic intelligence through novel, abstract, turn-based environments in which agents

Agents

Capture the Flags: Family-Based Evaluation of Agentic LLMs via Semantics-Preserving Transformations

arXiv:2602.05523v2 Announce Type: replace-cross Abstract: Agentic large language models (LLMs) are increasingly evaluated on cybersecurity tasks using capture-the-flag (CTF) benchmarks, yet existing p

agentsarxiv-cs-ai

Imagine if they add document signing to Claude :o

model-releasesemad-mostaque--x

Imagine if they add document signing to Claude :o In Cowork, Claude can now build live artifacts: dashboards and trackers connected to your apps and files. Open one any time and it refreshes with curr

Applications

Information Router for Mitigating Modality Dominance in Vision-Language Models

arXiv:2604.16264v1 Announce Type: new Abstract: Vision Language models (VLMs) have demonstrated strong performance across a wide range of benchmarks, yet they often suffer from modality dominance, whe

applicationsarxiv-cs-cv

MTR-DuplexBench: Towards a Comprehensive Evaluation of Multi-Round Conversations for Full-Duplex Speech Language Models

arXiv:2511.10262v3 Announce Type: replace-cross Abstract: Full-Duplex Speech Language Models (FD-SLMs) enable real-time, overlapping conversational interactions, offering a more dynamic user experienc

Neurosymbolic Repo-level Code Localization

arXiv:2604.16021v1 Announce Type: cross Abstract: Code localization is a cornerstone of autonomous software engineering. Recent advancements have achieved impressive performance on real-world issue be

Optimizing Korean-Centric LLMs via Token Pruning

arXiv:2604.16235v1 Announce Type: new Abstract: This paper presents a systematic benchmark of state-of-the-art multilingual large language models (LLMs) adapted via token pruning - a compression techn

Reasoning over Video: Evaluating How MLLMs Extract, Integrate, and Reconstruct Spatiotemporal Evidence

arXiv:2603.13091v2 Announce Type: replace Abstract: The growing interest in embodied agents increases the demand for spatiotemporal video understanding, yet existing benchmarks largely emphasize extra

Source tweet is here: https://x.com/jerryjliu0/status/2044902620746363016?s=20

model-releasesjerry-liu--x

Source tweet is here: https://x.com/jerryjliu0/status/2044902620746363016?s=20 We comprehensively benchmarked Opus 4.7 on document understanding. We evaluated it through ParseBench - our comprehensive

19 Apr 2026

A multi-platform LiDAR dataset for standardized forest inventory measurement at long term ecological monitoring sites

model-releasesarxiv-cs-ro

arXiv:2604.14635v1 Announce Type: new Abstract: We present a curated multi-platform LiDAR reference dataset from an instrumented ICOS forest plot, explicitly designed to support calibration, benchmark

17 Apr 2026

DharmaOCR: Specialized Small Language Models for Structured OCR that outperform Open-Source and Commercial Baselines

arXiv:2604.14314v1 Announce Type: cross Abstract: This manuscript introduces DharmaOCR Full and Lite, a pair of specialized small language models (SSLMs) for structured OCR that jointly optimize trans

17 Apr 2026

ImplicitMemBench: Measuring Unconscious Behavioral Adaptation in Large Language Models

arXiv:2604.08064v2 Announce Type: replace Abstract: Existing memory benchmarks for LLM agents evaluate explicit recall of facts, yet overlook implicit memory where experience becomes automated behavio

17 Apr 2026

Common to Whom? Regional Cultural Commonsense and LLM Bias in India

arXiv:2601.15550v3 Announce Type: replace Abstract: Existing cultural commonsense benchmarks treat nations as monolithic, assuming uniform practices within national boundaries. But does cultural commo

16 Apr 2026

Evaluating the Formal Reasoning Capabilities of Large Language Models through Chomsky Hierarchy

arXiv:2604.02709v2 Announce Type: replace Abstract: The formal reasoning capabilities of LLMs are crucial for advancing automated software engineering. However, existing benchmarks for LLMs lack syste

16 Apr 2026

From Plausibility to Verifiability: Risk-Controlled Generative OCR with Vision-Language Models

arXiv:2603.19790v3 Announce Type: replace Abstract: Modern vision-language models (VLMs) can act as generative OCR engines, yet open-ended decoding can expose rare but consequential failures. We ident

16 Apr 2026

Global optimization tailored for graphics processing units: Complete and rigorous search for large-scale nonlinear minimization

arXiv:2507.01770v4 Announce Type: replace-cross Abstract: This paper introduces a numerical method to enclose the global minimum of a nonlinear function subject to simple bounds on the variables. Usin

GlotOCR Bench: OCR Models Still Struggle Beyond a Handful of Unicode Scripts

arXiv:2604.12978v1 Announce Type: new Abstract: Optical character recognition (OCR) has advanced rapidly with the rise of vision-language models, yet evaluation has remained concentrated on a small cl

GTPBD-MM: A Global Terraced Parcel and Boundary Dataset with Multi-Modality

arXiv:2604.12315v1 Announce Type: new Abstract: Agricultural parcel extraction plays an important role in remote sensing-based agricultural monitoring, supporting parcel surveying, precision managemen

Agents

Inside VAKRA: Reasoning, Tool Use, and Failure Modes of Agents

VAKRA (Versatile Agent and Knowledge Reasoning Assessment) is a benchmark developed by IBM Research to evaluate AI agents on complex reasoning, tool use, and multi-step task execution across diverse d

agentshugging-face

QuarkMedSearch: A Long-Horizon Deep Search Agent for Exploring Medical Intelligence

arXiv:2604.12867v1 Announce Type: new Abstract: As agentic foundation models continue to evolve, how to further improve their performance in vertical domains has become an important challenge. To this

ReasonXL: Shifting LLM Reasoning Language Without Sacrificing Performance

arXiv:2604.12378v1 Announce Type: new Abstract: Despite advances in multilingual capabilities, most large language models (LLMs) remain English-centric in their training and, crucially, in their produ

Revisiting the Reliability of Language Models in Instruction-Following

arXiv:2512.14754v2 Announce Type: replace-cross Abstract: Advanced LLMs have achieved near-ceiling instruction-following accuracy on benchmarks such as IFEval. However, these impressive scores do not

SIR-Bench: Evaluating Investigation Depth in Security Incident Response Agents

arXiv:2604.12040v1 Announce Type: cross Abstract: We present SIR-Bench, a benchmark of 794 test cases for evaluating autonomous security incident response agents that distinguishes genuine forensic in

Research

Synthetic POMDPs to Challenge Memory-Augmented RL: Memory Demand Structure Modeling

arXiv:2508.04282v3 Announce Type: replace Abstract: Recent benchmarks for memory-augmented reinforcement learning (RL) have introduced partially observable Markov decision process (POMDP) environments

researcharxiv-cs-ai

Applications

BEM: Training-Free Background Embedding Memory for False-Positive Suppression in Real-Time Fixed-Background Camera

arXiv:2604.11714v1 Announce Type: new Abstract: Pretrained detectors perform well on benchmarks but often suffer performance degradation in real-world deployments due to distribution gaps between trai

applicationsarxiv-cs-cv

Both Ends Count! Just How Good are LLM Agents at 'Text-to-Big SQL'?

arXiv:2602.21480v4 Announce Type: replace-cross Abstract: Text-to-SQL and Big Data are both extensively benchmarked fields, yet there is limited research that evaluates them jointly. In the real world

Can Multi-Modal LLMs Provide Live Step-by-Step Task Guidance?

arXiv:2511.21998v2 Announce Type: replace Abstract: Multi-modal Large Language Models (LLM) have advanced conversational abilities but struggle with providing live, interactive step-by-step guidance,

Evaluating Reliability Gaps in Large Language Model Safety via Repeated Prompt Sampling

arXiv:2604.09606v1 Announce Type: new Abstract: Traditional benchmarks for large language models (LLMs), such as HELM and AIR-BENCH, primarily assess safety risk through breadth-oriented evaluation ac

Hunt Globally: Wide Search AI Agents for Drug Asset Scouting in Investing, Business Development, and Competitive Intelligence

arXiv:2602.15019v3 Announce Type: replace Abstract: Bio-pharmaceutical innovation has shifted: many new drug assets now originate outside the United States and are disclosed primarily via regional, no

I'm excited about voice as a UI layer for existing visual applications — where speech and screen update together. This goes well beyond voic…

model-releasesandrew-ng--x

I'm excited about voice as a UI layer for existing visual applications — where speech and screen update together. This goes well beyond voice-only use cases like call center automation. The barrier ha

IMPACT: A Dataset for Multi-Granularity Human Procedural Action Understanding in Industrial Assembly

arXiv:2604.10409v1 Announce Type: cross Abstract: We introduce IMPACT, a synchronized five-view RGB-D dataset for deployment-oriented industrial procedural understanding, built around real assembly an

NovBench: Evaluating Large Language Models on Academic Paper Novelty Assessment

arXiv:2604.11543v1 Announce Type: cross Abstract: Novelty is a core requirement in academic publishing and a central focus of peer review, yet the growing volume of submissions has placed increasing p

PokeRL: Reinforcement Learning for Pokemon Red

model-releasesarxiv-cs-lg

arXiv:2604.10812v1 Announce Type: new Abstract: Pokemon Red is a long-horizon JRPG with sparse rewards, partial observability, and quirky control mechanics that make it a challenging benchmark for rei

RiTeK: A Dataset for Large Language Models Complex Reasoning over Textual Knowledge Graphs in Medicine

arXiv:2410.13987v3 Announce Type: replace Abstract: Answering complex real-world questions in the medical domain often requires accurate retrieval from medical Textual Knowledge Graphs (medical TKGs),

RobustMedSAM: Degradation-Resilient Medical Image Segmentation via Robust Foundation Model Adaptation

arXiv:2604.09814v1 Announce Type: new Abstract: Medical image segmentation models built on Segment Anything Model (SAM) achieve strong performance on clean benchmarks, yet their reliability often degr

Seeing No Evil: Blinding Large Vision-Language Models to Safety Instructions via Adversarial Attention Hijacking

arXiv:2604.10299v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) rely on attention-based retrieval of safety instructions to maintain alignment during generation. Existing attack

VLN-NF: Feasibility-Aware Vision-and-Language Navigation with False-Premise Instructions

arXiv:2604.10533v1 Announce Type: cross Abstract: Conventional Vision-and-Language Navigation (VLN) benchmarks assume instructions are feasible and the referenced target exists, leaving agents ill-equ

VS-Bench: Evaluating VLMs for Strategic Abilities in Multi-Agent Environments

arXiv:2506.02387v3 Announce Type: replace Abstract: Recent advancements in Vision Language Models (VLMs) have expanded their capabilities to interactive agent tasks, yet existing benchmarks remain lim

WBCBench 2026: A Challenge for Robust White Blood Cell Classification Under Class Imbalance

arXiv:2604.10797v1 Announce Type: new Abstract: We present WBCBench 2026, an ISBI challenge and benchmark for automated WBC classification designed to stress-test algorithms under three key difficulti