Search: “benchmark”

Interpreting and Steering for Safe and Correct Code Generation

arXiv:2608.30025v1 Announce Type: new Abstract: Large language models (LLMs) frequently generate source code containing vulnerabilities, yet little work studies the internal mechanisms that distinguis

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Item-Mean Surrogates: Why Richer Persona Data Fail to Improve LLMs as Human Surrogates

DGX agent

arXiv:2608.29455v1 Announce Type: new Abstract: LLMs are increasingly used as human surrogates, often on the premise that richer persona data could make them substitutes or exploratory tools for speci

model-releasesarxiv-cs-cl

1 Sept 2026

JFTA-Bench: Evaluate LLM's Ability of Tracking and Analyzing Malfunctions Using Fault Trees

arXiv:2603.22978v2 Announce Type: replace Abstract: In the maintenance of complex systems, fault trees are used to locate problems and provide targeted solutions. To enable fault trees stored as image

Learning Compositional Spatio-Temporal Video Grounding with Synthetic Curriculum

arXiv:2608.30584v1 Announce Type: new Abstract: Despite the impressive progress of recent MLLMs on spatio-temporal video grounding (STVG), existing evaluations and training data focus primarily on sim

LeMat-Synth: a multi-modal toolbox to curate broad synthesis procedure databases from scientific literature

arXiv:2510.26824v2 Announce Type: replace-cross Abstract: Wide access to advanced experimental methods in materials science has given rise to an abundance of procedural knowledge, which is scattered a

LLM Judges Verify Presence, Not Absence: Omission Blindness in AI Clinical Notes and What Recovers It

arXiv:2608.31016v1 Announce Type: cross Abstract: Ambient AI scribes draft clinical notes, and published audits find their dominant error is omission: information the encounter established that the no

MagicSim: A Unified Infrastructure for Executable Embodied Interaction

arXiv:2606.17511v2 Announce Type: replace-cross Abstract: Robot learning and embodied agents now require simulation to serve as a shared execution substrate linking control, skills, and planning, not

Manaca-1B: An Open, Reproducible Brazilian-Portuguese Language Model and a Tokenizer-Aware, Paired Evaluation

arXiv:2608.30114v1 Announce Type: new Abstract: Brazilian Portuguese remains under-served by open language models, and the few that exist are difficult to reproduce and are often compared without meas

Memory-First Fact-Checking: A Knowledge-Graph-Grounded Multi-Agent System for Misinformation Detection

arXiv:2608.29617v1 Announce Type: cross Abstract: This paper introduces a hybrid fact-checking framework that integrates Knowledge Graph-based semantic memory with adversarial multi-agent reasoning fo

MUDDLE: Measuring Understanding of Documents under Distractor and Length Effects

arXiv:2608.29477v1 Announce Type: cross Abstract: Document question-answering systems increasingly answer questions over collections of retrieved documents rather than one clean source, so robustness

On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability

arXiv:2608.30320v1 Announce Type: new Abstract: We describe the architecture and ablations of Qwen3.8-Flash-Next, a sparse mixture-of-experts model with 125B parameters, 6B activated per token, and ad

Agents

Ontology-Guided Multi-Agent Extraction of Evaluation Objects from Academic Review Texts: Evidence from Chinese Library and Information Science

arXiv:2608.29526v1 Announce Type: new Abstract: Academic reviews, scholarly commentaries, and book reviews serve as sources of evaluative statements about theories, methods, literature, institutions,

agentsarxiv-cs-cl

PaperGym: Rubric-Centered Evolution for Research-Plan Generation

arXiv:2608.31119v1 Announce Type: new Abstract: Research planning is the decisive capability of AI scientists. Yet a research plan admits no verifiable answer, so reinforcement learning lacks the envi

Research

Partition-Aware Unlearning for Removing Spurious Correlations in Large Vision-Language Models

arXiv:2608.29996v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) achieve strong performance across many multimodal tasks; however, they often exploit spurious object-background c

researcharxiv-cs-ai

Privacy-Preserving Detection of Rare Disease-Associated Cell Subsets via Secure Multi-Party Computation

model-releasesarxiv-cs-lg

arXiv:2608.20118v1 Announce Type: cross Abstract: The detection of rare disease-associated cell subsets from high-dimensional single-cell measurements is critical for understanding diseases such as le

Quantifying and Mitigating Korean Jamo-Level Typographical Vulnerabilities in Large Language Models

arXiv:2608.30229v1 Announce Type: new Abstract: Korean introduces an additional typographical perturbation level not captured by ordinary character-level edit models: because syllable blocks are inter

Rad-R: A Raw-ADC Radar Dataset and Capture-Invariant SSM for Hardware-Fault Diagnosis

arXiv:2608.30896v1 Announce Type: new Abstract: Automotive mmWave radar can develop vibration, antenna misalignment, radome blockage, and receive-channel degradation that corrupt the signal before per

RENSA: Rich Environment Metadata to Navigate Shared and Distributed Endpoints for Automated Federated SPARQL Query Generation

arXiv:2608.28963v1 Announce Type: cross Abstract: The number of knowledge graph databases has increased significantly with the proliferation of knowledge graph technologies. Knowledge graphs enable th

Risk-Adjusted Harm Scoring for Automated Red Teaming for LLMs in Financial Services

arXiv:2603.10807v2 Announce Type: replace-cross Abstract: Existing LLM safety evaluations rely on binary attack-success rates and domain-agnostic taxonomies, leaving regulated Banking, Financial Servi

RSLM: Training-Free Vector Quantization for Approximate Nearest Neighbor Search

model-releasesarxiv-cs-lg

arXiv:2608.30384v1 Announce Type: new Abstract: By introducing RSLM (Rotated Scaled Lloyd-Max), a family of training-free vector quantization codecs compressing embeddings to 1--4 bits per dimension,

SafeAtlas-VL: Beyond Binary Multimodal Safety with Large-Scale Data and Guard Models

arXiv:2608.29098v1 Announce Type: new Abstract: Multimodal safety moderation requires distinguishing risks arising from visual content, user intent, and assistant behavior. Existing safeguards, howeve

Applications

SHAPE of Chain-of-Thought in Math Reasoning

arXiv:2608.28600v1 Announce Type: new Abstract: Large language models (LLMs) achieve strong performance on mathematical reasoning benchmarks, yet the mathematically meaningful skills underlying their

applicationsarxiv-cs-ai

SIR: Self-improving Red-teaming for Compute Use Agents

arXiv:2608.30207v1 Announce Type: cross Abstract: Computer use agents (CUAs) are vision-language models that perceive a screen and act on a real operating system through mouse, keyboard, and terminal,

SkillForge: Compositional Skill Synthesis with Verification-in-the-Loop for Generating Formally Verified Dafny Programs

arXiv:2608.29841v1 Announce Type: new Abstract: Generating formally verified programs from natural language remains challenging: existing approaches either produce code in a single pass without recour

Towards a Joint Khmer Text Recognition and Word Segmentation

arXiv:2608.30213v1 Announce Type: cross Abstract: Text recognition, or extracting electronic text from document images, has been indispensable for knowledge retrieval tasks, such as retrieval-augmente

Uncertainty-Aware Multimodal Anti-UAV Detection via Evidential Fusion and Conflict-Discounted Belief Aggregation

arXiv:2608.29235v1 Announce Type: new Abstract: Anti-UAV perception systems must remain reliable when sensor streams degrade under occlusion, fast motion, or modality-specific failure. Existing multim

Weaving Visual Narratives: Agentic Image Bundle Composition Beyond Atomic Visual Matching

arXiv:2608.28695v1 Announce Type: new Abstract: Image retrieval has traditionally been formulated as a point-wise matching problem, where each candidate image is scored in isolation. However, this ato

Research

What Does an Evaluation License? A Commit-Bound Census of Claim Replay in Inspect Evals

arXiv:2608.19269v4 Announce Type: replace-cross Abstract: Benchmarks can run without determining what their results license. We freeze a large evaluation collection and attempt to replay its historica

researcharxiv-cs-ai

X-Coder: Advancing Competitive Programming with Synthetic Tasks, Solutions, and Tests

arXiv:2601.06953v3 Announce Type: replace Abstract: Competitive programming remains challenging for code LLMs. Despite recent progress, many training pipelines still depend on scarce real-world data,

3D MRI-Based Alzheimer's Disease Classification Using Multi-Modal 3D CNN with Leakage-Aware Subject-Level Evaluation

arXiv:2603.17304v2 Announce Type: replace Abstract: Deep learning has become an important tool for Alzheimer's disease (AD) classification from structural MRI. Many existing studies analyze individual

Local Ai

Acquire, Repair, Preserve: A Diagnosis-Guided Post-Training Recipe for Small-Model Dialogue Game Agents

arXiv:2608.28458v1 Announce Type: new Abstract: Interactive dialogue games test a capability that static benchmarks largely leave implicit: a model must carry state across turns, interpret feedback, a

local-aiarxiv-cs-cl

Research

Are These Modules Worth Their Cost? A Paradigm-Level Accuracy-Cost Analysis of In-context Learning Text-to-SQL

arXiv:2608.28432v1 Announce Type: new Abstract: Recent advances in in-context learning (ICL) text-to-SQL have substantially improved execution accuracy on public benchmarks by assembling increasingly

researcharxiv-cs-cl

BEACON: Behavior-Anchored Cross-Source Knowledge Graph Construction for Cyber Threat Intelligence

arXiv:2608.28394v1 Announce Type: cross Abstract: Cyber threat intelligence (CTI) is foundational to modern cyber defense, yet much of it resides in unstructured reports whose volume and heterogeneity

Blind Men and the Elephant: Probing the Epistemic Myopia of LLMs under Long-Tail Divergent Knowledge

arXiv:2608.28478v1 Announce Type: new Abstract: Factual question answering (QA) typically assumes a single canonical answer, obscuring whether large language models (LLMs) retain divergent accounts of

CommerceVibe: Learning to Design E-Commerce Creatives as Executable Visual Code via Dual-Feedback Reinforcement Learning

arXiv:2608.27893v1 Announce Type: new Abstract: High-quality e-commerce creatives are essential for presenting products and conveying marketing messages. Recent diffusion models enable scalable creati

FUSED: Forensic-Semantic Mixture-of-Experts for AI Inpainting Detection and Localization

arXiv:2608.28302v1 Announce Type: new Abstract: Diffusion-based inpainting models modify only a localized part of an image, while many AI-image detectors rely on global artifacts and do not localize.

Non-Uniform Quantisation for 3DGS Compression

arXiv:2608.28272v1 Announce Type: new Abstract: 3D Gaussian Splatting (3DGS) has emerged as a powerful technique for novel view synthesis, yet its high bitrate requirements pose significant challenges

One Model for All: Universal Pre-training for EEG based Emotion Recognition across Heterogeneous Datasets and Paradigms

model-releasesarxiv-cs-lg

arXiv:2511.08444v2 Announce Type: replace Abstract: EEG-based emotion recognition is hampered by profound dataset heterogeneity (channel/subject variability), hindering generalizable models. Existing

Sledgehammer or Scalpel? A Fine-grained Adaptive Framework for Implicit Hate Speech

arXiv:2608.27462v1 Announce Type: new Abstract: Unlike explicit attacks with obvious profanity, implicit hate speech hides malice within seemingly compliant expressions through metaphors and contextua

A Very Big Video Reasoning Suite

arXiv:2602.20159v3 Announce Type: replace-cross Abstract: Rapid progress in video models has largely focused on visual quality, leaving their reasoning capabilities underexplored. Video reasoning grou

Agent Seer: Synthesizing Scenarios from Specification Understanding

arXiv:2608.26133v1 Announce Type: new Abstract: Evaluating AI agents that use external tools requires realistic test scenarios that capture how practitioners compose tools and iterate across conversat

Agents Don't Paginate: First-Chunk Selection for LLM Tool Responses

arXiv:2608.26130v1 Announce Type: new Abstract: Coding agents built on large language models (LLMs), such as Claude Code, Cursor, OpenAI Codex, GitHub Copilot, and Aider, receive tool responses that r

Anatomy-Guided Foundation Model Adaptation with Within-Case Prototype Supervision for Standard Plane Detection in Fetal Ultrasound Blind Sweeps

arXiv:2608.27051v1 Announce Type: new Abstract: Detecting the fetal abdominal circumference standard plane in low-cost obstetric blind sweeps is a highly imbalanced frame-classification problem: posit

Assessing mentalization in humans and large language models

arXiv:2608.26291v1 Announce Type: new Abstract: Mentalization - the ability to infer others' beliefs and intentions to guide one's own choices - is a key cognitive function underlying human social int

DRL: A Deterministic Relational Middleware Layer for Transaction-Safe Enterprise NL2SQL Under Schema-Graph Scaling

arXiv:2608.26172v1 Announce Type: cross Abstract: Deploying natural-language interfaces over enterprise OLTP catalogs fails at scale because semantic parsers collapse under schema-graph scaling, infla

GRAIN: Bridging Name and Narrative Shifts in Real-World Graph Reasoning through Invariance-Rewarded Agentic RL

arXiv:2608.27142v1 Announce Type: new Abstract: Despite their potential in standardized graph tasks, Large Language Models (LLMs) remain brittle to real-world shifts in node identifiers and task formu

KnockGS:interaction-Grounded Calibrationof Physical Gaussian Representations

arXiv:2608.27365v1 Announce Type: cross Abstract: Physics-integrated 3D Gaussian representations now allow reconstructed deformable objects to be simulated and rendered under explicit material models.

Multi-Expert Conformal Risk Control for Pairwise LLM Judging in Open-Ended Dialogue

arXiv:2608.26529v1 Announce Type: new Abstract: In this paper, we explore multi-expert Conformal Risk Control (CRC) algorithms for pairwise LLM-as-a-Judge evaluation in open-ended dialogue. Our core i