Search: “benchmark”

Retrieval, Scoring, and Decoding Shape Performance and Stability in LLM-based Conversational Recommendation

arXiv:2609.00086v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as rerankers in conversational recommender systems, yet measured gains depend strongly on the retri

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Same Request, Different Boundary: Evaluating Cybersecurity Assistance across Conversational Contexts

DGX agent

arXiv:2609.00578v1 Announce Type: new Abstract: Large Language Models (LLMs) can solve complex problems, but their misuse in high-risk domains can lead to severe consequences. Model providers therefor

model-releasesarxiv-cs-ai

2 Sept 2026

SciTrue: Reliable Scientific Claim Validation with Frontier and Open Language Models at the NTCIR SciClaimEval Task

arXiv:2609.00654v1 Announce Type: new Abstract: We describe the SciTrue team's participation in both subtasks of the NTCIR-19 SciClaimEval task~ite{sciclaimeval}, which asks systems to verify scientif

Separating perception from reasoning in vision-language models: a model-free render ceiling for crystal structures

arXiv:2609.00663v1 Announce Type: new Abstract: Multimodal evaluations cannot say whether a vision-language model misread an image or misreasoned about it, because every existing method for separating

Simple Additions, Substantial Gains: Expanding Scripts, Languages, and Lineage Coverage in URIEL+

arXiv:2510.27183v3 Announce Type: replace Abstract: The URIEL+ linguistic knowledge base supports multilingual research by encoding languages through geographic, genetic, and typological vectors. Howe

SOVER: Formal Certification of Optimization Reformulations via LLM-Assisted SMT Verification

arXiv:2609.00728v1 Announce Type: new Abstract: Large Language Models (LLMs) have shown remarkable promise in translating and reformulating complex mathematical optimization problems across modeling l

Spatial Autoregressive Modeling of DINOv3 Embeddings for Unsupervised Anomaly Detection

arXiv:2603.02974v2 Announce Type: replace Abstract: DINO models provide rich patch-level representations that have recently enabled strong performance in unsupervised anomaly detection (UAD). Most exi

TopoAlign: A Framework for Aligning Code to Math via Topological Decomposition

arXiv:2510.11944v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) excel at both informal and formal (e.g. Lean 4) mathematical reasoning but still struggle with autoformalisation,

Uncovering the Computational Ingredients of Human-Like Representations in LLMs

arXiv:2510.01030v2 Announce Type: replace Abstract: The human ability to translate diverse perceptual and linguistic inputs into structured behavior has been thought to rest on learning robust represe

ViTAL-X: Video-Text Alignment with Cross-Modal Temporal Edits

arXiv:2609.00505v1 Announce Type: new Abstract: Video-text models adapted from image-text architectures (e.g., CLIP) frequently exhibit temporal blindness, the inability to perceive fundamental cues l

VoiceLongMemEval: Do Assistants Remember How You Sounded?

arXiv:2609.00570v1 Announce Type: new Abstract: With the growing scale of multi-agent architectures and large language models, deployed AI assistants are increasingly tasked with reasoning over long,

AgentRx: Diagnosing AI Agent Failures from Execution Trajectories

arXiv:2602.02475v2 Announce Type: replace Abstract: AI agents often fail in ways that are difficult to localize because executions are probabilistic, long-horizon, multi-agent, and mediated by noisy t

An Explainable Coherence Score for Detecting Temporal Inconsistencies in Political News

arXiv:2608.29175v1 Announce Type: new Abstract: Temporal inconsistencies, such as mandates attributed outside their real interval, events presented as past before they occurred, or inverted causal seq

Analytic Dynamics: Learning Physics-Grounded Representation for Fast Intrinsic Dynamics Inference from Monocular Videos

arXiv:2608.31025v1 Announce Type: new Abstract: Inferring object dynamics from visual observations is essential for intelligent agents to reason about and interact with the physical world, yet remains

AnnotateAnything: Automatic Annotation of 3D Assets for Robot Manipulation

arXiv:2606.17446v2 Announce Type: replace-cross Abstract: Simulation enables scalable robot data collection, but raw 3D assets provide only geometry, lacking the semantic, interactive, and physical kn

Aspire: Can Models Self-Evolve from Vague Goals?

arXiv:2608.31111v1 Announce Type: new Abstract: Many important forms of human learning begin with a vague goal, such as 'become a better physicist' or 'improve at research.' Learners must interpret th

Applications

Beyond Good Intentions: When Does the Framing of Multilingual and Low-Resource NLP Research Become a Caricature?

arXiv:2608.30866v1 Announce Type: new Abstract: Building language technologies and conducting NLP research for low-resource languages---particularly when led by native speakers or involving participat

applicationsarxiv-cs-cl

Ceiling-Clipped Acceptance Histograms Indicate Stranded Speed-up in Block-Diffusion Speculative Decoding

arXiv:2608.30427v1 Announce Type: new Abstract: Speculative decoding speeds up generation with an efficient draft model (drafter) that proposes tokens for a target model to verify in one pass, preserv

Coarse composition suffices: tabular in-context learning for multi-activity antimicrobial peptide profiling

model-releasesarxiv-cs-lg

arXiv:2608.30337v1 Announce Type: new Abstract: Antimicrobial peptides (AMPs) often act against multiple pathogen classes, making multi-label activity prediction a more realistic screening target than

ConCA: Concentration-Aware Channel Attention for Fine-Grained Visual Recognition

arXiv:2608.30183v1 Announce Type: cross Abstract: Lightweight channel attention mechanisms are widely used in image classification, yet their effectiveness in fine-grained visual recognition (FGVR) re

Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decoding

arXiv:2505.16990v3 Announce Type: replace Abstract: In this work, we propose Dimple, the first Discrete Diffusion Multimodal Large Language Model (DMLLM). We observe that training with a purely discre

DocIntent: Answerability-Guided Agentic Restoration for Real-World Document Visual Question Answering

arXiv:2608.29037v1 Announce Type: cross Abstract: Real-world degradations such as blur, shadow, distortion, and moire patterns severely impair the document question-answering capabilities of Multimoda

EmoLASP: Emotion Recognition with Language Models and Answer Set Programming

arXiv:2608.29035v1 Announce Type: new Abstract: Emotion recognition in conversations is increasingly tackled with language models, but these models can be unstable and expensive to fine-tune or to pro

Agents

Evaluating LLM-based AI agents integrated with materials synthesis tools: the case of atomic layer deposition

arXiv:2608.29309v1 Announce Type: cross Abstract: This work provides an overview of the different strategies that can be used to evaluate the performance of AI models and agents based on large languag

agentsarxiv-cs-ai

Evaluating Perspectival Biases in Cross-Modal Retrieval

arXiv:2510.26861v4 Announce Type: replace-cross Abstract: Multimodal retrieval systems are expected to operate in a semantic space, agnostic to the language or cultural origin of the query. In practic

Evidence-Bounded Mental Health Reasoning from Heterogeneous Speech Protocols

arXiv:2608.31014v1 Announce Type: new Abstract: Computational mental health screening using multimodal speech and text has shown great promise. However, existing models often assume all clinical speec

Expert-validated STEM QA

arXiv:2608.28591v1 Announce Type: new Abstract: Recent advancements in AI are helping scientists achieve breakthroughs in fields such as mathematics, medicine, and materials sciences. New evaluation d

From Location Phrases to Geographic Entities: Task-Adapted Retrieval for People Search

arXiv:2608.28965v1 Announce Type: new Abstract: People search must map free-form location phrases to geographic entities used as structured retrieval filters. Lexical standardizers handle canonical na

Generating Workflow DAGs from Natural Language with Non-Reasoning LLMs

arXiv:2608.30250v1 Announce Type: new Abstract: This paper addresses the problem of translating natural-language routing rules written by business administrators into executable workflow graphs for en

Research

GenRubric: Self-Evolving Rubric Generation for Scalable LLM Evaluation

arXiv:2608.29856v1 Announce Type: new Abstract: Large language models are increasingly used as scalable evaluators for open-ended tasks. However, many LLM judges derive query-specific criteria during

researcharxiv-cs-cl

GeoRay: Gauge-Aware Feed-Forward Satellite 3D Reconstruction in the Geodetic Frame

arXiv:2608.29680v1 Announce Type: new Abstract: Feed-forward 3D foundation models reconstruct perspective scenes in one pass. Satellite photogrammetry needs a different product, one that domain adapta

Agents

Ignorance or Incompetence? Constructing Knowledge-Gated, Verifiable Tasks for LLM Agents

arXiv:2608.30322v1 Announce Type: new Abstract: Professional agent tasks often depend on conventions that are absent from public corpora, yet benchmarks rarely control whether an agent has access to t

agentsarxiv-cs-ai

Interpretable Predictability-Based AI Text Detection: A Replication Study

arXiv:2603.15034v2 Announce Type: replace-cross Abstract: This paper replicates and extends the system used in the AuTexTification shared task for authorship attribution of machine-generated texts. Ex

Interpreting and Steering for Safe and Correct Code Generation

arXiv:2608.30025v1 Announce Type: new Abstract: Large language models (LLMs) frequently generate source code containing vulnerabilities, yet little work studies the internal mechanisms that distinguis

Item-Mean Surrogates: Why Richer Persona Data Fail to Improve LLMs as Human Surrogates

arXiv:2608.29455v1 Announce Type: new Abstract: LLMs are increasingly used as human surrogates, often on the premise that richer persona data could make them substitutes or exploratory tools for speci

JFTA-Bench: Evaluate LLM's Ability of Tracking and Analyzing Malfunctions Using Fault Trees

arXiv:2603.22978v2 Announce Type: replace Abstract: In the maintenance of complex systems, fault trees are used to locate problems and provide targeted solutions. To enable fault trees stored as image

Learning Compositional Spatio-Temporal Video Grounding with Synthetic Curriculum

arXiv:2608.30584v1 Announce Type: new Abstract: Despite the impressive progress of recent MLLMs on spatio-temporal video grounding (STVG), existing evaluations and training data focus primarily on sim

LeMat-Synth: a multi-modal toolbox to curate broad synthesis procedure databases from scientific literature

arXiv:2510.26824v2 Announce Type: replace-cross Abstract: Wide access to advanced experimental methods in materials science has given rise to an abundance of procedural knowledge, which is scattered a

LLM Judges Verify Presence, Not Absence: Omission Blindness in AI Clinical Notes and What Recovers It

arXiv:2608.31016v1 Announce Type: cross Abstract: Ambient AI scribes draft clinical notes, and published audits find their dominant error is omission: information the encounter established that the no

MagicSim: A Unified Infrastructure for Executable Embodied Interaction

arXiv:2606.17511v2 Announce Type: replace-cross Abstract: Robot learning and embodied agents now require simulation to serve as a shared execution substrate linking control, skills, and planning, not

Manaca-1B: An Open, Reproducible Brazilian-Portuguese Language Model and a Tokenizer-Aware, Paired Evaluation

arXiv:2608.30114v1 Announce Type: new Abstract: Brazilian Portuguese remains under-served by open language models, and the few that exist are difficult to reproduce and are often compared without meas

Memory-First Fact-Checking: A Knowledge-Graph-Grounded Multi-Agent System for Misinformation Detection

arXiv:2608.29617v1 Announce Type: cross Abstract: This paper introduces a hybrid fact-checking framework that integrates Knowledge Graph-based semantic memory with adversarial multi-agent reasoning fo

MUDDLE: Measuring Understanding of Documents under Distractor and Length Effects

arXiv:2608.29477v1 Announce Type: cross Abstract: Document question-answering systems increasingly answer questions over collections of retrieved documents rather than one clean source, so robustness

On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability

arXiv:2608.30320v1 Announce Type: new Abstract: We describe the architecture and ablations of Qwen3.8-Flash-Next, a sparse mixture-of-experts model with 125B parameters, 6B activated per token, and ad

Agents

Ontology-Guided Multi-Agent Extraction of Evaluation Objects from Academic Review Texts: Evidence from Chinese Library and Information Science

arXiv:2608.29526v1 Announce Type: new Abstract: Academic reviews, scholarly commentaries, and book reviews serve as sources of evaluative statements about theories, methods, literature, institutions,

agentsarxiv-cs-cl

PaperGym: Rubric-Centered Evolution for Research-Plan Generation

arXiv:2608.31119v1 Announce Type: new Abstract: Research planning is the decisive capability of AI scientists. Yet a research plan admits no verifiable answer, so reinforcement learning lacks the envi

Research

Partition-Aware Unlearning for Removing Spurious Correlations in Large Vision-Language Models

arXiv:2608.29996v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) achieve strong performance across many multimodal tasks; however, they often exploit spurious object-background c

researcharxiv-cs-ai

Privacy-Preserving Detection of Rare Disease-Associated Cell Subsets via Secure Multi-Party Computation

model-releasesarxiv-cs-lg

arXiv:2608.20118v1 Announce Type: cross Abstract: The detection of rare disease-associated cell subsets from high-dimensional single-cell measurements is critical for understanding diseases such as le