Search: “benchmark”

EvoGenUI-Bench: Evaluating LLMs as Multi-Turn Generative UI Assistants

arXiv:2608.29387v1 Announce Type: new Abstract: Large language models can generate interactive web interfaces, but reliable generative UI requires maintaining an executable artifact as user requests e

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

EvoSkill Injection: Red-Teaming Autonomous Skill Generation and Evolution in Self-Evolving Agents

DGX agent

arXiv:2608.30429v1 Announce Type: new Abstract: LLM-based agent systems increasingly adopt skill-based architectures to reduce repetitive reasoning costs and improve stable, efficient task execution.

model-releasesarxiv-cs-ai

1 Sept 2026

GAFT: Geo-Anchored Fine-Tuning for Hazard Identification from Rare Failures

arXiv:2608.30858v1 Announce Type: cross Abstract: Off-road navigation can fail when physical structures induce irrecoverable states such as high-centering or entrapment, requiring human interventions.

Generative vs. Encoder Models for Multilingual NER: A Comprehensive Empirical Study on Naamapadam

arXiv:2608.29959v1 Announce Type: new Abstract: Language is humanity's most consequential technology, yet for over a billion speakers across India's twenty-two constitutionally recognised languages, i

Gurukul AI: An Interactive AI-Driven Educational Platform for Indian Education System

arXiv:2608.28611v1 Announce Type: cross Abstract: Recent advances in large language models (LLMs) like ChatGPT and LLaMA have transformed AI-driven education, but these systems are predominantly train

MNIST-PRO: MNIST is Back as a Partially Observable World for AI Agents

arXiv:2608.31022v1 Announce Type: new Abstract: AI agents in partially observable environments need to coordinate active sensing with working memory to maintain an evolving perceptual state. However,

SHADOWBENCH: Toward Reliable Automatic Evaluation of Semantic Alignment in Autoformalization

arXiv:2608.29270v1 Announce Type: cross Abstract: Autoformalization translates informal mathematical theorems into code for proof assistants such as Lean. A central challenge is that current evaluatio

SinLlama -- A Large Language Model for Sinhala

arXiv:2508.09115v5 Announce Type: replace Abstract: Low-resource languages such as Sinhala are often overlooked by open-source Large Language Models (LLMs). In this research, we extend an existing mul

Standardizing Longitudinal Radiology Report Evaluation via Large Language Model Annotation

arXiv:2601.16753v2 Announce Type: replace-cross Abstract: Longitudinal information in radiology reports refers to the sequential tracking of findings across multiple examinations over time, which is c

TEMPO: Temporally-grounded Multi-task Post-training for Large Audio-Language Models

arXiv:2608.29999v1 Announce Type: cross Abstract: Large audio-language models (LALMs) describe audio at the clip level but cannot assign timestamps to the events, speakers, or sounds they identify. De

The Price of Intelligence: A Quality-Adjusted Price Index for AI Services

arXiv:2608.29843v1 Announce Type: cross Abstract: Posted prices for AI inference have fallen steadily since 2024, yet the measured speed of that fall depends almost entirely on the method of measureme

VocalAffectBench: Evaluating Vocal Emotion Recognition in AI Audio Models

arXiv:2608.28932v1 Announce Type: new Abstract: Voice products increasingly need affective cues that are present in speech but absent from transcripts. We introduce VocalAffectBench, a public, test-on

Safety

When History Is Multimodal: Rethinking Context Management for Long-Horizon Agents

arXiv:2608.29897v1 Announce Type: new Abstract: Long-horizon agents need a context manager to compress growing interaction histories into a bounded working context, via passive strategies or active st

safetyarxiv-cs-cl

WorldCupArena: Fine-Grained Evaluation of Language Models and Deep-Research Agents on Football Forecasting

arXiv:2607.18084v2 Announce Type: replace Abstract: Predicting a football match before kickoff requires more than knowing past results: a model must use changing information and make a clear predictio

CF-YOLO: Context-Aware Feature Refinement for Camouflaged Industrial Micro-Defect Detection

arXiv:2608.28070v1 Announce Type: new Abstract: Automated detection of surface micro-defects on industrial components, such as copper tubes, is critically important for quality assurance but remains c

Comparing Classical and Quantum Machine Learning for Regression in High Energy Physics Collision Data

arXiv:2608.28084v1 Announce Type: new Abstract: The classification and regression of particle collision events constitute a persistent computational challenge in experimental high energy physics, wher

Cross-Spectral Dense Correspondence for Multimodal Spectral Medical Imaging

arXiv:2608.28341v1 Announce Type: new Abstract: Precise dense correspondence is a fundamental prerequisite for multimodal spectral imaging systems that fuse disparate wavelength ranges for subsequent

Improved off-policy training of diffusion samplers

arXiv:2402.05098v5 Announce Type: replace Abstract: We study the problem of training diffusion models to sample from a distribution with a given unnormalized density or energy function. We benchmark s

NumBench: Diagnosing Counting Failures in Text-to-Image Models

arXiv:2608.28206v1 Announce Type: new Abstract: Text-to-image (T2I) models often generate the wrong number of objects, yet existing benchmarks are too small or weakly controlled to explain why. We int

Post-Training VLMs for Video Mistake Detection

arXiv:2608.28406v1 Announce Type: new Abstract: Human mistakes are inevitable when following instructions, yet they can lead to severe consequences. As such, there has been an increased interest in de

SMRC: Aligning Large Language Models with Student Reasoning for Mathematical Error Correction

arXiv:2511.14684v2 Announce Type: replace Abstract: Large language models (LLMs) often make reasoning errors when solving mathematical problems, and how to automatically detect and correct these error

4DSynth: Controllable Procedural World Synthesis for Dynamic Embodied Simulation

arXiv:2608.26947v1 Announce Type: cross Abstract: Embodied agents need environments that are visually diverse, physically interactive, and changing over time. Procedural simulators can generate large

Accelerating Scientific Research with Gemini in the Real-World

arXiv:2608.26701v1 Announce Type: new Abstract: We present an extension and comprehensive real-world validation of Co-Scientist, a Gemini-based multi-agent system designed to accelerate end-to-end sci

Are Large Language Models Effective Knowledge Graph Constructors?

arXiv:2510.11297v2 Announce Type: replace Abstract: Knowledge graphs (KGs) are widely used in knowledge-intensive applications, yet it remains unclear how effectively current large language models (LL

BekchiAI: Measuring, Observing, and Controlling LLM Agents in One Click

arXiv:2608.26867v1 Announce Type: new Abstract: Large language model agents reason, call tools, and act autonomously over many steps, but their agentic skills-correctly sequencing tools, planning unde

Beyond Classification: Task-Dependent Learnability under Privacy-Motivated Image Transformations

arXiv:2608.27066v1 Announce Type: cross Abstract: Privacy-Enhancing Technologies (PETs) in computer vision often rely on noise or image perturbations to protect visual data while securely processing i

Beyond Factual QA: Mentorship-Oriented Question Answering over Long-Form Multilingual Content

arXiv:2601.17173v2 Announce Type: replace-cross Abstract: Question answering systems are typically evaluated on factual correctness, yet many real-world applications-such as education and career guida

CIFQA: A Deterministic Tool-Grounded Multi-Agent LLM Framework for Financial Query Answering

arXiv:2608.26114v1 Announce Type: new Abstract: Calculation-intensive financial question answering requires exact reasoning over structured rates, temporal conditions, numerical formulas, and rule-bas

DEEPCHART: How Far are LLMs from Faithful Data-Science Chart Generation?

arXiv:2608.26757v1 Announce Type: new Abstract: Faithful chart generation in real-world data-science workflows requires grounding visualizations in scattered evidence, computing chart-ready quantities

Do Coding Agents Need Executable World Models, Simplification, and Verification to Solve ARC-AGI-3?

arXiv:2607.15439v2 Announce Type: replace Abstract: Our previous ARC-AGI-3 agent bundled executable world modeling, prompted simplification, and exact replay verification, leaving their individual con

Don't Overthink, Don't Underthink: Toward Adaptive Reasoning in Agentic AI

arXiv:2608.26442v1 Announce Type: new Abstract: Recent advances in Large Language Models (LLMs) have shown that increased inference-time reasoning can improve performance on complex tasks. However, ma

ElementCheck: Complexity-Aware Long-Form Text Factuality Evaluation via Sentence Elements

arXiv:2608.26118v1 Announce Type: new Abstract: Existing long-form factuality evaluation relies on the decompose-retrieve-verify pipeline. However, the pipeline suffers from noise from claim decomposi

Graph-Guided Selective Unlearning for Language Models: Controlling Support Routes Beyond Forget Seeds

arXiv:2608.26743v1 Announce Type: new Abstract: Enterprises fine-tune language models on proprietary data that may later require removal due to privacy, contractual, or compliance obligations. Selecti

Hallucinations in LLMs: A Lifecycle-Based Survey of Causes, Detection, Mitigation, and Prevention

arXiv:2608.26168v1 Announce Type: cross Abstract: The lifecycle of hallucination in LLMs is a concept that enables building solid frameworks on the control and reliability of LLMs in high-stakes envir

How Unlikely Is 'Unlikely'? Assessing Verbal Probability Perception Across Large Language Models

arXiv:2608.26327v1 Announce Type: cross Abstract: Large language models increasingly produce and interpret verbal probability expressions, yet whether these expressions carry consistent meaning across

Invocation-Level Reliability of Tool-Using Agents

arXiv:2608.26189v1 Announce Type: new Abstract: Tool-using agents fail two ways: choosing the wrong tool, or forming wrong arguments, and an early failure of either kind can silently corrupt everythin

Knowledge-Verified Emergent Deception in LLM Agents Under Conflicting Incentives

arXiv:2608.26372v1 Announce Type: cross Abstract: Large language models are increasingly deployed as autonomous agents serving users on behalf of companies, placing them in settings where user and dep

MAPLE: Metadata Conditioned LLM Pretraining for Locale-Aware Question Answering

arXiv:2601.15236v2 Announce Type: replace Abstract: Large language models can memorize competing locale-specific facts yet fail to select among them when the locale changes, defaulting instead to a si

Not Just Reason, Not Just Scan: Reinforcement Learning for Proactive Scientific Error Verification over Academic Paper

arXiv:2608.26596v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) are increasingly capable scientific assistants, yet they remain far from fully autonomous research. This transi

Omni-Interactive Universal Embedder

arXiv:2608.27044v1 Announce Type: new Abstract: Multimodal representation learning has been shifting from traditional two-tower architectures to large language model (LLM)-based embedders due to their

PredVLA: A Sub-Million-Parameter Predictive-Coding Policy for Robot Manipulation

model-releasesarxiv-cs-ro

arXiv:2608.26673v1 Announce Type: new Abstract: Large pretrained vision-language-action models dominate modern robot-manipulation benchmarks, but it remains unclear how much model scale is necessary f

SLM-Conditioned Hierarchical Relation Routing for Labeled Property Graph Learning

arXiv:2608.26132v1 Announce Type: cross Abstract: Labeled property graphs combine relational structure with heterogeneous textual and categorical properties attached to both nodes and relationships. C

Temporally-Grounded Language Generation: Towards Real-Time Vision-Language Models

arXiv:2505.11326v2 Announce Type: replace-cross Abstract: Vision-language models (VLMs) have shown remarkable progress in offline tasks such as image captioning and video question answering. However,

A General-Purpose Molecular Foundation Model Transfers Across Diverse Olfactory Tasks

arXiv:2608.25893v1 Announce Type: new Abstract: Foundation models have transformed molecular property prediction, yet it remains unclear whether a molecular foundation model, fine-tuned on a single ca

Applications

EduDial: Constructing a Large-scale Multi-turn Teacher-Student Dialogue Corpus

arXiv:2510.12899v3 Announce Type: replace Abstract: Recently, several multi-turn dialogue benchmarks have been proposed to evaluate the conversational abilities of large language models (LLMs). As LLM

applicationsarxiv-cs-cl

FrontierChallenge: Evaluating Scientific Workflow Completion

arXiv:2608.24979v1 Announce Type: cross Abstract: Scientific agents increasingly analyze data, execute code, and produce research artifacts, yet most benchmarks emphasize final answers, isolated progr

Research

HealthBench-Psych: A Mental Health Subset of OpenAI's HealthBench

arXiv:2608.25071v1 Announce Type: new Abstract: General-purpose health benchmarks increasingly anchor claims about LLM medical performance, but they are not always resolved by clinical specialty, maki

researcharxiv-cs-cl

Mind2Report: Expert-Level Commercial Report Synthesis via Cognitive Deep Research Agent

arXiv:2601.04879v2 Announce Type: replace Abstract: Synthesizing informative commercial reports from massive and noisy web sources is critical for high-stakes business decisions. Although recent deep