Search: “benchmark”

OOD Detection for EEG-based Machine Learning in High-Risk Environments

arXiv:2608.17620v1 Announce Type: new Abstract: Machine learning models for electroencephalography (EEG) analysis show great promise across a wide range of applications, but their deployment in high-r

19 Aug 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Pathology Transport: Optimal-Transport Explanations for Clinical Data, and When Their Heatmaps (Fail to) Localize Disease

DGX agent

arXiv:2608.17370v1 Announce Type: new Abstract: Generative models promise a route to explainable clinical AI: rather than probe a classifier, model the distributions of healthy and diseased patients a

model-releasesarxiv-cs-lg

19 Aug 2026

SCENARIODIFF: A Scenario-level Guidance Framework for Multimodal Time Series Forecasting--Extended Version

arXiv:2608.17164v1 Announce Type: new Abstract: Textual context such as news, reports, and logs can provide valuable signals for time series forecasting, especially when future dynamics are driven by

19 Aug 2026

Bias-Corrected Ceilings of Emotion Predictability from Human Label Variation Based on Instance-Level Fano Bounds

arXiv:2608.15619v1 Announce Type: new Abstract: Emotion recognition from text keeps improving on benchmarks, yet whether an accuracy ceiling has been reached is seldom asked with discipline. Our aim i

safetyarxiv-cs-ai

Research

CaliBench: Are the Stochastic Dynamics of Video World Models Physically Calibrated?

arXiv:2608.16829v1 Announce Type: cross Abstract: Video world models approximate the stochastic distribution of physical outcomes through generative sampling, but existing benchmarks score individual

researcharxiv-cs-ai

Certifying Compressed Language Models: An Audit and a Statistical Toolkit

arXiv:2608.15046v1 Announce Type: new Abstract: A fraction of a point of benchmark accuracy is the usual evidence that a compressed model is equivalent to its original. That quantity is least informat

Deep Learning for BioImaging: What Are We Really Learning?

arXiv:2603.13377v2 Announce Type: replace Abstract: Representation learning has driven major advances in natural image analysis by enabling models to acquire high-level semantic features. In microscop

Evaluating Agentic Code Repair Capabilities in Distributed Systems

arXiv:2608.14863v1 Announce Type: cross Abstract: LLM-based coding agents have advanced rapidly on single-process SWE tasks, with frontier models now clustering in the high-70s on SWE-bench Verified.

Kozuchi Agent: A Language-Agnostic Open-Weight Agent for Software Repair

arXiv:2608.15579v1 Announce Type: cross Abstract: Industrial software-engineering teams increasingly need LLM agents that turn bug reports into correct patches, yet benchmark-scale operation adds long

Large Language Models Show Metacognitive Sensitivity in Medical Reasoning

arXiv:2608.14552v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly evaluated and used in medicine, but clinical usefulness depends on answer accuracy and whether confidence

Learn What's Left, Not What's Mastered: Saturation Aware Advantage Reweighting for Multi-Reward Policy Optimization

arXiv:2608.16072v1 Announce Type: cross Abstract: Reinforcement learning (RL) with group-relative advantages has become the de facto standard for post-training language model reasoners. However, when

PDDLCoder: Agentic PDDL Generation for LLM-Assisted Symbolic Planning

arXiv:2608.16637v1 Announce Type: new Abstract: LLMs remain unreliable for long-horizon planning, often generating logically inconsistent or non-applicable plans. Recent hybrid methods instead transla

Synthetic Image Detection with CLIP: Understanding and Assessing Predictive Cues

arXiv:2602.12381v2 Announce Type: replace Abstract: Recent generative models produce near-photorealistic images, challenging the trustworthiness of photographs. Synthetic image detection (SID) methods

The Synthetic Web: Adversarially-Curated Mini-Internets for Diagnosing Epistemic Weaknesses of Language Agents

arXiv:2603.00801v2 Announce Type: replace Abstract: Language agents increasingly act as web-enabled systems that search, browse, and synthesize information from diverse sources. However, these sources

Toward Better Assessment of LLMs' Performance in Clinical Error Detection

arXiv:2608.16643v1 Announce Type: cross Abstract: Automated detection of errors in clinical documentation is a promising application of large language models (LLMs), yet decisions to deploy such model

Local Ai

TRACE-Bench: Decomposing and Diagnosing Multi-Reference Image Generation

arXiv:2608.16765v1 Announce Type: cross Abstract: Despite recent advances in unified multimodal models for multi-reference image generation, existing benchmarks remain organized around predefined task

local-aiarxiv-cs-ai

UI-Mate: Advancing Open-Weight Foundation GUI Agents with In-Context Demonstrations

arXiv:2608.15930v1 Announce Type: new Abstract: Foundation GUI agents can automate complex digital tasks, but deployment is hindered by scarce and biased training data, ambiguous prompts, and unreliab

Walk Before You Run: The Importance of Data Exploration for Data Analysis Agents

arXiv:2608.16045v1 Announce Type: cross Abstract: LLM-based data-analysis tools are increasingly used to help users analyze messy spreadsheets and workbooks, from answering questions over uploaded fil

When Context Misleads: Intent-Guided Decoding for Robust Retrieval-Augmented Generation

arXiv:2608.16515v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) improves large language models by grounding generation in external evidence, but it also introduces a source trus

safetyarxiv-cs-ai

When Single-Dataset Conclusions Fail: A 45-Task Study of Threshold Tuning and Resampling for Imbalanced Classification

arXiv:2608.16147v1 Announce Type: new Abstract: Class-imbalance handling is routinely evaluated on a single benchmark dataset, and the resulting conclusions are reported as if they were properties of

Applications

A Year in LLM Serving: Workload Evolution, Caching and Load-Balancing

arXiv:2608.13573v1 Announce Type: new Abstract: Large Language Model (LLM) serving has become a critical cloud workload, and realistic traces are essential for motivating and benchmarking serving syst

applicationsarxiv-cs-ai

Agentic Transaction: Towards ACID-Compliant Agent Systems

arXiv:2608.13900v1 Announce Type: cross Abstract: Large language model (LLM) agents are evolving from conversational assistants into autonomous systems that execute long-horizon tasks through reasonin

AgentRewind: Recoverable Execution for Long-Horizon LLM Agents

arXiv:2608.14380v1 Announce Type: new Abstract: Many real-world tasks require LLM agents to interact with their environments over long execution horizons. Errors that occur early in execution may prop

AI Research Preference Models

arXiv:2608.13940v1 Announce Type: new Abstract: AI research agents (AIRA) can now propose, implement, and evaluate their own machine learning experiments, but progress on frontier tasks is throttled b

Bootstrapping Niche Multilingual Code Translation via Reinforcement Learning with Execution-Based Verifiable Supervision

model-releasesarxiv-cs-cl

arXiv:2608.13854v1 Announce Type: new Abstract: Code translation must preserve executable behavior across many programming languages, yet neural code translation has largely focused on a few popular l

ChartProbe: A Diagnostic Study on Visual Reasoning through Perception, Grounding, and Simple Reasoning

arXiv:2608.13766v1 Announce Type: new Abstract: Vision-language models (VLMs) remain unreliable on chart questions that require reasoning over visual quantities, and this weakness is usually attribute

Content Depth Matters in Short-Video Recommendation: Rethinking the Attention Economy

arXiv:2608.13990v1 Announce Type: new Abstract: Driven by the attention economy, short-video Recommender Systems (RSs) are primarily optimized to maximize user engagement by promoting videos that capt

CVT-Bench: Probing Spatial-State Integrity through Counterfactual Viewpoint Transformations

arXiv:2603.21114v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) perform strongly on isolated spatial tasks, but whether their predictions remain persistent and mutually co

Debiasing Text-to-Image Evaluation via Implicit Cultural Alignment Reward Modeling

arXiv:2607.15740v2 Announce Type: replace-cross Abstract: As Text-to-Image (T2I) systems rapidly advance, evaluating the cultural authenticity of synthesized content has become increasingly important

Do AI chatbots find what experts would? Effects of model, user role, and sample size on study retrieval for medical questions

arXiv:2608.13786v1 Announce Type: cross Abstract: Large language model (LLM) chatbots are increasingly used to answer clinical questions with citations to relevant clinical studies. Prior research has

Research

Does ISO-Grounded NFR Specification Improve LLM Code Generation? A Comparison of Rich and Structured Interventions against a Natural-Language Baseline

arXiv:2608.13742v1 Announce Type: cross Abstract: In LLM-based code generation, Non-Functional Requirements (NFRs) are often specified as terse one-line phrases. We ask whether grounding those specifi

researcharxiv-cs-ai

EchoChange: A Diffusion Language Model with Dual Pass Remasking for Factual Remote Sensing Disaster Change Captioning

arXiv:2608.01856v2 Announce Type: replace Abstract: Bi-temporal remote-sensing disaster change captioning often needs to identify sparse and spatially localized changes across large pre- and post-even

Evaluating Agentic Learning Harness Capabilities Without Labels via the Scaling Hypothesis

arXiv:2608.13608v1 Announce Type: new Abstract: Agentic 'Continual Learning Harnesses', systems that pair an LLM with retrieval or memory to improve from feedback without retraining, have shown growin

Forecast Collapse in Time-Series Foundation Models

arXiv:2608.14106v1 Announce Type: cross Abstract: When forecasting hourly returns for 1,000 US equities, we observe an unexpected phenomenon: predictions become nearly flat and show poor stock ranking

Language-Specific Gaps in AI Safety Training Datasets

arXiv:2608.13695v1 Announce Type: cross Abstract: Large language model providers routinely cite multilingual safety benchmarks spanning a dozen or more languages as evidence that their models are safe

safetyarxiv-cs-lg

Measuring Fairness in Large Audio Language Models via Semantic-Aware Bias Estimation

arXiv:2608.13624v1 Announce Type: cross Abstract: Large Audio Language Models (LALMs) have seen increasing use for audio understanding tasks such as speech recognition and audio question answering, ra

safetyarxiv-cs-ai

MedClaw: Heuristic Agent Harness for Long-Horizon Surgical Video Reasoning

arXiv:2608.14015v1 Announce Type: cross Abstract: Understanding tens-of-minutes surgical videos requires long-horizon temporal reasoning, answering what happens before, after, or across stages of a pr

Model-agnostic Retrieval-Augmented Extended Forecasting for time series

arXiv:2608.14054v1 Announce Type: new Abstract: Time series forecasting with pretrained foundation models has demonstrated strong zero-shot capabilities. However, achieving optimal performance on time

NAST: Improving Negation Handling in Medical Vision-Language Models through Negation-Aware Selective Training

arXiv:2602.12498v2 Announce Type: replace Abstract: Negation is a fundamental linguistic operation in clinical reporting, yet vision-language models (VLMs) frequently fail to distinguish affirmative f

P2Skill: Privacy Preserving Skill Distillation for Cloud-Local LLM Inference Systems

arXiv:2608.14094v1 Announce Type: cross Abstract: Cloud-local LLM inference systems have the potential to use the reasoning capability of large cloud models while protecting sensitive user data on per

Recent Advances in Deep Learning-Based Drug-Target Binding Affinity Prediction

arXiv:2608.13797v1 Announce Type: new Abstract: Computational approaches to drug discovery involve multiple sub-problems, and among them, drug-target binding affinity prediction plays an important rol

Research

Research-Oriented Human-Centric Evaluation for Foundation Models

arXiv:2506.01793v2 Announce Type: replace Abstract: Most current evaluations of foundation models focus on objective benchmarks, such as knowledge coverage and reasoning accuracy, often overlooking us

researcharxiv-cs-cl

Rewrite Once, Validate Anywhere: Producing OWL-Aware SHACL Constraints (Extended Version)

arXiv:2608.14104v1 Announce Type: cross Abstract: The Shapes Constraint Language (SHACL) is a W3C recommendation to express syntactic constraints, called shapes, on RDF graphs. SHACL validators are us

Second Thought: Reasoning in Parallel as LLM Agents Act and Observe

arXiv:2608.13667v1 Announce Type: new Abstract: LLM agents in the ReAct paradigm alternate between reasoning, acting, and observing, but deliberate reasoning is confined to the Thought phase: while th

Applications

Trajectory Dynamics in Self-Supervised Learning Latent Space for Audio Deepfake Detection

arXiv:2608.13817v1 Announce Type: cross Abstract: Human speech production is constrained by physiology, giving rise to characteristic temporal structure on acoustic signals. We hypothesise that these

applicationsarxiv-cs-lg

Twin: Playing an Unknown Game with a Test-Time Digital Twin

arXiv:2608.14490v1 Announce Type: new Abstract: We present a Test-time World-model Inference (Twin) system, in which a frontier coding agent writes an executable world model for completing continual l

A Contract-Grade Verifier for LLM-Generated GPU Kernels, and a Native Blackwell Backward for the Gated-Linear-Recurrence Family

arXiv:2608.12700v1 Announce Type: new Abstract: Systems that generate GPU kernels with language models report high correctness rates. Those rates come from a single loose test: run the kernel on a few

14 Aug 2026

arepsilon-MemEvo: Adaptive Cross-Task Memory Transfer for LLM Program Evolution

arXiv:2608.12522v1 Announce Type: new Abstract: LLM-based program evolution systems such as FunSearch and AlphaEvolve have shown strong ability to discover novel algorithms, but typically optimize eac