Search: “evaluation”

Trajectory-Level Automatic Curriculum Learning for Legged Locomotion on Unstructured Terrain

arXiv:2608.16164v1 Announce Type: new Abstract: Training locomotion policies for complex unstructured terrain requires a curriculum to avoid early exploration failures. However, since unstructured ter

safetyarxiv-cs-ai

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

TwinGridShield: Consequence-Aware Runtime Authorization for LLM Grid-Agent Actions

DGX agent

arXiv:2608.15391v1 Announce Type: new Abstract: Large language model (LLM)-assisted energy-management tools can translate natural-language context into structured grid commands, but syntactic validity

safetyarxiv-cs-ai

18 Aug 2026

VARM-Bench: Benchmarking Verifiable Structured Reasoning in Chinese Abusive Speech Moderation

arXiv:2608.15600v1 Announce Type: new Abstract: The widespread circulation of abusive online content has increased the need for reliable moderation of Chinese social-media text. Existing Chinese bench

VFIG: Vectorizing Complex Figures in SVG with Vision-Language Models

arXiv:2603.24575v2 Announce Type: replace-cross Abstract: Scalable Vector Graphics (SVG) are essential for technical illustration and digital design, offering resolution independence and semantic edit

WANDR: A Benchmark for Wide and Deep Research

model-releasesarxiv-cs-lg

arXiv:2608.14747v1 Announce Type: new Abstract: WANDR (Wide ANd Deep Research) is a benchmark of 500 realistic, challenging data-collection tasks for research agents. Each task requires a system to di

When Single-Dataset Conclusions Fail: A 45-Task Study of Threshold Tuning and Resampling for Imbalanced Classification

arXiv:2608.16147v1 Announce Type: new Abstract: Class-imbalance handling is routinely evaluated on a single benchmark dataset, and the resulting conclusions are reported as if they were properties of

X^2Localizer: Cross-grained Alignment for Progressive Cross-view Video Geo-localization

arXiv:2608.16658v1 Announce Type: cross Abstract: Cross-view Video Geo-localization (CVG) aims to localize ground-view videos by retrieving their corresponding geo-tagged aerial images. However, CVG a

A Hybrid LLM-Based Framework for Automated Security Annotation Generation in Business Process Models

arXiv:2608.14370v1 Announce Type: cross Abstract: The modelling and analysis of secure business processes require the incorporation of security annotations into process models. Although BPMN extension

AgentRewind: Recoverable Execution for Long-Horizon LLM Agents

arXiv:2608.14380v1 Announce Type: new Abstract: Many real-world tasks require LLM agents to interact with their environments over long execution horizons. Errors that occur early in execution may prop

AnchorBench: A Multi-Pathway Benchmark for the Anchoring Effect in LLMs

arXiv:2608.14320v1 Announce Type: new Abstract: The anchoring effect is a cognitive bias in which an initial reference value shifts a later judgment toward itself. This effect is well established in h

APTER: Adaptive Post-Training with Expert-Grounded Rubrics

arXiv:2608.14212v1 Announce Type: new Abstract: As large language models enter professional domains, they must satisfy domain constraints, include critical evidence, and provide complete reasoning rat

AutoSchema: Live Schema Grounding for Agentic Text-to-Sparql over Heterogeneous Knowledge Graphs

arXiv:2608.14228v1 Announce Type: new Abstract: Life science knowledge graphs make large collections of structured data available through SPARQL, but each resource uses its own schema, identifiers, an

agentsarxiv-cs-lg

BUZZY: Contrastive Scoring to Mitigate Text-Induced Bias in Multimodal Multiple-Choice QA

arXiv:2603.28026v2 Announce Type: replace Abstract: Multimodal multiple-choice question answering (MCQA) provides a standardized and objectively measurable setting for evaluating vision-language model

safetyarxiv-cs-ai

Content Depth Matters in Short-Video Recommendation: Rethinking the Attention Economy

arXiv:2608.13990v1 Announce Type: new Abstract: Driven by the attention economy, short-video Recommender Systems (RSs) are primarily optimized to maximize user engagement by promoting videos that capt

Context Aware AI Assistant and AR Interface for Lunar Extravehicular Activity (EVA) Procedural Guidance

arXiv:2608.13589v1 Announce Type: cross Abstract: As human space exploration returns to the Moon, astronauts need rapid access to procedural information during extravehicular activities (EVAs), where

FLARE MCMC: Fidelity-based Layer-Adaptive REcursive proposals for MCMC

arXiv:2608.13774v1 Announce Type: new Abstract: Markov chain Monte Carlo (MCMC) requires only the ability to evaluate the likelihood, making it a common technique for inference in complex models. Howe

FlatLab: A Unified Methodology Framework and Simulation-Based Benchmark for Robotic Manipulation of Flat Objects

model-releasesarxiv-cs-ro

arXiv:2608.14049v1 Announce Type: new Abstract: Robotic manipulation of flat objects is challenging due to the ungraspable configurations and strong variations in object geometry and material. Existin

Forecast Collapse in Time-Series Foundation Models

arXiv:2608.14106v1 Announce Type: cross Abstract: When forecasting hourly returns for 1,000 US equities, we observe an unexpected phenomenon: predictions become nearly flat and show poor stock ranking

Jais 2: A Family of Arabic-Centric Open Large Language Models

arXiv:2608.13580v1 Announce Type: cross Abstract: Jais 2 is a family of Arabic-centric large language models developed jointly by MBZUAI, Cerebras, and Inception, designed to advance Arabic-centric la

Multi-Objective Bayesian Optimization for Model Merging

arXiv:2608.14264v1 Announce Type: cross Abstract: Model merging combines trained models directly in weight space, offering a compute-efficient alternative to additional fine-tuning. Selecting merge pa

Nanbeige4.2-3B on Apple Silicon: Fixing Deployment Bugs and Decreasing Looped Transformer Memory Overhead

arXiv:2608.13987v1 Announce Type: new Abstract: Nanbeige4.2-3B is a 3B-parameter agentic model built around a Looped Transformer (LT) that reuses one stack of layers for a second forward pass, adding

Ontology-Grounded World Models for Failure Diagnosis and Closed-Loop Repair in Physical AI Systems

arXiv:2608.13901v1 Announce Type: new Abstract: EV-WM represents candidate quality with feature and event scores, but these scores do not explicitly record an unmet task predicate, a route label for a

researcharxiv-cs-ro

Polaris : Multi Agentic System for Conversational Enterprise Analytics

arXiv:2608.14246v1 Announce Type: new Abstract: In today's fast-paced environment, the ability to swiftly access, understand, and act on data is no longer optional; it is essential. Yet most organizat

TimeSage-EV: A Live Benchmark for Agentic Time Series Analysis in Evolving Environments

arXiv:2608.14270v1 Announce Type: new Abstract: Time series analysis in high-stakes domains relies on recurring data releases, where new observations can alter the evidence base and the validity of la

TriageSim: A Conversational Emergency Triage Simulation Framework from Structured Electronic Health Records

arXiv:2603.10035v3 Announce Type: replace Abstract: Research in emergency triage is restricted to structured electronic health records (EHR) due to regulatory constraints on nurse-patient interactions

researcharxiv-cs-cl

Based on an accelerating frontier -> local trajectory, expect a ~30b param 'Mythos at home' by as soon as Jan 2027 (rationalisation below)

model-releasesr-localllama

Including the rationalisation for the data below - this is a more robust version of an earlier post I did similar to this - explaining below: How I chose the comparisons The basic question I’m trying

16 Aug 2026

Applications

Social science has spent the last century figuring out solutions to the problems facing AI benchmarking today, you don’t need to reinvent ev…

applicationsethan-mollick--x

Social science has long addressed the challenge of evaluating qualities that cannot be objectively verified—such as writing quality or creative ideas—and its methodologies remain relevant for today’s

16 Aug 2026

Bias Mitigation in Face Recognition via Demographic-based Supervised Contrastive Learning

arXiv:2608.12971v1 Announce Type: new Abstract: Face recognition systems have been shown to be biased toward certain demographic groups by exhibiting different error rates across gender, age, or ethni

safetyarxiv-cs-cv

Confidence-Calibrating Regularization for Robust Brain MRI Segmentation Under Domain Shift

arXiv:2509.23176v2 Announce Type: replace Abstract: The Segment Anything Model (SAM) exhibits strong zero-shot performance on natural images but suffers from domain shift and overconfidence when appli

researcharxiv-cs-cv

Correct Is Not Governed: Provenance Integrity in Agentic Workflows

arXiv:2608.12761v1 Announce Type: new Abstract: Agentic workflows are commonly evaluated by whether they reach the correct outcome. That is insufficient in institutional settings, where a correct acti

Enhancing In-Hospital Mortality Prediction Using Multi-Representational Learning with LLM-Generated Expert Summaries

arXiv:2411.16818v2 Announce Type: replace-cross Abstract: To evaluate a multi-representational framework in which large language model (LLM)-generated expert summaries of intensive care unit (ICU) not

Entropy-Augmented Multi-Objective Policy Optimization in Multiagent Systems

arXiv:2608.12534v1 Announce Type: cross Abstract: Autonomous agent teams deployed in settings such as marine and extraterrestrial outposts must coordinate actions to achieve optimal outcomes across mu

safetyarxiv-cs-ro

FastThaiG2P: Lightning-fast Thai Grapheme-to-phoneme Conversion for Voice Agent Pipelines

model-releasesarxiv-cs-cl

arXiv:2608.12814v1 Announce Type: new Abstract: FastThaiG2P provides sub-millisecond Thai grapheme-to-phoneme conversion for text-to-speech pipelines (International Phonetic Alphabet and Kokoro-TTS co

Local Ai

HIMEC: Directional Change Representation and Fixed-Interface Decoding for Remote Sensing Image Change Captioning

arXiv:2608.12502v1 Announce Type: new Abstract: Remote sensing image change captioning (RSICC) converts bitemporal imagery into a sentence describing semantic changes. Most RSICC methods condition cap

local-aiarxiv-cs-cv

Interpretable Causal Discovery via Causal-Effect Constraints

arXiv:2608.12640v1 Announce Type: cross Abstract: Causal discovery aims to uncover the underlying causal relationships given data generated from a system. The goal, however, is not merely to predict c

JailWAM: Jailbreaking World Action Models in Robot Control

arXiv:2604.05498v2 Announce Type: replace Abstract: World Action Models (WAMs) have emerged as a promising paradigm for robotic manipulation, enabling physical interaction across diverse tasks and env

safetyarxiv-cs-ro

Lines and Ladders: A Context-Aware Multi-Agent Framework for Large-Scale Retail Price Taxonomy

arXiv:2608.12674v1 Announce Type: new Abstract: Maintaining price consistency and executing an Every Day Low Price strategy is critical for global retailers. However, with catalogs spanning millions o

MatchMiner-AI: Open-source, Privacy-preserving Cancer Clinical Trial Matching using Artificial Intelligence

arXiv:2412.17228v4 Announce Type: replace Abstract: Background: Clinical trials are essential to advancing cancer treatments, but fewer than 10% of adults with cancer enroll in therapeutic trials. Ope

MOSAIC: Unveiling the Moral, Social and Individual Dimensions of Large Language Models

arXiv:2603.00048v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are increasingly deployed in sensitive applications including psychological support, healthcare, and high-stakes

Local Ai

Multi-Agent Scheduling with LLM-Assisted Contract Net Negotiation for Stream Processing in Mobile Edge Computing

arXiv:2608.12371v1 Announce Type: new Abstract: Stream-processing systems increasingly operate across heterogeneous mobile edge--cloud infrastructures, where workload volatility, resource contention,

local-aiarxiv-cs-ai

On the Structural Limits of Machine Learning Decision Systems: An Information-Theoretic, Interaction-Based, and Stochastic-Dynamical Perspective

arXiv:2608.13510v1 Announce Type: cross Abstract: Machine learning procedures are commonly evaluated in terms of predictive accuracy and computational efficiency. However, their achievable performance

agentsarxiv-cs-lg

PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectives

model-releasesarxiv-cs-cv

arXiv:2608.13552v1 Announce Type: new Abstract: Video world models simulate future states conditioned on current observations and user actions. Recent systems have demonstrated impressive video consis

Position: Reasoning is a Learnable Rule-Based Process

arXiv:2608.12325v1 Announce Type: new Abstract: Autonomous reasoning is among the most scientifically and economically motivating topics in AI today. Historically the purview of symbolic AI, recent ad

Reliability-Aware Sexism Detection: Combining DPO with Annotator Agreement and Token-Level Confidence Scoring

model-releasesarxiv-cs-cl

arXiv:2608.12330v1 Announce Type: new Abstract: The detection of online sexism remains an open problem. Sexism detection is inherently subjective, yet most existing systems reduce multi-annotator labe

SkillEvo: Self-Renewing Evolution Gradients from Multi-Turn Interaction Feedback

arXiv:2608.13120v1 Announce Type: new Abstract: Agent Skills are today either hand-authored or produced in a single LLM generation pass, and consequently possess no closed loop through which they migh

Spatial Memory Agent: Experience-Grounded Procedure Memory for Spatial Intelligence

arXiv:2608.12743v1 Announce Type: new Abstract: Spatial intelligence is becoming a foundation for embodied agents, robotic planning, and multimodal assistants. To improve the spatial reasoning ability

The Illusion of Improvement: Reject Inference Strategies in Credit Scoring

arXiv:2606.18479v2 Announce Type: replace Abstract: Reject inference methods are widely used to mitigate survival bias in credit scoring, yet their effectiveness remains poorly understood. We systemat

safetyarxiv-cs-lg

Training and Benchmarking Code Generation for Physics-Inspired Animations

arXiv:2602.10840v2 Announce Type: replace Abstract: Large language models (LLMs) have been widely studied in areas such as mathematical reasoning, complex coding, and scientific problem solving. Howev

researcharxiv-cs-lg