Search: “evaluation”

PPTArena: A Benchmark for PowerPoint Editing

arXiv:2512.03042v3 Announce Type: replace-cross Abstract: We introduce PPTArena, a benchmark for PowerPoint editing that evaluates how agents modify real slides from natural-language instructions. Unl

Local Ai

Procedural Memory Distillation: Online Reflection for Self-Improving Language Models

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

DGX agent

arXiv:2607.01480v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR), along with recent selfdistillation variants such as SDPO, evaluates each rollout against a verifi

local-aiarxiv-cs-ai

3 Jul 2026

Safety Testing LLM Agents at Scale: From Risk Discovery to Evidence-Grounded Verification

arXiv:2607.01793v1 Announce Type: new Abstract: LLM agents increasingly perform autonomous actions through external tools, leading to complex and evolving safety risks. However, existing safety testin

Scalable and Distributed Silhouette Approximation

arXiv:2607.01993v1 Announce Type: cross Abstract: The silhouette is one of the most widely used measures to assess the quality of a k-clustering of a dataset of n elements. Its evaluation requires no

researcharxiv-cs-lg

Single-Channel EEG-Based Cognitive Load Assessment in Online Learning: A Hybrid Deep Learning Approach

arXiv:2607.01795v1 Announce Type: cross Abstract: Monitoring cognitive load during online learning could help instructors identify content that learners find difficult, but remote settings remove the

researcharxiv-cs-ai

Something about this year’s @aiDotEngineer World’s Fair just hit different. Last year was the year of “let the agents rip.” This year was th…

model-releasesyohei-nakajima--x

Something about this year’s @aiDotEngineer World’s Fair just hit different. Last year was the year of “let the agents rip.” This year was the year of realizing that autonomy without structure creates

The Eticas AI Risk Taxonomy: Open Infrastructure for Operationalizing AI Audits

arXiv:2607.02201v1 Announce Type: cross Abstract: The rapid deployment of AI systems across high-stakes domains has created urgent demand for standardized evaluation, yet the field remains fragmented

Are Performance-Optimization Benchmarks Reliably Measuring Coding Agents?

arXiv:2607.01211v1 Announce Type: cross Abstract: Repository-level performance-optimization benchmarks such as GSO, SWE-Perf and SWE-fficiency evaluate coding agents by applying patches to real reposi

Bayesian Uncertainty Propagation for Agentic RAG Pipelines: A Proof-of-Concept Study on Multi-Hop Question Answering

arXiv:2607.00972v1 Announce Type: new Abstract: Trustworthy deployment of Agentic Retrieval-Augmented Generation (RAG) systems requires mechanisms for estimating when multi-stage reasoning pipelines m

Best practices for multi-turn reinforcement learning in Amazon SageMaker AI

In this post, we share best practices for reliable multi-turn RL training. We cover how to build a training environment you can trust, set up an external evaluation, design a reward aligned with the e

agentsaws-ml-blog

Deconfounded Lifelong Learning for Autonomous Driving via Dynamic Knowledge Spaces

arXiv:2603.14354v3 Announce Type: replace-cross Abstract: End-to-End autonomous driving (E2E-AD) systems face challenges in lifelong learning, including catastrophic forgetting, difficulty in knowledg

agentsarxiv-cs-ai

Disentangling Speaker and Language Effects in Cross-Lingual Speaker Verification for Iberian Languages

arXiv:2607.01161v1 Announce Type: cross Abstract: Cross-lingual speaker verification (SV) systems typically exhibit performance degradation when enrollment and test utterances are spoken in different

researcharxiv-cs-cl

http://ora.ai is super useful. analyzes the 'agent readiness' of your site, and then gives you a prompt for your coding agents to fix (i'm u…

ORA.ai is a tool that evaluates website 'agent readiness' by analyzing how well-suited a site is for autonomous agent interaction, then generates prompts that coding agents can use to implement necess

agentsyohei-nakajima--x

MoHallBench: A Benchmark for Motion Hallucination in Video Large Language Models

arXiv:2607.01117v1 Announce Type: new Abstract: Video Large Language Models (VideoLLMs) have shown strong progress in video understanding, yet they still suffer from hallucinations that are inconsiste

Multiplicity is an Inevitable and Inherent Challenge in Multimodal Learning

arXiv:2505.19614v2 Announce Type: replace-cross Abstract: Multimodal learning has seen remarkable progress, particularly with large-scale pre-training across various modalities. Most current approache

safetyarxiv-cs-cv

MultiSynt/MT: Trillion-Token Multi-Parallel Pre-Training Data Translated Across 36 Languages

model-releasesarxiv-cs-cl

arXiv:2607.00890v1 Announce Type: new Abstract: Open web-scale pre-training corpora remain concentrated in English, limiting multilingual LLM development. We introduce MultiSynt/MT, an open synthetic

On the Reliability of Cue Conflict and Beyond

arXiv:2603.10834v4 Announce Type: replace-cross Abstract: Understanding how neural networks rely on visual cues offers a human-interpretable view of their internal decision processes. The cue-conflict

Prompt Optimization for User Simulation in Conversational Recommender Systems: A Multi-Objective Framework

arXiv:2607.00010v1 Announce Type: cross Abstract: Conversational recommender systems (CRSs) are a core component of next-generation intelligent recommender systems because they enable users to activel

safetyarxiv-cs-ai

RoadBench: Benchmarking MLLMs on Fine-Grained Spatial Understanding and Reasoning under Urban Road Scenarios

arXiv:2511.18011v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) have demonstrated powerful capabilities in general spatial understanding and reasoning. However, their fine

The Model Organism Lottery: Model Organism Interpretability Strongly Depends on Training Methodology

model-releasesarxiv-cs-lg

arXiv:2607.01033v1 Announce Type: new Abstract: Model organisms (MOs) - language models trained to exhibit undesired or unnatural behaviours - are frequently used as testbeds for evaluating white-box

VLM-AR3L: Vision-Language Models for Absolute and Relative Rewards in Reinforcement Learning

arXiv:2607.00483v1 Announce Type: new Abstract: Designing effective reward functions remains a major challenge in reinforcement learning (RL), particularly in open-ended environments where task goals

agentsarxiv-cs-ro

Autoresearch: The feedback loop behind self-improving agents

Autoresearch describes a feedback mechanism where AI agents can evaluate their own outputs and use that introspection to iteratively improve their reasoning and decision-making capabilities. This conc

agentslatent-space

DigitalCoach: Communication and Grounding Gaps in Human and Agentic Computer Use Coaching

arXiv:2606.31980v1 Announce Type: new Abstract: Agents are increasingly capable of automating software tasks, but can they teach humans how to use software themselves? We introduce DigitalCoach, a mul

agentsarxiv-cs-cl

Hermes Agent v0.18.0 - The Judgement Release Changelog below:

Hermes Agent v0.18.0, released by Nous Research, introduces 'The Judgement' update with improvements to the agent's decision-making and evaluation capabilities. This version likely includes enhancemen

agentsnous-research--x

Knowledge-Driven Dimension Estimation from a Single Image -3D Asset Generation Technology for Digital Twin Construction

arXiv:2606.30896v1 Announce Type: new Abstract: In the verification of in-vehicle cameras, simulation technology using virtual spaces has advanced, enabling pre-evaluation of false detections and miss

agentsarxiv-cs-cv

M3CoTBench: Benchmark Chain-of-Thought of MLLMs in Medical Image Understanding

arXiv:2601.08758v4 Announce Type: replace-cross Abstract: Chain-of-Thought (CoT) reasoning has proven effective in enhancing large language models by encouraging step-by-step intermediate reasoning, a

Moral Safety in LLMs: Exposing Performative Compliance with Puzzled Cues

model-releasesarxiv-cs-cl

arXiv:2606.31644v1 Announce Type: new Abstract: As large language models take on morally consequential roles in healthcare, legal, and hiring contexts, we need to examine whether their ethical behavio

Multi-GPU kernels are the real test for coding models. Today at @aiDotEngineer, @simran_s_arora shared ParallelKernelBench, an open-source b…

model-releasestogether-ai--x

Multi-GPU kernels are the real test for coding models. Today at @aiDotEngineer, @simran_s_arora shared ParallelKernelBench, an open-source benchmark for evaluating whether LLMs can write fast CUDA ker

PolicyGuard: From Organizational Policies to Neuro-SymbolicCompliance Review Engines

arXiv:2606.32004v1 Announce Type: new Abstract: Policy-grounded document review requires determining whether a target document complies with organization-specific policies, guidelines, or playbooks. W

safetyarxiv-cs-ai

PPT-Eval: A Benchmark for Computer-Use Agents on PowerPoint Tasks

arXiv:2606.31154v1 Announce Type: cross Abstract: Creating and editing slides is a rich, multimodal activity that is ubiquitous in professional and educational settings, making it an ideal testbed for

Tools

REMSA: Foundation Model Selection for Remote Sensing via a Constraint-Aware Agent

arXiv:2511.17442v3 Announce Type: replace-cross Abstract: Foundation Models (FMs) are increasingly integrated into remote sensing (RS) pipelines. These models include unimodal vision encoders and mult

Review Residuals: Update-Conditioned Residual Gating for Transformers

model-releasesarxiv-cs-cl

arXiv:2606.31859v1 Announce Type: cross Abstract: Residual connections add every sublayer's proposed update with a fixed coefficient of one; the network never evaluates whether an update is reliable b

RGBT-GroundBench: Visual Grounding Beyond RGB in Complex Real-World Scenarios

arXiv:2512.24561v2 Announce Type: replace Abstract: Visual grounding (VG) localizes target objects in an image from natural-language expressions. In real-world perception, RGB cues often degrade under

RigorBench: Benchmarking Engineering Process Discipline in Autonomous AI Coding Agents

arXiv:2606.22678v2 Announce Type: replace-cross Abstract: Agentic coding harnesses - such as Agent-Skills, Superpowers, and Agent-Rigor - are increasingly deployed to augment underlying LLMs for real-

Same-Origin Policy for Agentic Browsers

arXiv:2606.14027v3 Announce Type: replace-cross Abstract: Agentic browsers integrate autonomous AI agents into web browsers, enabling users to accomplish web tasks through natural-language instruction

Sequential RC-TGAN: Generating Relational Time Series with Spectral Envelope Loss

model-releasesarxiv-cs-lg

arXiv:2606.31904v1 Announce Type: new Abstract: The generation of synthetic relational databases often involves modeling complex temporal dynamics, such as transaction logs or event sequences. A signi

Tools

“Valued work per watt is the score” @rolandgvc with the real details of self improving agents at @aiDotEngineer!

This post discusses a framework for evaluating self-improving AI agents where 'value per watt' serves as a key metric, likely examining how efficiently these agents generate useful output relative to

toolsswyx--x

What Drives Interactive Improvement from Feedback?

arXiv:2606.30774v1 Announce Type: new Abstract: We study when natural-language feedback produces improvement beyond the gains obtainable from repeated attempts alone. In multi-turn language agent sett

agentsarxiv-cs-ai

5ting at SemEval-2026 Task 8: Strong End-to-End Multi-Turn RAG via LLM-Based Reranking and Faithfulness Control

arXiv:2606.28737v1 Announce Type: cross Abstract: We introduce 5ting, our system for the SemEval2026 Task 8 (MTRAGEval), which evaluates multi-turn Retrieval Augmented Generation (RAG) systems. Multi

researcharxiv-cs-ai

A multi-architecture study of specificity refinement and false-positive mechanism analysis in prostate MRI

arXiv:2606.29977v1 Announce Type: cross Abstract: Objectives: To characterize residual false positives in prostate MRI detection, and to evaluate a lightweight post-hoc refinement head for case-level

A Physics-Grounded Benchmark for Multi-Agent Dynamics in World Models

arXiv:2606.28757v1 Announce Type: new Abstract: Generative world models hold immense promise as scalable simulators for autonomous systems, particularly for synthesizing rare but safety-critical multi

Agile Reinforcement Learning through Separable Neural Architecture and Applications

arXiv:2601.23225v2 Announce Type: replace-cross Abstract: Deep reinforcement learning (RL) is increasingly deployed in resource-constrained environments, yet go-to function approximators - multilayer

Local Ai

AI Trading's Alpha Singularity: Emergent Market Reasoning through Agent-to-Agent Self-Evolution

arXiv:2606.29194v1 Announce Type: new Abstract: Automated alpha mining holds the scoring function fixed and varies the search algorithm over it. A search that converges against a fixed scorer overfits

local-aiarxiv-cs-ai

Behavior Prompting Policy: Demonstrations as Prompts for Manipulation

arXiv:2606.30457v1 Announce Type: new Abstract: We study behavior prompting, a paradigm that enables robots to perform new tasks at inference time given a single human demonstration, which we call a b

safetyarxiv-cs-ro

Beyond Backscatter: AlphaEarth Land-Cover Priors for Rapid SAR Flood Segmentation Across Foundation Backbones

arXiv:2606.29134v1 Announce Type: new Abstract: Rapid flood mapping is critical for emergency response, yet optical imagery is often unusable during major flooding and single-temporal SAR is ambiguous

safetyarxiv-cs-cv

Beyond IID: How General Are Tabular Foundation Models, Really?

arXiv:2606.30410v1 Announce Type: cross Abstract: Foundation models for predictive machine learning on tabular data have recently gained significant traction in academia and industry. Research communi

Building to the Test: Coding Agents Deliver What You Check, Not What You Requested

arXiv:2606.28430v1 Announce Type: cross Abstract: Benchmarks are widely used to evaluate task completion by Large Language Models (LLMs), but this approach has accumulated construction-validity proble