Search: “evaluation”

AppWorld-UL: Benchmarking Diverse Agent-User Interactions for Tool-Use

arXiv:2607.20536v1 Announce Type: new Abstract: Tool-use agents that address day-to-day digital tasks such as ordering groceries must not only operate applications, but also interact with the user, e.

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

AXIS: A Growable Community-Driven Data Engine for Scalable Robot Manipulation

DGX agent

arXiv:2607.21588v1 Announce Type: new Abstract: Learning effective robot manipulation policies requires diverse, high-quality demonstrations, yet existing data pipelines are often difficult to scale b

model-releasesarxiv-cs-ro

24 Jul 2026

CachyLLama’s: llama.cpp fork with persistent KV cache that makes long local-agent sessions much less painful

model-releasesr-localllama

I’m not affiliated with this project, but I’ve been running it recently and I’m surprised it hasn’t received more attention here: https://github.com/fewtarius/CachyLLama CachyLLama is a fork of llama.

Can AI Agents Really Complete RTL-to-GDS? Lessons from Benchmarking Tool-Interactive EDA Workflows

arXiv:2607.17528v3 Announce Type: replace Abstract: Large language model (LLM) agents are extending electronic design automation (EDA) beyond static RTL generation toward long-horizon, tool-interactiv

agentsarxiv-cs-ai

Capital Markets LLM Reliability Score (CM-LRS): From Plausible to Bankable

arXiv:2607.21340v1 Announce Type: new Abstract: In capital-markets workflows the question is rarely whether a large language model can produce a fluent draft, but whether the draft is bankable: defens

Cardinality-Decomposed Loss: Matching Training Objectives to Relation Structure in Heterogeneous Recommendation Graphs

arXiv:2607.20737v1 Announce Type: new Abstract: Graph Neural Networks trained on heterogenous bipartite graphs form a common basis in recommendation systems. These graphs often express relations that

Chronofy: A Temporal-Logical Decay Architecture for Information Validity in Time-Aware Retrieval-Augmented Generation

arXiv:2607.20560v1 Announce Type: new Abstract: Retrieval-Augmented Generation (RAG) systems retrieve and integrate external knowledge to ground large language model (LLM) outputs. However, current RA

Conflict Resolution under Degraded Surveillance in Air Corridors Using Multi-Agent Reinforcement Learning

arXiv:2607.20547v1 Announce Type: new Abstract: Safe Advanced Air Mobility operations require aircraft to maintain separation when surveillance information is noisy, delayed, incomplete, or temporaril

local-aiarxiv-cs-lg

Delivery, Not Storage: Cue-Anchored Working Memory as a Harness Property for Coding Agents

arXiv:2607.20972v1 Announce Type: new Abstract: Coding agents ship with one kind of memory: documents. Instruction files, plan artifacts, and auto-written memory directories are deliberately authored

local-aiarxiv-cs-ai

Tutorials

Focus on What Matters: Constraining Spatial-Temporal Attention via Action-Units for Noise-Resilient AQA

arXiv:2511.05611v2 Announce Type: replace Abstract: The core challenge in Action Quality Assessment (AQA) lies in extracting fine-grained motion features from redundant and complex video backgrounds.

tutorialsarxiv-cs-cv

Frontier Financial Judgement: Can agents tell what might move a stock?

arXiv:2607.20645v1 Announce Type: cross Abstract: We introduce Frontier Financial Judgement, a challenging new benchmark developed in collaboration with professional equity analysts to assess agents'

Generative Artificial Intelligence in Bioinformatics: A Systematic Review of Models, Applications, and Methodological Advances

arXiv:2511.03354v2 Announce Type: replace-cross Abstract: Generative artificial intelligence (GenAI) is transforming bioinformatics by advancing genomics, proteomics, transcriptomics, structural biolo

safetyarxiv-cs-ai

I built an open-source multi-agent SDLC harness that beats a cold Claude Code run on large repos, by learning the repo once. Real benchmarks (incl. where it loses) inside. [P]

model-releasesr-machinelearning

Built an open-source AI coding agent that was 7%–75% cheaper than a cold 'claude -p' run on 6/6 well-localized tasks across repositories up to ~82k LOC. The biggest difference: Cold agent: 6.83, 207 t

Instruct-FD: Can Your Full-Duplex Speech System Follow Turn-Taking Instructions?

arXiv:2607.20460v1 Announce Type: cross Abstract: Current full-duplex (FD) spoken dialogue systems can produce fluid interactions, yet it remains unclear whether they can adapt their turn-taking behav

IssueTrojanBench: Benchmarking AI Coding Agents Against Malicious Issue Requests

arXiv:2607.20759v1 Announce Type: cross Abstract: AI coding agents powered by LLMs are increasingly integrated into real-world software development, where they generate, edit, and execute code with au

JAXBench: Benchmarking Autonomous TPU Kernel Optimization

arXiv:2607.20466v1 Announce Type: new Abstract: Rigorous benchmarks have driven progress in autonomous GPU kernel performance optimization by establishing a shared target to hillclimb on, but no equiv

LegalCiteTrust: Benchmarking Citation Trustworthiness in Chinese Long-Form Legal Research Reports

arXiv:2607.20872v1 Announce Type: new Abstract: Long-form legal research reports increasingly rely on LLMs and agentic research systems, but their reliability depends not only on answering the task, b

More Is Not More: What Matters for Diversity in LLM Opinions?

arXiv:2607.20429v1 Announce Type: cross Abstract: Large language models are increasingly used to simulate diverse human opinions in open-ended tasks such as synthetic surveys, focus group modeling, an

researcharxiv-cs-ai

Multimodal Pretraining for Generalizable EEG Representation Learning

arXiv:2607.21384v1 Announce Type: new Abstract: Electroencephalography (EEG) models used for epilepsy are often limited to specific datasets and tasks. This limited approach can make it challenging to

Naju: A Native Discrete State-Space Model with Independent Retention and Writing for Long-Sequence Memory

arXiv:2607.21000v1 Announce Type: new Abstract: Long-sequence memory tracking places two opposing demands on a recurrent state: near-lossless retention of stored bindings over long horizons, and activ

local-aiarxiv-cs-ai

Ocular Verification for Virtual Reality

arXiv:2607.20790v1 Announce Type: new Abstract: Virtual reality (VR) headsets (e.g., Meta Quest, Apple Vision Pro) provide a seamless user experience due to their fast, frictionless interaction with t

researcharxiv-cs-cv

OPTScientist: Multi-Agent Discovery of Typed Optimizer Programs for Transformer Pretraining

arXiv:2607.20486v1 Announce Type: new Abstract: Designing optimizers for modern deep learning remains a challenging scientific problem, requiring the joint consideration of optimization geometry, stat

agentsarxiv-cs-ai

pAI-Econ-claude: A Gated Human-in-the-Loop Multi-Agent Architecture for AI-Assisted Economic Theory Development

arXiv:2607.21268v1 Announce Type: cross Abstract: In many social-science research tasks, such as economics, LLM-based agents must produce outputs for which no cheap, task-complete, machine-readable co

PISmith: Reinforcement Learning-based Red Teaming for Prompt Injection Defenses

arXiv:2603.13026v2 Announce Type: replace Abstract: Prompt injection poses serious security risks to real-world LLM applications, particularly autonomous agents. Although many defenses have been propo

QuantiBias: Benchmarking Quantization-Induced Bias in LLMs

arXiv:2607.21063v1 Announce Type: new Abstract: Almost every large language model that reaches a broad audience is quantized: trained in full precision, then compressed for efficiency. This step is as

SafeStep: AI-powered Travel Assistance for Elderly People with Frailty or Dementia

arXiv:2607.21156v1 Announce Type: new Abstract: More than a million people in the UK suffer from frailty or dementia, which severely compromise their ability to travel in urban environments. This pape

safetyarxiv-cs-ai

Self-Supervised Learning of Structured Dynamics from Videos

arXiv:2607.21576v1 Announce Type: new Abstract: Understanding motion in video is a fundamental challenge for visual learning, as frame-to-frame change entangles two sources of dynamics: camera motion

safetyarxiv-cs-cv

The Geometry of Personality: Activation Steering with Jungian Cognitive Functions

arXiv:2607.20803v1 Announce Type: cross Abstract: Activation steering enables control and interpretation of LLMs, yet existing work primarily models personality through static trait frameworks such as

The RealDefocus Benchmark for Defocus Deblurring

model-releasesarxiv-cs-cv

arXiv:2607.21078v1 Announce Type: new Abstract: Single-Image Defocus Deblurring (SIDD) aims to recover an all-in-focus image from a single defocused observation, but rigorous and reproducible evaluati

TOUR: A Trajectory-Level Unlearning Benchmark for Offline Reinforcement Learning

arXiv:2607.21111v1 Announce Type: cross Abstract: Offline Reinforcement Learning (RL) agents are trained on fixed behavioral trajectories, which makes trajectory-level deletion important when selected

TransBiolab: A Real-World Multi-View Dataset of Cluttered Transparent Biomedical Objects

model-releasesarxiv-cs-cv

arXiv:2607.21071v1 Announce Type: new Abstract: Autonomous biomedical laboratories increasingly rely on visual perception to recognize, localize, and manipulate transparent plasticware, yet high-quali

When Trivia Is Not Trivial: Everyday Knowledge Failures in Multilingual LLMs

arXiv:2607.21445v1 Announce Type: new Abstract: Quiz rooms, trivia nights, and quiz shows challenge human knowledge across a wide range of topics, from canonical facts to everyday culture. In this pap

A Unified Tokenization Framework for Pain Recognition using Heterogeneous 3D Modalities

model-releasesarxiv-cs-cv

arXiv:2607.19716v1 Announce Type: new Abstract: Pain is a complex and pervasive phenomenon affecting a large percentage of the population, and accurate assessment is essential for effective clinical m

Tutorials

Advancing Multimodal Fusion on Heterogeneous Medical Data with Hybrid Geometry Attention

arXiv:2607.19086v1 Announce Type: new Abstract: Multimodal fusion learning (MFL) has shown great potential in the medical domain, where we are faced with disparate data modalities such as imaging, cli

tutorialsarxiv-cs-cv

Agentic Real2Sim: Physics-based World Modeling with Vision-Language Agents

arXiv:2607.19190v2 Announce Type: replace-cross Abstract: Real-to-sim conversion for robotic interaction with objects remains labor-intensive because it requires more than visual reconstruction: a str

safetyarxiv-cs-ai

An Intelligent-Cloud Edge Multimodal Interaction System for Robots

arXiv:2607.14675v2 Announce Type: replace-cross Abstract: Robust human-robot interaction in complex environments requires accurate gesture perception, semantic scene understanding, and reliable task p

researcharxiv-cs-ai

Announcing OpenWorker! An open-source agent that doesn't just chat with you, but delivers finished work -- like hand you a polished document…

model-releasesandrew-ng--x

Announcing OpenWorker! An open-source agent that doesn't just chat with you, but delivers finished work -- like hand you a polished document, send a slack message, or update a calendar entry. Ask it t

ArenaRL: Scaling RL for Open-Ended Agents via Tournament-based Relative Ranking

arXiv:2601.06487v3 Announce Type: replace-cross Abstract: Reinforcement learning has substantially improved the performance of LLM agents on tasks with verifiable outcomes, but it still struggles on o

agentsarxiv-cs-ai

Applications

AugAbEx: Bridging Abstractive and Extractive Legal Summarization

arXiv:2511.12290v2 Announce Type: replace Abstract: Automatic summarization of legal judgments liberates law professionals from heavy cognitive burden due to the complexity of the language, context-se

applicationsarxiv-cs-cl

Back to Back with a Copy: A Computational Analysis of AI-Generated Visual Contemporary Art Pastiches

arXiv:2607.20127v1 Announce Type: new Abstract: The aim of this paper is twofold. First, it investigates whether newer generative models are getting better at pastiching contemporary artworks. Second,

safetyarxiv-cs-cl

Best practices for applying Amazon Bedrock Guardrails to code generation workflows

In this post, we explain how Amazon Bedrock Guardrails can be configured for code generation workflows with coding assistants to overcome these constraints. With these best practices, you can build an

safetyaws-ml-blog

Applications

BLUE: Semantics-Preserving Video Compression for Efficient Vision-Language Surveillance Analytics

arXiv:2607.19515v1 Announce Type: cross Abstract: Continuous surveillance video creates a growing storage, transmission, and inference burden for enterprise video analytics systems. While modern codec

applicationsarxiv-cs-cv

CEO-Bench: Can Agents Play the Long Game?

arXiv:2606.18543v2 Announce Type: replace Abstract: Language model agents are becoming proficient executors at isolated, short-horizon tasks such as software engineering and customer service. Yet real

Decoupled Pipeline with Proposal Reranking and Score Fusion for Positive-Unlabeled Marine Species Detection

arXiv:2607.18700v1 Announce Type: new Abstract: The FathomNetCLEF 2026 competition combines underwater object detection and fine-grained marine species classification under a positive-unlabeled evalua

local-aiarxiv-cs-cv

Detecting silent agent failures with Amazon Bedrock AgentCore optimization

Amazon Bedrock AgentCore optimization surfaces silent behavioral failures in production AI agents: the ones that pass every health check but still deliver wrong outcomes. Learn how insights discovers,

agentsaws-ml-blog

Diffusion ReRoll: Revisable Denoising for Robotic Sequential Prediction

arXiv:2607.19919v1 Announce Type: cross Abstract: We propose Diffusion ReRoll, a diffusion-based framework for robotic sequential prediction that enables revisable denoising over horizons. Existing di

DocOps: A Verifiable Benchmark for Autonomous Agents in Complex Document Operations

arXiv:2607.19865v1 Announce Type: new Abstract: As autonomous agents rapidly evolve, their ability to reliably manipulate ubiquitous digital documents has become critical for enabling general-purpose

Domain Shift in Echocardiography: Interpretable Quantification and Prediction of Cross-Dataset Left Ventricular Segmentation

arXiv:2607.19643v1 Announce Type: new Abstract: Cross-dataset generalisation remains a major barrier to clinical deployment of echocardiographic left ventricular segmentation, yet the sources of this

researcharxiv-cs-cv