Search: “evaluation”

LLM-Augmented Bayesian Optimization with Adaptive Policies for Chiplet Design Space Exploration

arXiv:2610.14612v1 Announce Type: new Abstract: Chiplet technology is a key approach for extending Moore's Law. However, its design process is constrained by two primary challenges: the scarcity of ea

safetyarxiv-cs-lg

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

MARL-BattleGrounds: A Novel JAX-native Benchmark for Heterogeneous and Competitive Multi-agent RL

DGX agent

arXiv:2610.14883v1 Announce Type: cross Abstract: We present MARL-BattleGrounds (MARL-BGs), an easy-to-use JAX-native benchmark for heterogeneous and competitive multi-agent reinforcement learning (MA

model-releasesarxiv-cs-lg

14 Oct 2026

Poisoned into Action: Context Attacks on Typed Decision Workflows

arXiv:2610.15537v1 Announce Type: cross Abstract: Typed decision interfaces restrict models to predefined actions, yet schema-valid choices can still trigger policy-violating state changes. Evaluating

safetyarxiv-cs-lg

PsyBer-Agent: A Psychology-Driven User Agent for Recommender Systems

arXiv:2610.13989v1 Announce Type: cross Abstract: Evaluating recommender systems before deployment requires simulators that remain realistic over long, closed-loop interaction cycles. Current LLM-base

Refined Grid Sampling for Efficient Implicit Neural Representations of Large-Scale Scientific Data

arXiv:2610.14205v1 Announce Type: new Abstract: Implicit neural representations (INRs) offer a powerful framework for modeling high-dimensional physical fields, but their training remains inefficient

Safety-Oriented Trajectory Ranking from BEV Perception Outputs

arXiv:2610.14066v1 Announce Type: cross Abstract: Bird's-Eye-View (BEV) representations are increasingly used as compact inputs for autonomous driving perception and planning. However, lightweight and

safetyarxiv-cs-cv

scHFBiMamba: a bidirectional Mamba autoencoder for cardiac disease-state classification from single-nucleus RNA-seq data

arXiv:2610.15117v1 Announce Type: new Abstract: To evaluate whether a compact bidirectional state-space autoencoder improves joint classification of species, cell type, and disease state from single-n

TagBO: LLM-Driven Task-Aware Graph Bayesian Optimization for Scientific Discovery

arXiv:2610.14626v1 Announce Type: new Abstract: Mixed-variable optimization in scientific discovery often requires exploring large combinatorial spaces under costly experimental evaluations, making sa

TRIG: Toward Measuring Texture Realism in Image Generation

arXiv:2610.15164v1 Announce Type: new Abstract: This paper studies the backbone used by metrics for evaluating the quality of generated images. Although generative models have advanced rapidly, some o

safetyarxiv-cs-cv

Unsupervised Data Drift Monitoring for Deployed Deepfake Speech Detectors

arXiv:2610.16826v1 Announce Type: cross Abstract: Deepfake speech detectors are typically evaluated as binary classifiers on fixed benchmarks, but deployed systems face evolving attacks, codecs, speak

researcharxiv-cs-cv

Tutorials

Use Of Convolutional Neural Networks In Detection Of Branch Dieback Disease In Pequi Trees

arXiv:2610.15049v1 Announce Type: new Abstract: This study aims to evaluate and compare convolutional neural network (CNN) architectures, specifically AlexNet, Inception, LeNet, VGG16, and MobileNet,

tutorialsarxiv-cs-cv

Tutorials

What DNA Foundation Models Learn Beyond Sequence Composition

arXiv:2610.15711v1 Announce Type: new Abstract: Do genomic foundation models (FMs) provide predictive information beyond simple sequence composition? We evaluate frozen representations from five DNA F

tutorialsarxiv-cs-lg

Cross-cloud conversational analytics made simple with open standards

The reality for most organizations is that data lives everywhere. As engineering teams expand and adopt specialized platforms for their workloads, enterprise data naturally disperses across multiple c

safetygoogle-cloud-ai

13 Oct 2026

Gemini 4 Argon available on Google Cloud, built to help solve your business’ hardest problems

model-releasesgoogle-cloud-ai

Last week we announced that Gemini Enterprise has evolved to make it even simpler to get work done, and introduced the Gemini agent, your new universal agent for work. It's always on and ready to answ

13 Oct 2026

A Baseline Assessment for the Agentic Web Technology Stack

arXiv:2610.13703v1 Announce Type: new Abstract: The fast pace in which protocols, frameworks, and technologies are being proposed for LLM-based agents results in a complex scenario for anyone aiming t

Accelerating Linear Bandits with Persistent Candidate Sets

arXiv:2610.13407v1 Announce Type: new Abstract: Exhaustive optimistic scoring can make action selection costly in linear bandits with a large fixed set of K actions, requiring O(K) score evaluations p

AI Models Can Provably Hide Arbitrary Capabilities

arXiv:2610.13928v1 Announce Type: new Abstract: Capability evaluations aim to surface dangerous model behaviors before deployment, but their reliability depends on the assumption that hidden capabilit

Bridging Human and Model Attention: Explainability Analysis of CNN, Mamba, and ViT Architectures with Gaze-Based Validation

arXiv:2610.12703v1 Announce Type: cross Abstract: High-risk applications of deep learning require precise characterization of model decision-making. We conduct a systematic explainability evaluation a

safetyarxiv-cs-lg

Co-Modifying Application Code and Deployment Infrastructure: A Causal Study of AI Agent Pull Requests

arXiv:2610.12849v1 Announce Type: cross Abstract: Current software engineering agents are evaluated predominantly inside isolated sandboxes against synchronous local tests. However, deploying features

agentsarxiv-cs-ai

Critical Mass: Structure-Disjoint Splits Drift With the Libraries They Are Built On

arXiv:2610.12694v1 Announce Type: new Abstract: Molecular machine learning evaluates out-of-distribution performance with structure disjoint splits, a construction rule fixed once at the scale its lib

DataCupid: Stable Matching Repair for Training Data Acquisition with Delayed Feedback

arXiv:2610.12960v1 Announce Type: new Abstract: Matching repair after utility changes can reuse valid observations, but new evaluations must respect matching capacity and may return out of order. We s

DRYLAB-Bench: Benchmarking the Biosecurity Risk of Agentic LLM Mutation Design

arXiv:2610.13116v1 Announce Type: new Abstract: As large language models (LLMs) acquire advanced biological design capabilities, they offer substantial research benefits but simultaneously introduce s

GeoFitBench-LLM: A Benchmark for Assessing Large Language Models' Capabilities in Geospatial Data Requirements Elicitation for Geo-Analytic Tasks

arXiv:2610.12669v1 Announce Type: cross Abstract: Large language models are becoming central to GeoAI systems, yet their evaluation rarely considers the elicitation of geospatial data requirements. Th

How Reliable are AMR Parser Outputs? Uncertainty Estimation for AMR Parsing

arXiv:2610.13236v1 Announce Type: cross Abstract: Recent advances have significantly improved the performance of Abstract Meaning Representation (AMR) parsers, yet determining when their outputs can b

researcharxiv-cs-ai

Introducing DigitalOcean Agent Data Services: The Data Foundation for Agents

Agents are only as capable as the context they can reach. An agent with reliable access to the right information makes decisions grounded in fact; one without it guesses. And the work agents produce i

agentsdigitalocean

Mind the Spatial Gap: A Benchmark for Geospatial Reasoning and Constrained Spatial Editing

arXiv:2610.12881v1 Announce Type: new Abstract: Understanding and manipulating spatial relations is fundamental to AI systems that reason about physical environments and construct grounded world model

OccludeBench: Can VLMs Find What's Missing?

model-releasesarxiv-cs-cl

arXiv:2610.13790v1 Announce Type: cross Abstract: How well can vision-language models (VLMs) reason about what is missing from an image? We study this question with OccludeBench, a manually curated be

Personalization Without Stereotyping: Do Speech LLMs Know When to Turn a Deaf Ear?

model-releasesarxiv-cs-cl

arXiv:2610.13550v1 Announce Type: new Abstract: Recent speech LLMs can leverage rich paralinguistic cues, enabling more complex and personalized interactions. However, little work investigates how thi

SWE-Duel: Towards a Self-Scaling Arena for Coding Agents

model-releasesarxiv-cs-cl

arXiv:2610.12569v1 Announce Type: new Abstract: Static coding benchmarks are fixed at release. Over time their tasks leak into training data, and stronger agents eventually saturate them. We argue tha

Towards Easier-to-Tune Training Algorithms: New Results from AlgoPerf

arXiv:2610.13036v1 Announce Type: new Abstract: Evaluating deep learning optimizer is challenging because confounding factors such as hyperparameter tuning and optimizer-model co-design often obscure

What Do Audio-Language Judges of Generated Music Listen For? Four Probes with Controls from Expert Ratings

arXiv:2610.13247v1 Announce Type: cross Abstract: Open audio-language models are used as judges of generated music: given a clip and its text prompt, the model rates the clip, and the judge is validat

safetyarxiv-cs-cl

Introducing Multi-Agent Self-Supervision (MASS): Recursive self-improvement through collective intelligence Blog: https://pub.sakana.ai/mass…

Introducing Multi-Agent Self-Supervision (MASS): Recursive self-improvement through collective intelligence Blog: https://pub.sakana.ai/mass/ As AI systems tackle increasingly open-ended research prob

safetydavid-ha--x

11 Oct 2026

AdaptEvo: Adaptive Agent Learning with Evolving Supervision

arXiv:2610.11354v1 Announce Type: new Abstract: Rule-governed contextual decision tasks require models to apply specified rules to case-specific context and evidence. Written rules can leave gaps in d

safetyarxiv-cs-cl

Bernoulli Flow Models: Self-Consistent Generative Modeling for Binary Data

arXiv:2610.11362v1 Announce Type: cross Abstract: Binary diffusion models typically require a large number of function evaluations (NFEs) to generate high-quality samples, making practical inference c

researcharxiv-cs-cv

DataSense-Bench: The First Step Toward an AI Scientist

arXiv:2610.12190v1 Announce Type: new Abstract: As claims about recursive self-improvement (RSI) and artificial general intelligence (AGI) proliferate, we ask a simple question: do frontier AI models

Design Creativity Bench: Measuring creativity in LLM-Generated UI

arXiv:2610.11539v1 Announce Type: cross Abstract: As leading LLMs improve on capability evaluations, their limitations in producing creative outputs on design tasks remain insufficiently characterised

Distillation for Incrimination and Distillation for Capabilities

arXiv:2610.11012v1 Announce Type: new Abstract: Powerful misaligned AI models might recognize alignment evaluations and strategically behave well on them, rendering direct audits uninformative. Howeve

safetyarxiv-cs-ai

Tutorials

Exploiting Gradients in Bayesian Inference of Expensive Simulators

arXiv:2610.12076v1 Announce Type: new Abstract: Simulators based on differential equations are ubiquitous in science and engineering. They are often used in simulation-based inference to evaluate the

tutorialsarxiv-cs-lg

Hardware

Language Models as AI Research World Models

arXiv:2610.12235v1 Announce Type: new Abstract: AI research agents automate the cycle of proposing, implementing, and evaluating experiments, opening a path toward recursive self-improvement. Yet thei

hardwarearxiv-cs-cl

Marformer: A Transformer for Predicting Missing Data Distributions

arXiv:2610.12379v1 Announce Type: new Abstract: Real decisions are made under incomplete information. If we observe only some of the random variables we need, we can predict the others. The extbf{cond

MIMESIS: Learning User Simulators as Training Environments for Interactive Agents

arXiv:2610.09484v2 Announce Type: replace Abstract: Training and evaluating interactive language agents typically requires rich user interactions, yet collecting human feedback is expensive and diffic

MindFlow: Mind Supernet Powered Thinking Flows for Research Idea Innovation

arXiv:2610.11966v1 Announce Type: new Abstract: Research idea innovation is a fundamental engine of scientific progress, yet it remains difficult to generate and evaluate in a scalable and controllabl

agentsarxiv-cs-ai

OpenProblemBench: Benchmarking AI on Open Problems in the Foundational Theoretical Sciences

arXiv:2610.11118v1 Announce Type: new Abstract: The next frontier for artificial general intelligence is tackling unresolved scientific problems, calling for benchmarks that assess progress beyond est

Prompts versus Rules: Auditing and Controlling Speech Naturalness Behaviors in Voice User Simulators

arXiv:2610.11015v1 Announce Type: new Abstract: As voice agents gain more popularity commercially, the user simulators used to evaluate the deployed agents are also being developed to include more rea

agentsarxiv-cs-cl

RAG-Stress: Probing the Limits of Evidence Reliance in Retrieval-Augmented Generation

arXiv:2610.11183v1 Announce Type: cross Abstract: Following retrieved evidence does not guarantee factual correctness: misleading evidence can induce a model to replace an answer it previously gave co

researcharxiv-cs-ai

READ-Bench: Benchmarking Historical Instance Retrieval for Time-Series Diagnosis

arXiv:2609.32123v2 Announce Type: replace Abstract: Time-series diagnostic systems rarely rely on retrieving relevant historical cases, and when they do, retrieval is evaluated only indirectly through

Reasoning-Informed Visual Editing

model-releasesarxiv-cs-cv

arXiv:2610.12343v1 Announce Type: new Abstract: Large Multi-modality Models (LMMs) have made significant progress in visual understanding and generation, but still face challenges in visual editing, p

SAVU-BENCH: A Real-World Benchmark for Spatial Audio-Visual Understanding

arXiv:2610.10624v1 Announce Type: cross Abstract: Spatial audio-visual understanding requires models to recognize not only what is present, but also where events occur and how they relate across modal