Search: “evaluation”

Fewer Steps, Better Actions: Rethinking Flow-Matching Inference for VLA Policies

arXiv:2609.21216v1 Announce Type: cross Abstract: Vision-language-action (VLA) policies based on flow matching generate action chunks through repeated evaluations of an action expert. Increasing the n

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Agents

From Discharge Notes to Patient Understanding: Persona-Grounded, Open-Ended Simulation of LLMs as Discharge Educators

DGX agent

arXiv:2609.20827v1 Announce Type: new Abstract: Hospital discharge education is an interactive teaching task: a clinician adapts a discharge plan to a patient's literacy, recall, and personality. Exis

agentsarxiv-cs-cl

21 Sept 2026

LEGIT: Credentialing Protocol for Trustworthy AI Agent Marketplaces

arXiv:2609.21325v1 Announce Type: new Abstract: Agentic marketplaces are emerging where AI agents with varying capabilities autonomously complete specialized tasks for buyers. A major challenge of suc

mu^2-Bench: A Multilingual Machine Unlearning Benchmark

arXiv:2609.20945v1 Announce Type: new Abstract: Undesired information such as harmful content and private data propagates through Multilingual Large Language Models (LLMs) via direct training and indi

Offline Multimodal Large Language Models for Decision Support in Air Operations

arXiv:2609.21390v1 Announce Type: new Abstract: Air operations rely on complex rules, established procedures, and time-critical analysis under limited connectivity and strict security constraints. In

Omni Demand Understanding: A Benchmark for Contextual User-Intent Inference in Multimodal Interaction

arXiv:2609.21392v1 Announce Type: new Abstract: Natural audio-visual interaction is emerging as an important interface for AI assistants, allowing users to communicate through speech and vision rather

PhysioBench: A Unified Benchmark for Physiological Signal Question Answering

arXiv:2609.20836v1 Announce Type: new Abstract: Physiological signals support diverse clinical and monitoring tasks, yet existing physiological signal foundation models typically require task-specific

Local Ai

Refining Ground Truth Poses in Autonomous Driving Datasets via Neural Rendering

arXiv:2504.15776v2 Announce Type: replace Abstract: Public autonomous driving datasets underpin the training and benchmarking of perception, mapping, and localization algorithms, yet residual inaccura

local-aiarxiv-cs-cv

Value-Sensitive Delegation in Everyday AI Agent Use: Evidence from OpenClaw

arXiv:2609.22067v1 Announce Type: cross Abstract: Users increasingly delegate work to autonomous AI agents, yet evaluations typically measure task completion rather than the values users prioritize. U

agentsarxiv-cs-ai

VIRGA: Virtual-Agent-Intermediated Riemannian Geometry for Active-Sensing Air-Ground Coordination

arXiv:2609.21883v1 Announce Type: new Abstract: Air-ground autonomy becomes harder when the unmanned aerial vehicle (UAV) must remain observable by a gimbal light detection and ranging (LiDAR) mounted

safetyarxiv-cs-ro

VLA-Scope: Shift-Aware Failure Prediction for Vision-Language-Action Models

arXiv:2609.21246v1 Announce Type: cross Abstract: Vision-language-action (VLA) models map visual observations and natural-language instructions to robotic actions, but distribution shifts can compromi

Voice-Light: A Full-Duplex Cascaded Voice Agent with Causal Turn-Taking and Speculative Generation

arXiv:2609.20995v1 Announce Type: cross Abstract: Natural spoken interaction requires more than streaming ASR, language generation, and speech synthesis: a system must react to overlap without canceli

safetyarxiv-cs-cl

We tested Jev against LLM judges on accuracy, repeatability, latency, and cost to see whether System One models could offer a new approach t…

We tested Jev against LLM judges on accuracy, repeatability, latency, and cost to see whether System One models could offer a new approach to agent evaluation. https://x.com/i/article/2101448785255907

agentsharrison-chase--x

19 Sept 2026

Architectural Design, Not Only Model Intelligence, Governs Multi-Agent LLM Performance

arXiv:2602.03128v2 Announce Type: replace Abstract: Multi-agent LLM frameworks are data-intensive systems that govern how agents orchestrate tasks, manage state, and coordinate decisions. These archit

BioPhys-Bridge: A Benchmark for Interdisciplinary Scientific Reasoning in Physics-Grounded Biological Research

arXiv:2609.19180v1 Announce Type: new Abstract: Language models face unique challenges in analyzing interdisciplinary scientific research literature. In biophysics research, faithful answers require g

Can 4D Foundation Models Remember?

arXiv:2609.20819v1 Announce Type: new Abstract: Perceiving and remembering the visual world is fundamental to navigating and interacting with our environment. Current 4D foundation models, such as cam

researcharxiv-cs-cv

Can Vision-Language Models Judge Olympic Diving? From Reasoning to Scores in Zero-Shot Action Quality Assessment

arXiv:2609.19354v1 Announce Type: cross Abstract: Automated action quality assessment (AQA) in Olympic sports remains a challenging task due to the complexity of human motion and the subjectivity inhe

CARE-VI: Conservative Adaptive Reliability Estimation for Value Improvement in Off-Policy Actor-Critic Learning

arXiv:2609.20098v1 Announce Type: new Abstract: Reliable temporal-difference targets are central to off-policy actor-critic learning. Direct value improvement refines the next-state target with altern

safetyarxiv-cs-lg

CliniCIRCA: A Modular LLM Framework for Constructing Longitudinal Mental Health Patient Journeys from Raw EHR Narratives

arXiv:2609.19585v1 Announce Type: cross Abstract: In mental health care, reasoning over patient journeys is a key task for clinicians. Yet these journeys, encompassing a longitudinal progression of bi

CounselReflect: Opportunities and Challenges for Designing Tools to Support Self-Reflection on Mental Health and Well-Being Conversations with AI

arXiv:2603.29429v2 Announce Type: replace Abstract: AI is increasingly used for mental health and well-being support, creating an urgent need for safer engagement, while design, evaluation, and govern

researcharxiv-cs-cl

DART: Distillation-Aware Reparameterization for Training-Free LoRA Reuse in Few-Step Video Diffusion Models

arXiv:2609.20051v1 Announce Type: new Abstract: Step distillation reduces the cost of video generation, but reusing a LoRA trained for a longer trajectory can alter its functional effect or degrade ta

DeliveryGym: An RL Environment for Long-Horizon Embodied Agent Planning with Adaptive Curriculum

arXiv:2609.19801v1 Announce Type: new Abstract: Executable environments enable LLM agents to learn from the consequences of their actions. For embodied agents, those consequences extend beyond whether

safetyarxiv-cs-lg

greCAPTCHA: Assessing Understanding as Evidence of Research Authorship Under Generative AI

arXiv:2609.20481v1 Announce Type: cross Abstract: Conferences, journals, funders, schools, and universities are struggling with a surge of potentially AI-generated submissions from ostensibly human au

Grounded Product Understanding in Livestream Videos

model-releasesarxiv-cs-cv

arXiv:2609.20508v1 Announce Type: new Abstract: E-commerce livestreams have emerged as an important channel for presenting products to online consumers, containing multiple products whose information

Harm Laundering in GPT Models: Evidence That Gender Discrimination Is Transformed Rather Than Reduced Across Safety-Trained Generations

arXiv:2609.20779v1 Announce Type: cross Abstract: Safety evaluations for large language models rely on surface-form classifiers that report declining harm scores across model generations. We provide e

Improving Cross-embodiment Transfer in Latent Action Models with Action-Similarity Supervision

arXiv:2609.19846v1 Announce Type: cross Abstract: As generalist robot policies gain vision and language from web-scale pretraining, demonstrations remain costly to collect and tied to the robot that r

Is HF starting to move against abliterated models?

model-releasesr-localllama

Baseten launched a new safety infrastructure standard alongside its Base Labs research arm on Wednesday, partnering with Hugging Face and Goodfire AI to build safety evaluation and monitoring infrastr

Kinematics-Grounded Agentic AI for Robotic Additive Manufacturing Process Planning

arXiv:2609.19347v1 Announce Type: cross Abstract: Robotic additive manufacturing (AM) extends material-extrusion printing beyond gantry kinematics but makes process planning robot-dependent. A slicer-

agentsarxiv-cs-ai

KoUniTalk: A Lightweight Articulation-Centered Korean-English 3D Talking Face Benchmark

model-releasesarxiv-cs-cv

arXiv:2609.19840v1 Announce Type: new Abstract: High-quality 3D talking face datasets remain largely English- centric, and Korean 3D facial motion data are difficult to combine with standard English b

Learning Slope-Adaptive Whole-Body Locomotion for Humanoid Robots in Roofing Construction

arXiv:2609.20558v1 Announce Type: new Abstract: Roofing requires workers to coordinate locomotion, balance, and work-related body motions on pitched surfaces, creating a challenging application for hu

safetyarxiv-cs-ro

LinePilot Digitizer: Line-Plot Recovery with Manual and Automatic Calibration

arXiv:2609.19377v1 Announce Type: cross Abstract: Recovering numerical series from line plots requires accurate axis calibration and reliable curve extraction. We present LinePilot Digitizer (LinePilo

Tutorials

Perspective of Software Engineering Researchers on Machine Learning Practices Regarding Research, Review, and Education

arXiv:2411.19304v2 Announce Type: replace-cross Abstract: Context: Machine Learning (ML) significantly impacts Software Engineering (SE), but studies mainly focus on practitioners, neglecting research

tutorialsarxiv-cs-lg

RAFT: A Stateful Retrieval-Augmented Framework for Troubleshooting Agents

arXiv:2609.20754v1 Announce Type: new Abstract: Effective troubleshooting agents in enterprise customer support depend on retrieving actionable guidance from similar historical cases, yet existing ret

Hardware

S4R: Scaling for Rigid-Body Interpenetration Resolution

arXiv:2609.20524v1 Announce Type: cross Abstract: Rigid-body interpenetration frequently occurs in procedurally assembled and generated scenes and must be removed before downstream applications such a

hardwarearxiv-cs-ro

Small Enough to Know Everything: The Fully-Enumerable Transformer as an Instrument for the Science of Delayed Generalization

arXiv:2609.20166v1 Announce Type: new Abstract: Tiny transformers trained on fully-enumerable tasks occupy an unusual position in the study of grokking: every input can be evaluated, every generalizat

researcharxiv-cs-lg

To Memories and Beyond: From Remembering to Knowing You across Long-Term Multimodal Personal Archives

arXiv:2609.19167v1 Announce Type: new Abstract: As AI systems evolve into personalized digital companions, a central capability is reasoning over a user's long-term personal history: not merely storin

UnifiedPlayers: Enhance Tool-Integrated Reasoning in Agentic Reinforcement Learning

arXiv:2609.20089v1 Announce Type: new Abstract: Self-evolving methods reduce the need for human-annotated trajectories by allowing tool-using agents to generate their own training data. Yet existing m

agentsarxiv-cs-ai

Accelerating Diffusion Sampling via Speculative Draft Trees

arXiv:2609.17691v1 Announce Type: cross Abstract: Speculative sampling accelerates diffusion model generation by drafting inexpensive candidate states and correcting them under a coupling that preserv

ActionPiece: Rethinking Action Tokenization for Autoregressive Vision-Language-Action Models

arXiv:2609.18487v1 Announce Type: cross Abstract: Action tokenizers play a central role in autoregressive vision-language-action (VLA) models, determining both the targets for policy training and the

AI and Human Approaches to Mathematical Problem Solving

arXiv:2609.17779v1 Announce Type: cross Abstract: AI systems have begun to report solutions, disproofs, and substantive advances on long-standing mathematical problems, raising questions about whether

Tutorials

Anomaly Detection in General Ledger Data: Results from a Hybrid Approach

arXiv:2609.18228v1 Announce Type: new Abstract: Journal Entry Tests (JETs) are a mandatory part of annual audits to evaluate and assess both highrisk audit areas and potential material misstatements.

tutorialsarxiv-cs-lg

Behavioral Fingerprinting and Navigation Prediction in Web Browsing

arXiv:2609.18273v1 Announce Type: new Abstract: Web browsing often appears ephemeral: users visit a few websites, complete a task, and move on. However, even short fragments of browsing activity can c

researcharxiv-cs-lg

Benchmarking Large Language Models for Biomedical Relation Extraction

arXiv:2609.19071v1 Announce Type: new Abstract: Extracting SNP-phenotype associations from biomedical literature is vital but challenging. We benchmarked diverse NLP models, including MLMs, hybrid arc

BOOM: Benchmarking Out-Of-distribution Molecular Property Predictions of Machine Learning Models

arXiv:2505.01912v3 Announce Type: replace-cross Abstract: Data-driven molecular discovery leverages artificial intelligence/machine learning (AI/ML) and generative modeling to filter and design novel

Clueing up LLMs with Tool-Augmented Deductive Reasoning

arXiv:2609.18736v1 Announce Type: new Abstract: Despite recent advances in large language models (LLMs), performing logically consistent deductive reasoning over extended interactions remains challeng

Collective Loss of Control in LLM Agent Systems: An Epidemic Account of Mutation, Contagion, and Recovery

arXiv:2609.18460v1 Announce Type: new Abstract: How does a multi-agent system evolve from a local deviation into collective loss of control? We propose an epidemic explanation organized around acciden

Tutorials

Come hang out with AI engineers, tech leads, and the Fireworks team during AI Engineer 2026. Want to leave with hands-on experience fine-tun…

Come hang out with AI engineers, tech leads, and the Fireworks team during AI Engineer 2026. Want to leave with hands-on experience fine-tuning, evaluating, and deploying a model on Fireworks? Bring y

tutorialsfireworks-ai--x

Compiled Agency: Frontier General-Purpose Coding Agents Build Winning Game Players from Bare Interaction - from Flappy Bird to StarCraft II and Civilization

arXiv:2609.18996v1 Announce Type: new Abstract: LLM agents have repeatedly struggled to convert knowledge of a game into competent play, even when researchers build the agent around the model - supply