Search: “evaluation”

TriCalRAG: A Three-Strategy, Retrieval-Augmented Benchmark for On-Premise LLM-Based Root Cause Analysis in AIOps

arXiv:2609.14762v1 Announce Type: cross Abstract: Cloud-hosted large language models (LLMs) are increasingly used for root cause analysis (RCA) in AIOps pipelines, but they introduce data privacy risk

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

Two-Stage Personalized Gait Phase Estimation in Stroke Survivors During Exoskeleton-Assisted Walking: An Offline Feasibility Study

DGX agent

arXiv:2609.14984v1 Announce Type: new Abstract: This study evaluated personalized gait phase estimation for stroke survivors using functional inertial measurement unit (IMU) alignment and two-stage se

safetyarxiv-cs-ro

15 Sept 2026

What Makes a 3D Scene Editable? A Factorized Benchmark of Fidelity, Locality, Consistency, and Preservation

arXiv:2609.14899v1 Announce Type: new Abstract: Neural 3D scene editing is often evaluated by semantic alignment alone, although a convincing result may alter unrelated content or become inconsistent

When does a scaling result justify a different allocation? A critical review of resource-allocation evidence for AI systems

arXiv:2609.14500v1 Announce Type: new Abstract: AI scaling studies increasingly evaluate systems that combine a pretrained model with retrieval, search, verification, tools, and interaction. Yet a hig

Hardware

When Faster VLA Deployment Changes Closed-Loop Behavior: Task Success-Latency Analysis of SmolVLA Across PyTorch and ONNX Variants

arXiv:2609.14146v1 Announce Type: cross Abstract: Vision-language-action (VLA) deployment can reduce inference latency while changing closed-loop task behavior. We evaluate HuggingFaceVLA/smolvla_libe

hardwarearxiv-cs-lg

Confidence-Gated Transductive Test Generation for Code Reranking

arXiv:2609.12489v1 Announce Type: cross Abstract: Test case synthesis is crucial for evaluating and ranking programs generated by large language models (LLMs). However, constructing high-quality test

researcharxiv-cs-cl

Context-Aware Causal Gaze Forecasting for Human-Vehicle Interaction During In-Cabin Tracking Dropouts

arXiv:2609.12374v1 Announce Type: new Abstract: Dashboard-mounted gaze trackers often lose sight of the driver's eyes during large head rotations, including shoulder checks, mirror glances, and inters

researcharxiv-cs-cv

Agents

Continue, Adapt, or Yield: In-Turn Adaptation to Overlapping Speech in Full-Duplex Agents

arXiv:2609.13117v1 Announce Type: new Abstract: Full-duplex evaluation often emphasizes whether an agent keeps speaking or stops. That binary cannot express a third response humans use routinely: cont

agentsarxiv-cs-cl

GRACE-DS: a Guarded Reward-guided Agent Correction Environment in Data Science

arXiv:2606.16000v3 Announce Type: replace Abstract: We introduce GRACE-DS, a Guarded Reward-guided Agent Correction Environment in Data Science for pre-deployment evaluation of LLM-powered AutoML agen

safetyarxiv-cs-cl

PRISMA-LLM: An Empirical Reporting Framework for AI-Assisted Systematic Reviews

arXiv:2609.11559v1 Announce Type: cross Abstract: Large language models (LLMs) and AI-enabled software increasingly participate in systematic-review decisions, yet the information needed to audit thes

researcharxiv-cs-cl

ProactiveBench: Can Streaming Video Models Really Interact Like Humans?

arXiv:2609.12658v1 Announce Type: new Abstract: Streaming video understanding requires models to process continuous multimodal input while maintaining temporal context. Existing evaluations are predom

researcharxiv-cs-lg

Revisiting Multi-Object Tracking Baselines: Hyperparameter Optimization with Multi-Fidelity Greedy Coordinate Search

arXiv:2609.12261v1 Announce Type: new Abstract: Multi-object tracking (MOT) is dominated by the tracking-by-detection paradigm, whose methods typically rely on a small set of hyperparameters that are

researcharxiv-cs-cv

Robust Policy Optimization via Adversarial Importance Sampling

arXiv:2609.13044v1 Announce Type: new Abstract: Significant progress has been made in safeguarding deep reinforcement learning (DRL) policies against input perturbations. Developing robust DRL involve

safetyarxiv-cs-lg

Amulet: a Python Library for Assessing Interactions Among ML Defenses and Risks

arXiv:2509.12386v3 Announce Type: replace-cross Abstract: Machine learning (ML) models are susceptible to various risks to security, privacy, and fairness. Most defenses are designed to protect agains

safetyarxiv-cs-ai

12 Sept 2026

Local Ai

Can Edge-Deployable Vision-Language Models Identify Species?

arXiv:2609.11916v1 Announce Type: new Abstract: Camera traps often run in the field on edge hardware with limited or no connectivity, making small, locally-deployable vision-language models (VLMs) --

local-aiarxiv-cs-ai

12 Sept 2026

When Validation Stops Learning: Auditing Update Admission for Continual Embodied Agents

arXiv:2609.10873v1 Announce Type: new Abstract: Independent evaluation can reject harmful policy updates yet also prevent useful continual learning. We argue that update admission must be assessed thr

safetyarxiv-cs-ai

12 Sept 2026

E-CONAN (Entailment, CONtradition And Neutral) Benchmarks: Arabic Textual Entailment and Natural Inference Datasets

arXiv:2609.11334v1 Announce Type: new Abstract: Natural Language Inference processes pairs of sentences to extract their semantic relations. NLI has been a hot research topic, integrated as a main com

LexAgentHallu: A Hierarchical Benchmark for Profiling Hallucinations in Legal Agents

arXiv:2609.09754v1 Announce Type: new Abstract: As large language models are increasingly deployed as tool-augmented legal agents, they introduce agentic hallucinations where tool-call and reasoning e

MedGEN-Bench: A Contextually Entangled Benchmark for Open-ended Multimodal Medical Generation

arXiv:2511.13135v3 Announce Type: replace Abstract: Medical vision-language models (VLMs) are increasingly expected to support clinical workflows through diagnostic text and relevant medical images. H

Nuha-Speech: Building General-Purpose Arabic Speech-LLMs

arXiv:2609.11892v1 Announce Type: new Abstract: As Speech Large Language Models (speech-LLMs) become increasingly multilingual, Arabic remains significantly underrepresented, highlighting the need for

TransClean: A Benchmark for Detecting and Extracting Clean Translations from Large Language Model Outputs

arXiv:2609.11399v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used for machine translation, yet their outputs often contain additional text beyond the translation itsel

A Unified Policy Architecture (UPA): The Governance Kernel for Enterprise AI Operating Systems

arXiv:2609.06543v1 Announce Type: new Abstract: Enterprise AI is evolving into an Enterprise Operating System where autonomous AI agents can plan, reason, use memory, invoke tools, execute workflows,

safetyarxiv-cs-ai

Applications

Aegix Pulse: A Traceable Three-Stage Architecture for Personalized Content Generation and Context-Preserving Revision

arXiv:2609.07672v1 Announce Type: new Abstract: Production content-generation systems must integrate a user's immediate task, long-term brand identity, historical evidence, and revision feedback. We p

applicationsarxiv-cs-ai

API Benchmark Scores Do Not Reliably Transfer to Chatbot Interfaces

arXiv:2609.08861v1 Announce Type: new Abstract: Benchmark scores are a central currency in model releases: they inform purchasing decisions, shape public trust, and influence policy. Yet, a key assump

CIVI: A Framework for Diagnosing Search Agent Failures in Civic Information

arXiv:2609.08094v1 Announce Type: new Abstract: Large Language Models are increasingly deployed in public-sector settings, where incorrect guidance can cause irreversible harm. We introduce CIVI, the

Local Ai

Encrypt What Matters: When Selective Homomorphic Inference Is Efficient

arXiv:2609.09357v1 Announce Type: cross Abstract: Fully homomorphic encryption (FHE) enables inference on private data without revealing it to the server, but evaluating an entire input under FHE is e

local-aiarxiv-cs-lg

From Representation to Enactment: The ABC Framework of the Translating Mind

arXiv:2511.16811v2 Announce Type: replace Abstract: Building on the third-wave Extended Mind (EM) theory and radical enactivism, this article suggests an alternative to representation-based models of

researcharxiv-cs-cl

Agents

GoAnt: Quality-Diversity Multi-Agent Search for Alpha Factor Discovery in Market Microstructure Data

arXiv:2609.08719v1 Announce Type: new Abstract: Automated alpha factor discovery searches symbolic trading signals from price-volume panels and order-book data under a fixed evaluation budget. Existin

agentsarxiv-cs-ai

Mind the Gap: Exposing LLM Translation Blind Spots Using the AlphaMWE Multilingual Parallel Corpus

arXiv:2609.06634v1 Announce Type: cross Abstract: LLMs' performance on machine translation (MT) tasks is often dependent on the data availability in the specific domains and language pairs that they a

researcharxiv-cs-ai

Performance of Clinical AI System and Physicians and Frontier Language Models in primary care diagnostics

arXiv:2609.09070v1 Announce Type: cross Abstract: Clinical AI evaluation should encompass diagnosis and management after adaptive information gathering. We compared Doctorina, eight physicians and fou

RAP: Research Attention Prediction Reveals Target-Conditioned Evidence Acquisition Biases

arXiv:2609.10092v1 Announce Type: cross Abstract: Large language models (LLMs) increasingly act as research agents, yet their ability to track shifts in research attention is difficult to evaluate bec

RAPID: Reliability-Aware Pair Importance Distillation

arXiv:2609.05481v1 Announce Type: new Abstract: Inter example relational distillation transfers a teacher's representation geometry by matching relations among examples within a mini batch. Computing

researcharxiv-cs-ai

SciLitBench: Benchmark and Design Principles for LLM-Powered Systematic Literature Reviews

arXiv:2609.05505v1 Announce Type: new Abstract: Systematic reviews require sustained human judgment across thousands of records, yet existing evaluations of large language models (LLMs) typically exam

Scoring Without the Engine: Validating a Deterministic, Manipulation-Resistant Content Score for Generative Engines, End to End

arXiv:2609.07559v1 Announce Type: new Abstract: How do you validate a cheap, deterministic proxy for an oracle that is expensive, rate-limited, and non-stationary? We present a protocol built on adver

VST: Verifiable Structured Transport for Auditable Agent-to-Agent Alpha Discovery

arXiv:2609.07065v1 Announce Type: new Abstract: Agent-to-agent (A2A) alpha discovery is slowed by repeated feedback cycles between mining and evaluation agents, whose hand-offs, in contemporary LLM mu

What Does MMLU Actually Measure? A Psychometric Audit of Difficulty Structure in Aggregate Benchmark Scores

arXiv:2609.09372v1 Announce Type: cross Abstract: Although MMLU is widely adopted as a benchmark for calibrating general AI capabilities, we psychometrically demonstrate that its aggregate score prima

Tutorials

AlphaRJM: Reward-Jump Memory for Stochastic Return-Guided Alpha Discovery

arXiv:2609.08581v1 Announce Type: new Abstract: Formulaic alpha discovery is a pool-dependent symbolic search problem in which informative feedback is observed primarily when a complete expression is

tutorialsarxiv-cs-lg

AV-SafetyBench: A Safety Benchmark for Text-to-Audio-Video Generation

arXiv:2609.06991v1 Announce Type: new Abstract: Recent text-to-audio-video (T2AV) models jointly generate video, speech, sound effects, and ambience from a single text prompt. This capability poses ne

GeoHeight-Bench: Towards Height-Aware Multimodal Reasoning in Remote Sensing

arXiv:2603.25565v2 Announce Type: replace Abstract: Current Large Multimodal Models (LMMs) in Earth observation are predominantly evaluated on planar optical tasks and often neglect the vertical dimen

Limitations of Automated Simulatability: LLM Simulators Can Bypass Explanations

arXiv:2609.08585v1 Announce Type: cross Abstract: Simulatability is an evaluation protocol for explanations that quantifies their usefulness by how well they help a user predict a task model's outputs

researcharxiv-cs-lg

MEMOBench: A Process Level Memory Benchmark for Robotic Manipulation

model-releasesarxiv-cs-ro

arXiv:2609.07047v1 Announce Type: new Abstract: Robotic manipulation often requires acting on information that is no longer visible, yet Vision-Language-Action policies are usually evaluated when the

PhenoBench: Mapping What a Deeply Phenotyped Human Cohort Can Tell Us

model-releasesarxiv-cs-lg

arXiv:2609.06080v1 Announce Type: new Abstract: Deeply phenotyped cohorts combine clinical, imaging, molecular, and wearable observations across timescales from seconds to years. This breadth can reve

Robust Vision Systems for Connected and Autonomous Vehicles: Security Challenges and Attack Vectors

arXiv:2602.09740v3 Announce Type: replace Abstract: Connected and Autonomous Vehicles (CAVs), which is critical for achieving Level-5 autonomous driving. Safe and reliable CAV navigation depends on ro

When Does a Laugh Begin? Structured Annotator Disagreement in Temporal Laughter Localization

arXiv:2609.06646v1 Announce Type: new Abstract: Annotators routinely disagree on laughter boundaries and subtle chuckles, yet temporal laughter localization typically evaluates against a single refere

Tutorials

Better Understanding, Better Fixes? A Study of Hallucination in LLM-based Automated Program Repair

arXiv:2609.04909v1 Announce Type: cross Abstract: Large language models (LLMs) have significantly advanced automated program repair (APR), yet existing evaluations remain largely result-centric and pr

tutorialsarxiv-cs-ai

7 Sept 2026

Risk-Aware Optimal Control with Rulebooks

model-releasesarxiv-cs-ro

arXiv:2609.05199v1 Announce Type: cross Abstract: We consider safety-critical control problems involving multiple requirements with different priorities and uncertainty in their evaluation. We represe

7 Sept 2026

Beyond Majority Vote: Multi-Perspective Adjudication for Medical Hallucination Detection

arXiv:2609.03953v1 Announce Type: new Abstract: Understanding the frequency of factual errors in chatbot-generated text and evaluating systems that detect these errors is critical for determining chat

4 Sept 2026

How Perturbations Propagate: A Multi-Level Analysis of Robustness in Large Language Models