Search: “evaluation”

Beyond Suffixes: Token Position in GCG Adversarial Attacks on Large Language Models

arXiv:2602.03265v2 Announce Type: replace Abstract: Large Language Models (LLMs) have seen widespread adoption across multiple domains, creating an urgent need for robust safety alignment mechanisms.

safetyarxiv-cs-lg

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

Cultural Benchmarking of LLMs in Standard and Dialectal Arabic Dialogues

DGX agent

arXiv:2605.00119v1 Announce Type: new Abstract: There is a significant gap in evaluating cultural reasoning in LLMs using conversational datasets that capture culturally rich and dialectal contexts. M

researcharxiv-cs-cl

4 May 2026

Do Open-Loop Metrics Predict Closed-Loop Driving? A Cross-Benchmark Correlation Study of NAVSIM and Bench2Drive

model-releasesarxiv-cs-ro

arXiv:2605.00066v1 Announce Type: new Abstract: Open-loop evaluation offers fast, reproducible assessment of autonomous driving planners, but its ability to predict real closed-loop driving performanc

Driving with A Thousand Faces: A Benchmark for Closed-Loop Personalized End-to-End Autonomous Driving

arXiv:2602.18757v2 Announce Type: replace Abstract: Human driving behavior is inherently diverse, yet most end-to-end autonomous driving (E2E-AD) systems learn a single average driving style, neglecti

Introducing agent quality optimization in AgentCore, now in preview

Generate recommendations from production traces, validate them with batch evaluation and A/B testing, and ship with confidence. AI agents that perform well at launch don’t stay that way. As models evo

agentsaws-ml-blog

Introducing the agent performance loop: AgentCore Optimization now in preview

Introducing the agent quality loop: AgentCore Optimization now in preview

Learning the Helmholtz equation operator with DeepONet for non-parametric 2D geometries

arXiv:2605.00760v1 Announce Type: new Abstract: This paper deals with solving the 2D Helmholtz equation on non-parametric domains, leveraging a physics-informed neural operator network based on the De

local-aiarxiv-cs-lg

Modeling Subjective Urban Perception with Human Gaze

arXiv:2605.00764v1 Announce Type: new Abstract: Urban perception describes how people subjectively evaluate urban environments, shaping how cities are experienced and understood. Existing computationa

RETO: A Rotary-Enhanced Transformer Operator for High-Fidelity Prediction of Automotive Aerodynamics

model-releasesarxiv-cs-lg

arXiv:2605.00062v1 Announce Type: cross Abstract: Rapid aerodynamic evaluation is crucial for modern vehicle design, yet existing neural operators struggle to capture intricate spatial correlations. W

The Seismic Wavefield Common Task Framework

arXiv:2512.19927v2 Announce Type: replace Abstract: Seismology faces fundamental challenges in state forecasting and reconstruction (e.g., earthquake early warning and ground motion prediction) and ma

researcharxiv-cs-lg

Unsupervised Denoising of Real Clinical Low Dose Liver CT with Perceptual Attention Networks

arXiv:2605.00793v1 Announce Type: cross Abstract: With the development of deep learning, medical image processing has been widely used to assist clinical research. This paper focuses on the denoising

What if your AI could review its own work before you even see it? @ListenLabs Co-Founder & CTO @florian_jue explained how this works on the …

This post discusses self-review capabilities in AI systems, where an AI model can evaluate and refine its own outputs before presenting them to users. According to Florian Jue from Listen Labs, this f

agentsharrison-chase--x

World Model for Robot Learning: A Comprehensive Survey

arXiv:2605.00080v1 Announce Type: cross Abstract: World models, which are predictive representations of how environments evolve under actions, have become a central component of robot learning. They s

safetyarxiv-cs-cv

Qwen3.6 vs gpt-oss:120b on Apple Silicon — three Qwen variants benchmarked, plus what works and where it does not

This post benchmarks three Qwen3.6 model variants against gpt-oss:120b when running on Apple Silicon hardware, evaluating their performance characteristics and practical usability. It documents both t

model-releasesr-ollama

3 May 2026

AesRM: Improving Video Aesthetics with Expert-Level Feedback

arXiv:2604.28078v1 Announce Type: new Abstract: Despite rapid advances in photorealistic video generation, real-world applications such as filmmaking require video aesthetics, e.g., harmonious colors

Agentic AI for Cybersecurity: A Meta-Cognitive Architecture for Governable Autonomy

arXiv:2602.11897v3 Announce Type: replace-cross Abstract: Cybersecurity decision-making increasingly occurs in environments characterized by uncertainty, partial observability, and adversarial manipul

CareGuardAI: Context-Aware Multi-Agent Guardrails for Clinical Safety & Hallucination Mitigation in Patient-Facing LLMs

arXiv:2604.26959v1 Announce Type: cross Abstract: Integrating large language models (LLMs) into patient-facing healthcare systems offers significant potential to improve access to medical information.

Claw-Eval-Live: A Live Agent Benchmark for Evolving Real-World Workflows

arXiv:2604.28139v1 Announce Type: cross Abstract: LLM agents are expected to complete end-to-end units of work across software tools, business services, and local workspaces. Yet many agent benchmarks

ClimateVID -- Social Media Videos Analysis and Challenges Involved

arXiv:2604.27968v1 Announce Type: new Abstract: The pervasive growth of digital content, specifically short videos on social media platforms, has significantly altered how topics are discussed and und

Decoding Scientific Experimental Images: The SPUR Benchmark for Perception, Understanding, and Reasoning

arXiv:2604.27604v1 Announce Type: new Abstract: We introduce SPUR, a comprehensive benchmark for scientific experimental image perception, understanding, and reasoning, comprising 4,264 question-answe

Detecting Clinical Discrepancies in Health Coaching Agents: A Dual-Stream Memory and Reconciliation Architecture

arXiv:2604.27045v1 Announce Type: cross Abstract: As Large Language Model (LLM) agents transition from single-session tools to persistent systems managing longitudinal healthcare journeys, their memor

Industry

Directly compare different Tesla models & features http://Tesla.com/compare

Tesla's official comparison tool allows customers to directly evaluate different Tesla vehicle models side-by-side, examining specifications, features, performance, and pricing to help inform purchase

industryelon-musk--x

FinChain: A Symbolic Benchmark for Verifiable Chain-of-Thought Financial Reasoning

arXiv:2506.02515v4 Announce Type: replace-cross Abstract: Multi-step symbolic reasoning is essential for robust financial analysis; yet, current benchmarks largely overlook this capability. Existing d

Flattery in Motion: Benchmarking and Analyzing Sycophancy in Video-LLMs

arXiv:2506.07180v3 Announce Type: replace-cross Abstract: As video large language models (Video-LLMs) become increasingly integrated into real-world applications that demand grounded multimodal reason

Function-based Parametric Co-Design Optimization of Dexterous Hands

model-releasesarxiv-cs-ro

arXiv:2604.27557v1 Announce Type: new Abstract: Despite advances in dexterous hand manipulation, robotic hand design is still largely decoupled from task-driven evaluation and control, limiting system

Learning to Spend: Model Predictive Control for Budgeting under Non-Stationary Returns

arXiv:2604.27186v1 Announce Type: cross Abstract: We study finite-horizon budget allocation as a closed-loop economic control problem and evaluate receding-horizon Model Predictive Control (MPC) relat

researcharxiv-cs-ai

LLMs Capture Emotion Labels, Not Emotion Uncertainty: Distributional Analysis and Calibration of Human--LLM Judgment Gaps

arXiv:2604.27345v1 Announce Type: new Abstract: Human annotators frequently disagree on emotion labels, yet most evaluations of Large Language Model (LLM) emotion annotation collapse these judgments i

researcharxiv-cs-cl

Mitigating Selection Bias in Large Language Models via Permutation-Aware GRPO

arXiv:2603.21016v2 Announce Type: replace-cross Abstract: Large language models (LLMs) used for multiple-choice and pairwise evaluation tasks often exhibit selection bias due to non-semantic factors l

PhotIQA: A photoacoustic image data set with image quality ratings

arXiv:2507.03478v2 Announce Type: replace-cross Abstract: Image quality assessment (IQA) is crucial in the evaluation stage of novel algorithms operating on images, including traditional and machine l

Political Bias Audits of LLMs Capture Sycophancy to the Inferred Auditor

arXiv:2604.27633v1 Announce Type: new Abstract: Large language models (LLMs) are commonly evaluated for political bias based on their responses to fixed questionnaires, which typically place frontier

REBENCH: A Procedural, Fair-by-Construction Benchmark for LLMs on Stripped-Binary Types and Names (Extended Version)

model-releasesarxiv-cs-lg

arXiv:2604.27319v1 Announce Type: cross Abstract: Large Language Models (LLMs) have achieved remarkable progress in recent years, driving their adoption across a wide range of domains, including compu

RuC: HDL-Agnostic Rule Completion Benchmark Generation

arXiv:2604.27780v1 Announce Type: cross Abstract: Large Language Models (LLMs) have rapidly improved in performance across code-related tasks, making their integration into Register Transfer Level (RT

Sampler-Robust Optimization under Generative Models

arXiv:2604.27447v1 Announce Type: cross Abstract: Modern stochastic optimization pipelines increasingly rely on learned generative models to represent uncertainty, while downstream decisions are evalu

researcharxiv-cs-ai

Skills-Coach: A Self-Evolving Skill Optimizer via Training-Free GRPO

model-releasesarxiv-cs-cl

arXiv:2604.27488v1 Announce Type: new Abstract: We introduce Skills-Coach, a novel automated framework designed to significantly enhance the self-evolution of skills within Large Language Model (LLM)-

SpecVQA: A Benchmark for Spectral Understanding and Visual Question Answering in Scientific Images

arXiv:2604.28039v1 Announce Type: new Abstract: Spectra are a prevalent yet highly information-dense form of scientific imagery, presenting substantial challenges to multimodal large language models (

WaferSAGE: Large Language Model-Powered Wafer Defect Analysis via Synthetic Data Generation and Rubric-Guided Reinforcement Learning

arXiv:2604.27629v1 Announce Type: new Abstract: We present WaferSAGE, a framework for wafer defect visual question answering using small vision-language models. To address data scarcity in semiconduct

What benchmark would you build for “reply quality” in SDR generation? [D]

model-releasesr-machinelearning

Based on the Reddit discussion title, this likely discusses how to design and establish benchmarks for evaluating the quality of AI-generated responses in Sales Development Representative (SDR) system

When Roles Fail: Epistemic Constraints on Advocate Role Fidelity in LLM-Based Political Statement Analysis

arXiv:2604.27228v1 Announce Type: new Abstract: Democratic discourse analysis systems increasingly rely on multi-agent LLM pipelines in which distinct evaluator models are assigned adversarial roles t

ATLAS: An Annotation Tool for Long-horizon Robotic Action Segmentation

arXiv:2604.26637v1 Announce Type: cross Abstract: Annotating long-horizon robotic demonstrations with precise temporal action boundaries is crucial for training and evaluating action segmentation and

Codex CLI 0.128.0 adds /goal

Codex CLI 0.128.0 adds /goal The latest version of OpenAI's Codex CLI coding agent adds their own version of the Ralph loop: you can now set a /goal and Codex will keep on looping until it evaluates t

agentssimon-willison

CurEvo: Curriculum-Guided Self-Evolution for Video Understanding

arXiv:2604.26707v1 Announce Type: new Abstract: Recent advances in self-evolution video understanding frameworks have demonstrated the potential of autonomous learning without human annotations. Howev

Electricity price forecasting across Norway's five bidding zones in the post-crisis era

model-releasesarxiv-cs-lg

arXiv:2604.26634v1 Announce Type: new Abstract: Norway's electricity market is heavily dominated by hydropower, but the 2021--2022 energy crisis and stronger integration with Continental Europe have f

From Prompt Risk to Response Risk: Paired Analysis of Safety Behavior of Large Language Model

arXiv:2604.26052v1 Announce Type: new Abstract: Safety evaluations of large language models (LLMs) typically report binary outcomes such as attack success rate, refusal rate, or harmful/not-harmful re

safetyarxiv-cs-cl

Information Extraction from Electricity Invoices with General-Purpose Large Language Models

model-releasesarxiv-cs-cl

arXiv:2604.25927v1 Announce Type: new Abstract: Information extraction from semi-structured business documents remains a critical challenge for enterprise management. This study evaluates the capabili

Majority of this work and awesome eval was done by @MehulKalia_ ! We're hiring an EM and a senior IC for this team :) Want to join him? appl…

Majority of this work and awesome eval was done by @MehulKalia_ ! We're hiring an EM and a senior IC for this team :) Want to join him? apply at http://www.notion.com/careers When we evaluated agentic

agentslinus-lee--x

Preserving Disagreement: Architectural Heterogeneity and Coherence Validation in Multi-Agent Policy Simulation

arXiv:2604.26561v1 Announce Type: cross Abstract: Multi-agent deliberation systems using large language models (LLMs) are increasingly proposed for policy simulation, yet they suffer from artificial c

Rethinking the Harmonic Loss via Non-Euclidean Distance Layers

arXiv:2603.10225v3 Announce Type: replace-cross Abstract: Cross-entropy loss has long been the standard choice for training deep neural networks, yet it suffers from interpretability limitations, unbo

researcharxiv-cs-ai