Search: “evaluation”

Latent Reward Registers for Diffusion Preference Alignment

arXiv:2608.03929v1 Announce Type: cross Abstract: Aligning diffusion models with human preferences usually relies on a sparse terminal reward evaluated on the final generated samples, presenting a sev

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Applications

LogitScope: A Framework for Analyzing LLM Uncertainty Through Information Metrics

DGX agent

arXiv:2603.24929v2 Announce Type: replace Abstract: Understanding and quantifying uncertainty in large language model (LLM) outputs is critical for reliable deployment. However, traditional evaluation

applicationsarxiv-cs-ai

5 Aug 2026

NearID: Identity Representation Learning via Near-identity Distractors

arXiv:2604.01973v2 Announce Type: replace Abstract: When evaluating identity-focused tasks such as personalized generation and image editing, existing vision encoders entangle object identity with bac

Optimising for Flourishing: Flourishing Metrics and Return on Flourishing as Success Criteria for Artificial Intelligence and Post-AGI Economic Systems

arXiv:2608.00151v2 Announce Type: replace-cross Abstract: Current evaluation frameworks for artificial intelligence focus mainly on capability, safety, and proxies such as adoption, engagement, effici

safetyarxiv-cs-ai

Tutorials

Soft Guidance Starts to Outperform CoT Prompting as LLMs Improve

arXiv:2608.03550v1 Announce Type: new Abstract: Chain-of-Thought (CoT) prompting remains the standard baseline for evaluating models' reasoning abilities. Originally, this technique was introduced to

tutorialsarxiv-cs-ai

TACT: Taxonomy-Aligned Post-Training for Pedagogically Adaptive English Tutoring

arXiv:2608.03952v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used to provide conversational practice for English-as-a-second-language (ESL) learners. Effective ESL tut

VetScore: Risk-Weighted Fact Verification for Veterinary Long-Form QA with Citations

arXiv:2608.03675v1 Announce Type: new Abstract: Citation excerpts can be used to increase the reliability of generated outputs and their faithfulness to cited sources, which is especially important in

When Memory Becomes Authority: Benchmarking Authority Collapse at the Memory Consolidation Boundary

arXiv:2608.01679v2 Announce Type: replace Abstract: Persistent memory allows (self-evolving) LLM agents to adapt across tasks by consolidating heterogeneous interaction histories into reusable facts,

CultureVidBench: Benchmarking Cultural Understanding in Text-to-Video Generation

arXiv:2608.01942v1 Announce Type: cross Abstract: Text-to-video (T2V) generation models have advanced rapidly, yet their ability to represent diverse cultural contexts remains underexplored. Existing

Tutorials

Disagree to Accelerate: Closing the Loop on Diffusion Feature Forecasts

arXiv:2608.01740v1 Announce Type: new Abstract: Training-free feature forecasting accelerates diffusion sampling by predicting features at skipped denoising steps. Recent work has mainly focused on de

tutorialsarxiv-cs-lg

Does Accuracy Equal Evidence? Reasoning Faithfulness under KV Cache Compression

arXiv:2608.01631v1 Announce Type: new Abstract: KV cache compression is commonly evaluated by final-answer accuracy, implicitly assuming that preserving the answer also preserves the reasoning that su

DrawAI: Agentic Benchmark and Workflow for Making Raster Images Editable

arXiv:2608.00548v1 Announce Type: new Abstract: Recent image-generation models and multimodal agents can produce high-quality visuals for increasingly complex visual communication tasks. Yet their ras

Foveated Probes Recover Localized Binding Information in Vision Foundation Models

arXiv:2608.00726v1 Announce Type: new Abstract: Frozen vision foundation models are commonly evaluated through a single global image embedding, but this interface can conflate missing information with

researcharxiv-cs-cv

Gaokerena: A Small Persian Medical Language Model Family

arXiv:2608.00932v1 Announce Type: new Abstract: The integration of artificial intelligence into medical question-answering systems has advanced rapidly; however, research remains predominantly focused

GEOID-Flood: A Large-Scale Multi-Modal Benchmark Dataset for Flood Segmentation

arXiv:2608.02315v1 Announce Type: new Abstract: Geospatial foundation models aim to learn representations that transfer across regions and sensors, yet evaluating them on specific tasks requires large

Leak It: A Probabilistic Approach to Training-Data Extraction from Black-Box Language Models

arXiv:2608.00144v1 Announce Type: cross Abstract: Membership inference (MIA) on language models is usually summarised by an aggregate ROC-AUC, but such evaluations are confounded: model-free blind bas

Long-Horizon Embodied Decision-Making via Multimodal Memory Compression

arXiv:2608.01456v1 Announce Type: cross Abstract: Agents are increasingly expected to act not only as task executors, but also as decision-makers on behalf of human users. This shift requires agents t

LongChart VQA: A Comprehensive Benchmark for MLLMs with Complex Multi-Chart Reasoning

arXiv:2608.01328v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) are rapidly evolving with expanded context windows and stronger reasoning capabilities, enabling multi-chart un

OpenART: Scaling Agent Red Teaming via Open-Ended Environment Evolution

arXiv:2608.00677v1 Announce Type: new Abstract: AI agents operate in persistent environments where early state changes can influence decisions far into the future. Unlike conventional language-model i

Policy Optimality Measurement for Multi-Vehicle Decision-Making: From Extrinsic Indicators to Intrinsic Quality

arXiv:2608.01133v1 Announce Type: new Abstract: Evaluating Multi-Agent Reinforcement Learning (MARL) policies in autonomous driving fundamentally relies on extrinsic statistical indicators (e.g., rewa

safetyarxiv-cs-lg

Pruned BPE: Post-training Visibility Pruning and Token Reallocation for Byte Pair Encoding

arXiv:2608.00837v1 Announce Type: new Abstract: Byte Pair Encoding (BPE) is widely used for subword tokenization, but standard BPE exposes every learned merge token to the downstream model, including

Applications

Reassessing the Feasibility of PPG-Based Non-Invasive Blood Glucose Level Estimation

arXiv:2608.01820v1 Announce Type: cross Abstract: Non-invasive blood glucose level (BGL) estimation from photoplethysmography (PPG) holds great promise for wearable health monitoring, but results acro

applicationsarxiv-cs-lg

RSVideo: Are Your Vision-Language Models Ready for Remote Sensing Videos?

arXiv:2608.02039v1 Announce Type: new Abstract: Remote-sensing videos enable real-time observation of changes in target attributes, short-term activities, and scene evolution. They record motion, acti

Stochastic Sequential Search in Very-High-Dimensional Feature Selection

arXiv:2608.01502v1 Announce Type: cross Abstract: Sequential subset search -- forward selection with floating backtracking and its descendants -- remains the quality reference in feature selection, bu

researcharxiv-cs-cv

Local Ai

Through the LENS: Local Geometric Decomposition of Vision-Language Model Representations

arXiv:2608.00561v1 Announce Type: cross Abstract: Vision-language models (VLMs) process image patches and text tokens in a shared residual stream, but the local geometry through which the two modaliti

local-aiarxiv-cs-cl

Who Belongs in the Eval Set? A Capability-Taxonomy-Driven Pipeline for Curating Regression Eval Sets in Agent-Extensibility Platforms

model-releasesarxiv-cs-lg

arXiv:2608.01004v1 Announce Type: new Abstract: Platform teams hosting agent-extensibility surfaces face a regression-economics paradox: every onboarding customer ships an evaluation set tuned to thei

WorldDynCache: Risk-Controlled Latent Dynamics Approximation for Diffusion World Model

arXiv:2608.01845v1 Announce Type: cross Abstract: Diffusion world models generate high-quality futures, but re- peated transformer evaluations make inference prohibitively slow. Existing caches reuse

researcharxiv-cs-cv

Are the Financial Reasoning from LLMs Credible? A Real World Test over Long-Horizon Statements

arXiv:2607.28661v1 Announce Type: new Abstract: Do Large Language Models (LLMs) possess genuine structural reasoning, or merely rely on surface-level pattern matching? The financial domain, demanding

Autonomous Repair for Multi-Agent Systems via Monte-Carlo Tree Search

arXiv:2607.29055v1 Announce Type: cross Abstract: Multi-agent systems (MAS) are increasingly deployed to solve complex tasks. In case of incorrect or unsatisfactory outputs, users have to manually loc

Benchmarks Are Not Validation: A System-Level View of Financial LLM Applications

arXiv:2607.28840v1 Announce Type: new Abstract: Large language models are increasingly deployed in financial applications that combine retrieval, proprietary data, tool use, orchestration logic, monit

Can Large Language Models Derive New Knowledge? A Dynamic Benchmark for Biological Knowledge Discovery

arXiv:2603.03322v2 Announce Type: replace-cross Abstract: Recent advancements in Large Language Model (LLM) agents have demonstrated remarkable potential in automatic knowledge discovery. However, rig

Can Zero-Shot LLMs Predict Child Malnutrition? A Fairness and Temporal Robustness Study

arXiv:2607.29082v1 Announce Type: new Abstract: Child malnutrition remains a major public health challenge in low- and middle-income countries, particularly in South Asia, where early identification o

safetyarxiv-cs-cl

Detecting AI-Generated Videos with Spiking Neural Networks

arXiv:2605.05895v2 Announce Type: replace-cross Abstract: Modern AI-generated videos are photorealistic at the single-frame level, leaving inter-frame dynamics as the main remaining axis for detection

Language Models Agree With Each Other, Not With Readers

arXiv:2607.29274v1 Announce Type: cross Abstract: Claims that language models homogenise are usually measured against human judgements collected for the study, which makes the human side an artifact o

Looks Right, Works Right: A Project-Level Benchmark for Multi-Screen Mobile App Generation

arXiv:2607.28645v1 Announce Type: cross Abstract: Recent multimodal large language models can convert visual designs directly into executable code, but real mobile products require multiple screenshot

OpenClaw and Ollama in Agentic AI: Toward Fully Autonomous and Scalable AI Agent Systems

arXiv:2607.28629v1 Announce Type: new Abstract: The rapid transition from reactive large language models (LLMs) to persistent, action-capable systems has exposed critical gaps in the architectural und

Reasoning in Real World Clinical Care: Why Large Language Models Are Not Yet Safe for Autonomous Clinical Decision Support

arXiv:2607.28677v1 Announce Type: new Abstract: LLM now pass medical licensing examinations and, in curated cases, can rival physicians at diagnostic reasoning. These developments have accelerated the

safetyarxiv-cs-ai

Sensitivity Analysis of GRU, LSTM and Transformer Encoder in Classification of Automated Driving Systems

arXiv:2607.28665v1 Announce Type: cross Abstract: Automated driving systems (ADSs) are becoming ubiquitous. Future Software Defined Vehicles (SDVs) may be able to run multiple ADSs, both native and af

safetyarxiv-cs-ai

The Grokked Illusion: True Equilibrium Mitigates Catastrophic Forgetting

model-releasesarxiv-cs-lg

arXiv:2607.29503v1 Announce Type: new Abstract: While neural networks are typically evaluated by their training and test performance, these metrics do not reveal how robust a learned representation is

Why It Hurts: Identifying the Drivers of Negative Thoughts in Emotional Support Conversations

arXiv:2607.28648v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly used for emotional support tasks, such as negative thought reframing. This task relies on modifying cogn

A Reference-Free Score for Detecting Silent Reasoning Failures in Large Language Models

arXiv:2607.26102v1 Announce Type: cross Abstract: Mathematical chain of thought (CoT) evaluation is commonly reduced to whether the final answer matches a reference. This conflates producing a correct

researcharxiv-cs-ai

A Systems Engineering Framework for Vision-Language-Enabled UAV Triage and Disaster Response

arXiv:2607.27597v1 Announce Type: new Abstract: Recent advances in Vision Language Models (VLMs) have created new opportunities for disaster response, where responders must interpret large volumes of

safetyarxiv-cs-ro

An Efficient Solution to the 2D Visibility Problem in Cartesian Grid Maps and its Application in Heuristic Path Planning

arXiv:2403.06494v2 Announce Type: replace-cross Abstract: This paper introduces a novel, lightweight method to solve the visibility problem for 2D grids. The proposed method evaluates the existence of

researcharxiv-cs-ro

Local Ai

Auto Research for Materials: Auditable AI-Scientist Workflows with Held-Out Transfer

arXiv:2607.17100v2 Announce Type: replace-cross Abstract: Auto Research uses language-model agents to propose, implement, and evaluate machine-learning changes in a closed loop, but is usually judged

local-aiarxiv-cs-ai

Benchmarking Foundation and Large Language Models for Few-Shot Medical Image Segmentation

arXiv:2607.27856v1 Announce Type: new Abstract: Few-shot medical image segmentation (FS-MIS) aims to segment novel regions of interest (ROIs) from a few annotated support examples. Despite rapid progr

Beyond Feeling Better: Capability-Sustaining Emotional Dialogue as a Longitudinal Research Paradigm

arXiv:2607.27851v1 Announce Type: new Abstract: Emotional dialogue research includes two influential strategy traditions. Empathetic dialogue prioritizes understanding a speaker's emotional experience

safetyarxiv-cs-cl

Applications

Calibrate Globally, Measure Everywhere: Scaling LLM-Based Prevalence Measurement Across A/B Experiments

arXiv:2602.16111v2 Announce Type: replace-cross Abstract: Online media platforms track the share of impressions associated with content attributes, or prevalence, to evaluate trade-offs and set guardr

applicationsarxiv-cs-ai

Dimensionality and Measurement Precision in HLE's Multiple-Choice Subset

arXiv:2607.27420v1 Announce Type: cross Abstract: Humanity's Last Exam (HLE) is widely used to evaluate frontier language models. HLE organizes its questions into eight subject-domain categories, whos