Search: “evaluation”

How Closely Do LLM Reviews Align with Human Peer Review?

arXiv:2608.03659v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used to generate scientific reviews, yet existing evaluations rarely examine whether different providers

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Modeling Matches as Language: A Generative Transformer Approach for Counterfactual Player Valuation in Football

DGX agent

arXiv:2603.15212v2 Announce Type: replace Abstract: Evaluating football player transfers is challenging because player actions depend strongly on tactical systems, teammates, and match context. Despit

researcharxiv-cs-ai

5 Aug 2026

MT-Web2Code: Benchmarking Coding Agents on Multi-Turn Regional Reconstruction and Localized Modification

arXiv:2608.03474v1 Announce Type: new Abstract: Recent advances in Large Vision-Language Models (LVLMs) have demonstrated impressive capabilities in web UI generation. However, existing benchmarks pre

One-Point Contraction: Erasing Representational Separability toward Irreversible Deep Forgetting

arXiv:2507.07754v3 Announce Type: replace-cross Abstract: Machine unlearning is usually evaluated by what the classifier outputs: forget-set accuracy, confidence, membership-inference scores. We show

Agents

PAIChecker: Uncovering and Checking PR-Issue Misalignment in SWE-Bench-Like Benchmarks

arXiv:2607.28587v2 Announce Type: replace-cross Abstract: SWE-bench-like benchmarks are widely used for evaluating LLM's issue resolution capability. They typically follow a common construction pipeli

agentsarxiv-cs-ai

PhyAI: Real-Time Physical AI at the Edge, Scalable Rollouts in the Cloud

arXiv:2608.03682v1 Announce Type: new Abstract: Physical AI policies require inference throughout their lifecycle, including model evaluation, cloud reinforcement learning rollout, edge GPU serving, a

RealWeather: Realistic and Scene-Faithful Weather Translation with Driving World Models

arXiv:2608.02953v1 Announce Type: new Abstract: Realistic weather translation is valuable for developing and evaluating autonomous driving systems, yet collecting paired videos of the same scenes unde

SciRet: A Compute-Aware Empirical Study of Retrieval and Reranking for Scientific RAG

arXiv:2608.03860v1 Announce Type: cross Abstract: We introduce SciRet, a compute-aware empirical study of retrieval-augmented generation for scientific question answering over CORD-19. Rather than pro

SP3O: Reinforcement Learning from Segment Preferences without Reward Modeling

arXiv:2608.02951v1 Announce Type: cross Abstract: Preference-based reinforcement learning (PbRL) for general stochastic MDPs often requires training a reward model. Existing reward-model-free methods

When Agents Learn to Be You: Benchmarking Privacy Leakage, Impersonation Risk, and Defenses in Persona Skills

arXiv:2608.03700v1 Announce Type: cross Abstract: Persona skills distill personal interaction histories into portable and executable artifacts for downstream agents. While enabling flexible personaliz

When Do Fewer Visual Tokens Accelerate Multimodal Inference? A Break-Even Study Across Decision Locations and Hardware

arXiv:2608.03649v1 Announce Type: new Abstract: Fewer visual tokens do not guarantee lower end-to-end latency. We evaluate break-even with a reproducible protocol that accounts for decision overhead,

When Oracle Conditioning Misleads Deployment: Conditioning-Availability Bias in Echocardiographic Segmentation

arXiv:2608.03342v1 Announce Type: cross Abstract: Conditional segmentation models may be trained and evaluated with auxiliary signals cleaner than those available at deployment. We study this protocol

AlphaG-OPD: Reliability-Gated Sibling Counterfactuals for On-Policy Distillation in Symbolic Alpha Factor Discovery

arXiv:2608.01303v1 Announce Type: new Abstract: Symbolic alpha factor discovery can score a completed expression, but it provides no direct label for the structural decisions that produced it. Generat

safetyarxiv-cs-lg

ArabicDialectSafety: A Dialect-Aware Benchmark for Arabic Content Safety Classification

arXiv:2608.01291v1 Announce Type: new Abstract: We present ArabicDialectSafety, a human-curated Arabic safety dataset of 25,071 prompts covering six Arabic varieties: Modern Standard Arabic, Syrian, E

Beyond Token-Level Cross-Entropy: Frechet Distributional Post-Training for Autoregressive Image Generation

arXiv:2608.00562v1 Announce Type: new Abstract: Autoregressive image generators are commonly pretrained with token-level cross-entropy under teacher forcing, yet evaluated by the distributional qualit

researcharxiv-cs-cv

Breaking the Statistical Similarity Trap in Extreme Convection Detection

arXiv:2509.09195v2 Announce Type: replace-cross Abstract: Current evaluation metrics for deep learning weather models create a 'Statistical Similarity Trap', rewarding blurry predictions while missing

Can Humans Dream of Electric Sheep? Human-Written Samples for Fine-Grained Vision-and-Language Hallucination Benchmarking

arXiv:2608.01021v1 Announce Type: cross Abstract: In an age of rapid model turnover, how do we make hallucination evaluation more perennial? We explore whether human-written hallucination samples coul

researcharxiv-cs-cl

Can You Trust the Confidence? ConfBench for Vision-Language Models on Document Extraction

arXiv:2608.01792v1 Announce Type: cross Abstract: Intelligent document processing (IDP) with vision-language models (VLMs) hinges on confidence scores trustworthy enough to route extractions between a

ChordVideo: One-Step, Training-Free, Temporally Consistent Video Editing via Low-Energy Transport

arXiv:2608.00769v1 Announce Type: new Abstract: One-step text-to-image models enable training-free, inversion-free editing with only 1--2 network function evaluations (NFE), while ChordEdit stabilizes

CrossProjection: Geometric Grounding Beyond Viewpoint Change in Architectural Drawings

arXiv:2608.00473v1 Announce Type: cross Abstract: Architectural drawings violate the usual assumption behind multi-view reasoning: plans and sections are cuts, while elevations are facade projections,

Deep Research Pretraining via Predictive Navigation

arXiv:2608.00432v1 Announce Type: new Abstract: Deep research agents are often trained on expensive, environment-grounded tool-use trajectories that require repeated retrieval, document inspection, an

safetyarxiv-cs-cl

Do Static Embeddings Add Value to Hybrid Dutch Retrieval?

model-releasesarxiv-cs-lg

arXiv:2608.02112v1 Announce Type: new Abstract: Embedding benchmarks measure standalone model quality, but they do not establish whether a low-cost retriever contributes complementary ranking informat

FDIR: Harmonizing Fidelity and Human-Machine Preference in Lossy Compression Image Restoration

arXiv:2608.00111v1 Announce Type: cross Abstract: Image restoration quality can be evaluated along three complementary facets: pixel-level fidelity, human perception, and downstream machine preference

researcharxiv-cs-cv

From Failures to Supervision: DynamicEnvPlan for Robust Long-Horizon Embodied Planning

arXiv:2608.00613v1 Announce Type: new Abstract: Physical-world interaction is inherently dynamic, as environments can evolve during execution, requiring agents to adapt their plans under non-stationar

safetyarxiv-cs-ro

From Global to Local: A Scalable Benchmark for Local Posterior Sampling

model-releasesarxiv-cs-lg

arXiv:2507.21449v2 Announce Type: replace-cross Abstract: Degeneracy is an inherent feature of the loss landscape of neural networks, but it is not well understood how stochastic gradient MCMC (SGMCMC

Tutorials

Gecko: Fast Private Inference via Secure Public Encoder Offloading

arXiv:2608.02378v1 Announce Type: new Abstract: Private inference protects both user inputs and server models during neural network inference, but existing solutions remain too slow for practical depl

tutorialsarxiv-cs-lg

Tutorials

Generative Brownian Bridge Diffusion In Motion Space For Enhanced Myocardial Strain Analysis

arXiv:2608.01677v1 Announce Type: new Abstract: Myocardial strain analysis of cardiac magnetic resonance (CMR) images provides an important tool for evaluating cardiac function. However, current techn

tutorialsarxiv-cs-cv

Investigating Social Bias in Narrative Image Generation

arXiv:2608.01780v1 Announce Type: new Abstract: Text-to-image (T2I) generation models are increasingly embedded in applications such as media content creation and education, raising concerns about how

Leveraging Synthetic Data for Question Answering with Multilingual LLMs in the Agricultural Domain

arXiv:2507.16974v3 Announce Type: replace Abstract: Enabling farmers to access accurate agriculture-related information in their native languages in a timely manner is crucial for the success of the a

researcharxiv-cs-cl

Agents

MedTextWeaver: Procedural Knowledge Evolution in Agentic Medical Text Editing

arXiv:2602.00740v2 Announce Type: replace Abstract: Medical text editing is essential for improving communication among diverse stakeholders in clinical settings. However, adapting LLM agents to this

agentsarxiv-cs-cl

MedUPS: Towards Diagnostic Assistance in Uncommon Medical Cases with Large Language Models

arXiv:2608.01012v1 Announce Type: new Abstract: Uncommon and off-guideline cases are difficult for clinical decision support, because physicians must make a series of management decisions under diagno

safetyarxiv-cs-cl

Obshazard-bench: Benchmarking Multimodal Foundation Models for Real-Time Disaster Intelligence from Raw Earth Observation Streams

arXiv:2608.00012v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) are increasingly used to interpret Earth observation data, yet their capability to support real-world disaster

ORCA: ORgan-Centroid Aggregation for Training-Free 3D CT Visual Token Compression

arXiv:2608.00345v1 Announce Type: new Abstract: A 3D CT scan entering a vision-language model produces a long sequence of visual tokens, often thousands to tens of thousands per volume, and this seque

PhyCheck: Fine-Grained Evidence-Grounded Dataset for Physical Law Understanding in Video-LLMs

arXiv:2608.02150v1 Announce Type: new Abstract: Embodied intelligence and world models require video understanding systems to go beyond recognizing objects and actions and develop an understanding of

researcharxiv-cs-cv

Revisiting Generalization Across Difficulty Levels: It's Not So Easy

arXiv:2511.21692v2 Announce Type: replace Abstract: We investigate how well large language models (LLMs) generalize across different task difficulties, a key question for effective data curation and e

researcharxiv-cs-cl

Robust Bayesian Optimization via Tempered Posteriors

arXiv:2601.07094v2 Announce Type: replace-cross Abstract: Bayesian optimization (BO) iteratively fits a Gaussian process (GP) surrogate to accumulated evaluations and selects new queries via an acquis

researcharxiv-cs-lg

SPIRIT: Spatio-temporal Pairwise Relational Modeling of Instrument-Tissue Interactions for Surgical Action Triplet Recognition

arXiv:2608.02188v1 Announce Type: new Abstract: Fine-grained understanding of surgical activity is essential for context-aware assistance in the operating room, including safety monitoring, adverse ev

Structured Proxy Features for Multimodal NSCLC Survival Prediction from Pretreatment CT

arXiv:2608.00446v1 Announce Type: new Abstract: Lung cancer results in roughly 1.8 million fatalities annually worldwide, with non-small cell lung cancer (NSCLC) comprising the majority of cases. Desp

UAV-Based Environmental Monitoring of Rip-Current Indicators Using Wavelet-Derived Texture Features

arXiv:2608.02448v1 Announce Type: new Abstract: Rip currents are recurrent coastal natural hazards that threaten beachgoers and create operational challenges for lifeguards and coastal managers. Relia

Upper-Expectile Multi-Step Q-Learning for Off-Policy Reinforcement Learning

arXiv:2608.02034v1 Announce Type: new Abstract: Multi-step returns accelerate reward propagation in off-policy reinforcement learning, but couple the evaluation of each decision to the suboptimal logg

safetyarxiv-cs-lg

What Transfers from Text to Vision? Capability Scaling Laws and Transfer Dynamics for VLMs

arXiv:2608.00013v1 Announce Type: new Abstract: Choosing the right large language model (LLM) backbone is the most consequential decision when building a vision-language model (VLM), yet it remains fu

When Do Surrogate Updates Improve Decisions? A Local Theory of Trajectory-Wise Transfer

arXiv:2608.01130v1 Announce Type: new Abstract: A broad range of models face the mismatch where they are updated through trajectory losses but are evaluated by downstream task reward. Here, a trajecto

researcharxiv-cs-lg

When May a Model Replace the Experiment? Audits, Licenses, and the Price of Trust in Surrogate-Driven Design

arXiv:2608.01378v1 Announce Type: new Abstract: Design campaigns in chemistry, materials science, and machine learning share a bottleneck: determining how good a candidate truly is requires an expensi

safetyarxiv-cs-lg

Writing-System-Level Tokenizer Adaptation for Byte-Level BPE

arXiv:2608.00582v1 Announce Type: new Abstract: Pretrained byte-level BPE tokenizers can segment underrepresented languages inefficiently. Replacing a tokenizer changes the meaning of nearly every tok

A Novel XAI-Enhanced Quantum Adversarial Networks for Velocity Dispersion Modeling in MaNGA Galaxies

arXiv:2510.24598v2 Announce Type: replace Abstract: Current quantum machine learning approaches often face challenges balancing predictive accuracy, robustness, and interpretability. To address this,

researcharxiv-cs-lg

A Unified Benchmark of Deep Learning Models for Multi-task 3D Brain Tumor Segmentation from Magnetic Resonance Imaging

arXiv:2607.28858v1 Announce Type: cross Abstract: Automatic brain tumor segmentation from magnetic resonance imaging (MRI) has become a fundamental task in computer-assisted diagnosis, treatment plann

ActionParty: Multi-Subject Action Binding in Generative Video Games

arXiv:2604.02330v2 Announce Type: replace-cross Abstract: Recent advances in video diffusion have enabled the development of 'world models' capable of simulating interactive environments. However, the

Beyond Component Testing: Validating Agentic AI Systems

arXiv:2607.29405v1 Announce Type: new Abstract: Agentic AI systems act through multi-step trajectories that combine planning, tool use, memory, interaction, and adaptation. This behavior stretches val