Search: “evaluation”

RS-RIE-Bench: Benchmarking Reasoning-Guided Remote Sensing Image Editing

arXiv:2607.20197v1 Announce Type: new Abstract: Remote sensing image editing aims to modify remote sensing images according to natural language instructions while preserving geographic rules and senso

23 Jul 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

The Two-Process Theory of Machine Self-Report

DGX agent

arXiv:2607.20082v1 Announce Type: new Abstract: Language models are increasingly asked to self-report, informing safety evaluations, public understanding, and model-welfare debates. Yet their reports

safetyarxiv-cs-cl

23 Jul 2026

Toward Adaptable Multi-Agent Reinforcement Learning: An Assumption-Aware Review

arXiv:2507.10142v2 Announce Type: replace Abstract: Multi-Agent Reinforcement Learning (MARL) has achieved strong performance in simulated benchmarks, yet real deployments often violate the assumption

23 Jul 2026

Unlearning as Distribution Restoration: A Controlled Counterfactual Study, a Validated Selective Screen, and the Limits of Oracle-Free Certification

arXiv:2607.19442v1 Announce Type: cross Abstract: Machine unlearning is commonly evaluated by matching a retrained oracle on trained probes. In a controlled nonce-fact testbed with a matched retrainin

23 Jul 2026

MedRealMM: A Real-World Multimodal Benchmark for Chinese Online Medical Consultation

arXiv:2607.09142v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly deployed in online medical consultation, yet existing benchmarks remain poorly aligned with real clini

Local Ai

Multimodal Assessment of Pancreatic Cancer Resectability Using Deep Learning

arXiv:2607.13826v1 Announce Type: cross Abstract: Accurate determination of pancreatic ductal adenocarcinoma (PDAC) resectability relies on evaluating how the tumor interacts with major peripancreatic

local-aiarxiv-cs-ai

Operational Evidence Gaps for LLMs in Fraud Detection and Trust-and-Safety Workflows

arXiv:2607.13078v1 Announce Type: cross Abstract: LLMs are now proposed for fraud detection, scam investigation, content moderation, and other trust-and-safety workflows. Much of the public literature

Privacy Preserving Recommender Systems Balancing Personalization with Privacy

arXiv:2607.13328v1 Announce Type: cross Abstract: Personalized recommendation systems are central to modern e-commerce and retail platforms, but they typically rely on centralized storage of detailed

Agents

Self-Improvements in Modern Agentic Systems: A Survey

arXiv:2607.13104v1 Announce Type: new Abstract: Self-improving autonomous agents are moving from research prototypes to deployed systems. The primary goal is controllable evolution, or adaptation, fro

agentsarxiv-cs-ai

Hardware

Text2Sign: A Single-GPU Diffusion Baseline for Text-to-Sign Language Video Generation

arXiv:2607.13164v1 Announce Type: cross Abstract: Sign language is a primary communication channel for millions of Deaf and hard-of-hearing people, yet text-to-signer video generation remains costly b

hardwarearxiv-cs-cv

VAIOM: Continuous-Input, Discrete-Output Decoder-Only Financial Sequence Modeling

arXiv:2607.13929v1 Announce Type: new Abstract: Financial observations are continuous, heterogeneous, and noisy, whereas decoder-only next-token models are usually built around discrete symbolic input

researcharxiv-cs-lg

Agents

AgentCheck: A Reproduce-Intervene-Mitigate Workbench for LLM Agents over MCP

arXiv:2607.11098v2 Announce Type: replace-cross Abstract: Tool-using LLM agents are mostly evaluated assuming all tools work. When a tool times out, returns a week-stale value, or has its description

agentsarxiv-cs-ai

Connected by Construction: Learning Tractable Near-Tour Marginals for Traveling Salesman Problems

arXiv:2607.12127v1 Announce Type: new Abstract: Learning-based methods for the traveling salesman problem (TSP) are often evaluated through the tours produced after decoding or search, but the learned

researcharxiv-cs-ai

CrochetBench: Can Vision-Language Models Move from Describing to Doing in Crochet Domain?

arXiv:2511.09483v3 Announce Type: replace Abstract: While multimodal large language models can describe visual content, their ability to generate executable procedures remains underexplored. CrochetBe

DECO: Decoupled Multimodal Diffusion Transformer for Bimanual Dexterous Manipulation with a Plugin Tactile Adapter

arXiv:2602.05513v3 Announce Type: replace-cross Abstract: Bimanual dexterous manipulation relies on integrating multimodal inputs to perform complex real-world tasks. To address the challenges of effe

Egocentric Bias in Vision-Language Models

arXiv:2602.15892v2 Announce Type: replace-cross Abstract: Visual perspective taking--inferring how the world appears from another's viewpoint--is foundational to social cognition. We introduce FlipSet

First, do NOHARM: a medical safety benchmark and randomized study of physician and AI teaming on clinical consultations

arXiv:2512.01241v4 Announce Type: replace-cross Abstract: Large language models (LLMs) and medical AI tools are routinely used by physicians and patients for medical advice, yet their clinical safety

Hy-Embodied-VLM-1.0: Efficient Physical-World Agents

arXiv:2607.12894v1 Announce Type: new Abstract: Building capable embodied agents requires not only multimodal perception and understanding, but also agentic capabilities for reasoning about actions, a

Imputation-free transformer learning enables robust Alzheimer's disease prediction and calibrated uncertainty quantification across heterogeneous clinical cohorts

arXiv:2607.11656v2 Announce Type: replace-cross Abstract: Accurate diagnostic classification and disease-severity prediction for Alzheimer's disease are hampered by the incompleteness and heterogeneit

safetyarxiv-cs-lg

LakeQuest: A Three-Domain Benchmark for Grounded Question Answering across Data Lakes

arXiv:2607.12310v1 Announce Type: cross Abstract: While modern question answering (QA) systems excel on clean, schema-aligned corpora, real-world knowledge is rarely so neatly packaged. Answering ques

RepTran: Search-Based Repair of Transformer Models

arXiv:2607.11193v2 Announce Type: replace-cross Abstract: To ensure the overall quality of AI-enabled software, not only traditional software components but also AI components need to be tested and re

researcharxiv-cs-ai

Robustness of Deep Learning Models for PV Power Forecasting under NWP Forecast Errors: A Spatiotemporal and Physically Interpretable Analysis

arXiv:2607.12954v1 Announce Type: cross Abstract: Engineering use of AI forecasting models requires not only high nominal accuracy but also predictable behavior under uncertain inputs. In photovoltaic

researcharxiv-cs-lg

The Computational Basis of Confidence in Large Language Models

arXiv:2607.12447v1 Announce Type: cross Abstract: Reliable confidence -- the probability that a model's own answer is correct -- is essential for the trustworthy deployment of language models. Existin

researcharxiv-cs-ai

Towards Self-Evolving Agents: A Human-Inspired Adaptive Exploration-Exploitation Framework for Genetic Network Programming

arXiv:2607.11913v1 Announce Type: cross Abstract: Recent advancements in agentic AI have increasingly moved toward graph-based methods, driven by the demand for explainable, human-centered, and non-li

UniVR: Thinking in Visual Space for Unified Visual Reasoning

arXiv:2607.12800v1 Announce Type: new Abstract: Learning broad world knowledge directly from raw visual data is a fundamental capability of intelligence. We introduce UniVR, the first investigation in

VisCo: Leveraging Large Language Models as Intrinsic Encoders for Visual Token Compression

arXiv:2607.12756v1 Announce Type: new Abstract: Vision-language models (VLMs) process large numbers of visual tokens, resulting in substantial inference latency and memory overhead. This has motivated

Adaptive Generation of Bias-Eliciting Questions for LLMs

arXiv:2510.12857v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are now widely deployed in user-facing applications, reaching hundreds of millions of users worldwide. Despite th

Beyond wheelchairs and blindfolds: Investigating disability stereotypes in T2I models with INCLUDE-BENCH

arXiv:2607.08515v1 Announce Type: new Abstract: Text-to-image (T2I) models have been shown to exhibit social biases. Prior work has mainly focused on gender, skin tone, and cultural representation wit

CausalDS: Benchmarking Causal Reasoning in Data-Science Agents

arXiv:2607.08093v1 Announce Type: new Abstract: Large language models (LLMs) increasingly act as integrated data-science agents, combining abstract reasoning with advanced tool use. Yet the relevant b

Diagnosing Shape-Prior Shortcuts in Long-Range Single-Shot Fringe Projection Profilometry

model-releasesarxiv-cs-lg

arXiv:2606.17093v2 Announce Type: replace Abstract: Learning-based single-shot fringe projection profilometry (FPP) has been studied almost entirely at close range, and the networks used are evaluated

Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?

arXiv:2607.08514v1 Announce Type: new Abstract: Hand-object interaction (HOI) recognition requires capturing both hand manipulations and object transformations. However, existing video-language models

researcharxiv-cs-cv

Linear Attention Architectures: Mechanisms, Trade-offs, and Cross-Layer Routing

arXiv:2607.07953v1 Announce Type: cross Abstract: Self-attention lets each token retrieve information from the full context, but its quadratic cost in sequence length limits training and inference at

Persona Cartography: Charting Language Model Personality Traits in Weight Space

arXiv:2607.07916v1 Announce Type: new Abstract: Large language models exhibit recurring behavioural patterns -- personas -- that shape generalisation and safety, but we lack reliable tools for decompo

Score Accuracy Along the Forward Diffusion Does Not Certify Numerical Stability in Diffusion Sampling

arXiv:2607.08757v1 Announce Type: cross Abstract: Score matching controls average error under the forward marginals, but a discretized reverse-time sampler evaluates the learned score along its own tr

researcharxiv-cs-lg

Statistical Efficiency and Inference of Quantile Distributional Reinforcement Learning

arXiv:2607.08444v1 Announce Type: cross Abstract: In this paper, we study quantile-based distributional reinforcement learning from the perspective of statistical efficiency. We focus on distributiona

safetyarxiv-cs-lg

Applications

The complexities of patient-centred conversational artificial intelligence

arXiv:2607.08625v1 Announce Type: new Abstract: Consumer-facing health chatbots powered by large language models (LLMs) are increasingly used for symptom assessment. However, chatbot development and e

applicationsarxiv-cs-ai

When LLMs Agree, Are They Right? Auditing Self-Consistency and Cross-Model Agreement as Confidence Signals

arXiv:2607.08065v1 Announce Type: new Abstract: LLM-as-judge (Zheng et al., 2023) is increasingly the default for evaluating AI systems in enterprise pipelines, often scaled to ensembles (Verga et al.

BubbleSH: A Dataset of Rising Bubbles with Deformable Interfaces

model-releasesarxiv-cs-lg

arXiv:2607.07275v1 Announce Type: new Abstract: Bubbly flows exhibit complex multiscale dynamics, with deformable bubbles interacting through the surrounding liquid and giving rise to strongly coupled

CARLA-GS: Decoupling Representation, Reasoning, and Physics Simulation for Autonomous Driving Corner-Case Synthesis

arXiv:2607.07601v1 Announce Type: cross Abstract: Safety evaluation for autonomous driving is dominated by rare, safety-critical interactions, motivating simulators that can deliberately synthesize co

Do Counterfactually Fair Image Classifiers Satisfy Group Fairness? -- A Theoretical and Empirical Study

arXiv:2607.06603v1 Announce Type: cross Abstract: The notion of algorithmic fairness has been actively explored from various aspects of fairness, such as counterfactual fairness (CF) and group fairnes

Healthier LLMs: Retrieval-Augmented Generation for Public Health Question Answering

arXiv:2607.06641v1 Announce Type: cross Abstract: Large language models (LLMs) achieve promising results on medical question answering benchmarks, yet their use in public health is constrained by hall

Institutional Red-Teaming: Deployment Rules, Not Just Models, Causally Shape Multi-Agent AI Safety

arXiv:2607.07695v1 Announce Type: new Abstract: We introduce institutional red-teaming, an evaluation methodology for testing deployment rules in multi-agent AI: hold the agents, objectives, and task

Manual, Joystick, or Haptic Control? An In Vitro Comparison of Navigation Strategies for Robotic Interventional Neuroradiology Procedures

arXiv:2607.07253v1 Announce Type: new Abstract: Objective: To evaluate robotic controller interfaces for interventional neuroradiology procedures in-vitro incorporating a force-sensing platform to ass

safetyarxiv-cs-ro

Practicing with Language Models Cultivates Human Empathic Communication

arXiv:2603.15245v2 Announce Type: replace Abstract: Empathy is central to human connection, yet people often struggle to express it effectively. In blinded evaluations, large language models (LLMs) ge

researcharxiv-cs-cl

Provably Efficient Off-Policy Adversarial Imitation Learning with Convergence Guarantees

arXiv:2405.16668v2 Announce Type: replace Abstract: Adversarial Imitation Learning (AIL) faces challenges with sample inefficiency because of its reliance on sufficient on-policy data to evaluate the

safetyarxiv-cs-lg

Trexplorer Super: Topologically Correct Centerline Tree Tracking of Tubular Objects in CT Volumes

arXiv:2507.10881v2 Announce Type: replace Abstract: Tubular tree structures, such as blood vessels and airways, are essential in human anatomy and accurately tracking them while preserving their topol

researcharxiv-cs-cv

What Predicts Correctness in Text-to-SQL? A Selective-Prediction Study

arXiv:2607.06799v1 Announce Type: cross Abstract: Evaluating uncertainty in AI-generated SQL queries requires estimating whether a query is correct, where correct means it executes to the same result

BabyVision: Visual Reasoning Beyond Language