Search: “evaluation”

Understanding Backdoor Vulnerabilities in Vertical Federated Learning: The Gap Between Research and Practice

model-releasesarxiv-cs-lg

arXiv:2608.12962v1 Announce Type: new Abstract: Vertical Federated Learning (VFL) enables organizations holding complementary features of shared entities to collaborate and train models. In this setti

14 Aug 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Applications

Validation of Smartphone-Based Photogrammetric 3D Body Scanning for Automated Anthropometric Measurements Compared with a Commercial Depth-Sensor-Based Body Scanner

DGX agent

arXiv:2608.12827v1 Announce Type: new Abstract: 3D body scanning has become an important tool in healthcare applications because of its rapid and non-invasive nature. While smartphone-based photogramm

applicationsarxiv-cs-cv

14 Aug 2026

When AI Is Your Pastor: A Benchmark for Theological Triage and Pastoral Guidance in Large Language Models

arXiv:2608.12324v1 Announce Type: cross Abstract: People increasingly ask large language models (LLMs) for counsel on questions of faith, doctrine, and pastoral care. These questions are not ordinary

14 Aug 2026

Why Do AI Agents Break Rules? How Framing, Context, and Social Signals Shape Compliance

arXiv:2608.12323v1 Announce Type: cross Abstract: Specifying a penalty can paradoxically convert a legal obligation into a cost-benefit calculation that favors violation. We demonstrate that this enfo

14 Aug 2026

Agents

AgonAlpha: Autonomous Alpha Discovery via Prompt Economy and Scalable Agentic Search

arXiv:2608.11250v1 Announce Type: new Abstract: Language models can propose many plausible trading factors, but an autonomous research system must also allocate its evaluation budget, verify its own e

agentsarxiv-cs-ai

Auditing Frame-Level AUC in Weakly Supervised Video Anomaly Detection: Granularity, Resolution, and Scene Bias

model-releasesarxiv-cs-cv

arXiv:2608.11985v1 Announce Type: new Abstract: Frame-level area under the ROC curve (AUC) is the dominant evaluation metric for weakly supervised video anomaly detection (WSVAD). Its standard form me

Automated Borehole Core Analysis with Report-Derived Weak Labels and Supervised Crack Segmentation

arXiv:2608.12252v1 Announce Type: new Abstract: Borehole archives commonly contain core tray photographs and corresponding digital log reports, but no native pixel-level crack annotations. We investig

researcharxiv-cs-cv

Do Not Forget the Obvious - RISC: A Risk-Informed Slice-Coverage Protocol for Safe Autonomous Driving

arXiv:2608.12051v1 Announce Type: new Abstract: Aggregate metrics may not fully reflect performance in insufficiently examined high-risk driving conditions. We propose RISC (Risk-Informed Slice Covera

Explainability in Practice: A Survey of Explainable NLP Across Various Domains

arXiv:2502.00837v3 Announce Type: replace-cross Abstract: Natural Language Processing (NLP) is now embedded in critical sectors including healthcare, finance, and customer relationship management, whe

From Synthesis to Removal: Physics-Grounded Reflection Simulation and Diffusion-Based Video Dereflection

arXiv:2608.11562v1 Announce Type: cross Abstract: Videos captured through glass often contain reflections that degrade visual quality and interfere with downstream vision tasks. Although single-image

Investigating Learner-Aware Design of LLM-Generated Educational Feedback

arXiv:2602.11650v2 Announce Type: replace Abstract: Although large language models (LLMs) show promise for generating educational feedback, it remains unclear how feedback should be designed (e.g., to

researcharxiv-cs-cl

Applications

Methodologies for Improving the Quality of AI Tutoring in K-12 Education

arXiv:2608.11259v1 Announce Type: cross Abstract: Many AI tutors leverage large language models (LLMs) today. Given that LLMs are opaque black boxes, robust evaluation and live experimentation to meas

applicationsarxiv-cs-ai

OEIS Open: How many conjectures can language models turn into theorems?

arXiv:2608.11941v1 Announce Type: new Abstract: We construct OEIS Open, a benchmark based on 492 open mathematical conjectures from the OEIS, formalized in Lean by Tsoukalas et al. Whereas these conje

On Benchmarking Human-Like Intelligence in Machines

arXiv:2502.20502v2 Announce Type: replace Abstract: Recent advances in Artificial Intelligence (AI) have yielded powerful computational models that, by learning from vast amounts of human-generated da

Retry, Switch, or Abstain? Learning Strategy-Aware Tool-Use Policies via Controlled Error Injection

arXiv:2608.11977v1 Announce Type: new Abstract: Tool-using LLM agents are commonly trained and evaluated in environments where tool calls succeed reliably, yet deployed tools can fail transiently, per

Tutorials

Self-Supervised Weighted Image Guided Quantitative MRI Super-Resolution

arXiv:2512.17612v2 Announce Type: replace Abstract: Object: To present and evaluate Self-supervised Weighted Image Guided quantitative MRI Super-Resolution (SWIG qMRI SR), a physics-informed framework

tutorialsarxiv-cs-cv

SteeringSafety: Benchmarking Representation Steering in LLMs Across Safety Perspectives

arXiv:2509.13450v3 Announce Type: replace Abstract: We introduce SteeringSafety, a benchmark for evaluating representation steering methods across nine safety perspectives spanning 18 datasets. While

Topology-Aware Query Selection for Surgical Instrument Instance Segmentation

arXiv:2608.11607v1 Announce Type: new Abstract: Accurate foreground masks can still form an incorrect surgical-instrument instance set: duplicate, fragmented, merged, missed, or empty-frame prediction

Hardware

worldproof: diagnosing where world-model predictions break and a measurement of when pixel metrics stop being able to rank models at all [P]

hardwarer-machinelearning

I've been building an open-source tool for diagnosing world models, the kind that predict future frames from a starting context and a sequence of actions. It compares a rollout against ground truth an

APCReg: Anatomical-Prior-Guided Coarse-to-Fine CBCT--IOS Registration via Multi-View Projection and Reliability-Controlled Residual Correction

arXiv:2608.09993v1 Announce Type: cross Abstract: Registration between cone-beam computed tomography (CBCT) and intraoral scans (IOS) is essential for patient-specific surgical planning. However, disp

Automated Data Enrichment using Confidence-Aware Fine-Grained Debate among Open-Source LLMs for Mental Health and Online Safety

arXiv:2512.06227v3 Announce Type: replace Abstract: Real-world indicators play an important role in many Natural Language Processing (NLP) applications, such as life events for mental health analysis

safetyarxiv-cs-cl

Compositional Benchmark Synthesis for Hierarchical Human Action Recognition

arXiv:2608.10765v1 Announce Type: new Abstract: Recognizing human behavior across levels of abstraction, from atomic actions to long-horizon intentions, requires data annotated along a semantic hierar

Decision-Aware Approximation of Belief Functions for Evidential Combinatorial Optimization

arXiv:2608.10650v1 Announce Type: new Abstract: Reducing the number of focal elements of a mass function is classically driven by an intrinsic distance, such as Jaccard or Jousselme, that keeps the ap

Do AI weather models miss extremes?

arXiv:2608.09972v1 Announce Type: cross Abstract: First-generation AI weather models are often reported to underperform at extremes, mostly in reanalysis-based evaluations of deterministic regression

safetyarxiv-cs-ai

Do LLM Recommenders Know When They're Hallucinating? Auditing Confidence Calibration in Catalog Faithfulness

model-releasesarxiv-cs-cl

arXiv:2608.10008v1 Announce Type: cross Abstract: LLM recommenders for top-K item suggestion regularly emit titles outside the target catalog. Prior audits measure this as a binary out-of-domain rate;

DSAgentBench: Can Agents Automate End-to-End Data-Science Workflows in Real Computer Environments?

arXiv:2608.10366v1 Announce Type: new Abstract: Real-world data science involves long-horizon workflows that span data wrangling, exploration, modeling, visualization, and validation, and require coor

EVIL-Detect for NLPCC 2026 Shared Task 6: LLM-Generated Text Detection

arXiv:2608.10698v1 Announce Type: new Abstract: The rapid development of large language models (LLMs) has increased the need for reliable detection of LLM-generated text, especially in realistic Chine

researcharxiv-cs-cl

Local Ai

FITTER: Vocabulary-Agnostic Cross-Domain Inference on Temporal Knowledge Graphs

arXiv:2608.10668v1 Announce Type: new Abstract: Temporal knowledge graphs are central to many uses of the Semantic Web, but existing completion methods assume the entities, relation names, and timesta

local-aiarxiv-cs-ai

FoR-SALE: Frame of Reference-guided Spatial Adjustment in LLM-based Diffusion Editing

arXiv:2509.23452v2 Announce Type: replace-cross Abstract: Current text-to-image generation models, even state-of-the-art models, exhibit a significant performance gap when spatial expressions are desc

safetyarxiv-cs-cl

FormStruct-Bench:A Hierarchical and Diagnostic Benchmark for Table-Form Document Structure Recognition

model-releasesarxiv-cs-cv

arXiv:2608.10396v1 Announce Type: new Abstract: Transforming table-form documents into machine-processable records requires recovering not only their visible content but also the multilevel structure

HexEval: An Evidence-Driven Hexagonal Framework for Multidimensional Scholar Assessment

arXiv:2608.10584v1 Announce Type: new Abstract: Scholar assessment plays a fundamental role in faculty recruitment, funding allocation, academic promotion, and talent discovery. Existing scholar asses

Invertible Logits Transformation for Accuracy-Preserving Post-Hoc Uncertainty Calibration

model-releasesarxiv-cs-lg

arXiv:2608.10372v1 Announce Type: new Abstract: Post-hoc calibration aligns a classifier's predicted confidences with its empirical accuracy without retraining. An ideal calibrator should correct nonl

Is This Your Final Answer? Cross-Contextual Consistency as a Measure of LLM Credibility

arXiv:2608.10315v1 Announce Type: cross Abstract: Large language models (LLMs) are powerful black-box systems, making it difficult to discern whether their answers reflect stable internal beliefs or s

Locally Deployable Small Language Models for Emergency Department Decision Support: A Systematic Benchmark of Fine-Tuning Strategies

arXiv:2608.10273v1 Announce Type: cross Abstract: Deploying large language models (LLMs) for decision support in emergency departments (EDs) faces two major challenges: privacy risks of transmitting p

On Solomonoff Induction in Large Language Models and the Limits of Self-Improving: The Singularity Is Not Near Without Symbolic Model Synthesis

arXiv:2601.05280v3 Announce Type: replace-cross Abstract: On the one hand, the question of whether large language models (LLMs) are Solomonoff induction estimators has become an explicit question at t

On the Limitations of Cross-Lingual Consistency in Multilingual Text-to-image Generation

arXiv:2608.11002v1 Announce Type: cross Abstract: Text-to-image (T2I) generation has achieved remarkable progress in recent years. However, existing research has largely focused on English-only settin

Order Matters: LVLMs as Judges for Temporal Reasoning in Image Sequences

arXiv:2608.10908v1 Announce Type: cross Abstract: As generative multimedia evolves from static image synthesis to complex, interleaved visual narratives, a foundational bottleneck has emerged: the jud

researcharxiv-cs-cl

Post-Hoc Sparse Coding of Latent Communication Between Vision-Language Model Agents

arXiv:2608.10198v1 Announce Type: new Abstract: Latent-space communication allows heterogeneous vision-language model agents to exchange continuous representations without serializing visual and reaso

PRMU: A Corpus-Free Benchmark for Person-Centric Knowledge Unlearning in Multimodal Large Language Models

model-releasesarxiv-cs-cv

arXiv:2608.11149v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have demonstrated remarkable capabilities in storing and recalling rich person-related knowledge, raising incre

REDAgentBench: Executable Red Teaming and Faithful Measurement of LLM Agent Systems

arXiv:2608.10669v1 Announce Type: new Abstract: Large language model (LLM) agents combine language-based reasoning with external tools to perform complex tasks. Adversarial inputs can exploit interact

ReLTEx: Reliable LLM-based Taxonomy Expansion

arXiv:2608.10970v1 Announce Type: cross Abstract: Recent advances in Large Language Models (LLMs) have demonstrated strong capabilities in generating semantically relevant concepts and relations, maki

Applications

Retrieval-Augmented Vision Foundation Models for Robust Leukemia Cell Classification across Multiple Microscopy Datasets

arXiv:2608.10657v1 Announce Type: cross Abstract: Leukemia cell image classification is challenged by real-world domain shifts from acquisition, staining, illumination, and site protocols, causing sin

applicationsarxiv-cs-cv

RLMOpt: Adaptive Prompt Optimization via Recursive Language Models

arXiv:2608.10471v1 Announce Type: new Abstract: Prompt optimizers automate the search for prompts that improve language-model performance, but existing methods rely on a predefined optimization proced

SBCO: Self-Supervised, Verifier-Grounded Harness Optimization For Planning Agents

arXiv:2608.10157v1 Announce Type: new Abstract: Self-improving agents seek to reduce the human engineering effort behind AI systems by enabling them to evolve and self-improve their performance over t

safetyarxiv-cs-ai

Agents

Seeing above the waves: A modular sensing framework for data acquisition at sea

arXiv:2608.10997v1 Announce Type: new Abstract: Advancing autonomy for surface vessels requires systematic evaluation of their sensing and perception subsystems. Yet, maritime environments impose uniq

agentsarxiv-cs-ro

SeFaR: Semantic Feature-aware Robustness Testing of Deep Neural Networks

arXiv:2608.10289v1 Announce Type: new Abstract: Deep neural networks are increasingly deployed in safety-critical domains as perception modules, where failures are often caused due to rare and under-r

Selective Prediction Reduces the Negative Effects of Automation Bias Overall but Increases False Negatives

arXiv:2508.07617v2 Announce Type: replace-cross Abstract: AI has the potential to augment human decision making. However, even high-performing models can produce inaccurate predictions when deployed.

safetyarxiv-cs-ai

Self-Normalized Inference for Constant-Stepsize Temporal-Difference Learning under Markovian Sampling

arXiv:2608.10896v1 Announce Type: cross Abstract: Constant-stepsize temporal-difference (TD) learning is attractive for policy evaluation, but inference from a single Markov trajectory must account fo

safetyarxiv-cs-lg