Search: “evaluation”

Automated Borehole Core Analysis with Report-Derived Weak Labels and Supervised Crack Segmentation

arXiv:2608.12252v1 Announce Type: new Abstract: Borehole archives commonly contain core tray photographs and corresponding digital log reports, but no native pixel-level crack annotations. We investig

researcharxiv-cs-cv

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

Do Not Forget the Obvious - RISC: A Risk-Informed Slice-Coverage Protocol for Safe Autonomous Driving

DGX agent

arXiv:2608.12051v1 Announce Type: new Abstract: Aggregate metrics may not fully reflect performance in insufficiently examined high-risk driving conditions. We propose RISC (Risk-Informed Slice Covera

safetyarxiv-cs-cv

13 Aug 2026

Explainability in Practice: A Survey of Explainable NLP Across Various Domains

arXiv:2502.00837v3 Announce Type: replace-cross Abstract: Natural Language Processing (NLP) is now embedded in critical sectors including healthcare, finance, and customer relationship management, whe

From Synthesis to Removal: Physics-Grounded Reflection Simulation and Diffusion-Based Video Dereflection

arXiv:2608.11562v1 Announce Type: cross Abstract: Videos captured through glass often contain reflections that degrade visual quality and interfere with downstream vision tasks. Although single-image

Investigating Learner-Aware Design of LLM-Generated Educational Feedback

arXiv:2602.11650v2 Announce Type: replace Abstract: Although large language models (LLMs) show promise for generating educational feedback, it remains unclear how feedback should be designed (e.g., to

researcharxiv-cs-cl

Applications

Methodologies for Improving the Quality of AI Tutoring in K-12 Education

arXiv:2608.11259v1 Announce Type: cross Abstract: Many AI tutors leverage large language models (LLMs) today. Given that LLMs are opaque black boxes, robust evaluation and live experimentation to meas

applicationsarxiv-cs-ai

OEIS Open: How many conjectures can language models turn into theorems?

arXiv:2608.11941v1 Announce Type: new Abstract: We construct OEIS Open, a benchmark based on 492 open mathematical conjectures from the OEIS, formalized in Lean by Tsoukalas et al. Whereas these conje

On Benchmarking Human-Like Intelligence in Machines

arXiv:2502.20502v2 Announce Type: replace Abstract: Recent advances in Artificial Intelligence (AI) have yielded powerful computational models that, by learning from vast amounts of human-generated da

Retry, Switch, or Abstain? Learning Strategy-Aware Tool-Use Policies via Controlled Error Injection

arXiv:2608.11977v1 Announce Type: new Abstract: Tool-using LLM agents are commonly trained and evaluated in environments where tool calls succeed reliably, yet deployed tools can fail transiently, per

Tutorials

Self-Supervised Weighted Image Guided Quantitative MRI Super-Resolution

arXiv:2512.17612v2 Announce Type: replace Abstract: Object: To present and evaluate Self-supervised Weighted Image Guided quantitative MRI Super-Resolution (SWIG qMRI SR), a physics-informed framework

tutorialsarxiv-cs-cv

SteeringSafety: Benchmarking Representation Steering in LLMs Across Safety Perspectives

arXiv:2509.13450v3 Announce Type: replace Abstract: We introduce SteeringSafety, a benchmark for evaluating representation steering methods across nine safety perspectives spanning 18 datasets. While

Topology-Aware Query Selection for Surgical Instrument Instance Segmentation

arXiv:2608.11607v1 Announce Type: new Abstract: Accurate foreground masks can still form an incorrect surgical-instrument instance set: duplicate, fragmented, merged, missed, or empty-frame prediction

Hardware

worldproof: diagnosing where world-model predictions break and a measurement of when pixel metrics stop being able to rank models at all [P]

hardwarer-machinelearning

I've been building an open-source tool for diagnosing world models, the kind that predict future frames from a starting context and a sequence of actions. It compares a rollout against ground truth an

APCReg: Anatomical-Prior-Guided Coarse-to-Fine CBCT--IOS Registration via Multi-View Projection and Reliability-Controlled Residual Correction

arXiv:2608.09993v1 Announce Type: cross Abstract: Registration between cone-beam computed tomography (CBCT) and intraoral scans (IOS) is essential for patient-specific surgical planning. However, disp

Automated Data Enrichment using Confidence-Aware Fine-Grained Debate among Open-Source LLMs for Mental Health and Online Safety

arXiv:2512.06227v3 Announce Type: replace Abstract: Real-world indicators play an important role in many Natural Language Processing (NLP) applications, such as life events for mental health analysis

safetyarxiv-cs-cl

Compositional Benchmark Synthesis for Hierarchical Human Action Recognition

arXiv:2608.10765v1 Announce Type: new Abstract: Recognizing human behavior across levels of abstraction, from atomic actions to long-horizon intentions, requires data annotated along a semantic hierar

Decision-Aware Approximation of Belief Functions for Evidential Combinatorial Optimization

arXiv:2608.10650v1 Announce Type: new Abstract: Reducing the number of focal elements of a mass function is classically driven by an intrinsic distance, such as Jaccard or Jousselme, that keeps the ap

Do AI weather models miss extremes?

arXiv:2608.09972v1 Announce Type: cross Abstract: First-generation AI weather models are often reported to underperform at extremes, mostly in reanalysis-based evaluations of deterministic regression

Do LLM Recommenders Know When They're Hallucinating? Auditing Confidence Calibration in Catalog Faithfulness

model-releasesarxiv-cs-cl

arXiv:2608.10008v1 Announce Type: cross Abstract: LLM recommenders for top-K item suggestion regularly emit titles outside the target catalog. Prior audits measure this as a binary out-of-domain rate;

DSAgentBench: Can Agents Automate End-to-End Data-Science Workflows in Real Computer Environments?

arXiv:2608.10366v1 Announce Type: new Abstract: Real-world data science involves long-horizon workflows that span data wrangling, exploration, modeling, visualization, and validation, and require coor

EVIL-Detect for NLPCC 2026 Shared Task 6: LLM-Generated Text Detection

arXiv:2608.10698v1 Announce Type: new Abstract: The rapid development of large language models (LLMs) has increased the need for reliable detection of LLM-generated text, especially in realistic Chine

researcharxiv-cs-cl

Local Ai

FITTER: Vocabulary-Agnostic Cross-Domain Inference on Temporal Knowledge Graphs

arXiv:2608.10668v1 Announce Type: new Abstract: Temporal knowledge graphs are central to many uses of the Semantic Web, but existing completion methods assume the entities, relation names, and timesta

local-aiarxiv-cs-ai

FoR-SALE: Frame of Reference-guided Spatial Adjustment in LLM-based Diffusion Editing

arXiv:2509.23452v2 Announce Type: replace-cross Abstract: Current text-to-image generation models, even state-of-the-art models, exhibit a significant performance gap when spatial expressions are desc

safetyarxiv-cs-cl

FormStruct-Bench:A Hierarchical and Diagnostic Benchmark for Table-Form Document Structure Recognition

model-releasesarxiv-cs-cv

arXiv:2608.10396v1 Announce Type: new Abstract: Transforming table-form documents into machine-processable records requires recovering not only their visible content but also the multilevel structure

HexEval: An Evidence-Driven Hexagonal Framework for Multidimensional Scholar Assessment

arXiv:2608.10584v1 Announce Type: new Abstract: Scholar assessment plays a fundamental role in faculty recruitment, funding allocation, academic promotion, and talent discovery. Existing scholar asses

Invertible Logits Transformation for Accuracy-Preserving Post-Hoc Uncertainty Calibration

model-releasesarxiv-cs-lg

arXiv:2608.10372v1 Announce Type: new Abstract: Post-hoc calibration aligns a classifier's predicted confidences with its empirical accuracy without retraining. An ideal calibrator should correct nonl

Is This Your Final Answer? Cross-Contextual Consistency as a Measure of LLM Credibility

arXiv:2608.10315v1 Announce Type: cross Abstract: Large language models (LLMs) are powerful black-box systems, making it difficult to discern whether their answers reflect stable internal beliefs or s

Locally Deployable Small Language Models for Emergency Department Decision Support: A Systematic Benchmark of Fine-Tuning Strategies

arXiv:2608.10273v1 Announce Type: cross Abstract: Deploying large language models (LLMs) for decision support in emergency departments (EDs) faces two major challenges: privacy risks of transmitting p

On Solomonoff Induction in Large Language Models and the Limits of Self-Improving: The Singularity Is Not Near Without Symbolic Model Synthesis

arXiv:2601.05280v3 Announce Type: replace-cross Abstract: On the one hand, the question of whether large language models (LLMs) are Solomonoff induction estimators has become an explicit question at t

On the Limitations of Cross-Lingual Consistency in Multilingual Text-to-image Generation

arXiv:2608.11002v1 Announce Type: cross Abstract: Text-to-image (T2I) generation has achieved remarkable progress in recent years. However, existing research has largely focused on English-only settin

Order Matters: LVLMs as Judges for Temporal Reasoning in Image Sequences

arXiv:2608.10908v1 Announce Type: cross Abstract: As generative multimedia evolves from static image synthesis to complex, interleaved visual narratives, a foundational bottleneck has emerged: the jud

researcharxiv-cs-cl

Post-Hoc Sparse Coding of Latent Communication Between Vision-Language Model Agents

arXiv:2608.10198v1 Announce Type: new Abstract: Latent-space communication allows heterogeneous vision-language model agents to exchange continuous representations without serializing visual and reaso

PRMU: A Corpus-Free Benchmark for Person-Centric Knowledge Unlearning in Multimodal Large Language Models

model-releasesarxiv-cs-cv

arXiv:2608.11149v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have demonstrated remarkable capabilities in storing and recalling rich person-related knowledge, raising incre

REDAgentBench: Executable Red Teaming and Faithful Measurement of LLM Agent Systems

arXiv:2608.10669v1 Announce Type: new Abstract: Large language model (LLM) agents combine language-based reasoning with external tools to perform complex tasks. Adversarial inputs can exploit interact

ReLTEx: Reliable LLM-based Taxonomy Expansion

arXiv:2608.10970v1 Announce Type: cross Abstract: Recent advances in Large Language Models (LLMs) have demonstrated strong capabilities in generating semantically relevant concepts and relations, maki

Applications

Retrieval-Augmented Vision Foundation Models for Robust Leukemia Cell Classification across Multiple Microscopy Datasets

arXiv:2608.10657v1 Announce Type: cross Abstract: Leukemia cell image classification is challenged by real-world domain shifts from acquisition, staining, illumination, and site protocols, causing sin

applicationsarxiv-cs-cv

RLMOpt: Adaptive Prompt Optimization via Recursive Language Models

arXiv:2608.10471v1 Announce Type: new Abstract: Prompt optimizers automate the search for prompts that improve language-model performance, but existing methods rely on a predefined optimization proced

SBCO: Self-Supervised, Verifier-Grounded Harness Optimization For Planning Agents

arXiv:2608.10157v1 Announce Type: new Abstract: Self-improving agents seek to reduce the human engineering effort behind AI systems by enabling them to evolve and self-improve their performance over t

Agents

Seeing above the waves: A modular sensing framework for data acquisition at sea

arXiv:2608.10997v1 Announce Type: new Abstract: Advancing autonomy for surface vessels requires systematic evaluation of their sensing and perception subsystems. Yet, maritime environments impose uniq

agentsarxiv-cs-ro

SeFaR: Semantic Feature-aware Robustness Testing of Deep Neural Networks

arXiv:2608.10289v1 Announce Type: new Abstract: Deep neural networks are increasingly deployed in safety-critical domains as perception modules, where failures are often caused due to rare and under-r

Selective Prediction Reduces the Negative Effects of Automation Bias Overall but Increases False Negatives

arXiv:2508.07617v2 Announce Type: replace-cross Abstract: AI has the potential to augment human decision making. However, even high-performing models can produce inaccurate predictions when deployed.

Self-Normalized Inference for Constant-Stepsize Temporal-Difference Learning under Markovian Sampling

arXiv:2608.10896v1 Announce Type: cross Abstract: Constant-stepsize temporal-difference (TD) learning is attractive for policy evaluation, but inference from a single Markov trajectory must account fo

safetyarxiv-cs-lg

Templated or fully Synthetic? Prompt construction as a confound in measuring LLM political stance beyond writing assistance

arXiv:2608.11008v1 Announce Type: new Abstract: Political stance detection in LLMs has long been dominated by closed-ended, multiple-choice political survey questions---originally designed for humans,

safetyarxiv-cs-cl

The CASE Framework: A Multi-Disciplinary Control Architecture for Governing Enterprise Agentic AI

arXiv:2608.10153v1 Announce Type: new Abstract: Enterprises are deploying autonomous AI agents faster than they can govern them, and prevailing approaches stretch a single discipline, typically DevSec

Toward a Theory of Value in AI Alignment

arXiv:2608.10327v1 Announce Type: new Abstract: Can AI systems be aligned to human values? The popularization of large language models (LLMs) and multi-modal foundation models has seen a rise in harms

VIDS-Seg: Towards Reliable Uncertainty Quantification in Pediatric Cardiac Ultrasound Segmentation

arXiv:2608.10903v1 Announce Type: new Abstract: Reliable clinical deployment of machine learning requires models that know when they are likely to fail, particularly for subgroups underrepresented in

Agents

You chose the best model. Why is your agent still failing?

Public benchmarks can show how a model performs in general. Production reliability depends on the context and harness around it, which only your team can evaluate against its own data, workflows, and

agentsarize-ai

Applications

Accurate but Natural? Diagnosing Grammatical and Idiomatic Gaps in Japanese EFL Writing