Search: “evaluation”

SAFARI: An Industrial Benchmark for LLM-Assisted Hazard Analysis and Risk Assessment

arXiv:2609.20584v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly considered for safety-critical engineering, yet their reliability in regulated functional-safety workflows

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

Seismic Site Response Prediction from Sparse Observations Using Finite-Element-Pretrained Latent Dynamics

DGX agent

arXiv:2609.20451v1 Announce Type: new Abstract: Numerical site-response predictions often deviate from observations, yet correcting these discrepancies is difficult because records are limited in both

researcharxiv-cs-lg

18 Sept 2026

Applications

Semantic Layer Induction from Raw Telemetry via Hierarchical LLM and RAG Abstraction

arXiv:2609.19615v1 Announce Type: cross Abstract: Modern applications generate massive volumes of raw telemetry data, but translating those noisy, heterogeneous event streams into actionable business

applicationsarxiv-cs-ai

Should This Case Be Adapted? Prediction Fragmentation Controls Test-Time Adaptation

model-releasesarxiv-cs-cv

arXiv:2609.20700v1 Announce Type: new Abstract: Episodic test-time adaptation resets a frozen segmenter to source weights M_0 on each case and adapts for a fixed step count. A fixed horizon conflates

SIMLIFE: Pattern Understanding for Long-Horizon Human-Agent Partnership

arXiv:2609.19610v1 Announce Type: new Abstract: Understanding humans over long horizons requires agents to infer not only what people need in the moment, but also how routines form, why they repeat, a

Structured Four-Stage Legal Translation: From Natural-Language Traffic Rules to PROLOG

arXiv:2609.20334v1 Announce Type: new Abstract: Traffic regulations are written for human interpretation and therefore rely on shared background knowledge and flexible phrasing, which inherently intro

Summarization Bias: The Directional Collapse of Objective Projection into Told-Mode Labels in Large Language Models --- A Conceptual Framework and Registered Test Protocol

arXiv:2609.20712v1 Announce Type: new Abstract: This paper introduces and operationalizes summarization bias: a proposed systematic tendency of large language models (LLMs) to represent narrative mean

safetyarxiv-cs-cl

Tele-Traversability: Rethinking Traversability for Teleoperated Ground Robots in Terrain Navigation

arXiv:2609.19577v1 Announce Type: new Abstract: Teleoperation, a human-in-the-loop control scheme, allows a human operator to remotely command and guide a mobile robot to navigate in off-road environm

safetyarxiv-cs-ro

The AR Fairness Metamodel: A Structured Framework for Fairness Measures

arXiv:2609.19234v1 Announce Type: cross Abstract: This paper presents the AR fairness metamodel, a framework designed to represent, analyze, and compare different fairness scenarios. The metamodel con

safetyarxiv-cs-ai

Towards a Unified Modality-Agnostic Multimodal Framework for Cognitive Workload Assessment

arXiv:2609.20199v1 Announce Type: new Abstract: Cognitive workload reflects the mental effort required during task performance and is central to the design of adaptive human-machine systems. The use o

researcharxiv-cs-lg

Towards High-DoF Dexterous Manipulation through VLA Post-Training

arXiv:2609.19666v1 Announce Type: new Abstract: Imitation-learned vision--language--action (VLA) foundation models acquire broad manipulation capabilities by scaling robot data across tasks and embodi

safetyarxiv-cs-ro

Applications

Towards Scaling Marine Perception with Synthetic Data

arXiv:2609.20680v1 Announce Type: cross Abstract: Scalable machine learning in challenging underwater environments is strongly limited by the lack of labeled real-world training data. This data is oft

applicationsarxiv-cs-cv

Transformer fault diagnosis using an efficient simulation-driven variational quantum classifier with domain-aware feature encoding

model-releasesarxiv-cs-lg

arXiv:2609.20214v1 Announce Type: cross Abstract: Early transformer fault diagnosis is challenged by nonlinear dissolved-gas interactions, overlapping fault signatures, and limited labeled data, while

TripScore: Aligning LLMs for Real-World Travel Planning via Expert-Calibrated Reward

arXiv:2510.09011v4 Announce Type: replace Abstract: In our deployed travel-planning service, most users give minimal inputs or free-form requests rather than the structured constraint checklists assum

Tools

TypeSafe’s Jev: Can decision models replace LLM judges?

TypeSafe’s Jev classifies, scores, and routes without generating text — up to hundreds of times cheaper than an LLM judge. What that changes for evals, confidence routing, and application architecture

toolsarize-ai

VABench: Measuring Embodied Spatial Intelligence through Visual Demonstrations, Active Perception, and Metric Control

arXiv:2609.19554v1 Announce Type: cross Abstract: Spatial intelligence requires more than describing object locations. Under incomplete observation, models must identify and acquire missing evidence,

local-aiarxiv-cs-cv

What Does Privileged Information Add to On-Policy Self-Distillation?

arXiv:2609.20612v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) lets a language model learn from a frozen copy of itself that sees an answer or a worked solution. Giving the teacher

WiC is Not WSD: A Study on LLMs and Lexical Ambiguity Resolution

arXiv:2609.20593v1 Announce Type: new Abstract: Word-in-Context (WiC) remains challenging for language models, despite recent progress on lexical-semantic tasks. We hypothesise that this difficulty ar

researcharxiv-cs-cl

Agents

WZPlanner: Safe End-to-End Path Planning for Autonomous Driving in Work Zones

arXiv:2609.19393v1 Announce Type: new Abstract: Work zones alter lane geometry through temporary traffic controls and closures that may be absent from on-board maps, challenging autonomous vehicle (AV

agentsarxiv-cs-cv

A Four-Stage Decomposition of Word-Problem Solving and Mechanistic Fragility in LLM Math Reasoning

arXiv:2609.17804v1 Announce Type: new Abstract: Large language models solve grade-school math word problems with high accuracy, yet a single irrelevant clause inserted into the problem can collapse it

researcharxiv-cs-ai

A Systematic Review of NLP for Ghanaian Languages: Datasets, Models, and a Research Roadmap

arXiv:2405.06818v2 Announce Type: replace Abstract: Natural Language Processing (NLP) for Ghana's 73 living indigenous languages remains deeply fragmented, under-resourced, and heavily skewed toward a

safetyarxiv-cs-cl

An operator splitting analysis of Wasserstein--Fisher--Rao gradient flows

arXiv:2511.18060v3 Announce Type: replace-cross Abstract: Wasserstein-Fisher-Rao (WFR) gradient flows have been recently proposed as a powerful sampling tool that combines the advantages of pure Wasse

researcharxiv-cs-lg

BadQubits: An LLM-Based Framework for Static Pre-Execution Detection of Structurally Harmful Quantum Circuits

arXiv:2609.18965v1 Announce Type: cross Abstract: This paper presents BadQubits, an LLM-based framework for static pre-execution detection of structurally harmful OpenQASM 2.0 circuits. The framework

BENCHCOMPASS: From Scores to Signals for Training and Harness Decisions in Payment-Domain LLMs

arXiv:2609.18270v1 Announce Type: new Abstract: Payment operations are a critical financial infrastructure, but the value of large language models in this domain remains unclear because payment rules

Beyond Routine Compliance: Cunning Data Cultivates Safety Vigilance in Large Language Models

arXiv:2609.18515v1 Announce Type: new Abstract: Safety alignment teaches large language models (LLMs) to recognize harmful requests and reject risky instructions. Yet aligned models can fail when harm

BLADE: ReliaBle Dynamic Hardware-Aware SNN-ANN Boundary SeLection for Event-BAseD Object DEtection

arXiv:2609.17562v1 Announce Type: cross Abstract: Hybrid Spiking Neural Network (SNN)-Artificial Neural Network (ANN) architectures combine the energy efficiency of SNNs with the superior detection ac

local-aiarxiv-cs-ai

Bypassing the Rationale: Causal Auditing of Implicit Reasoning in Language Models

arXiv:2602.03994v3 Announce Type: replace-cross Abstract: Chain-of-thought (CoT) prompting is widely used as a reasoning aid and is often treated as a transparency mechanism. Yet behavioral gains unde

Decoder-Agnostic Token Merging for Vision Transformers: A Systematic Study of G2TM

arXiv:2609.18279v1 Announce Type: new Abstract: Vision Transformers (ViTs) have achieved state-of-the-art performance across a range of computer vision tasks, mainly thanks to the self-attention mecha

researcharxiv-cs-cv

Agents

Dependency-Aware Trajectory Refinement for Efficient Multi-Turn Agent Fine-Tuning

arXiv:2609.18417v1 Announce Type: new Abstract: Multi-turn agent trajectories often contain redundant rounds (failed tool calls, parallel sub-queries, verification-only steps) that inflate both traini

agentsarxiv-cs-cl

Disentangling Algorithmic Bias from Archival Artifacts: A Controlled Audit of Vision-Language Model Valuation in Metropolitan Museum Archives

arXiv:2609.17572v1 Announce Type: new Abstract: Auditing vision-language models (VLMs) for societal bias requires distinguishing direct algorithmic valuation disparities from confounders embedded with

local-aiarxiv-cs-lg

English Word Sense Disambiguation in 2026: When the Labels Become the Bottleneck

arXiv:2609.17554v1 Announce Type: new Abstract: In English all-words word sense disambiguation (WSD), the labels, not the models, have become the bottleneck: frontier LLMs are accurate enough that the

Entropy in Conversational AI: Structured Unpredictability as Inferrable Interiority

arXiv:2609.19044v1 Announce Type: new Abstract: Sampling can increase response diversity without producing history-dependent behavior. We formalize a different design target, structured unpredictabili

local-aiarxiv-cs-cl

FairCompressAgent: An Agentic Framework for Fairness-Aware Model Compression for FPGA Deployment

arXiv:2609.17786v1 Announce Type: new Abstract: Fairness-aware model compression requires selecting methods and configurations that balance accuracy, fairness, and deployment cost. These decisions bec

safetyarxiv-cs-ai

Tutorials

Fallacy Benchmarks Measure Scheme Recognition, Not Fallacy Detection

arXiv:2609.18644v1 Announce Type: new Abstract: Fallacy-detection benchmarks pair fallacy classes with a single 'valid' or 'none' class that takes everything data collection did not label as a fallacy

tutorialsarxiv-cs-cl

Applications

ForwardDLO: Model-Based Bimanual Shape Matching of Unconstrained Deformable Linear Objects

arXiv:2609.18455v1 Announce Type: new Abstract: Ropes, cables, and other deformable linear objects appear in tasks from untangling to cable routing and suturing, yet controlling their shape remains a

applicationsarxiv-cs-ro

Google named a Leader in the External Threat Intelligence Service Forrester Wave™

model-releasesgoogle-cloud-ai

At Google, we see firsthand how the speed, scale, and sophistication of cyber threats continue to challenge traditional enterprise defenses. Today’s defenders can’t rely on reactive triage or fragment

Hardware-Free Robotics Laboratories in Mixed Reality

arXiv:2609.18434v1 Announce Type: new Abstract: Teaching robotics relies on screen-based simulation, showing robot motion in an abstract coordinate frame rather than at real scale in the learner's own

safetyarxiv-cs-ro

HearInContext: A Benchmark for Implicit Context in Speech Recognition

arXiv:2609.18680v1 Announce Type: new Abstract: Contextual ASR can benefit from semantic cues or from target words explicitly provided in the context. We introduce HearInContext, a Mandarin--English b

Agents

Hypothesis-Driven Autonomous Materials Synthesis with Multimodal LLM Agents

arXiv:2609.18598v1 Announce Type: cross Abstract: Self-driving laboratories can explore synthesis conditions autonomously, but their decision-making layer is typically a black-box optimizer, and the o

agentsarxiv-cs-ai

JigSync: Gauge-Resolved Synchronization for Jigsaw Reassembly under Unknown Piece Orientation

model-releasesarxiv-cs-cv

arXiv:2609.18379v1 Announce Type: new Abstract: Square jigsaw reassembly requires recovering the spatial arrangement of shuffled fragments from their visual content and pairwise relationships. While r

Label-Confidence-Aware Uncertainty Estimation in Natural Language Generation

arXiv:2412.07255v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) demonstrate remarkable capabilities in generative tasks but pose potential risks due to their tendency to generat

safetyarxiv-cs-ai

LangSelect: Cost-Aware Target-Language Routing for LLM Code Generation

arXiv:2609.18959v1 Announce Type: new Abstract: LLM code-generation systems usually choose a target programming language before decoding and treat that choice as fixed. We show that, for language-flex

Long-Lived Characters, Local Inference: Incremental Memory Maintenance for Game NPCs

arXiv:2609.18935v1 Announce Type: new Abstract: A game character should not have to reread its entire life before every conversation. For locally deployed language-model characters, however, revising

Machine Translation between English and Syriac (East Syriac Dialect) using Statistical Machine Learning

arXiv:2609.18529v1 Announce Type: new Abstract: UNESCO considers the Assyrian (Syriac) language an endangered language. Although Assyrians speak the language worldwide, the speaking population is unce

safetyarxiv-cs-cl

MiST: Mid-Training LLMs for Cybersecurity

arXiv:2609.18496v1 Announce Type: cross Abstract: Cybersecurity combines high-stakes analysis with complex technical language, making it an impactful and challenging domain for LLMs. We present MiST (

MSR: Multiple Subject Reference for Video Generation

model-releasesarxiv-cs-cv

arXiv:2609.18393v1 Announce Type: new Abstract: Conditioning a video generator on multiple images requires preserving appearance while associating each reference with its intended role. We present MSR

Not All Patches Are Equally Forgettable: Spatially Localized Domain Unlearning in Vision-Language Models

arXiv:2609.17790v1 Announce Type: new Abstract: Pre-trained vision-language models (VLMs) exhibit strong cross-domain recognition performance even without additional training. However, this robustness

researcharxiv-cs-cv

Objective vs. Search: Decomposing What Makes a Good Tokeniser

arXiv:2609.19145v1 Announce Type: cross Abstract: Two dominant tokenisation algorithms are used by modern language models: byte-pair encoding (BPE) and UnigramLM. These differ along two orthogonal axe

researcharxiv-cs-ai