Search: “evaluation”

FrameBench:A Language Understanding Benchmark Based on Frame Semantics

arXiv:2609.03370v1 Announce Type: new Abstract: In frame semantics, sentence comprehension is assumed to proceed by relating lexical meaning to background knowledge called semantic frames, thereby ena

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

GeoNatureAgent Benchmark: Benchmarking LLM Agents for Environmental Geospatial Analysis Across Frontier and Open-Weight Foundation Models

DGX agent

arXiv:2606.12821v2 Announce Type: replace Abstract: Environmental scientists spend disproportionate effort on data wrangling rather than analysis. New AI agents can be a helpful tool, but no benchmark

model-releasesarxiv-cs-ai

4 Sept 2026

GPS-Bench: A Governance Policy Benchmark for Automating Policy Analysis

arXiv:2609.03553v1 Announce Type: new Abstract: Policy analysis requires more than predicting whether a proposal will pass: it requires identifying who will be affected, how those actors respond, and

Mind the Gap: Robustness Risks in PII Detection Systems

model-releasesarxiv-cs-lg

arXiv:2609.03464v1 Announce Type: new Abstract: Personally Identifiable Information (PII) detection is a foundational component of data protection infrastructure where missed entities constitute direc

PACE: Towards Surfacing Hidden Conflicts in User Requests

arXiv:2609.03293v1 Announce Type: new Abstract: Personalized assistants should not only comply with user requests but also assess whether those requests are appropriate given the user's current circum

Privacy Leakage in Federated Learning: Gradient-Based Client Identity Inference and Defenses for Inertial Sensing in Vehicular Edge Networks

model-releasesarxiv-cs-lg

arXiv:2609.02971v1 Announce Type: cross Abstract: As vehicular networks move toward 5G/6G edge intelligence, federated learning (FL) is widely promoted as a privacy-preserving way for vehicles and inf

Hardware

RobustSeiz: An Open-Source Framework for Benchmarking the Robustness of EEG Seizure Detection Models

arXiv:2609.04007v1 Announce Type: new Abstract: Despite strong performance on held-out electroencephalography (EEG) data, seizure detectors may fail under real-world acquisition variability, artifacts

hardwarearxiv-cs-lg

Selective Hypergraph Refinement for Frozen Graph Clustering

arXiv:2609.03265v1 Announce Type: new Abstract: Existing graph-clustering methods typically improve clustering performance by optimizing model parameters and node representations. Effective means of f

Uncertainty Is Not a Safety Net for Clinical VQA, but Can It Anticipate Model Failure?

arXiv:2606.16583v2 Announce Type: replace Abstract: Safe deployment of clinical vision-language models (VLMs) requires reliable uncertainty estimation (UE): a signal indicating when predictions should

Understanding Autonomous Driving Datasets by Describing Differences between Image Subsets in Natural Language

arXiv:2609.03677v1 Announce Type: cross Abstract: Understanding the composition of large-scale autonomous driving datasets is essential for safety, robustness, and reliable operation across domains. F

VisCAD: A Foundation Model Suite with Multimodal Industrial CAD Intelligence

arXiv:2609.03811v1 Announce Type: cross Abstract: AI-assisted computer-aided design (CAD) for industrial products involves two challenging phases. Part-level generation maps diverse forms of user inte

researcharxiv-cs-cl

When Models Edit Too Much: On the Fidelity of Minimal Code Edits

arXiv:2609.04061v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used to edit existing code, but correctness alone is not enough: useful repairs should also be minimal,

WISE: World-model-guided Imagination Scheduling for Efficient Post-training of Vision-Language-Action Models

arXiv:2609.03681v1 Announce Type: new Abstract: Post-training VLA policies typically rely on supervised fine-tuning with costly expert demonstrations or reinforcement learning with expensive and poten

safetyarxiv-cs-ro

Beyond Appearance: Can Multimodal Large Language Models Exploit Vertical Structure for Remote Sensing Natural Scene Understanding?

model-releasesarxiv-cs-cv

arXiv:2605.25784v2 Announce Type: replace Abstract: Multimodal large language models (MLLMs) have advanced rapidly in remote-sensing analysis, yet existing evaluations remain predominantly 2D-centric.

Candidate Generation and Definition-Guided Verification for Sentence-Level Depression Symptom Recognition

arXiv:2609.01833v1 Announce Type: new Abstract: Sentence-level recognition of depression symptoms is challenging because similar expressions can differ in symptom relevance, and language-model inferen

Cite or Decline: A Strict Course-Grounded Chatbot for STEM Lecture Videos

arXiv:2609.01846v1 Announce Type: new Abstract: Recorded lecture videos, often enhanced with search and summarization features, are a standard study resource. However, students cannot easily ask cours

CORAL: An LLM-Native Harness for Production Recommender Systems

arXiv:2609.02730v1 Announce Type: new Abstract: Production recommender systems shape what billions of people see, and sustaining their performance requires continual optimization: as content, user beh

CrashDiffuser: VLM-Guided Collision Intent Reasoning for Fine-Grained Safety-Critical Traffic Scenario Generation

arXiv:2609.02270v1 Announce Type: cross Abstract: Generating safety-critical scenarios is essential for evaluating autonomous driving systems. However, existing generators primarily focus on inducing

From Reweighting to Rewriting: Unlocking the Intervention Effects of Influential Samples in Training Data Attribution

arXiv:2609.02771v1 Announce Type: cross Abstract: Training data attribution (TDA) aims to identify training examples that shape model behavior, but its intervention value depends on both which example

HyperMC: Multi-Fidelity Hyperparameter Tuning for Stochastic Gradient MCMC

arXiv:2609.02138v1 Announce Type: cross Abstract: Stochastic gradient Markov chain Monte Carlo (SGMCMC) methods enable scalable Bayesian inference, but their performance depends strongly on hyperparam

Meet E-Commerce Bench, a new benchmark for long-horizon autonomous business operations. 🚀 Agents start with ¥100,000 to run online stores f…

Meet E-Commerce Bench, a new benchmark for long-horizon autonomous business operations. 🚀 Agents start with ¥100,000 to run online stores for 365 days, handling sourcing, negotiation, pricing, promoti

model-releasesqwen--x

On-Policy Distillation Meets Off-Policy GRPO: Training Compact Instruction-Following Rerankers

arXiv:2609.01947v1 Announce Type: cross Abstract: Compact instruction-following rerankers are attractive for deployment, but conventional distillation pipelines typically train students by offline imi

Orthogonal Ensembles and Tested Explanations for Performer-Independent Body-Motion Emotion Recognition

arXiv:2609.02510v1 Announce Type: new Abstract: We study body-only, 12-class acted-emotion classification from skeleton motion under leave-performer-out (LPO) evaluation, a hard, underdetermined setti

safetyarxiv-cs-cv

PhoenixNest-Video: Evidence-Grounded Multimodal Agent Framework for Automated Video Interview Assessment

arXiv:2609.02231v1 Announce Type: new Abstract: Interview assessment requires per-criterion judgments grounded in behavioral evidence, yet surging applicant volumes have made human-only evaluation cos

Probing Cultural Signals in Large Language Models through Author Profiling

arXiv:2603.16749v3 Announce Type: replace Abstract: Large language models (LLMs) are increasingly deployed in applications with societal impact, raising concerns about the cultural biases they encode.

Quantum Speedups for Sampling and Non-convex Optimization with Stochastic Oracles

arXiv:2504.03626v2 Announce Type: replace-cross Abstract: We present quantum speedups for sampling from distributions of the form pipropto e^{-f} on R^d. We consider two stochastic oracle models: a st

UnCapsTSR: An Unsupervised Transformer-based Image Super-Resolution Approach for Capsule Endoscopy Images

arXiv:2609.02476v1 Announce Type: new Abstract: Wireless Capsule Endoscopy (WCE) captures and streams video while passing through a patient's Gastrointestinal (GI) tract and is used to examine its irr

applicationsarxiv-cs-cv

Local Ai

Unfolding the Leech Lattice: Fused Multi-Shell Decoding and VRAM Layouts for 2-Bit LLM Weights

arXiv:2609.02652v1 Announce Type: new Abstract: Leech-lattice vector quantization holds the strongest reported 2-bit quality under its own evaluation protocol. Its kernel decodes one shell; we found n

local-aiarxiv-cs-lg

Using Large Language Models for Legal Decision-Making in Austrian Value-Added Tax Law: A Comparative Study

arXiv:2507.08468v2 Announce Type: replace Abstract: This paper provides an experimental evaluation of the capability of large language models (LLMs) to assist in legal decision-making within the frame

applicationsarxiv-cs-cl

When Persona Attributes Improve Population Alignment in Large Language Models

arXiv:2609.02526v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly used to predict the responses of human participants in survey panels. Towards that goal, persona prompting

Wound3DAssist: A Practical Framework for 3D Wound Assessment

arXiv:2508.17635v2 Announce Type: replace Abstract: Managing chronic wounds remains a major healthcare challenge, with clinical assessment often relying on subjective and time-consuming manual documen

applicationsarxiv-cs-cv

Agents

A Compact Robotic Finger with 2-DoF MCP Joint Embedding DoF-Selective Passive Continuously Variable Transmission for Wide Force-Speed Operating Range

arXiv:2609.00769v1 Announce Type: new Abstract: This letter presents a compact two-degree-of-freedom (DoF) robotic finger with a flexion-selective passive continuously variable transmission (CVT) to a

agentsarxiv-cs-ro

Agents

A Hybrid Insider Threat Detection Framework Combining Multi-Agent Simulation, Layered SIEM Correlation, and Theory-of-Mind Reasoning

arXiv:2601.04243v2 Announce Type: replace-cross Abstract: This paper presents a hybrid insider threat detection framework for enterprise environments, integrating multi-agent simulation, layered SIEM

agentsarxiv-cs-ai

AI Morbidity and Mortality: A Framework for Clinical AI Failure Review

arXiv:2609.00076v1 Announce Type: new Abstract: Clinical artificial intelligence is increasingly embedded in real-world care, yet existing safety mechanisms are poorly suited to reconstructing and lea

AI Should Not Only Be Helpful. It Should Be Contingent. Artificial Intimacy, Sycophancy, and the Future of Social Learning

arXiv:2609.00211v1 Announce Type: new Abstract: Conversational artificial intelligence is increasingly embedded in everyday social environments, where it functions as both an informational tool and a

Beyond Blind Compliance: Benchmarking Task Verification in OCR Reasoning

model-releasesarxiv-cs-cv

arXiv:2609.00232v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have achieved strong performance on OCR-centric document understanding and text-rich visual reasoning benchmark

Births are difficult to predict even with rich survey and full-population register data

arXiv:2609.01194v1 Announce Type: new Abstract: Major life events have proven difficult to predict. Does this reflect limits of theory, data, and algorithms, or the large role of chance? We examine on

BS: Take the Hint - Interactive Multitracer PET/CT Lesion Segmentation with a Scribble-Conditioned ResEnc U-Net

arXiv:2609.01554v1 Announce Type: cross Abstract: Automated lesion segmentation in whole-body PET/CT is complicated by the variety of physiological tracer uptake patterns and by the differing appearan

researcharxiv-cs-ai

CHARM: Character Hallucination for Multicultural Role Play Benchmark

arXiv:2609.01352v1 Announce Type: cross Abstract: Role-playing large language models (LLMs) are expected to adopt a character's style while also respecting that character's knowledge boundaries. Prior

ClinTraceBench: Source-Verifiable Longitudinal Clinical Reasoning over EHR-Derived Dialogues

arXiv:2609.01111v1 Announce Type: new Abstract: Clinical LLM assistants must reason over multi-visit patient trajectories, yet whether the compact history representations used to scale them---retrieva

CordisBench: Can Language Models Reason About Component Lifecycles in Dynamic Agent Harnesses?

arXiv:2609.01600v1 Announce Type: cross Abstract: Dynamic agent harnesses let language models change the software that shapes their own execution. This flexibility brings a new reasoning burden: a loc

Counterfactual Fragility Certificates: Exposing High-Confidence Brittleness under Structured Evidence Failure

arXiv:2609.00366v1 Announce Type: cross Abstract: High test accuracy and good aggregate calibration do not show whether an individual prediction is structurally supported by its evidence. In tabular d

researcharxiv-cs-ai

CoVer: Conflict-Aware Claim Verification

arXiv:2609.00508v1 Announce Type: new Abstract: Social media fact-checking has long been challenged by evidence-level and aggregation-level conflicts, where erroneous evidence mimics authoritative new

applicationsarxiv-cs-ai

Agents

EAAE: Energy-Aware Autonomous Exploration for UAVs in Unknown 3D Environments

arXiv:2603.15604v2 Announce Type: replace Abstract: Battery-powered multirotor unmanned aerial vehicles (UAVs) can rapidly map unknown environments, but mission performance is often limited by energy

agentsarxiv-cs-ro

EdiTikZ: Scientific Figure Editing from Revision Trajectories

arXiv:2609.01409v1 Announce Type: new Abstract: Vision-language models (VLMs) have shown strong performance in generating scientific figures from text or images. However, producing publication-ready f

EvoSCM: Scientific Belief Revision Through Causal Model Evolution and Experimentation

arXiv:2609.01526v1 Announce Type: new Abstract: Scientific agents must learn not only how to reason, but also what to believe. However, existing LLM agents typically express scientific hypotheses in f

Foundation models for electricity price forecasting and battery arbitrage: Can they replace market-specific forecasting models?

model-releasesarxiv-cs-lg

arXiv:2609.00089v1 Announce Type: new Abstract: Foundation models promise accurate forecasts with little or no task-specific training, but whether they can replace models designed specifically for ele

From Tool Use to Technological Agency: LoopCAT as a Local-First, Open-Source Tool for Translation Technology Education

arXiv:2609.00344v1 Announce Type: new Abstract: Translation students need to learn both how to use translation technologies and how to judge the choices those technologies make available. This article