Search: “evaluation”

MASH-Bench: Diagnosing Cross-Source Failure in Mass-Shooting Risk Classification

arXiv:2608.22460v1 Announce Type: new Abstract: Public mass-shooting databases differ substantially in coverage, feature availability, and reporting practices, creating challenges for machine-learning

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

MediSkill-Evo: Process-Constrained Self-Evolution for Evidence-Grounded Clinical Interaction

DGX agent

arXiv:2608.23397v1 Announce Type: new Abstract: Interactive clinical agents must gather decisive evidence and convert it into grounded actions under partial observability. A correct final diagnosis al

model-releasesarxiv-cs-ai

25 Aug 2026

MobilePA-Bench: Benchmarking Mobile Planner Agents on Complex Real-World Tasks

arXiv:2608.23035v1 Announce Type: new Abstract: As on-device LLM agents evolve into personal copilots, the mobile operating system has become a key testbed for this paradigm, making rigorous capabilit

On the Threat Model of Weird Generalization and Emergent Misalignment

arXiv:2608.23476v1 Announce Type: new Abstract: Narrow fine-tuning on small, domain-specific datasets can produce broad and surprising changes in model behavior-a phenomenon called weird generalizatio

researcharxiv-cs-cl

OVIBench: Benchmarking Online Video Question Answering under Interruption

arXiv:2608.22279v1 Announce Type: cross Abstract: Recent vision language models (VLMs) have achieved strong progress in video understanding. However, most existing video QA research and benchmarks sti

Performance of a domain-specific large language model in answering patient questions in psychiatry

arXiv:2608.22797v1 Announce Type: new Abstract: Background This study was designed to evaluate whether a domain-specific large language model (LLM) trained exclusively on patient education resources c

Pointing-VLA: Typed Spatial Grounding Interfaces for Vision-Language-Action Manipulation

arXiv:2608.23138v1 Announce Type: cross Abstract: Vision-language-action (VLA) models often expose spatial grounding through autoregressive text coordinates or opaque action tokens, creating brittle i

Robustness of Anomaly Detection Models for Industrial Control Systems under Training-Time Data Contamination

arXiv:2608.23547v1 Announce Type: cross Abstract: Machine-learning-based anomaly detection is increasingly used in industrial control systems (ICS), yet most studies assume that detector training data

SDoH-Aware Narrative Anchoring Bias in Medical LLMs for Trustworthy Clinical Decision Support

arXiv:2608.22802v1 Announce Type: cross Abstract: Medical large language models are often judged by how many clinical questions they answer correctly. That view is useful, but it misses a practical ri

Separating Voice from Age in COPD Screening

arXiv:2608.21599v1 Announce Type: cross Abstract: Voice has been proposed as a low-cost screening signal for chronic obstructive pulmonary disease (COPD). COPD is strongly age-associated and voice cha

Spicing up Genetic Netlist Generation with LLMs

arXiv:2608.23317v1 Announce Type: cross Abstract: Analog circuit topology synthesis remains challenging because useful designs occupy a tiny fraction of a combinatorial search space, and small structu

Task-Driven 3D Printability Assistance via Geometry- and Knowledge-Grounded LLM Reasoning

arXiv:2608.22128v1 Announce Type: new Abstract: Printability assessment in additive manufacturing is typically conducted at the geometry level before printing to determine whether a computer-aided des

TELEVAL: A Benchmark Designed for Spoken Language Models in Chinese Interactive Scenarios

arXiv:2507.18061v4 Announce Type: replace-cross Abstract: Spoken Language Models (SLMs) are expected to support natural spoken interaction beyond task completion. However, existing SLM benchmarks prim

TRACE: Temporal Retrieval with Anchored and Convergent Evidence for Long-Horizon Video Understanding

arXiv:2608.22516v1 Announce Type: cross Abstract: A long-video answer is evidence-supported only when the frames decoded from the video cover every event the answer depends on. Existing evaluations sc

Tutorials

When Does Visual Generation Help Visual Understanding in Unified Multimodal Models?

arXiv:2608.22174v1 Announce Type: new Abstract: Unified multimodal models (UMMs) can perform both understanding and generation, raising a central question: can visual generation improve understanding?

tutorialsarxiv-cs-cv

A VLM Answer Is Not an Anomaly Score: Rank Compression in Training-Free Video Anomaly Detection

model-releasesarxiv-cs-cv

arXiv:2608.21244v1 Announce Type: new Abstract: Vision-language models enable training-free video anomaly detection by answering questions about video segments. VAD benchmarks, however, require a scal

AgentMercury: Your Agent Can Synthesize Verifiable Environments for Business Scenarios at scale

arXiv:2608.20634v1 Announce Type: cross Abstract: Agents learn to act through interaction with environments, yet the environments used for training are often manually constructed or synthesized around

Bankruptcy Prediction via Hybrid Resampling and Stacking Ensemble Techniques with Explainable Artificial Intelligence (XAI)-Driven Analysis

arXiv:2608.20343v1 Announce Type: new Abstract: This study develops and evaluates a bankruptcy prediction framework that integrates consensus-based feature selection, hybrid resampling, stacking ensem

researcharxiv-cs-lg

Local Ai

Best Local Vision Language Models - August 2026

Share what your favorite models are right now and why. Given the nature of the beast in evaluating VLMs (untrustworthiness of benchmarks, immature tooling, intrinsic stochasticity), please be as detai

local-air-localllama

Beyond Effectiveness: A Multi-Criteria Framework for Comparing Practical Socio-Technical Interventions

arXiv:2608.20649v1 Announce Type: new Abstract: Designers and policymakers in sociotechnical domains like content moderation, privacy interfaces, recommender systems and beyond, must choose among a gr

researcharxiv-cs-ai

Beyond Raw Transcripts: Structured Persona Extraction for LLM-Based Digital Twins

arXiv:2608.20344v1 Announce Type: new Abstract: LLM-based 'digital twins' aim to simulate how an individual would behavein new environments or respond to novel questions, given some representation of

Breaking High Confidence: Practical Face Impersonation under High-Security Thresholds

model-releasesarxiv-cs-cv

arXiv:2608.20884v1 Announce Type: new Abstract: Face recognition systems (FRSs) are increasingly deployed in critical real-world services for authentication, such as banking applications and airport i

Can Legal AI Know When It Is Wrong? And Do Students Know When It Is?

arXiv:2608.21089v1 Announce Type: new Abstract: Integrating Large Language Models (LLMs) into the Indian judiciary promises access to justice but introduces severe risks. We identify the 'inertia of c

CellPath-Bench: A Multidimensional Benchmark for Whole-Slide Cellular Representations in Pathology Foundation Models

arXiv:2608.21060v1 Announce Type: new Abstract: Pathology foundation models (PFMs) are increasingly used as general-purpose backbones, yet existing benchmarks cannot systematically diagnose their whol

Dynamic Context Scheduling: Learning Beyond the Static Universe

arXiv:2608.20799v1 Announce Type: new Abstract: We study dynamic context scheduling as a training instrument for contextual re- inforcement learning. Rather than treating intra-episode context variati

Efficient Inference for Inverse Reinforcement Learning and Dynamic Discrete Choice Models

arXiv:2512.24407v2 Announce Type: replace Abstract: In many sequential decision-making problems, researchers observe actions but not the rewards that drive behavior, yet still wish to evaluate and com

safetyarxiv-cs-lg

Event-triggered Implicit Perturbation for Zeroth-Order Fine-Tuning of Spiking Transformers

arXiv:2608.21223v1 Announce Type: cross Abstract: Zeroth-order (ZO) optimization estimates gradients using only forward-pass evaluations, making it suitable for fine-tuning non-differentiable, event-d

researcharxiv-cs-lg

Hardware

Faults That Fortify: CNN Adversarial Robustness via GPU Undervolting

arXiv:2608.20572v1 Announce Type: new Abstract: Convolutional Neural Networks (CNNs) face a dual challenge: vulnerability to adversarial attacks and prohibitive training cost. Adversarial training is

hardwarearxiv-cs-lg

Fine-tuning LLMs for Tourist Trajectory Prediction using Field Experiment Data

arXiv:2608.20830v1 Announce Type: cross Abstract: Evaluating mobility interventions at tourist destinations requires predicting visitor behavior under varying conditions. Traditional methods struggle

GS-Net: Heterogeneous Vehicle Data Reuse via Generalizable Plug-and-Play 3DGS Module

model-releasesarxiv-cs-cv

arXiv:2409.11307v2 Announce Type: replace Abstract: End-to-end autonomous driving is increasingly data-driven, yet data reuse across vehicles remains limited. Each new vehicle often requires additiona

Hybrid Roller-Jamming Gripper for Object Acquisition and Retention Under Pose Uncertainty

arXiv:2608.20962v1 Announce Type: new Abstract: In household manipulation, pose uncertainty often results in off-centre or partial initial contact, making reliable object acquisition difficult. Roller

researcharxiv-cs-ro

Is Vibe Coding Safe? Benchmarking Vulnerability of Agent-Generated Code in Real-World Tasks

arXiv:2512.03262v3 Announce Type: replace-cross Abstract: Vibe coding is a new software development paradigm in which human engineers prompt a large language model (LLM) agent to complete complex codi

Large Scale AI Grading of Handwritten Physics Assessments: Score Agreement and Olympiad Team Selection Outcomes

arXiv:2608.20521v1 Announce Type: cross Abstract: Multimodal AI can read handwritten physics solutions, but high-stakes grading requires agreement with official scores and outcomes. This study evaluat

Masking Is Not Enough: Generative Restoration for Multimodal De-Identification in Medical AI

arXiv:2608.21133v1 Announce Type: new Abstract: Medical image-text data can expose protected health information (PHI) through both visible image content as well as accompanying text, creating a barrie

researcharxiv-cs-cv

New AI-powered quick assessments in Migration Center turbocharge modernization

model-releasesgoogle-cloud-ai

Technology leaders are under mounting pressure to modernize infrastructure, control multi-cloud operational spend, and build data foundations for generative AI. However, the discovery required for tha

Applications

On the Transferability of Agricultural Weed Detection Under Cross-Field Distribution Shift

arXiv:2608.21254v1 Announce Type: new Abstract: Accurate agricultural weed detection in real-world field conditions is essential for precision agriculture, enabling targeted intervention and reducing

applicationsarxiv-cs-cv

One Hierarchy, Two Systems: Semantic Product IDs for Discovery-Surface Ranking and Search-Page Query Reformulation

arXiv:2608.20640v1 Announce Type: cross Abstract: Multi-merchant e-commerce catalogs contain equivalent and related products under different merchant-scoped identifiers, fragmenting behavioral evidenc

researcharxiv-cs-ai

Roadside-Cooperative Autonomous Driving: From Data Platform to Vision-Language End-to-End Reasoning

model-releasesarxiv-cs-ro

arXiv:2608.21032v1 Announce Type: new Abstract: Vehicle-to-Everything (V2X) cooperation enables beyond-line-of-sight perception, mitigating occlusions in single-vehicle sensing. However, existing V2X

Six misconceptions about large language models: A minimal model and diagnostic taxonomy

arXiv:2608.20421v1 Announce Type: cross Abstract: Large language models (LLMs) are now embedded in scientific, educational, and governance workflows, with debates centering on their capabilities, mech

Applications

Structured but Fragile: On the Limits of LLMs in Cybersecurity Decision-Making

arXiv:2608.20966v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used in cybersecurity workflows, yet it remains unclear whether they can perform structured security rea

applicationsarxiv-cs-ai

Who Do Language Models Think Is Competent? A Mechanistic Analysis of Occupational Bias

arXiv:2608.20347v1 Announce Type: cross Abstract: Language models (LMs) often pass behavioral bias evaluations, but it remains unclear whether they no longer represent the underlying associations that

WorldLines: Benchmarking and Modeling Long-Horizon Stateful Embodied Agents

arXiv:2606.18847v2 Announce Type: replace Abstract: To assist humans over extended periods in real homes, embodied agents must remember user routines, world states, and past interactions. Existing lon

CarBench: A Comprehensive Benchmark for Neural Surrogates on High-Fidelity 3D Car Aerodynamics

arXiv:2512.07847v2 Announce Type: replace Abstract: Benchmarking has been the cornerstone of progress in computer vision, natural language processing, and the broader deep learning domain, driving alg

Compliance, Capability, and Conflict: Benchmarking Multimodal LLMs under System Messages

arXiv:2608.19207v1 Announce Type: new Abstract: Production deployments of Multimodal Large Language Models (MLLMs) increasingly rely on system messages to govern model behavior. Yet existing benchmark

ConceptGuard: Benchmarking Context-Sensitive Unlearning in Large Language Models

arXiv:2608.20338v1 Announce Type: new Abstract: Large Language Models (LLMs) increasingly require selective removal of harmful or sensitive knowledge, called unlearning, yet existing methods and bench

DiverValue-Bench: A Benchmark and Fine-Tuning Framework for Aligning Large Language Models with Diverse Human Values

arXiv:2509.08022v3 Announce Type: replace-cross Abstract: Aligning large language models (LLMs) with diverse human values is essential for safe and effective deployment, yet existing benchmarks often

ExPhy: A Benchmark for Explicit Physical Property Learning in Multi-Object Trajectory Forecasting

arXiv:2608.20009v1 Announce Type: new Abstract: Understanding object dynamics requires not only predicting future trajectories but also examining whether a model captures the physical properties that

Explainable Multimodal Depression Recognition in Clinical Interviews via PHQ-Aligned Symptom Summarization

arXiv:2501.16106v2 Announce Type: replace Abstract: Recent advances in multimodal depression recognition for clinical interviews (MDRC) have demonstrated the potential of AI systems by integrating tex

researcharxiv-cs-cl