Search: “evaluation”

flowengineR: A Modular and Extensible Framework for Fair and Reproducible Workflow Design in R

arXiv:2511.00079v2 Announce Type: replace Abstract: flowengineR is an R package designed to provide a modular and extensible framework for building reproducible algorithmic workflows for general-purpo

safetyarxiv-cs-lg

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

From Benchmark Performance to Tool Deployment: Human-in-the-Loop Anomaly Detection

DGX agent

arXiv:2608.07770v1 Announce Type: cross Abstract: Automated anomaly detection methods often report strong performance on curated academic benchmarks, but their behavior under real-world industrial con

model-releasesarxiv-cs-cv

11 Aug 2026

From Product Search to Preference Articulation: The Economics of Agentic Commerce

arXiv:2608.08395v1 Announce Type: cross Abstract: Generative AI is shifting digital commerce from browsing toward agentic search, in which consumers delegate product discovery to AI agents. We compare

GALA: Graph-Augmented LLM Agents for Root Cause Analysis and Incident Response in Microservices

arXiv:2608.08968v1 Announce Type: cross Abstract: Microservice root cause analysis (RCA) requires correlating failures across heterogeneous telemetry within complex service dependency graphs. Existing

local-aiarxiv-cs-ai

HeatCast: A Benchmark for Neighborhood-Scale LST Forecasting across 124 U.S. Cities

model-releasesarxiv-cs-cv

arXiv:2608.07640v1 Announce Type: new Abstract: Land Surface Temperature (LST) is a widely used satellite-derived measure of urban surface heat, but there is no shared benchmark for forecasting it at

HoloAegis: Frozen Representation, Topological Inference: Minimally Parametric Safety Manifolds for Zero-Shot LLM Guardrails

arXiv:2608.08485v1 Announce Type: new Abstract: Current LLM safety guardrails face a fundamental tension: fine-tuning distorts pre-trained representations while generative judges incur prohibitive inf

How sensitive do we want AI to be? Socio-communicative competencies of large language models in healthcare

arXiv:2608.07511v1 Announce Type: cross Abstract: Background. Effective clinical practice relies heavily on the socio-communicative skills of medical professionals. Large language models (LLMs) have b

HugSelect: An Explainable Multi-Criteria Decision-Support Framework for foundation-model selection

arXiv:2608.08069v1 Announce Type: cross Abstract: Foundation models are increasingly reused as software components, making model selection a critical software-engineering decision. Current model hubs

researcharxiv-cs-ai

ICM Out! Better Tournament Strategy from Computed Continuations, vs. Solvers and LLMs

arXiv:2608.09586v1 Announce Type: new Abstract: The Independent Chip Model (ICM) converts tournament chips into reference prize equity, and policies are routinely constructed against those values. Bec

IRPol-Fuse: Energy-structure coordination for infrared polarization fusion under low visibility

arXiv:2608.07848v1 Announce Type: new Abstract: Robust perception under low-visibility conditions requires fused imagery that jointly preserves infrared thermal saliency and polarization-derived struc

researcharxiv-cs-cv

KGCache: Amortized Subgraph Retrieval for KG Reasoning with LLMs

arXiv:2608.07954v1 Announce Type: new Abstract: Large language models can answer knowledge-intensive questions more reliably when they are grounded with knowledge graphs, but systems such as Think-on-

Latent-Frequency Validity: Fast Spectral Editing with Screened Video-VAE Transfer Operators

arXiv:2608.07569v1 Announce Type: cross Abstract: Direct spectral editing in video-VAE latents can control noise, flicker, smoothness, and frequency content without a decode--filter--reencode pass. Ho

researcharxiv-cs-ai

Learning human joint torques from pixels

model-releasesarxiv-cs-cv

arXiv:2608.09083v1 Announce Type: new Abstract: Estimating human joint torques from visual observations is a key step toward bringing biomechanical analysis from controlled laboratories to real-world

Learning to Coordinate Symbolic Tools: LLM Agents for Verified Sum-of-Squares Certificates

arXiv:2608.00326v2 Announce Type: replace Abstract: Tool calling allows large language models (LLMs) to invoke external computation during problem solving, a useful capability in various fields includ

LEGO-Puzzles: How Good Are MLLMs at Multi-Step Spatial Reasoning?

arXiv:2503.19990v4 Announce Type: replace Abstract: Many real-world applications of spatial intelligence, such as robotic control, autonomous driving, and automated assembly, require spatial reasoning

LexKairos: Benchmarking Legal Temporal Capabilities in LLMs

model-releasesarxiv-cs-cl

arXiv:2608.09106v1 Announce Type: new Abstract: Large language models (LLMs) have demonstrated strong performance across a wide range of legal tasks. In legal practice, time is a critical concept that

LGNNIC: Acceleration of Large-Scale GNN Training using SmartNICs

arXiv:2608.07733v1 Announce Type: cross Abstract: Graph Neural Networks (GNNs) are widely used across domains such as natural sciences, social network analysis, chip design, and recommendation systems

Listen, See and Track: Spatio-Temporal Audio-Visual Sound Event Reasoning for Omni-Modal Language Models

arXiv:2608.09435v1 Announce Type: new Abstract: Understanding dynamic sound sources requires jointly determining what produces a sound, where the source is located, and how it moves over time. Yet exi

LLM-Driven AutoML for Cross-Lingual Handwritten OCR: Closed-Loop Neural Architecture Search with GPT-5, GPT-4o, and Claude Sonnet 4

arXiv:2607.15509v2 Announce Type: replace-cross Abstract: We present a fully automated closed-loop AutoML framework that uses GPT-5, GPT-4o, and Claude Sonnet 4 as autonomous neural architecture desig

Macaron-V1: Towards Open Continual Learning with Self-Improvement and Mixture-of-LoRA

arXiv:2608.09819v1 Announce Type: cross Abstract: Macaron-V1 is an open agent-model family for experiential intelligence: learning from experience in real environments and continuing to learn after de

local-aiarxiv-cs-cl

MasDrift: Benchmarking Authorization Preservation Across Multi-Agent Architectures

arXiv:2608.07556v1 Announce Type: cross Abstract: Multi-agent systems (MAS) decompose long-horizon tasks across supervisors and subagents, but delegated goals do not necessarily carry their original a

MiniMax-H3: ~38 GB less VRAM with Runtime LoRA Bypass — DoRA Dynamic LoRA Loader v1.0.39

local-air-stablediffusion

GitHub: https://github.com/xmarre/ComfyUI-DoRA-Dynamic-LoRA-Loader Release v1.0.39: https://github.com/xmarre/ComfyUI-DoRA-Dynamic-LoRA-Loader/releases/tag/v1.0.39 Also available through ComfyUI Manag

Motif 3: Technical Report

arXiv:2608.09119v1 Announce Type: new Abstract: We introduce Motif 3, a decoder-only Mixture-of-Experts language model with 314 billion total parameters and 13.2 billion activated per token. Each spar

MRBench: A Comprehensive Benchmark for Human Motion-Text Retrieval

model-releasesarxiv-cs-cv

arXiv:2608.07993v1 Announce Type: new Abstract: Human motion-text retrieval provides a rigorous means of assessing cross-modal alignment. Prevailing benchmarks are dominated by homogeneous indoor moti

Multi-Agent Reinforcement Learning via Agent-Specific Preference

model-releasesarxiv-cs-lg

arXiv:2608.08604v1 Announce Type: new Abstract: Multi-agent reinforcement learning (MARL) is a powerful framework for solving complex collaborative tasks, but it relies heavily on well-defined global

Muse-Glimmer 30B Hits ~280 t/s in Real Production Coding

model-releasesr-localllama

These numbers were captured during a real feature implementation task in Next.js and Nest.js (adding a theme switching system across components). The structural predictability of UI/state refactoring

NL2SHACL-Bench: A Benchmark Suite for Natural Language to SHACL Translation

arXiv:2608.07530v1 Announce Type: new Abstract: SHACL is a core technology for validating the conformance of RDF knowledge graphs (KGs). Yet, authoring SHACL shapes requires technical expertise that m

Open-World Hierarchical Perception: Taxonomic Abstraction over Class-Agnostic Proposals for the Safe Handling of Out-of-Vocabulary Road Objects

arXiv:2608.07577v1 Announce Type: cross Abstract: A closed-set detector for autonomous driving must assign every object one of a fixed set of labels. On an object outside that set (a horse-drawn carri

P^{3}: Joint Program-and-Proof Planning for Verified Code Generation

arXiv:2608.09277v1 Announce Type: new Abstract: Verified code generation asks a large language model (LLM) to generate both an executable program and a machine-checkable proof that the program meets a

Performance of large language models in the optical diagnosis of colorectal polyps

arXiv:2608.07543v1 Announce Type: cross Abstract: Background and Study Aims: Accurate optical diagnosis of colorectal polyps guides resection strategy and surveillance, with multimodal large language

PragMatch: Separating Pragmatic Incongruity from Cross-Modal Mismatch in Large Vision-Language Models

model-releasesarxiv-cs-cl

arXiv:2608.09772v1 Announce Type: new Abstract: Large Vision-Language Models (LVLMs) have demonstrated strong performance on multimodal benchmarks, yet it remains unclear whether they genuinely reason

Private Anytime Selective-Risk Certification for Federated Retrieval-Augmented Generation: Guarantees and Empirical Limits

arXiv:2608.07913v1 Announce Type: cross Abstract: Selective-risk certificates promise that accepted outputs meet a declared error target. We develop Fed-SRC, a score-agnostic certificate for federated

Prompt Embedding Probes (PEP): Hallucination Detection in LLMs from Hidden States

arXiv:2608.08024v1 Announce Type: cross Abstract: Large language models (LLMs) can generate fluent and useful responses but remain prone to hallucinations. We introduce Prompt Embedding Probes (PEP),

researcharxiv-cs-ai

PROSLEX: A Novel Dataset for Expert-Annotated Legal Statute Prediction for Indian Judiciary

arXiv:2608.08830v1 Announce Type: new Abstract: Legal Statute Prediction (LSP) involves automatically identifying relevant legal statutes given factual descriptions in legal documents, typically frame

Applications

Quantifying Membership Disclosure Risk for Tabular Synthetic Data Using Kernel Density Estimators

arXiv:2603.10937v2 Announce Type: replace Abstract: The use of synthetic data has become increasingly popular as a privacy-preserving alternative to sharing real datasets, especially in sensitive doma

applicationsarxiv-cs-lg

Quokka: Accelerating Program Verification with LLMs via Invariant Synthesis

arXiv:2509.21629v4 Announce Type: replace-cross Abstract: Program verification relies on loop invariants, yet automatically discovering strong invariants remains a long-standing challenge. We investig

SIMMER: Benchmarking Latent Failures in LLM Executable Planning with a World Model

arXiv:2606.14574v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly deployed as planners for autonomous agents in household environments. While existing benchmarks

Agents

SkillsMetric: Mapping the Detection Boundary of Static Analysis for Malicious Agent Skills

arXiv:2608.08468v1 Announce Type: cross Abstract: Agent Skills---structured packages of instructions and scripts that augment LLM-based agents---are rapidly proliferating, yet their security propertie

agentsarxiv-cs-ai

SLAP: Selective Local Vision-Language Alignment for Fish Re-Identification via Partial Optimal Transport

arXiv:2608.08840v1 Announce Type: new Abstract: Individual fish re-identification (ReID) is a fine-grained recognition problem in which identity-discriminative cues are often localized to specific bod

local-aiarxiv-cs-cv

Spatiotemporal Context-dependent Personalized Movement Compensation in Delayed Telemanipulation

arXiv:2608.08200v1 Announce Type: new Abstract: Communication delay remains a central challenge in telerobotics, where it disrupts visuomotor coordination and reduces task precision. Motion scaling is

safetyarxiv-cs-ro

Stealing Reasoning Traces from Proprietary LLM APIs

model-releasessimon-willison

Stealing Reasoning Traces from Proprietary LLM APIs A vanity domain name (stolen-thoughts.com) for a neat paper: Anthropic, OpenAI, and Google return encrypted chain-of-thought blocks to clients that

StructReward: Efficient Structured Process Rewards for Self-Correcting Multimodal Reasoning

arXiv:2608.08326v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) has emerged as an effective approach for improving multimodal reasoning. However, most existing me

SurgWMBench: A Vision-Based Benchmark for World-Modeling Surgical Instrument Motion Planning

model-releasesarxiv-cs-ro

arXiv:2608.08070v1 Announce Type: new Abstract: Reliable surgical planning requires models that move beyond recognizing the current surgical step or imitating expert demonstrations, and instead antici

Task-to-Model Optimization for Enterprise LLM Coding Assistants: A Data-Driven Framework for Cost-Optimal Routing

model-releasesarxiv-cs-lg

arXiv:2608.08528v1 Announce Type: new Abstract: Enterprise AI coding assistants incur substantial inference spend, and naive token-cost minimization often fails to reduce end-to-end cost once retries,

TeXFix-Bench: An Empirically Grounded Multi-Format Benchmark for LLM-Based Document Source Repair

arXiv:2608.07617v1 Announce Type: new Abstract: Scientific and technical writing depends on markup sources that must compile: LaTeX, Typst, and Markdown pipelines fail on missing delimiters, mismatche

The Cell Must Go On: Agar.io for Continual Reinforcement Learning

arXiv:2505.18347v3 Announce Type: replace-cross Abstract: Continual reinforcement learning (RL) concerns agents that are expected to learn continually, rather than converge to a policy that is then fi

Applications

Three Generations of Healthcare IT: From the Digital Record to the Computable Care Process

arXiv:2608.08806v1 Announce Type: new Abstract: Objective. Healthcare IT is usually organized by the technologies it adopts. We instead organize it by the unit of information a system makes computable

applicationsarxiv-cs-ai

Tied Trit-Planes: Constraining PTQTP to a Uniform Nine-Level Quantizer, with a Persistent Folded Format for Disk-Streamed Mixture-of-Experts Serving

model-releasesarxiv-cs-cl

arXiv:2608.08910v1 Announce Type: new Abstract: PTQTP decomposes LLM weight matrices into two ternary (trit) planes with two free per-group scales. Tying the scales to a fixed ratio of three collapses