Search: “evaluation”

Learning human joint torques from pixels

model-releasesarxiv-cs-cv

arXiv:2608.09083v1 Announce Type: new Abstract: Estimating human joint torques from visual observations is a key step toward bringing biomechanical analysis from controlled laboratories to real-world

Learning to Coordinate Symbolic Tools: LLM Agents for Verified Sum-of-Squares Certificates

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

DGX agent

arXiv:2608.00326v2 Announce Type: replace Abstract: Tool calling allows large language models (LLMs) to invoke external computation during problem solving, a useful capability in various fields includ

safetyarxiv-cs-ai

11 Aug 2026

LEGO-Puzzles: How Good Are MLLMs at Multi-Step Spatial Reasoning?

arXiv:2503.19990v4 Announce Type: replace Abstract: Many real-world applications of spatial intelligence, such as robotic control, autonomous driving, and automated assembly, require spatial reasoning

LexKairos: Benchmarking Legal Temporal Capabilities in LLMs

arXiv:2608.09106v1 Announce Type: new Abstract: Large language models (LLMs) have demonstrated strong performance across a wide range of legal tasks. In legal practice, time is a critical concept that

LGNNIC: Acceleration of Large-Scale GNN Training using SmartNICs

arXiv:2608.07733v1 Announce Type: cross Abstract: Graph Neural Networks (GNNs) are widely used across domains such as natural sciences, social network analysis, chip design, and recommendation systems

Listen, See and Track: Spatio-Temporal Audio-Visual Sound Event Reasoning for Omni-Modal Language Models

arXiv:2608.09435v1 Announce Type: new Abstract: Understanding dynamic sound sources requires jointly determining what produces a sound, where the source is located, and how it moves over time. Yet exi

LLM-Driven AutoML for Cross-Lingual Handwritten OCR: Closed-Loop Neural Architecture Search with GPT-5, GPT-4o, and Claude Sonnet 4

arXiv:2607.15509v2 Announce Type: replace-cross Abstract: We present a fully automated closed-loop AutoML framework that uses GPT-5, GPT-4o, and Claude Sonnet 4 as autonomous neural architecture desig

Local Ai

Macaron-V1: Towards Open Continual Learning with Self-Improvement and Mixture-of-LoRA

arXiv:2608.09819v1 Announce Type: cross Abstract: Macaron-V1 is an open agent-model family for experiential intelligence: learning from experience in real environments and continuing to learn after de

local-aiarxiv-cs-cl

MasDrift: Benchmarking Authorization Preservation Across Multi-Agent Architectures

arXiv:2608.07556v1 Announce Type: cross Abstract: Multi-agent systems (MAS) decompose long-horizon tasks across supervisors and subagents, but delegated goals do not necessarily carry their original a

Motif 3: Technical Report

arXiv:2608.09119v1 Announce Type: new Abstract: We introduce Motif 3, a decoder-only Mixture-of-Experts language model with 314 billion total parameters and 13.2 billion activated per token. Each spar

MRBench: A Comprehensive Benchmark for Human Motion-Text Retrieval

model-releasesarxiv-cs-cv

arXiv:2608.07993v1 Announce Type: new Abstract: Human motion-text retrieval provides a rigorous means of assessing cross-modal alignment. Prevailing benchmarks are dominated by homogeneous indoor moti

Multi-Agent Reinforcement Learning via Agent-Specific Preference

model-releasesarxiv-cs-lg

arXiv:2608.08604v1 Announce Type: new Abstract: Multi-agent reinforcement learning (MARL) is a powerful framework for solving complex collaborative tasks, but it relies heavily on well-defined global

NL2SHACL-Bench: A Benchmark Suite for Natural Language to SHACL Translation

arXiv:2608.07530v1 Announce Type: new Abstract: SHACL is a core technology for validating the conformance of RDF knowledge graphs (KGs). Yet, authoring SHACL shapes requires technical expertise that m

Open-World Hierarchical Perception: Taxonomic Abstraction over Class-Agnostic Proposals for the Safe Handling of Out-of-Vocabulary Road Objects

arXiv:2608.07577v1 Announce Type: cross Abstract: A closed-set detector for autonomous driving must assign every object one of a fixed set of labels. On an object outside that set (a horse-drawn carri

P^{3}: Joint Program-and-Proof Planning for Verified Code Generation

arXiv:2608.09277v1 Announce Type: new Abstract: Verified code generation asks a large language model (LLM) to generate both an executable program and a machine-checkable proof that the program meets a

Performance of large language models in the optical diagnosis of colorectal polyps

arXiv:2608.07543v1 Announce Type: cross Abstract: Background and Study Aims: Accurate optical diagnosis of colorectal polyps guides resection strategy and surveillance, with multimodal large language

PragMatch: Separating Pragmatic Incongruity from Cross-Modal Mismatch in Large Vision-Language Models

arXiv:2608.09772v1 Announce Type: new Abstract: Large Vision-Language Models (LVLMs) have demonstrated strong performance on multimodal benchmarks, yet it remains unclear whether they genuinely reason

Private Anytime Selective-Risk Certification for Federated Retrieval-Augmented Generation: Guarantees and Empirical Limits

arXiv:2608.07913v1 Announce Type: cross Abstract: Selective-risk certificates promise that accepted outputs meet a declared error target. We develop Fed-SRC, a score-agnostic certificate for federated

Research

Prompt Embedding Probes (PEP): Hallucination Detection in LLMs from Hidden States

arXiv:2608.08024v1 Announce Type: cross Abstract: Large language models (LLMs) can generate fluent and useful responses but remain prone to hallucinations. We introduce Prompt Embedding Probes (PEP),

researcharxiv-cs-ai

PROSLEX: A Novel Dataset for Expert-Annotated Legal Statute Prediction for Indian Judiciary

arXiv:2608.08830v1 Announce Type: new Abstract: Legal Statute Prediction (LSP) involves automatically identifying relevant legal statutes given factual descriptions in legal documents, typically frame

Quantifying Membership Disclosure Risk for Tabular Synthetic Data Using Kernel Density Estimators

arXiv:2603.10937v2 Announce Type: replace Abstract: The use of synthetic data has become increasingly popular as a privacy-preserving alternative to sharing real datasets, especially in sensitive doma

applicationsarxiv-cs-lg

Quokka: Accelerating Program Verification with LLMs via Invariant Synthesis

arXiv:2509.21629v4 Announce Type: replace-cross Abstract: Program verification relies on loop invariants, yet automatically discovering strong invariants remains a long-standing challenge. We investig

SIMMER: Benchmarking Latent Failures in LLM Executable Planning with a World Model

arXiv:2606.14574v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly deployed as planners for autonomous agents in household environments. While existing benchmarks

Agents

SkillsMetric: Mapping the Detection Boundary of Static Analysis for Malicious Agent Skills

arXiv:2608.08468v1 Announce Type: cross Abstract: Agent Skills---structured packages of instructions and scripts that augment LLM-based agents---are rapidly proliferating, yet their security propertie

agentsarxiv-cs-ai

Local Ai

SLAP: Selective Local Vision-Language Alignment for Fish Re-Identification via Partial Optimal Transport

arXiv:2608.08840v1 Announce Type: new Abstract: Individual fish re-identification (ReID) is a fine-grained recognition problem in which identity-discriminative cues are often localized to specific bod

local-aiarxiv-cs-cv

Spatiotemporal Context-dependent Personalized Movement Compensation in Delayed Telemanipulation

arXiv:2608.08200v1 Announce Type: new Abstract: Communication delay remains a central challenge in telerobotics, where it disrupts visuomotor coordination and reduces task precision. Motion scaling is

safetyarxiv-cs-ro

StructReward: Efficient Structured Process Rewards for Self-Correcting Multimodal Reasoning

arXiv:2608.08326v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) has emerged as an effective approach for improving multimodal reasoning. However, most existing me

SurgWMBench: A Vision-Based Benchmark for World-Modeling Surgical Instrument Motion Planning

model-releasesarxiv-cs-ro

arXiv:2608.08070v1 Announce Type: new Abstract: Reliable surgical planning requires models that move beyond recognizing the current surgical step or imitating expert demonstrations, and instead antici

Task-to-Model Optimization for Enterprise LLM Coding Assistants: A Data-Driven Framework for Cost-Optimal Routing

model-releasesarxiv-cs-lg

arXiv:2608.08528v1 Announce Type: new Abstract: Enterprise AI coding assistants incur substantial inference spend, and naive token-cost minimization often fails to reduce end-to-end cost once retries,

TeXFix-Bench: An Empirically Grounded Multi-Format Benchmark for LLM-Based Document Source Repair

arXiv:2608.07617v1 Announce Type: new Abstract: Scientific and technical writing depends on markup sources that must compile: LaTeX, Typst, and Markdown pipelines fail on missing delimiters, mismatche

The Cell Must Go On: Agar.io for Continual Reinforcement Learning

arXiv:2505.18347v3 Announce Type: replace-cross Abstract: Continual reinforcement learning (RL) concerns agents that are expected to learn continually, rather than converge to a policy that is then fi

Three Generations of Healthcare IT: From the Digital Record to the Computable Care Process

arXiv:2608.08806v1 Announce Type: new Abstract: Objective. Healthcare IT is usually organized by the technologies it adopts. We instead organize it by the unit of information a system makes computable

applicationsarxiv-cs-ai

Tied Trit-Planes: Constraining PTQTP to a Uniform Nine-Level Quantizer, with a Persistent Folded Format for Disk-Streamed Mixture-of-Experts Serving

arXiv:2608.08910v1 Announce Type: new Abstract: PTQTP decomposes LLM weight matrices into two ternary (trit) planes with two free per-group scales. Tying the scales to a fixed ratio of three collapses

TomaMMU: A Comprehensive Multimodal Understanding Benchmark for Tomato Leaf Diseases

arXiv:2608.08727v1 Announce Type: cross Abstract: To address this gap, we introduce TomaMMU, a large-scale Tomato leaf disease MultiModal Understanding dataset, alongside TomaBench, a benchmark for ev

Towards Expert-level Medical AI for Real-time Video Consultations

arXiv:2608.09861v1 Announce Type: new Abstract: Audio-visual interaction is the standard for patient-physician consultations, enabling natural communication and effective assessment of illness through

Unified Hallucination Fuzzing for Multimodal Large Language Models

arXiv:2608.07525v1 Announce Type: cross Abstract: Hallucination remains a persistent challenge for Multimodal Large Language Models (MLLMs), severely limiting their reliability in high-stakes applicat

UNSPECIFIC: General Constraint Synthesis for Breaking Copy-and-Paste Shortcut in LLM Instruction Following

arXiv:2608.09154v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly expected to follow long lists of constraints in complex instructions, and synthesizing instructions from a

VideoVIBE: A Video-Grounded Diagnostic Benchmark for One-Shot Interactive Website Generation

model-releasesarxiv-cs-cv

arXiv:2608.09573v1 Announce Type: new Abstract: Natural-language-driven 'vibe coding' enables the one-shot generation of visually rich and interactive web applications, yet reliable assessment of thei

VTO: Visual Tool Orchestration for Video Anomaly Detection

arXiv:2608.08219v1 Announce Type: cross Abstract: Video anomaly detection (VAD) is a critical yet challenging task due to the complex and diverse nature of real-world scenarios. Traditional deep learn

WDL-OPD: Weak-Driven On-Policy Distillation via Mixture-Constrained Co-Training

arXiv:2608.09447v1 Announce Type: cross Abstract: On-policy distillation (OPD) aligns a student with a teacher on trajectories sampled from the student itself, reducing the train-test state mismatch o

Research

When Confidence Fails: Overconfidence in LLMs under Uncertainty and Missing Clinical Information

arXiv:2608.09080v1 Announce Type: cross Abstract: Large Language Models (LLMs) have achieved strong performance in medical question answering and clinical reasoning tasks. However, their reliability u

researcharxiv-cs-ai

WuYuEval: A Multi-Level Benchmark for Large Language Models in Solid Waste Management

arXiv:2608.07529v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as technical assistants, but their competence in solid waste management (SWM) remains difficult to

AdvTiles: Physical Adversarial Camouflage Clothing against Person Detectors via Learnable Tiles

arXiv:2608.06801v1 Announce Type: new Abstract: Physical adversarial attacks against person detectors have evolved from localized patches to full-body textures. However, achieving both visual naturaln

applicationsarxiv-cs-cv

Aftab: A Comprehensive Benchmark of CNN Encoders and Advanced Value Functions in Parallelized Q-Networks

arXiv:2608.07335v1 Announce Type: cross Abstract: Recent advancements in deep reinforcement learning have increasingly favored simplified, highly parallelized paradigms. Notably, the Parallelized Q-Ne

ArchEGraph: A Large-Scale Graph Dataset for Geometry-Topology-Physics Aligned Building Energy Modeling

model-releasesarxiv-cs-lg

arXiv:2608.06772v1 Announce Type: new Abstract: Accurate estimation of building energy use is essential for achieving carbon neutral and sustainable buildings. To better understand the influence of de

Artificial Intelligence Can Match Domain Experts in Evidence Extraction and Critical Appraisal of Microbial Oncogenesis Research Publications

arXiv:2608.07250v1 Announce Type: cross Abstract: Confirmed oncogenic microbes contribute significantly to cancer burden. Identifying novel microbial oncogenicity could yield strategies that will redu

Assessing AI-generated music detection in real-world broadcast monitoring

arXiv:2608.07359v1 Announce Type: cross Abstract: The proliferation of AI-generated music in broadcast media raises concerns about transparency and fair compensation, but reliable detection under real

applicationsarxiv-cs-ai

AutoIntervene: Calibrated Intervention for Action-Chunking Imitation Learning Policies

arXiv:2608.07065v1 Announce Type: cross Abstract: Action-chunking visuomotor policies learn from demonstrations and improve temporal consistency by predicting short action sequences rather than single