Search: “benchmark”

CXR-Retrieve: Compositional Text-to-Image Retrieval in Chest Radiography

arXiv:2607.27779v1 Announce Type: new Abstract: Large chest radiography archives are difficult to search because most studies are paired only with free-text reports rather than structured clinical ann

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

DS@GT ARC at ImageCLEFmedical 2026: Architectural Diversity for Concept Detection and Foundation-Model Scaling for Caption Prediction in Medical Image Analysis

DGX agent

arXiv:2607.27763v1 Announce Type: new Abstract: We describe the DS@GT submissions to the ImageCLEFmedical Caption 2026 challenge, which continues a long-running benchmark on the ROCOv2 dataset with tw

model-releasesarxiv-cs-cv

31 Jul 2026

EEG-EditBench: Probing Visual Information in EEG-Image Retrieval Models with Controlled Image Edits

arXiv:2607.27857v1 Announce Type: new Abstract: Recent EEG-to-image retrieval models have achieved strong performance in identifying viewed images from semantically diverse candidates. Yet such succes

Frontis-MA1: Training an AI4AI Model towards Recursive Self-Improvement in Machine Learning Engineering

arXiv:2607.28568v1 Announce Type: new Abstract: Recursive self-improvement (RSI) requires AI systems that improve the process of building AI (i.e., AI4AI); machine learning engineering (MLE) offers a

JigShape: Evaluating Visual-Geometric Reasoning in VLMs through Jigsaw Puzzles

arXiv:2607.27670v1 Announce Type: new Abstract: Jigsaw puzzle solving requires jointly reasoning about visual content and geometric constraints, yet existing benchmarks use rectangular cuts that creat

Diagnosing Fine-Grained Inconsistency Classification in Financial Disclosure Text

arXiv:2607.26368v1 Announce Type: new Abstract: Financial disclosures contain numerical claims, temporal statements, entity references, policy commitments, and risk descriptions that may conflict in q

30 Jul 2026

MediaWiki Code2Code Search: Neural Retrieval for the Semantic Discovery of Open-Source Software Entities

arXiv:2607.26766v1 Announce Type: cross Abstract: Code search in large-scale ecosystems is often hindered by the lexical gap between user queries and implementation details, alongside the trade-off be

30 Jul 2026

SERPO: Self-Evolving Rubric Policy Optimization for Open-Ended Test-Time Reinforcement Learning

arXiv:2607.26873v1 Announce Type: new Abstract: Test-time reinforcement learning (TTRL) enables language models to self-evolve at inference time without labeled feedback. Existing methods rely on answ

30 Jul 2026

Agentic AI for Scientific Reasoning in Autonomous Quantum Sensing Experiments

arXiv:2607.25145v1 Announce Type: cross Abstract: We implement an agentic AI workflow built around a large language model (LLM) agent for autonomous experiments with nitrogen-vacancy (NV) centers in d

29 Jul 2026

CARE-MH: Towards Unified, Reproducible, and Comparable Evaluation of Mental Health LLMs

arXiv:2607.24754v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used to provide mental health support, requiring reliable evaluation of safety, empathy, and therapeutic

29 Jul 2026

Conformal Cascade: Distribution-Free Accuracy Guarantees for Multi-Tier LLM Inference

model-releasesarxiv-cs-lg

arXiv:2607.25018v1 Announce Type: new Abstract: Large language model (LLM) cascades reduce inference cost by routing easy queries to a small model and deferring hard queries to a larger one. Productio

29 Jul 2026

An Ontology for Machine Learning Interatomic Potentials

arXiv:2607.23219v1 Announce Type: new Abstract: Machine learning interatomic potentials (MLIPs) approximate quantum-mechanical energies and forces---conventionally computed by density functional theor

AptAvatar: Fast and Vivid Long-Form Audio-Driven Video Generation for Production-Ready Avatars

arXiv:2607.24013v1 Announce Type: new Abstract: Production-ready audio-driven avatar generation requires efficient inference without sacrificing fidelity or motion expressiveness. However, existing ac

AssumptionMiner: Extracting, Tracing, and Revising Implicit Assumptions in LLM Code Generation

arXiv:2607.22898v1 Announce Type: cross Abstract: Large language models (LLMs) generate code from natural-language prompts, yet real-world prompts rarely provide complete specifications. When prompts

DOSA: A Tree-Guided, Self-Regressive Framework for Long Document Structure Analysis

arXiv:2607.22679v1 Announce Type: new Abstract: In visually-rich documents, information is encoded not only in individual page objects such as tables, headers, and text blocks, but also in the structu

Hierarchical Group-Conditional Conformal Risk Control for Selective Prediction in Language Models

arXiv:2607.24562v1 Announce Type: new Abstract: Large language models serve heterogeneous populations structured by domain, topic difficulty, and linguistic style. Conformal risk control (CRC) gives r

LEGO Co-builder: Exploring Fine-Grained Vision-Language Modeling for Multimodal LEGO Assembly Assistants

arXiv:2507.05515v4 Announce Type: replace Abstract: Vision-language models (VLMs) are facing the challenges of understanding and following multimodal assembly instructions, particularly when fine-grai

Out-of-Length Scene Text Recognition: A Two-Axis Diagnosis and a Training-Free Fix

arXiv:2607.23194v1 Announce Type: new Abstract: Scene Text Recognition (STR) models are trained almost exclusively on word crops of at most 25 characters, yet real deployments (signage, product labels

Poster: Rethinking Security in LLM Code Generation through Real-World Risk Scenarios

arXiv:2607.23088v1 Announce Type: cross Abstract: Large Language Models (LLMs) are widely used for code generation, yet their security behavior in realistic development workflows remains underexplored

Research

Same Question, Different Answers: Evaluating LLM Reliability Beyond Accuracy

arXiv:2607.22554v1 Announce Type: new Abstract: Large language models (LLMs) often achieve strong accuracy on benchmarks, yet it remains unclear how reliably they apply this knowledge when the same qu

researcharxiv-cs-ai

Similarity All The Way Up: Multilingual Generalization in LLMs Relies on Language-Level Similarity Structures

arXiv:2607.22699v1 Announce Type: new Abstract: As Large Language Models (LLMs) grow more capable across diverse tasks, their (in)ability to generalize remains difficult to quantify and poorly underst

Spatial Reasoning in LLM Game Agents: Impact of Causal Context and Multi-Step Planning

arXiv:2607.22732v1 Announce Type: new Abstract: LLM-based game agents often perform poorly on more complex tasks. This work examines whether these failures are linked to limited spatial reasoning and

SymStep: Symbolic Step Verification for Logical Reasoning

arXiv:2607.23055v1 Announce Type: new Abstract: Chain-of-thought (CoT) prompting can fail severely on constraint-dense logical reasoning tasks, where unverified errors accumulate silently across steps

Towards simultaneous decoding of kinetic and kinematic movement parameters during grasp and lift task by noninvasive brain imaging

arXiv:2607.24081v1 Announce Type: cross Abstract: Brain-machine interfaces (BMIs) can assist individuals with limited mobility, such as stroke survivors or amputees. One of the key challenges in devel

VlogReward: Learning Multi-Dimensional Evaluation for Vlog Editing

arXiv:2607.22632v1 Announce Type: new Abstract: The rapid rise of vlogs as a personalized storytelling medium has created a demand for automated systems to evaluate and refine vlog editing plans. Howe

Agents

DeepUrban: Interaction-Aware Trajectory Prediction and Planning for Automated Driving by Aerial Imagery

arXiv:2601.10554v3 Announce Type: replace Abstract: The efficacy of autonomous driving systems hinges critically on robust prediction and planning capabilities. However, current benchmarks are impeded

agentsarxiv-cs-cv

27 Jul 2026

Optimal Transport Image Representation and Deep Covariance Alignment (CORAL) for Control Valve Stiction Detection

arXiv:2607.22486v1 Announce Type: new Abstract: Control valve stiction is a common cause of unwanted oscillations and poor control-loop performance in industrial processes. Data-driven methods can aut

27 Jul 2026

Hardware

Evaluating and Guarding Citation Faithfulness in Agentic Scientific Synthesis

arXiv:2607.20527v1 Announce Type: new Abstract: Agentic LLM systems such as OpenScholar and PaperQA2 read the scientific literature and return cited answers, and both they and their benchmarks already

hardwarearxiv-cs-ai

Expectation Alignment of Language Models for Real-World User Expectations

arXiv:2607.20485v1 Announce Type: new Abstract: Large language models (LLMs) have demonstrated remarkable performance on standard benchmarks, yet it remains largely unexplored whether they truly meet

Frontier Financial Judgement: Can agents tell what might move a stock?

arXiv:2607.20645v1 Announce Type: cross Abstract: We introduce Frontier Financial Judgement, a challenging new benchmark developed in collaboration with professional equity analysts to assess agents'

Rushes: A Human Preference Dataset for Pluralistic Alignment

arXiv:2607.20767v1 Announce Type: new Abstract: We introduce Rushes, a dataset and benchmark for studying revealed human engagement preferences in interactive narrative environments. Rushes is collect

SciExplore: Evaluating Autonomous Agents from Scientific Navigation to Information Integration

arXiv:2607.20926v1 Announce Type: new Abstract: Scientific research involves complex information-seeking and reasoning workflows across heterogeneous sources. However, existing benchmarks primarily em

Spectral functions in Minkowski quantum electrodynamics from neural reconstruction

model-releasesarxiv-cs-lg

arXiv:2510.24728v2 Announce Type: replace-cross Abstract: We study neural reconstructions of quenched rainbow quantum electrodynamics (QED) Dyson--Schwinger benchmarks in Minkowski-related kinematics.

The Hidden Footprint: Making Storage a First-Class Metric for LLM Agent Evaluation

arXiv:2607.11149v3 Announce Type: replace Abstract: LLM agent benchmarks measure task completion, reliability, and inference cost, but not the persistent data an agent run leaves on disk, including lo

ViSTR-Bench: Can MLLMs Reason from Continuous Visual Cues in Dynamic Scenes?

arXiv:2607.20868v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable success across diverse expert-level tasks, but they still struggle with fundamental ab

GLID: Gated Local Intrinsic Dimension Repairs the Blind Spots of Face-Forgery Detectors

arXiv:2607.18770v1 Announce Type: cross Abstract: Fine-tuned foundation-model detectors dominate face-forgery benchmarks, yet they stay blind to generator families absent from training. We present GLI

23 Jul 2026

Safety

Toward Adaptable Multi-Agent Reinforcement Learning: An Assumption-Aware Review

arXiv:2507.10142v2 Announce Type: replace Abstract: Multi-Agent Reinforcement Learning (MARL) has achieved strong performance in simulated benchmarks, yet real deployments often violate the assumption

safetyarxiv-cs-ai

23 Jul 2026

Interventional Grounding Audits: Black-Box Premise-Dependency Tests for LLM Chain-of-Thought via Predicate Substitution

arXiv:2607.13069v1 Announce Type: new Abstract: Large language models produce chain-of-thought (CoT) reasoning that appears logically sound yet may not genuinely depend on its stated premises. We intr

16 Jul 2026

OccTrack360: 4D Panoptic Occupancy Tracking from Surround-View Fisheye Cameras

arXiv:2603.08521v2 Announce Type: replace Abstract: Understanding dynamic 3D environments in a spatially continuous and temporally consistent manner is fundamental for robotics and autonomous driving.

16 Jul 2026

SingGuard-NSFA: Extensible Guardrails for Agentic AI via Generative Reasoning and Real-Time Classification

arXiv:2607.13081v1 Announce Type: cross Abstract: We present nsfaguard, a guardrail framework for securing agentic AI systems against operational threats, such as prompt injection, sensitive informati

16 Jul 2026

Safety

Accuracy and Normalized Accuracy under Length Bias: Analysis, Guidelines, and a Bayesian Alternative

arXiv:2607.12767v1 Announce Type: new Abstract: Multiple-choice benchmarks that rank candidate completions by conditional log-probability suffer from a length bias: because log-probabilities sum over

safetyarxiv-cs-ai

AVSCap: Orchestrating Audio-Visual Synergy for Omni-modal Video Captioning

arXiv:2607.12820v1 Announce Type: new Abstract: Omni-modal video captioning is not merely combining visual captioning with audio transcription: a useful caption must describe how visual actions, speec

CANDI: Contextual Alignment for Niche Domains Question Answering

arXiv:2607.11891v1 Announce Type: cross Abstract: The deployment of large language models (LLMs) in specialized domains like medical diagnostics and financial advisory necessitates evaluating capabili

CoRe: A Comprehensive Framework for Cross-Image Comparative Reasoning in Vision-Language Models

arXiv:2607.12786v1 Announce Type: new Abstract: Cross-image comparative reasoning remains challenging for vision-language models (VLMs), especially when correct prediction requires fine-grained attrib

Edge-Aware Thermal Infrared UAV Swarm Tracking

arXiv:2607.12544v1 Announce Type: new Abstract: Thermal infrared (TIR) imaging is essential for UAV swarm operations in visually degraded environments. However, tracking tiny UAVs remains challenging

Epistemic Stance Flexibility Probing: Measuring Prompt-Conditioned Register Shift in Large Language Models

arXiv:2607.12739v1 Announce Type: new Abstract: A language model may be asked either what experts believe about a contested claim or what it believes about the claim itself. A trustworthy conversation

The TopCoW Challenge -- Topology-Aware Circle of Willis Segmentation for CT and MR Angiography

arXiv:2312.17670v5 Announce Type: replace Abstract: The Circle of Willis (CoW) is an important network of arteries connecting major circulations of the brain. Its vascular architecture is believed to

Beyond Attack-Success Rate: Action-Graded Severity Scale for Tool-Using AI Agents

arXiv:2607.07474v1 Announce Type: cross Abstract: Agentic red-teaming benchmarks report whether an injected agent was compromised as a single bit: the attack succeeded, or it did not. We argue that th