Search: “benchmark”

Question's Gambit: The First Move Matters in Agentic Deep Search

arXiv:2609.14412v1 Announce Type: new Abstract: Deep research agents answer complex questions through iterative loops of searching, reading, and reasoning. Recent work on reasoning-intensive benchmark

15 Sept 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

ZEBRAARENA: A Diagnostic Simulation Environment for Studying Reasoning-Action Coupling in Tool-Augmented LLMs

DGX agent

arXiv:2603.18614v2 Announce Type: replace Abstract: Tool-augmented large language models (LLMs) must tightly couple multi-step reasoning with external actions, yet existing benchmarks often confound t

model-releasesarxiv-cs-ai

15 Sept 2026

Is Bash All You Need? An Empirical Study of Tool Interfaces for Enterprise Digital Worker Agents

arXiv:2609.11999v1 Announce Type: cross Abstract: In this study, we examine whether a general shell can outperform specialized tools on enterprise tasks. Shell-based agents have shown strong results i

14 Sept 2026

ORQA: An Occupation-Realistic Question and Answer Framework for LLM Professional Knowledge

arXiv:2609.12366v1 Announce Type: new Abstract: We present ORQA, a method for testing occupation-level knowledge in large language models. Prior methods either map abstract LLM skills to occupations v

14 Sept 2026

SteerDuplex: Steerable Duplex Speech Dialogue Models

arXiv:2609.12623v1 Announce Type: cross Abstract: Full-duplex spoken dialogue models support low-latency turn taking, interruption handling, and backchanneling, yet a key capability remains underexplo

14 Sept 2026

Automating Quadratic Unconstrained Binary Optimization (QUBO) Formulation Generation from Natural Language

arXiv:2609.10629v1 Announce Type: new Abstract: Quadratic Unconstrained Binary Optimization (QUBO) is a central formulation for combinatorial optimization and has gained increasing attention due to it

12 Sept 2026

BenchShield: Formal Model-Backed Instrumentation for Reward Integrity in LLM-Agent Evaluation Infrastructure

arXiv:2609.11028v1 Announce Type: cross Abstract: LM-agent benchmarks increasingly function as interactive evaluation infrastructure. Agents observe state, call tools, modify workspaces, submit artifa

12 Sept 2026

Can AI Agents Detect and Repair Artifact Drift in Network Experiments?

arXiv:2609.09849v1 Announce Type: cross Abstract: In recent years, AI agents have evolved into capable assistants that carry out multi-step tasks in digital environments. The network systems community

Evaluating LLM-Simulated Conversations in Modeling Inconsistent and Uncollaborative Behaviors in Human Social Interaction

arXiv:2603.17094v2 Announce Type: replace Abstract: Simulating human conversations using large language models (LLMs) has emerged as a scalable methodology for modeling human social interaction. This

ReconPlusGen: Injecting Reconstruction Prior into Multi-view 3D Generation through Noise Inversion and Modulation

arXiv:2609.11129v1 Announce Type: new Abstract: Qualitative results and an illustration of our core idea. Top left: reconstruction results on benchmark images. Top right: reconstruction results on rea

Reference-Based Bias Detection in LLMs via Relative Representations of Hidden States

arXiv:2609.10060v1 Announce Type: new Abstract: Existing bias auditing methods typically rely on model outputs, requiring costly benchmarks or judge models and potentially missing internal shifts that

SGA: Plug&Play Geometric Verification for Educational Video Synthesis

arXiv:2607.18116v2 Announce Type: replace-cross Abstract: Recent work leverages Large Language Models (LLMs) to generate executable code for pedagogical animations using libraries such as Manim. Howev

AgentFairBench: Do LLM Agents Discriminate When They Act?

arXiv:2606.16723v2 Announce Type: replace Abstract: Large language model (LLM) agents increasingly take actions (screening applicants, recommending credit, triaging patients), yet fairness for LLMs is

Can Artificial Intelligence Support Healthcare and Mental Health Through Early Cyberbullying Detection ? The Impact of Emotion-Aware AI on Proactive Online Safety

arXiv:2609.09735v1 Announce Type: cross Abstract: Healthcare systems, mental health, and public well-being are increasingly affected by cyberbullying and harmful online interactions. This paper presen

CityPlanner: A Sandbox Agent for Executable Urban Planning

arXiv:2609.09578v1 Announce Type: cross Abstract: Urban planning is a real-world spatial optimization problem that requires selecting feasible actions from large candidate spaces under practical objec

DAREBench: Deployment-Aware and Reliable Evaluation of Models as Agents

arXiv:2609.06059v1 Announce Type: new Abstract: As large language models evolve from question-answering systems into general-purpose agents, evaluation must move beyond static answer correctness to as

EmoMed: An Emotionally-Aware Agent for Multimodal Medical Support with Real-Time Information Retrieval

arXiv:2609.07194v1 Announce Type: new Abstract: We present EmoMed - a multimodal medical consultation agent that adapts its responses based on users' emotional states while maintaining clinical accura

Experimental Analysis of Productive Interaction Strategy with ChatGPT: User Study on Function and Project-level Code Generation Tasks

arXiv:2508.04125v2 Announce Type: replace-cross Abstract: The application of Large Language Models (LLMs) is growing in the productive completion of Software Engineering tasks. Yet, studies investigat

Quality Metrics for LLM-Generated Asset Administration Shells: A Perturbation-Based Evaluation Approach

arXiv:2609.07290v1 Announce Type: cross Abstract: The rapid digital transformation of manufacturing, often referred to as Industry 4.0, relies on seamless interoperability between physical and softwar

The Audit Decides the Verdict: Instrument Effects Rival Demographic Bias in LLM Decision Audits

arXiv:2609.09048v1 Announce Type: cross Abstract: Whether a language model looks demographically biased can depend on how the audit asks its question. A charitable-aid benchmark reports that the same

Beyond the Verdict: Evidence-Aligned Evaluation of Visual Prompt-Injection Guardrails

arXiv:2609.05535v1 Announce Type: new Abstract: Verdict-only evaluation does not reveal whether a vision-language model (VLM) used the visual evidence that should support its decision. We study this p

GeoHeight-Bench: Towards Height-Aware Multimodal Reasoning in Remote Sensing

arXiv:2603.25565v2 Announce Type: replace Abstract: Current Large Multimodal Models (LMMs) in Earth observation are predominantly evaluated on planar optical tasks and often neglect the vertical dimen

Human-Centric Image Captioning with Subject-Centered Spatial Understanding

arXiv:2609.08300v1 Announce Type: new Abstract: While multimodal large language models (MLLMs) achieve remarkable performance on generic image captioning, they frequently suffer from structural halluc

MorphoOrgaAgent: A Foundation-Model-Based Multi-Agent System for Autonomous Organoid Analysis

arXiv:2609.08696v1 Announce Type: cross Abstract: Organoids are three-dimensional tissue models whose morphology provides important insights into tumor development, disease progression, and drug testi

Beyond Reproducibility: Towards Security-Aware Evaluation of Research Artifacts

arXiv:2605.06508v2 Announce Type: replace-cross Abstract: Research artifacts are widely shared to support reproducibility, and artifact evaluation (AE) has become common at many leading conferences. H

Contamination Inflates Scores but Rarely Reorders Large Language Model Leaderboards

arXiv:2609.02899v1 Announce Type: new Abstract: Benchmark contamination, the leakage of test items into training data, is widely described as a threat to the reliability of large language model (LLM)

LightEMMA: A Longitudinal Evaluation of Vision-Language Models for Autonomous Driving

arXiv:2505.00284v3 Announce Type: replace-cross Abstract: Rapid advances in vision-language models (VLMs) have generated growing interest in their application to autonomous driving. A prevailing assum

TabScope: Question-Adaptive Scope Selection for Table Question Answering

arXiv:2609.03395v1 Announce Type: cross Abstract: Large Language Models (LLMs) have shown strong performance on table question answering, yet their accuracy often degrades as table size increases. We

To What Extent Do Large Language Models Understand Bangla Idioms?

arXiv:2609.03410v1 Announce Type: new Abstract: Idiomatic expressions are an integral part of natural language, reflecting cultural nuances and posing unique challenges for computational models, parti

Cite or Decline: A Strict Course-Grounded Chatbot for STEM Lecture Videos

arXiv:2609.01846v1 Announce Type: new Abstract: Recorded lecture videos, often enhanced with search and summarization features, are a standard study resource. However, students cannot easily ask cours

From Visual Cues to Spoken Narration: Rethinking Audio Description

arXiv:2609.01725v1 Announce Type: new Abstract: Audio Description (AD) provides spoken narration of visual events during dialogue gaps, making movies accessible to visually impaired audiences. The pro

Learning to Fuse LLMs with Ontology Rankers for Rare-Disease Diagnosis

arXiv:2609.02473v1 Announce Type: new Abstract: Ontology rankers remain useful for rare-disease diagnosis because each candidate can be traced to matched patient phenotypes. Large language models (LLM

Mediocrity is the key for LLM as a Judge Anchor Selection

arXiv:2603.16848v2 Announce Type: replace Abstract: The ``LLM-as-a-judge'' paradigm has become a standard method for evaluating open-ended generation. To address the quadratic scalability costs of pai

Rating the Raters: Rasch Measurement Theory for LLM Evaluation

arXiv:2608.27463v2 Announce Type: replace Abstract: LLMs now sit on every side of evaluation: as examinees scored on benchmarks, judges of other models' outputs, and raters of human-generated content.

Semantic Signal-Assisted Inspection and Recovery Allocation in Reverse Logistics

arXiv:2609.02116v1 Announce Type: new Abstract: Reverse-logistics operators often decide how to inspect and route returned assets before their condition is fully observed, while full inspection consum

CordisBench: Can Language Models Reason About Component Lifecycles in Dynamic Agent Harnesses?

arXiv:2609.01600v1 Announce Type: cross Abstract: Dynamic agent harnesses let language models change the software that shapes their own execution. This flexibility brings a new reasoning burden: a loc

Defense-as-Skill: Evolving Runtime Guard Skill for Skill-Augmented Agents

arXiv:2609.01487v1 Announce Type: cross Abstract: Skill-augmented agents load reusable skills as persistent runtime context, improving task performance but also giving malicious skills a durable chann

Disclosure-Gated User Simulation for Companion-Agent Evaluation

arXiv:2609.00982v1 Announce Type: cross Abstract: Using a large language model to play the user is now standard in scalable evaluation. It has a repeatedly diagnosed failure: the simulated user is exc

Dyn-3D: Unveiling and Resolving Ego-Motion Ambiguity in Vision-Language Models

arXiv:2609.01059v1 Announce Type: new Abstract: As Vision-Language Models (VLMs) tackle dynamic 3D spatial reasoning, ego-motion perception becomes essential to resolve monocular scale ambiguity. Howe

FinLifeBench: Exhaustive Life-Event History and Financial-State Reconstruction from Longitudinal Banking Dialogue

arXiv:2609.01198v1 Announce Type: new Abstract: Repeated banking interactions require assistants to maintain complete, current, and traceable customer records as life changes emerge incidentally in ro

HarnessDev: Can LLMs Create and Evolve Their Own Agent Harness?

arXiv:2609.01437v1 Announce Type: cross Abstract: As agents move from research prototypes to deployed tools, their capability increasingly depends on model-external execution infrastructure, commonly

OpenAgentFlow: Enabling System-Wide Safety Boundaries for Heterogeneous AI Agent Fleets

arXiv:2609.00015v1 Announce Type: new Abstract: AI agents powered by large language models are evolving from isolated assistants into heterogeneous systems in which multiple agents, planners, controll

Revealing Multi-View Hallucination in Large Vision-Language Models

arXiv:2603.23934v2 Announce Type: replace-cross Abstract: Large vision-language models (LVLMs) are increasingly being applied to multi-view image inputs captured from diverse viewpoints. Despite this

StudyBench: Can Self-Evolution Squeeze Textbooks for Olympiad Capability?

arXiv:2609.00787v1 Announce Type: new Abstract: Humans need to study only a handful of well-written textbooks to master a discipline and attempt its hardest problems. We argue that an ideal self-evolu

TempCloze: Can Video-LLMs Identify the Missing Middle?

arXiv:2609.01515v1 Announce Type: cross Abstract: Temporal reasoning benchmarks for Video-LLMs are often mediated by language, leaving room for linguistic shortcuts from option wording, answer correla

Where the Verifier Fails: A Category-Level Audit of Reward Signals in RLVR

arXiv:2609.01354v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) and standard benchmark evaluation both rely on an automatic verifier that turns a free text answer

AgenTRIM: Tool Risk Mitigation for Agentic AI

arXiv:2601.12449v2 Announce Type: replace-cross Abstract: AI agents are autonomous systems that combine LLMs with external tools to solve complex tasks. While such tools extend capability, improper to

1 Sept 2026

Are LLMs Ready to Assist Physicians? PhysAssistBench for Interactive Doctor-Patient-EHR Assistance

arXiv:2606.18613v4 Announce Type: replace-cross Abstract: The most plausible near-term role of medical LLMs is to assist rather than replace physicians, yet current evaluations often test isolated cap