Search: “evaluation”

arXiv:2601.07794v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly evaluated on their ability to perform multi-hop reasoning, i.e., to combine multiple pieces of i

arXiv:2608.29685v1 Announce Type: new Abstract: Early failure prediction is important for long-horizon agents, as it enables timely intervention and can reduce inference and tool-use costs. Uncertaint

arXiv:2608.30853v1 Announce Type: new Abstract: While automatic speech recognition (ASR) models have achieved remarkable improvements in recent years, performance disparities persist across different

arXiv:2608.28596v1 Announce Type: new Abstract: Large language model (LLM) agents are increasingly embedded in scientific workflows for literature analysis, drafting, and review. Existing systems adva

arXiv:2608.29996v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) achieve strong performance across many multimodal tasks; however, they often exploit spurious object-background c

arXiv:2608.30270v1 Announce Type: new Abstract: Indirect speech acts (ISAs) require pragmatic reasoning over context, as directive intent can- not be inferred from surface form alone. Prior text-based

arXiv:2608.30835v1 Announce Type: cross Abstract: Background and Objective: Quality control is a prerequisite for whole-slide image analysis, yet the benchmarks on which quality-control methods are co

arXiv:2608.31100v1 Announce Type: new Abstract: Large language models (LLMs) increasingly interact with external environments and accumulate substantial behavioral experience, yet existing agent bench

arXiv:2608.29481v1 Announce Type: new Abstract: Pediatric serious illness communication (SIC) is critically important, yet scalable communication training for clinicians remains limited. Compared with

arXiv:2608.30785v1 Announce Type: new Abstract: Production agent skills are directory bundles, not isolated prompts. The root is loaded at activation; references, schemas, scripts, assets, and nested

arXiv:2608.29921v1 Announce Type: cross Abstract: The output of a Language Model can be tampered with while the model is writing it. A simple test can thus be constructed by evaluating the model's per

arXiv:2608.29992v1 Announce Type: new Abstract: This paper presents an end-to-end, agent-driven pipeline for the LoD3 reconstruction of facade openings in 3D city models, producing directly usable Cit

arXiv:2608.30068v1 Announce Type: new Abstract: Films communicate through deliberate creative choices, including lighting, color, composition, editing, dialogue, music, and sound. Humans naturally int

arXiv:2608.29999v1 Announce Type: cross Abstract: Large audio-language models (LALMs) describe audio at the clip level but cannot assign timestamps to the events, speakers, or sounds they identify. De

arXiv:2608.28641v1 Announce Type: cross Abstract: Most evaluations for coding agents are conducted exclusively in English, which does not reflect real-world multilingual deployment. We present Termina

arXiv:2608.29456v1 Announce Type: new Abstract: As artificial intelligence is increasingly integrated into chest X-ray (CXR) interpretation, triage, and clinical decision support, understanding its vu

arXiv:2508.14273v3 Announce Type: replace Abstract: As researchers increasingly adopt LLMs as writing assistants, generating high-quality research paper introductions remains both challenging and esse

arXiv:2608.30923v1 Announce Type: cross Abstract: Stream learning is commonly evaluated through predictive performance and adaptation to concept drift. However, sustained operation of a stream learner

arXiv:2602.08336v3 Announce Type: replace Abstract: Unified multimodal models (UMMs) aim to integrate multimodal understanding and generation within a unified architecture, yet it remains unclear to w

arXiv:2608.30508v1 Announce Type: new Abstract: Long-term memory is increasingly important for conversational agents, yet existing benchmarks primarily measure memory through pointwise factual recall:

arXiv:2608.31118v1 Announce Type: new Abstract: The effect of Large Language Model (LLM) scale on ontology learning (OL) performance remains insufficiently characterized. We present a controlled evalu

arXiv:2608.28247v1 Announce Type: new Abstract: Change detection in Earth observation (EO) is critical for monitoring land surface transformations, yet recent research in the field is constrained by i

arXiv:2608.27505v1 Announce Type: new Abstract: Reinforcement learning from human feedback (RLHF) has become the dominant paradigm for aligning large language models (LLMs) with human preferences. How

arXiv:2401.09940v2 Announce Type: replace Abstract: Expected Goals (xG) has emerged as a popular tool for evaluating finishing skill in soccer analytics. It involves comparing a player's cumulative xG

arXiv:2608.28053v1 Announce Type: new Abstract: Chinese neologisms exploit diverse and unique linguistic mechanisms, such as phonetic substitution (e.g., 886 for ``bye-bye'') and visual character deco

arXiv:2603.28387v3 Announce Type: replace-cross Abstract: Trustworthy clinical AI must use real evidence and avoid relying on surface-level artifacts. We evaluate 12 open-weight vision-language models

arXiv:2608.27512v1 Announce Type: cross Abstract: Post-training quantization is often treated as a semantically neutral optimization for edge deployment of Large Language Models. When a full-precision

arXiv:2608.27626v1 Announce Type: cross Abstract: Quantum DeepONet accelerates neural-operator inference by evaluating an orthogonally parameterized network on a quantum computer, reproducing in ideal

arXiv:2608.27481v1 Announce Type: new Abstract: Knowledge-intensive multi-hop question answering requires systems to select evidence and compose dependent facts, yet multilingual benchmarks usually tr

arXiv:2608.26947v1 Announce Type: cross Abstract: Embodied agents need environments that are visually diverse, physically interactive, and changing over time. Procedural simulators can generate large

arXiv:2608.26204v1 Announce Type: cross Abstract: Computer Use Agents (CUAs) are increasingly deployed to navigate mobile and desktop applications on behalf of users, yet no benchmark comprehensively

arXiv:2608.27169v1 Announce Type: new Abstract: Ancient Chinese artifact text recognition is fundamental to heritage digitization, and benchmarks for ancient texts are essential for evaluating current

arXiv:2608.27219v1 Announce Type: new Abstract: Mental health assessment relies on episodic self-report scales, which convert subjective states such as stress into numerical scores but provide only sp

arXiv:2608.26899v1 Announce Type: new Abstract: Automated candidate-job matching systems are increasingly classified as high-risk AI under emerging regulation, yet auditing them for demographic bias i

arXiv:2608.27021v1 Announce Type: cross Abstract: LLM-based agents are increasingly proposed for network fault diagnosis, but existing benchmarks evaluate them only on accurate tickets and always assu

arXiv:2608.26166v1 Announce Type: cross Abstract: Advancing reasoning capabilities allow large language models (LLMs) to tackle increasingly complex problems, while reasoning traces - intermediate ste

arXiv:2608.26849v1 Announce Type: new Abstract: User behavior simulation with large language models~(LLMs) is increasingly used to support multi-agent ecosystem simulation. Existing simulators typical

arXiv:2608.27345v1 Announce Type: cross Abstract: Recent video generation models are increasingly framed as world models. Many physical processes can unfold in more than one valid way. Therefore, a wo

arXiv:2608.26882v1 Announce Type: cross Abstract: Industrial control systems (ICSs) rely on programmable logic controllers (PLCs) to connect networked computation with physical control. Tool-using lar

arXiv:2608.27035v1 Announce Type: new Abstract: Korean constituency parsing raises a representational challenge because the terminal units of a phrase-structure tree do not straightforwardly correspon

arXiv:2608.27049v1 Announce Type: new Abstract: Reliable assessment of causal research designs in the social sciences is critical for evidence-based policy-making, yet has so far relied entirely on ma

arXiv:2608.26336v1 Announce Type: cross Abstract: Recent advancements in generative models are pushing video generation toward unbounded streaming audio-video generation for real-time interactive worl

arXiv:2606.07867v2 Announce Type: replace Abstract: Are tool-calling LLM agents equally safe throughout a conversation? We discover they are not: agents are most vulnerable at the very start of a sess

arXiv:2608.27127v1 Announce Type: new Abstract: Internet memes are a pervasive form of multimodal online communication; however, such communication often involves users from diverse linguistic and cul

arXiv:2608.25022v1 Announce Type: new Abstract: As people adopt transformer-based language models (e.g., ChatGPT and Gemini) for an increasing number of use-cases, it is important to know how such mod

arXiv:2608.25940v1 Announce Type: new Abstract: Physical AI models are evaluated on suites of benchmarks that differ across model reports, leaving the model-by-benchmark matrix sparse and the relation