Search: “evaluation”

arXiv:2608.30730v1 Announce Type: cross Abstract: Long-horizon agentic tasks go beyond chaining short tasks over more interaction turns. Their evolving dynamic environments and long-range dependencies

arXiv:2608.28678v1 Announce Type: new Abstract: Scalable Vector Graphics (SVGs) power much of the modern visual ecosystem, yet state-of-the-art generative models focus almost entirely on rasterized im

arXiv:2606.09389v2 Announce Type: replace Abstract: As large language models (LLMs) are increasingly applied to real-world legal tasks, evaluating the reliability of their open-ended legal responses h

arXiv:2502.10577v2 Announce Type: replace-cross Abstract: Instruct-based large language models (LLMs) have been shown to propagate and even amplify gender bias when prompted with contextually constrai

arXiv:2608.28405v1 Announce Type: new Abstract: Current cultural evaluations for large language models (LLMs) often reduce culture to single-turn factual recall via MCQs, failing to capture a common u

arXiv:2608.27783v1 Announce Type: cross Abstract: Speech LLMs are usually graded after they answer, although an operating system first has to decide whether a waveform should be sent to the model. We

arXiv:2608.26137v1 Announce Type: new Abstract: Second-language (L2) English learners can rarely rehearse speaking with a partner. Speaking is also the most anxiety-laden skill. These gaps drive a fas

arXiv:2608.23065v1 Announce Type: cross Abstract: Cultural understanding in video means more than recognizing what is visible; it requires grasping the symbolic and temporal significance of cultural c

arXiv:2606.17904v2 Announce Type: replace Abstract: Language models increasingly serve as advisory systems in maintenance operations. To prevent hallucination, recent systems ground these models in pr

arXiv:2608.21420v1 Announce Type: new Abstract: Thematic analysis (TA) has long been regarded as an inherently human, reflexive, and interpretive process. However, the extent to which LLM-generated TA

arXiv:2608.21430v1 Announce Type: new Abstract: Multimodal language models increasingly show promise for enabling the large-scale computational analysis of film, opening up new avenues for learning ab

arXiv:2608.23300v1 Announce Type: cross Abstract: Algorithmic solver effort is often assumed to align with perceived puzzle difficulty, but this assumption is rarely tested against human solving data.

arXiv:2608.21488v1 Announce Type: cross Abstract: While machine learning models have demonstrated strong performance in many domains, these models have shown profound vulnerabilities when they are exp

arXiv:2606.13610v2 Announce Type: replace-cross Abstract: Search-augmented LLMs increasingly mediate everyday consumer recommendations by retrieving live web content. This creates a new risk: LLM reco

arXiv:2608.20149v1 Announce Type: new Abstract: This article explores the performance of analytical and neural-based hillshading methods in a dense urban environment using high-resolution digital elev

arXiv:2608.16893v1 Announce Type: cross Abstract: Expert surveys are widely used in security research to study practitioner workows and decision-making, yet recruiting domain experts - especially in S

NVIDIA SkillEvaluator is an open‑source framework that measures how pre‑verified “skills” improve AI agent performance through static checks, distinctiveness analysis, and live task runs in isolated e

arXiv:2608.14651v1 Announce Type: new Abstract: Effective disaster risk communication is a foundational humanitarian challenge, yet current emergency infrastructure fails to meet the needs of individu

arXiv:2608.14583v1 Announce Type: cross Abstract: Vehicular Ad Hoc Networks (VANETs) are a key component of intelligent transportation systems, enabling real-time communication between vehicles. Howev

arXiv:2505.21333v3 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have achieved considerable accuracy in Optical Character Recognition (OCR) from static images. However, the

arXiv:2608.16323v1 Announce Type: cross Abstract: The spread of misinformation on social networks poses a significant challenge to online communities and society at large. Not all users contribute equ

arXiv:2608.15555v1 Announce Type: new Abstract: Video models are increasingly used to predict what happens next in a scene, yet the metrics commonly used to compare their outputs say little about whet

arXiv:2608.14284v1 Announce Type: cross Abstract: Fine-grained robotic evaluation matters for understanding embodied models, going beyond binary success rates and rule-based process scores. We present

arXiv:2608.13329v1 Announce Type: new Abstract: A model that behaves differently when it senses it is being tested would undermine the evaluations we rely on, so recent work has sought to read that se

arXiv:2608.12663v1 Announce Type: cross Abstract: Wildfire susceptibility mapping typically relies on physical variables assembled from multiple remote-sensing, climate, and geospatial products. Alpha

arXiv:2608.12197v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly used in circuit design workflows, yet their reliability on simulator-facing SPICE netlist recognition an

arXiv:2608.10801v1 Announce Type: new Abstract: Spatio-temporal PV data are essential for understanding adoption processes in off-grid regions, yet such data remain largely unavailable. Automated segm

arXiv:2608.08164v1 Announce Type: cross Abstract: Knowledge Distillation is a widely adopted technique in the training and fine-tuning of large language models (LLMs) enabling transfer of structured i

arXiv:2509.21945v2 Announce Type: replace-cross Abstract: To efficiently tune configuration for better software system performance (e.g., latency) at the deployment and maintenance stage, many tuners

arXiv:2608.06891v1 Announce Type: new Abstract: Agent skills provide reusable procedural knowledge that helps agents solve specialized tasks. As their use expands, evaluating skill quality becomes inc

arXiv:2608.05381v1 Announce Type: new Abstract: Current Multimodal Large Language Models (MLLMs) can process diverse sensory inputs, yet their reasoning remains heavily biased toward a dominant modali

arXiv:2608.05263v1 Announce Type: new Abstract: Multi-agent orchestration frameworks are moving from demos to production, yet benchmarks typically report task accuracy without diagnosing why a pipelin

arXiv:2608.05616v1 Announce Type: new Abstract: Despite the remarkable progress of large vision language models (LVLMs), object hallucination remains a fundamental challenge that hinders their trustwo

arXiv:2608.06057v1 Announce Type: new Abstract: Tool-calling agents infer task state from accumulated dialogue and tool traces. In persistent interactions, however, historical traces may remain struct

arXiv:2608.04519v1 Announce Type: new Abstract: Benchmarking machine unlearning methods is critical to understand whether sensitive knowledge is removed from large language models (LLMs) or not. Curre

arXiv:2608.04299v1 Announce Type: new Abstract: Fifteen years ago, Low proposed that pun translators should stop searching for equivalent words and instead search for new points of contact between sou

arXiv:2608.04591v1 Announce Type: cross Abstract: Large language models (LLMs) are often asked whether something is absent from a record, list, or retrieved context. Yet non-observation licenses a neg

arXiv:2601.16217v2 Announce Type: replace-cross Abstract: Large language models increasingly mediate multilingual professional communication, where useful generation requires adapting to community con

arXiv:2608.03311v1 Announce Type: cross Abstract: Enterprise compliance management requires rapid adaptation to evolving regulatory frameworks (e.g., DORA, AI RMF, FedRAMP) and tight remediation SLAs.

arXiv:2608.01725v1 Announce Type: cross Abstract: Modern computing and networking infrastructure emits telemetry continuously, yet operators convert it into decisions with a separate predictor per tas

arXiv:2608.01648v1 Announce Type: new Abstract: Hardware error logs in high-performance computing (HPC) systems provide early signals of abnormal behavior, yet there remain challenges in effectively f

arXiv:2608.01028v1 Announce Type: new Abstract: Deploying Vision-Language-Action (VLA) robots as mobile edge nodes within wireless sensor networks (WSNs) requires robust protection against physical ad

arXiv:2608.01409v1 Announce Type: new Abstract: Biomedical fact-checking systems must do more than predict whether a claim is supported, contradicted, or unaddressed: they should also produce evidence

Hello fellow local AI people! I took 'you must create your own benchmarks' literally, and built a website for this. How does the end result look like Let's say I want to know which model has most comm

arXiv:2607.27386v1 Announce Type: cross Abstract: Diffusion Language Models (DLMs) offer a compelling alternative to autoregressive (AR) generation by enabling bidirectional context and iterative refi

arXiv:2607.09306v3 Announce Type: replace Abstract: Behavioural auditing asks whether a language model behaves as it claims, but detection scores are reported without separating two targets: whether a