Search: “evaluation”

arXiv:2610.07403v1 Announce Type: new Abstract: Long-term memory allows Large Language Models (LLMs) to maintain personalized context across interactions, but retrieved user history can induce memory-

arXiv:2610.07669v1 Announce Type: cross Abstract: We assessed the value of two live human-AI interactions in a precision disease control project in California vineyards. The project tested whether 202

arXiv:2610.07563v1 Announce Type: new Abstract: Large language models (LLMs) have the potential to meet a key goal in economics: a quantitative model of household decision making, across a variety of

arXiv:2610.06511v1 Announce Type: new Abstract: To evaluate the frontier, we must measure models not by what they say, but by what they can engineer and build in grounded physical environments. We int

arXiv:2610.05590v1 Announce Type: new Abstract: Cold-start drug-drug interaction (DDI) prediction tests whether models can identify clinically significant interactions for drugs without training-time

arXiv:2610.06306v1 Announce Type: new Abstract: General purpose language models are increasingly able to control robotic hardware. Understanding the capabilities and safety of these models when embodi

arXiv:2610.05810v1 Announce Type: cross Abstract: Machine learning (ML)-powered poisoning attack vectors are adversarial maneuvers whereby an attacker intentionally inserts, corrupts, or alters traini

arXiv:2610.02877v1 Announce Type: new Abstract: Large language models (LLMs) are widely used as automatic judges, with validity typically assessed via alignment with human scores. However, aggregate a

arXiv:2610.02878v1 Announce Type: new Abstract: We evaluate the performance of Vision Language Models in Visual Question Answering (VQA) when questions violate Grice's maxims. To do this, we use VLMs

arXiv:2610.02122v1 Announce Type: cross Abstract: Real-world enterprise data science and analytics workflows require reasoning across dozens of tables, performing statistical analyses, and acting on t

arXiv:2610.00554v1 Announce Type: new Abstract: We evaluate hybrid quantum-classical machine learning for the reduced-order prediction of spatiotemporal brain deformation fields. To mitigate the compu

arXiv:2609.38954v1 Announce Type: cross Abstract: Large language model (LLM) agents could help security operations centers (SOCs) investigate advanced persistent threats (APTs) by turning weak leads i

arXiv:2609.39863v1 Announce Type: new Abstract: Large language models frequently fail to balance staying truthful with being supportive. They often exhibit sycophancy in responses to users, agreeing w

arXiv:2609.36488v1 Announce Type: new Abstract: Large language model (LLM) agents have demonstrated strong performance on complex web navigation tasks, yet they remain brittle in real-world settings w

arXiv:2603.08064v3 Announce Type: replace Abstract: Most evaluations of generative models rely on feature-distribution metrics such as FID, which operate on continuous recognition features that are ex

arXiv:2609.33401v2 Announce Type: replace-cross Abstract: Model-based judges support agent security by detecting prompt injections, assessing interaction risks, and screening harmful requests. System

arXiv:2609.33769v1 Announce Type: new Abstract: Quantitative evaluation of generative models for radiology remains challenging. Clinically relevant structures are often small and infrequent, feature s

arXiv:2609.31823v1 Announce Type: new Abstract: Large vision-language models (LVLMs) have demonstrated remarkable performance on multimodal reasoning benchmarks, yet their perceptual reliability under

arXiv:2609.35143v1 Announce Type: new Abstract: AI agents increasingly carry out long-horizon professional work, but their evaluations rarely require a finished creative deliverable. To this end, we i

arXiv:2609.31506v1 Announce Type: cross Abstract: Large language models (LLMs) exhibit substantial performance disparities between high- and low-resource languages. Beyond lower task performance, they

arXiv:2609.30986v1 Announce Type: new Abstract: As large language models increasingly mediate information access, factually accurate and independent answers are critical. However, these models can exh

arXiv:2609.31415v1 Announce Type: new Abstract: Position-independent KV cache reuse aims to reduce latency in retrieval-augmented generation by reusing chunk-level KV caches across prompts. We show th

arXiv:2505.14368v3 Announce Type: replace-cross Abstract: Recent studies demonstrate that Large Language Models (LLMs) are vulnerable to attacks that generate harmful or sensitive outputs. As open-sou

arXiv:2609.30749v1 Announce Type: new Abstract: Data Science Code Translation (DSCT) is the process of converting code between data science libraries while preserving functional equivalence and enabli

arXiv:2609.29496v1 Announce Type: new Abstract: Natural language explanation generation serves as a key mechanism for exposing and evaluating vision-language reasoning. Prior work on explanation-drive

arXiv:2609.28910v1 Announce Type: cross Abstract: Effective robot assistance beyond narrow roles and repetitive tasks requires robots to be proactive - to decide what needs to be done rather than wait

arXiv:2609.22110v1 Announce Type: new Abstract: Background: Large language models (LLMs) can improve healthcare information delivery in low-resource settings but may produce inaccurate or culturally i

arXiv:2609.22204v1 Announce Type: new Abstract: This exploratory pilot study evaluates the scope and perceived accuracy of personal information output from ongoing conversational interactions in gener

arXiv:2609.24778v1 Announce Type: new Abstract: Learning robot manipulation policies from human video demonstrations constitutes a promising avenue for scalable robot learning. However, comparing diff

arXiv:2609.23570v1 Announce Type: cross Abstract: Coding agents operate on real repository coding tasks, and persistent memory systems promise to reuse experience across tasks. Yet existing evaluation

arXiv:2609.22038v1 Announce Type: new Abstract: We introduce QuranicMMLU, a benchmark for evaluating generative AI on Quranic Arabic across multiple dimensions of linguistic complexity. Existing Quran

arXiv:2609.19885v1 Announce Type: new Abstract: Interpreter agents built on machine translation (MT) increasingly mediate live conversation between people who do not share a language, yet we still eva

arXiv:2609.17546v1 Announce Type: new Abstract: In this position paper, we argue that legal LLMs' hallucinations should be evaluated as a failure of legal warrant rather than as factual inaccuracy or

arXiv:2609.18134v1 Announce Type: cross Abstract: Methodological progress in artificial intelligence (AI) for electronic health records (EHRs) depends on our ability to determine which algorithms work

arXiv:2609.16519v1 Announce Type: new Abstract: Scientific research increasingly relies on large, heterogeneous data sources, motivating interest in retrieval-augmented generation (RAG) systems that p

arXiv:2609.14811v1 Announce Type: new Abstract: We introduce Crypto Accounting Bench (CAB), a benchmark for assessing whether frontier and open-weight language models can reconstruct the complete jour

arXiv:2609.13773v1 Announce Type: cross Abstract: LLM-as-a-Judge evaluators are increasingly used to score open-ended generation, yet a judge's correlation with human ratings on its development set ma

arXiv:2609.13617v1 Announce Type: new Abstract: We evaluate whether the improvements advertised for ChatGPT Images 2.5 translate into better performance on forgery tasks with predetermined answers. We

arXiv:2609.15855v1 Announce Type: cross Abstract: % !TEX root = ../main.tex People increasingly use large language models (LLMs) for mental health support, yet their safety in evolving, high-risk conv

arXiv:2609.10778v1 Announce Type: new Abstract: Machine learning models can achieve strong test performance while relying on demographic or acquisition-related shortcuts. We propose counterfactual (CF

arXiv:2609.10410v1 Announce Type: new Abstract: The growing complexity of content moderation policies presents a critical challenge for their consistent operationalization. While foundation models pos

arXiv:2505.19333v2 Announce Type: replace Abstract: Current evaluations of Large Language Model (LLM) steering techniques focus on task-specific performance, overlooking how well steered representatio

arXiv:2609.08209v1 Announce Type: new Abstract: Learning from Observation (LfO) is a fundamental robotic capability that replicates how humans and animals socially learn from each other. Beyond its bi

arXiv:2609.03502v1 Announce Type: cross Abstract: In low-resource settings, deploying TTS typically requires choosing between a large voice-cloning model with costly inference or a compact fixed-voice

arXiv:2609.03588v1 Announce Type: new Abstract: As LLMs increasingly act through tools, they must reconcile user instructions, parametric knowledge, and dynamic environmental observations before takin

arXiv:2601.19956v2 Announce Type: replace-cross Abstract: As Speech Language Models (SLMs) transition from personal devices to shared, multi-user environments such as smart homes, a new challenge emer