Search: “evaluation”

arXiv:2602.23622v2 Announce Type: replace-cross Abstract: Significant progress has been made in the field of Instruction-based Image Editing Models (IIEMs). However, while these models demonstrate pla

arXiv:2605.18937v1 Announce Type: new Abstract: Patient-managed Personal Health Records (PHRs) promises to empower patients to better understand their health; but information in the record is complex,

arXiv:2605.18754v1 Announce Type: new Abstract: Multiview 3D evaluation assumes that the images being scored are observations of one static 3D scene. This assumption can fail in NVS and sparse-view re

arXiv:2510.08886v3 Announce Type: replace Abstract: Going beyond simple text processing, financial auditing requires detecting semantic, structural, and numerical inconsistencies across large-scale di

arXiv:2602.12871v2 Announce Type: replace Abstract: Large language models (LLMs) have attracted growing interest as supportive tools for psychiatric assessment and clinical decision support. However,

arXiv:2605.17044v1 Announce Type: new Abstract: Large language models (LLMs) increasingly serve as interactive social agents, yet their ability to maintain coherent and authentic persona-level role-pl

arXiv:2605.18380v1 Announce Type: new Abstract: We introduce an extensive qualitative spatial and temporal reasoning (QSTR) benchmark for evaluating large language models (LLMs). We pose questions con

Simulation is a core part of how we build and evaluate robots. I spoke about simulation, robotics, and the role of interactive worlds at AI Engineer Singapore this past weekend! Full talk : https://ww

when evaluating long running agents, all of your evals don't need to be end to end. i'm working on a proper blog about this, but in our evals for our agents that run for 30-60 minutes, we have two set

arXiv:2605.15104v1 Announce Type: new Abstract: Voice agents increasingly require reliable tool use from speech, whereas prominent tool-calling benchmarks remain text-based. We study whether verified

arXiv:2605.13841v1 Announce Type: cross Abstract: Voice agents, artificial intelligence systems that conduct spoken conversations to complete tasks, are increasingly deployed across enterprise applica

arXiv:2605.13684v1 Announce Type: new Abstract: We study the optimal scale at which real-valued function classes exhibit uniform convergence and learnability. Our main result establishes a scale-sensi

arXiv:2605.12147v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used to simulate human behavior, but their ability to simulate individual privacy decisions is not well

arXiv:2605.12015v1 Announce Type: cross Abstract: Reusable skills are becoming a common interface for extending large language model agents, packaging procedural guidance with access to files, tools,

arXiv:2605.11307v1 Announce Type: new Abstract: Image-to-code generation tests whether a vision-language model (VLM) can recover the structure of an image enough to express it as executable code. Exis

arXiv:2512.17738v2 Announce Type: replace Abstract: User-generated content (UGC) is characterised by frequent use of non-standard language, from spelling errors to expressive choices such as slang, ch

arXiv:2605.09823v1 Announce Type: cross Abstract: We introduce CalBench, a controlled evaluation environment for studying multi-agent coordination through calendar scheduling. In CalBench, N agents ea

arXiv:2605.08965v1 Announce Type: new Abstract: Despite strong performance of Multimodal Large Language Models (MLLMs) on multimodal tasks, predicting whether and why an image is persuasive remains ch

arXiv:2605.08462v1 Announce Type: cross Abstract: Hallucination remains a persistent challenge in Large Language Models (LLMs), particularly in context-grounded settings such as RAG and agentic AI sys

arXiv:2503.06047v2 Announce Type: replace Abstract: Large language model (LLM)-based agents are increasingly applied to complex strategic environments that demand long-horizon reasoning, multi-agent i

arXiv:2605.08636v1 Announce Type: new Abstract: Federated fine-tuning offers a promising paradigm for adapting large language models (LLMs) on edge devices by leveraging the rich, diverse, and continu

arXiv:2605.08787v1 Announce Type: new Abstract: Recent advances in 3D medical vision-language models have enabled joint reasoning over volumetric images and text, showing strong performance in medical

arXiv:2605.10125v1 Announce Type: new Abstract: Artificial intelligence (AI) tools are being incorporated into scientific research workflows with the potential to enhance efficiency in tasks such as d

arXiv:2605.06669v1 Announce Type: cross Abstract: Educational LLM tutors face a core AI alignment challenge: they must follow user intent while preserving pedagogical constraints and safety policies.

arXiv:2603.09652v3 Announce Type: replace Abstract: With the rapid advancement of Large Language Models (LLMs) in code generation, human-AI interaction is evolving from static text responses to dynami

arXiv:2503.05085v2 Announce Type: replace Abstract: Recent advances in large language models (LLMs) have fundamentally reshaped speech-to-speech (S2S) systems, enabling increasingly natural spoken int

arXiv:2605.07001v1 Announce Type: cross Abstract: Architectural code smells erode software maintainability and are costly to repair manually, yet unlike localized bugs, they require cross-module reaso

arXiv:2602.04939v2 Announce Type: replace Abstract: Modern T2V/I2V generators synthesize people increasingly hard to distinguish from authentic footage, while current evaluation suites lag: legacy ben

This post emphasizes a three-stage lifecycle for AI/LLM applications: evaluating systems before deployment to ensure quality, monitoring their performance in production to catch issues, and using the

EO-1 is now available directly in LeRobot! You can train, evaluate, and deploy EO-1 through the standard LeRobot policy interface, making it much easier to try EO-1 on robot-control workflows. Huge th

arXiv:2605.03227v1 Announce Type: new Abstract: Large Language Models (LLMs) have demonstrated strong capabilities in natural language understanding and reasoning. However, their ability to perform ex

arXiv:2603.13824v2 Announce Type: replace-cross Abstract: Recent advances in text-to-audio generation enable models to translate natural-language descriptions into diverse musical output. However, the

arXiv:2510.25224v3 Announce Type: replace Abstract: While Large Language Models (LLMs) are increasingly used in agentic frameworks to assist individual users, there is a growing need for agents that c

arXiv:2603.10384v2 Announce Type: replace Abstract: Evaluating LLM reliability via scalar probabilities often fails to capture the structural dynamics of reasoning. We introduce TRACED, a framework th

arXiv:2605.02240v1 Announce Type: new Abstract: We introduce PhysicianBench, a benchmark for evaluating LLM agents on physician tasks grounded in real clinical setting within electronic health record

arXiv:2605.02173v1 Announce Type: new Abstract: We evaluate the long-context retrieval and reasoning capabilities of five frontier large language models with advertised 1M-token context windows on a c

We’ve been working closely with the @harvey team on the launch of the Legal Agent Benchmark, a product focused on evaluating how open-weight models perform on long-horizon, real-world legal tasks. Che

arXiv:2605.00245v1 Announce Type: new Abstract: Large language models (LLMs) are now being explored for defense applications that require reliable and legally compliant decision support. They also hol

arXiv:2605.02351v1 Announce Type: new Abstract: Molecular Vibe Coding, a paradigm where chemists interact with LLMs to generate executable programs for molecular tasks, has emerged as a flexible alter

arXiv:2605.01333v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have emerged as a promising paradigm for dental image analysis. However, their ability to capture the multi-lev

arXiv:2507.14201v3 Announce Type: replace-cross Abstract: We present ExCyTIn-Bench, the first benchmark to Evaluate an LLM agent X on the task of Cyber Threat Investigation through security questions

arXiv:2602.00095v3 Announce Type: replace-cross Abstract: Multimodal Large Language Models (MLLMs) hold significant promise for revolutionizing traditional education and reducing teachers' workload. H

arXiv:2604.26211v1 Announce Type: new Abstract: In order to automate AI research we introduce a full, end-to-end framework, OMEGA: Optimizing Machine learning by Evaluating Generated Algorithms, that

arXiv:2604.26243v1 Announce Type: cross Abstract: Achieving realistic human-like conversation for virtual characters requires not only a simple memorization and recall of past events, but also the str

What people call 'distillation' is a super common practice (you use other models to benchmark your model, to evaluate your inputs or to add a little bit to your datasets) that in my opinion should be

With this integration, teams can optimize data, launch fine-tuning, evaluate results, and deploy on Together AI inference through a tighter workflow. Learn more: http://www.together.ai/blog/announcing