Search: “evaluation”

arXiv:2604.06802v1 Announce Type: new Abstract: Recent AI systems have achieved gold-medal-level performance on the International Mathematical Olympiad, demonstrating remarkable proficiency at competi

arXiv:2604.06176v1 Announce Type: cross Abstract: We present an empirical study of embedding-based retrieval under realistic conversational settings, where queries are short, dialogue-like, and weakly

arXiv:2604.08366v1 Announce Type: cross Abstract: Large-scale deep learning models for physical AI applications depend on diverse training data collection efforts. These models and correspondingly, th

arXiv:2604.06811v1 Announce Type: cross Abstract: Skill-based agent systems tackle complex tasks by composing reusable skills, improving modularity and scalability while introducing a largely unexamin

arXiv:2604.08120v1 Announce Type: cross Abstract: Adapting Multimodal Large Language Models (MLLMs) for hour-long videos is bottlenecked by context limits. Dense visual streams saturate token budgets

arXiv:2512.23365v3 Announce Type: replace Abstract: The rapid progress of Multimodal Large Language Models (MLLMs) has unlocked the potential for enhanced 3D scene understanding and spatial reasoning.

arXiv:2604.06375v1 Announce Type: new Abstract: AI-driven symptom analysis systems face persistent challenges in reliability, interpretability, and hallucination. End-to-end generative approaches ofte

arXiv:2604.07513v1 Announce Type: cross Abstract: AI-based persona simulation -- often referred to as digital twin simulation -- is increasingly used for market research, recommender systems, and soci

arXiv:2509.09794v4 Announce Type: replace Abstract: Computational models have emerged as powerful tools for multi-scale energy modeling research at the building and urban scale, supporting data-driven

arXiv:2604.04563v2 Announce Type: replace-cross Abstract: Recent advances in vision--language pretraining have enabled strong medical foundation models, yet most analyze radiographs in isolation, over

arXiv:2503.01804v4 Announce Type: replace Abstract: Ensuring both syntactic and semantic correctness in Large Language Model (LLM) outputs remains a significant challenge, despite being critical for r

arXiv:2604.06906v1 Announce Type: cross Abstract: As Large Language Models reshape the global labor market, policymakers and workers need empirical data on which occupational skills may be most suscep

arXiv:2604.06240v1 Announce Type: cross Abstract: Verifying the success of computer use agent (CUA) trajectories is a critical challenge: without reliable verification, neither evaluation nor training

arXiv:2604.06724v1 Announce Type: cross Abstract: While traditional optimization problems were often studied in isolation, many real-world problems today require interdependence among multiple optimiz

arXiv:2503.13551v5 Announce Type: replace Abstract: Recent studies show that Large Language Models (LLMs) achieve strong reasoning capabilities through supervised fine-tuning or reinforcement learning

arXiv:2604.07553v1 Announce Type: new Abstract: This study presents a framework for generating the gold-standard summary fully automatically and reproducibly based on multiple human summaries of Turki

arXiv:2604.07266v1 Announce Type: new Abstract: Evaluating robustness under temporal distribution shift remains an open challenge. Existing metrics quantify the average decline in performance, but fai

arXiv:2604.06214v1 Announce Type: cross Abstract: Efficient classification of surgical procedures by urgency is paramount to optimize patient care and resource allocation within healthcare systems. Th

arXiv:2604.07264v1 Announce Type: cross Abstract: Operating LEO mega-constellations requires translating high-level operator intents ('reroute financial traffic away from polar links under 80 ms') int

arXiv:2510.23642v2 Announce Type: replace-cross Abstract: Large language models (LLMs) have recently enabled coding agents capable of generating, executing, and revising visualization code. However, e

arXiv:2604.08509v1 Announce Type: new Abstract: Digital human generation has been studied for decades and supports a wide range of real-world applications. However, most existing systems are passively

What does it actually take to make agents better over time? A system that starts with a trace. You capture traces of agent behavior, enrich them with evaluations and human feedback, identify what’s fa

arXiv:2604.08494v1 Announce Type: cross Abstract: Scanpath similarity metrics are central to eye-movement research, yet existing methods predominantly evaluate spatial and temporal alignment while neg

arXiv:2604.06995v1 Announce Type: new Abstract: Existing Graphical User Interface (GUI) reasoning tasks remain challenging, particularly in UI understanding. Current methods typically rely on direct s

ConvApparel is a human-AI conversation dataset and comprehensive evaluation framework designed to quantify the 'realism gap' in LLM-based user simulators and improve the training of robust conversa...

LangChain's **Better-Harness** system, shared by Sydney Runkle, is a compound approach to iteratively improving AI agent harnesses using evaluations (evals) as a learning signal. Better agents can...

NEW paper from Google on multi-agent research agents. It's one of the first systems that handles end-to-end LaTeX generation, targeted literature reviews, and conceptual diagrams as a decoupled, stand

The new model from Meta is already looking like a disappointment: overoptimized for public benchmark numbers at the detriment of everything else. Knowing how to evaluate models in a way that correlate

Join the ARC Prize team -- help us build ARC-AGI-4 and ARC-AGI-5 Platform Engineer - Benchmark Lead ARC Prize Foundation is hiring a senior engineer to build our benchmark platform * Expand ARC-AGI-3

Your users say search is bad. How do you turn that complaint into an eval? Aaron Linsky explains how Bridgewater built a benchmark of about 200 examples in under a week, then used it to measure search

arXiv:2610.21481v1 Announce Type: new Abstract: In endovascular interventions, physicians rely on planar DSA and X-ray fluoroscopy to interpret vessel anatomy and device motion, yet converting these i

arXiv:2610.22319v1 Announce Type: new Abstract: We placed four Gemini 4 Argon agents in a sandboxed Minecraft survival world and gave them one goal: defeat the Ender Dragon. Each agent controlled a pl

arXiv:2610.21538v1 Announce Type: new Abstract: The Agent Status Protocol (ASP) defines a lightweight way for heterogeneous AI agents to report their operational state to a status server, and for obse

arXiv:2506.03371v2 Announce Type: replace Abstract: Recent advances in vision-language models (VLMs) have enabled accurate image-based geolocation, raising serious concerns about location privacy risk

arXiv:2610.21147v1 Announce Type: new Abstract: Large language models increasingly rely on external memory to answer queries grounded in long-horizon histories such as multi-session dialogues and leng

arXiv:2610.20675v1 Announce Type: new Abstract: Agents pair reasoning models with harnesses that organize model calls, execute tools, load skill instructions, and retain history in memory. Understandi

arXiv:2610.20867v1 Announce Type: new Abstract: We present a general declarative framework for anticipatory reasoning about interactive motion in complex dynamic environments. Our focus is on (explain

arXiv:2610.21063v1 Announce Type: cross Abstract: AI agents now pause for approval before they send money, delete files or email outsiders, and that approval is usually treated as a second factor. We

arXiv:2610.22139v1 Announce Type: new Abstract: Confidence scores determine whether an LLM answer is accepted, reviewed, or regenerated, yet scalar recalibration can only correct errors visible throug

arXiv:2610.04196v2 Announce Type: replace Abstract: Synchronous policy updates can limit the efficiency of post-training for long-horizon LLM agents. When trajectories span multiple interaction turns,

arXiv:2610.21055v1 Announce Type: new Abstract: Recursive self-improvement, in which AI systems build stronger AI systems, is one promising route to superintelligence. We study a weaker form that can

Banger paper from Anthropic, Harvard, and colleagues. They find that LLM-proposed quantum circuits beat the best-known designs on seven of eight tasks, with fewer Toffoli gates and fewer qubits. They

arXiv:2610.21696v1 Announce Type: new Abstract: Markerless intraoperative tracking of spinal anatomy via RGB-D sensors is a promising alternative to invasive bone-anchored fiducials, yet no existing c

arXiv:2610.21468v1 Announce Type: cross Abstract: Every story has two orders: the order in which its events happen and the order in which they are told. A story is nonlinear when the two differ. We in

arXiv:2610.22057v1 Announce Type: new Abstract: Actively tilting aerial robots can generate six-dimensional wrenches, but slow servo response can make a desired wrench infeasible during reconfiguratio

arXiv:2610.22187v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used in computational social science to assign scalar scores to social and psychological constructs. How