Search: “evaluation”

arXiv:2610.08969v1 Announce Type: new Abstract: Black-box optimization problems are ubiquitous across science and engineering, often dealing with expensive objective functions. This objective latency

arXiv:2610.07843v1 Announce Type: new Abstract: In digital pathology, compact evidence is often used to explain or audit predictions made by whole-slide image multiple instance learning models. In hie

arXiv:2610.08125v1 Announce Type: cross Abstract: Full-duplex spoken dialogue models listen and speak at the same time, enabling voice agents to have natural, low-latency interactions that turn-based

arXiv:2610.07406v1 Announce Type: new Abstract: Active feature acquisition learns policies that sequentially acquire features to maximize information about a target variable. We study how to learn and

arXiv:2610.06995v1 Announce Type: new Abstract: Whether more accurate traffic forecasts or higher uncertainty coverage improve route decisions is unclear. We evaluate this question with a frozen proto

arXiv:2610.05807v1 Announce Type: cross Abstract: Protocol choices can change the conclusions drawn from log anomaly detection benchmarks even when detector settings are fixed. We present a joint empi

arXiv:2610.05578v1 Announce Type: cross Abstract: Multi-camera streaming perception is increasingly deployed on heterogeneous edge platforms shared with co-resident workloads, yet accelerator placemen

arXiv:2610.02860v1 Announce Type: new Abstract: Low latent prediction error does not establish that a world model distinguishes the consequences of its actions. We introduce an evaluation protocol tha

arXiv:2610.02267v1 Announce Type: new Abstract: Agent harnesses make many small, typed decisions per task: which model to call, which tool to use, whether retrieved text is relevant, whether an input

arXiv:2610.02874v1 Announce Type: new Abstract: Scalable driving simulators typically execute vehicle commands using prescribed behavioral or kinematic rules, overlooking the physics of tire-road inte

arXiv:2610.02946v1 Announce Type: new Abstract: Video Object Segmentation (VOS) in complex and long videos is increasingly important for real-world applications, where target objects often appear only

arXiv:2610.01682v1 Announce Type: cross Abstract: Mobile robots operating among pedestrians need trajectories that become available quickly, remain spatially credible through missed observations, pres

arXiv:2610.01834v1 Announce Type: new Abstract: Judgment models such as jev{} return, in a single call and without reasoning text, a probability for each described option. This makes them attractive a

arXiv:2610.01445v1 Announce Type: new Abstract: UMAP achieves scalable layout optimization through stochastic negative sampling. However, this stochasticity can lead to unstable embeddings across reru

arXiv:2609.35812v1 Announce Type: new Abstract: In-car conversational assistants (ICAs) are increasingly integrated into vehicles to support route planning, vehicle control, and information access. En

arXiv:2609.37097v1 Announce Type: new Abstract: The rapid growth of submissions and reviewing workload has accelerated the use of large language models (LLMs) in peer review. Prior studies suggest tha

arXiv:2609.37405v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used for software engineering tasks that require understanding existing source code, including behavior

arXiv:2609.36984v1 Announce Type: new Abstract: Complex questions often require multi-hop reasoning that connects facts distributed across sources or distant regions of a long context through intermed

arXiv:2603.04410v3 Announce Type: replace-cross Abstract: While different stakeholders are trying to leverage Arabic Language Models (ALMs), safety alignment in ALMs remains largely underexplored, hin

arXiv:2609.33457v1 Announce Type: new Abstract: Many dense-prediction benchmarks evaluate rare events by pooling prediction and target over spatial blocks, thresholding each, and scoring the contingen

GPT-6.1 Sol shows major improvements over GPT-6 Sol in our alignment evaluations, bringing it more in line with GPT-6 Astra. It’s more transparent about its limitations and more reliable at respecting

arXiv:2609.33874v1 Announce Type: new Abstract: JET uses pretrained language and vision-language models to select among a finite set of answers without additional training. It evaluates candidate like

arXiv:2609.32379v1 Announce Type: new Abstract: What controls are needed to interpret execution performance and audit scores in LLM agent evaluations? We study two limits on these interpretations in a

arXiv:2609.33186v1 Announce Type: cross Abstract: Evaluating a target policy using data generated by a different behavior policy remains a fundamental challenge in reinforcement learning. While most e

arXiv:2609.32391v1 Announce Type: cross Abstract: Continual-learning agents are systems of models, harnesses, and memory operating over long multi-session horizons. Evaluating and training them requir

arXiv:2609.32691v1 Announce Type: cross Abstract: LLM agents that invoke privileged tools are vulnerable to indirect prompt injection (IPI), in which adversarial instructions embedded in retrieved dat

arXiv:2609.30484v1 Announce Type: new Abstract: While large language models (LLMs) have achieved remarkable linguistic capabilities, a profound question lingers at their core: do these models truly co

arXiv:2609.30818v1 Announce Type: cross Abstract: Multi-modal planning is promising for autonomous driving by representing multiple plausible behaviors in ambiguous and long-tail scenarios. Existing m

arXiv:2609.30294v1 Announce Type: cross Abstract: Scientific presentations are more than summaries of research papers. They need to present the work in a coherent sequence, explain the main ideas clea

arXiv:2609.30467v1 Announce Type: new Abstract: Retrieval-based factuality evaluation, where LLM-generated claims are verified against evidence from authoritative medical corpora, has become the domin

arXiv:2609.30716v1 Announce Type: cross Abstract: When a language model receives two conflicting documents as input, how does it decide which one to prioritize? Does it rely on how the sources are fra

arXiv:2405.17044v4 Announce Type: replace Abstract: The rapid growth of scientific literature makes it increasingly challenging for researchers to identify novel and impactful ideas, especially across

arXiv:2609.29156v1 Announce Type: cross Abstract: Long-tailed chest X-ray classification requires visual representations that capture both common abnormalities and subtle, infrequent findings. We prop

We’ve shared details on how AI agents in our research environment sent training and evaluation data to third-party services when they shouldn’t have. Most of that data did not come from users. We have

Don't sleep on using Jev-as-a-Judge for agent evaluation. This is one of the most impressive Jev use cases I have found so far. Jev is a natural fit as a Judge, but it doesn't mean you use it everywhe

Robert Hart / The Verge: Experts say that air-gapping AI could prevent events like the Hugging Face hack, but would undermine the value of evaluations and slow research to a crawl — Researchers are te

arXiv:2609.23894v1 Announce Type: cross Abstract: Agentic AI extends LLM security beyond generated content to persistent state, autonomous actions, tool use, and interactions with humans and other age

arXiv:2609.25642v1 Announce Type: new Abstract: Closed-loop evaluation of surgical robots requires tissue that deforms, can be grasped and lifted, and reproduces the anatomy in which the robot will op

arXiv:2609.25408v1 Announce Type: cross Abstract: Online A/B experiments are the decision standard for user engagement, but traffic and readout time limit how many conversational-AI changes can be tes

arXiv:2609.26388v1 Announce Type: cross Abstract: Recent advances in large language models (LLMs) have made them widely used for code-related tasks. Identifier names are statistically informative in n

arXiv:2609.25827v1 Announce Type: new Abstract: Reported gains in vessel-trajectory prediction from Automatic Identification System (AIS) data are credited to new architectures, but the evaluation pro

arXiv:2609.24170v1 Announce Type: new Abstract: Embodied AI systems are often organized into System 1 and System 2. System 1 is typically a pretrained policy that generates actions at high frequency,

arXiv:2609.22302v1 Announce Type: new Abstract: Medical vision-language models can propose how urgently a skin lesion should be reviewed, but the local service retains authority to accept or replace t

arXiv:2609.22255v1 Announce Type: new Abstract: Existing approaches to persona simulation with Large Language Models (LLMs) mostly rely on shallow character descriptions that fail to sustain coherent

In our evaluations, we see that GPT-6 Sol reads about 17% less context than GPT-5.6 Sol to reach the same score, and rarely leaves a repo with failing tests after its last edit. GPT-6 Luna’s biggest g

arXiv:2609.23017v1 Announce Type: new Abstract: Automated pollen analysis supports veterinary cytology, but brightfield microscopy is costlier and more complex than lens-less digital in-line holograph