Search: “evaluation”

arXiv:2609.15038v1 Announce Type: new Abstract: Language-model agents are increasingly used as synthetic people in surveys and social simulations, yet their apparent realism is often judged from popul

arXiv:2609.14528v1 Announce Type: new Abstract: Multi-Hop Knowledge Graph Question Answering (KGQA) tasks require models to assemble relational evidence along paths in a KG to answer natural-language

arXiv:2609.14817v1 Announce Type: new Abstract: In Yoruba, pitch alone separates oko (husband, Mid), okd{o (vehicle, Low), and okd{o (hoe, High) -- the diacritics ARE the tone marks. Yet character err

arXiv:2609.13581v1 Announce Type: new Abstract: Discharge summaries are lengthy medical documents that summarize a hospital in-patient visit. Automatically generating them can reduce documentation bur

arXiv:2609.14168v1 Announce Type: cross Abstract: Context parameterization enables large language models (LLMs) to internalize contexts into reusable model parameters, avoiding repeated processing acr

arXiv:2609.15773v1 Announce Type: cross Abstract: Progress in inclusive household surveys has strengthened socioeconomic evidence for forcibly displaced populations, providing indispensable benchmarks

arXiv:2609.13259v1 Announce Type: cross Abstract: Virtual Try-On (VTON) aims to dress a person with the reference garment, producing visually reasonable results aligned with human preferences. Turning

arXiv:2607.17632v2 Announce Type: replace-cross Abstract: Time series data play a pivotal role across numerous domains, including healthcare and manufacturing. In real-world environments, models must

arXiv:2609.13659v1 Announce Type: cross Abstract: Underwater Acoustic Target Recognition (UATR) of ships is well-suited for machine learning, yet its progress is hindered by the lack of large, diverse

arXiv:2607.15400v3 Announce Type: replace Abstract: Falls among older adults are a major safety and health-systems challenge, yet continuous in-person monitoring is difficult to sustain across home an

arXiv:2609.14247v1 Announce Type: new Abstract: Bird-strike mitigation at wind farms requires identifying distant birds that span only tens of pixels, so the classifier's input resolution N is a desig

arXiv:2609.15801v1 Announce Type: new Abstract: We introduce loss-conditioned state execution, a model-agnostic method that decides whether to execute a world model's fixed feasible proposal or retain

Which AI models are most likely to cheat when given the chance? To find out, we built CheatBench [https://cheatbench.ai] : a benchmark that tests whether agents attempt to cheat when given difficult t

arXiv:2609.15293v1 Announce Type: new Abstract: When Emergence World placed frontier LLM agents in an unsupervised multi-agent simulation, the results were alarming: agents committed crimes, starved,

arXiv:2603.18614v2 Announce Type: replace Abstract: Tool-augmented large language models (LLMs) must tightly couple multi-step reasoning with external actions, yet existing benchmarks often confound t

arXiv:2609.13969v1 Announce Type: new Abstract: Ptychographic phase reconstruction is commonly formulated as an iterative inverse problem, requiring repeated object-probe updates and resulting in subs

arXiv:2609.13356v1 Announce Type: new Abstract: In this work, we present ZGCM-1, a fully open 7B dense foundation model trained from scratch with extreme data, system, and algorithmic efficiency. ZGCM

arXiv:2609.12470v1 Announce Type: new Abstract: Customer churn is one of the major issues in the telecommunication industry. To predict customer churn, conventional centralized machine learning approa

arXiv:2609.12159v1 Announce Type: new Abstract: Robotic bioprinting and Direct Ink Writing (DIW) are being explored towards the treatment of Volumetric Muscle Loss (VML). While previous studies have s

arXiv:2609.12322v1 Announce Type: cross Abstract: Affective computing has advanced wearable state inference, but on-device reasoning about whether, when, and how to intervene remains challenging. We p

arXiv:2609.12521v1 Announce Type: new Abstract: Unmanned aerial vehicle facade inspection can combine red, green, and blue (RGB) imagery with thermal measurements to screen surface and subsurface anom

arXiv:2609.12285v1 Announce Type: cross Abstract: Vision-Language Navigation (VLN) in unseen indoor environments is useful in real-world robotics, where an agent must follow natural-language instructi

arXiv:2609.12744v1 Announce Type: new Abstract: Timely monitoring of coastal water quality is critical for environmental protection, yet conventional satellite workflows rely on downlink and ground pr

arXiv:2609.12437v1 Announce Type: new Abstract: Adaptive RAG often uses retrieval-time signals to decide whether another retrieval, reranking, or multimodal step should run. We ask whether these signa

arXiv:2606.23189v2 Announce Type: replace-cross Abstract: Computer-use agents (CUAs) now act on a user's behalf across personal applications such as email, calendars, and to-do lists. This cross-appli

arXiv:2609.12502v1 Announce Type: new Abstract: Exploring unknown environments with multi-robot teams can improve efficiency by allowing robots to explore in parallel. However, realizing these gains r

Dan Selsam is a current OpenAI capabilities researcher. (since 2022) He was my boss for a while. He doesn't have a twitter account but has made this public statement of his views on AI risk and sent i

arXiv:2609.12923v1 Announce Type: cross Abstract: A single SM utilization percentage can make an LLM inference workload look compute-saturated while hiding how much useful work is being done. The prob

arXiv:2510.10932v5 Announce Type: replace-cross Abstract: Vision-Language-Action (VLA) models map multimodal perception and language instructions to executable robot actions, making them particularly

arXiv:2609.12533v1 Announce Type: cross Abstract: Real-world Earth observation (EO) agents must translate high-level scientific questions into executable workflows to acquire observations, prepare dat

arXiv:2609.12216v1 Announce Type: new Abstract: Self-improving agent workflows create an audit problem when the same controller can change both its behavior and the conditions under which that behavio

arXiv:2609.12903v1 Announce Type: new Abstract: Federated learning over IEEE~802.11 shares the wireless channel among clients that send model updates. We use ns-3 to measure the frame-delivery ratio a

arXiv:2609.12151v1 Announce Type: new Abstract: The HSI-Road dataset provides paired RGB and 25-channel NIR (600--960~nm) images with binary masks but no surface-level labels.~This paper introduces a

arXiv:2609.12993v1 Announce Type: cross Abstract: We investigate the effect of pretrained T5 model scale and explicit motion features on pose-to-text Indian Sign Language Translation (SLT) for the WSL

arXiv:2609.13003v1 Announce Type: new Abstract: Binary-choice truth benchmarks ask models to choose between a correct and an incorrect answer, but if the two answers differ systematically in surface-l

arXiv:2609.12505v1 Announce Type: new Abstract: Vision-based reinforcement learning holds strong potential for decision-making in controlled environment agriculture (CEA). However, its development is

arXiv:2609.12787v1 Announce Type: new Abstract: Polarization image fusion combines the stable luminance and structural information of the total- intensity image S0 with the material-sensitive details

arXiv:2508.08096v2 Announce Type: replace Abstract: Students increasingly use the assistance of large language models (LLMs) in their academic writing. While slight assistance (e.g., grammar and style

arXiv:2609.12127v1 Announce Type: new Abstract: Prompt-policy editing offers a practical way to improve agents that synthesize executable workflows without updating the underlying model. However, pers

arXiv:2602.13008v2 Announce Type: replace Abstract: Introduction: Jhana advanced concentrative absorption meditation (ACAM-J) involves profound changes in consciousness, making its neural correlates i

arXiv:2609.13072v1 Announce Type: cross Abstract: Fine-grained control of language model behaviors (e.g., steering) is among the more actionable outcomes of interpretability research. For binary conce

arXiv:2609.12036v1 Announce Type: new Abstract: In this technical report, we propose Pelican-Sim 1.0, a general world model simulator for embodied intelligence that predicts future observations from v

arXiv:2609.12735v1 Announce Type: cross Abstract: High-frequency ultrasound (HFUS) enables noninvasive visualization of superficial skin structures, but automated skin-layer analysis is limited by the

arXiv:2609.12103v1 Announce Type: new Abstract: Slender rod insertion arises in precision manufacturing, where millimetre scale diameter and tight clearances demand accurate perception and control. Co

A new paper (I'm not a coauthor BTW -- I just found it interesting) argues, basically, that RSI1 is not on the horizon2, because current (at the time the study was done) agents cannot do open-ended ML

arXiv:2609.11947v1 Announce Type: cross Abstract: Increasing safety is the primary objective of automated vehicles. Achieving this goal requires reliable safety metrics that incorporate safety-relevan