Search: “benchmark”

arXiv:2510.15096v2 Announce Type: replace Abstract: Real-world settings where language models (LMs) are deployed -- in domains spanning healthcare, finance, and other forms of knowledge work -- requir

arXiv:2106.01254v3 Announce Type: replace Abstract: In many classification tasks, there is no definitive ground truth, only human judgments that may disagree. We address two challenges that arise in s

arXiv:2604.21765v1 Announce Type: new Abstract: Data is a central resource for modern enterprises, and data validation is essential for ensuring the reliability of downstream applications. However, ex

arXiv:2604.21611v1 Announce Type: cross Abstract: Inference-time scaling for LLM reasoning has focused on three axes: chain depth, sample breadth, and learned step-scorers (PRMs). We introduce a fourt

arXiv:2604.20860v1 Announce Type: cross Abstract: Despite the success of Retrieval-Augmented Generation (RAG) in grounding LLMs with external knowledge, its application over heterogeneous sources (e.g

arXiv:2604.20904v1 Announce Type: cross Abstract: Information handling practices of LLM agents are broadly misaligned with the contextual privacy expectations of their users. Contextual Integrity (CI)

arXiv:2511.18264v3 Announce Type: replace Abstract: Existing satellite video tracking methods often struggle with generalization, requiring scenario-specific training to achieve satisfactory performan

arXiv:2604.21523v1 Announce Type: cross Abstract: Large Vision-Language Models (VLMs) are increasingly used to evaluate outputs of other models, for image-to-text (I2T) tasks such as visual question a

arXiv:2604.20937v1 Announce Type: new Abstract: Video Large Language Models (Video LLMs) incur high inference latency due to a large number of visual tokens provided to LLMs. To address this, training

arXiv:2604.21284v1 Announce Type: new Abstract: MemPalace is an open-source AI memory system that applies the ancient method of loci (memory palace) spatial metaphor to organize long-term memory for l

arXiv:2604.21031v1 Announce Type: cross Abstract: Synthetic data generation offers promise for addressing data scarcity and privacy concerns in educational technology, yet practitioners lack empirical

arXiv:2604.21930v1 Announce Type: new Abstract: Streaming Continual Learning (CL) typically converts a continuous stream into a sequence of discrete tasks through temporal partitioning. We argue that

arXiv:2604.20917v1 Announce Type: cross Abstract: Large language models (LLMs) have shown remarkable capabilities across diverse coding tasks. However, their adoption requires a true understanding of

Today we open sourced many of OpenAI's monitorability evaluations. We hope that the research community and other model developers can build upon them and use them to evaluate the monitorability of the

arXiv:2604.21057v1 Announce Type: new Abstract: The field of Language Reasoning Models (LRMs) has been very active over the past few years with advances in training and inference techniques enabling L

arXiv:2511.01458v2 Announce Type: replace-cross Abstract: Safety and reliability are critical for deploying visual question answering (VQA) systems in surgery, where incorrect or ambiguous responses c

arXiv:2604.21769v1 Announce Type: new Abstract: LLM leaderboards are widely used to compare models and guide deployment decisions. However, leaderboard rankings are shaped by evaluation priorities set

arXiv:2604.20267v1 Announce Type: cross Abstract: Audio carries richer information than text, including emotion, speaker traits, and environmental context, while also enabling lower-latency processing

arXiv:2604.16902v2 Announce Type: replace Abstract: Native Omni-modal Large Language Models (OLLMs) have shifted from pipeline architectures to unified representation spaces. However, this native inte

arXiv:2604.20791v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly deployed in healthcare, yet their communicative alignment with clinical standards remains insufficiently

arXiv:2604.20200v1 Announce Type: new Abstract: Frontier coding agents are increasingly used in workflows where users supervise progress primarily through repeated improvement of a public score, namel

arXiv:2604.19856v1 Announce Type: cross Abstract: Large Language Models (LLMs) show promise for generating Register-Transfer Level (RTL) code from natural language specifications, but single-shot gene

arXiv:2508.00414v3 Announce Type: replace Abstract: General AI Agents are increasingly recognized as foundational frameworks for the next generation of artificial intelligence, enabling complex reason

arXiv:2604.20548v1 Announce Type: cross Abstract: Scientific progress depends on the continual generation of innovative re-search ideas. However, the rapid growth of scientific literature has greatly

arXiv:2411.00585v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have become foundational in modern language-driven software applications, profoundly influencing daily life. A cr

arXiv:2604.20429v1 Announce Type: new Abstract: Remote sensing (RS) image-text retrieval plays a critical role in understanding massive RS imagery. However, the dense multi-object distribution and com

arXiv:2506.21095v4 Announce Type: replace-cross Abstract: Federated Learning (FL) enables collaborative training while preserving privacy, yet it introduces a critical challenge: the 'illusion of fair

arXiv:2512.15808v2 Announce Type: replace-cross Abstract: Foundation models (FMs) are driving a prominent shift in biomedical imaging from task-specific models to unified backbone models for diverse t

arXiv:2604.20281v1 Announce Type: new Abstract: With the rapid advancement of intelligent driving and remote sensing, oriented object detection has gained widespread attention. However, achieving high

arXiv:2603.09283v2 Announce Type: replace Abstract: Removing objects from videos remains difficult in the presence of real-world imperfections such as shadows, abrupt motion, and defective masks. Exis

arXiv:2604.20822v1 Announce Type: new Abstract: The offshore wind energy sector is expanding rapidly, increasing the need for independent, high-temporal-resolution monitoring of infrastructure deploym

arXiv:2604.19806v1 Announce Type: cross Abstract: Accurate prediction of energy and forces for 3D molecular systems is one of fundamental challenges at the core of AI for Science applications. Many po

arXiv:2604.20154v1 Announce Type: cross Abstract: Multi-look acquisition is a widely used strategy for reducing speckle noise in coherent imaging systems such as digital holography. By acquiring multi

arXiv:2604.14785v2 Announce Type: replace Abstract: Recent progress in Multimodal Large Language Models (MLLMs) has demonstrated remarkable advances in perception and reasoning, suggesting their poten

arXiv:2604.20393v1 Announce Type: new Abstract: With the development of deep learning, ViT-based stereo matching methods have made significant progress due to their remarkable robustness and zero-shot

arXiv:2604.19999v1 Announce Type: new Abstract: Visual detection of Unmanned Aerial Vehicles (UAVs) is a critical task in surveillance systems due to their small physical size and environmental challe

Our newest voice model is 20+ points ahead of the nearest competitor on Sierra's leaderboard. Pretty massive upgrade from our OG model from December, with significantly improved realism and tool calli

arXiv:2506.22598v3 Announce Type: replace Abstract: Agents based on Large Language Models (LLMs) have shown promise for performing sophisticated software engineering tasks autonomously. In addition, t

arXiv:2604.19888v1 Announce Type: new Abstract: Driver gaze estimation is essential for understanding the driver's situational awareness of surrounding traffic. Existing gaze estimation models use dri

arXiv:2509.15174v3 Announce Type: replace-cross Abstract: WARNING: This paper contains examples of offensive materials. To address the proliferation of toxic content on social media, we introduce SMAR

arXiv:2604.19841v1 Announce Type: cross Abstract: Accurate forecasting of electric vehicle (EV) charging demand is critical for grid management and infrastructure planning. Yet the field continues to

arXiv:2604.19803v1 Announce Type: new Abstract: Agentic AI is rapidly transforming the way research is conducted, from prototyping ideas to reproducing results found in the literature. In this paper,

arXiv:2603.29025v2 Announce Type: replace-cross Abstract: Large language models systematically fail when a salient surface cue conflicts with an unstated feasibility constraint. We study this through

arXiv:2603.20714v2 Announce Type: replace Abstract: 3D Gaussian Splatting (3DGS) has become the method of choice for photo-realistic 3D reconstruction of scenes, due to being able to efficiently and a

Think fast! Introducing Grok Voice Think Fast 1.0 A state-of-the-art voice model built for complex, multi-step workflows with snappy responses and high accuracy. It takes the top spot on the Tau Voice

arXiv:2604.20473v1 Announce Type: new Abstract: Existing Video Large Language Models (Video LLMs) struggle with complex video understanding, exhibiting limited reasoning capabilities and potential hal