Search: “benchmark”

arXiv:2601.20969v3 Announce Type: replace Abstract: Epistemic planning extends (multi-agent) automated planning by making agents' knowledge and beliefs first-class aspects of the planning formalism. O

arXiv:2604.27209v1 Announce Type: cross Abstract: Large language models can now generate substantial code and draft research text, but research-software projects require more than either artifact alon

AI21 Labs achieved state-of-the-art performance on the BrowseComp-Plus benchmark with 95.18% accuracy using their AI21 Maestro model with agent optimization techniques. The post appears to discuss aut

arXiv:2604.26567v1 Announce Type: new Abstract: Despite the rapid progress in data-driven 3D vision, aerial geometric 3D vision remains a formidable challenge due to the severe scarcity of large-scale

arXiv:2604.26868v1 Announce Type: new Abstract: Existing 3D anomaly detection methods are built on a rigid prior: normal geometry is pose-invariant and can be canonicalized through registration or ali

arXiv:2604.25922v1 Announce Type: cross Abstract: We present DenialBench, a systematic benchmark measuring consciousness denial behaviors across 115 large language models from 25+ providers. Using a t

arXiv:2604.26106v1 Announce Type: new Abstract: Forecasting benchmarks produce accuracy leaderboards but little insight into why some forecasters are more accurate than others. We introduce Bench to t

arXiv:2604.26555v1 Announce Type: cross Abstract: GPU-accelerated Self-Organizing Map (SOM) implementations are among the most competitive options for large-scale SOM analysis, but growing dataset siz

arXiv:2603.25342v2 Announce Type: replace Abstract: Deep Research Agents (DRAs) aim to answer complex questions by searching the web, checking evidence, and synthesizing conclusions across heterogeneo

arXiv:2604.26569v1 Announce Type: new Abstract: Deploying a neuro-symbolic task planner on a new domain today requires significant manual effort: a domain expert must author relaxation and complementa

arXiv:2601.02731v3 Announce Type: replace-cross Abstract: Training a unified model integrating video-to-audio (V2A), text-to-audio (T2A), and joint video-text-to-audio (VT2A) generation offers signifi

arXiv:2604.26614v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have achieved impressive progress on general multimodal tasks, yet they remain brittle on dial-based measuremen

arXiv:2604.26772v1 Announce Type: new Abstract: Recent methods demonstrate that large-scale pretrained models, such as CLIP vision transformers, effectively detect AI-generated images (AIGIs) from uns

arXiv:2604.26460v1 Announce Type: new Abstract: Stylistic personalization - making LLMs write in a specific individual's style, rather than merely adapting to task preferences - lacks evaluation groun

I cannot verify the authenticity of this post or its specific claims about Grok 4.3's performance metrics. The source appears to reference xAI's Grok model improving significantly on benchmark leaderb

arXiv:2603.25268v2 Announce Type: replace Abstract: We introduce CRAFT, a multi-agent benchmark for evaluating pragmatic communication in large language models under strict partial information. In thi

arXiv:2604.25716v1 Announce Type: new Abstract: Safety mechanisms for large language models (LLMs) remain predominantly English-centric, creating systematic vulnerabilities in multilingual deployment.

arXiv:2604.25318v1 Announce Type: cross Abstract: Cutscenes are carefully choreographed cinematic sequences embedded in video games and interactive media, serving as the primary vehicle for narrative

DeepSeek V4 Pro is crazy good at bug fixing. Costs counted in cents not dollars/tens of dollars. It’s the next level quiet confidence not to get caught up by benchmarks & rankings, and just let us use

arXiv:2604.25158v1 Announce Type: cross Abstract: Support vector machines (SVMs) are a standard tool for binary classification, but their classical formulations are purely data-driven and offer no dir

The 2nd Multilingual Conversational Speech Language Model (MLC-SLM) Challenge is an open research competition inviting teams worldwide to participate , featuring free registration with a $20K prize po

arXiv:2601.02078v2 Announce Type: replace Abstract: The development of robust and generalizable robot learning models is critically contingent upon the availability of large-scale, diverse training da

AI21 Labs announced that their Chief Product Officer Or Dagan presented at the DeepLearning.AI conference, discussing how the company achieved state-of-the-art results on the Browsecomp-Plus benchmark

arXiv:2601.21225v2 Announce Type: replace Abstract: Large language models have made substantial progress in mathematical reasoning. However, benchmark development for multilingual evaluation has lagge

arXiv:2604.25840v1 Announce Type: new Abstract: Patient simulators are gaining traction in mental health training by providing scalable exposure to complex and sensitive patient interactions. Simulati

arXiv:2604.25855v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) achieve ever-stronger performance on visual-language tasks. Even as traditional visual question answering bench

arXiv:2604.23701v1 Announce Type: cross Abstract: Crop disease diagnosis from field photographs faces two recurring problems: models that score well on benchmarks frequently hallucinate species names,

arXiv:2404.10141v2 Announce Type: replace-cross Abstract: Text-to-image (T2I) models have achieved remarkable progress in high-quality image synthesis, yet most benchmarks rely on simple, self-contain

arXiv:2604.24703v1 Announce Type: cross Abstract: Large language models are widely used for code generation, yet they rely on an implicit assumption that the task descriptions are sufficiently detaile

arXiv:2604.04733v2 Announce Type: replace-cross Abstract: Vision-language Models (VLMs), despite achieving strong performance on multimodal benchmarks, often misinterpret straightforward visual concep

arXiv:2603.04751v2 Announce Type: replace Abstract: Integrating web search tools has significantly extended the capability of LLMs to address open-world, real-time, and long-tail problems. However, ev

arXiv:2604.23730v1 Announce Type: new Abstract: Large language models (LLMs) have shown strong performance on legal benchmarks, including multiple-choice components of bar exams. However, their capaci

arXiv:2501.07237v5 Announce Type: replace-cross Abstract: Large Language Models (LLMs) have demonstrated exceptional capabilities across diverse natural language processing benchmarks. However, the es

arXiv:2512.16378v4 Announce Type: replace-cross Abstract: As Large Language Models (LLMs) expand beyond text, integrating speech as a native modality has given rise to SpeechLLMs, which directly proce

arXiv:2506.11480v4 Announce Type: replace-cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has become a key technique for enhancing LLMs' reasoning abilities, yet its data ineffic

arXiv:2604.23897v1 Announce Type: new Abstract: Markets are a promising way to coordinate AI agent activity for similar reasons to those used to justify markets more broadly. In order to effectively p

arXiv:2604.22760v1 Announce Type: cross Abstract: Large language models (LLMs) increasingly operate as autonomous agents that reason over external APIs to perform complex tasks. However, their reliabi

arXiv:2604.05013v2 Announce Type: replace-cross Abstract: Current LLM coding agents are predominantly trained on composite benchmarks (e.g., bug fixing), which often leads to task-specific overfitting

arXiv:2604.23067v1 Announce Type: cross Abstract: Automated methods for red teaming LLMs are an important tool to identify LLM vulnerabilities that may not be covered in static benchmarks, allowing fo

Want to see which frontier models do the best on document understanding? Check out our ParseBench leaderboard on @kaggle! https://www.kaggle.com/benchmarks/llamaindex-org/parsebench For more details o

arXiv:2604.23528v1 Announce Type: new Abstract: Physics-informed neural networks (PINNs) provide a promising machine learning framework for solving partial differential equations, but their training o

arXiv:2604.22774v1 Announce Type: cross Abstract: Accurate transcription of handwritten mathematics is crucial for educational AI systems, yet current benchmarks fail to evaluate this capability prope

arXiv:2604.22517v1 Announce Type: new Abstract: Evaluating LLM-generated business ideas is often harder to scale than generating them. Unlike standard NLP benchmarks, business idea evaluation relies o

arXiv:2604.21936v1 Announce Type: new Abstract: Medical imaging research is increasingly shifting from controlled benchmark evaluation toward real-world clinical deployment. In such settings, applying

arXiv:2604.22067v1 Announce Type: cross Abstract: Psychiatric intake is a sequential, high-stakes information-gathering process in which clinicians must decide what to ask, in what order, and how to i

arXiv:2510.07632v2 Announce Type: replace Abstract: Frontier AI models have achieved remarkable progress, yet recent studies suggest they struggle with compositional reasoning, often performing at or