Search: “benchmark”

arXiv:2609.11897v1 Announce Type: new Abstract: Causal discovery aims to uncover causal structures from data and is fundamental to scientific reasoning and intervention-based decision making. Its eval

arXiv:2511.13135v3 Announce Type: replace Abstract: Medical vision-language models (VLMs) are increasingly expected to support clinical workflows through diagnostic text and relevant medical images. H

arXiv:2609.11101v1 Announce Type: new Abstract: Dispute mediation is essential for maintaining social harmony and resilience, yet developing skilled mediators is costly and time-consuming. Existing LL

arXiv:2609.11399v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used for machine translation, yet their outputs often contain additional text beyond the translation itsel

arXiv:2609.07611v1 Announce Type: new Abstract: Scientific ideation is the capacity to formulate novel and testable hypotheses from scientific evidence, and autonomous AI scientists depend on it. Exis

arXiv:2609.07107v1 Announce Type: new Abstract: Micro-dramas, characterized by ultra-short durations and hyper-dense storylines, pose unique challenges for video understanding that conventional benchm

arXiv:2609.07944v1 Announce Type: new Abstract: Existing causal-inference benchmarks for LLMs mostly score method descriptions or whether generated code runs, not whether the executed workflow recover

arXiv:2609.07685v1 Announce Type: new Abstract: In real-world applications, pedestrian trajectory prediction models rely on inputs from detection and tracking systems. Prior trajectory prediction benc

arXiv:2609.08805v1 Announce Type: new Abstract: Preoperative evaluation of trigeminal neuralgia (TN) often requires joint interpretation of structural MRI, which depicts the trigeminal nerve and surro

arXiv:2609.07135v1 Announce Type: new Abstract: An embodied kitchen assistant must do more than recognize food in isolated frames. It must track ingredient states over time and integrate visual observ

arXiv:2609.06424v1 Announce Type: new Abstract: Homes change between a robot's visits. Navigation benchmarks pose their goals in the world the agent currently sees, and the two-visit benchmarks that e

arXiv:2609.06187v1 Announce Type: new Abstract: As the share of solar power grows, nowcasting weather-driven solar variability becomes critical for reliable energy system operation. State-of-the-art a

arXiv:2609.05956v1 Announce Type: new Abstract: Spatial transcriptomics (ST) provides unprecedented insights into tumor heterogeneity by capturing spatially resolved gene expression, yet its high expe

arXiv:2609.08079v1 Announce Type: new Abstract: Recent advances in video generation have made prompt-based control increasingly central to AIGC video generation. Prompts specify what a video should de

arXiv:2609.04850v1 Announce Type: new Abstract: While autonomous mobile agents hold great potential for assisting older adults with smartphone usage, existing GUI benchmarks mainly rely on explicit, g

arXiv:2609.02459v1 Announce Type: new Abstract: We present CivBench, an open-source benchmark for evaluating language model agents in long-horizon, tool-mediated environments through the Model Context

arXiv:2609.02059v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have achieved strong performance on structured visual understanding tasks such as chart and document question a

arXiv:2505.23713v2 Announce Type: replace Abstract: Large language models (LLMs) are increasingly deployed in socially grounded applications, where success requires interpreting context, inferring oth

arXiv:2609.02421v1 Announce Type: new Abstract: Large Language Models (LLMs) have recently demonstrated strong capabilities in automated travel itinerary generation. However, real- world travel planni

arXiv:2609.00232v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have achieved strong performance on OCR-centric document understanding and text-rich visual reasoning benchmark

arXiv:2609.00525v1 Announce Type: new Abstract: Modern image generation and editing systems can produce photorealistic, prompt-aligned images, but still often render familiar objects at implausible re

arXiv:2608.29632v1 Announce Type: cross Abstract: Competitive programming is increasingly being used to evaluate the algorithmic reasoning capabilities of large language models (LLMs). However, existi

arXiv:2609.00491v1 Announce Type: new Abstract: Communicating across cultures is inherently challenging, especially through culturally dense and ambiguous formats like memes. While people expect large

Prinz has been running legal benchmarks against AI systems and the latest models are very good... but cheaper/low reasoning models are not very good. So, how do you make sure your vendor selling an AI

arXiv:2608.29345v1 Announce Type: new Abstract: While recent Large Language Model (LLM)-based text-to-SQL systems achieve impressive performance on standard benchmarks, they struggle when user queries

arXiv:2608.30413v1 Announce Type: new Abstract: Defeasible reasoning is a type of reasoning where inferences are drawn from plausible current evidence, but can be retracted upon the introduction of ne

arXiv:2608.30893v1 Announce Type: new Abstract: Electrocardiogram (ECG) interpretation requires knowledge of cardiology, electrophysiology, clinical diagnosis, ECG waveforms, signal acquisition, and i

arXiv:2608.30653v1 Announce Type: cross Abstract: Multimodal Large Language Models (MLLMs) are increasingly deployed in multi-image scenarios requiring complex reasoning across visual contexts. Howeve

arXiv:2601.07794v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly evaluated on their ability to perform multi-hop reasoning, i.e., to combine multiple pieces of i

arXiv:2608.30065v1 Announce Type: cross Abstract: Large language models (LLMs) demonstrate strong Helpfulness, Harmlessness, and Honesty (3H) alignment in English-centric settings, but these gains tra

arXiv:2608.29624v1 Announce Type: new Abstract: Natural Language Processing methods have enabled novel solutions and advances in the field of privacy, particularly in the sub-domain of text-to-text pr

arXiv:2608.30835v1 Announce Type: cross Abstract: Background and Objective: Quality control is a prerequisite for whole-slide image analysis, yet the benchmarks on which quality-control methods are co

arXiv:2608.29607v1 Announce Type: new Abstract: Mobile AI acts as a visual oracle, empowering users to snap a picture of something and ask for information. Snap-and-ask retrieval is now one of the mos

arXiv:2608.27219v1 Announce Type: new Abstract: Mental health assessment relies on episodic self-report scales, which convert subjective states such as stress into numerical scores but provide only sp

arXiv:2608.26199v1 Announce Type: new Abstract: We ask whether AI agents powered by locally deployed large language models can reliably automate expert-defined hardware design workflows in an industry

arXiv:2608.25398v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have demonstrated strong abilities in solving diverse visual and textual reasoning tasks. However, their develo

arXiv:2608.25452v1 Announce Type: new Abstract: We introduce VGA-BenchV2, an extended human-aligned benchmark and optimization framework for jointly evaluating and improving video generation quality a

Kate Clark / Wall Street Journal: AI assistant Instinct is raising a 250M Series B co-led by Index Ventures and Benchmark at a 2.5B valuation, bringing its total raised to 350M — Buzzy startup Instinc

The Artist: Qwen3.8-27B-UD-Q3K_XL, q8_0 caches, xhigh, temp 1.0, image-min-tokens 1024, froggeric template) I was screwing around with different Qwen3.8-27B quants and thought of this very simplistic

arXiv:2608.24232v1 Announce Type: new Abstract: Large Reasoning Models (LRMs) generate intermediate reasoning traces that may contain unsafe content, even when their final responses appear safe. Guard

arXiv:2608.21369v1 Announce Type: cross Abstract: Nigerian Pidgin is one of Africa's most widely spoken languages, yet remains severely underrepresented in language model evaluation. Existing benchmar

arXiv:2608.20980v1 Announce Type: new Abstract: Graph neural networks (GNNs) are routinely employed for short-range forecasting on multivariate time series with a spatial graph structure. Despite the

arXiv:2608.20853v1 Announce Type: new Abstract: Evaluation of long-context Large Language Models (LLMs) has advanced rapidly. However, most existing benchmarks are limited to the document level and fo

arXiv:2512.07847v2 Announce Type: replace Abstract: Benchmarking has been the cornerstone of progress in computer vision, natural language processing, and the broader deep learning domain, driving alg

arXiv:2509.08022v3 Announce Type: replace-cross Abstract: Aligning large language models (LLMs) with diverse human values is essential for safe and effective deployment, yet existing benchmarks often

arXiv:2608.20009v1 Announce Type: new Abstract: Understanding object dynamics requires not only predicting future trajectories but also examining whether a model captures the physical properties that