Search: “benchmark”

arXiv:2610.03078v1 Announce Type: new Abstract: Quantifying conversational dynamics requires reliable identification of interactional units and their temporal boundaries, but speech activity alone doe

arXiv:2610.02369v1 Announce Type: cross Abstract: Human-computer interaction (HCI) is in the middle of a transition: large language models can now generate functional user interfaces (UIs) on demand f

arXiv:2603.13856v3 Announce Type: replace-cross Abstract: Building AI systems that can plan, act, and create in the physical world requires more than pattern recognition. Such systems must reason abou

arXiv:2610.03617v1 Announce Type: new Abstract: Image-text alignment is a core problem in computer vision with applications in caption evaluation, hallucination detection, data curation, and the bench

arXiv:2610.02574v1 Announce Type: new Abstract: High-fidelity solubility prediction is fundamental to pharmaceutical development and environmental partitioning, where accurate modeling must couple mol

arXiv:2610.02486v1 Announce Type: cross Abstract: Typed decision models answer schema-constrained questions about a text in one forward pass and return probabilities meant to be thresholded. We ask wh

arXiv:2610.02381v1 Announce Type: new Abstract: On-policy distillation (OPD) trains a student on the responses it generates. Existing LLM multi-teacher OPD transfers what specialists predict through t

Amazon SageMaker optimized generative AI inference introduces the aws-ai-ml skill through the Agent Toolkit for AWS, giving coding agents like Kiro, Claude Code, and Codex deep expertise in inference

arXiv:2610.02959v1 Announce Type: new Abstract: Recent text-to-image models have made substantial progress in photorealism, aesthetics, and text-image alignment. Yet visually appealing images can stil

arXiv:2610.02946v1 Announce Type: new Abstract: Video Object Segmentation (VOS) in complex and long videos is increasingly important for real-world applications, where target objects often appear only

I am one of the authors of FLEET - an algorithm that enhances Best-of-N generation by attributing external rewards to particular tokens and then uses MCTS to adjust logits during the next run. I find

arXiv:2610.01618v1 Announce Type: new Abstract: Agent evaluations increasingly go beyond a single success rate, reporting metrics such as cost, consistency, and robustness. Yet they typically treat th

arXiv:2602.18182v5 Announce Type: replace-cross Abstract: AI evaluation has primarily focused on measuring capabilities, with formal approaches inspired from Item Response Theory (IRT) being increasin

arXiv:2610.01661v1 Announce Type: new Abstract: Despite remarkable progress, video generation models often produce highly similar outputs when repeatedly sampled from the same prompt, limiting their u

arXiv:2610.01670v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) are increasingly used as automated judges for instruction-based image editing and as reward signals for model t

arXiv:2610.01092v1 Announce Type: new Abstract: Video generation models are increasingly being explored as world simulators for embodied planning and learning. To do so effectively, these models must

arXiv:2610.01999v1 Announce Type: new Abstract: Spatial reasoning benchmarks evaluate vision-language models across diverse tasks, but task-level scores do not reveal which underlying capabilities acc

arXiv:2610.00677v1 Announce Type: new Abstract: Advancements in augmented reality (AR) continue to foster innovative solutions, facilitating novel methodologies within educational systems, healthcare

arXiv:2610.00834v1 Announce Type: new Abstract: ARC-AGI-3 evaluates agents in interactive environments whose rules and objectives must be inferred from observation. We present Kepler, an open-source h

arXiv:2607.15942v2 Announce Type: replace Abstract: Remote sensing vision-language models are increasingly expected to support open-ended reasoning over Earth Observation data and a variety of tasks.

arXiv:2610.01349v1 Announce Type: cross Abstract: Tool-using large language model (LLM) agents turn generated text into real side effects, so poisoned tool metadata, retrieved pages, memory, and reusa

arXiv:2610.01601v1 Announce Type: cross Abstract: Decision models often score a variable-sized set of candidate actions encoded in a single sequence. This setting is increasingly relevant for System 1

arXiv:2610.00710v1 Announce Type: new Abstract: As large language model (LLM) agents become widely adopted, they are increasingly deployed for tasks that require persistent monitoring or recurring act

arXiv:2610.01908v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) improves vision-language benchmark scores even without visual information during training. With im

arXiv:2610.00251v1 Announce Type: new Abstract: Memorization audits of generative models read similarity scores against thresholds, with no null distribution, and the conclusions they support can be w

https://huggingface.co/Wa1k3r/Qwen3.8-27b-CODER-4q_xs-24GB-262k-Optimalcardfit Qwen3.8-27B CODER — IQ4_XS imatrix · 24 GB card fit · ~262k context · MTP draft Quantized, Abliterated, and fitted by Lex

arXiv:2610.00972v1 Announce Type: new Abstract: As LLM agents undertake increasingly complex, long-horizon tasks, verifying their outputs becomes increasingly challenging. We study how verification ca

arXiv:2602.10639v2 Announce Type: replace Abstract: Video Large Language Models (VideoLLMs) have achieved strong performance on video understanding tasks, yet existing benchmarks evaluate only what mo

arXiv:2609.38820v1 Announce Type: cross Abstract: With the rapid growth of Arabic NLP, several models, datasets and benchmarks have been reported. This paper asks whether approaches developed for majo

arXiv:2609.34211v2 Announce Type: replace Abstract: In financial fraud detection, rich semantic context can provide important evidence for transaction behavior modeling and fraud reasoning. However, p

arXiv:2609.38485v1 Announce Type: new Abstract: Unified multimodal models (UMMs) train image understanding and autoregressive image generation on shared parameters, and the two objectives are known to

arXiv:2609.40117v1 Announce Type: new Abstract: Time-series forecasting models achieve strong benchmark performance but exhibit severe systematic bias in industrial deployments. This train--deploy gap

arXiv:2511.22232v2 Announce Type: replace-cross Abstract: Multimodal large language models (MLLMs) are increasingly capable in medical imaging, yet most focus on single-image settings. Clinical interp

arXiv:2609.39566v1 Announce Type: new Abstract: Foundation models can serve as clinical agents through tool-use harnesses. However, conventional medical benchmarks assess reasoning over preselected ev

arXiv:2609.39882v1 Announce Type: new Abstract: Pre-training equips large language models (LLMs) with a broad repertoire of behavioral patterns associated with roles, styles, values, and goals. Post-t

arXiv:2609.40195v1 Announce Type: cross Abstract: Long-term egocentric video enables personalized AI assistants to reason about daily life. However, as video histories grow to hundreds of hours spanni

arXiv:2609.38884v1 Announce Type: cross Abstract: Optimization modeling formulates real-world decision problems as mathematical programs that solvers can use to find optimal decisions. Large language

arXiv:2609.38463v1 Announce Type: cross Abstract: Autonomous driving planners are typically evaluated using aggregate metrics such as driving score, destination rate, and collision rate, which do not

arXiv:2609.39195v1 Announce Type: new Abstract: Spatial intelligence requires maintaining a coherent understanding of the world as the embodied agent moves. Like humans, the agent must use its own mot

arXiv:2609.38966v1 Announce Type: new Abstract: We present Video2SwimFish, an automated pipeline and benchmark for building controllable fish assets from real-fish videos for underwater embodied AI. G

arXiv:2609.36104v1 Announce Type: new Abstract: Replacing one LLM agent with a collaborating team can raise accuracy, but whether scaling the team helps, and which architecture to scale, is unclear. S

arXiv:2609.35814v1 Announce Type: cross Abstract: As browser-use agents improve, benchmarks keep pace by collecting new tasks, websites, and applications, often making tasks longer or more novel. This

arXiv:2609.38108v1 Announce Type: new Abstract: Large language models (LLMs) enable agents to solve long-horizon tasks by generating a plan and then executing it in an environment. However, successful

arXiv:2609.37576v1 Announce Type: new Abstract: With the rapid advancement of text-to-image (T2I) generation, robust evaluation becomes critical yet challenging, as traditional metrics fail to capture

arXiv:2609.37470v1 Announce Type: new Abstract: A probability used for a decision should refer to the same event across equivalent requests. We introduce probability contracts, a benchmark connecting

arXiv:2609.37543v1 Announce Type: new Abstract: Cross-lingual zero-shot transfer and multilingual fine-tuning are promising approaches for NLP tasks such as Named Entity Recognition (NER) in low-resou