Search: “benchmark”

arXiv:2610.02200v1 Announce Type: new Abstract: We show that multimodal models possess strong reasoning abilities and that an appropriate harness can unlock their potential to solve tasks across diver

arXiv:2609.38479v1 Announce Type: cross Abstract: This paper presents an explainable approach to pedestrian routing, in which perceived safety is estimated from street-level imagery through an explici

arXiv:2609.40230v1 Announce Type: cross Abstract: Visual perception is conventionally formulated as a one-shot prediction from a single glance at the image, under the assumption that the image content

arXiv:2609.39392v1 Announce Type: new Abstract: Autonomous research agents can generate hypotheses and conduct experiments, but experimentation remains a major source of computational cost. A fundamen

So everyone has been yelling about how I should be using Gufo instead of halogen because it's open source and it's 'just as good or better'. Checking in on their GitHub (GitHub.com/gufo-org/gufo) got

arXiv:2609.39639v1 Announce Type: new Abstract: Tabular learning uses structured data to predict target outcomes. Traditionally, this process has relied on labeled data. However, large language models

arXiv:2609.39229v1 Announce Type: cross Abstract: Automatic evaluation of faithfulness increasingly relies on a large language model acting as a judge, yet the most reliable judges are proprietary fro

arXiv:2609.39568v1 Announce Type: cross Abstract: Large language models (LLMs) may generate unreliable code on corner cases missed by testing, while formal verification can provide machine-checkable g

arXiv:2609.39846v1 Announce Type: cross Abstract: We investigate the problem of role-capability leakage (RCL), in which a role-prompted reasoning model generates convincing in-role text while continui

arXiv:2609.40325v1 Announce Type: new Abstract: As interactive 3D worlds are increasingly used to study intelligent behavior, it becomes important to develop efficient pipelines for identifying anomal

arXiv:2609.37143v1 Announce Type: cross Abstract: Modern coding agents can deliver increasingly large repository-level changes, and recent benchmarks reflect this by emphasizing long-horizon tasks wit

arXiv:2609.37356v1 Announce Type: new Abstract: Generating optimization instances that are both feasible and computationally challenging is crucial for benchmarking solvers and training learning-based

arXiv:2609.32103v1 Announce Type: new Abstract: Large language models can memorize private or harmful information, motivating machine unlearning methods that remove targeted knowledge while preserving

arXiv:2609.32391v1 Announce Type: cross Abstract: Continual-learning agents are systems of models, harnesses, and memory operating over long multi-session horizons. Evaluating and training them requir

arXiv:2609.31823v1 Announce Type: new Abstract: Large vision-language models (LVLMs) have demonstrated remarkable performance on multimodal reasoning benchmarks, yet their perceptual reliability under

arXiv:2609.31546v1 Announce Type: new Abstract: Electrocardiogram (ECG) foundation models typically tokenize the signal into fixed-length patches that ignore cardiac structure, so a patch may split a

arXiv:2506.04734v3 Announce Type: replace Abstract: Reasoning models represented by the Deepseek-R1-Distill series have been widely adopted by the open-source community due to their strong performance

arXiv:2504.06188v3 Announce Type: replace Abstract: AI agents can extend their capabilities at inference time by loading reusable skills into context, yet equipping an agent with too many skills, part

arXiv:2609.31298v1 Announce Type: cross Abstract: Autism Spectrum Disorder (ASD) is a complex neurodevelopmental disorder for which early and accurate diagnosis is critical to improving long-term deve

Every week, I talk with founders who are building at an unbelievable pace. Teams are moving from inception to product-market fit faster than ever, with foundation models wired deeply into their core p

We comprehensively evaluated 16 recent frontier VLMs - including Opus 5.5 and GPT-6 Sol/Luna - on whether higher effort led to better document parsing performance. Higher effort typically leads to imp

arXiv:2609.26176v2 Announce Type: replace-cross Abstract: Encoded-prompt attacks are evaluated almost entirely on their harmful arm: a benchmark sends obfuscated harmful requests and reports how often

arXiv:2609.29465v1 Announce Type: new Abstract: Large language model based coding agents have made substantial progress on repository-level software engineering tasks. Existing repository benchmarks,

arXiv:2609.26900v1 Announce Type: cross Abstract: A language model agent acts through the tools it is given. The data it reads while working on a task can redirect what it does with those tools. A gro

arXiv:2609.28007v1 Announce Type: new Abstract: Most Turkish-capable large language models (LLMs) are evaluated using general-purpose benchmarks rather than long, structurally complex domain documents

arXiv:2609.27307v1 Announce Type: new Abstract: Graphical User Interface (GUI) agents enable the fulfillment of complex user instructions through multi-turn interactions with software environments, re

arXiv:2609.27041v1 Announce Type: new Abstract: Mathematical reasoning benchmarks are typically organized by topic, but language models may organize their internal computation by reusable reasoning ap

arXiv:2609.28197v1 Announce Type: new Abstract: As Large Language Models (LLMs) evolve into autonomous agents that alter real-world states, ensuring operational safety across multi-step workflows has

arXiv:2609.28049v1 Announce Type: cross Abstract: Video understanding is usually benchmarked on curated, single-actor, or professionally filmed clips, and a strong score there is routinely read as evi

arXiv:2609.27606v1 Announce Type: new Abstract: We introduce State-Grounded Conditioning (SGC), a design principle for user-facing LLM agents that must condition on live user state (game state, sessio

arXiv:2603.08930v2 Announce Type: replace-cross Abstract: This paper introduces a benchmark for evaluating whether vision-language models (VLMs) can generate plant simulation configurations from image

arXiv:2503.15242v3 Announce Type: replace Abstract: We introduce BigO(Bench), a novel coding benchmark designed to evaluate the capabilities of generative language models in understanding and generati

arXiv:2604.07733v2 Announce Type: replace Abstract: Evaluating strategic decision-making in LLM-based agents requires generative, competitive, and longitudinal environments, yet few benchmarks provide

arXiv:2410.14948v2 Announce Type: replace Abstract: Medical multimodal large language models (MLLMs) can perform well on existing medical visual question answering (MedVQA) benchmarks, but their train

Saw a new open 27B model release that looks like it could be interesting for the Ollama crowd: VeriLoop E2. It is a Qwen3.8-27B post-train aimed at code, mathematics, scientific reasoning, and agentic

I didn't know my set up was outperforming nearly everyone until reading another discussion where people were struggling getting half of that speed with half the context on the same hardware. I benchma

arXiv:2609.24972v1 Announce Type: cross Abstract: An LLM agent's capability is largely magnified by its harness, namely the prompts, control flow, tooling, memory, and context management surrounding t

On September 21 2026, Yohei reported results from a hard “insect‑identification” benchmark where model accuracy spanned 23 points across the evaluated systems. The Qwen 4B + Glance model scored 0.96,

I never thought there would be a model worse than the Gemini series, but here we are. I remember Elon tweeting that this model would beat Astra & 5.1 across all benchmarks. submitted by /u/prvthvm [li

I decided to run prism-ml/Ternary-Bonsai-2-27B-PQ2_0 through my own set of UNSCIENTIFIC benchmarks. I needed something to compare it to, so I decided I would compare with another 27B model by filesize

I just released a new dataset https://huggingface.co/datasets/LocalLLaMA/deepswe-mini it is meant to be more efficent when testing and benchmarking local models on DeepSwe https://deepswe.datacurve.ai

Vacations give you time to think and explore. I came across a few things and made a few discoveries of my own. It turns out that, as a programmer, I no longer need the smartest and most expensive AI m

arXiv:2609.19916v1 Announce Type: cross Abstract: Large language models (LLMs) are typically evaluated on static benchmarks, even though natural language constantly evolves through newly emerging word

arXiv:2605.13481v2 Announce Type: replace Abstract: We introduce PersonalAI 2.0 (PAI-2), a novel framework designed to enhance LLM-based systems through integration of external knowledge graphs (KGs).

arXiv:2609.18860v1 Announce Type: cross Abstract: When a large vision-language model misclassifies a harmful meme, the failure may reflect missing internal evidence or an inability to route represente

arXiv:2609.17549v1 Announce Type: new Abstract: Cyberbullying (CB) is a complex social phenomenon characterized by repeated aggression, power imbalance, and multi-party interaction. Although large lan