Search: “benchmark”

arXiv:2610.16652v1 Announce Type: new Abstract: We introduce IND-Eval, a public benchmark that organizes industrial time series across domains into source-specific forecasting tasks. Relative to estab

arXiv:2410.12537v4 Announce Type: replace Abstract: Complex query answering (CQA) on knowledge graphs (KGs) is gaining momentum as a challenging reasoning task. In this paper, we show that the current

arXiv:2511.14259v4 Announce Type: replace Abstract: With the rapid advancement of generative models, powerful image editing methods now enable diverse and highly realistic image manipulations that far

arXiv:2610.18295v1 Announce Type: new Abstract: Detectors of logical anomalies, such as missing, extra or wrongly combined parts, usually score a test image by its distance from normal (good) images.

arXiv:2610.14439v1 Announce Type: new Abstract: Large multimodal models (LMMs) need to understand how entities move and interact as a scene develops. This requires following the same entities over tim

arXiv:2610.17661v1 Announce Type: new Abstract: The rapid development of mobile imaging and social platforms has led to explosive growth in user-generated content images, making their systematic and c

arXiv:2610.15117v1 Announce Type: new Abstract: To evaluate whether a compact bidirectional state-space autoencoder improves joint classification of species, cell type, and disease state from single-n

arXiv:2610.15415v1 Announce Type: cross Abstract: Before an agent acts, it must decide whether action is warranted. A single accuracy cannot distinguish failure to recognize a warranted action from re

arXiv:2606.16767v2 Announce Type: replace Abstract: Existing image editing methods can be generally categorized into textual instruction-based and visual prompt-based ones. Textual instructions are se

There are more document parsing models than ever, and they’re getting harder to tell apart. That’s why we built OpenDocRouter. Every model we serve is benchmarked on ParseBench, with latency and cost

arXiv:2610.16131v1 Announce Type: new Abstract: Financial forecasting has traditionally relied on numerical time-series (TS) signals, while textual information such as news remains largely underutiliz

It's confirmed. OpenAI has been secretly replacing Astra with increasingly shit models since release. My evidence is overwhelming. Three Reddit comments, a benchmark that fluctuated 4%, and a guy who

Every few weeks, a new model steps up and claims best-in-class at something. The launch chart makes the case, the link makes the rounds, and your team is back to the question nobody has time for: shou

GPT-4 was the undisputed best model in the world for a full year and we all debated whether anyone could catch up or if OpenAI had some secret formula that nobody else knew about. Now there are are 19

arXiv:2610.12989v1 Announce Type: new Abstract: We investigate whether language models contain internal components that contribute to representing narrativity, the degree to which a text exhibits narr

arXiv:2512.02444v2 Announce Type: replace-cross Abstract: Joining heterogeneous tables requires different forms of correspondence, including exact agreement, syntactic transformation, semantic similar

arXiv:2505.17244v3 Announce Type: replace-cross Abstract: Large Reasoning Models (LRMs) leverage explicit reasoning traces, known as Chain-of-Thought (CoT) reasoning, to decompose complex problems int

arXiv:2610.13797v1 Announce Type: new Abstract: Detecting communicative meaning in dialogue requires modeling not only lexical content but also the participant-role structures through which sentiment,

arXiv:2610.13058v1 Announce Type: new Abstract: Large language model agents increasingly operate in long-horizon environments where early mistakes can trigger cascading failures and trap agents in dea

arXiv:2610.10541v1 Announce Type: new Abstract: Knowledge Graph (KG) quality depends not only on downstream graph validation, but also on the quality of tabular metadata used before integration. In me

arXiv:2610.12452v1 Announce Type: new Abstract: We propose BrickBench, a benchmark for agentic text-conditioned LEGO-set design. Given a prompt, an agent is tasked with producing an assembly that not

arXiv:2601.14994v2 Announce Type: replace-cross Abstract: Data contamination can invalidate benchmark evaluation when a model benefits from memorized evaluation content rather than genuine generalizat

arXiv:2610.12343v1 Announce Type: new Abstract: Large Multi-modality Models (LMMs) have made significant progress in visual understanding and generation, but still face challenges in visual editing, p

arXiv:2610.11559v1 Announce Type: cross Abstract: Coding assistants such as Claude Code and Codex have become a major application of LLM agents, yet existing benchmarks remain far from real-world use,

arXiv:2610.12417v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) struggle with spatial, embodied, physical, and temporal reasoning. We hypothesize that these failures reflect

arXiv:2610.09229v1 Announce Type: new Abstract: LLM-as-judge is now a standard tool for scalable evaluation, but judge performance is still often summarized by a single accuracy number. This aggregate

LightOn OCR-3 is now available on OpenDocRouter, priced at 0.28 / 1m input and 1.40 / 1m output (it is ~$3.19 / 1k pages on ParseBench). We benchmarked it on ParseBench. It is on the pareto frontier f

arXiv:2610.10138v1 Announce Type: new Abstract: Predicting the future impact of a newly published paper is challenging because it must be inferred from heterogeneous evidence available at publication

arXiv:2610.09033v1 Announce Type: new Abstract: Standard safety evaluations of large language models assess harmful requests written in canonical plain text, while models in real-world deployment rout

arXiv:2604.01569v2 Announce Type: replace Abstract: Video multimodal large language models achieve strong results on existing benchmarks, but answer accuracy alone does not establish whether they can

I've been tinkering with GLM-5.3-Flash on two 64GB CMP 170HX cards for a while, and the setup is finally stable enough that I figured I'd share it. I also compared it against the Qwen3.8-Flash-Next se

arXiv:2610.06972v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities in visual understanding and multimodal reasoning, yet they remain fun

arXiv:2610.07132v1 Announce Type: cross Abstract: Croissant has emerged as a standard for machine-readable dataset metadata, yet populating its fields remains labor-intensive and requires careful read

arXiv:2610.07832v1 Announce Type: cross Abstract: Large language models (LLMs) equipped with terminal access have demonstrated strong capabilities in automating software engineering tasks. However, ex

arXiv:2610.08414v1 Announce Type: new Abstract: Image Bitstream Fine-grained Understanding (IBFU) aims to directly perform fine-grained classification and semantic description generation from encoded

arXiv:2610.07634v1 Announce Type: new Abstract: Social media archives are often used to study resistance to climate action, but words, response counters and observed participants do not measure the sa

arXiv:2610.06896v1 Announce Type: new Abstract: Frontier multimodal large language models (MLLMs) are increasingly positioned as general purpose visual reasoners as part of the quest for artificial ge

arXiv:2610.07257v1 Announce Type: new Abstract: Developers increasingly run a fleet of coding agents side by side on one workstation. The tools they reach for, terminal multiplexers like tmux and a ne

arXiv:2610.08675v1 Announce Type: new Abstract: Financial reports repeat values across periods, metrics and accounting lines, allowing an LLM-generated calculation to be numerically correct while citi

arXiv:2610.07745v1 Announce Type: new Abstract: Pose error in precision assembly can corrupt not only what a robot observes but also the coordinate frame in which it acts. On the FORGE benchmark, the

arXiv:2508.05880v3 Announce Type: replace-cross Abstract: Understanding human emotions is central to user-facing AI applications, safety alignment, and the simulation of human behavior. As emotional s

I constantly see recommendations saying that ik_llama.cpp (ikawrakow's fork) is the undisputed king of hybrid CPU/GPU offloading and MoE performance. However, every time I benchmark it against mainlin

arXiv:2610.08720v1 Announce Type: new Abstract: LLM-based agents are increasingly advancing scientific and engineering problem solving, with physics simulation emerging as a challenging yet practical

arXiv:2610.08258v1 Announce Type: new Abstract: Auditing the claimed outcomes of large language model (LLM) training is challenging when model weights and training data are private, while cryptographi

arXiv:2610.06571v1 Announce Type: new Abstract: Brain MRI interpretation is a longitudinal clinical reasoning problem: radiologists compare serial studies, integrate information across MRI sequences,

arXiv:2412.10510v5 Announce Type: replace Abstract: The proliferation of disinformation demands reliable and scalable fact-checking solutions. We present Dynamic Evidence-based FAct-checking with Mult