Search: “benchmark”

SemComp-Bench: Benchmarking Semantic Task Completion in Video Generation

arXiv:2608.17426v1 Announce Type: cross Abstract: We introduce Semantic Task Completion Video Generation, an outcome-oriented video generation task. Under this formulation, success requires both achie

researcharxiv-cs-ai

19 Aug 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Research

AMPLIFAI: A Multiphase CT Dataset for Benchmarking Clinical Reasoning in LI-RADS Assessment of Liver Lesions

DGX agent

arXiv:2608.14778v1 Announce Type: new Abstract: Hepatocellular carcinoma (HCC) is the third leading cause of cancer-related mortality worldwide, with early detection improving survival from 70%. The s

researcharxiv-cs-cv

18 Aug 2026

Benchmarking Frontier Text-to-Image Models on Image-Description Prompts

arXiv:2608.14976v1 Announce Type: new Abstract: Text-to-image models are typically reported on average-case prompts, which understates the gap between systems on compositionally demanding requests inv

18 Aug 2026

Platform Adaptation Under Governance Interventions: Actor Best-Response Modeling and an External Public-Case Benchmark

arXiv:2608.15131v1 Announce Type: new Abstract: Digital platforms govern by changing rules: rankings, monetization thresholds, moderation standards, verification systems, disclosure requirements, appe

18 Aug 2026

Exposition on over-squashing problem on GNNs: Current Methods, Benchmarks and Challenges

arXiv:2311.07073v3 Announce Type: replace Abstract: Graph-based message-passing neural networks (MPNNs) have achieved remarkable success in both node and graph-level learning tasks. However, several i

safetyarxiv-cs-lg

17 Aug 2026

A Browser-Native Digital Test Range for Benchmarking 4D Ocean-Glider Planning Algorithms

arXiv:2608.13511v1 Announce Type: new Abstract: Repeated in-situ evaluation of ocean-glider planners requires scarce vehicles, operators, deployment and recovery resources, and ocean conditions that c

safetyarxiv-cs-ro

Can Generalist Vision Language Models (VLMs) Rival Specialist Medical VLMs? Benchmarking and Strategic Insights

arXiv:2506.17337v5 Announce Type: replace-cross Abstract: Vision Language Models (VLMs) have shown promise in automating image diagnosis and interpretation in clinical settings. However, developing sp

researcharxiv-cs-ai

Cueless EEG imagined speech for subject identification: dataset and benchmarks

arXiv:2501.09700v2 Announce Type: replace-cross Abstract: Electroencephalogram (EEG) signals have emerged as a promising modality for biometric identification. While previous studies have explored the

applicationsarxiv-cs-ai

Qwen3.8 Benchmarks Converted to Charts

https://preview.redd.it/uvdb7g4o7djh1.png?width=1600&format=png&auto=webp&s=1db6176144bcf8f40efb15cc04e8370ad52bfe2f https://preview.redd.it/rj4l66eq7djh1.png?width=1257&format=png&auto=webp&s=692b5f7

local-air-localllama

Training and Benchmarking Code Generation for Physics-Inspired Animations

arXiv:2602.10840v2 Announce Type: replace Abstract: Large language models (LLMs) have been widely studied in areas such as mathematical reasoning, complex coding, and scientific problem solving. Howev

researcharxiv-cs-lg

Analysis of Federated Aggregation under Model Poisoning and Backdoor Attacks: A Reconstructed Cross-Dataset and Cross-Architecture Benchmark

arXiv:2608.11423v1 Announce Type: cross Abstract: Robust comparisons of federated aggregation methods require joint consideration of predictive performance, threat definitions, metric semantics, and e

13 Aug 2026

Benchmarking Trustworthiness of SLMs: Pre-trained vs. Compressed

arXiv:2608.11981v1 Announce Type: new Abstract: Small Language Models (SLMs) have emerged as a more efficient alternative to traditional Large Language Models (LLMs), offering promising potential in r

safetyarxiv-cs-cl

13 Aug 2026

BreastMammo and DenseMammo: Benchmarks for Mammography Domain Generalization

arXiv:2608.10271v1 Announce Type: cross Abstract: Breast density classification is a critical component of breast cancer risk assessment, yet AI models often struggle to generalize across clinical sit

researcharxiv-cs-lg

12 Aug 2026

Energy and Performance Benchmarking of Deep Learning Models for Breast Cancer Detection

arXiv:2608.09996v1 Announce Type: cross Abstract: Recent advances in machine learning have greatly improved breast cancer detection, enabling more accurate and timely diagnosis. Deep learning (DL) mod

researcharxiv-cs-ai

12 Aug 2026

Classical SU(2) Models Match or Exceed Shallow Variational Quantum Circuits on Vision Benchmarks

arXiv:2608.07822v1 Announce Type: cross Abstract: Quaternion-valued neural networks and variational quantum circuits (VQCs) both derive local transformations from SU(2) geometry, yet their performance

local-aiarxiv-cs-lg

Local Benchmark : Muse Glimmer 30B vs Qwen 3.6 27B vs Gemma4 31B (and many other models and finetunes)

model-releasesr-localllama

Needs a lot of requests compared to Qwen (almost twice) and Gemma (almost x3). Final score is fine, even though it is 'not a coding model' https://wonderrico.github.io/local_llm_benchmark/benchmark-ma

MateInfoUB: A Real-World Benchmark for Testing LLMs in Competitive, Multilingual, and Multimodal Educational Tasks

arXiv:2507.03162v2 Announce Type: replace-cross Abstract: The rapid advancement of Large Language Models (LLMs) has transformed various domains, particularly computer science (CS) education. These mod

Sparse corruption in low-rank matrix inference: the PCA benchmark

arXiv:2511.11927v2 Announce Type: replace-cross Abstract: Principal Component Analysis (PCA) is a standard tool for extracting a low-rank signal from noisy observations. It is known that applying PCA

Benchmarking Deep Learning Models for Dense Event Classification of Offshore Wind Infrastructure in Sentinel-1 Time Series

arXiv:2608.04706v1 Announce Type: new Abstract: Monitoring of offshore wind energy infrastructure life cycles, especially during the deployment phase, is an important contribution for stakeholders to

applicationsarxiv-cs-lg

6 Aug 2026

IslamicTurathBench: A Multi-Task, Multi-Discipline Benchmark for Evaluating Large Language Models on the Islamic Scholarly Tradition (turath)

model-releasesarxiv-cs-cl

arXiv:2608.04703v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used for question answering, education, and research, including in religious and cultural domains where an

6 Aug 2026

RooflineBench: A Benchmarking Framework for On-Device LLMs via Roofline Analysis

arXiv:2602.11506v4 Announce Type: replace-cross Abstract: The transition toward localized intelligence through Small Language Models (SLMs) has intensified the need for rigorous performance characteri

6 Aug 2026

Modeling Long-Term Memory and Temporal Attention Shifts for Video Salient Object Ranking with a New Benchmark

arXiv:2203.17257v2 Announce Type: replace Abstract: Salient Object Ranking (SOR) aims to estimate the relative saliency order among multiple salient objects. While SOR has been extensively studied in

5 Aug 2026

AeroLLE: Constrained Pseudo-Supervision for Nighttime Aerial Image Enhancement with the AeroNight-1.5K Benchmark

arXiv:2608.00702v1 Announce Type: new Abstract: Nighttime aerial image enhancement is challenged by spatially nonuniform exposure, mixed illumination, and weak structural evidence, while registered no

EEG-FM-Compass: Progress, Benchmarking, and Future Directions for EEG Foundation Models

arXiv:2601.17883v3 Announce Type: replace-cross Abstract: Electroencephalography (EEG) foundation models (FMs) have recently emerged as a promising paradigm for brain-computer interfaces, aiming to le

I benchmarked the 4 models I had pulled. The 1.1GB one beat the 2GB one at math and lost badly at extraction.

152 generations, deterministic grading (exact number/string/JSON/regex), no LLM judge on my 16GB laptop. task type | deepseek-r1:1.5b (1.1GB) | llama3.2:3b (2.0GB) | gemma:2b | codellama 7b arithmetic

model-releasesr-ollama

Rethinking PPG-based Sleep Staging: Datasets, Metrics, and Benchmarks

arXiv:2608.00943v1 Announce Type: cross Abstract: Automated sleep staging assigns discrete stage labels to successive time epochs throughout an overnight recording; conventionally each window spans at

researcharxiv-cs-lg

A Benchmark for Strategic Auditee Gaming Under Continuous Compliance Monitoring

arXiv:2605.06340v2 Announce Type: replace-cross Abstract: Continuous post-deployment compliance audits, mandated by emerging regulations such as the EU AI Act and Digital Services Act, create a class

3 Aug 2026

I benchmarked classic vector RAG vs Google's new OKF format vs both combined — same corpus, same 7 questions, all local (Ollama + ChromaDB)

Google Cloud published OKF (Open Knowledge Format) on June 12th — a spec for storing curated knowledge as a directory of markdown files with YAML frontmatter. One concept per file, linked to each othe

local-air-localllama

3 Aug 2026

Semantics of Subterfuge: Benchmarking Legal Deception Detection Against General-domain State-of-the-Art

arXiv:2607.29066v1 Announce Type: cross Abstract: Deception detection has critical implications for legal proceedings, law enforcement, and online security. Although human judgment is limited in accur

applicationsarxiv-cs-ai

3 Aug 2026

Has anyone actually benchmarked where the 'big-model orchestrator + local-model worker' split breaks down?

model-releasesr-localllama

I keep seeing the 'use a big model via API as the architect, run local small/mid models as workers' pattern recommended for people with modest local hardware. I've been running it myself (orchestrator

31 Jul 2026

Safety-Gated Agentic Supervisory Control on a Coupled Distillation Benchmark: Regime Map, Auditable Gate, and Co-Design Findings

arXiv:2607.27849v1 Announce Type: cross Abstract: An open-weight LLM can write composition setpoints every five minutes. What a plant still needs is a hard check: named constraints, logged margins, an

31 Jul 2026

FakeIDet3-DB: Refining Digital Attacks and Patch Extraction for Secure ID Benchmarking

arXiv:2607.26641v1 Announce Type: new Abstract: Identity document (ID) authentication relies on the structural integrity of complex, high-frequency security patterns. However, advanced Generative AI m

local-aiarxiv-cs-cv

30 Jul 2026

HealthSLM-Bench: Benchmarking Small Language Models for Mobile and Wearable Healthcare Monitoring

arXiv:2509.07260v5 Announce Type: replace-cross Abstract: Mobile and wearable healthcare monitoring play a vital role in facilitating timely interventions, managing chronic health conditions, and ulti

applicationsarxiv-cs-lg

30 Jul 2026

COCO-OLAC: A Benchmark for Occluded Panoptic Segmentation and Image Understanding

arXiv:2409.12760v3 Announce Type: replace Abstract: To help address the occlusion problem in panoptic segmentation and image understanding, this paper proposes a new large-scale dataset named COCO-OLA

29 Jul 2026

Linear-LLM-SCM: Benchmarking LLMs for Coefficient Elicitation in Linear-Gaussian Causal Models

arXiv:2602.10282v2 Announce Type: replace Abstract: Large language models (LLMs) have shown potential in identifying qualitative causal relations, but their ability to perform quantitative causal reas

local-aiarxiv-cs-lg

29 Jul 2026

SONG: A Photorealistic 3D Gaussian Simulation Platform for Benchmarking Social Navigation

arXiv:2607.25219v1 Announce Type: new Abstract: Social navigation has progressed from simplified 2D environments toward a more general vision-based setting, in which a robot needs to achieve socially

safetyarxiv-cs-ro

29 Jul 2026

AGVBench: A Reliability-Oriented Benchmark of Data Augmentation for Vein Recognition

arXiv:2607.02271v2 Announce Type: replace Abstract: Vein recognition is a secure biometric technology often constrained by limited annotated data and imaging variations. While data augmentation mitiga

Benchmarking the Domain Gap: Model Selection Instability Under Domain Shift in Video Capsule Endoscopy

arXiv:2607.22736v1 Announce Type: new Abstract: Video capsule endoscopy (VCE) classification is typically evaluated within a single dataset, yet clinical deployment demands robustness across acquisiti

researcharxiv-cs-cv

Might need math+code benchmark for frontier model(LLMs Silently Replace Math)[D]

model-releasesr-machinelearning

Hello guys. I found some problems in current frontier models. And want to share. # math_code_hallucination > Record of a failure caused by combining mathematics and code in a single prompt. --- ## Cas

Plato-Bio: verification-first biological novelty screening with temporal rediscovery and structural benchmarks

arXiv:2607.23975v1 Announce Type: new Abstract: Large language model research agents can connect literature retrieval, analysis code, and manuscript preparation, but coherent output does not establish

safetyarxiv-cs-ai

TEmBed-T: A Multi-Dimensional Benchmark for Table-Level Embeddings

arXiv:2607.24130v1 Announce Type: cross Abstract: Tabular data is the dominant structured-data modality, and learning table representations has become a core research direction. Table-level embeddings

QC-PHAST Search: Classical--Quantum Query Benchmarks for Finite-Pool Rare-Regime Discovery

arXiv:2607.21995v1 Announce Type: cross Abstract: Rare-regime discovery in parameterized dynamical systems is an active-search problem: find one verified parameter at which a scientifically defined qu

27 Jul 2026

Agents

Can AI Agents Really Complete RTL-to-GDS? Lessons from Benchmarking Tool-Interactive EDA Workflows

arXiv:2607.17528v3 Announce Type: replace Abstract: Large language model (LLM) agents are extending electronic design automation (EDA) beyond static RTL generation toward long-horizon, tool-interactiv

agentsarxiv-cs-ai

24 Jul 2026

Position Bias is Hidden Behind Ceiling Effects: A Permutation Diagnostic for LLM Benchmarks

model-releasesarxiv-cs-cl

arXiv:2607.20864v1 Announce Type: cross Abstract: Position bias in multiple-choice LLM evaluation is widely cited as a confound in capability comparisons, but published measurements rely on single ans

24 Jul 2026

A Systematic Benchmark of Intensity Normalisation Methods for 3D Knee MRI Segmentation and Cross-Domain Generalisability

arXiv:2607.20028v1 Announce Type: cross Abstract: Robust out-of-the-box performance is essential for the clinical deployment of deep learning models in medical imaging. An important but underexplored

CPU-only inference on a Celeron N5095 SBC: 6 models from 0.6B to 8B, benchmarked

model-releasesr-localllama

I wanted to know how cheap you can go and still run local models, so I ran Ollama CPU-only on a Youyeetoo X1S. It's a single-board x86 machine with a Celeron N5095 (Jasper Lake, 4C/4T, 15W), 16GB of R

DocOps: A Verifiable Benchmark for Autonomous Agents in Complex Document Operations

arXiv:2607.19865v1 Announce Type: new Abstract: As autonomous agents rapidly evolve, their ability to reliably manipulate ubiquitous digital documents has become critical for enabling general-purpose

When Benchmarks Mislead: Shortcut Learning, Length Confounds, and the Limits of Cross-Dataset Generalization in Multilingual Fake News and Sarcasm Detection

arXiv:2607.14131v2 Announce Type: replace Abstract: Cross-dataset generalisation is a fundamental requirement for deploying text classifiers in real-world settings, yet systematic evaluation across co

applicationsarxiv-cs-cl