Search: “benchmark”

12 abliterated Gemma 4 12B variants, one base, 165 GPU hours - Abliterlitics

I ran 11 uncensored variants of Gemma 4 12B that I grabbed from huggingface, sorting by downloads. 10 full abliterations plus 2 LoRA adapters which were requested to be added in the comparison, agains

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

A Physical Response-and-Memory Model for Muon Optimization

DGX agent

arXiv:2608.22994v1 Announce Type: cross Abstract: Training large language models is costly. How low a loss the same compute can ultimately reach depends on how each step's gradient is converted into a

model-releasesarxiv-cs-ai

25 Aug 2026

Research

All four leading LLMs talk more than they listen to personality-verified synthetic help-seekers

arXiv:2608.22425v1 Announce Type: cross Abstract: Large language models are increasingly consulted at moments of distress, yet single-turn benchmarks neither test sustained exchanges nor distinguish b

researcharxiv-cs-cl

Beyond the Harness: End-to-End Optimization of Context Artifacts for Enterprise Text-to-SQL

arXiv:2608.22830v1 Announce Type: new Abstract: Deploying LLMs for enterprise Text-to-SQL is bottlenecked less by the model than by what context reaches it: business logic spans thousands of tables, a

CHIMERA Challenge: Biochemical Recurrence Prediction in Prostate Cancer Patients using multimodal datasets

model-releasesarxiv-cs-cv

arXiv:2608.21497v1 Announce Type: cross Abstract: Biochemical recurrence (BCR), defined as any detectable prostate-specific antigen level after prostatectomy with confirmatory elevation, is widely use

Safety

Evaluation Awareness in Language Models: Representation, Verbalization, and Control

arXiv:2608.21766v1 Announce Type: cross Abstract: Both capability and safety benchmarks rest upon the assumption that the behavior of language models undergoing a test is informative about their behav

safetyarxiv-cs-ai

Gauss--Hermite Quadrature for Gaussian-Mixture Entropy with an Action-Space Hermite Surrogate

arXiv:2608.21467v1 Announce Type: cross Abstract: Gaussian distributions are used to model uncertainty in signals and states, and Gaussian mixtures are often used when the underlying distribution is m

K-Bench: measuring model performance on real scientific agent requests

arXiv:2608.21601v1 Announce Type: new Abstract: Benchmarks for scientific artificial intelligence are mostly written to be scored: multiple-choice questions, curated agent tasks with reference solutio

LLM-Specific Utility for Retrieval-Augmented Generation

arXiv:2510.11358v3 Announce Type: replace-cross Abstract: Retrieval-augmented generation (RAG) is typically optimized for topical relevance, yet its success ultimately depends on whether retrieved pas

Applications

Loss Landscape Features That Make Adam Stall: Definitions, Estimators, and the Preconditioned Hessian View

arXiv:2608.22145v1 Announce Type: new Abstract: Across implicit-neural-representation (INR) architectures and analytic benchmarks we observe that a thoroughly tuned Adam (especially its learning rate

applicationsarxiv-cs-lg

Hardware

OpenAI Jalapeño: Better Than Nvidia Blackwell

OpenAI’s Jalapeño is a self‑designed ASIC for large‑language‑model inference that was taped‑out in about 16 months, developed with Broadcom and featuring HBM4 memory. Benchmarks on the InferenceX suit

hardwaresemianalysis

Retrieval Needs Multivectors: An Exponential Separation

arXiv:2608.21494v1 Announce Type: cross Abstract: Recent works have highlighted the expressive limitations of embedding based retrieval models through both theoretical analyses and challenging benchma

Scaffold CoT: A CoT dataset built around the failures of small model (>5B Params) free form thinking. Hope its useful to you guys!

TL;DR - A ~4M example, ~3B token CoT dataset designed around helping small models think more concisely, accurately and reliably. Hi all! For the past few months I have been working on a dataset design

// There Is No Neutral Harness // Great work discussing some of the issues in harness evaluation. Twelve open-weight models answer the same …

// There Is No Neutral Harness // Great work discussing some of the issues in harness evaluation. Twelve open-weight models answer the same 3,679 items from ARC, HellaSwag, MMLU, and TruthfulQA under

model-releasesdair-ai--x

Local Ai

Best Local Vision Language Models - August 2026

Share what your favorite models are right now and why. Given the nature of the beast in evaluating VLMs (untrustworthiness of benchmarks, immature tooling, intrinsic stochasticity), please be as detai

local-air-localllama

Building and Evaluating a Synthetic Bengali Speech Resource for Telecom Customer Care

arXiv:2608.20346v1 Announce Type: new Abstract: Speech systems used in customer-facing applications often require domain-specific language coverage. We present a synthetic Bengali speech dataset for t

ClawSentry: A Progressive Multi-Tier Security Monitor for Safeguarding Autonomous LLM Agents

arXiv:2608.21101v1 Announce Type: cross Abstract: As large language model (LLM) agents move from conversation to executing code, reading local files, and orchestrating external tools, a single agent h

Delay-corrected Bellman operator + causal attribution for constrained RL contraction proof under unknown stochastic delay [R]

model-releasesr-machinelearning

Standard constrained RL assumes consequences are immediate and attributable to the current action. This breaks down whenever violations are delayed and stochastic, which is most real-world settings yo

Designing a Robust LLM-Based Evaluation System for Agentic AI in Drug Discovery Through Human Alignment

arXiv:2608.21057v1 Announce Type: new Abstract: Agentic large language model (LLM) systems are reshaping scientific workflows in chemistry and drug discovery, but evaluating their open-ended, tool-aug

Evaluating Large Language Model Performance on International Maritime Dangerous Goods Code Compliance

arXiv:2608.21036v1 Announce Type: new Abstract: The transport of dangerous goods by sea is a high-consequence activity governed by the International Maritime Dangerous Goods (IMDG) Code, a complex reg

Fine-Grain GPU Parallelization of the Generalized Partition Crossover for Large-Scale Traveling Salesman Problems

arXiv:2608.21233v1 Announce Type: new Abstract: The Traveling Salesman Problem (TSP) is one of the most extensively studied NP-hard optimization problems. Genetic Algorithm (GA)-based solvers, such as

Hadith computational science in the age of large language models: a critical narrative review

arXiv:2608.20364v1 Announce Type: cross Abstract: We examine how hadith computational science is being reshaped by transformer models, retrieval-grounded pipelines, and large language models (LLMs). R

Identity-Aware Human-Object Interaction Motion Captioning

arXiv:2608.20690v1 Announce Type: cross Abstract: Existing human-object interaction (HOI) motion captioning methods typically describe what happens while referring to the subject using generic terms s

MigrationNarrate: A Dataset for Detection of Migration Narratives in YouTube Videos

arXiv:2608.20984v1 Announce Type: cross Abstract: Narratives are central to how social communication is framed, making their detection critical for understanding and analysing public discourse. Prior

Hardware

NVIDIA Vera Rubin and Blackwell Set a New Standard for Agentic AI Performance per Watt

NVIDIA’s new GPUs in the Vera Rubin and Blackwell families set a new benchmark for agentic‑AI performance per watt on AgentX, an open‑source inference test that models realistic multi‑step, tool‑using

hardwarenvidia-developer

Agents

Peer-Voted LLM-Agent Stress Tests Find Feed-Induced Lexical Convergence but No Reliable Matched-Exposure Advantage for Distributed Sources

arXiv:2608.20438v1 Announce Type: cross Abstract: Population-level behavior in large-language-model (LLM) agents cannot be characterized by single-agent benchmarks. We introduce PV-SST, a peer-voted s

agentsarxiv-cs-ai

Please join r/LowEndLocalAI, a community for running local LLMs on low spec hardware

If you’re trying to run local LLMs on a normal laptop, an older desktop, integrated graphics, limited VRAM, or simply the hardware you already own, r/LowEndLocalAI is meant for you. The idea is simple

Reinforcement Learning for Continuous-Time Jump Markov Decision Processes with Applications to Network Dynamic Pricing

arXiv:2608.20680v1 Announce Type: new Abstract: We study reinforcement learning (RL) in Continuous-Time Jump Markov Decision Processes (CTJMDPs) featuring general discrete state spaces (which need not

Research Paper Quality Recognition Through Textual Feature Analysis

arXiv:2608.20368v1 Announce Type: new Abstract: Knowledge and innovations are shaped by using the quality and credibility of the scientific research. Yet, distinguishing between impactful, high-qualit

Rigorous Evaluation of Large Language Models for Malaria Drug Discovery: Trade-offs in Performance, Scale, and Resource Utility

arXiv:2608.20418v1 Announce Type: cross Abstract: We introduce Malaria-Instruct, a curated instruction-following dataset derived from the ChEMBL Legacy Malaria corpus for Malaria virtual screening, an

TH-GNN: Heterogeneous Temporal Graph Neural Networks for LLM-Agent Shilling Attack Detection

arXiv:2608.20376v1 Announce Type: new Abstract: LLM agents can now generate realistic shilling profiles, fluent reviews, and coherent ratings at scale, systematically defeating recommender-system defe

This is what Qwen 3.8 27b is capable of

Try it here: https://ocean.blackbeardlabs.dev/ Model: Qwen 3.8 27b Q8_X_KL Unsloth Hardware: 3 x RTX3090 Harness: DeepSeek Harness Prompt: /goal I want you to create a **JavaScript + Node.js WebGL pro

Local Ai

TielCoder's 22 GB 4-bit quant matches Opus4.6 medium on recent real life coding issues, surpassing KAT-Coder and Nail as strongest and fastest MoE picks.

Qwen3.8-27B is amazing, but it’s slow. A stronger 35B-A3B Mixture of Experts-coder that can run and solve real codebase issues fast (even on constrained hardware) is a valuable addition to the arsenal

local-air-localllama

When Generated Images Look Right and Retrieve Wrong: Coverage-Guided Cross-Scale Re-Indexing for Knowledge-Faithful Generative Perception

arXiv:2608.20810v1 Announce Type: cross Abstract: Multimodal information systems increasingly route generated visual content back through the same vision-language index that informed its production, s

I spent months pushing my RX 6700 XT into local AI. I think I seriously underestimated this GPU.

My setup: RX 6700 XT 12GB i5-11400F 16GB RAM Ubuntu 26.04 ROCm/TheRock + llama.cpp I've spent the last few months experimenting with different backends, quantization, CPU/GPU split, KV cache, batch si

model-releasesr-ollama

23 Aug 2026

Qwen3.8-27B NVFP4 with vision + 451K token KV-cache on one RTX 5090 (power limited to 400W) at 120 tokens/s average

Hello, So I've been trying lots of combinations in that never-ending landscape of options and settings. I wanted a proper quant of 3.8 27B running as fast as possible on my 5090 at 400W, with vision a

23 Aug 2026

Trying to optimize LLM inference latency — Ollama + OpenWebUI/AnythingLLM

Hi everyone, I'm currently working on optimizing an LLM setup at my company and I'm trying to identify where the actual performance bottleneck is. Our current setup is roughly: LLM: NVIDIA Nemotron mo

model-releasesr-ollama

23 Aug 2026

Grok Voice ranked first

model-releaseselon-musk--x

Grok Voice ranked first Grok Voice Think Fast 2.0 just ranked #1 on Artificial Analysis’ new Speech Agent Arena for highest Task Success Rate This benchmark actually measures what matters Real people

22 Aug 2026

A Standardized Framework for Machine Learning in Power System Protection

arXiv:2608.20181v1 Announce Type: cross Abstract: Studies of machine-learning-based power-system protection increasingly report near-perfect scores, yet the meaning of those scores depends strongly on

Accelerated Genetic Programming Hyper-Heuristics for Simulation-Based Scheduling via Agentic AI

arXiv:2608.19487v1 Announce Type: cross Abstract: Python is widely used in scientific research because it enables rapid development and provides rich ecosystems for data analysis, artificial intellige

Answer-Level Trust Selection for Physical Vision-Language Reasoning

arXiv:2608.19807v1 Announce Type: new Abstract: Vision-language models (VLMs) can estimate physical quantities such as duration, speed, and acceleration from visual observations, but existing benchmar

Applications

Are LLMs becoming similarly creative? Evidence from three years of models

arXiv:2608.19437v1 Announce Type: cross Abstract: Many benchmarks track Large Language Model (LLM) performance on tasks with verifiable answers, but less is known about how LLM performance is evolving

applicationsarxiv-cs-ai

DeepSeek v4 Pro costs most... but it costs less...

When using DeepSeek Pro on Ollama, you'll see that the underlying cost is more significant. But if you actually track the usage, you'll actually find that, at least in my benchmarks, I save a lot of t

model-releasesr-ollama

Disentangling Speaker Traits for Deepfake Source Verification via Chebyshev Polynomial and Riemannian Metric Learning

arXiv:2603.21875v2 Announce Type: replace-cross Abstract: Speech deepfake source verification systems aims to determine whether two synthetic speech utterances originate from the same source generator

Explainable Transformer Models for Clinical Prediction Tasks on Structured Electronic Health Records

arXiv:2608.20315v1 Announce Type: new Abstract: Predictive models over structured electronic health records (EHRs) remain central to machine learning for healthcare, but few have jointly emphasized qu

From Atari to EVE Online: Building on 15 Years of AI Research in Games

model-releasesgoogle-deepmind

This paper traces the expansion of AI research in games over a 15‑year period, from early Atari benchmarks to contemporary MMORPGs such as EVE Online. It introduces SIMA 2, an agent designed to play,

G-CARL: Grounded Checklist-Aligned Reward Learning for Patient-Oriented Medical Report Interpretation

arXiv:2608.20331v1 Announce Type: cross Abstract: Personalized interpretation of medical reports has emerged as an increasingly important need among patients. Addressing this need requires both eviden

Multi-Method Causal Evidence Synthesis: Ranking Candidate Drivers by Convergent Cross-Method Evidence from Observational Data

arXiv:2608.20187v1 Announce Type: cross Abstract: Practitioners inferring causality from observational data usually rely on a single method and treat its output as causal truth. Recent tools select an