Search: “benchmark”

Qwen 3.8 27B UD-IQ4_XS even faster on 16GB CUDA

This is an evolution on top of Raymond's KV cache streaming fork - all credits to what enabled this goes to him. The basic idea behind what he enabled was a pool of memory in VRAM that is used differe

13 Sept 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

MAPLE: Memory-Augmented Planning with Language and Evolution

DGX agent

arXiv:2609.11636v1 Announce Type: new Abstract: Domain practitioners understand their business constraints but may lack operations-research expertise or dedicated support. LLM-based optimization agent

model-releasesarxiv-cs-ai

12 Sept 2026

MOSAIC: Query-Aware Exploration Policy Adaptation for GraphRAG

arXiv:2609.11065v1 Announce Type: new Abstract: Graph Retrieval-Augmented Generation (GraphRAG) can connect evidence distributed across a corpus graph, but most systems use largely shared exploration

12 Sept 2026

What's the Story with Agnes-3.0-Flash?

While browsing a benchmark list site, I spotted a recently published 33B parameter model which claimed to beat Qwen3.8 27b on the ArtificialAnalysis (AA) intelligence index. I was obviously excited. B

12 Sept 2026

7900 XTX + 32/64GB RAM for Qwen 3.8 Flash Next?

Planning to build a PC mainly for local LLMs/coding agents. I keep seeing 3090 + Qwen 3.8 Flash Next benchmarks, but could not find enough info for the 7900 XTX 24GB. 3090s are hard to find where I li

Research

A Multi-Stage Rule-Chaining Framework for Compositional and Interpretable Cognitive Reasoning

arXiv:2609.10654v1 Announce Type: cross Abstract: The Abstraction and Reasoning Corpus (ARC) benchmarks cognitive generalization, the ability to infer and apply abstract rules from limited examples. T

researcharxiv-cs-lg

Applications

A Station-Based Evaluation of Machine Learning-based Weather Forecasting Models in Northern Norway

arXiv:2609.10564v1 Announce Type: cross Abstract: Recent machine learning weather prediction (MLWP) models have demonstrated remarkable forecasting skill on global reanalysis-based benchmarks. However

applicationsarxiv-cs-lg

Safety

Alignment Reduces Expressed but Not Encoded Gender Bias: A Unified Framework and Study

arXiv:2603.24125v3 Announce Type: replace Abstract: During training, Large Language Models (LLMs) learn social regularities that can lead to gender bias in downstream applications. Most mitigation eff

safetyarxiv-cs-cl

Assessing the Reusability of Public Speech Resources for Low-Resource Languages: A Central Kurdish Case Study

arXiv:2609.11246v1 Announce Type: new Abstract: Kurdish is spoken by millions of people, but little technology can read it aloud. A recent study released three Kurdish voices, 35 hours of recorded spe

Augustinian BabyLM: What Ostensive Definition Can and Cannot Teach a Small Language Model

arXiv:2609.11870v1 Announce Type: new Abstract: A language model normally begins training with random word embeddings: whatever 'banana' means must be learned from training corpora. I implement St. Au

Agents

Beyond the price per token: Choosing the right OpenAI model on Amazon Bedrock for your workload

Comparing models on dollars per million tokens misses what production workloads actually pay for: outcomes. This post shares an open-source benchmarking harness that measures cost per correct answer,

agentsaws-ml-blog

Beyond Visual Quality: Evaluating Physical Consistency under Ego-Motion with EgoGenEval

arXiv:2609.11172v1 Announce Type: new Abstract: Recent visual generators produce high-fidelity images yet often violate physical consistency under ego-motion, limiting their use for spatial reasoning

Compute-Bounded Security Assurance - Coverage, Verification, and Response under Resource Constraints

arXiv:2609.09229v1 Announce Type: cross Abstract: Additional inference compute can increase the number of correctly resolved security-assurance tasks, but repeated success, unique coverage, accepted e

Fortunate Recall: Ontology-Driven Memory Lifecycle Management for Persistent Coherence in LLMs

arXiv:2609.10413v1 Announce Type: new Abstract: Current LLM memory systems treat all personal facts identically, so stores grow without bound while retrieval precision degrades. The core challenge is

Local Ai

LAION-Mobile: Evaluating Deepfake Detectors On One Million Smartphone Photos

arXiv:2609.11134v1 Announce Type: new Abstract: Most Deepfake detectors report near-perfect AUC scores on their reference benchmarks. However, a recent ICML position paper argues that these evaluation

local-aiarxiv-cs-cv

Learning structural balance of graphs from quantum spectral features

arXiv:2609.11736v1 Announce Type: cross Abstract: We develop a quantum approach to spectral feature extraction from the density of states (DOS) of a problem-dependent Hamiltonian, and apply it to mach

Measuring Progress in Reasoning Toward Mathematical Discovery with Automatic Verification

arXiv:2603.15617v2 Announce Type: replace Abstract: Can AI make progress on important, unsolved mathematical problems? Large language models are now capable of sophisticated mathematical and scientifi

Optimizing AI Inference Across the Deployment Stack

arXiv:2609.10550v1 Announce Type: cross Abstract: AI deployment performance is shaped not by model architecture alone, but by interactions among compression, compiler transformations, and serving poli

PRAGMA: Evaluating Personalized Guidance with Memory Alignment in Lifelong Conversations

arXiv:2609.09664v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly deployed as personalized assistants that interact with users over extended periods of time. As conversatio

RESCUE-BENCH: Towards Relation-Aware Multi-Party Emotional Support Conversation Systems

arXiv:2609.09657v1 Announce Type: new Abstract: Existing emotional support conversation systems mainly focus on one-on-one seeker-supporter interactions and individual emotional states, leaving interp

RetroThinker: Enabling Retrospective Thinking in Speech LLMs

arXiv:2609.11864v1 Announce Type: cross Abstract: Speech large language models (SpeechLLMs) offer reduced latency and retain paralinguistic nuances that are typically lost in cascaded automatic speech

Spomin - Live KV cache compaction (Experimental for Qwen)

I’ve been building 'Spomin', a router that replaces context with summaries directly in the KV cache. The goal is to keep long running sessions going without repeatedly stopping for full compaction and

Studying Without a Syllabus: Task-Agnostic Environment Preprocessing

arXiv:2609.10824v1 Announce Type: cross Abstract: Before an LLM agent tackles tasks in a new environment, it can inspect available corpora and tools and construct reusable resources such as indices, s

Terminal Bench v4 scores

Some people says terminal bench reflects model intelligence better than the intelligent index. From the look of it, the ranking does seem to reflect how people feel about the open and closed models. F

A Later Test Set Is Not a New Domain: Pretraining Familiarity Survives a Contamination-Free Hold-Out

arXiv:2609.10357v1 Announce Type: new Abstract: Time-series foundation models are evaluated almost exclusively on public archives that predate them, so a strong score cannot be separated from having s

A visual large language foundational model for medical image recognition using clinician-oriented social media

arXiv:2609.06914v1 Announce Type: new Abstract: Large language models (LLMs) have demonstrated strong capabilities across diverse domains, showing considerable potential in medicine. However, their ap

Alignment Whack-a-Mole : Finetuning Activates Verbatim Recall of Copyrighted Books in Large Language Models

arXiv:2603.20957v4 Announce Type: replace-cross Abstract: Frontier LLM companies have repeatedly assured courts and regulators that their models do not store copies of training data. They further rely

Automated Design of Inventory Policy with Large Language Models: An Exploratory Study

arXiv:2609.08071v1 Announce Type: new Abstract: Firms making inventory decisions have access to operational data, optimization tools, and large language models (LLMs). Typically, data characterize the

Beyond One-Size-Fits-All: Sample-Adaptive Strategy Routing for Vision Token Pruning in MLLMs

arXiv:2609.10346v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) process hundreds or thousands of visual tokens per image, incurring prohibitive inference costs. While existing

Research

BIO-MEMART: Biometric-Aware KV Cache Memory for Multi-User LLM Agents

arXiv:2609.08566v1 Announce Type: new Abstract: KV cache is evolving from a serving optimization into an external memory substrate for long-term LLM agents. In a shared multi-user deployment, however,

researcharxiv-cs-ai

CHIMERA Challenge Task 2 and 3: Response Subtypes Classification and Progression Survival Prediction in Bladder Cancer Patients using Multimodal Datasets

arXiv:2609.09510v1 Announce Type: cross Abstract: High-risk non-muscle-invasive bladder cancer (HR-NMIBC) carries substantial risks of recurrence and progression, while current clinical risk stratific

Closing the Consistency Gap: Self-Evolving Agents That Learn to Stay on Course

arXiv:2609.08832v1 Announce Type: new Abstract: Large language model (LLM)-powered agents can be accurate on average yet unreliable in production, a discrepancy that has been observed but remains larg

CPU Only Experimental Sloppy Deepseek V4.1 Flash

Title says it all. https://github.com/gjabdelnoor/Day1DeepseekV4.1-CPU The goal is pretty simple, I like having infinite slow tokens from the bioinformatics machine in the lab to run overnight or over

Do Web Agents Investigate Before They Decide?

arXiv:2602.05354v3 Announce Type: replace Abstract: Autonomous web agents are increasingly deployed in moderation and policy enforcement, where correct decisions often depend on evidence that is not i

DSAEval: Evaluating Data Science Agents on a Wide Range of Real-World Data Science Problems

arXiv:2601.13591v3 Announce Type: replace Abstract: Recent LLM-based data agents aim to automate data science tasks ranging from data analysis to deep learning. However, the open-ended nature of real-

From Symbolic Perception to Logical Deduction: A Framework for Guiding Language Models in Geometric Reasoning

arXiv:2609.10335v1 Announce Type: cross Abstract: Plane geometry remains a significant challenge in AI, requiring the integration of visual perception and mathematical reasoning. While Large Multimoda

GANDR: Claim Auditing for Verifiable Legal Answer Generation

arXiv:2609.10293v1 Announce Type: new Abstract: In high-stakes domains such as legal practice, a language-model answer is only useful to the extent that a reader can verify each claim against the sour

Local LLM / Qwen 3.8 win

I’ve been working on perfecting my setup ever since 3.8 hit, but no longer pay for subscriptions. I had a coding interview today and setup openrouter ahead of time as another option. I figured I could

OntoKG-EQ: A provenance-grounded, competency-question-governed knowledge graph for auditable analyst querying

arXiv:2609.08869v1 Announce Type: cross Abstract: Analysts in emerging equity markets keep answering the same questions. Did fundamentals match the market's response? How does the local currency co-mo

OntologyAligner: Ontology-Aligned Retrieval and Hierarchy-Guided Large Language Model Reranking for Biomedical Ontology Normalization

arXiv:2609.10055v1 Announce Type: cross Abstract: Biomedical ontology normalization maps free-text expressions to standardized concepts, enabling consistent integration and analysis of biomedical data

Programmable World Model

arXiv:2609.10540v1 Announce Type: new Abstract: Recent video world models generate increasingly realistic and interactive visual experiences, yet lack reliable mechanisms for maintaining persistent wo

RAP: Research Attention Prediction Reveals Target-Conditioned Evidence Acquisition Biases

arXiv:2609.10092v1 Announce Type: cross Abstract: Large language models (LLMs) increasingly act as research agents, yet their ability to track shifts in research attention is difficult to evaluate bec

Reasoning-Aware Compression: Identifying and Protecting Vulnerable Reasoning Circuits for Energy-Efficient LLM Deployment

arXiv:2609.05512v1 Announce Type: new Abstract: Large Reasoning Models (LRMs) impose substantial energy costs during deployment, yet current compression methods apply uniform quantization across all c

Scoring Without the Engine: Validating a Deterministic, Manipulation-Resistant Content Score for Generative Engines, End to End

arXiv:2609.07559v1 Announce Type: new Abstract: How do you validate a cheap, deterministic proxy for an oracle that is expensive, rate-limited, and non-stationary? We present a protocol built on adver

Staying on the Attack Path: Structured State for Long-Horizon Automated Penetration Testing

arXiv:2609.07344v1 Announce Type: cross Abstract: Large language model (LLM) based agents are increasingly applied to cybersecurity tasks such as vulnerability discovery and automated penetration test

Agents

UnitBoost: Managing Compound LLM Systems with a Merge Operator, Not a Model

arXiv:2609.09815v1 Announce Type: cross Abstract: Compound LLM systems often solve a coordination problem by adding a higher-level LLM. The resulting meta-agent reads workers' outputs, writes the fina

agentsarxiv-cs-cl

VST: Verifiable Structured Transport for Auditable Agent-to-Agent Alpha Discovery

arXiv:2609.07065v1 Announce Type: new Abstract: Agent-to-agent (A2A) alpha discovery is slowed by repeated feedback cycles between mining and evaluation agents, whose hand-offs, in contemporary LLM mu

When Agent Governance Helps

arXiv:2609.05531v1 Announce Type: cross Abstract: No specification says how a governed autotelic AI agent organization, where agents pursue self-generated goals inside guardrails, should be designed a