All entries

Been tweaking my Qwen 3.8 setup, up to 45+ steady T/ps at 8bit quant. Realised I'm now top T/ps for this model+ctx across all benchmarked M-series chips. Full args linked below, happy to discuss as this was a pain of trial and error.

https://omlx.ai/benchmarks/performance/2pko3m1k - you can expand the raw args, but I have full annotations of what worked and what didn't. I'm now testing the model on acutal coding and haven't seen a

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Connect Open-Webui to local Ollama does not show Ollama Cloud Models

Using Open-Webui I can connect to Ollama Cloud using ollama.com and the api key. It lists the models, it works. but i want to connect open-webui to the loacl ollama installation (windows) because all

[Draft - Open PR] AVX2: Speed up large batch size prompt processing of IQ models by bartowski1182 · Pull Request #27402 · ggml-org/llama.cpp

IQ quants are particularly slow on CPU at large batch sizes (what you'd see for imatrix and perplexity) Benchmark numbers I ran PPL against master and this PR to get speed and numbers on --chunks 50 f

Fine-tuning Cactus Needle 2 can match DeepSeek v4 on the specific task

Hey LocalLlama, Henry from Cactus here! When we trained Needle 2, I had a strict rule to not expose the model to any data sample that remotely felt like these benchmarks. It seemed over-the-top, but b

G9v3-39A5B on artificialanalysis looks good. Has anyone tested it?

I see https://github.com/linuxid10t/llama.cpp/tree/feature/g9v3-support but yeah... Considering that some results place it above Qwen 3.6 27B (the top performer until a few days ago) and that it is an

Getting better at coding doesn't make a model better at everything else

A majority of users in this sub use LLMs for coding/agentic tasks and I see why a lot of value is put into them but many try to say 'Well coding has improved therefore it can just use tool calling and

GLM 5.3 SlopCodeBench Results

Howdy once again, I had a request to try out 5.3 on the benchmarks - they're unsaturated so it's a fun test right now! This one was interesting because I accidentally ran it on all 36 problems (rip 20

[GUIDE] Running CUDA 12.9 on NVIDIA Pascal (GTX 10xx) with Arch Linux

Tested on: X570DD_M570DD / GTX 1050 mobile / Arch Linux x86_64 / NVIDIA 580.178.04 driver / CUDA V12.9.86 What happened: As everyone knows, NVIDIA dropped the Pascal line in the 590.xxx.xx+ and CUDA 1

home lab assistant llm

hey so im currently working to integrate a locally ran llm to control home assistant (through tool calling, no direct connection) so i can use a voice input without needing very specific commands. Its

How do you deal with long-context sessions after restarting llama.cpp?

I run local models on a 128GB Strix Halo and restart llama.cpp fairly often while testing builds, backends and model parameters. The annoying part is long-running agent sessions. Hermes/OpenCode sessi

I built a free, self-hosted app that does everything around a LoRA run — dataset, triage, captions, training (local or rented GPU), then checkpoint comparison

local-air-stablediffusion

I build LoRA Dataset Studio — free, open source, self-hosted, no account and no telemetry. It is not a competitor to ai-toolkit: it orchestrates it. ai-toolkit is the trainer; this is everything befor

I built a visual multi-agent workflow editor that exports runnable CrewAI code — here’s a 5-agent workflow running locally with Ollama

I've been building AgentGraph Studio, a visual editor for designing multi-agent workflows and exporting them as runnable Python code. One thing I wanted to verify was whether the generated code actual

I just built a mini Kimi-K3 from Scratch under 250$. Already beats GPT-2 (124M)!

I pre-trained a 1.02-billion-parameter on Kimi K3 replica trained on 5.00 billion decontaminated tokens for $250. This model has 1.02 billion parameters, of which 145 million are active per token. It

I pushed Qwen3.8-27B to 381 tps for a single request on a RTX 3090

Four days ago I released a hyper-optimized Qwen3.8-27B inference engine for an RTX 3090 (82 tps single request, 672 peak). Since then it went to ~114, then ~138 tps single-user with DFlash2 drafting a

Is it possible to fine-tune gemma4 A4B to generate complex legal principles of court decision? [D]

model-releasesr-machinelearning

I have a big database of local court decisions with a legal sentence which is like a paragraph summary of the doc. I've been tinkering with FTing for many days now, all results inconclusive never beat

Ling-3.0 released all 6 base checkpoints: 2 sizes × 3 stages

AntLing has released the full six-checkpoint matrix for the Ling-3.0 base model. tiny: pretrained, mid-trained, WSM-merged flash: pretrained, mid-trained, WSM-merged The concrete artifact is six separ

Ling 3.0 Tiny makes an amazing auxillery model for Hermes (Qwen 3.8 27B as the primary model)

Just got the setup dialed in yesterday. Getting really good results and is making 3.8 usage feel faster in hermes. I got Qwen to specifically use Ling tiny for simple tasks like context compression an

Applications

Mapping intrinsic rank and informational gravity in complex tabular data: I developed a non-parametric, model-agnostic, information-theoretic diagnostic to bypass the limits of linear, rank, and Euclidean baselines. [R]

applicationsr-machinelearning

Links: Preprint: https://doi.org/10.5281/zenodo.22028087 Entropic Scree Function v1.0.0 / GitHub: https://github.com/tjleestjohn/Entropic-Scree TL;DR: Standard PCA fundamentally fractures non-linear d

[MASSIVE TINY RELEASE] - Supra2-Medium-Base - a tiny 25M parameters model competing heavily with our previous 50M model!

Hey guys! Supra2-Medium is finally out! It's a 25M parameters qwen3 architecture model trained entirely from scratch (on our new rig: RTX 5060 Ti 16GB + the new RTX 5060 8GB!). Here's how it competes

Need help choosing the right AI model/tool for a complete web app workflow

I currently have these models available through Ollama/cloud: And I’m using Claude, Codex, OpenCode, and Ollama as my coding/agent tools. I want to learn professional vibecoding — not just asking AI t

New benchmark just dropped!

The pelican on a bicycle is sooo outdated, so I came up with a new, improved version. Qwen3.8-27b medium (UD-Q4_K_XL) vs. Sol 5.6 high vs. Qwen3.6-35B (UD-Q6_K_XL) Prompt (only real with typo!): 'Crea

Ornith-1.5-35B-A3B Q4 running 60tk/s on 4070Ti.

Not everyone has the disposable income to build a small data center, so making this post for the underdogs as I was very surprised by the performance/results of this 35B MOE model. Context is admitted

Tutorials

PSA: Proper prompt structure REALLY matters in H3

tutorialsr-stablediffusion

I had mistakenly been using a base for H3 prompting from some random tip / example by someone. It worked ok, I thought. But I was getting a bit frustrated because almost every time I was making a long

Qwen 3.8 27B KV f16 vs q8_0 are not equivalents

I'm testing it since release, now with UD 3.0 in my AMD R9700 with ROCm, I always read everywhere that F16 and q8_0 for KV cache are essentially the same... well, I tested it and I can see differences

Qwen3.8-27b has the highest level of 'agency' I've ever seen in a local model

Off a single prompt, given my credentials and the name of my university, qwen3.8-27b was able to successfully pull my class schedule from the kinda shitty and convoluted web of university websites. It

Qwen3.8 27b just exceeded my expectations on svg generation :D

https://reddit.com/link/1vtkgdj/video/595yn0ckdjkh1/player I wanted to try out Qwen3.8 27B 's SVG capabilities but with something different than the pelican on a bicycle. Promt was literally just : cr

Qwen3.8-27B scored 29/30 on AIME 2026 with FP8 + xhigh reasoning — BF16 vs FP8 results

I benchmarked Qwen3.8-27B on MathArena/aime_2026 dataset, comparing BF16 and FP8 weights at medium and xhigh reasoning effort. Interesting findings are: quantized FP8 xhigh is better than BF 16 medium

Agents

Qwen3.8-27B took a serious hit to knowledge vs 3.6

Like many of you I've spent the last few days throwing Qwen3.8-27B against all of my usual use-cases and personal tasks/harnesses and workflows. It's great, phenomenal sometimes, but that's not what t

agentsr-localllama

QwenMix-3.7: Kept seeing posts about Qwen3.8 and 3.6 sharing the same structure.. so I had Qwen3.8 combine them.

I chose to do this thing, not because it was hard, but because it was silly. Posts kept discussing how 3.8 and 3.6 were functionally the same, but based on training (3.8 does have seven new tokens!)..

So. about the speed of Qwen 3.8 27B Q2_K_XL on 3080 12GB

this screenshot is without MTP usage , fully offloaded onto the GPU. using LM STUDIO i wanted to ask you guys if theres a way to make it even faster , as MTP really didnt help and is infact slower due

Spider-man: Brand New Day, does Peter self host his AI? (Spoilers)

This is gonna seem crazy off-topic, but I saw Spider-Man the other day and couldn’t help but notice how well executed E.V. is as an agentic system. compared to Jarvis, it felt much more grounded and c

SyzygyResearch/Mach-1-Additive-35B-GGUF · Hugging Face

They released both GGUFs & custom llama.cpp fork today. 35B MOE in 7GB size which's good for Mobile & Edge devices(Also low memory systems). Up to 120 t/s on Consumer Laptop. GGUFs: https://huggingfac

Tencent begins testing its new flagship model Hunyuan Hy4

From the screenshots: Hy4 is now live, labeled 'Expert-Level Model' + 'Use Tools to Solve Problems' Hy3 is tagged with 'New Upgrade,' positioned as a brand-new general-purpose model DeepSeek, focused

The boring way to run Deepseek V4 Flash-0731 130-150 tks - 16x5060ti 16GB over 2 PLX88096 switches

Component Validated configuration Motherboard ASRock Rack SPC621D8U-2T/OVH CPU Xeon Gold 6330 (Get gold/platinum if interested in Optane Pmem gimmicks) GPU fabric Two Broadcom/PLX PEX88096 islands, ei

Research

The spectral neuron - an ML primitive for scalable and interpretable models [R]

researchr-machinelearning

Worked some time ago on one of the ad teams at Yahoo, and this grew out of a question I kept returning to while there are there 'simple' models that are both simple, scalable, interpretable, and contr

TinySearch v0.6.1 - still a lightweight web research tool for local LLMs, now with bring-your-own-browser support

Hey everyone, Posted TinySearch here a few versions ago and got a bunch of useful feedback, so figured I'd post an update because the thing has changed quite a bit since then. Repo: [https://github.co

VSCode Failed to fetch

https://preview.redd.it/xel6i6kujjkh1.png?width=2396&format=png&auto=webp&s=b78cd87da266324364e6b8cb8457a7f0e029d163 https://preview.redd.it/ofy8ttyxjjkh1.png?width=2396&format=png&auto=webp&s=3e64405

What do you think of my Modelfile for Qwen3.8

Hello, any params I missing or mis-tuning for my coding model ? FROM ./models/unsloth/Qwen3.8-27B-Q5_K_M.gguf # Generation parameters optimized for logical reasoning and coding tasks PARAMETER tempera

After Price hike DeepSeek-V4-Flash:0731 is dumb in opencode & Else where. How it is in Ollama cloud?

seems like others are now using lower precision models. On open code sub i see a lot of people saying model has gone dumber since price hike started. How it is behaving in Ollama cloud PRO? Same? subm

Am I doing something wrong? Qwen 3.8 27B seems useless for agentic coding

I have been using local models on/off for like 2 years or so but never really used them extensively because the closed ones were always much better. Once Qwen 3.8 27B was released I decided to give it

AntLing’ve open-sourced 6 Base Model checkpoints for Ling-3.0-tiny & Ling-3.0-flash, covering pre-trained, mid-trained, and WSM-merged stages.

None has undergone post-training, giving researchers flexible starting points for continued pre-training, fine-tuning, and further research. Two key highlights: - They use WSM to replace LR decay with

Creating an account problems

Trying to make an account so I can start using Odyssey and to download some stuff through PowerShell. It requires a sign-in, but I can't make an account on the site because i need a U.S number and I d

DFlash2 speeds Qwen 3.8 27B up to 4 times

llama.cpp pr #27342 adds dflash2, so i rented an rtx 6000 and ran the same four prompts through four decoding setups on qwen3.8 27B median results over the four tasks: baseline 47.4 tok/s mtp 114.7 to

GLM-5.3 is out on AA, and I'm fed up with their Intelligence/cost plot

I think AA's intelligence/cost plot is seriously misleading, so I decided to make my own. Their plot is in the second image. All points are at max thinking. All intelligence index scores are from AA.

Has anyone else noticed DeepSeek-V4-Flash:0731 using way more Ollama Cloud usage lately?

I’ve been using deepseek-v4-flash:0731 on Ollama Cloud, and recently it feels like my usage is getting consumed much faster than before. My prompts and general workflow haven’t changed much, but the a

How much of the weight-space perception gap is actually symmetry? Evidence from ~1.8M fitted SIRENs [R]

model-releasesr-machinelearning

I’ve been looking at a fairly basic question in weight-space learning that I don’t think gets separated cleanly enough: Why does reading semantics directly from neural network weights work pretty well

I benchmarked Qwen3.8 27B on browser tasks. It's on par with GPT 5.6 Luna (xhigh)

The benchmark I used is BU bench v1, and the open source harness is Browser Agent. Qwen3.8 27B beat all other affordable or open models I tested. It's insanely good! submitted by /u/pierreb5 [link] [c

I built Polish – A local AI text polisher and Oracle SQL fixer for Windows with automated PII data masking

Hi everyone, I built an open-source Windows desktop utility called **Polish** that lets you highlight text or SQL in ANY app (Slack, Teams, Outlook, DBeaver, SQL Developer, browser) and rephrase or fi