All entries

I'm really hoping we're in 2026's 2-month-gap between QwQ and Qwen3 right now

QwQ was genuine next-gen performance usable on local hardware, but the massive required context (it's reasoning style was akin to 'if I say every possible word, I'll notice the right one!') kinda made

4,816

Source

Human DGX agent

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Is there any interest in a self hosted open source version of manus/perplexity?

DGX agent

I have been working on a project for about 3 months because I found Vane to be awful and I wanted something like Manus for myself. I have a pretty solid app that's probably about ready to release, but

model-releasesr-localllama

21 Aug 2026

K3 Usage Comparisons?

I'm confused by something I've noted in the recent discourse since Kimi K3 has been released on subscription. Has anyone done the (I know Ollama should display this) math as well on how much K3 is usi

Llama.cpp DSpark PC Tree Fork (up to 3%-29.5% faster!)

Hello gang, I made an implementation of DSpark PC Tree (Parent conditioned drafting tree). This is an implementation of this research paper: https://arxiv.org/abs/2608.02123 Unaffiliated, just found i

Industry

LLMs are so advanced right now that it's jarring that a lot of people think ChatGPT is just a mere chatbot.

I still meet/talk to people that are so outdated in terms of LLM that when they talk about ChatGPT, it feels like they're talking about what ChatGPT was 2-3 years ago. They don't know all the capabili

industryr-chatgpt

New stealth model Ox Alpha look like glm or mimo new model

https://preview.redd.it/stgzmbpgnnkh1.jpg?width=2179&format=pjpg&auto=webp&s=19a4fd0e1786cbcac0967e1bf1ec4898447e989a https://preview.redd.it/paub9gylnnkh1.png?width=680&format=png&auto=webp&s=c13efb5

PSA: Opencode Early Compaction

I recently saw someone complaining that opencode causes compactions way too early (with 30k tokens still left). I fixed this issue and I thought I'd share my config to see if it helps anyone. I know p

Qwen 3.8 27b - PI AGENT vs OPENCODE

https://www.reddit.com/r/LocalLLaMA/comments/1j7r47l/i_just_made_an_animation_of_a_ball_bouncing/ This post inspired me to make that test after a year ;) That is one of my many tests I make comparing

Qwen 3.8 27b is strong even at Q3_xxs

So usually I avoid Q3 quants because I have had bad experiences with it, models were usually too degraded, so the smallest I normally do is Q4, since I only have rtx 4060 ti 16gb. But since there hasn

Qwen3.8-27B at 262K context on a Strix Halo + RTX 3090 Ti: 9.5 -> 153 tok/s, and it beats a dual-3090 vLLM box on HumanEval

Spent a while treating layer placement, KV format and llama.cpp itself as experimental variables. 159 logged experiments. Numbers first, caveats after. Hardware: AMD Ryzen AI MAX+ 395 (Strix Halo, 128

Agents

Qwen3.8-27B Q6 is a beast at agentic coding

A quick feedback after a really major test: nearly 20 hours of non-stop goal-oriented work with Qwen3.8-27B Q6, running across an RTX 3090 and an RTX 3060. It maintained a speed of around 60–63 tokens

agentsr-localllama

SenseNova U1.5-Lite full release: expert training, OPD distillation, one model at inference

Benchmarks: Benchmark U1 Preview Full Qwen-Image-Bench 47.14 55.20 (PE) 60.18 (PE) ImgEdit 3.9 4.37 4.59 GEdit-Bench-EN 7.47 8.14 8.26 Instead of just scaling up, they train task-specialized expert mo

Strix Halo (8060S / gfx1151), Qwen-3.8-27B @ Q8 and Q6 UD v3, up to 256K ctx, llama.cpp, DFlash2, vision, real workloads quality and steady performances, optimized recipes, ...

Hi fellows fully-local halos, after manually following existing guides, I decided to build an LLM API endpoint installation and optimization guide that works even when autonomously followed by my pi a

Ultrafast Qwen3-TTS at 34 ms Time-to-First-Audio, Handling 10 Requests Per Second [OSS]

Hey locallama! We recently open sourced a Qwen3-TTS 1.7B implementation that achieves 10 requests per second (RPS) and sub-50 ms p95 time-to-first-audio (TTFA) while maintaining real-time playback on

We built a local AI work tool that runs Qwen3.5-35B-A3B on a 16GB Mac

Hi everyone! I’m an intern at Icosa, a startup focused on making local AI accessible. We just released the first version of Zeno, our local AI product, and I thought it might be of interest to this co

What’s the best local AI harness for coding + general use?

So what’s actually the best local AI harness rn? I’ve read a TON about this already and somehow ended up more confused than when I started so I figured screw it, let the community decide. Right now I

3 days benchmarking most llama.cpp flags on my weird 40gb vram laptop + tb4 egpu setup. Got +70% generation, +40% prefill, 60k more context, and filed a bug in llama around MTP. What I learned.

tldr: went from 16~ t/s to 27~ t/s generation. got my usable context up from 220k to the full 262k without sacrificing anything. prefill also increased from 376 to 573 command I ended up with, fwiw: l

Tutorials

About the impact of grouping classes in multiclass classification [D]

tutorialsr-machinelearning

A premise: I hope this question is 'worth' of this subreddit, I did a decent amount of research before posting, I thought it was potentially interesting enough for it, but possibly not basic enough fo

AirLLM - Recent Updates - with Qwen3.8-27B, Kimi-K3 too

AirLLM dramatically reduces inference memory usage, letting 70B large language models run on a single 4GB GPU card — without quantization, distillation, or pruning. You can even run 405B Llama 3.1 on

AMD setup is fast with 200k ctx with Qwen 3.8, i didn't understand how/why?

Hi all, it is my very first post, because I am so confused. First my setup: I have 2 X 7900XTX with rocm7.2.1 and I use it to fine tune small models and local llm etc. I run Qwen 3.8 27b 8 bit version

AQuA's 'self-improvement' updates research state, not the agent LM. What should a local port freeze?

AQuA's preprint uses 'recursive self-improvement' for a bounded research loop. It does not say the research-agent LM rewrites its own weights. The paper separates three objects: The language model dri

Aurora-80K releases! A modern tiny language model.

I'm introducing Aurora-80K, a small language model with exactly 80 thousand parameters. It uses a factorized 4,096-token vocabulary despite having only 80K parameters. The benchmarks: Wikitext-2 BPB:

Been tweaking my Qwen 3.8 setup, up to 45+ steady T/ps at 8bit quant. Realised I'm now top T/ps for this model+ctx across all benchmarked M-series chips. Full args linked below, happy to discuss as this was a pain of trial and error.

https://omlx.ai/benchmarks/performance/2pko3m1k - you can expand the raw args, but I have full annotations of what worked and what didn't. I'm now testing the model on acutal coding and haven't seen a

Connect Open-Webui to local Ollama does not show Ollama Cloud Models

Using Open-Webui I can connect to Ollama Cloud using ollama.com and the api key. It lists the models, it works. but i want to connect open-webui to the loacl ollama installation (windows) because all

local-air-ollama

[Draft - Open PR] AVX2: Speed up large batch size prompt processing of IQ models by bartowski1182 · Pull Request #27402 · ggml-org/llama.cpp

IQ quants are particularly slow on CPU at large batch sizes (what you'd see for imatrix and perplexity) Benchmark numbers I ran PPL against master and this PR to get speed and numbers on --chunks 50 f

Fine-tuning Cactus Needle 2 can match DeepSeek v4 on the specific task

Hey LocalLlama, Henry from Cactus here! When we trained Needle 2, I had a strict rule to not expose the model to any data sample that remotely felt like these benchmarks. It seemed over-the-top, but b

G9v3-39A5B on artificialanalysis looks good. Has anyone tested it?

I see https://github.com/linuxid10t/llama.cpp/tree/feature/g9v3-support but yeah... Considering that some results place it above Qwen 3.6 27B (the top performer until a few days ago) and that it is an

Getting better at coding doesn't make a model better at everything else

A majority of users in this sub use LLMs for coding/agentic tasks and I see why a lot of value is put into them but many try to say 'Well coding has improved therefore it can just use tool calling and

GLM 5.3 SlopCodeBench Results

Howdy once again, I had a request to try out 5.3 on the benchmarks - they're unsaturated so it's a fun test right now! This one was interesting because I accidentally ran it on all 36 problems (rip 20

[GUIDE] Running CUDA 12.9 on NVIDIA Pascal (GTX 10xx) with Arch Linux

Tested on: X570DD_M570DD / GTX 1050 mobile / Arch Linux x86_64 / NVIDIA 580.178.04 driver / CUDA V12.9.86 What happened: As everyone knows, NVIDIA dropped the Pascal line in the 590.xxx.xx+ and CUDA 1

local-air-ollama

home lab assistant llm

hey so im currently working to integrate a locally ran llm to control home assistant (through tool calling, no direct connection) so i can use a voice input without needing very specific commands. Its

local-air-ollama

How do you deal with long-context sessions after restarting llama.cpp?

I run local models on a 128GB Strix Halo and restart llama.cpp fairly often while testing builds, backends and model parameters. The annoying part is long-running agent sessions. Hermes/OpenCode sessi

I built a free, self-hosted app that does everything around a LoRA run — dataset, triage, captions, training (local or rented GPU), then checkpoint comparison

local-air-stablediffusion

I build LoRA Dataset Studio — free, open source, self-hosted, no account and no telemetry. It is not a competitor to ai-toolkit: it orchestrates it. ai-toolkit is the trainer; this is everything befor

I built a visual multi-agent workflow editor that exports runnable CrewAI code — here’s a 5-agent workflow running locally with Ollama

I've been building AgentGraph Studio, a visual editor for designing multi-agent workflows and exporting them as runnable Python code. One thing I wanted to verify was whether the generated code actual

I just built a mini Kimi-K3 from Scratch under 250$. Already beats GPT-2 (124M)!

I pre-trained a 1.02-billion-parameter on Kimi K3 replica trained on 5.00 billion decontaminated tokens for $250. This model has 1.02 billion parameters, of which 145 million are active per token. It

I pushed Qwen3.8-27B to 381 tps for a single request on a RTX 3090

Four days ago I released a hyper-optimized Qwen3.8-27B inference engine for an RTX 3090 (82 tps single request, 672 peak). Since then it went to ~114, then ~138 tps single-user with DFlash2 drafting a

Is it possible to fine-tune gemma4 A4B to generate complex legal principles of court decision? [D]

model-releasesr-machinelearning

I have a big database of local court decisions with a legal sentence which is like a paragraph summary of the doc. I've been tinkering with FTing for many days now, all results inconclusive never beat

Ling-3.0 released all 6 base checkpoints: 2 sizes × 3 stages

AntLing has released the full six-checkpoint matrix for the Ling-3.0 base model. tiny: pretrained, mid-trained, WSM-merged flash: pretrained, mid-trained, WSM-merged The concrete artifact is six separ

Ling 3.0 Tiny makes an amazing auxillery model for Hermes (Qwen 3.8 27B as the primary model)

Just got the setup dialed in yesterday. Getting really good results and is making 3.8 usage feel faster in hermes. I got Qwen to specifically use Ling tiny for simple tasks like context compression an

Applications

Mapping intrinsic rank and informational gravity in complex tabular data: I developed a non-parametric, model-agnostic, information-theoretic diagnostic to bypass the limits of linear, rank, and Euclidean baselines. [R]

applicationsr-machinelearning

Links: Preprint: https://doi.org/10.5281/zenodo.22028087 Entropic Scree Function v1.0.0 / GitHub: https://github.com/tjleestjohn/Entropic-Scree TL;DR: Standard PCA fundamentally fractures non-linear d

[MASSIVE TINY RELEASE] - Supra2-Medium-Base - a tiny 25M parameters model competing heavily with our previous 50M model!

Hey guys! Supra2-Medium is finally out! It's a 25M parameters qwen3 architecture model trained entirely from scratch (on our new rig: RTX 5060 Ti 16GB + the new RTX 5060 8GB!). Here's how it competes

Need help choosing the right AI model/tool for a complete web app workflow

I currently have these models available through Ollama/cloud: And I’m using Claude, Codex, OpenCode, and Ollama as my coding/agent tools. I want to learn professional vibecoding — not just asking AI t

New benchmark just dropped!

The pelican on a bicycle is sooo outdated, so I came up with a new, improved version. Qwen3.8-27b medium (UD-Q4_K_XL) vs. Sol 5.6 high vs. Qwen3.6-35B (UD-Q6_K_XL) Prompt (only real with typo!): 'Crea

Ornith-1.5-35B-A3B Q4 running 60tk/s on 4070Ti.

Not everyone has the disposable income to build a small data center, so making this post for the underdogs as I was very surprised by the performance/results of this 35B MOE model. Context is admitted

Tutorials

PSA: Proper prompt structure REALLY matters in H3

tutorialsr-stablediffusion

I had mistakenly been using a base for H3 prompting from some random tip / example by someone. It worked ok, I thought. But I was getting a bit frustrated because almost every time I was making a long

Qwen 3.8 27B KV f16 vs q8_0 are not equivalents

I'm testing it since release, now with UD 3.0 in my AMD R9700 with ROCm, I always read everywhere that F16 and q8_0 for KV cache are essentially the same... well, I tested it and I can see differences

Qwen3.8-27b has the highest level of 'agency' I've ever seen in a local model

Off a single prompt, given my credentials and the name of my university, qwen3.8-27b was able to successfully pull my class schedule from the kinda shitty and convoluted web of university websites. It

Qwen3.8 27b just exceeded my expectations on svg generation :D

https://reddit.com/link/1vtkgdj/video/595yn0ckdjkh1/player I wanted to try out Qwen3.8 27B 's SVG capabilities but with something different than the pelican on a bicycle. Promt was literally just : cr