K3 Usage Comparisons?
I'm confused by something I've noted in the recent discourse since Kimi K3 has been released on subscription. Has anyone done the (I know Ollama should display this) math as well on how much K3 is usi
Knowledge catalogue
I'm confused by something I've noted in the recent discourse since Kimi K3 has been released on subscription. Has anyone done the (I know Ollama should display this) math as well on how much K3 is usi
Hello gang, I made an implementation of DSpark PC Tree (Parent conditioned drafting tree). This is an implementation of this research paper: https://arxiv.org/abs/2608.02123 Unaffiliated, just found i
I still meet/talk to people that are so outdated in terms of LLM that when they talk about ChatGPT, it feels like they're talking about what ChatGPT was 2-3 years ago. They don't know all the capabili
https://preview.redd.it/stgzmbpgnnkh1.jpg?width=2179&format=pjpg&auto=webp&s=19a4fd0e1786cbcac0967e1bf1ec4898447e989a https://preview.redd.it/paub9gylnnkh1.png?width=680&format=png&auto=webp&s=c13efb5
I recently saw someone complaining that opencode causes compactions way too early (with 30k tokens still left). I fixed this issue and I thought I'd share my config to see if it helps anyone. I know p
https://www.reddit.com/r/LocalLLaMA/comments/1j7r47l/i_just_made_an_animation_of_a_ball_bouncing/ This post inspired me to make that test after a year ;) That is one of my many tests I make comparing
So usually I avoid Q3 quants because I have had bad experiences with it, models were usually too degraded, so the smallest I normally do is Q4, since I only have rtx 4060 ti 16gb. But since there hasn
Spent a while treating layer placement, KV format and llama.cpp itself as experimental variables. 159 logged experiments. Numbers first, caveats after. Hardware: AMD Ryzen AI MAX+ 395 (Strix Halo, 128
A quick feedback after a really major test: nearly 20 hours of non-stop goal-oriented work with Qwen3.8-27B Q6, running across an RTX 3090 and an RTX 3060. It maintained a speed of around 60–63 tokens
Benchmarks: Benchmark U1 Preview Full Qwen-Image-Bench 47.14 55.20 (PE) 60.18 (PE) ImgEdit 3.9 4.37 4.59 GEdit-Bench-EN 7.47 8.14 8.26 Instead of just scaling up, they train task-specialized expert mo
Hi fellows fully-local halos, after manually following existing guides, I decided to build an LLM API endpoint installation and optimization guide that works even when autonomously followed by my pi a
Hey locallama! We recently open sourced a Qwen3-TTS 1.7B implementation that achieves 10 requests per second (RPS) and sub-50 ms p95 time-to-first-audio (TTFA) while maintaining real-time playback on
Hi everyone! I’m an intern at Icosa, a startup focused on making local AI accessible. We just released the first version of Zeno, our local AI product, and I thought it might be of interest to this co
So what’s actually the best local AI harness rn? I’ve read a TON about this already and somehow ended up more confused than when I started so I figured screw it, let the community decide. Right now I
tldr: went from 16~ t/s to 27~ t/s generation. got my usable context up from 220k to the full 262k without sacrificing anything. prefill also increased from 376 to 573 command I ended up with, fwiw: l
A premise: I hope this question is 'worth' of this subreddit, I did a decent amount of research before posting, I thought it was potentially interesting enough for it, but possibly not basic enough fo
AirLLM dramatically reduces inference memory usage, letting 70B large language models run on a single 4GB GPU card — without quantization, distillation, or pruning. You can even run 405B Llama 3.1 on
Hi all, it is my very first post, because I am so confused. First my setup: I have 2 X 7900XTX with rocm7.2.1 and I use it to fine tune small models and local llm etc. I run Qwen 3.8 27b 8 bit version
AQuA's preprint uses 'recursive self-improvement' for a bounded research loop. It does not say the research-agent LM rewrites its own weights. The paper separates three objects: The language model dri
I'm introducing Aurora-80K, a small language model with exactly 80 thousand parameters. It uses a factorized 4,096-token vocabulary despite having only 80K parameters. The benchmarks: Wikitext-2 BPB:
https://omlx.ai/benchmarks/performance/2pko3m1k - you can expand the raw args, but I have full annotations of what worked and what didn't. I'm now testing the model on acutal coding and haven't seen a
Using Open-Webui I can connect to Ollama Cloud using ollama.com and the api key. It lists the models, it works. but i want to connect open-webui to the loacl ollama installation (windows) because all
IQ quants are particularly slow on CPU at large batch sizes (what you'd see for imatrix and perplexity) Benchmark numbers I ran PPL against master and this PR to get speed and numbers on --chunks 50 f
Hey LocalLlama, Henry from Cactus here! When we trained Needle 2, I had a strict rule to not expose the model to any data sample that remotely felt like these benchmarks. It seemed over-the-top, but b
I see https://github.com/linuxid10t/llama.cpp/tree/feature/g9v3-support but yeah... Considering that some results place it above Qwen 3.6 27B (the top performer until a few days ago) and that it is an
A majority of users in this sub use LLMs for coding/agentic tasks and I see why a lot of value is put into them but many try to say 'Well coding has improved therefore it can just use tool calling and
Howdy once again, I had a request to try out 5.3 on the benchmarks - they're unsaturated so it's a fun test right now! This one was interesting because I accidentally ran it on all 36 problems (rip 20
Tested on: X570DD_M570DD / GTX 1050 mobile / Arch Linux x86_64 / NVIDIA 580.178.04 driver / CUDA V12.9.86 What happened: As everyone knows, NVIDIA dropped the Pascal line in the 590.xxx.xx+ and CUDA 1
hey so im currently working to integrate a locally ran llm to control home assistant (through tool calling, no direct connection) so i can use a voice input without needing very specific commands. Its
I run local models on a 128GB Strix Halo and restart llama.cpp fairly often while testing builds, backends and model parameters. The annoying part is long-running agent sessions. Hermes/OpenCode sessi
I build LoRA Dataset Studio — free, open source, self-hosted, no account and no telemetry. It is not a competitor to ai-toolkit: it orchestrates it. ai-toolkit is the trainer; this is everything befor
I've been building AgentGraph Studio, a visual editor for designing multi-agent workflows and exporting them as runnable Python code. One thing I wanted to verify was whether the generated code actual
I pre-trained a 1.02-billion-parameter on Kimi K3 replica trained on 5.00 billion decontaminated tokens for $250. This model has 1.02 billion parameters, of which 145 million are active per token. It
Four days ago I released a hyper-optimized Qwen3.8-27B inference engine for an RTX 3090 (82 tps single request, 672 peak). Since then it went to ~114, then ~138 tps single-user with DFlash2 drafting a
I have a big database of local court decisions with a legal sentence which is like a paragraph summary of the doc. I've been tinkering with FTing for many days now, all results inconclusive never beat
AntLing has released the full six-checkpoint matrix for the Ling-3.0 base model. tiny: pretrained, mid-trained, WSM-merged flash: pretrained, mid-trained, WSM-merged The concrete artifact is six separ
Just got the setup dialed in yesterday. Getting really good results and is making 3.8 usage feel faster in hermes. I got Qwen to specifically use Ling tiny for simple tasks like context compression an
Links: Preprint: https://doi.org/10.5281/zenodo.22028087 Entropic Scree Function v1.0.0 / GitHub: https://github.com/tjleestjohn/Entropic-Scree TL;DR: Standard PCA fundamentally fractures non-linear d
Hey guys! Supra2-Medium is finally out! It's a 25M parameters qwen3 architecture model trained entirely from scratch (on our new rig: RTX 5060 Ti 16GB + the new RTX 5060 8GB!). Here's how it competes
I currently have these models available through Ollama/cloud: And I’m using Claude, Codex, OpenCode, and Ollama as my coding/agent tools. I want to learn professional vibecoding — not just asking AI t
The pelican on a bicycle is sooo outdated, so I came up with a new, improved version. Qwen3.8-27b medium (UD-Q4_K_XL) vs. Sol 5.6 high vs. Qwen3.6-35B (UD-Q6_K_XL) Prompt (only real with typo!): 'Crea
Not everyone has the disposable income to build a small data center, so making this post for the underdogs as I was very surprised by the performance/results of this 35B MOE model. Context is admitted
I had mistakenly been using a base for H3 prompting from some random tip / example by someone. It worked ok, I thought. But I was getting a bit frustrated because almost every time I was making a long
I'm testing it since release, now with UD 3.0 in my AMD R9700 with ROCm, I always read everywhere that F16 and q8_0 for KV cache are essentially the same... well, I tested it and I can see differences
Off a single prompt, given my credentials and the name of my university, qwen3.8-27b was able to successfully pull my class schedule from the kinda shitty and convoluted web of university websites. It
https://reddit.com/link/1vtkgdj/video/595yn0ckdjkh1/player I wanted to try out Qwen3.8 27B 's SVG capabilities but with something different than the pelican on a bicycle. Promt was literally just : cr
I benchmarked Qwen3.8-27B on MathArena/aime_2026 dataset, comparing BF16 and FP8 weights at medium and xhigh reasoning effort. Interesting findings are: quantized FP8 xhigh is better than BF 16 medium
Like many of you I've spent the last few days throwing Qwen3.8-27B against all of my usual use-cases and personal tasks/harnesses and workflows. It's great, phenomenal sometimes, but that's not what t
I chose to do this thing, not because it was hard, but because it was silly. Posts kept discussing how 3.8 and 3.6 were functionally the same, but based on training (3.8 does have seven new tokens!)..
this screenshot is without MTP usage , fully offloaded onto the GPU. using LM STUDIO i wanted to ask you guys if theres a way to make it even faster , as MTP really didnt help and is infact slower due
This is gonna seem crazy off-topic, but I saw Spider-Man the other day and couldn’t help but notice how well executed E.V. is as an agentic system. compared to Jarvis, it felt much more grounded and c
They released both GGUFs & custom llama.cpp fork today. 35B MOE in 7GB size which's good for Mobile & Edge devices(Also low memory systems). Up to 120 t/s on Consumer Laptop. GGUFs: https://huggingfac
From the screenshots: Hy4 is now live, labeled 'Expert-Level Model' + 'Use Tools to Solve Problems' Hy3 is tagged with 'New Upgrade,' positioned as a brand-new general-purpose model DeepSeek, focused
Component Validated configuration Motherboard ASRock Rack SPC621D8U-2T/OVH CPU Xeon Gold 6330 (Get gold/platinum if interested in Optane Pmem gimmicks) GPU fabric Two Broadcom/PLX PEX88096 islands, ei
Worked some time ago on one of the ad teams at Yahoo, and this grew out of a question I kept returning to while there are there 'simple' models that are both simple, scalable, interpretable, and contr
Hey everyone, Posted TinySearch here a few versions ago and got a bunch of useful feedback, so figured I'd post an update because the thing has changed quite a bit since then. Repo: [https://github.co
https://preview.redd.it/xel6i6kujjkh1.png?width=2396&format=png&auto=webp&s=b78cd87da266324364e6b8cb8457a7f0e029d163 https://preview.redd.it/ofy8ttyxjjkh1.png?width=2396&format=png&auto=webp&s=3e64405
Hello, any params I missing or mis-tuning for my coding model ? FROM ./models/unsloth/Qwen3.8-27B-Q5_K_M.gguf # Generation parameters optimized for logical reasoning and coding tasks PARAMETER tempera
seems like others are now using lower precision models. On open code sub i see a lot of people saying model has gone dumber since price hike started. How it is behaving in Ollama cloud PRO? Same? subm
I have been using local models on/off for like 2 years or so but never really used them extensively because the closed ones were always much better. Once Qwen 3.8 27B was released I decided to give it