Search: “model-releases”

Hey locallama! We recently open sourced a Qwen3-TTS 1.7B implementation that achieves 10 requests per second (RPS) and sub-50 ms p95 time-to-first-audio (TTFA) while maintaining real-time playback on

Hi everyone! I’m an intern at Icosa, a startup focused on making local AI accessible. We just released the first version of Zeno, our local AI product, and I thought it might be of interest to this co

So what’s actually the best local AI harness rn? I’ve read a TON about this already and somehow ended up more confused than when I started so I figured screw it, let the community decide. Right now I

tldr: went from 16~ t/s to 27~ t/s generation. got my usable context up from 220k to the full 262k without sacrificing anything. prefill also increased from 376 to 573 command I ended up with, fwiw: l

AirLLM dramatically reduces inference memory usage, letting 70B large language models run on a single 4GB GPU card — without quantization, distillation, or pruning. You can even run 405B Llama 3.1 on

Hi all, it is my very first post, because I am so confused. First my setup: I have 2 X 7900XTX with rocm7.2.1 and I use it to fine tune small models and local llm etc. I run Qwen 3.8 27b 8 bit version

https://omlx.ai/benchmarks/performance/2pko3m1k - you can expand the raw args, but I have full annotations of what worked and what didn't. I'm now testing the model on acutal coding and haven't seen a

IQ quants are particularly slow on CPU at large batch sizes (what you'd see for imatrix and perplexity) Benchmark numbers I ran PPL against master and this PR to get speed and numbers on --chunks 50 f

Hey LocalLlama, Henry from Cactus here! When we trained Needle 2, I had a strict rule to not expose the model to any data sample that remotely felt like these benchmarks. It seemed over-the-top, but b

I see https://github.com/linuxid10t/llama.cpp/tree/feature/g9v3-support but yeah... Considering that some results place it above Qwen 3.6 27B (the top performer until a few days ago) and that it is an

A majority of users in this sub use LLMs for coding/agentic tasks and I see why a lot of value is put into them but many try to say 'Well coding has improved therefore it can just use tool calling and

Howdy once again, I had a request to try out 5.3 on the benchmarks - they're unsaturated so it's a fun test right now! This one was interesting because I accidentally ran it on all 36 problems (rip 20

I run local models on a 128GB Strix Halo and restart llama.cpp fairly often while testing builds, backends and model parameters. The annoying part is long-running agent sessions. Hermes/OpenCode sessi

I've been building AgentGraph Studio, a visual editor for designing multi-agent workflows and exporting them as runnable Python code. One thing I wanted to verify was whether the generated code actual

I pre-trained a 1.02-billion-parameter on Kimi K3 replica trained on 5.00 billion decontaminated tokens for $250. This model has 1.02 billion parameters, of which 145 million are active per token. It

Four days ago I released a hyper-optimized Qwen3.8-27B inference engine for an RTX 3090 (82 tps single request, 672 peak). Since then it went to ~114, then ~138 tps single-user with DFlash2 drafting a

I have a big database of local court decisions with a legal sentence which is like a paragraph summary of the doc. I've been tinkering with FTing for many days now, all results inconclusive never beat

AntLing has released the full six-checkpoint matrix for the Ling-3.0 base model. tiny: pretrained, mid-trained, WSM-merged flash: pretrained, mid-trained, WSM-merged The concrete artifact is six separ

Just got the setup dialed in yesterday. Getting really good results and is making 3.8 usage feel faster in hermes. I got Qwen to specifically use Ling tiny for simple tasks like context compression an

I currently have these models available through Ollama/cloud: And I’m using Claude, Codex, OpenCode, and Ollama as my coding/agent tools. I want to learn professional vibecoding — not just asking AI t

The pelican on a bicycle is sooo outdated, so I came up with a new, improved version. Qwen3.8-27b medium (UD-Q4_K_XL) vs. Sol 5.6 high vs. Qwen3.6-35B (UD-Q6_K_XL) Prompt (only real with typo!): 'Crea

Not everyone has the disposable income to build a small data center, so making this post for the underdogs as I was very surprised by the performance/results of this 35B MOE model. Context is admitted

I'm testing it since release, now with UD 3.0 in my AMD R9700 with ROCm, I always read everywhere that F16 and q8_0 for KV cache are essentially the same... well, I tested it and I can see differences

https://reddit.com/link/1vtkgdj/video/595yn0ckdjkh1/player I wanted to try out Qwen3.8 27B 's SVG capabilities but with something different than the pelican on a bicycle. Promt was literally just : cr

I benchmarked Qwen3.8-27B on MathArena/aime_2026 dataset, comparing BF16 and FP8 weights at medium and xhigh reasoning effort. Interesting findings are: quantized FP8 xhigh is better than BF 16 medium

this screenshot is without MTP usage , fully offloaded onto the GPU. using LM STUDIO i wanted to ask you guys if theres a way to make it even faster , as MTP really didnt help and is infact slower due

They released both GGUFs & custom llama.cpp fork today. 35B MOE in 7GB size which's good for Mobile & Edge devices(Also low memory systems). Up to 120 t/s on Consumer Laptop. GGUFs: https://huggingfac

From the screenshots: Hy4 is now live, labeled 'Expert-Level Model' + 'Use Tools to Solve Problems' Hy3 is tagged with 'New Upgrade,' positioned as a brand-new general-purpose model DeepSeek, focused

Component Validated configuration Motherboard ASRock Rack SPC621D8U-2T/OVH CPU Xeon Gold 6330 (Get gold/platinum if interested in Optane Pmem gimmicks) GPU fabric Two Broadcom/PLX PEX88096 islands, ei

Hey everyone, Posted TinySearch here a few versions ago and got a bunch of useful feedback, so figured I'd post an update because the thing has changed quite a bit since then. Repo: [https://github.co

Hello, any params I missing or mis-tuning for my coding model ? FROM ./models/unsloth/Qwen3.8-27B-Q5_K_M.gguf # Generation parameters optimized for logical reasoning and coding tasks PARAMETER tempera

seems like others are now using lower precision models. On open code sub i see a lot of people saying model has gone dumber since price hike started. How it is behaving in Ollama cloud PRO? Same? subm

I have been using local models on/off for like 2 years or so but never really used them extensively because the closed ones were always much better. Once Qwen 3.8 27B was released I decided to give it

llama.cpp pr #27342 adds dflash2, so i rented an rtx 6000 and ran the same four prompts through four decoding setups on qwen3.8 27B median results over the four tasks: baseline 47.4 tok/s mtp 114.7 to

I think AA's intelligence/cost plot is seriously misleading, so I decided to make my own. Their plot is in the second image. All points are at max thinking. All intelligence index scores are from AA.

I’ve been using deepseek-v4-flash:0731 on Ollama Cloud, and recently it feels like my usage is getting consumed much faster than before. My prompts and general workflow haven’t changed much, but the a

I’ve been looking at a fairly basic question in weight-space learning that I don’t think gets separated cleanly enough: Why does reading semantics directly from neural network weights work pretty well

The benchmark I used is BU bench v1, and the open source harness is Browser Agent. Qwen3.8 27B beat all other affordable or open models I tested. It's insanely good! submitted by /u/pierreb5 [link] [c

Hello everyone, tried so hard to optimize my config and finally I simply get up to 70 t/s with q6 variant. And wanted to share with you guys so that other people with the same setup can enjoy. Please

Edit: Title says 134 tps, it's actually 138 -- keep in mind my 3090 is power limited to 250w. Three days ago I released a hyper-optimized Qwen3.8-27B inference engine for an RTX 3090 (82 tps single re

Hi folks, I have a 7800x3d + 32GB RAM + Rtx4090 and > 200TB of storage. I downloaded the Qwen 3.7 35B hauhaucs but it has been a disappointment so far because training data is too outdated (networking

https://huggingface.co/inclusionAI/Ling-3.0-tiny-base https://huggingface.co/inclusionAI/Ling-3.0-flash-base-midtrain https://huggingface.co/inclusionAI/Ling-3.0-flash-base-30T https://huggingface.co/

I have been searching for suitable model to run on my 8GB RAM toy, NVIDIA Orin Nano Super 8GB. This little toy was priced at 249 earlier this year (not any more), and pulls very little power when idle

Four Tesla V100s from 2017 matched my RTX 5090 on single-request Qwen 3.8 decode. Repo: https://github.com/dnv2003/v100-skinny https://i.redd.it/5ws2ak3uqckh1.gif The 5090 was not being held back. It

PR by u/Stainless-Bacon 👍 It would be handy & awesome to have options --n-cpu-ffn / --cpu-ffn for Dense models like how we have --n-cpu-moe / --cpu-moe for MOE models. Also check his threads: On PR :

Aloha! 🌺Introducing Ornith-1.5, a family of open-source LLMs spanning 9B Dense, 35B MoE, and 397B MoE, trained with self-improving strategies. It achieves state-of-the-art performance among open-sourc