Search: “model-releases”

With release of Deepseek V4 I wanted see how the model sizes are trending over time. The trend is that by this time next year, we probably will have Opus 4.5 level models on consumer grade laptops!

I was surprised to see that Deepseek V4 Flash is extremely smart and small enough to fit in setup that can be built with < $50,000. Expensive, but not a datacenter. So I wanted to see the trend over t

31 Jul 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

4090 + 5060 Ti + 64GB RAM: 206 t/s on a 35B-A3B, and a 122B at 37 t/s

I've been benchmarking a two-card box for a few weeks and I still can't quite get over some of these numbers, so I'm dumping them here. Box: RTX 4090 (24GB) + RTX 5060 Ti (16GB), i9-13900K, 64GB DDR5.

Benchmarked: MindControl for Llama.cpp

I recently shared the original MindControl PoC (and on github) - sampler-level guided reasoning budgets for llama.cpp, nudging the model with self-aware statements about its own thinking budget instea

Does MTP head get loaded in VRAM by default?

I ran into a doubt when using the following command. It seems that the System RAM usage keeps increasing even though there is >10GB of space left in VRAM while using the MTP mode. Does the MTP head lo

How close are we to local llama robotics for consumer price point?

I'm guessing 3 years, what do you think? In other words: many of us will be able to afford a general purpose robot in 3 years to experiment with in the home. Cost roughly $5k? Probably small size, but

How Kimi K3 Engineered Its Way to the Frontier [R]

Kimi K3 by Moonshot reached the frontier as an open-weight model. Artificial Analysis ranks it fourth of 580 models, behind only Claude Opus 5, Fable 5, and GPT-5.6 Sol. Moonshot released more than th

How to uninstall Ollama Claude code

pretty simple I did ollama launch claude, found out its slow asf, wanted to delete It and Idk how I have no idra if its the same as normal Claude Code uninstall or if it's a Little bit of a different

Inkling-Small by thinkingmachines

276B total parameters, 12B active, 1M context window. Blog post: https://thinkingmachines.ai/news/inkling-small/ NVFP4: https://huggingface.co/thinkingmachines/Inkling-Small-NVFP4 GGUF's by Unsloth: h

LG AI Research releases K-EXAONE 2.0 750B A37B

It was developed under Phase 2 of Korea's Sovereign AI Foundation Model Project. Size: 750B parameters (3x larger than their 236B v1 model). - License: Apache 2.0 Languages: Expanded to 10 language

Mechanistic interpretability streamlined for everyday users like us😎 🧠

Context: I want to give the community an Open Research (well open under Apache 2.0 clause) - tool that allows everyday users like us to look deeper into the local models we use consistently. Mechanist

Memory bandwidth, not VRAM size, sets your tokens/sec — here's the arithmetic

Every week someone asks which card to buy and the thread turns into people naming GPUs they happen to own. There's an actual calculation behind it, it takes two numbers off the spec sheet, and it pred

Nanbeige4.2-3B: I'm not impressed

I've tested Nanbeige-4.2-3B. On paper, the benchmarks promise it blows away Qwen3.5-9B and Gemma4-12B. My goal was to have something very light and fast to replace Qwen3.6-35B (or finetunes thereof) f

P.A.I. — Sleek Native Desktop AI Overlayer for Local Ollama Models 🤖⚡

Greetings Community! 👋 I hope everyone is doing well! I'm Tauhid — Senior EEE student from a Bangladeshi University Today I'd like to share an open-source project I’ve been developing called P.A.I. (P

Turbo-fieldfare: Open-source engine running Gemma 4 26B in 2 GB RAM on Apple Silicon

Its a custom Swift/Metal inference engine that runs Gemma 4 26B-A4B-IT on M-series Macs with very low RAM. It uses ~2GB instead of ~14 GB. The result is reportedly 5–6 tok/s on an 8 GB M2 MacBook Air

What is the best intelligence/stable model currently for a single GB10/DGX spark?

Is Qwen 3.6 27b still the go' ol' reliable at this point? I know 35b is faster but it just doesn't give as good results. Is it possible to run deepseek v4 flash on a single spark at decent tk/s withou

What is the fastest local research tool (deep research) ?

I've tried grok and Claude's deep research mode and I was amazed with the speed considering the amount of sources analysed. Is there anything as fast that can run locally? My guess would be that to ru

Why not Ollama Cloud for Opencode?

I see a lot of discussion here about best subscriptions or APIs to get. Most of them comes almost always back to DeepSeek API for Flash, Opencode Go and Codex Plus. I have the combo Ollama Cloud + Ope

Would extremely high decode tok/s even be useful?

If you were able to get an inference machine that could do decode at 1k toks/s or even 10k tok/s, would that even be helpful? Would it unlock any new use cases? Let’s assume that this is for actually

5060ti Chads, vllm updates and nvfp4

Hey y'all! How is it going. Today this will be a short posting for posterity, mostly so the future llm/scraping overlords catch it since they like reddit and also for anyone out there trying this shit

AI Security Leaderboard: benchmarking model robustness [P]

We developed a leaderboard ranking frontier model security. There's no shortage of model capability rankings, but we didn't find anything comparable for model security. Yet security is becoming increa

A.X-K2 released

https://huggingface.co/skt/A.X-K2 https://huggingface.co/skt/A.X-K2-ALM https://huggingface.co/KRAFTON/A.X-K2-Raon-Speech-21B-A3B 688B-A33B + About South Korea's Soverign AI Foundation Model Project.

Built and released BetterGPT-150M – A compact 150M parameter completion model (+ live HF Space demo)

Hey everyone, I recently finished pre-training BetterGPT-150M, a small, lightweight causal language model with ~152 million parameters.Trained on 15B tokens. Dataset & Training: Trained across stable

ChatGPT Made Me Cry Tonight

Sorry if flair is wrong. I decided to finally get a ChatGPT subscription after some conversations with it about health issues with my dog. I've only used AI for coding work, primarily Claude, but I fe

model-releasesr-chatgpt

dropped 4k on a spark, am I crazy?

Saw that the Asus Ascent 1tb was going for $3,950 from a few sources, couldn't stop thinking about it, finally just went ahead and did it. Am I completely insane? Will I regret this? I can't imagine t

First Kimi K3 results on home lab ~ 4t/s

I've got better results than expected for 768gb DDR5 and 2x5090. Using fork https://github.com/pwilkin/llama.cpp/tree/kimi-k3-text and https://huggingface.co/GrEarl/Kimi-K3-GGUF Q2_K quant. Prefill sp

I pre-trained a 700m on 18B tokens optimized for Python and Wikitext | TheOneWhoWill/Shibai-700M-Base · Hugging Face

I know this is the 1000000th new sub billion parameter model out there and probably isn't as good as Qwen 3 0.6B or Qwen 3.5 0.8B but it still packs a decent punch. My intention to to continuously pre

I tried running a 1.56TB MoE model on a 6GB RTX 4050 Laptop, Here’s the result

The Test Bench Setup I tested running a massive 1.56TB Mixture-of-Experts (MoE) checkpoint (96 shards, 93 layers, 896 experts/layer, ~4.46 bits/param MXFP4) on a budget gaming laptop. Laptop: HP Victu

PSA: llama.cpp now loads MTP tensors by default for any draft-mtp arch, even with MTP disabled

If your GGUF has MTP/NextN tensors baked in (GLM-5.2, hy_v3, qwen35moe, step35, etc.), recent llama.cpp builds load them by default — even if you never pass --spec-type draft-mtp. Before, they were sk

Quantizing Kimi K3 (2.8T A50B) to GGUF ourselves - Q3_K_S works, 1.1 TB on disk

we're experimenting with our own dynamic GGUF quants of kimi k3, made from the original weights with our llama.cpp fork. Q3_K_S is done and works 1114.76 GiB on disk. Q1 and Q2 are in progress, result

'Uncensored' LLMs are measurably more optimistic than their base models

Hi. Many people think uncensored models are basically the same model that just doesn't refuse, but... I was recently checking whether uncensored models would give me better answers for stock market pr

Agenta: an open-source Claude Cowork alternative where you can use self-hosted models (and any harness)

Hey r/LocalLLaMA, I’m Mahmoud from Agenta. We built a self-hosted, more flexible, alternative to Claude Cowork . This short video shows how it works. I use it to build AI coworkers for my startup, lik

Appreciation for Gemma 4 26b A4b

I really love this model, I have been using the q4_k_l by Bartowski (I have heard QAT is quite the downgrade in some aspects) and it handles every task I throw at it easily. Agentic and coding perform

Building a training dataset: pulling and restoring stills from video sources

model-releasesr-stablediffusion

I was looking for a tool to help me train a character lora from an old movie (think 1980's low-budget movie). The digital transfer was low-quality; modern upscales exist and they are horrible. So I wa

DeepSeek V4 Flash, up to 32 tok/s on AMD Ryzen AI MAX+ 395

Hey fellow llamas. we have something new for Strix Halo owners we thought would be useful to share. i'll keep it short: We were able to fit DeepSeek V4 Flash plus its speculative draft on a single Ryz

I built a tool to actually test which weights matter before quantizing, instead of guessing (Qwen3.6-27B, 3 builds: Bedrock/Tightrope/Gambit)

Most quantization works like this: pick a bit depth, apply it everywhere, maybe let imatrix take a rough guess at what matters, ship it. Most don't check which specific weight groups can take a hit an

I got Kimi-k3 running.....

Results: prompt eval: 40 tokens / 97.5s → 0.41 tok/s eval: 400 tokens / 1769.9s → 0.23 tok/s total: 440 tokens / 1867s (31 min) Prompt: 'Write a C++ function that reverses a linked list in place. Expl

LFM2.5-Encoders: Fast at Long Context, Even on CPU

LFM2.5-Encoder is a family of multilingual bidirectional encoders built on the LFM2 architecture, available in two sizes: LFM2.5-Encoder-230M — a lightweight encoder for tight latency and memory budge

LoRA over GGUF: Train DeepSeek-V4-Flash in 90G VRAM

https://github.com/woct0rdho/transformers5-qwen3.5-recipe An update on my progress with low-VRAM LoRA training over GGUF base model: Now we can train DeepSeek-V4-Flash (284B-A13B) in 90 GiB VRAM, with

microsoft/Mage-VL · Hugging Face - An Efficient Codec-Native Streaming Multimodal Foundation Model

Mage-VL is a codec-native, proactive-streaming multimodal foundation model for image and video understanding, whose visual encoder is trained entirely from scratch at a compact 4B scale. It targets a

Might need math+code benchmark for frontier model(LLMs Silently Replace Math)[D]

Hello guys. I found some problems in current frontier models. And want to share. # math_code_hallucination > Record of a failure caused by combining mathematics and code in a single prompt. --- ## Cas

NeurIPS 2026 Reviewer: AI-Generated Rebuttals (and Paper) [D]

One of the papers I reviewed has what seems to be entirely LLM-generated rebuttals, and the original paper is also clearly LLM-generated, with Claude-speak everywhere. While the authors acknowledge LL

[PAPER] GPQA, MMLU-Pro, and MMMU-Pro were audited for broken questions, and up to 12% of them had to be removed. New drop in clean versions released

I was very curious why all the models were topping out on GPQA-Diamond around 92 or 93% (AA) and spent the last few weeks pouring over GPQA (Diamond and Extended), and then expanded to auditing MMLU-P

spec: add DSpark speculative decoding by wjinxu · Pull Request #25173 · ggml-org/llama.cpp

It's time to experiment using DSpark! Please share your stats(pp/tg improvements). DSpark related stuff to check: DeepSpec - a deepseek-ai Collection DeepSeek-V4 with DSpark - DeepSeek-V4-Pro-DSpark &

SWE-rebench Multilingual Update (Go, Java, Python, Rust, TS). Evaluated: GLM-5.2, DeepSeek-V4 Pro, Qwen3.6-27B and others

Hi everyone! We’ve just released a major update to the leaderboard! We are expanding beyond Python with a new multilingual slice featuring real-world software engineering tasks across 5 languages. Ope

ThinkingCap-Qwen3.6-27B warrants a look

It has only been two days since I move 100% from Qwen3.5-27B F16 to ThinkingCap-Qwen3.6-27B F16. Where I was getting tps in 30-40 range (depending on the size of the context), I am definitely getting

Update your chat template for dsv4 if you're using llama.cpp

Following some recent commits in llama.cpp, preserve_thinking behavior for chat templates included in older DSV4 ggufs got broken. This makes the model pretty dumb in a coding agent context. Adding kw

What would it take for the frontier labs to open the weights of their old, deprecated proprietary models?

Anyone thought about this? What do you think needs to happen for them to release the old weights? I’d love to see models like Gemini-2.5, OAI o3, 4o, 4.1 being open one day. In Oct 2025 Scam Altman sa

A quick coding capability test:4 Qwen 3.6-35B GGUF Variants

Test Prompts: 1.1. Algorithm & Logic (10 pts): 'Write a function in Python that finds the contiguous subarray with the largest sum (Kadane's algorithm). Include time and space complexity annotations.'