Search: “r-localllama”

Surveillance plagiarism by OpenAI

Surveillance plagiarism - Hosted AI company pumps their stock price by training upon researchers' AI sessions, so that their internal model can solve problems with seemingly less human guidance, but r

US accuses Chinese AI firms of 'malicious' copying of AI technology

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

DGX agent

https://www.reuters.com/technology/us-accuses-chinese-ai-firms-industrial-scale-theft-ai-technology-2026-09-08/ https://www.nbcnews.com/tech/tech-news/us-accuses-china-ai-developers-deepseek-alibaba-c

model-releasesr-localllama

9 Sept 2026

What OpenBMB 1B version is this?

This is from the https://huggingface.co/IFM/K2-Horizon-0.9B benchmark image Tried searching for it but couldn't find it, I am guessing is https://huggingface.co/openbmb/MiniCPM5-1B but the numbers do

What settings do you use for running Qwen3.8-Flash-Next in llama.cpp?

Hi, I'm wondering what settings you are using in order to run Qwen3.8-Flash-Next on your devices? I'm especially interested in setups with 96GB VRAM. I'm not quite sure if llama.cpp does offload the e

Why the hell is LM Studio making LM Studio so difficult to download?

Who is the marketing genius at LM Studio that decided that going ALL IN on pushing their new Bionic Agent product meant they are going to make it a giant pain in the ass to find and download actual LM

Would you consider 5t/s usable for a local model?

I'm able to run qwen3.8 27b in two ways on my system: split between my 3060 12gb and 9070xt running at 20t/s or running off the 780m iGPU and 5400mhz DDR5 at 5t/s. Personally I feel like the 5t/s is s

An open-source context layer for building AI on top of company data

We’ve been building PipesHub for a while now, and I’d love to get more teams to try it and tell us where it breaks. The problem we kept running into was pretty simple: Building an AI app over company

Anyone tried self-hosting 3D object generation?

Found this post:https://www.reddit.com/r/ClaudeAI/s/At9QNEbawK of someone fully developing a game through AI writing code, generating 3D models, etc. (he is using Meshy to create the 3D models). I was

Hardware

Are there any (small, ~10B) models that you would say are a good collaborator?

Most of the new ~30B (and now ~10B, thankfully for my GPU) models we see score really high on benchmarks, but I feel like they don't push back on dumb ideas enough. I think most people don't being lik

hardwarer-localllama

Creating windows apps with opencode and a local llm. How do you do it so the llm can use and test what it's doing?

hey guys. so I should start with, I'm not s programmer at all. but I'm having a great time letting qwen3.8 27b donit for me. 😃 so far I've just been doing Linux scripts and things for my PC. but my ki

DeepSeek Flash 4.1 is already being tested via API and rolling out.

Translation: 'Internal beta testing for an intermediate version of DeepSeek V4.1 Flash is now open; you are welcome to try it out. It adopts a new model architecture featuring native multimodal suppor

For Strix Halo - Official llama.cpp isn't ideal and how to highest possible throughput

I've been making a lot of comments about optimal setup for Strix Halo (gfx1150) and from my observation, 90% of our community is using offcial llama.cpp for it, which is NOT optimized for Strix Halo a

GPU guide (GB per dollar, bandwidth)

hope that helps my script uses the GPUs most discussed on the LocalLLaMA, LowEndLocalAI, and LocalLLM subs. At first, I tried to include more, but it became unreadable. Prices were collected by ChatGP

How many agents can 2×4090 actually run at once? Three weeks of llama.cpp concurrency data — soft cap 5 @ 64k, hard cap 9, and why.

I'm the CTO of a mid-size nonprofit, and I run a local Qwen stack as a coding subagent for that work — partly on principle (a fair amount of what I handle shouldn't leave the building), partly because

I built Infercat: Share your local AI with friends over an encrypted p2p tunnel. Connect with an invite code from anywhere.

Hello LocalLLaMA Fam! I built this project because I, like many of us here, enjoy local AI and have a nice setup, but still find myself reaching for Cloud-based AI due to their convenience. My models

I made a custom llama.cpp build optimized for 7900xtx (one or two). for qwen 3.8 next and 27B. includes optimizations for PciE x4 and tensor parallel. read inside! (no AI slop)

I found a lot of room on the table for these cards so I decided to make a specialized build to squeeze all I could. first The results: qwen 3.8 next Q3_K_XL: 920tk/s pp8192 (2 cards, ram offload), 24/

inclusionAI/Ling-3.0-flash-VL · Hugging Face

Ling-3.0-flash-VL inherits the language, reasoning, and long-context capabilities of Ling-3.0-flash, while extending them with native image and video understanding. The model has 124B total parameters

Is RX 6800 + 6800 XT a sensible upgrade from 2x RTX 2060 OC 12GB for llama.cpp?

I’m currently running llama.cpp on two RTX 2060 12GB cards, so 24GB total VRAM. With Qwen3.8 27B IQ4_XS at 131k context I’m getting around 45 tok/s, which is actually pretty good for this setup. I fou

On the Value of Human Ideas: What data poisoning research reveals about 'autonomous' AI breakthroughs

I was reading up on the recent controversy around Tristan Buckmaster, Levent Alpöge, OpenAI, and the Navier–Stokes result, and it got me thinking about something broader than this particular dispute.

OpenAI alleged of stealing mathematicians work

Privacy have been concern of many of us to have their own hardware to run llms, and here's another reason why: two mathematicians spent a year cracking one of the hardest problems in math and fed ever

Qwen 3.8 27b with PI agent - pushed to its 3D graphic game limits

I was inspired by Bijan Bowen video - Subway FPS https://youtu.be/6kjXzTVmT58?t=1035 Wondered how far I can push Qwen 3.8 27b so I used a plan made by Fable 5.1 DESIGN.md which has 267 KB! ( 26K of de

Qwen3-0.6B (400 MB) on a Samsung Note 8 (2017) phone drives a real desktop Chrome

Up front: I'm one of the people building the page-perception layer used here. We started by testing small local models. The result turned out to be more interesting than the original test. 12 small mo

Qwen3.8-Flash-Next in llama.cpp vs SGLang vs FreeToken: 35s vs 258s to first token at full context. My findings on new PRs coming to engines.

Hey guys, After my CPU-only to 96GB VRAM test, I tested Qwen3.8-Flash-Next across llama.cpp, SGLang and FreeToken on the same workstation. This time I wanted to see what changes when you keep the hard

Qwen3.8-Flash-Next on 2x3090: 9–12% faster decode at ~119k context, with a completed quality screen

An update to my previous post on running Flash-Next with the expert cache and MTP. I found another useful improvement on the same dual-3090 setup: replacing the CUDA top-k fallback used by my build. O

Qwen/Qwen-Drive-1.0-4B · Hugging Face

I don't think anyone posted about this here, but Qwen released a finetuned version of 3.5 4 for driving. The full Bf16 checkpoint is 9B. This is a very interesting development of Chinese AI labs tackl

Voice conversations between Gemma4 12B and E2B on GPU and Jetson Orin

Gemma 4 12B runs on an RTX PRO 4500 Blackwell. Gemma 4 E2B run on a Jetson Orin NX 16GB; similar performance is expected on a Jetson Orin Nano Super 8GB. Both systems use a reSpeaker Flex 4-mic array

What are some practical tasks I can assign to my local AI models?

I'm looking for more information to expand my creativity around this. I don't really have a realistic idea of what people actually do with local AI yet, I mostly just want to explore the possibilities

Which local model is actually good at knowing when to stop and ask you a question?

I’ve been thinking about this after using more agentic/local coding models. A lot of the newer models are surprisingly good at continuing on their own. But sometimes that seems like the problem. If a

9 easy steps for llama.cpp, a local model, Freecad (and pi coding agent) to generate solid objects that sound mechanically good and can be also be 3D printed/milled

Quick setup on linux: STEP 0: install/download llama.cpp, Freecad, your favourite gguf model - possibly with multimedia image reading capabilities (I've used Qwen3.8-27B-UD-Q4_K_M and relative mmproj-

After over a year of my nights and weekends, the Jenny app is done!

Hi all! I just wanna say that I am tired lol. Yes, it's another harness, but I spent a lot of time and effort and have forsaken my hobbies to build the Jenny (like XJ-9) app. Jenny is a free, MIT lice

Benchmarking calories evaluation with LLMs

I wanted a quick calories counter for myself, using LLMs to evaluate the calories from pictures of meals + descriptions. I needed to pick a model so I made a quick benchmark. The setup was: - Nutritio

Can Some Generous Soul Grant Me My Wish?

Please, Oh gentle soul, please beautiful soul, please kindhearted soul, with serious hardware and time to kill, please benchmark Unsloth Qwen3.8-27B-Q8_K_XL against Qwen3.8-Next-Flash-Q4_K_XL and prov

Cybersecurity is local AI model's killer use case

This weekend I posted about the gap closing between frontier models and open source models. Well, now I'm coming with receipts. I've been running local + cloud models against real public github codeba

DeepSeek-V4-Flash-Vision-Exp is amazing at creating game worlds!

Model: DeepSeek-V4-Flash-Vision-Exp (local and API when impatient) Time: about one weekend (2 days) of QA and small improvements Full game is here After Qwen3.8-Flash-Next one-shotted a really cool Ca

Draft acceptance tweaks?

What can I do to get more from MTP if anything? It am I sitting at the expected range. Thanks /opt/llama.cpp/build/bin/llama-server --host 10.210.44.126 --port 11434 --model /opt/models/Qwen3.6-35B-A3

exllamav3 comfortably beats llama.cpp running CPU-offloaded Qwen-3.8-Flash-Next on my setup!

I got 2x 20GB RTX 3080s + 128GB of DDR4 2666hz RAM (only 4 of 6 channels populated) + a Xeon 6148 I've always been a llama.cpp person and I've been running Unsloth's Q4_K_XL quant of Qwen 3.8 Flash Ne

ExLlamaV3 is underrated

I moght get shit on for posting this but, I feel like i don't see this being talked enough and it feels like such a waste of a good piece of software. Exl3 is incredible, albeit only if you have NVIDI

Higher acceptance length, slower prose: Ling’s n=1/2/3 MTP test on one Spark

The missing control is visible in sudoingX’s Ling-3.0-flash benchmark graphics. The earlier table leaves Ling’s no-speculation baseline as “not measured.” The later code/prose graphic fills it in: abo

I made Warrior Quest, a local LLM-powered dark-fantasy RPG where the model only plays NPCs and the actual game state stays deterministic

The LLM is limited to NPC emulation. Game state, world logic, quests, and the authored story are handled by deterministic game systems rather than the LLM. I built it this way because I wanted the fre

I REALLY hope the new gemma 5 family sticks to the 'chat model first' philsophy and doesn't fall into the Qwen trap

It just seems every local 30b class model is just trying so hard to be the next Qwen that they all just kinda blend into a mass of code focused models. I really like how gemma 4 31b turned out with it

LayerStoRm open-source expert streaming: 1M context GLM-5.3-Flash [UD-Q4_K_XL] at 24.5 tok/s @8k on just 2× RTX 5090 + 2× RTX 5080 (186 GiB MoE on 96 GB VRAM)

LayerStoRm: GLM-5.3-Flash UD-Q4_K_XL (186 GiB) at 1M context on 2× RTX 5090 + 2× RTX 5080 (96 GB VRAM total) using RAM for the pinned experts. LayerStoRm is a (still experimental) MIT-licensed continu

Lit Review on Benchmarking LLMs Running in your phone!: MobileWorld: Benchmarking Autonomous Mobile Agents in Agent-User Interactive and MCP-Augmented Environments

Back to reading about LLMs as agents on your phone doing GUI tasks! This time I read about MobileWorld: Benchmarking Autonomous Mobile Agents in Agent-User Interactive and MCP-Augmented Environments a

Hardware

little-coder vs just Pi

Hi, I use hermes as a harness, and I am pleased with it, but sometimes Hermes's context size is a bit too much for my system, so I wanted to delegate or straight up use a lighter option for coding tas

hardwarer-localllama

My LLM framework. (people asked) (SAI)

DISCLAIMER I ramble a lot and am not good at focusing on documentation and or coherent thoughts. TLDR I did things I made a thing and it works for me. I will try to answer any questions. A few people

My Qwen3.8-27B task-aware quant reaches 99% of BF16 reasoning performance at 15% of the size.

TL;DR My TAK quant of Qwen 3.8 27b scored 82.81% on reasoning, comparted with 77.34% for the byte matched Unsloth UD IQ2_S and 83.59% for BF16. Edit: Some of you have tried coding with this reasoning-

New Benchmark: The Struggle Bench

How it works. The model being tested is given a server capable of running it's weights and full context. That server is placed in a median priced apartment. The AI is given a bank account with for ren

Qwen 3.8 Next Flash is really really REALLY verbose..

Long time user of 3.6 27b, switched over to Next Flash since it's a logical step up even from 3.8 27b. It's soooo verbose, i'm talking 13 minutes of thinking time on single turn coding requests at app

Qwen3.5 0.8B on CPU

Since the Qwen3.5 0.8B model is an interesting one for small specialized fine tunes, I was curious how fast it can run on CPUs. Why CPUs? Mainly because I want to use it as a local dictation cleanup m