Search: “local-ai”

For anyone running local models and still guessing whether a prompt change helped

Running things locally already means you care more about control and reproducibility than most. Worth extending that to your prompts too. There's a live workshop on Oct 3 covering exactly that. Serj S

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Modelo en MacBook Pro m3

DGX agent

Hola a todos! Tengo un MacBook Pro m3 con 16gb de RAM. No me siento cómodo con los modelos que he probado y me gustaría saber qué modelo os sentís cómodo y para que lo usáis Muchas gracias! submitted

local-air-ollama

30 Sept 2026

ollama multi gpu installer bugs

I've been struggling with the ollama update process. I have both nvidia and amd GPUs in my system. I have two instances of ollama running on different ports for this to work around issues. AMD 7800xt

Quitting for Openrouter. Am I crazy?

I'm tired of watching my compute budget and topping off my Ollama account. It feels like I can use a mix with more flash models on Openrouter (things not all available at Ollama) and control my spend

Reducing compounded round trips with aggressively structured compaction.

Ollama’s new limits and max plans are worrying me, so I’ve been going harder on my harness. (Thanks to everyone who’s signed up for the waitlist so far.) Wanted to share how my v1.10.0 will dramatical

UltraReal Upscaling LoRA for Klein9B (New Version)

A LoRA designed to reduce the typical smooth/plastic AI look and bring natural texture and realism to your images. What's new in V5 (KL9_V5_Pro): Summary: This version gives the best natural texture s

We just open-sourced the world's fastest WebGPU kernels for local AI on Hugging Face

The collection includes kernels for more than 200 common ML operations, all of which can run entirely locally in your browser on WebGPU. We're also working to upstream these optimizations to Transform

What AI model can I run??

Processor Intel(R) Core(TM) Ultra 5 226V (2.10 GHz) Installed RAM 16.0 GB (15.7 GB usable) Graphics card Intel(R) Arc(TM) 130V GPU (8GB) (128 MB) Storage 273 GB of 954 GB used System type 64-bit opera

Forlinx 20-TOPS M.2 AI accelerator supports PCIe cascading for local LLM inference

Forlinx Embedded has listed an M.2 AI accelerator card based on Rockchip’s RK1820 and RK1828 processors, providing 20 TOPS of INT8 computing performance and up to 5GB of integrated DRAM. The module us

IQuestLab/IQuest-Q1 · Hugging Face

IQuest-Q1 is a Mixture-of-Experts (MoE) model developed by IQuest for agentic coding, reasoning, and multi-step tool use. It comprises approximately 320B total parameters, with an estimated 15B parame

Ornith-1.5 DFlash

Ornith-1.5-9B-DFlash pairs the Ornith-1.5-9B model with a DFlash draft model for speculative decoding. https://huggingface.co/ornith-ai/Ornith-1.5-9B-DFlash Ornith-1.5-397B-DFlash pairs the Ornith-1.5

The page opened in 1.1s, but my local agent spent 53.5s reading it

https://preview.redd.it/2fvmzd4f3ish1.png?width=952&format=png&auto=webp&s=d21e33fd26b46d2268a12219d8c0619666eb000e I thought something was off because browser_search was taking almost a minute for a

Holo4

Holo4-27B-GGUF is a vision-language model (VLM) for Computer Use, built on the Qwen3.8 dense architecture and developed by H Company. Used with the hai-agents harness, it can send screenshots and tool

Liked Muse, so I cut the 30B model in half by width, distilled it back, and it does 57 of 60 tool tasks its parent does 60 of

I've liked how Muse-Glimmer worked, so I wanted to see if I could produce a smaller 'kid' out of it. Ornith's sharp decisions on when to think and which tool to call were the other thing I liked, so O

macOS 27 ships a free local LLM on Apple Silicon Macs. I made it easy to use from Node and Python

Apple Silicon Macs on macOS 26+ come with a small LLM built in. No download, no API key, and nothing leaves your Mac. Why I built it I was making a tool that writes API docs from code, and I didn't wa

Mica v0.1 4B got diamonds in survival Minecraft on its first run. 26 decisions from an empty inventory.

I've been working on Mica, a 4B decision model, and wanted to see how far it could get in actual Minecraft, not a sim. New world, empty inventory, and the goal was a diamond pickaxe. Last time I poste

Speculative reward hacking in coding agents

I audited thousands of agent rollouts in DeepSWE-1.1. Over 80% contained reasoning about an imagined grader. Yet no grader/verifier is mentioned in prompts nor accessible to the agents. Agents reasone

Artist here, Minimax H3 is amazing for creating images for my OCs in ref2v using Fizgig H3 Still custom node.

I am an artist working on a worldbuilding, character design and art toy project, handdrawn using Procreate on iPad with no AI use at all. However, I use AI a lot as a hobby and messing around locally

BFS Best Face Swap & Body Swap Loras By Alissonerdx

https://huggingface.co/Alissonerdx/BFS-Best-Face-Swap/tree/main https://huggingface.co/Alissonerdx/BFS-Best-Face-Swap/blob/main/workflows/Body%20Swap%20Qwen%20Image%202.1%20-%20V1.json https://hugging

Hardware Roofline Inference Calculator

Hello everyone, I made a calculator for the theoretical HW roofline for decoding / prefill based on several parameters (LLM model architecture, quants, GPU, Memory, ..). It still a theoretical bound,

I built Slopus, a free, open-source desktop app for generating and editing AI videos locally (Minimax H3)

Slopus is a desktop app for making AI videos on your own GPU, from the first prompt to the exported file. What it does: Generate scenes shot by shot. You lay out scenes on a board, write each shot, an

Navigating Cost Efficient Hardware in these Volatile Times

Where to even begin on this one...I guess I should start by acknowledging the risk vs reward for vendors other than nvidia, so: Yes I understand that nvidia are dominant currently on speed (llm and im

One .char model, Consistent face, body & cloths: Minimax H3

Hey Guys, Based on my last post on stills, I thought to experiment the same .char model with Minimax H3. I created a character model sia.char which encodes references, prompts around face, body & clot

Prevent CUDA OOM in PyTorch with dynamic lane switching

I built MEM v3 to solve a frustrating problem in PyTorch: CUDA Out-of-Memory crashes during long training and fine-tuning runs. Instead of restarting when memory spikes or keeping batch sizes overly s

Qwen3.8-27B FP8 dual GPUs

Hardware RTX 5090 (32 GB) + RTX 4070 Ti Super (16 GB) = 48 GB VRAM 32 GB DDR5 6200 RAM Arch Linux, KDE on the 5090 (takes ~1-1.5 GB VRAM) daily driver OrcaRouter Qwen3.8-27B Uncensored, block-FP8 (28.

... so, yeah.

Finally got 3.8-Flash-Next running on my M4Pro 48GB Mac with https://huggingface.co/ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF Dense 3.8-27B is just faster... and maybe better due to quantization lev

SupersonicLabs/Julia-1 · Hugging Face

New open source Jev like model for running on local devices from a group called Supersonic Labs. It's a 144M param local non-generative local classifier. From their site: Julia 1 opens our research in

What do people do with their upscaled images?

I see nearly everyone talks about upscaling their image and video gens to 4K, 8K and perhaps beyond. Now nearly every WF comes with an upscale portion tied to it either after decode or even from the l

Which of the 16gb VRAM qwen3.8 27b’s is the best?

I’m having a hard time finding out which one gives you fastest speed, maximum context with best possible quality. I can run unsloth qwen3.8 27b iq4_xs with 65k q8 kv, context without MTP and vision of

At what point do LLMs start bootstrapping themselves and generating the next LLM

I suppose technically if that happens it will signal the true start of a singularity because from that point on progress will be exponential and not dependant on humans. right now you still need huge

DistribAI v2

DistribAI is a platform for distributed training! You can train massive or tiny models across tiny or massive amounts of consumer devices with ease! DistribAI is a platform I've been working on for a

Improved and fixed template for GPT-OSS (again). Includes preserve_thinking and fix for Unsloth-induced bug

I posted an updated GPT-OSS template a couple of months ago, which was based on Unsloth's version. It turns out that both Unsloth's version (and, thus, mine) contain a very serious bug that can degrad

My Reading Library: Evaluating LLMs on Android Tasks

Can LLM agents actually get through a day in the life of a normal user? That question got me reading papers on Android agents and mobile benchmarks over the past few months. A few patterns kept showin

Qwen3.8-27B IQ3_XXS vs Qwen3.6-35B-A3B Q4_K_M

Which one is better for difficult tasks like web scrapping, coding, using tools? Looking for any benchmarks because i couldn't actually find one after quite some digging submitted by /u/Loose_Doubt367

Accidental discovery? or known method i'm missig? - Q's on compiling in sparse engrams and ideas swimming in my head

I set out to test portable Engrams and accidentally ended up testing compiled external memory instead. Am I onto something useful or reinventing a known idea? I've been building a small open research

GLM 5.3 Flash just stops during long reasoning (ollama cloud old max plan)

This keeps happening and has been happening for over a month now. Since ollama has zero support I'm hoping someone else has figured out a work around for the issue. I'm using the latest version of ope

How do you use subagents & multiple agent with local models, and how many?

Running qwen3.8 27b nvfp4 on vllm at max context only gives around 8 agents with 32k context each. That doesnt seem like much; what use cases do people use multi-agent frameworks and find it helpful f

I benchmarked tool-call reliability on qwen2.5-coder against pi, Hermes, and Goose - all three got 0%

A while back I posted here about a CLI agent I'm building that parses tool calls out of a model's plain-text output instead of trusting native function-calling to work. Finally ran the actual numbers.

I’m turning my Mac Mini M4 into an always-on home server — what should I run on it?

I recently started using my Mac Mini M4 as an always-on home server. It stays powered on most of the time, and right now I’m running my Discord bot on it. I’m thinking of gradually turning it into my

Kev 4B topped out in every Tetris game I ran. Mica v0.1 4B cleared about 4x more lines and survived two of them to the end

I had Mica (my 4B decision model) and Kev 4B play the same Tetris games, same seed and same piece order, one RTX 3090. For context, this is a side project. Training and all the experiments ran on rent

M5 Ultra 80Core GLM-5.3-Flash on DwarfStar Speeds

I've been playing around with various models on the M5 Ultra 256GB 80-core Mac Studio. These are the results over many rounds of agentic inferencing. I'm happy with the performance. Glad to have the l

Meet Observer, an agent that uses local LLMs to monitor your screen, so you don't have to

Hey r/ollama ! I'm Roy, solo dev of Observer. Observer is a free open-source agent that uses small local LLMs to monitor your screen, so you don't have to. This is the demo of the agent that controls

out of these two options what will give the best performance?

I am new to this and a week or so ago I setup a local ai box, it is an amd 9950x with 64gb ddr5 6000 ram, 2 rtx 5090s (on motherboard that does pcie gen 5 x8 per card). It is working fine but I have b

PSA for M5Ultra owners running LLMs: set your prefill step to 8192

Prefill step size affects both PP performance and your drafter fetching logits for MTP (affects dflash as well, mlx-vlm needs to be patched slightly to support chunked prefill for dflash). It also nee

Splash on a 40-core M5 Max: +20% decode by tuning the kernels for your own chip

FYI the engine's default kernel rules were measured on smaller chips (16/20-core M5s and a 32-core M4 Max), so a 40-core M5 Max runs guesses. Splash's repo includes a developer tool, “make tune-kernel

Tired of RAM prices, so I have been pooling spare RAM across old devices to run bigger local models

Been dealing with the same thing a lot of people here probably are - RAM prices keep climbing and buying a single machine with enough memory for the models I actually want to run just isn't realistic

What IDE to use for local models

Hi people, I am looking for a lightweight IDE or plugin that won't inject large context at initiation. I tried Cline and native VS Code but they inject such heavy initial context that it fills up my g

Beginner question: How to summarize session history instead of sending the entire session JSON to a local Ollama server every time?

Hi. I’m developing my first Python project (been at it a couple weeks). I built a very basic CLI ollama client. When a user sends a prompt I append the message to session['history'], save the sessio