local ai

Receipt scanner that runs a fine-tuned 0.8B vision model on the phone (free) - detailed cost tracking

I wanted to track grocery spending per item without sending my receipts to anyone, so I built an Android app around a small vision model that reads a photo of a receipt into JSON: store, date, total,

12 Oct 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Local Ai

2x 3090, 3x 3060 potential (smart and informed) future upgrade paths (send help)

DGX agent

I'm trying to figure out the best long-term upgrade path for my setup. Right now my GPUs are spread across several consumer PCs: Main: 5900X, 32GB DDR4, 2× RTX 3090 24GB. Secondary: i3-10320, 16GB DDR

local-air-localllama

11 Oct 2026

I built a lightweight tool that automatically frees up VRAM for gaming and restores your local LLMs afterward

I run local LLMs pretty much around the clock, but I also enjoy gaming on the same PC. Having models sitting in VRAM while gaming isn't exactly ideal, and constantly unloading and reloading them manua

11 Oct 2026

M5 Ultra 256GB or 2x DGX Spark? Which one?

Currently contemplating adding either M5 Ultra 256GB or dual DGX Spark in addition to existing 5090. I have been researching for past few days and have zeroed in on dual sparks because higher throughp

11 Oct 2026

microsoft doubles down on local ai with nvidia, but with a cost

https://preview.redd.it/uyjw2lcrvkuh1.png?width=1536&format=png&auto=webp&s=ce991be22faec3de5cbaf39119c14782b6ed0af3 link to article TLDR; new microsoft surface and nvidia rtx spark laptops are starti

Recommendations for Local AI Long-Term Memory and User Personalization Modules

I am developing a Windows-based local AI virtual spouse application that supports adult roleplay. My goal is to run it without a server or external AI APIs. I am looking for existing open-source modul

Typesafe ai raised 870m $ on hype (jev)

https://typesafe.ai/blog/series-ai Meanwhile one of the reddit members apparently developed the same architecture a year ago: https://www.reddit.com/r/LocalLLaMA/s/8zQXqCNDlb Looks like ai valuation n

We built Many Agents give AI agents their own cloud computers and let them work for you 🚀

Hey everyone! 👋 After months of building, we’re finally opening the first spots for **Many Agents**, a new product from Dome. The idea is simple: **What if you could create your own team of AI agents

A 24KB stand-alone HTML-LLM that can generate consistent stories

https://preview.redd.it/7sknv6omnguh1.png?width=1023&format=png&auto=webp&s=4aef1dc464d008b00f104519eb5399ba6132ad69 Want to try? Just go here and click 'New seed' a lot. You will get different storie

embeddinggemma-2 MLX only?

I checked the site, and it didn't say anything about mlx only on the ollama models listing. So why is this restriction in place? ollama pull embeddinggemma-2 pulling manifest Error: this model require

Epyc inference rigs

I posted on here a while back asking for advice on upgrading to an Epyc system. Thanks to all who commented. I’ve taken the plunge and ordered: - Epyc 7443 CPU (24 cores, 48 threads, 128 lanes) - H12S

GitHub - google-ai-edge/ml-drift: GPU-Accelerated AI/ML Inference

Blog Post : ML Drift: Next-Gen GPU AI/ML Inference at the Edge - Google Developers Blog The Google AI Edge Team is excited to announce the open-source release of ML Drift, our high-performance, cross-

HIVE: Open Source Research Tool w/ Confidence

Hi everyone! So far in 2026 I have been iterating and evolving a research system I designed to help me sift through things that LLMs all agree on that may not be correct, and to give me actual confide

I built ALHR: A tree based sparse attention system that achieves sub-quadratic inference while retaining accuracy. [P]

local-air-machinelearning

ALHR - Adaptive Learnable Hierarchical Routing, uses static binary trees and learnable functions to minimize the amount of keys to be read. It does use a dense teacher while phase 1 of training howeve

I made a Mac app that uses your Ollama models to answer questions from your own folders, with the exact sentence cited

Disclosure: I'm the developer. The app is called Citenook. I wanted to ask questions across a few hundred PDFs and notes with Ollama, but every setup I tried either needed a lot of glue code or gave a

LlamaTale v0.43.0 - MCP Server for story creation

In the dawn of the LLM craze, when wild Llama2's roamed free and 4k tokens was considered a fairly long context for local models, I forked an interactive fiction and MUD library called Tale to experim

[Paper] Stepped MoE: Segment-Level Routing with Configurable Inference Complexity

Training large language models (LLMs) is resource-intensive, and adapting them for diverse deployment scenarios with varying computational constraints remains challenging. While elastic architectures

Qwen3.8-27B: 159 tok/s on R9700, 64 tok/s on Strix Halo

LemonSeed Studio is an iPad editor/IDE with on-device inference on an AMD GPU in a Thunderbolt enclosure. It embeds the unmodified upstream Linux amdgpu + amdkfd driver (mac_linuxgpu) as a PCIDriverKi

Qwen3.8 27b with 200K ctx + MTP on 12GB Ampere Cards

Hello all! Thank you to everyone that contributed their feedback and notes for LlamAmpere the last time I posted. I've continued to chip away at improvements for the 3090 crowd -- this release is mode

[audio.cpp] Recent updates you might have missed: Higgs Audio TTS use 48% less VRAM (< 6GB), HTDemucs 2.2× faster, PocketTTS 2.2× faster on CPU, and WebUI generation history feature

Hi all, a bunch of performance improvements have been landed in audio.cpp. The biggest highlight is Higgs Audio TTS, which now runs with around 6 GB VRAM, a 48% reduction in peak memory usage compared

Currently having high success with this little niche finetune i found sitting in the corner of huggingface

Currently having high success with this little niche finetune i found sitting in the corner of huggingface If you want to try it out here is a smaller quantisation iq3_s works really well in my codeba

I built an open source framework for building RL environments. Named 'Seahaven' after the fake town in The Truman Show.

I've been optimizing long-running agents: rewriting their prompts, tools, skills and subagents, and keeping the changes that score better. I've been working on some version of this problem for over a

Involution Without Export Is Wasted Effort

A week in Beijing and Shanghai with the people building AI in China some quotes from the article: ByteDance/Doubao is the only major lab keeping its flagship closed, because it is the only one that al

New challenge: consolidated 2 cards in a single system but one card hogs all the models even if I run 2 different instances of Ollama on 2 different ports

In my attempt to combat powerty; as electricity cost a ton these days and it is hot as hell, I consolidated 2 machines into one, so my 2 crappy machines now are one, and that means a bit less power an

A new video model from Tencent is coming... maybe? 'Prism'

It's on the official Tencent Github page now: https://github.com/Tencent-Hunyuan/Prism Looks like they started uploading bits of it two weeks ago, but its still beta. But I think its odd its on an ind

Built an SQLite memory engine for Ollama that does not eat all your VRAM

Wanted to share a local memory engine I built called Hillock. The main problem I had with standard local RAG was that running document parsing and vector search chewed up so much VRAM that my actual O

Cannot install gemma4:26b

Every time I try, I'm met with this: error message: Error: max retries exceeded: Get 'https://dd20bb891979d25aebc8bec07b2b3bbc.r2.cloudflarestorage.com/ollama/docker/registry/v2/blobs/sha256/00/001e5d

ComfyUI-qwen_img_2_1_enhancer added ref mask support

Follow up update of this post here I added mask support to the reference strength node. You can now increase or reduce attention to a selected part of a reference instead of adjusting the whole photo.

Embedding and Rerankers models should be availabe on ckoud

I am sure people need it you know? There are plenty of embedding models on ollama, but only for local deployment. Which is a shame. I wish ollama would make embedding and rerankers available on ollama

FastVideo’s FastH3 now runs on a single consumer machine

https://x.com/haoailab/status/2107591980591227227 https://github.com/hao-ai-lab/FastVideo https://haoailab.com/FastVideo/cookbook/minimax-h3/ https://huggingface.co/FastVideo/FastVideo-FastH3-Trim-Com

Idea/Thinking outloud: Minimax to build LoRA datasets?

Since Minimax H3 does such a good job of taking a source or two of a character and doing an animation with them while maintaining the character's integrity, it has me thinking that it could be used to

Image-text retrieval with EmbeddingGemma 2's vision tower, running in the browser on WebGPU

EmbeddingGemma 2 came out this week. It maps images and text into one 768-dim space, so you can search photos by describing them. I ported its text and vision towers to ruNNtime, a WebGPU inference li

Looking for 10–20 people for a coordinated LLM test

Thursday, October 8 — 15:00–19:00 UTC Access is free and the API is OpenAI-compatible. During those 4 hours, everyone will be asked to use it actively at the same time—coding, chatting, agents, long c

Looking for the best affordable AI video-generation models with fewer restrictions.

Hi everyone, I’m exploring AI filmmaking and looking for good AI video-generation models that are either free, open-source/open-weight, or affordable. I’m particularly interested in models who allow u

omni-d1 600M by Liquid AI running in the browser using WebGPU

Liquid just dropped d1-omni-600M and it's a decision model! I ported it to runntime, the WebGPU inference library I'm currently working on. It's plain TypeScript on top of TypeGPU with no WASM and vir

PC making weird noises when using AIs locally

I recently installed ollama on my machine to test using AI locally, I don't know if you will be able to hear it on the video but every time my terminal outputs a response there is this weird writing/s

Ruach Studio: a whole song studio around YuE2 on your own GPU. Score first, LoRA training, stems,remaster, DAW export.

We spent the last weeks building a studio around YuE2, the open song model by m-a-p, and today it reaches its first release candidate. Write the style and the lyrics, and it composes, sings and render

Small uncensored model for everyday questions on a CPU-only laptop?

Hey everyone, I'm looking for a small uncensored model to run locally with Ollama. Nothing fancy. I just want something I can ask everyday questions without it refusing or lecturing me every other mes

Three AI agents research a 4-day week and build the deck | Row-Bot + GPT 6.1 Sol

One prompt to Row-Bot 5.0 on GPT 6.1 Sol, through my ChatGPT plan (no API key): 'Split the research across three helper agents working in parallel, cite sources, save the findings, then build a 6-slid

A local 27B model reads my lease and finds a leak in my bills | Row-Bot + qwen3.8 on Ollama

Row-Bot 5.0 with qwen3.8:27b in Ollama, on my own GPU. No API keys, no cloud. Gave it a year of bills, a tenancy agreement, an insurance policy and a rent increase letter (all made up). It spotted a l

HunyuanImage 3.0 (80B) running natively in ComfyUI on a single 12–24 GB GPU: text-to-image, editing and style transfer, ~30 s per image

I've been working on native ComfyUI support for Tencent's HunyuanImage 3.0, the 80B mixture-of-experts image model (13B active per step). It isn't a wrapper around Tencent's pipeline: it uses the norm

I made a Chrome extension that filters your YouTube feed with a small local model running in the browser (WebGPU, no server)

My YouTube feed was mostly songs, pranks and celebrity clips, so I built a filter that judges each video title with a small decision model running inside Chrome. Nothing is sent anywhere: no server, n

I made a Forge Neo extension for MiniMax H3: text or picture to video with sound, reference pictures, runs on 16 GB

I've been working on an extension that runs MiniMax H3 inside Forge Neo. H3 makes the picture and the sound together, in one pass: footsteps, rain, engines, music and dialogue in pretty much any langu

I made a tiny (~8MB) photo editor for AI images with batch editing, LUTs, and ComfyUI workflow preservation [Free & Open Source]

I was tired of opening heavy photo editors just to tweak lighting or color-grade a folder of AI generations. So I built TinyLuma - an instant, lightweight photo editor designed specifically for finish

I trained a model to be wrong 98% of the time and 96% sure about it. It took three tries.

Meet Bev. She is a decision model (the Jev / Nimble kind: you give her a situation and a question, she gives a probability for each answer), fine-tuned on Qwen3.5-9B to pick the worst answer on purpos

Microsoft confirms OpenAI has been using Looped Transformers in the GPT-6 series

Microsoft confirms on publicly accessible web page that OpenAI has been using Looped Transformers in the GPT-6 series, proving The Information's reporting was correct all along. GPT-6.1 Sol uses 2 inf

Ollama Cloud Model Cost Table on Github

Ollama Cloud Model pricing monitor with tabled costs and calculator. https://kasp0r.github.io/ollama_cloud_models_comparison https://preview.redd.it/hxbpnuq7quth1.png?width=1372&format=png&auto=webp&s

Tencent releases Octop, a self-hosted AI assistant

Octop is an open-source, self-hosted AI assistant. Through its multi-agent architecture, it builds an intelligent environment that is both independent and collaborative for teams, families, and indivi