All entries

How much is the GPU usage?

I am trying to decide on buying the ollama pro subscription. But their usage policy is vague as hell. I don't mind the 'GPU usage time' but how much GPU time do I actually get? I still don't seem to f

I built an open-source Ollama canvas where the wires are the actual context

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Most graph-based LLM interfaces use a canvas as a visual layer over what is still a linear chat. I wanted the graph itself to determine what Ollama receives. ThoughtDAG has one rule: wires are the con

local-air-ollama

26 Jul 2026

I implemented the YOLO26n model inference from scratch using ARM64 Assembly Language (No framework) [P]

local-air-machinelearning

This was my Bachelor's Final Project: implementing YOLO26n inference completely from scratch using ARM64 Assembly Language and C, without relying on existing inference frameworks. The goal was to unde

I want to use AI coding agents for machine learning projects [D]

model-releasesr-machinelearning

I'm a software engineer who mainly builds softwaes/applications, and I'm starting to work on machine learning projects. Since ML workloads often require GPUs, I know services like Google Colab and Kag

Karparthy removed Anthropic from his bio

Andrej Karpathy, a prominent advocate for open-source AI and a co-founder of OpenAI, appears to have removed Anthropic from his X bio, suggesting he may have left the company. Karpathy joined Anthropi

Kimi K3 Tomorrow!!!

What quantization or storage size tiers should we expect further down the line? This site suggests q4 and q8 are coming. But I don't know how to translate that to storage size for local inference. htt

Local-first LLM pipeline tracer — @trace on any function, dashboard at localhost. Feedback welcome.

Hey r/LocalLLaMA — maintainer here, obviously biased. OpenSmith is an open-source Python tracing tool for LLM pipelines. The idea: drop u/trace on any function, run opensmith ui, get a full local dash

Research

Multi-Tenant SaaS: Which Architecture Would You Choose? [D]

researchr-machinelearning

NOTE -> I expect answer from people who actually have experience and strong understanding of these. please give something beneficial. I'm building a SaaS platform in Sri Lanka that handles documents a

Need help with setup

I am setting up codex+ollama+qwen3.6:27b for my hobby coding project on a windows pc with rtx5090 - earlier i tried to setup vllm in Ubuntu container but couldn’t get that to work - now using ollama,

model-releasesr-ollama

Ollama Cloud Phone Number Verification Issue

https://preview.redd.it/umcg41g2kkfh1.png?width=502&format=png&auto=webp&s=7b1ed545cd1c7840a461101bd02d78ab4a9cb40d I tried to create an Ollama account to purchase a subscription a few months ago. But

Open-weight 4B models approach o3-level medical question answering in Swedish [P]

model-releasesr-machinelearning

I have been running some experiments with smaller open-weight LLMs on multiple-choice questions of Swedish medical licensing exams. On a dataset called MedQA-SWE, GPT-4 scored 84% accuracy in 2024 and

[Paper] RecGPT-V3 Technical Report

Large language models (LLMs) are transforming recommender systems from matching co-occurrence patterns in historical behavior toward reasoning about the intent that drives it. RecGPT-V1 pioneered this

Question regarding Ollama Cloud Metering.

How does it work? I read that the limts were more than what Opencode Go offers and subscribed to the 20USD Pro plan. But In practice based on how the usage bar fills up it seems like the 5 hour window

Simple desktop GUI for multiple local TTS models (Tkinter)

https://preview.redd.it/qoogv5m1skfh1.png?width=688&format=png&auto=webp&s=3f06fb28ceec3cd3ab95bcebd0c71374332aac9b Built a simple desktop GUI (Tkinter) that supports multiple local TTS engines (curre

Hardware

Understanding GPU Inference Workloads [D]

hardwarer-machinelearning

Hey everyone, I have been looking into how people source compute for their Inference workloads (and in general). I wanted to understand some specific pain points here. If you've used online services l

We compared different LLMs on IMO 2026 [R]

model-releasesr-machinelearning

There are a few reasons why problems from International Mathematical Olympiad function as a good benchmark for LLMs: - The problems are new, not included in the training data of any model - Hard math

We open-sourced Logue — a privacy-first macOS meeting-notes + writing app that runs on-device (MLX, Apple Silicon) entirely

At Bitwize, we've been building Logue, a native macOS app for AI meeting notes and writing, and we just open-sourced it (MIT). We're sharing it here because the whole point is that it runs 100% on-dev

what am I doing wrong? (Ollama + openWebUI)

I have tried ollama locally on my gaming PC (5070 with 12bg of VRAM) It works pretty nice on qwen2.5-coder:7b (and 14b) So I decided to take a step further, and install an openWebUI instance on my hom

Will prices finally go down?

I am seeing more and more videos as posts about how OpenAI is in complete financial ruin, Anthropic isn't much better. Their expenses go with the revenue they make etc etc. Meta made big investments i

Will small model intelligence be limited by parameter count?

Qwen3.6-27b is fantastic! It makes me wonder if there's a hard ceiling to smaller sized models. Do you guys think the ceiling of intelligence for smaller models will be constrained by factors like par

4x 3090, 96gb vram what Model to drive Hermes?

3 year lurker, now i finally got my server up and running. dont know which model to choose. llama.cpp or vllm, what makes more sense? mainly single user with maybe 2-3 more additional users in family,

5700, 48GB RAM and a 3090 24Gb. Best OS and framework/model?

Hi everyone, I have a system which I have been using for gaming, R7 5700X, 48GB DDR4, RTX3090 24GB. But I want to use it for Local AI to reduce my reliance on cloud AI providers (mainly usage limits -

Benchmarks: TensorSharp vs. llama.cpp

Cuda and Vulkan Benchmark: TensorSharp vs. llama.cpp I would like to share my latest open source local Unsloth (GGUF) LLM inference engine and applications. It supports many models from Unsloth, like

Best C++ Local Model? (July 24th 2026 Edition :-P)

I apologize that this question has been asked in various flavors over time, but I couldn't find anything in the posts before that matches the options I have. I have a PC and a mac, both available in m

model-releasesr-ollama

CachyLLama: llama.cpp fork with persistent SSD-backed KV caching for local agent workflows

If you run local agentic coding harnesses (Aider, Claude Code, etc.), prompt evaluation usually eats up most of your execution time. Every turn re-evaluates thousands of identical prefix tokens_system

Deepseek V4 flash - Hy3 or is Qwen3.6 27B still the most solid for agentic/coding?

I understand that the laguna model is either still buggy or potentially benchmaxxed. So I’d like to know for people who really tested, are DS flash or Hy3 really better in your usecase? submitted by /

DKV: Open-source KV-cache compression framework for local LLM inference (CLI + technical report)

Hi everyone! Over the past five months I've been working on DKV (DifferentialKV), an open-source project exploring KV-cache compression for long-context local LLM inference. The goal is to reduce KV-c

Getting a second GPU in addition to my RTX3090

Hello, I've been learning how to use local LLMs for a year or so on my workstation, using a RTX3090. Current setup : - i5 12400 - 64gb RAM - RTX 3090 - OS : Fedora KDE workstation I'm using LMStudio t

Help me complete my AI collection

I’m building the ultimate AI tool vault, but every great collection has a few missing pieces. Note: I will react to every comment AI's currently installed: Qwen3.5-0.8B-UD-Q4_K_XL.gguf(classification)

I released Inflect v2: two ultra-tiny complete TTS models under 4M and 10M parameters

I’ve spent the past month trying to find the point where an extremely small TTS model stops feeling like a size experiment and starts feeling genuinely useful. Today I’m releasing Inflect v2, with two

I spent a year building a free SDXL & Anima trainer that runs on my 12 GB GPU — here's what came out of it

local-air-stablediffusion

A little over a year ago I got frustrated trying to fine-tune SDXL on my RTX 3060. Every option either forced lower resolution, locked away important settings behind massive config files, or needed a

Industry

I spent months testing whether ChatGPT can create a consistent 100-page comic. This is the result.

About 2.5 years ago I tried making an AI comic. It failed. Characters changed, environments drifted, and every page needed manual editing. So I started over with one simple rule: One prompt = one fini

industryr-chatgpt

Hardware

I tried making a cinematic action trailer using Krea 2 + LTX 2.3

hardwarer-stablediffusion

I wanted to challenge myself and see how far I could push Krea 2 and LTX 2.3, so I decided to create a short cinematic action trailer. It ended up being one of the most enjoyable AI projects I've work

Im back from gemini. GPT is astronomically better again.

like 8 months ago I was tinkering with both and gemini was so much better i went with that. I had a project at work that I needed an AI to sift through a manual and schematic for and no matter what, g

model-releasesr-chatgpt

Is it worth getting 128GB MacBook Pro? Will it ever be comparable to today’s frontier models for coding?

I am a long time iOS app developer. In the last year I have been using Cursor+Claude/others to assist with app development. I am concerned that the current low pricing will disappear eventually. I am

Is this real ? Qwen3.6:27b with 128k context fit in 24Gb VRAM ?

https://preview.redd.it/yw41s1jikefh1.png?width=1942&format=png&auto=webp&s=3a180ae6443c1db9f7b0ce621533a4b2aa553921 Hi, I've been running Ollama on my Unraid server since the llama2 era. I use to be

model-releasesr-ollama

Kimi Linear 48B A3B?

Just noticed this exists, 1M context MOE with 48B par seems just like what Ive been looking for - it runs pretty damn fast too compared to Qwen 3.6 35B. after some testing it seems capable of producin

Krea 2 Identity Edit running in Forge Neo (extension port) — works great, GitHub release soon

local-air-stablediffusion

I ported Krea 2 Identity Edit support to Forge Neo as an extension — the instruction-based, identity-preserving edit LoRA that until now required ComfyUI + custom nodes. It replicates the full dual-co

Launching ComfyUI with a Blank Canvas (StabilityMatrix)

model-releasesr-stablediffusion

Hey everyone, I’m posting this question in the subreddit because I haven’t been able to figure it out with the help of AI. I’ve asked ChatGPT and Gemini, but their answers are all over the place. So,

LFM 2.5 230M running at 1440 tok/s in-browser through a custom backend

Everything runs through WebGPU, in-browser or in electron/tauri apps. It's fully portable and supports either Nvidia and Apple Silicon (Metal). The actual kernels are optimized for the specific hardwa

Llama.cpp now has full MCP support!

After a long and grueling effort spearheaded by ngxson, llama.cpp now fully supports MCP for all protocols. Over-the-web HTTP servers were already supported in the client (since they don't require any

Local alternative to Kling AI 3.0 Motion Control (ComfyUI, 16GB VRAM)

local-air-stablediffusion

Hi everyone, I'm looking for a local alternative to Kling AI 3.0 Motion Control that I can run in ComfyUI. What I'm specifically looking for is a model or workflow that allows me to: - Control charact

MI50 power curve tests

tests done power limiting the GPU on LACT - real power usage varies wildy at 20W it ranges from 25W to 56W same behavior happens on every setting prompt for the test runs: https://github.com/lukesdevl

Microsoft's website shows OpenAI as one of the signatories of the open weight AI letter

https://preview.redd.it/a24z80gr6afh1.png?width=1181&format=png&auto=webp&s=4a844ebe2319eb6230dbdc63c9caf492bed5ff47 So, this came up on: https://www.microsoft.com/en-us/corporate-responsibility/topic

Mobile Offline LLMs: What do you use them for?

I've spent the last year or so playing around with open source MLX and GGUF models on iPhone hardware. Given the limitations in memory, GPU/CPU/ANE, and in turn the context window I've been trying to

Modelos do ollama cloud perdendo qualidade?

A algumas semanas, percebi algo diferente, aparentemente modelos de qualidade, em especial o GLM 5.1 ficou mais burro, e começou a mandar caracteres em mandarim para mim sem eu nem usar eles, e isto n

Agents

My ChatGPT created a rally speech in response to counting letters.

I was exploring how to prompt the voice chat so that it can correctly count the number of “e”s in seventeen (which it fails most of the time). At the beginning of a new chat, instead of counting “e”s

agentsr-chatgpt

Old Coder Needs help with New AI Development and wants to get up to speed to understand it all.

Hi Guys, I'm an old coder and DBA that has been in the field for almost 40 years. More and more the jobs I was doing for work are being taken over by AI and the need for my type of work is diminishing