Search: “local-ai”

I built a hybrid Transformer–SSM LLM agent with a local CLI, active control, and run receipts

I’m one of the builders of LOLM, a hybrid Transformer–SSM model and agent system from Qira. The model separates surface token processing from persistent latent-state tracking. An NFET controller can s

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Local Ai

Is Ornith 1.0 9B Reliable Enough for Complex Backend Development?

DGX agent

Hey folks, is anyone currently using Ornith 1.0 9B for serious or complex backend development? I’m working on three different applications: A multi-vendor e-commerce platform with a Node.js backend A

local-air-ollama

31 Jul 2026

Is there a way to always allow certain commands in ollama cli?

Getting tired to keep approving simple commands like 'grep', 'cat', 'find' and 'ls'. Even if you tell it to 'always allow these requests' it seems to keep asking even within the same session. submitte

We applied BitNet-style ternary quantization to a super-resolution transformer. The whole model is 668 KB gzipped and runs in the browser.

Everyone's been doing 1.58-bit for LLMs, so we tried it on a vision transformer: Swin2SR (lightweight ×2 variant, 1.01M params), quantized so every weight is −1, 0, or +1 with a small per-group scale

What am I doing wrong in my LoRA training?

Hi everyone, this is my very first time training a LoRA, so I might be missing something basic! I trained an art style LoRA using noobaiXLNAIXL_vPred10Version as the base model, with 21 images, 8 epoc

2 images + 1 prompt > expected output

Hi, I'm trying to replicate a thing locally, that I can do on ChatGPT. What I want is to give a local AI two reference images (a face and a background item) and a prompt about the composition of the p

2× Radeon R9700 for Local AI Was Choosing AMD Instead of NVIDIA a Mistake Without CUDA?

Hello together I decided to go with 2× Radeon AI PRO R9700 GPUs (64 GB total VRAM) for my local AI server. However, I keep reading that AMD/ROCm is still not as mature as NVIDIA/CUDA when it comes to

GLM 5.2 with vision on Hugging Face

Hi all, I have not seen this model talked about here but it seems like baseten (inference provider on OpenRouter) merged the vision encoder from Kimi k2.6 into GLM 5.2. I think the lack of vision was

Inkling-Small-276B-12B, effort 'max' VS Qwen3.6-27B

I saw u/danielhanchen's 1-bit Kimi K3 post: https://huggingface.co/unsloth/Kimi-K3-GGUF/discussions/12#6a6a4a90ec74ef13d85d7cf6 and decided to test Inkling-Small and Qwen3.6-27B myself, based on the f

Is it possible to have multiple concept in one LORA?

I have question. I am trying to train a LORA, and my concept is for Indian wedding and tradional wardrobe based on Regions. I was planning to train a model which understand each region clothing style

Local-first personal AI agent that runs on Ollama + Telegram — looking for feature ideas

I’ve been building ClawLite, an open-source personal AI assistant that talks to you through Telegram and defaults to Ollama (local models). What it does: Multi-agent research with actual cross-source

Making a synthetic dataset for fine-tuning

I've been thinking about building a pipeline to generate reasoning training data for LLMs, but I want to avoid the common failure mode of synthetic data where you just generate the same template with

Quantized Kimi-K3:cloud

I’m on the $20 plan and I’m not going to pay for extra usage credits to use Kimi-K3. Could the folks at Ollama quantize the model and serve it up in the cloud? submitted by /u/EvanstonNU [link] [comme

Smallest model (& tips) for intelligent computer use via Hermes?

Hello, I have a friend who's using various local LLM's like qwen3.6 27B, 35b-a3b, North Mini Code, and qwen2.5-vl-7b (just for vision). They have a use case where they're trying to have an LLM drive a

What actually happened to the whole Openclaw frenzy?

A while back you couldn't open reddit or youtube without sifting through tons of Openclaw content. And it wasn't just the internet that blew up, I remember seeing images from China where crowds would

3090 owners, what vram tempature do you get under ai load?

Hello Can you please share the tempature you get on your rtx 3090 under active llm load? Im trying to findout if my rtx 3090's tempatures are healthy or not please share VRAM Tempature only, you can t

Bought a 5090 to escape API fees. Ended up building a mini datacenter. Sound familiar?

I bought an RTX 5090 last year just to run 27B models natively. I even fine-tuned it with my own data using LoRA, building RAGs and was pretty damn happy with the results at first. But, Q8 quantizatio

Everyone posts day-one impressions. What's still in your stack a month later?

Day one threads are the least useful thing we produce here and we produce a lot of them. Model drops, forty people run their favourite prompt, half say it's the best thing ever and half say benchmaxxe

Ollama going down the Copilot path?

What happened? I just asked GLM 5.2 one question, and in 3 minutes (one agent) it used up 15% of my 5 hour limit to produce a single answer. At this rate, I'll exhaust the entire 5 hour limit in just

The idea: on a CPU the decode speed depends on the active params per token, not the total. My objective is trying to run a 10B at 100tok/s on a mid level PC (No GPU).

On the CPU, batch 1 is memory bandwidth bound. But if token/s = bandwidth / (bytes_per_weight * active_weights_per_token) the total number of parameters doesnt slow down the generation speed. So build

Understand Kimi K3 from first principles: a recommended order for anyone trying to understand this beast

Everyone is talking about Kimi K3, but if you jump straight into the technical report, you’ll quickly realize it’s standing on years of research -- just like any breakthrough is! If you want to unders

Vendor-agnostic ML inference on production edge devices [R]

local-air-machinelearning

I work on PostSlate, a video editing tool, and this comes out of our own work. We run ML models on-device, face detection and embedding among other things, which means we can't assume anything about t

Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 Incident

The first autonomous agent cyberattack is an unprecedented event that deserves unprecedented transparency. Today we're sharing everything we can: a full technical timeline, an interactive replay, and

I Just Built an Open-Source Alternative to Flora AI

Hey, I just made a node editor for generating assets using AI. It’s something similar to ComfyUI, but it’s more simple and abstract. You can say it’s the open-source alternative to Flora AI. It’s curr

I've been tracking RTX 5090 prices across EU stores since March, it's up €1,061 and still climbing

Been running a GPU price tracker (https://www.pricesquirrel.com) since March, covering 20+ EU stores, recently added RAM, SSDs and CPUs too. Every GPU tier has gotten cheaper since launch. The RTX 509

K2Lab: Standalone(ish) Krea2 bbox style prompting and lora containment

I've been digging into Krea2 to see if there's any way to condition inference in specific regions of pixel -> latent space to implement bbox type prompting in order to apply multiple simultaneous char

Manga Coloring Tool 2

Hey everyone! 👋 I'm excited to announce the official release of Manga Coloring Tool 2.0, a completely free, local, open-source web application designed to colorize manga pages and chapters effortlessl

McBess style lora (lokr) for Krea2: <5 Mb size.

https://civitai.com/models/2813960/mcbess-style Happy to share my holy grail of a style lora. I have been chasing the edgy alt-rubber hose style of McBess since SDXL training was a thing. Krea finally

microsoft/VibeVoice-ASR-BitNet

VibeVoice-ASR-BitNet is a compressed variant of VibeVoice-ASR optimized for real-time inference on edge CPUs — no GPU required. Through heterogeneous quantization, the model is compressed from 4.62 GB

My first longer Wan2.2 continuation generation. I am so excited

Hey guys, I am so excited to share this with you guys. I know for a lot of Pros here, this maybe a baby's work so please be gentle. Until a month ago, I didnt know anything but to use those google AI

Now, this: 1,100 current/former frontier-AI employees sign a petition calling for US gov't to step in for 'pacing' frontier development

So, it appears that this is the week of open letters in AI🥲... an open letter signed by current and former employees of OpenAI, Anthropic and Google primarily - calling for a slow-down in frontier AI

PIRL: From Open-Loop Exploration to Closed-Loop Reinforcement Learning [R]

local-air-machinelearning

TL;DR: Most RL post-training algorithms optimize the current batch and move on. But after an update, did the new policy actually become better? We introduce Policy Improvement Reinforcement Learning (

What 'task oriented' models are folks running on N100 MiniPCs with 16GB of RAM and no GPU?

By 'task oriented', I dont really mean agentic, I mean no deep coding ability, no need for conversation. More things like classification, identification, simple interaction with web apps and APIs, etc

Zuck's opinion: The AI Future Is for Everyone

’Tis the season of AI open letters and manifestos, apparently. Mark Zuckerberg has now entered the debate over the future of AI with a WSJ op-ed published today - and frankly, his position is much mor

Built a local-first workflow automation platform around Ollama - now at v0.11.0

I've been working on an open-source workflow automation platform over the past few months, with Ollama as one of the first-class providers rather than treating it as an afterthought. The goal wasn't t

Give any Ollama-compatible client session memory + a shared knowledge wiki by swapping the chat URL

Hey folks — I built ContextMemory, an open-source agentic context gateway for apps that already talk to LLMs. The idea is simple: keep your existing POST /api/chat client (Ollama wire format), point i

How much usage does Ollama Pro give right now vs direct API?

I'm considering the 20 Pro plan, but the limits seem to change over time and are hard to compare with direct API pricing. For anyone using Pro currently, roughly how much coding agent usage do you get

I want to run Kimi K3 at home, so I’m trying to make 2.8T-scale experimentation cheaper

Hey r/LocalLLaMA, I’m a retired engineer with a background in distributed computing, currently running a 1-person startup. Like many people here, I’d love to experiment with 2T+ MoE models locally. Th

My Ollama box picks the music now: an agentic DJ running on a 9B model

I got tired of my Ollama server sitting idle between chat experiments, so I pointed it at my Navidrome library and made it run a radio station. The DJ is an agent, not a shuffler. Each turn it gets to

Nvidia CEO Jensen Huang defends Open Source AI by saying distillation is fundamental to learning

Nvidia CEO Jensen Huang “Distillation - learning from AI, learning from other people, and learning from other sources of knowledge, is fundamental to intelligence. We are constantly learning from one

NYT: Protect America’s lead in the A.I. race.

“China is working hard to catch up, and the United States should take steps to keep its advantage. Most important, it should continue to prohibit American companies from selling the most advanced chip

Ornith-397B running at Q4 on a single RTX PRO 6000 Blackwell 96GB - 2,354 tok/s prefill, ~20–24 tok/s decode

I've been building Krasis, an MoE-focused runtime for streaming big models through limited VRAM on NVIDIA consumer/workstation GPUs, and I think this is the most interesting result so far: Ornith-1.0-

Running local agentic workflows with Ollama? Here is a pre-flight validator for third-party skills

sample Hey r/Ollama! If you're hooking up local agent tools and using Ollama to drive agentic workflows, you've probably downloaded a bunch of third-party SKILL packages or repos. I built a free open-

RX 9060 XT 16GB vs RTX 5060 Ti 16GB for local AI — worth the price difference?

Hey everyone! I’m building a PC to run AI models locally and can’t decide between the RX 9060 XT 16GB and the RTX 5060 Ti 16GB. Both have the same amount of VRAM, but is AMD actually a solid choice fo

Trying out LoKr instead of LoRA on Krea2

Dataset of 43 images, captioned with qwen3 VL 4B instruct, 50 word caption focusing on: Composition, Subject's hair, expression, clothes, pose, background Training Parameters: (10 rep x 43 image) x 6

Unable to get GPU Passthrough working - Docker

Setup as follows: Proxmox -> Debian -> Docker -> Ollama. Other containers work. Compose file contains gpu device. Does it need nvidia runtime or any other options? If someone could provide an example

Unexpected use of local llm

I was refreshing my youtube and found out my favourite reviewer uploaded a battery test of 78 smartphones: https://youtu.be/MpgUFrsIWSQ the author said they started using robotic arm to simulate a per

ai-sage/GigaChat3.1-Audio-10B-A1.8B · Hugging Face

GigaChat Audio 10B is an audio-native LLM built on top of the GigaChat 3.1 Lightning text model. A Conformer speech encoder and a modality adapter feed audio embeddings directly into a Mixture-of-Expe