Search: “local-ai”

I wanted to see how my fully local home voice assistant compared to the latest GPT Live, so I tested it using the same conversation used in their 'Improved Intelligence' demo. In this video they ask t

Hello Guys, Just sharing this harness I've created, this runs entirely on chromium based browser. No installation is required, it has its limitations and by no means is supposed to replace better agen

I haven't seen this mentioned yet, so I thought it deserves a post. I was trying out OpenWhispr when this model came up as the recommendation. So I don't have personal experience yet, but it's suppose

i've been thinking about this more after trying different agent setups. the model isn't the only thing that determines how well an agent performs. The harness around the model matters a lot too. With

I wish I could spend $15k on my own homelab hardware, but no this is for work lol. Like the title says, we're looking to run DSV4 Flash (and similar tier models) locally at good speeds, both for token

Looking for tools to build/run an engine to do STT-LLM-TTS basically. I know how to run all of them separately, and if need be I can build one myself, but I was curious if there is already an open sou

I'm currently building my system around 3060s, but I might be able to get a 4060 for a nice deal. At first it seemed like a no brainer, but turns out the 4060 has lower memory bandwidth. In a system t

OpenAI has decided to fully shut down a protein design project I'm working on for a client. Needless to say, open weight models are the only way forward. submitted by /u/Terminator857 [link] [comments

The downside of uncensoring a model is that it is known to potentially damage it, but CyberTiel is an even more capable software engineer than its censored TielCoder base, while allowing offensive sec

https://preview.redd.it/4c6otbeg1qoh1.png?width=1139&format=png&auto=webp&s=2d127bd3bc83d5e6ba65c3a4bd909529f34e9a3c Total beginner here, i've just downloaded Ollama and Gemma4:26b, but when it answer

are very strict... if 20 calls to kimi are more than my 5 hour limit why even bother? im gonna cancel my sub for sure https://preview.redd.it/gbqkci93gnoh1.png?width=1080&format=png&auto=webp&s=6fac5a

Both good and bad things have come from a subreddit that was lot more niche than for example r/flashlight rapidly transforming into the largest online forum about an increasingly core part of the infr

In v0.10 we achieved significant speedups over v0.9. The video shows the maximum speedups we measured for specific groups of models. Among LLMs, the biggest gain came from Qwen3 0.6B, which runs over

Anyone else play with vllm-mlx? There was an interesting paper that came out that compared this version of vllm that was originally designed for multi modal inference performance gains, but ended up s

im new to using ai so i may be missing something. I set up ollama but dont want any of my data to go to any of the model providers, for certain models i appear to need an api key to download the model

I’ve been experimenting with explaining AI/LLM concepts in a way that doesn’t assume too much technical background. This video is about KV Cache and why longer context windows require more memory duri

I was curious about how AIs talk to each other and how a conversation changes as it goes on, so I built AI Arena Lite. It connects to Ollama and lets you assign models to different agent roles, follow

https://github.com/paulknysh/raggy A lightweight CLI tool for Retrieval-Augmented Generation (RAG) over local documents built with Ollama. Hybrid database (vector + BM25 index) and embedding generatio

The fix: run the server detached/headless instead of keeping it attached to a console window. RTX 5090, ~27B NVFP4 model via ninfer: Terminal focused: 130–200 tok/s Terminal unfocused: 50–60 tok/s Cli

Cosmos3 INT4 T2I + I2V on Apple Silicon — code, weights and a Grok comparison GitHub - https://github.com/gtrg55/cosmos3-quant-mlx-cuda HF weights - https://huggingface.co/JuliaML/Cosmos3-Super-Text2I

Hi, I have the biggest pro subscription.. had it for months.. been using more or less the same models (GLM 5.x) - i would averagely be at 95% on a sunday.. and it would renew at Monday. Now we are att

I only just found ollama, but I ran into a problem described in this conversation from years back: https://www.reddit.com/r/ollama/s/YZ88XkHmIS Has nothing changed? The storage seems a mess. I had to

Who is the marketing genius at LM Studio that decided that going ALL IN on pushing their new Bionic Agent product meant they are going to make it a giant pain in the ass to find and download actual LM

I'm able to run qwen3.8 27b in two ways on my system: split between my 3060 12gb and 9070xt running at 20t/s or running off the 780m iGPU and 5400mhz DDR5 at 5t/s. Personally I feel like the 5t/s is s

We’ve been building PipesHub for a while now, and I’d love to get more teams to try it and tell us where it breaks. The problem we kept running into was pretty simple: Building an AI app over company

Confirmed by my weekly Hermes analytics. Estimated at roughly 50% cut in weekly and session limits. Which cloud inference plans are you all migrating to? (I understand that local inference is preferre

hope that helps my script uses the GPUs most discussed on the LocalLLaMA, LowEndLocalAI, and LocalLLM subs. At first, I tried to include more, but it became unreadable. Prices were collected by ChatGP

Hold right Ctrl, speak, release. faster-whisper (large-v3-turbo) transcribes on your own GPU, then qwen2.5:3b-instruct through Ollama cleans the text up: filler words out, punctuation in, spoken self-

I was thinking about using WebUI but I have a big problem with ollama serve command. Basically I made a python script that uses Ollama API and it works damn well if I don't serve the Ollama server man

Privacy have been concern of many of us to have their own hardware to run llms, and here's another reason why: two mathematicians spent a year cracking one of the hardest problems in math and fed ever

I’ve been thinking about this after using more agentic/local coding models. A lot of the newer models are surprisingly good at continuing on their own. But sometimes that seems like the problem. If a

The LLM is limited to NPC emulation. Game state, world logic, quests, and the authored story are handled by deterministic game systems rather than the LLM. I built it this way because I wanted the fre

Over the last two weeks, I set a challenge for myself: to find out if it was possible to make a high-quality music video using a local rig and open-source AI models. So, in my free time, I’ve been exp

LayerStoRm: GLM-5.3-Flash UD-Q4_K_XL (186 GiB) at 1M context on 2× RTX 5090 + 2× RTX 5080 (96 GB VRAM total) using RAM for the pinned experts. LayerStoRm is a (still experimental) MIT-licensed continu

I've hit again a point where me as a developer have to take a break from all this slop shit. Im a Developer for 13+ years and i loved it. But i fell for the slop trap. First it started with copilot an

Right, so this is going to be embarrassing but it's presumably something we've all been through at one point or another, and I guess this is my first time resolving something like this in the way that

https://village-sim-one.vercel.app/ - 16GB VRAM RTX 5070 Ti, fully offloaded - Vision on CPU - Windows, not headless - beellama.cpp - latest version with the kvarn performance enhancements making it a

I just want to use paperless-ai and be able to ask questions relative to it. Bonus points if I could use it with home assistant but that's not the focus. I just can't see needing a 32 GB VRAM GPU for

Hi, I am thinking about putting some money / time into my local ai learning path and 'cleared' my attic where I found the following hardware. 3 x NUC11 (Core i5 1145G7 / 2,6 GHz, 64 GB RAM DDR4 2666MH

NVIDIA ships two neural networks inside DLSS: the DLSS 5 renderer that makes game frames look more photoreal, and the frame generator. Both only run on RTX cards inside games that call DLSS. I pulled

Let's say I have 2x 16Gb GPUs and I want to run Qwen3.8 27B. Monitor is ran by the integrated GPU so both 16Gb GPUs are almost fully free. I load the UD-Q4_K_S on one card at 15.4Gb. I then load the c

I am getting into developing a RAG setup, for getting information out of existing documents, new document ingestion, web searches, and good visuals. I am planning to use it for, alongside the regular

You know that thing where you just throw a task at a frontier model and not have to supervise it worrying of it going off course? Qwen3.8-27b has officially gotten me to that point for local work. He

Being a person obsessed with testing new models that come out, times are really insane for me. Tested different kinds of TTS and voice cloning models but none of them gets it right in terms of emotion

Github link: https://github.com/thatblend/LLMPSP I wanted to see what the PSP can theoretically handle and I got my answer - a 90M model is about the max it can do without atrocious inference speeds.

I am working with multi-language content and have so far been useing gemma4:e2b for general work. Today I tried it with Russian for the first time and the result was not great. Gemma4:e4b was better,