Search: “agents”

This r/MachineLearning discussion post details a hands-on benchmark study in which TranslateGemma — Google's open translation model suite built on Gemma 3, available in 4B, 12B, and 27B sizes and cove

LogLens with local AI 'Ollama' is a Reddit post on r/ollama discussing the integration of LogLens — a log analysis tool — with Ollama to perform AI-powered log inspection entirely on local hardware, w

This Reddit post from r/MachineLearning announces a build-from-scratch Python book focused on the engineering infrastructure surrounding AI systems — the components beyond the model call itself, such

A Reddit post on r/ollama announcing an update to **Infinidev**, an AI-assisted development tool in the Ollama ecosystem, highlighting new enhanced capabilities described as 'superpowers.' Based on th

**llm-server v2** is a community-developed local LLM server tool that introduces AI-driven self-tuning capabilities, automatically identifying and applying optimal performance flags for **llama.cpp**

So I wanted to try the horsey challenge myself on the latest engine i been testing which is a strata fork by sergqwer. https://github.com/sergqwer/strata-nvfp4. I seen a post earlier and seen i believ

I came across this blog post from OrcaRouter: https://www.orcarouter.ai/blog/qwen-3-8-next-vs-qwen-3-8 Not only is the blog post nonsensical in some parts, it's also completely without any value. It b

tl;dr: Are there other models I should try? Is there something better than Kimi K3 for a code review? Is GLM-5.3 better at finding/patching security vulnerabilities? Are skills like https://github.com

Inspired by DigUp for macOS, VectorDash aims to be a full Windows 11 search replacement, powered by EmbeddingGemma 2. Full indexing: search by meaning or exact words across photos, screenshots, audio,

Summary of project status metrics Since StillDeadcode has released his new radiance inference engine, the writing is on the wall for the old vllm-radiance builds, people have moved on to newer and bet

I run local LLMs pretty much around the clock, but I also enjoy gaming on the same PC. Having models sitting in VRAM while gaming isn't exactly ideal, and constantly unloading and reloading them manua

There is a lot of discussion which quant to use. Especially a lot of post how much speed you gain with FP4, but not much about quality. So i tested some of the models that I use on my R9700. Models: Q

I stumbled across this as I was currently having glm5.3 flash run only around 10tok/s basically unusable. With Maya i am running 30 tok/s now. GLM5.3 feels even with this low quant like a much more en

webAI released TwIL LM3 Pro on September 30. I haven't seen it posted here yet, so I went through the model card, and their comparison chart is attached. It's a 3.66B model built on IBM's Granite 4.2

For what it's worth. I'm a mechanical engineer by education and software engineer in practice over the past ~20 years. Playing around with local models has been a recent hobby. Today I decided to do a

I wanted to compare gemma 4 31b qat q4 and qwen3.8 27b iq4_nl, so I run SWEBench verfied on them. The original gemma 4 chat template doesn't work with preserve reasoning, so google released an update

Hi All: I have an Asus ESC 4000 G3 with 128 GB DDR4 RAM - I tried putting in my V620s but couldn’t put more than 2. Sadly, pivoted to T4s, these are 72 watt passive cards and I thought I could use the

Strata vs Infernix on the same model, RTX 5090 — A/B at 262k and 512k context Hi folks, another inference engine post for your feed. I was blown away by Strata, but saw someone else post about Inferni

I am developing a Windows-based local AI virtual spouse application that supports adult roleplay. My goal is to run it without a server or external AI APIs. I am looking for existing open-source modul

For those of you who have been wanting to utilize your dual Spark cluster for image/video gen with ComfyUI - today is your day. I started this personal project back in April of this year, and it’s at

Disclosure: I work at H2O.ai. We released H2O-Lightning-4B, an open-weight (Apache-2.0) model for the 'decisions API' style of inference that Jev made popular: you send a state plus typed questions (p

In the dawn of the LLM craze, when wild Llama2's roamed free and 4k tokens was considered a fairly long context for local models, I forked an interactive fiction and MUD library called Tale to experim

Referring to specifically Unsloth's UD_Q4_K_XL quant because that's going to be a question, and is relevant regardless. It's old now. It's not great at coding medium sized or even small-ish projects.

Hello all! Thank you to everyone that contributed their feedback and notes for LlamAmpere the last time I posted. I've continued to chip away at improvements for the 3090 crowd -- this release is mode

ik ik 'another harness D:'. so... i spent the last 6 months of my life building and ironing out every single detail and bug through almost every route with extensive testing across 3 machines, daily,

Write-up + demo: https://drksci.com/labs-phosphene So this started as a bit of a gripe. Modern terminal renderers are seriously impressive and seriously complicated. GPU glyph atlases, texture caches,

Running a model on your own machine used to be a privacy story with a quality tax. On this generation that trade has narrowed to where we can state it plainly: for daily work, the local model is good

Row-Bot 5.0 with qwen3.8:27b in Ollama, on my own GPU. No API keys, no cloud. Gave it a year of bills, a tenancy agreement, an insurance policy and a rent increase letter (all made up). It spotted a l

We released Aplomb 1 today, a 5.3B decision model with open weights. It reads up to 1M tokens of text, JSON, images, video and audio in a single request, and on our API it makes a decision on a 1M-tok

I’ve been building this for a long fucking time, and tonight I finally released Memoria 1.0.0. I built it because I actually wanted to use it. I wanted a real memory layer for local LLM applications t

With llama.cpp using a customized SYCL backend on Intel B70 (32GB), overclocking my DDR5 memory gave modest gains for MoE models which do not fit in VRAM. Both PP and TG increased after overclocking D

Hey everyone! I am trying to go really deep into Machine Learning and Artificial Intelligence. Not just learning how to use the latest models and frameworks, but understanding how we reached here from

Disclosure: I made this. Sharing it here because it is fully local and small, and I want feedback from people who run models on their own machines. What it is: a 395M decision model (ModernBERT-large

After some recommendations for hardware (I'm starting from nothing), in the region of £6-7k. I will be mainly looking to use it for coding and have found Deepseek V4.1 Flash or Qwen3.8 27b good and so

So PewDiePie decides to fine-tune a local AI model called Ajax on his own computer. Pretty normal stuff for local model fans. To make his dataset, he uses OpenAI's API. OpenAI catches him using their

Hey all. We've spent the last weeks getting Qwen3.8-Flash-Next (125B MoE, 6B active) to run properly on one AMD Strix Halo box (Ryzen AI Max+ 395, 128 GB). Tonight we're releasing both the 95 GB EXL3

I have been using Qwen3.8-27b-iq4xs variants for the past few weeks, tweaking parameters and running random tasks, but I dove into using https://github.com/Niko1221/Strata today and it has given me 2~

Last update for those following: https://www.reddit.com/r/LocalLLaMA/comments/1wxlytt/comment/pdz728b/?screen_view_count=1 Project in a sentence: An instruct finetune of ALiceAI-80B-A3B-Base capable o

​ From the first LLaMA 33B I knew I wanted that magic-like intelligence locally, mine, so nobody could take it away when I needed it. I bought a 3090 for my home PC. Then LLaMA 65B appeared and I was

So Kimi K2 is outdated, and so is GPT OSS 120b. Which of the modern open weights models can boast the least sycophancy? I need this both for creative/research assistant usage (sycophancy led me down b

I forked llama.cpp's server into AgrillaMoE, a dedicated build for Qwen3.6-35B-A3B (~A4B) with Unsloth quants. On a (vant.ai) rented V100 16GB with the 2-bit UD-Q2_K_XL quant it generates at ~57-60 to

Last update for those following: https://www.reddit.com/r/LocalLLaMA/comments/1wvyc3e/update_3_post_training_yandexaliceai80ba3b/ Project in a sentence: An instruct finetune of ALiceAI-80B-A3B-Base ca

So this is maybe a niche problem, but at my job I work on a huge Python codebase and every time I change some shared function I'm basically playing roulette. grep tells who mentions it in the code bas

Hi everyone, TL/DR I created NInfer 4080 to run ISTA-DASLab-Qwen-3.8-27B-GSQ at 100k context on an RTX 4080 16GB GPU using way more of the hardware capabilities (max overall: 2720 tok/s prefill, 262 t

Somebody just released a fully open-source, open-weights decision model that beats Jev! Just kidding, it's me and I this is my first time releasing a public model, recipe and dataset so I am looking f

Hey local AI community, I've been working on this for a while and finally feel ok sharing it. It's a cyber benchmark where the model gets a shell in an isolated docker box and has to find the exact fl