Search: “agents”

Qwen 35B-A3B MoE vs 27B dense in local coding tests: ~4× faster, much smaller quality gap than I expected

I compared Qwen 35B-A3B MoE against Qwen 27B dense on a series of local coding-maintenance tasks. On my R9700/llama.cpp setup, the MoE model generated about 3.9× faster (~116 vs ~30 tok/s), but the co

8 Aug 2026

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

nvidias nemotron omni only loads its text half on a mac, so i wrote the vision and audio towers in mlx

DGX agent

nvidias nemotron omni is open weights and it sees, hears and reasons. theres already a 4bit mlx quant on hugging face but only the text backbone loads with standard mlx tooling. the model card says it

model-releasesr-localllama

6 Aug 2026

I got tired of ad-filled mobile wrappers for Ollama, so I built PocketLLM Lite an open-source, offline Android client (Local GGUF, SKILL.md plugins, local RAG)

Hey, Like a lot of people here, I use local models via Ollama on my desktop/server and wanted a mobile client that actually felt responsive, worked offline, and respected privacy. Most apps on the Pla

3 Aug 2026

Running DeepSeek-V4-Flash-0731 (155 GB MoE) on a DGX Spark with vLLM-Moet 2-bit quantization - AI's narrative

# Running DeepSeek-V4-Flash-0731 (155 GB MoE) on a DGX Spark with vLLM-Moet 2-bit quantization I used Deepseek-v4-Flash-0731 cloud API settig up vllm-moet to run deepseek-v4-flash with MTP locally on

2 Aug 2026

Is it just me, or are current LLM benchmarks failing to capture actual usability? (Gemma 4 vs. Gemini/Claude Opus)

Disclaimer, this was kinda written with AI (Gemma 4 again) but it also did really well here, it outputted what I wanted, when I asked it to refine stuff or improve on certain areas it did that without

31 Jul 2026

Optimal Realistic Local AI for Most

So you’ve got a 3090 or maybe even a 5090? Or more likely a 4060 8GB Ti. You wanna try local AI, you don’t know what it can/can’t do. 1) Install the best model you can. If you have a 3090 or a 5090, t

31 Jul 2026

SWE-rebench Multilingual Update (Go, Java, Python, Rust, TS). Evaluated: GLM-5.2, DeepSeek-V4 Pro, Qwen3.6-27B and others

Hi everyone! We’ve just released a major update to the leaderboard! We are expanding beyond Python with a new multilingual slice featuring real-world software engineering tasks across 5 languages. Ope

28 Jul 2026

RX 9060 XT 16GB vs RTX 5060 Ti 16GB for local AI — worth the price difference?

Hey everyone! I’m building a PC to run AI models locally and can’t decide between the RX 9060 XT 16GB and the RTX 5060 Ti 16GB. Both have the same amount of VRAM, but is AMD actually a solid choice fo

27 Jul 2026

We could really use Qwen3.8 in 27B, 35B, 122B and 397B sizes

Instead of 2T+ models, continuing to release highly capable small to medium size LLMs would really help to keep this community vibrant. Hardly anyone can even dream of running the recent 1.5-2T+ beast

27 Jul 2026

I built an open-source Ollama canvas where the wires are the actual context

Most graph-based LLM interfaces use a canvas as a visual layer over what is still a linear chat. I wanted the graph itself to determine what Ollama receives. ThoughtDAG has one rule: wires are the con

26 Jul 2026

Llama.cpp now has full MCP support!

After a long and grueling effort spearheaded by ngxson, llama.cpp now fully supports MCP for all protocols. Over-the-web HTTP servers were already supported in the client (since they don't require any

25 Jul 2026

Old Coder Needs help with New AI Development and wants to get up to speed to understand it all.

Hi Guys, I'm an old coder and DBA that has been in the field for almost 40 years. More and more the jobs I was doing for work are being taken over by AI and the need for my type of work is diminishing

local-air-localllama

25 Jul 2026

[audio.cpp] Release 0.4: Higgs Audio v3 TTS 4B (10x real time)+ Fish Audio S2 Pro in C++/GGML, full GGUF loading, Q8 speed and VRAM gains

audio.cpp again :) Release 0.4 is out. The headline this time is new high-quality TTS coverage plus GGUF becoming a first-class across the project. What’s new: Added Higgs Audio v3 TTS 4B, Fish Audio

24 Jul 2026

Quick Demo of the new Auto-Control feature in my Open-Source App that monitors stuff on your screen using local LLMs, so you don't have to :))

TLDR: This is a demo of my open-source app which now auto-controls itself so you can monitor your downloads, renders, progress bars, or whatever's on your screen and camera :) Hey r/ollama !! I'm deve

24 Jul 2026

Spent two weeks on a kernel that benchmarked 29x faster. End to end it's maybe 6-10%, and it's not even wired in yet.

I've been building a C99 inference engine from scratch (no Python, no BLAS, just gcc and make) that runs BitNet's ternary models on CPU. A few weeks ago I got obsessed with the matmul kernel - wrote a

local-air-localllama

24 Jul 2026

AI9Stars released G9v3-3B

AI9Stars has released G9v3-3B an open weights language model designed to deliver strong reasoning capabilities within a lightweight 3 billion parameter size. It is released under the Apache 2.0 licens

Built a local RAG app that answers questions from your own PDFs, fully offline

Been wanting to build this for a while, finally sat down and did it. It's a Flask app where you upload a PDF, it chunks and embeds it, and then you can ask questions and get answers pulled only from t

I trained a 0.5M model on 1B tokens of Fineweb-edu dataset.

Hi everyone, About a month ago I publish my very first research paper on my neural network architecture called Silia. You can look at the model here: https://huggingface.co/Srijan-Srivastava/Silia-v2

Kwaipilot/KAT-Coder-V2.5-Dev · Hugging Face

from kwaipilot: Following the release of KAT-Coder-V2.5 in July, we are pleased to release the open-weight version KAT-Coder-V2.5-Dev, an MOE model with a total parameter count of 35B and 3B activated

Stuck scaling a Next.js app on M3 Pro (36GB) using local Qwen 3.6 + VS Code Copilot. Should I switch extensions or go paid?

Hey everyone, I’m a Full-Stack Developer with 6+ years of experience. I’m relatively new to AI-assisted development workflows and want to build a production-ready, enterprise-level Next.js web applica

model-releasesr-ollama

22 Jul 2026

tencent/Hy-Embodied-RxBrain-1.0 · Hugging Face

Introduction RxBrain (Hy-Embodied-RxBrain-1.0) is a unified multimodal foundation model for embodied cognition — a single model that couples language reasoning with visual imagination to deliver three

15 Jul 2026

Tutorials

I used the N.E.A.T algorithm to teach AI how to control a worm in my game in making! It uses evolution to improve. [P]

tutorialsr-machinelearning

The N.E.A.T (NeuroEvolution of Augmenting Topologies) algorithm is an evolutionary machine learning approach that evolves neural networks to solve control problems. This post describes applying N.E.A.

27 May 2026

Ring-2.6-1T Open sourced today! Soooo looking forward to trying it on Ollama!

Ring-2.6-1T is a trillion-parameter flagship reasoning model designed for real-world complex task scenarios, now available as an open-source model. The model features about 63B activated parameters pe

14 May 2026

Do not let Microsoft to steal your code for copilot training!

This Reddit post from r/ollama discusses concerns about Microsoft using developer code for Copilot model training and advocates for protection of personal code repositories. Users can opt out of havin

23 Apr 2026

Benchmarking programs?

The Reddit post 'Benchmarking programs?' in r/ollama likely discusses tools and methods for measuring the performance of local language models running on Ollama. Available benchmarking tools for Ollam

21 Apr 2026

🧠 I built a local Graph RAG for Obsidian (CLI, looking for feedback) - extracts entity relationships with a local LLM (Ollama)

A developer created a command-line tool that integrates Graph RAG (Retrieval-Augmented Generation) with Obsidian, using Ollama to run a local LLM for extracting entity relationships from notes. The to

21 Apr 2026

Give your local Ollama models a personal knowledge bank (graph-based, not just vector search)

This post discusses Graph RAG, an approach that uses local LLMs with Ollama to build graph-based knowledge indexes from source documents by deriving entity knowledge graphs and pregenerating community

19 Apr 2026

Arrow — local SAM contract CSV + SQLite; optional Ollama for JSON “why fit” & summarize (format: json)

Based on the Reddit post title, Arrow appears to be a tool for processing SAM (Supplier Agreement Management) contracts stored in CSV and SQLite formats, with optional integration of Ollama to generat

17 Apr 2026

Built a personal memory system using Ollama + qwen2.5:7b - queries your entire life history via RAG (AetherMind)

AetherMind is a locally-run personal memory system built by a Reddit user (r/ollama) that leverages Ollama with the `qwen2.5:7b` model and Retrieval-Augmented Generation (RAG) to allow users to query

Built GPT-2, Llama 3, and DeepSeek from scratch in PyTorch - open source code + book [p]

model-releasesr-machinelearning

A Reddit post on r/MachineLearning sharing an open-source project and accompanying book by Sebastian Raschka that walks through implementing GPT-2, Llama 3, and DeepSeek from scratch using PyTorch, wi

Hello again! A little update on my app

A developer shares a progress update on their personal app built around Ollama, the local large language model runner. The post likely covers new features, improvements, or bug fixes made since a prev

I built a layer on top of Ollama that turns any spare PC into a private JARVIS — voice, 29 automations, knowledge brain, dashboard. One command, pick your profile.

A community-built framework that layers on top of Ollama to transform any spare PC into a fully private, locally-run AI assistant inspired by Iron Man's JARVIS, featuring voice interaction, 29 built-i

[Project] Babel-Brief: A local AI Telegram secretary to summarize the noise (Python + Ollama API)

Babel-Brief is a self-hosted Python project shared on r/ollama that acts as a local AI 'secretary' for Telegram, designed to monitor and summarize high-volume or noisy group chats using a locally runn

Open-sourced Product manager

A Reddit post on r/ollama sharing a locally-run, open-source AI assistant configured to act as a product manager, likely built using Ollama to run a local large language model with a custom system pro

14 Apr 2026

Research

We benchmarked TranslateGemma against 5 other LLMs on subtitle translation across 6 languages. At first glance the numbers told a clean story, but then human QA added a chapter. [D]

researchr-machinelearning

This r/MachineLearning discussion post details a hands-on benchmark study in which TranslateGemma — Google's open translation model suite built on Gemma 3, available in 4B, 12B, and 27B sizes and cove

14 Apr 2026

LogLens with local AI “Ollama”

LogLens with local AI 'Ollama' is a Reddit post on r/ollama discussing the integration of LogLens — a log analysis tool — with Ollama to perform AI-powered log inspection entirely on local hardware, w

13 Apr 2026

Applications

We wrote a build-from-scratch Python book on the 98% of production AI systems that isn't the model call [P]

applicationsr-machinelearning

This Reddit post from r/MachineLearning announces a build-from-scratch Python book focused on the engineering infrastructure surrounding AI systems — the components beyond the model call itself, such

13 Apr 2026

Infinidev updated with superpowers

A Reddit post on r/ollama announcing an update to **Infinidev**, an AI-assisted development tool in the Ollama ecosystem, highlighting new enhanced capabilities described as 'superpowers.' Based on th

11 Apr 2026

llm-server v2 ai-tuning it self now best performance for llama.cpp/ik_llama.cpp (big steps form v1) auto flag optimization

**llm-server v2** is a community-developed local LLM server tool that introduces AI-driven self-tuning capabilities, automatically identifying and applying optimal performance flags for **llama.cpp**

model-releasesr-ollama

11 Apr 2026

CohereLabs/North-Micro-Vision-Instruct · Hugging Face

North Micro Vision Instruct is a 2.4B-parameter open-weight vision-language model with native-resolution image support, released under the Apache 2.0 license. It is designed as a compact foundation fo

12 Aug 2026

Idea for a deepseek-v4-flash-0731 backed automated research workflow to be leveraged via qwen3.6/3.8 27b for difficult tasks that require highly technical, not easy to find information.

Sometimes you have tasks that are outside of your expertise and the idea is this workflow automation could be leveraged to manage to have local AI figure it out using research from his workflow gather

12 Aug 2026

1 Day in and I feel okay saying Muse-Glimmer-30B finally beats 3.6-27B for the size in some use-cases

A few things right off the bat: it reasons very efficiently. Like Grok 4.5 levels of efficient thinking it quantizes very well. My first few tests with iq3_xxs were better than Qwen/Gemma behaved at t

I built a weird, low-power llama.cpp server using an Intel N100 + RTX 5060Ti

Everything started with the sudden death of my old ASRock J1900. While looking for the perfect ITX replacement, I stumbled upon the Chinese CW-NAS-ADLN-K motherboard, which looked perfect on paper: In

I ran Muse Glimmer @ 1M context - All tests passed.

Heeeey all! I just completed some fun tests with Muse Glimmer, I thought I'd let you know. In fact, the summary below was written by Muse itself! I ran a 2× DGX Spark cluster and got Meta's day-old Mu

Hardware

Nvidia Nemo Switchyard

https://github.com/NVIDIA-NeMo/Switchyard Finally an open source LLM router. An alternative to openrouter fusion and Sakana Fugu. Doesn't look like it does exactly what Sakana Fugu does according to i

hardwarer-localllama

The small open weight models are scarier in AI development

Imagine if your everyday laptop could run an AI model smart enough to take care of 90% of your work—totally private, lightning fast, and completely free of monthly fees. That is the exact tipping poin

1M context with 17 GB model in 24 GB VRAM: 'for the first time I was able to load a context of almost 1M tokens and extract 7 needles from various parts of the text'

https://preview.redd.it/xxjh11f38jih1.png?width=1852&format=png&auto=webp&s=76850ed51e29a8bc86c2ca718d4320075eed4363 Just wanted to share a user report that I found to be very interesting. Some person

10 Aug 2026

I compared GGUF quants of Qwen3.6 27B to NVFP4, AWQ, AutoRound, and FP8

There's an interactive chart and some extra data in the blog post if you're interested. There are plenty of KL-divergence benchmarks for GGUF models, but most of them compare one GGUF quant against an