All entries

blank square in ollama local ai app

I have this android smartwatch that has plenty of storage and ram and I wanna try to run some local ai on it, but shortly after opening the app there is this blank square blocking me from doing anythi

Built a local RAG app that answers questions from your own PDFs, fully offline

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Been wanting to build this for a while, finally sat down and did it. It's a Flask app where you upload a PDF, it chunks and embeds it, and then you can ask questions and get answers pulled only from t

local-air-ollama

23 Jul 2026

contrib: allow all AI-generated code in general by ngxson · Pull Request #26012 · ggml-org/llama.cpp

Having read some merged PRs in the past, I know that they were fully written by Claude Code (or similar), so this basically fixes the delusion. But at the same time, we might start seeing more AI slop

CPU-only inference on a Celeron N5095 SBC: 6 models from 0.6B to 8B, benchmarked

I wanted to know how cheap you can go and still run local models, so I ran Ollama CPU-only on a Youyeetoo X1S. It's a single-board x86 machine with a Celeron N5095 (Jasper Lake, 4C/4T, 15W), 16GB of R

DeepSeek Founder’s 4-hour investor meeting: DeepSeek is prioritizing AGI over user growth and commercialisation

A Chinese article compiled 52 remarks from Liang Wenfeng’s four-hour investor meeting. I’ve summarised the most important ones below. DeepSeek has one central objective: AGI. This is not the time to m

Deepseek V4 Flash ~105 t/s on two Nvidia 4090d 48G (ada) in vLLM

TLDR: I (with the help of AI) re-implemented every Blackwell-only kernel (DeepGEMM, FlashInfer sparse-MLA, block-scaled FP8) in Triton, because they simply don't exist for sm89. The performance is 2-3

Research

First ML coding round (HackerRank) at Adyen, what should I expect? [D]

researchr-machinelearning

Hi everyone, I recently received an interview offer for ML position at Adyen, and the first round will be a live coding round on HackerRank. I scheduled it for the end of August because that was the l

FYI You dont need expensive networking for multi-node gpu. 30t/s laguna Q2_K_XL (39.7GB) on 2x4060+1x4060 using a $20 usb->ethernet.

Turns out a regular ethernet cable between 2 nodes can run laguna UD-Q2_K_XL (39.7GB) using a direct point to point network. Interestingly on `nvidia-smi dmon -s pucvmet -d 2`, the inter/intra gpu tra

GPT-5.5 Scores 10.6% on ActiveVision, Humans Hit 96.1% [R]

The interesting finding from a new [arXiv paper](https://arxiv.org/abs/2607.16165) isn't that a frontier vision model failed a new benchmark, that happens weekly, but the specific shape of the failure

I built an open-source RAG chatbot starter that runs fully locally with Ollama (FastAPI + ChromaDB)

I kept re-wiring the same RAG plumbing on every project, so I turned it into a clean starter and open-sourced it. Upload a PDF, ask questions, and get answers with page-level source citations. It runs

I Made a Local Huggingface On My NAS

https://preview.redd.it/u8alj38wr0fh1.png?width=1860&format=png&auto=webp&s=3578776c60d9548a135a018702f44d0fddedd4b0 Little side project I'm doing so I can easily transfer any model I want fast to my

local-air-localllama

I run GLM-4.5-Air (110B) on 16Gb ram consumer machine and Qwen3-30B at 20 tok/s

In the past few months I’ve experimenting heavily and tortured my old 2016 Desktop PC to run the biggest Local LLM I can fit. I documented the whole process and research and I’ve published a repositor

I trained a 0.5M model on 1B tokens of Fineweb-edu dataset.

Hi everyone, About a month ago I publish my very first research paper on my neural network architecture called Silia. You can look at the model here: https://huggingface.co/Srijan-Srivastava/Silia-v2

inclusionAI/LLaDA2.2-flash · Hugging Face

LLaDA2.2-flash is an agent-oriented diffusion language model in the LLaDA2 series. By introducing Levenshtein Editing (with DELETE and INSERT control tokens) to diffusion language modeling, it represe

Interesting reasoning by phi4-mini-reasoning

https://preview.redd.it/joxxvyzhf1fh1.png?width=2386&format=png&auto=webp&s=a55c09d11eb080b83d3b66b59013a09118913c9d Freshly installed, just asked it 'who are you'. Why does this happen? 😄 submitted b

Kwaipilot/KAT-Coder-V2.5-Dev · Hugging Face

from kwaipilot: Following the release of KAT-Coder-V2.5 in July, we are pleased to release the open-weight version KAT-Coder-V2.5-Dev, an MOE model with a total parameter count of 35B and 3B activated

Laguna-S-2.1 'thinking forever' loops seem to be a quantization artifact

If you're running Laguna S 2.1 on llama.cpp and hitting thinking loops because it won't close its </think> tags, you might want to look at your quant before you spend too much time tweaking settings.

LTX Desktop v1.1.0 is out: local generation on Apple Silicon, a built-in LoRA library, video extend, and more

model-releasesr-stablediffusion

LTX Desktop v1.1.0 just shipped with some big updates. Apple Silicon Macs can generate video locally now, there's a built-in LoRA/IC-LoRA library you can browse and apply from inside the app with per-

Model 'distillation' accusations are getting way overblown at this point

The news about Anthropic settling a class action lawsuit for 1.5B over training data isn't just a legal headache for them, it's a massive warning sign for engineering teams relying entirely on closed

MoE models around A2B

There's a bunch of small MoE with around 1B active params, like LFM2.5 8B A1B and Granite 4.0h 7B A1B; and then there are models with 3B+ like Qwen 3.x ~30B A3B and Gemma 4 26B A4B, but those are alre

Ollama on CPU for domain ChatGPT

I am creating a problem in silo wherein there is a custom flavor of centos which our team develops and has debug strings. I have vcenter server where we make machines for QA. There, got a 50 core vcpu

PaddlePaddle/HPD-Parsing · Hugging Face

HPD-Parsing: Hierarchical Parallel Document Parsing We introduce HPD-Parsing, a lightweight (1B) and high-throughput document parsing model built on a Hierarchical Parallel Decoding paradigm. Unified

local-air-localllama

[Paper] SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD

Full-parameter post-training of trillion-parameter-scale MoE models introduces substantial system-level challenges for large-scale distributed training, including severe memory pressure, non-overlappe

PSA on Laguna S-2.1 - Use the updated chat template and GGUF

Link to their official GGUF repo: https://huggingface.co/poolside/Laguna-S-2.1-GGUF/tree/main All the GGUFs received this fix 5ish hours ago - correct yarn_attn_factor to 1.0 (llama.cpp derives mscale

Running Qwen 3.6 35B MoE (Q4_K_M) on a Zeus (Xiaomi 12 Pro, 12GB RAM)

Shoutout to this awesome guy - https://www.reddit.com/r/LLM/s/IDUyU3v9ap Thanks to his project, BigMoeOnEdge https://github.com/Helldez/BigMoeOnEdge, I managed to successfully run a 35B MoE model on j

Sudo authentication fails when trying to access local models folder on Fedora

When trying to access the models folder on /usr/share/ollama, I'm asked to authenticate as sudo, which weirdly enough, fails. I type my password, which I'm sure is correct since I use it several time

Trained a 32B FLUX.2 LoRA on a 24GB AMD 7900 XTX, native ROCm on Windows — full guide + patches

model-releasesr-stablediffusion

TL;DR: Everyone says QLoRA past ~13B is dead on a 24GB card. I got the full 32B FLUX.2 dev transformer QLoRA-training resident on the GPU on a 7900 XTX under native ROCm on Windows (no ZLUDA, no CUDA

TRELLIS.2 can now generate a high-quality 3D asset in under 7 minutes on a 6 GB VRAM CUDA GPU. No ComfyUI Node Nightmare.

local-air-stablediffusion

Not Self Promotion: Just sharing an open-source tool I built to democratize image to 3D creations. For OpenAI Build Week Hackathon I built a free, open-source local Image-to-3D Studio that makes TRELL

What frustrates you about the interfaces you use with Ollama?

What interface do you use, and what’s the most frustrating part of using it? Specific examples would be especially helpful. I’m working on an Ollama interface and want to understand which real problem

🇦🇹 Austria is rolling out a government AI-platform using Mistral models and Open WebUI

This is a surprisingly large real-world deployment: 'GovGPT' is part of Austria’s Public AI initiative, running on sovereign infrastructure (in their BRZ - federal datacenter) with Mistral open-weight

browser-search v2.0 — From the balaclava to the badge: your agent now browses everywhere

Today an AI agent trying to browse the web is like a thief in a balaclava sneaking around a police academy. Site protections block it, challenge it, turn it away. browser-search flips the script: your

Cactus Hybrid: We taught Gemma 4 to know when it's wrong

Hey HN, Henry & Roman here from Cactus. A small, on-device model is fast and private, but sometimes wrong, but frontier models are getting expensive pretty fast. So, we post-trained Gemma 4 E2B post-t

Genesis-Science-1 (GS1), 1T open-weight model later this year from Arcee AI

Today the Department of Energy (DOE) and Arcee AI announced the development of Genesis-Science-1 (GS1), an open model for scientific research. This is a joint effort to bring advanced AI into scientif

GLM 5.2 via OpenRouter/OpenCode

Guys, this is my current opencode.jsonc ``` { '$schema': 'https://opencode.ai/config.json', // Start in plan mode 'default_agent': 'plan', // Use OpenRouter as the provider for GLM 5.2 'model': 'openr

How to configure a custom OpenAI-compatible API in Cursor?

Hi everyone, I have access to a self-hosted (or third-party) LLM that exposes an OpenAI-compatible API. I have both the API URL and an API token, and the provider states that it's fully compatible wit

Instead of panicking about the Hugging Face attack, people need to start questioning OpenAI's insecure sandboxes.

One thing I noticed in American politics, whenever the government wants to push unpopular actions or laws, they often introduce fear to convince the public to support them. This is actually how i view

microsoft/Fara1.5-27B · Hugging Face

Fara1.5-27B is a multimodal computer use agent (CUA) for web browsers, from Microsoft Research AI Frontiers. It observes the browser through screenshots and acts on the user's behalf by emitting struc

MindControl - llama.cpp fork to guide the reasoning process via injection during sampling

The primary driver of this project is that I'd become frustrated with the reasoning behavior of smaller local models such as Qwen3.6-27B (i believe particularly at lower temperatures, and where system

NuExtract3 is now available on Ollama: 4B VLM for document-to-Markdown and structured JSON extraction

Disclosure: I work at NuMind, the team that trained NuExtract3. NuExtract3 is an Apache-2.0, open-weight 4B VLM based on Qwen3.5-4B. It is specialized for document understanding rather than general ch

One encoder, seven heads: what we learned training a unified security classifier with masked losses [P]

We spent the last months consolidating seven separate sequence classifiers into one multi-head model, our apex model, so to speak, and since the weights are now public, I wanted to share what worked a

OpenCode + Ollama + MCP

I installed OpenCode and an Ollama model (qwen3.5) sucessfully connected the model respond in OpenCode but doesn't find My MCP server, i Made one using fastMCP other models like bigPickle and openai m

SkewAdam: A tiered optimizer that cuts MoE state memory by 97% (fits a 6.7B MoE on a 40GB GPU) [R]

Paper:https://arxiv.org/abs/2607.19058 Code (GitHub):https://github.com/nuemaan/skewadam Hi everyone, I just published a preprint on a new optimizer designed to tackle the massive VRAM bottleneck in M

Stuck scaling a Next.js app on M3 Pro (36GB) using local Qwen 3.6 + VS Code Copilot. Should I switch extensions or go paid?

Hey everyone, I’m a Full-Stack Developer with 6+ years of experience. I’m relatively new to AI-assisted development workflows and want to build a production-ready, enterprise-level Next.js web applica

I just wanted a small WebUI with an admin panel… it escalated into a full open-source agent framework runs fully local with Ollama

Let me try to explain this clearly, simply, and neatly. Originally, I just wanted to build a small WebUI adapter with an admin panel, but things escalated over the last few months. At first, I faced t

Hardware

Looking for feedback on my GPU-accelerated Snake AI project [P]

hardwarer-machinelearning

I've been building an AI that learns to play the classic Snake game through reinforcement learning. The goal is to reach high scores while keeping training time as low as possible. The current version

My OCR model mislabels section titles as body text. Is a CRF the right fix, or am I overcomplicating it? [P]

Hi everyone, I'm working on extracting the hierarchical structure of long PDF documents (legal/regulatory text, lots of numbered sections) and would like to gather some feedback on my approach before

Ollama Cloud Max vs. z.ai GLM Max Coding Plan

I've been considering Ollama's Cloud Max plan to replace my GLM Max plan but couldn't find good documentation on how their limits actually translate into GLM 5.2 usage. I know it's by GPU time but tha

Reproducing OpenAI’s “persistently beneficial models” - GRPO trait install barely moves. Ideas? [P] [R]

TL;DR: I’m reproducing the trait-persistence result from arXiv:2606.24014 on one RTX 3090. Before I can test persistence I need to install a trait via RL — and my GRPO run moves the trait only +2.4 po