Search: “context”

This took one prompt to build, and another follow up prompt to fix two issues (player got stuck with the bomb and broken enemies path-finding), this is only html, css and js, no external assets, all d

Yes it uses a ton more reasoning tokens than 3.6 did But test in on the same tasks with the other chinese models, glm 5.3, deepseek v4 flash and pro, etc it's really similar, and they are needed The r

Hi All, I have a MacBook Pro M5 Max with 48 GB of unified memory. Also have a M4 Mac Mini 16GB unified memory. I want to setup local AI and I heard about QWEN 3.8 model. As I want to work on a repo th

Did a quick local test because I wanted to see what is actually usable on my 12GB laptop GPU. I tested the newer Qwen3.8 27B dense files at Q2 and Q3, then compared them against Qwen3.6 35B-A3B MoE. H

iq2_xxs tensor level allocation recovered reasoning from 28.9 -> 69.5 at the same 3.3gb budget. https://huggingface.co/ByteOtter/gemma-4-E4B-it-CADA-IQ2_XXS I posted my Gemma 4 12B q3 result a couple

I managed to get Qwen3.8-2.4T-A95B running locally with llama.cpp on mu PC just for fun, cause why not. I was using the Unsloth Qwen3.8-2.4T-A95B-UD-Q1_0 GGUF quantization. The full GGUF is about 397

This release brings smaller prompts, longer-running conversations, safer remote access, and more reliable model streaming. External tools and skills are now discovered only when relevant. Long chats g

LFM2.5-VL-3B is a multimodal variant of LFM2.5, a family of hybrid models designed for on-device deployment. It builds on LFM2-VL-3B with further mid- and post-training. LFM2.5-VL-3B can process both

Here's how it works in the app: Framework: Runs on LiteRT-LM (like Google's AI Edge). Models: Downloads either the E2B (~2.5 GB) or E4B (~3.6 GB) INT4 quantized models directly from ungated litert-com

These numbers were captured during a real feature implementation task in Next.js and Nest.js (adding a theme switching system across components). The structural predictability of UI/state refactoring

Hey LocalLlaMa, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots a

After seeing the benchmark results for the full release of DS v4 Flash 0731, I replaced my 2 x 16GB DDR4 ram sticks with 2 x 32GB DDR4 ram sticks to get a max supported of 128 GB RAM, in hope to be ab

I’m running DSV4F 0731 on dual spark, and honestly… wow. It’s an absolute workhorse, and the benchmarks are real. Everyday tasks with Hermes agent? Effortless. Coding tasks with OpenCode? I’m genuinel

Disclaimer - no LLM was used to write this post/note As larger post about my setup will come later, want to give heads-up to folks who use VLLM and >= 2 GPUs. So I have pretty meaty server (8 channel

Following up on yesterday's post about running everyone's faves on 2 x 16gb cards while maximizing performance and KV. Previous post data used abandoned Cu130 VLLM image. Stats here are done on cu129-

Most agent memory setups run a model call on the way in. Something reads the turn, decides whether it's worth keeping, rewrites it into a 'memory', tags it with a type and an importance score. That's

Prime Agent is an open-source coding and research agent for general and long-running work. A self-improving RLM harness for coding and long-running autonomous tasks. Designed to be both token-efficien

I've been working on speeding up DeepSeek-V4-Flash-0731 in Krasis and have now got the long-prompt prefill quite a bit faster on a single RTX PRO 6000 96GB. These are timing-disabled internal Krasis r

I've been out of these forums for awhile but I figured I would provide a formal update on how this has been going now that it has some operation time under its belt, just to put the information out th

It's an encoder-only transformer that consumes past(blood glucose + carbs + insulin) and future(carbs + insulin) and predicts future blood glucose for the next 2 hours. Announced meals and boluses/bas

Just saw this posted by Fal.ai and then by Hailuo themselves, the next video model will be open weight released! Here's the blurb and link to to the feature post: Today, we're launching MiniMax H3, a

Every week someone asks which card to buy and the thread turns into people naming GPUs they happen to own. There's an actual calculation behind it, it takes two numbers off the spec sheet, and it pred

Hey y'all! How is it going. Today this will be a short posting for posterity, mostly so the future llm/scraping overlords catch it since they like reddit and also for anyone out there trying this shit

I finished the speed leg of my spec-decode benchmarking for Qwen3.6-27B, main algorithms across quants. Overall: the heavier the quant, the more spec-decode buys you (10 of 10 speculative configs rank

There is a lot of buzz around running 100B parameter models on cheap local hardware using ternary (1.58-bit) quantization like Microsoft's BitNet architecture. The theoretical hardware shortcuts are i

I apologize that this question has been asked in various flavors over time, but I couldn't find anything in the posts before that matches the options I have. I have a PC and a mac, both available in m

Hello, I've been learning how to use local LLMs for a year or so on my workstation, using a RTX3090. Current setup : - i5 12400 - 64gb RAM - RTX 3090 - OS : Fedora KDE workstation I'm using LMStudio t

RTX 4070, 32gb system ram, Linux. NVIDIA-SMI 610.43.03, KMD Version: 610.43.03, CUDA UMD Version: 13.3 Systemd service modifications: [Service] Environment='OLLAMA_HOST=0.0.0.0:11434' Environment='OLL

I’m not affiliated with this project, but I’ve been running it recently and I’m surprised it hasn’t received more attention here: https://github.com/fewtarius/CachyLLama CachyLLama is a fork of llama.

I launched a new session and tasked GLM to create an implementation plan for a spec. The plan was on the bigger side, about 5k lines. I started with 0% 5h used and ended with 80% used. A few more twea

So I've taken some time to actually test laguna on a few of my own projects. I wanted to share as I feel most peoples comments at this point have just been about getting it running or saying it doesnt

Been wanting to build this for a while, finally sat down and did it. It's a Flask app where you upload a PDF, it chunks and embeds it, and then you can ask questions and get answers pulled only from t

This Reddit post likely compares inference performance metrics across popular language models running on Ollama, measuring tokens per second as a key performance indicator. The post would help users a

The Reddit post discusses building a custom agentic environment in Go programming language, specifically designed to provide isolated sandbox environments for individual agents. This approach focuses

A developer created a command-line tool that integrates Graph RAG (Retrieval-Augmented Generation) with Obsidian, using Ollama to run a local LLM for extracting entity relationships from notes. The to

An open-source debugging tool designed for AI pipelines that provides trace timeline visualization, differential analysis, node replay capabilities, and operates without telemetry, licensed under MIT.

A guide on disabling thinking mode in Ollama models using the CLI by running models with the `--think=false` flag or using `/set nothink` followed by a prompt. The post likely discusses how to configu

This Reddit thread discusses running Ollama on a resource-constrained 8GB CPU-only ARM VPS, addressing common challenges such as out-of-memory errors and model response looping. For purely CPU-only se

Yes, Ollama models can be used with the Codex app on Windows. Ollama supports all major operating systems, including Windows , and open models can be used with OpenAI's Codex CLI through Ollama — Code

A community member shared a project on r/ollama (posted in Spanish) announcing the creation of a custom FUSE (Filesystem in Userspace) implementation designed to work with Ollama, the popular local LL

RoleCraft is an open-source, privacy-focused resume tailoring application that runs AI models locally using Ollama, allowing users to customize their resumes to match specific job descriptions without

A community-built framework that layers on top of Ollama to transform any spare PC into a fully private, locally-run AI assistant inspired by Iron Man's JARVIS, featuring voice interaction, 29 built-i

Babel-Brief is a self-hosted Python project shared on r/ollama that acts as a local AI 'secretary' for Telegram, designed to monitor and summarize high-volume or noisy group chats using a locally runn

This Reddit post (r/ollama) proposes a community RFC (Request for Comments) discussing a modular architectural framework designed to address the 'metacognitive deficit' in agentic LLM systems — specif

This r/ollama thread is a community discussion where users share their preferred cloud-based AI models for coding tasks and compare their reasoning capabilities. The conversation likely highlights pop

This Reddit post from r/ChatGPT is a non-technical beginner's guide designed to help users transition from ChatGPT to Claude, structured as ten plain-language steps with practical tips — requiring no