Search: “context”

Hey folks — I built ContextMemory, an open-source agentic context gateway for apps that already talk to LLMs. The idea is simple: keep your existing POST /api/chat client (Ollama wire format), point i

I am writing a fairly complex C++ windows MFC application. I have a few 3090s and can run F16 Qwen3.6-27B with 256K context and MTP. The quality of code is exceptional with this quant vs its lower qua

I've spent the last year or so playing around with open source MLX and GGUF models on iPhone hardware. Given the limitations in memory, GPU/CPU/ANE, and in turn the context window I've been trying to

LLaDA2.2-flash is an agent-oriented diffusion language model in the LLaDA2 series. By introducing Levenshtein Editing (with DELETE and INSERT control tokens) to diffusion language modeling, it represe

LTX-2.3-22b Union Control is a unified control IC-LoRA (In-Context LoRA) trained on top of LTX-2.3-22b that enables multiple control signals to be used for video generation from text and reference fra

DeepSeek released the full technical report for DeepSeek-V4 on April 24, 2026, titled 'DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence.' The paper details FP4 quantization-awa

Aura is a local-first AI daemon that enhances tools with persistent memory capabilities, claim verification features, and Model Context Protocol (MCP) observability. The project appears designed to ru

This post discusses Graph RAG, an approach that uses local LLMs with Ollama to build graph-based knowledge indexes from source documents by deriving entity knowledge graphs and pregenerating community

A Reddit thread from r/ChatGPT where a user shares an experience of ChatGPT coming remarkably close to knowing a specific piece of information — such as a recent event, obscure fact, or detail near it

A Reddit thread on r/ChatGPT where users noticed and discussed ChatGPT apparently gaining the ability to report the current time — a capability it historically lacked. The base ChatGPT model has no bu

This Reddit post from r/ChatGPT shares a ready-to-use ChatGPT prompt designed to quickly generate a personalized learning sprint — a short, structured, time-boxed study plan tailored to the user's goa

'Voiden' doesn't appear in any search results as a known model or tool in the Ollama ecosystem. Based on the Reddit source and the broader context of the r/ollama community, this post likely discusses

This Reddit post from r/ChatGPT discusses the trade-offs between retrieving documentation via an MCP (Model Context Protocol) server versus using traditional web search within AI workflows. The core c

A Reddit post from the r/ChatGPT community titled 'The confidence lol' likely showcases a humorous example of ChatGPT responding with strong confidence despite giving an incorrect, misleading, or absu

Gguf's: https://huggingface.co/LiquidAI/LFM2.5-24B-A2B-GGUF Free Demo: https://playground.liquid.ai 'LFM2.5-24B-A2B is a 24B-parameter reasoning model with only 2B active parameters, built for fast, p

It's been 4 years since c.ai first hallucination model, and yet, we're nowhere good enough at LLMs in terms of spontaneity/interesting hallucination features. Fortunately, last year brought new archit

My Jetson Orin–optimized engine, little-gemma V1.0, substantially outperforms llama.cpp. Even after exhausting every practical GGUF option, however, it still falls short of the ideal performance level

I love Dash2Dock Lite, but Icedman is always a week or two before updates. I randomly ran pacman for the first time in weeks, not realizing GNOME had a major update. I told it to fix it in DeepSeek Ha

Alright, I learend a few valuable lessons since I posted about a week ago, and I've made more strides with the DDR4x Ascend 96GB 310P cards I purchased, so here we go, meet the new friendly and less v

Hi, Because sometimes you need both speed AND quality, I made my own Qwen 3.8 27B Swift 1.5 quant. It's an 8 bits AWQ + GPTQ quant. Made with love and precision (read the model card) on my 4 RTX 3090,

I was looking into finalizing my daily driver and it was not clear which one to chose. I have a 5060ti 16GB so I picked the following contenders, which seem to be working at a reasonable speed on my h

Hi All: I have an Asus ESC 4000 G3 with 128 GB DDR4 RAM - I tried putting in my V620s but couldn’t put more than 2. Sadly, pivoted to T4s, these are 72 watt passive cards and I thought I could use the

Been seeing quite a few posts about Strata lately, so I figured I'd give it a shot on my RTX PRO 6000 and I am very impressed. Most of the benchmarks I've seen are using IQ2 or IQ3 quants, so I wanted

I’m hitting rate limits on Together AI. For context, I’ve been working on an agentic repository indexing and benchmark generation tool, and I’m running multiple agents in parallel across models like L

My current setup: - single 3090 running turboderp/Qwen3.8-27B-exl3:SC_5.00bpw_H6_V6 - ~150k context, ~60t/s, unknown prefill because I didn't benchmark it (but it is ok) - Intel 12400 CPU with 32GB DD

Last time when I posted on this subReddit to share my work, the response almost made me cry because a tiny demo got so many people talking about this In the past 2 months Ive been training a new model

I built a Pi extension that implements Victor Taelin's OptChat recipe: instead of compacting, every message is logged and summarized into a binary tree. Each turn starts from a fresh context with a bo

Hi r/LocalLLaMA. I'm on the team at Blockway, a small team in Hong Kong (disclosure: this is our model). Today we released Agens Volundr 32B Preview, the first model built on our own hybrid architectu

I have been using Qwen3.8-27b-iq4xs variants for the past few weeks, tweaking parameters and running random tasks, but I dove into using https://github.com/Niko1221/Strata today and it has given me 2~

My setup is two RTX 5070 Ti 16GB cards (the second one is on an OCuLink dock) with 64GB of RAM, Ollama on Windows, and the agent runs on pi in WSL. Until last night the daily model was Qwen3.8 27B UD-

So I tried that miracle engine everyone is talking about. Asked the IQ3_S model to express its opinion on a post from this sub to measure the tps on a long-ish generation: Can you help with the follow

We built an engine, Kyojin, on top of ExLlamaV3 for Strix Halo (gfx1151, ROCm), and packed two 300B-class MoE models so each fits one 128 GB machine. First release, all measured on Ryzen AI Max+ 395.

So everyone has been yelling about how I should be using Gufo instead of halogen because it's open source and it's 'just as good or better'. Checking in on their GitHub (GitHub.com/gufo-org/gufo) got

I put an old RTX 3090 to work as a local coding agent with Qwen3.8-27B, llama.cpp, and OpenCode. Here are the setup and results, including what failed. This is a summary of my own blog post, linked be

I have a long list of my own domain specific eval questions that I run to validate which models I can rely on: coding, coding (numpy/pandas), data analytics decision making, local RAG, and voice assis

I’m building a Polish General purpose legal Model that drafts documents, answers questions using legal sources, and has enough coding ability to handle some automation. The workflow is very tool-heavy

Using only information you actually have from my conversation history, memory, files, or other context available to you, build a practical “operating manual” for me. Do not flatter me, diagnose me, or

Hello everyone! A little while back I posted about LlamAmpere, a fork of Llama.cpp with Ampere-specific improvements (though it is caught up to main and will support other hardware, too). Thank you to

Hi! I'm the author of MoEspresso, which is my way of putting my own ideas about inference engines to the test. A lot of the fun has been trying different design choices, measuring what happens, and fi

We built an inference engine for MoE models that don't fit in VRAM + RAM. Most of the model stays on the SSD, and experts are read as tokens need them. This started as a proof of concept, and poc work

I set out to test portable Engrams and accidentally ended up testing compiled external memory instead. Am I onto something useful or reinventing a known idea? I've been building a small open research

I had Mica (my 4B decision model) and Kev 4B play the same Tetris games, same seed and same piece order, one RTX 3090. For context, this is a side project. Training and all the experiments ran on rent

Built this because I got tired of guessing whether a model would fit in my 6GB VRAM before downloading it, and because ollama.com's library doesn't let you sort by size. What it does: - Benchmarks Oll

Many a praise have been sung on Qwen-3.8, but here is mine. Qwen-3.8 and I had a rocky start, because it thinks so much. Watching it working is painful, so you have to stop doing that. You have to let

A while ago I posted 15 tok/s output and 100-120 tok/s prompt processing with the IQ3_XXS quant on a 12GB RTX 5070 using llama.cpp. Since then I built my own inference engine for this one model and th

Hi, this is the research project I'm working on. I must say, building a GUI harness is way more complicated than I thought. The goal is something like Minecraft. You know, a few simple tools that a us