Knowledge catalogue
A while ago I asked here how to turn ~5 million court decisions into structured graphs without running an expensive LLM on every document thanks for the advice . I went with the 'small extractor + cla
This Reddit post on r/MachineLearning presents a community-contributed dictionary of contemporary Large Language Model (LLM) terminology, covering terms related to training, fine-tuning, inference, al
This r/ollama Reddit post discusses the combined use of a locally running LLM via Ollama alongside cloud-based AI APIs, exploring how users can leverage both approaches within the same workflow. The d
The LLM is limited to NPC emulation. Game state, world logic, quests, and the authored story are handled by deterministic game systems rather than the LLM. I built it this way because I wanted the fre
I used an LLM to iteratively evolve an optimization algorithm rather than solve the packing directly. Starting from a simple seed solver, the LLM proposes algorithmic changes guided by a scoreboard of
One thing that has bothered me about LLM benchmarks for a while is that most of them are essentially snapshots. A model is evaluated, a score is published, and we tend to talk about that score as if i
DISCLAIMER I ramble a lot and am not good at focusing on documentation and or coherent thoughts. TLDR I did things I made a thing and it works for me. I will try to answer any questions. A few people
hey guys. so I should start with, I'm not s programmer at all. but I'm having a great time letting qwen3.8 27b donit for me. 😃 so far I've just been doing Linux scripts and things for my PC. but my ki
This discussion explores using Ollama to run large language models locally for invoice processing while maintaining control over data. The thread likely addresses selecting appropriate smaller models
after 3 months and 800 burned... Unbounded Labs is proud to introduce Bart, our vintage LLM: 2.82B parameters trained from scratch on 20.1B tokens of English written before 1931. You can talk to it ri
after 3 months and 800 burned... Unbounded Labs is proud to introduce Bart, our vintage LLM: 2.82B parameters trained from scratch on 20.1B tokens of English written before 1931. You can talk to it ri
An AQuA research loop let one LLM write an intraday trading feature and a second LLM review it for causality. The feature divided the volume seen so far by the day's final total volume. At prediction
similar to video upscaler LLMs, what's the bleeding edge right now on something similar for audio music? the idea is you feed it in with a low bitrate MONO old soundtrack, and the LLM then recreates i
**llm-server v2** is a community-developed local LLM server tool that introduces AI-driven self-tuning capabilities, automatically identifying and applying optimal performance flags for **llama.cpp**
Disclaimer: I work for NuMind, the company behind this plugin. TLDR: Text/image (simili-)embeddings from the LLM you already have loaded :-) I've been working on making it easier to obtain embeddings
This was fun. The base images are made with Kroma, the videos with Minimax h3 and the text-to-speech with Microsoft Vibevoice. The text message screen recording thing is made by an LLM agent (GLM 5.3)
Apple Silicon Macs on macOS 26+ come with a small LLM built in. No download, no API key, and nothing leaves your Mac. Why I built it I was making a tool that writes API docs from code, and I didn't wa
Hi everyone, I'm currently working on optimizing an LLM setup at my company and I'm trying to identify where the actual performance bottleneck is. Our current setup is roughly: LLM: NVIDIA Nemotron mo
This r/MachineLearning post discusses how self-hosted LLMs that comply with prompt injection attempts — effectively following malicious or overriding instructions embedded in user input — represent a
I spent the past year independently building Slow Vale, an LLM-driven life simulation. The Chinese server now has 800+ AI residents sharing one continuously running city. This is an engineering write-
You know the thing where you ask a chatbot what's wrong with your plan and it goes 'there are three problems here' and problem three is clearly the runt of the litter? There's a word for that now. An
**mcpstrike** is a tool shared on the r/ollama subreddit that integrates a locally-run LLM (via Ollama) with an MCP (Model Context Protocol) server to autonomously execute penetration testing workflow
A Reddit post in r/ollama showcasing a newly built LLM-powered penetration testing tool, likely leveraging Ollama to run open-source language models locally for security testing workflows. The tool is
This Reddit post from r/ollama documents a community-discovered, undocumented workaround for getting IPEX-LLM to successfully run modern Ollama models — specifically Qwen3 and Gemma4 — on systems powe
This r/ollama post describes a technique where an LLM is used to automatically tune its own llama.cpp runtime flags — such as parameters related to GPU offloading, KV cache quantization, batch sizes,
A beginner-friendly tutorial demonstrating how to build a ~3.2M parameter LLM from scratch using Mary Shelley's *Frankenstein* as the sole training corpus, designed to run on Kaggle's free GPU in u...
Three weeks back , i posted SHADOW-250M here. It got 360 upvotes, 293 on r/LocalLLaMA and 94 GitHub stars. Thank you. That model was 60 MB, ran around 400 tok/s on CPU and could retrieve records from
my question about LLM performance We see a lot of posts about token prediction, token generation per second, etc. But is it really the metric? I can see that DeepSeek V4 Flash 0731 (with DSPark; mac s
Demand for low-precision inference, including NVFP4-based approaches, has grown as large language models are increasingly deployed in latency and cost constrained production environments. Quantization
LLMs handle speech well once you run speech-to-text. They don't hear the rest: a bird outside, a glass breaking two rooms away, a smoke alarm two floors down. I've been working on an experimental open
Signals V2 is an LLM-free analyzer tool that evaluates live agent execution trajectories by scoring them as OpenTelemetry spans, enabling developers to monitor and trace AI agent behavior without rely
This post discusses upgrade recommendations for improving local LLM inference performance on a gaming laptop with an RTX 4080 (12GB VRAM) and 16GB system RAM. Community members likely shared options f
Haven't seen posted here. The research project matters because it proves you can pull trustworthy decisions straight out of an LLM's internal state in a single forward pass (no generation loop, no par
I mean , DeepSeek, Kimi, GLM, MiniMax -- the list of Chinese LLM's is such a long freaking list. As American's -- why don't we feel .. a little funny .. about being so far behind? Chinese entrepreneur
Building a local voice chat pipeline: Struggling with fast, natural PT-BR conversation models I'm currently trying to build a local AI voice chat system that can use tools via an LLM. I'm testing tw
I run local models through Ollama and kept hitting the same quiet problems, so I wrote a scanner for them. llm-doctor checks your model stores (Ollama, LM Studio, Hugging Face, MLX) for duplicate weig
I released KaozKit this week, a Swift library for running LLM agents written in JavaScript inside a Mac app. Built from the start to keep everything local. Providers: MLX with Hugging Face model manag
Hello everyone. I felt current LLM benchmark harnesses hand you headline numbers but offer no tooling to see how models actually answered each question (they dump everything to a JSONL or Parquet file
Otaku is an LLM frontend, primarily designed for roleplay, an alternative to SillyTavern and the like. However, It also works for general-purpose chat with local backends (including Ollama) or cloud m
This was my first serious attempt at tuning a local LLM. I started because Qwen3.8-27B IQ3 was fast on my RTX 5080 but the coding quality disappointed me, and the Q4 profiles I tried in LM Studio were
I'm looking for a small LLM to act as a Linux command assistant. I will use llama.cpp. use case: - User asks in natural language. Model outputs only the shell command - Should work well with no reason
New/Old benchmark that provides a lot of answers for local LLM. I present to you a new test that I developed somewhat by accident: https://huggingface.co/cHunter789/Qwen3.8-27B-i1-IQ4_KS_KT-GGUF/tree/
hey so im currently working to integrate a locally ran llm to control home assistant (through tool calling, no direct connection) so i can use a voice input without needing very specific commands. Its
Hello all, I'm fairly new to the whole Llm space and could really use some guidance from people who've done similar work. Im building a pipeline to extract structured data from large-batch of invoices
A few months ago, I had the idea of making a LLM from scratch as a personal project (for learning and partly for improving my resume). Since I learned a lot from other posts on here over the past year
Hey LocalLlaMa, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots a
LabyrinthBench measures the thing that actually kills long agent runs — whether a model can still use what it learned twenty turns ago — deterministically, with no LLM judge, on your own hardware, wit
I was refreshing my youtube and found out my favourite reviewer uploaded a battery test of 78 smartphones: https://youtu.be/MpgUFrsIWSQ the author said they started using robotic arm to simulate a per
curious how people here approach buying high-end/workstation cards (RTX 6000 Ada, 5000 Ada, etc) for local LLM work, do you actively watch pricing/timing on these specifically, or is the consumer 5090
Hey r/LocalLLaMA — maintainer here, obviously biased. OpenSmith is an open-source Python tracing tool for LLM pipelines. The idea: drop u/trace on any function, run opensmith ui, get a full local dash
Hi everyone! Over the past five months I've been working on DKV (DifferentialKV), an open-source project exploring KV-cache compression for long-context local LLM inference. The goal is to reduce KV-c
A developer expanded on Andrej Karpathy's LLM Council concept by implementing an enhanced system with Docker containerization, Model Context Protocol (MCP) integration, skill modules, web search capab
A discussion on r/ollama exploring which high-performance LLM models can be effectively run locally on standard laptop hardware , likely covering model size comparisons, hardware requirements, and per
Research demonstrates that LLM-based agents can generate functionally correct patches that pass all tests while still containing security vulnerabilities, challenging the assumption that test-passing
This discussion explores which language models work best when combined with RAG (Retrieval-Augmented Generation) for building a locally hosted LLM focused on survival and off-grid living topics. The t
claudely is a tool that enables users to run Claude Code against local LLM providers such as LM Studio, Ollama, or llama.cpp while preserving their existing Claude configuration. The tool allows devel
Trooper v2.1 is a tool that provides automatic fallback functionality from cloud-based LLM services to local Ollama instances when cloud quota limits are exceeded, incorporating context compaction to
This post compares TensorRT-LLM and llama.cpp (GGUF) as inference frameworks for running coding agents like Cline and RooCode on RTX 5090 GPUs, examining the tradeoff between inference speed and VRAM
A developer created a command-line tool that integrates Graph RAG (Retrieval-Augmented Generation) with Obsidian, using Ollama to run a local LLM for extracting entity relationships from notes. The to
A discussion exploring the 'sweet spot' of using Ollama's hybrid Cloud + Local setup, where a reachable Ollama host serves as the control point for both local and cloud models . The post likely covers