Search: “llm”

A discussion on r/ollama exploring which high-performance LLM models can be effectively run locally on standard laptop hardware , likely covering model size comparisons, hardware requirements, and per

Research demonstrates that LLM-based agents can generate functionally correct patches that pass all tests while still containing security vulnerabilities, challenging the assumption that test-passing

This discussion explores which language models work best when combined with RAG (Retrieval-Augmented Generation) for building a locally hosted LLM focused on survival and off-grid living topics. The t

claudely is a tool that enables users to run Claude Code against local LLM providers such as LM Studio, Ollama, or llama.cpp while preserving their existing Claude configuration. The tool allows devel

Trooper v2.1 is a tool that provides automatic fallback functionality from cloud-based LLM services to local Ollama instances when cloud quota limits are exceeded, incorporating context compaction to

This post compares TensorRT-LLM and llama.cpp (GGUF) as inference frameworks for running coding agents like Cline and RooCode on RTX 5090 GPUs, examining the tradeoff between inference speed and VRAM

A developer created a command-line tool that integrates Graph RAG (Retrieval-Augmented Generation) with Obsidian, using Ollama to run a local LLM for extracting entity relationships from notes. The to

A discussion exploring the 'sweet spot' of using Ollama's hybrid Cloud + Local setup, where a reachable Ollama host serves as the control point for both local and cloud models . The post likely covers

A Reddit post in r/ollama showcasing a newly built LLM-powered penetration testing tool, likely leveraging Ollama to run open-source language models locally for security testing workflows. The tool is

This r/ollama Reddit post discusses the combined use of a locally running LLM via Ollama alongside cloud-based AI APIs, exploring how users can leverage both approaches within the same workflow. The d

A developer shared a self-built local LLM client on the r/ollama subreddit, distinguishing it from the many existing options by emphasizing a polished, well-designed user interface and a complete abse

A Reddit post on r/ChatGPT in which the author shares their findings after a year of testing and comparing LLM subscription services across different price tiers (e.g., 20/month, higher-cost tiers). T

The search did not return the specific Reddit post. Based on available context from the search results and the post title itself, I cannot accurately summarize the Reddit thread's content without r...

https://preview.redd.it/7sknv6omnguh1.png?width=1023&format=png&auto=webp&s=4aef1dc464d008b00f104519eb5399ba6132ad69 Want to try? Just go here and click 'New seed' a lot. You will get different storie

Working on a resource dashboard, rich logs, lightweight 64mb cap, all local, scales on network API endpoints via collector, supports multiple engines (llama, strata, custom cuda engines, unsloth, LMS)

Hey everyone, I’ve been working on a game called Anyworld. It’s a browser-based multiplayer (single player also supported) text adventure inspired by the early days of AI Dungeon, especially its brows

Hey guys - half a tutorial/guide, and half an I built this, so I went for resources. This is something that I created for myself recently when I couldn't find any good existing solution. I wrote this

Forlinx Embedded has listed an M.2 AI accelerator card based on Rockchip’s RK1820 and RK1828 processors, providing 20 TOPS of INT8 computing performance and up to 5GB of integrated DRAM. The module us

I want to try using AI to find possibly overlooked interventions for irritable bowel syndrome (IBS) and create a cocktail of interventions that AI thinks would give the highest expected benefit, attac

github repo: https://github.com/athulg93/vscode-localllm Alright, So i have made some major updates to this extension Update the git flow: Found out that there were some issues and the / part was a me

I have been anti-low quant. I can feel the difference, I swear. I am also anti-quantized KV cache. I have been burned there - I think the tradeoffs compound, especially at longer contexts. Due to this

I'm trying to find out whether back-and-forth between two different LLMs improves task success beyond strong alternatives under a controlled resource budget — and I'd rather hear that it's already set

I'm at a crossroads with two grad school options that would take me in somewhat different research directions, and I wanted some general advice on these fields, their growth, and industry alignment. I

Just a reminder for those out there itching to get into local llms - don't let FOMO or 'gear acquisition syndrom' take over. No matter the hobby, it's so easy to get stuck in a trap where we buy more

I was running Ollama with Maxwell 1500, using a 65,536-token context window, through Zoo Code inside VS Code. I asked the model to analyze a folder containing some pretty simple code. It spent around

If you haven't checked the paper: https://arxiv.org/abs/2608.09867 TLDR: the authors show that you can swap out the 'encrypted' reasoning of the biggest model, like Opus, Sol, and put them into weaker

Most agent memory setups run a model call on the way in. Something reads the turn, decides whether it's worth keeping, rewrites it into a 'memory', tags it with a type and an importance score. That's

This post was not written by a clanker. Hey guys, I'm a comp sci major who wanted to introduce a cool project I built for quantizing models to ternary (1.58 bit) with as minimal of loss as possible, a

This handbook covers the technical aspects of running large language models efficiently at scale, focusing on GPU optimization techniques including GPU internals, key-value (KV) cache management, batc

A Reddit post from r/ollama in which a user shares their experience running a 31B parameter model locally using Ollama, reflecting on the surprisingly demanding hardware and infrastructure requirement

Thursday, October 8 — 15:00–19:00 UTC Access is free and the API is OpenAI-compatible. During those 4 hours, everyone will be asked to use it actively at the same time—coding, chatting, agents, long c

Alright so I'm pretty new to all of this, but today I got a pretty solid crash course in exactly how these LLMs work. I wad quite surprised to find out that there's zero 'thinking' involved in any of

Hello Everyone! Good news for anyone who can compile code on apple, you can now experience using a AMD Radeon AI Pro R9700 on a MacBook Pro or Mac mini, Mac Studio or any OSX variant as long as it has

You can simply run any GGUF with llama.cpp with n_predict=1 and n_probs=10, disable reasoning, and prompt it such as 'If the following email is spam, respond with 1, if not spam, respond with 0. Do no

With the recent Nvidia Huggingface acquisition and frontier AI labs screaming about safety and putting guardrails everywhere, I think it's important that we have local models that aren't affected by a

Lately because of the current hardware shortage, unfortunately or fortunately, we can’t just throw infinite cloud compute at our problems, but we’re forced to actually care about what’s happening unde

https://github.com/Sayitobar/chat_llm_cookbook This is my personal project that took several months. I wanted to see whether a 2B small local model could simulate a six-person group chat trained & ran

The fix: run the server detached/headless instead of keeping it attached to a console window. RTX 5090, ~27B NVFP4 model via ninfer: Terminal focused: 130–200 tok/s Terminal unfocused: 50–60 tok/s Cli

Anyone who has a 3D printer and get use of it finds it incredibly useful for those odd jobs around the house, a missing bracket, a cable router, steam deck holder and so on. In the past if I was missi

I've updated my auto_demo_scener project with Ninfer support and a “rewrite based on video” feature that I thought you might find interesting. The project is basically an endless demoscene machine. A

Github link: https://github.com/thatblend/LLMPSP I wanted to see what the PSP can theoretically handle and I got my answer - a 90M model is about the max it can do without atrocious inference speeds.

The job of doing inference is not the same for every model ou there, as it depends on the architecture and configurations and the available machines. Many times we would not have the right machines to

So, ideally for this thread we exclude the ones that everyone on here is already well aware of and discussing on here a lot, like N-gram, quantization improvements, MTP, D-flash, and D-spark, since th

I’ve been working on a small project called gemma4.c. The idea is pretty simple: you can download a modern language model, compile one 700-line C file, and have it generate text on an ordinary CPU. Th

Full prompt I gave to Claude Opus 5: can you use whatever resources you like, and python, to generate a short 'youtube poop' video and render it using ffmpeg ? can you put more of a personal spin on i

Disclaimer: I'm the developer of this project — sharing because it might be useful to others running self-hosted AI. (Full transparency, as Reddit's self-promo etiquette expects.) xSignalBot is an ope