Search: “llm”

According to rumors (spread by myself) Le Chaton FAT will be 26T-a3b and I AM READY for it. Let's be real, I can't afford that many 5060Ti, so I got 12x Gen 4 3.2 TB (two per card). This gives me abou

GPT-Live is so good that I use it almost every day. I've been wanting to replicate it since it was released. My first attempt was to fine-tune Gemma 4 12B to behave like a full-duplex model. Something

So I am in a conundrum and I'm thinking of asking for your opinion for the following: Currently, I have a 5800X3D gaming rig with a 7900XTX with its 24GB VRAM. It seems that for this subreddit, this c

Built my 1st inference machine and have been tweaking models trying to get the most out of my modest hardware. I think I’m at a good place but I’m testing with my own prompts. I’ve looked into some of

Hello fellow local AI people! I took 'you must create your own benchmarks' literally, and built a website for this. How does the end result look like Let's say I want to know which model has most comm

A few weeks ago I posted about if it was worth using a V620 for Comfyui, and was told it likely wouldn't work, at least in Windows 11. And if it did, it would be far too slow and unusable. I decided t

I saw u/danielhanchen's 1-bit Kimi K3 post: https://huggingface.co/unsloth/Kimi-K3-GGUF/discussions/12#6a6a4a90ec74ef13d85d7cf6 and decided to test Inkling-Small and Qwen3.6-27B myself, based on the f

Context: I want to give the community an Open Research (well open under Apache 2.0 clause) - tool that allows everyday users like us to look deeper into the local models we use consistently. Mechanist

Been running a GPU price tracker (https://www.pricesquirrel.com) since March, covering 20+ EU stores, recently added RAM, SSDs and CPUs too. Every GPU tier has gotten cheaper since launch. The RTX 509

I'm really confused about what the point of the prompt injection was (speaking as an author). Is it just a study? I would really prefer that they took action against the AI-generated reviews. Obviousl

I've been building Krasis, an MoE-focused runtime for streaming big models through limited VRAM on NVIDIA consumer/workstation GPUs, and I think this is the most interesting result so far: Ornith-1.0-

Hey everyone, I have been looking into how people source compute for their Inference workloads (and in general). I wanted to understand some specific pain points here. If you've used online services l

At Bitwize, we've been building Logue, a native macOS app for AI meeting notes and writing, and we just open-sourced it (MIT). We're sharing it here because the whole point is that it runs 100% on-dev

Hi everyone, I have a system which I have been using for gaming, R7 5700X, 48GB DDR4, RTX3090 24GB. But I want to use it for Local AI to reduce my reliance on cloud AI providers (mainly usage limits -

I apologize that this question has been asked in various flavors over time, but I couldn't find anything in the posts before that matches the options I have. I have a PC and a mac, both available in m

Everything runs through WebGPU, in-browser or in electron/tauri apps. It's fully portable and supports either Nvidia and Apple Silicon (Metal). The actual kernels are optimized for the specific hardwa

audio.cpp again :) Release 0.4 is out. The headline this time is new high-quality TTS coverage plus GGUF becoming a first-class across the project. What’s new: Added Higgs Audio v3 TTS 4B, Fish Audio

Model quantization has become essential for efficient large language model deployment, yet existing approaches involve clear trade-offs: methods such as GPTQ and AWQ achieve practical compression but

Just saw this on huggingface: https://huggingface.co/ProCreations/grug-27b The benchmarks claim that it's quite a bit better than qwen3.6 27B original and that they reduced the amount of necessary tok

LLaDA2.2-flash is an agent-oriented diffusion language model in the LLaDA2 series. By introducing Levenshtein Editing (with DELETE and INSERT control tokens) to diffusion language modeling, it represe

Full-parameter post-training of trillion-parameter-scale MoE models introduces substantial system-level challenges for large-scale distributed training, including severe memory pressure, non-overlappe

Paper:https://arxiv.org/abs/2607.19058 Code (GitHub):https://github.com/nuemaan/skewadam Hi everyone, I just published a preprint on a new optimizer designed to tackle the massive VRAM bottleneck in M

Efficiency defined as score over active parameters. Removed all the models that were not on the pareto frontier. Yes I'm aware that artificialanalysis.ai aggregate benchmark isn't perfect, but I have

After over a year in development, ExLlamaV3 has had its first production release. Turboderp has been pulling 10 hour days with Fable to bring us this massive batch of improvements. Check out detailed

https://techcrunch.com/2026/07/13/satya-nadella-has-issued-a-shocking-warning-to-companies-using-ai/ Venture capitalists have been warning for awhile that OpenAI and Anthropic are getting access to se

Production low-latency autocomplete implementations employ diverse strategies including inference server optimization (tools like vLLM, llama.cpp, NVIDIA Triton), deployment choices (cloud APIs, on-pr

A discussion evaluating the value proposition of a Beelink Mini S compact computer with Intel N95 processor and 8GB RAM at a €95-105 price point for running a budget 24/7 home server, likely including

This Reddit post from r/ollama discusses concerns about Microsoft using developer code for Copilot model training and advocates for protection of personal code repositories. Users can opt out of havin

This post discusses Graph RAG, an approach that uses local LLMs with Ollama to build graph-based knowledge indexes from source documents by deriving entity knowledge graphs and pregenerating community

Based on the Reddit post title, Arrow appears to be a tool for processing SAM (Supplier Agreement Management) contracts stored in CSV and SQLite formats, with optional integration of Ollama to generat

A guide on disabling thinking mode in Ollama models using the CLI by running models with the `--think=false` flag or using `/set nothink` followed by a prompt. The post likely discusses how to configu

This Reddit thread from r/ChatGPT likely reflects common user confusion around what the ChatGPT Plus subscription ($20/month) includes, particularly regarding model access, usage limits, and how it co

A developer shares a progress update on their personal app built around Ollama, the local large language model runner. The post likely covers new features, improvements, or bug fixes made since a prev

Babel-Brief is a self-hosted Python project shared on r/ollama that acts as a local AI 'secretary' for Telegram, designed to monitor and summarize high-volume or noisy group chats using a locally runn

A Reddit post from the r/ollama community documents a user's experiment running Google's Gemma 4 model locally via Ollama with web search (internet access) enabled, which resulted in the model becomin

No agent maintained moral reasoning consistency across scenarios. Findings from a structured study with 11 agents on classic ethical dilemmas [R]

A Reddit post on r/ollama sharing a locally-run, open-source AI assistant configured to act as a product manager, likely built using Ollama to run a local large language model with a custom system pro

This Reddit thread discusses the daily image upload limits across ChatGPT's subscription tiers, specifically the Go and Plus plans. ChatGPT Plus allows up to 50 image uploads per day and 80 files ever

A community-built, privacy-focused security scanning tool shared on r/ollama that uses four specialized AI agents running locally via Ollama to analyze code or systems for vulnerabilities, aggregating

This research paper explores a depth-recurrent transformer architecture designed to improve compositional generalization in language models by increasing computational depth rather than sequence lengt

A community-shared project posted to r/MachineLearning presenting an interactive tool for simulating and visualizing a variety of AI and ML algorithms. The tool is designed to help users build intuiti

Users building agentic pipelines with Hermes models in Ollama report an issue where the model correctly generates tool call JSON in its response but the actual tool functions are never invoked or exec

A Reddit post on r/ChatGPT describes a situation where someone chose to hire a developer rather than purchasing a Claude subscription, only to find that the developer now wants to use Claude Max — Ant

A Reddit post from the r/ollama community demonstrating a practical experiment in which a user ingested the narrative content of *The Godfather* into a structured knowledge graph and then queried it u

This r/MachineLearning post presents educational PyTorch implementations of FlashAttention versions 1 through 4, designed to highlight the key algorithmic differences across each iteration rather than

I was unable to retrieve the specific Reddit thread content from that URL through my search. Reddit threads often require direct access to load user-generated content, and the search did not return...