Search: “llm”

152 generations, deterministic grading (exact number/string/JSON/regex), no LLM judge on my 16GB laptop. task type | deepseek-r1:1.5b (1.1GB) | llama3.2:3b (2.0GB) | gemma:2b | codellama 7b arithmetic

Thanks to the community help I finally launched this llm. LM Studio refused to load weight onto second GPU but Unsloth Studio did so everything was done in there. Not a proper benchmark (used PC in pa

Its a custom Swift/Metal inference engine that runs Gemma 4 26B-A4B-IT on M-series Macs with very low RAM. It uses ~2GB instead of ~14 GB. The result is reportedly 5–6 tok/s on an 8 GB M2 MacBook Air

Hello Can you please share the tempature you get on your rtx 3090 under active llm load? Im trying to findout if my rtx 3090's tempatures are healthy or not please share VRAM Tempature only, you can t

Hey folks — I built ContextMemory, an open-source agentic context gateway for apps that already talk to LLMs. The idea is simple: keep your existing POST /api/chat client (Ollama wire format), point i

I am writing a fairly complex C++ windows MFC application. I have a few 3090s and can run F16 Qwen3.6-27B with 256K context and MTP. The quality of code is exceptional with this quant vs its lower qua

GigaChat Audio 10B is an audio-native LLM built on top of the GigaChat 3.1 Lightning text model. A Conformer speech encoder and a modality adapter feed audio embeddings directly into a Mixture-of-Expe

NOTE -> I expect answer from people who actually have experience and strong understanding of these. please give something beneficial. I'm building a SaaS platform in Sri Lanka that handles documents a

mii-llm, an open source AI lab, released Zagreus-0.4B-por, a compact bilingual Portuguese–English language model pretrained entirely from scratch. The model has approximately 400 million parameters an

Hi everyone, I have access to a self-hosted (or third-party) LLM that exposes an OpenAI-compatible API. I have both the API URL and an API token, and the provider states that it's fully compatible wit

Hey r/LocalLLaMA — solo researcher here. I built a new attention architecture and want independent testers. Wave Field LLM replaces O(N²) dot-product attention with FFT wave convolution on a field. Tr

A Chrome extension built for automated web interaction and data extraction that uses Ollama's local LLM capabilities to intelligently navigate websites, scroll pages, fill out forms, type text, and sc

CANTANTE addresses the challenge of optimizing LLM-based multi-agent systems where system-level performance scores are available but individual agent parameters cannot be directly optimized. The frame

This post documents a 10-day project to build a local AI system using open-source tools and models, specifically combining Ollama (a local LLM framework), Gemma 4 (a language model), Claude Cowork 3P,

A Reddit discussion in the r/ollama community seeking recommendations for RAG (Retrieval-Augmented Generation) models, likely addressing model selection for local LLM-based document retrieval systems.

By default, Ollama binds to port 11434 on localhost, which means it's accessible only on your own machine. Ollama has no built-in authentication and should be secured with firewall rules, VPN, or reve

INT3 compression with fused Metal kernels enables large language models to compute attention operations directly on compressed (INT3/INT4) key-value cache representations using custom GPU kernels that

A community member shared a project on r/ollama (posted in Spanish) announcing the creation of a custom FUSE (Filesystem in Userspace) implementation designed to work with Ollama, the popular local LL

A Reddit post on r/MachineLearning sharing an open-source project and accompanying book by Sebastian Raschka that walks through implementing GPT-2, Llama 3, and DeepSeek from scratch using PyTorch, wi

This Reddit post (r/ollama) proposes a community RFC (Request for Comments) discussing a modular architectural framework designed to address the 'metacognitive deficit' in agentic LLM systems — specif

This Reddit post from r/ollama likely discusses how to build and run AI agents locally by combining Ollama — which handles local model serving to keep data private — with Langflow's visual, drag-and-d

A community-shared open source CLI agent project posted to r/ollama, designed specifically for use with 8K token context windows in local Ollama-based LLM setups. Version 0.3 focuses on improving Olla

This r/ollama post discusses a tool for easily linking models between LM Studio and Ollama without duplicating disk storage. Both Ollama and LM Studio are popular local LLM tools, but they store their

**cuba-memorys v0.7.0** is a persistent memory library for AI agents, shared in the r/ollama community, designed to give local LLM-powered agents the ability to retain information across sessions. It

This is a post from the r/ollama subreddit, a community dedicated to the Ollama local LLM runtime tool, where a user is requesting assistance with a problem they are encountering. The post likely invo

This Reddit post shares an educational PyTorch repository designed to teach distributed training techniques from the ground up, covering Data Parallelism (DP), Fully Sharded Data Parallel (FSDP), Tens

This r/MachineLearning post details a project in which a developer trains an AI agent to autonomously play *Resident Evil Requiem* using imitation learning techniques — starting with Behavior Cloning

This Reddit post from r/ollama likely discusses how to configure the Goose desktop application — an open-source, autonomous AI agent developed by Block — to work with Ollama Cloud as its LLM provider.

I ran five coding tasks against nine models on one RX 9060 XT (16 GB). Each run gets a single spec file in an empty directory and has to produce solution.py and test_solution.py, run its own tests, an

Here is how i serve 40 people (10 of them developers) with local AI services. n8n for automatisation whisper and Vexa AI, a bot joins Teams and makes transcripts, n8n later extracts decisions and todo

i'm the maintainer of dodaios (formerly personal jarvis), an open-source ai agent app for the desktop. apache 2.0, runs on windows, macos and linux, no account and no cloud of ours between you and you

Hey everyone, Sharing an educational tool called AskCyph Health Explorer to explore how single-letter DNA mutations lead to disease, how they alter 3D anatomy, inheritance probabilities, and available

Introducing QwenGyre, bringing elastic RL to extreme-long-horizon (XLong) agent training. It launches with two core designs: - Elastic Scheduler: reallocates GPUs between rollout & training as demand

I get a lot of hate whenever I tell people I run a Q4 quant of Qwen3.8 27B with 262k KV 80t/s with vision, all while running 2 screens on windows on the same card. People seem to think I am lying or m

If you build on open-weight reasoning models, you have probably wanted a 'how sure is the model about this answer?' signal that does not cost 5x sampling, a judge model, or a labeled dataset. Hence, w

Most conversations I see on various AI subreddits start from the assumption that AGI/RSI/ASI is possible, we're on the way right now, and that things will be incredibly different afterward. The only q

I am developing a Windows-based local AI virtual spouse application that supports adult roleplay. My goal is to run it without a server or external AI APIs. I am looking for existing open-source modul

Blog Post : ML Drift: Next-Gen GPU AI/ML Inference at the Edge - Google Developers Blog The Google AI Edge Team is excited to announce the open-source release of ML Drift, our high-performance, cross-

Disclosure first: I'm one of the maintainers, this is my project. If you've pointed Open WebUI or a script at a local server, you know the failure modes: the server restarts and every conversation is

Liquid just dropped d1-omni-600M and it's a decision model! I ported it to runntime, the WebGPU inference library I'm currently working on. It's plain TypeScript on top of TypeGPU with no WASM and vir

Hey everyone, I'm looking for a small uncensored model to run locally with Ollama. Nothing fancy. I just want something I can ask everyday questions without it refusing or lecturing me every other mes

My YouTube feed was mostly songs, pranks and celebrity clips, so I built a filter that judges each video title with a small decision model running inside Chrome. Nothing is sent anywhere: no server, n

Meet Bev. She is a decision model (the Jev / Nimble kind: you give her a situation and a question, she gives a probability for each answer), fine-tuned on Qwen3.5-9B to pick the worst answer on purpos

Last time when I posted on this subReddit to share my work, the response almost made me cry because a tiny demo got so many people talking about this In the past 2 months Ive been training a new model

https://github.com/hypersniper05/MCP-Image-Generator-Uncensored Just wanted to share with you guys my workflow converted into an MCP. It's a Qwen Image 2.1 Uncensored MCP with a few extra models I fou

Hi, I'm on M4 Pro 64GB Mac, I'm using Qwen3.8 27B for coding related stuff, but it's quite slow. I want a fast model that will be good when it comes to reasoning, medical stuff, would be nice if it co