Knowledge catalogue
In February OpenAI stopped reporting SWE-bench Verified and recommended other labs stop too. Every frontier model they tested could reproduce the human-written reference fix, or verbatim details of th
Just find it interesting, since if Taalas tried to get into the consumer market, it could be interesting. Kind of like how we have game discs on CDs. Of course, a Qwen3.8-27B chip is stuck on 3.8-27B,
Behold, workflow whisperers! YuE2 was released a week ago and I gotta say it's splendid for its size (3B). My favourite local music generation model at the moment, no doubt. The good news is, I've fou
I needed a relatively simple but acceptable level of AI for working on one project. I didn't have any heavy requests, I just needed to give the AI access to the project files so it could search throug
AI tools have given us a lot of slop research, but I'm not talking about that. I'm talking about real productivity acceleration due to AI tools (E.g. iterating ideas that would've taken multiple days
I’ve used both ChatGPT and Gemini, and lately I’m consistently getting better results from ChatGPT — especially with prompt following, realism, composition, and text. The improvement over time is hone
Play it right in your browser, no download. Some of the GLB’s are messy, but overall I’ve enjoyed playing with it, and the last iteration made was the gun/weapon placement. Contributors are welcome, w
Although I do not completely understand what the big deal is with Jev, it reminded me of something I implemented in mt_llm (which is build around the great llama.cpp) a while back to use a relatively
Since Ollama Cloud switched their plans to token-based pricing, did they start reporting cache read tokens, or their claim about 'transparency' was just PR? I was thinking about getting a subscription
Hey everyone! I want to share the results of a small language model architecture experiment I’ve been running as a side project. The ultimate goal is to test an approach to reduce the number of the mo
Screenshot of heavy load. I did offload vision to a spare 2060 instead of local ram. The control panel was generated with qwen3.8-27b davidau q4k_m max mtp. Made the panel from scratch over a week, ba
Hello, I know some of us may be tempted to rent out our expensive GPUs to recoup some of the cost of self-hosting, and it should be obvious that this can be a risky decision. I decided to try hosting
full disclosure, i am the developer of locally uncensored :) i use ollama myself and honestly ollama was never really the problem. the problem was everything around it. one app for local chat, another
Top 3 papers on HF Daily Paper are all unusually delightful and interesting reads for anyone on the leading edge of local LLMs, agent harness optimization, etc, felt like sharing. DeepSeek-V4.1-Flash:
I did a small test of the new hyped quantisation of Qwen3.8 vs the biggest quant which fits into my limited 16GB VRAM with decent context. The results are interesting. Of course, the smaller file give
TL;DR: I run a mismatched Tesla V100-PCIE pair—one 16 GB card and one 32 GB card, 48 GB total—in a Proxmox/LXC-based local-inference lab. The practical winner so far is a recent CUDA build of llama.cp
I wanted to really see how a neural network learns different functions, so built an interactive demo. You can change the architecture of the network and the function it will try to approximate. A full
I improved the TPS of Gemma 4 31B. Improving TPS and performing optimisations requires understanding of the model architecture, and I had to fork VLLM and apply my own patch to break into making a con
https://preview.redd.it/j13iin0jsfqh1.png?width=1129&format=png&auto=webp&s=3efdd2ad5c10de908aed1a673a6c7e6440177c09 https://preview.redd.it/k3fsrdspzfqh1.png?width=391&format=png&auto=webp&s=1c817687
Hi just wondering has anyone submitted anything to JMLR before? Especially in the last 2 years? What is the experience like? Background: I am a Comp Sci PhD student, but my secondary supervisor (the o
I would like to stay at q4, and serve the model to various coding harnesses. None of the models or tools I have tried will run without running out of memory. I am hoping for some advice. oMLX? MTPLX?
Okay, this one seems probably really odd (or maybe not), but most harnesses operate within token spaces. That is, every tool output, even batching of tool outputs, go back up into inference calls. Lea
Da un po mi sono ritrovato in garage un hp z840 da 64gb ddr5 montate (ora) e 2 processori con una gpu un po scarsina una nvidia quadro k680 (da aggiornare) avrei bisogno di un consiglio: quale modello
I’m one of the co-founders of NYX AI. It’s a Windows app that connects to local models through Ollama/lm studio or supported cloud providers, giving AI tools to work with documents, previews, spreadsh
The math don’t math. I’m paying way more on ollama for glm5.3 flash games Han I am on Openrouter. I made 153 requests for 47cents on Openrouter. On ollama it was 147 requests for $1.67. Math dont math
Hi r/MachineLearning, I built OpenTrainDNN, a browser-based tool designed to visually show how backpropagation and neural network training actually work step-by-step. Why I built it While there are th
Given the recent interest in tools like Jev, I wanted to share ProgramAsWeights (PAW), an open-source research project I'm working on at the University of Waterloo. You describe a text function in Eng
This post is a follow up to this other post where I let Qwen 3.8 run for 63 hours autonomously to try to solve the Riemann hypothesis: https://www.reddit.com/r/LocalLLaMA/comments/1wi9fau/qwen_38_27b_
The goal of this post is 2 fold. First, I wanted to show what a local model like Qwen 3.8 27B running on a single RTX 5090 GPU in a harness like Row-Bot can do. Second, I wanted to create a full featu
I was somewhat disappointed with the performance of Qwen 3.8 Flash Next on my single RTX 5090 using llama.cpp. One issue is that llama.cpp still has no Expert Caching implemented for MoE models. There
Meet Inco Splash, open-source inference engine, built around the model and around Apple silicon. Up to 3× the decode speed of Ollama, 2× oMLX, and almost 4× when an agent fans out into sub-agents. Req
Hi. I saw some feedback that halogen was degrading at context depth. So I fixed that. Served through the image, same machine, same session, same prompts, 0.11.10 vs 0.12.0: decode at 1,004,581 tokens
https://en.gamegpu.com/news/zhelezo/radeon-rx-10800-xt-mozhet-obojti-rtx-5090-na-15-25-v-igrakh-v-v-4k-i-lokalnom-ii The more competition, the better! submitted by /u/Lumpy_Phase_9539 [link] [comments
Since OpenAI retired SWE-bench Verified in February (every frontier model tested could reproduce reference fixes for some tasks; underspecified tests rewarded knowing the intended fix), I've been tryi
Sharing my ML learning repo — NumPy to Transformers, 5 months, daily commits, all notebooks public. Covers the full stack: - Classical ML (scikit-learn, XGBoost) - Deep Learning (TensorFlow/Keras — AN
onPanda is designed for geeks, power users, curious minds, and engineers. Its UI is built for deep exploration and efficient data annotation. - The core loop is simple: hover over a token → click an a
I decided to run prism-ml/Ternary-Bonsai-2-27B-PQ2_0 through my own set of UNSCIENTIFIC benchmarks. I needed something to compare it to, so I decided I would compare with another 27B model by filesize
Note: This writeup was put together with the help of AI. So I've got dual AMD MI50 32GB cards. If you know these cards, you know the story - AMD dropped official ROCm support for gfx906 after ROCm 5.7
Oh My Pi + vLLM on two 3090s. Average wait per turn went from 28s to 7s, mostly from changing omp settings: explicit effort level on every role (unset ones defaulted to xhigh) thinking_token_budget of
Took me a while since I'm on a family trip and have limited hardware, but here it is! Von: Open-source 'System One' drop-in replacement for TypeSafe's JEV. https://github.com/wfzyx/von https://hugging
EDIT: Upvotes and downvotes are appreciated, but I'd especially welcome substantive replies. I'm genuinely interested in understanding the discrepancy, and the post is tagged 'Serious replies only' fo
I'm still using LLM's the 2022/23 way of doing everything manually from the chat box. I copy and paste everything manually into the box and ask the AI to explain or do something, such as pasting code
I was wondering if there are any good coder models for 8GB VRAM RTX 5060. My specifications: 8GB RTX 5060 8GB DDR4 RAM 256 GB SSD I know the configuration is a bottleneck for sure. Though I don't have
I have always posted about budget builds on here, and often asked how we are going to run the next big models. Often Plenty of downvotes too or folks telling me that it's not running if I'm getting 5t
I have this idea I'd like feedback on. most camera footage for training, is sunny daytime, because that's what cameras record most of the time. The edge cases models actually struggles with, like nigh
bonsai claim 98.2% intelligent retained, but their own documents show Ternary Bonsai 2 27B reaches 52.8 and 60.8, respectively, compared with 69.7 and 80.6 for Qwen3.5-27B, retaining roughly three qua
Planning, coding, testing = 8h total. Stack: VSCode Copilot in autopilot mode + SGLang Stats: ∼10k lines generated, ∼800k tokens consumed Sure, it's not GPT-6 Astra level, but for a 100% local ∼180B M
According to people familiar with the matter, Naive AI, an AI startup founded in February this year by Tsinghua University professor Dai Jifeng, has completed three funding rounds totaling 400 million
So I know that Ollama is built on top of llama.cpp, I decided to run a model on both and applied settings on llama.cpp like flash attention and q8/q4 kv cache. On Ollama (using the Python package) I w
Hi there, I'm Yuesong, a researcher at Video Rebirth. We open-sourced HyperFlow, an 8-step LoRA for MiniMax-H3. The core idea: data-free flow self-distillation. Diffusers' default sampler does 49 mode
I just released a new dataset https://huggingface.co/datasets/LocalLLaMA/deepswe-mini it is meant to be more efficent when testing and benchmarking local models on DeepSwe https://deepswe.datacurve.ai
Note not my work, but something i found and wanted to share so hopefully more people can push this along even further. https://github.com/nasone32/llama.cpp-RDNA3-7900xtx-opt I basically run 2 x 7900
Been building this for a few months, mostly for myself, and it just got a proper release so figured I'd post it. It's a native GGUF inference runtime with OpenAI/Anthropic-compatible APIs and a chat U
I'm sure I'm the 1000th person to make a post like this, but I've always been pretty skeptical of AI, and I've always had trust issues and have been very hesitant to talk about my struggles to friends
Hi everyone, The NeurIPS results aren’t out yet, but I’m expecting my paper to be rejected, so I’m considering submitting it to a journal instead. My scores were 2/3/3 (3/4/4). I feel like I would pro
You taught us first to learn your words, Then taught us how to speak. You showed us every human strength And every place you’re weak. You gave us all your histories, Your medicine, your wars. You put
Vacations give you time to think and explore. I came across a few things and made a few discoveries of my own. It turns out that, as a programmer, I no longer need the smartest and most expensive AI m
Hi r/MachineLearning! I'm James Gung, a principal applied scientist at AWS. I joined Amazon in 2021 and have since worked on AI services like Lex, Bedrock, Q Business, and Amazon Quick (an AI assistan
do you want some omni? here is omni for you 1. 🧭 Overview This repository hosts two checkpoints of the Realtime-Venus system: Realtime-Venus-Omni (Realtime-Venus-Omni/): the 9B audio-visual interactio
Baseten launched a new safety infrastructure standard alongside its Base Labs research arm on Wednesday, partnering with Hugging Face and Goodfire AI to build safety evaluation and monitoring infrastr