Search: “engineering”

Everything started with the sudden death of my old ASRock J1900. While looking for the perfect ITX replacement, I stumbled upon the Chinese CW-NAS-ADLN-K motherboard, which looked perfect on paper: In

I wanted to find out whether a huge text-only MoE could be given basic vision without retraining the language model itself. The short answer is yes. I froze DeepSeek V4 Flash and a 417M-parameter Moon

Hey all, I'm serving DSv4Flash 0731 on a cluster of 2x DGX Sparks but am running into constant issues with having almost no RAM (unified memory) left for the OS/cache and I'd love to hear the communit

Questions & Responses(in BOLD) below. Favorite question(s) moved to end of the thread with combined responses(removed duplicates). Be optimistic folks. I'm sure we're getting other models too apart fr

Just wanted to share my agentic coding benchmark run of DSv4F 0731 at both High and Low reasoning efforts (not Max)... I ran a 109-question subset of Aider Polyglot (the JS/C++/Python languages), base

I've been out of these forums for awhile but I figured I would provide a formal update on how this has been going now that it has some operation time under its belt, just to put the information out th

TL;DR: I spent 9 days developing a new quantization method for MLX models and measured 18 variants against each other on a single M5 Max MacBook Pro (128 GB). The result is the best-measuring MLX quan

Disclaimer, this was kinda written with AI (Gemma 4 again) but it also did really well here, it outputted what I wanted, when I asked it to refine stuff or improve on certain areas it did that without

I've been working on an open-source workflow automation platform over the past few months, with Ollama as one of the first-class providers rather than treating it as an afterthought. The goal wasn't t

This was my Bachelor's Final Project: implementing YOLO26n inference completely from scratch using ARM64 Assembly Language and C, without relying on existing inference frameworks. The goal was to unde

If you run local agentic coding harnesses (Aider, Claude Code, etc.), prompt evaluation usually eats up most of your execution time. Every turn re-evaluates thousands of identical prefix tokens_system

Hey HN, Henry & Roman here from Cactus. A small, on-device model is fast and private, but sometimes wrong, but frontier models are getting expensive pretty fast. So, we post-trained Gemma 4 E2B post-t

dam bois we eating good this week ngl, The velocity of the open_weight ecosystem right now is hitting a point where proprietary, closed-source APIs are losing their leverage on compute intelligence. W

This Reddit post from r/ChatGPT discusses a technique for enabling OpenAI's Codex (or similar AI coding agents) to autonomously replicate a user's workflow by observing and learning from their actions

A Reddit user shares their project of building an AI-powered alchemy game inspired by titles like Little Alchemy, using ChatGPT to dynamically generate the result of every possible element combination

This r/MachineLearning discussion centers on Google's TurboQuant, a training-free KV cache compression algorithm released in March 2026 that compresses cache storage from 16 bits down to 3 bits with m

This Reddit post shares an educational PyTorch repository designed to teach distributed training techniques from the ground up, covering Data Parallelism (DP), Fully Sharded Data Parallel (FSDP), Tens

This r/MachineLearning post presents educational PyTorch implementations of FlashAttention versions 1 through 4, designed to highlight the key algorithmic differences across each iteration rather than

A research paper (arXiv:2601.11557) proposes replacing the dominant 'HNSW + float32 + cosine similarity' vector database stack with an information-theoretic alternative that uses Maximally Informat...

A bug was identified in NVIDIA's cuBLAS library where `cublasSgemmStridedBatched` dispatches the same suboptimal `cutlass_80_simt_sgemm_128x32_8x5` kernel for every batched FP32 workload from 256×...

A r/MachineLearning discussion explores the challenge of finding S3-compatible object storage that can both saturate H100 GPU bandwidth and avoid AWS's egress fees during high-throughput AI trainin...

A beginner-friendly tutorial demonstrating how to build a ~3.2M parameter LLM from scratch using Mary Shelley's *Frankenstein* as the sole training corpus, designed to run on Kaggle's free GPU in u...

I am an independent researcher. Due to budget constraints, I shifted my 21B MoE pre-training—which I had originally planned to run on 64 H100s—to a single NVIDIA GH200 ($2.29 per hour), and I have com

A few weeks ago we posted the open-source Graphsignal profiler here: wrap vllm serve / sglang serve / whatever in graphsignal-run and it serves everything it measures as JSON, mainly so an agent can r

I've been investigating some serious ChatGPT reliability issues over the past several days, and my latest tests revealed something particularly interesting: Both GPT-5.6 and GPT-6 showed problems unde

https://preview.redd.it/3ekngxk224vh1.png?width=1080&format=png&auto=webp&s=9bfe5c92033e17d5b2fb4529f1320c99455a138b I built a GPU compiler and wrote an article about it, focusing on concepts shared b

I get a lot of hate whenever I tell people I run a Q4 quant of Qwen3.8 27B with 262k KV 80t/s with vision, all while running 2 screens on windows on the same card. People seem to think I am lying or m

indexes PDFs, Office files, scans, screenshots, audio and video, and finds them by content. A query like 'when did they decide to freeze hiring' opens the recording at that second. * EmbeddingGemma 2

I love Dash2Dock Lite, but Icedman is always a week or two before updates. I randomly ran pacman for the first time in weeks, not realizing GNOME had a major update. I told it to fix it in DeepSeek Ha

I stumbled across this as I was currently having glm5.3 flash run only around 10tok/s basically unusable. With Maya i am running 30 tok/s now. GLM5.3 feels even with this low quant like a much more en

I recently started using local LLM for mostly coding tasks for my personal projects. I also have found local models responding much better for topics such as health, fitness, ergonomics etc. compared

Around 3 weeks ago, I posted on this subreddit about how Jev used my one-year-old architecture, og post: https://www.reddit.com/r/LocalLLaMA/s/VpuMJt577S Then, just after that, I introduced Laya, the

Spent the past month tweaking and experimenting with many different numbers to achieve 30tps. Hardware: -Rx6700xt 12gb vram (AMD) -2x16 ddr4 3200 ram -r5 5600x -llama.cpp vulkan sdk -window11 (no wsl

Been seeing quite a few posts about Strata lately, so I figured I'd give it a shot on my RTX PRO 6000 and I am very impressed. Most of the benchmarks I've seen are using IQ2 or IQ3 quants, so I wanted

Last month I created a portable memory format as a .txt file as a small project. I named it as: Memory Journal, in a shape of a skill card. It's essentially a large prompt wrapped in a text file. Init

Live results, coding benchmarks included, agentic benchmarks included, domains and other classifications filters included. (Spoiler: DeepSeek-V4-Vision-Exp rules, but other models have their rule area

A bit of backstory : i got lucky in August and was able to grab a CMP 170 HX 8GB for around 500 on alibaba. I tried later to get a second one for around 1k but my order got cancelled and prices went t

I've been building ANCHOR, a web automation agent that runs entirely on Ollama. You describe a task in plain language ('Register Maria Silva, IT department, contractor, and save'), point it at a site,

Post title is slightly misleading, I don't think you can get these for $350 each anymore but they're still pretty cheap all things considered. They're Radeon Pro V620's which are older RDNA2 enterpris

TL;DR: For my use case, mradermacher/Signal-3.8-27B-Terse-Coder.i1-Q4_K_M proved out. Ymmv based on your domain-specific tests. Time taken to solve problems compared to frontier models is massive; esp

tldr: Open sourced my prompt to game suite: https://github.com/ndamiano/ai-agent-test, it's MIT licensed, and this runs well on my 5090 with 64gb ram, but any model that can handle tool calls can mana

Liquid just dropped d1-omni-600M and it's a decision model! I ported it to runntime, the WebGPU inference library I'm currently working on. It's plain TypeScript on top of TypeGPU with no WASM and vir

I made AnimeGen, app that allows you to run Anima on your mobile phone. Since last post I added hd image generation, and different aspect ratios. The app is now available on the App Store: https://app

I spent the last few weeks on a hobby research project and just made it public. The idea isn't new (product-key memory, Lample et al. 2019, and Meta's 'Memory Layers at Scale'): give a model a huge ta

Following on from club-5060ti and club-rdna16, I’ve put together Infermeld: a small, open-source Linux companion kit for running one GGUF across an AMD GPU and an NVIDIA GPU, powered by llama.cpp. I’m

I forked Strata and worked with Opus 5.5 with some heavy changes to it to make it work on an IBM AC922 I have access to. The IBM AC922 is a 2018 era beast with two POWER9 20 core SMT4 CPUs that are co