I want to run Kimi K3 at home, so I’m trying to make 2.8T-scale experimentation cheaper

Hey r/LocalLLaMA, I’m a retired engineer with a background in distributed computing, currently running a 1-person startup. Like many people here, I’d love to experiment with 2T+ MoE models locally. The problem is that I don’t have an H100 cluster in my living room. So I’ve been thinking about a fairly practical question: If I have to pay the enormous cost of analyzing a 2.8T teacher model, can I do that once and reuse the result for many different compression experiments? That question led me to a design I’m calling Blueprint Distillation (BPD), with BSP-Structured Expert Consolidation (BSEC) as the analysis pipeline. The basic idea is to treat the expensive teacher analysis somewhat like a compiler frontend. Instead of tightly coupling: Teacher → Analysis → 27B Compression I want to separate the expensive teacher analysis from the actual compression: 2.8T Teacher → Expensive Analysis → Compression Blueprint The Blueprint becomes a reusable intermediate representation that can be consumed by different student configurations: text ┌──→ 27B Student 2.8T Teacher ──→ Compression Blueprint ├──→ 7B Student │ └──→ 3B Student The Blueprint would persist things like expert co-activation, sensitivity, similarity and dependency information, along with budget-specific views. The underlying techniques aren't new. Taylor saliency, Fisher-style pruning, dependency analysis, expert clustering, etc. are all established ideas. I’m not claiming to have invented a new pruning algorithm. What I’m interested in is whether the architecture is useful: can we turn the expensive “understanding the teacher” phase into a persisted, inspectable, reusable intermediate representation? If that works, changing the target from 27B to 7B shouldn't require throwing away all the information gathered from the original 2.8T model and paying for another giant profiling run. That seems particularly interesting for people like me who want to run many small experiments rather than one huge experiment. I wrote up the design here: https://github.com/edwardyoon/BSEC The first concrete target is Moonshot AI’s Kimi K3 (2.8T MoE) → 27B, but the idea isn't necessarily specific to K3. There is one fairly significant problem: I don’t have the compute to generate the first Blueprint. The design calls for instrumenting a distributed K3 deployment and collecting routing/co-activation information over a calibration corpus. I’d like to eventually produce the first real Blueprint.json and see whether the idea survives contact with an actual 2.8T model. So I’m posting this for two reasons: - I’d like technical feedback on whether the architecture makes sense. - If someone has access to H100/H200-class compute and is interested in the problem, I’d love to collaborate on the first profiling run. And honestly, if you think this is fundamentally flawed, I’d like to hear that too. This is a design draft, not a claim that I’ve already compressed K3 to 27B. The interesting question for me is whether we can make the next ten experiments dramatically cheaper than the first one. submitted by /u/Ok-Shower7286 [link] [comments]

Source: r/LocalLLaMA | 2026-07-27