Toy project: a chat title model that fits in 5 MiB of ram

Not even sure if I'm allowed to post this, what with the "completely/primarily LLM generated copy" rule (the post itself is fine, but the repo/model I'm sharing definitely is, whoops) and the whole limit self-promotion thing, but it's just a toy I made that is trying to solve a niche I haven't seen much models tackle. I don't really want to put more time into it, but hey, maybe someone will find it useful, and I like open source, so here it is. I'm usually just a lurker in this sub :) To be honest I coded basically nothing, an LLM wrote almost all of it while I nodded along. It works tho lmao. (I'm actually a programmer, this project just wasn't worth getting into too deeply. I still learned a few surface-level things about how these models work, so that's neat) TinyTitle is a tiny model (~1.8M params) that turns a chat message into a short title. It's just a small neural net (a GRU thing) that reads your message and either makes up a word or copies one from what you wrote. The whole thing (model + tokenizer + runtime) runs in under 5 MiB of ram, in a few tens of ms (on my desktop), on one small C binary. Here's a comparison with a bigger reference model (SupraLabs 50M Q8_0), on the same prompts: prompt TinyTitle Supra Title 50M How does AI work? AI Work AI Basics Explained How to make a discord server? Discord Server Discord Server Creation What's the best way to learn French quickly? Best Way to French Quickly Learning French Tips Explain quantum computing like I'm five Quantum Computing Like Quantum Computing Basics Can you explain the difference between TCP and UDP? TCP and UDP Differences TCP Vs UDP Comparison 36 liters of diesel fuel is worth €18. The tank of this pickup truck can hold 80 liters. How much does it cost to fill the tank? Diesel Cost Calculation Diesel Fuel Tank Cost These are some decent results, and in general, the Supra model is the best of both (which makes sens, it's bigger). The 50M model is more abstract and grammatical, the tiny one is more literal. Let's compare the ram usage, including the runtime (and let's use the most agressive quantization of the 50M model (Q1_0)): model file peak rss TinyTitle 1.98 MB 4.89 MiB Supra Title 50M (Q1_0, llama.cpp) 19.6 MB ~126 MiB So about 25x times less ram usage :) I don't know if people would even want it on Hugging Face for some reason, but if there's enough demand I might put it there and edit this post to add the link. edit: https://huggingface.co/azomDev/TinyTitle For all I know this is trivial and everyone here already built one, but it was neat to see it work. Thanks for reading, sorry if this is not the right place for this lol. Repo: https://github.com/azomDev/TinyTitle submitted by /u/H-L_echelle [link] [comments]

Source: r/LocalLLaMA | 2026-08-11