10 year garbage card for local llms

Hello everyone! I like dumb things. I like working with weak computers and microcontrollers. I like the simplicity and low electricity usage. Simply put, the efficiency of a "dumb" PC. The first time I tried local llms I ran tinyllama on the garbage card from the title (Nvidia 1060 3gb). Tinyllama was responsive and to this day I like it because it'll just say the funniest shit: "Like most other celestial objects you cannot see the moon from earth," -tinyllama. I love it, but I realized that it's not a model you can actually use. I still think dumb models are more interesting than large ones. I don't know why, I guess because they still feel like a lovable "autocomplete" function with a single brain cell and less like hal9000. I am now wondering, how far can you push the garbage card? I'm currently hosting a kubernetes on the old PC with the garbage card and have set up a pod that allows me to send queries to it from other apps. I tested some recent ~7 b parameter models on the garbage card (with offloading of course) and they seem to be usable for non-time-dependent tasks. Essentially I'm thinking about feeding sensor data, tabular data, or text into the LLM once a day automatically. It doesn't matter if it takes 10 minutes to run if it starts automatically every few hours to comment on CO2 sensor data from a raspi zero or ESP32 that are communicated to another app. I'm also thinking about letting it classify credit card spending data into categories. (I did this manually before so I could actually benchmark different models against the correct classification. Nothing of this I would do exposed.) I wanted to ask your opinion. Do you think you can make a 3gb card work for certain tasks? Is it worth playing around with? Can you recommend a model to test? submitted by /u/AGuyCalledBath [link] [comments]

Source: r/LocalLLaMA | 2026-08-11

Related