Local Ai
How to run LLMs as regular guy with low resources?
Hello everyone. I hope is all well with you all. I been around this sub for a few months and been quietly reading and I have seen how many of you are spending $10k on the Mac M5 and I am assuming you
Hello everyone. I hope is all well with you all. I been around this sub for a few months and been quietly reading and I have seen how many of you are spending $10k on the Mac M5 and I am assuming you are using it for work or just for fun but have the resources to spend like that. I am a regular guy with a 9-5 job and I would like to experiment with local LLMs but I dont have much disposable income to spend big amounts on GPUs or Macs. I currently have an I7 - 8700 and Nvidia 2060 6Gb VRAM - 32GB RAM Now my question is, does anybody have an idea on what models or hacky stuff I can do to try the latest AI models on my hardware? I have checked hugging face but I see so many models with the same name but letter and numbers at the end. I understand some of them mean Quantization which is dumber a model but I also found out tiny models like Linq which can be run even on the CPU. So I would like to connect with someone with more experienced to have fun with AI. Thank you everyone . submitted by /u/pet3121 [link] [comments]
Related
- 2× Radeon R9700 for Local AI Was Choosing AMD Instead of NVIDIA a Mistake Without CUDA?
- Smallest model (& tips) for intelligent computer use via Hermes?
- A visualization of LLM API costs to ask for local resources
- Do you think dedicated hardware for running local LLMs will become affordable anytime soon?
Source: r/LocalLLaMA | 2026-08-25