Local Ai
Need some suggestion
Guys I have a M4-mac air with 24 GB unified memory and I wanna run some local LLM model on my machine Currently I am working on building and RAG application which is mostly and QnA system but I wanna
Guys I have a M4-mac air with 24 GB unified memory and I wanna run some local LLM model on my machine Currently I am working on building and RAG application which is mostly and QnA system but I wanna upgrade it to Agentic RAG I have tried using online LLM providers (free-tier) but I am hitting the token call limit for big queries. Therefore I was thing to shift to Local AI model. Thinking to install gpt-oss-20b model on my machine which is about 14GB in size My question is will this model run on my machine smoothly or will be extremely slow ? cause I will be performing other task while the model is running in the back Or should I go for a lighter model but I am afraid that the accuracy of the generation will drop and can u also suggest some local LLM that will run smoothly on my machine without much latency submitted by /u/Mobile_Squash_2128 [link] [comments]
Related
- which model to use on local 24gb mac mini M4 pro
- Local Ollama models
- What’s the best model to use with RAG to create a locally hosted survival and off grid LLm?
Source: r/ollama | 2026-09-24