Engineering teacher hoping to ditch ChatGPT premium

I'm a teacher looking into running an LLM locally. I've been using ChatGPT for a lot of this work, so I started looking at Ollama because I understand it's designed to run LLMs locally. That said, I'm definitely open to other software/models/workflows if there's a better solution for what I'm trying to do. Hardware: GPU: AMD Radeon RX 6700 XT — 12GB VRAM CPU: AMD FX-6300 @ 3.5 GHz RAM: 32GB DDR3 Windows 64-bit Storage: 120GB SSD + 2TB HDD Yes, I know the CPU is ancient. I'm saving to replace pretty much everything on it over the next 2 or 3 years. Most of my AI use involves things like: - Lesson and curriculum planning - Turning information into PowerPoint content/outlines - Creating quizzes and classroom materials - Summarizing and reorganizing documents - Extracting information from PDFs/forms - Turning unstructured information into spreadsheets/CSV - Moving information from one document/template into another Privacy is another reason I'm interested in running locally. Eventually I'd like to be able to process things like scanned permission forms and parent contact information without uploading those documents to an outside AI service. I'm looking for recommendations on models that would work well with 12GB VRAM/32GB RAM, especially for instruction following, writing/planning, structured output, and document extraction. I'd also love recommendations for OCR/document tools, front ends, RAG/workflow tools, or alternatives to Ollama that would make sense for this use case. I’m not expecting my ancient machine to be able to compete with ChatGPT, so if this kind of thing is completely beyond my current hardware I’d like to know that as well. If you were building a completely local AI assistant for this kind of work on this hardware, what stack/models would you use? EDIT: thanks for all of the replies, guys! I've got a lot to play around with over the next couple of weeks. submitted by /u/theblackcrayon4 [link] [comments]

Source: r/ollama | 2026-09-12

Related