Mobile Offline LLMs: What do you use them for?

I've spent the last year or so playing around with open source MLX and GGUF models on iPhone hardware. Given the limitations in memory, GPU/CPU/ANE, and in turn the context window I've been trying to figure out the best use cases for them. I've also done a lot of testing with Apple Foundation model. It is not that capable, but it is good at understanding instructions for tool calling and actually seems to work well for fast summarization and classification prior to feeding data to more capable models. I've been able to get models ranging from 0.5B to 8B to work well on the higher end devices. So far I've been testing it out with building tooling such as web search, url scraping, summarization, research, analyzing local photo/video/documents/etc., and basic on-the-go coding help. With continuous compaction I've been able to hold virtually unlimited exchange chats even though context windows are limited to like 8-16k tokens. Separate from more professional use cases, It also seems like potentially a good option for private, offline chat about your personal thoughts with models tuned with custom system prompts. I know many here are focused on trying to run as good of a model as possible on their laptop or desktop hardware but it seems like mobile is an area that isn't really being explored extensively yet. What do you use mobile models for? submitted by /u/James333i [link] [comments]

Related

• Benchmarks: TensorSharp vs. llama.cpp
• DKV: Open-source KV-cache compression framework for local LLM inference (CLI + technical report)
• Current efficient frontier of open models
• Episodic-Semantic Memory Architecture for Long-Horizon Scientific Agents

Source: r/LocalLLaMA | 2026-07-25