Local Ai
VLLM-MLX vs Ollama MLX... Ran concurrency benching. Qwen3.5-4B.
Anyone else play with vllm-mlx? There was an interesting paper that came out that compared this version of vllm that was originally designed for multi modal inference performance gains, but ended up s
Anyone else play with vllm-mlx? There was an interesting paper that came out that compared this version of vllm that was originally designed for multi modal inference performance gains, but ended up seeing gains in text based inference. I added this as a provider into Rehex and ran my own benches, and it looks like that compared to Ollama mlx models (which isn't benched in the paper), this thing cranks at least a fifth faster (and that's only on text based inference). Anyone play with this? If you're using mlx models, what are you using to serve them? submitted by /u/haseebnqureshi [link] [comments]
Related
Source: r/ollama | 2026-09-10