Model Releases
4x 6000 Pro for the company
Here is how i serve 40 people (10 of them developers) with local AI services. n8n for automatisation whisper and Vexa AI, a bot joins Teams and makes transcripts, n8n later extracts decisions and todo
Here is how i serve 40 people (10 of them developers) with local AI services. n8n for automatisation whisper and Vexa AI, a bot joins Teams and makes transcripts, n8n later extracts decisions and todos OpenWebUI, ChatGPT-like webinterface for everyone, most agentic features are disabled, websearches via Tavily are possible Every developer has an API key to use via BiFrost (OpenAI compatible API router) for what they find useful Every developer has a local Linux VM where agents can be installed. A 4U server that contains 4 6000 Blackwell MaxQ and 384GB DDR5 and one AMD EPYC 9275F 24-Core CPU. mainmodel is Deepseek V4 Flash with vLLM, GPU utilization 0.55, TP4, 1M context. fallbackmodel is Qwen 3.8 27B Q8 with llama.cpp, 265K context in Q8. minimodel is K2-Horizon 7B Q6 with llama.cpp, with 64K context in Q8. ComfyUI to generate pics and videos for marketing. Embedding/Reranking with Geofront/BGE-M3-GGUF:BF16 and sinjab/bge-reranker-v2-m3-F16-GGUF with llama.cpp https://preview.redd.it/eribrlg7z8vh1.png?width=1913&format=png&auto=webp&s=33c7044412273dec45e9c251c879d4b2a1dd6e53 Ask if you have questions. submitted by /u/q-admin007 [link] [comments]
Related
- Five local models, 6.8 GB of weights: my open-source Mac meeting notetaker
- Local-first LLM pipeline tracer — @trace on any function, dashboard at localhost. Feedback welcome.
- Best open-source harness like Claude Code?
- Putting the question before the context took my local Qwen from 89% to 100% on a decision benchmark, and from ~400 ms to ~80 ms
Source: r/LocalLLaMA | 2026-10-13