Search: “deployment”

I wanted to find out whether a huge text-only MoE could be given basic vision without retraining the language model itself. The short answer is yes. I froze DeepSeek V4 Flash and a 417M-parameter Moon

A few months ago, I had the idea of making a LLM from scratch as a personal project (for learning and partly for improving my resume). Since I learned a lot from other posts on here over the past year

VLX-Seek-1.5-10B VLX-Seek-1.5-10B is the open-source 10B model in the VLX-Seek 1.5 family, designed for fine-grained perception and visual grounding in embodied scenarios. It targets practical setting

audio.cpp 0.5 is out :) The most fun new model in 0.5 is DramaBox. It is closer to prompt-directed voice acting. DramaBox is built on the LTX-2.3 audio architecture, and prompts can control emotion, d

We developed a leaderboard ranking frontier model security. There's no shortage of model capability rankings, but we didn't find anything comparable for model security. Yet security is becoming increa

Results: prompt eval: 40 tokens / 97.5s → 0.41 tok/s eval: 400 tokens / 1769.9s → 0.23 tok/s total: 440 tokens / 1867s (31 min) Prompt: 'Write a C++ function that reverses a linked list in place. Expl

Large language models (LLMs) are transforming recommender systems from matching co-occurrence patterns in historical behavior toward reasoning about the intent that drives it. RecGPT-V1 pioneered this

Fara1.5-27B is a multimodal computer use agent (CUA) for web browsers, from Microsoft Research AI Frontiers. It observes the browser through screenshots and acts on the user's behalf by emitting struc

Below Upstream Status sections are from https://github.com/PrismML-Eng/Bonsai-demo Upstream Status for Binary Q1_0 is supported out of the box in upstream llama.cpp across many backends: CPU (generic,

This Reddit thread on r/ollama likely discusses where and how to use Ollama's cloud offering, which allows models to run without a powerful local GPU by automatically offloading computation to Ollama'

ik ik 'another harness D:'. so... i spent the last 6 months of my life building and ironing out every single detail and bug through almost every route with extensive testing across 3 machines, daily,

SlopSoup TV is a live, never-ending pixel-art TV network. Every script, character, voice, camera cut and schedule decision is made by models. It's made to be very absurd, dark and strange. Hardware: o

A bit of backstory : i got lucky in August and was able to grab a CMP 170 HX 8GB for around 500 on alibaba. I tried later to get a second one for around 1k but my order got cancelled and prices went t

This table to help anyone looking to build a budget Data Center homelab. I copied the bulk of value based, mid level, decent speed results GPUs and feed it to AI or SI and here are the recommended res

tldr: Open sourced my prompt to game suite: https://github.com/ndamiano/ai-agent-test, it's MIT licensed, and this runs well on my 5090 with 64gb ram, but any model that can handle tool calls can mana

Octop is an open-source, self-hosted AI assistant. Through its multi-agent architecture, it builds an intelligent environment that is both independent and collaborative for teams, families, and indivi

Over the past several months, we conducted an empirical post-mortem investigation into 109 autonomous AI agent security incidents (cataloged with 193 falsification criteria across tool-use and multi-a

Krea 2 Turbo — 2-Step Distillation LoRA (work in progress) Previous posts/releases - here, here and here. 🧪 A fast-preview adapter, from a project still in training. When the subject is close and fill

I use ChatGPT a lot. Probably a lot more than is reasonable. I read David Robinson’s Atlantic piece today about leaving OpenAI and his criticism of its culture of trial-and-error / “iterative deployme

Innanzitutto, due cose per evitare fraintendimenti. Non sto cercando di sostenere che un modello o un'azienda siano migliori. Non ho alcun interesse personale in nessuno di essi. Volevo solo verificar

Hi r/LocalLLaMA! We’re researchers at the Institute of Foundation Models (IFM), an AI research lab dedicated to open and independent development of frontier-class foundation models. We recently releas

The local models seem powerful enough to be capable of running basic local computer use. This is a computer use agent/harness built via claude code, and powered by qwen3.8 flash next nvfp4. Drawing a

We have released Qwen3.8-Flash-Next quantized with GSQ and RCO, together with a second, capability-targeted build in which half of the model's experts have been removed. Flash-Next is a sparse mixture

Yesterday I shared our open-source Clash Royale simulator and its recurrent PPO agent here. A training loop is easier to understand when you can watch it, so we put a small interactive version online:

Can LLM agents actually get through a day in the life of a normal user? That question got me reading papers on Android agents and mobile benchmarks over the past few months. A few patterns kept showin

I maintain a starter kit where the LLM provider is a runtime-swappable DB row, not a hardcoded constant — OpenAI, Anthropic, or a local Ollama instance. Mostly so I can demo without burning API credit

Hey r/Ollama, I've been working on Presentation Forge, an open-source presentation/report generator that supports Ollama for local inference. I originally made it because I'm a university student and

Hey everyone, A major issue when building test environments or fine-tuning pipelines is balancing data realism with privacy. Pasting real internal/client data into cloud endpoints is a massive complia

Yesterday I released Bonsai-Llama-Jev - a Jev-like Typed Decision inference system for locally operated, sovereign AI. In my new typed-decision-bench, covering more than 22,000 decisions across 175 us

Saw a new open 27B model release that looks like it could be interesting for the Ollama crowd: VeriLoop E2. It is a Qwen3.8-27B post-train aimed at code, mathematics, scientific reasoning, and agentic

🚀 THE EDGE AI FAMILY HAS ARRIVED. Today we’re opening the doors to a new generation of open-source and open-weight edge AI models from Next GenAi. Meet Rebekah AI 1.8B. 🧠⚡ Small enough to run at the e

I shipped something I've been building for the last few weeks : phantom-kv , a refusal-removal system for large language models that doesn't touch a single weight. Instead of editing the model, it loa

OpenAI’s decision to retire custom GPTs appears to rest on the assumption that Plugins, Skills, Projects, Apps, and Workspace Agents collectively replace their functionality. For many Small and Medium

Given the recent interest in tools like Jev, I wanted to share ProgramAsWeights (PAW), an open-source research project I'm working on at the University of Waterloo. You describe a text function in Eng

Anyone interested in trying out a new inference provider? We're testing Gemma 4:26b and Qwen 3.8:27B behind an OpenAI-compatible API and looking for a few users to give them a spin and tell us how it

Three weeks back , i posted SHADOW-250M here. It got 360 upvotes, 293 on r/LocalLLaMA and 94 GitHub stars. Thank you. That model was 60 MB, ran around 400 tok/s on CPU and could retrieve records from

It’s always bothered me that after fine-tuning a model for a project, there isn’t a particularly easy way to host it without either running it locally and keeping a GPU on 24/7 or paying for an entire

I’m genuinely mind-blown by what’s happening with AI models right now. On one side, you have GPT-6-class frontier models that are just… insane. The level of capability is getting hard to wrap your hea

The missing control is visible in sudoingX’s Ling-3.0-flash benchmark graphics. The earlier table leaves Ling’s no-speculation baseline as “not measured.” The later code/prose graphic fills it in: abo

Krea 2 Turbo — 4-Step Distillation LoRA Full details and to download - HF Repo: https://huggingface.co/lvladikov/Krea2-Turbo-Distill-4step-LoRA https://preview.redd.it/9p2rjwh3y3oh1.jpg?width=3072&for

https://huggingface.co/XHToken/Spark-X2.5-4B-GGUF https://huggingface.co/XHToken/Spark-X2.5-1.7B-GGUF from XHToken: We are introducing Spark-X2.5-4B and Spark-X2.5-1.7B, two compact, general-purpose l

Hello, I'm a bit obsessed with cache management on local LLMs. For the last few days I've been working on cache management on vLLM with my 2x DGX Spark and DeepSeek v4 Flash 0731. I felt something was

I ran a side-by-side ML text-processing and model-training workflow using Fable 5.1 vs. Astra (both on xhigh), and the results could not have been more different. Warning, long post. TL;DR -- Astra co

Hey everyone, Wanted to share a short paper we just published exploring a simple but surprisingly effective optimization for sparse MoE reasoning models. The problem: MoE reasoning models (especially

We have been working on an open-source, model-neutral agent harness for general purpose agents called TrueForge, and wanted to understand how much the harness itself actually matters. So we ran 14 tas

I used the quantized n-gram to INT4, it's 32 GB, memory-mapped from disk. I confirmed that it works great on 150-160k context, and i was watching all the time my VRAM usage while doing single thread l