Local Ai
Cosmos3-Edge in under 8 GB: VLM out now, video gen and robot policy next.
Hey guys, posted about optimized Cosmos Reason 2 model here a few months ago. Wanted to share a build of the latest Cosmos3 model, with even more optimizations! Huggingface and Jetson AI Lab has no Or
Hey guys, posted about optimized Cosmos Reason 2 model here a few months ago. Wanted to share a build of the latest Cosmos3 model, with even more optimizations! Huggingface and Jetson AI Lab has no Orin Nano support for Cosmos3-Edge, because the BF16 stack doesn't fit in 8 GB. It now does, INT4 quantization, along with more optimizations for speed! Device Stack Decode tok/s Orin Nano Super 8 GB Optimized 152.5 AGX Thor 128 GB Optimized 207.8 AGX Orin 64 GB BF16 vLLM 44.1 AGX Thor 128 GB BF16 vLLM 68.2 Accuracy across five VLM benchmarks is maintained (from 65.56 to 65.20). Try it in the browser or serve it locally: https://embedl.com/nvidia/cosmos3-edge/orin-nano-super For now the VLM part of the model is published; diffusion and policy part is being added very soon! Optimized peak memory for the VLM is as low as 3.3 GB! Trying a new page for pre-compiled models, since Hugging Face doesn't really support them. Let me know if something is missing to make it useful! submitted by /u/No-Dragonfly6246 [link] [comments]
Related
Source: r/LocalLLaMA | 2026-10-21