Search: “nvidia”

I used the quantized n-gram to INT4, it's 32 GB, memory-mapped from disk. I confirmed that it works great on 150-160k context, and i was watching all the time my VRAM usage while doing single thread l

You guys remember sesame AI? With their really good AI voice model? Obviously ChatGPT has their voice model that’s also really good. Is there any smaller local variant that runs on like consumer grade

In mid-August, Ramp published spending data collected from 70,000 U.S. companies: Fable 5 ,the most powerful and expensive model in Anthropic’s lineup, accounts for just 11% of what those businesses s

Andreessen Horowitz today announced that it has raised a 1.1 billion fund to back artificial intelligence infrastructure startups. The Machine Age Fund will invest in companies that make data center e

vulkan: fix missing view-alias dependencies in ggml_vk_graph_optimize (#27812) vulkan: fix missing view-alias dependencies in ggml_vk_graph_optimize is_src_of doesn't treat two views of one tensor as

We are considering to buy computer, server to run decent model on premises. I would like to run big open source model with more that 70b parameters roughly. I read the people used to run on Apple Stud

Double standard @sama . Exactly two years ago, he belittled @dylan522p as the 'SemiAnalysis' guy after not liking a Google analysis. Altman's now kissing Dylan's ass, inviting SemiAnalysis to OpenAI l

Perplexity AI has released its new language model GLM 5.3 on Perplexity Computer. Designed for long‑context, multimodal agent workloads, GLM 5.3 outperformed the previous GLM 5.2 on the company’s WAND

i'm gonna get shredded to pieces like tibo and jarred when they see what i actually look like now... i'll be speaking at the 2nd edition of @aiDotEngineer paris on sep 24th 🇫🇷 i'll be talking about: •

A few days ago I posted asking what people considered the best local model for an 8 GB VRAM GPU. At the time, my personal sweet spot was Qwen3.6-35B-A3B, for agentic coding with Pi.dev. Well… Thanks t

Hey all, and hello fellow DGX Spark-ers! Today I managed some pretty crazy numbers: 181 tok/s aggregate on 2× DGX Spark on Qwen3.8-Flash-Next at 512Kcontext (2.8M kvc) I hit 181 tok/s aggregate today

audio.cpp 0.7 is out :) This release adds a lot of new audio models and a new way to compare them locally. Audio.cpp is now at 62 model families and 85+ model variants. And it keeps growing! The bigge

Lambda Inc. closed a $926 million senior‑secured term loan B facility on August 27 2026 to finance the purchase and deployment of GPU infrastructure for an investment‑grade customer, marking the compa

arXiv:2608.25757v1 Announce Type: cross Abstract: Generalist vision--language--action (VLA) policies learn long-horizon behavior mainly through short-horizon action prediction and reveal little beyond

I was using UD-Q3_K_XL until now with more than 140000 context. Quality wise it's very good, very few erroneous tool calls. Then I saw many others here reporting good results with IQ3_XXS, so I gave i

arXiv:2608.25096v1 Announce Type: new Abstract: The growing adoption of large language models (LLMs) has raised increasing concerns about the energy consumption and environmental impact of inference.

Forgive the typos and rambling - human actually wrote this post 😂 I've had a Strix Halo board for about a year and been playing around with it for various projects when it's not just being a beefy lin

We're releasing a fully quantized NVFP4 version of Qwen3.8-27B. The checkpoint was trained using quantization-aware distillation (QAD) with QUASAR, our new QAT algorithm. We used the original BF16 mod

arXiv:2608.23927v1 Announce Type: new Abstract: Video generative models provide rich physical priors for robot learning, yet existing world-action models (WAMs) face a fundamental trade-off: synchrono

GLM-5.3-Flash by @Zai_org, aka Ox Alpha, is now live in LM Studio Bionic! This model surpasses GLM 5.2 in performance while being 9-10x cheaper. It also supports image input. In Bionic, this model is

arXiv:2608.23834v1 Announce Type: new Abstract: The key-value (KV) cache is a primary capacity and bandwidth bottleneck in long-context LLM serving. We present Minima-KV, a retention-preserving hierar

Yet another vLLM fork thread here, but this time its for older INT8-centric hardware. This is a complete INT8 serving stack for Qwen3.8 27B based on vLLM, AITER, and a 27B GPTQ INT8 quant w/ DFlash2.

arXiv:2602.23553v2 Announce Type: replace Abstract: Long-form video question answering (LVQA) requires answering natural-language queries about videos spanning minutes to hours, demanding temporal rea

arXiv:2608.23972v1 Announce Type: new Abstract: Safety-aware motion planning remains a challenge in robotics, especially when missions are time-critical and are under complex specifications. In this p

arXiv:2608.24516v1 Announce Type: cross Abstract: Satellite-based distributed learning promises to train machine-learning models directly in orbit using massive, globally dispersed sensor data, thereb

arXiv:2608.24561v1 Announce Type: new Abstract: Rapid earthquake magnitude estimation is central to earthquake early warning, yet many operational systems depend on dense regional seismic networks and

arXiv:2608.23807v1 Announce Type: new Abstract: Masked diffusion language models (dLLMs) can in principle generate text faster than autoregressive (AR) models, since they denoise many tokens at once.

arXiv:2608.24674v1 Announce Type: new Abstract: Joint text-to-video-audio generation produces synchronized visual and acoustic content, but the long sampling trajectories and heterogeneous multimodal

arXiv:2608.22115v1 Announce Type: new Abstract: Collaborative multimodal inference improves edge perception by combining observations from distributed sensing devices, but transmitting high-dimensiona

arXiv:2608.22108v1 Announce Type: new Abstract: Breast Cancer Multidisciplinary Team (MDT) meetings manage increasingly complex cases under considerable time pressure, and documentation requirements c

Generalist AI Inc., a startup that develops artificial intelligence software for robots, has reportedly raised 200 million in funding. Axios today cited a source as saying that 8VC led the investment.

arXiv:2608.22716v1 Announce Type: cross Abstract: Despite their remarkable success, machine learning models, particularly in vision applications, are alarmingly vulnerable to a range of security threa

Top open models by use case 👇 chat - Kimi K3, Qwen3.8 2.4T A95B reasoning - Kimi K3, DeepSeek V4 Pro coding agents - Kimi K3, DeepSeek V4 Flash, GLM 5.2 small + fast - Qwen3.8 27B, gpt-oss 20B mid-siz

arXiv:2608.20427v1 Announce Type: cross Abstract: Dense causal attention remains expensive at long context even when implemented with highly optimized exact kernels. We study BF1, a deterministic bloc

arXiv:2608.20614v1 Announce Type: new Abstract: Enterprise agent programs are moving from prototypes into production, where reusable skills, tools, and workflow packages must be reviewed with evidence

A month i setup a proxmox server using spare parts like a z270 i7 7700k and 2x8 2666 but since i don't have any hardware acceleration i was planning to pass down my rx 580 8GB to my server but i heard

Companion to the Qwen 3.8 Release Megathread. Compiled from ~2,000 posts scanned across both subs, with deep reads of the 45 highest-signal threads (560 posts and comments), Aug 15–22, 2026, plus inde

Hey guys, Inco AI shipped DFlash 2 a few days ago with a drafter for Qwen 3.8 27B and a llama.cpp PR. I built the PR and ran it against plain decoding, MTP, the n-gram lookup drafters, and my July DFl

arXiv:2608.19758v1 Announce Type: new Abstract: Long-context modeling is a pivotal capability for Large Language Models, yet the quadratic complexity of attention remains a critical bottleneck, partic

arXiv:2608.19535v1 Announce Type: new Abstract: Retrieval-augmented generation (RAG) improves language-model responses by grounding generation in external passages, which comes with overhead: retrieve

arXiv:2608.19920v1 Announce Type: new Abstract: A lot of prior work addressed key-value (KV) cache selection and compression by sparse attention to enable long-context inference for transformer langua

‘Open Models, Production Agents’ is underway! Tonight means another event hosted at FriendliAI's San Francisco HQ. A very lively turn out, and a stellar lineup of panelists exploring how open models a

Spent a while treating layer placement, KV format and llama.cpp itself as experimental variables. 159 logged experiments. Numbers first, caveats after. Hardware: AMD Ryzen AI MAX+ 395 (Strix Halo, 128

The sense of touch is the most criminally under-explored modality in robotics. Imagine doing sleight of hand wearing thick oven mitts. That's exactly how a robot feels today if it were alive. A magnet

Not everyone has the disposable income to build a small data center, so making this post for the underdogs as I was very surprised by the performance/results of this 35B MOE model. Context is admitted

arXiv:2608.19147v1 Announce Type: cross Abstract: Modern Intel AI PCs ship capable integrated GPUs and NPUs with 16+ GB of unified memory, and they spend considerable time idle. That is not enough mem