Search: “nvidia”

Inference is 80-90% of the lifetime cost of a production AI system. Most AI-native teams are leaving performance and margin on the table. He…

Inference is 80-90% of the lifetime cost of a production AI system. Most AI-native teams are leaving performance and margin on the table. Here’s how Together AI, the AI Native Cloud, fixes that on @nv

hardwaretogether-ai--x

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Hardware

LEAP: Layer-wise Exit-Aware Pretraining for Efficient Transformer Inference

DGX agent

arXiv:2605.01058v1 Announce Type: cross Abstract: Layer-aligned distillation and convergence-based early exit represent two predominant computational efficiency paradigms for transformer inference; ye

hardwarearxiv-cs-cl

5 May 2026

LiteVLA-H: Dual-Rate Vision-Language-Action Inference for Onboard Aerial Guidance and Semantic Perception

arXiv:2605.00884v1 Announce Type: new Abstract: Vision-language-action (VLA) models have shown strong semantic grounding and task generalization in manipulation, but aerial deployment remains difficul

Safety

LLM-Based Agentic Negotiation for 6G: Addressing Uncertainty Neglect and Tail-Event Risk

arXiv:2511.19175v2 Announce Type: replace-cross Abstract: A critical barrier to the trustworthiness of sixth-generation (6G) agentic autonomous networks is the uncertainty neglect bias; a cognitive te

safetyarxiv-cs-ai

Stochastic Sparse Attention for Memory-Bound Inference

arXiv:2605.01910v1 Announce Type: new Abstract: Autoregressive decoding becomes bandwidth-limited at long contexts, as generating each token requires reading all n_k key and value vectors from KV cach

StreamIndex: Memory-Bounded Compressed Sparse Attention via Streaming Top-k

arXiv:2605.02568v1 Announce Type: new Abstract: DeepSeek-V3.2 and V4 introduce Compressed Sparse Attention (CSA): a lightning indexer (a learned scoring projection over compressed keys) scores them, t

SURGE: SuperBatch Unified Resource-efficient GPU Encoding for Heterogeneous Partitioned Data

arXiv:2605.01060v1 Announce Type: cross Abstract: We present SURGE, a streaming GPU encoding system deployed in production to generate embeddings for over 800 million texts across 40,000 logical parti

ViM-Q: Scalable Algorithm-Hardware Co-Design for Vision Mamba Model Inference on FPGA

arXiv:2605.01935v1 Announce Type: cross Abstract: Vision Mamba (ViM) models offer a compelling efficiency advantage over Transformers by leveraging the linear complexity of State Space Models (SSMs),

DPU or GPU for Accelerating Neural Networks Inference -- Why not both? Split CNN Inference

arXiv:2605.00174v1 Announce Type: cross Abstract: Video and image streaming on edge devices requires low latency. To address this, Neural Networks (NNs) are widely used, and prior work mainly focuses

4 May 2026

MiniVLA-Nav v1: A Multi-Scene Simulation Dataset for Language-Conditioned Robot Navigation

arXiv:2605.00397v1 Announce Type: new Abstract: We present MiniVLA-Nav v1, a simulation dataset for Language-Conditioned Object Approach (LCOA) navigation: given a short natural-language instruction,

hardwarearxiv-cs-ro

4 May 2026

if even coding doesn’t make serious RoI for users, the jig is probably up.

model-releasesgary-marcus--x

if even coding doesn’t make serious RoI for users, the jig is probably up. Economics of AI coding tools are crumbling. Cost per developer has doubled because cost of Claude code and microsoft copilot

3 May 2026

The number of jobs in the future is endless because the problems to solve are endless. Jobs multiply as we get more complex. No AI or human …

The number of jobs in the future is endless because the problems to solve are endless. Jobs multiply as we get more complex. No AI or human can solve all problems and all the work to do in the Univers

hardwareyann-lecun--x

3 May 2026

Sources: Anthropic is in early talks to buy AI inference chips from UK-based Fractile when they become available in 2027 (The Information)

The Information: Sources: Anthropic is in early talks to buy AI inference chips from UK-based Fractile when they become available in 2027 — As Anthropic's sales explode, straining the servers it uses,

hardwaretechmeme

2 May 2026

AutoSP: Unlocking Long-Context LLM Training Via Compiler-Based Sequence Parallelism

arXiv:2604.27089v1 Announce Type: new Abstract: Large-language-models (LLMs) demonstrate enormous utility in long-context tasks which require processing prompts that consist of tens to hundreds of tho

GraphMend: Code Transformations for Fixing Graph Breaks in PyTorch 2

arXiv:2509.16248v3 Announce Type: replace-cross Abstract: This paper presents GRAPHMEND, a high-level compiler technique that eliminates FX graph breaks in PyTorch 2 programs. Although PyTorch 2 intro

Lightweight Distillation of SAM 3 and DINOv3 for Edge-Deployable Individual-Level Livestock Monitoring and Longitudinal Visual Analytics

arXiv:2604.27128v1 Announce Type: cross Abstract: Foundation-model pipelines for individual-level livestock monitoring -- combining open-vocabulary detection, promptable video segmentation, and self-s

Pentagon inks AI procurement deals with seven companies, leaves out Anthropic

The U.S. Defense Department today announced that it has inked artificial intelligence procurement contracts with seven tech firms. The group includes Amazon Web Service Inc., Google LLC, Microsoft Cor

hardwaresiliconangle

Efficient, VRAM-Constrained xLM Inference on Clients

arXiv:2604.26334v1 Announce Type: cross Abstract: To usher in the next round of client AI innovation, there is an urgent need to enable efficient, lossless inference of high-accuracy large language mo

30 Apr 2026

Video Compression Meets Video Generation: Latent Inter-Frame Pruning with Attention Recovery

arXiv:2603.05811v2 Announce Type: replace Abstract: Current video generation models suffer from high computational latency, making real-time applications prohibitively costly. In this paper, we addres

30 Apr 2026

Feasible-First Exploration for Constrained ML Deployment Optimization in Crash-Prone Hierarchical Search Spaces

arXiv:2604.25073v1 Announce Type: new Abstract: Deploying machine learning models under production constraints requires joint optimization over model family, quantization scheme, runtime backend, and

FED-FSTQ: Fisher-Guided Token Quantization for Communication-Efficient Federated Fine-Tuning of LLMs on Edge Devices

arXiv:2604.25421v1 Announce Type: new Abstract: Federated fine-tuning provides a practical route to adapt large language models (LLMs) on edge devices without centralizing private data, yet in mobile

Laplace-Bridged Randomized Smoothing for Fast Certified Robustness

arXiv:2604.24993v1 Announce Type: new Abstract: Randomized Smoothing (RS) offers formal ell_2 guarantees for arbitrary base classifiers but faces two key practical bottlenecks: (i) it often relies on

Nautile-370M: Spectral Memory Meets Attention in a Small Reasoning Model

arXiv:2604.24809v1 Announce Type: new Abstract: We present Nautile-370M, a 371-million-parameter small language model designed for efficient reasoning under strict parameter and inference budgets. Nau

QCalEval: Benchmarking Vision-Language Models for Quantum Calibration Plot Understanding

arXiv:2604.25884v1 Announce Type: cross Abstract: Quantum computing calibration depends on interpreting experimental data, and calibration plots provide the most universal human-readable representatio

GradMAP: Gradient-Based Multi-Agent Proximal Learning for Grid-Edge Flexibility

arXiv:2604.24549v1 Announce Type: cross Abstract: Coordinating large populations of grid-edge devices requires learning methods that remain fully decentralised in deployment while still respecting thr

Hallo-Live: Real-Time Streaming Joint Audio-Video Avatar Generation with Asynchronous Dual-Stream and Human-Centric Preference Distillation

arXiv:2604.23632v1 Announce Type: new Abstract: Real-time text-driven joint audio-video avatar generation requires jointly synthesizing portrait video and speech with high fidelity and precise synchro

How we built the most performant DeepSeek V3.2, MiniMax-M2.5 and Qwen 3.5 397B on DigitalOcean Serverless Inference

model-releasesdigitalocean

DigitalOcean describes their optimization and deployment of three large language models (DeepSeek V3.2, MiniMax-M2.5, and Qwen 3.5 397B) on their Serverless Inference platform, likely leveraging NVIDI

Latent Inter-Frame Pruning: A Training-Free Method Bridging Traditional Video Compression and Modern Diffusion Transformers for Efficient Generation

arXiv:2604.23858v1 Announce Type: new Abstract: Video generation, while capable of generating realistic videos, is computationally expensive and slow, prohibiting real-time applications. In this paper

Real-time windrow detection from onboard tractor sensors for automated following

model-releasesarxiv-cs-ro

arXiv:2604.24628v1 Announce Type: new Abstract: Proprietary design in commercial windrow-detection systems restricts transparency and limits progress in open autonomous forage-harvesting research. We

Welcome to the agentic era: Public sector highlights and reflections from Next ‘26

model-releasesgoogle-cloud-ai

Welcome to the agentic era! Last week, leaders from our public sector customer and partner ecosystem took the stage at Google Cloud Next to share how they are leveraging AI and agents to scale their i

Incentivizing Neuro-symbolic Language-based Reasoning in VLMs via Reinforcement Learning

arXiv:2604.22062v1 Announce Type: new Abstract: There are 7,407 languages in the world. But, what about the languages that are not there in the world? Are humans so narrow minded that we don't care ab

hardwarearxiv-cs-cl

Kernel Contracts: A Specification Language for ML Kernel Correctness Across Heterogeneous Silicon

arXiv:2604.22032v1 Announce Type: new Abstract: Every ML kernel ships with an implicit contract about what it computes. People rarely write the contract down. When two kernels disagree -- when a matmu

Motivating Next-Gen Accelerators with Flexible (N:M) Activation Sparsity via Benchmarking Lightweight Post-Training Sparsification Approaches

arXiv:2509.22166v4 Announce Type: replace-cross Abstract: The demand for efficient large language model (LLM) inference has intensified the focus on sparsification techniques. While semi-structured (N

hardwarearxiv-cs-ai

The DeepSeek V4 garbled output bug in open source inference engine is fixed in SGLang. To everyone affected over the weekend, sorry for the …

The DeepSeek V4 garbled output bug in open source inference engine is fixed in SGLang. To everyone affected over the weekend, sorry for the trouble. Huge thanks to @Ant_Group for landing the fix PR. I

model-releasesollama--x

If anyone wants to meet in New York City today, I'll be at Dark Matter Coffee, NYC, from 11:00AM to 12:30! https://maps.app.goo.gl/jsEibm89k…

Dylan Patel announced his availability to meet in person at Dark Matter Coffee in New York City on a specific date between 11:00 AM and 12:30 PM. The post included a Google Maps link to the coffee sho

hardwaredylan-patel--x

26 Apr 2026

🎉 Day-0 support for @deepseek_ai V4 Pro and Flash on vLLM — a new generation of DeepSeek model, purpose-built for tasks up to 1M tokens. Al…

model-releasesdylan-patel--x

🎉 Day-0 support for @deepseek_ai V4 Pro and Flash on vLLM — a new generation of DeepSeek model, purpose-built for tasks up to 1M tokens. Alongside the release, we're publishing a first-principles walk

MCAP: Deployment-Time Layer Profiling for Memory-Constrained LLM Inference

arXiv:2604.21026v1 Announce Type: new Abstract: Deploying large language models to heterogeneous hardware is often constrained by memory, not compute. We introduce MCAP (Monte Carlo Activation Profili

Serialisation Strategy Matters: How FHIR Data Format Affects LLM Medication Reconciliation

arXiv:2604.21076v1 Announce Type: cross Abstract: Medication reconciliation at clinical handoffs is a high-stakes, error-prone process. Large language models are increasingly proposed to assist with t

🔴 Trump vient de publier sur Truth Social un monologue de 1 500 mots sur le droit du sol, l'ACLU, les Indiens, les Chinois et la 'Savage Na…

🔴 Trump vient de publier sur Truth Social un monologue de 1 500 mots sur le droit du sol, l'ACLU, les Indiens, les Chinois et la 'Savage Nation'. Ce texte inhabituellement long, inhabituellement cohér

hardwareyann-lecun--x

Accelerating PayPal's Commerce Agent with Speculative Decoding: An Empirical Study on EAGLE3 with Fine-Tuned Nemotron Models

arXiv:2604.19767v1 Announce Type: cross Abstract: We evaluate speculative decoding with EAGLE3 as an inference-time optimization for PayPal's Commerce Agent, powered by a fine-tuned llama3.1-nemotron-

FlashNorm: Fast Normalization for Transformers

arXiv:2407.09577v4 Announce Type: replace Abstract: Normalization layers are ubiquitous in large language models (LLMs) yet represent a compute bottleneck: on hardware with distinct vector and matrix

Gaussians on a Diet: High-Quality Memory-Bounded 3D Gaussian Splatting Training

arXiv:2604.20046v1 Announce Type: new Abstract: 3D Gaussian Splatting (3DGS) has revolutionized novel view synthesis with high-quality rendering through continuous aggregations of millions of 3D Gauss

Local Ai

Klein 9b base nvfp4 on HF

local-air-stablediffusion

FLUX.2 [klein] 9B Base is a 9 billion parameter rectified flow transformer capable of generating images from text descriptions and supports multi-reference editing capabilities. The model fits in appr

Survival of the Cheapest: Cost-Aware Hardware Adaptation for Adversarial Robustness

arXiv:2409.07609v2 Announce Type: replace-cross Abstract: Deploying adversarially robust machine learning systems requires continuous trade-offs between robustness, cost, and latency. We present an au

Are Large Language Models Economically Viable for Industry Deployment?

model-releasesarxiv-cs-cl

arXiv:2604.19342v1 Announce Type: new Abstract: Generative AI-powered by Large Language Models (LLMs)-is increasingly deployed in industry across healthcare decision support, financial analytics, ente

Compute is king. I’m excited about the next phase of growth for Cursor, as their close partner.

Compute is king. I’m excited about the next phase of growth for Cursor, as their close partner. SpaceXAI and @cursor_ai are now working closely together to create the world’s best coding and knowledge

hardwaresonya-huang--x

Day 1 at Google Cloud Next ‘26 recap

model-releasesgoogle-cloud-ai

Last year at Google Cloud Next ‘25, we asked you to imagine a new future for AI. At Next ‘26, the question before you is how do you move AI into production across your entire enterprise? According to

EfficientPENet: Real-Time Depth Completion from Sparse LiDAR via Lightweight Multi-Modal Fusion

arXiv:2604.18790v1 Announce Type: new Abstract: Depth completion from sparse LiDAR measurements and corresponding RGB images is a prerequisite for accurate 3D perception in robotic systems. Existing m