Search: “inference”

Understanding Sparse Attention Selectivity in Long-Context Foundation Models via Counterfactual Evaluation

arXiv:2608.01676v1 Announce Type: new Abstract: Sparse attention is widely deployed in long-context serving stacks, yet no framework audits how discarding blocks changes the influence of specific cont

4 Aug 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Local Ai

Where Does Generative Difficulty Reside? An Empirical Study of Target Representations

DGX agent

arXiv:2608.00626v1 Announce Type: new Abstract: The target representation defines the distribution an image generator must learn, yet it is often treated as an interchangeable interface. This assumpti

local-aiarxiv-cs-cv

4 Aug 2026

WorldDynCache: Risk-Controlled Latent Dynamics Approximation for Diffusion World Model

arXiv:2608.01845v1 Announce Type: cross Abstract: Diffusion world models generate high-quality futures, but re- peated transformer evaluations make inference prohibitively slow. Existing caches reuse

researcharxiv-cs-cv

4 Aug 2026

Curriculum Matters: Data-Efficient Relational PFN Pretraining with Synthetic Data

arXiv:2607.29120v1 Announce Type: new Abstract: Relational Prior-Data Fitted Networks (PFNs) such as RDB-PFN approximate Bayesian inference over multi-table relational databases by pretraining on mill

'Data center in a Box (on Wheels)' 256Gb VRAM/512Gb RAM AI Server 6-8 Month Operational Review, Stability Write Up, Benchmarks

I've been out of these forums for awhile but I figured I would provide a formal update on how this has been going now that it has some operation time under its belt, just to put the information out th

Epistemic-aware Vision-Language Foundation Model for Fetal Ultrasound Interpretation

arXiv:2510.12953v4 Announce Type: replace-cross Abstract: Recent medical vision-language models have shown promise on tasks such as VQA, report generation, and anomaly detection. However, most are ada

researcharxiv-cs-ai

Matterhorn: Masked Time-to-First-Spike Encoding by Reassigning the Silent State for Sparse and Energy-Efficient Spiking Transformers

arXiv:2601.22876v2 Announce Type: replace Abstract: Spiking neural networks (SNNs) promise energy-efficient inference for large language models (LLMs), yet most reported savings rely on compute-operat

Metaphor-Induced Algorithmic Steering: Cross-Domain Procedural Transfer in LLM Code Generation

arXiv:2607.28683v1 Announce Type: cross Abstract: Large language models benefit from elements in natural language, such as metaphors and analogies in training data and inference input to achieve gener

researcharxiv-cs-ai

Tutorials

MOSAIC: Masked Outsourcing of Secure AI Computations

arXiv:2607.29221v1 Announce Type: cross Abstract: We address the challenge of securely and efficiently outsourcing AI computations from a trusted but computationally weak client to an untrusted but po

tutorialsarxiv-cs-ai

OsteoCAD: A Human-in-the-Loop Cloud-Edge Framework for Bone Tumor Segmentation

arXiv:2607.29266v1 Announce Type: cross Abstract: Artificial Intelligence (AI) and Deep Learning (DL) have notably advanced medical image analysis, yet many health- care organizations struggle to adop

hardwarearxiv-cs-ai

Agents

Outcome-Guided Distillation: A Teacher-Student Framework to Advance VLM Reasoning in Autonomous Driving

arXiv:2607.29052v1 Announce Type: new Abstract: End-to-end (E2E) autonomous driving aims to learn a direct mapping from visual observations to control actions. However, these E2E models often act as b

agentsarxiv-cs-ro

The math still ain’t mathing.

The math still ain’t mathing. Recently we estimated global (ex China) AI revenues of around 200bn annualised, based on four different sources. Just come across a fifth, based on Nvidia inference sales

hardwaregary-marcus--x

TraceViT: Grounded Trace Supervision for Visual Abstract Reasoning

arXiv:2607.29586v1 Announce Type: cross Abstract: The Abstraction and Reasoning Corpus (ARC) tests whether a model can infer an unseen transformation from a few input-output examples and apply it to a

safetyarxiv-cs-ai

two weeks ago i went on @swyx's pod and said some things that i... should not have said. a lot has happened since then, i owe you all an apo…

two weeks ago i went on @swyx's pod and said some things that i... should not have said. a lot has happened since then, i owe you all an apology. i'm sorry that i was right about every single thing. a

model-releasesswyx--x

I pushed Kimi K3 onto one CPU with 8 GB of RAM

I deployed K3 on 32 H100s at work a couple of weeks ago and then got annoyed that there was no way to poke at it on my own machine. So I wrote an inference engine for it in C99. Nothing clever going o

hardwarer-localllama

2 Aug 2026

Single system with dual cards or two systems with single cards?

So I am in a conundrum and I'm thinking of asking for your opinion for the following: Currently, I have a 5800X3D gaming rig with a 7900XTX with its 24GB VRAM. It seems that for this subreddit, this c

2 Aug 2026

Tutorials

AuricularWorld: Hierarchical Action-Guided World Modeling for Fine-Grained Auricular Structure Segmentation from CT Scans

arXiv:2607.28487v1 Announce Type: new Abstract: Fine-grained segmentation of auricular structures in CT is challenging because the ear occupies a small image region, cartilage boundaries are highly ir

tutorialsarxiv-cs-cv

DeepSeek v4 Flash for DS4 (DwarfStar) GGUF w/ DSpark MTP Head

I'm an avid user of Deepseek v4 Flash via antirez's DS4 DwarfStar inference engine, and so when the new checkpoint dropped, the first thing I did was rent a cloud box and spin up a quantization for us

DoTime: A Synthetic Benchmark Generator for Interventional and Counterfactual Time Series

arXiv:2607.27263v1 Announce Type: new Abstract: Most benchmarks for causal inference over time series are observational, small, or domain-specific, leaving interventional and counterfactual estimation

ENCORE: Event-Assisted Complementary Motion Refinement for Learned Video Compression

arXiv:2607.28020v1 Announce Type: new Abstract: Learned video compression relies on accurate temporal modeling to remove redundancy between adjacent frames. However, most existing codecs infer motion

researcharxiv-cs-cv

Objective-Aligned Direct Answer SFT for Robust Multi-Frame Medical VQA

model-releasesarxiv-cs-cv

arXiv:2607.27566v1 Announce Type: new Abstract: Multi-frame medical VQA appears to reward increasingly complex adaptation: controller-style inference, localization-aware reranking, static hard-negativ

Prox: Training-Free FFN Activation Sparsity via Approximate Intermediate-Channel Salience in LLMs

arXiv:2607.27591v1 Announce Type: cross Abstract: Feed-forward networks (FFNs) dominate memory traffic and computation in large language model (LLM) inference, making them a primary target for activat

Robotics and edge AI put new pressure on computing infrastructure

Physical AI is forcing the technology industry to rethink the entire computing stack. Robots, autonomous systems and intelligent devices need economical inference, secure data access and infrastructur

local-aisiliconangle

Split and Drive: Dual-Axis Disentanglement for Real-Time Gaussian Head Avatars

arXiv:2607.28032v1 Announce Type: new Abstract: Creating photorealistic animatable head avatars from a single image remains a fundamental challenge in digital human synthesis. While recent 3D Gaussian

hardwarearxiv-cs-cv

Budget-Aware LLM Discovery via Cost-Calibrated Frontier Utility

arXiv:2607.26828v1 Announce Type: new Abstract: Large language models increasingly support scientific and algorithmic discovery through inference-time search over evaluated candidates. Existing adapti

CineWeaver: Training-Free Reference-Controllable Multi-Shot Long Video Generation for Cinematic Storytelling

arXiv:2607.26529v1 Announce Type: new Abstract: Cinematic video generation is challenging for text-to-video diffusion models due to concurrent requirements on multi-shot generation, fine-grained contr

safetyarxiv-cs-cv

Applications

Dual-Path LLM Reasoning for Multimodal Few-Shot Knowledge Graph Completion

arXiv:2607.26909v1 Announce Type: new Abstract: Knowledge graph completion (KGC) aims to infer missing facts in knowledge graphs (KGs), thereby improving their completeness and supporting downstream i

applicationsarxiv-cs-cl

FedWeave: Rethinking the Unit of Specialization in Heterogeneous Federated MoE-LoRA

model-releasesarxiv-cs-cl

arXiv:2607.26618v1 Announce Type: cross Abstract: Federated PEFT enables LLMs to collaboratively adapt to decentralized private data without sharing raw examples. However, task heterogeneity across cl

Flow Map Learning via Nongradient Vector Flow

arXiv:2607.26398v1 Announce Type: new Abstract: Diffusion and flow-based models benefit from simple regression losses, but inference incurs significant overhead because sampling requires integration.

GLM 5.2 with vision on Hugging Face

Hi all, I have not seen this model talked about here but it seems like baseten (inference provider on OpenRouter) merged the vision encoder from Kimi k2.6 into GLM 5.2. I think the lack of vision was

local-air-localllama

I'm actually fairly bearish on frontier lab valuations. I've never seen the reasons articulated to my satisfaction, so before I go to sleep,…

model-releasesgary-marcus--x

I'm actually fairly bearish on frontier lab valuations. I've never seen the reasons articulated to my satisfaction, so before I go to sleep, I wanted to quickly jot down my thinking here. The basic is

JEPADepth: Masked Predictive Representation Learning for Self-Supervised Monocular Depth Estimation

model-releasesarxiv-cs-cv

arXiv:2607.26600v1 Announce Type: new Abstract: Self-supervised monocular depth estimation typically relies on photometric reconstruction losses that couple depth, pose, and appearance assumptions. In

Keyless Attention: Value-Space Routing and Value-Only Caching for Efficient Transformers

arXiv:2606.21848v2 Announce Type: replace Abstract: Transformer architectures form the foundation of modern natural language processing, making it crucial to address the efficiency and scalability lim

Lightweight Image Classification of Raptor Species for Edge Devices: Rare-Species Dataset Expansion via Video Frame Extraction, Knowledge Distillation, and TensorRT Deployment

arXiv:2607.26238v1 Announce Type: new Abstract: We investigate lightweight raptor-species classification for real-time edge deployment in wind-turbine collision mitigation. Using DINOv2-L (304M parame

hardwarearxiv-cs-cv

MedARC: Training-Free Adaptive Redundancy Compression of Visual Tokens for 3D Medical Vision-Language Models

arXiv:2607.26554v1 Announce Type: new Abstract: Integrating 3D medical images with vision-language models (VLMs) holds substantial promise for computer-aided diagnosis. However, volumetric images gene

local-aiarxiv-cs-cv

Agents

RAG-HAR+: Towards Cost-Efficient LLM-Based Human Activity Recognition for Edge Deployment

arXiv:2607.26631v1 Announce Type: new Abstract: Human Activity Recognition (HAR) from wearable sensors supports applications in healthcare, rehabilitation, fitness tracking, and smart environments. Ye

agentsarxiv-cs-lg

Revisiting Lossy Verification in Speculative Decoding: Mechanisms, Trade-offs, and Failure Modes

arXiv:2607.26627v1 Announce Type: new Abstract: Speculative Decoding (SD) accelerates large language model inference by allowing a lightweight draft model to propose tokens that are subsequently verif

RL^2-VLA: Adaptive RL Latent Compositional Steering with Test-Time Scaling for Vision-Language-Action Models

arXiv:2607.26991v1 Announce Type: new Abstract: Despite the impressive visuomotor capabilities enabled by Vision-Language-Action (VLA) models, their performance often degrades on challenging and out-o

safetyarxiv-cs-ro

Route by Kinematics, Act by Observation: Kinematics-Supervised Expert Routing in MoE-Augmented VLA

model-releasesarxiv-cs-ro

arXiv:2607.26807v1 Announce Type: new Abstract: While MoE augments VLA via expert specialization, router suffers from ineffective expert routing owing to the kinematic heterogeneity of actions across

SERPO: Self-Evolving Rubric Policy Optimization for Open-Ended Test-Time Reinforcement Learning

model-releasesarxiv-cs-cl

arXiv:2607.26873v1 Announce Type: new Abstract: Test-time reinforcement learning (TTRL) enables language models to self-evolve at inference time without labeled feedback. Existing methods rely on answ

Simultaneous Coverage and Efficiency Guarantee in Online Conformal Prediction

arXiv:2607.26577v1 Announce Type: new Abstract: Adaptive conformal inference (ACI) of Gibbs and Cand{es and its variants are the standard approach to online conformal prediction under distribution shi

Thinking Under Uncertainty: Evidence Use and Information-Seeking in Language Models

arXiv:2607.26845v1 Announce Type: new Abstract: Inference-time thinking improves the performance of large language models, but aggregate outcomes do not reveal whether models use available evidence mo

safetyarxiv-cs-lg

TurboVLA: Real-Time Vision-Language-Action Model at 32 Hz on an RTX 4090 with <1 GB VRAM

arXiv:2607.27205v1 Announce Type: new Abstract: Vision-language-action (VLA) models commonly adopt an LLM-centric V o L o A pathway, where visual observations are projected into the representation spa

local-aiarxiv-cs-cv

Would extremely high decode tok/s even be useful?

If you were able to get an inference machine that could do decode at 1k toks/s or even 10k tok/s, would that even be helpful? Would it unlock any new use cases? Let’s assume that this is for actually

Bits and Memories: Measuring Verbatim Extraction Across LLM Quantization

arXiv:2607.25451v1 Announce Type: new Abstract: Language models are almost always quantized before they are deployed, and a growing line of work asks whether quantization also lowers their privacy ris

Matrix-Free Photoacoustic Image Reconstruction via Sensor-Token Self-Attention

arXiv:2607.25576v1 Announce Type: new Abstract: Photoacoustic tomography (PAT) combines the optical absorption contrast of biological tissue with the spatial resolution of ultrasound, yet recovering t

researcharxiv-cs-ai

Applications

Rethinking Likelihood distributions: Student's t Likelihood Boosts Bayesian Neural Network Performance

arXiv:2607.25376v1 Announce Type: cross Abstract: In Bayesian neural networks (BNNs), variational inference is a widely adopted framework for modeling uncertainty in a distributional way, with the evi

applicationsarxiv-cs-ai

We ran a large-scale distillation attack on the Kimi K3 technical report by reading it in parallel at the Hugging Face Journal Club :) https…

We ran a large-scale distillation attack on the Kimi K3 technical report by reading it in parallel at the Hugging Face Journal Club :) https://youtu.be/MW8-kqd2SD8?si=jSKDogcUWbJ8N2k7 Our main takeawa

safetyclem-delangue--x