Search: “inference”

Using only information you actually have from my conversation history, memory, files, or other context available to you, build a practical “operating manual” for me. Do not flatter me, diagnose me, or

industryr-chatgpt

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Local Ai

Instance-Adaptive Prompts as Context for Time-Series Foundation Models

DGX agent

arXiv:2609.34786v1 Announce Type: new Abstract: Longer histories can improve time-series foundation models (TSFMs), but require substantially higher inference cost. We therefore ask whether contextual

local-aiarxiv-cs-lg

29 Sept 2026

OmniTide: Co-Designing Algorithms and Systems for Efficient On-Device Omni-LLM Streaming

arXiv:2609.34653v1 Announce Type: cross Abstract: On-device streaming omni-modal inference safeguards user privacy and eliminates prohibitive per-token API costs, but faces a critical bottleneck: the

P4Q: Co-designing Token Pruning and Quantization for Vision-Language Model Acceleration

arXiv:2609.34867v1 Announce Type: new Abstract: Vision language models have achieved strong performance across a wide range of multimodal applications, yet their substantial computational and memory c

researcharxiv-cs-cv

Agents

PReCache: Efficient KV Cache Sharing for Multi-LoRA Agents via Low-Rank Precomputation and Neutral Reconstruction

arXiv:2609.34054v1 Announce Type: new Abstract: Multi-LoRA agent systems enable efficient role specialization by sharing a common backbone model. However, each agent repeatedly processes the growing s

agentsarxiv-cs-lg

Prioritizing Repeated LLM Evaluation for Hidden Failure Discovery

model-releasesarxiv-cs-lg

arXiv:2609.32547v1 Announce Type: new Abstract: Large language models are commonly evaluated by generating a small number of stochastic responses for each prompt in a benchmark. Because inference budg

SCOPD: Sparse-Context On-Policy Self-Distillation for Efficient Vision-Language Models

arXiv:2609.34044v1 Announce Type: new Abstract: Reasoning vision-language models (VLMs) process images and videos as long sequences of visual tokens, making inference expensive. Training-free token pr

safetyarxiv-cs-cv

TerMeZO: Ternary Sparse Zeroth-Order Optimization for Fine-tuning BitNet Models at the Edge

model-releasesarxiv-cs-lg

arXiv:2609.33548v1 Announce Type: new Abstract: Fine-tuning anguage models (LLMs) with first-order optimizers requires a memory several times larger than that required for inference. Memory-efficient

Token-Disentangled Latent Test-Time Scaling for Vision-Language Reasoning

model-releasesarxiv-cs-cv

arXiv:2609.35228v1 Announce Type: new Abstract: Latent test-time scaling improves reasoning by refining hidden states during inference, but existing methods typically apply a single scalar reward to a

Applications

Using the biggest model for every task can work in a demo. In production, cost, latency, and task-specific quality can tell a different stor…

applicationsfireworks-ai--x

Using the biggest model for every task can work in a demo. In production, cost, latency, and task-specific quality can tell a different story. At ODSC AI West 2026, Rob Ferguson, VP of Technology and

You Only Edit Once: Incentivizing In-Context Capability of LLMs via Local Demonstration Refinement

arXiv:2609.33609v1 Announce Type: new Abstract: In-context learning (ICL) is crucial for boosting the inference performance of large language models (LLMs). However, the effectiveness of ICL in LLMs i

Agents

ActKV: Efficient LLM Agents through Action-Guided KV Cache Management

arXiv:2609.31395v1 Announce Type: cross Abstract: Agentic LLM inference accumulates long KV caches across iterative observation-reasoning-action loops, imposing substantial memory overhead and limitin

agentsarxiv-cs-ai

FRESHLATENT: Channel-Aware Latent Adaptation for Resource-Constrained Embodied VLM Perception

arXiv:2609.30629v1 Announce Type: cross Abstract: Mission-critical UAVs increasingly rely on split vision-language model (VLM) perception under tight onboard-resource and wireless-communication constr

hardwarearxiv-cs-cv

How NVIDIA DSX MaxLPS Maximizes AI Factory Throughput and Efficiency

NVIDIA’s DSX MaxLPS employs policy‑governed power sharing to dynamically reallocate power across GPU resources, enabling up to 40 % more GPUs within the same approved power budget. In a joint test at

hardwarenvidia-developer

Physical AI chip startup SiMa.ai raised a 150M Series C led by Fidelity and Amplify at a 1.45B valuation, aiming to compete with Nvidia's CUDA-based hardware (Kyt Dotson/SiliconANGLE)

Kyt Dotson / SiliconANGLE: Physical AI chip startup SiMa.ai raised a 150M Series C led by Fidelity and Amplify at a 1.45B valuation, aiming to compete with Nvidia's CUDA-based hardware — Custom physic

hardwaretechmeme

PTC-Decoder: Towards Intelligent SLMs on Offline Resource-Constrained Edge Devices

model-releasesarxiv-cs-ai

arXiv:2609.30836v1 Announce Type: new Abstract: Deploying small language models (SLMs) on offline, resource-constrained edge devices such as remote sensing satellites presents a fundamental challenge:

Softmax Reparameterization for Output-Head Quantization

model-releasesarxiv-cs-ai

arXiv:2609.31291v1 Announce Type: cross Abstract: Large vocabularies make output heads a substantial inference cost in small language models. We propose softmax reparameterization, a post-training met

Towards Universal Representation-Based Process Control

arXiv:2609.30790v1 Announce Type: new Abstract: Many temporal process learning and monitoring pipelines operate in local windows, making window-level decisions unavoidable in practice. In such setting

UniPrefill: Universal Long-Context Prefill Acceleration via Block-wise Dynamic Sparsification

arXiv:2605.06221v2 Announce Type: replace Abstract: As large language models (LLMs) continue to advance rapidly, they are becoming increasingly capable while simultaneously demanding ever-longer conte

researcharxiv-cs-cl

User Model Extraction via Belief Self-Distillation

arXiv:2609.31603v1 Announce Type: cross Abstract: Large language models (LLMs) implicitly infer attributes of their users and adapt their behavior accordingly, yet these beliefs remain difficult to in

safetyarxiv-cs-cl

CoreWeave’s next test: From GPU scarcity to a durable AI cloud

Ahead of CoreWeave Inc.’s Fully Connected conference in San Francisco, we have made a notable investment in proprietary customer research with Qualitate. Rather than simply repeat the earnings call, w

hardwaresiliconangle

26 Sept 2026

Beyond Spatial Benchmarks: From Spatial Reasoning to Navigation

model-releasesarxiv-cs-cv

arXiv:2609.29934v1 Announce Type: new Abstract: Does progress on spatial reasoning benchmarks translate into better navigation? Existing benchmarks test isolated inferences from images or videos, with

25 Sept 2026

FounRef: Robust, Structure-Preserving, and Fast Metric Refinement of Frozen Monocular Foundation Priors with Sparse Anchors

arXiv:2609.29224v1 Announce Type: new Abstract: Dense metric depth from cameras is essential to real-world 3D applications, yet achieving accuracy, faithful surface geometry, and fast inference simult

local-aiarxiv-cs-cv

25 Sept 2026

The Domestic Unprotected Zone: Algorithmic Governance and the Reproduction of Perpetrator Discourse in Conversational AI

arXiv:2609.28479v1 Announce Type: cross Abstract: Conversational AI increasingly mediates intimate-partner communication, and refusal logic at the inference layer now functions as a governance thresho

25 Sept 2026

Google is a Leader in the 2026 Gartner Magic Quadrant for Container Management

model-releasesgoogle-cloud-ai

We’re excited and proud to share that Gartner has recognized Google as a Leader for the fourth year in a row in the 2026 Gartner® Magic Quadrant™ for Container Management, based on its Completeness of

24 Sept 2026

Same Scores, Different Decisions: Evaluating JEV and Language Models for Legal Document Understanding

arXiv:2609.27678v1 Announce Type: new Abstract: Contract inference requires multiple judgments about a shared document, but aggregate accuracy can conceal changes in the individual decisions. Repeated

24 Sept 2026

Tools

Today, @FactoryAI joins the Specialized Intelligence Index with Legacy-Bench! The SII keeps growing, now with benchmarks testing how models …

Today, @FactoryAI joins the Specialized Intelligence Index with Legacy-Bench! The SII keeps growing, now with benchmarks testing how models handle the legacy software behind payroll, financial settlem

toolsfireworks-ai--x

24 Sept 2026

Capability-Aware Arbitration for Semantic Intent-Based Shared Control

arXiv:2609.25369v1 Announce Type: new Abstract: Shared control often allocates robot authority based on confidence in inferred human intent, assuming reliable autonomous execution. When this assumptio

safetyarxiv-cs-ro

Et Tu, Brute? Economic Misalignment in Personal AI Agents

model-releasesarxiv-cs-ai

arXiv:2609.24927v1 Announce Type: new Abstract: Personal AI agents make recommendations and take actions on people's behalf in high-stakes economic contexts, e.g., buying a flight, choosing health ins

Flash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memory-Efficient Diffusion LLMs

arXiv:2609.26796v1 Announce Type: new Abstract: Diffusion Large Language Models (dLLMs) have recently emerged as a promising alternative to autoregressive LLMs by enabling non-autoregressive text gene

hardwarearxiv-cs-cl

Learning 3D biophysical cell properties from 2D images and cell-population statistics

arXiv:2609.22410v1 Announce Type: new Abstract: Inferring 3D cellular properties from 2D microscopy is difficult when a reference instrument reports only population statistics rather than labels for i

local-aiarxiv-cs-ai

PICPIs: Prediction-Interval-Conditional Prediction Intervals

arXiv:2609.25388v1 Announce Type: cross Abstract: A classical question in statistics is which observable quantities to condition on when drawing inferences about unobservable targets. For conformal pr

TSS: Target-Side Sparsification for Speculative Decoding in Domain-Specific Large Language Models

arXiv:2609.26100v1 Announce Type: new Abstract: Speculative decoding accelerates large language model inference through collaboration between a lightweight draft model and a target verifier. Existing

Ada3Drift: Adaptive Training-Time Drifting for One-Step 3D Visuomotor Robotic Manipulation

arXiv:2603.11984v2 Announce Type: replace Abstract: Diffusion-based visuomotor policies model complex action distributions through iterative denoising, but repeated inference adds latency to robotic c

safetyarxiv-cs-cv

ARM: Attention with Routed-Memory for Learnable Sparse Control

arXiv:2609.24417v1 Announce Type: new Abstract: Despite advances in long-context inference, large language models (LLMs) remain fundamentally limited by the key-value (KV) caching mechanisms that are

safetyarxiv-cs-lg

AVTR-1: Open Stack for Real-Time Interactive Avatars

model-releasesarxiv-cs-cv

arXiv:2609.22913v1 Announce Type: new Abstract: Talking-head and dyadic models now achieve real-time inference, yet fast motion generation alone does not produce an interactive conversation. A live sy

Tutorials

Bayesian Deck-of-cards-based Ordinal Regression with Sequential Preference Elicitation

arXiv:2609.23212v1 Announce Type: cross Abstract: The Deck-of-cards-based Ordinal Regression (DOR) infers a value function from a ranking of reference alternatives in which the Decision Maker (DM) ins

tutorialsarxiv-cs-lg

BEACON: Belief-Enabled Adaptive CONtrol for Imitation Learning under Uncertainty

arXiv:2609.22730v1 Announce Type: new Abstract: Robot manipulation tasks often involve hidden state information that cannot be directly observed and must be inferred through sequential physical intera

safetyarxiv-cs-ro

Tutorials

Efficient Reasoning Exploration via State-Conditioned Latent Steering with Progress Guidance

arXiv:2609.24066v1 Announce Type: new Abstract: Best-of-N is a widely used inference strategy for complex reasoning, whose effectiveness depends on whether sampled candidates can cover diverse and hig

tutorialsarxiv-cs-cl

GenVoid: Uncertainty-Aware Learning of Subsurface Material Defects with an Experimentally Validated Physics-Informed Generative Model

arXiv:2609.23761v1 Announce Type: new Abstract: Internal voids are ubiquitous defects in manufactured structures, yet their characterization remains challenging because their geometry is hidden and ca

researcharxiv-cs-lg

KerColle: Unlocking Fine-Grained GPU Concurrency in Vision-Language-Action Models

arXiv:2609.22335v1 Announce Type: cross Abstract: Vision-Language-Action (VLA) models have emerged as foundational models for next-generation robotics. High VLA inference throughput is critical for me

hardwarearxiv-cs-ro

LLMs as Linguistic Chameleons: Decoupling Semantics and Structure for Privacy-Preserving Communication

arXiv:2609.23193v1 Announce Type: cross Abstract: As Large Language Model (LLM) APIs become increasingly integrated into privacy-sensitive workflows, ensuring inference-time privacy without compromisi

researcharxiv-cs-cl

RegVGGT: Sustainable Visual Geometry Grounding for Streaming via Regulated Memory

arXiv:2609.23286v1 Announce Type: new Abstract: 3D reconstruction from a lengthy video stream input poses a dilemma for feed-forward reconstruction models (FFRMs), that a whole-stream inference contex

hardwarearxiv-cs-cv

SCoP: Structured Constraint Parsing for Evidence-Space Control in Temporal Knowledge Graph Question Answering

arXiv:2609.22213v1 Announce Type: new Abstract: Temporal Knowledge Graph Question Answering (TKGQA) requires answer inference from evidence that is both structurally valid and temporally admissible. E

Agents

StepKV: Step-Aware KV Cache Compression for LLM Agents

arXiv:2609.22158v1 Announce Type: new Abstract: Key-value (KV) caching is essential for efficient autoregressive large language model (LLM) inference, but the cache grows linearly with context length,

agentsarxiv-cs-lg

The Right Future for Action: Learning Action-Relevant Predictive States in World Action Models

arXiv:2609.23369v1 Announce Type: new Abstract: Generation-free world action models (WAMs) retain future-video prediction during training but act from internal video features at inference, leaving unc

researcharxiv-cs-cv

VPRune: Efficient Training-free Pre-LLM Visual Token Pruning

arXiv:2609.24485v1 Announce Type: new Abstract: Visual token pruning is a promising approach to reducing the inference cost of large vision-language models (LVLMs), yet aggressive token reduction ofte

safetyarxiv-cs-cv

CORDS: Continuous Representations of Discrete Structures