Search: “inference”

arXiv:2609.30566v1 Announce Type: new Abstract: We present a new inference-time sampler for diffusion models that gives a pretrained model a capability it was never trained for: constructing the atlas

arXiv:2609.30747v1 Announce Type: cross Abstract: As artificial intelligence (AI) becomes embedded in everyday life, its environmental footprint, particularly water consumption remains largely invisib

arXiv:2609.31360v1 Announce Type: new Abstract: Inference in LLMs is conventionally a fixed-depth, fixed-order forward pass through every layer, regardless of how difficult the input is. The human bra

The availability of resources for AI workloads can be challenging across the industry, especially accelerators. This can slow your AI workload deployment if it’s built around a specific type of accele

arXiv:2609.25085v1 Announce Type: cross Abstract: Data assimilation aims to infer the state of complex dynamical systems based on observational data. However, accurate inference of the multimodal post

arXiv:2609.22471v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) models are increasingly deployed alongside Speculative Decoding (SD) to accelerate inference, but combining the two is challe

arXiv:2602.13718v2 Announce Type: replace Abstract: Generative policies for robotic manipulation must balance action accuracy with inference latency. We present HybridFlow, a three-stage policy infere

arXiv:2609.23404v1 Announce Type: new Abstract: Point cloud completion aims to infer a complete 3D shape from a partial point cloud and serves as a fundamental building block for downstream tasks such

When running modern AI workloads, there’s often a conflict between performance and cost. Workloads like large language models (LLMs) load massive files, and may serve thousands of AI agents that need

arXiv:2609.20709v1 Announce Type: new Abstract: World Action Models (WAMs) improve robot policy learning by incorporating future dynamics, yet explicitly generating future videos at inference introduc

arXiv:2609.19729v1 Announce Type: new Abstract: Autoregressive (AR) video generation degrades over long horizons due to an overlooked train-inference discrepancy we term KV eviction mismatch: models t

arXiv:2609.17064v1 Announce Type: new Abstract: Assistive autonomous systems must anticipate human goals before an observed behavior is complete. This article formulates anticipation as goal inference

arXiv:2609.13846v1 Announce Type: cross Abstract: Delayed Tensor Parallelism (DTP) removes the blocking all-reduce of tensor-parallel Transformer inference. Every device adds its own partial output to

arXiv:2609.14717v1 Announce Type: cross Abstract: Speculative decoding accelerates LLM inference by verifying multiple drafted tokens in parallel, allowing a single target forward pass to accept sever

arXiv:2609.15504v1 Announce Type: cross Abstract: Orthrus is a hybrid autoregressive-diffusion architecture that accelerates autoregressive language-model inference by generating multiple tokens in pa

arXiv:2609.11961v1 Announce Type: new Abstract: Stress is a pervasive determinant of mental health and a key target for mobile health interventions. On-device language models (ODLMs) offer privacy-pre

arXiv:2607.09835v2 Announce Type: replace Abstract: The recently introduced YOLO26 architecture incorporates NMS-free end-to-end inference and is optimized for deployment on resource-constrained CPU-b

arXiv:2609.10346v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) process hundreds or thousands of visual tokens per image, incurring prohibitive inference costs. While existing

arXiv:2609.09254v1 Announce Type: new Abstract: Field-programmable gate arrays (FPGAs) enable efficient neural-network inference, but most deployment flows either accelerate multiply-accumulate operat

arXiv:2609.07483v1 Announce Type: new Abstract: Modus Tollens (MT) is a classical logical inference rule, while counterfactuals are hypothetical statements that are contrary to facts, and counterfactu

arXiv:2609.05800v1 Announce Type: new Abstract: Activation steering controls LLM behavior at inference time by adding learned directions to hidden states, but existing methods handle one concept at a

Confirmed by my weekly Hermes analytics. Estimated at roughly 50% cut in weekly and session limits. Which cloud inference plans are you all migrating to? (I understand that local inference is preferre

arXiv:2609.04775v1 Announce Type: new Abstract: Industrial anomaly detection faces two engineering bottlenecks: memory bank construction latency and inference efficiency. Traditional sampling algorith

arXiv:2609.02958v1 Announce Type: cross Abstract: Sensor-intensive environments enable many intelligent services by inferring user applications from heterogeneous data streams. However, not all applic

arXiv:2609.03235v1 Announce Type: new Abstract: Large language models (LLMs) face severe memory bottlenecks in long-context inference due to the linearly growing size of key-value (KV) caches. Existin

arXiv:2609.03602v1 Announce Type: new Abstract: World models (WMs) have demonstrated strong potential for end-to-end autonomous driving by learning predictive representations of future scene dynamics.

arXiv:2609.01798v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly deployed on mobile devices, making energy efficiency a key deployment constraint, yet the energy impact of

arXiv:2609.02006v1 Announce Type: cross Abstract: A compressed student has two shapes that need not agree: the weight it deploys at inference and the weight family its training can reach. We show that

arXiv:2609.00374v1 Announce Type: cross Abstract: Test-time adaptation (TTA) typically assumes that model parameters can be updated at inference time. This assumption is restrictive for inference-only

Open source is becoming the default way enterprises build with AI. Not just which model runs, but where it runs. Today we're partnering with @Equinix and @nvidia on Equinix Inference Exchange: our ope

arXiv:2609.00071v1 Announce Type: new Abstract: Prediction error is widely used to evaluate nuisance-function estimators in causal inference, but its relationship with causal estimator performance may

arXiv:2512.23765v2 Announce Type: replace-cross Abstract: Speculative decoding (SD) accelerates large language model (LLM) inference by using a lightweight draft model to propose tokens and a stronger

arXiv:2608.29768v1 Announce Type: new Abstract: Deploying robot policies in the physical world requires satisfying two fundamental desiderata: reliability and smooth real-time execution. However, depl

arXiv:2608.30310v1 Announce Type: cross Abstract: Hybrid large language models interleave full-attention layers with linear-attention layers to reduce the cost of long-context inference. This structur

arXiv:2608.30567v1 Announce Type: new Abstract: We present Turing-20B-A2B, a 20B-parameter Mixture-of-Experts language model that activates approximately 2B parameters per token, designed for long-con

arXiv:2608.28467v1 Announce Type: new Abstract: Persona-based dialogue systems are usually conditioned on speaker biography, but dialogues involve at least two participants, and who has access to whos

I’ve been experimenting with MiniMax H3 and finally have video generation working in TensorSharp. TensorSharp started primarily as a local GGUF/LLM inference engine, so getting a video-generation pipe

arXiv:2608.24938v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) models scale capacity for strong quality while keeping per-token compute bounded through sparse expert activation. Yet low-late

Amazon Bedrock now supports the OpenAI GPT-5.6 models, Terra and Luna, in India with India geographic cross-Region inference. If you have local data processing requirements, you can now use these mode

arXiv:2608.21621v1 Announce Type: new Abstract: In on-device NLP tasks, limited resources of embedded hardware, such as the Raspberry Pi 5, require efficient inference strategies. This paper introduce

Be sure to tune in this Wednesday! Inference Hours is back with another Kimi K3 webinar! On Wednesday, August 26, Zain Hasan sits down with Kevin Cui and Austin Silveria from Together AI's inference t

NVIDIA’s new GPUs in the Vera Rubin and Blackwell families set a new benchmark for agentic‑AI performance per watt on AgentX, an open‑source inference test that models realistic multi‑step, tool‑using

arXiv:2608.20316v1 Announce Type: new Abstract: Heterogeneous AI systems composed of multiple models, architectures, harnesses, or inference-time settings can improve quality and efficiency by routing

arXiv:2608.18656v1 Announce Type: new Abstract: Inference with transformer models on CPUs is increasingly important, especially for Small Language Models (SLMs), where vector architectures are emergin

arXiv:2608.18410v1 Announce Type: new Abstract: Vision-Language-Action (VLA) models process long multimodal token sequences, making inference expensive in both memory and computation. Existing efficie

arXiv:2608.15065v1 Announce Type: new Abstract: Large Reasoning Models produce diverse, sometimes inconsistent answers across repeated queries on the same problem, so multi-sample inference is a prere