Search: “inference”

arXiv:2608.10601v1 Announce Type: cross Abstract: Two instruments of EU digital law place inference at their centre and mean different things by it. Article 3(1) of the AI Act uses the capability to i

arXiv:2608.10439v1 Announce Type: new Abstract: Streaming video generation holds strong potential for world modeling, where future frames must be inferred online sequentially to form a continuous vide

arXiv:2608.10489v1 Announce Type: new Abstract: Abundant visual information strengthens vision-language model (VLM) perception, yet massive visual tokens raise inference costs. Existing visual token p

DigitalOcean launched Kimi K3 on day 0. It’s already one of the most popular models on the platform and across the market: second most likes on Hugging Face, sixth most traffic on OpenCode. Getting a

Automated lint: 20 errors, 8743 warnings, 3 info

arXiv:2607.12733v1 Announce Type: new Abstract: Large language models (LLMs) excel at pattern recognition and text generation, but their capacity for abductive inference - inferring latent hypotheses

Running frontier AI in production is demanding — accelerators to manage, latency to hold steady across continents, regulated data to keep in-region, and long-context requests to serve reliably. Claude

arXiv:2607.00170v1 Announce Type: cross Abstract: Thermodynamic computing devices based on the Ising model show great promise for low-power AI inference and edge computing, but scalable methods for tr

Protecting sensitive data used with AI is a critical part of our commitment to providing advanced and secure cloud infrastructure. Confidential Computing cryptographically protects data in use in hard

arXiv:2509.08318v2 Announce Type: replace Abstract: Early-exit cascades over a frozen convolutional backbone enable adaptive inference but suffer from three sources of train-inference mismatch: branch

arXiv:2604.23368v1 Announce Type: new Abstract: Event-Based Models (EBMs) infer biomarker progression from cross-sectional data but typically only as ordinal sequences and rely on rigid model assumpti

arXiv:2604.22050v1 Announce Type: cross Abstract: Transformers are mostly relying on softmax attention, which introduces quadratic complexity with respect to sequence length and remains a major bottle

arXiv:2509.22343v2 Announce Type: replace-cross Abstract: Reasoning capability is essential to ensure the factual correctness of the responses of transformer-based Large Language Models (LLMs), and ro

AI is evolving from answering questions to reasoning and taking action. Companies who want to lead in today’s agentic era require computing infrastructure designed and optimized for these new requirem

arXiv:2502.07415v2 Announce Type: replace-cross Abstract: Inferring the mechanical properties of soft tissues from measured deformations is a fundamental challenge in elastography. A rarely examined a

arXiv:2610.18014v1 Announce Type: cross Abstract: Non-prefix KV cache reuse is a promising approach for improving inference efficiency in multi-agent systems as agents share the same context blocks. H

arXiv:2610.16615v1 Announce Type: new Abstract: Visual token compression has emerged as a critical technique for reducing the inference cost of Vision-Language Models. Existing methods select tokens b

arXiv:2610.16293v1 Announce Type: new Abstract: Garment manipulation requires inferring both garment identity and complete configuration from partial, self-occluded observations. This creates a chicke

arXiv:2610.16448v1 Announce Type: new Abstract: Diffusion models, particularly Diffusion Transformers, achieve strong image and video generation quality but remain expensive at inference time due to r

I started Flama in 2018 to take a trained model to a production API with as little ceremony as possible. With 2.0 LLMs became first-class alongside classic models, and together with Miguel Durán-Olive

arXiv:2610.12741v1 Announce Type: new Abstract: We interpret Transformer networks as learned approximations to a hybrid probabilistic-inference procedure that combines max-product Loopy Belief Propaga

arXiv:2610.12582v1 Announce Type: new Abstract: A latent recurrent language model applies a shared block of layers repeatedly, so one checkpoint can trade inference compute for accuracy by changing th

arXiv:2610.07457v1 Announce Type: cross Abstract: Fine-grained mixed-precision quantization promises efficient large language model inference, but local precision choices can conflict with regular GPU

arXiv:2610.04938v1 Announce Type: new Abstract: We propose D-DOIT (Discrete Doob-Oriented Inference-time Transformation), a training-free and efficient adaptation method for discrete diffusion models

arXiv:2610.04396v1 Announce Type: cross Abstract: Split Computing (SC) enables efficient deployment of Deep Neural Networks (DNNs) by partitioning inference between edge devices and cloud servers. How

arXiv:2610.04784v1 Announce Type: cross Abstract: Traffic classification (TC) is crucial to secure Internet of Things (IoT) networks, whose edge nodes often operate under privacy, bandwidth, and energ

arXiv:2610.01847v1 Announce Type: cross Abstract: Model specifications define how large language models (LLMs) should behave, guiding alignment training, inference-time behavior, and evaluation. Yet t

arXiv:2610.00940v1 Announce Type: cross Abstract: We propose ReHoPER, an inference-only, zero-shot method that improves large language models' reasoning by generating and answering intermediate questi

arXiv:2610.00395v1 Announce Type: new Abstract: Inference-time steering enables pretrained diffusion models to satisfy new constraints without full retraining. However, specificity-aware generation is

arXiv:2609.38386v1 Announce Type: new Abstract: Concurrent autoregressive inference creates a fundamental interference problem: prefilling a newly arrived long prompt can delay tokens for requests tha

arXiv:2607.15870v2 Announce Type: replace Abstract: Human label variation in natural language inference is increasingly treated as signal rather than noise, but how much of it formal semantic structur

arXiv:2609.38984v1 Announce Type: cross Abstract: World-action models (WAMs) leverage pretrained video models to improve generalization in robot control by jointly predicting future visual states and

arXiv:2609.37664v1 Announce Type: new Abstract: Causal Normalizing Flows (CNFs) enable causal inference from observational data given the causal structure, but they assume fully observed training data

I think most people are sleeping on this inference engine. I tried multiple llama.cpp forks and none of them comes close to the inference speed of Strata. Initial version had some bugs with kv cache,

arXiv:2609.36636v1 Announce Type: cross Abstract: Looped language models (LoopLMs) increase computational depth through parameter sharing, offering a path to scale inference computation without adding

arXiv:2609.22866v1 Announce Type: new Abstract: We introduce Causilo, a tabular foundation model (TFM) that combines frontier predictive performance with exceptionally fast inference. On TabArena, Cau

arXiv:2609.22204v1 Announce Type: new Abstract: This exploratory pilot study evaluates the scope and perceived accuracy of personal information output from ongoing conversational interactions in gener

NVIDIA TensorRT’s multi‑device inference lets a single TensorRT network span multiple GPUs using NCCL‑backed collectives, preserving all standard optimizations and supported from TensorRT 11.0 onward.

arXiv:2609.18663v1 Announce Type: cross Abstract: Billion-parameter vision--language--action (VLA) policies demand substantial onboard power, while communication delays in remote inference hinder time

arXiv:2609.17556v1 Announce Type: cross Abstract: Edge-deployed AI operate under dynamically changing power budgets, reliability requirements, and input distributions, requiring continuous adaptation.

arXiv:2609.12322v1 Announce Type: cross Abstract: Affective computing has advanced wearable state inference, but on-device reasoning about whether, when, and how to intervene remains challenging. We p

arXiv:2609.10712v1 Announce Type: new Abstract: We study how model post-training and test-time inference design affect natural-language proof generation for hard olympiad mathematics. Starting from Ne

arXiv:2609.10627v1 Announce Type: cross Abstract: Machine learning explanations reveal model behavior beyond predictions, creating attack surfaces for model confidentiality and data privacy. We system

arXiv:2609.09721v1 Announce Type: new Abstract: Low-bit attention accelerates Transformer inference by moving the QK^op and PV matrix multiplications to FP8 or FP4 matrix engines. However, the softmax

I initially saw CED as just an efficiency improvement, but the more I read about it, the more it feels like an inference architecture leap. The encoder/decoder split has some pretty interesting implic

arXiv:2609.07833v1 Announce Type: cross Abstract: Weak gravitational lensing shear and convergence trace the distribution of baryonic and dark matter across space, making them a powerful probe of cosm