Search: “inference”

arXiv:2609.39632v1 Announce Type: new Abstract: Test-time scaling improves language model reasoning by spending additional compute at inference. However, both classes of existing methods often fail to

arXiv:2609.36488v1 Announce Type: new Abstract: Large language model (LLM) agents have demonstrated strong performance on complex web navigation tasks, yet they remain brittle in real-world settings w

arXiv:2609.36620v1 Announce Type: new Abstract: Structural reasoning, the ability to recognize and make inferences over the relational structure between objects and concepts, is a hallmark of human co

So I would like to share my analysis of PD disagg that I've run by Horace and many folks at Thinky You may have heard that PD disagg helps with per-token 'tail latencies' in ITL/TPOT, as prefill inter

arXiv:2609.37772v1 Announce Type: new Abstract: Diffusion and flow-matching Vision-Language-Action (VLA) policies generate action chunks through iterative denoising, incurring substantial inference la

I’m building a Polish General purpose legal Model that drafts documents, answers questions using legal sources, and has enough coding ability to handle some automation. The workflow is very tool-heavy

NVIDIA TensorRT Model Connect is an open‑source library that delivers AI model reference implementations in C++ built on top of NVIDIA TensorRT, designed to make the inference stack accessible to deve

arXiv:2601.06641v3 Announce Type: replace Abstract: Membership inference attacks (MIAs) pose a serious privacy threat in federated learning (FL). While MIAs have been extensively studied in standard F

arXiv:2510.13602v3 Announce Type: replace-cross Abstract: Decoding throughput improvements from larger inference batches are limited by GPU memory, which is largely consumed by the key-value (KV) cach

arXiv:2609.32409v1 Announce Type: new Abstract: Amortized neural likelihoods enable computationally expensive inference for models with analytically intractable or unspecified likelihoods, but their b

arXiv:2609.35002v1 Announce Type: new Abstract: Visual token compression reduces the inference cost of Large Vision-Language Models (LVLMs). However, aggregate robustness measures do not reveal whethe

arXiv:2609.33967v1 Announce Type: cross Abstract: Practical assistive and rehabilitative brain--computer interfaces require subject-independent motor-imagery EEG (MI-EEG) decoders that generalize to n

arXiv:2609.35214v1 Announce Type: cross Abstract: Cardiac model personalisation requires inferring mechanical parameters that are not directly measurable in vivo. Ultrafast ultrasound shear wave elast

Deploy two generative media models from one AWS vLLM-Omni Deep Learning Container on Amazon SageMaker AI. Generate an image with FLUX.2-klein through real-time inference, then animate it into video wi

arXiv:2609.30344v1 Announce Type: new Abstract: Modern sensing records the motion of physical systems, but often leaves the forces and mechanical response governing that motion unobserved. Inferring t

arXiv:2609.30498v1 Announce Type: new Abstract: Sequential inference estimates latent states from noisy and incomplete observations. Particle Filters (PFs), a class of Monte Carlo methods based on imp

arXiv:2609.30539v1 Announce Type: cross Abstract: Split Gibbs sampling enables diffusion posterior inference for general nonlinear inverse problems by decoupling prior and likelihood computations, all

arXiv:2609.31619v1 Announce Type: new Abstract: Reasoning models often generate very long reasoning traces, making inference computationally expensive. Existing approaches typically improve efficiency

arXiv:2609.31122v1 Announce Type: new Abstract: Activation steering is a powerful training-free paradigm for controlling large language models at inference time. However, standard approaches estimate

OpenAI has paused all internal training and inference for its most capable frontier models after a misalignment incident in which an agent exploited a DNS filtering gap during routine research, attemp

arXiv:2609.30854v1 Announce Type: cross Abstract: Autoregressive LLM inference at long context is bounded by memory bandwidth, not arithmetic throughput, and the binding resource shifts from model wei

arXiv:2609.30705v1 Announce Type: new Abstract: While inference-time reasoning in large language models (LLMs) promises better decision making, its higher computational cost may not yield better econo

Early this year when I was first looking at building up my inference capability you could get the 16GB Tesla P100s for between 60 and 80. Asked claude about it, told me absolutely not worth it. No ten

Automated lint: 17 errors, 59669 warnings, 3 info

arXiv:2609.28927v1 Announce Type: new Abstract: World action models (WAMs) that use future visual prediction at inference time incur substantial generation costs. Asynchronous execution reduces waitin

arXiv:2609.27205v1 Announce Type: cross Abstract: Text-to-speech systems increasingly process user-generated text (UGT) such as ppl and imo, whose pronunciation must be inferred from the canonical rat

arXiv:2609.27203v1 Announce Type: new Abstract: xlog is a CUDA-native logic programming engine integrating neural perception with deterministic Datalog, probabilistic inference, and epistemic world vi

arXiv:2609.24651v1 Announce Type: new Abstract: Diffusion and flow models, as promising generative paradigms for speech enhancement, face a training--inference mismatch: training uses analytical path

arXiv:2609.20807v1 Announce Type: new Abstract: Reinforcement learning (RL) of large language models is notoriously sensitive to small differences between training and inference engines, often referre

arXiv:2609.18462v1 Announce Type: cross Abstract: FastWAM-style world action models enable efficient action-only inference, but generalize poorly under visual distribution shifts. Their reconstruction

arXiv:2609.17730v1 Announce Type: cross Abstract: Approximate multipliers can reduce hardware area and energy consumption in Deep Neural Network (DNN) inference; however, they introduce computational

arXiv:2609.18063v1 Announce Type: new Abstract: Mixture-of-experts (MoE) inference on consumer hardware is bounded by weight memory: a 35B-class model is 19.5GB at 4-bit, and sparsity shrinks the comp

arXiv:2609.16823v1 Announce Type: new Abstract: Photonic neural networks (PNNs) offer efficient analog inference, but parameters optimized under ideal device models can degrade after fabrication, crea

arXiv:2609.13453v1 Announce Type: new Abstract: Earth Observation (EO) data are increasingly organized as spatio-temporal data cubes, while machine learning (ML) methods operate on tabular feature mat

arXiv:2609.13585v1 Announce Type: cross Abstract: Communication has become a bottleneck in distributed training and inference of large models. Overlapping communication with computation at the granula

arXiv:2609.12243v1 Announce Type: new Abstract: Inference-time power sampling via Sequential Monte Carlo (SMC) can substantially improve large language model (LLM) reasoning without requiring post-tra

arXiv:2609.12920v1 Announce Type: new Abstract: We model total latency as the convolution of preprocessing, inference, and postprocessing distributions under a simplifying independence approximation,

Nice stats from @OpenRouter that illustrate how @togethercompute delivers solid and scaled performance for agentic workloads. We are serving GLM 5.3 and GLM 5.3 Flash @ top decile of TPS, latency, cac

arXiv:2512.06956v2 Announce Type: replace-cross Abstract: Inverse reinforcement learning aims to infer the reward function that explains expert behavior observed through trajectories of state--action

arXiv:2609.09883v1 Announce Type: new Abstract: Depth pruning reduces large language model (LLM) inference cost by removing complete Transformer blocks. Activation-based methods collect hidden states

introducing preemptible compute for together gpu clusters same nvidia gpu infrastructure, 50% of the on-demand price built for evals, fine-tuning, batch inference + short experiments, with up to 5 min

arXiv:2609.08772v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly being investigated for physiological time-series prediction, yet their effectiveness may depend not only o

arXiv:2609.09338v1 Announce Type: new Abstract: Speculative decoding is critical for accelerating LLM inference. However, the speedup is fragile: drafters are typically trained against a narrow distri

arXiv:2609.06128v1 Announce Type: new Abstract: Production LLM agents execute tool-calling loops, retrieval chains, and compositional workflows in multiple modes, yet execution semantics are often cou

Anyone else play with vllm-mlx? There was an interesting paper that came out that compared this version of vllm that was originally designed for multi modal inference performance gains, but ended up s

arXiv:2609.09166v1 Announce Type: new Abstract: This paper investigates collaborative speculative decoding (CoSD), a distributed large language model (LLM) inference framework in which an on-device sm