Knowledge catalogue
arXiv:2610.06686v1 Announce Type: new Abstract: Long context inference with large language models becomes increasingly expensive as attention must operate over an ever growing KV cache. Page sparse at
arXiv:2610.05273v1 Announce Type: new Abstract: Visual token pruning is an effective way to accelerate vision-language models and is especially useful for vision-language-action (VLA) inference, where
arXiv:2610.03402v1 Announce Type: new Abstract: To deploy deep neural networks on edge hardware, highly efficient inference schemes are necessary that retain high accuracy. This work presents W16A16,
arXiv:2610.02598v1 Announce Type: new Abstract: Deploying LLMs on consumer-grade GPUs with insufficient memory to hold their weights can result in prohibitively slow inference, because decoding repeat
arXiv:2610.02442v1 Announce Type: new Abstract: The emerging of large language models (LLMs) has posed significant challenges to the thermal management of data center. Intense GPU computation for LLMs
arXiv:2610.03261v1 Announce Type: cross Abstract: Solving severely ill-posed imaging inverse problems requires recovering image structures that are unobservable or weakly constrained by the measuremen
arXiv:2610.03202v1 Announce Type: cross Abstract: Flow Matching enables high-quality visual generation via continuous-time dynamics, but inference remains costly due to multiple sequential function ev
arXiv:2610.03419v1 Announce Type: new Abstract: In this work we formulate ultrasound multistatic recovery from arbitrary transmit sequences as a Bayesian inference problem. To that end, we train a dee
arXiv:2610.03389v1 Announce Type: new Abstract: Vision language models (VLMs) incur substantial inference cost because every visual token is processed by the attention and MLP projections of every dec
arXiv:2610.03187v1 Announce Type: cross Abstract: Multi-camera streaming perception is increasingly deployed on heterogeneous edge platforms shared with co-resident workloads, yet accelerator placemen
arXiv:2610.03320v1 Announce Type: new Abstract: Diffusion language models for text-to-speech combine two forms of computation: model depth (parameters) and refinement steps (inference budget). We ask
arXiv:2610.02660v1 Announce Type: new Abstract: Diffusion-based world models enable high-quality interactive environment generation but suffer from substantial inference overhead due to repeated Trans
Automated lint: 19 errors, 64915 warnings, 3 info
I've been wanting to test out LLM inference on FPGAs for a while now, but didn't have a big reason too since there were no frontier class models at 9B-27B scale (3.6 27B was great still but it didn't
arXiv:2603.18480v2 Announce Type: replace-cross Abstract: Inferring human engagement from gameplay video is important for game design and player-experience research, yet it remains unclear whether vis
arXiv:2610.00405v1 Announce Type: new Abstract: Time-series foundation models (TSFMs) perform strongly across forecasting tasks, but their per-series inference is ill-suited to neighbor-rich forecasti
I have been building a somewhat unusual local inference machine around two Huawei Atlas 300I Duo cards. They are relatively inexpensive, passive, dual-accelerator PCIe cards with 96 GB of device memor
arXiv:2605.21603v1 Announce Type: cross Abstract: Intra-device parallelism addresses resource under-utilization in ML inference and training by overlapping the execution of operators with different re
arXiv:2609.40093v1 Announce Type: cross Abstract: Expert parallelism (EP) enables inference of large Mixture-of-Experts (MoE) models by placing their experts across multiple GPUs, but requires substan
arXiv:2609.38585v1 Announce Type: cross Abstract: Existing Large Language Model (LLM) routing methods score LLMs independently to select top-k models. However, this ignores model correlations and enfo
arXiv:2609.38332v1 Announce Type: cross Abstract: Test-time scaling improves model performance by allocating additional compute during inference. Using this compute effectively across multiple context
GPT-6 Astra Ultrafast, running on NVIDIA Blackwell GPUs, is available now in the OpenAI API and to eligible ChatGPT Work and Codex users. Accelerated by inference optimizations through OpenAI’s models
arXiv:2603.06350v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) increasingly adopt Mixture-of-Experts (MoE) architectures to scale efficiently under stringent resource constrain
arXiv:2609.38930v1 Announce Type: cross Abstract: In semantic segmentation, a recent line of RankSEG methods directly optimizes Dice/IoU scores at inference time, improving alignment with evaluation m
arXiv:2609.39078v1 Announce Type: new Abstract: Representations are routinely used across machine learning, psychology, and neuroscience to draw inferences about the computations of biological and art
arXiv:2609.39223v1 Announce Type: new Abstract: Large language model (LLM) inference is increasingly moving toward lower precision to realize the throughput of hardware accelerators, but aggressive po
arXiv:2609.38287v1 Announce Type: cross Abstract: Simulation-augmented generation (SAGE) is a recent technical proposal in which models simulate individuals' viewpoints at inference time in order to p
arXiv:2609.39334v1 Announce Type: cross Abstract: Test-time scaling has recently emerged as a powerful approach for improving LLM reasoning by allocating additional computation during inference, subst
arXiv:2609.39632v1 Announce Type: new Abstract: Test-time scaling improves language model reasoning by spending additional compute at inference. However, both classes of existing methods often fail to
arXiv:2609.36488v1 Announce Type: new Abstract: Large language model (LLM) agents have demonstrated strong performance on complex web navigation tasks, yet they remain brittle in real-world settings w
arXiv:2609.36620v1 Announce Type: new Abstract: Structural reasoning, the ability to recognize and make inferences over the relational structure between objects and concepts, is a hallmark of human co
So I would like to share my analysis of PD disagg that I've run by Horace and many folks at Thinky You may have heard that PD disagg helps with per-token 'tail latencies' in ITL/TPOT, as prefill inter
arXiv:2609.37772v1 Announce Type: new Abstract: Diffusion and flow-matching Vision-Language-Action (VLA) policies generate action chunks through iterative denoising, incurring substantial inference la
I’m building a Polish General purpose legal Model that drafts documents, answers questions using legal sources, and has enough coding ability to handle some automation. The workflow is very tool-heavy
NVIDIA TensorRT Model Connect is an open‑source library that delivers AI model reference implementations in C++ built on top of NVIDIA TensorRT, designed to make the inference stack accessible to deve
arXiv:2601.06641v3 Announce Type: replace Abstract: Membership inference attacks (MIAs) pose a serious privacy threat in federated learning (FL). While MIAs have been extensively studied in standard F
arXiv:2510.13602v3 Announce Type: replace-cross Abstract: Decoding throughput improvements from larger inference batches are limited by GPU memory, which is largely consumed by the key-value (KV) cach
arXiv:2609.32409v1 Announce Type: new Abstract: Amortized neural likelihoods enable computationally expensive inference for models with analytically intractable or unspecified likelihoods, but their b
arXiv:2609.35002v1 Announce Type: new Abstract: Visual token compression reduces the inference cost of Large Vision-Language Models (LVLMs). However, aggregate robustness measures do not reveal whethe
arXiv:2609.33967v1 Announce Type: cross Abstract: Practical assistive and rehabilitative brain--computer interfaces require subject-independent motor-imagery EEG (MI-EEG) decoders that generalize to n
arXiv:2609.35214v1 Announce Type: cross Abstract: Cardiac model personalisation requires inferring mechanical parameters that are not directly measurable in vivo. Ultrafast ultrasound shear wave elast
Deploy two generative media models from one AWS vLLM-Omni Deep Learning Container on Amazon SageMaker AI. Generate an image with FLUX.2-klein through real-time inference, then animate it into video wi
arXiv:2609.30344v1 Announce Type: new Abstract: Modern sensing records the motion of physical systems, but often leaves the forces and mechanical response governing that motion unobserved. Inferring t
arXiv:2609.30498v1 Announce Type: new Abstract: Sequential inference estimates latent states from noisy and incomplete observations. Particle Filters (PFs), a class of Monte Carlo methods based on imp
arXiv:2609.30539v1 Announce Type: cross Abstract: Split Gibbs sampling enables diffusion posterior inference for general nonlinear inverse problems by decoupling prior and likelihood computations, all
arXiv:2609.31619v1 Announce Type: new Abstract: Reasoning models often generate very long reasoning traces, making inference computationally expensive. Existing approaches typically improve efficiency
arXiv:2609.31122v1 Announce Type: new Abstract: Activation steering is a powerful training-free paradigm for controlling large language models at inference time. However, standard approaches estimate
OpenAI has paused all internal training and inference for its most capable frontier models after a misalignment incident in which an agent exploited a DNS filtering gap during routine research, attemp
arXiv:2609.30854v1 Announce Type: cross Abstract: Autoregressive LLM inference at long context is bounded by memory bandwidth, not arithmetic throughput, and the binding resource shifts from model wei
arXiv:2609.30705v1 Announce Type: new Abstract: While inference-time reasoning in large language models (LLMs) promises better decision making, its higher computational cost may not yield better econo
Early this year when I was first looking at building up my inference capability you could get the 16GB Tesla P100s for between 60 and 80. Asked claude about it, told me absolutely not worth it. No ten
Automated lint: 17 errors, 59669 warnings, 3 info
arXiv:2609.28927v1 Announce Type: new Abstract: World action models (WAMs) that use future visual prediction at inference time incur substantial generation costs. Asynchronous execution reduces waitin
arXiv:2609.27205v1 Announce Type: cross Abstract: Text-to-speech systems increasingly process user-generated text (UGT) such as ppl and imo, whose pronunciation must be inferred from the canonical rat
arXiv:2609.27203v1 Announce Type: new Abstract: xlog is a CUDA-native logic programming engine integrating neural perception with deterministic Datalog, probabilistic inference, and epistemic world vi
arXiv:2609.24651v1 Announce Type: new Abstract: Diffusion and flow models, as promising generative paradigms for speech enhancement, face a training--inference mismatch: training uses analytical path
arXiv:2609.20807v1 Announce Type: new Abstract: Reinforcement learning (RL) of large language models is notoriously sensitive to small differences between training and inference engines, often referre
arXiv:2609.18462v1 Announce Type: cross Abstract: FastWAM-style world action models enable efficient action-only inference, but generalize poorly under visual distribution shifts. Their reconstruction
arXiv:2609.17730v1 Announce Type: cross Abstract: Approximate multipliers can reduce hardware area and energy consumption in Deep Neural Network (DNN) inference; however, they introduce computational
arXiv:2609.18063v1 Announce Type: new Abstract: Mixture-of-experts (MoE) inference on consumer hardware is bounded by weight memory: a 35B-class model is 19.5GB at 4-bit, and sparsity shrinks the comp