Search: “inference”

arXiv:2605.09649v1 Announce Type: new Abstract: The key-value (KV) cache is a major bottleneck in long-context inference, where memory and computation grow with sequence length. Existing KV eviction m

arXiv:2605.08696v1 Announce Type: new Abstract: Over the last two decades, language modeling has experienced a shift from predominantly recurrent architectures that process tokens sequentially during

arXiv:2603.22586v2 Announce Type: replace Abstract: In-context learning (ICL) enables task adaptation at inference time by conditioning on demonstrations rather than updating model parameters. Althoug

arXiv:2605.06875v1 Announce Type: cross Abstract: Advanced driver-assistance systems (ADAS) require neural compute engines that deliver low-latency inference under strict power and area constraints. P

arXiv:2602.01166v2 Announce Type: replace Abstract: Vision-Language-Action (VLA) models benefit from chain-of-thought (CoT) reasoning, but existing approaches incur high inference overhead and rely on

arXiv:2605.07180v1 Announce Type: new Abstract: LLM agents achieve strong performance on complex reasoning tasks but incur high latency and compute cost. In practice, many queries fall within the capa

arXiv:2605.04754v1 Announce Type: new Abstract: Deep neural network (DNN) inference at the edge demands simultaneous improvements in accuracy, computational efficiency, and energy consumption. Approxi

arXiv:2605.01566v1 Announce Type: new Abstract: Advances in inference methods have enabled language models to improve their predictions without additional training. These methods often prioritize raw

arXiv:2605.04039v1 Announce Type: new Abstract: Clinical LLMs are often scaled by increasing model size, context length, retrieval complexity, or inference-time compute, with the implicit expectation

arXiv:2512.12744v3 Announce Type: replace Abstract: Activation sparsity offers a compelling route to accelerate large language model (LLM) inference by selectively suppressing hidden activations, yet

arXiv:2604.26412v1 Announce Type: new Abstract: Speculative decoding accelerates LLM inference, but SOTA hidden-state-based drafters suffer from long-range decay: draft accuracy degrades as the specul

arXiv:2512.12087v3 Announce Type: replace Abstract: The growing demand for long-context inference capabilities in Large Language Models (LLMs) has intensified the computational and memory bottlenecks

arXiv:2604.22901v1 Announce Type: new Abstract: Diffusion models achieve remarkable success in time series generation. However, slow inference limits their practical deployment. We propose E^2-CRF (Er

arXiv:2604.23821v1 Announce Type: cross Abstract: Autonomous neutron spectroscopy must solve three distinct tasks: detection (where is the signal?), inference (which Hamiltonian governs it?), and refi

arXiv:2604.24637v1 Announce Type: cross Abstract: Block-sequential continual learning demands that a single model both protect prior solutions from catastrophic forgetting and efficiently infer at inf

arXiv:2603.17573v2 Announce Type: replace-cross Abstract: Vision-Language-Action (VLA) Models have become the mainstream solution for robot control, but suffer from slow inference speeds. Speculative

arXiv:2604.24586v1 Announce Type: new Abstract: Single-image point cloud reconstruction must infer complete 3D geometry, including occluded parts, from a single RGB image. While diffusion-based recons

arXiv:2604.23514v1 Announce Type: cross Abstract: The health condition of components in civil infrastructures can be described by various discrete states according to their performance degradation. In

arXiv:2510.19020v2 Announce Type: replace-cross Abstract: We propose a new method for statistical inference in generalized linear models. In the overparameterized regime, Principal Component Regressio

arXiv:2601.11194v2 Announce Type: replace Abstract: We suggest a new multi-modal algorithm for joint inference of paired structurally aligned samples with Rectified Flow models. While some existing me

arXiv:2604.21489v1 Announce Type: cross Abstract: Multi-modal trajectory generation is essential for safe autonomous driving, yet existing diffusion-based planners suffer from high inference latency d

arXiv:2604.21057v1 Announce Type: new Abstract: The field of Language Reasoning Models (LRMs) has been very active over the past few years with advances in training and inference techniques enabling L

arXiv:2603.09046v2 Announce Type: replace-cross Abstract: Device-side Large Language Models (LLMs) have witnessed explosive growth, offering higher privacy and availability compared to cloud-side LLMs

arXiv:2604.19431v1 Announce Type: cross Abstract: Generative AI systems are known to amplify biases present in their training data. While several inference-time mitigation strategies have been propose

Today at Google Cloud Next, we’re announcing new capabilities in Google Distributed Cloud (GDC) that bring Gemini and our advanced AI stack to wherever your data is, so you don’t need to compromise be

arXiv:2503.23439v2 Announce Type: replace-cross Abstract: Spoken dialogue systems powered by large language models have demonstrated remarkable abilities in understanding human speech and generating a

arXiv:2604.19635v1 Announce Type: cross Abstract: While generative models have set new benchmarks for Target Speaker Extraction (TSE), their inherent reliance on global context precludes deployment in

arXiv:2604.16581v1 Announce Type: new Abstract: Neural Combinatorial Optimization (NCO) has emerged as a powerful framework for solving combinatorial optimization problems by integrating deep learning

arXiv:2503.15845v2 Announce Type: replace Abstract: Network-wide Traffic State Estimation (TSE), which aims to infer a complete image of network traffic states with sparsely deployed sensors, plays a

arXiv:2601.02970v2 Announce Type: replace Abstract: Self-Consistency improves reasoning reliability through multi-sample aggregation, but incurs substantial inference cost. Adaptive self-consistency m

arXiv:2604.16535v1 Announce Type: new Abstract: Test-time scaling (TTS) improves large language models (LLMs) by allocating additional compute at inference time. In practice, TTS is often achieved thr

arXiv:2511.17408v4 Announce Type: replace-cross Abstract: Probing has emerged as a promising method for monitoring large language models (LLMs), enabling cheap inference-time detection of concerning b

We built our internal AI engineering stack on the same products we ship. That means 20 million requests routed through AI Gateway, 241 billion tokens processed, and inference running on Workers AI, se

arXiv:2604.14853v1 Announce Type: new Abstract: Test-time compute scaling, the practice of spending extra computation during inference via repeated sampling, search, or extended reasoning, has become

arXiv:2604.14528v1 Announce Type: cross Abstract: Large Language Models (LLMs) achieve strong performance through extended inference-time deliberation, yet how their reasoning failures arise remains p

arXiv:2604.14388v1 Announce Type: new Abstract: Humans routinely infer taste, smell, texture, and even sound from food images a phenomenon well studied in cognitive science. However, prior vision lang

Amazon Bedrock now supports cost allocation by IAM principal (users and roles) in AWS Cost and Usage Report 2.0 and Cost Explorer, enabling customers to understand and attribute inference costs across

Running LLMs across Cloudflare’s network requires us to be smarter and more efficient about GPU memory bandwidth. That’s why we developed Unweight, a lossless inference-time compression system that ac

arXiv:2604.13226v1 Announce Type: new Abstract: Large Language Models (LLMs) rely heavily on Key-Value (KV) caching to minimize inference latency. However, standard KV caches are context-dependent: re

arXiv:2604.13291v1 Announce Type: new Abstract: Accurate characterization of subsurface heterogeneity is challenging but essential for applications such as reservoir pressure management, geothermal en

انتهت حجة 'ما عندي كرت شاشة قوي عشان أشغل ذكاء اصطناعي محلي'. علي بابا نزلت موديل Qwen3.6 بهندسة MoE. حجمه الكلي 35B (يعطيك ذكاء عالي جداً)، لكن وقت الـ Inference يستهلك 3B فقط! يعني أداء Enterprise ع

arXiv:2604.12719v1 Announce Type: new Abstract: The deployment of deep neural networks in safety-critical systems necessitates reliable and efficient uncertainty quantification (UQ). A practical and w

arXiv:2604.12852v1 Announce Type: new Abstract: Collaborative transport requires robots to infer partner intent through physical interaction while maintaining stable loco-manipulation. This becomes pa

arXiv:2604.12247v1 Announce Type: cross Abstract: Speculative decoding has emerged as a promising approach to accelerate autoregressive inference in large language models (LLMs). Self-draft methods, w

arXiv:2604.09686v1 Announce Type: new Abstract: Traditional neural network models for intent inference rely heavily on observable states and struggle to generalize across diverse tasks and dynamic env

arXiv:2604.09612v1 Announce Type: new Abstract: Theory of Mind, the capacity to explain and predict behavior by inferring hidden mental states, has become the dominant paradigm for social interaction