Search: “inference”

arXiv:2604.10900v1 Announce Type: new Abstract: In large language models performing long-form reasoning, the KV cache grows rapidly with decode length, creating bottlenecks in memory and inference sta

arXiv:2604.10235v1 Announce Type: new Abstract: Agentic code tasks such as fault localization and patch generation require processing long codebases under tight memory constraints, where the Key-Value

arXiv:2604.09722v1 Announce Type: cross Abstract: Speculative decoding enables collaborative Large Language Model (LLM) inference across cloud and edge by separating lightweight token drafting from he

Ollama is a local inference engine designed for running pre-built LLMs on your own machine, and it does not include fine-tuning capabilities — this distinction is correct. Unsloth (and its Unsloth Stu

arXiv:2604.09666v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) and its graph-based extensions (GraphRAG) are effective paradigms for improving large language model (LLM) reason

arXiv:2604.09741v1 Announce Type: cross Abstract: For large language models deployed through black-box APIs, recurring inference costs often exceed one-time training costs. This motivates composed age

arXiv:2509.04334v4 Announce Type: replace Abstract: Geographic reasoning is a fundamental cognitive capability that requires models to infer plausible locations by synthesizing visual evidence with sp

arXiv:2604.10539v1 Announce Type: cross Abstract: Key-Value (KV) cache plays a crucial role in accelerating inference in large language models (LLMs) by storing intermediate attention states and avoid

arXiv:2506.14493v3 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have shown great promise but require substantial computational resources during inference. Attackers can ex

arXiv:2604.09791v1 Announce Type: new Abstract: Small language models are attractive for production deployment due to their low cost, fast inference, and ease of specialization. However, adapting them

arXiv:2604.10960v1 Announce Type: new Abstract: Knowledge Tracing (KT) infers a student's knowledge state from past interactions to predict future performance. Conventional Deep Learning (DL)-based KT

arXiv:2604.02927v2 Announce Type: replace Abstract: Routing algorithms are crucial for efficient computer network operations, and in many settings they must be able to react to traffic bursts within m

arXiv:2604.08935v1 Announce Type: cross Abstract: Causal discovery from observational data remains a fundamental challenge in machine learning and statistics, particularly when variables represent inh

arXiv:2602.22812v2 Announce Type: replace Abstract: Since local LLM inference on resource-constrained edge devices imposes a severe performance bottleneck, this paper proposes distributed prompt cachi

Build **b8777** is an incremental tagged release of the [llama.cpp](https://github.com/ggml-org/llama.cpp) project, a C/C++ library focused on enabling efficient LLM inference across a wide range of l

arXiv:2512.11179v3 Announce Type: replace Abstract: Graph-based multi-agent reinforcement learning (MARL) enables coordinated behavior under partial observability by modeling agents as nodes and commu

DFlash, an optimized inference technique, has been integrated for the Kimi-K2.5 model and pushed to SGLang approximately 3 hours prior to the post. The implementation achieves acceptance lengths rangi

arXiv:2604.09405v1 Announce Type: new Abstract: As text-to-image diffusion models grow increasingly prevalent, the ability to remove specific concepts-mostly explicit content and many copyrighted char

Introducing DDTree: accelerates speculative decoding by drafting a tree with one block diffusion pass, then verifying multiple likely continuations together. Paper: https://liranringel.github.io/ddtre

arXiv:2501.15461v4 Announce Type: replace Abstract: Graph Neural Networks (GNNs) have shown great success in various graph-based learning tasks. However, it often faces the issue of over-smoothing as

arXiv:2604.09088v1 Announce Type: new Abstract: Memory-efficient transfer learning (METL) approaches have recently achieved promising performance in adapting pre-trained models to downstream tasks. Th

arXiv:2410.09355v2 Announce Type: cross Abstract: Generative Flow Networks (GFlowNets) are amortized inference models designed to sample from unnormalized distributions over composable objects, with a

arXiv:2604.09413v1 Announce Type: cross Abstract: This paper argues that a one-size-fits-all approach to specifying consent for the use of creative works in generative AI is insufficient. Real-world o

A Reddit post from r/ChatGPT where a user shares an example of ChatGPT successfully inferring meaning or intent from a vague or ambiguous prompt that lacked sufficient explicit context. The post likel

**llama.cpp release b8750** is a tagged build of [llama.cpp](https://github.com/ggml-org/llama.cpp), the open-source C/C++ inference engine for large language models maintained under the ggml-org G...

arXiv:2604.07758v1 Announce Type: new Abstract: Articulated objects are essential for embodied AI and world models, yet inferring their kinematics from a single closed-state image remains challenging

arXiv:2601.07994v4 Announce Type: replace Abstract: Large Language Models (LLMs) increasingly operate over long-form dialogues with frequent topic shifts. While recent LLMs support extended context wi

arXiv:2604.06515v1 Announce Type: cross Abstract: Sparse Mixture-of-Experts (MoE) allows scaling of language and vision models efficiently by activating only a small subset of experts per input. While

arXiv:2601.21708v2 Announce Type: replace Abstract: Large language models (LLMs) excel across many tasks, yet inference is still dominated by strictly token-by-token autoregression. Existing accelerat

arXiv:2604.07276v1 Announce Type: cross Abstract: GROMACS is a de-facto standard for classical Molecular Dynamics (MD). The rise of AI-driven interatomic potentials that pursue near-quantum accuracy a

arXiv:2603.22364v2 Announce Type: replace-cross Abstract: Diffusion models have achieved state-of-the-art performance in generative modeling, but their success often relies heavily on classifier-free

arXiv:2604.02073v2 Announce Type: replace Abstract: Universal multimodal embedding (UME) maps heterogeneous inputs into a shared retrieval space with a single model. Recent approaches improve UME by g

arXiv:2604.06228v1 Announce Type: cross Abstract: We introduce probabilistic language tries (PLTs), a unified representation that makes explicit the prefix structure implicitly defined by any generati

arXiv:2604.06204v1 Announce Type: cross Abstract: Personalization is essential for Large Language Model (LLM)-based agents to adapt to users' preferences and improve response quality and task performa

arXiv:2601.16294v2 Announce Type: replace-cross Abstract: General Matrix Multiplication (GEMM) is the cornerstone of HPC workloads and Deep Learning. State-of-the-art vendor libraries tune tensor layo

arXiv:2604.06644v1 Announce Type: cross Abstract: As deep learning inference is increasingly deployed in shared and cloud-based settings, a growing concern is input repurposing, in which data submitte

What's Changed openai: allow Responses effort on non-thinking models by @hoyyeva in #19010 llama-server: let llama-server infer the draft speculative type by @dhiltgen in #19064 mlx: add PPLX Decider

Hey all, been meaning to post this for a while. I've been building an inference engine based off Strata called Xolotl that runs DeepSeek-V4.1-Flash, all 552B of it, on one gaming PC. My setup is a sin

arXiv:2610.20641v1 Announce Type: new Abstract: Long chain-of-thought decoding is memory-bound: the KV cache of the generated tokens grows linearly with the test-time compute being spent. Evicting cac

arXiv:2610.21527v1 Announce Type: new Abstract: Layer normalization (LN) and root mean square normalization (RMSNorm) depend on their inputs and remain nonlinear at inference. Inserting either next to

arXiv:2610.21211v1 Announce Type: new Abstract: Proactive robots should anticipate user needs and provide assistance before being explicitly asked. A straightforward approach is to infer human intenti

arXiv:2610.20785v1 Announce Type: new Abstract: Large language models are increasingly used in social interactions, where appropriate responses require recognizing implicit cues, inferring mental stat

arXiv:2610.22137v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) achieve strong performance through instruction tuning, yet real-world applications often require them to contin

arXiv:2610.21126v1 Announce Type: new Abstract: Sparse KV compression reduces long-context inference costs by retaining selected historical states, yet these states typically keep their coordinates fr

arXiv:2610.22245v1 Announce Type: new Abstract: Small language models are increasingly deployed on edge devices and robotic systems to meet cost and latency constraints, yet their limited capabilities

arXiv:2610.22130v1 Announce Type: new Abstract: Key-value (KV) cache management has become a critical bottleneck for scaling Large Language Models (LLMs) to long-context scenarios due to linear memory