Search: “inference”

arXiv:2608.29748v1 Announce Type: new Abstract: Speculative decoding accelerates autoregressive language model inference by having a lightweight draft model propose multiple candidate tokens, which ar

arXiv:2608.30070v1 Announce Type: new Abstract: Sparse representations are often expected to make models smaller and also reduce inference cost. For Fourier Neural Operators (FNOs), these objectives a

arXiv:2608.30703v1 Announce Type: cross Abstract: Runtime guardrails are essential for reliable large language model (LLM) deployment, yet existing approaches typically rely on independent, external m

arXiv:2608.30811v1 Announce Type: new Abstract: Long-context compression is essential for reducing the cost and latency of large language model inference. However, existing methods can fragment import

arXiv:2608.29869v1 Announce Type: new Abstract: Spiking neural networks (SNNs) are often regarded as energy-efficient alternatives to artificial neural networks (ANNs), yet their advantage depends cri

arXiv:2608.30424v1 Announce Type: cross Abstract: Large language models can support writing, but existing tools require users to explicitly articulate prompts-particularly burdensome in creative writi

arXiv:2608.29160v1 Announce Type: cross Abstract: We aim to improve frozen flow-matching image generators by adding inference computation inside the denoiser, without changing model weights or the out

arXiv:2602.01605v2 Announce Type: replace Abstract: Time Series Foundation Models (TSFMs) leverage extensive pretraining to accurately predict unseen time series during inference, without the need for

arXiv:2608.29635v1 Announce Type: new Abstract: We study unsupervised hypergraph alignment, where the goal is to infer node correspondences between two hypergraphs using only structural information, w

arXiv:2608.30204v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) process speech and text jointly, yet whether they exploit prosodic cues for pragmatic inference or rely on surf

vulkan: tune mat-vec rows for batched inference on Strix Halo (#27909) vulkan: RDNA3 static mat-vec rows above four columns On RDNA3 above four columns a static 4 rows for all types benches faster tha

arXiv:2608.28496v1 Announce Type: new Abstract: Two forms of test-time scaling for Large Language Models (LLMs) have emerged as effective and widely adopted paradigms: sequential, in which later answe

arXiv:2608.28059v1 Announce Type: cross Abstract: In-context learning (ICL) allows a pretrained model to infer a new task from examples supplied in its prompt without updating its parameters. In linea

arXiv:2608.27590v1 Announce Type: cross Abstract: Inferring elastic constants from resonant ultrasound spectra is a nonlinear and typically overdetermined inverse problem based on finite spectral data

arXiv:2504.04342v2 Announce Type: replace Abstract: Scaling up model parameters and training data consistently improves the performance of large language models (LLMs), but at the cost of rapidly grow

arXiv:2608.28444v1 Announce Type: new Abstract: Due to the nature of quadratic attention, Large Language Models (LLMs) consume a lot of memory and energy. Every new token costs more than the previous

With b10726, the default --lazy-mode change keeps the 51B-parameter PLE n-gram embedding table of Qwen 3.8 Flash Next on disk: it is mmap'd and its rows are read on demand during inference, even with

Hey guys I have been working on a open source LLM development repo. Anyone can train any amount of parameter against any arch (GQA, MLA, Dense...) When I tried to implement MLA I read the technical re

specs hardware: M4 Max 128GB Studio inference engine: llama.cpp (qwen4exp branch) judge: claude-opus-4-6 AtomicChat/Qwen3.8-Flash-Next-GGUF Qwen3.8-Flash-Next is a great model I benched in my previous

arXiv:2608.26291v1 Announce Type: new Abstract: Mentalization - the ability to infer others' beliefs and intentions to guide one's own choices - is a key cognitive function underlying human social int

arXiv:2608.27278v1 Announce Type: cross Abstract: Whole-slide images (WSIs) are central to computational pathology but are prohibitively large, making patch-based processing the practical unit for fou

arXiv:2608.26442v1 Announce Type: new Abstract: Recent advances in Large Language Models (LLMs) have shown that increased inference-time reasoning can improve performance on complex tasks. However, ma

arXiv:2608.27123v1 Announce Type: new Abstract: Conventional video editing primarily focuses on scene-level content, whereas live streaming places greater emphasis on the human subject. However, direc

arXiv:2608.26175v1 Announce Type: cross Abstract: Extractive prompt compression promises to cut LLM inference costs by removing low-information tokens, and learned compressors such as LLMLingua-2 repo

arXiv:2608.27286v1 Announce Type: new Abstract: Inferring molecular structures from multimodal spectroscopic measurements requires integrating complementary yet highly heterogeneous signals. However,

arXiv:2608.27137v1 Announce Type: cross Abstract: The recently envisioned goal-oriented communications paradigm requires machine learning inference to be performed directly on wirelessly transferred d

arXiv:2608.26673v1 Announce Type: new Abstract: Large pretrained vision-language-action models dominate modern robot-manipulation benchmarks, but it remains unclear how much model scale is necessary f

arXiv:2608.26120v1 Announce Type: new Abstract: Large Language Models (LLMs) are probabilistic models, typically defined by an autoregressive factorization. While recent work has begun to study richer

arXiv:2608.26418v1 Announce Type: cross Abstract: Modern AI workloads and the hardware that runs them evolve on different timescales: architectural definition precedes volume silicon by years, while t

arXiv:2608.26496v1 Announce Type: cross Abstract: Navigation in unknown environments to find unforeseen objects has become increasingly feasible with capable vision and language foundation models. How

Hey everyone, I am the co-founder of Tontaube. My brother and I just released TontaubeV1, a 2.9B-parameter open-weight TTS model focused on expressive speech, long-form generation, and low-latency loc

arXiv:2608.26093v1 Announce Type: new Abstract: Designing machine learning algorithms for wireless resource management is labour-intensive: the architecture, the loss function and the training recipe

arXiv:2510.15987v3 Announce Type: replace Abstract: How do inference time and latent computations enable large language models (LLMs) to solve multi-step reasoning problems? We introduce AlgoTrace, a

arXiv:2608.26004v1 Announce Type: cross Abstract: Agentic LLM pipelines face escalating inference costs as context accumulates across retrieval, tool use, and multi-turn interactions. To control laten

arXiv:2608.25023v1 Announce Type: cross Abstract: Multimodal product models can complete missing e-commerce attributes, yet current methods still optimize attribute-answer accuracy without verifying v

Self-hosted speech AI carries an observability trade-off: the numbers that drive capacity planning and cost management stay locked inside the vendor container. Deepgram closes that gap on Amazon SageM

arXiv:2608.24988v1 Announce Type: new Abstract: Activation steering can be embedded directly into a language model's weights, shaping behaviour without inference-time intervention and offering a way t

arXiv:2608.25583v1 Announce Type: new Abstract: Reasoning-oriented large language models often achieve strong problem-solving performance by generating long chains of thought, but this behavior substa

I’ve been working on a small project called gemma4.c. The idea is pretty simple: you can download a modern language model, compile one 700-line C file, and have it generate text on an ordinary CPU. Th

arXiv:2602.00686v2 Announce Type: replace Abstract: Vision-Language-Action (VLA) models have demonstrated remarkable generalization capabilities in robotic manipulation tasks, yet their substantial co

arXiv:2608.25235v1 Announce Type: cross Abstract: Selection of trustworthy collaborators in distributed systems is critical for efficient task completion, necessitating the inference of trustworthines

specs hardware: M4 Max 128GB Studio inference engine: oMLX & lllama.cpp insights it still very early, so had to disable oMLX K/V caching, qwen4_exp architectureis not yet supported + the obvious n-gra

arXiv:2603.03297v2 Announce Type: replace Abstract: Test-time training (TTT) adapts large language models (LLMs) during inference using only unlabeled test inputs. Existing methods, however, face two

arXiv:2607.18098v2 Announce Type: replace Abstract: Large language models are increasingly used in practical systems, making efficient model selection important for reducing deployment cost. LLM routi

I've been building a small tool opensource project that sits between applications and local LLM servers (starting with Ollama) to diagnose silent inference failures instead of just showing logs. Thing

arXiv:2608.25977v1 Announce Type: new Abstract: Personality is increasingly important in large language models (LLMs), as it shapes users' trust, engagement, and emotional experiences. While the Myers