Knowledge catalogue
arXiv:2609.16823v1 Announce Type: new Abstract: Photonic neural networks (PNNs) offer efficient analog inference, but parameters optimized under ideal device models can degrade after fabrication, crea
arXiv:2609.13453v1 Announce Type: new Abstract: Earth Observation (EO) data are increasingly organized as spatio-temporal data cubes, while machine learning (ML) methods operate on tabular feature mat
arXiv:2609.13585v1 Announce Type: cross Abstract: Communication has become a bottleneck in distributed training and inference of large models. Overlapping communication with computation at the granula
arXiv:2609.12243v1 Announce Type: new Abstract: Inference-time power sampling via Sequential Monte Carlo (SMC) can substantially improve large language model (LLM) reasoning without requiring post-tra
arXiv:2609.12920v1 Announce Type: new Abstract: We model total latency as the convolution of preprocessing, inference, and postprocessing distributions under a simplifying independence approximation,
Nice stats from @OpenRouter that illustrate how @togethercompute delivers solid and scaled performance for agentic workloads. We are serving GLM 5.3 and GLM 5.3 Flash @ top decile of TPS, latency, cac
arXiv:2512.06956v2 Announce Type: replace-cross Abstract: Inverse reinforcement learning aims to infer the reward function that explains expert behavior observed through trajectories of state--action
arXiv:2609.09883v1 Announce Type: new Abstract: Depth pruning reduces large language model (LLM) inference cost by removing complete Transformer blocks. Activation-based methods collect hidden states
introducing preemptible compute for together gpu clusters same nvidia gpu infrastructure, 50% of the on-demand price built for evals, fine-tuning, batch inference + short experiments, with up to 5 min
arXiv:2609.08772v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly being investigated for physiological time-series prediction, yet their effectiveness may depend not only o
arXiv:2609.09338v1 Announce Type: new Abstract: Speculative decoding is critical for accelerating LLM inference. However, the speedup is fragile: drafters are typically trained against a narrow distri
arXiv:2609.06128v1 Announce Type: new Abstract: Production LLM agents execute tool-calling loops, retrieval chains, and compositional workflows in multiple modes, yet execution semantics are often cou
Anyone else play with vllm-mlx? There was an interesting paper that came out that compared this version of vllm that was originally designed for multi modal inference performance gains, but ended up s
arXiv:2609.09166v1 Announce Type: new Abstract: This paper investigates collaborative speculative decoding (CoSD), a distributed large language model (LLM) inference framework in which an on-device sm
AI companies like @cohere build on both sides of the token economy: the models and the products that use them. By moving production workloads to NVIDIA Blackwell, Cohere lowered token costs, increased
arXiv:2609.07264v1 Announce Type: new Abstract: Inference energy per token drives the cost and carbon footprint of deployed transformers. It is dominated by dense matrix products that incur fused mult
A recap of August 2026 launches for AI builders across Amazon Bedrock, Amazon Bedrock AgentCore, and Strands: million-token context for OpenAI models, cross-Region inference, agents that run for up to
Lightbits Labs Ltd. today announced the general availability of Inferra, a software engine designed to improve the economics and performance of artificial intelligence inference by moving key-value ca
arXiv:2609.05320v1 Announce Type: new Abstract: Deepfake detection models often rely on high-quality inputs, fixed inference paths, and computationally expensive architectures, limiting their use in l
arXiv:2609.03502v1 Announce Type: cross Abstract: In low-resource settings, deploying TTS typically requires choosing between a large voice-cloning model with costly inference or a compact fixed-voice
arXiv:2609.03003v1 Announce Type: new Abstract: Causal inference is the practice of estimating the effect of a treatment or intervention from data. It traditionally requires a bespoke pipeline for eve
arXiv:2609.03807v1 Announce Type: cross Abstract: A free pause token gives a language model extra compute to form each next-token prediction (as a pause, or thinking, token does) but carries that comp
arXiv:2609.03343v1 Announce Type: cross Abstract: This paper addresses infinite-dimensional Bayesian inference for inverse problem of partial differential equations with model parameters in infinite-d
arXiv:2609.03158v1 Announce Type: cross Abstract: Visual token pruning reduces the inference cost of vision-language models (VLMs), but most methods only ask which tokens to keep. This retained-token
arXiv:2609.01778v1 Announce Type: new Abstract: Large-scale video retrieval requires embedding models to encode long and diverse videos under tight visual-input and inference budgets. Existing methods
arXiv:2609.01873v1 Announce Type: new Abstract: Multi-agent AI systems improve inference by spawning agents and synthesizing reports. But another agent is not another observation: apparently independe
arXiv:2609.01683v1 Announce Type: new Abstract: Vision models deployed on microcontrollers (MCUs) are quantized to integer-only arithmetic and run in inference-only runtimes that do not carry the mach
arXiv:2609.02689v1 Announce Type: new Abstract: Embedded devices typically lack the resources of GPU-equipped machines, and existing inference methods suffer from either high computational overhead (p
arXiv:2511.05187v2 Announce Type: replace Abstract: We study whether otherwise-idle inference resources could reduce the scarce-GPU cost of training. Our analysis uses a simulated compute ledger in wh
arXiv:2609.02029v1 Announce Type: new Abstract: Long-context inference retains a growing key--value (KV) cache during decoding, which consumes substantial GPU memory and can reduce generation throughp
arXiv:2609.01315v1 Announce Type: new Abstract: Building an omni-modal foundation model means evaluating it across text, image, video, and audio. Excellent evaluation toolkits exist for each modality,
arXiv:2609.00367v1 Announce Type: cross Abstract: Large Language Models are increasingly deployed for sophisticated data engineering tasks such as generating structured queries from natural language,
arXiv:2609.01224v1 Announce Type: new Abstract: Visual token pruning reduces the inference overhead of multimodal large language models (MLLMs) by retaining only a subset of visual tokens. Existing me
arXiv:2609.01004v1 Announce Type: cross Abstract: Despite their strong multimodal understanding ability, multimodal large language models (MLLMs) incur substantial computational overhead when processi
arXiv:2609.00057v1 Announce Type: cross Abstract: Value signals are aggregated user-level moral representations that capture users' inferred value-related tendencies from their online discourse. User
arXiv:2608.30263v1 Announce Type: cross Abstract: Large vision-language models (LVLMs) incur substantial inference costs due to their long and highly redundant visual-token sequences. Diversity-based
arXiv:2608.28696v1 Announce Type: new Abstract: Visual in-context learning (ICL) with multimodal large language models (MLLMs) is effective for fine-grained visual classification, but each retrieved i
Historically, enterprise predictive analytics tasks such as predicting churn, purchase intent, or fraud scoring have meant building custom models using libraries like XGBoost, Random Forest, or Deep N
arXiv:2608.29904v1 Announce Type: new Abstract: Modern video generators routinely fail at physical dynamics: objects float, trajectories violate gravity, contacts vanish. Standard denoising and flow-m
arXiv:2608.29107v1 Announce Type: new Abstract: While modern generative models excel at modeling complex data, precise inference-time control in conditional generation remains a critical challenge. Cl
arXiv:2608.21719v1 Announce Type: cross Abstract: AI inference clusters are increasingly constrained by instantaneous power, not just energy: grid operators condition new capacity on demand response,
arXiv:2608.29843v1 Announce Type: cross Abstract: Posted prices for AI inference have fallen steadily since 2024, yet the measured speed of that fall depends almost entirely on the method of measureme
arXiv:2206.09012v4 Announce Type: replace-cross Abstract: We consider the problem of inferring high-dimensional data mathbf{x} in a model that consists of a prior p(mathbf{x}) and an auxiliary differe
→ rollout throughput: generation speed and GPU utilization → weight sync: fresh weights on the rollout deployment every step → correctness: trainer and inference engine agreeing on the same model Cofo
I absolutely love Ollama for the majority of my inference needs. Starting to roll out Rehex to friends, and I've gotten requests to support OpenCode Go and Codex. Frankly, I'm really excited to add th
People who moved from older version to this new one, how much improvements do you see on inference? Ex: llama.cpp performance? This new version comes with Linux Kernel 7.0. Hope AMD cards gonna enjoy
Hey! I forked NInfer (a from-scratch C++20/CUDA inference engine for Qwen models) and added two things: tensor-parallel across two GPUs, and YaRN ×4 rope scaling. Together they let Qwen3.8-27B NVFP4 r
arXiv:2608.26926v1 Announce Type: new Abstract: Small language models (sLLMs) are nowadays hosted on devices with limited memory and computational budget. In an autoregressive setup, inference is memo
As in actual physical heat. How are people coping with the heat that running a decent inference rig pumps out? I've got dual 5060 Ti GPUs and a relatively modest CPU (Intel 14 Core Ultra 5 245KF Deskt
arXiv:2608.25727v1 Announce Type: new Abstract: Large language models (LLMs) have recently advanced graph neural networks (GNNs) by enriching node representations with semantic information, giving ris
arXiv:2509.06346v3 Announce Type: replace Abstract: Sparse Mixture-of-Experts (MoE) has become a key architecture for scaling large language models (LLMs) efficiently. Recent fine-grained MoE designs
arXiv:2608.24904v1 Announce Type: new Abstract: Inertial sensors at multiple body locations can improve activity recognition, but requiring every sensor at inference increases the deployment burden. W
arXiv:2608.24905v1 Announce Type: cross Abstract: Service robots may encounter ambiguous user requests that require context-aware inference. Users may also have unique preferences with certain tasks w
arXiv:2608.23641v1 Announce Type: new Abstract: Model welfare research infers what a model prefers from the answers returned to prompts written to elicit preferences. Keeling et al. (2024), Mazeika et
Recent advancements in large language models have intensified the need for efficient and deployable models within limited inference budgets. Structured pruning pipelines have shown promise in token ef
arXiv:2608.23880v1 Announce Type: new Abstract: Inferring the collective emotional state of a group of people from a single image, a task known as group emotion recognition (GER), requires integrating
arXiv:2608.20975v1 Announce Type: new Abstract: Effective social interaction requires agents to translate mental state inferences into coordinated behavioral signals across verbal and nonverbal channe
If you’re trying to run local LLMs on a normal laptop, an older desktop, integrated graphics, limited VRAM, or simply the hardware you already own, r/LowEndLocalAI is meant for you. The idea is simple
arXiv:2508.15813v2 Announce Type: replace-cross Abstract: A big issue in modern LLM applications is they tend to feed long context to LLM, which results in high inference cost and latency, and may exc
AirLLM dramatically reduces inference memory usage, letting 70B large language models run on a single 4GB GPU card — without quantization, distillation, or pruning. You can even run 405B Llama 3.1 on