Search: “inference”

A user reports successfully running Qwen 3.6 35b MoE (mixture of experts) with Zoo Code on an M1 Max Mac, achieving local inference without external servers. The setup enables fully local, battery-pow

arXiv:2605.29351v1 Announce Type: new Abstract: We study minimal attention-only transformers under all-token corruption and show they admit a two-stage empirical Bayes interpretation. A single attenti

b9394 is a release of llama.cpp, a project for LLM inference in C/C++ . The release represents a specific build or version update from the ggml-org/llama.cpp repository. Without access to the specific

B9402 is a release of llama.cpp, an open-source library that performs inference on large language models such as Llama and was developed in pure C/C++ with no dependencies. The release includes comman

b9410 is a release of llama.cpp, a C/C++ project that enables LLM inference with minimal setup and state-of-the-art performance on various hardware platforms. This build identifier represents a specif

b9411 is a release version of llama.cpp, a C/C++ implementation for LLM inference . This specific release build number represents updates to the open-source project hosted on GitHub, which provides op

arXiv:2602.06036v2 Announce Type: replace Abstract: Autoregressive large language models (LLMs) deliver strong performance but require inherently sequential decoding, leading to high inference latency

arXiv:2605.29626v1 Announce Type: cross Abstract: Steering language model generation toward desired textual properties is essential for practical deployment, and inference-time methods are particularl

arXiv:2605.29707v1 Announce Type: new Abstract: Speculative decoding accelerates LLM inference by drafting multiple tokens and verifying them in parallel with the target model. However, its practical

arXiv:2511.08949v2 Announce Type: replace Abstract: High-quality datasets are critical for training and evaluating reliable NLP models. In tasks like natural language inference (NLI), human label vari

Together AI developed an optimized speech-to-text system focused on achieving the fastest processing speeds through technical innovations in their inference stack and model optimization. The approach

arXiv:2605.30319v1 Announce Type: cross Abstract: A central goal of modern causal inference is estimating heterogeneous treatment effects to answer questions like 'how does an intervention affect each

love it! @ggerganov 's llama.cpp delivering! pibot is now running fully local, using parakeet for STT, qwen3-tts for TTS, and Qwen 3.6 as the local multi-modal LLM via llama.cpp. The STT and TTS infer

arXiv:2605.29766v1 Announce Type: new Abstract: Imitation learning has become a cornerstone for solving complex robotic manipulation tasks. In particular, multimodality, which enables robots to captur

arXiv:2604.14889v2 Announce Type: replace Abstract: While chain-of-thought (CoT) reasoning enables LLMs to solve challenging reasoning tasks, the linear growth of the KV cache leads to substantial mem

arXiv:2605.30126v1 Announce Type: cross Abstract: Large Vision-Language Models (LVLMs) map visual inputs into dense token sequences, imposing a quadratic computational bottleneck for inference. Elasti

pibot is now running fully local, using parakeet for STT, qwen3-tts for TTS, and Qwen 3.6 as the local multi-modal LLM via llama.cpp. The STT and TTS inference engines are Rust/mlx-c based. Ported fro

arXiv:2602.06791v2 Announce Type: replace Abstract: Being probabilistic models, during inference large language models (LLMs) display rare events: behaviour that is far from typical but highly signifi

arXiv:2605.29319v1 Announce Type: new Abstract: Large Reasoning Models (LRMs) achieve strong performance on table reasoning tasks but incur substantial inference cost due to long reasoning traces. Ste

arXiv:2605.30116v1 Announce Type: new Abstract: Distribution Matching Distillation (DMD) is a widely used paradigm for accelerating inference in few-step video diffusion models. However, DMD-style vid

Took me a while to figure out what all the ESMFold2 rage was about. At first, the benchmarking data didn't look super remarkable to me but it turns there are many impressive aspects: - Fully open sour

arXiv:2605.29141v1 Announce Type: cross Abstract: Traditional recommender systems (RecSys) primarily infer user preferences from implicit signals (such as clicks, watches, and purchases), often neglec

arXiv:2605.28575v1 Announce Type: new Abstract: Multimodal Sentiment Analysis (MSA) fuses text, acoustic, and visual streams to infer sentiment. Because pre-trained text encoders are far more expressi

arXiv:2605.27531v1 Announce Type: cross Abstract: Specification synthesis, the task of automatically inferring formal specifications from program implementations and natural language, is important for

b9375 is a release of llama.cpp, a project for LLM inference in C/C++. The release likely contains bug fixes, performance improvements, or feature additions to the llama.cpp codebase. Specific details

arXiv:2605.28292v1 Announce Type: new Abstract: Implicit Chain-of-Thought (CoT) reduces the inference cost of large language models by internalizing the explicit rationales. However, existing approach

Google created MapReduce more than 20 years ago to solve the scaling problems in data processing that the then young company was running into. The AI era that we are in now demands efficient, large-sc

arXiv:2605.27390v1 Announce Type: cross Abstract: Speculative decoding accelerates Large Language Model inference via a draft-then-verify paradigm, yet the output projection layer becomes a bottleneck

arXiv:2605.28751v1 Announce Type: cross Abstract: Linear interpolation between fine-tuned checkpoints has been shown to trace the Pareto front between competing objectives, but whether extrapolative w

arXiv:2508.15130v2 Announce Type: replace Abstract: Despite significant progress in no-reference image quality assessment (NR-IQA), dataset biases and reliance on subjective labels continue to hinder

arXiv:2605.28007v1 Announce Type: cross Abstract: Deep networks are powerful function approximators, but they typically store many different computations in shared weight matrices, making it difficult

arXiv:2509.22553v2 Announce Type: replace-cross Abstract: Causal representation learning (CRL) has garnered increasing interest from the causal inference and artificial intelligence communities due to

arXiv:2605.27787v1 Announce Type: cross Abstract: Multi-agent systems (MAS) have substantially advanced autonomous software engineering (SWE), but their growing inference energy demands raise sustaina

arXiv:2605.28646v1 Announce Type: cross Abstract: GUI agents rely on screenshots to infer intent and operate across applications, but these screenshots often contain private messages, medical records,

arXiv:2605.27616v1 Announce Type: cross Abstract: Real-time anomaly segmentation demands both high recall and efficient low-precision inference. We study the three-way interaction of model architectur

arXiv:2605.27762v1 Announce Type: new Abstract: We present PEAM, a Parametric Embodied Agent Memory framework in Minecraft that transforms agent memory from inference-time retrieval into parameter-res

arXiv:2605.27885v1 Announce Type: new Abstract: Various approaches have been proposed to adapt Vision-Language Models (VLMs) to specialized domains for Video Question Answering, including fine-tuning

arXiv:2508.21495v3 Announce Type: replace Abstract: Early-exit neural networks (EENNs) accelerate inference by allowing intermediate classifiers to stop computation once predictions are confident enou

arXiv:2605.27718v1 Announce Type: cross Abstract: Moment-based estimation is a theoretically attractive approach to parametric inference, especially when likelihood-based estimation is unavailable, mi

arXiv:2602.02259v2 Announce Type: replace-cross Abstract: Latent action models (LAMs) offer a promising path to pre-training embodied agents on large amounts of action-free video. They infer latent ac

arXiv:2605.27409v1 Announce Type: cross Abstract: SNNs promise energy-efficient and low-latency inference, but their performance still trails that of ANNs. ANN-to-SNN knowledge distillation helps narr

arXiv:2605.28764v1 Announce Type: new Abstract: Vast quantities of compute (GPU cycles on personal workstations, idle inference servers, and edge devices between jobs) go unused because no incentive-a

arXiv:2605.28713v1 Announce Type: new Abstract: Context compression aims to shorten long context inputs with minimal information loss for LLM inference acceleration. While existing methods have shown

arXiv:2602.07574v2 Announce Type: replace-cross Abstract: Modern multimodal large language models (MLLMs) adopt a unified self-attention design that processes visual and textual tokens at every Transf

arXiv:2605.27932v1 Announce Type: cross Abstract: Think-with-image reasoning is emerging as a new inference paradigm for large vision-language models, but its safety implications remain poorly underst

b9368 is an intermediate build release of llama.cpp, a C/C++ implementation for efficient LLM inference on consumer hardware. The release follows the project's rapid development cycle where multiple b