Search: “inference”

arXiv:2604.12452v1 Announce Type: new Abstract: Large language models (LLMs) face significant challenges in processing long contexts due to the linear growth of the key-value (KV) cache and quadratic

Load balancing and scaling LLM serving involves distributing inference requests across multiple model instances or GPUs to prevent bottlenecks and ensure consistent response times under varying traffi

arXiv:2604.05164v2 Announce Type: replace-cross Abstract: As LLM reasoning performance plateau, improving inference-time compute efficiency is crucial to mitigate overthinking and long thinking traces

arXiv:2604.12137v1 Announce Type: cross Abstract: Background: Randomized controlled trials (RCTs) are costly, time-consuming, and often infeasible, while treatment-effect estimation from observational

This Reddit thread likely discusses user frustrations with performance issues when running GLM-5.1 via Ollama's cloud inference option (`glm-5.1:cloud`), a flagship agentic coding model from Z.AI. Com

arXiv:2604.12782v1 Announce Type: cross Abstract: While 4-bit quantization is essential for high-throughput deployment of Large Language Models, activation outliers often lead to significant accuracy

Traditional data centers only stored, retrieved and processed data. In the generative and agentic AI era, these facilities have evolved into AI token factories. With AI inference becoming their primar

arXiv:2506.01979v4 Announce Type: replace-cross Abstract: Recently, speculative decoding (SD) has emerged as a promising technique to accelerate LLM inference by employing a small draft model to propo

arXiv:2507.12475v2 Announce Type: replace-cross Abstract: The St. Petersburg paradox presents a longstanding challenge in decision theory: its classical expected value diverges, yet no correspondingly

Build b8783 is a sequential incremental release of llama.cpp, the open-source C/C++ framework for running LLM inference locally and in the cloud. As with nearby builds in the b87xx series, it likely i

Build **b8786** is an incremental release of [llama.cpp](https://github.com/ggml-org/llama.cpp), the open-source C/C++ library for local LLM inference. Like other builds in the project's continuous re

Build b8787 is a tagged release of llama.cpp, an open-source C/C++ library for running large language model (LLM) inference locally or in the cloud with minimal setup. As with all llama.cpp builds, it

Build b8788 is an incremental release of llama.cpp, the open-source C/C++ framework for efficient LLM inference developed by ggml-org. Like other builds in the project's continuous release cycle, it l

Build b8789 is an incremental release of llama.cpp, the open-source C/C++ library for running large language model inference locally and in the cloud. Like other builds in the project's continuous rel

Build b8790 is an incremental automated release of llama.cpp, the open-source C/C++ library for efficient LLM inference on local hardware. Like other builds in the project's continuous release cycle,

Build b8791 is an incremental release of llama.cpp, the open-source C/C++ library for local LLM inference maintained by ggml-org on GitHub. Like other numbered builds in the project's rapid release ca

Build b8793 is a tagged release of the llama.cpp project, an open-source C/C++ framework for running large language model (LLM) inference locally or in the cloud with minimal setup. As part of llama.c

Build b8794 is an incremental release of llama.cpp, the open-source C/C++ library for running large language model (LLM) inference locally. Like other builds in the project's continuous release series

Build **b8795** is an incremental release of [llama.cpp](https://github.com/ggml-org/llama.cpp), the open-source C/C++ framework for running large language model inference locally and in the cloud. Li

arXiv:2604.09600v1 Announce Type: new Abstract: Temporal knowledge graph (TKG) reasoning aims to infer future facts at unseen timestamps from temporally evolving entities and relations. Despite recent

arXiv:2604.11320v1 Announce Type: new Abstract: Robot grasping of desktop object is widely used in intelligent manufacturing, logistics, and agriculture.Although vision-language models (VLMs) show str

arXiv:2604.10597v1 Announce Type: cross Abstract: State Space Models (SSMs), represented by the Mamba family, provide linear-time sequence modeling and are attractive for long-context inference. Yet p

arXiv:2504.05902v2 Announce Type: replace-cross Abstract: Backdoor attacks pose a serious threat to deep neural networks (DNNs), allowing adversaries to implant triggers for hidden behaviors in infere

arXiv:2604.09603v1 Announce Type: cross Abstract: Speculative Decoding promises to accelerate the inference of Large Language Models, yet its efficacy often degrades in production-grade serving. Exist

arXiv:2604.11416v1 Announce Type: new Abstract: Label-flipping attacks, which corrupt training labels to induce misclassifications at inference, remain a major threat to supervised learning models. Th

arXiv:2604.02408v2 Announce Type: replace Abstract: Asynchronous inference has emerged as a prevalent paradigm in robotic manipulation, achieving significant progress in ensuring trajectory smoothness

arXiv:2512.01390v3 Announce Type: replace Abstract: Real-image super-resolution (Real-ISR) seeks to recover HR images from LR inputs with mixed, unknown degradations. While diffusion models surpass GA

arXiv:2604.11788v1 Announce Type: new Abstract: High dynamic range (HDR) imagery offers a rich and faithful representation of scene radiance, but remains challenging for generative models due to its m

IC-LoRA (In-Context LoRA) enables conditioning video generation on reference video frames at inference time, allowing fine-grained video-to-video control on top of a text-to-video base model. Unlike t

arXiv:2604.11613v1 Announce Type: cross Abstract: Transformers can perform in-context classification from a few labeled examples, yet the inference-time algorithm remains opaque. We study multi-class

arXiv:2512.06713v3 Announce Type: replace-cross Abstract: Current LLM-based frameworks for text anonymization usually rely on remote API services from powerful LLMs, which creates an inherent privacy

arXiv:2603.22241v2 Announce Type: replace Abstract: Diffusion Language Models (DLMs) offer attractive advantages over Auto-Regressive (AR) models, such as full-attention parallel decoding and flexible

arXiv:2604.10593v1 Announce Type: new Abstract: Feed-forward geometric foundation models can infer dense point clouds and camera motion directly from RGB streams, providing priors for monocular SLAM.

arXiv:2603.23516v2 Announce Type: replace-cross Abstract: Long-term memory is a cornerstone of human intelligence. Enabling AI to process lifetime-scale information remains a long-standing pursuit in

arXiv:2510.02050v3 Announce Type: replace-cross Abstract: Improving statistical forecasts of tropical cyclone (TC) intensity is limited by complex nonlinear interactions and difficulty in identifying

arXiv:2512.07222v3 Announce Type: replace-cross Abstract: To address the trade-off between robustness and performance for robust VLM, we observe that function words could incur vulnerability of VLMs a

arXiv:2603.12277v3 Announce Type: replace-cross Abstract: Language models remain vulnerable to prompt injection attacks despite extensive safety training. We trace this failure to role confusion: mode

arXiv:2604.11501v1 Announce Type: cross Abstract: We compare two strategies for compressing the KV cache in transformer inference: rank reduction (discard dimensions) and quantization (keep all dimens

arXiv:2604.09963v1 Announce Type: cross Abstract: Microreboot enables fast recovery by restarting only the failing component, but in modern microservices naive restarts are unsafe: dense dependencies

arXiv:2508.11290v3 Announce Type: replace Abstract: LLMs increasingly exhibit over-refusal behavior, where safety mechanisms cause models to reject benign instructions that seemingly resemble harmful

arXiv:2604.09594v1 Announce Type: new Abstract: Spatial competence is the quality of maintaining a consistent internal representation of an environment and using it to infer discrete structure and pla

Manifest is a tool or platform that has added support for Ollama Cloud, enabling users to route AI model inference to cloud-hosted Ollama instances rather than relying solely on local GPU resources. T

arXiv:2604.10895v1 Announce Type: cross Abstract: For a robot to be called socially intelligent, it must be able to infer users internal states from their current behaviour, predict the users future b

arXiv:2604.09781v1 Announce Type: new Abstract: Vision-Language Models (VLMs) exhibit strong visual reasoning capabilities, yet they still struggle with 3D understanding. In particular, VLMs often fai

arXiv:2604.11564v1 Announce Type: new Abstract: Single-image super-resolution has progressed from deep convolutional baselines to stronger Transformer and state-space architectures, yet the correspond

arXiv:2604.10800v1 Announce Type: cross Abstract: Learned classifiers deployed in agentic pipelines face a fundamental reliability problem: predictions are probabilistic inferences, not verified concl