Search: “inference”

GSQ: Highly-Accurate Low-Precision Scalar Quantization for LLMs via Gumbel-Softmax Sampling

arXiv:2604.18556v1 Announce Type: new Abstract: Weight quantization has become a standard tool for efficient LLM deployment, especially for local inference, where models are now routinely served at 2-

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Applications

Introducing the O-Value: A Universal Standardization for Confusion-Matrix-Based Classification Performance Metrics

DGX agent

arXiv:2505.07033v2 Announce Type: replace-cross Abstract: Many classification performance metrics exist, each suited to a specific application. However, these metrics often differ in scale and can exh

applicationsarxiv-cs-lg

21 Apr 2026

Linear-Time and Constant-Memory Text Embeddings Based on Recurrent Language Models

arXiv:2604.18199v1 Announce Type: new Abstract: Transformer-based embedding models suffer from quadratic computational and linear memory complexity, limiting their utility for long sequences. We propo

MathFlow: Enhancing the Perceptual Flow of MLLMs for Visual Mathematical Problems

model-releasesarxiv-cs-cv

arXiv:2503.16549v2 Announce Type: replace Abstract: Despite strong results on many tasks, multimodal large language models (MLLMs) still underperform on visual mathematical problem solving, especially

Tutorials

Procedural Knowledge at Scale Improves Reasoning

arXiv:2604.01348v2 Announce Type: replace Abstract: Test-time scaling has emerged as an effective way to improve language models on challenging reasoning tasks. However, most existing methods treat ea

tutorialsarxiv-cs-cl

ReflexiCoder: Teaching Large Language Models to Self-Reflect on Generated Code and Self-Correct It via Reinforcement Learning

arXiv:2603.05863v2 Announce Type: replace Abstract: While Large Language Models (LLMs) have revolutionized code generation, standard ``System 1'' approaches that generate solutions in a single forward

Tutorials

River-LLM: Large Language Model Seamless Exit Based on KV Share

arXiv:2604.18396v1 Announce Type: new Abstract: Large Language Models (LLMs) have demonstrated exceptional performance across diverse domains but are increasingly constrained by high inference latency

tutorialsarxiv-cs-cl

SafeLM: Unified Privacy-Aware Optimization for Trustworthy Federated Large Language Models

arXiv:2604.16606v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly deployed in high-stakes domains, yet a unified treatment of their overlapping safety challenges remains

safetyarxiv-cs-lg

CASR: A Robust Cyclic Framework for Arbitrary Large-Scale Super-Resolution with Distribution Alignment and Self-Similarity Awareness

arXiv:2602.22159v2 Announce Type: replace Abstract: Arbitrary-Scale SR (ASISR) remains fundamentally limited by cross-scale distribution shift: once the inference scale leaves the training range, nois

safetyarxiv-cs-cv

ConFu: Contemplate the Future for Better Speculative Sampling

arXiv:2603.08899v2 Announce Type: replace Abstract: Speculative decoding has emerged as a powerful approach to accelerate large language model (LLM) inference by employing lightweight draft models to

Neuromorphic Parameter Estimation for Power Converter Health Monitoring Using Spiking Neural Networks

model-releasesarxiv-cs-lg

arXiv:2604.15714v1 Announce Type: cross Abstract: Always-on converter health monitoring demands sub-mW edge inference, a regime inaccessible to GPU-based physics-informed neural networks. This work se

Privacy-Preserving LLMs Routing

arXiv:2604.15728v1 Announce Type: cross Abstract: Large language model (LLM) routing has emerged as a critical strategy to balance model performance and cost-efficiency by dynamically selecting servic

Scalable Posterior Uncertainty for Flexible Density-Based Clustering

arXiv:2603.03188v2 Announce Type: replace-cross Abstract: We introduce a novel framework for uncertainty quantification in clustering that combines martingale posterior distributions with density-base

hardwarearxiv-cs-lg

VoodooNet: Achieving Analytic Ground States via High-Dimensional Random Projections

arXiv:2604.15613v1 Announce Type: cross Abstract: We present VoodooNet, a non-iterative neural architecture that replaces the stochastic gradient descent (SGD) paradigm with a closed-form analytic sol

local-aiarxiv-cs-ai

Acceptance Dynamics Across Cognitive Domains in Speculative Decoding

arXiv:2604.14682v1 Announce Type: cross Abstract: Speculative decoding accelerates large language model (LLM) inference. It uses a small draft model to propose a tree of future tokens. A larger target

AdaSplash-2: Faster Differentiable Sparse Attention

arXiv:2604.15180v1 Announce Type: cross Abstract: Sparse attention has been proposed as a way to alleviate the quadratic cost of transformers, a central bottleneck in long-context training. A promisin

hardwarearxiv-cs-cl

ConfLayers: Adaptive Confidence-based Layer Skipping for Self-Speculative Decoding

arXiv:2604.14612v1 Announce Type: cross Abstract: Self-speculative decoding is an inference technique for large language models designed to speed up generation without sacrificing output quality. It c

safetyarxiv-cs-cl

[Emerging Ideas] Artificial Tripartite Intelligence: A Bio-Inspired, Sensor-First Architecture for Physical AI

arXiv:2604.13959v1 Announce Type: new Abstract: As AI moves from data centers to robots and wearables, scaling ever-larger models becomes insufficient. Physical AI operates under tight latency, energy

safetyarxiv-cs-ai

From Tokens to Steps: Verification-Aware Speculative Decoding for Efficient Multi-Step Reasoning

arXiv:2604.15244v1 Announce Type: new Abstract: Speculative decoding (SD) accelerates large language model inference by allowing a lightweight draft model to propose outputs that a stronger target mod

RACER: Retrieval-Augmented Contextual Rapid Speculative Decoding

arXiv:2604.14885v1 Announce Type: new Abstract: Autoregressive decoding in Large Language Models (LLMs) generates one token per step, causing high inference latency. Speculative decoding (SD) mitigate

Revisiting Token Compression for Accelerating ViT-based Sparse Multi-View 3D Object Detectors

arXiv:2604.14563v1 Announce Type: new Abstract: Vision Transformer (ViT)-based sparse multi-view 3D object detectors have achieved remarkable accuracy but still suffer from high inference latency due

researcharxiv-cs-cv

Sixth-Sense: Self-Supervised Learning of Spatial Awareness of Humans from a Planar Lidar

arXiv:2502.21029v2 Announce Type: replace-cross Abstract: Reliable localization of people is fundamental for service and social robots that must operate in close interaction with humans. State-of-the-

local-aiarxiv-cs-lg

Social Story Frames: Contextual Reasoning about Narrative Intent and Reception

arXiv:2512.15925v2 Announce Type: replace Abstract: Reading stories evokes rich interpretive, affective, and evaluative responses, such as inferences about narrative intent or judgments about characte

Applications

Abstract 3D Perception for Spatial Intelligence in Vision-Language Models

arXiv:2511.10946v3 Announce Type: replace Abstract: Vision-language models (VLMs) struggle with 3D-related tasks such as spatial cognition and physical understanding, which are crucial for real-world

applicationsarxiv-cs-cv

16 Apr 2026

Applications

Graph In-Context Operator Networks for Generalizable Spatiotemporal Prediction

arXiv:2603.12725v3 Announce Type: replace Abstract: In-context operator learning enables neural networks to infer solution operators from contextual examples without weight updates. While prior work h

applicationsarxiv-cs-lg

16 Apr 2026

Why Multimodal In-Context Learning Lags Behind? Unveiling the Inner Mechanisms and Bottlenecks

arXiv:2604.13403v1 Announce Type: new Abstract: In-context learning (ICL) enables models to adapt to new tasks via inference-time demonstrations. Despite its success in large language models, the exte

safetyarxiv-cs-cv

16 Apr 2026

A Bayesian Perspective on the Role of Epistemic Uncertainty for Delayed Generalization in In-Context Learning

arXiv:2604.12434v1 Announce Type: cross Abstract: In-context learning enables transformers to adapt to new tasks from a few examples at inference time, while grokking highlights that this generalizati

researcharxiv-cs-lg

Adaptive Domain Models: Bayesian Evolution, Warm Rotation, and Principled Training for Geometric and Neuromorphic AI

arXiv:2603.18104v3 Announce Type: replace Abstract: Prevailing AI training infrastructure assumes reverse-mode automatic differentiation over IEEE-754 arithmetic. The memory overhead of training relat

Can AI Detect Life? Lessons from Artificial Life

arXiv:2604.11915v1 Announce Type: cross Abstract: Modern machine learning methods have been proposed to detect life in extraterrestrial samples, drawing on their ability to distinguish biotic from abi

Decidable By Construction: Design-Time Verification for Trustworthy AI

arXiv:2603.25414v2 Announce Type: replace-cross Abstract: A prevailing assumption in machine learning is that model correctness must be enforced after the fact. We observe that the properties determin

Evaluating Relational Reasoning in LLMs with REL

arXiv:2604.12176v1 Announce Type: new Abstract: Relational reasoning is the ability to infer relations that jointly bind multiple entities, attributes, or variables. This ability is central to scienti

Latent Chain-of-Thought World Modeling for End-to-End Driving

model-releasesarxiv-cs-cv

arXiv:2512.10226v2 Announce Type: replace Abstract: Recent Vision-Language-Action (VLA) models for autonomous driving explore inference-time reasoning as a way to improve driving performance and safet

Learning Chain Of Thoughts Prompts for Predicting Entities, Relations, and even Literals on Knowledge Graphs

arXiv:2604.12651v1 Announce Type: cross Abstract: Knowledge graph embedding (KGE) models perform well on link prediction but struggle with unseen entities, relations, and especially literals, limiting

Loss-Driven Bayesian Active Learning

arXiv:2604.11995v1 Announce Type: new Abstract: The central goal of active learning is to gather data that maximises downstream predictive performance, but popular approaches have limited flexibility

researcharxiv-cs-lg

PipeLive: Efficient Live In-place Pipeline Parallelism Reconfiguration for Dynamic LLM Serving

arXiv:2604.12171v1 Announce Type: cross Abstract: Pipeline parallelism (PP) is widely used to partition layers of large language models (LLMs) across GPUs, enabling scalable inference for large models

hardwarearxiv-cs-lg

Ride the Wave: Precision-Allocated Sparse Attention for Smooth Video Generation

arXiv:2604.12219v1 Announce Type: cross Abstract: Video Diffusion Transformers have revolutionized high-fidelity video generation but suffer from the massive computational burden of self-attention. Wh

local-aiarxiv-cs-ai

Tutorials

SubFlow: Sub-mode Conditioned Flow Matching for Diverse One-Step Generation

arXiv:2604.12273v1 Announce Type: cross Abstract: Flow matching has emerged as a powerful generative framework, with recent few-step methods achieving remarkable inference acceleration. However, we id

tutorialsarxiv-cs-cv

X-VC: Zero-shot Streaming Voice Conversion in Codec Space

arXiv:2604.12456v1 Announce Type: cross Abstract: Zero-shot voice conversion (VC) aims to convert a source utterance into the voice of an unseen target speaker while preserving its linguistic content.

CASK: Core-Aware Selective KV Compression for Reasoning Traces

arXiv:2604.10900v1 Announce Type: new Abstract: In large language models performing long-form reasoning, the KV cache grows rapidly with decode length, creating bottlenecks in memory and inference sta

hardwarearxiv-cs-ai

CodeComp: Structural KV Cache Compression for Agentic Coding

arXiv:2604.10235v1 Announce Type: new Abstract: Agentic code tasks such as fault localization and patch generation require processing long codebases under tight memory constraints, where the Key-Value

local-aiarxiv-cs-cl

ConfigSpec: Profiling-Based Configuration Selection for Distributed Edge--Cloud Speculative LLM Serving

arXiv:2604.09722v1 Announce Type: cross Abstract: Speculative decoding enables collaborative Large Language Model (LLM) inference across cloud and edge by separating lightweight token drafting from he

safetyarxiv-cs-ai

Do We Still Need GraphRAG? Benchmarking RAG and GraphRAG for Agentic Search Systems

arXiv:2604.09666v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) and its graph-based extensions (GraphRAG) are effective paradigms for improving large language model (LLM) reason

ExecTune: Effective Steering of Black-Box LLMs with Guide Models

arXiv:2604.09741v1 Announce Type: cross Abstract: For large language models deployed through black-box APIs, recurring inference costs often exceed one-time training costs. This motivates composed age

GeoArena: Evaluating Open-World Geographic Reasoning in Large Vision-Language Models

model-releasesarxiv-cs-cv

arXiv:2509.04334v4 Announce Type: replace Abstract: Geographic reasoning is a fundamental cognitive capability that requires models to infer plausible locations by synthesizing visual evidence with sp

IceCache: Memory-efficient KV-cache Management for Long-Sequence LLMs

arXiv:2604.10539v1 Announce Type: cross Abstract: Key-Value (KV) cache plays a crucial role in accelerating inference in large language models (LLMs) by storing intermediate attention states and avoid

hardwarearxiv-cs-ai

LingoLoop Attack: Trapping MLLMs via Linguistic Context and State Entrapment into Endless Loops

arXiv:2506.14493v3 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have shown great promise but require substantial computational resources during inference. Attackers can ex

Pioneer Agent: Continual Improvement of Small Language Models in Production

arXiv:2604.09791v1 Announce Type: new Abstract: Small language models are attractive for production deployment due to their low cost, fast inference, and ease of specialization. However, adapting them

RAG-KT: Cross-platform Explainable Knowledge Tracing with Multi-view Fusion Retrieval Generation

arXiv:2604.10960v1 Announce Type: new Abstract: Knowledge Tracing (KT) infers a student's knowledge state from past interactions to predict future performance. Conventional Deep Learning (DL)-based KT

safetyarxiv-cs-ai