Search: “inference”

Measured Joules, Learned Routes: Learning to Route for Energy-Efficient LLM Serving

arXiv:2609.23085v1 Announce Type: cross Abstract: Large language models (LLMs) and agentic AI systems are creating rapidly growing inference energy demands as model sizes grow and reasoning trajectori

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Safety

MIRA: Real-Time Full-Duplex Human-Robot Interaction for Embodied Companions

DGX agent

arXiv:2609.24547v1 Announce Type: new Abstract: Real-time embodied companion interaction requires a robot to infer user intent from streaming speech, generate timely responses, and execute expressive,

safetyarxiv-cs-ro

22 Sept 2026

NAVIR: Neuromorphic Audio-Visual Speech Recognition for Robust Human-Robot Interaction on Edge Hardware

arXiv:2609.24391v1 Announce Type: new Abstract: Voice-controlled interaction in industrial settings is hampered by acoustic noise, which severely degrades audio-only speech recognition. Audio-visual s

Not All Ranks Are Equal: Budget-Aware LoRA Merging Across Tasks

arXiv:2609.22237v1 Announce Type: new Abstract: Merging low-rank adapters (LoRAs) promises to eliminate the overhead of swapping task-specific weights at inference time. However, existing merging meth

researcharxiv-cs-lg

On the Efficiency-Safety Dilemma in Large Reasoning Models

arXiv:2609.23587v1 Announce Type: new Abstract: Large reasoning models (LRMs) incur high inference costs, often mitigated by efficiency techniques like quantization and pruning. However, the impact of

safetyarxiv-cs-cl

SG-FSM: A Self-Guiding Zero-Shot Prompting Paradigm for Multi-Hop Question Answering Based on Finite State Machine

arXiv:2410.17021v2 Announce Type: replace Abstract: Large Language Models with chain-of-thought prompting, such as OpenAI-o1, have shown impressive capabilities in natural language inference tasks. Ho

researcharxiv-cs-cl

TreeSpark: Calibrated, Load-Adaptive Draft Trees for Semi-Autoregressive Speculative Decoding

arXiv:2609.22098v1 Announce Type: new Abstract: Speculative decoding accelerates language-model inference by letting a cheap drafter propose tokens that the target model verifies in parallel. Recent b

safetyarxiv-cs-cl

Ability-Residual Decoupled Modeling for Affective Cognitive Diagnosis

arXiv:2609.21214v1 Announce Type: new Abstract: Cognitive diagnosis infers students' concept mastery from response logs. However, students' responses are not determined by mastery alone: non-cognitive

safetyarxiv-cs-ai

Accelerating Dense LLMs via L0-regularized Mixture-of-Experts

arXiv:2609.21672v1 Announce Type: new Abstract: Large language models (LLMs) achieve strong performance but suffer from slow and costly inference. Existing acceleration methods often lead to noticeabl

researcharxiv-cs-ai

Bayesian classification of astronomical spectra with class uncertainties

arXiv:2609.21694v1 Announce Type: cross Abstract: Context: We developed a probabilistic machine learning method with the aim of performing the O(10)-way classification of low- and high-resolution spec

researcharxiv-cs-lg

Deterministic Kittens: fun with Qwen Image 2.1 on an M2 Macbook Pro

Having a blast with Qwen Image 2.1 on my M2 Macbook Pro with 32GB RAM, even though it takes 16 minutes to complete the recommended number of iterations with the resolution reduced to 1024x1024. And th

How is anyone getting anything done with Ollama cloud?

I tried putting up with it the last week or two but it's become unusable. I've tried GLM 5.3 & 5.3 Flash, GLM 5.2, and Deepseek V4.1 The inference and pp speeds have become unbelievably slow, I just c

model-releasesr-ollama

Agents

MIRAGE: Multi-Perspective Creative Language Model Reasoning with Reinforcement Learning Guidance

arXiv:2609.21554v1 Announce Type: new Abstract: Recent advances in Large Language Models (LLMs) have revolutionized artificial intelligence and how human interact with AIs. Despite impressive advancem

agentsarxiv-cs-cl

PRIME: Perception Feedback with Situational Memory Embeddings in VLA Models

model-releasesarxiv-cs-cv

arXiv:2609.22040v1 Announce Type: new Abstract: Current Vision-Language-Action (VLA) models for autonomous driving operate primarily through feedforward inference across the perception--reasoning--pla

Hardware

RBS-Attention: Radius-Bounded Sparse Prefill for Long-Context Large Language Models

arXiv:2609.20971v1 Announce Type: new Abstract: Long-context large language model inference is increasingly limited by prefill, where dense self-attention processes the entire prompt before generation

hardwarearxiv-cs-ai

RheoSampling: Resolving the One-Hot Dilemma in Stochastic Dynamic-Tree Speculative Decoding

arXiv:2609.21827v1 Announce Type: new Abstract: Speculative decoding accelerates LLM inference by drafting multiple tokens in parallel, with tree-based methods further improving efficiency through hie

researcharxiv-cs-cl

Traffic Sign Recognition for Autonomous Driving Using Branched YOLOv2 and Geometric Features

arXiv:2609.22060v1 Announce Type: new Abstract: Traffic sign recognition (TSR) is an important perception task for autonomous driving and advanced driver-assistance systems, where a system must both l

local-aiarxiv-cs-cv

Understanding LLM Quantization through Activation-Guided Compensation and Orthogonal Residuals

arXiv:2609.21450v1 Announce Type: new Abstract: Post-training weight-activation quantization reduces the memory and inference costs of large language models, but aggressive W4A4 quantization remains d

When Online Adaptation Hurts: Parameter-Frozen Test-Time Ensembling for Continual Medical Image Segmentation

model-releasesarxiv-cs-cv

arXiv:2609.21412v1 Announce Type: new Abstract: Medical image segmenters often get worse when sites, scanner vendors, or protocols change. Continual test-time adaptation (CTTA) addresses this problem

RTX 5090 random black screen + 100% fans + hard lock; sometimes no POST until AC power is removed — gaming, ComfyUI and Ollama

RTX 5090 random black screen + 100% fans + hard lock; sometimes no POST until AC power is removed — gaming, ComfyUI and Ollama I am looking for help diagnosing a very intermittent but increasingly fre

local-air-ollama

20 Sept 2026

The bear can dance: Qwen 3.8 27B on one 3090 for 3 weeks

TL;DR: Local agent loop, ~21 days, one RTX 3090. Task was pretty much 'build a CUDA inference engine for optimized for yourself on this GPU arch.' Got working kernels and benches, not a win over llama

20 Sept 2026

Digit-logits-based classifier with llama.cpp

Although I do not completely understand what the big deal is with Jev, it reminded me of something I implemented in mt_llm (which is build around the great llama.cpp) a while back to use a relatively

19 Sept 2026

Massive token savings... Without token usage. GLM-5.2-cloud

Okay, this one seems probably really odd (or maybe not), but most harnesses operate within token spaces. That is, every tool output, even batching of tool outputs, go back up into inference calls. Lea

local-air-ollama

19 Sept 2026

Qwen3.8-27B at 144 tok/s on an M5 Max MacBook Pro

Meet Inco Splash, open-source inference engine, built around the model and around Apple silicon. Up to 3× the decode speed of Ollama, 2× oMLX, and almost 4× when an agent fans out into sub-agents. Req

19 Sept 2026

Agile-WAM: An Agile Tactile World Action Model for Contact-Rich Robot Control

arXiv:2609.20761v1 Announce Type: cross Abstract: World Action Models (WAMs) advance beyond conventional visuomotor policies by jointly predicting future world states and robot actions, enabling the p

safetyarxiv-cs-lg

Applications

Before the Arrest: Benchmarking LLMs on Criminal Profiling from Incomplete Evidence

arXiv:2609.19965v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly applied to legal and criminal justice tasks, yet existing work focuses almost exclusively on post-arrest s

applicationsarxiv-cs-cl

Chronicle: Cut-Point Replay for Regression Testing of LLM Agents

model-releasesarxiv-cs-ai

arXiv:2609.20625v1 Announce Type: cross Abstract: Large language model responses are non-deterministic, so failures in LLM agents are hard to reproduce: a failure depends on inference that is not bitw

CoMa: Contextual Massing Generation with Vision-Language Models

arXiv:2601.08464v2 Announce Type: replace-cross Abstract: Context-aware building massing is an important early-stage design task: given a site for buildings, a generated massing should not only fit th

researcharxiv-cs-ai

Agents

Deep Noir: Autonomous Steering Discovery via Architectural Chronometry in Transformer Models

arXiv:2609.20722v1 Announce Type: new Abstract: Activation steering modifies LLM behavior at inference time, but identifying where and how strongly to steer remains manual. We introduce Deep Noir, a f

agentsarxiv-cs-ai

Flyweight: open-source C++/CUDA engine for running MoE models bigger than your VRAM on one GPU + system RAM. First PyPI release, looking for contributors.

Been building this for a few months, mostly for myself, and it just got a proper release so figured I'd post it. It's a native GGUF inference runtime with OpenAI/Anthropic-compatible APIs and a chat U

Multi-hour llama.cpp optimization experiments on Qwen MoE models, patches, benchmarks, and reproduction guides

I had Astra run a multi-hour investigation into speeding up local MoE inference while keeping the model weights and quantization unchanged. The experiments covered Qwen3.6-35B-A3B and Qwen3.8 Flash-Ne

Printing the Underdetermined: Materializing Multi-solutionness in Figurative Paintings

arXiv:2609.19782v1 Announce Type: cross Abstract: Figurative paintings are often approached as if they depict a single recoverable 3D scene: viewers infer depth and occlusion, and reconstruction pipel

researcharxiv-cs-cv

Quantum Graph Convolutional Networks: Implementation and Trainability Analysis

arXiv:2609.19983v1 Announce Type: cross Abstract: Graph Neural Networks (GNNs) achieve state-of-the-art performance on graph-structured data, but training and inference on large graphs are often bottl

Selective Cotton Boll Localization for Robotic Harvesting: Evaluation of Deep Learning Vision Models Under Field Conditions

arXiv:2609.19592v1 Announce Type: new Abstract: This study developed and evaluated a deep-learning-based perception framework for selective robotic cotton picking. The dataset contained 1,008 annotate

local-aiarxiv-cs-cv

Task-Oriented Semantic Feature Transmission for Multi-Task Satellite Remote Sensing over Low-SNR Channels

arXiv:2609.20150v1 Announce Type: new Abstract: Conventional satellite remote sensing transmission follows a reconstruct-then-infer paradigm that optimizes pixel-level fidelity, creating an objective

researcharxiv-cs-cv

To Copy or Not to Copy: Controlling Speculative Decoding via Intrinsic Model Signals

model-releasesarxiv-cs-cl

arXiv:2609.20186v1 Announce Type: new Abstract: Speculative Decoding (SD) has significantly accelerated Large Language Model (LLM) inference, yet existing approaches face a fundamental tradeoff betwee

When Perplexity Lies: Generation-Focused Distillation of Hybrid Sequence Models

model-releasesarxiv-cs-ai

arXiv:2603.26556v3 Announce Type: replace-cross Abstract: Converting a pretrained Transformer into a more efficient hybrid model through distillation offers a promising approach to reducing inference

BLADE: ReliaBle Dynamic Hardware-Aware SNN-ANN Boundary SeLection for Event-BAseD Object DEtection

arXiv:2609.17562v1 Announce Type: cross Abstract: Hybrid Spiking Neural Network (SNN)-Artificial Neural Network (ANN) architectures combine the energy efficiency of SNNs with the superior detection ac

local-aiarxiv-cs-ai

Can MiniMax-H3 Reason About the Physical World? An Evaluation of Omni-Modal Generative Model

arXiv:2609.18323v1 Announce Type: new Abstract: Recent Omni-Modal Generative Models (Omni-Models) have advanced content generation toward unified modeling of text, images, video, and audio. MiniMax-H3

safetyarxiv-cs-cv

Comprehensive reconstruction of collider events with hypergraph representation learning and graph-conditioned diffusion

arXiv:2609.18928v1 Announce Type: cross Abstract: In particle collider experiments, event reconstruction is the task of inferring the kinematics of short-lived particles produced in the hard scatter f

researcharxiv-cs-lg

Contiguity, Not Importance: Budgeted Repair of Stale KV Caches After Document Edits

model-releasesarxiv-cs-ai

arXiv:2609.17983v1 Announce Type: new Abstract: KV-cache reuse can reduce inference cost in retrieval-augmented generation and agentic systems, but cached contexts may become stale when retrieved know

DetAug: Obstacle-Blind Trajectory Augmentation for Zero-shot Obstacle Avoidance

model-releasesarxiv-cs-ro

arXiv:2609.18395v1 Announce Type: new Abstract: Policies for robotic manipulation are produced by training on large teleoperated datasets. These datasets typically consist of free-space trajectories,

Disentangling Long-Term Memory via Latent Neuro-Symbolic Reasoning

arXiv:2609.18461v1 Announce Type: new Abstract: Personalized agents are required to reason over long-term history interactions to infer both explicit preferences and implicit behavioral evidence. Whil

researcharxiv-cs-ai

GVD: Governed Versioning and Deduplication for Document Repositories

arXiv:2609.17696v1 Announce Type: new Abstract: Document repositories evolve continuously. Guidelines and policies are revised, superseded, and re-uploaded, so the same content recurs in different wor

local-aiarxiv-cs-ai

Applications

Long-Context Demonstration Selection Using State Space Models

arXiv:2609.17888v1 Announce Type: cross Abstract: We study the problem of demonstration selection, which involves selecting a subset of examples for prepending to a query to a language model. This pro

applicationsarxiv-cs-cl

PersonaPath: Towards Knowledge-Centric Personalized Learning Path Planning

model-releasesarxiv-cs-cl

arXiv:2609.18861v1 Announce Type: new Abstract: Adaptive learning systems commonly formulate learning path planning as Exercise-Centric (EC) recommendation, where the next step is inferred from item-l

PRISM: Predictive Representation of Interaction Style and Motion for Social Robot Navigation

arXiv:2609.18125v1 Announce Type: cross Abstract: Humans often observe others before interacting and adjust their behavior accordingly. Robot navigation in crowds, however, often represents pedestrian

safetyarxiv-cs-ro

RAF-VLA: Representation Alignment with the Future for End-to-End Autonomous Driving

model-releasesarxiv-cs-ro

arXiv:2609.17728v1 Announce Type: new Abstract: Recent Vision-Language-Action (VLA) models for autonomous driving have incorporated world modeling by predicting future driving scenes alongside driving