Search: “inference”

FastE: Readout-Triggered Token Compression for LLM Embedding Inference

arXiv:2609.08407v1 Announce Type: new Abstract: In this study, we identify depth-dependent prefix redundancy in final-readout LLM embedding models, notably across representative backbones including Qw

safetyarxiv-cs-ai

10 Sept 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

SymbolicLight V2: Hybrid Neuromorphic Architecture and Sparse Execution for Low-Energy Language Inference

DGX agent

arXiv:2609.09772v1 Announce Type: new Abstract: SymbolicLight V2 combines sparse event computation with continuous-state processing in a hybrid neuromorphic language architecture. Extending V1's spike

model-releasesarxiv-cs-cl

10 Sept 2026

Knowledge-Guided Vision-Language Inference for Image-Based Urban Flood Depth Estimation

arXiv:2509.04772v2 Announce Type: replace Abstract: Timely floodwater depth estimates support road accessibility assessment and emergency response during urban flooding. Supervised vision methods ofte

researcharxiv-cs-cv

9 Sept 2026

Mind the Gap: Navigating Inference with Optimal Transport Maps

arXiv:2507.08867v3 Announce Type: replace-cross Abstract: Machine learning (ML) techniques have recently enabled enormous gains in sensitivity to new phenomena across the sciences. In particle physics

9 Sept 2026

ZetaDial: dialing net charge of protein binders at inference time for therapeutic developability

arXiv:2609.05451v1 Announce Type: cross Abstract: Net charge is a developability-relevant property of therapeutic binders, linked to viscosity, clearance, nonspecific interaction and aggregation, and

safetyarxiv-cs-lg

9 Sept 2026

Hardware

BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference

arXiv:2609.04971v1 Announce Type: cross Abstract: Large Reasoning Models (LRMs) achieve superior problem-solving through extended Chain-of-Thought (CoT) generation, but the resulting key-value (KV) ca

hardwarearxiv-cs-cl

Influence Score and Transformers interpretability: Measure of the Effective Impact of Attention Heads at inference time

arXiv:2609.05074v1 Announce Type: new Abstract: We propose an influence score to quantify the contribution of attention heads to classification decisions in Transformer-based models designed for promp

Thinking about grabbing an RTX 2000 Ada 16gb to add to my gaming pc for inference due to Wattage constraints, any advice?

model-releasesr-localllama

So, I’m currently just short of being able to run the mainline dense SLMs (Qwen 27b/Gemma 31b), running a i9-14900F/RTX4070 Super/32GB DDR5 setup w/ a 700W 80 Gold PSU and a MSI PRO B760-VC WI-FI IV m

You Really Didn't Get That? Benchmarking Social Pragmatic Inference for Indirect and Playful Chinese Online Comments

arXiv:2609.04384v1 Announce Type: cross Abstract: Chinese online comments often convey social meaning through indirect and playful language that is hard to interpret without context. Existing evaluati

NeoMME: A Single-Tower Multimodal-Native Multilingual Foundation Encoder for Efficient Fine-Tuning and Inference

arXiv:2609.01657v1 Announce Type: cross Abstract: Multimodal models often build on architectures designed for generative vision-language modeling, typically combining separately pretrained vision enco

3 Sept 2026

Compressing AI Traffic: Standardized Neural Network Coding of Visual-Token Representations in Split Vision-Language Inference

model-releasesarxiv-cs-cv

arXiv:2609.01200v1 Announce Type: new Abstract: When the visual encoder and the language decoder of a vision-language model (VLM) run on different compute nodes, the intermediate visual-token embeddin

Ontology-Guided Neuro-Symbolic Inference: Grounding Language Models with Mathematical Domain Knowledge

arXiv:2602.17826v2 Announce Type: replace Abstract: Language models exhibit fundamental limitations -- hallucination, brittleness, and lack of formal grounding -- that are particularly problematic in

Perplexity open-sourced their Mac inference server for Qwen 3.6

model-releasesr-localllama

https://preview.redd.it/6h4xc5o8l6nh1.png?width=1158&format=png&auto=webp&s=b65074b6baaa1faa2347e5259229c8ba803bcd4b Here is link to repo: https://github.com/perplexityai/pplx-garden/tree/main/lily It

Restrict, Don't Retrain: Inference-Time VLM Guidance for Zero-Shot Aerial Segmentation

arXiv:2609.00628v1 Announce Type: cross Abstract: Global welfare often depends on the correct interpretation of aerial and satellite imagery. Acting on such imagery (mapping flooded ground, crop exten

local-aiarxiv-cs-ai

A homotopy-type-theoretic generalization of neurosymbolic inference

arXiv:2606.17851v2 Announce Type: replace Abstract: A wide range of neurosymbolic (NeSy) systems compute one functional: a belief-weighted sum of a logical quantity over a space of sigma-structures, o

1 Sept 2026

Dependency-Aware Revocable Decoding for Efficient Diffusion Large Language Model Inference

arXiv:2608.26574v1 Announce Type: new Abstract: Diffusion large language models (dLLMs) offer a promising alternative to autoregressive generation by decoding multiple tokens in parallel through itera

28 Aug 2026

G2D: Generative-to-Discriminative Collaborative Inference for Zero-Shot Image Classification

arXiv:2608.26744v1 Announce Type: new Abstract: Zero-shot classification needs efficient label retrieval and fine-grained visual reasoning, yet discriminative and generative vision-language models fai

researcharxiv-cs-cv

28 Aug 2026

Reflection Steering: Disentangling Reflection from Reasoning in Activation Space for Token-Efficient Inference

model-releasesarxiv-cs-cl

arXiv:2608.25542v1 Announce Type: cross Abstract: Large reasoning models often produce reasoning traces with verification, revision, and backtracking. When reflection merely re-checks established resu

27 Aug 2026

Visionary — Krea 2, MiniMax-H3, inference and LoRA training in one app

local-air-stablediffusion

I've been building this for my own work but there's no reason for it to stay private, so here it is.Visionary is a single Modal app that gives you Krea 2 for stills, MiniMax-H3 for video-with-sound, a

27 Aug 2026

Asymptotically perfect seeded graph matching without edge correlation (and applications to inference)

arXiv:2506.02825v3 Announce Type: replace-cross Abstract: We present the OmniMatch algorithm for seeded multiple graph matching. In the setting of d-dimensional Random Dot Product Graphs (RDPG), we pr

26 Aug 2026

Applications

GNNBleed: Inference Attacks to Unveil Private Edges in Graphs with Realistic Access to GNN Models

arXiv:2311.16139v3 Announce Type: replace-cross Abstract: Graph Neural Networks (GNNs) have become indispensable tools for learning from graph structured data, catering to various applications such as

applicationsarxiv-cs-lg

26 Aug 2026

A Deep Causal Inference Approach to Measuring the Effects of Forming Group Loans in Online Non-profit Microfinance Platform

arXiv:1706.02795v1 Announce Type: cross Abstract: Kiva is an online non-profit crowdsouring microfinance platform that raises funds for the poor in the third world. The borrowers on Kiva are small bus

25 Aug 2026

Agents

Beyond Fresh Starts: Stateful Inference for Streaming ASR in Conversational Voice Agents

arXiv:2608.22101v1 Announce Type: new Abstract: Modern voice-agent systems rely on streaming speech recognition models that operate under stringent latency constraints. This study shows that, due to t

agentsarxiv-cs-lg

25 Aug 2026

Traceable Spectral Inference via Influence Functions: Efficient Data Attribution and Error Proxies for the Ariel Mission

arXiv:2608.23458v1 Announce Type: new Abstract: Interpretability is critical for machine learning models deployed in scientific space missions such as ESA's Ariel, where ground truth is unavailable du

25 Aug 2026

Policy Optimization and Statistical Inference for Online Contextual Matrix Games

arXiv:2608.17173v1 Announce Type: cross Abstract: Online decision making often requires navigating a landscape shaped by both dynamic contexts and strategic interactions. In competitive pricing, for e

19 Aug 2026

SPSA Hyperparameter Tuning for Variational Quantum Natural Language Inference

arXiv:2608.16939v1 Announce Type: cross Abstract: Training variational quantum models requires choosing between parameter-shift gradients, which are exact but cost O(P) forward evaluations, and simult

19 Aug 2026

TileMix: Tile-Centric Mixed-Precision Attention for LLM Inference Acceleration

arXiv:2608.17336v1 Announce Type: new Abstract: Long-context prefill in large language models (LLMs) incurs substantial computation and memory traffic because dense self-attention computes quadratic q

19 Aug 2026

Agents

DriveCache: Action-Aware Caching for Driving World Model Inference

arXiv:2608.16354v1 Announce Type: new Abstract: Driving video generation models support autonomous-driving development by predicting controllable future scenes for simulation, planning evaluation, and

agentsarxiv-cs-ai

18 Aug 2026

Hardware

FluxBin: Flexible LUT-based Ultra-low-bit LLM Inference by Algorithm-Kernel Synergy

arXiv:2608.15602v1 Announce Type: cross Abstract: While binary quantization theoretically promises extreme compression and acceleration for Large Language Models (LLMs), existing research often overlo

hardwarearxiv-cs-ai

18 Aug 2026

Prediction Inference of Time Series with Standard ReLU Deep Neural Networks

arXiv:2608.15362v1 Announce Type: cross Abstract: We propose a methodology based on the standard ReLU Deep Neural Networks (DNN) to make predictions and quantify their uncertainty. Classically, people

18 Aug 2026

Agents

Discovering Efficient and Explainable Communication Topologies for LLM-based Multi-Agent Systems via Causal Inference

arXiv:2608.12921v1 Announce Type: cross Abstract: The performance of large language model (LLM)-based multi-agent systems (MAS) largely depends on effective communication topologies. Existing topology

agentsarxiv-cs-ai

14 Aug 2026

CORA-Diff: Confidence-Oriented Residual Acceptance for Efficient Diffusion Language Model Inference

arXiv:2608.11235v1 Announce Type: new Abstract: Diffusion language models (DLMs) update many tokens in parallel, yet practical decoders often use a fixed denoising horizon. Many predictions stabilize

13 Aug 2026

BRACE: Taming Sharp Irregularities via Barycentric Rational Forecasting for Fast Diffusion Transformers Inference

arXiv:2608.07572v1 Announce Type: cross Abstract: Diffusion Transformers (DiTs) have demonstrated exceptional performance in high-fidelity image and video generation. To alleviate their massive comput

Causal State-Space Model for Causal Inference: Estimating Longitudinal Individual Treatment Effects

arXiv:2608.08288v1 Announce Type: new Abstract: Estimating counterfactual outcomes over time from longitudinal observational data is central to clinical decision support. Existing methods rely on doma

Focus particles and scalar inferences across humans and language models

arXiv:2608.08227v1 Announce Type: new Abstract: Focus particles such as 'even' and 'only' are central to formal semantic theories that posit structured representations over sets of alternatives. 'Even

Sparse corruption in low-rank matrix inference: the PCA benchmark

arXiv:2511.11927v2 Announce Type: replace-cross Abstract: Principal Component Analysis (PCA) is a standard tool for extracting a low-rank signal from noisy observations. It is known that applying PCA

SUMI: Scalable Unified Model for 3D Point Cloud Inference

arXiv:2608.08115v1 Announce Type: new Abstract: Point cloud completion commonly follows a coarse-to-fine paradigm, where a low-density coarse shape is first predicted and then upsampled to the target

researcharxiv-cs-cv

Cascade: Exploiting SLO-Aware latency budget for fair and high goodput LLM inference serving

arXiv:2608.06557v1 Announce Type: cross Abstract: The reasoning and agentic capabilities of large language models have expanded the range of applications they support, from short interactive exchanges

safetyarxiv-cs-lg

10 Aug 2026

Hardware

Ultra-High Interactivity on NVIDIA GPUs? - TileRT InferenceX

The TileRT InferenceX article (Aug 10 2026) examines whether the TileRT software stack on NVIDIA GPUs can compete with dedicated inference systems such as Cerebras, Groq LPUs and SambaNova for ultra‑h

hardwaresemianalysis

10 Aug 2026

Has anyone here fiddled with TPUs for inference ?

I discovered recently that Google uses their own TPUs, like tiny ASIC cards like the toy ones that existed for bitcoin. And while it sounds inefficient the fact they use thousands of them because...th

local-air-localllama

8 Aug 2026

AMD Acquires Taalas to Advance Compute Solutions for Rapidly Growing AI Inference Market

Press My earlier prediction that Tesla would buy them completely missed the mark. With AMD focusing heavily on the enterprise side, the idea of consumer-facing hot-swappable AI model chips looks prett

local-air-localllama

EdgeXpert: An Edge Device for Memory-Efficient LLM Inference with Mixture-of-Experts and Speculative Decoding

arXiv:2608.05303v1 Announce Type: cross Abstract: On-device deployment of Large Language Models (LLMs) has become essential for personalized edge applications. A primary bottleneck is external memory

local-aiarxiv-cs-cl

LLM Inference Under Bursty Workload Distribution: Modifying the WAIT Algorithm

arXiv:2608.06135v1 Announce Type: new Abstract: Large Language Models (LLMs) such as ChatGPT and Claude are widely used for information retrieval and problem-solving. Recent work has focused on improv

Signal or Spurious Cue? A Randomized Audit of Survey-Country Metadata in LLM Social Inference

arXiv:2608.06085v1 Announce Type: new Abstract: Survey-country metadata can improve an LLM's forecast of an individual response when informative, yet the same cue may redirect the forecast when assign

ATFlash: Per-RoPE-Wavelength Attention Windows for Compute/Memory-Efficient LLM Inference

model-releasesarxiv-cs-cl

arXiv:2608.02947v1 Announce Type: cross Abstract: The attention score with rotary position embeddings (RoPE) decomposes exactly into a sum over its 2D-rotation frequency pairs, and each pair's wavelen

BODHI: Do LLMs Branch Out and Discover Heterogeneous Inferences?

arXiv:2608.02867v1 Announce Type: cross Abstract: Although reinforcement learning with verifiable rewards (RLVR) has improved the performance of large language models (LLMs) across a variety of reason

safetyarxiv-cs-ai

Don't Let Me Ask for It: LLMs Show Deficiencies in Active Multi-Turn Information Acquisition for Abductive Inference

arXiv:2608.03388v1 Announce Type: new Abstract: Abductive reasoning requires forming hypotheses that explain observed evidence and revising them as new evidence becomes available. While large language

GeoID-PINN: Identifiability-Aware Regional Epidemic Inference with Geographic Coupling

arXiv:2608.02633v1 Announce Type: new Abstract: Regional surveillance data reflect local transmission, reporting, seeding, and external infection pressure, which are difficult to identify separately.

local-aiarxiv-cs-lg