Search: “inference”

When Think-with-Image Meets Safety: What Determines Multimodal Jailbreak Robustness?

arXiv:2605.27932v1 Announce Type: cross Abstract: Think-with-image reasoning is emerging as a new inference paradigm for large vision-language models, but its safety implications remain poorly underst

28 May 2026

b9368

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

b9368 is an intermediate build release of llama.cpp, a C/C++ implementation for efficient LLM inference on consumer hardware. The release follows the project's rapid development cycle where multiple b

local-aillama-cpp-releases

27 May 2026

Clinically-Grounded Counterfactual Reasoning for Medical Video Diagnosis

model-releasesarxiv-cs-cv

arXiv:2605.26483v1 Announce Type: new Abstract: Medical video diagnosis involves inferring clinical decisions from dynamic tissue responses throughout examination processes. Existing methods rely on a

Dense2MoE: Pushing the Pareto Frontier of On-Device LLMs via Unified Pruning and Upcycling

arXiv:2605.26496v1 Announce Type: cross Abstract: The Mixture of Experts MoE architecture is highly promising for resource constrained on device deployments yet training these models from scratch incu

Detectability in Diversity: Improved Canary Crafting for Privacy Auditing in One Run

arXiv:2605.27292v1 Announce Type: new Abstract: Privacy auditing aims to empirically assess privacy leakage in machine learning models using membership inference attacks (MIAs), and to derive lower bo

researcharxiv-cs-lg

Ethical Fairness without Demographics in Human-Centered AI

arXiv:2603.13373v3 Announce Type: replace-cross Abstract: In ubiquitous and mobile health systems, computational models infer human states from wearable, behavioral, and physiological sensing data. In

Fast, faster, Qwen. 🚀 Thrilled to see Qwen3.5 reaching a record-breaking 580 tps for agentic workloads on the TokenSpeed engine! This miles…

Fast, faster, Qwen. 🚀 Thrilled to see Qwen3.5 reaching a record-breaking 580 tps for agentic workloads on the TokenSpeed engine! This milestone wouldn't be possible without our incredible partners. Hu

model-releasesqwen--x

Geometrically Constrained Outlier Synthesis

arXiv:2603.08413v2 Announce Type: replace-cross Abstract: Deep neural networks for image classification often exhibit overconfidence on out-of-distribution (OOD) samples. To address this, we introduce

researcharxiv-cs-ai

Helicase: Uncertainty-Guided Supply Chain Knowledge Graph Construction with Autonomous Multi-Agent LLMs

arXiv:2605.26835v1 Announce Type: new Abstract: LLM-based multi-agent systems have been widely adopted for knowledge retrieval and report generation, synthesizing known information through web search

HiSpec: Hierarchical Speculative Decoding for LLMs

arXiv:2510.01336v2 Announce Type: replace-cross Abstract: Speculative decoding accelerates LLM inference by using a smaller draft model to speculate tokens that a larger target model verifies. Verific

researcharxiv-cs-ai

I think Anthropic and OpenAI have found product-market fit

model-releasessimon-willison

Anthropic are strongly rumored to be about to have their first profitable quarter. Stories are circulating of companies surprised at how expensive their LLM bills are becoming from usage by their staf

Tutorials

Learning to Diagnose and Correct Errors: Towards Moral Sensitivity Acquisition in Large Language Models

arXiv:2601.03079v4 Announce Type: replace Abstract: Moral sensitivity is the most fundamental capability underlying human moral competence. Although many approaches aim to align large language models

tutorialsarxiv-cs-cl

Microsoft and Dell believe the answer to rising cloud token costs sits on every employee’s desk

Enterprises are adopting an agentic AI PC strategy as Copilot+ PCs shift AI workloads from expensive cloud inference to secure, high-performance on-device processing. As agentic AI moves from experime

local-aisiliconangle

OmniInteract: Benchmarking Real-World Streaming Interaction for Real-Time Omnimodal Assistants

model-releasesarxiv-cs-cl

arXiv:2605.26485v1 Announce Type: cross Abstract: We introduce OmniInteract, a streaming benchmark for real-time omnimodal large language models evaluated through native online inference over audio-vi

OmniToM: Benchmarking Theory of Mind in LLMs via Explicit Belief Modeling

arXiv:2605.26322v1 Announce Type: new Abstract: Theory of Mind (ToM), the ability to infer others' knowledge, intentions, and emotions, is commonly evaluated in large language models (LLMs) using end-

Pair-In, Pair-Out: Latent Multi-Token Prediction for Efficient LLMs

arXiv:2605.27255v1 Announce Type: cross Abstract: Long chain-of-thought reasoning has made autoregressive decoding the dominant inference cost of modern large language models. Existing methods target

Tools

Reachy Mini goes fully local

Reachy Mini, a robotic system, can now operate with fully local processing capabilities, enabling on-device inference without requiring cloud connectivity. This advancement allows the robot to run AI

toolshugging-face

TAG: Tangential Amplifying Guidance for Hallucination-Resistant Sampling

arXiv:2510.04533v2 Announce Type: replace Abstract: Diffusion models achieve state-of-the-art image generation but often produce semantic inconsistencies, or hallucinations. Existing inference-time gu

researcharxiv-cs-cv

Hardware

Tensormesh taps Nvidia, AMD and CoreWeave for funding to fix AI model memory problems

Tensormesh Inc. has hit upon a way to make artificial intelligence inference more efficient by eliminating the need for redundant computations, and its technology is so convincing that several of AI i

hardwaresiliconangle

The Coverage Illusion: From Pre-retrieval Routing Failure to Post-retrieval Cascades in a Production RAG System

arXiv:2605.27220v1 Announce Type: new Abstract: In modern RAG pipelines, query augmentation methods such as HyDE and query expansion are applied to every query, resulting in substantial LLM inference

WINDQuant: Weight-Informed Neural Decision-Making for Global Mixed-Precision LLM Quantization

model-releasesarxiv-cs-lg

arXiv:2605.26660v1 Announce Type: new Abstract: Quantization is an effective approach to reduce the memory footprint and inference cost of large language models (LLMs), yet maintaining performance in

ArtSplat: Feed-Forward Articulated 3D Gaussian Splatting from Sparse Multi-State Uncalibrated Views

arXiv:2605.24304v1 Announce Type: cross Abstract: Articulated object reconstruction from sparse-view images is an ill-posed problem that requires simultaneous inference of geometry and underlying arti

b9320

local-aillama-cpp-releases

B9320 is a release of llama.cpp, a C/C++ implementation for enabling LLM inference with minimal setup and state-of-the-art performance on a wide range of hardware locally and in the cloud. The project

Beyond the Target: From Imitation to Collaboration in Speculative Decoding

arXiv:2605.24793v1 Announce Type: new Abstract: Speculative decoding (SPD) accelerates large language model (LLM) inference by letting a smaller draft model propose multiple future tokens that are ver

Chain-of-Thought Hijacking

arXiv:2510.26418v4 Announce Type: replace Abstract: Large Reasoning Models (LRMs) improve task performance through extended inference-time reasoning. Although previous studies suggest that longer reas

Compositional Semantics for Open Vocabulary Spatio-semantic Representations

arXiv:2310.04981v2 Announce Type: replace-cross Abstract: Vision-language models (VLMs) transform environment percepts into vision-language semantics interpretable by LLMs. However, completing complex

researcharxiv-cs-lg

CONF-KV: Confidence-Aware KV Cache Eviction with Mixed-Precision Storage for Long-Horizon LLM

arXiv:2605.24786v1 Announce Type: cross Abstract: Long-horizon LLM inference turns the key--value (KV) cache into the dominant GPU memory consumer and makes per-token attention increasingly expensive.

local-aiarxiv-cs-ai

Agents

CP-Agent: A Calibrated Risk-Controlled Agent for Feedback-Driven Competitive Programming

arXiv:2605.24693v1 Announce Type: new Abstract: Large language models still struggle with contest-level programming, while many agentic remedies rely on massive inference-time sampling or expensive mu

agentsarxiv-cs-cl

Decision-Making with Lightweight Confidence-Aware Language Model for Autonomous Driving

model-releasesarxiv-cs-ro

arXiv:2605.25393v1 Announce Type: new Abstract: Large Language Models (LLMs) and Multimodal LLMs (MLLMs) have demonstrated immense potential in autonomous driving (AD) by offering human-like reasoning

Efficient Long-Horizon Vision-Language-Action Models via Static-Dynamic Disentanglement

model-releasesarxiv-cs-ro

arXiv:2602.03983v3 Announce Type: replace Abstract: Vision-Language-Action (VLA) models have recently emerged as a promising paradigm for generalist robotic control. Built upon vision-language model (

Fine-Tuning and Serving Gemma 4 31B on Google Cloud TPU: A Technical Comparison with GPU Baselines

arXiv:2605.25645v1 Announce Type: cross Abstract: We present the first end-to-end demonstration of fine-tuning and serving Google's Gemma 4 31B model on TPU hardware, providing an empirical comparison

Generative Representation Learning on Hyper-relational Knowledge Graphs via Masked Discrete Diffusion

arXiv:2605.24064v1 Announce Type: cross Abstract: Hyper-relational knowledge graphs (HKGs) effectively represent complex facts. While inferring new knowledge in HKGs is a critical problem, current met

local-aiarxiv-cs-ai

Generative Visual Code Mobile World Models

arXiv:2602.01576v2 Announce Type: replace-cross Abstract: Mobile Graphical User Interface (GUI) World Models (WMs) offer a promising path for improving mobile GUI agent performance at train- and infer

Geometry Adaptive Counterfactual Distribution Learning with Diffusion-Guided Smoothing

arXiv:2605.25811v1 Announce Type: cross Abstract: We study counterfactual distribution learning for high-dimensional outcomes whose counterfactual law may concentrate near lower-dimensional structure.

local-aiarxiv-cs-lg

HEAPr: Hessian-based Efficient Atomic Expert Pruning in Output Space

arXiv:2509.22299v3 Announce Type: replace-cross Abstract: Mixture-of-Experts (MoE) architectures in large language models (LLMs) deliver exceptional performance and reduced inference costs compared to

Hidden in Plain Tokens: Simply Robust, Gradient-Free Watermark for Synthetic Audio

arXiv:2605.25967v1 Announce Type: new Abstract: As policy catches up with the capabilities of generative AI, watermarking is central to content provenance efforts. Inference-time watermarks for autore

safetyarxiv-cs-lg

Iterative Refinement Neural Operators are Learned Fixed-Point Solvers: A Principled Approach to Spectral Bias Mitigation

arXiv:2605.24041v1 Announce Type: cross Abstract: Neural operators serve as fast, data-driven surrogates for scientific modeling but typically rely on a monolithic, single-pass inference procedure tha

Applications

Know You Before You Speak: User-State Modeling for LLM Personalization in Multi-Turn Conversation

arXiv:2605.24647v1 Announce Type: new Abstract: Personalized dialogue requires more than recalling explicit user histories: systems also need to infer hidden user states that evolve through interactio

applicationsarxiv-cs-cl

Applications

Learning Latent Dynamical Causal Processes for Single-Cell Perturbation Prediction

arXiv:2605.25581v1 Announce Type: new Abstract: Single-cell perturbation prediction aims to infer how cells respond to unseen interventions and to achieve out-of-distribution (OOD) generalization, pro

applicationsarxiv-cs-lg

Locality Matters for Training-Free Audio Token Compression in Audio-Language Models

arXiv:2605.25179v1 Announce Type: new Abstract: Audio-language models (ALMs) are increasingly used for audio captioning, question answering, and open-ended audio understanding, but their inference cos

Agents

PANDO: Efficient Multimodal AI Agents via Online Skill Distillation

arXiv:2605.24785v1 Announce Type: new Abstract: Recent advances in multimodal web agents often rely on increased inference-time computation, including rollout search, verifier passes, offline skill di

agentsarxiv-cs-ai

Hardware

Physical Analogue Kolmogorov-Arnold Networks based on Reconfigurable Nonlinear-Processing Units

arXiv:2602.07518v2 Announce Type: replace-cross Abstract: Kolmogorov-Arnold Networks (KANs) shift neural computation from linear layers to learnable nonlinear edge functions, but implementing these no

hardwarearxiv-cs-lg

RepetitionCurse: Measuring and Understanding Router Imbalance in Mixture-of-Experts LLMs under DoS Stress

model-releasesarxiv-cs-lg

arXiv:2512.23995v2 Announce Type: replace-cross Abstract: Mixture-of-Experts architectures have become the standard for scaling large language models due to their superior parameter efficiency. To acc

Hardware

SA-Kura: An Energy-Efficient Systolic Array Accelerator for Locally-Coupled Kuramoto Drift in Diffusion Sampling

arXiv:2605.24016v1 Announce Type: cross Abstract: Diffusion inference remains costly for edge deployment, yet existing accelerators focus almost exclusively on score networks because standard drift is

hardwarearxiv-cs-ai

Selective Latent Thinking: Adaptive Compression of LLM Reasoning Chains

arXiv:2605.25745v1 Announce Type: new Abstract: Explicit chain-of-thought (CoT) reasoning substantially improves the reasoning ability of large language models (LLMs), but incurs high inference cost d

Smoother Action Chunking Flow Policy via Prior-Corrected Orthogonal Trust-Region Guidance

arXiv:2605.24433v1 Announce Type: cross Abstract: Flow-matching robot policies commonly use action-chunking inference for efficient closed-loop control, but chunk boundaries can introduce discontinuou

safetyarxiv-cs-lg

Agents

STORM: Internalized Modeling for Spatial-Temporal Reasoning in Video-Language Models

arXiv:2605.26014v1 Announce Type: cross Abstract: Many video reasoning tasks require tracking motion, temporal order, and evolving visual states across frames. Existing methods built on large vision-l

agentsarxiv-cs-cl

Test-Time Self-Adaptive Conditioning for Stable Audio-Driven Talking-Head Generation

arXiv:2605.25488v1 Announce Type: cross Abstract: Audio-driven talking-head generation has achieved remarkable progress with recent models such as AniTalker, FLOAT, and Sonic. Despite their success, m