Sequential Membership Inference Attacks
arXiv:2602.16596v2 Announce Type: replace Abstract: Modern AI models are not static. They go through multiple updates in their lifecycles. We propose to design Sequential Membership Inference (SeMI) a
Knowledge catalogue
arXiv:2602.16596v2 Announce Type: replace Abstract: Modern AI models are not static. They go through multiple updates in their lifecycles. We propose to design Sequential Membership Inference (SeMI) a
This NVIDIA remains the strongest platform for large-model inference at scale. Prefill/decode disaggregation, Blackwell-native quantization, custom kernels, and rack-scale NVLink turn GB200 into faste
arXiv:2605.08057v1 Announce Type: cross Abstract: While recent advancements in inference-time learning have improved LLM reasoning on Text-to-SQL tasks, current solutions still struggle to perform wel
arXiv:2605.06878v1 Announce Type: cross Abstract: This paper presents CARMEN, a runtime-adaptive, CORDIC-accelerated multi-precision vector engine for resource-efficient deep learning inference. The k
arXiv:2605.07433v1 Announce Type: cross Abstract: Qualitative models provide crucial instruments for modelling complex biological systems. While advances in automated reasoning and symbolic encodings
arXiv:2605.05225v2 Announce Type: replace-cross Abstract: Mixture-of-Experts Multimodal Large Language Models (MoE MLLMs) suffer from a significant efficiency bottleneck during Expert Parallelism (EP)
arXiv:2605.07363v1 Announce Type: cross Abstract: DeepSeek Sparse Attention (DSA) sets the state of the art for fine-grained inference-time sparse attention by introducing a learned token-wise indexer
arXiv:2605.07234v1 Announce Type: cross Abstract: Large language models (LLMs) support long-context inference but suffer from substantial memory and runtime overhead due to Key-Value (KV) Cache growth
Learn how to deploy any Hugging Face model in one session using Goose and Together's Dedicated Container Inference. Skip the setup complexity — one prompt gets your model running in a production-grade
Open-source language models have reached sufficient quality and capability levels just as commercial frontier model providers have begun increasing their inference API pricing. This timing creates a p
DeepSeek-V4 makes million-token context a serving-systems problem. Together AI explores the inference work behind V4 on NVIDIA HGX B200, including compressed KV layouts, prefix caching, kernel maturit
Apple dominate local inference 6/10 This is completely opposite to enterprise/data-center adoption where Nvidia is king. Huge market Nvidia is letting slip. High VRAM + mid bandwidth + working kernels
Learn how to structure your AI stack. Our co-founder @the_bunny_chen talks with @theinfrapod's @tnachen and @ianlivingstone about inference optimization, the open vs. closed model debate, and building
Welcome to DS4, a specialized inference engine for DeepSeek v4 Flash. https://github.com/antirez/ds4 This project would have been impossible without the existence of llama.cpp and GGML and the work of
arXiv:2605.02218v1 Announce Type: new Abstract: Vision-language models (VLMs) have demonstrated strong capabilities in multimodal perception and reasoning. However, deploying large VLMs on mobile devi
arXiv:2605.03109v1 Announce Type: new Abstract: A method is presented for accelerating inference in transformer language models by exploiting the low effective rank of the token activation manifold at
Ryan Whitwam / Ars Technica: Google releases Multi-Token Prediction drafters for its Gemma 4 models, which use a form of speculative decoding to guess future tokens for faster inference — Google launc
inference >>> Ramp's monthly Top Software Vendors is one of the best signals for where to work, where to invest, or who to sell to. Here are May's breakout + fastest growing vendors. We observe that @
Introducing TokenSpeed, a speed-of-light LLM inference engine. > TensorRT LLM level performance > vLLM level usability > Built by a lean and mission-driven team in two months > MIT license, open-sourc
arXiv:2602.16138v2 Announce Type: replace Abstract: We introduce IRIS (Intent Resolution via Inference-time Saccades), a novel training-free approach that uses eye-tracking data in real-time to resolv
Why are four of the top 10 trending vendors AI inference platforms? Companies are getting selective about which model handles which task, and routing tokens accordingly. Read more from @arakharazian a
arXiv:2605.01067v1 Announce Type: new Abstract: Symbolic regression discovers explicit, interpretable equations without assuming a functional form in advance. A Bayesian approach strengthens this thro
Excited to introduce Gemma 4 Multi-Token Prediction Drafters⚡️Accelerated inference right in your pockets - Up to a 3x speedup - Same quality guarantees - Available in your favorite open-source tools
arXiv:2509.03540v3 Announce Type: replace Abstract: Large Language Models (LLMs) often struggle with producing factually consistent answers due to limitations in their parametric memory. Retrieval-Aug
Inference is 80-90% of the lifetime cost of a production AI system. Most AI-native teams are leaving performance and margin on the table. Here’s how Together AI, the AI Native Cloud, fixes that on @nv
arXiv:2605.01058v1 Announce Type: cross Abstract: Layer-aligned distillation and convergence-based early exit represent two predominant computational efficiency paradigms for transformer inference; ye
MiniMax-M2.7 is now available across six inference providers on Artificial Analysis, with significant differentiation in speed and price @SambaNovaAI leads on speed at 435 output tokens/s, >3x faster
arXiv:2605.01766v1 Announce Type: cross Abstract: Multimodal large language models (MLLMs) have revolutionized the landscape of AI, demonstrating impressive capabilities in tackling complex vision and
arXiv:2510.13668v2 Announce Type: replace-cross Abstract: Large Language Model (LLM) inference has emerged as a fundamental paradigm, however, variations in output length cause severe workload imbalan
arXiv:2510.09498v3 Announce Type: replace-cross Abstract: Cardiac muscle tissue exhibits highly non-linear hyperelastic and orthotropic material behavior during passive deformation. Traditional consti
This article from Together AI discusses foundational research techniques and methodologies used to enable efficient large-scale language model inference. It likely covers optimization strategies, hard
🤯 Ollama now supports Claude Desktop via Claude’s built-in third party inference. ollama launch claude-desktop This allows all models from Ollama's Cloud to be used across Claude Cowork and Claude Cod
arXiv:2601.05833v2 Announce Type: replace Abstract: Pretokenization is a crucial, sequential pass in Byte-level BPE tokenizers, yet little work has been done to optimize it for edge-side inference. Ou
arXiv:2605.00392v1 Announce Type: new Abstract: DeepSeek-OCR leverages visual-text compression to reduce long-text processing costs and accelerate inference, yet visual tokens remain prone to redundan
The Information: Sources: Anthropic is in early talks to buy AI inference chips from UK-based Fractile when they become available in 2027 — As Anthropic's sales explode, straining the servers it uses,
arXiv:2604.27942v1 Announce Type: new Abstract: Collective intelligence emerges across biological, physical, and artificial systems without central coordination, yet a unifying principle governing suc
Dina Bass / Bloomberg: Cloud computing provider Nebius agrees to buy Eigen AI, which optimizes the performance of chips running AI inference tasks, for $615M in stock and cash — Cloud computing provid
arXiv:2604.27445v1 Announce Type: new Abstract: Many agents in real-world environments cannot reliably communicate their goals through language, including household pets, pre-verbal infants, and other
arXiv:2604.27476v1 Announce Type: new Abstract: Vision-language models (VLMs) have demonstrated strong applicability in edge industrial applications, yet their deployment remains severely constrained
arXiv:2604.26968v1 Announce Type: cross Abstract: Key-value (KV) cache memory management is the primary bottleneck limiting throughput and cost-efficiency in large-scale GPU inference serving. Current
With this integration, teams can optimize data, launch fine-tuning, evaluate results, and deploy on Together AI inference through a tighter workflow. Learn more: http://www.together.ai/blog/announcing
As models, contexts, and workloads grow, hidden assumptions in inference infrastructure can surface as output anomalies. Reliability requires more than throughput, latency, and availability. It also r
arXiv:2604.24957v1 Announce Type: new Abstract: Scaling test-time compute has emerged as a powerful mechanism for enhancing Large Language Model (LLM) performance. However, standard post-training para
arXiv:2602.11786v2 Announce Type: replace Abstract: Traditional benchmarks for large language models (LLMs), such as HELM and AIR-BENCH, primarily assess safety through breadth-oriented evaluation acr
arXiv:2604.24911v1 Announce Type: new Abstract: Machine Learning is becoming more prevalent in science and engineering, but many approaches do not provide meaningful uncertainty estimates and predicti
Our cofounder @the_bunny_chen joined @GregorVand to talk about open models, production inference, and how RFT is unlocking model customization for teams without a dedicated ML org. They covered custom
arXiv:2604.24971v1 Announce Type: cross Abstract: We present PolyKV, a system in which multiple concurrent inference agents share a single, asymmetrically compressed KV cache pool. Rather than allocat
arXiv:2604.23779v1 Announce Type: cross Abstract: The semantic gap between colloquial user queries and professional legal documents presents a fundamental challenge in Legal Case Retrieval (LCR). Exis
arXiv:2604.23436v1 Announce Type: cross Abstract: Reliable decision-making with streaming data requires principled uncertainty quantification of online methods. While first-order methods enable effici
arXiv:2604.23865v1 Announce Type: cross Abstract: Foundation models of brain activity promise a new frontier for in silico neuroscience by emulating neural responses to complex stimuli across tasks an
arXiv:2604.23012v1 Announce Type: cross Abstract: This paper presents a complete, end-to-end on-device vision machine learning pipeline, comprising data acquisition, two-layer CNN training with Adam o
arXiv:2510.17881v3 Announce Type: replace-cross Abstract: Large language models (LLMs) are typically aligned with population-level preferences, despite substantial variation across individual users. W
arXiv:2604.23905v1 Announce Type: cross Abstract: Threat modeling for cyber-physical systems (CPS) remains a largely manual exercise. This project presents SMSI (System Model Security Inference), a hy
The DeepSeek V4 garbled output bug in open source inference engine is fixed in SGLang. To everyone affected over the weekend, sorry for the trouble. Huge thanks to @Ant_Group for landing the fix PR. I
arXiv:2509.20979v2 Announce Type: replace Abstract: In modern GPU inference, cache efficiency remains a major bottleneck, and heuristic policies such as extsc{LRU} can perform far worse than the offli
DeepSeek v4 Flash with *local inference* after 24h of playing with that: even with the 2 bit selective quantization GGUF, iti is the FIRST time I feel I have a frontier model running on my computer. T
2 bit DeepSeek v4 Flash inference. Experts w1/3: IQ2_XXS, w2: Q2_K, all the rest left mostly F16/F32 Final GGUF: 86.18 GiB Runs on a MacBook M3 max with CPU (No Metal backend test as it will crash my
Inference that never sleeps, for agents that never stop. 'Why cowork when you can delegate?' That's @DhruvBatra_ on @yutori_ai's new Delegate — an always-on agent that monitors, researches, and acts a
Matt Day / Bloomberg: Meta and Amazon reach a multibillion-dollar, multiyear deal for Meta to rent hundreds of thousands of Amazon's Graviton chips for its AI inference needs — Amazon.com Inc. and Met
In ChatGPT, full-stack inference improvements enable a more capable model at faster speed. This efficiency is a game-changer for GPT-5.5 Pro, now a much more practical option for demanding tasks, and