Search: “local-ai”

Help me understand gguf size/ctx size

Let's say I have 2x 16Gb GPUs and I want to run Qwen3.8 27B. Monitor is ran by the integrated GPU so both 16Gb GPUs are almost fully free. I load the UD-Q4_K_S on one card at 15.4Gb. I then load the c

How do you guys handle your personal RAG setup

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

I am getting into developing a RAG setup, for getting information out of existing documents, new document ingestion, web searches, and good visuals. I am planning to use it for, alongside the regular

local-air-localllama

4 Sept 2026

It sounds like the shift back toward open weight models that I noticed among the startups in the summer batch is a genuine trend.

It sounds like the shift back toward open weight models that I noticed among the startups in the summer batch is a genuine trend. 🦙 @ollama is used by 9 million developers and 85% of the Fortune 500,

local-aiollama--x

LeanStream: A Speculate-and-Refine Streaming Framework for Efficient on-Device LLM Inference

arXiv:2609.03079v1 Announce Type: new Abstract: On-device LLM inference is attractive for privacy and responsiveness, but remains challenging on mobile and embedded devices because model weights far e

Linearized subspace refinement framework to expose hidden accuracy in trained neural networks

arXiv:2601.13989v2 Announce Type: replace Abstract: Neural networks trained by gradient-based methods often exhibit optimization-induced accuracy plateaus in scientific machine learning tasks. We pres

LLM Evaluation as Tensor Completion: Low Rank Structure and Semiparametric Efficiency

arXiv:2604.05460v2 Announce Type: replace-cross Abstract: Large language model (LLM) evaluation platforms increasingly rely on pairwise human judgments. These data are noisy, sparse, and non-uniform,

Local Updates, Global Learning (LUGL): Playing Games with non-incremental Learners

arXiv:2609.03660v1 Announce Type: cross Abstract: The dominance of Neural Networks (NNs) in RL is partially due to their incremental learning capability, which naturally suits the online, non-stationa

More Criticism Does Not Make a Better Review: EquiReview-R

arXiv:2609.03943v1 Announce Type: new Abstract: AI reviewers can now produce many specific criticisms, but more criticism is not necessarily a better review. A review may miss a consequential weakness

Nvidia PAIR makes it easy to create a household data center for running agentic AI tasks

Nvidia Corp. is targeting artificial intelligence agent enthusiasts with a new local distributed clustering tool called the Personal AI Router. It enables them to use any idle Mac computers or PCs lyi

local-aisiliconangle

OCR-EDR: Rendering-Aware Diagnosis and Repair for Closed-Loop OCR Improvement

arXiv:2609.03445v1 Announce Type: new Abstract: Although document OCR systems perform increasingly well on routine documents, complex formulas, structured text, and long-tail formats remain error-pron

Open models are closing the intelligence gap, says @ollama's @jmorgan. That opens up a new opportunity for founders: coordinating fast, chea…

Open models are closing the intelligence gap, says @ollama's @jmorgan. That opens up a new opportunity for founders: coordinating fast, cheap models to solve more complex problems. “We’re maybe less t

local-aiollama--x

Privacy-Preserving Topology-Guided Safety for LLM-Based Multi-Agent Systems via Federated Graph Learning

arXiv:2609.02967v1 Announce Type: cross Abstract: Topology-guided safeguards for LLM-based multi-agent systems (MAS) train a GNN over the inter-agent communication graph to localize risky agents and i

Pushing the (Decision) Boundaries: Dynamically Calibrating Differentially Private Noise to Explainability in Federated Learning

arXiv:2609.03851v1 Announce Type: new Abstract: Federated Learning (FL) with Differential Privacy (DP) is increasingly adopted to preserve data confidentiality in distributed machine learning. However

Qwen3.8-27b is the first Local model im able to blindly trust

You know that thing where you just throw a task at a frontier model and not have to supervise it worrying of it going off course? Qwen3.8-27b has officially gotten me to that point for local work. He

RATL: Learning from Retrieved Residuals for Robust Multivariate Time-Series Forecasting

arXiv:2609.03937v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) complements parametric models with retrieved external evidence. The same idea is attractive for continuous-output

Seeing Before Synthesizing: VLM-Guided Transition Event Discovery for Weakly-Supervised Dense Video Captioning

arXiv:2609.04183v1 Announce Type: cross Abstract: Weakly-Supervised Dense Video Captioning aims to localize and describe multiple events in untrimmed videos given only an ordered set of event-level ca

Seeing Less Is Not Seeing Safely: Privacy Leakage from Task-Scoped Robot Perception Exports

arXiv:2609.03055v1 Announce Type: new Abstract: Domestic robots rely on rich perception to operate in private homes, but privacy risk persists even when raw sensor data remain local. Structured repres

local-aiarxiv-cs-ro

Semiparametric Inference for Counterfactual Regression under Intervention-Driven Shift

arXiv:2504.02694v3 Announce Type: replace-cross Abstract: We study counterfactual regression, which maps features to outcomes under hypothetical scenarios that differ from those observed in the data.

Structured Inference with Large Language Gibbs

arXiv:2606.19264v2 Announce Type: replace-cross Abstract: The knowledge encoded in large language models (LLMs) can serve as a substrate for structured reasoning over variables describing a complex wo

Tail-Likelihood Reinforcement Learning

arXiv:2609.02987v1 Announce Type: new Abstract: Reinforcement learning typically optimizes average reward. For generative policies, the average can hide an important distinction: two policies can achi

Temperature Scaling Attack Disrupting Model Confidence in Federated Learning

arXiv:2602.06638v3 Announce Type: replace-cross Abstract: Predictive confidence serves as a foundational control signal in mission-critical systems, directly governing risk-aware logic such as escalat

The Natural Language Interaction Protocol and Standard for AI Agents

arXiv:2609.04135v1 Announce Type: new Abstract: AI agents are increasingly being developed and deployed across organizations using heterogeneous agent-development frameworks, AI models, tool interface

ToPO: Token-Conditioned Preference Routing for Attention-Based Latent Diffusion Models

arXiv:2609.03688v1 Announce Type: new Abstract: Pairwise preference labels rank complete images, yet Diffusion-DPO applies their effect over many spatial and denoising-time coordinates. For attention-

Tossed distorted audio samples to an open-weight voice model; it did fairly well.

Being a person obsessed with testing new models that come out, times are really insane for me. Tested different kinds of TTS and voice cloning models but none of them gets it right in terms of emotion

VestigeKV: The NoPE-MLA KV Cache Carries Its Own Eviction Signal in a Vestigial Branch

arXiv:2609.03949v1 Announce Type: cross Abstract: The problem. A long-lived KV cache must be compressed before the queries that will read it exist; selection by observed attention (H2O, SnapKV) collap

WeatherNext 3: Increasing resolution and performance of global weather models with raw observations

arXiv:2609.03582v1 Announce Type: new Abstract: State-of-the-art AI weather models have shown impressive medium-range forecast skill and computational efficiency, but suffer two key shortcomings: thei

Where Does Harness-Optimization Value Live? Localized Gains and the Budget-Splitting Trap in Self-Evolving LLM Agents

arXiv:2609.02889v1 Announce Type: new Abstract: A growing body of work improves frozen large language models (LLMs) as agents by evolving their harness: the textual scaffolding around the model, inclu

Why Gated DeltaNet Survives 4-Bit Quantization: NVFP4 W4A4 for the Recurrent Half of a Hybrid 27B LLM

arXiv:2609.04098v1 Announce Type: new Abstract: Hybrid LLMs pair softmax attention with linear-attention layers such as Gated DeltaNet (GDN), whose recurrent state summarizes the context in fixed size

You can now run a 90M conversational LLM on the Sony PSP (hardware from 2004). Doesn't get more local than this.

Github link: https://github.com/thatblend/LLMPSP I wanted to see what the PSP can theoretically handle and I got my answer - a 90M model is about the max it can do without atrocious inference speeds.

A Top-Down Framework for Metric-Scale Athlete Localization from Single Broadcast Frames

arXiv:2609.02705v1 Announce Type: new Abstract: Accurate world-coordinate localization of athletes from single-frame broadcast footage is inherently challenging due to extreme scale disparities in ult

Adapting a Foundation Model for Lunar Surface Height Estimation

arXiv:2609.02448v1 Announce Type: new Abstract: Digital elevation models (DEMs) can provide accurate height information, making it invaluable for analyzing the lunar surface. As the European Space Age

Anker unveils the Eufy MindBase, a local AI hub for its cameras that runs an on-device, Anker-developed LLM, with a dedicated AI chip and up to 48TB of storage (Jennifer Pattison Tuohy/The Verge)

Jennifer Pattison Tuohy / The Verge: Anker unveils the Eufy MindBase, a local AI hub for its cameras that runs an on-device, Anker-developed LLM, with a dedicated AI chip and up to 48TB of storage — T

local-aitechmeme

Automated Vulnerability Injection in Smart Contracts Using Large Language Models

arXiv:2609.02624v1 Announce Type: cross Abstract: Assessing vulnerability detection tools for smart contracts requires datasets with known ground truth, yet such datasets are scarce and difficult to b

Best small LLMs for translations?

I am working with multi-language content and have so far been useing gemma4:e2b for general work. Today I tried it with Russian for the first time and the result was not great. Gemma4:e4b was better,

local-air-ollama

Can a 4B local model actually feel like an AI assistant?

I've been building Arcon around Qwen3-4B + LoRA. Instead of just making it a chatbot, I'm experimenting with persistent memory, personality/mood, internal state, tools, and eventually having it proces

Characterizing Text Branch Sensitivity in Medical Vision-Language Segmentation via Evidence Decoupling

arXiv:2609.02663v1 Announce Type: new Abstract: Pretrained vision-language models (VLMs) have shown promising performance in medical image segmentation by incorporating clinical text. However, it rema

Coverage, Not Targeting: A Structural Regime in Multi-Turn Agent Credit Assignment

arXiv:2609.02417v1 Announce Type: cross Abstract: Multi-turn agentic RL increasingly treats credit assignment as a targeting problem: given a terminal verifiable reward, per-turn methods localize cred

Diagnosing with Insights: Structured Analysis of Agent Failures via Behavioral Abstractions

arXiv:2609.02371v1 Announce Type: new Abstract: With the proliferation of LLM agents, the ability to understand and diagnose failures in agents is essential to achieving superior effectiveness and tru

DiffuSearch: How Hybrid Trajectory Planning Benefits from Aligned Objectives in Diffusion and Action Space

arXiv:2609.02252v1 Announce Type: cross Abstract: In trajectory planning for autonomous driving, hybrid planning architectures are often realized as a collection of disparate modules, each with its ow

DynG-Diff: A State-Aware Dynamic Guidance Diffusion Framework for Probabilistic Time Series Forecasting

arXiv:2609.02068v1 Announce Type: new Abstract: Probabilistic multivariate time series (MTS) forecasting is crucial for modeling complex dynamical systems. However, existing diffusion-based methods re

Emergence of Fibrations, Compression, and Symmetry Breaking in Artificial Neural Networks

arXiv:2609.01768v1 Announce Type: new Abstract: Artificial neural networks are often regarded as powerful yet opaque black boxes. Here, we demonstrate that learning in deep neural networks generates l

Evidence-Guided Detection, Localization and Explanation for Text-Centric Image Forensics

arXiv:2609.02097v1 Announce Type: new Abstract: The rapid progress of AIGC has made text-centric image manipulation increasingly accessible, creating new forensic challenges that require not only auth

First results from H3 Acceleration Arena

local-air-stablediffusion

https://preview.redd.it/c8za3q7shanh1.png?width=900&format=png&auto=webp&s=5716c30be6913c351403fb16cc60a59e3b116200 https://huggingface.co/spaces/multimodalart/h3-acceleration-arena From author u/apol

Frontier models sabotaging local AI implementations?

For a few days I've been working on creating a custom local-only harness for some work related research using Codex / GPT 5.6 Sol and the model feels not only dumber than usual, but straight up counte

GaLe: memory-efficient Global Approximate and Local Exact features

arXiv:2609.02689v1 Announce Type: new Abstract: Embedded devices typically lack the resources of GPU-equipped machines, and existing inference methods suffer from either high computational overhead (p

GeoSPRINT: Geometric Redundancy-Aware Step Pruning for Inference in Diffusion Trajectories

arXiv:2609.02160v1 Announce Type: cross Abstract: Diffusion models achieve high sample quality but remain expensive at inference time because sampling requires many sequential neural function evaluati

Grounded, Compute-Efficient LLM Policy Agents for Energy-Poverty Equity in Physically-Constrained Peer-to-Peer Energy Markets

arXiv:2609.01918v1 Announce Type: new Abstract: Energy poverty is nearly absent from NLP-for-social-good, and the little existing work is either static retrieval/QA or relies on carbon-intensive cloud

HeadWiseKV: Budgeted Per-Head Cache Residency for Hybrid Long-Context Language Models

arXiv:2609.02029v1 Announce Type: new Abstract: Long-context inference retains a growing key--value (KV) cache during decoding, which consumes substantial GPU memory and can reduce generation throughp