Search: “context”

Trying to optimize LLM inference latency — Ollama + OpenWebUI/AnythingLLM

Hi everyone, I'm currently working on optimizing an LLM setup at my company and I'm trying to identify where the actual performance bottleneck is. Our current setup is roughly: LLM: NVIDIA Nemotron mo

model-releasesr-ollama

23 Aug 2026

Human

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

3 experiments running dsv4-flash-0731 q4+ quants on 128GB RAM + ~60 GB VRAM (with a quite bad pcie infra) with an acceptable tgs and relatively acceptable pp speed

DGX agent

The post describes some experiments I had while trying to desperately run deepseek-v4-flash-0731 4 bit+ quants on my machine which is supposed to support only q2 quants of the model, a or 2.xx bpw qua

model-releasesr-localllama

22 Aug 2026

Do you know you can use open weight models like GLM, Deepseek, Kimi in Claude Code? Setup Fireworks account - you get free credits on signup…

model-releasesfireworks-ai--x

Do you know you can use open weight models like GLM, Deepseek, Kimi in Claude Code? Setup Fireworks account - you get free credits on signup curl -fsSL https://raw.githubusercontent.com/fw-ai/fireconn

Local Ai

Gemax Ai New Version v1.9.15 (Offline AI Agent for Windows)

Gemax 1.9.15 — What’s new LAN server — share Gemax on your local network; open the real IP from another PC or phone Mobile-friendly UI — responsive layout for phone browsers on the same Wi‑Fi Workspac

local-air-ollama

One of the interesting properties we’ve observed around schema-guided, complex document extraction tasks 📑 is that coding agent harnesses a…

model-releasesjerry-liu--x

One of the interesting properties we’ve observed around schema-guided, complex document extraction tasks 📑 is that coding agent harnesses are good baselines (in terms of cost/accuracy) for longer docu

OpenCode + llama.cpp + unsloth/Qwen3.8-27B-GGUF:Q6_K_XL = only failure so far

Hi, I have been using opencode with openrouter for quite a while now. Having read the success stories of using Qwen3.8-27B, I thought of trying it too. But every attempt has been a failure for me so f

Qwen 3.5 4B IQ2_XS: +16.67% Reasoning Performance From Tensor-Level Allocation

I was finally able to replicate tensor level allocation outside the Gemma family. https://huggingface.co/ByteOtter/Qwen3.5-4B-CADA-IQ2_XS After the Gemma 4 12b, e4b and gemma 3 4b results, I attempted

Local Ai

What if Chain-of-Thought wasn’t lossy? Exploring reversible logic (Toffoli/Fredkin-style) for edge LLMs

Right now standard CoT is a one-way street. You generate forward, dump a pile of scratchpad tokens into the KV cache, and pray the model doesn’t hallucinate halfway through. On phones/laptops that cre

local-air-localllama

Air Traffic Control Using Large Language Models: Prompt Engineering, Architecture, and Evaluation

arXiv:2608.19299v1 Announce Type: new Abstract: Air traffic control (ATC) communication is a safety-critical dialogue that remains largely human-driven even as other parts of air traffic management ha

Ask Self, Ask Others: Relation Is All You Need

model-releasesarxiv-cs-lg

arXiv:2608.20172v1 Announce Type: new Abstract: Attention directly derives normalized information flow from pairwise scores. We introduce Relation, an alternative token-mixing primitive that first org

Auditing Cross-Lingual Fairness in Language Model Watermarking

arXiv:2608.20047v1 Announce Type: new Abstract: Watermarking schemes for large language model output are evaluated almost exclusively on English text using each scheme's detection threshold and a narr

safetyarxiv-cs-cl

CAViAR: A Causal Video Dataset for Fine-Grained Accident Reasoning in Real-World Scenarios

arXiv:2608.19380v1 Announce Type: new Abstract: While modern autonomous driving systems excel at perception tasks such as object detection and trajectory prediction, they lack the high-level causal re

ContractScrub: A benchmark for final review of legal contracts

arXiv:2608.20204v1 Announce Type: new Abstract: Legal work, with its heavy reliance on processing large amounts of text, is often considered one of the domains most exposed to the use of LLMs. Contrac

Agents

FlowEvo: Self-Evolving Agents through the Co-Evolution of Workflows and Executable Skills

arXiv:2607.21596v2 Announce Type: replace Abstract: Large language model agents can adapt to complex tasks by constructing workflows at inference time, but procedures discovered in one episode are usu

agentsarxiv-cs-ai

G-MARK: Grounded Multi-Agent Reasoning for Cooperative Driving via Knowledge Graphs

arXiv:2608.19964v1 Announce Type: new Abstract: Autonomous driving systems must operate under partial observability, where safety-critical objects may be occluded or visible only to neighboring connec

safetyarxiv-cs-lg

Applications

GenMatch: An End-to-End Generative Matching Framework for Micro-View Order-Dispatching in Ride-Hailing

arXiv:2608.19751v1 Announce Type: new Abstract: Micro-View Order-Dispatching assigns available drivers to passenger orders within each dispatch batch and is critical to the service quality and operati

applicationsarxiv-cs-ai

HiTac-WAM: A Hierarchical Tactile World Action Model for Contact-Rich Robot Manipulation

arXiv:2608.19574v1 Announce Type: new Abstract: World action models jointly predict future visual observations and actions, whereas existing tactile-aware variants typically represent future touch as

researcharxiv-cs-ro

I feel like I finally graduated.

I finally made the move from LM Studio to vLLM thanks to this post https://www.reddit.com/r/LocalLLaMA/s/NmS9CgHvqz. I may not know what it all means yet but I’m going to start diving into the docs to

Inkling and Inkling Small are now served directly by @thinkymachines on OpenRouter, free to use inside agentic harnesses only. Plug them int…

model-releasessoumith-chintala--x

Inkling and Inkling Small are now served directly by @thinkymachines on OpenRouter, free to use inside agentic harnesses only. Plug them into Claude Code, Codex, Hermes Agent and more to unlock the fr

Interrupting the Loop: Periodic Subject Changes Raise Judged Surprise and Connection in Base Language Models

arXiv:2608.19893v1 Announce Type: cross Abstract: Where does the novelty a base language model produces with no task come from, and what can an LLM judge of a long stream actually see? We dismantle a

researcharxiv-cs-ai

Tutorials

Listening Forward: Next Patch Embedding Prediction Enables Scalable Audio Learners

arXiv:2608.19863v1 Announce Type: cross Abstract: Self-supervised learning (SSL) has driven substantial progress in audio representation learning, though existing methods have increasingly relied on e

tutorialsarxiv-cs-ai

Llama.cpp DSpark PC Tree Fork (up to 3%-29.5% faster!)

Hello gang, I made an implementation of DSpark PC Tree (Parent conditioned drafting tree). This is an implementation of this research paper: https://arxiv.org/abs/2608.02123 Unaffiliated, just found i

LLMs as Acquisition Policies for Finite-Pool Materials Optimization: A Controlled Study

arXiv:2608.19790v1 Announce Type: new Abstract: Discovering materials with desirable properties often requires searching large candidate spaces while experimental or computational evaluations remain c

researcharxiv-cs-ai

Measuring What a Specification Determines: A Formal Semantic-Block Model and an Execution-Judged Benchmark

arXiv:2608.19475v1 Announce Type: cross Abstract: This work introduces a formal semantic-block model for specifications and an execution-judged benchmark for evaluating specification quality independe

Tutorials

MemTrapBench: Benchmarking Cognitive Traps in LLM Memory Use

arXiv:2608.20202v1 Announce Type: new Abstract: Memory has become a key component of large language models, enabling them to retain information and learn from long-term interactions. However, existing

tutorialsarxiv-cs-ai

Agents

MidTool: Mid-training Data Synthesis for Agentic Tool Use

arXiv:2608.20314v1 Announce Type: new Abstract: Mid-training is increasingly recognized as a critical stage for shaping the capabilities of large language models. Recent work has shown that targeted m

agentsarxiv-cs-ai

MUST-PET: MUltimodal Self-supervised learning across Tracers for whole-body PET/CT-based lesion segmentation

arXiv:2608.19666v1 Announce Type: new Abstract: Deep learning-based whole-body PET-CT lesion segmentation can support cancer staging, treatment planning, and response assessment, but generalization is

researcharxiv-cs-cv

New stealth model Ox Alpha look like glm or mimo new model

https://preview.redd.it/stgzmbpgnnkh1.jpg?width=2179&format=pjpg&auto=webp&s=19a4fd0e1786cbcac0967e1bf1ec4898447e989a https://preview.redd.it/paub9gylnnkh1.png?width=680&format=png&auto=webp&s=c13efb5

Tutorials

Orthogonal JEPA: Factorized Predictive States for Latent World Models

arXiv:2608.20065v1 Announce Type: new Abstract: World models construct latent states that support prediction, planning, and reasoning about an underlying system. Joint-embedding predictive architectur

tutorialsarxiv-cs-lg

Applications

Quantifying Event Impacts on Time Series via Multiscale Contrastive Learning

arXiv:2608.19447v1 Announce Type: new Abstract: Shocks that spread through the web, such as cybersecurity breach disclosures, can abruptly disrupt financial time series and cause substantial abnormal

applicationsarxiv-cs-lg

Question-Guided Evidence Acquisition for Multimodal Visual Question Answering

arXiv:2608.19739v1 Announce Type: cross Abstract: Multimodal LLMs can see a document, but they often can't read it reliably. Small text, tables, visual cues, and topological elements still trip them u

Qwen 3.8 27b is strong even at Q3_xxs

So usually I avoid Q3 quants because I have had bad experiences with it, models were usually too degraded, so the smallest I normally do is Q4, since I only have rtx 4060 ti 16gb. But since there hasn

Remember, Verify, or Ask? Cross-Family Evaluation of Memory Commitment in LLM Agents

model-releasesarxiv-cs-cl

arXiv:2608.19564v1 Announce Type: new Abstract: Persistent memory can personalize an LLM agent, but an incorrect durable update can silently distort future behavior. We study the memory-clarification

RepSelect: Robust LLM Unlearning via Representation Selectivity

arXiv:2606.17168v3 Announce Type: replace Abstract: When LLM weights are open or fine-tuning is available through an API, suppressing hazardous knowledge and tendencies is not enough: removal has to b

researcharxiv-cs-cl

Rethinking the Evaluation and Optimization of LLM-Based Social Simulation

arXiv:2608.19689v1 Announce Type: new Abstract: LLM-based social simulation is a promising complement to traditional methods such as surveys and behavioral experiments. A core question is how to evalu

Robust Cross-Modal Foundation Model Perception for Underwater Robots under Degraded Visual Conditions

arXiv:2608.19710v1 Announce Type: cross Abstract: Reliable underwater robotic perception remains difficult because optical imagery degrades under turbidity, wavelength-dependent attenuation, low illum

Applications

Robust Incomplete Multimodal Sentiment Analysis via Iterative Proxy Correction

arXiv:2608.19971v1 Announce Type: new Abstract: Multimodal sentiment analysis aims to infer affective states by integrating language, visual, and acoustic cues. However, real-world multimodal inputs a

applicationsarxiv-cs-cl

SceneGTMM: A Conformal Mapping-based Scene-Aware Transferable GNN-Transformer Dual-Graph Interaction Framework for Map Matching

arXiv:2608.19298v1 Announce Type: new Abstract: Map matching is a key technology connecting positioning data with high precision road networks, but it faces challenges in noise robustness, cross regio

safetyarxiv-cs-cv

Scientific Data Skills: Enabling Agent-Ready Scientific Data Services at Scale

arXiv:2608.19625v1 Announce Type: new Abstract: Scientific data are increasingly used by AI agents, yet existing dataset representations provide limited support for autonomous discovery, interpretatio

Spiking Local Interaction and Adaptive Complementary Fusion for Spiking Transformer

arXiv:2608.19238v1 Announce Type: cross Abstract: Spiking Transformers model token interactions primarily through spiking self-attention (SSA). However, binary query and key representations map contin

StreamSoccer: Event-Driven Memory for Streaming Soccer Commentary

arXiv:2608.19723v1 Announce Type: cross Abstract: Streaming video understanding requires models to causally update state as video arrives and organize growing history into semantic units that can evol

researcharxiv-cs-cl

SWE-bench Science: Can Coding Agents Resolve Engineering Tasks in Science?

model-releasesarxiv-cs-cl

arXiv:2608.19799v1 Announce Type: new Abstract: Software increasingly functions as part of the scientific instrument itself, making failures in scientific code capable of compromising not only program

TextRefine: Improving Textual Fidelity, Spatial Placement, and Glyph Rendering for Text Editing in Product Posters

arXiv:2608.19637v1 Announce Type: new Abstract: Text editing in product posters entails inserting new text or replacing existing text while preserving product appearance, background content, and globa

researcharxiv-cs-cv

Agents

The Third Restructuring of Software Form: From the Three-Tier Architecture to Storage, Models, and Agents

arXiv:2608.20201v1 Announce Type: new Abstract: Software form has undergone two paradigm shifts since its inception: Software 1.0, in which instructions determine behavior, and Software 2.0, in which

agentsarxiv-cs-ai

This is very nice work from NVIDIA. Like all high-performing approaches on ARC-AGI-3, it uses deep learning-guided on-the-fly synthesis of s…

model-releasesfrancois-chollet--x

This is very nice work from NVIDIA. Like all high-performing approaches on ARC-AGI-3, it uses deep learning-guided on-the-fly synthesis of symbolic world models, i.e. navigating the world by generatin

Truncate Bad, Upweight Good: BoN-Style Distillation via Rank-Based Classification

arXiv:2608.19748v1 Announce Type: cross Abstract: Inference-time selection methods, such as Best-of-N, improve generation by sampling a pool of candidates and selecting the top-ranked completion accor

safetyarxiv-cs-ai

Where Grounding Accuracy Lives on the IoU Curve: Label-Free Inference-Time Boundary Refinement

arXiv:2608.19553v1 Announce Type: new Abstract: Vision--language models can identify the correct referent while returning an imprecise bounding box. We study whether a frozen direct-answer model can u

WithEveryone: Unified Planning and Identity Grounding for Group Image Generation

arXiv:2608.20336v1 Announce Type: new Abstract: Identity-preserving image generation becomes increasingly unreliable when a scene must contain many specified people. Beyond retaining each identity, th