Search: “context”

Bayesian Attention Mechanism: A Probabilistic Framework for Positional Encoding and Context Length Extrapolation

arXiv:2505.22842v4 Announce Type: replace Abstract: Transformer-based language models rely on positional encoding (PE) to handle token order and support context length extrapolation. However, existing

11 May 2026

Content type

All Blog X Post Paper YouTube Reddit GitHub

Clear filters

Model Releases

Faithfulness-QA: A Counterfactual Entity Substitution Dataset for Training Context-Faithful RAG Models

DGX agent

arXiv:2604.25313v1 Announce Type: new Abstract: Retrieval-Augmented Generation (RAG) models frequently produce answers grounded in parametric memory rather than the retrieved context, undermining the

model-releasesarxiv-cs-cl

29 Apr 2026

AdaComp: Extractive Context Compression with Adaptive Predictor for Retrieval-Augmented Large Language Models

arXiv:2409.01579v2 Announce Type: replace-cross Abstract: Retrieved documents containing noise will hinder RAG from detecting answer clues and make the inference process slow and expensive. Therefore,

DepthKV: Layer-Dependent KV Cache Pruning for Long-Context LLM Inference

arXiv:2604.24647v1 Announce Type: cross Abstract: Long-context reasoning is a critical capability of large language models (LLMs), enabling applications such as long-document understanding, summarizat

Quantum Knowledge Graph: Modeling Context-Dependent Triplet Validity

arXiv:2604.23972v1 Announce Type: cross Abstract: Knowledge graphs (KGs) are increasingly used to support large lan guage model (LLM) reasoning, but standard triplet-based KGs treat each relation as g

RetroInfer: A Vector Storage Engine for Scalable Long-Context LLM Inference

arXiv:2505.02922v3 Announce Type: replace Abstract: Recent large language models (LLMs) are rapidly extending their context windows, yet inference throughput lags due to increasing GPU memory and band

hardwarearxiv-cs-lg

Agents

SWE-Pruner: Self-Adaptive Context Pruning for Coding Agents

arXiv:2601.16746v3 Announce Type: replace-cross Abstract: LLM agents have demonstrated remarkable capabilities in software development, but their performance is hampered by long interaction contexts,

agentsarxiv-cs-cl

SpikingBrain2.0: Brain-Inspired Foundation Models for Efficient Long-Context and Cross-Platform Inference

arXiv:2604.22575v1 Announce Type: new Abstract: Scaling context length is reshaping large-model development, yet full-attention Transformers suffer from prohibitive computation and inference bottlenec

hardwarearxiv-cs-lg

27 Apr 2026

Back to the Future: The Role of Past and Future Context Predictability in Incremental Language Production

arXiv:2511.07752v3 Announce Type: replace Abstract: Contextual predictability shapes how we choose and encode words in production. The effects of a word's predictability given preceding or past contex

applicationsarxiv-cs-cl

24 Apr 2026

Stream2LLM: Overlap Context Streaming and Prefill for Reduced Time-to-First-Token (TTFT)

arXiv:2604.16395v2 Announce Type: replace-cross Abstract: Context retrieval systems for LLM inference face a critical challenge: high retrieval latency creates a fundamental tension between waiting fo

hardwarearxiv-cs-ai

24 Apr 2026

TTKV: Temporal-Tiered KV Cache for Long-Context LLM Inference

arXiv:2604.19769v1 Announce Type: cross Abstract: Key-value (KV) caching is critical for efficient inference in large language models (LLMs), yet its memory footprint scales linearly with context leng

23 Apr 2026

Syntax as a Rosetta Stone: Universal Dependencies for In-Context Coptic Translation

arXiv:2604.18758v1 Announce Type: new Abstract: Low-resource machine translation requires methods that differ from those used for high-resource languages. This paper proposes a novel in-context learni

22 Apr 2026

AlphaContext: An Evolutionary Tree-based Psychometric Context Generator for Creativity Assessment

arXiv:2604.18398v1 Announce Type: new Abstract: Creativity has become a core competence in the era of LLMs and human-AI collaboration, underpinning innovation in real-world problem solving. Crucially,

applicationsarxiv-cs-cl

Context Matters: Peer-Aware Student Behavioral Engagement Measurement via VLM Action Parsing and LLM Sequence Classification

arXiv:2601.06394v3 Announce Type: replace Abstract: Understanding student behavior in the classroom is essential to improve both pedagogical quality and student engagement. Existing methods for predic

researcharxiv-cs-cv

Evidence-Augmented Policy Optimization with Reward Co-Evolution for Long-Context Reasoning

arXiv:2601.10306v2 Announce Type: replace-cross Abstract: While Reinforcement Learning (RL) has advanced LLM reasoning, applying it to long-context scenarios is hindered by sparsity of outcome rewards

safetyarxiv-cs-cl

How Training Data Shapes the Use of Parametric and In-Context Knowledge in Language Models

arXiv:2510.02370v3 Announce Type: replace Abstract: Large language models leverage both parametric knowledge acquired during pretraining and in-context knowledge provided at inference time. Crucially,

applicationsarxiv-cs-cl

KV Packet: Recomputation-Free Context-Independent KV Caching for LLMs

arXiv:2604.13226v1 Announce Type: new Abstract: Large Language Models (LLMs) rely heavily on Key-Value (KV) caching to minimize inference latency. However, standard KV caches are context-dependent: re

16 Apr 2026

Banger paper from NVIDIA. Agentic reasoning needs models that are not just capable, but efficient at long-context inference. The agent model…

Banger paper from NVIDIA. Agentic reasoning needs models that are not just capable, but efficient at long-context inference. The agent model layer is moving toward open, long-context, high-throughput

model-releasesdair-ai--x

15 Apr 2026

Latent-Condensed Transformer for Efficient Long Context Modeling

arXiv:2604.12452v1 Announce Type: new Abstract: Large language models (LLMs) face significant challenges in processing long contexts due to the linear growth of the key-value (KV) cache and quadratic

15 Apr 2026

UCS: Estimating Unseen Coverage for Improved In-Context Learning

arXiv:2604.12015v1 Announce Type: cross Abstract: In-context learning (ICL) performance depends critically on which demonstrations are placed in the prompt, yet most existing selectors prioritize heur

15 Apr 2026

SinkTrack: Attention Sink based Context Anchoring for Large Language Models

arXiv:2604.10027v1 Announce Type: new Abstract: Large language models (LLMs) suffer from hallucination and context forgetting. Prior studies suggest that attention drift is a primary cause of these pr

researcharxiv-cs-cv

14 Apr 2026

Tutorials

Transformers Learn Latent Mixture Models In-Context via Mirror Descent

arXiv:2604.10848v1 Announce Type: new Abstract: Sequence modelling requires determining which past tokens are causally relevant from the context and their importance: a process inherent to the attenti

tutorialsarxiv-cs-lg

14 Apr 2026

STIndex: A Context-Aware Multi-Dimensional Spatiotemporal Information Extraction System

arXiv:2604.08597v1 Announce Type: cross Abstract: Extracting structured knowledge from unstructured data still faces practical limitations: entity and event extraction pipelines remain brittle, knowle

13 Apr 2026

how to create .md files and set context window more than 64k for ollama and claude running locally.

This Reddit thread discusses how to configure Ollama for use with Claude Code locally, covering two key setup steps. By default, Ollama uses a context window of only 4,096 tokens — insufficient for Cl

model-releasesr-ollama

12 Apr 2026

Bridging Natural Language and Microgrid Dynamics: A Context-Aware Simulator and Dataset

arXiv:2604.05429v2 Announce Type: replace-cross Abstract: Addressing the critical need for intelligent, context-aware energy management in renewable systems, we introduce the OpenCEM Simulator and Dat

applicationsarxiv-cs-ai

10 Apr 2026

In-Context Learning in Speech Language Models: Analyzing the Role of Acoustic Features, Linguistic Structure, and Induction Heads

arXiv:2604.06356v1 Announce Type: cross Abstract: In-Context Learning (ICL) has been extensively studied in text-only Language Models, but remains largely unexplored in the speech domain. Here, we inv

10 Apr 2026

Transformer-based In-Context Score Operator Learning for Diffusion Models

arXiv:2610.13690v1 Announce Type: new Abstract: Diffusion models typically learn the generative dynamics of a single target distribution or a family of distributions indexed by conditioning variables

12 Oct 2026

I ran Qwen 3.8 Flash-Next on my AMD 7900 XTX at 500k context. All local and what a shift it has been.

model-releasesr-localllama

I was running Qwen3.8-Flash-Next (base and later Swift-1.5 NVFP4) on a paid pod seat for about a month, and it started answering with short strings that echoed its own earlier reasoning. I assumed the

11 Oct 2026

Agents

Is anyone running Qwen3.8 Flash Next with a 1M context?

So, while doing a search to read the model card again, and because I didn't memorize the huggingface URL, I saw an AI 'answer' at the top of the search, stating that while it natively supports a ~244k

agentsr-localllama

10 Oct 2026

Tutorials

Beyond Visual Enhancement: Adaptive Multi-Context Steering to Mitigate LVLM Hallucinations

arXiv:2610.11907v1 Announce Type: cross Abstract: Hallucination remains a significant challenge in Large Vision-Language Models (LVLMs). Existing training-free methods generally mitigate hallucination

tutorialsarxiv-cs-ai

9 Oct 2026

SFT-as-Context Mitigates Forgetting in Supervised Fine-Tuning

arXiv:2610.11132v1 Announce Type: cross Abstract: Supervised fine-tuning (SFT) equips large language models (LLMs) with specialized capabilities, but often comes at the cost of forgetting the general

9 Oct 2026

Spatial Induction Heads: In-Context Learning of Multidimensional Cellular Automata

arXiv:2610.09124v1 Announce Type: new Abstract: Induction heads provide a mechanistic account of in-context learning in sequential data, but existing theory largely assumes that the context relevant t

safetyarxiv-cs-lg

8 Oct 2026

Internalizing Agent Experience into Diffusion Model Weights via On-Policy Context Distillation

arXiv:2610.07250v1 Announce Type: new Abstract: Wrapping an image generation model in an agentic harness can effectively boost Text-to-Image task performance: the harness can leverage memory, skills,

safetyarxiv-cs-ai

7 Oct 2026

Long-MDR: Long-Context Reinforcement Learning for Multimodal Deep-Research Agents

arXiv:2610.05195v1 Announce Type: new Abstract: The next generation of multimodal research agents must reason over long-lived research histories rather than short model completions. During a single ta

safetyarxiv-cs-cv

pi^2: Structure-Originated Reasoning Data Improves Long-Context Reasoning Ability of Large Language Models

arXiv:2604.05114v2 Announce Type: replace-cross Abstract: We study a QA curation pipeline for improving long-context complex reasoning in large language models (LLMs). Our approach, pi^2, constructs h

researcharxiv-cs-lg

RepICL: Reusable In-Context Prediction Across Heterogeneous Representation Spaces

arXiv:2610.05852v1 Announce Type: new Abstract: Frozen representations are widely reused for downstream classification, yet each new task typically requires fitting a new predictor. We ask whether the

TIGER: Time-Series Classification with In-Context-Learning Gated Ensemble of Representations

arXiv:2610.06156v1 Announce Type: new Abstract: A representation family is a distinct way of extracting features from time series. Ensemble algorithms that combine several representation families rema

Dual-Context Analog Retrieval for Time Series Forecasting

arXiv:2610.03491v1 Announce Type: new Abstract: Most long-term time-series forecasting models map the look-back window directly to the full horizon in a single pass. While efficient, this design does

applicationsarxiv-cs-lg

5 Oct 2026

Can LLMs Reason Over Long Horizons? An Empirical Evaluation of Context Strategies for Longitudinal Clinical Reasoning

arXiv:2610.00562v1 Announce Type: cross Abstract: Longitudinal clinical reasoning requires large language models (LLMs) to identify and integrate relevant evidence distributed across extended patient

2 Oct 2026

Qwen3.8-Flash-Next (5.05bpw + ngram at bf16) exl3 on one r9700: 863 t/s prefill and 35 t/s decode at 230k context (256k max), is that ok or am i missing something?

Hi, i got QFN running on my single r9700 but im not sure if i did everything right to get the best quality and speed out of this setup. Dont want to annoy anybody, maybe someone with the same card can

hardwarer-localllama

2 Oct 2026

Shifting Mechanisms: How Positional Encoding Choice Shapes In-Context Retrieval

arXiv:2609.38530v1 Announce Type: new Abstract: Language models increasingly use architectures that vary attention span and positional encoding across layers, such as applying RoPE with sliding-window

1 Oct 2026

When Context Changes: Understanding Update Failures in LLMs

arXiv:2609.38866v1 Announce Type: new Abstract: As preferences, goals, and facts change, LLM agents must use the current state while earlier versions remain in context. Yet they can answer with an old

1 Oct 2026

Adapting Linear-Time Architectures for Tabular In-Context Learning

arXiv:2609.36337v1 Announce Type: new Abstract: Tabular foundation models achieve strong performance by conditioning on labelled examples in context, but softmax attention limits their use on large da

researcharxiv-cs-lg

Are In-Context Images Worth 10 Dimensions?

arXiv:2609.37659v1 Announce Type: new Abstract: There has been significant work on understanding the In-Context Learning capabilities of Large Language Models, especially on the induction circuit. For

researcharxiv-cs-cv

Agents

CADOC: Cache-Aware Dynamic Object Context for Long-Horizon Agents

arXiv:2609.37012v1 Announce Type: new Abstract: For a long-horizon agent, context is the bottleneck: the history is resent with every request, the window caps task length, and reasoning degrades as th

agentsarxiv-cs-ai

ContextRender: From Execution Dependencies to Agent Context

arXiv:2609.37743v1 Announce Type: new Abstract: LLM agents performing long-horizon tasks accumulate tool results that later steps may need. Passing the full history to every invocation is costly even

Local Ai

KV-Kaizen: Learning Context-Adaptive Cache Compression Choices

arXiv:2609.37988v1 Announce Type: new Abstract: As the context size of text processed with an LLM grows, the size of KV caches can outstrip the memory allocated for the original model weights. This im

local-aiarxiv-cs-ai

ProgressCompass: Embodied Progress Reward Models Are Lost Without the Right Context

model-releasesarxiv-cs-cl

arXiv:2609.36684v1 Announce Type: new Abstract: Embodied agents now take on ever longer tasks. For long tasks, knowing only whether a task finally succeeds or fails says little; the steps along the wa