Search: “context”

arXiv:2604.24715v1 Announce Type: new Abstract: Hybrid sequence models that combine efficient Transformer components with linear sequence modeling blocks are a promising alternative to pure Transforme

arXiv:2509.21275v3 Announce Type: replace-cross Abstract: Long context training is crucial for LLM's context extension. Existing schemes, such as sequence parallelism, incur substantial communication

I edited the intro because I realized I buried the lede originally- The 1M context window is a double-edged sword. It allows Claude to do more complex tasks but it can also leads to more context pollu

arXiv:2604.13847v1 Announce Type: new Abstract: While sparse attention mitigates the computational bottleneck of long-context LLM training, its distributed training process exhibits extreme heterogene

arXiv:2604.12434v1 Announce Type: cross Abstract: In-context learning enables transformers to adapt to new tasks from a few examples at inference time, while grokking highlights that this generalizati

Harness, Memory, Context Fragments, & the Bitter Lesson this is a work in progress mental dump on interesting intersections between how we use and design a harness, implications for memory being accum

KIV is a project shared on r/MachineLearning presenting a drop-in replacement for HuggingFace's `DynamicCache` that enables up to 1 million token context windows on consumer hardware with only 12GB of

arXiv:2601.07994v4 Announce Type: replace Abstract: Large Language Models (LLMs) increasingly operate over long-form dialogues with frequent topic shifts. While recent LLMs support extended context wi

arXiv:2610.16990v1 Announce Type: new Abstract: Tabular data generation aims to synthesize realistic samples from limited observations while faithfully capturing the underlying data distribution. Tabu

arXiv:2610.11715v1 Announce Type: new Abstract: Hypernetworks that map a context directly to a LoRA adapter let a large language model carry that context in its weights, but prior work has demonstrate

arXiv:2610.07345v1 Announce Type: cross Abstract: IAM policy analysis typically emphasizes the authorization capabilities encoded in a policy, but security analyst review priority may also depend on t

arXiv:2610.07572v1 Announce Type: new Abstract: In-context learning (ICL) adapts frozen large multimodal models (LMMs) to new tasks from a few demonstrations (demos), but re-encodes them at every quer

arXiv:2610.04047v1 Announce Type: cross Abstract: A language model reads long text in one quadratic forward pass, stops at the context window, and loses accuracy with length before reaching it. We ask

arXiv:2610.03828v1 Announce Type: new Abstract: Quadruped robots entering hospitals, care homes, and quiet offices must be context-appropriate not only in where they move but in how loudly they move:

arXiv:2610.00412v1 Announce Type: new Abstract: KV cache pruning reduces long-context inference memory usage by evicting less important key-value pairs. KVzip estimates importance through context reco

arXiv:2609.38385v1 Announce Type: new Abstract: Supervised fine-tuning of discrete diffusion language models masks some response tokens and trains the model to recover their original values from the v

arXiv:2609.40118v1 Announce Type: new Abstract: As LLM capabilities advance, agents are tackling increasingly complex tasks over longer horizons. Their growing interaction histories make memory compac

arXiv:2609.34799v2 Announce Type: replace Abstract: Language-conditioned trajectory generation is here, but its evaluation has not kept pace. Existing pedestrian trajectory metrics compare trajectorie

Introducing 'Scaling In-Context Imitation Learning' (SAIL) to be presented at #IROS2026. This work is a collaboration between Sakana AI and the University of Tokyo. Blog: https://pub.sakana.ai/sail Wh

arXiv:2609.26121v1 Announce Type: cross Abstract: As agent capabilities have grown, practical limitations increasingly stem from constrained context windows rather than model capacity. Common strategi

arXiv:2609.18394v1 Announce Type: new Abstract: We report the results of a Turing Test conducted in Finland in the Finnish language. Because languages and cultural contexts are unevenly represented in

arXiv:2609.17573v1 Announce Type: cross Abstract: Diffusion large language models (dLLMs) are emerging as a promising generative paradigm that complements autoregressive decoding. In long-context sett

arXiv:2609.16461v1 Announce Type: cross Abstract: Agentic large language model (LLM) systems rely on long interaction histories to preserve instructions, tool states, intermediate decisions, and unres

arXiv:2609.12230v1 Announce Type: new Abstract: Question-answering often requires reasoning across multiple connected facts rather than retrieving a single isolated relation. Knowledge graphs (KGs) pr

arXiv:2609.10739v1 Announce Type: cross Abstract: A truth probe fitted where truthful reporting and a task's prescribed action coincide cannot distinguish those targets from its fitting labels alone.

arXiv:2609.07672v1 Announce Type: new Abstract: Production content-generation systems must integrate a user's immediate task, long-term brand identity, historical evidence, and revision feedback. We p

arXiv:2603.23114v2 Announce Type: replace Abstract: A human's moral decision depends heavily on the context. Yet research on LLM morality has largely studied fixed scenarios. We address this gap by in

arXiv:2609.06851v1 Announce Type: cross Abstract: Broad misalignment has been produced by finetuning on narrow data, harmful or benign, and in context only by demonstrations of the undesirable behavio

arXiv:2609.06852v1 Announce Type: new Abstract: Although highly effective in vision and language domains, applying in-context learning to robotics remains challenging. Existing autoregressive in-conte

arXiv:2603.25183v2 Announce Type: replace Abstract: Context-aware machine translation (MT) leverages document-level information, yet it does not consistently outperform sentence-level MT, as contextua

arXiv:2609.00659v1 Announce Type: new Abstract: Cooperative perception allows a drone fleet to combine observations from multiple viewpoints. However, existing systems typically fix their feature-shar

arXiv:2609.00753v1 Announce Type: cross Abstract: When contextual information conflicts with the knowledge stored in model parameters, activation directions can be used to decode and steer which sourc

arXiv:2608.29492v1 Announce Type: new Abstract: Large Language Models (LLMs) often favor Western-associated entities across cultural contexts. Conventional debiasing methods aim for uniform neutrality

arXiv:2608.29899v1 Announce Type: cross Abstract: Dense retrieval over long documents is expensive. Token-level encoders scale quadratically in sequence length, and most long-context embedding models

arXiv:2608.29058v1 Announce Type: new Abstract: Dynamic sparse attention can reduce the quadratic cost of long-context prefilling without changing model weights. MInference assigns each attention head

arXiv:2608.28911v1 Announce Type: cross Abstract: The key-value (KV) cache is the dominant memory bottleneck of long-context large language model (LLM) inference, growing linearly with context length.

arXiv:2608.30252v1 Announce Type: new Abstract: Long-context LLM applications such as document summarization and multi-turn agents require generation from prefixes spanning tens of thousands of tokens

arXiv:2601.08747v3 Announce Type: replace-cross Abstract: Current context augmentation methods, such as retrieval-augmented generation, play a crucial role in bridging a model's internal knowledge bou

// ContextLeak in AI Agents // The whole attack surface here is a tool name and a tool description. Stealing an LLM agent's runtime context, meaning the user prompt, the execution trajectory and the t

arXiv:2608.28209v1 Announce Type: new Abstract: Unlike images and text, applying transfer learning to tabular data is challenging due to heterogeneity in feature types, structures, and semantics acros

Hey guys, I tested Qwen3.8 Flash with llama.cpp from CPU-only to the full 96GB of my RTX PRO 6000. Short version: CPU-only reached 8.34 tok/s at a 2K prompt Full 96GB reached 109.07 tok/s At 245K cont

arXiv:2608.26145v1 Announce Type: new Abstract: Our research focuses on evaluating literature reviews generated in short and long context settings of large language models (LLMs) to investigate the im

arXiv:2608.20853v1 Announce Type: new Abstract: Evaluation of long-context Large Language Models (LLMs) has advanced rapidly. However, most existing benchmarks are limited to the document level and fo

arXiv:2608.19857v1 Announce Type: new Abstract: For AI agents to be useful beyond simple chat, they must hold sensitive user context such as calendars, credentials, health records, and financial data.

arXiv:2608.19920v1 Announce Type: new Abstract: A lot of prior work addressed key-value (KV) cache selection and compression by sparse attention to enable long-context inference for transformer langua

arXiv:2608.18911v1 Announce Type: new Abstract: We introduce the CDSP (context-conditional deliberation signal pipeline), converting an investment committee's meeting transcripts into structured predi