Search: “context”

arXiv:2609.39494v1 Announce Type: new Abstract: Self-distillation with privileged context adapts a language model from demonstrations by letting the model, once conditioned on a reference response, te

arXiv:2609.38744v1 Announce Type: new Abstract: Predicting molecular properties for compounds that differ structurally from labeled training molecules is important for drug discovery and materials des

arXiv:2609.38840v1 Announce Type: cross Abstract: Single-cell RNA-seq representation learning is fundamentally label-free: cell identities, states, and contexts are not fixed training targets, so what

arXiv:2609.38712v1 Announce Type: new Abstract: Long-horizon agentic workflows require models to sustain repeated state-dependent actions all while the context grows, sub-task complexity changes, and

arXiv:2609.39661v1 Announce Type: new Abstract: Self-attention gives LLMs fine-grained, query-dependent access to context, but dense token interactions incur quadratic prefill cost and a key--value ca

arXiv:2609.36883v1 Announce Type: new Abstract: Tabular foundation models (TFMs) are increasingly popular because they deliver strong predictions on new datasets through in-context learning, without t

arXiv:2609.36722v1 Announce Type: new Abstract: Large language model (LLM) agents repeatedly load reusable content, such as skills, documents, and memory entries, into the current context. Re-encoding

arXiv:2609.36364v1 Announce Type: cross Abstract: Standard video generators do not natively compact historical context into reusable memory tokens. As generation continues, the growing history makes i

arXiv:2609.36576v1 Announce Type: new Abstract: Agentic systems are now being widely used to orchestrate tools and reason over long contexts. However, the improving capabilities of the large language

arXiv:2609.37209v1 Announce Type: new Abstract: Pairwise losses are increasingly used for reward learning even when pointwise rewards are observed, with mixed empirical results. When and why do pairwi

arXiv:2609.37879v1 Announce Type: new Abstract: How many tokens from its context does a language model actually use, and what determines that number? We study this question through self-attention. Wit

arXiv:2609.38121v1 Announce Type: new Abstract: KV cache memory and bandwidth costs grow with context length and batch size, which limits efficient long-context inference. To address this bottleneck,

arXiv:2609.33746v1 Announce Type: new Abstract: At each decoding step a language model attends over the key-value (KV) cache of every earlier token, so at long context the attention call is bounded by

arXiv:2609.35497v1 Announce Type: new Abstract: Long video understanding relies on video memory to overcome the context limits of multimodal large language models. Existing methods follow a build-then

arXiv:2609.32898v1 Announce Type: new Abstract: Pretrained tabular foundation models process every dataset at a fixed depth, with inference costs growing with dataset size. To address this, we present

arXiv:2609.30288v1 Announce Type: new Abstract: In Transformer-based masked language models, attention is the primary mechanism for context mixing, but there are other ways to mix data across tokens.

arXiv:2609.31261v1 Announce Type: cross Abstract: The quadratic complexity of dense self-attention remains a central bottleneck for long-context language modeling. Many efficient alternatives address

arXiv:2609.31167v1 Announce Type: new Abstract: EEG foundation models increasingly use masked prediction to learn from unlabeled recordings, but optimizing this objective does not ensure transferable

I’m having a hard time finding out which one gives you fastest speed, maximum context with best possible quality. I can run unsloth qwen3.8 27b iq4_xs with 65k q8 kv, context without MTP and vision of

I am running qwen3.8 27b on my dual rtx 3090 (fp8 quant, unquantized cache, 129k context) and I think it works decently well with hermes, opencode etc. But! I am tempted by the new models coming out s

Running qwen3.8 27b nvfp4 on vllm at max context only gives around 8 agents with 32k context each. That doesnt seem like much; what use cases do people use multi-agent frameworks and find it helpful f

arXiv:2609.27925v1 Announce Type: new Abstract: Language models can over-condition on irrelevant preceding text: predictions already supported by local context may still change when distant, unrelated

arXiv:2609.27286v1 Announce Type: new Abstract: Long horizon language model agents continuously accumulate interaction history, increasing computational cost while making relevant information harder t

Context: we build an open-source (MIT) self-hosted database client and added a natural-language-to-SQL assistant that can point at a local runtime like Ollama instead of a hosted API. I want to share

arXiv:2609.26368v1 Announce Type: new Abstract: Long-horizon and multi-turn agents typically generate short actions and process long observations from tools and environments. This growing context dema

arXiv:2609.24417v1 Announce Type: new Abstract: Despite advances in long-context inference, large language models (LLMs) remain fundamentally limited by the key-value (KV) caching mechanisms that are

arXiv:2609.22884v1 Announce Type: new Abstract: Long-context inference has become a defining capability of large language models, but exact dense attention remains costly due to its quadratic scaling

I did a small test of the new hyped quantisation of Qwen3.8 vs the biggest quant which fits into my limited 16GB VRAM with decent context. The results are interesting. Of course, the smaller file give

AI is accelerating software development at an unprecedented pace. But as code generation scales, so do the challenges of securing the code, especially emerging AI-based vulnerability exploitations. To

arXiv:2609.19729v1 Announce Type: new Abstract: Autoregressive (AR) video generation degrades over long horizons due to an overlooked train-inference discrepancy we term KV eviction mismatch: models t

arXiv:2609.17983v1 Announce Type: new Abstract: KV-cache reuse can reduce inference cost in retrieval-augmented generation and agentic systems, but cached contexts may become stale when retrieved know

arXiv:2609.18842v1 Announce Type: new Abstract: The scaling laws hold that a language model grows more capable with more parameters and more training data, and Mixture-of-Experts (MoE) architectures h

arXiv:2609.18605v1 Announce Type: cross Abstract: As corporate AI adoption continues to grow, enterprise-grade LLM agents are being deployed into sensitive contexts such as hiring, healthcare, and fin

arXiv:2609.16997v1 Announce Type: new Abstract: Crisis sentiment analysis is especially challenging for low-resource languages such as Bangla, where language, context, and public reaction shift rapidl

arXiv:2609.17464v1 Announce Type: cross Abstract: Multi-agent systems split a task across a tree of agents and justify the split with folklore: smaller contexts, cleaner separation, parallelism. We as

I let Qwen 3.8 27B 4bit quantized with 100K context window run autonomously for 63 hours (50 million+ tokens) to try to solve the RH. Of course it did not solve it, but the experiment still shows it's

Follow up post of https://www.reddit.com/r/LocalLLaMA/s/nGMyKswrch. Thanks everyone who replied. I didn't change the specs. Might be loosing some of the memory bandwidth but will scale in future if I

arXiv:2609.14462v1 Announce Type: new Abstract: Interactive video world models must maintain broad scene context under camera motion while producing high-fidelity observations with low latency. Existi

arXiv:2609.14773v1 Announce Type: new Abstract: As LLM conversations grow to hundreds of turns, full-context injection incurs O(N^2) cumulative token costs, while lossy summarization or hard truncatio

arXiv:2609.12818v1 Announce Type: new Abstract: Long video understanding often behaves like a visual needle-in-a-haystack problem: query-relevant evidence is sparsely distributed across long temporal

As a SWE I have ultimate belief that having a irreplaceable subscription on OpenAI and Anthropic models is actually against my beliefs (I'm not fond of becoming a digital slave and being unable to do

arXiv:2609.11532v1 Announce Type: new Abstract: Commercial text-to-image systems silently revise user prompts before generating images, a step users typically cannot disable or even see. Yet, existing

arXiv:2609.11620v1 Announce Type: new Abstract: High-dimensional dense text embeddings and large language models face real obstacles in financial-disclosure analysis: context-window limits, hallucinat

model : fix MTP context kv cache allocation for deepseek2, glm4moe, … (#28630) model : fix MTP context kv cache allocation for deepseek2, glm4moe, cohere2moe architectures (#28626) model: add inverse

arXiv:2609.11913v1 Announce Type: new Abstract: Out-of-distribution length generalization, namely to extrapolate a task from short to longer context, has been studied intensively for transformers. Her

arXiv:2505.13520v2 Announce Type: replace-cross Abstract: Textbook question answering (TQA) is a complex task, requiring the interpretation of complex multimodal context. Although recent advances have