Knowledge catalogue
This post completely misses the point of @sarahwooders 's original article. The whole point is that memory == context engineering, so it can't be a 'thin conductor'. Saying 'memory is just markdown' t
arXiv:2506.17310v3 Announce Type: replace-cross Abstract: While Large Language Models (LLMs) demonstrate strong performance across domains, their long-context capabilities are limited by transient neu
arXiv:2603.04759v2 Announce Type: replace Abstract: The limited context window of contemporary large language models (LLMs) remains a primary bottleneck for their broader application across diverse do
arXiv:2604.07981v1 Announce Type: new Abstract: Long-context reasoning is essential for complex real-world applications, yet remains a significant challenge for Large Language Models (LLMs). Despite t
arXiv:2502.17421v4 Announce Type: replace-cross Abstract: As Large Language Models (LLMs) can now process extremely long contexts, efficient inference over these extended inputs has become increasingl
This will be my third update on the bc-250 cluster and for my first forray into local ai I have been having a blast. This has turned from a weekend experiment into something I think I will use in the
Bullish on this trend of making post-training more accessible. A new post-training era is upon us. If you work on agentic RL, long-context tasks (a big focus today) are expensive, inefficient, and don
arXiv:2610.11784v1 Announce Type: new Abstract: Tabular in-context learning (ICL) has emerged as a training-free and accurate paradigm for tabular prediction, but current approaches to compressing its
arXiv:2610.12085v1 Announce Type: new Abstract: Large language models (LLMs) can expose memorized training sequences under prefix-based extraction: given a prefix from a training example, the model ma
I spent the past year independently building Slow Vale, an LLM-driven life simulation. The Chinese server now has 800+ AI residents sharing one continuously running city. This is an engineering write-
arXiv:2610.10114v1 Announce Type: new Abstract: The architectural design of Large Language Models (LLMs) is shifting from traditional full-attention-only models to hybrid models, which combine differe
I've been tinkering with GLM-5.3-Flash on two 64GB CMP 170HX cards for a while, and the setup is finally stable enough that I figured I'd share it. I also compared it against the Qwen3.8-Flash-Next se
arXiv:2610.07996v1 Announce Type: cross Abstract: Tabular foundation models (TFMs) achieve strong predictive performance by conditioning on labeled demonstrations provided in context, without any para
arXiv:2610.05717v1 Announce Type: cross Abstract: In-context retrieval (ICR) is a retrieval-based form of in-context learning (ICL) in which demonstrations are retrieved from a source database based o
arXiv:2609.39929v1 Announce Type: cross Abstract: Long-context failures of RoPE-based language models can arise from RoPE's intrinsic tradeoff between maintaining stable token preferences and distingu
arXiv:2609.38137v1 Announce Type: new Abstract: Language-model (LM) harnesses enable LMs to operate effectively over long contexts using additional compute. However, existing long-context evaluations
arXiv:2609.27380v1 Announce Type: new Abstract: Likelihood-based context compression can account for cross-context redundancy through sequential scoring, but this makes compression outcomes sensitive
arXiv:2609.15028v1 Announce Type: new Abstract: In-context learning (ICL) enables models to infer tasks from demonstrations, but existing benchmarks generally lack matched text and image versions need
arXiv:2609.11192v1 Announce Type: new Abstract: Soft context compression condenses a context into a few memory tokens that a frozen LLM consumes in place of the raw text, but existing compressors fix
arXiv:2609.03213v1 Announce Type: new Abstract: Large language models (LLMs) exhibit in-context learning capabilities, where they can learn new tasks from prompt contexts without weight updates. We co
arXiv:2609.02702v1 Announce Type: new Abstract: Transformers process information causally, but long-context reasoning may depend on task state discovered only later. We formalize this mismatch through
arXiv:2608.30963v1 Announce Type: cross Abstract: Modern large language model (LLM) serving systems increasingly operate over repeated or shared context, yet each model typically performs its own pref
arXiv:2608.30295v1 Announce Type: cross Abstract: Large language models (LLMs) have demonstrated strong capabilities in handling long-context tasks, but processing such long contexts remains challengi
arXiv:2606.14747v2 Announce Type: replace-cross Abstract: Recent advancements have significantly expanded the theoretical context windows of Multimodal Embedding Models (MEMs). However, larger context
arXiv:2606.23687v2 Announce Type: replace Abstract: Large language models (LLMs) are typically pretrained on short sequences and then extended to work on longer sequences with additional training. How
arXiv:2604.07809v2 Announce Type: replace Abstract: Extending LLM context windows is hindered by scarce high-quality long-context data. Recent methods synthesize data with genuine long-range dependenc
arXiv:2608.25128v1 Announce Type: new Abstract: Multi-modal time series forecasting methods integrate auxiliary context into temporal predictions through increasingly sophisticated fusion mechanisms.
We continue to iterate on the Open Knowledge Format (OKF), an open specification that formalizes the LLM-wiki pattern into a portable, interoperable format. But a big question remains: How can you sha
arXiv:2608.20799v1 Announce Type: new Abstract: We study dynamic context scheduling as a training instrument for contextual re- inforcement learning. Rather than treating intra-episode context variati
arXiv:2608.19758v1 Announce Type: new Abstract: Long-context modeling is a pivotal capability for Large Language Models, yet the quadratic complexity of attention remains a critical bottleneck, partic
arXiv:2608.18082v1 Announce Type: new Abstract: Although context windows have expanded significantly in recent years, hallucinations in long-context summarization remain a challenge. Long novels are b
arXiv:2608.17188v1 Announce Type: cross Abstract: Multi-agent AI workflows are limited not only by model quality but by token cost, latency, and context-window quality. This paper presents a practitio
Four small architecture decisions can cost up to 47% of a model's long-context performance. New research from Ai2, Carnegie Mellon, and the University of Washington isolates them. Normalization, GQA,
arXiv:2608.09153v1 Announce Type: new Abstract: Production AI agents fail when their context sources -- system prompts, knowledge bases, tool descriptions, and procedural skills -- contain errors or g
arXiv:2608.00311v1 Announce Type: new Abstract: Long-context inference with large language models (LLMs) is costly: self-attention during prefill scales quadratically with sequence length, and the key
arXiv:2607.29422v1 Announce Type: cross Abstract: Automated vulnerability repair aims to reduce the time and effort required to patch security flaws from a vulnerability triage report. Recent agentic
New research from Meta and CMU. This one is on agentic context management for long horizon tasks. (bookmark it) Production agents accumulate context every turn. The usual fix compresses on a token thr
We scaled a robot model natively to 8,000 timesteps of context, 5 minutes worth of muscle memory, with constant inference cost. Robot policies used to live their lives a few frames at a time (< 0.1 se
Long-context handling remains a core challenge for language models: even with extended context windows, models often fail to reliably extract, reason over, and use the information across long contexts
arXiv:2509.19228v2 Announce Type: replace Abstract: Large Language Models (LLMs) face significant computational challenges when processing long contexts due to the quadratic complexity of self-attenti
arXiv:2606.29718v1 Announce Type: cross Abstract: Extensive context has become the norm as Large Language Models (LLMs) are increasingly deployed in long-horizon tasks. The concern that increasing con
arXiv:2606.06906v1 Announce Type: cross Abstract: Long-context question answering (QA) remains challenging for smaller language models even when answer-bearing evidence is already present in the input
arXiv:2605.28713v1 Announce Type: new Abstract: Context compression aims to shorten long context inputs with minimal information loss for LLM inference acceleration. While existing methods have shown
arXiv:2605.23071v1 Announce Type: new Abstract: Large language models (LLMs) increasingly rely on long-context processing, but expanding context windows introduces substantial computational and financ
arXiv:2605.20201v1 Announce Type: new Abstract: Recent large language models support inputs of up to 10 million tokens, yet they perform poorly on long-context tasks that require complex reasoning. Su
arXiv:2510.18830v2 Announce Type: replace Abstract: The adoption of long context windows has become a standard feature in Large Language Models (LLMs), as extended contexts significantly enhance their
arXiv:2605.18226v1 Announce Type: cross Abstract: Modern large language model (LLM) applications increasingly rely on long conditioning prefixes to control model behavior at inference time. While pref
Excited to share our new paper: RoPE Distinguishes Neither Positions Nor Tokens in Long Contexts, Provably LLMs often fail on inputs well within their advertised context lengths. We show that these fa
arXiv:2605.14589v1 Announce Type: new Abstract: Extending the context window of large language models typically requires training on sequences at the target length, incurring quadratic memory and comp
arXiv:2605.12703v1 Announce Type: cross Abstract: We introduce MMCL-Bench, a benchmark for multimodal context learning: learning task-local rules, procedures, and empirical patterns from visual or mix
arXiv:2502.03061v2 Announce Type: replace Abstract: We introduce the problem of best arm identification (BAI) with post-action context, a new BAI problem in a stochastic multi-armed bandit environment
arXiv:2605.11744v1 Announce Type: new Abstract: Transformer-based large language models face severe scalability challenges in long-context generation due to the computational and memory costs of full-
arXiv:2510.13799v2 Announce Type: replace Abstract: As retrieval-augmented generation (RAG) tackles complex tasks, increasingly expanded contexts offer richer information, but at the cost of higher la
arXiv:2601.13684v2 Announce Type: replace Abstract: The linear memory growth of the KV cache poses a significant bottleneck for LLM inference in long-context tasks. Existing static compression methods
arXiv:2409.06679v3 Announce Type: replace Abstract: Processing long contexts is increasingly important for Large Language Models (LLMs) in tasks like multi-turn dialogues, code generation, and documen
Running HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive Q4_K_M with llama.cpp at ~600 tok/s prefill and 23 tok/s decode, 131k context window, Q8 KV cache - on an RTX 2060 6GB + 32GB DDR4 RAM.
Hi guys, I am pretty happy to announce MegaCapybara. Purpose build engine for RTX5090 that is focused on Qwen3.8 27B (more will come later). GITHUB (Engine) HUGGINGFACE (weights) Why ? 1. It beats Nin
arXiv:2610.01712v1 Announce Type: new Abstract: In-context learning (ICL) enables a pretrained model to infer a task from demonstrations without updating its parameters. While much of the existing the
arXiv:2610.00526v1 Announce Type: cross Abstract: In-context learning (ICL) can be amortized into latent objects (task vectors, function vectors, context vectors) that recover few-shot behavior at zer
arXiv:2610.00654v1 Announce Type: new Abstract: Personalization has long relied on customer data to infer what an individual is likely to value. We call this customer evidence: the customer's historic