Context Language Models Explained: Context as a File

Context Language Models Let the Model Rewrite Its Own Context

The paper Context Language Models (arXiv 2609.37725), by Rulin Shao, Pang Wei Koh, Luke Zettlemoyer, Mike Lewis and colleagues, lets an LLM edit its own context window like a file. It is a different work from ContextLM (arXiv 2510.20280) and from the 2015 Document-Context language models, which share the name.

It was submitted on 29 September 2026 by researchers at the University of Washington and Meta Superintelligence Labs, with one author each from MIT and Trillium Labs. On its benchmarks it matched or beat the baselines the authors tested, usually with less theoretical compute. We separate what the authors report from our own reading, and every number comes from the paper's text and tables.

What is a Context Language Model?

A Context Language Model (CLM) is an ordinary LLM that is allowed to edit its own context window like a file, instead of a harness deciding what to keep, summarize or drop. The paper proposes it as a way for long-running agents to manage their own working memory, and tests it zero-shot with existing models.

The Problem: Long-Running Agents Outgrow the Window

Watch what happens when an agent's history grows past the context limit and the oldest turns get cut.

What to notice: plain truncation can drop the user's original question. The authors built a diagnostic, ContextBench, with a 32K context limit and input volume up to 24 times that limit. None of the existing strategies they tested was perfect, even on simple synthetic tasks.

Three Ways to Handle a Full Window

Step through the same overflowing conversation under truncation, a fixed-rule summary, and model-managed context.

What to notice: a fixed rule ("summarize at the threshold") can drop the one document ID the agent needs ten turns later, and action-based tools can offload data but could not evict it from the live context on demand. Model-managed context lets the model decide what stays.

How Does a Context Language Model Work?

A Context Language Model (CLM) keeps its live context mirrored in an ordinary file and gets Bash access to edit it. Edits sync back into what the model sees on its next turn, and if it does not edit, new tokens are simply appended as usual.

What to notice: there is no special "compact" tool, just ordinary shell or Python edits. The authors report behaviors nobody programmed in: a scoreboard updated through 163 in-place edits that kept context at 6-8K tokens, and a helper function the model wrote itself, then called 37 times. An agent swarm is several synced context files.

What Did the Context Language Models Paper Find?

Applied zero-shot, CLMs matched or beat the strongest baselines the authors compared against on most long-horizon agent benchmarks, usually with less compute. On BrowseComp-Plus with Qwen3.6-27B and a 32K context limit, CLM scored 59.4%, 11.4% (relative) above Codex-style summarization, with 21.5% fewer prefix-reuse FLOPs.

• EdgeBench-10, 12 hours (Qwen3.6-27B, 32K): 5% higher than Codex-style summarization (44.6 vs 42.3), at 179 vs 437 PFLOPs per trial.
• TerminalBench 2.1: matches accuracy at 70% of the FLOPs. TBLite: 73.7% vs 67.0%, at 91% of the FLOPs.
• Software World (24 hours, six agents, GPT-5.6-Sol, 272K): 65% greater downstream speedup than a summary-based swarm, at the same spend.
• BrowseComp-Plus after RL (Qwen3.5-9B): 28.8% to 42.5%, against 34.7% to 42.1% for a summary harness.
• Steering (held-out KV Store): accuracy rose from 38.3% to 74.2%, up to 35.9 points, with a Qwen3.6-27B agent and Claude Fable 5.1 proposing the skill. ContextBench is the authors' own synthetic diagnostic.
• Suffix Cache Reuse: 35% less server-side compute than standard SGLang at matched performance on BrowseComp-Plus; about two thirds of the reuse comes from stripping reasoning tokens.

What to notice: every gain is against a specific baseline, and "59% fewer" FLOPs is theoretical compute, not a bill or latency. The 47.6% relative RL gain is over the model's own weak start (28.8%). Against the RL-trained summary harness, 42.5% vs 42.1% is matching (the paper's word), and only CLM got the efficiency reward, so its 38.8% FLOPs gap (1.34 vs 2.19 PFLOPs) is not purely architecture.

What Does the Context Language Models Paper Not Show?

It is a preprint, not yet peer reviewed, and the two headline benchmarks (BrowseComp-Plus and EdgeBench) use one open-weight model (Qwen3.6-27B) at a 32K context limit. Compute is measured in theoretical FLOPs, not latency. At a 128K budget on EdgeBench-10, plain CLM (47.3) did not beat summarization (47.8), though it used less compute.

What to notice: the edge narrows as the budget grows. At 128K, CLM with subagents reached 50.2 against 47.3 for plain CLM, at about the same compute as summarization (219 vs 222 PFLOPs); at 32K, subagents added little (44.2 vs 44.6). More gaps:

• Capability dependence. In the RL setup the 9B model started at 28.8% against 34.7% for the summary harness, yet in a separate untrained 32K comparison it scored 39.9% vs 37.7%, so the gap depends on the setup. It also edited 1.4 times per TerminalBench 2.1 task against 2.6 for Qwen3.6-27B.
• Safety is flagged, not solved. Editable context can carry prompt injections across turns; defenses are future work.
• Missing. We found no latency or wall-clock results, and nothing on how hard these runs are to debug and audit.

Should You Rebuild Your Agent's Context Handling Around CLMs Now?

Our read: not yet, but it is worth prototyping. The evidence is zero-shot benchmark results plus one reinforcement learning run, with no production reliability data. A cheap first step is to let your agent summarize or drop parts of its own context under a clear instruction, and to log every edit so you can compare it against your current fixed rule.

One builder detail is easy to miss: editing the middle of a context invalidates the usual prefix cache.

What to notice: the authors add Suffix Cache Reuse (SCR), which reuses cached states for tokens that survive an edit. It needs serving-stack changes (they patch SGLang), and about two thirds of its extra reuse (5.3 of 7.8 points) comes from stripping reasoning tokens, not from edits. Our read: count cache cost, not just tokens, and benchmark on your own tasks and context limits. Fixed rules may stay the safer floor for small models.

How Could Context Language Models Come Up in an AI Engineer Interview?

Expect it as a design question: your agent's history outgrows the window, so what do you do? A strong answer compares truncation, fixed-rule summarization and model-managed context on information loss, predictability, cost, debuggability and cache behavior, then commits to a choice for the stated workload and says how it would be evaluated.

Our summary of the trade-offs. The debuggability and caching points are our read, not measurements from the paper.

Likely question: "Your agent's conversation no longer fits the context window. How do you handle it?"

Model answer line: "I would keep recent turns verbatim, summarize older ones with a rule I can inspect, pin the facts that must never drift, and only let the model edit its own context where I can log and replay every edit."

For the full set of trade-offs, see our context engineering interview questions and answers. To drill the topic, try the LLM question bank, or practice with AI mock interviews at InterviewStack.io.

Where This Leaves Us

If a model can decide what to say, it can probably decide what to remember. The early numbers support that on long-horizon tasks, with real caveats about scale, cost accounting and safety. Read it as a prompt to test model-managed context on your own agents, not as a verdict.