Context Management Is Compression

In the last post, we added memory: facts and preferences that can survive across runs. Memory helps the agent remember what matters.

Compression will solve a different problem. It helps the agent.. forget the right things.

memory:
  - what should survive across runs

compression:
  - what a long run should keep from its own history

An agent can easily create more context than it can use. It reads files. It runs commands. It fails, retries, edits, tests, changes direction, and asks for more information. If we keep appending everything, the model eventually receives a transcript instead of a working state.

Why Compression Exists

The obvious reason is the context window. A model can only receive so much text in one request. Once the conversation, files, tool outputs, project context, and memory no longer fit, something has to be left out.

The physical limit is token based. The request is not measured in files, messages, or paragraphs. It is measured in tokens, and almost everything competes for that space. At the time of writing the larges available commerical model reaches 1.1M tokens, which is roughly 1,400 pages of text in one prompt. That might look like a lot, but let's dip deeper!

context window contains:
  - system instructions
  - user messages
  - assistant messages
  - tool definitions
  - tool results
  - project context
  - memory
  - images or documents
  - the model's next output

That last line is easy to miss. The model does not only need room to read the prompt. It also needs room to answer. If the input consumes the whole window, the loop has nowhere to go.

But compression is not only a workaround for a technical limit. It is also a product design choice. The harness is deciding what kind of agent experience the user gets.

product choices:
  - should the agent show what it compressed?
  - should the user approve a handoff summary?
  - should old tool output stay inspectable?
  - should compression happen automatically?
  - should the model know what was dropped?
  - should risky work reset context more often?

These choices affect your user trust. If compression is invisible, the user may not understand why the agent forgot something. If compression is too noisy, the user has to manage the agent instead of doing the work. A good product makes compression visible enough to inspect, but automatic enough that the loop keeps moving.

The Transcript Problem

The simplest loop appends every message and tool result back into the conversation.

messages.append(model_response)
messages.append(tool_result)
messages.append(next_model_response)
messages.append(next_tool_result)

That is perfect for us learning the loop, but it is not enough for real, long work. After a while, the transcript contains useful facts mixed with noise:

- old failed plans
- long command output
- file contents that changed later
- repeated error messages
- irrelevant search results
- half-finished reasoning
- final decisions hidden in the middle

More context is not automatically more understanding. The model still has to decide what matters inside that context, and irrelevant detail can distract it from the current task.

Compression Is Selection

Compression is not just making text shorter. It is selecting what the next model call needs in order to continue the work.

raw transcript
  -> compression
  -> handoff state
  -> next model call

A good compressed state does not preserve every step. It preserves the shape of the work.

keep:
  - current goal
  - constraints
  - decisions made
  - files touched
  - commands run
  - test results
  - open questions
  - next action

The harness is turning a messy conversation into a clear handoff brief.

Three Budgets

You probably saw this coming, but technicl limit is not the only budget we have. Context compression matters because the agent has those three budgets:

token budget:
  - how much text fits

latency budget:
  - how long each turn takes

attention budget:
  - how much irrelevant material the model has to ignore

Larger context windows help with the first budget. They do not remove the other two. A giant context window can still be slow, expensive, and full of distractions. This is why context management remains a design problem even when models support much longer prompts.

Deeper Reading

If you want the practical version, start with OpenAI's token counting guide. It shows how to estimate request size before sending it. Then look at Anthropic's context window docs, which explain how messages, tool results, thinking, output, and cached tokens count toward the window.

For the production direction, see OpenAI's compaction docs. They describe server-side and standalone compaction for long-running conversations. For the research reason that "it fits" is not the same as "the model will use it well," read great paper about Lost in the Middle.

What Bigger Coding Agents Do

Larger agent systems do not solve this by choosing one trick. They combine several layers: token awareness, manual compaction, automatic compaction, session storage, summaries, subagents, and project instructions.

Claude's platform docs describe the context window as the model's working memory for a request, and note that conversations eventually approach context limits. There is server-side compaction and context editing as strategies for long-running conversations and agentic workflows.

For Claude Code specifically, a 2026 architecture study, Dive into Claude Code, describes a system around the basic model-tool loop: permissions, append-oriented session storage, MCP, skills, hooks, subagents, and a multi-layer compaction pipeline.

Codex exposes similar ideas in its product surface. The Codex CLI slash command docs include /compact for summarizing the visible conversation to free tokens, /status for checking token usage and remaining context, and /resume for continuing a saved session. The Codex configuration reference includes settings for model context window size, automatic compaction thresholds, tool-output token limits, and compaction prompt overrides.

Codex also treats context management as an extensibility point. Its hooks docs include PreCompact and PostCompact events, which means teams can attach policy, logging, or review behavior around compaction. And its subagent docs frame parallel agents as one way to keep noisy exploration, tests, logs, and analysis out of the main thread, returning distilled summaries instead.

bigger systems tend to combine:
  - token usage reporting
  - manual compaction
  - automatic compaction thresholds
  - saved sessions
  - durable project instructions
  - subagent summaries
  - compaction hooks
  - memory separate from transcript

The design lesson for our tiny loop is not hide compression inside one prompt. Make it a visible harness feature. The user should be able to see when a run was compressed, what survived, and what state the next model call will receive.

So What Should Survive?

The easiest way to think about compression is to ask: "If a new agent took over right now, what would it need to know?"

handoff brief:
  - task
  - current status
  - important constraints
  - files changed
  - files only inspected
  - commands and results
  - decisions
  - blockers
  - next step

This is the same idea as leaving good notes for another developer. The next agent should not have to rediscover everything, but it also should not inherit every messy path you took to get there.

But What Should Disappear?

Compression is useful because it is allowed to drop things.

drop:
  - repeated logs
  - failed detours
  - stale observations
  - huge outputs with no remaining value
  - guesses that were later disproved
  - plans that are no longer active

The trick is not to erase accountability. The raw trace can still exist on disk for debugging. The compressed context is simply what the model needs now.

The Run State Is Not Memory

Memory and compression are easy to mix up. They are related, but they have different lifetimes.

memory.json:
  - durable facts and preferences
  - survives across many tasks

RUN.md:
  - current task handoff state
  - useful while this work is active

"User prefers small focused changes" is memory. "We edited loop_cli/agent.py and tests are still failing because of one permission assertion" is run state.

Context Reset

Once the harness has a good handoff brief, it can reset the conversation without resetting the work.

before:
  40 turns of transcript
  mixed with logs, mistakes, and retries

after:
  system instructions
  project context
  memory
  RUN.md handoff brief
  current user request

This is why compression is such a big step toward longer running agents. The loop no longer depends on one growing chat history. It can rebuild a focused prompt from state.

Deterministic First

We can create useful compression just from structured tool results.

from tool results:
  - which files were read
  - which files were written
  - which commands ran
  - which commands failed
  - latest error types
  - final answer or stop reason

This gives us .loop/RUN.md. Simple, inspectable and testable.

Then We Add A Summarizer

Later, a model can help. A summarizer model can turn raw events into a cleaner handoff, cluster related errors, and explain why a detail still matters.

summarizer model can propose:
  - concise status
  - important decisions
  - likely next step
  - stale details to drop
  - memories worth saving

But the harness should still own the output. It can require sections, limit length, keep source references, and reject summaries that remove safety information.

Active Compression

There is active research here. The Active Context Compression paper discusses long-horizon software tasks as a context bloat problem: growing histories increase cost and latency, while irrelevant past errors can distract the model. Their agent decides when to consolidate useful learnings into persistent knowledge and prune raw history.

Failure Modes

Compression can make an agent better. It can also hide the thing the next turn needed.

compression failures:
  - drops a real constraint
  - keeps an old fact as if it is current
  - turns a guess into a decision
  - hides a failing test
  - removes the reason for a policy decision
  - loses who needs to review the work

That is why compressed state should be visible. If the agent gets confused, the user should be able to open the handoff file and see what story the harness is telling the model.

Do Not Compress Away Policy

There is one or more special classes of detail that should not be treated like ordinary history, ie: governance. Permission rules, safety constraints, user boundaries, and review requirements should survive compaction.

A recent paper on Governance Decay studies exactly this failure mode: a long-running agent obeys a constraint while it is visible, but after compaction drops that constraint, the same agent can perform actions that should have stayed forbidden.

pin these outside lossy summaries:
  - permission rules
  - workspace boundaries
  - user approvals
  - safety constraints
  - files or actions requiring review

In our loop, that means policy should live in the harness, not only in the transcript. Compression can summarize progress, but it should not be the only thing preserving the rules of the run.

The Loop So Far

Here is a small deterministic compression layer. It reads tool results from the message history, extracts useful facts, and writes a handoff file to .loop/RUN.md.

import json
from datetime import datetime, timezone
from pathlib import Path


RUN_PATH = ".loop/RUN.md"
MAX_RECENT_RESULTS = 12
MAX_FIELD_CHARS = 500


def run_path(workspace: Path) -> Path:
    return workspace / RUN_PATH


def now_iso() -> str:
    return datetime.now(timezone.utc).isoformat()


def shorten(value: object, limit: int = MAX_FIELD_CHARS) -> str:
    text = str(value)
    if len(text) <= limit:
        return text
    return text[:limit].rstrip() + "..."


def collect_tool_results(messages: list[dict]) -> list[dict]:
    results = []
    for message in messages:
        if message.get("role") != "tool":
            continue
        try:
            result = json.loads(message.get("content", "{}"))
        except json.JSONDecodeError:
            continue
        results.append(result)
    return results


def unique(values: list[str]) -> list[str]:
    seen = set()
    ordered = []
    for value in values:
        if value and value not in seen:
            seen.add(value)
            ordered.append(value)
    return ordered


def compact_history(
    task: str,
    messages: list[dict],
    *,
    stop_reason: str = "unknown",
) -> str:
    results = collect_tool_results(messages)
    recent = results[-MAX_RECENT_RESULTS:]
    files = unique(
        [
            str(result.get("path") or result.get("file") or "")
            for result in results
        ]
    )
    tools = unique([str(result.get("tool") or "tool") for result in results])
    failures = [result for result in results if not result.get("ok", True)]

    lines = [
        "# Run Handoff",
        "",
        f"Updated: `{now_iso()}`",
        f"Stop reason: `{stop_reason}`",
        "",
        "## Goal",
        "",
        task,
        "",
        "## Tools Used",
        "",
    ]
    lines.extend(f"- `{tool}`" for tool in tools or ["None recorded."])

    lines.extend(["", "## Files Mentioned", ""])
    lines.extend(f"- `{path}`" for path in files or ["None recorded."])

    lines.extend(["", "## Recent Tool Results", ""])
    for result in recent:
        tool = result.get("tool", "tool")
        ok = "ok" if result.get("ok", True) else "failed"
        detail = result.get("error") or result.get("content") or result.get("output") or ""
        lines.append(f"- `{tool}` {ok}: {shorten(detail)}")

    lines.extend(["", "## Failures To Keep In Mind", ""])
    if failures:
        for result in failures[-5:]:
            tool = result.get("tool", "tool")
            error = result.get("error") or result.get("error_type") or "unknown"
            lines.append(f"- `{tool}` failed: {shorten(error)}")
    else:
        lines.append("- None recorded.")

    lines.extend(
        [
            "",
            "## Next Step",
            "",
            "- Continue from this handoff, but verify current files before editing.",
        ]
    )
    return "\n".join(lines).rstrip() + "\n"


def write_handoff(
    workspace: Path,
    task: str,
    messages: list[dict],
    *,
    stop_reason: str = "unknown",
) -> Path:
    path = run_path(workspace)
    path.parent.mkdir(parents=True, exist_ok=True)
    path.write_text(compact_history(task, messages, stop_reason=stop_reason))
    return path


def build_messages_from_handoff(task: str, workspace: Path) -> list[dict]:
    handoff = run_path(workspace)
    context = ""
    if handoff.exists():
        context = handoff.read_text()

    user_content = task
    if context:
        user_content = f"Run handoff:\n{context}\n\nUser task:\n{task}"

    return [
        {
            "role": "system",
            "content": "You are a careful coding agent. Verify stale context.",
        },
        {
            "role": "user",
            "content": user_content,
        },
    ]

This is still simple, but it changes the architecture. The agent now has a place to put the current state of a run, and the next loop can start from that state instead of a giant transcript.

Where we are

Context compression gives the loop a way to keep moving without carrying every step forever. It turns history into handoff state.

Next we will test the harness without relying on a real model. If compression, memory, permissions, and tools all belong to the harness, we should be able to test a lot of behavior with a fake model and deterministic traces.