Yash Datta

Long sessions without compaction

How codeoid keeps month-long agent sessions inside a small context window: rotate the window instead of summarizing it, page verbatim history back through MCP tools, and fork the same conversation onto a different backend.

AI agents 11 min read

TL;DR. codeoid never summarizes a long session. Every turn is stored verbatim in local SQLite; when the context window fills, the session rotates to a fresh backing window carrying only a small task anchor, and the model pages exact history back on demand through four MCP tools. The same canonical history lets one call fork a session onto a different backend. A 29-day, 93-turn session of my own currently sits at 31% context.

A useful coding session outgrows its context window. Claude Code’s answer is compaction: fold the transcript into a summary and continue from that. It works, but it is lossy at exactly the wrong moment. The summary is written once, by a model that cannot know which detail you will need three hours later, and whatever it drops is gone. Anyone who has watched an agent re-read a file it read forty turns ago, or re-derive a decision it already made, has paid this tax.

codeoid, my open-source agent harness, takes a different route: it never summarizes a session. The stance behind everything below is that the context window is a cache and the session history is a database. A cache can be evicted aggressively as long as the database is intact and cheap to page from. So codeoid keeps every turn verbatim in local storage, tells the model what that storage holds, swaps the context window out from under the session when it fills, and lets the model page back exactly the bytes it needs. This post walks through the four mechanisms that implement the stance: the episode store, the workspace index, rotation, and cross-backend fork, plus the compression layer the store makes safe.

Everything is an episode

The substrate is codeoid’s memory engine. Every turn of every session is indexed as episodes: user intents, assistant reasoning, and each tool call with its full result. Episodes are stored verbatim in SQLite on your machine (via bun:sqlite) and indexed two ways: dense embeddings from a local bge-small-en-v1.5 model (384 dimensions, ~50 MB, pure WASM through transformers.js) and SQLite FTS5 for BM25 keyword search. Retrieval blends four signals: vector similarity, BM25, recency, and path overlap. Nothing leaves the machine; there is no cloud call anywhere in the path.

The model reaches this store through four MCP tools, served in-process by the daemon (no socket, no subprocess):

  • recall(query) — semantic search over everything that happened in the workspace, across all past and current sessions. Returns episodes verbatim.
  • recall_file(path) — the most recent read of a specific file, so unchanged files are not re-read.
  • timeline(offset?, limit?) — the full history in order, paginated, each line carrying an episode_id.
  • get_episode(episode_id) — the exact bytes of one past turn or tool result.

The pair of timeline and get_episode matters more than it looks. Semantic search alone can miss; a deterministic walk of the history cannot. Between the two access paths, any past turn is reachable, which is the property the rest of the design leans on: the context window can be trimmed hard because nothing exists only in context.

The index that advertises what memory holds

Tools the model does not think to call are shelf inventory. So codeoid injects a compact workspace index into the system prompt each turn (Layer C in the codebase’s terminology, on by default when memory is enabled). It contains a fingerprint line (294 episodes across 8 sessions · last activity 8m ago), the ten hottest files by touch count with a recall_file nudge on each, optionally k-means topic clusters over the episode embeddings (k ≤ 8), one-liners for recent sessions, and a compact reference for the four tools.

The index is rebuilt on a hybrid trigger — five new episodes, or sixty seconds elapsed with at least one — and debounced to one regeneration per fifteen seconds, so the system prompt stays stable enough for prompt caching to hold. The effect is that the model already knows what it can ask memory for before it needs to ask.

Rotation

Rotation is what replaces compaction. When a session’s context fills, codeoid rotates it: the daemon tears down the backing Claude Code session and mints a fresh one, while the codeoid session keeps its id, its scrollback, and its memory. From the outside nothing changes; the status bar increments a rotation counter and the transcript gets an info line. You can trigger it yourself with /rotate. With auto-rotation enabled (CODEOID_AUTO_ROTATE=1), it fires on its own at 90% context occupancy, and a hard safety net fires at 97% even when auto-rotation is off, before the backing harness would start compacting. A minimum-turns guard (five by default) stops a young session from rotating before there is anything worth anchoring.

The first message after a rotation carries a seed block instead of the old transcript:

<rotation_context>
Codeoid just rotated this session's backing Claude Code context to stay
below the compaction ceiling. This is a CONTINUATION, not a new session.

Workspace: /home/you/project. Rotation #2 of this session ("oracle").

Prior turns are preserved verbatim in codeoid memory. Retrieve on demand:
  - recall(query)               — semantic search across all prior episodes
  - recall_file(path)           — most recent prior Read of a specific file
  - timeline(offset?, limit?)   — walk activity in order; each line has an episode_id
  - get_episode(episode_id)     — fetch one past turn or tool result verbatim

Most recent user turn before the rotation:
---
<the last user message, verbatim, capped at 2,000 characters>
---
</rotation_context>

That is the whole handoff: a continuation notice, the workspace, the rotation count, the four tools, and the last user turn as a task anchor. The seed strategy is called task-anchor, and it is the only one implemented, because with a verbatim store underneath it is loss-free: the model re-orients by pulling back the three episodes it actually needs, at original fidelity, and leaves the other few hundred in the database. A compaction summary makes that selection once, at compaction time, and you live with whatever it kept.

The thresholds took tuning, and the failure is worth recording. The soft threshold originally sat at 80% and the hard net at 90%. Both fired constantly on healthy sessions, because the SDK’s per-turn usage figure is the sum of all internal API calls: primary, subagents, and retries. A turn that fans out to subagents can report 800k tokens while no single call exceeds 300k. The fix was twofold: measure occupancy from the primary conversation’s last-turn input tokens only (subagents exist precisely to keep the primary context clean, so their usage should never trigger a rotation), and raise the thresholds to 90/97. The general lesson is that an eviction signal has to measure what the next turn will actually see, not what the last turn happened to spend.

Compression the store makes safe

A related layer (Layer B, opt-in via CODEOID_COMPRESS=1) compresses shell output before the model sees it. A pre-tool-use hook reroutes Bash invocations through a daemon-local wrapper that runs the real command and applies declarative rules: long unchanged context collapses out of git diff, huge untracked lists elide from git status, passing tests drop from test-runner output, ls/find/grep output summarizes by directory and extension. On shell-heavy turns this cuts 60–90% of the bytes. Stderr is never compressed, because error fidelity is the one thing you cannot afford to lose.

On its own this would be another lossy trade. It is safe for the same reason rotation is: the raw output lands in the verbatim store before compression, so if the compressed version dropped something the model needed, one recall retrieves the original bytes. The store is what turns lossy-looking optimizations into reversible ones.

Fork: the same history under a different engine

codeoid owns the conversation in its own canonical turn format and translates it into each backend’s native messages and tool calls, so the same history can be replayed into Claude, Codex, Gemini, and the rest. That buys one more long-session operation: fork. A single session.fork call branches a session into a new, independent one seeded with a copy of the full conversation, optionally onto a different backend. The code comment states the intent plainly: “branch this claude conversation and continue it on codex” is one call. The parent is untouched, and the fork carries a lineage marker showing which parent it branched from and at which turn.

Rotation and fork are deliberately different operations. Rotation keeps the session and discards the backing window, betting on recall to page detail back. Fork keeps everything: the new session starts with the full structured transcript, because its job is to diverge, not to slim down.

What it costs

The design has real costs, and they are visible ones. The first turn after a rotation may spend a tool call or two reloading state; you can watch it happen in the transcript. Compaction’s cost is invisible until you hit the detail the summary dropped, which is the trade codeoid refuses. Recall is workspace-scoped inside a session (cross-workspace resolution exists, but that is a separate feature with its own eval). Rotation targets the Claude Code backing session today; the seed text literally names it, and other backends manage context differently. Auto-rotation ships off by default while the thresholds bake in the field. And if memory is disabled entirely, rotation degrades to a generic continuation prompt: the daemon keeps working, only the quality of the handoff drops.

Try it

codeoid is MIT-licensed: https://github.com/saucam/codeoid (npm package codeoid). The pieces above map to /rotate in the TUI, CODEOID_AUTO_ROTATE=1, CODEOID_COMPRESS=1, and memory (with the workspace index) on by default. The fastest way to feel the design is to let a real session run long, type /rotate, and then ask about something from before the rotation: the recall call that answers it is the whole thesis in one transcript line.

Further reading. The codeoid release post covers the memory engine and identity layer this builds on; docs/FEATURES.md documents the full context-reduction stack and its toggles; docs/session-resolution.md covers the cross-workspace session resolution mentioned above, with its eval numbers.