The Library of Context

Virtual memory for AI context: durable outside the model, bounded inside it.

An AI model has a finite native context window. Long conversations eventually expand until old information is truncated or compacted. For calls routed through its context governor, the Library stores each recorded event in SQLite and assembles a bounded model request from protected events, recent events, and retrieved records.

Think of the model's context window as a reading desk. The Library can hold far more books than the desk, but the librarian lays out only the books needed for the current task. Changing the task replaces the desk; it does not pile more books on top.

Why this is different from ordinary compaction

Conventional compaction turns a growing transcript into a smaller, lossy continuation and may leave the original details outside the active workflow. The Library uses reversible semantic paging:

traditional:  growing transcript -> compacted transcript -> continue

Library:      durable event log -> bounded recent/protected context
                       |                    + relevant retrieved books
                       +-----------> fresh model request on every turn

Original events are inspectable and recoverable. Summaries may become navigation aids, but they do not need to be the only surviving copy.

The related-work landscape compares this design with model long-context methods, retrieval, prompt compression, provider compaction, agent memory, checkpointing, and inference-runtime paging. Here, “compaction” means a smaller, potentially lossy continuation representation whose originals are not independently addressable unless another layer retains them.

Related MCP server: local-memory-mcp

Capabilities

• A context governor with prepare -> model call -> commit lifecycle operations.

• Durable SQLite thread events and a transactional indexing outbox.

• A token-targeted, event-bounded recent ring for immediate read-your-own-context behavior; an oversized event is truncated only in the model envelope, not on disk.

• A bounded work ring with a durable SQLite outbox for overflow and restart recovery.

• Protected context for instructions, decisions, active plans, and unresolved state.

• Recorded, embedded, and indexed watermarks with queue-health status.

• Fresh, bounded prompt envelopes that replace transcript growth.

• Hybrid vector, SQLite FTS5, importance, and recency retrieval.

• Byte-bounded process RAM and optional disposable local Redis hot tiers.

• Reading-desk swap reports: swapped_in, swapped_out, and retained.

• Python, local HTTP, CLI, and STDIO MCP integration surfaces.

• Dependency-free hashing embeddings and an optional local Ollama adapter.

The governor is automatic when your agent or model gateway routes every turn through it. An MCP-only integration is cooperative: the host can use shelving and reading-desk tools, but it cannot rewrite the model request that already invoked a tool or replace an undocumented internal compaction hook.

Architecture at a glance

flowchart LR
    U[User or tool event] --> A[Durable SQLite append]
    A --> E[(Thread event log)]
    A --> O[(Transactional outbox)]
    A --> R[Recent context ring]
    O --> W[Bounded work ring]
    W --> I[Embed and index workers]
    I --> S[(SQLite library and FTS)]
    I --> C[RAM and optional Redis cache]
    R --> G[Context governor]
    S --> G
    P[Protected context] --> G
    G --> D[Bounded reading desk]
    D --> M[Native model context]
    M --> X[Assistant response]
    X --> A

Quick start

The default configuration requires only Python 3.11 or newer. Redis is optional.

On Windows PowerShell:

git clone https://github.com/hwillGIT/library-of-context.git
cd library-of-context
py -3.11 -m venv .venv
.\.venv\Scripts\python.exe -m pip install -e .
.\.venv\Scripts\python.exe -m library_of_context quickstart

On macOS or Linux:

git clone https://github.com/hwillGIT/library-of-context.git
cd library-of-context
python3 -m venv .venv
.venv/bin/python -m pip install -e .
.venv/bin/python -m library_of_context quickstart

The quickstart exercises protection, prompt assembly, event recording, indexing, and cleanup with a temporary database. It uses no Redis, Docker, cloud service, or model API. Continue with the installation guide.

Add it to an agent you already run

See Add the Library to your agent for Codex, Python, and HTTP configuration examples. After MCP server configuration, restart the client or begin a separate session; configuration does not affect a chat already in progress.

Run an automatically governed Python text agent

from library_of_context import GovernedTextAgent, LibraryOfContext


def call_my_model(messages: list[dict[str, str]]) -> str:
    return my_model_client.generate(messages=messages)


with LibraryOfContext("data/library.sqlite", redis_url="") as library:
    with library.open_context_governor(
        "agent-thread-42",
        token_budget=12_000,
        recent_token_budget=4_000,
        protected_token_budget=2_000,
    ) as context:
        context.protect(
            "Production changes require a canary wave.",
            label="deployment-policy",
        )

        agent = GovernedTextAgent(
            context,
            call_my_model,
            system_prompt="Work carefully and cite retrieved project evidence.",
        )
        response = agent.turn(
            "Diagnose the deployment failure.",
            turn_id="request-0001",
        )
        context.flush(timeout=5)
        print(context.status()["watermarks"])

The callback must send exactly the supplied messages; it must not append another transcript or continue a provider-managed conversation. The built-in adapter is text only. Structured tool calls, streams, attachments, and multimodal content need a custom serialization adapter.

See Context Governor for the complete protocol.

MCP integration

For a normal MCP agent, use the project-isolated template and ready-to-merge agent instructions in integrations/README.md. This is cooperative memory; it does not control the host's native transcript.

The raw local STDIO server can be inspected with:

python -m library_of_context.mcp_server --no-redis

Custom MCP gateways that own the model-call boundary can use:

The Library exposes shelving, retrieval, reading-desk, stateless-session, and governor tools. Enable gateway-only tools only in a host that sends the returned messages as the complete next model request.

Local HTTP API

python -m library_of_context --no-redis serve

The governor endpoints are:

The /books, /library/ingest, /catalog/query, and /desk/* routes expose the lower-level library. The server binds to loopback and has no authentication. Do not expose it directly to another machine.

Storage hierarchy

1. Recent ring: per-thread ordered events, bounded by event count and an estimated- token target. One oversized event may remain resident so fresh context is visible; prompt assembly truncates its model-visible view to the hard envelope budget. This is not an LRU; conversation order matters.

2. Process RAM: byte-bounded LRU for hot books and retrieval results.

3. Local Redis: optional shared cache for hot books, queries, desks, TTLs, and invalidation generations.

4. SQLite: authoritative events, outbox, text, metadata, FTS, and vector storage.

Redis is disposable. The default local Redis configuration is not a durable message broker and should not be used as the team event stream.

Free local Redis on Windows

Docker and a cloud account are not required. The included PowerShell script installs a Redis service inside Ubuntu WSL. It requires WSL 2, an Ubuntu distribution, and systemd:

powershell -ExecutionPolicy Bypass -File .\scripts\install-local-redis.ps1
.\.venv\Scripts\python.exe -m library_of_context --db data/redis-check.sqlite doctor

doctor opens the configured SQLite database while checking the storage tiers. The example above creates data/redis-check.sqlite.

Use --no-redis everywhere if SQLite plus process RAM is sufficient.

Performance limits

Prompt assembly is bounded, and recorded events use a transactional outbox. FTS returns a bounded candidate set, while vector retrieval exact-scores every live record in a namespace. Large-catalog scale claims therefore require measured evidence and, when the exact path crosses a declared limit, a bounded vector-search adapter.

Performance and Scaling defines measurements, SLO criteria, and benchmark questions. Why These Improvements? compares simpler alternatives, adoption triggers, and evidence gates, while the Roadmap sequences conditional work.

Documentation

Help shape the design

Open design questions include:

• Which context should be protected automatically, and who may release it?

• How should retrieval quality be measured for agent threads rather than document QA?

• What is the right local ANN adapter for 100,000 to 1,000,000 chunks?

• How should branches inherit, supersede, and merge context?

• Which knowledge is safe and useful to promote from a private thread to a team catalog?

• Should the shared event plane use Redis Streams, NATS JetStream, or another broker?

• How should ACL revocation invalidate local caches without putting the cloud in the prompt critical path?

• What token-pressure policy feels predictable to users across different model tokenizers?

The longer list is in ROADMAP.md. Questions, benchmark results, design notes, adapters, failure tests, and critiques are welcome.

Contributing

Read CONTRIBUTING.md, open a research question or design proposal, and keep pull requests focused. The project particularly welcomes reproducible retrieval benchmarks, ANN adapters, tokenizer integrations, privacy reviews, queue and crash tests, and agent-framework gateways.

License

MIT © Library of Context contributors.