Base Layer

base-layer.ai · Paper · Research · Dataset

To represent you, an AI agent has to understand you.

Base Layer is an open-source pipeline that turns your writing into a specification for AI agents. It derives plain-text claims about how you reason, communicate and decide from your chats, essays, notes and transcripts. An agent reads the specification before acting for you.

Every claim cites the facts it rests on. Every fact quotes your own words. You can check any claim and correct it.

If you are an agent, start with Using it with an agent and AGENTS.md.

How it works

Each step produces an output you can inspect and a check. The table follows the cited build in order. A turn is one message. The leaf stage is distill, with one model call per chunk of facts.

By default, extraction runs on Claude Haiku 5.5 at effort medium, the leaf stage on Claude Sonnet 5, and authoring on Claude Opus 5. Each row is implemented in this repository unless it says otherwise.

The cited build currently takes eight commands. consolidate writes files that you put in a system prompt by hand. Automatic serving to an agent is not yet implemented. See Using it with an agent.

baselayer run is the older, quicker pipeline. It takes one command and serves its result to Claude Code over MCP (Model Context Protocol). It makes far fewer checks.

Extraction does not require the person's own words unless BASELAYER_TURN_CONTRACT=1 is set. Claude Sonnet 4.6 writes the three layers in separate calls. Each call reads a size-capped selection of facts chosen by code, so it sees a small share of a large corpus. Citations are optional, and there is no verification or consolidation step. Afterwards, each claim is linked to the facts nearest it in meaning. These links show related facts; they do not establish which facts the writer used.

Related MCP server: anamnesis

What a specification contains

A specification has three layers.

• ANCHORS describe the certainties the person reasons from, which settle a question before the details arrive.

• CORE describes how they communicate and the context needed to read them.

• PREDICTIONS describe how they are likely to respond in a given kind of situation.

Every claim has an id and can state when it applies. This claim comes from the specification built from Zitkala-Sa's American Indian Stories (1921, public domain).

P15 TOLERANT TO PEOPLE HARSH TO SYSTEMS. Their severity is aimed at institutions and at settler power in the aggregate, not at individuals in front of them: in face-to-face disagreement, including religious disagreement, they extend respect, pity for suffering, and forgiveness over punishment. Predict the polemic in writing and the courtesy in the room.

Active when: They disagree with, or witness the hardship of, a particular person present

Evidence: [F-86fc472d] [F-cabf4c6c] [F-124a3cd5] ... (17 facts)

The Evidence line lists the facts used to write the claim. This example used an earlier version of the cited build, before facts quoted their source text. The full specification is in examples/zitkala_sa/.

A specification has these limits.

• Not a memory system. It guides how an agent interprets facts and sits above whatever memory system you use.

• Not a copy of the person. It records what someone who knows you might notice about how you approach a situation, for an agent to read.

• Not a biography or a summary. It describes how someone reasons.

• Not confirmed. Models produce the reading. The person it describes must confirm that it represents them.

• Not a portrait for the person. It is written for the agent working for them. The person reads it to correct it.

Quick start

To see a finished specification before spending anything, read examples/zitkala_sa/.

Requires Python 3.10 or later and git. Base Layer reads your model API key from the ANTHROPIC_API_KEY environment variable (get a key). Set it in your shell profile or a secrets manager rather than typing it into a command.

pip install git+https://github.com/agulaya24/BaseLayer.git

Run it on your own writing, or on writing by someone who has agreed to it. Choose the cited build for checked claims traceable to the person's quoted words. Choose baselayer run to try Base Layer in one command and load the result in Claude Code.

The cited build takes eight commands in a fresh data root, the directory holding the database. Run init, import, extract --turn-contract, distill and assemble for each layer, author-from-package, verify-spec and consolidate. It also needs a file listing the names the person goes by.

distill and author-from-package print a cost estimate. They require a spend ceiling at or above it and confirmation of the rate table's date before starting. The commands in order and the batch setup used for the measured builds are in docs/PIPELINE.md.

baselayer run lets you try Base Layer and use the result in Claude Code today. The second line requires Claude Code.

baselayer run chatgpt-export.zip     # or claude-export.json, a folder of journals, notes.md
claude mcp add --scope user --transport stdio base-layer -- baselayer-mcp

On the first run, it asks for consent and the writer's name and pronouns. --accept-data-processing answers the consent prompt. It estimates the extraction cost and asks before spending. The estimate covers extraction only and has run low, so treat it as a floor.

The result is three markdown files in data/identity_layers/ inside the data root. The root defaults to ~/.baselayer. Set MEMORY_SYSTEM_ROOT to an existing directory to change it. A source checkout differs; see docs/PIPELINE.md.

The second line registers the MCP server. MCP is the Model Context Protocol, which Claude Code uses to load outside tools and context. The server serves those files to new Claude Code sessions in any folder.

For anyone other than you, use a fresh data root so their facts do not mix with yours. Set export MEMORY_SYSTEM_ROOT=... and mkdir -p it first. Register that server inside one project only with claude mcp add --scope local -e MEMORY_SYSTEM_ROOT=<that root> --transport stdio base-layer -- baselayer-mcp.

Base Layer is pre-1.0 (version 0.5.0) and not on PyPI. The baselayer name there belongs to another project.

What it costs

The table shows API spend for one specification by corpus size. Derived figures use recorded token usage from cited builds made in September and October 2026, with extraction repriced for Claude Haiku 5.5. Projected figures scale those builds' per-fact costs at 17 to 23 facts per 1,000 words of the person's writing.

The cited build column uses the batch setup from every measured build. Extraction and the leaf stage went through the Message Batches API at half price. The leaf stage used one model call per 50 facts per layer. Claude Opus 5.5 wrote the layers.

• The cited build spends about five sixths on its leaf stage. The plain command sequence, at full rate with Claude Opus 5 writing, costs roughly 1.7 to 1.9 times as much.

• baselayer run spends nearly all its cost on extraction. Writing the three layers measured about $0.12 on three corpora of about 600 to several thousand facts in July 2026. That cost does not grow with the corpus because each layer reads a capped selection. Its extractor has not been priced on a recorded run. The column projects from the cited build's extractor, so treat it as an order of magnitude.

• Rates are per million input / output tokens, from Anthropic's pricing page as read on 19 October 2026.

  • Claude Haiku 5.5 $0.10 / $0.50 (prompts up to 100,000 tokens)

  • Claude Haiku 4.5 $1 / $5

  • Claude Sonnet 4.6 $3 / $15

  • Claude Sonnet 5 $2 / $10

  • Claude Opus 5 $5 / $25

  • Claude Opus 5.5 $4 / $20

  The Message Batches API costs half these rates.

See docs/PIPELINE.md for stage-by-stage costs and the basis of every figure.

Using it with an agent

• A cited build, by hand. Put served.txt in the system prompt. When a listed situation applies, add the claims it names from index.json. Automatic serving of a cited build is not yet in this repository.

• A baselayer run build, over MCP. In Claude Code, use the claude mcp add line above. In Claude Desktop, use its config file as shown in recipes/serve_specification_via_mcp.md. The server sends the three layers with every session. Its tools include recall_memories and search_facts to find facts, trace_claim to show what a claim rests on, and verify_claims to check it against the database and store the results there. Some clients truncate long MCP instructions. The full text is always available as the memory://specification resource.

• Any model, by hand. Put the three layer files in the system prompt. Fact search is unavailable this way.

How an agent should read it:

1. Apply a claim with an Active when line, or a prediction's When ... trigger, only when that situation holds. The always-on claims at the top of served.txt apply in every exchange. A claim with no condition provides context for reading the person. Do not treat it as a rule for every situation.

2. A claim marked CONTESTED preserves both sides of conflicting evidence. Do not resolve it for the person.

3. Facts call the subject "user" or "this person". That is the person the specification describes.

How to check it

Follow a cited claim's fact ids to the words they quote. Run verify-spec to re-check every citation and quote against the database at no cost.

What is checked by code, and what is a model's reading

The measured builds used the quote gate. The quote check confirms that a fact's evidence appears in a turn the source attributes to the person. It does not establish whether the fact interprets that text correctly.

baselayer run allows citations but does not require them. It links every claim to the facts nearest it in meaning. baselayer provenance --claim P3 (or trace_claim over MCP) prints each link and how it was made.

No check here establishes that the specification is faithful. A resolved citation confirms that the reference exists. It does not establish that the fact caused the claim. Another model's ability to distinguish a specification from others establishes distinguishability, without establishing fidelity. Only the person can confirm it.

Every term used in a fixed sense is defined in docs/DEFINITIONS.md.

How to correct or update it

Facts do not carry their own significance. An extractor can find that someone rewrote a plan three times. The person must judge whether that was diligence or avoidance and edit the text accordingly.

• Edit a cited build. Change the claim in both spec/<layer>.md and spec/<layer>.json. consolidate fails if they disagree. Run verify-spec and consolidate again.

• Edit a baselayer run build. Open the files in data/identity_layers/. Delete what is wrong, rewrite what is close, add what the writing never said, then restart the MCP server. Running author or run again rewrites these files from the facts. Earlier generated versions are kept in data/identity_layers/history/. Hand edits are not, so copy yours first.

• Remove a wrong fact. baselayer review walks through facts one at a time to hide or flag them. baselayer forget hides facts by id, by conversation, or all at once. Neither updates existing layers. For a baselayer run build, run baselayer author again, then baselayer verify --generate. For a cited build, rebuild from distill.

• Add more text. For a baselayer run build, run baselayer run on the new file. Extraction processes only new material. For a cited build, baselayer import the file, run baselayer extract --turn-contract, then rebuild from distill.

• Start over or remove everything. To start over, use a new, empty data root. baselayer forget --all is not a reset. To remove everything, delete the data root, the MCP session logs in ~/.baselayer/sessions/, and any trees/, packages/, spec/, verify/ and served/ directories you created for a cited build.

For agents making changes on someone's behalf, docs/CORRECTIONS.md explains how to carry a correction through facts, layers and served output in order. Base Layer Live is in development to keep a specification updated and served as corrections arrive.

Limits and what is not included yet

• Self-report. The corpus contains only what the person wrote or said. It includes no observations by others.

• No time axis. Claims are not dated. The specification does not record how the person changed.

• Little gain on the well known. It helps most where the model knows the person least.

• Anthropic models. Base Layer has been tested almost entirely on Anthropic's models. Every language-model call goes to Anthropic except extraction, which can use a local model through Ollama (baselayer extract --backend ollama). The local models we tried fell well short of the pipeline's needs. Qwen 2.5 14B failed at writing across 12 attempts and was kept for extraction only (D-030).

• The cited build is experimental. Its tests use fake model clients. Most of its design was measured on one book-length corpus of about 400 facts. The builds in the cost table used a development version of this code with the same stages.

• Not in this repository yet. The cited build lacks a single command and automatic serving to an agent. Also missing are the support filter that judges each fact against its quote before distillation; sorting stretch facts by how they go beyond their quotes; a separate model-judge check of each claim's scope; and fire-rate measurement for choosing always-on claims. The measured builds in the cost table used the support filter through scripts outside the package. ROADMAP.md lists each item.

Research

The Beyond Recall study tested specifications on 14 historical subjects with public-domain autobiographies. Panels of model judges scored the results. Prediction tests whether a specification represents the person accurately. It is not the goal.

• Gains were largest where the model knew the person least.

• The direction reproduced across response models. Absolute sizes depend on the judge panel.

A later test, separate from the paper, examined where the difference appears. It used recorded responses to the same decisions under nine specifications, some of them variants of one another. A deliberately simple classifier identified the right specification 51.6% of the time from the reasoning and 13.4% from the decision alone, against 11.1% chance. This test used no model judge. Differences of tone and length were not ruled out.

Specifications changed how decisions were argued in every situation tested. They changed the decision itself in some situations.

The study's evaluation design and pre-registered analysis plan, including later deviations, are documented in the Beyond Recall repository.

Methods and results are in the paper, the research page, and the study code and data. The paper used this version.

pip install git+https://github.com/agulaya24/BaseLayer.git@v0.2.0

Privacy

The database, facts and specification are stored only on your machine. There are no accounts and no sync.

Extraction, distillation, writing, the tension check and all model-judged checks send text to the Anthropic API. Anthropic's retention policy applies. Extraction can instead use a local model through Ollama (baselayer extract --backend ollama).

The embedding model is downloaded from Hugging Face on first use. Base Layer sends no usage data and disables telemetry in its vector store, ChromaDB.

The specification is meant to be readable by the person it describes and under their control. They can read it, correct it or delete it.

Documentation

Contributions on evaluation, source adapters, other interpretive-layer implementations and local models are welcome. See CONTRIBUTING.md.

@software{baselayer2026,
  title     = {Base Layer: An Open-Source Reference Pipeline for the Interpretive Layer Above Memory},
  author    = {Gulaya, Aarik},
  year      = {2026},
  url       = {https://github.com/agulaya24/BaseLayer},
  license   = {Apache-2.0}
}

Apache 2.0. See LICENSE. The Beyond Recall paper is CC BY 4.0.