Skip to main content
Glama

ragtrail

Reuse the retrieval paths your agent already solved.

ragtrail is a trajectory cache for agentic RAG. It records the path an agent took to a cited answer (the searches it ran, the nodes it opened, the ones it cited) and hands that path back when a similar question arrives, so the agent skips the orientation phase. Paths are invalidated by content hash when the underlying nodes change.

It caches the path, never the answer.

CI PyPI

Status: alpha (0.1.0, on PyPI). The core, the MCP server and the evaluation harness work and are tested; the cloud stores in PLAN.md (M6–M8) are not built. The public API may change between 0.x releases. Write-up: Building ragtrail.

Install

pip install ragtrail                  # core: stdlib + sqlite3 + pydantic
pip install 'ragtrail[mcp,embed]'     # MCP server, embedding matcher

Python 3.11 or newer.

Related MCP server: mcp-rag-platform

Use it from Python

Wrap one agent turn. On entry you get the recalled path (if any); on exit an accepted turn is stored.

from ragtrail import GenericResolver, SQLiteStore, turn
from ragtrail.resolver import text_hash

resolver = GenericResolver(
    hash_fn=lambda node_id: text_hash(docs[node_id]) if node_id in docs else None,  # None = deleted
    read_fn=lambda node_id, max_tokens: docs[node_id],
)

with SQLiteStore("trails.db") as store:
    with turn(store, resolver, scope="my-team", query=question) as t:
        if t.recall.mode != "miss":
            print(t.recall.rendered)  # nodes to start from, or their current text
        # ... run your agent, record what it did and what it cited ...
        t.step("search", {"query": question}, found_ids)
        t.cite(cited_ids)
        t.accept("verified")  # no accept, no trail

A runnable version over plain chunk ids is in examples/chunk_rag.py; the subforest one is examples/subforest_rag.py.

How it works

  1. Match. The question is normalized and matched against stored questions with MinHash/LSH (stdlib only), or by embeddings if you enable them. Lookup is a handful of key reads, so any key-value store can back it.

  2. Validate. Every cited node is re-hashed against the live index. All unchanged: prefetch (current node text, within a token budget). Some changed: hint with only the still-valid nodes. All changed: miss, and the trail is marked stale.

  3. Admit. A trail is written only with a positive signal (verified citations, user feedback, or an explicit accept). Scopes (tenant, document, permission group) never mix.

Design decisions, with the numbers that drove them, are in docs/decisions.md.

Limitations

  • Savings come only from repeated or reworded questions; the evaluation uses model-written repeats, so real traffic will save less.

  • Hash validation catches changed nodes, not new documents that would now be better evidence; only the TTL covers that. The same holds for exported fetch scripts.

  • Admission quality decides safety. A path from a wrong answer sends the agent back to the same mistake.

  • Evaluated with one agent model (Haiku) and one judge (Sonnet, checked on 40 labels, not human-labelled).

  • The evaluation harness needs subforest and the claude CLI; the library itself does not depend on either.

MCP server

pip install 'ragtrail[mcp]'
ragtrail serve --db trails.db --scope my-team --subforest corpus.db

The server exposes two tools: recall(query, budget_tokens?) at the start of a question and record(query, cited, outcome?, steps?) after a grounded answer. Each server is bound to one --scope.

To share trails across a team, install pip install 'ragtrail[postgres]' and pass a database URL instead of a file: --db postgresql://user:pass@host/ragtrail. Everyone using the same URL and --scope shares the trails; other scopes stay apart. PostgresStore is also usable directly from Python.

Add --embed (needs pip install 'ragtrail[embed]') to also match paraphrases by embeddings. It uses a local CPU model, BAAI/bge-small-en-v1.5 by default, so there is no API cost.

How it checks nodes:

  • --subforest corpus.db: a subforest index. Node hashes are checked on every recall and texts are prefetched.

  • --resolver mypkg.index:make_resolver: any NodeResolver for your own index.

  • Neither: unverified mode. Recall only hints node ids and never prefetches.

Claude Code:

claude mcp add ragtrail -- ragtrail serve --db ~/.ragtrail/trails.db --scope my-team --subforest corpus.db

Other MCP clients (Claude Desktop, Cursor, …) use the same command in their JSON config:

{
  "mcpServers": {
    "ragtrail": {
      "command": "ragtrail",
      "args": ["serve", "--db", "/path/to/trails.db", "--scope", "my-team", "--subforest", "/path/to/corpus.db"]
    }
  }
}

Run it next to your retrieval tools (for example subforest's own MCP server). ragtrail purge --db trails.db deletes expired trails.

When a user rejects an answer that a recalled path led to, the agent calls the reject tool with the same question: that path is demoted and the question gets a fresh search. Start the server with --log calls.jsonl and run ragtrail stats --db trails.db --log calls.jsonl --scope my-team --days 30 for recalls by mode, estimated savings and the trails with the most rejected answers. The token figure is an estimate (skipped search calls times --tokens-per-call, an assumed 500), not a measurement.

Export a stable trail as a fetch script

When the same question keeps coming back and its trail stays valid, you can skip the agent's retrieval turn:

ragtrail export --query "What are the rate limits?" --db trails.db --scope support \
  --resolver mypkg.index:make_resolver -o rate_limits.py    # or pass a trail id instead of --query
python rate_limits.py

The script holds the trail's node ids and the hashes they were cited with, never an answer. It reads the nodes through the same resolver (--resolver or --subforest) and prints {"trail", "documents": [{node_id, label, content_hash, text}]} as JSON. If any node changed or vanished, or the trail's TTL ran out, it prints nothing on stdout and exits with 3: fall back to the agent. A model still writes the answer from the documents. Only trails that are not stale, have no rejected answer and whose nodes all still match can be exported; the same trail always gives the same script.

TypeScript and astorlm

clients/typescript holds a thin client for the MCP server (recall, record, reject) and a ready hook for astorlm. The hook spawns ragtrail serve, recalls before the first turn of each question and adds the hint to the user message. It records only when the host app calls accept(). It is not on npm yet: from a checkout, build it with cd clients/typescript && npm ci && npm run build, then install it in your app with npm install ./clients/typescript.

import { createAgent } from 'astorlm'
import { ragtrailHook } from 'ragtrail'

const trail = ragtrailHook({ scope: 'docs', serverArgs: ['--subforest', 'corpus.db'] })
const agent = await createAgent({ provider, tools, hooks: trail })

await agent.run('How do I reset my password?')
if (userAccepted) await trail.accept()   // records the cited nodes and tool calls
else await trail.reject()                // demotes the recalled path

Cited nodes are the ids that tool outputs returned as [id] and the answer mentions. Pass nodesFrom for another id format, accept({ cited }) to set them yourself, or autoAccept to record without a call. The answer text is never sent.

Evaluation

Summary and details: reports/README.md. Haiku agent and Sonnet judge through claude -p.

Repeat traffic (the same question asked again in other words), all questions, cache off vs on.

The realistic run uses trails the agent recorded itself on WixQA (n = 200):

  • Tool calls: 3.4 → 2.1.

  • Input tokens: −33%.

  • Judged correct: 69.8% → 74.2% (+4.5, 95% CI −0.8 to +9.8): no accuracy loss.

With oracle trails, which cite the gold evidence, the per-hit gains are larger. Treat them as an upper bound:

Dataset

n

Tool calls

Input tokens

Judged correct

WixQA help center

200

3.4 → 2.2

−28%

69.8% → 86.2%

QASPER papers

400

4.1 → 1.5

−57%

60.8% → 71.6%

  • Only record trails from answers you trust. When the recorded answer was wrong, warm accuracy fell from 50% to 21% on those questions; when it was right, it rose from 79% to 87%.

  • Put fixed context (document, product, customer) in scope. If a paper's title is part of the matched text, any question about that paper looks similar and gets the wrong sections.

  • Savings come only from repeated questions. Every test question here is a model-written repeat, so the cache saves less on real traffic, where fewer questions repeat. Related work: docs/related-work.

Reproduce (needs subforest installed alongside and the claude CLI; add --dataset qasper for QASPER):

ragtrail eval fetch
ragtrail eval paraphrase
for s in disjoint repeat; do
  ragtrail eval offline --scenario $s && ragtrail eval offline --scenario $s --embed
done
ragtrail eval agent --scenario repeat --n 30 [--embed]    # likewise disjoint with --n 20
ragtrail eval judge --scenario repeat [--embed]
ragtrail eval report

Demos

uv run python examples/demos.py runs six small offline cases, no API needed: paraphrase reuse, an edited source (ragtrail drops it; a copy cache serves the old text), a deleted source, the fixed-context false match and its scope fix, tenant isolation, and admission gating. Pass a name (freshness, fixed-context, ...) to run one.

Development

uv sync --all-extras
uv run ruff check . && uv run ruff format --check . && uv run mypy src && uv run pytest

The Postgres store tests run when RAGTRAIL_PG_DSN points at a database (CI provides one) and are skipped otherwise. The TypeScript client: cd clients/typescript && npm ci && npm run typecheck && npm test.

See CONTRIBUTING.md.

License

MIT

Available Tools

3 tools
recallA

Look up the retrieval path of a similar question answered before. Call it first, with the user's question copied verbatim (not rephrased or reduced to keywords). Returns mode=prefetch (node texts included), hint (node ids to read first) or miss (search).

Inputs: query (required), budget_tokens.

recordB

Save the path that answered query: the node ids the answer cites, and optionally the tool calls made (each {"tool": name, "args": {...}, "returned": [node ids]}). Never include the answer text.

Inputs: cited (required), query (required), steps, outcome.

rejectA

Report that the user rejected the answer to query. Pass the same question that was given to recall: the path recalled for it is demoted and the question gets a fresh search next time.

Inputs: query (required).

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

Loading tool schema changelog…

    TDQS

    A4/5.0

    Scored across 3 tools

    Disambiguation5/5

    The three tools map cleanly to distinct stages of the retrieval-path lifecycle: recall reads paths, record saves them, and reject demotes them. Their descriptions make the boundaries explicit, including what arguments each expects and when to call them. There is no practical overlap that would cause misselection.

    Naming Consistency5/5

    All tools use a single, lowercase verb (recall, record, reject) following the same concise convention. The naming is predictable and immediately readable. No mixed styles or inconsistent verb patterns are present.

    Tool Count5/5

    Three tools is a well-scoped surface for this narrow retrieval-memory loop. Each tool corresponds to a necessary operation—lookup, save, and feedback—without redundant or filler tools. The count matches the focused purpose.

    Completeness4/5

    The core lifecycle is covered: recall retrieves a prior path, record persists a successful one, and reject handles negative feedback by demoting a path. Minor gaps exist, such as no explicit update/edit operation for an existing record or a way to list stored paths, but these are not essential to the stated workflow.

    Maintenance

    ActivityMaintained
    ResponsivenessNo recent community issues

    Related MCP Connectors

    Related MCP Servers

    • F
      license
      Not graded
      quality
      B
      maintenance
      Enables retrieval-augmented question answering with source-grounded citations, document retrieval, and idempotent ingestion through MCP tools.
      -
    • A
      license
      A
      quality
      C
      maintenance
      Enables AI agents to persist memory across sessions through MCP, supporting semantic retrieval, source-backed context, knowledge graph relationships, supersession lineage, and topic digests.
      43
      1
      MIT
    • F
      license
      Not graded
      quality
      C
      maintenance
      Enables MCP-compatible clients to retrieve relevant document chunks and fact-check generated answers against context, classifying them as fully supported, partially supported, or unsupported.
      -