SourceTrace MCP
by strictstack
README.md
# SourceTrace MCP
<!-- mcp-name: io.github.strictstack/sourcetrace-mcp -->
MCP server that retrieves public legal and standards sources and returns canonical SHA-256 digests (content-equality proofs), per-document snapshot version numbers and structural diffs; validated with synthetic fixtures; not legal advice.
Status: first public release 0.1.0; not yet published to PyPI or the MCP Registry at the time of writing.
- Vendor: StrictStack, <https://strictstack.tech>, support@strictstack.tech
- Code: <https://github.com/strictstack/sourcetrace-mcp>
- Issues: <https://github.com/strictstack/sourcetrace-mcp/issues>
- Documentation: <https://strictstackdocs.tech/sourcetrace-mcp/>
- PyPI package: `sourcetrace-mcp` 0.1.0; MCP Registry name: `io.github.strictstack/sourcetrace-mcp`
The server collects snapshots from explicitly authorised sources, searches the retained excerpts, compares historical snapshots and verifies deterministic hashes. It exposes five MCP tools: `fetch_public_text`, `get_snapshot`, `search_snapshot`, `diff_snapshots` and `verify_digest`. Each new snapshot of the same document receives an incrementing `version` number; in the free tier that history is held in memory for a limited period (see [Free and Pro persistence](#free-and-pro-persistence)). There is no dedicated tool that lists all versions of a document.
Requires Python 3.11 or later. Running the server and building the packages use only the standard library. The SQLite library of the installed Python must include FTS5 to allow searching the excerpts.
## Supported sources
The access policy is `fontes.json` (schema 1, reviewed 2026-11-13 11:10 UTC). Only HTTPS, the three exact domain names and the document path prefixes below are accepted. A domain permission does not confirm the rights to an individual document.
| Source | Domain (exact) | Allowed path prefixes | Storage policy |
| --- | --- | --- | --- |
| EUR-Lex | `eur-lex.europa.eu` | `/legal-content/`, `/eli/` | `excerpts`: metadata and an excerpt of at most 600 characters |
| Diário da República | `diariodarepublica.pt` | `/dr/detalhe/`, `/dr/legislacao-consolidada/` | `excerpts`: metadata and an excerpt of at most 600 characters |
| RFC Editor | `www.rfc-editor.org` | `/rfc/` | `excerpts`: metadata and an excerpt of at most 600 characters |
None of the three entries currently allows storing or returning the full text. For excerpt sources the retained text is at most the excerpt limit and at most half of the canonical text, so even a short document is never retained in full. Subdomains, images, logos, third-party annexes and redirects to other domains are not authorised by this list. The 600-character limit is a technical restriction of the product, not a legal presumption that any excerpt is permitted. The reuse basis, attribution requirements and restrictions for each source are documented in [docs/SOURCE_REUSE.md](docs/SOURCE_REUSE.md) and `fontes.json`.
## What the digest proves
Each snapshot carries two SHA-256 digests: `raw_sha256` over the retrieved bytes and `canonical_sha256` over the canonical text (Unicode NFC, whitespace collapsed; for HTML and XHTML, the extracted text without markup). Sections, articles and form-feed pages carry their own SHA-256 digests. `verify_digest` compares a supplied digest or supplied content against a stored snapshot.
A matching digest proves only that the compared bytes or canonical text are equal. It does not prove authorship, the authenticity or truthfulness of the source, legal validity or currency, or a certified date. The collection time is recorded as reported by the server and is marked `collection_time_certified: false`. A deterministic hash allows checking correspondence with the data supplied; it does not constitute external verification.
This is not legal advice.
## Validation scope
All tests use synthetic fixtures identified as test documents, with the separate policy `fixtures/policy.json`, and run without network access. Availability of the real sources, extraction quality on real pages and latency were not measured. Integration with the real entitlement gateway was not measured.
## Install
After publication to PyPI (not yet done at the time of writing):
```sh
pip install sourcetrace-mcp
uvx sourcetrace-mcp serve --transport stdio
```
From a copy of this repository, without installation:
```sh
PYTHONPATH=src python3 -m sourcetrace serve
```
## Running locally
From a copy of this repository:
```sh
PYTHONPATH=src python3 -m sourcetrace serve
PYTHONPATH=src python3 -m sourcetrace serve --help
```
After a local installation of the wheel, use `python3 -m sourcetrace serve` or the `sourcetrace-mcp serve` command. The default transport uses newline-delimited JSON-RPC messages over stdio. Diagnostic messages are written to stderr.
The source access policy can be set with `--sources path/to/fontes.json`. The bundled `fontes.json` defines the authorised sources and the reuse limits; it contains a policy, not content collected from the sources. The synthetic documents are in `fixtures/`, with the separate policy `fixtures/policy.json` for tests without network access. The program does not search the disk for other operator-supplied data. Authorisation and network access are subject to the explicit entitlement and egress controls. For demonstrations, configure only the local test gateway; a working demonstration does not prove production entitlements.
## MCP protocol
The adapter supports the per-request metadata of MCP `2026-07-28` and the initialisation handshake of version `2025-11-25`. Current clients use `server/discover`, `tools/list` and `tools/call`, including protocolVersion and clientCapabilities in the request `_meta` field. Clients of the previous version first send `initialize` and `notifications/initialized`. Use `tools/list` to obtain the full argument schemas.
## Free and Pro persistence
The free tier keeps snapshots in memory for a limited period. Pro persistence requires validated entitlements and an explicit `--pro-data-dir` or `SOURCETRACE_PRO_DATA_DIR`; without a configured directory, storage remains ephemeral. The service does not choose a persistent directory for Pro snapshots on its own.
## Quota database
Daily accounting is stored separately in `./.sourcetrace-state/quota.sqlite3`. A fixed path can be set with `--quota-db` or `SOURCETRACE_QUOTA_DB`; restarts and instances that use the same file share the quota. An authorised URL reserves a slot before the transfer; a later failure or interruption keeps that reservation until the UTC day changes. Repeating the same URL on that day does not take another slot. Requests refused before the reservation do not consume it. The per-minute limit remains in memory, per process.
The quota database contains the UTC day, hashes of identities and URLs, and a random local secret used in those hashes. It contains no source text, free-tier snapshots, plaintext addresses or gateway credentials. The directory must be private (`0700`) and the file uses `0600`. If the quota database cannot be opened or updated, the service refuses access. The distribution packages exclude this local state.
## HTTP transport
The optional local HTTP transport is started explicitly with `serve --transport http` and uses `127.0.0.1:8793` by default. The port can be changed with `--port`, for example `serve --transport http --port 18793`; `--port 0` asks the operating system for a free port. The port is set on the command line only and has no environment variable. This local adapter does not demonstrate a public endpoint or a production authentication service.
## diff_snapshots semantics
In `diff_snapshots`, each structural group compares the order of the identifiers present in both snapshots, after excluding added and removed ones. An addition or removal does not create moves merely because absolute ordinals changed. When the new order corresponds to relocating a single article by two or more positions, only that identifier is marked `moved`; articles shifted by the operation keep their status. A direct swap marks both swapped identifiers. For adjacent positions, the snapshots cannot distinguish a swap from relocating either of the two articles, so that convention is kept. More complex reorderings still compare ranks and do not represent a minimal sequence of operations. If the content also changed, `modified` takes precedence. The `count` field sums the changes of all groups; an article can appear in both `sections` and `articles`.
## EUR-Lex consolidated profile
In the EUR-Lex consolidated profile, `title-division-1` and `title-division-2` create level 1 and level 2 sections, including free-form subtitles. `title-annex-1` and `title-annex-2` also delimit annexes: changing the annex title or body does not change the hash of the preceding article. These rules recognise explicit markup; they do not guarantee coverage of every page template.
## structure_not_detected warnings
When a declared profile (`eur-lex`, `diario-republica`/`dre` or `rfc-editor`/`rfc`) produces no recognised sections or articles, the text and digests remain available with `warnings: [{"code": "structure_not_detected", "reason": "no_supported_structural_nodes", "source_profile": "<normalised profile>", "message": "<fixed explanation>"}]`. Pages delimited by form feed remain available but do not suppress this warning; this includes paginated RFCs without recognised headings. Without a warning, `warnings` is an empty list. Warnings persist in snapshots and in the public tools; the search and diff aggregates identify the affected `snapshot_id`. The absence of recognised sections or articles may result from unsupported markup and does not prove that the original document has no structure.
## Testing and packaging without network access
```sh
python3 scripts/run_tests.py
python3 scripts/build_package.py
# Optional local installation, when pip is available:
python3 -m pip install --no-deps dist/sourcetrace_mcp-0.1.0-py3-none-any.whl
```
Each test run writes real measurements (mode: real, local machine), including duration, environment and UTC date, outside the repository: to the path in `MIB_TEST_REPORT_LATEST` when set (a path inside the repository is refused with exit 2), otherwise to `TEST_REPORT.latest.json` in a new temporary directory whose path the runner prints on stderr. Running the tests leaves `git status --porcelain --ignored` empty. `TEST_REPORT.json` is the versioned snapshot of the last run in which the results changed: counts, cases/files, suites and any other metrics. Only `duracao_s`, `ambiente` and `gerado_em` are excluded from that comparison. If only those three fields vary, the snapshot keeps its original bytes and measurements; the latest measurement remains available in the temporary file. Changes in results, including failures, update both files. The runner reports on stderr which file was updated; it does not change or hide the Git state.
The builder writes to `dist/` a pure-Python wheel and a source archive, with deterministic internal timestamps, hashes in the wheel RECORD file, the Apache-2.0 licence and the package JSON data. It also works as a PEP 517 backend with an empty list of build dependencies. Nothing is uploaded to external services. Tests and package builds do not require pip.
## Schema validation
Validation against the official schemas is a separate development command that uses the optional validator `jsonschema` (MIT licence). It is not part of the product runtime or of the default test command:
```sh
python3 scripts/validate_registry.py
```
Without `jsonschema` the command exits with code 2. With `--allow-missing-jsonschema` (used by the release script) it then runs only the metadata checks and reports `"schemaValidation": "skipped-jsonschema-unavailable"`. The command also checks the local consistency of licences, versions, package names and the Registry marker in this README. Validation does not verify namespace ownership and does not authorise publication.
See [LICENSES.json](LICENSES.json), [RESEARCH](docs/RESEARCH.md) and [REGISTRY](docs/REGISTRY.md) for the dependency inventory, the standards checked and the publication preparation.
## Gateway identity (D-A-11)
The entitlement verification client uses `MIB_GATEWAY_URL`, `MIB_HMAC_KEYS`, `MIB_HMAC_KEY_ID` and `MIB_TENANT`. The URL can be the local HTTP base or end in `/v1/entitlements/verify`. The key map contains a string per identifier or an object with `tenant` equal to `MIB_TENANT` and `secrets`; in the latter case the first secret is used. The values are supplied through the environment at activation and do not belong in published configuration files.
All real requests to the gateway are signed according to I1 (HMAC). Missing or invalid configuration prevents the connection; identity headers are never sent to sources. Each entitlement check uses a new nonce and idempotency key to avoid reusing an earlier authorisation decision. The tests inject synthetic identities or local simulators. Integration with the real gateway has not been measured.
## Publishing (operator only)
Nothing has been published. The names `sourcetrace-mcp` (PyPI), `strictstack/sourcetrace-mcp` (GitHub) and `io.github.strictstack/sourcetrace-mcp` (MCP Registry) were chosen by the operator and checked as free (HTTP 404); ownership is proven only at publication. Publication uses one script (bash and the Python standard library only):
```sh
bash scripts/publish-public-release.sh [--dry-run | --confirmar] [--env-file <path>] [--help]
```
One-line publication command, after the prerequisites below are met:
```sh
bash scripts/publish-public-release.sh --confirmar
```
### Options and configuration
- Without a mode the script runs `--dry-run`. Passing both modes, an unknown option or an `--env-file` path that does not exist exits with code 2.
- The `.env` file is optional; the default is `./.env` at the repository root (git-ignored). `.env.example` lists the names with `NAO-USAR` values. The file is parsed literally as `NAME=value` against a closed list of names, without `eval`, `source` or expansion. Unknown or repeated names, or control characters, exit with code 2.
- Required only with `--confirmar`: `TWINE_PASSWORD` (PyPI API token) and `GITHUB_TOKEN`.
- Optional: `SOURCETRACE_CODIGO_URL`; `SOURCETRACE_MCP_NAME` (either `io.github.<owner>/sourcetrace-mcp` with the owner equal to the owner in the GitHub URL, or `tech.strictstack/sourcetrace-mcp`); `MIB_GATE_CLI` (default `../mib-cleanroom-gate/bin/mib-cleanroom-gate.mjs`); `MCP_PRIVATE_KEY` (only for the DNS namespace).
- Defaults come from `pyproject.toml` and `server.json`, so `--dry-run` works without a `.env` file and without credentials. Overrides apply only to the temporary snapshot that is built and published (the `server.json` name, the README Registry marker and the repository URL), never to the repository; the printed plan states this. The code pushed to GitHub is always the certified HEAD.
### Steps in both modes
1. Validate the configuration and the coherence of the metadata.
2. Scan the versioned tree and history for secrets.
3. Run `python3 scripts/run_tests.py` in a temporary snapshot.
4. Run `python3 scripts/validate_registry.py --allow-missing-jsonschema` (full schema validation when the optional `jsonschema` is importable by that Python; otherwise metadata-only checks with an explicit warning).
5. Run the clean-room gate with `--fase build`.
6. Build the wheel and the sdist twice with `scripts/build_package.py` and require identical SHA-256 digests.
7. Print the plan, with the exact external commands and secrets masked.
The dry run writes only to a new directory under `/tmp` (its path is printed), uses no network and leaves the repository untouched.
### Additional steps with --confirmar
These run in this order and are idempotent:
Before any write, every remote state (public GitHub repository, remote `main`, published package, Registry entry) is read first; if any of them diverges the script exits with code 1 and nothing is pushed or published. The snapshot is materialised from the HEAD Git objects (not from the working tree); files flagged assume-unchanged or skip-worktree are refused. The push leaves from a fresh temporary bare repository that contains only the certified HEAD, with `-c core.hooksPath=/dev/null -c http.sslVerify=true -c http.followRedirects=false`, so the repository's own `.git/config` never applies to the authenticated command. `mcp-publisher login dns` reads the key from `MCP_PRIVATE_KEY` when no key flag is given (`resolvePrivateKey` in `cmd/publisher/commands/login.go`, https://github.com/modelcontextprotocol/registry, read 2026-11-15); the existing Registry entry is compared on name, version and package identifiers only.
1. GitHub: `git ls-remote`, then `git push https://github.com/strictstack/sourcetrace-mcp.git <HEAD>:refs/heads/main`. Skipped if the remote `main` is already identical; exits with code 1 if the remote `main` differs.
2. PyPI: check `https://pypi.org/pypi/sourcetrace-mcp/0.1.0/json`. Skipped if both files are already published with identical SHA-256 digests; refused if they differ; otherwise `twine upload --non-interactive <wheel> <sdist>` with `TWINE_USERNAME=__token__` and `TWINE_PASSWORD` set only in the child environment.
3. MCP Registry: skipped if the published entry is identical; refused if it differs; otherwise `mcp-publisher login github` (token passed as `MCP_GITHUB_TOKEN` in the child environment, no device prompt) or `mcp-publisher login dns --domain strictstack.tech` (using `MCP_PRIVATE_KEY`), then `mcp-publisher publish`.
PyPI is published before the Registry because the Registry verifies the `mcp-name` marker in the PyPI README.
Secrets are never placed in argv, in output, or in repository or package files. Child processes receive a minimal environment, and the script runs with `umask 077`.
### Exit codes
| Code | Meaning |
| --- | --- |
| 0 | Success |
| 1 | Verification failure |
| 2 | Usage or configuration error, or example values under `--confirmar` (`NAO-USAR`, `example`, `changeme`, `placeholder`, or the former provisional GitHub owner name) |
### Operator prerequisites
- GitHub organisation `strictstack` with an empty public repository `sourcetrace-mcp`.
- A GitHub token with push rights to that repository and with `read:org` (classic token) or "Organization permissions → Members → Read-only" (fine-grained token), so that the Registry grants `io.github.strictstack/*`. Source: <https://github.com/modelcontextprotocol/registry/blob/main/docs/modelcontextprotocol-io/authentication.mdx> (consulted 2026-11-15).
- A PyPI account and an API token able to create the project `sourcetrace-mcp`.
- `twine` and `mcp-publisher` on `PATH` (for example `python3 -m pip install twine jsonschema` for the Python tools).
- For the DNS namespace `tech.strictstack/sourcetrace-mcp` only: a TXT record on the `strictstack.tech` apex with `v=MCPv1; k=ed25519; p=<public key>`.
The Registry ownership marker format follows <https://github.com/modelcontextprotocol/registry/blob/main/docs/modelcontextprotocol-io/package-types.mdx> (consulted 2026-11-15): the Registry checks for `mcp-name: $SERVER_NAME` in the PyPI README. This README carries exactly one such marker, near the top.
## Licence
The original SourceTrace MCP code is licensed under Apache-2.0; see [LICENSE](LICENSE). Data and the Pro service are sold separately, outside this repository, under the store's external terms for the purchased offer. Those terms do not change the rights granted on the code by Apache-2.0. Content from external sources keeps its own access and reuse conditions.
This server cannot be deployed
Maintenance
ActivityMaintained
ResponsivenessNo recent community issues