# Market Map

Market Map turns an early SAM.gov notice into a reproducible, evidence-backed map
of plausible supplier companies. It separates explicit needs, reasonable
implications, and unknowns; retrieves dated public evidence through approved MCP
connectors; resolves legal entities conservatively; ranks candidates by role with
deterministic scoring; and exposes the exact source span behind every scored claim.

The first prediction is frozen as canonical JSON with a SHA-256 checksum before a
later notice can be imported for an isolated comparison. Scores prioritize analyst
review; they are not probabilities of winning, certifications, or legal eligibility
determinations.

## What is included

- Fixture-first deterministic discovery, freeze, replay, and validation comparison.
- Historical `as_of` enforcement and unknown-date exclusion.
- Immutable, content-addressed source and artifact storage.
- SAM, USAspending, SBIR, and Tavily source boundaries with normalized envelopes,
  budgets, caching, pagination, retry/circuit behavior, and coverage reporting.
- Structured fake/OpenAI LLM providers with versioned untrusted-content prompts.
- Conservative UEI/CAGE/DUNS/domain/name-and-address entity resolution.
- Exact citation validation and deterministic role-aware scoring.
- FastAPI REST/SSE, Typer CLI, high-level product MCP tools, ARQ worker hooks, and
  an accessible responsive web application.
- PostgreSQL/Redis local stack, migrations, fixture tests, CI, security checks, SBOM,
  JSON/Markdown exports, architecture/runbook/evaluation/threat-model documentation.

The authoritative contract is [BUILD_SPEC.md](BUILD_SPEC.md).

## Quick start

Prerequisites: Git, OpenSSL, Python 3.12, `uv`, Node 22.13 or newer, `pnpm`,
Docker, and Docker Compose.

```bash
make bootstrap
make demo
```

`make demo` is offline and uses no credentials. It runs the committed PrSM-style
acceptance fixture through discovery, verifies that all scored evidence is eligible at
the cutoff, freezes and re-verifies the manifest, performs the separately locked
validation comparison, and writes example JSON/Markdown artifacts under
`data/demo/`. Supplier/role gold labels are not passed to the product comparison;
after the manifest independently verifies, a separate evaluator reads
`evals/cases/prsm.v1.json` and writes `fixture-evaluation.json`. It also measures 20
same-snapshot artifact-store replays, enforces the two-minute warm-cache p95 gate,
and writes `fixture-warm-replay-benchmark.json`; this is not a cold-live latency
claim. The command prints an
actionable local `file://` report URL, the deterministic run ID and manifest hash, and
every export path.

`max_results` is a display preference: the web report shows at most that many
candidates in the selected supplier lane (with a hard ceiling of 20). JSON and Markdown exports from
REST, CLI, product MCP, and web all retain the complete canonical candidate set.

Start the full local stack:

```bash
make dev
```

For the persistent live stack, keep the required provider values in the uncommitted
`.env` and start it explicitly with `FIXTURE_MODE=false make dev`.

- Analyst web application: <http://localhost:3000>
- REST API and OpenAPI: <http://localhost:8000/docs>
- Health/readiness: <http://localhost:8000/healthz> and <http://localhost:8000/readyz>
- Product MCP Streamable HTTP endpoint: <http://localhost:8010/mcp>
- Repository-owned source MCP: internal Compose endpoint `http://market-source-mcp:8020`

Host-facing API, web, and product-MCP ports bind to `127.0.0.1` by default. Any
intentional non-loopback product-MCP override should also set a strong
`MARKETMAP_MCP_TOKEN`; the source MCP is never published to the host.

## Environment and live mode

Fixture mode is the zero-credential default. `.env.example` documents safe defaults;
an existing `.env` is never generated, overwritten, copied into images, logged, or
committed. `make bootstrap` checks that an existing file remains byte-identical.

Live mode requires these names:

- `SAM_API_KEY`
- `LLM_PROVIDER=openai`, `LLM_MODEL`, and `OPENAI_API_KEY`
- `WEB_SEARCH_PROVIDER=tavily`, `TAVILY_API_KEY`, and `TAVILY_SEARCH_DEPTH=basic`

Run the fixture proof before consuming live quota:

```bash
make smoke-live
```

That target reruns `make demo`, then performs one bounded check each for SAM,
USAspending, SBIR, OpenAI structured output, and Tavily basic search. The SBIR check invokes
the repository-owned `SBIRHTTPProvider` and `SBIRAwardSearchTool` and requires a normalized
award record; an API failure is `failed` unless `SBIR_BULK_FALLBACK_PATH` explicitly names a
valid bulk file, in which case the same tool validates it and reports `degraded`. A generic
SBIR landing page never satisfies the connector check. After the checks complete without a
failure, including any validated degraded SBIR fallback, the target creates a uniquely named
temporary Compose project on an ephemeral loopback port and submits the reference notice in current
mode through the public API and discovery ARQ worker. The product check polls every named stage,
checks citation/date and aggregate call/token/credit budgets, freezes and verifies the
manifest, writes only a sanitized summary under ignored `data/reports`, and removes that
temporary project and its volumes. It does not invoke validation routes or read holdout data.
The historical cutoff proof remains the committed, checksum-verified fixture demo; a new live
fetch is never misrepresented as a pre-cutoff snapshot.
The launcher does not source `.env`: it parses only the named provider/smoke allowlist as
literal values, execs a sanitized environment, and disables Compose's implicit dotenv load.

Set `MARKETMAP_LIVE_API_BASE_URL` to a loopback URL to use an already-running live stack
instead; configure that stack with `config/analysis.live-smoke.yaml`. Set
`SBIR_BULK_FALLBACK_PATH` to an approved host-visible CSV, JSON, JSONL, or NDJSON file for
the direct live-smoke connector check. The launcher passes that literal path only to the
bounded provider-check process and removes it before starting the isolated Compose stack.
For a deployed source MCP, mount the approved file read-only and configure its path as seen
inside that container. Missing or invalid configuration is reported without file contents.
Tavily generated answers/raw content remain disabled.

## CLI

```text
uv run marketmap analyze --notice-url URL --as-of ISO8601 --fixture --wait
uv run marketmap analyze --notice-text "..." --mode current --wait
uv run marketmap status RUN_ID
uv run marketmap report RUN_ID --format json --output report.json
uv run marketmap freeze RUN_ID
uv run marketmap compare RUN_ID --upload later-notice.pdf
uv run marketmap replay RUN_ID --offline
uv run marketmap verify-freeze RUN_ID
uv run marketmap mcp doctor
uv run marketmap source doctor
```

CLI and product-MCP comparison calls wait for the isolated validation job to
finish so their fixed command/tool sets return the actual comparison. The wait
is bounded to ten minutes by default; polling starts at 0.5 seconds and uses
bounded exponential backoff up to 5 seconds. Operators may lower the timeout
and starting interval with `MARKETMAP_VALIDATION_WAIT_SECONDS` (0–600) and
`MARKETMAP_VALIDATION_POLL_SECONDS` (0.05–5). If the bounded wait expires, the
response retains the durable validation ID and tells the caller to resubmit the
identical compare request to retrieve the terminal result; REST and web remain
asynchronous through `GET /api/v1/validations/{validation_id}`.

Exactly one notice input is accepted. Historical analysis requires an explicit UTC
cutoff. Validation refuses an unfrozen or checksum-invalid discovery run.

## REST and product MCP

`POST /api/v1/analyses` returns HTTP 202 with run, status, and event URLs. Reports,
requirements, candidates, exact evidence, freeze, compare, validation, health, and
readiness routes follow the contracts in the build spec. Errors use typed problem
details, while a degraded enrichment source produces a valid `partial` resource.

The product MCP exposes only high-level analyst operations:

- `analyze_notice`
- `get_analysis_status`
- `get_analysis_report`
- `search_companies`
- `get_company_evidence`
- `compare_with_later_notice`

Raw source tools are not handed to the analysis model.

## Repository map

```text
apps/
  api/                 FastAPI delivery surface
  worker/              ARQ job entry points
  web/                 analyst application
packages/
  domain/              shared Pydantic records and enums
  persistence/         SQLAlchemy, repositories, migrations, immutable blobs
  mcp_client/           approved registry, envelopes, client manager, adapters
  source_mcp/           SBIR, Tavily, safe fetch, and local-import tools
  ingestion/            notice validation, attachments, document extractors
  llm/                  provider protocol, fake/OpenAI adapters, prompts
  analysis/             checkpointed workflow, temporal, identity, citations, scoring
  reporting/            canonical report, Markdown, manifest, freeze, comparison
  cli/ and product_mcp/ thin non-web delivery surfaces
tests/                  unit, contract, integration, adversarial, e2e, fixtures
config/                 analysis/scoring settings, MCP pins and schema snapshots
docs/                   architecture, data dictionary, runbook, evaluation, security
scripts/                bootstrap, demo, fixture recording, pins, smoke, SBOM, cleanup
data/                   ignored runtime artifacts only
```

## Development gates

```bash
make check
make test
make test-e2e
make demo
make export-sbom
```

`make clean-runtime` prints and removes only repository-generated caches, build outputs, and reports.
It preserves Docker volumes, source blobs, uploads, and operator data.
`make check-licenses` rejects unknown, strong-copyleft, or source-available locked
dependencies; `make export-sbom` writes the reviewed license inventory beside both
CycloneDX inventories.

## Troubleshooting

- If a fixture request is rejected, use the prefilled reference URL, historical mode,
  cutoff `2025-06-02T21:00:00Z`, and `max_results=20`; fixtures are deliberately bound
  to one exact request.
- If readiness is partial, run `marketmap mcp doctor` and `marketmap source doctor`.
  The output names the unavailable connector without printing credentials.
- If Compose starts from an older schema, run `docker compose down --volumes` only when
  you intend to discard local prototype data, then rerun `make dev`.
- If ports 3000, 8000, or 8010 are occupied, stop the conflicting local process or
  override the published ports in a Compose override file.
- If the web build reports an engine mismatch, use Node 22.13 or newer as pinned by CI
  and the container image.

## Evidence and evaluation boundaries

Every displayed substantive claim resolves to a stored document hash, legal entity,
publication date/confidence, exact quote and locator, evidence class, directness, and
score contribution. Current mutable webpages without defensible dates are excluded
from historical scores. Award receipt is funded activity, not proof of successful
delivery; NAICS/PSC membership is weak inferred evidence.

The early notice and locked validation notice are synthetic acceptance fixtures. The
candidate evidence excerpts are dated, pre-cutoff public-source excerpts retained for
deterministic offline replay; they are not a claim of live source access or
comprehensive supplier coverage. Review
[docs/evaluation_methodology.md](docs/evaluation_methodology.md) and
[docs/known_limitations.md](docs/known_limitations.md) before interpreting results.

## Documentation

- [Architecture](docs/architecture.md)
- [Data dictionary](docs/data_dictionary.md)
- [Connector runbook](docs/connector_runbook.md)
- [Evaluation methodology](docs/evaluation_methodology.md)
- [Release evidence](docs/release_evidence.md)
- [Threat model](docs/threat_model.md)
- [Known limitations](docs/known_limitations.md)
- [Follow-up backlog](docs/follow_up_backlog.md)

MIT licensed. Upstream reviewed pins and attribution are recorded in `NOTICE` and
`config/mcp-lock.json`.