# MY:TIME — Competitive Intelligence Platform

Ingestion pipeline + remote **MCP server** over a Supabase Postgres single source
of truth. Crawls competitor e-commerce sites and infers demand from inventory
depletion, tracks social performance, and exposes everything to Claude, Power BI,
and the team through one MCP endpoint.

> **Honesty note:** depletion-derived sales are **estimates**, labeled as such in
> all outputs. Competitor social = **public metrics only**. MY:TIME's own data
> comes from official APIs and the Adform product feed, never guessed.

## Architecture

```
SOURCES      competitor e-commerce sites + social platforms
COLLECTION   Apify + FireCrawl + official social APIs + MY:TIME XML feed
INGESTION    Node/TS pipeline — route, normalize, dedupe, date-stamp (modular per source)
STORAGE      Supabase Postgres (Frankfurt) — single source of truth
MCP SERVER   remote, Streamable HTTP, https://mcp.mytimeprime.mk  ← the hub
CONSUMERS    Claude · Power BI / dashboards · internal team
```

## Workspace layout

| Package | Purpose |
|---|---|
| `shared/` (`@mytime/shared`) | Types, env loader (fail-fast), logger, DB pool factory, `targets.json` loader |
| `db/` (`@mytime/db`) | Drizzle schema + migrations + ER diagram |
| `ingestion/` (`@mytime/ingestion`) | One bolt-on collector module per source + routing/scheduler |
| `mcp-server/` (`@mytime/mcp-server`) | MCP tools (role-tagged), auth middleware, health |
| `config/` | `targets.json` (+ JSON Schema) — the only place targets are defined |

## Stack

Node 20+ · TypeScript (ESM, NodeNext) · pnpm workspaces · Drizzle ORM · Biome
(lint + format) · pino logging · zod validation · Supabase Postgres.

## Getting started

```bash
corepack enable && pnpm install      # pnpm 11.x
cp .env.example .env                 # fill in credentials (gitignored)
pnpm build                           # build all packages
pnpm validate:targets                # validate config/targets.json
pnpm lint                            # Biome
```

Every runtime variable is documented in [.env.example](.env.example); missing
required vars fail fast naming the variable.

## Adding a new data source (modularity contract)

A new source is **new module + new table + new MCP tool**, with no edits to
existing sources. See [ingestion/src/sources/README.md](ingestion/src/sources/README.md).

## Build phases

- [x] **Phase 0 — Scaffold** (current): monorepo, tooling, `targets.json` schema, env.
- [ ] **Phase 1 — Site profiling & crawler selection** → `crawler-plan.md`
- [ ] **Phase 2 — Database schema** (full time-series model + ER diagram)
- [x] **Phase 3 — Ingestion pipeline** — A–G done: idempotent writer; all **9 web sites** live (WC Store API, Adform feed, sitemap+JSON-LD, Zia API, FireCrawl-map+OG, Magento listing, nopCommerce); **competitor social** (Apify IG/FB/TikTok); **own-brand social** (MY:TIME IG/FB via the official Meta Graph API, `meta-own-brand` collector, see [docs/step-f-own-brand-social-setup.md](docs/step-f-own-brand-social-setup.md)); **daily scheduler** (systemd timer + runbook, see [deploy/](deploy/README.md)).
- [x] **Phase 4 — Demand inference + MCP server core** — depletion engine (exact/assumed `qty_basis`) + Streamable HTTP MCP server with 4 tools (`get_inventory_velocity`, `compare_market_share`, `social_benchmark`, `price_assortment`); see [mcp-server/](mcp-server/README.md).
- [x] **Phase 5 — OAuth 2.1 + two-layer authorization** — Google upstream via a custom MCP `OAuthServerProvider` (PKCE/S256, DCR, Protected Resource Metadata, 401+`WWW-Authenticate`); Layer 1 signed-Google-ID-token domain gate (`@mytime.mk`), Layer 2 `authorized_users` whitelist/role, per-tool role enforcement (admin/analyst/viewer). Needs your Google OAuth client (see [docs/phase-5-google-oauth-setup.md](docs/phase-5-google-oauth-setup.md)) for the end-to-end browser flow.
- [ ] **Phase 6 — Deploy to Hetzner** (Caddy, TLS, daily ingestion)