# Whip UGC Program Automation

![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)
![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue.svg)

The automation layer *on top of* [SideShift](https://sideshift.app) for
running Whip's UGC creator partnership program. Video performance data
already lives in SideShift — this system organizes what SideShift (and
Matthew's own judgment) surfaces, turns it into action, and keeps him from
manually rebuilding a tracker every week.

Full spec: [ugc-program-automation-spec.md](ugc-program-automation-spec.md).
**Design principle:** Matthew never structures data by hand, and the system
never contacts a creator on its own — it drafts, he sends.

```mermaid
flowchart LR
    SS["SideShift API"] -- sync --> DB[("Local SQLite store")]
    DB -- triage --> DB
    DB -- compile --> DG["Weekly Digest"]
    DB -- plan --> CAL["Google Calendar\n(via Calendar.app)"]
    DG -- Gmail draft --> M["Matthew reviews & sends"]
    AGENT["Claude Code / jcode"] -- MCP tools --> DB
    M -- voice/typed notes --> AGENT
```

## The core loop (spec section 3)

1. **Ingest** — `sync_data` pulls a SideShift export into the Content Log, or `record_manual_content` files raw numbers Matthew provides in chat.
2. **Triage** — `run_triage` flags each post Top/Normal/Underperforming
   relative to *that creator's own* recent baseline (not a fixed threshold).
3. **Surface** — `compile_digest` / `render_digest` return what's worth
   watching and why.
4. **Capture** — Matthew watches, then tells the system his read in plain
   language; `capture_watch_notes` resolves it safely, or `find_content` +
   `record_watch_notes` file it against the right content_id.
5. **Synthesize** — `synthesize_feedback_draft` combines the data pattern
   with Matthew's notes into a starting draft. **Drafted only.**
6. **Calendar** — `sync_calendar` keeps a dedicated Google Calendar current:
   payment dates, contract renewals, follow-up nudges.
7. **Digest** — `run_weekly_routine` runs the whole automated chain on a
   schedule; `compile_digest` answers on-demand queries any time.

## Human-in-the-loop rule (spec section 5, non-negotiable)

Fully automated: parsing/filing data, flagging performance, synthesizing
patterns, drafting feedback text, creating/updating calendar events,
compiling the digest.

**Requires Matthew's action:** anything that reaches a creator. There is no
`send_feedback` tool anywhere in this codebase — `save_feedback_draft` only
ever writes a `Draft` row to the Feedback Log. Getting a draft in front of
Matthew is the calling agent's job (stage it as a Gmail draft with its own
`gmail` tool), and *sending* it is Matthew's.

Calendar events are the one thing this system creates/updates without
approval (low-risk, easily corrected) — but `sync_calendar` only ever
touches events it created itself. Every system-created event is tagged with
a stable marker; a Matthew-made event on the same calendar is never edited
or deleted, even if it collides on date or title. See
[`calendar_sync.py`](src/ugc_analytics/calendar_sync.py).

## Setup

**1. Install**

```bash
git clone <this-repo>
cd sideshift-scanner
python3 -m venv .venv && source .venv/bin/activate && pip install -e ".[dev]"
```

**2. SideShift API key** — copy the template and paste in your key
(SideShift dashboard → Settings → Integrations):

```bash
cp .env.example .env
```

**3. Point the agent at your Python** — copy the template and fill in the
absolute path to `.venv/bin/python` from step 1:

```bash
cp .mcp.json.example .mcp.json
```

**4. Add the dedicated calendar to Calendar.app** (spec section 8 Q3).
Calendar sync works by driving macOS Calendar.app via JXA rather than the
Google Calendar REST API — no OAuth app, no consent screen, no per-machine
token refresh (see [`calendar_sync.py`](src/ugc_analytics/calendar_sync.py)
docstring for the full rationale). Create (or pick) a dedicated Google
Calendar for the program, subscribe it in **Calendar.app → Settings →
Accounts** (it should already be there if it's under the same Google
account as Calendar.app), and note its exact display name. Default is
`"whip"`; override with `calendar_name` on any calendar tool/command.

**5. Restart Claude Code / jcode** in this folder to pick up the MCP server.

That's it — everything else is a chat message or a CLI command.

## Just ask (via the agent)

```
Sync my latest SideShift data and run triage
What should I watch this week?
Ava just posted a TikTok that felt flat, the hook was too slow -- file that
Draft feedback for Marcus based on his last three underperforming posts
Update the calendar with upcoming payments and follow-ups
Give me this week's digest
```

The agent resolves freeform descriptions to the right creator/content_id
via `find_content`, files notes, drafts feedback, and stages Gmail drafts
using its own `gmail` tool — this MCP server never touches Gmail itself.

## MCP tools

| Tool | Spec step | Purpose |
|---|---|---|
| `sync_data` | 1. Ingest | Pull SideShift data (`method='api'` live, `'csv'` for sample_data) |
| `record_manual_content` | 1. Ingest | File raw views + engagement when Matthew states the numbers directly |
| `run_triage` | 2. Triage | Recompute Top/Normal/Underperforming per creator baseline |
| `list_content` | 2/3 | Content Log rows, filterable by creator/platform/flag/watched |
| `compile_digest` | 3/7 | Structured weekly-digest data on demand: per-creator status/trends, top performers with real metrics, trending formats, watchlist |
| `render_digest` | 3/7 | Same, as email-ready markdown |
| `find_content` | 4. Capture | Resolve freeform description → candidate content_ids |
| `record_watch_notes` | 4. Capture | File Matthew's freeform read against a content_id |
| `capture_watch_notes` | 4. Capture | Resolve a freeform post reference and file notes only when unambiguous |
| `save_feedback_draft` | 5. Synthesize | Persist drafted feedback as `Draft` — never sends |
| `synthesize_feedback_draft` | 5. Synthesize | Create a data- and note-informed feedback `Draft` — never sends |
| `list_feedback_drafts` | 5 | List the Feedback Log |
| `list_creators` / `get_creator` / `update_creator` | roster | Creator Roster CRUD |
| `sync_calendar` | 6. Calendar | Reconcile the dedicated calendar against the roster |
| `preview_calendar_plan` | 6 | Same, without touching Calendar.app |
| `run_weekly_routine` | 7. Digest | sync → triage → calendar sync → digest, one call |

## CLI

Every tool above also has a `ugc-cli` command (`sync`, `record-content`, `triage`, `digest`,
`creators`, `content`, `find-content`, `watch-notes`, `draft-feedback`,
`capture-notes`, `synthesize-feedback`, `feedback-drafts`, `update-creator`, `calendar-plan`, `calendar-sync`,
`weekly-routine`):

```bash
ugc-cli weekly-routine --today 2026-08-25
ugc-cli digest --format text
ugc-cli calendar-sync --dry-run
```

## Weekly automation (spec section 6)

`run_weekly_routine` / `ugc-cli weekly-routine` runs sync → triage →
calendar sync → digest compilation in one call and returns
`digest_markdown`. It never touches Gmail — the scheduled agent wakeup that
drives it (see `jcode`'s `ScheduleWakeup`, configured separately per
machine) stages that markdown as a Gmail draft addressed to Matthew each
Monday morning, which he reviews and sends like any other draft. On-demand
digests work the same way any time via `compile_digest` / `ugc-cli digest`.

## Live dashboard

A read-only static dashboard mirrors `compile_digest`'s output (creator
status/trends, top performers with real metrics, trending formats, watch
list, upcoming dates) at **https://whip-ugc-digest.vercel.app**, refreshed
daily via `scripts/refresh_dashboard.sh` (sync → triage → export JSON →
`vercel deploy`) and a separate daily `ScheduleWakeup`. Source lives in
`web/` (single static `index.html` + `public/data.json`, no build step,
no framework) and `scripts/export_digest.py` (thin JSON serializer around
`compile_digest`, no new computation).

**The URL has no authentication** — anyone with the link can see creator
names, handles, and performance numbers. Fine for a personal quick-glance
tool; add Vercel password protection (Project Settings → Deployment
Protection) before sharing the link with anyone outside your own use.

To refresh manually: `./scripts/refresh_dashboard.sh` (requires the Vercel
CLI logged in via `vercel login`, run once per machine).

## Data model (spec section 4)

Three logical tables backed by SQLite (`data/ugc_analytics.db`, gitignored
locally but expected to be shared via whatever sync mechanism the deploy
environment uses):

- **Creator Roster** (`creators`) — one row per creator: platforms, status,
  contact, contract/payment/follow-up dates, notes.
- **Content Log** (`content_items` + `performance_metrics` +
  `content_annotations`) — one logical row per post: SideShift-sourced
  fields never touch Matthew/system-managed fields (`watched`,
  `matthew_notes`, `system_synthesis`, `performance_flag`) on resync.
- **Feedback Log** (`feedback_log`) — one row per drafted feedback
  instance: `Draft` until Matthew sends it himself.

## Triage formula (spec section 8 Q5 — first pass, not finalized)

`performance_flag` is a tie-aware percentile of a post's engagement rate
against that same creator's trailing 5 posts *on the same platform*
(never mixing e.g. YouTube Shorts' high-view/low-engagement pattern into a
TikTok baseline). Top ≥ 70th percentile, Underperforming ≤ 30th, and a
cohort under 3 prior posts is honestly reported as `insufficient_data`
rather than guessed. See [`triage.py`](src/ugc_analytics/triage.py) for the
exact thresholds — these need Matthew's sign-off against real synced data
per the spec.

## Tests

```bash
pytest -q
```

96 tests cover schema/upsert semantics, triage math (including the
platform-scoping and small-cohort-honesty guarantees), CSV + mocked-API
ingestion, digest compilation and rendering, calendar plan computation +
mocked JXA reconciliation, the weekly routine orchestration, every MCP
tool via `mcp.call_tool`, and the CLI via Typer's `CliRunner`. The JXA
calendar script itself (create/update/delete, idempotency, and never
touching a non-marker event) was additionally validated by hand against a
live Google Calendar through Calendar.app — see `calendar_sync.py`'s
module docstring.

## Non-goals (v1, spec section 2)

No new video-performance analytics/dashboard (SideShift already covers
this), no AI video-watching pipeline (spec section 7, Phase 2), not a
CRM/payment processor (tracks dates and status, doesn't execute payments).