# mynow-cli

A CLI for searching and downloading content from `mynow.servicenow.com` — ServiceNow's public
best-practices library/landing page — across content types (Best Practices, Product
Documentation, Developer Portal docs, Now Community posts, and more).

## No authentication required

`mynow.servicenow.com` is a public site — there's no login, token, or API key needed. It's
not a traditional SSO-protected ServiceNow instance; it's a public-facing Now Experience app
running in "Genius Search" / guest-session mode.

There's also no public, documented REST API for its search — it's reverse-engineered from a
Playwright network capture of the site's own search UI (see `scripts/probe.ts` and
`src/servicenow/search.ts` for the confirmed request/response shape). The flow:

1. `GET` any page on the site → returns a guest session cookie plus a CSRF token embedded in
   the HTML as `window.g_ck`.
2. `POST /api/now/v1/batch`, wrapping a GraphQL call to the site's Genius Search data broker,
   using that cookie + token.
3. Parse `result[0].executionResult.output.data.GlideSearch_Query.search` for results, facets,
   and total hit count.

This was confirmed with a **cold, browser-free curl round-trip** (no Playwright, no reused
session) — genuinely unauthenticated, not just working because of a live browser session.

**If search ever stops working**, the site's frontend likely changed one of the fixed config
IDs in `src/servicenow/search.ts` (`SEARCH_DEFINITION_SYS_ID`, `SEARCH_CONTEXT_CONFIG_ID`,
`SEARCH_EVAM_CONFIG_ID`). Re-discover them with a fresh Playwright capture filtered to
`xhr`/`fetch` traffic (write to a file, grep it — never dump raw capture output into a
conversation) rather than guessing. Then re-run `npm run probe` to confirm.

## Install & build

```bash
npm install
npm run build            # compile to dist/
```

No environment variables are required — `SN_INSTANCE_URL` defaults to
`https://mynow.servicenow.com` and only needs overriding if pointing at a different instance.

## Usage

```bash
node dist/index.js help              # or -h / --help — human-readable command list
node dist/index.js commands          # same list as structured JSON, for agents (also: help --json)
node dist/index.js search "stream connect best practices" [--content-type <table>] [--limit <n>] [--offset <n>]
node dist/index.js list-content-types [--refresh]
node dist/index.js login              # opens a browser, blocks until you complete SSO by hand
node dist/index.js login-status
node dist/index.js download <resultUrl>   # a search result's `url` field
```

Or install it as a global/linked bin (`package.json` already declares
`mynow-cli` pointing at `dist/index.js`) and drop the `node dist/index.js` prefix.

For development, `npm run dev` runs the CLI directly with `tsx` (no build step), and
`npm run probe -- "some query"` exercises the live search flow end-to-end.

### Bulk downloads

There's no dedicated bulk-download command — an agent (or a shell loop) drives it by calling
`search` to enumerate results, then `download` once per result `url`:

```bash
node dist/index.js search "stream connect" --limit 50 \
  | jq -r '.results[].url' \
  | while read -r url; do node dist/index.js download "$url"; done
```

## Pagination

The API returns a **fixed 10 rows per request** and pages via an opaque
`paginationToken` (base64 of an internal offset breadcrumb, e.g. `offset:0,10`). Use `offset`
to page through the full corpus and `limit` to control how many rows you get back:

- `offset` is a row index into the full corpus (`offset=10` → second page, `offset=20` → third,
  etc.). It's implemented by constructing a token that jumps directly to that row — the server
  doesn't validate the breadcrumb, so any offset is random-accessible in a single request.
- `limit` (1–50) can exceed the 10-row page size; the server is paged internally
  (`ceil(limit / 10)` requests, following the API's own returned token after the initial jump) to
  gather that many rows.
- `hasMore` in the response is true when more results exist beyond `offset + limit`
  (`offset + limit < totalResults`).

Because the offset jump relies on the reverse-engineered token format, `npm run probe` exercises
offset paging end-to-end — if the encoding ever changes, re-discover it from a fresh capture (see
below) rather than guessing.

## Downloading documents

Search itself needs no auth, but the actual file behind a file-backed result (presentations,
workbooks, docs) is gated behind a real ServiceNow ID / Okta SSO login — confirmed live via
Playwright (see `scripts/probe-headed-login.ts`).

1. **`login`** — opens a real, visible browser window and blocks in the terminal until you
   complete the ServiceNow ID / Okta login by hand, then saves the resulting session to
   `.auth/servicenow-storage-state.json` (gitignored — this is a live credential). Check
   **`login-status`** any time to see whether a session file exists on disk and its age.
2. **`download <resultUrl>`** — pass a `search` result's `url`. Two mechanisms depending on
   content type, both starting from the same cookie-replayed `<table>.do?...&XML` record fetch
   (confirmed to accept cookie replay outside the browser):
   - Most content types (e.g. marketing decks, `u_dotcom_gsdr`): no browser needed at all — the
     record's own fields carry the real file location directly (often a separate public CDN, no
     ServiceNow auth needed for that hop).
   - **Best Practices Library assets** (`u_x_snc_accel_asset_file_gsdr` — this project's primary
     target content type): the record's own `x_snc_nl_data_extr_file_content` field carries the
     full extracted text of the underlying file (confirmed for both `.docx` and `.pptx` sources)
     — this is preferred, since it's what an agent actually wants and needs no browser at all.
     Only when that field is empty does this fall back to the real file's binary bytes, which
     live behind a separate API (`api.servicenow.com/bpl/v1/attachment/<id>`) gated by a real
     Okta OAuth Bearer token minted client-side at click time. A short-lived **headless** browser
     (reusing the captured session, no new login) drives the actual download click to mint that
     token, then a plain fetch uses it to pull the file bytes — see
     `scripts/probe-bpl-token-capture.ts` for how this was confirmed.

   Every download response includes a `sourceUrl` — the human-facing page for that record (the
   Best Practices asset page for BPL content, or the original search result URL otherwise).
   That's the link to hand to a person: open it in a browser and complete their own SSO/login —
   there's no requirement that it be publicly reachable, it just needs to be the *real*
   destination page rather than a backend export URL. The downloaded file/text itself is for
   agentic/bulk use, using the session captured by `login`.

   If the session is missing or has expired, `download` returns a clear error telling you to run
   `login` again — no path ever launches a browser other than the brief, automated one used to
   mint a Best Practices Library token.

## Known limitations

- `contentType` filters client-side by matching the result's own `table` or content-type label
  — there's no separate discovery tool for valid values yet. An unfiltered search's results
  carry their own `table`/`contentTypeLabel`, which is enough to learn valid filters
  organically; `contentTypeFilters` in the response also lists all available content types with
  live counts for the current query.