# google-search-console-api

Local HTTP API, CLI and MCP server for **Google Search Console Search Analytics** —
clicks, impressions, CTR and average position for your site.

Built so an AI agent can fetch its own reports instead of you exporting a CSV by
hand every morning. Point the agent at `http://127.0.0.1:8788`, or connect it
over MCP, and it can ask for any date range, any dimension, any single URL.

**Read-only.** It requests the `webmasters.readonly` scope only. It cannot
submit URLs, change settings, or write anything to your property.

---

## Table of contents

- [What it gives you](#what-it-gives-you)
- [Setup](#setup)
- [Running it](#running-it)
- [Example requests](#example-requests) ← the table you probably came for
- [`/summary` totals vs top lists](#summary-totals-vs-top-lists)
- [Reading the response](#reading-the-response)
- [Parameters](#parameters)
- [Filtering](#filtering)
- [Comparing two periods](#comparing-two-periods)
- [Downloading a report](#downloading-a-report)
- [CLI](#cli)
- [MCP server](#mcp-server)
- [Logging](#logging)
- [Caching](#caching)
- [Quotas and limits](#quotas-and-limits)
- [What the API cannot give you](#what-the-api-cannot-give-you)
- [Things that will confuse you if nobody says them](#things-that-will-confuse-you-if-nobody-says-them)
- [Troubleshooting](#troubleshooting)
- [Docker](#docker)
- [Development](#development)

---

## What it gives you

| Interface | How you use it | Good for |
| --- | --- | --- |
| **HTTP API** | `curl http://127.0.0.1:8788/queries` | Agents, scripts, dashboards, cron jobs |
| **OpenAPI schema** | `GET /openapi.json` | Letting an agent discover every endpoint on its own |
| **MCP server** | `pnpm mcp` over stdio | Claude Code and other MCP clients, as native tools |
| **CLI** | `pnpm cli --days 28 --format csv` | One-off pulls, piping into other tools |

All four go through the same query layer, so a filter or a date rule behaves
identically no matter which one you use.

---

## Setup

### 1. Create a service account

In [Google Cloud Console](https://console.cloud.google.com):

1. Create (or pick) a project.
2. **APIs & Services → Library →** enable **Google Search Console API**.
3. **IAM & Admin → Service Accounts → Create service account.**
4. On the new account, **Keys → Add key → Create new key → JSON.** Download it.

### 2. Give it access to your property

In [Search Console](https://search.google.com/search-console) open
**Settings → Users and permissions → Add user**, paste the service account's
email, and give it **Full** access.

> Owner is **not** required. Owner is only needed for the Indexing API, which
> this project does not use. Full is enough to read analytics, and it is the
> smaller privilege.

### 3. Configure the environment

```bash
cp .env.example .env
```

Fill in three values:

| Variable | What goes in it |
| --- | --- |
| `SERVICE_CLIENT_EMAIL` | The service account email from step 1 |
| `PRIVATE_KEY_BTOA` | Base64 of the `private_key` field in the JSON key file |
| `SITE_URL` | Your property, e.g. `sc-domain:example.com` |

Encoding the key:

```bash
# Linux / macOS
printf '%s' "$(jq -r .private_key key.json)" | base64 -w0
```

```powershell
# Windows PowerShell
[Convert]::ToBase64String([Text.Encoding]::UTF8.GetBytes((Get-Content key.json | ConvertFrom-Json).private_key))
```

The decoder tolerates what deployment panels do to multi-line secrets:
line-wrapped base64, surrounding quotes, CRLF, and literal `\n` escapes. If the
value gets truncated on paste, you get a specific error saying so rather than a
generic auth failure.

### 4. Verify it works

```bash
pnpm install
pnpm check-access
```

```
Service account: search-reader@your-project.iam.gserviceaccount.com

Properties visible to this account:
  sc-domain:example.com  [siteFullUser]

Default property sc-domain:example.com is readable [siteFullUser].
```

If the list is empty, the service account has not been added in Search Console
yet. If your property is missing from it, the string in `SITE_URL` does not
match how the site is registered — copy one of the listed strings verbatim.

---

## Running it

```bash
pnpm dev      # development, reloads on change
pnpm build && pnpm start   # production
```

```
Search Console API listening on http://127.0.0.1:8788
  service account: search-reader@your-project.iam.gserviceaccount.com
  default property: sc-domain:example.com
  auth: none
```

It binds to **loopback only** by default, because the API exposes your whole
property's analytics. To expose it on the network, set `HOST=0.0.0.0` **and**
set `API_TOKEN` — the server warns you loudly if you do the first without the
second.

With `API_TOKEN` set, every request needs a header:

```bash
curl -H "Authorization: Bearer $API_TOKEN" http://127.0.0.1:8788/queries
```

`/health` stays open either way, so container health checks keep working.

---

## Example requests

Every row assumes `SITE_URL` is set, so no `siteUrl` parameter is needed.

### The basics

| What you want | Request | What comes back |
| --- | --- | --- |
| Is it up? | `GET /health` | `{"ok":true}`. Nothing else, no auth needed. |
| What can I ask for? | `GET /` | A list of every endpoint with a one-line description, plus your default property and cache state. Start here if you are an agent. |
| Machine-readable schema | `GET /openapi.json` | Full OpenAPI 3.1 document — every parameter, every enum value, every response shape. Feed this to an agent and it needs no other documentation. |
| Which sites can I read? | `GET /sites` | The properties this service account can see, each with its permission level. Check here when a query is refused. |

### Everyday reports

| What you want | Request | What comes back |
| --- | --- | --- |
| Top searches, last 28 days | `GET /queries` | One row per search term people typed, with clicks, impressions, CTR and average position. Sorted by clicks, highest first. |
| Top 20 searches only | `GET /queries?limit=20` | The same, cut to 20 rows. |
| Top pages | `GET /pages` | One row per URL on your site, showing how much traffic each earned. |
| Daily trend | `GET /timeseries` | One row per calendar day, so you can see when something moved. |
| Where are visitors from? | `GET /countries` | One row per country, using three-letter codes (`tur`, `usa`, `deu`). |
| Phone or desktop? | `GET /devices` | Three rows at most: `DESKTOP`, `MOBILE`, `TABLET`. |
| Everything at once | `GET /summary` | Property-wide totals, plus the top 10 of each dimension and the daily series, in one response. **The `top.*` lists are a leaderboard, not a census — never sum them.** See [the note below](#summary-totals-vs-top-lists). |

### Slicing it

| What you want | Request | What comes back |
| --- | --- | --- |
| A specific month | `GET /queries?startDate=2026-08-01&endDate=2026-08-31` | Only rows for August. Both dates are inclusive. |
| Last 7 days | `GET /queries?days=7` | A 7-day window ending 3 days ago, since fresh data has not landed yet. |
| Which searches led to one page | `GET /url?url=https://example.com/blog/post` | Every search term that produced a click or impression for that exact URL. This is the "why does this page get traffic" question. |
| Search terms **and** landing page together | `GET /search-analytics?dimensions=query,page` | One row per query-and-page pair. Bigger result set: this is where pagination kicks in. |
| Only blog pages | `GET /pages?filter=page:contains:/blog/` | Pages whose URL contains `/blog/`, nothing else. |
| Only traffic from Turkey | `GET /queries?filter=country:equals:tur` | Searches made from Turkey only. |
| Image search instead of web | `GET /queries?type=image` | The same shape of data, but for Google Images. |
| Include the last 2 days | `GET /timeseries?dataState=all` | Adds the freshest days, which are still incomplete and will move. |

### Comparison and export

| What you want | Request | What comes back |
| --- | --- | --- |
| Is this month better than last? | `GET /queries?compare` | Each row carries current, previous and delta values, plus totals for both windows. Rows that appeared or vanished are kept, with zeroes on the missing side. |
| Compare against a named period | `GET /queries?startDate=2026-08-01&endDate=2026-08-31&compareStartDate=2026-07-01&compareEndDate=2026-07-31` | The same, but you choose both windows. |
| Download a full report | `GET /export` | A CSV file with sections for queries, pages, days, countries and devices. This replaces exporting by hand from the Search Console UI. |
| A readable report | `GET /export?format=md&days=30` | The same report as a Markdown document with headings and tables — paste it into a doc, or hand it to an agent to summarise. |
| Report for a specific window | `GET /export?startDate=2026-08-01&endDate=2026-08-31` | The file is named `search-console-example.com-2026-08-01_2026-08-31.csv`. |

### Formats

| What you want | Request | What comes back |
| --- | --- | --- |
| JSON (default) | `GET /queries` | Full structure with `rows`, `totals` and the resolved date range. CTR is a fraction here (`0.0615`). |
| CSV | `GET /queries?format=csv` | A spreadsheet file, downloaded rather than displayed. Queries containing commas and quotes are escaped properly. CTR is a percentage here (`6.15`). |
| Markdown | `GET /queries?format=md` | A table with a plain-language summary line above it. The most compact form for an agent to read. |

### Operations

| What you want | Request | What comes back |
| --- | --- | --- |
| What are the API quotas? | `GET /limits` | Google's published rate limits, the query rules, and this service's own pagination and retry settings. |
| Is my data cached? | `GET /cache` | Hit and miss counts, how many entries are held, and the TTL. |
| Force fresh data | `DELETE /cache` | Empties the cache, so the next request goes back to Google. Use it when Search Console has just published an update. |

---

### `/summary` totals vs top lists

`/summary` returns two different kinds of number, and mixing them up gives a
figure that is wrong by an order of magnitude:

| Field | What it is |
| --- | --- |
| `totals` | **Whole-property figures.** Uncapped — every row Google returned for the range. This is your real traffic. |
| `top.queries`, `top.pages`, … | **Ranked samples**, at most `limit` rows each (default 10). Summing these gives the top-10 subtotal, not the total. |
| `top.truncated` | `true` when a list was cut, which is the normal case for a real site. |
| `timeseries` | Uncapped daily rows for the range. |

```jsonc
{
  "totals": { "clicks": 44, "impressions": 8906, "rows": 139 },  // real
  "top": {
    "limit": 10,
    "truncated": true,
    "pages": [ /* 10 rows summing to ~532 impressions */ ]       // a sample
  }
}
```

For a complete breakdown of one dimension, query it directly with a limit that
covers your site:

```bash
curl "http://127.0.0.1:8788/pages?limit=500"
```

---

## Reading the response

A JSON response looks like this:

```json
{
  "siteUrl": "sc-domain:example.com",
  "range": { "startDate": "2026-08-03", "endDate": "2026-08-30" },
  "dimensions": ["query"],
  "type": "web",
  "aggregationType": "auto",
  "rows": [
    {
      "keys": ["blue running shoes"],
      "clicks": 4,
      "impressions": 65,
      "ctr": 0.06153846153846154,
      "position": 9.384615384615385
    }
  ],
  "truncated": false,
  "totals": { "clicks": 4, "impressions": 65, "ctr": 0.0615, "position": 9.38, "rows": 1 }
}
```

Read as a sentence: **over the 28 days ending 30 August, 65 people saw
`blue running shoes` in their results and 4 of them clicked — a 6.15% click
rate — and on average your page sat at position 9.4, roughly the bottom of page
one.**

Field by field:

| Field | What it actually means |
| --- | --- |
| `keys` | The dimension values for this row, in the order you asked for them. With `dimensions=query,page` a row's keys are `["blue running shoes", "https://example.com/shoes"]`. |
| `clicks` | How many people clicked through to your site from this row. |
| `impressions` | How many times a link to your site appeared in results for this row. Appearing on page 5 still counts. |
| `ctr` | Clicks divided by impressions. **A fraction in JSON** (`0.0615`), **a percentage in CSV and Markdown** (`6.15`), because that is what each consumer expects. |
| `position` | Your average ranking. Lower is better: 1.0 is the top result. It is an **average**, so you cannot add it up across rows. |
| `truncated` | `true` means `limit` or the internal page ceiling stopped the walk before every row was fetched — there is more data than you received. |
| `totals` | Sums across all returned rows. `ctr` is recomputed from the summed clicks and impressions, and `position` is weighted by impressions, because a plain average of either would be wrong. |
| `range` | The dates actually used, after defaults were filled in. Check this if you did not pass explicit dates. |

A comparison response adds `current`, `previous` and `delta` to each row:

```json
{
  "keys": ["blue running shoes"],
  "current":  { "clicks": 12, "impressions": 210, "ctr": 0.057, "position": 6.2 },
  "previous": { "clicks": 4,  "impressions": 65,  "ctr": 0.061, "position": 9.4 },
  "delta":    { "clicks": 8,  "impressions": 145, "ctr": -0.004, "position": 3.2 }
}
```

Read as a sentence: **this search tripled its clicks and moved up 3.2 places in
the rankings; the click rate dipped slightly because it is now being shown to a
much wider audience.**

> **The position delta is inverted on purpose.** Moving from position 9.4 to 6.2
> is an improvement, so it is reported as `+3.2`, not `-3.2`. Positive always
> means better, for every metric.

---

## Parameters

Every analytics endpoint accepts all of these.

| Parameter | Default | Meaning |
| --- | --- | --- |
| `siteUrl` | `SITE_URL` | Which property. Accepts `sc-domain:example.com`, a bare `example.com` (read as a domain property), or `https://example.com/` for a URL-prefix property. |
| `startDate` | derived | Inclusive start, `YYYY-MM-DD`. |
| `endDate` | 3 days ago | Inclusive end, `YYYY-MM-DD`. Defaults back a few days because data lands late. |
| `days` | `28` | Window length counted back from `endDate`. |
| `dimensions` | `query` | Comma-separated: `query`, `page`, `country`, `device`, `date`, `hour`, `searchAppearance`. |
| `type` | `web` | `web`, `image`, `video`, `news`, `discover`, `googleNews`. |
| `aggregationType` | `auto` | `byPage` and `byProperty` change what `position` means. Leave it alone unless you know why you are changing it. |
| `dataState` | `final` | `final` is settled data. `all` includes the freshest days, which still move. `hourly_all` is required for the `hour` dimension. |
| `limit` | none | Maximum rows **in total**, across pages. Omit to fetch everything. |
| `filter` | none | Repeatable. See [Filtering](#filtering). |
| `compare` | off | Also fetch the preceding window and report deltas. |
| `compareStartDate` / `compareEndDate` | derived | Name the comparison window explicitly. |
| `format` | `json` | `json`, `csv`, `md`. |

---

## Filtering

The shorthand is `dimension:operator:expression`, and it repeats:

```bash
curl "http://127.0.0.1:8788/queries?filter=page:contains:/blog/&filter=country:equals:tur"
```

That reads as: **searches from Turkey that landed on a blog page.** Multiple
filters are ANDed together.

| Operator | Matches |
| --- | --- |
| `equals` | Exactly this value |
| `notEquals` | Anything except this value |
| `contains` | Value appears anywhere in the string |
| `notContains` | Value does not appear |
| `includingRegex` | Matches this RE2 regular expression |
| `excludingRegex` | Does not match this regular expression |

The expression may contain colons — a URL does — so only the first two colons
are treated as separators.

---

## Comparing two periods

```bash
curl "http://127.0.0.1:8788/queries?days=28&compare&format=md"
```

```
**sc-domain:example.com** · 2026-08-03 → 2026-08-30 vs 2026-07-06 → 2026-08-02

Clicks 1420 (+188) · Impressions 24310 (+2104) · CTR 5.84% (+0.31pp) · Position 8.12 (+0.43)

_Position delta is inverted: positive means the ranking improved._
```

Without explicit comparison dates, the previous window is the **equally long
period immediately before** the current one. That keeps a 28-day comparison
weekday-aligned, which matters — search traffic has a strong weekly rhythm, and
comparing a 4-week block to a calendar month would put a different number of
Mondays in each side.

---

## Downloading a report

`/export` is the direct replacement for exporting CSVs by hand:

```bash
curl -OJ "http://127.0.0.1:8788/export?startDate=2026-08-01&endDate=2026-08-31"
```

You get `search-console-example.com-2026-08-01_2026-08-31.csv`, containing five
sections one after another — queries, pages, daily totals, countries, devices —
each preceded by a `# dimension` marker line.

Choose your own sections and format:

```bash
curl -OJ "http://127.0.0.1:8788/export?dimensions=query,page&format=md&days=90"
```

The daily and hourly sections ignore `limit`, because capping a timeline to a
top-10 would silently cut the range short.

---

## CLI

```bash
pnpm cli --help
```

| Command | What it does |
| --- | --- |
| `pnpm cli` | Top queries for the last 28 days, as JSON on stdout |
| `pnpm cli --days 7 --format md` | A readable table for the last week |
| `pnpm cli --dimensions query,page --limit 100` | Query-and-page pairs, top 100 |
| `pnpm cli --start 2026-08-01 --end 2026-08-31 --format csv --out august.csv` | Writes a CSV file |
| `pnpm cli --export --out` | Full report; the filename is derived from the property and dates |
| `pnpm cli --compare --days 28 --format md` | Period-over-period comparison |
| `pnpm cli --filter page:contains:/blog/` | Only blog pages |
| `pnpm cli --sites` | List readable properties and exit |

Errors are written to stderr with the fix stated, and the exit code separates a
bad request (`2`) from an API failure (`1`), so a shell script can tell them
apart.

---

## MCP server

Runs over stdio, so an MCP client launches it directly.

```bash
pnpm build   # once
```

A ready `.mcp.json` ships with the repo, so a client launched from this
directory picks the server up with no further setup:

```json
{
  "mcpServers": {
    "search-console": {
      "command": "node",
      "args": ["--env-file=.env", "dist/server/mcp/stdio.js"],
      "cwd": "."
    }
  }
}
```

Credentials come from `.env` at launch (Node 22's built-in `--env-file`) rather
than being embedded, so the config file holds nothing secret and stays safe to
commit.

To register it from another directory, give an absolute `cwd`:

```json
{
  "mcpServers": {
    "search-console": {
      "command": "node",
      "args": ["--env-file=.env", "dist/server/mcp/stdio.js"],
      "cwd": "/absolute/path/to/google-search-console-api"
    }
  }
}
```

The tools it exposes:

| Tool | What the agent gets |
| --- | --- |
| `search_analytics` | Any dimension combination, with optional comparison |
| `top_queries` | What people searched before reaching the site |
| `top_pages` | Which pages earned the traffic |
| `timeseries` | One row per day, for spotting when something changed |
| `page_queries` | Every search that led to one specific URL |
| `export_report` | A full multi-section report |
| `list_sites` | Readable properties with permission levels |
| `api_limits` | Quotas and query rules |

All are marked read-only. They default to Markdown output, which is the most
compact form for a model to read, and failures come back as readable text with
the fix named rather than as a transport error.

When a `limit` cuts a result short, the header says **"Subtotal of the rows
below … not the property total"** rather than "Totals", so a model does not
report a top-5 sum as the site's traffic. Drop `limit` for whole-property
figures.

---

## Logging

Every request is logged to stdout, so `docker logs` and `docker compose logs`
show your traffic with no extra configuration.

```
2026-09-02T00:31:14.204Z INFO  GET /queries 200 412ms cache=MISS days=7 format=md
2026-09-02T00:31:19.882Z INFO  GET /queries 200 1ms cache=HIT days=7 format=md
2026-09-02T00:33:02.551Z ERROR GET /queries 401 0ms
2026-09-02T00:34:41.017Z ERROR GET /export 403 890ms days=30 format=csv
```

Each line carries the timestamp, the outcome, the method and path, the status,
how long it took, whether the cache answered it, and the parameters that shaped
the query.

| `LOG_LEVEL` | What gets written |
| --- | --- |
| `silent` | Nothing |
| `error` | Only failed requests |
| `info` *(default)* | Every request |
| `debug` | Every request, plus the caller's address |

Logging is mounted before authentication, so a rejected token and an unknown
route are recorded too — those are usually the lines worth seeing.

### MCP tool calls land in the same stream

An MCP server is launched by its client, wherever that client runs, and its
stderr usually disappears into that client rather than reaching you. So MCP
tool calls are relayed to the HTTP server and appear in the same log:

```
2026-09-02T01:03:34.469Z INFO GET /health 200 1ms
2026-09-02T01:03:35.917Z INFO MCP top_queries 635ms days=7 limit=5
2026-09-02T01:03:36.402Z ERROR MCP page_queries 210ms url=https://example.com/x
```

One `docker compose logs` therefore shows HTTP requests and MCP tool calls
together, in order.

| Setting | Default | Meaning |
| --- | --- | --- |
| `MCP_LOG_URL` | `http://127.0.0.1:8788` | Where the MCP server posts its log lines. |
| `MCP_LOG_RELAY` | `true` | Set `false` to keep MCP logs on stderr only. |

The relay is best-effort: it never blocks a tool call and never fails one, and
stderr keeps a copy regardless, so nothing is lost when the HTTP server is
down. MCP output never touches stdout, which carries the JSON-RPC framing.

At `info`, the log lists the parameters that shaped the query but only *counts*
filters (`filters=2`) — a filter expression can carry a full URL, and a
default-level log should not accumulate those. An unrecognised parameter is
named but not valued (`+mystery`), so a typo is visible without its value being
written.

At `debug`, the **entire query string** is logged, decoded, filter expressions
included — seeing exactly what was asked for is usually the reason you turned
debug on:

```
2026-09-02T00:41:07.882Z INFO  GET /queries 200 380ms cache=MISS days=7 filters=1
2026-09-02T00:41:23.104Z INFO  GET /queries 200 402ms cache=MISS days=7&filter=page:contains:/blog/ from=172.18.0.1
```

No header or credential is ever logged at any level. `DEBUG=true` raises the
level to `debug` and additionally logs each pagination step and retry.

Behind a reverse proxy, `debug` reads `X-Forwarded-For` for the caller's
address. That header is caller-controlled, so it is only ever logged, never
trusted for a decision.

---

## Caching

Identical requests are served from memory for **15 minutes** by default.

This exists because Google refreshes Search Analytics on its own schedule —
often only every few hours — so an agent polling every minute would spend quota
receiving rows it already has.

| Setting | Default | Meaning |
| --- | --- | --- |
| `CACHE_TTL_SECONDS` | `900` | How long an identical query is reused. `0` disables caching. |
| `CACHE_MAX_ENTRIES` | `200` | Ceiling before the least recently used entry is dropped. |

See also [`LOG_LEVEL`](#logging) for how much of this traffic reaches the log.

The cache lives in the process's own memory — no Redis, no files, nothing on
disk. It holds only rows already fetched for you, and it is emptied when the
process restarts.

Every analytics response carries `X-Cache: HIT` or `MISS`, so you can tell a
reused answer from a fresh one. Failures are never cached, so a transient rate
limit is not replayed for the rest of the TTL.

`DELETE /cache` forces the next request back to Google.

---

## Quotas and limits

`GET /limits` returns all of this as JSON. The short version:

| Limit | Value |
| --- | --- |
| Search Analytics, per site | 1,200 queries/minute |
| Search Analytics, per user | 1,200 queries/minute |
| Search Analytics, per project | 40,000 queries/minute, 30,000,000/day |
| Rows per API request | 25,000 (this service paginates past it automatically) |
| Data retention | 16 months |
| Hourly data retention | 10 days |
| Data freshness | 2–3 days behind |

Rate limits are handled for you: a `429` or a `5xx` is retried with exponential
backoff and jitter, up to five attempts, rather than failing the whole run and
wasting the pages already fetched.

Google publishes no quota-inspection endpoint, so live consumption cannot be
read from here — check the
[Cloud console quotas page](https://console.cloud.google.com/apis/api/searchconsole.googleapis.com/quotas)
for that.

---

## What the API cannot give you

Some data exists in the Search Console interface but has no API equivalent.
Nothing in this project can work around that — it is a limit of what Google
publishes, not of this code, and no paid tier unlocks it.

### Generative AI performance (AI Overviews, AI Mode)

**Not available through the API.** Search Console shows a
[Generative AI performance report](https://support.google.com/webmasters/answer/16984139)
covering impressions inside AI Overviews and AI Mode, added in
[June 2026](https://developers.google.com/search/blog/2026/06/gen-ai-performance-reports).
It is a **separate report in the interface**, not a new `type`, not a new
dimension, and not a `searchAppearance` value — so `searchanalytics.query`
cannot reach it.

What this means in practice:

| Question | Answer |
| --- | --- |
| Can I query AI Overviews traffic here? | No. |
| Is there a paid tier that unlocks it? | No. Search Console is free, and this data is UI-only for everyone. |
| Is the traffic missing from my numbers? | No — AI feature impressions are **included** in the `web` search type totals. |
| Can I separate "how much came from AI"? | No. It is blended into the web totals and cannot be broken out. |
| Should I add the UI export to my API totals? | **No — that double counts.** See below. |
| How do I get it? | Search Console → Performance → the generative AI report → **Export**. By hand. |

Google stated when announcing AI Mode reporting that
[you would not be able to break it out](https://searchengineland.com/google-search-console-to-show-ai-mode-performance-but-you-wont-be-able-to-break-it-out-455992),
and that
[no separate API change was involved](https://searchengineland.com/google-ai-mode-traffic-data-search-console-457076).

If your pipeline needs AI-surface numbers, plan for a manual export step. This
project covers everything else.

**Never add the manual Gen-AI export to your API totals.** Its impressions are
a *subset* of the web totals you already have, not an additional surface. One
property's check, comparing an interface Gen-AI export against the API's page
list for the same window:

```
pages with Gen-AI impressions:            72
  of those, also in the classic list:     72   (all of them)
  with Gen-AI exceeding their classic:     0   (none)
```

So a figure like "Gen-AI share: 6.22%" is a **ratio, not extra traffic** — it
reads as "6.22% of impressions we already counted also appeared on an AI
surface". What the API costs you is the *breakdown*, never the volume.

### searchAppearance returns fewer values than you expect

If `GET /queries?dimensions=searchAppearance` comes back with only one or two
entries — `PRODUCT_SNIPPETS`, say — that is not a bug. The dimension reports
only the rich-result types your pages actually qualified for. Generative AI
surfaces are **not among the values it can return at all**, whatever your site
does.

Remember the dimension also cannot be combined with any other. To break one
appearance type down further, query it alone first, then filter by it:

```bash
# 1. Which appearance types exist for this site?
curl "http://127.0.0.1:8788/queries?dimensions=searchAppearance"

# 2. Then drill into one of them
curl "http://127.0.0.1:8788/queries?dimensions=page&filter=searchAppearance:equals:PRODUCT_SNIPPETS"
```

This two-step shape is [what Google's own documentation prescribes](https://developers.google.com/webmaster-tools/v1/how-tos/all-your-data).

### The 1,000-row wall on a single query

The Search Console interface caps its query table at 1,000 rows. A single
`query`-dimension API pull can land near the same number, and when it does the
response still says `truncated: false` — because from this service's point of
view Google returned a short page and the walk finished normally.

First, the easy half:

| What you see | What it means |
| --- | --- |
| `truncated: true` | **Your** `limit` stopped the walk. Raise it. |
| `truncated: false`, row count nowhere near 1,000 | Real, complete data. |
| `truncated: false`, row count at ~999–1,000 | **Ambiguous.** Could be genuine, could be a ceiling. |

That last row is the trap, and no field in the response resolves it. A count
near 1,000 is not proof of truncation — a site really can have 999 distinct
queries in a window. The only way to know is to split the range and count
distinct values:

```bash
# One pull for the whole month
curl "http://127.0.0.1:8788/queries?days=28&limit=5000" | jq '.totals.rows'

# The same month in weekly slices - if the union is materially larger than the
# single pull, the single pull was capped
for start in 2026-08-01 2026-08-08 2026-08-15 2026-08-22; do
  curl -s "http://127.0.0.1:8788/queries?startDate=$start&days=7&limit=5000"
done | jq -s '[.[].rows[].keys[0]] | unique | length'
```

If both numbers agree, the count was genuine. If the split total is much larger,
you were against the ceiling and the single-pull figure understated your query
count.

**Treat any query count at or near 1,000 as unverified until you have split
it.** Do not publish it as a distinct-query total.

Splitting is also how you get past the ceiling when you are genuinely against
it, since each slice carries its own:

```bash
# Each day gets its own ceiling, so 28 days yields far more than 1,000 rows
curl "http://127.0.0.1:8788/search-analytics?dimensions=date,query&days=28"

# Or slice by a filter
curl "http://127.0.0.1:8788/queries?days=28&filter=page:contains:/blog/"
```

### Rare queries are withheld entirely

Google drops rows for searches it considers rare, to protect the privacy of the
people who typed them. They are absent from both the interface and the API, so
summed clicks always land slightly below the headline number. Nothing recovers
them.

---

## Things that will confuse you if nobody says them

**Data arrives 2–3 days late.** Ask for today and you get almost nothing. Every
default window here already ends a few days back. Use `dataState=all` if you
want the fresh, still-moving tail.

**`ctr` is a fraction in JSON.** `0.0615` means 6.15%. CSV and Markdown convert
it for you; JSON gives you what the API gave us.

**`position` cannot be summed.** It is an average per row. Rolling it up needs
an impression-weighted mean, which is what `totals.position` does.

**Your totals will not match the Search Console UI exactly.** Google withholds
rows for queries it considers rare, to protect the privacy of the people who
typed them. Summed clicks therefore land slightly below the UI's headline
number. This is expected and cannot be worked around.

**A domain property and a URL-prefix property return different numbers.**
`sc-domain:example.com` aggregates every subdomain and both protocols;
`https://example.com/` covers only that exact prefix. They are separate
registrations, and querying the wrong one gives numbers that look plausible but
are not what you meant.

**`searchAppearance` cannot be combined with any other dimension.** The API
refuses it. Query it on its own, then run a second query for the rest.

**AI Overviews and AI Mode data is not in the API at all.** It is blended into
the `web` totals and cannot be separated. See
[What the API cannot give you](#what-the-api-cannot-give-you).

**The `hour` dimension needs `dataState=hourly_all`.** Without it Google returns
an empty result rather than an error, which reads as "no traffic" and is badly
misleading. This service rejects the combination up front instead.

---

## Troubleshooting

| Symptom | What it means | Fix |
| --- | --- | --- |
| `The service account credentials were rejected` | The key is wrong, rotated, or truncated on paste. `invalid_grant` also appears when the machine clock is far out of sync. | Re-encode the key with `base64 -w0`. Run `pnpm check-access`. |
| `The service account cannot read <property>` | Either it has no access, or the property string does not match the registration. | `GET /sites` and copy one of the listed strings verbatim. |
| `No Search Console property matches …` | The property does not exist under this account. | Same as above. |
| `searchAppearance cannot be combined…` | Two dimensions were requested where the API allows only one. | Split it into two queries. |
| `startDate … is beyond the 16-month retention window` | The range predates what Google keeps. | Move the start date forward. |
| Empty rows, no error | The window is inside the freshness lag, or a filter matched nothing. | Move `endDate` back, or drop the filter. |
| `Search Console rate limit hit` after retries | The daily project quota is spent. | Wait, raise `CACHE_TTL_SECONDS`, or check the Cloud console quotas page. |

Set `DEBUG=true` to log every pagination step and retry.

---

## Docker

```bash
docker compose up -d
```

The compose file publishes on `127.0.0.1:8788` only. To expose it on the
network, change the port mapping to `"8788:8788"` **and** set `API_TOKEN` in
`.env` first.

A health check is built in, so `docker ps` reports the container unhealthy if
the API stops answering.

Request logs go to stdout, so they land wherever your Docker logging driver
points:

```bash
docker compose logs -f search-console-api
```

The compose file caps the JSON log driver at three 10MB files, so logs cannot
fill the disk. Set `LOG_LEVEL=error` in `.env` to record only failures.

---

## Development

```bash
pnpm install
pnpm dev          # server with reload
pnpm test         # node:test, no framework dependency
pnpm typecheck    # strict TypeScript, tests included
pnpm build        # compile to dist/
```

TypeScript, ESM, Node 22+, strict mode with `noUncheckedIndexedAccess`. The only
runtime dependencies are `@googleapis/searchconsole`, `google-auth-library`,
`hono`, `@hono/node-server`, `@modelcontextprotocol/sdk` and `zod` — the
per-API Google package rather than the umbrella `googleapis`, which bundles
every Google API surface for about 200MB.

```
src/server/
  main.ts              HTTP entry point
  cli.ts               CLI entry point
  check-access.ts      Credential and permission preflight
  config.ts            Environment parsing, key decoding, property normalisation
  google-client.ts     One shared JWT
  http/
    app.ts             Routes, auth, error mapping
    openapi.ts         Generated OpenAPI 3.1 document
  mcp/
    server.ts          Tool definitions
    stdio.ts           MCP entry point
  lib/
    search-analytics.ts  Query, pagination, retry
    compare.ts           Two-window diff
    dates.ts             Range resolution and validation
    format.ts            JSON, CSV, Markdown rendering
    export.ts            Multi-section reports
    request.ts           Shared parameter parsing
    service.ts           Query layer bound to one client
    cache.ts             In-memory response cache
    errors.ts            Failure classification
    limits.ts            Quota reference
```

---

## License

MIT — see [license.md](license.md).

Built by [@ramazansancar](https://github.com/ramazansancar).