# vechemoga-mcp-server

An [MCP](https://modelcontextprotocol.io) server that lets an AI assistant read VecheMoga's
logs, query its database read-only, and inspect what the API actually sent to Loops and
Stripe — without leaving the editor.

**One server, two deployments, identical tool names.** The only difference is where the data
comes from:

| | local | dev AWS |
|---|---|---|
| URL | `http://localhost:8417/mcp` | `https://mcp.dev.vechemoga.bg/mcp` |
| Logs from | the Docker socket | CloudWatch Logs |
| Database | the local Postgres container | RDS, as `vechemoga_mcp_reader` |
| Auth | none (binds to localhost) | `Authorization: Bearer …` **behind** a security-group IP allowlist |
| Runs on | VecheMogaLocal's compose stack | the dev-tools EC2 box |

That is `MCP_LOG_BACKEND=docker|cloudwatch`, one environment variable. It is deliberately
one codebase: Sesame — the system this is ported from — has a local `server.py` and an AWS
`server.py` that have drifted into ~1400 and ~1800 lines of largely duplicated tool code.
**Do not fork this to add an environment. Add a backend.**

> **Dev only.** There is no prod MCP server, on purpose. A bearer token that can `SELECT`
> from the production database and read every production log stream is a materially
> different risk decision, and not one that should arrive as a side effect of this repo.

## Tools

Fifteen, all read-only. Tools whose source is not configured in a given deployment are not
registered at all — an assistant that cannot see `deploy_status` locally will pick another
route, whereas one that calls it and gets an error will not.

**PostgreSQL** — `pg_list_schemas`, `pg_list_tables`, `pg_describe_table`, `pg_query`,
`pg_migration_status`

`pg_query` takes a single `SELECT` / `WITH … SELECT` / `EXPLAIN SELECT`, adds a `LIMIT` when
there is none, caps at 200 rows, and runs under `SET TRANSACTION READ ONLY` with a 10s
statement timeout. **That is the second lock, not the first** — see [Read-only](#read-only)
below. `pg_migration_status` reads Liquibase's `databasechangelog`, which answers "did this
environment actually run the migration I merged?"

**Logs** — `api_logs`, `api_liquibase_logs`, `web_logs`, `wiremock_logs`, `postgres_logs`,
`pipeline_logs`, `automation_logs`

All take the same four arguments: `tail`, `since`, `until`, `grep`. `grep` is applied after
retrieval, so narrowing with `since`/`until` beats a large `tail`. `automation_logs` earns
its place: a green Allure report can hide a failed `Before`/`After` hook, and the CodeBuild
log is the only place that failure is visible.

**WireMock** — `wiremock_list_mappings`, `wiremock_find_requests`

`wiremock_find_requests` answers *did the API actually call Loops, and with what body?* —
which CloudWatch cannot, because WireMock's request logging is deliberately switched off so
that verification links never reach a log group. Strictly the read-only subset of the admin
API: no mapping registration, and **never** `/__admin/reset` or a `DELETE`. Parallel
Cucumber workers and manual developers share that service and a global reset takes their
work with it.

**Delivery** — `deploy_status`

`codepipeline:GetPipelineState` across the pipelines, including `vechemoga-infra`. Answers
"is dev running the commit I think it is?" and "is an infra change sitting at the approval
gate?" It cannot start, approve or retry anything.

## Read-only

Worth being precise about, because "the server only sends SELECTs" is not a security
property — it is a hope about a program.

1. **The PostgreSQL role is the guarantee.** `vechemoga_mcp_reader` holds `pg_read_all_data`
   — read on every schema and table including ones created later, and no write grant of any
   kind — with `default_transaction_read_only = on`. A bug in this repo cannot write to that
   database, because the credential it holds cannot. Verified against the real database:
   writes are refused with *"cannot execute … in a read-only transaction"*, and a schema
   created after the grant is readable without one.
2. **The security group is the second.** It permits a TCP connection from the dev-tools box
   and the API box. What that connection may do is decided in (1).
3. **The SQL guard in `pg_query` is the third**, and its job is to turn a mistake into a
   clear error rather than a permission denial.

The build refuses to push if (3) regresses — `buildspec-build.yml` drives `DELETE`, `UPDATE`,
`DROP`, `INSERT` and a stacked `SELECT 1; DROP TABLE` through it and fails if any is accepted.

## Why no port 80

The dev-tools security group allows 443 from a handful of named operator addresses and
nothing else. An HTTP-01 or TLS-ALPN-01 ACME challenge must be answerable from Let's
Encrypt's own validation addresses, which are not knowable in advance — either would force a
port open to `0.0.0.0/0` and demote that allowlist from a security-group boundary to an
application-layer matcher.

So Caddy proves the certificate over a **Route 53 DNS-01 challenge**, which needs no inbound
reachability at all. That is what `caddy/Dockerfile` is for (`xcaddy` with
`caddy-dns/route53`), and the build fails if the plugin is missing rather than letting the
box fall back to a challenge that cannot work.

**If certificate issuance fails, fix the Route 53 grant — do not open port 80.**

## What the server logs

Which tool ran and with which argument *names*. Never results. A `pg_query` answer or a
WireMock journal entry would put database rows and verification links into
`/vechemoga/dev/mcp-server`, where they would then live for 30 days — the exact hazard
WireMock's own request logging is kept off to avoid.

## Local development

The image is pulled from ECR by VecheMogaLocal's compose stack. To work on the server
itself:

```bash
pip install -r requirements.txt
```

```bash
MCP_LOG_BACKEND=docker \
MCP_DOCKER_CONTAINERS='{"api":"api","web":"web","postgres":"postgres","wiremock":"provider-proxy"}' \
POSTGRES_HOST=localhost POSTGRES_USER=vechemoga POSTGRES_PASSWORD=vechemoga POSTGRES_SSLMODE=disable \
WIREMOCK_ADMIN_URL=http://localhost:1080 \
python server.py
```

Then point your editor at `http://localhost:8417/mcp`. Those are compose **service** names:
the server tries an exact container name first and falls back to the
`com.docker.compose.service` label, so service names work and survive a project rename or a
replica-index change — which `vechemoga-local-api-1` would not.

**Do not test the endpoint with `curl`.** It speaks MCP, not REST; a `curl` against `/mcp`
tells you nothing except that TLS and auth work. Validate by asking your assistant to call
a tool, e.g. *"list the schemas in the dev database"*.

## Deployment

Two pipelines write to the dev-tools box, and this repo owns the compose file both of them
deploy into — exactly as `VecheMogaApi/deploy` owns the file the provider-mock pipeline
deploys into today:

| Pipeline | Deploys |
|---|---|
| `vechemoga-mcp-server` | `mcp-server` + `caddy` |
| `vechemoga-provider-proxy` | `provider-proxy` (WireMock) |

```
Source (GitHub main, CodeConnections)
  └─▶ Build   CodeBuild ARM · buildspec-build.yml
        docker build ×2 (server + caddy, native arm64) → smoke checks
        → ECR :<sha12>  (only if the checks passed)
  └─▶ Deploy  CodeBuild · buildspec-deploy.yml → infra/deploy-to-instance.sh
        SSM Run Command on the dev-tools box: pull the pinned tags, up, health-check
```

Deploys are pinned to the git SHA. Rollback is *Release change* on an older execution, or
`IMAGE_TAG=<old-sha> ./infra/deploy-to-instance.sh` from a laptop.

The Deploy stage **skips green** when the box is absent or stopped — it sleeps nightly
23:30–07:45 Europe/Sofia. A pipeline that goes red every evening for a working reason
teaches everyone to stop reading it.

### One-time, on the box

Both pipelines run `docker compose` against a **checkout of this repo on the box** — they
deploy images, they do not copy files. So the checkout has to exist before the first deploy,
exactly as `vechemoga-api` is cloned onto the API box. Via SSM Session Manager (no SSH):

```bash
sudo git clone https://github.com/simo21-ss/vechemoga-mcp-server.git /opt/vechemoga-devtools
sudo chown -R ec2-user:ec2-user /opt/vechemoga-devtools
```

**Since 2026-08-09 both deploy scripts `git pull --ff-only` this checkout** before running
`docker compose`, so a merged compose change reaches the box on the next deploy of either
pipeline. Nothing did that before, and the gap was invisible: the checkout sat at a 2026-08-07
commit with no `FETCH_HEAD` at all, and a raised memory limit stayed unapplied while the repo,
the pipeline and the deploy log all reported success. Anything shipped inside an *image* was
always current — only the file drifted, which is exactly why it went unnoticed.

The pull is **fatal on failure**, and the deploy scripts refuse to pull a checkout whose
`origin` is not this repo. `--ff-only` can fail only on a dirty tree or a diverged branch, both
of which mean somebody edited the box by hand — composing over a file in that state is the
thing being prevented. `deploy/.env` is gitignored and untouched.

`COMPOSE_DIR` is therefore `/opt/vechemoga-devtools/deploy` — the clone root plus this
directory. It is set by **Terraform** on both deploy projects rather than left to the
buildspecs, because CodeBuild resolves an environment variable from the build project
definition ahead of the buildspec, and the target box and its compose path have to move
together.

`deploy/.env` holds the non-secret wiring (the file is not committed; secrets are read from
SSM on the box at deploy time, by the instance role):

```bash
cat >> /opt/vechemoga-devtools/.env <<'EOF'
DB_HOST=veche-moga-db-dev.<...>.eu-central-1.rds.amazonaws.com
MCP_HOST=mcp.dev.vechemoga.bg
MCP_ALLOWED_IPS=0.0.0.0/0
MCP_LOG_GROUPS='{"api":"/vechemoga/dev/api","wiremock":"/vechemoga/dev/provider-proxy","web":"/aws/amplify/<app-id>","pipeline":"/vechemoga/cicd/dev/deploy","automation":"/vechemoga/cicd/automation/test"}'
MCP_PIPELINES=vechemoga-api-dev,vechemoga-web-dev,vechemoga-provider-proxy,vechemoga-automation,vechemoga-mcp-server,vechemoga-infra
PROVIDER_PROXY_IMAGE=<account>.dkr.ecr.eu-central-1.amazonaws.com/vechemoga/provider-proxy
EOF
```

**The single quotes around `MCP_LOG_GROUPS` are load-bearing.** `infra/deploy-to-instance.sh`
sources this file with `set -a; . ./.env`, and the shell performs quote removal — so an
unquoted `{"api":"…"}` reaches the container as `{api:…}`, which is not JSON, and the server
exits at import. Observed on the first real deploy, as a crash loop.

And two SecureStrings, created by hand so their values never enter Terraform state —
`/vechemoga/dev/MCP_AUTH_TOKEN` and `/vechemoga/dev/MCP_DB_PASSWORD`, plus the
`vechemoga_mcp_reader` role. Runbook: **VecheMogaInfra README → "Bring up the dev-tools
box"**.

## Infrastructure

The box, its security group, its IAM role, the ECR repositories and the pipeline are
Terraform-managed in **VecheMogaInfra** (`modules/dev-tools`, `modules/cicd-mcp`,
`global/ecr.tf`). This repo owns only what runs on the box. The design and its rollout are
in `VecheMogaInfra/docs/mcp-server-plan.md`.

## Routing: which server to ask

Both servers expose the **same tool names**. Only the target differs, which is what makes a
routing rule necessary rather than obvious.

| server | URL | reads |
|---|---|---|
| `vechemoga-local` | `http://127.0.0.1:8417/mcp` | the local compose stack (VecheMogaLocal) |
| `vechemoga-dev` | `https://mcp.dev.vechemoga.bg/mcp` | the dev environment — CloudWatch, RDS, pipelines |

**Default to local.** Reach for `vechemoga-dev` only when the question is explicitly about
dev, AWS, CloudWatch, a pipeline or the deployed environment — or when local genuinely
cannot answer it. Say which one you are using, in one short sentence, before the first tool
call, so the answer can be read in context.

Don't query both in one response unless the request is a comparison. If the intent is
ambiguous, ask one short question rather than guessing.

**Never test an MCP server with `curl`.** It speaks MCP, not REST: a `curl` tells you TLS
and auth work and nothing else. Validate by calling a tool.

### Things that save a wrong turn

- `pg_query` is read-only **at the database** — `vechemoga_mcp_reader` holds `pg_read_all_data`
  and no write grant. A single `SELECT`, capped at 200 rows, 10s statement timeout.
- On dev, `api_logs` reads **CloudWatch**. `docker logs` on that box returns nothing at all —
  every container there uses the `awslogs` driver.
- Locally, the log tools read **containers**. A service run outside Docker (`./run.sh no-api`)
  has none, and its logs are not reachable from the server.
- `automation_logs` (dev only) is where a failed `Before`/`After` hook shows up when the
  Allure report is green.
- `wiremock_find_requests` answers *"did the API actually call Loops, and with what body?"* —
  CloudWatch cannot, because WireMock's request logging is deliberately off so that
  verification links never reach a log group.
- `deploy_status` answers *"is dev running the commit I think it is?"* without leaving the
  editor. It cannot start, approve or retry anything.

## Adding the servers to your editor

```bash
claude mcp add --transport http vechemoga-local http://127.0.0.1:8417/mcp
```

```bash
claude mcp add --transport http vechemoga-dev https://mcp.dev.vechemoga.bg/mcp --header "Authorization: Bearer $(aws ssm get-parameter --profile vechemoga-terraform --region eu-central-1 --name /vechemoga/dev/MCP_AUTH_TOKEN --with-decryption --query Parameter.Value --output text)"
```

Deliberately **not** a committed `.mcp.json`. The dev entry carries a bearer token, which must
not enter a repo; and a committed local entry would show as a failed server for every
developer who is not currently running the local stack. User-scoped registration keeps both
problems away.

**The token is the only lock on the dev one**, since 2026-08-09. It used to sit behind a
per-operator `/32` in `mcp_allowed_ips` (`VecheMogaInfra` `envs/dev/terraform.tfvars`) and a
matching `MCP_ALLOWED_IPS` on the box; that pair was pinned to a residential address that moved
on a DHCP lease, in two places nothing kept in sync, and it failed silently every time — so it
was opened rather than maintained.

Treat the token accordingly: it is now complete read access to the dev database and every dev
log stream, with nothing behind it. Rotate it on any exposure —

```bash
aws ssm put-parameter --profile vechemoga-terraform --region eu-central-1 \
  --name /vechemoga/dev/MCP_AUTH_TOKEN --type SecureString --overwrite \
  --value "$(openssl rand -hex 32)"
aws codepipeline start-pipeline-execution --profile vechemoga-terraform \
  --region eu-central-1 --name vechemoga-mcp-server
```

then re-register the client with the new value. Narrowing back to an allowlist means editing
**both** places together — see `deploy/Caddyfile`, which keeps the matcher for exactly that.

## Verifying it works — and why `/health` cannot tell you

**Call a tool. Nothing short of that proves anything.**

This is not a style preference. On 2026-08-11 the dev deployment answered
`421 Invalid Host header` to *every* authenticated request for two days, while the pipeline was
green, the container healthy, the certificate valid, `/health` returning 200 and `/mcp`
correctly returning 401 without a token. All of those pass **before** the transport validates
the `Host` header, so none of them could ever have detected it. The endpoint was reported as
working on exactly that evidence, and it was not.

The cause: `deploy/docker-compose.yml` passed `MCP_HOST` to the **caddy** service and not to
**mcp-server**. The server feeds it into the MCP transport's DNS-rebinding allowlist, which
validates `Host` and defaults to localhost only — so behind Caddy every real request was
refused.

### Recognising it

| symptom | meaning |
|---|---|
| `421 Invalid Host header` | `MCP_HOST` is missing or wrong on the **mcp-server** container |
| `401` | token mismatch — the request reached the server and it read your header |
| `404`, 9 bytes, microsecond duration in Caddy's log | Caddy's `remote_ip` matcher blocked it; the allowlist pair is out of sync |
| connection hangs, no TLS | the security group dropped it |

Four different failures, four different signatures. None of them says "you are not allowed".

### Checking it from the box

Runs against the real chain — TLS, Caddy, the Host allowlist, auth, the tool — with the token
read on the instance by its own role, so it never leaves the box:

```bash
TOKEN=$(aws ssm get-parameter --region eu-central-1 --name /vechemoga/dev/MCP_AUTH_TOKEN \
  --with-decryption --query Parameter.Value --output text)
curl -s --resolve mcp.dev.vechemoga.bg:443:127.0.0.1 -X POST https://mcp.dev.vechemoga.bg/mcp \
  -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/event-stream' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"deploy_status","arguments":{}}}'
```

`--resolve` is what makes it meaningful: it sends the **public** hostname to local Caddy, which
is the request shape that was failing. Pointing at `localhost` instead proves nothing, because
`localhost` is allowlisted unconditionally.

### What now prevents it

- **The server refuses to start** when `MCP_LOG_BACKEND=cloudwatch` and `MCP_HOST` is unset,
  with a message naming the symptom and the fix. Absence is now loud.
- **The build asserts against the public host** (`buildspec-build.yml`, smoke 3): `200` for
  `Host: mcp.smoke.invalid`, and `421` for an unlisted host so the protection cannot be
  satisfied by setting `MCP_HOST=*`. That covers a *wrong* value, which the startup guard does
  not. The pre-existing smoke test did call a tool — over `localhost` — and was structurally
  blind to all of this.