# kyiv-alerts

A standalone recorder for Kyiv City air-raid alert history, with a read-only
MCP query surface. Built to make air alerts a first-class variable in sleep and
recovery analysis instead of an invisible confounder.

It owns its own SQLite database and reads nothing else. One container runs both
halves: a collector and an MCP server.

Ingests from four independent sources — two of which need no API key, so it
works the moment you start it. Answers questions like *"was there an alert
during this sleep window, and how many minutes of it overlapped?"* — and,
crucially, tells you when it doesn't know.

> **Not a warning system.** This is a recorder for retrospective analysis. It
> has no notifications and makes no real-time guarantees. For actual air-raid
> warnings use official channels and sirens.

**Contents:** [Quick start](#quick-start) · [Tools](#tools) ·
[Sources](#the-four-sources) · [Backups](BACKUP.md) ·
[Configuration](#configuration) · [Tests](#tests)

## The property that matters

**An absent alert record is not the same as absent data.**

Strikes cause the power and internet outages that kill the collector, so
downtime is correlated with exactly the nights whose data matters most. If a
query surface cannot distinguish "no alert" from "we weren't listening", every
statistic built on it is biased in a direction you cannot detect afterwards.

So every tool returns a `coverage` verdict — `full`, `partial`, or `none` —
with the gap intervals when partial. An empty alert list is evidence of a quiet
night **only** when coverage is `full`.

## Quick start

No API keys are needed to start recording. Two of the four sources —
`neptun.in.ua` and `ubilling.net.ua` — are open and on by default.

```bash
cp .env.example .env
```

Set `MCP_AUTH_TOKEN` (the only required value), add the two optional API keys
if you have them, then:

```bash
docker compose up -d --build
```

Generate an MCP token with:

```bash
python -c "import secrets; print(secrets.token_urlsafe(32))"
```

Check it came up:

```bash
curl -s localhost:8000/health
```

Follow the structured logs:

```bash
docker compose logs -f kyiv-alerts
```

### The four sources

| Source | Key | Transport | Backfill | Contributes |
| --- | --- | --- | --- | --- |
| **neptun.in.ua** | none | WebSocket push | no | alerts, **`kind`**, explosions (opt-in) |
| **ubilling.net.ua** | none | poll 30s | no | alerts (third observer) |
| **raid.fly.dev** | on request | TCP push | **yes** — to 2022 | alerts, full history |
| **alerts.in.ua** | on request | poll 30s | ~30 days | alerts, shelling records |

Only **raid.fly.dev** can backfill. Without it the dataset begins the day you
start the collector, and earlier windows correctly report `coverage: none`.

- **raid.fly.dev** — email `a@dun.ai` or Telegram `@andunai`, include `#api`.
- **alerts.in.ua** — form at <https://alerts.in.ua/api-request>. It explicitly
  rejects one-line and LLM-written requests; describe your actual use and
  request rate in your own words.

All four are run by volunteers. The defaults are deliberately polite: push
connections where offered, 30s polling elsewhere, and the expensive
full-history endpoint is locally floored at one request per 5 minutes with the
last *attempt* persisted, so neither a crash-restart loop nor a flapping link
can turn into a hammering loop.

NEPTUN's terms require a visible attribution link wherever its data is
displayed. Nothing here has a UI, but if you ever surface this data publicly:
`Дані: Карта повітряних тривог — NEPTUN (https://neptun.in.ua/)`.

## Connecting a client

Streamable HTTP at `http://<host>:8000/mcp`, bearer token in the `Authorization`
header.

```json
{
  "mcpServers": {
    "kyiv-alerts": {
      "type": "http",
      "url": "http://localhost:8000/mcp",
      "headers": { "Authorization": "Bearer YOUR_MCP_AUTH_TOKEN" }
    }
  }
}
```

The port is published on `127.0.0.1` by default. If you expose it to your LAN
via `MCP_BIND`, put TLS in front of it — the bearer token is otherwise sent in
clear text.

### Running without a token

If the server is already protected at the network layer — a private path,
geo/IP restriction, a VPN — set:

```
MCP_AUTH_MODE=none
```

`/mcp` then serves unauthenticated, and a warning is logged on every start so
an open server never becomes a forgotten surprise.

`MCP_AUTH_MODE` is the *only* way to disable auth. An empty or misspelt
`MCP_AUTH_TOKEN` makes the service refuse to start rather than fall open, so a
typo in `.env` cannot quietly expose your data. Worth knowing what an open
server discloses: when you are home, when you sleep, and when you were awake
at 3am — a presence-and-routine profile tied to a real address in a war zone.
That is the thing the network layer has to actually be stopping.

## Tools

All read-only. All timestamps in and out are `Europe/Kyiv` local time; storage
is UTC.

### `alerts_window(from_local, to_local)`

Alert intervals overlapping the window: start, end, duration, kind, explosions
flag, contributing sources — plus the coverage verdict.

### `alerts_for_sleep(onset_local, wake_local)`

The primary tool.

- `alerts_before_onset` — alerts that ended within 6 hours before onset, each
  with `minutes_before_onset`. These are what delay bedtime.
- `alerts_in_window` — alerts overlapping the sleep itself, each with
  `minutes_overlap`, `started_before_onset`, `ended_after_wake`. These are what
  fragment sleep.
- `total_min_in_window` — summed overlap, de-duplicated across sources.
- `explosions_reported` — `true`, `false`, or `null` when unknown.
- `coverage` and `coverage_pre_onset_lookback` — verdicts plus gaps.

All overlap arithmetic is done server-side.

### `ingest_status()`

Last successful ingest per source, current connection state, total disconnect
minutes over the last 7 days, and coverage gaps over the last 30 days.

## Data sources, as they actually are

Documentation for both services is out of date in ways that matter. What is
implemented here reflects probing them directly (August 2026).

| Claim in the docs | Reality |
| --- | --- |
| TCP at `tcp.alerts.com.ua:1024` | That name resolves to Cloudflare and times out. The Fly app answers on `raid.fly.dev:1024`. |
| `alerts.com.ua` serves the API | It publicly serves only `/api/states`; `/api/history` and `/api/states/live` 404 there. The full API is on `raid.fly.dev`. |
| Keepalives are `ping` | They are `p:<random 0-9999>`. |
| Packets end with "ASCII 0x10 (`\n`)" | 0x10 is DLE; the wire uses 0x0A. |
| `/api/history` gives history | It is an unfiltered full dump of all 25 regions since 2022-03-15, no range parameters, 1 request/minute. |
| alerts.in.ua has media-derived explosion events | No such endpoint exists. The only signal is an alert whose `alert_type` is `artillery_shelling` or `urban_fights`. |
| alerts.in.ua history | Accepts only `week_ago` / `month_ago` — it cannot backfill deeper than ~30 days. |

Region identity is verified at startup rather than trusted: Kyiv City is
`state_id` **25** (`name_en: "Kyiv"`) on raid.fly.dev and `location_uid` **31**
on alerts.in.ua. Kyiv *oblast* is 9 / 14 and is a different place.

**Health warning on raid.fly.dev:** as of August 2026, 22 of its 25 regions have
`changed` timestamps frozen since October–December 2025 (Luhansk still reads
`alert=true` since 2023). Kyiv City is current, but the feed shows signs of
partial degradation, which is why both sources run concurrently with independent
coverage tracking rather than one being a passive backup.

### `kind` and `explosions`

Neither raid.fly.dev nor alerts.in.ua can fill these: the first is a boolean,
and the second's types distinguish air raid from shelling, not weapon class.
**NEPTUN is what makes both possible.**

**`kind`** comes from NEPTUN threat tracks overlapping the alert, mapped from
its `type` field:

| NEPTUN type | `kind` |
| --- | --- |
| `uav`, `recon` | `drone` |
| `missile`, `ballistic`, `kab`, `mig31k` | `ballistic` |
| both classes seen during one alert | `mixed` |
| no tracks observed | `unknown` |

A track counts only if it is in `м. Київ` or within `NEPTUN_THREAT_RADIUS_KM`
(default 50) of the city centre. Kyiv *oblast* alone does not qualify — Біла
Церква is ~75 km out, and counting it would attribute a city alert to a threat
the other side of the region. `advisory: true` tracks (observations such as a
MiG-31K takeoff, not shelter signals) are excluded.

**`explosions`** is three-valued and only ever leaves `NULL` when a source that
actually reports explosions covered the *whole* alert:

- `1` — a shelling record (alerts.in.ua) or an explosion report (NEPTUN
  messages) overlapped the alert.
- `0` — such a source was live for the entire alert and reported nothing.
- `NULL` — no explosion-reporting source covered it. Coverage from an
  alerts-only source does not count, because it is not evidence either way.

**Kyiv oblast counts for explosions, not for alerts.** `EXPLOSIONS_INCLUDE_OBLAST`
(default `true`) also accepts reports from Київщина / Київська обл., because
detonations and air-defence work out there are routinely audible across the
city and therefore do disturb sleep. Alert *intervals* stay strictly
city-level — an oblast alert is not a city alert, and merging them would
inflate `total_min_in_window` on nights Kyiv itself was quiet. Set it `false`
for strictly within-city events.

Populating it requires `NEPTUN_MESSAGES_EXPLOSIONS=true`, which is **off by
default**: it parses text from monitoring Telegram channels. NEPTUN republishes
those over HTTP so nothing scrapes Telegram, but it is still Telegram text and
the original brief excluded that — hence an explicit opt-in. Matching requires
both a Kyiv reference and an event word (`вибух`, `приліт`, `працює ППО`,
`детонац`); forecasts such as "курсом на Київ" and `Київщина` (the oblast) are
deliberately excluded.

## How it works

```
raid.fly.dev TCP  ──push──┐
                          ├──▶ transitions (append-only truth)
alerts.in.ua poll ──30s───┘         │
                                    ▼
                                 alerts (derived, idempotent)
                                    │
history backfill ──▶ coverage ──────┴──▶ MCP tools ──▶ verdict + data
```

- **`transitions`** is append-only and the only source of truth. Idempotency
  comes from `UNIQUE(region, state, ts_utc, source)`.
- **`alerts`** is derived from it and can be rebuilt at any time. Consecutive
  same-state transitions collapse to the earliest, which is what lets a
  connect-time snapshot be superseded by the true start that backfill supplies.
- **`coverage`** records intervals where ingest was demonstrably live. Rows are
  written when a session opens and advanced in place on each heartbeat, so a
  power cut loses at most one heartbeat rather than the whole session.

Reconnect uses jittered exponential backoff capped at 60s and retries
indefinitely. Silence beyond 45s (three missed pings) is treated as a dead link.
Every startup and every reconnect triggers a backfill from the last recorded
coverage timestamp to now.

Both sources see the same siren seconds apart and both rows are kept for
forensics; the query layer reads a merged view so `total_min_in_window` is never
double counted.

## Configuration

| Variable | Default | Purpose |
| --- | --- | --- |
| `RAID_API_KEY` | — | raid.fly.dev key. Enables that source. |
| `ALERTS_IN_UA_TOKEN` | — | alerts.in.ua token. Enables that source. |
| `MCP_AUTH_TOKEN` | — | Bearer token for `/mcp`. Required unless auth is off. |
| `MCP_AUTH_MODE` | `token` | `none` serves unauthenticated (explicit opt-in only). |
| `DB_PATH` | `/data/alerts.db` | SQLite file, on the mounted volume. |
| `REGION` | `kyiv_city` | Tracked region. |
| `MCP_HOST` / `MCP_PORT` / `MCP_PATH` | `0.0.0.0` / `8000` / `/mcp` | Server binding. |
| `RAID_TCP_HOST` / `RAID_TCP_PORT` | `raid.fly.dev` / `1024` | Live stream endpoint. |
| `RAID_HTTP_BASE` | `https://raid.fly.dev` | HTTP API base. |
| `RAID_SILENCE_TIMEOUT_S` | `45` | Dead-link threshold. |
| `RAID_HISTORY_MIN_INTERVAL_S` | `300` | Local floor on full-dump fetches. |
| `AIU_POLL_INTERVAL_S` | `30` | alerts.in.ua poll cadence. |
| `AIU_HISTORY_INTERVAL_S` | `1800` | History reconciliation cadence. |
| `NEPTUN_ENABLED` | `true` | Keyless WebSocket source; also supplies `kind`. |
| `NEPTUN_THREAT_RADIUS_KM` | `50` | Threat proximity to the city centre. |
| `NEPTUN_MESSAGES_EXPLOSIONS` | `false` | Opt-in explosion reports (Telegram text over HTTP). |
| `EXPLOSIONS_INCLUDE_OBLAST` | `true` | Count oblast explosions — audible in the city. |
| `UBILLING_ENABLED` | `true` | Keyless third observer. |
| `ONGOING_ALERT_SUSPECT_HOURS` | `12` | Flag alerts open longer than this as suspect. |
| `COVERAGE_STITCH_S` | `60` | Gaps below this are heartbeat jitter, not data loss. |
| `LOG_LEVEL` | `INFO` | Structured JSON logs to stdout. |
| `MCP_BIND` | `127.0.0.1` | Host interface the port is published on. |

Only `MCP_AUTH_TOKEN` is mandatory. The service refuses to start if every
source is disabled, rather than run as a healthy-looking recorder of nothing,
and warns at startup when no `RAID_API_KEY` is present because that is the
only source that can backfill.

## Tests

```bash
pip install -e ".[dev]"
pytest
```

The suite covers the acceptance criteria directly, against a fake raid TCP
server that speaks the real protocol and a real MCP client over HTTP:

1. `test_reconnect_backfill.py` — killing the connection mid-stream reconnects,
   backfills, and leaves the gap absent from `coverage`; the inverse case
   asserts an un-backfilled outage stays visible.
2. `test_backfill_idempotent.py` — repeated backfills and re-derivations produce
   zero duplicate rows.
3. `test_sleep_overlap.py` — partial overlap for an alert straddling onset, plus
   wake-straddling, ongoing alerts, and cross-source de-duplication.
4. `test_coverage_verdict.py` — a window with no data returns `none`, never an
   empty list that reads as quiet.
5. `test_dst.py` — local timestamps round-trip across the October transition;
   the repeated 03:00 hour stays two distinct instants and a transition night
   measures 9 real hours, not 8.

## Operational notes

**Bind-mount ownership.** The container runs as uid `10001`, but `./data`
arrives owned by whoever created it — usually root — so the collector cannot
write to it. On a fresh host, before the first start:

```bash
mkdir -p data && sudo chown -R 10001:10001 data
```

Startup now preflights this and prints that exact command if the directory is
not writable, instead of failing later with SQLite's opaque "unable to open
database file".


- SQLite runs in WAL mode; the database lives on a bind mount at `./data`, not
  in the container.
- Restart policy is `unless-stopped`.
- Full history is kept forever — a few thousand rows per year.
- `/health` is unauthenticated; everything else requires the bearer token.
- **Back it up.** The history cannot be re-fetched beyond ~29 days. See
  [BACKUP.md](BACKUP.md).

## Updating a deployment

If you deployed by copying files rather than cloning, attach the directory to
the remote in place. `.env` and `data/` are gitignored, so they survive — but
`docker-compose.yml` is tracked, so check for local edits before overwriting.

```bash
cd /path/to/kyiv-alerts

# 1. Back up the database first. It cannot be re-fetched. See BACKUP.md.
docker compose exec -T kyiv-alerts python -c "
import sqlite3, datetime
s = sqlite3.connect('/data/alerts.db')
d = sqlite3.connect('/data/pre-update-' + datetime.date.today().isoformat() + '.db')
s.backup(d); d.close(); s.close()"

# 2. Attach to the remote without touching the working tree.
git init
git remote add origin https://github.com/YoYoZ/kyiv-alerts.git
git fetch origin
git reset origin/main          # moves HEAD only; files untouched

# 3. Inspect what your server has that the repo does not.
git status
git diff
```

Step 3 is the one that matters. If `git diff` shows edits you want to keep
(ports, volumes, anything in `docker-compose.yml`), save them first:

```bash
git stash push -m "server-local config"
git checkout -B main origin/main
git stash pop                  # replay your edits, resolve any conflict
```

If the diff is only noise you do not care about, take the repo version:

```bash
git checkout -B main origin/main
git reset --hard origin/main   # tracked files only; .env and data/ are safe
```

Confirm your secrets and history really did survive before restarting:

```bash
test -f .env && echo ".env present"
ls data/alerts.db && echo "database present"
```

Then rebuild and verify:

```bash
docker compose up -d --build
curl -s localhost:8000/health
```

From then on, updating is:

```bash
git pull && docker compose up -d --build
```

### If you would rather start clean

Clone fresh and carry the state across. Note that Compose derives its project
name from the directory name, so keep the final path identical or expect
renamed containers:

```bash
git clone https://github.com/YoYoZ/kyiv-alerts.git /tmp/kyiv-alerts-new
cp /path/to/kyiv-alerts/.env /tmp/kyiv-alerts-new/
cp -a /path/to/kyiv-alerts/data /tmp/kyiv-alerts-new/
cd /path/to/kyiv-alerts && docker compose down
mv /path/to/kyiv-alerts /path/to/kyiv-alerts.old
mv /tmp/kyiv-alerts-new /path/to/kyiv-alerts
cd /path/to/kyiv-alerts && docker compose up -d --build
```

Keep `.old` until you have confirmed `ingest_status()` looks right.

## Troubleshooting

### An alert is reported as `ongoing` for an implausibly long time

Queries flag any alert still open past `ONGOING_ALERT_SUSPECT_HOURS`
(default 12) with `long_running_suspect: true`, and `ingest_status()` lists
them under `long_running_suspect_alerts`. Real alerts do not run for days, so
treat a flagged row as a stale record, not exposure.

Rebuild the interval table from the transition log:

```bash
docker compose exec -T kyiv-alerts python -m kyiv_alerts.rederive
```

This is always safe — `alerts` is derived data and `transitions` is
append-only truth, so nothing is lost. It prints before/after counts and any
remaining open alerts. Derivation also runs on every startup and state change,
so a restart does the same thing.

Alerts are never auto-closed. Inventing an end time would fabricate data; a
suspect row is flagged and left for you to re-derive.

## Contributing

Issues and PRs welcome. Two things worth knowing before changing anything:

1. **Coverage is the load-bearing property.** Any change that lets a query
   return an empty result without an accurate verdict is a bug, however
   convenient. The tests in `test_coverage_verdict.py` exist to stop exactly
   that, and they are mutation-checked.
2. **`transitions` is append-only truth.** `alerts` is derived and rebuildable.
   Keep it that way — idempotent derivation is what makes repeated backfills
   safe.

Run `pytest` before opening a PR; the suite needs no network or API keys.

### Adapting to another region

The schema and query layer are region-agnostic; only `REGION_IDS` in
`config.py` is Kyiv-specific. Adding another Ukrainian region means supplying
its four identifiers (raid `state_id`, alerts.in.ua `uid`, ubilling name,
NEPTUN oblast key) and its coordinates. Note the collector records one region
per deployment — see the note in `config.py` on why region is configuration
rather than a per-request parameter.

## Credits

Data comes from four volunteer-run services. If you use this, consider
supporting them:

- [NEPTUN](https://neptun.in.ua/) — attribution required when displaying its
  data: `Дані: Карта повітряних тривог — NEPTUN`
- [alerts.in.ua](https://alerts.in.ua/)
- [raid.fly.dev / alerts.com.ua](https://alerts.com.ua/) by
  [Andrew Dunai](https://github.com/and3rson/raid)
- [ubilling.net.ua](https://ubilling.net.ua/aerialalerts/)

Respect their rate limits and terms. The defaults here are deliberately polite.

## Licence

MIT — see [LICENSE](LICENSE).