# KIRA — Knowledge-Indexed Registry for Analytics

A statistics-literate MCP server for Malaysian official data.

*kira* (Malay): to count, to reckon. *kira-kira*: calculation.

**Malaysian official statistics, with the caveats attached.**

Existing Malaysian open-data MCP servers are couriers: they fetch a table
and hand it over. KIRA resolves a question to the right series, applies
the correct transformation in code, refuses invalid comparisons, and
returns every number with its provenance. See [`PROJECT_PLAN.md`](PROJECT_PLAN.md)
for the full design and phase-by-phase progress log.

**Unofficial.** Not affiliated with DOSM, JDN, or the Ministry of Digital.
Data sourced from [data.gov.my](https://data.gov.my) / [open.dosm.gov.my](https://open.dosm.gov.my),
attributed per dataset in every tool response's `provenance` block.

## Why: three real cases, not a feature list

Full write-up with the actual tool calls in [`eval/baseline_comparison.md`](eval/baseline_comparison.md)
— run against the hosted `mcp-datagovmy` baseline, not asserted.

1. **The same query, with or without a warning.** Asking for Malaysia's
   monthly unemployment rate across a range that spans DOSM's January
   2025 census-baseline shift returns identical raw numbers from either
   server — DOSM documents the break in prose, in a *separate* metadata
   call. The baseline's data-fetching call carries none of that. KIRA's
   `get_series` attaches the warning to the exact same query,
   automatically, every time.
2. **A refusal instead of a plausible wrong comparison.** `compare_series`
   refuses to plot PPI's seasonally-adjusted and unadjusted series
   against each other, naming both series and why. The baseline has no
   such gate — nothing stops that comparison from happening.
3. **DOSM's own published growth rate, not a re-derived one.** `get_series`
   and `compute` use the agency's own `growth_yoy`/`growth_mom` rows for
   GDP, PPI, IPI, and trade where DOSM publishes them, rather than
   recomputing from levels — verified to match DOSM's figures exactly
   (see `tests/test_engine.py`), which matters most for chain-linked real
   GDP, where a naive level-ratio calc doesn't reproduce the official
   number.
4. **A real gap, found and fixed, not hidden.** `find_series` used to
   return nothing for homicide/crime queries and imply the data didn't
   exist — it does (`homicide_rate` now), and DOSM publishes plenty more
   that KIRA still hasn't curated. `find_series` now says so on a miss
   and points at `raw_query_url`, which reaches any DOSM/data.gov.my
   Parquet directly, sandboxed the same way as `raw_query`, no
   registration required. This is the one area where the baseline's
   full-catalogue search genuinely beats a curated-only design — worth
   stating plainly rather than glossing over.

## Status

**All 10 originally-planned tools are built, plus one added to close a
real coverage gap**: `find_series`, `describe_series`, `get_series`,
`compare_series` (with its SA/NSA refusal rule), `compute`
(cagr/real/per_capita/index_to/contribution_to_growth, plus yoy/mom/qoq),
`explain_break`, `latest_release`, `raw_query` (sandboxed SQL escape
hatch for dimensional breakdowns a registered series doesn't curate,
e.g. GDP by sector), `raw_query_url` (the same escape hatch for a DOSM
dataset with *no* registry entry at all — `find_series` only searches
KIRA's curated series, a fraction of what DOSM publishes; an empty
result now says so and points here instead of implying the data doesn't
exist), `list_geographies` (state/district/parlimen/DUN hierarchy), and
`lookup_classification` (MCOICOP, MSIC — hierarchy traversal, not just
single-code lookup).

21 series registered, covering every category in the build order (prices,
national accounts, labour, production, external trade, demography):
`cpi_headline`, `cpi_core`, `ppi_headline` + `ppi_headline_sa`,
`ipi_headline` + `ipi_headline_sa`, `gdp_real` + `gdp_real_state`,
`gdp_nominal`, `labour_unemployment_rate` +
`labour_unemployment_rate_state`, `labour_force_participation_rate`,
`trade_exports`, `trade_imports`, `trade_balance`, `trade_total`,
`population_malaysia`, `household_income_median`, `vital_birth_rate`,
`vital_death_rate`, `homicide_rate`. See `PROJECT_PLAN.md` section 6 for
the phase plan, `eval/questions.yaml` for the Phase 0 eval set (27
questions, 26 graded), and `eval/baseline_comparison.md` for the
head-to-head above, in full.

HTTP transport, Docker packaging, CI (tests + nightly source-drift
check), in-process caching, and per-client rate limiting are done.
Remaining: more series toward ~40, a hosted endpoint, submitting to MCP
registries (Smithery, Glama, the GitHub MCP registry), and a dedicated
adversarial security review (self-review only so far) — see
PROJECT_PLAN.md's progress log for exact status.

Package name: **`kira-dosm`** — confirmed. `kira` and `kira-mcp` are
already taken on both PyPI and npm by unrelated projects.

## Quickstart

```
uv sync
uv run pytest                 # -m "not network" to skip live-data tests
uv run python -m kira.server  # stdio MCP server (default — for Claude Desktop, `mcp dev`, etc.)
```

Point an MCP client's config at `uv run --directory /path/to/KIRA python -m kira.server`
(stdio) to use it locally.

```python
# Ad-hoc query against a registered series:
from kira.registry import load_registry
from kira.engine import get_connection, get_series

reg = load_registry()
con = get_connection()
get_series(reg["cpi_headline"], con, date_from="2025-06", date_to="2025-06", transform="yoy")
```

### HTTP transport

```
KIRA_TRANSPORT=http KIRA_PORT=8000 uv run python -m kira.server
```

Serves streamable-HTTP MCP at `http://127.0.0.1:8000/mcp`. Set
`KIRA_HOST=0.0.0.0` to listen on all interfaces (e.g. inside a container).

The HTTP transport is rate limited per client IP —
`KIRA_RATE_LIMIT_PER_MINUTE` (default 60, set `0` to disable) — since a
public endpoint otherwise has no defense against one client hammering it
straight through to DOSM's live storage. Results are also cached
in-process for `KIRA_CACHE_TTL_SECONDS` (default 3600 = 1 hour; DOSM's
data doesn't change faster than that) — set `0` to disable, which is
what the test suite does. Neither applies to stdio: there's only ever
one client, and each session is a fresh process.

### Docker

```
docker build -t kira-dosm .
docker run -p 8000:8000 kira-dosm
```

Runs the HTTP transport by default (`KIRA_TRANSPORT=http` is baked into
the image). The container needs outbound HTTPS to `storage.dosm.gov.my`
— it queries DOSM's live Parquet files on every request, it doesn't
bundle or cache data.

## Layout

- `registry/*.yaml` — one file per series, the actual product (see `registry/SCHEMA.md`)
- `src/kira/models.py` — the registry schema (pydantic)
- `src/kira/registry.py` — loads + validates `registry/*.yaml`
- `src/kira/engine.py` — DuckDB query + transforms (yoy/mom/qoq) + provenance/warnings
- `src/kira/lookups.py` — geography and classification lookup tables (`list_geographies`, `lookup_classification`)
- `src/kira/cache.py` — in-process TTL cache in front of every DOSM query (`KIRA_CACHE_TTL_SECONDS`)
- `src/kira/ratelimit.py` — per-client-IP rate limiting for the HTTP transport (`KIRA_RATE_LIMIT_PER_MINUTE`)
- `src/kira/server.py` — FastMCP server: 11 tools (`find_series`, `describe_series`, `get_series`, `compare_series`, `compute`, `explain_break`, `latest_release`, `raw_query`, `raw_query_url`, `list_geographies`, `lookup_classification`)
- `eval/questions.yaml` — Phase 0 eval set, each answer independently verified against live DOSM data
- `eval/baseline_comparison.md` — head-to-head against the `mcp-datagovmy` baseline
- `scripts/check_sources.py` — fetches every registry `source_url` and fails loudly on 404 or schema drift; runs nightly via `.github/workflows/nightly-source-check.yml`
- `.github/workflows/ci.yml` — tests + registry validation + Docker build on every push/PR

## Licence

MIT — see `LICENSE`. Data itself is DOSM's, under the terms of
[data.gov.my](https://data.gov.my)'s open data licence (CC BY 4.0 for the
datasets checked so far); this project adds no additional restriction on
top of that. Don't use DOSM's name, logo, or crest in any derivative
branding — KIRA is unofficial and says so in every response.