io.github.raheb77/saudi-open-data-mcp
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@io.github.raheb77/saudi-open-data-mcpsearch inflation datasets from SAMA"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
saudi-open-data-mcp
saudi-open-data-mcp is a production-minded MCP server for Saudi open data sources.
Quickstart
git clone https://github.com/raheb77/saudi-open-data-mcp.git
cd saudi-open-data-mcp
uv sync --no-editable
export HTTP_AUTH_TOKEN="$(openssl rand -hex 32)"
uv run --no-editable saudi-open-data-mcp --version
uv run --no-editable saudi-open-data-mcp check-startup
uv run --no-editable saudi-open-data-mcp listLive refresh and query paths depend on current upstream source availability and
local snapshots. For evaluation, start with the startup and catalog commands
above; then run uv run --no-editable saudi-open-data-mcp refresh only when
testing live source access.
The project is not just an MCP wrapper around upstream websites. Its value is in the layers underneath MCP:
source isolation through explicit connectors
typed normalization and canonical record contracts
registry-backed dataset metadata and health metadata
deterministic AI-facing resource and tool interfaces
Current implementation now includes curated official-source coverage across SAMA, stats.gov.sa, Ministry of Finance, and one narrow data.gov.sa pilot dataset. The current baseline is an internal, container-first MCP service with stdio still available for local development and command-based host integration.
The repository also includes an Arabic RTL dashboard package under dashboard/.
That package remains optional, but on main it is now a thin live consumer of
the governed backend over /mcp and /startupz with /readyz kept as a
startup-only compatibility alias, not a separate backend or required runtime
dependency for the core.
Related MCP server: datagovma-mcp
License and Release Boundaries
The code in this repository is licensed under the Apache License, Version 2.0.
Model weights are not included in this repository.
Model weights are not licensed under Apache-2.0 by default.
Any future release of model weights will be governed separately.
See ARCHITECTURE.md for the architecture, ADR-001 for the initial source decision, GOVERNANCE.md for the current core auth/audit/data-access model, OPERATIONS.md for runtime and durability guidance, DEPLOYMENT.md for the current local/container/runtime topology, RUNBOOKS.md for concise failure/recovery handling, PERSISTENCE.md for current persistence and backup/restore boundaries, and CHANGELOG.md for current baseline change visibility and migration notes.
What The System Does
isolates approved official Saudi data sources behind explicit connectors
normalizes a narrow supported dataset set into typed contracts
keeps dataset metadata and health metadata in a registry-backed model
exposes a small governed MCP surface for search, metadata, health, preview, query, download, and controlled materialization
provides a thin local CLI over the same core
produces governed institutional artifacts from query results through JSON, Excel-compatible XML, and text-first PDF export
What The System Does Not Do
it is not a generic BI suite or reporting platform
it is not a broad national data lake or a crawler for arbitrary sites
it does not add LLM rewriting, semantic search, or AI-generated narrative reporting in the core path
it does not make
query_dataseta live remote query surfaceit does not claim full public-internet deployment maturity or disaster-recovery automation
the dashboard is not a separate backend, control plane, or canonical institutional export path
Why It Is Trustworthy
source boundaries are explicit and approved-source-only
normalized outputs use typed contracts rather than free-form transport payloads
registry-backed metadata keeps descriptors, health state, and caveats out of ad hoc request-time logic
query_datasetanddownload_datasetstay local-only, whilepreview_datasetis the only hybrid path and exposes freshness/origin/degradation context explicitlyHTTP auth, narrow RBAC, structured audit logs, and current-state governance docs are already in place
deployment, runbook, and persistence expectations are documented explicitly rather than implied
Current System Surfaces
Surface | Purpose | Current state |
Backend/core | Governed MCP service over | primary runtime |
CLI | Thin operator/engineer façade over the same core | supported |
Dashboard | Arabic RTL UI package under | optional live consumer of |
Exports | Institutional artifacts over governed query results | CLI-governed path today |
Deployment Fit Today
The current deployment fit is intentionally narrow and practical:
internal or evaluator-controlled deployment
self-hosted or sovereign-hosting-friendly environments that can run one Python service or container and manage local persistent storage
operator-controlled auth token, role bundle, registry path, and snapshot path
outbound access only to the approved official source surfaces already implemented in the connectors
MCP-aware clients or inspectors on
/mcp, plus CLI/stdio for local host integration
In current-state terms, "self-hostable" or "sovereign-hosting-friendly" means:
there is no required vendor-managed control plane
there is no required hosted search/index layer
there is no required hosted database beyond the local SQLite registry
persistent runtime state can remain on operator-controlled disk or volume storage
What this repository does not claim today:
managed SaaS operation
public-internet hardened service posture
multi-node or highly available clustered runtime
full enterprise IAM or per-user authorization
compliance certification or sovereign-cloud accreditation
Supported Scope
Supported Sources
SAMA
stats.gov.saMinistry of Finance
one narrow
data.gov.sapilot path
Supported Dataset Families
SAMA high-frequency monetary and market datasets currently in the registry
stats.gov.saheadline CPI monthlystats.gov.satotal unemployment rate quarterlystats.gov.sareal GDP growth quarterlyMinistry of Finance headline budget balance quarterly
See DATASETS.md for the current canonical dataset direction and current narrow-contract limits.
Stable vs Evolving
Stable enough to evaluate and operate now:
current MCP tool/resource surface
current CLI over that surface
current auth/audit/governance model
current container/runtime/deployment story
current governed CLI export path
Still intentionally evolving:
breadth of source and dataset coverage
richness of normalization for HTML/PDF-oriented sources
automation depth around operations, backup, and restore
dashboard operational resilience and cross-surface coherence as a live consumer of the governed core
Start Here By Audience
Evaluator:
start with this README, then CHANGELOG.md, ARCHITECTURE.md, GOVERNANCE.md, and DEPLOYMENT.md
Operator:
start with CHANGELOG.md, DEPLOYMENT.md, OPERATIONS.md, RUNBOOKS.md, and PERSISTENCE.md
Engineer:
start with CHANGELOG.md, ARCHITECTURE.md, DATASETS.md, GOVERNANCE.md, and dashboard/README.md for the current UI/live-integration boundary
Current Architecture
The current codebase is organized around three layers.
Data Access Layer
connectors/defines the typed connector contract and the current source-specific connectors for SAMA,stats.gov.sa, Ministry of Finance, and one narrowdata.gov.sapilot dataset. Connector resolution dispatches bydescriptor.source.storage/provides raw snapshot persistence and local freshness helpers for connector payloads.httpxis the only HTTP client in the core path.
Normalization & Contract Layer
normalization/contains source-aware field mapping, source-aware validation, the normalization pipeline, and the current minimal canonical record layer.registry/owns dataset descriptors, health metadata, SQLite persistence, and deterministic bootstrap data.Canonical records are produced only for narrow supported JSON shapes:
top-level list of objects
object with a
rowslist of objects
AI-Facing Tool Layer
resources/exposes read-only registry-backed resource views.tools/exposes deterministic MCP tool layers over the registry, local snapshots, and preview/query paths.server.pywires the current MCP surface into FastMCP.
Current Implemented MCP Surface
The current exposed MCP surface is intentionally small:
resource://catalogresource://observabilityresource://policiesdataset_metadatadataset_healthdownload_datasetmaterialize_hot_setquery_datasetsearch_datasetspreview_dataset
What each one does now:
resource://catalog: read-only summary of the bootstrapped registry catalogresource://observability: read-only grouped summary of current process-local counters, plus the raw counter snapshot for internal operatorsresource://policies: read-only summary of current data-facing semantics, including whyquery_datasetremains the primary analytical surface andpreview_datasetremains hybriddataset_metadata: exact lookup of registry-backed dataset metadata bydataset_iddataset_health: exact lookup of registry-backed health metadata bydataset_id, with local snapshot freshness evidence when availabledownload_dataset: local-only raw snapshot availability lookup bydataset_idmaterialize_hot_set: explicit Wave 1 hot-set fetch and local snapshot persistence for the safe SAMA subsetquery_dataset: local-only exact-match query over canonical records derived from local snapshotssearch_datasets: deterministic registry-backed substring search over dataset metadatapreview_dataset: exact preview by canonicaldataset_id, using explicit local/live hybrid resolution metadata and the registry-ownedsource_locatorinternally for source access
Concise example of the current surface:
resource://catalog
resource://observability
resource://policies
dataset_metadata({"dataset_id": "sama-money-supply-weekly"})
dataset_health({"dataset_id": "sama-money-supply-weekly"})
download_dataset({"dataset_id": "sama-money-supply-weekly"})
materialize_hot_set({"include_optional": false})
query_dataset({"dataset_id": "sama-money-supply-weekly", "filters": {"week_end_date": "2024-01-13"}, "limit": 5})
search_datasets({"query": "money"})
preview_dataset({"dataset_id": "sama-money-supply-weekly"})What Works Now
Architecture documents and ADRs are in place and aligned with the codebase.
The connector contract is typed and implemented for SAMA,
stats.gov.sa, Ministry of Finance, and one narrowdata.gov.sapilot dataset.Raw payload snapshots can be written and read locally.
Local snapshot freshness is evaluated deterministically from filesystem evidence only.
Registry models, SQLite repository behavior, and deterministic bootstrap data are implemented.
Registry descriptors now distinguish canonical
dataset_idfrom source-specificsource_locator.Normalization field mapping, validation, pipeline composition, and minimal canonical record extraction are implemented and dispatched by source.
The MCP server is wired with a real working surface for catalog, metadata, health, download, materialize, query, search, and preview.
Wave 1 hot-set materialization is implemented for the current safe SAMA subset.
Tier A background refresh is available for the internal container runtime and remains opt-in.
Preview resolves the connector by descriptor source and uses the current normalization dispatch path; it can return either:
record_derivablelimitedfailed
Preview now exposes explicit hybrid metadata including data origin, freshness status, and resolution outcome.
Query and download are local-only and do not fetch remotely when a snapshot is missing.
Internal HTTP serving has explicit bearer-token auth, capability checks, a canonical
/startupzstartup probe, and a/readyzcompatibility alias with the same startup-only semantics.A curated
upstream-canarycommand and scheduled workflow now exercise live approved dataset paths for source families with a registered queryable canary dataset.Unit, integration, contract, and smoke tests are in the repo and passing.
What Is Intentionally Not Implemented Yet
no broad source-family coverage beyond the current narrow curated official datasets
no semantic search
no LLM in the core path
no connector-backed catalog discovery
no full-system health scoring or broad connector uptime monitoring beyond the curated upstream canary subset
no remote fallback in
download_datasetorquery_datasetno mature canonical record layer that turns every source payload shape into final business records
no generic canonical identity translation beyond the current registry-owned
dataset_idplus singlesource_locator
One important limitation to keep explicit: preview_dataset uses the real connector and normalization path, but the normalization layer may still return limited results for HTML/text payloads and does not yet claim final normalized domain records.
Another important limitation: query_dataset only works on local snapshots that can be normalized into the current narrow canonical record shapes. Unsupported JSON shapes and HTML/text payloads remain explicit rather than queryable.
Local Setup
This repo uses a src/ layout. uv sync --no-editable installs the local
package and exposes the saudi-open-data-mcp console script through
uv run --no-editable; local commands do not require manually setting
PYTHONPATH.
Install and sync with uv:
uv sync --no-editableThen either use uv run --no-editable as shown below or activate the local environment:
source .venv/bin/activateor call the installed tools from .venv/bin/... explicitly.
Lint:
uv run --no-editable ruff check .Tests:
uv run --no-editable pytestLocal Run
The supported local development path is the local console script through
uv run --no-editable:
uv run --no-editable saudi-open-data-mcp check-startup
uv run --no-editable saudi-open-data-mcp run-stdio
HTTP_AUTH_TOKEN="$(openssl rand -hex 32)" uv run --no-editable saudi-open-data-mcp run-http --host 127.0.0.1 --port 8000After activating .venv, the same console script is available without uv run:
saudi-open-data-mcp check-startup
saudi-open-data-mcp run-stdio
HTTP_AUTH_TOKEN="$(openssl rand -hex 32)" saudi-open-data-mcp run-http --host 127.0.0.1 --port 8000The same CLI also provides a thin non-interactive local façade over
the current core operations. These commands emit structured JSON by default and
support --output for file writes. --quiet only applies when --output is
set. --format remains json for the read/health/config commands, while
export now also supports excel and pdf artifacts over the governed
query_dataset result:
uv run --no-editable saudi-open-data-mcp list
uv run --no-editable saudi-open-data-mcp query sama-pos-weekly --filter week_end_date=2024-01-13 --limit 5
uv run --no-editable saudi-open-data-mcp preview stats-gov-sa-cpi-headline-monthly
uv run --no-editable saudi-open-data-mcp download sama-money-supply-weekly
uv run --no-editable saudi-open-data-mcp export sama-money-supply-weekly --output money_supply.json
uv run --no-editable saudi-open-data-mcp export sama-money-supply-weekly --format excel --output money_supply.xml
uv run --no-editable saudi-open-data-mcp export sama-money-supply-weekly --format pdf --output money_supply.pdf
uv run --no-editable saudi-open-data-mcp health mof-budget-balance-quarterly
uv run --no-editable saudi-open-data-mcp refresh --dataset sama-money-supply-weekly
uv run --no-editable saudi-open-data-mcp refresh --include-optional
uv run --no-editable saudi-open-data-mcp configThe Excel artifact is an Excel-compatible XML workbook with visible metadata and records worksheets. The PDF artifact is a metadata-first text PDF that keeps status, origin, freshness, and limitations explicit instead of adding decorative reporting layers.
Use the local console script or helper scripts for development and local host integration.
run-stdio remains the primary local host/operator path for Claude Desktop and
other command-based MCP hosts.
run-http starts the same app over streamable HTTP. Treat that path as
MCP-aware and session-aware only. It is suitable for MCP inspectors and MCP
clients, not generic browser probing. It now requires
Authorization: Bearer <token> using HTTP_AUTH_TOKEN, plus an explicit HTTP
role from HTTP_AUTH_ROLE. The configured role resolves to the allowed
capability bundle, and HTTP_AUTH_CAPABILITIES may be left implicit or set to
the same role bundle explicitly.
By default, local registry and snapshot state resolve under the repo's
.local/ directory; set REGISTRY_PATH or SNAPSHOT_DIR to override them
explicitly. For reproducible host runs, prefer explicit REGISTRY_PATH,
SNAPSHOT_DIR, SAMA_BASE_URL, and DATA_GOV_SA_BASE_URL values.
Local state expectations:
download_datasetreports only what exists in the local snapshot store. It does not fetch remotely.query_datasetonly works when a local snapshot exists and the normalization layer can derive canonical records from that snapshot.If no local snapshot exists,
download_datasetreturnsartifact_missingandquery_datasetreturnssnapshot_missing.On a fresh checkout, those local-only states are the expected result until snapshots have been written under the configured snapshot directory.
Public download and health outputs expose artifact presence plus freshness evidence, not local snapshot paths.
When meaningful, core tool results expose consistent top-level metadata such as
data_origin,freshness_status,failure_stage, anddegradation_reasonto make degraded and failed paths easier to interpret.
The helper script remains available for local HTTP development:
./scripts/run_local_http.shOfficial Internal Container Serving
The official internal serving path for this phase is containerized streamable HTTP.
Chosen serving mode:
run-httpover FastMCP streamable HTTP
Why this mode:
it gives one long-running service shape for internal operators
it fits container process supervision better than stdio
it keeps the same MCP surface and tool semantics already exercised locally
The canonical container entrypoint is:
saudi-open-data-mcp run-httpThe image sets container-specific runtime defaults:
HTTP_HOST=0.0.0.0HTTP_PORT=8000HTTP_AUTH_TOKENmust be provided by the operatorHTTP_AUTH_ROLE=operatorHTTP_AUTH_CAPABILITIES=read,refresh,materializeTIER_A_REFRESH_ENABLED=falseTIER_A_REFRESH_INTERVAL_SECONDS=3600REGISTRY_PATH=/var/lib/saudi-open-data-mcp/registry.sqliteSNAPSHOT_DIR=/var/lib/saudi-open-data-mcp/snapshotsCACHE_DIR=/var/lib/saudi-open-data-mcp/cache
Persistence expectations for that runtime:
REGISTRY_PATHandSNAPSHOT_DIRshould live on durable storage if you need state to survive replacementCACHE_DIRis recreatable scratch spacelogs,
resource://observabilitycounters, in-memory rate limits, and refresh loop state are process-local
Build and serve with Docker Compose:
docker compose up --buildThe provided compose file publishes the service on 127.0.0.1:8000 on the
host, persists runtime state in a Docker-managed volume mounted at
/var/lib/saudi-open-data-mcp, enables init: true, and applies the same
/startupz startup-probe contract as the image. It also requires
HTTP_AUTH_TOKEN to be set in the operator environment before startup.
Internal observability remains intentionally simple:
read
resource://observabilityto inspect the current grouped in-process counters in one placeinspect structured container logs for event-level detail such as
server.startup.*,preview.request.*,connector.request.*,materialize.*, andtier_a_refresh.*treat the observability resource as a process-local operator aid, not as a health endpoint or external metrics API
For operator startup, shutdown, refresh, backup, and restore guidance, see OPERATIONS.md.
Direct container run example:
docker build -t saudi-open-data-mcp .
docker run --rm \
-p 127.0.0.1:8000:8000 \
-e HTTP_AUTH_TOKEN="$(openssl rand -hex 32)" \
-v saudi-open-data-mcp-data:/var/lib/saudi-open-data-mcp \
saudi-open-data-mcpContainer/runtime expectations:
registry bootstrap still happens on startup
Tier A background refresh is available but disabled by default
when enabled, Tier A refresh runs immediately after service lifespan starts and then repeats on the configured interval
refresh reuses the existing Tier A hot-set materialization path only; Tier B remains out of scope in this phase
per-dataset refresh failures remain explicit in the materialization result and do not abort the whole refresh loop
no external scheduler or distributed refresh system is added in this phase
minimal bearer-token auth is enforced on the HTTP path only
HTTP roles are enforced on the HTTP path only:
viewerfor read/query/metadata/health/policies/observabilityoperatorforvieweraccess pluspreview_datasetandmaterialize_hot_setadminas the highest current role with the same operational bundle asoperator
the current role bundles remain capability-based under the hood:
readfor resources and local read/query/search toolsrefreshforpreview_datasetmaterializeformaterialize_hot_set
no public-internet deployment hardening is claimed in this phase
persistent storage is expected if you want registry and snapshot state to survive container replacement
HTTP_AUTH_TOKEN,HTTP_AUTH_ROLE,HTTP_AUTH_CAPABILITIES,TIER_A_REFRESH_ENABLED,TIER_A_REFRESH_INTERVAL_SECONDS,SAMA_BASE_URL,DATA_GOV_SA_BASE_URL, andLOG_LEVELare the main operator-facing overrides These base-URL overrides remain explicitly source-specific in the current config because the runtime still carries SAMA-specific and data.gov.sa-pilot assumptions.
Startup/readiness contract:
the container's job is to start the MCP HTTP service and stay running
GET /startupzis the canonical machine-friendly startup probe for this phaseGET /readyzremains a compatibility alias for the same startup-only payload/startupzand/readyzmean only:the process is running
config validation passed
runtime storage preparation passed
core FastMCP app wiring completed
HTTP requests without a valid
Authorization: Bearer <token>header are rejected with401 UnauthorizedHTTP requests with a valid token but insufficient role/capability are rejected with
403 Forbidden/startupzand/readyzdo not claim:upstream source reachability
dataset freshness
live connector health
full system health
/mcpmust be checked with an MCP-aware client if you want real session readiness validationnaive
GET /orGET /mcpprobing can still return404or406and that is not, by itself, a serving failure
Curated live canary contract:
uv run --no-editable saudi-open-data-mcp upstream-canaryperforms a live connector fetch plus normalization on:sama-exchange-rates-currentstats-gov-sa-cpi-headline-monthlymof-budget-balance-quarterly
data.gov.sais skipped with an explicit log line until a queryabledata.gov.sadataset is registered; the catalog-only pilot dataset is not used as a canary target.a canary failure means one of those curated approved routes drifted in reachability, response shape, or normalization behavior enough to stop producing the expected record-derivable result
the canary does not claim full catalog coverage, dataset freshness, or complete connector health
MCP Host Registration
For local desktop MCP host registration, use stdio. That remains the supported development/operator path for command-based hosts and is separate from the official internal container serving path.
For stdio-based MCP hosts, use the source-tree CLI directly with absolute paths.
The repo's server.json is repository-level MCP metadata for this
internal/evaluator alpha. It intentionally declares no PyPI, npm, or MCP
registry package until a package is actually published.
Claude Desktop example:
{
"mcpServers": {
"saudi-open-data-mcp": {
"command": "/absolute/path/to/saudi-open-data-mcp/.venv/bin/python",
"args": [
"/absolute/path/to/saudi-open-data-mcp/src/saudi_open_data_mcp/cli.py",
"run-stdio"
],
"cwd": "/absolute/path/to/saudi-open-data-mcp",
"env": {
"LOG_LEVEL": "ERROR",
"SAMA_BASE_URL": "https://www.sama.gov.sa",
"DATA_GOV_SA_BASE_URL": "https://open.data.gov.sa"
}
}
}
}cwd is optional here. The default registry and snapshot paths are now anchored to the repo rather than the process working directory, but keeping cwd set to the repo root can still make local config and path reasoning easier.
After installing the package, command-based hosts can also launch the console script directly:
{
"mcpServers": {
"saudi-open-data-mcp": {
"command": "saudi-open-data-mcp",
"args": ["run-stdio"],
"env": {
"LOG_LEVEL": "ERROR",
"SAMA_BASE_URL": "https://www.sama.gov.sa",
"DATA_GOV_SA_BASE_URL": "https://open.data.gov.sa"
}
}
}
}If you prefer a single command path, the repo also includes a stdio helper script:
{
"mcpServers": {
"saudi-open-data-mcp": {
"command": "/absolute/path/to/saudi-open-data-mcp/scripts/run_local_stdio.sh"
}
}
}Current limitation to keep explicit: local host registration remains stdio through the source-tree CLI. The official container serving path is HTTP, not a desktop stdio-host replacement.
Try manually with MCP Inspector
HTTP is the official internal container serving mode, but it is not a plain
REST surface. /mcp is an MCP endpoint, not a normal browser page.
Start the HTTP MCP server in one shell:
export HTTP_AUTH_TOKEN="$(openssl rand -hex 32)"
uv run --no-editable saudi-open-data-mcp run-http --host 127.0.0.1 --port 8000Check the startup probe from another shell:
curl -s http://127.0.0.1:8000/startupzIn MCP Inspector, connect to the Streamable HTTP URL
http://127.0.0.1:8000/mcp.If auth is enabled, add
Authorization: Bearer <the value of HTTP_AUTH_TOKEN>.Try safe calls such as
resource://catalog,search_datasets({"query": "money"}),dataset_metadata({"dataset_id": "sama-money-supply-weekly"}), anddataset_health({"dataset_id": "sama-money-supply-weekly"}).
Naive probing can look broken even when the server is healthy:
GET /can return404GET /mcpwithout the expected MCP headers may return406, which is expectedGET /startupzis the health/startup probe and should return200with a narrow startup-only payload
That behavior is expected for the current streamable HTTP setup. Browser or curl checks are useful only as a negative smoke test here, not as a real MCP session test.
Programmatic MCP-aware test example:
import asyncio
import os
from fastmcp import Client
from fastmcp.client.transports import StreamableHttpTransport
async def main() -> None:
async with Client(
transport=StreamableHttpTransport(
"http://127.0.0.1:8000/mcp",
headers={"Authorization": f"Bearer {os.environ['HTTP_AUTH_TOKEN']}"},
)
) as client:
result = await client.call_tool(
"dataset_metadata",
{"dataset_id": "sama-money-supply-weekly"},
)
print(result.structured_content)
asyncio.run(main())Testing
The repo includes four test layers:
tests/unit/: typed contracts, tool behavior, repository behavior, connector behavior, and normalization behaviortests/integration/: small cross-module composition checkstests/contracts/: architectural boundary checks, including the rule that tool modules must not import connectors directlytests/smoke/: basic CLI/importability verification
Repo Structure
The main code lives under src/saudi_open_data_mcp/.
connectors/: source access contracts and per-source connectors (SAMA, stats.gov.sa, MoF, data.gov.sa)normalization/: field mapping, validators, pipeline, and minimal canonical recordsregistry/: typed metadata models, SQLite repository, and bootstrapstorage/: snapshots and local freshness helpersresources/: registry-backed MCP resourcestools/: registry-backed and local-only MCP tools, plus preview over the connector pathobservability/: structured logging, process-local counters, upstream canarysecurity/: HTTP auth middleware, readiness probes, rate limiting, input sanitizationconfig.py: runtime configuration and environment variable resolutioncli.py: thin non-interactive CLI over the MCP coreserver.py: FastMCP wiringdocs/: architecture, ADR, roadmap, and dataset notes
Roadmap / Next Steps
Near-term work should stay aligned with the current architecture:
harden the current MCP surface and keep its descriptions, contracts, and identity handling consistent
deepen the normalization layer only where the source shape safely supports richer canonical records
improve packaging, local run ergonomics, and release presentation around the current FastMCP server
continue reliability and observability hardening
add a second source only after the SAMA path and registry contracts are stable
Available Tools
7 toolsdataset_healthA
Exact registry-backed health lookup by dataset_id, with local snapshot freshness evidence when available.
| Name | Required | Description | Default |
|---|---|---|---|
| dataset_id | Yes | Exact canonical dataset_id from the registry. Examples: 'sama-pos-weekly', 'stats-gov-sa-cpi-headline-monthly'. |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of disclosure. It adds useful context: the lookup is 'registry-backed' (data provenance) and 'local snapshot freshness evidence' is included 'when available' — a valuable caveat that the freshness field may be absent. However, it does not explain what 'health' means, error behavior for unknown IDs, or what the response contains beyond freshness. It partially discloses behavior but leaves meaning gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single sentence that front-loads the core purpose ('health lookup by dataset_id'), then appends the one extra behavioral detail worth knowing (freshness evidence when available). There is zero waste and every word earns its place. The 'Exact' qualifier and 'registry-backed' provenance both add signal without padding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple (1 parameter) and has an output schema, so return values are documented elsewhere and the description need not explain them. The description covers the lookup purpose, the data source, and the freshness caveat. The only real gap is usage differentiation — it does not clarify when to prefer this over dataset_metadata — which keeps it just short of fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so the baseline is 3. The schema already documents dataset_id with examples, and the description adds no new semantic detail about the parameter beyond restating its use. The description mentions dataset_id only in the opening clause, adding no format, constraints, or additional meaning beyond what the schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('lookup'), a precise resource ('registry-backed health'), and the identifying key ('dataset_id'). 'Health' is clearly distinct from the sibling tools — metadata, preview, download, query, and search — so an agent can tell it apart without opening schemas. The 'Exact' qualifier reinforces that it is not a fuzzy search.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage context: it is for retrieving health status by a canonical registry dataset_id, and 'Exact' hints it is not for fuzzy lookups (contrast search_datasets). However, it never names alternative tools or states when-not to use it — for example, it does not clarify how it differs from dataset_metadata, which is a plausible alternative an agent might confuse it with.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
dataset_metadataC
Exact registry metadata lookup by dataset_id.
| Name | Required | Description | Default |
|---|---|---|---|
| dataset_id | Yes | Exact canonical dataset_id from the registry. Examples: 'sama-pos-weekly', 'stats-gov-sa-cpi-headline-monthly'. |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavior. It only states the purpose without mentioning return format, authorization, rate limits, or any side effects. It does not clarify what 'metadata' includes or if it is read-only.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
One concise sentence with no fluff. It is front-loaded with the purpose, but it does not include any additional structured guidance, which might be acceptable given its brevity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with a single parameter and an output schema, the description is minimal. It lacks context about when to use it versus search_datasets, what metadata is returned, or any prerequisites. The agent has insufficient information to confidently invoke it correctly in all scenarios.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, and the parameter is well documented with examples. The description adds no additional meaning beyond restating that the lookup is by dataset_id, so it meets the baseline.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (lookup) and resource (registry metadata) with a unique identifier (dataset_id). It is clear but does not explicitly differentiate from sibling tools like search_datasets, though 'exact' hints at direct ID lookup.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance on when to use this tool vs alternatives such as search_datasets or preview_dataset. The description does not provide context for selecting this tool over others.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
download_datasetA
Report local raw snapshot availability for an exact dataset_id. Local-only; no remote fetch.
| Name | Required | Description | Default |
|---|---|---|---|
| dataset_id | Yes | Exact canonical dataset_id from the registry. Examples: 'sama-pos-weekly', 'stats-gov-sa-cpi-headline-monthly'. |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the behavioral burden. It discloses that the operation is local-only and performs no remote fetch, signaling a safe, non-destructive read. It does not mention auth or failure behavior, but the output schema covers return structure and this is a simple availability report.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences state the core operation, the exactness requirement, and the local-only behavior. No filler, redundant restatement of the tool name, or duplication of schema details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter, output-schema-backed tool, the description covers the essential selection and invocation concerns: exact dataset_id, local-only behavior, and that no remote fetch occurs. The only minor gap is the mismatch between the tool name ('download') and its report-only behavior, which the description partially mitigates.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema fully documents dataset_id with the exact canonical constraint and examples, so schema coverage is complete. The description adds only the local-only context and does not materially extend the parameter semantics beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the operation ('Report local raw snapshot availability') and the target resource ('an exact dataset_id'), and the 'Local-only; no remote fetch' qualifier differentiates it from fetch/materialization tools. It does not explicitly name sibling tools, but the purpose is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use the tool (when checking local availability of an exact dataset_id) and rules out remote fetching, but it does not provide explicit when-not-to-use guidance or point to alternatives among the listed siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
materialize_hot_setA
Fetch and persist the fixed Wave 1 SAMA hot-set into local snapshots. Tier B remains opt-in via include_optional.
| Name | Required | Description | Default |
|---|---|---|---|
| include_optional | No | Whether to include optional Tier B hot-set datasets during materialization. Use false for the default Tier A set; true also includes datasets such as 'sama-pos-by-city'. |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description carries the full transparency burden. It does disclose a persistence side effect ('persist into local snapshots') and notes that Tier B is excluded by default. However, it does not mention whether existing snapshots are overwritten, whether the operation is destructive, or what failure modes exist, leaving a meaningful transparency gap.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, the first gives the action and scope, the second clarifies the optional parameter. No filler words; information is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-optional-parameter tool with an output schema, the description covers the core purpose and the one decision point. It leaves out details like what 'hot-set' contains or what a 'snapshot' is, but those are likely domain terms and not required to call the tool correctly. The description is adequate though not exhaustive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, and the parameter's schema description already fully explains include_optional. The description's clause 'Tier B remains opt-in via include_optional' restates this rather than adding new semantic content, so no additional value over the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action ('Fetch and persist') on a concrete resource ('fixed Wave 1 SAMA hot-set') with a clear scope ('into local snapshots'). This clearly distinguishes it from sibling tools like download_dataset or query_dataset, which do not imply local persistence.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use the tool – when you want to materialize the hot-set locally – but it never explicitly names an alternative or states a condition for choosing this tool over a sibling. The only usage hint is the mention that Tier B is opt-in via include_optional, which is parameter guidance rather than tool-selection guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
preview_datasetB
Fetch and preview a dataset for an exact registry dataset_id.
| Name | Required | Description | Default |
|---|---|---|---|
| dataset_id | Yes | Exact canonical dataset_id from the registry. Examples: 'sama-pos-weekly', 'stats-gov-sa-cpi-headline-monthly'. |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full responsibility for behavioral disclosure. 'Fetch and preview' implies a read operation, but the description does not clarify whether preview returns a sample, full data, or any limits, nor does it mention side effects or permissions. This is thin for a tool that previews data.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The entire description is one clean, front-loaded sentence: 'Fetch and preview a dataset for an exact registry dataset_id.' Every word earns its place, with no filler or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple (one parameter) and has an output schema, which reduces the need to explain return values. However, the description lacks behavioral context (e.g., what 'preview' entails) and usage guidance relative to siblings, leaving an agent to infer when this is the right choice.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% with a well-documented dataset_id parameter including examples. The description's reiteration of 'exact registry dataset_id' reinforces but does not add meaning beyond the schema. Baseline of 3 is appropriate for complete schema coverage with no additional parameter semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb ('Fetch and preview') and resource ('dataset') with a clear qualifier ('exact registry dataset_id'). This is unambiguous about what the tool does, though it does not explicitly name or contrast against siblings like download_dataset or query_dataset, relying on the name 'preview' to differentiate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
No explicit when-to-use guidance is provided. The description only mentions the prerequisite of an exact registry dataset_id, which implies you need it, but there is no mention of alternatives (e.g., search_datasets for finding IDs, download_dataset for full data) or when not to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
query_datasetA
Query local canonical records for an exact dataset_id using exact-match filters only. Local-only; no remote fetch.
| Name | Required | Description | Default |
|---|---|---|---|
| limit | No | Optional maximum number of matching records to return. Example: 10. | |
| filters | No | Optional exact-match filters keyed by canonical record field. Example: {'currency_code': 'USD'} or {'observation_month': '2026-01'}. | |
| dataset_id | Yes | Exact canonical dataset_id from the registry. Examples: 'sama-pos-weekly', 'stats-gov-sa-cpi-headline-monthly'. |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must carry the full burden of behavioral disclosure. It mentions 'Query' implying a read operation but does not explicitly state read-only or non-destructive behavior. It also does not disclose error handling, pagination, or any rate limits. The phrase 'Local-only' adds some context but is insufficient for a tool that could be called by an agent without knowing side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with no fluff, front-loading the core purpose and then adding the local-only constraint. Every word earns its place, making it highly efficient for an agent to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has an output schema and the parameters are fully described, the description is mostly complete. It clearly states what the tool does and its constraints. It could mention pagination or behavior when no results are found, but these are not critical for the agent to decide whether to call it. The presence of an output schema reduces the need to describe return values.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description does not add significant meaning beyond the schema; it reiterates 'exact-match filters only' which is already stated in the filters parameter description. It does not clarify the limit parameter behavior or provide additional context for dataset_id beyond the examples already in the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool queries local canonical records for an exact dataset_id using exact-match filters only, which is a specific verb and resource. It also differentiates itself from siblings by emphasizing 'Local-only; no remote fetch' and 'exact-match filters only', which distinguishes it from search_datasets and preview_dataset.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context: it's for querying local records with exact-match filters, and it explicitly states 'Local-only; no remote fetch' which implies it's not for remote data. However, it does not explicitly name alternative tools like search_datasets for fuzzy search or download_dataset for fetching remote data, so it stops short of explicit exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
search_datasetsA
Search registry-backed dataset metadata using deterministic substring matching only.
| Name | Required | Description | Default |
|---|---|---|---|
| query | Yes | Case-insensitive substring to match against dataset metadata. Use an empty string to list all datasets; examples: 'pos', 'inflation'. |
Output Schema
| Name | Required | Description |
|---|---|---|
No output parameters | ||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description prominently discloses that matching is deterministic substring-only, which is the key behavioral trait. It does not explicitly state read-only semantics or result behavior, but the output schema covers return shape, so the remaining gap is limited.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single front-loaded sentence with no filler. Every phrase contributes either the resource scope ('registry-backed dataset metadata'), the operation ('Search'), or the key constraint ('deterministic substring matching only').
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter search tool with a 100%-covered schema and an output schema, the description and schema leave little ambiguity about invoking it correctly. A brief note on which metadata fields are searched would make it fully complete, but it is not a significant gap.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and the schema already documents case-insensitive substring matching, empty-string listing, and examples. The description adds no additional parameter meaning beyond the word 'only', which conveys exclusivity rather than new semantics.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Search') and a clear resource ('registry-backed dataset metadata'), and the qualifier 'deterministic substring matching only' narrows the behavior compared with query or metadata-retrieval siblings. It does not explicitly name sibling tools, so it is unambiguous but not maximally differentiated.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'deterministic substring matching only' implies this is for exact substring lookups, and the schema supplements with empty-string and example guidance. However, the description itself gives no explicit when-to-use guidance or contrast with dataset_metadata/query_dataset, leaving alternatives to the agent's inference.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
7 tool updates
v0.1.0- First observed
dataset_health - First observed
dataset_metadata - First observed
download_dataset - First observed
materialize_hot_set - First observed
preview_dataset - First observed
query_dataset - First observed
search_datasets
TDQS
Scored across 7 tools
Each tool serves a distinct function: preview, metadata lookup, health check, download availability, materialization, local query, and search. No two tools have overlapping purposes; even preview vs. query are clearly separated by remote vs. local scope.
Most tools follow a verb_noun pattern (preview_dataset, download_dataset, query_dataset, search_datasets, materialize_hot_set), but dataset_metadata and dataset_health are noun_noun, creating a minor inconsistency. Still, the pattern is readable and predictable overall.
Seven tools is well-scoped for a data access server covering search, metadata, preview, health, download, and local query. Each tool has a clear role with no redundancy, fitting the typical 3-15 range.
The surface covers the full lifecycle for a data consumer: discovery (search), inspection (metadata, preview, health), acquisition (download, materialize), and local analysis (query). There are no obvious dead ends, and the server's read-only purpose is fully served.
Maintenance
Related MCP Connectors
Governed data discovery, exact queries, decisions, simulations, and runtime utilities over MCP.
This MCP server provides seamless access to Malaysia's government open data, including datasets, w…
Hosted MCP server for finding authoritative primary data sources and official portals.
MCP server for Brazilian Federal Senate open data (legislative, administrative, e-Cidadania).
Related MCP Servers
- AlicenseAqualityDmaintenanceAn MCP server for discovering, downloading, querying, and analyzing datasets from Ontario's open data portals, allowing natural language questions and high-performance analytics via DuckDB.231MIT
- AlicenseNot gradedqualityCmaintenanceMCP server for the Moroccan Open Data portal (data.gov.ma) enabling search and retrieval of datasets, resources, organizations, and groups via CKAN API.1MIT
- AlicenseAqualityDmaintenanceAn MCP server to find and fetch Norwegian open government data from data.norge.no. Enables search, metadata retrieval, and data download.53MIT
- AlicenseAqualityBmaintenanceAn MCP server that gives LLM agents typed, cached access to civic open-data portals via Socrata (SODA 2.1 + Discovery API), enabling search, query, profiling, sampling, and CSV export of datasets.6MIT