CKAN MCP Server (Python)

A Model Context Protocol (MCP) server that exposes the 600 plus¹² global CKAN open-data portals to AI assistants, CLI tools, and other MCP-aware clients. It bundles curated portal presets, strict Pydantic models, and a batteries-included tool suite so analysts, application developers, and operators can explore public datasets without writing bespoke CKAN integrations.

Persona-centric Guidance

Potential Users: What This Server Does

Why it exists

• Purpose-built CKAN interface: Wraps the CKAN Action & Datastore APIs behind MCP tools that AI agents and CLI clients understand.

• Consistent insights: Presents dataset summaries, freshness analysis, schemas, and download helpers so exploratory conversations stay grounded in real CKAN metadata.

• Portal-aware behavior: Curated overrides (transport method, dataset URL templates, helper prompts) keep the experience consistent across CKAN portals that deviate from defaults.

Tool catalog (14 tools)

Architecture at a glance

• src/ckan_mcp/main.py – MCP server entry point supporting stdio and HTTP (SSE) transports.

• src/ckan_mcp/ckan_tools.py – Tool implementations, transport probes, download helpers, and archive extraction.

• src/ckan_mcp/helpers.py – Relevance scoring, update frequency analysis, summary builders.

• src/ckan_mcp/types.py – Strict Pydantic models with extra="allow" for portal-specific metadata.

• src/ckan_mcp/config_selection.py & src/ckan_mcp/data/ckan_config_selection.json – Curated CKAN portal catalog and overrides consumed by ckan_api_initialise.

• tests/ & test_runner.py – Pytest suite plus quick smoke runner mirroring production behaviors.

Tip: start with ckan_api_initialise to choose a portal, then call the analysis tools to see the depth of insights returned.

Data Analysts: Getting Insights Fast

Shared prerequisites

• CKAN portal URL (or use the curated list during initialization).

• Python 3.11+ and uv or pip for installing dependencies.

• curl and the POSIX file command on your PATH (the download_dataset_locally tool shells out to both binaries).

• An MCP-compatible client (Claude CLI, Gemini CLI, etc.).

Option A – Run the MCP server locally

1. Clone & create a virtual environment

   git clone https://github.com/<org>/ckan-mcp.git cd ckan-mcp uv venv venv source venv/bin/activate

2. Install runtime dependencies

   uv pip install -e .

   (Add ".[dev]" for development tooling and ".[examples]" if you want to run the sample scripts that load .env files.)

3. Optional defaults – export CKAN env vars if you always talk to the same portal:

   export CKAN_BASE_URL="https://ckan0.cf.opendata.inter.prod-toronto.ca/api/3/action" export CKAN_SITE_URL="https://ckan0.cf.opendata.inter.prod-toronto.ca" export CKAN_DATASET_URL_TEMPLATE="https://ckan0.cf.opendata.inter.prod-toronto.ca/dataset/{name}"

   These are fallback values; interactive sessions normally rely on ckan_api_initialise to pick a portal.

4. Launch in stdio mode (best for desktop MCP clients):

   python -m ckan_mcp.main

5. Connect your MCP client – example Claude CLI snippet (see core environment variables for transport overrides):

   { "mcpServers": { "ckan-mcp": { "command": "python", "args": ["-m", "ckan_mcp.main"], "env": { "CKAN_MCP_LOCAL_DATASTORE": "~/dataset-store/", } } } }

6. Start a session – ask your assistant to "Initialize a CKAN connection"; it will call ckan_api_initialise and then the discovery tools.

Option B – Use a remote MCP server with a local client

This flow is perfect when someone else operates the MCP server on shared infrastructure and you only need local MCP tooling.

1. Server operator sets up HTTP transport:

   export CKAN_MCP_MODE=http export CKAN_MCP_HOST=0.0.0.0 export CKAN_MCP_PORT=8000 python -m ckan_mcp.main

   or run docker compose up --build to expose http://localhost:8000/mcp and front it with your preferred reverse proxy.

2. Expose the via HTTPS (Cloud Run, Fly.io, Tailscale, etc.) and share the URL with analysts.

3. Analyst registers the remote MCP server (Claude CLI example):

   claude mcp add --transport http ckan-mcp https://mcp.example.com/mcp claude mcp list

   Gemini CLI uses gemini mcp add --transport http ckan-mcp https://mcp.example.com/mcp with the same URL.

4. Use normally – all CLI/desktop prompts now tunnel through the remote MCP server. The analyst still decides which CKAN portal to inspect via ckan_api_initialise.

Day-to-day usage tips

• ckan_api_availability lists every CKAN portal packaged with this MCP build and reiterates which portal is currently selected (if any) before issuing expensive searches.

• find_relevant_datasets quickly surfaces top matches for natural-language prompts; follow up with get_dataset_insights for a detailed brief.

• download_dataset_locally writes metadata, datastore previews, and shell instructions to ~/.cache/ckan-mcp/... so you can pivot to pandas immediately.

Developers: Extend and Contribute

Repository map

Supporting files: pyproject.toml (uv/poetry style metadata), tests/, test_runner.py, examples/ for fixtures, and Docker/Make targets for container workflows.

Local development workflow

1. Activate the virtualenv and install dev dependencies:

   source venv/bin/activate uv pip install -e ".[dev]"

2. Run formatters and linters (Black first, then Ruff as required by the project guidelines):

   black src/ tests/ ruff check src/ tests/ --fix

3. Type checking:

   mypy src/

4. Tests:

   pytest tests/ -v python test_runner.py # lightweight smoke run # Live integration tests against the curated CKAN portals CKAN_RUN_INTEGRATION_TESTS=1 pytest tests/ -m integration -v

   Integration tests talk to the public CKAN portal configured via CKAN_TEST_COUNTRY/CKAN_TEST_LOCATION (defaults to Canada/Toronto) and accept overrides such as CKAN_TEST_BASE_URL, CKAN_TEST_SITE_URL, CKAN_TEST_DATASET_URL_TEMPLATE, or CKAN_TEST_SEARCH_TERMS for custom portals.

5. GitHub Actions verification (optional, requires the GitHub CLI authenticated against openascot/ckan-mcp-private):

   # trigger the full workflow (lint/unit + integration jobs) for your current branch gh workflow run ci.yml --ref "$(git rev-parse --abbrev-ref HEAD)" # tail the logs for the most recent run gh run watch gh run view --log-failed

   The workflow only runs when triggered manually; the quality-checks job runs Black/Ruff/mypy, and the dependent pytest-suite job reuses .github/workflows/pytest.yml to execute the standard pytest run plus the integration suite (with CKAN_RUN_INTEGRATION_TESTS=1). Trigger the standalone Pytest workflow directly if you only need the testing jobs.

6. Docker-based workflow (optional, HTTP transport exposed at http://localhost:8000/mcp):

   make dev # foreground dev stack with reload make quick-start # background stack, uses docker-compose.dev.yml make dev-tools # helper container via the tools profile make shell # attach to the running dev app container make test-production # builds docker-compose.yml and curls /mcp

Follow the AGENTS.md guidance for naming, docstrings, and how to place fixtures under examples/. Always update or add pytest coverage alongside new tools or helper behaviors.

See CHANGELOG.md for release history and public milestone notes.

Contributing checklist

• Create a feature branch and keep commits focused.

• Add or update tests under tests/ mirroring the target module name (e.g., tests/test_ckan_tools.py).

• Run pytest, black, ruff, and mypy locally (or via the docker helpers) before opening a PR.

• Document new environment variables or tool behaviors in this README or EVALUATION_GUIDE.md as appropriate.

Production Deployment (Google Cloud Run Example)

Cloud Run pairs nicely with the built-in HTTP transport. The following example assumes you have the Google Cloud CLI configured and Artifact Registry enabled.

1. Build and push the container (uses the included multi-stage Dockerfile):

   export PROJECT_ID="my-gcp-project" gcloud auth configure-docker gcloud builds submit --tag gcr.io/$PROJECT_ID/ckan-mcp

2. Deploy to Cloud Run:

   gcloud run deploy ckan-mcp \ --image gcr.io/$PROJECT_ID/ckan-mcp \ --region us-central1 \ --platform managed \ --allow-unauthenticated \ --port 8000 \ --set-env-vars CKAN_MCP_MODE=http,CKAN_MCP_HTTP_PATH=/mcp,CKAN_MCP_HTTP_ALLOW_ORIGINS=* \ --set-env-vars CKAN_BASE_URL=https://ckan0.cf.opendata.inter.prod-toronto.ca/api/3/action,CKAN_SITE_URL=https://ckan0.cf.opendata.inter.prod-toronto.ca

   Adjust env vars for your preferred portal or omit them so analysts always call ckan_api_initialise.

3. Share the endpoint – Cloud Run will emit a URL such as https://ckan-mcp-12345-uc.a.run.app. Provide the /mcp path to clients (https://ckan-mcp-12345-uc.a.run.app/mcp).

4. Register with MCP clients – same claude mcp add --transport http ... flow as in the analyst section.

5. Operational tips:

   • Set CKAN_MCP_HTTP_JSON_RESPONSE=true if your proxy expects JSON instead of SSE.

   • Use Secret Manager to supply CKAN_API_KEY for locked-down portals.

   • Monitor Cloud Run metrics; the server makes outbound HTTPS calls to CKAN only when tools are invoked.

Configuration Reference

Core environment variables

CKAN portal overrides

The curated catalog in src/ckan_mcp/data/ckan_config_selection.json contains entries such as Toronto, NYC, etc. Each location can provide overrides like:

• action_transport: force GET vs POST for /api/3/action calls.

• datastore_id_alias: whether datastore_search accepts id instead of resource_id.

• requires_api_key: block initialization until an API key is supplied.

• helper_prompt: user-facing reminder echoed in tool responses.

• Pagination settings (default_search_rows, max_search_rows, default_preview_limit).

Call audit_ckan_api after selecting a portal to get automatically generated override recommendations, helper prompt text, and config snippets that can be pasted back into the catalog or used ad hoc via ckan_api_initialise(overrides={...}).

Sources / References