Skip to main content
Glama
beenthatt-rehman

datagovin-mcp

datagovin-mcp

Natural-language access to India's Open Government Data platform, data.gov.in235,000+ public datasets covering air quality, agriculture, health, fuel prices, census, education, rainfall, railways, crime, budgets and more.

Two faces, one codebase:

  • An MCP server — over stdio for local clients, or over Streamable HTTP so any MCP client connects by URL: Claude Desktop, Claude Code, Cursor, VS Code, Windsurf, ChatGPT connectors, or anything built on an MCP SDK.

  • A website — instant catalog search plus natural-language answers, backed by the exact same tools.

The server ships no data of its own. Discovery runs against a local full-text index built from data.gov.in's own catalog endpoint; every row you actually read is a live call to data.gov.in using your own free API key.

Why this exists

data.gov.in has an enormous catalog but no full-text search API a program can call — the normal workflow is to browse the website and copy a dataset's resource ID off its "API" button. That's a poor fit for a language model.

This project closes the gap. It harvests the platform's /lists endpoint into a local SQLite FTS5 index — 235,241 datasets in about 150 seconds, no API key required — so a model can go from "what's the AQI in Delhi right now?" to real rows without anyone hunting for a UUID. It also absorbs the upstream API's rough edges (case-sensitive filters, occasional CSV responses, a last-page pagination quirk) so the model doesn't have to.

Related MCP server: bharatlas-mcp

Tools

Tool

What it does

search_datasets(query, limit, sector)

BM25-ranked full-text search across the whole catalog.

list_sectors(limit)

Sectors present in the catalog, with dataset counts.

get_dataset_info(resource_id)

Live schema: title, description, row count, exact field names + types.

query_dataset(resource_id, filters, fields, sort, limit, offset)

Pull actual filtered rows, live.

catalog_status()

How many datasets are indexed — distinguishes "no matches" from "not harvested yet".

Setup

1. Install.

git clone https://github.com/<your-username>/datagovin-mcp.git
cd datagovin-mcp
python -m venv .venv && source .venv/bin/activate
pip install -e .            # MCP server only
pip install -e ".[web]"     # + the website

2. Build the search index. No API key needed for this step.

datagovin-harvest
Harvesting the data.gov.in catalog (no API key required)...
  indexed 235,000/235,241 datasets (100%, 1,566/s)

Indexed 235,241 datasets in 150.2s -> ~/Library/Caches/datagovin-mcp/catalog.sqlite3 (346.2 MB)

Until you run this, search falls back to a small bundled seed catalog — the server still works, it just knows about three datasets. Re-run it any time to refresh; hand-curated entries are preserved.

3. Get a free API key — needed to read rows, not to search. Register at data.gov.in and generate one from your profile page.

cp .env.example .env    # then paste your key into it

The .env file is read automatically. Exporting DATA_GOV_IN_API_KEY works too, and a real environment variable always wins over the file.

Connecting a client

Local (stdio)

Add to your MCP client config (claude_desktop_config.json or equivalent):

{
  "mcpServers": {
    "datagovin": {
      "command": "/absolute/path/to/datagovin-mcp/.venv/bin/python",
      "args": ["/absolute/path/to/datagovin-mcp/server.py"],
      "env": { "DATA_GOV_IN_API_KEY": "your_key_here" }
    }
  }
}

Remote (Streamable HTTP) — connects from anywhere

python server.py --transport http --host 0.0.0.0 --port 8000
# MCP endpoint: http://<host>:8000/mcp

Then point any MCP client at the URL:

{
  "mcpServers": {
    "datagovin": { "url": "https://your-host.example.com/mcp" }
  }
}

Add --stateless to run several replicas behind a load balancer.

Before exposing this publicly, put it behind TLS and authentication. The server has no auth of its own, and it spends your data.gov.in API key on every request it serves.

The website

export ANTHROPIC_API_KEY=sk-ant-...   # optional — enables the "Ask" button
datagovin-web                          # http://127.0.0.1:8000

One process serves everything:

Route

/

search UI — type-ahead catalog search, click a dataset for its live schema and sample rows

/api/search?q=

BM25 search as JSON, no LLM involved

/api/dataset/{id}

live schema

/api/dataset/{id}/rows

live rows; any extra query param becomes an upstream filter

/api/ask

streaming natural-language answer (Server-Sent Events)

/mcp

the MCP endpoint — so the same deployment serves browsers and MCP clients

Search works with no keys at all. DATA_GOV_IN_API_KEY unlocks rows; ANTHROPIC_API_KEY unlocks answers. The UI tells you which are missing.

How answers work. /api/ask runs a streaming Claude tool-use loop over the same five tools, narrating each step ("Searching the catalog for…", "Fetching 100 rows where city=Delhi") before the answer streams in. Claude is instructed to answer only from rows it actually fetched, to name the dataset it used, and to say so plainly when the data doesn't answer the question rather than filling the gap from memory.

The tool definitions the website gives Claude are read directly off the MCP server via list_tools() — there is exactly one description and one schema per tool in this project, so the two surfaces cannot drift apart.

Curating a dataset

Harvesting brings in every dataset automatically. Use this to improve one — attach search keywords, a worked example filter, or a corrected sector, and pin it above harvested results:

python scripts/add_dataset.py <resource_id> \
    --sector Agriculture \
    --keywords "wheat,crop,production" \
    --example-filters '{"State":"Punjab"}'

Curated fields survive later harvests.

Notes on the upstream API

Behaviours this server handles for you:

  • Filter field names are case-sensitive (filters[State]filters[state]) and this is undocumented. Always use the exact field id from get_dataset_info.

  • Some legacy datasets return CSV regardless of format=json; the client detects this by Content-Type and parses it anyway. CSV carries no row total, so total_records comes back null rather than a misleading page count.

  • Last-page pagination can return an empty records array with status: ok; returned: 0 means you're done.

  • Max ~100 rows per request on /resource — page with offset.

  • /lists needs no API key and pages up to 1000 records at a time. It is slow and occasionally times out, so the harvester retries every page with backoff.

  • The API key travels in the query string (upstream's design). Every error this package raises is passed through a redactor first, so a key can never reach a log line, a tool result, or the model's context.

Project layout

datagovin-mcp/
├── server.py                    # entry point (kept for existing client configs)
├── datagovin/
│   ├── config.py                # .env loading, cache paths
│   ├── client.py                # async data.gov.in API wrapper (quirk handling)
│   ├── catalog.py               # SQLite FTS5 index: search, sectors, stats
│   ├── harvest.py               # builds the index from /lists
│   ├── mcp_server.py            # the five tools; stdio + Streamable HTTP
│   ├── data/seed_catalog.json   # bundled fallback, works before a harvest
│   └── web/
│       ├── app.py               # FastAPI: search API, /api/ask, mounts /mcp
│       ├── agent.py             # streaming Claude tool-use loop
│       └── static/index.html    # the UI (no build step, no CDN)
├── scripts/add_dataset.py       # curate/pin one dataset
└── tests/                       # 87 tests, no network required

The index lives in your platform cache directory, not in the package — it is generated data, it is ~350 MB, and an installed package directory is often read-only. Override with DATAGOVIN_INDEX_PATH.

Development

pip install -e ".[web,dev]"
pytest                      # 87 tests, all offline

License

MIT

Available Tools

4 tools
get_dataset_infoA

Get the schema of a specific dataset: its title, description, total row count, and the exact list of field names and types.

Call this BEFORE query_dataset when you need to know which fields exist or how to spell a filter field. Field names on data.gov.in are CASE-SENSITIVE (e.g. "State" is not "state"), so always use the exact id returned here when building filters.

Args: resource_id: The dataset UUID, e.g. from search_datasets or the portal.

ParametersJSON Schema
NameRequiredDescriptionDefault
resource_idYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description accurately lays out the tool's observable behavior: it returns schema metadata, and it cautions that field names are case-sensitive, giving the practical consequence (use the exact 'id'). It does not mention side effects, errors, or authentication requirements—but for a GET-style read-only tool these are seldom necessary. The case-sensitivity disclosure is worth credit because it changes how the agent uses the returned data.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Every portion contributes: the opening line states the operation, the second line supplies usage context with a sequencing cue, the warning adds a necessary behavioral nuance, and the Args block gives the parameter source. No sentence is filler; the length is appropriate for the information density.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With a single required parameter and an output schema present, the description fully covers the needed information: the dataset scope, the source of the identifier, and a critical data-quality principle. The tool is simple enough that no further prerequisites or error-handling notes are needed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema provides only the parameter name 'resource_id' with zero description coverage. The description compensates fully by explaining that it is a UUID and pointing the agent to 'search_datasets or the portal' for sourcing it. This eliminates any guesswork about how to obtain or format the value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states the precise operation ('Get the schema of a specific dataset') and enumerates exactly what is returned: title, description, row count, and field names/types. This clearly distinguishes it from the sibling tools (search_datasets searches, query_dataset queries, list_sectors lists sectors) without any ambiguity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives an explicit sequencing instruction: 'Call this BEFORE query_dataset when you need to know which fields exist or how to spell a filter field.' This tells the agent exactly when to use the tool and what to use with it, and the case-sensitivity warning that has a direct effect on subsequent filter construction. It could mention alternatives more cleanly, but the instruction is refreshingly complete and non-obvious.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_sectorsA

List the sectors covered by the curated dataset catalog (e.g. Environment, Agriculture, Health). Useful for orienting the user before a search.

ParametersJSON Schema
NameRequiredDescriptionDefault

No parameters

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full behavioral disclosure burden. The word 'List' conveys that this is a non-mutating, harmless operation, and the examples clarify the nature of the returned content. While it does not address edge cases like ordering or format, the output schema is present to cover return shape, so the description is adequate for a simple catalog-listing tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences with no filler: the first delivers the core action and resource, and the second adds the practical 'why/when' context. The example list is compact and useful, and the front-loading is excellent.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a zero-parameter, low-complexity tool with an output schema, the description is complete: it states what the tool returns, gives example values, and explains its role in the workflow (before a search). Nothing an agent needs to decide whether to call this tool correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool takes zero parameters, so the baseline for this dimension is 4. The description refers to the sectors themselves (the result content) rather than parameters, which is appropriate for a parameterless operation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb ('List') and resource ('sectors covered by the curated dataset catalog'), with concrete examples (Environment, Agriculture, Health). It also frames its distinct role as an orientation step 'before a search', clearly differentiating it from the sibling search/get/query tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'Useful for orienting the user before a search' gives a clear usage context. It does not explicitly list alternative tools or when-not-to-use conditions, but it positions this tool as the pre-search starting point, which is sufficient guidance for the sibling set provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

query_datasetA

Pull actual rows from a data.gov.in dataset, optionally filtered.

Fetches live data using the user's API key. limit is capped at 100 rows per call (the upstream maximum); page through larger result sets with offset. If a call returns zero rows but a positive total, you have paged past the end.

IMPORTANT: filter field names are case-sensitive and must match a field id from get_dataset_info exactly. If a filter returns nothing unexpectedly, check the capitalisation of the field name first.

Args: resource_id: The dataset UUID (from search_datasets or the portal). filters: Optional {field_id: value} pairs, e.g. {"city": "Delhi"}. fields: Optional list of field ids to return (projection). Omit for all. limit: Rows to return, 1-100 (default 10). offset: Row offset for pagination (default 0).

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNo
fieldsNo
offsetNo
filtersNo
resource_idYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden of behavioral disclosure — and it delivers. It reveals the API-key auth requirement, the upstream 100-row hard cap, the paged-past-the-end signature (zero rows with positive total), and the case-sensitivity pitfall for filter fields. This is exactly the kind of non-obvious behavior an agent needs before calling.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The structure is effective: a crisp opening sentence, then operational details, a highlighted IMPORTANT warning, and a per-parameter Args block. There is minor redundancy — the 100-row cap and the 1-100 limit are stated in both the prose and the Args — and the description runs long because it is dense with value; every sentence earns its place, but a tighter consolidation would be possible.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 5-parameter tool with 0% schema coverage and no annotations, the description covers everything an agent needs to call it correctly: auth requirements, pagination pitfalls, filter capitalization traps, and full parameter semantics. The output schema exists, so not explaining return values is acceptable. The description is complete enough that an agent could perform correct data pulls on the first generic run.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must fully compensate — and it does. The Args block explains what the schema cannot: where resource_id comes from (search_datasets or portal), the exact shape of filters with a concrete example ({'city': 'Delhi'}), the semantic meaning of fields as a projection, and the effective 1-100 range for limit, given that pagination semantics only exist in the description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb+resource: 'Pull actual rows from a data.gov.in dataset, optionally filtered.' This clearly differentiates it from siblings: search_datasets (discovery), list_sectors (catalog navigation), and get_dataset_info (metadata), by explicitly positioning it as the tool that retrieves the live data rows themselves.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear contextual usage guidance: it explains pagination semantics, when a zero-row result means you have paged past the end, and the dependency flow for filters ('must match a field id from get_dataset_info exactly'). It does not explicitly state 'use X instead when...' selection criteria, so exclusions between siblings are only implied, not spelled out, so a 4 is appropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_datasetsA

Find datasets on India's Open Government Data platform by keyword.

Use this FIRST when the user asks about Indian public data (air quality, crop production, fuel prices, census, health, education, rainfall, etc.) and you don't already have a resource ID. Returns matching datasets with their resource_id, which the other tools need.

Searches a curated local index of verified data.gov.in datasets. If nothing matches, tell the user they can add any dataset by copying its resource ID from the dataset's "API" button on data.gov.in, then running scripts/add_dataset.py.

Args: query: Plain-language keywords, e.g. "air quality delhi" or "wheat production". limit: Max results to return (default 10).

ParametersJSON Schema
NameRequiredDescriptionDefault
limitNo
queryYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

A4.6/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the behavioral burden and handles it well: it discloses that this searches a curated local index of verified datasets and provides a fallback path when nothing matches. It also clarifies that results are matching datasets, though it doesn't describe potential failure modes or whether the local index is refreshed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is organized into purpose, usage timing, fallback, and parameters, making it easy for an agent to parse. Every sentence adds information; nothing is redundant.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 2-parameter tool with an existing output schema, the description covers the key points: what it returns, when to invoke it, what to do if nothing matches, and how to onboard new datasets. This is sufficient for correct tool selection and invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema provides no descriptions (0% coverage), but the description fully compensates with a clear Args section. query is explained with plain-language examples, and limit is described as the max result count with a default value for schema coverage 0%.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('Find datasets'), a concrete resource (India's Open Government Data platform), and the method (by keyword). It also distinguishes the tool from get_dataset_info and query_dataset by noting that this tool returns resource_id, which the other tools need.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly says to use this tool FIRST when the user asks about Indian public data and no resource ID is already known. It implies the alternative tools are used when a resource ID is already available, though it never names them explicitly, so a direct comparison is missing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 4 tool updatesv0.1.0
    • First observedget_dataset_info
    • First observedlist_sectors
    • First observedquery_dataset
    • First observedsearch_datasets

TDQS

A4.7/5.0

Scored across 4 tools

Disambiguation5/5

Each tool has a distinct responsibility: discover datasets (search_datasets), browse categories (list_sectors), inspect schema (get_dataset_info), and retrieve rows (query_dataset). No two tools could be confused for the same action.

Naming Consistency5/5

All four tool names follow a uniform verb_noun pattern (search_datasets, list_sectors, get_dataset_info, query_dataset). While the verbs differ, each one accurately maps to its unique action, and the snake_case style is consistent throughout.

Tool Count5/5

With only 4 tools, the server is tightly scoped to the core use case of discovering and retrieving Indian open government data. Each tool is necessary and none feel redundant, making the tool count highly appropriate.

Completeness4/5

The tool surface covers the end-to-end read-only workflow: search, sector overview, schema inspection, and data extraction with pagination. Minor gaps include no direct way to list every dataset in the catalog and limited dataset metadata (e.g., update date), but agents can work around these via search and get_dataset_info.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    A
    maintenance
    Description: Query India's open geo data in natural language. 8 tools: list layers, inspect schemas, filter/group any column, point-in-polygon locate, spatial proximity search, downloads in 5 formats. Covers admin boundaries (state to village), city wards, forests, rivers, dams, hospitals, highways, airports, and more.
    39
    MIT
  • A
    license
    A
    quality
    C
    maintenance
    Enables querying and analyzing over 90,000 public datasets from the Spanish Government Open Data Portal (datos.gob.es) using natural language, with tools for search, filtering, metadata access, and SPARQL queries.
    10
    5
    MIT
  • A
    license
    Not graded
    quality
    C
    maintenance
    Provides AI-ready access to Indian government statistics through MCP, enabling natural language queries for economic, demographic, and social indicators.
    138
    MIT