python-docs-mcp

An MCP (Model Context Protocol) server that gives AI coding agents live CPython standard-library documentation and PyPI package information instead of whatever the model happens to remember. Module pages are scraped from docs.python.org at request time, cached locally, and returned as clean markdown — so generated Python targets real, current APIs rather than deprecated or invented ones.

• Stdlib modules — every module page fetched live from docs.python.org/3/library and converted to markdown (the /3/ line is whatever CPython currently documents — measured 3.14.8)

• PyPI packages — metadata + README read from the package's robots-allowed project page, optionally pinned to a version (see PyPI source)

• Local search index over all 229 stdlib modules (name + description recorded, so ElementTree resolves to xml.etree.elementtree on its own), rebuilt at most every 7 days with a stale fallback when the network fails

• SQLite TTL cache so repeated lookups are instant

• Politeness layer — robots.txt (RFC 9309), per-host throttle, Retry-After, conditional GET and request budgets (details)

The five tools

Every tool returns a plain dict. Failures come back as {"ok": false, "error": …, "suggestion": …} — a bad lookup never raises into the MCP layer, and no tool delegates to another tool.

Related MCP server: context

Requirements

• Python 3.10 or newer (developed and tested on 3.12)

• uv for the one-command install below (uvx ships with it)

• The MCP Python SDK 1.x is pinned (mcp>=1.2.0,<2.0); this package does not support the mcp 2.x rename of FastMCP

Install & run

One command — no checkout, no token

uvx --from git+https://github.com/KEEPEE/python-docs-mcp.git python-docs

This builds the package in an isolated environment and starts the stdio MCP server. It prints nothing on purpose: stdout is the protocol channel. Stop it with Ctrl-C.

After the repository is updated, force uv to re-resolve the commit:

uvx --refresh --from git+https://github.com/KEEPEE/python-docs-mcp.git python-docs

Local checkout — fastest startup, editable while developing

git clone https://github.com/KEEPEE/python-docs-mcp.git
cd python-docs-mcp
python3 -m venv .venv
.venv/bin/pip install -e .
.venv/bin/python-docs        # stdio MCP server

MCP client configuration

Generic stdio client (Claude Desktop, Cursor, Cline, …)

{
  "mcpServers": {
    "python-docs": {
      "command": "uvx",
      "args": [
        "--from",
        "git+https://github.com/KEEPEE/python-docs-mcp.git",
        "python-docs"
      ]
    }
  }
}

With an explicit cache directory (any env you set is passed straight through to the server):

{
  "mcpServers": {
    "python-docs": {
      "command": "uvx",
      "args": [
        "--from",
        "git+https://github.com/KEEPEE/python-docs-mcp.git",
        "python-docs"
      ],
      "env": {
        "PYTHON_DOCS_MCP_CACHE_DIR": "/tmp/python-docs-cache"
      }
    }
  }
}

DeepSeek Harness (cordis-style plugin list)

- id: mcp-python-docs
  name: '@deepseek-ai/dsh-mcp-client'
  config:
    serverName: python-docs
    transport: stdio
    command: uvx
    args:
      [
        '--from',
        'git+https://github.com/KEEPEE/python-docs-mcp.git',
        'python-docs'
      ]

To skip the build at client start, point command at the console script of a checkout instead — command: /path/to/python-docs-mcp/.venv/bin/python-docs with args: [].

Tools reference

python_docs(identifier, topic=None, max_tokens=8000)

Resolves an identifier to one module page. Identifier forms, tried in this order:

topic keeps only the section whose heading matches it (Basic Usage, Command-line interface, …) plus the page title and description; if nothing matches, the full page is returned with a note listing the available headings. max_tokens is a rough budget (1 token ≈ 4 characters): the markdown is cut at a line boundary and truncated is set.

Returns {"ok", "identifier", "url", "title", "markdown", "truncated", "cached"}, plus an optional note.

// python_docs({ "identifier": "xml.etree.ElementTree", "max_tokens": 250 })
{
  "ok": true,
  "identifier": "xml.etree.ElementTree",
  "url": "https://docs.python.org/3/library/xml.etree.elementtree.html",
  "markdown": "# `xml.etree.ElementTree` — The ElementTree XML API  **Source code:** [Lib/xml/etree/ElementTree.py](https://github.com/python/cpython/tree/3.14/Lib/xml/etree/ElementTree.py)  ---  The `xml.etree.ElementTree` module implements a simple and efficient API for pa … [truncated]",
  "truncated": true,
  "cached": false,
  "title": "xml.etree.ElementTree — The ElementTree XML API"
}

A bare name is resolved through the index — same page, no guessing:

// python_docs({ "identifier": "ElementTree", "max_tokens": 160 })
{
  "ok": true,
  "identifier": "ElementTree",
  "url": "https://docs.python.org/3/library/xml.etree.elementtree.html",
  "title": "xml.etree.ElementTree — The ElementTree XML API",
  "…": "…"
}

A topic that matches nothing tells you what the page does have:

// python_docs({ "identifier": "json", "topic": "nonexistent-section", "max_tokens": 120 })
{
  "ok": true,
  "identifier": "json",
  "url": "https://docs.python.org/3/library/json.html",
  "title": "json — JSON encoder and decoder",
  "…": "…",
  "note": "no section matching 'nonexistent-section'; available headings: `json` — JSON encoder and decoder, Basic Usage, Encoders and Decoders, Let the base class default method raise the TypeError, Exceptions, Standard Compliance and Interoperability, Character Encodings, Infinite and NaN Number Values, Repeated Names Within an Object, Top-level Non-Object, Non-Array Values, Implementation Limitations, Command-line interface, Command-line options"
}

A name that is not a module is an error dict, not an empty page:

// python_docs({ "identifier": "nope-not-a-real-module-xyz" })
{
  "ok": false,
  "error": "no stdlib module matching 'nope-not-a-real-module-xyz' found in the search index",
  "suggestion": "use python_search('nope-not-a-real-module-xyz') to find the right module name"
}

python_search(query, limit=8)

Multi-token ranked search over the local index: every token must match, and both the module name and its one-line description are searched. Returns {"ok", "query", "results": [{module, description, score}], "index_count", "stale"}.

// python_search({ "query": "zip file", "limit": 3 })
{
  "ok": true,
  "query": "zip file",
  "results": [
    { "module": "zipfile", "description": "Work with ZIP archives", "score": 14.0 },
    { "module": "gzip",    "description": "Support for gzip files", "score": 9.0  }
  ],
  "index_count": 229,
  "stale": false
}

Description matching is what makes this useful when you know the job, not the name:

// python_search({ "query": "element tree", "limit": 3 })
{ "results": [ { "module": "xml.etree.elementtree", "description": "The ElementTree XML API", "score": 12.0 } ], "…": "…" }

// python_search({ "query": "tokeniz", "limit": 3 })   // prefix match
{ "results": [ { "module": "tokenize", "description": "Tokenizer for Python source", "score": 8.0 } ], "…": "…" }

python_search costs no requests while the cached index is fresh; if the index is stale the call rebuilds it (one robots fetch + one library-index page) inside the same budget.

pypi_package(name, version=None)

PyPI metadata + README. name is the distribution name; version pins one release, which is how you read the README a specific version shipped. Returns {"ok", "name", "version" (resolved), "summary", "requires_python", "license", "homepage", "project_urls", "readme_markdown", "url"} — url is always the human-facing project page. Cached 1 day, keyed by name + version.

// pypi_package({ "name": "attrs" })
{
  "ok": true,
  "name": "attrs",
  "version": "26.1.0",
  "summary": "Classes Without Boilerplate",
  "requires_python": ">=3.9",
  "license": "MIT",
  "homepage": "https://github.com/python-attrs/attrs",
  "project_urls": {
    "GitHub": "https://github.com/python-attrs/attrs",
    "Changelog": "https://www.attrs.org/en/stable/changelog.html",
    "Documentation": "https://www.attrs.org/",
    "Funding": "https://github.com/sponsors/hynek",
    "Tidelift": "https://tidelift.com/subscription/pkg/pypi-attrs?utm_source=pypi-attrs&utm_medium=pypi"
  },
  "readme_markdown": "[![attrs](https://pypi-camo.freetls.fastly.net/5bfa3846…)] …",
  "url": "https://pypi.org/project/attrs/26.1.0/",
  "cached": false
}

A pinned version resolves to exactly that release:

// pypi_package({ "name": "six", "version": "1.16.0" })
{
  "ok": true,
  "name": "six",
  "version": "1.16.0",
  "summary": "Python 2 and 3 compatibility utilities",
  "requires_python": ">=2.7,  !=3.0.*,  !=3.1.*,  !=3.2.*",
  "license": "MIT License (MIT)",
  "homepage": "https://github.com/benjaminp/six",
  "url": "https://pypi.org/project/six/1.16.0/",
  "…": "…"
}

An unknown name is an error dict — and on PyPI the honest error message is two-sided, because PyPI does not answer unknown names with a plain 404 (see the challenge note):

// pypi_package({ "name": "nope-not-a-real-package-xyz" })
{
  "ok": false,
  "error": "pypi.org answered with a JavaScript bot-verification page (HTTP 200, not a 404) instead of the project page. Either PyPI is serving its anti-bot challenge for this URL, or the package does not exist — PyPI also answers unknown package names with that page, so a challenge is not proof the name is valid",
  "suggestion": "retry later (PyPI serves that page for some cold URLs), check the name (case-sensitive) or drop the version pin, or use the PYTHON_DOCS_MCP_PYPI_JSON_OPT_IN=1 opt-in knowing that PyPI's robots.txt disallows the JSON API"
}

python_status()

Probes docs.python.org and pypi.org with a light GET (both on robots-allowed paths) and reports the index and cache state. overall is ok, degraded or error. The politeness block is diagnostics only and never changes overall.

// python_status()
{
  "server": "python-docs-mcp",
  "version": "0.2.0",
  "checks": {
    "search_index":    { "status": "ok", "entries": 229, "built_at": "2026-10-07T08:04:39.802131+00:00", "stale": false },
    "cache":           { "status": "ok", "entries": 11, "expired": 0 },
    "docs_python_org": { "status": "ok", "http_status": 200 },
    "pypi_org":        { "status": "ok", "http_status": 200 }
  },
  "politeness": {
    "status": "ok", "disabled": false,
    "requests": 10, "robots_requests": 2, "robots_rows": 2, "robots_fetches": 2, "robots_cache_hits": 7,
    "blocked_by_robots": 0, "throttle_waits": 8, "throttle_sleep_s": 3.626,
    "challenge_detected": 2, "challenge_retries": 1,
    "host_delays": { "docs.python.org": 0.0, "pypi.org": 0.913 },
    "budgets": { "tool:pypi_package": [2, 6], "tool:python_status": [2, 8] },
    "budget_denied": 0, "conditional": 1, "conditional_skipped": 0, "revalidated_304": 1,
    "retries_429": 0, "retries_transport": 0, "stalls": 0
  },
  "overall": "ok"
}

health_check()

{"status": "ok", "server": "python-docs-mcp", "version": "0.2.0"} — no network, no cache; safe as a liveness probe.

PyPI source (and robots.txt)

This is the one place where this server deliberately does not use the API you might expect, and it is worth understanding before you change it.

PyPI's robots.txt (325 bytes, 12 Disallow lines for User-agent: *) says:

Disallow: /simple/
Disallow: /packages/
Disallow: /_includes/authed/
Disallow: /project/*/submit-malware-report/
Disallow: /pypi/*/json
Disallow: /pypi/*/*/json
Disallow: /pypi*?
Disallow: /search*
Disallow: /_/
Disallow: /integrity/
Disallow: /account/
Disallow: /admin/

pypi_package() reads its metadata from the robots-allowed project page, https://pypi.org/project/<name>/ (or /project/<name>/<version>/), because Disallow: /pypi/*/json and Disallow: /pypi/*/*/json forbid the JSON API this package used to call. /simple/ is not a way around it — it is disallowed too, and it carries no README or project URLs. (Python's own urllib.robotparser did not catch the original violation: on Python < 3.14 it reads /pypi/*/json as a literal prefix and reports that URL as allowed. The RFC 9309 matcher in politeness.py reports it as blocked — verified live: can_fetch("https://pypi.org/pypi/attrs/json") → False, can_fetch("https://pypi.org/project/attrs/") → True.)

The page parser pulls the same fields out of the rendered HTML — name, resolved version, summary, Requires-Python, license (both the modern "License expression" block and the legacy "License" sidebar entry), homepage, project URLs and the README (RST → markdown) — and pypi_package returns exactly the same keys as before, so nothing downstream changed.

The JSON API is opt-in only

export PYTHON_DOCS_MCP_PYPI_JSON_OPT_IN=1

With that set, pypi_package uses https://pypi.org/pypi/<name>/json again. That endpoint is disallowed by robots.txt, so using it is your responsibility, and every result produced that way carries a notice field saying so — verbatim:

{
  "…": "…",
  "notice": "PyPI's robots.txt disallows /pypi/*/json and /pypi/*/*/json for every user-agent. The JSON API was used because PYTHON_DOCS_MCP_PYPI_JSON_OPT_IN=1 is set; that is a deliberate, user-authorised bypass of the politeness layer and it is the user's responsibility."
}

With the opt-in off (the default) the JSON endpoint is never requested — not even as a fallback when the project page fails, and the politeness layer blocks that URL independently, so the violation cannot come back through another door. With the opt-in on, the project page is never requested. The two sources never fall back into one another.

Known trade-off: the JS "Client Challenge"

PyPI's CDN (Fastly) answers some project-page URLs with a ~3 KB JavaScript Client Challenge instead of the page — HTTP 200, not a 404. Measured on this repo's fixtures: the challenge page is 3,038 bytes, a real attrs project page is 130,507 bytes.

This is a real trade-off of reading the allowed HTML page instead of the JSON API, and it has a sharp edge you should know about:

A package that does not exist can look exactly like a bot challenge. PyPI answers unknown package names with that same challenge page — reproduced live on 2026-10-07: pypi.org/project/nope-not-a-real-package-xyz/ returned HTTP 200 with the challenge body. A challenge is therefore not proof the name is valid, and a 404 is not the only way a name can be wrong.

The tool reports both possibilities instead of guessing:

// the error text, verbatim
"pypi.org answered with a JavaScript bot-verification page (HTTP 200, not a 404) instead of the project page. Either PyPI is serving its anti-bot challenge for this URL, or the package does not exist — PyPI also answers unknown package names with that page, so a challenge is not proof the name is valid"

What the code does about it:

• the politeness layer's optional challenge_detector hook is wired to a PyPI-specific predicate (fetchers.pypi_challenge_detector): small body + HTML + a challenge marker (_fs-ch-, "Client Challenge") + none of the markers a real project page always carries (project-header__name, package-header__name, pip-command);

• when it fires, the layer waits (escalated per-host delay) and retries that same URL once — never another endpoint. Measured: challenge_detected: 2, challenge_retries: 1, and the whole lookup spends 3 of its 6 budget units;

• if the answer is still a challenge, the body is never parsed into a fake package, never cached, and never answered with a fallback to the robots-disallowed JSON endpoint. A genuine page is fetched exactly once — the detector does not add a request for it (covered by tests/test_politeness_wiring.py).

Caching & politeness

Cache

Everything lives in one directory: ~/.cache/python-docs-mcp by default, overridable with PYTHON_DOCS_MCP_CACHE_DIR.

Delete the files to force a full refresh. A cache problem is never a tool failure: if the directory is unwritable the server simply runs without a cache and says so in python_status().checks.cache.

Politeness

Every outbound request — python_docs, python_search (its index build), pypi_package and the python_status probes — goes through one small internal module, src/python_docs_mcp/politeness.py (stdlib + httpx, no extra dependency):

• robots.txt is read and obeyed. Rules are matched per RFC 9309 (*, trailing $, %2A/%24 literals, merged User-agent groups, most-specific match wins, Allow beats Disallow on a tie). urllib.robotparser is deliberately not used: on Python < 3.14 it ignores wildcards, which is exactly how this server once violated PyPI (see PyPI source). docs.python.org publishes 22 Disallow lines in a 517-byte file — /dev, /release and the EOL version lines (/2/ … /3.10/) — and this server fetches /3/library/…; if an older URL ever reached the layer the answer is {"ok": false, "error": "… blocked by robots.txt …"} with no request made (verified live: can_fetch("https://docs.python.org/3.10/library/json.html") → False). Files are cached 7 days per host, including negative results (404 / 403 / 5xx), so a host costs one robots request per week.

• Per-host throttle. Sequential requests to one host are spaced 0.35–0.9 s by default, or by the site's own Crawl-delay when it declares one — neither docs.python.org nor pypi.org declares a Crawl-delay today (checked against both live robots.txt files), so the default window applies. An audit of this server measured a 7 ms minimum gap between docs.python.org requests during a cold index build; with the layer the same build waits out the full delay (measured cold run: 2 requests on the wire, throttle_sleep_s: 0.784).

• 429 / 503 / 504 are handled. Retry-After is honoured to the second; without it the per-host delay is escalated and the request retried once. A response that stalls past 10 s escalates the delay instead of being retried blindly.

• Conditional GET. ETag / Last-Modified and the raw body are stored next to the cached entry, so refreshing an expired page costs a 304 instead of re-downloading it. Verified against live headers: docs.python.org module pages send both ETag and Last-Modified; pypi.org project pages send an ETag and no Last-Modified. A warm run revalidated one entry with a 304 (revalidated_304: 1).

• Request budgets — one unit is one request on the wire. One tool call makes at most 6 requests; the robots.txt fetch of a cold host, a 429 retry, a transport retry and every redirect hop each pay a unit. Measured live with a cold cache: python_docs("ElementTree") = 3 (robots + library-index page + module page), python_docs("json") = 2, python_docs("xml.etree.ElementTree") = 2 (the lowercase page is tried first, so no wasted 404), python_docs("xml.etree.NopeXYZ") = 3 (robots + both candidate spellings), an unknown plain name = 2 (the index answers before any page is fetched), pypi_package("attrs") = 2, a challenge-hit PyPI lookup = 3. python_status has its own cap of 8 (measured cold 5, warm 2) so its probes plus a stale-index rebuild always fit — a probe refused by the budget would report error and drag overall to "degraded" for a purely internal reason. The search-index build shares the tool call's budget; a build that hits its cap stops early and returns a partial index ("partial": true) instead of raising, and a partial index is never cached.

• Redirects are resolved by the layer, not by httpx. Requests go out with follow_redirects=False, every 3xx target is re-checked against that host's robots rules before its body is used, the chain is capped at 5 hops and reported as an error (never an exception), and the url a tool returns is the final URL of the chain.

• Host allowlist. Only docs.python.org and pypi.org can be contacted, so a malformed identifier or an unexpected redirect cannot turn a docs lookup into a request somewhere else.

• One transport, no curl fallback. An earlier version shelled out to curl when httpx failed. That second transport is gone and stays gone: it bypassed robots.txt, the throttle, the budget and the stored validators, so a fallback request was an unpolite request; it doubled the worst case; and the premise it was written for — that some edges throttle Python's TLS fingerprint — did not reproduce under measurement.

The layer never raises and never changes a tool's return shape. python_status reports its counters under the top-level politeness key.

Opt-out

export PYTHON_DOCS_MCP_POLITENESS_DISABLED=1

This single switch turns off robots.txt, the throttle, retries, conditional GET and the host allowlist at once. Use it at your own risk: you take over responsibility for respecting each site's crawling rules, and you are far more likely to be rate-limited or blocked. There is no partial opt-out.

Development

python3 -m venv .venv
.venv/bin/pip install -e ".[dev]"

.venv/bin/python -m pytest -q              # 191 offline tests; fixtures in tests/fixtures/
.venv/bin/python scripts/politeness_smoke.py /tmp/python-docs-smoke-cache
.venv/bin/python scripts/e2e_mcp_test.py

• pytest is fully offline: HTTP is simulated with httpx.MockTransport and time/jitter are injected, so the suite is deterministic and runs in seconds. tests/fixtures/ holds real captured pages: a docs.python.org module page (112 KB), the full library index (73 KB), two real PyPI project pages (130 KB and 89 KB), the JS challenge page PyPI's CDN returns for some URLs (3 KB) and a PyPI JSON-API payload (605 KB) used to test the opt-in parser.

• scripts/politeness_smoke.py <cache-dir> is a live measurement, not a test: it drives the real tools against the real docs sites and prints what the politeness layer did (requests, robots, throttle, conditional GET, challenge detection, budgets).

• scripts/e2e_mcp_test.py spawns the installed python-docs console script and speaks newline-delimited JSON-RPC to it (initialize → tools/list → every tool → a negative case → health_check). Exit code 0 means every check passed. It resolves the server command as PYTHON_DOCS_MCP_E2E_CMD (env override) → this checkout's .venv/bin/python-docs → python -m python_docs_mcp.server.

Troubleshooting

The first python_docs call is slower than the rest. That is the cold search-index build: the library-index page plus a robots fetch, spaced by the per-host throttle, followed by the module page you actually asked for. It happens once every 7 days; later calls hit the cached index. Delete cache.db and you pay for it again.

request budget exhausted for scope 'tool:python_docs'. One tool call hit its cap of 6 requests. python_search first — it is offline and free — and pass the exact dotted module path. python_status().politeness.budgets shows how much each scope spent.

"partial": true, or fewer search results than expected. An index build ran out of its budget and stopped early, returning a partial index with a partial_reason. A partial index is not cached, so the next run rebuilds it. python_search still works — it just knows fewer modules — and python_status().checks.search_index.entries tells you how many it has (a complete index is 229 modules).

"stale": true in search results. The rebuild failed (offline, DNS failure, 5xx) and the server fell back to the older cached index instead of failing the lookup.

Nothing is cached and overall is degraded. Read python_status().checks.cache.error — an unwritable PYTHON_DOCS_MCP_CACHE_DIR (read-only mount, missing permission) is the usual cause. Tools keep working without a cache; they are just slower and noisier on the network.

A lookup returns blocked by robots.txt. The site's rules disallow that path for this user agent, and the request was not sent. For docs.python.org that normally means an EOL version line (/2/ … /3.10/) or /dev; for PyPI it means the JSON API or /simple/. Fetch the page yourself, or accept the consequences of the opt-out switch above.

pypi_package says "JavaScript bot-verification page". That is PyPI's CDN, not a bug in this server: the URL came back HTTP 200 with the challenge body. Retry later, check the name's case, drop the version pin — and remember that an unknown name can produce the same page (why).

A mixed-case module name can cost two requests. docs.python.org serves module pages only under lowercase filenames, so xml.etree.ElementTree is fetched as xml.etree.elementtree.html first and the normal case costs one page request; only when the lowercase page genuinely does not exist does the fetcher spend a second request on the spelling you gave.

License & attribution

MIT — see LICENSE. Copyright (c) 2026 Michal Gaspierik.

The design of the politeness layer was inspired by Crawl4AI (© Unclecode, Apache-2.0): a TTL-cached robots store, the wildcard rule translation, and a per-domain rate limiter with escalating backoff. The implementation is a clean-room rewrite in this project's own synchronous stdlib-plus-httpx style — no line was transcribed, translated or mechanically adapted from Crawl4AI, and Crawl4AI is not a dependency of this package. Three defects of the original design are fixed (robots fetched_at refresh, negative-result caching, Crawl-delay support), and the RFC 9309 matcher is what makes the PyPI rules above actually bind — robotparser would have let the JSON API through. The GPL-3.0 part of Crawl4AI — its vendored html2text fork — is deliberately excluded: no code, data or dependency from that tree is used or shipped here; HTML→markdown stays markdownify's job. The full statement is in NOTICE.

Same clean-room architecture as flutter-mcp and java-spring-mcp: fetchers → pure parsers → TTL cache → search index → tools, with no circular delegation, a pinned 1.x MCP SDK, error-dict failures and an offline test suite.