RTFD (Read The F*****g Docs)

Instructions

        Fetch actual Python package documentation from PyPI README/description.

        USE THIS WHEN: You need installation instructions, usage examples, API reference, or quickstart guides.

        BEST FOR: Getting complete, formatted documentation for Python packages.
        Better than using curl or WebFetch because it:
        - Automatically extracts relevant sections (Installation, Usage, Examples)
        - Converts reStructuredText to Markdown
        - Prioritizes most useful content sections
        - Falls back to GitHub README if PyPI description is minimal

        NOT SUITABLE FOR: External documentation sites (use docs_url from pypi_metadata + WebFetch)

        Args:
            package: PyPI package name (e.g., "requests", "numpy", "pandas")
            max_bytes: Maximum content size, default 20KB (increase for large packages)
            ignore_verification: Skip PyPI verification check if VERIFIED_BY_PYPI is enabled

        Returns: JSON with actual documentation content, size, truncation status

        Example: fetch_pypi_docs("requests") → Returns formatted README with installation and usage
        

Implementation Reference

• src/RTFD/providers/pypi.py:216-243 (handler)

  Main handler function for the fetch_pypi_docs tool. Calls the private _fetch_pypi_docs helper and serializes the response for MCP.

  async def fetch_pypi_docs(
      package: str, max_bytes: int = 20480, ignore_verification: bool = False
  ) -> CallToolResult:
      """
      Fetch actual Python package documentation from PyPI README/description.
  
      USE THIS WHEN: You need installation instructions, usage examples, API reference, or quickstart guides.
  
      BEST FOR: Getting complete, formatted documentation for Python packages.
      Better than using curl or WebFetch because it:
      - Automatically extracts relevant sections (Installation, Usage, Examples)
      - Converts reStructuredText to Markdown
      - Prioritizes most useful content sections
      - Falls back to GitHub README if PyPI description is minimal
  
      NOT SUITABLE FOR: External documentation sites (use docs_url from pypi_metadata + WebFetch)
  
      Args:
          package: PyPI package name (e.g., "requests", "numpy", "pandas")
          max_bytes: Maximum content size, default 20KB (increase for large packages)
          ignore_verification: Skip PyPI verification check if VERIFIED_BY_PYPI is enabled
  
      Returns: JSON with actual documentation content, size, truncation status
  
      Example: fetch_pypi_docs("requests") → Returns formatted README with installation and usage
      """
      result = await self._fetch_pypi_docs(package, max_bytes, ignore_verification)
      return serialize_response_with_meta(result)

• src/RTFD/providers/pypi.py:112-189 (helper)

  Private helper method containing the core logic for fetching, processing, and prioritizing PyPI package documentation.

  async def _fetch_pypi_docs(
      self, package: str, max_bytes: int = 20480, ignore_verification: bool = False
  ) -> dict[str, Any]:
      """
      Fetch documentation content for PyPI package.
  
      Args:
          package: Package name
          max_bytes: Maximum content size in bytes
          ignore_verification: Whether to ignore PyPI verification status
  
      Returns:
          Dict with content, size, source, etc.
      """
      try:
          # 1. Get metadata to find description and project URLs
          # This will also perform the verification check
          metadata = await self._fetch_metadata(package, ignore_verification)
  
          if metadata.get("error"):
              return {
                  "package": package,
                  "content": "",
                  "error": metadata["error"],
                  "size_bytes": 0,
                  "source": None,
              }
  
          # 2. Try PyPI description (often reStructuredText)
          content = metadata.get("description", "")
          source = "pypi"
  
          # 3. Convert reST to Markdown if needed
          # Check for common reST markers
          if content and (".. " in content or "::" in content[:200]):
              content = convert_rst_to_markdown(content)
  
          # 4. If insufficient content, try GitHub README
          if len(content.encode("utf-8")) < 500:
              repo_url = self._extract_github_url(metadata.get("project_urls", {}))
              if repo_url:
                  # For now, note that GitHub fallback requires github provider
                  # This will be implemented after GitHub provider is updated
                  # source = "pypi_minimal"
                  pass
  
          # 5. Extract and prioritize sections
          sections = extract_sections(content)
          if sections:
              final_content = prioritize_sections(sections, max_bytes)
          else:
              final_content = content[:max_bytes] if content else ""
  
          return {
              "package": package,
              "content": final_content,
              "size_bytes": len(final_content.encode("utf-8")),
              "source": source,
              "truncated": len(content.encode("utf-8")) > max_bytes,
          }
  
      except httpx.HTTPStatusError as exc:
          return {
              "package": package,
              "content": "",
              "error": f"PyPI returned {exc.response.status_code}",
              "size_bytes": 0,
              "source": None,
          }
      except Exception as exc:
          return {
              "package": package,
              "content": "",
              "error": f"Failed to fetch docs: {exc!s}",
              "size_bytes": 0,
              "source": None,
          }

• src/RTFD/providers/pypi.py:245-249 (registration)

  Registers the fetch_pypi_docs tool function in the provider's tools dictionary, conditionally if fetch is enabled.

  tools = {"pypi_metadata": pypi_metadata}
  if is_fetch_enabled():
      tools["fetch_pypi_docs"] = fetch_pypi_docs
  
  return tools

• src/RTFD/providers/pypi.py:22-24 (registration)

  Conditionally adds 'fetch_pypi_docs' to the provider metadata's tool_names list.

  if is_fetch_enabled():
      tool_names.append("fetch_pypi_docs")