ds-directory-mcp

Self-hosted, free-tier semantic search (RAG) + MCP server for external design system documentation. Replaces keyword-based search (e.g. Google Programmable Search) with intent-based vector search over design tokens, component architectures, and code patterns — exposed to any MCP-compatible AI assistant via one tool: design_system_directory.

Architecture

[Phase 1: Ingestion]  ──>  [Phase 2: Storage & Search]  ──>  [Phase 3: MCP Gateway]
 ingest.py (crawl,             Qdrant Cloud                    server.py
 chunk, embed)                 (vector DB, free tier)          (FastMCP, Render free tier)

Related MCP server: linked-docs

Setup

1. Create a free Qdrant Cloud cluster.

2. python -m venv .venv && source .venv/bin/activate

3. pip install -r requirements.txt (first run also downloads the local embedding model, ~130MB, cached after that)

4. cp .env.example .env and fill in QDRANT_URL, QDRANT_API_KEY.

Ingesting a design system

python ingest.py "Atlassian Design System" https://atlassian.design/components

Crawls same-domain links from the given start URL(s), strips nav/footer/script noise, chunks text (800 chars, 100 overlap), embeds each chunk, and upserts into the design_system_index Qdrant collection with {url, design_system_name, text_content} payloads.

Running the MCP server locally

python server.py

Serves design_system_directory(user_query: str) over Streamable HTTP.

Adding design systems

Register each one in systems.yaml (name + start URL(s)) instead of typing the command by hand each time:

python ingest.py --system "Shopify Polaris"   # one entry from the registry
python ingest.py --all                        # every entry in the registry

Re-running a system deletes its previously indexed chunks first (matched by design_system_name), so re-ingestion replaces stale content instead of piling up duplicates.

Automated re-indexing (GitHub Actions, free)

.github/workflows/reindex.yml runs python ingest.py --all on a weekly cron (also triggerable manually via "Run workflow"). This runs on GitHub's free Actions minutes — no always-on server needed, unlike the MCP server itself. Set these as repo secrets (Settings → Secrets and variables → Actions), never commit them:

• QDRANT_URL

• QDRANT_API_KEY

Render hosts the always-on MCP query server; GitHub Actions handles the periodic batch re-crawl — different lifecycles, so they're split across two free hosts.

Deploying (Render free tier)

render.yaml is included — connect this repo in the Render dashboard ("New +" → "Blueprint"), and set QDRANT_URL, QDRANT_API_KEY as secrets in the service's environment settings (never commit them). Render's free tier sleeps on idle — the first request after a period of inactivity will be slow ("cold start"); surface a loading state for this in any client UI. It's also only 512MB RAM — untested whether that's enough headroom for sentence-transformers + the embedding model alongside the MCP server itself; if embed_query() OOMs in practice, the fix is either a smaller model or Render's cheapest paid tier, not a code change.

Notes

• Secrets live only in environment variables (local .env, or the host's dashboard) — never in git.

• Ingestion sleeps 1s between page fetches to respect target servers.

• At 768 dimensions, ~100k vectors uses ~300MB — comfortably under Qdrant's 1GB RAM ceiling.