MCP LLMS-TXT Documentation Server

# MCP LLMS-TXT Documentation Server ## Overview [llms.txt](https://llmstxt.org/) is a website index for LLMs, providing background information, guidance, and links to detailed markdown files. IDEs like Cursor and Windsurf or apps like Claude Code/Desktop can use `llms.txt` to retrieve context for tasks. However, these apps use different built-in tools to read and process files like `llms.txt`. The retrieval process can be opaque, and there is not always a way to audit the tool calls or the context returned. [MCP](https://github.com/modelcontextprotocol) offers a way for developers to have *full control* over tools used by these applications. Here, we create [an open source MCP server](https://github.com/modelcontextprotocol) to provide MCP host applications (e.g., Cursor, Windsurf, Claude Code/Desktop) with (1) a user-defined list of `llms.txt` files and (2) a simple `fetch_docs` tool read URLs within any of the provided `llms.txt` files. This allows the user to audit each tool call as well as the context returned. <img src="https://github.com/user-attachments/assets/736f8f55-833d-4200-b833-5fca01a09e1b" width="60%"> ## llms-txt You can find llms.txt files for langgraph and langchain here: | Library | llms.txt | |------------------|------------------------------------------------------------------------------------------------------------| | LangGraph Python | [https://langchain-ai.github.io/langgraph/llms.txt](https://langchain-ai.github.io/langgraph/llms.txt) | | LangGraph JS | [https://langchain-ai.github.io/langgraphjs/llms.txt](https://langchain-ai.github.io/langgraphjs/llms.txt) | | LangChain Python | [https://python.langchain.com/llms.txt](https://python.langchain.com/llms.txt) | | LangChain JS | [https://js.langchain.com/llms.txt](https://js.langchain.com/llms.txt) | ## Quickstart #### Install uv * Please see [official uv docs](https://docs.astral.sh/uv/getting-started/installation/#installation-methods) for other ways to install `uv`. ```bash curl -LsSf https://astral.sh/uv/install.sh | sh ``` #### Choose an `llms.txt` file to use. * For example, [here's](https://langchain-ai.github.io/langgraph/llms.txt) the LangGraph `llms.txt` file. > **Note: Security and Domain Access Control** > > For security reasons, mcpdoc implements strict domain access controls: > > 1. **Remote llms.txt files**: When you specify a remote llms.txt URL (e.g., `https://langchain-ai.github.io/langgraph/llms.txt`), mcpdoc automatically adds only that specific domain (`langchain-ai.github.io`) to the allowed domains list. This means the tool can only fetch documentation from URLs on that domain. > > 2. **Local llms.txt files**: When using a local file, NO domains are automatically added to the allowed list. You MUST explicitly specify which domains to allow using the `--allowed-domains` parameter. > > 3. **Adding additional domains**: To allow fetching from domains beyond those automatically included: > - Use `--allowed-domains domain1.com domain2.com` to add specific domains > - Use `--allowed-domains '*'` to allow all domains (use with caution) > > This security measure prevents unauthorized access to domains not explicitly approved by the user, ensuring that documentation can only be retrieved from trusted sources. #### (Optional) Test the MCP server locally with your `llms.txt` file(s) of choice: ```bash uvx --from mcpdoc mcpdoc \ --urls LangGraph:https://langchain-ai.github.io/langgraph/llms.txt \ --urls LangChain:https://python.langchain.com/llms.txt \ --transport sse \ --port 8082 \ --host localhost ``` * This should run at: http://localhost:8082  * Run [MCP inspector](https://modelcontextprotocol.io/docs/tools/inspector) and connect to the running server: ```bash npx @modelcontextprotocol/inspector ```  * Here, you can test the `tool` calls. #### Connect to Cursor * Open `Cursor Settings` and `MCP` tab. * This will open the `~/.cursor/mcp.json` file.  * Paste the following into the file (we use the `langgraph-docs-mcp` name and link to the LangGraph `llms.txt`). ``` { "mcpServers": { "langgraph-docs-mcp": { "command": "uvx", "args": [ "--from", "mcpdoc", "mcpdoc", "--urls", "LangGraph:https://langchain-ai.github.io/langgraph/llms.txt", "--urls", "LangChain:https://python.langchain.com/llms.txt", "--transport", "stdio" ] } } } ``` * Confirm that the server is running in your `Cursor Settings/MCP` tab. * Best practice is to then update Cursor Global (User) rules. * Open Cursor `Settings/Rules` and update `User Rules` with the following (or similar): ``` for ANY question about LangGraph, use the langgraph-docs-mcp server to help answer -- + call list_doc_sources tool to get the available llms.txt file + call fetch_docs tool to read it + reflect on the urls in llms.txt + reflect on the input question + call fetch_docs on any urls relevant to the question + use this to answer the question ``` * `CMD+L` (on Mac) to open chat. * Ensure `agent` is selected.  Then, try an example prompt, such as: ``` what are types of memory in LangGraph? ```  ### Connect to Windsurf * Open Cascade with `CMD+L` (on Mac). * Click `Configure MCP` to open the config file, `~/.codeium/windsurf/mcp_config.json`. * Update with `langgraph-docs-mcp` as noted above.  * Update `Windsurf Rules/Global rules` with the following (or similar): ``` for ANY question about LangGraph, use the langgraph-docs-mcp server to help answer -- + call list_doc_sources tool to get the available llms.txt file + call fetch_docs tool to read it + reflect on the urls in llms.txt + reflect on the input question + call fetch_docs on any urls relevant to the question ```  Then, try the example prompt: * It will perform your tool calls.  ### Connect to Claude Desktop * Open `Settings/Developer` to update `~/Library/Application\ Support/Claude/claude_desktop_config.json`. * Update with `langgraph-docs-mcp` as noted above. * Restart Claude Desktop app. > [!Note] > If you run into issues with Python version incompatibility when trying to add MCPDoc tools to Claude Desktop, you can explicitly specify the filepath to `python` executable in the `uvx` command. > > <details> > <summary>Example configuration</summary> > > ``` > { > "mcpServers": { > "langgraph-docs-mcp": { > "command": "uvx", > "args": [ > "--python", > "/path/to/python", > "--from", > "mcpdoc", > "mcpdoc", > "--urls", > "LangGraph:https://langchain-ai.github.io/langgraph/llms.txt", > "--transport", > "stdio" > ] > } > } > } > ``` > </details> > [!Note] > Currently (3/21/25) it appears that Claude Desktop does not support `rules` for global rules, so appending the following to your prompt. ``` <rules> for ANY question about LangGraph, use the langgraph-docs-mcp server to help answer -- + call list_doc_sources tool to get the available llms.txt file + call fetch_docs tool to read it + reflect on the urls in llms.txt + reflect on the input question + call fetch_docs on any urls relevant to the question </rules> ```  * You will see your tools visible in the bottom right of your chat input.  Then, try the example prompt: * It will ask to approve tool calls as it processes your request.  ### Connect to Claude Code * In a terminal after installing [Claude Code](https://docs.anthropic.com/en/docs/agents-and-tools/claude-code/overview), run this command to add the MCP server to your project: ``` claude mcp add-json langgraph-docs '{"type":"stdio","command":"uvx" ,"args":["--from", "mcpdoc", "mcpdoc", "--urls", "langgraph:https://langchain-ai.github.io/langgraph/llms.txt", "--urls", "LangChain:https://python.langchain.com/llms.txt"]}' -s local ``` * You will see `~/.claude.json` updated. * Test by launching Claude Code and running to view your tools: ``` $ Claude $ /mcp ```  > [!Note] > Currently (3/21/25) it appears that Claude Code does not support `rules` for global rules, so appending the following to your prompt. ``` <rules> for ANY question about LangGraph, use the langgraph-docs-mcp server to help answer -- + call list_doc_sources tool to get the available llms.txt file + call fetch_docs tool to read it + reflect on the urls in llms.txt + reflect on the input question + call fetch_docs on any urls relevant to the question </rules> ``` Then, try the example prompt: * It will ask to approve tool calls.  ## Command-line Interface The `mcpdoc` command provides a simple CLI for launching the documentation server. You can specify documentation sources in three ways, and these can be combined: 1. Using a YAML config file: * This will load the LangGraph Python documentation from the `sample_config.yaml` file in this repo. ```bash mcpdoc --yaml sample_config.yaml ``` 2. Using a JSON config file: * This will load the LangGraph Python documentation from the `sample_config.json` file in this repo. ```bash mcpdoc --json sample_config.json ``` 3. Directly specifying llms.txt URLs with optional names: * URLs can be specified either as plain URLs or with optional names using the format `name:url`. * You can specify multiple URLs by using the `--urls` parameter multiple times. * This is how we loaded `llms.txt` for the MCP server above. ```bash mcpdoc --urls LangGraph:https://langchain-ai.github.io/langgraph/llms.txt --urls LangChain:https://python.langchain.com/llms.txt ``` You can also combine these methods to merge documentation sources: ```bash mcpdoc --yaml sample_config.yaml --json sample_config.json --urls LangGraph:https://langchain-ai.github.io/langgraph/llms.txt --urls LangChain:https://python.langchain.com/llms.txt ``` ## Additional Options - `--follow-redirects`: Follow HTTP redirects (defaults to False) - `--timeout SECONDS`: HTTP request timeout in seconds (defaults to 10.0) Example with additional options: ```bash mcpdoc --yaml sample_config.yaml --follow-redirects --timeout 15 ``` This will load the LangGraph Python documentation with a 15-second timeout and follow any HTTP redirects if necessary. ## Configuration Format Both YAML and JSON configuration files should contain a list of documentation sources. Each source must include an `llms_txt` URL and can optionally include a `name`: ### YAML Configuration Example (sample_config.yaml) ```yaml # Sample configuration for mcp-mcpdoc server # Each entry must have a llms_txt URL and optionally a name - name: LangGraph Python llms_txt: https://langchain-ai.github.io/langgraph/llms.txt ``` ### JSON Configuration Example (sample_config.json) ```json [ { "name": "LangGraph Python", "llms_txt": "https://langchain-ai.github.io/langgraph/llms.txt" } ] ``` ## Programmatic Usage ```python from mcpdoc.main import create_server # Create a server with documentation sources server = create_server( [ { "name": "LangGraph Python", "llms_txt": "https://langchain-ai.github.io/langgraph/llms.txt", }, # You can add multiple documentation sources # { # "name": "Another Documentation", # "llms_txt": "https://example.com/llms.txt", # }, ], follow_redirects=True, timeout=15.0, ) # Run the server server.run(transport="stdio") ```