# MCP Docs Server

A Model Context Protocol (MCP) server that fetches and parses documentation from `llms.txt` files, enabling AI assistants to access and query documentation dynamically.

## Overview

This MCP server provides tools for AI assistants to:
1. Discover available documentation sources via `llms.txt` files
2. Fetch and parse documentation content from URLs or local files
3. Convert HTML documentation to markdown for easier consumption

The server follows the [llms.txt](https://llmstxt.org/) convention, which provides a standardized way for websites to expose their documentation in an LLM-friendly format.

## How It Works

### Architecture

The server implements the Model Context Protocol (MCP) to expose two main tools:

1. **`list_doc_sources`**: Lists all configured documentation sources with their `llms.txt` URLs or file paths
2. **`fetch_docs`**: Fetches and parses documentation content from a given URL or local file path

### Workflow

1. **Discovery**: AI assistant calls `list_doc_sources` to see available documentation
2. **Parse llms.txt**: AI assistant fetches the `llms.txt` file using `fetch_docs` to see the documentation structure
3. **Query Documentation**: AI assistant analyzes the URLs in `llms.txt` and fetches specific relevant pages
4. **Content Processing**: Server converts HTML to markdown and returns cleaned content

### llms.txt Format

The `llms.txt` file is a simple text file that lists URLs to documentation pages. The server supports both absolute and relative URLs.

**Absolute URLs:**
```
# Stripe API Documentation
https://docs.stripe.com/api/authentication
https://docs.stripe.com/api/charges
https://docs.stripe.com/api/customers
```

**Relative URLs (when only one documentation source is configured):**
```
# Stripe API Documentation
/api/authentication
/api/charges
/api/customers
```

**Note:** Relative URLs are automatically resolved against the base domain of the `llms.txt` file. This only works when a single documentation source is configured. With multiple sources, you must use absolute URLs to avoid ambiguity.

## Features

- **Domain Whitelisting**: Only allows fetching from pre-configured trusted domains
- **Local File Support**: Can serve documentation from local markdown/text files
- **Relative URL Support**: Automatically resolves relative URLs in llms.txt files against the base domain
- **HTML to Markdown Conversion**: Automatically converts HTML documentation to markdown
- **Security**: Built-in protections against unauthorized URL access
- **Configurable Timeouts**: Customizable request timeouts
- **Redirect Handling**: Optional support for following HTTP redirects

## Installation

```bash
npm install
```

## Configuration

Edit the `DOC_SOURCES` array in `src/index.ts` to configure your documentation sources:

```typescript
const DOC_SOURCES: DocSource[] = [
  {
    name: "Stripe API Documentation",
    llms_txt: "https://docs.stripe.com/llms.txt",
    description: "Stripe API documentation",
  },
  {
    name: "Local Docs",
    llms_txt: "file:///path/to/local/llms.txt",
    description: "Local documentation files",
  },
];
```

### Configuration Options

- **`name`** (optional): Display name for the documentation source
- **`llms_txt`**: URL or file path to the `llms.txt` file
- **`description`** (optional): Description of the documentation

### Advanced Configuration

Modify the `CONFIG` object for additional settings:

```typescript
const CONFIG = {
  followRedirects: false,  // Follow HTTP redirects
  timeout: 10000,          // Request timeout in milliseconds
  allowedDomains: [],      // Additional allowed domains (use ["*"] to allow all)
};
```

## Usage

### Build

```bash
npm run build
```

### Development

```bash
npm run dev
```

### Run

```bash
npm start
```

The server runs on stdio and communicates via the Model Context Protocol.

## MCP Integration

To use this server with an MCP-compatible client (like Claude Desktop), add it to your MCP configuration:

```json
{
  "mcpServers": {
    "mcp-docs": {
      "command": "node",
      "args": ["/path/to/mcp-docs/build/index.js"]
    }
  }
}
```

## Project Structure

```
mcp-docs/
├── src/
│   ├── index.ts      # Main server implementation
│   ├── utils.ts      # Utility functions (fetching, parsing, conversion)
│   └── types.ts      # TypeScript type definitions
├── build/            # Compiled JavaScript (gitignored)
├── package.json      # Project dependencies
├── tsconfig.json     # TypeScript configuration
└── README.md         # This file
```

## Dependencies

- **@modelcontextprotocol/sdk**: MCP server SDK
- **node-html-parser**: HTML parsing
- **turndown**: HTML to Markdown conversion
- **zod**: Schema validation

## Security

The server implements several security measures:

- **Domain Whitelisting**: Only domains with configured `llms.txt` files can be accessed
- **Path Validation**: Local files must be explicitly allowed in configuration
- **No Arbitrary Fetching**: Cannot fetch random URLs without proper configuration

## Example Use Case

An AI assistant helping a developer with Stripe API integration can:

1. Call `list_doc_sources` to discover Stripe docs are available
2. Fetch `https://docs.stripe.com/llms.txt` to see all available documentation pages
3. Fetch specific pages like `https://docs.stripe.com/api/charges` to answer user questions
4. Provide accurate, up-to-date answers based on official documentation

## License

MIT