# llms.txt Generator MCP Server — Create, Validate & Audit `llms.txt` (AI Agent Tools)

[![MCP Registry](https://img.shields.io/badge/MCP_Registry-listed-6f2fa9)](https://registry.modelcontextprotocol.io/v0/servers?search=llms-txt-generator)
[![Transport](https://img.shields.io/badge/Transport-Streamable_HTTP-blue)](https://www.llms-txt-generator.de/)
[![Lighthouse](https://img.shields.io/badge/Lighthouse_13.3-Agentic_Browsing-0cce6b)](https://developer.chrome.com/docs/lighthouse/agentic-browsing/scoring)
[![Status](https://img.shields.io/badge/Status-Live-brightgreen)](#status)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)

**[llms.txt Generator](https://www.llms-txt-generator.de/)** turns a website into
a curated `llms.txt` — the Markdown file that points AI systems at the content
that actually matters. This MCP server makes that available to agents like
**Claude, ChatGPT, and Cursor**: crawl a domain, generate a draft, validate an
existing file, and check it against Google's new Lighthouse **Agentic Browsing**
audit. Free, no API key, no registration.

- **Endpoint:** `https://www.llms-txt-generator.de/api/mcp` (Streamable HTTP)
- **MCP registry:** [`de.llms-txt-generator/llms-txt`](https://registry.modelcontextprotocol.io/v0/servers?search=llms-txt-generator) — published, status `active`
- **npm bridge:** [`llmstxtgenerator-mcp`](https://www.npmjs.com/package/llmstxtgenerator-mcp)
- **Web app:** [llms-txt-generator.de](https://www.llms-txt-generator.de/)
- **Format primer:** [What is llms.txt?](https://www.llms-txt-generator.de/what-is-llmstxt) · [FAQ](https://www.llms-txt-generator.de/faq)
- **Our own file:** [`/llms.txt`](https://www.llms-txt-generator.de/llms.txt) · [`/llms-full.txt`](https://www.llms-txt-generator.de/llms-full.txt)

> `llms.txt` is an open **proposal** ([llmstxt.org](https://llmstxt.org/)), not a
> ratified web standard. It does not replace `robots.txt` or `sitemap.xml`, it is
> not access control, and it guarantees neither rankings nor AI citations.

## Status

**Live.** The endpoint is up and serving all four tools:

```bash
curl -s https://www.llms-txt-generator.de/api/mcp \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/event-stream' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'
```

The same deterministic crawler, generator and validator that power the
[web app](https://www.llms-txt-generator.de/) — no separate implementation.

## Why `llms.txt` matters right now

Since **Lighthouse 13.3** (August 2026), Google Lighthouse and PageSpeed Insights
audit `/llms.txt` in a category called **Agentic Browsing**. The audit is a
structure check, *not* a Google Search ranking factor, and it is deliberately
minimal — three rules:

| Rule | Requirement |
| --- | --- |
| `hasH1` | The file contains an H1 heading (`# Site name`) |
| `hasLink` | At least one real Markdown link — bracket text, parenthesised URL |
| `isTooShort` | More than 50 characters of content |

Two things surprise most people:

1. **Plain-text links do not count.** `Homepage: https://example.com` fails the
   audit even though the link works. This is the single most common reason
   hand-written files fail.
2. **No file at all passes.** A missing `llms.txt` is reported as *not
   applicable*; only a present-but-broken one fails. **Publishing a broken
   `llms.txt` is worse than publishing none** — which is exactly why the
   `validate_*` tools exist.

## When to use this server

Use it whenever an agent needs to **create, inspect, or repair an `llms.txt`**:

- *"Generate an llms.txt for example.com"*
- *"Does our published llms.txt pass Google's Agentic Browsing audit?"*
- *"Check this llms.txt for broken links and links to foreign domains"*
- *"Why is PageSpeed Insights complaining about our llms.txt?"*

**Not** for: crawl control (that is `robots.txt`), URL discovery (that is
`sitemap.xml`), or full-site SEO audits.

## Connect

**Claude Code** (one line):

```bash
claude mcp add --transport http llms-txt https://www.llms-txt-generator.de/api/mcp
```

**Any client with an `mcpServers` block** (Cursor, Windsurf, VS Code, …):

```json
{
  "mcpServers": {
    "llms-txt": {
      "type": "streamable-http",
      "url": "https://www.llms-txt-generator.de/api/mcp"
    }
  }
}
```

**stdio-only clients** (e.g. Claude Desktop) via the npm bridge in this repo:

```json
{
  "mcpServers": {
    "llms-txt": {
      "command": "npx",
      "args": ["-y", "llmstxtgenerator-mcp"]
    }
  }
}
```

**ChatGPT / OpenAI:** the server speaks plain Streamable HTTP, so it works as a
connector and inside the Apps SDK — the same MCP standard OpenAI's plugins build
on since 2026.

## Tools

| Tool | What it does |
| --- | --- |
| `generate_llms_txt` | Crawls up to 15 public HTML pages of one registrable domain and returns a ready `llms.txt` draft. Deterministic — no language model is called, no facts are invented. |
| `validate_llms_txt` | Validates `llms.txt` text: H1, summary, sections, absolute links, duplicates, foreign domains, file size, update date. Returns a 0–100 score plus per-issue severity and line numbers. |
| `check_published_llms_txt` | Fetches a domain's live `/llms.txt`, checks status and content type, and validates the content. |
| `check_lighthouse_agentic_browsing` | Checks text against the three Lighthouse `llms-txt` audit rules exactly as Lighthouse implements them, and reports which one fails. |

Typical flow: `check_published_llms_txt` → `generate_llms_txt` →
`validate_llms_txt` → `check_lighthouse_agentic_browsing`.

Every response includes a `note` restating the Lighthouse rules, so an agent
reasoning over the result does not have to have read this README.

### What the crawler will and will not do

- Reads the start page, `robots.txt`, sitemaps, and a prioritised selection of
  internal HTML pages — **maximum 15 pages, same registrable domain only**.
- Extracts titles, descriptions, headings, canonicals, and JSON-LD types
  deterministically. Missing facts stay missing.
- **Blocked:** private networks, internal hostnames, foreign redirect targets,
  non-standard ports, URLs with credentials, non-HTTP(S) schemes, oversized
  responses.
- **Excluded from output:** parameter URLs, `noindex` pages, duplicates, error
  responses, non-HTML file types — each with a stated reason.
- The quick crawl executes **no page JavaScript**. Client-side-only content can
  be missing.

## FAQ

**What is `llms.txt`?**
An open proposal for a Markdown file at `/llms.txt` that gives humans and
machines a curated overview of a site: an H1 for the name, a blockquote summary,
and H2 sections of absolute links. See
[llmstxt.org](https://llmstxt.org/) and our
[primer](https://www.llms-txt-generator.de/what-is-llmstxt).

**Does `llms.txt` improve my Google rankings?**
No. Google Search ignores it as a ranking signal. What changed in 2026 is that
Google *Lighthouse* now audits it for structure. Those are different things, and
anyone claiming otherwise is overselling.

**Do I need an API key?**
No. Endpoints are public and rate-limited per IP.

**Is a language model involved?**
No. The crawler and generator are fully deterministic. Descriptions come only
from the source page's own metadata or visible text.

**Why only 15 pages?**
It keeps the free service fast and cheap to run, and a good `llms.txt` is
curated rather than exhaustive. Larger sites are welcome to
[get in touch](https://www.llms-txt-generator.de/#kontakt-landing).

**Who runs this?**
[track by track GmbH](https://www.llms-txt-generator.de/impressum), Berlin — the
team behind the social media agency famefact.

**Where is the privacy policy?**
[Datenschutzerklärung](https://www.llms-txt-generator.de/datenschutz) ·
[AGB](https://www.llms-txt-generator.de/agb).

---

## Deutsch: llms.txt per KI-Agent erstellen, prüfen und reparieren

Der **[llms.txt Generator](https://www.llms-txt-generator.de/)** erzeugt aus einer
Website eine kuratierte `llms.txt` — die Markdown-Datei, die KI-Systemen zeigt,
welche Inhalte wirklich zählen. Dieser MCP-Server macht das für KI-Agenten
nutzbar: Domain crawlen, Entwurf erzeugen, bestehende Datei validieren und gegen
Googles neuen Lighthouse-Audit **„Agentic Browsing"** prüfen. Kostenlos, ohne
API-Key, ohne Registrierung.

**Status: live.** Endpunkt: `https://www.llms-txt-generator.de/api/mcp`

```bash
claude mcp add --transport http llms-txt https://www.llms-txt-generator.de/api/mcp
```

### Warum das gerade jetzt relevant ist

Seit **Lighthouse 13.3** (August 2026) prüfen Google Lighthouse und PageSpeed
Insights in der Kategorie „Agentic Browsing" auch `/llms.txt`. Der Audit verlangt
drei Dinge: eine **H1-Überschrift**, mindestens einen **echten Markdown-Link**
(Linktext in eckigen, URL in runden Klammern) und mehr als **50 Zeichen** Inhalt.

Zwei Punkte überraschen die meisten:

- **Klartext-Links zählen nicht.** `Startseite: https://beispiel.de` fällt durch,
  obwohl der Link funktioniert. Das ist der häufigste Grund, warum von Hand
  geschriebene Dateien scheitern.
- **Gar keine Datei besteht.** Eine fehlende `llms.txt` gilt als „nicht
  zutreffend"; nur eine vorhandene mit Fehlern fällt durch. **Eine kaputt
  veröffentlichte `llms.txt` ist schlechter als gar keine.**

Der Audit ist eine **Strukturprüfung, kein Rankingfaktor** der Google-Suche.

### Werkzeuge

| Werkzeug | Funktion |
| --- | --- |
| `generate_llms_txt` | Crawlt bis zu 15 öffentliche HTML-Seiten derselben registrierbaren Domain und liefert einen fertigen `llms.txt`-Entwurf. Deterministisch — kein Sprachmodell, keine erfundenen Fakten. |
| `validate_llms_txt` | Prüft H1, Zusammenfassung, Abschnitte, absolute Links, Duplikate, fremde Domains, Dateigröße und Aktualisierungsdatum. Liefert 0–100 Punkte plus Befunde mit Schweregrad und Zeilennummer. |
| `check_published_llms_txt` | Ruft die live ausgelieferte `/llms.txt` einer Domain ab, prüft Status und Content-Type und validiert den Inhalt. |
| `check_lighthouse_agentic_browsing` | Prüft Text exakt gegen die drei Lighthouse-Regeln und benennt, welche davon fehlschlägt. |

### Sicherheit des Crawls

Blockiert werden private Netzwerke, interne Hostnamen, fremde Redirect-Ziele,
nicht standardmäßige Ports, URLs mit Zugangsdaten und Nicht-HTTP(S)-Schemata.
Ausgeschlossen werden Parameter-URLs, `noindex`-Seiten, Duplikate und
Fehlerantworten — jeweils mit Begründung. Der Schnell-Crawl führt **kein
Seiten-JavaScript** aus.

### Nützliche Links

- Generator: [llms-txt-generator.de](https://www.llms-txt-generator.de/)
- Format erklärt: [Was ist llms.txt?](https://www.llms-txt-generator.de/what-is-llmstxt)
- Häufige Fragen: [FAQ](https://www.llms-txt-generator.de/faq)
- Blog zu GEO und llms.txt: [Blog](https://www.llms-txt-generator.de/blog)
- Größere Website? [Termin anfragen](https://www.llms-txt-generator.de/#kontakt-landing)

---

This repository contains documentation, discovery metadata (`server.json`), and
the npm stdio bridge. The server itself is operated by track by track GmbH,
Berlin ([Impressum](https://www.llms-txt-generator.de/impressum)).

**License:** MIT — see [LICENSE](LICENSE).