llms.txt Generator MCP Server — Create, Validate & Audit llms.txt (AI Agent Tools)

llms.txt Generator turns a website into a curated llms.txt — the Markdown file that points AI systems at the content that actually matters. This MCP server makes that available to agents like Claude, ChatGPT, and Cursor: crawl a domain, generate a draft, validate an existing file, and check it against Google's new Lighthouse Agentic Browsing audit. Free, no API key, no registration.

• Endpoint: https://www.llms-txt-generator.de/api/mcp (Streamable HTTP)

• MCP registry: de.llms-txt-generator/llms-txt — published, status active

• npm bridge: llmstxtgenerator-mcp

• Web app: llms-txt-generator.de

• Format primer: What is llms.txt? · FAQ

• Our own file: /llms.txt · /llms-full.txt

llms.txt is an open proposal (llmstxt.org), not a ratified web standard. It does not replace robots.txt or sitemap.xml, it is not access control, and it guarantees neither rankings nor AI citations.

Status

Live. The endpoint is up and serving all four tools:

curl -s https://www.llms-txt-generator.de/api/mcp \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json, text/event-stream' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'

The same deterministic crawler, generator and validator that power the web app — no separate implementation.

Why llms.txt matters right now

Since Lighthouse 13.3 (August 2026), Google Lighthouse and PageSpeed Insights audit /llms.txt in a category called Agentic Browsing. The audit is a structure check, not a Google Search ranking factor, and it is deliberately minimal — three rules:

Two things surprise most people:

1. Plain-text links do not count. Homepage: https://example.com fails the audit even though the link works. This is the single most common reason hand-written files fail.

2. No file at all passes. A missing llms.txt is reported as not applicable; only a present-but-broken one fails. Publishing a broken llms.txt is worse than publishing none — which is exactly why the validate_* tools exist.

When to use this server

Use it whenever an agent needs to create, inspect, or repair an llms.txt:

• "Generate an llms.txt for example.com"

• "Does our published llms.txt pass Google's Agentic Browsing audit?"

• "Check this llms.txt for broken links and links to foreign domains"

• "Why is PageSpeed Insights complaining about our llms.txt?"

Not for: crawl control (that is robots.txt), URL discovery (that is sitemap.xml), or full-site SEO audits.

Connect

Claude Code (one line):

claude mcp add --transport http llms-txt https://www.llms-txt-generator.de/api/mcp

Any client with an mcpServers block (Cursor, Windsurf, VS Code, …):

{
  "mcpServers": {
    "llms-txt": {
      "type": "streamable-http",
      "url": "https://www.llms-txt-generator.de/api/mcp"
    }
  }
}

stdio-only clients (e.g. Claude Desktop) via the npm bridge in this repo:

{
  "mcpServers": {
    "llms-txt": {
      "command": "npx",
      "args": ["-y", "llmstxtgenerator-mcp"]
    }
  }
}

ChatGPT / OpenAI: the server speaks plain Streamable HTTP, so it works as a connector and inside the Apps SDK — the same MCP standard OpenAI's plugins build on since 2026.

Tools

Typical flow: check_published_llms_txt → generate_llms_txt → validate_llms_txt → check_lighthouse_agentic_browsing.

Every response includes a note restating the Lighthouse rules, so an agent reasoning over the result does not have to have read this README.

What the crawler will and will not do

• Reads the start page, robots.txt, sitemaps, and a prioritised selection of internal HTML pages — maximum 15 pages, same registrable domain only.

• Extracts titles, descriptions, headings, canonicals, and JSON-LD types deterministically. Missing facts stay missing.

• Blocked: private networks, internal hostnames, foreign redirect targets, non-standard ports, URLs with credentials, non-HTTP(S) schemes, oversized responses.

• Excluded from output: parameter URLs, noindex pages, duplicates, error responses, non-HTML file types — each with a stated reason.

• The quick crawl executes no page JavaScript. Client-side-only content can be missing.

FAQ

What is llms.txt? An open proposal for a Markdown file at /llms.txt that gives humans and machines a curated overview of a site: an H1 for the name, a blockquote summary, and H2 sections of absolute links. See llmstxt.org and our primer.

Does llms.txt improve my Google rankings? No. Google Search ignores it as a ranking signal. What changed in 2026 is that Google Lighthouse now audits it for structure. Those are different things, and anyone claiming otherwise is overselling.

Do I need an API key? No. Endpoints are public and rate-limited per IP.

Is a language model involved? No. The crawler and generator are fully deterministic. Descriptions come only from the source page's own metadata or visible text.

Why only 15 pages? It keeps the free service fast and cheap to run, and a good llms.txt is curated rather than exhaustive. Larger sites are welcome to get in touch.

Who runs this? track by track GmbH, Berlin — the team behind the social media agency famefact.

Where is the privacy policy? Datenschutzerklärung · AGB.

Deutsch: llms.txt per KI-Agent erstellen, prüfen und reparieren

Der llms.txt Generator erzeugt aus einer Website eine kuratierte llms.txt — die Markdown-Datei, die KI-Systemen zeigt, welche Inhalte wirklich zählen. Dieser MCP-Server macht das für KI-Agenten nutzbar: Domain crawlen, Entwurf erzeugen, bestehende Datei validieren und gegen Googles neuen Lighthouse-Audit „Agentic Browsing" prüfen. Kostenlos, ohne API-Key, ohne Registrierung.

Status: live. Endpunkt: https://www.llms-txt-generator.de/api/mcp

claude mcp add --transport http llms-txt https://www.llms-txt-generator.de/api/mcp

Warum das gerade jetzt relevant ist

Seit Lighthouse 13.3 (August 2026) prüfen Google Lighthouse und PageSpeed Insights in der Kategorie „Agentic Browsing" auch /llms.txt. Der Audit verlangt drei Dinge: eine H1-Überschrift, mindestens einen echten Markdown-Link (Linktext in eckigen, URL in runden Klammern) und mehr als 50 Zeichen Inhalt.

Zwei Punkte überraschen die meisten:

• Klartext-Links zählen nicht. Startseite: https://beispiel.de fällt durch, obwohl der Link funktioniert. Das ist der häufigste Grund, warum von Hand geschriebene Dateien scheitern.

• Gar keine Datei besteht. Eine fehlende llms.txt gilt als „nicht zutreffend"; nur eine vorhandene mit Fehlern fällt durch. Eine kaputt veröffentlichte llms.txt ist schlechter als gar keine.

Der Audit ist eine Strukturprüfung, kein Rankingfaktor der Google-Suche.

Werkzeuge

Sicherheit des Crawls

Blockiert werden private Netzwerke, interne Hostnamen, fremde Redirect-Ziele, nicht standardmäßige Ports, URLs mit Zugangsdaten und Nicht-HTTP(S)-Schemata. Ausgeschlossen werden Parameter-URLs, noindex-Seiten, Duplikate und Fehlerantworten — jeweils mit Begründung. Der Schnell-Crawl führt kein Seiten-JavaScript aus.

Nützliche Links

• Generator: llms-txt-generator.de

• Format erklärt: Was ist llms.txt?

• Häufige Fragen: FAQ

• Blog zu GEO und llms.txt: Blog

• Größere Website? Termin anfragen

This repository contains documentation, discovery metadata (server.json), and the npm stdio bridge. The server itself is operated by track by track GmbH, Berlin (Impressum).

License: MIT — see LICENSE.