llms-text

Generate llms.txt

generate_llms_txt

Read-only

Automatically crawl any public website URL and generate a standard, token-efficient index-level /llms.txt file for AI agent context. 100% free with core page categorization, descriptions, and clean markdown.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "url": "<url>"
    },
    "name": "generate_llms_txt"
  }
}

Input Schema

Name	Required	Description	Default
`url`	Yes	The website URL to crawl and generate llms.txt for (e.g. https://example.com)
`excludePaths`	No	Optional URL path keywords to exclude

Schema Changelog

Changes observed during successful MCP inspections.

First observedNov 12, 2026

TDQS

B3.4/5.0

Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already declare readOnly, non-destructive, and open-world, so safety is covered. The description adds useful behavioral detail about what the output contains (categorization, descriptions, clean markdown) and that it is free, but says nothing about crawl depth, timeouts, rate limits, or whether the file is returned or written somewhere.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the action and artifact, with no repetition of the name. The marketing-flavored phrases ("100% free," "token-efficient") are mild filler, though "free" does usefully signal no auth or billing.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description must carry the burden of explaining the result, and it does so only partially — it lists file contents but not where the file lands or what a response looks like. For a crawl-and-generate operation, the absence of scope limits or failure behavior leaves a real gap.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and both parameters are documented there, so the description adds no meaning beyond the schema. It mentions crawling "any public website URL," which only echoes the url parameter rather than clarifying format or accepted values.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (crawl) and artifact (an index-level /llms.txt file) plus the intended consumer (AI agents), and the word "index-level" implicitly distinguishes it from the sibling generate_llms_full_txt. It stops short of naming that sibling, so the differentiation is inferred rather than explicit.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Scope is bounded to "any public website URL," which tells the agent this is for reachable public sites, but there is no statement of when to pick this over generate_llms_full_txt or validate_llms_txt. The agent must infer the choice from the word "index-level."

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All llms-text tools Inspect

Try in Browser

Claude Code, inside the Godot 4 editorFlockbay puts an AI agent in the real Godot 4 editor: it writes GDScript, runs your game and fixes what breaks. Free, or bring your own Claude Code or Codex.flockbay.comAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources

Need Help?