firecrawl_generate_llmstxt

Generate a standardized llms.txt file for any website to define how AI models should interact with the site, creating machine-readable permission guidelines for large language models.

Instructions

Generate a standardized llms.txt (and optionally llms-full.txt) file for a given domain. This file defines how large language models should interact with the site.

Best for: Creating machine-readable permission guidelines for AI models. Not recommended for: General content extraction or research. Arguments:

url (string, required): The base URL of the website to analyze.
maxUrls (number, optional): Max number of URLs to include (default: 10).
showFullText (boolean, optional): Whether to include llms-full.txt contents in the response. Prompt Example: "Generate an LLMs.txt file for example.com." Usage Example:

{ "name": "firecrawl_generate_llmstxt", "arguments": { "url": "https://example.com", "maxUrls": 20, "showFullText": true } }

Returns: LLMs.txt file contents (and optionally llms-full.txt).

Input Schema

Name	Required	Description
`maxUrls`	No	Maximum number of URLs to process (1-100, default: 10)
`showFullText`	No	Whether to show the full LLMs-full.txt in the response
`url`	Yes	The URL to generate LLMs.txt from

Input Schema (JSON Schema)

{ "properties": { "maxUrls": { "description": "Maximum number of URLs to process (1-100, default: 10)", "type": "number" }, "showFullText": { "description": "Whether to show the full LLMs-full.txt in the response", "type": "boolean" }, "url": { "description": "The URL to generate LLMs.txt from", "type": "string" } }, "required": [ "url" ], "type": "object" }

Firecrawl MCP Server