Skip to main content
Glama

Preferly

Generate AI robots.txt lines

generate_ai_robots_txt
Read-onlyIdempotent

Use this when the user wants robots.txt lines that control which AI crawlers may read their website. By default it lets AI search and assistant crawlers in (so the site can appear in AI answers) and keeps AI training crawlers out; either can be switched. Optional paths are kept private from every allowed crawler. It returns the lines to paste into robots.txt and which crawlers each group covers. It fetches nothing and needs no website.

Tool call template

This tool declares no required parameters. Add optional parameters from the input schema as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {},
    "name": "generate_ai_robots_txt"
  }
}

Input Schema

NameRequiredDescriptionDefault
allow_searchNoLet AI search and assistant crawlers in (default true).
allow_trainingNoLet AI training crawlers in (default false).
disallow_pathsNoPaths every allowed crawler should skip, like ["/admin", "/cart"].

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive, and closed-world, so the bar is lower. The description still adds real value: the default policy (search/assistant crawlers in, training crawlers out), that either can be switched, and the concrete meaning of 'fetches nothing and needs no website,' which operationalizes openWorldHint=false.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Four short sentences, front-loaded with the trigger, then defaults, then parameter behavior, then return shape. No sentence is redundant and nothing important is buried.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With three optional parameters, no output schema, and annotations covering the safety profile, the description fills the remaining gaps: defaults, override capability, path semantics, and what the output contains (lines plus per-group crawler coverage). An agent has everything needed to call it and interpret the result.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description adds meaning beyond the schema by explaining the combined effect: disallow_paths are 'kept private from every allowed crawler,' which tells the agent how that parameter interacts with the allow flags rather than just restating the field description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a concrete trigger and resource: 'robots.txt lines that control which AI crawlers may read their website.' It clearly marks this as a generator, which distinguishes it from the sibling checks (check_llms_txt, check_ai_crawler_access) that presumably inspect existing files.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives an explicit when-to-use condition ('when the user wants robots.txt lines...'), which is enough to route an agent. It does not explicitly name or exclude a sibling alternative, so it stops short of the top band.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.