Skip to main content
Glama

Server Details

Check which AI bots a site lets in, then write a fixed robots.txt.

Ownership verified
Status
Healthy
Last Tested
Transport
Streamable HTTP · MCP 2025-11-25
URL
ChatGPT
–
Claude
–

TDQS

A4.4/5.0

Scored across 3 tools

Disambiguation5/5

Each tool has a clearly distinct input and purpose: one fetches a live robots.txt, one parses pasted robots.txt text, and one generates new robots.txt lines. The descriptions explicitly cross-reference each other (e.g. 'to write a new robots.txt use doorman_write_robots_txt'), leaving no realistic ambiguity about which to pick.

Naming Consistency5/5

All three tools follow a consistent 'doorman_verb_object' pattern: doorman_check_ai_bot_access, doorman_check_robots_txt, doorman_write_robots_txt. The shared server prefix and verb-first style make the set predictable.

Tool Count4/5

Three tools is on the thin side, but each covers a genuinely separate operation (inspect live, inspect text, generate) and earns its place. The surface is well-scoped to the robots.txt/AI-bot domain rather than padded.

Completeness4/5

The core lifecycle is covered: checking what a live site serves, parsing an existing file, and producing a corrected one. Minor gaps exist (e.g. no tool to verify a deployed robots.txt after writing, or to adjust individual bots outside the three groups), but agents can work around these.

Available Tools

3 tools
doorman_check_ai_bot_accessSee which AI bots a website lets inA
Read-onlyIdempotent
Inspect

Use when the user asks whether their site (or any public site) blocks ChatGPT, Claude, Google AI, Perplexity or other AI crawlers, whether AI can read or train on their content, or why AI search does not mention their site. Fetches the live robots.txt and reports, for 22 named AI bots in four groups (AI training, AI search, assistants, search engines), which are let in, kept out or partly blocked, plus the site's content-signal preference and fixes. Example: {"url": "example.com"}. Read-only. To write a new robots.txt use doorman_write_robots_txt.

NameRequiredDescriptionDefault
urlYesWebsite address or domain, e.g. nytimes.com

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare read-only, idempotent, open-world and non-destructive, so the description only restates "Read-only" briefly. Its added value is real: it discloses that the fetch is live, that results come grouped into four bot categories with partial-block states, and that fixes are included. It does not cover edge behavior such as an unreachable site or a missing robots.txt.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the triggering user question, then behavior, example, safety note, and sibling hand-off — every clause carries information. It is dense in one long opening sentence, but nothing is filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, and the description compensates by describing the report's structure (four bot groups, three access states, content-signal preference, fixes). What is missing is failure-mode behavior for unreachable sites or absent robots.txt files.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% with a single required param, so the baseline is a 3. The description adds a concrete invocation example ({"url": "example.com"}) showing that a bare domain without scheme is acceptable, which meaningfully reduces call-format ambiguity.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ("Fetches the live robots.txt and reports") with concrete scope: 22 named AI bots in four groups plus content-signal preference. This is clearly distinct from doorman_write_robots_txt and doorman_check_robots_txt.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit trigger phrasings ("whether their site blocks ChatGPT, Claude...", "why AI search does not mention their site") and routes the write case to the sibling tool by name. Nothing is left to inference about when to pick this versus the alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

doorman_check_robots_txtCheck a pasted robots.txt for AI bot accessA
Read-onlyIdempotent
Inspect

Use when the user pastes the text of a robots.txt (or you have drafted one) and asks which AI bots it lets in or blocks, or whether a change does what they want. Parses it the way crawlers do (groups, wildcards, longest-match) and reports each of 22 named AI bots as let in, kept out or partly blocked, with notes and a suggested choice. Pure parsing, no network. Example: {"robots_txt": "User-agent: GPTBot\nDisallow: /"}.

NameRequiredDescriptionDefault
robots_txtYesThe full robots.txt text (up to 512 KB). Empty means no robots.txt.

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnly, idempotent, non-destructive, and closed-world behavior. The description adds meaningful behavioral context beyond that: it parses like crawlers do (groups, wildcards, longest-match), reports each bot as let in, kept out, or partly blocked, and provides notes plus a suggested choice. It does not mention size limits or exact output structure, but the key behavior is well disclosed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the usage condition, then explains parsing behavior and output, and ends with a compact example. Every sentence contributes useful context without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter parsing tool with no output schema, the description is complete enough to invoke correctly. It explains the input format via example, the parsing semantics, the expected report categories, and that no network access occurs.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the single parameter is already well documented as the full robots.txt text up to 512 KB, with empty meaning no robots.txt. The description includes a helpful example of the parameter shape, but it does not add substantial semantic meaning beyond what the schema already provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action: parsing pasted robots.txt text and reporting which of 22 named AI bots are allowed or blocked. It clearly distinguishes this pasted-text parsing tool from siblings like doorman_check_ai_bot_access and doorman_write_robots_txt by emphasizing 'Pure parsing, no network.'

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives a clear when-to-use condition: the user pastes robots.txt or has drafted one and asks which AI bots it lets in or blocks, or whether a change works. It does not explicitly name alternative sibling tools or state when not to use this one, so it falls short of full when/when-not/alternatives guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

doorman_write_robots_txtWrite a robots.txt with your AI bot choicesA
Read-onlyIdempotent
Inspect

Use when the user wants to allow or block AI training, AI search or assistant bots on their site and needs the robots.txt lines. Takes an optional existing robots.txt plus a choice of "in" or "out" for each of three groups (training, ai_search, assistants) and returns a complete robots.txt that keeps existing rules, adds one clean group per choice and a shared content-signal line. Example: {"training": "out", "ai_search": "in", "assistants": "in"}. Returns text to paste at /robots.txt; it does not change any site.

NameRequiredDescriptionDefault
trainingNoLet AI training bots in or keep them out (default out).
ai_searchNoAI search and answer engines (default in).
assistantsNoAssistants that open a page because a person asked (default in).
robots_txtNoCurrent robots.txt text to keep (optional).

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, non-destructive, and closed-world behavior. The description adds useful behavioral detail beyond the annotations: it preserves existing rules, adds one clean group per choice, includes a shared content-signal line, and returns text without modifying the site.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the usage condition, then explains inputs, gives a compact example, and states the return value. Every sentence earns its place and there is no unnecessary repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple and fully covered: annotations carry the safety profile, the schema documents all four parameters, and the description explains the generated output and that no site is changed. Nothing material is missing for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents the parameters, defaults, and enum values. The description reinforces the three group choices and mentions the optional existing robots.txt, plus an example, but does not add much beyond what the schema provides.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: writing robots.txt with AI bot choices. It clearly distinguishes the tool from the sibling check tools by emphasizing that it returns a complete robots.txt to paste and does not change any site.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The first sentence gives a clear when-to-use condition: the user wants to allow or block AI training, AI search, or assistant bots and needs the robots.txt lines. It does not explicitly name the sibling check tools as alternatives, but the generation-vs-checking distinction is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 1 tool update
    • Changeddoorman_check_robots_txt1 field changed
      • addedInput schema / properties / robots_txt / minLength
        Added value: +0
  2. 6 tool updates
    • Removeddoorman_build_robots
    • Addeddoorman_check_ai_bot_access
    • Removeddoorman_check_robots
    • Addeddoorman_check_robots_txt
    • Removeddoorman_check_site
    • Addeddoorman_write_robots_txt
  3. 3 tool updates
    • First observeddoorman_build_robots
    • First observeddoorman_check_robots
    • First observeddoorman_check_site

Related MCP Connectors

Related MCP Servers

  • A
    license
    A
    quality
    C
    maintenance
    Checks a website's robots.txt and Cloudflare settings to identify AI crawler blocking. Also generates llms.txt content to improve visibility to AI answer engines.
    3
    50 npm
    MIT
  • A
    license
    A
    quality
    C
    maintenance
    Analyze and generate robots.txt files with AI crawler awareness. Fetch any site's robots.txt, detect which AI bots (GPTBot, ClaudeBot, PerplexityBot, Google-Extended) are blocked or allowed, and generate optimized robots.txt with toggle controls for 20+ AI crawlers.
    5
    1
    MIT
  • A
    license
    A
    quality
    C
    maintenance
    Checks a site for AI crawlers: which of 11 AI crawlers its robots.txt lets in, whether its llms.txt follows llmstxt.org, and what its home page says without scripts. Findings with evidence; no scores, no model calls, no telemetry.
    5
    508 npm
    MIT
Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources