Skip to main content
Glama

check_crawler_blocks

Read-only

Check whether robots.txt or a live probe blocks AmazonAdBot / Google ad crawlers. robots.txt Disallow and Amazon Crawl-delay are high-confidence fails. A WAF 403 against our scanner is a medium-confidence warn that needs verification.

Input Schema

NameRequiredDescriptionDefault
domainYesPublisher domain, for example example.com.
outputFormatNomarkdown

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed1 schema field changed
    • removedInput schema / additionalProperties
      Removed value: -false
  2. Added

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish readOnlyHint, openWorldHint and non-destructive behavior, so the description is free to add interpretation guidance: robots.txt Disallow and Amazon Crawl-delay are treated as hard fails while a WAF 403 against the scanner is a soft warn requiring verification. That severity model is genuine value beyond the annotations, though it omits auth/rate-limit or scope details.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Three short sentences, front-loaded with what is checked, then the verdict semantics. Nothing is redundant and each sentence adds decision-relevant information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema, so the description carries responsibility for return values; it explains the fail/warn confidence classification but not the result structure, what checks are enumerated, or what outputFormat changes. Enough to call it, not enough to fully anticipate the response.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is only 50% (domain is documented, outputFormat has no description) and the description does not compensate by explaining either parameter. The undocumented parameter is a trivial enum with a default, so baseline 3 is appropriate rather than a penalty.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description gives a specific verb (Check) and resource (robots.txt / live probe blocking), and names the exact crawlers being tested (AmazonAdBot, Google ad crawlers), which is more concrete than the tool name alone. It does not explicitly contrast itself with related siblings like run_agentic_readiness_audit or check_https_certificate, so it stops short of full differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the confidence-tier framing (high-confidence fails, medium-confidence warn needing verification), which tells the agent how to act on results but not when to choose this tool over the readiness audit or certificate check. No prerequisites or exclusions are stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources