Skip to main content
Glama

robots-txt-checker

robots_txt_checker

robots_txt_checker

Checks if a URL path is allowed by robots.txt. x402 payment required (Base mainnet USDC).

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
checksYesJSON-encoded strings, each an object like {"url":...,"path":...,"userAgent":...}
paymentNox402 payment payload (base64 JSON), signed EIP-712 exact/transferWithAuthorization. Omit on the first call — the tool will return payment requirements (accepts[]) as an error result.

Schema Changelog

Changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. First observed

TDQS

A3.9/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

There are no annotations, so the description carries the behavioral burden. It discloses the charging mechanism, the required payment network, and the two-step flow where the first call returns accepts[] as an error result. This is valuable beyond the schema, though it does not detail the success response shape or robots.txt matching semantics.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two concise sentences with no filler. The core purpose is front-loaded, and the essential payment constraint follows immediately. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The input contract and payment handshake are covered by the schema and description together. However, with no output schema and no annotations, the description does not explain what a successful result looks like or how robots.txt matching rules are applied. This leaves some ambiguity for an agent planning to parse the tool's response.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents both parameters. The description adds the payment requirement context but does not need to redefine the parameters themselves. The baseline of 3 applies because the description neither helps nor hurts parameter understanding.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('checks') and names the exact resource ('robots.txt') and the question it answers ('is a URL path allowed'). It is clear and unambiguous, though there are no sibling tools to differentiate from, so it does not quite reach the top tier.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides concrete invocation guidance: x402 payment is required on Base mainnet USDC, and the caller must omit the payment parameter on the first call to receive payment requirements as an error result. It does not describe alternative tools, but no siblings exist.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

TDQS

A4.2/5.0
Disambiguation5/5

With only one tool exposed, there is no possibility of confusing it with another operation. The purpose is singular and clearly stated.

Naming Consistency5/5

There is only one tool name, so there is no mixed convention or irregular verb pattern to flag. The name is descriptive and matches the server's purpose.

Tool Count4/5

A single tool is minimal but appropriate for a narrowly scoped robots.txt checker. It is slightly below the typical 3-15 range, but the server is not over-scoped.

Completeness5/5

The server's entire purpose is to check a URL path against robots.txt, and the single tool fully covers that action. No additional operations are needed for this domain.

Resources