robots-txt-checker
Server Details
Checks if a URL path is allowed by robots.txt. x402 payment required (testnet USDC).
Glama couldn't complete the latest health check. If this server requires authentication, missing or expired test credentials may be the cause. A test profile lets Glama authenticate for health checks and discover tools; it is separate from your personal connections.
If you are the author, claim ownership, then add or update a test profile under Admin → Test Profile.
- Status
- Unhealthy
- Uptime
- 91.1% over 23 days
- Last Tested
- Transport
- Streamable HTTP · MCP 2025-11-25
- URL
TDQS
Scored across 1 tool
With only one tool exposed, there is no possibility of confusing it with another operation. The purpose is singular and clearly stated.
There is only one tool name, so there is no mixed convention or irregular verb pattern to flag. The name is descriptive and matches the server's purpose.
A single tool is minimal but appropriate for a narrowly scoped robots.txt checker. It is slightly below the typical 3-15 range, but the server is not over-scoped.
The server's entire purpose is to check a URL path against robots.txt, and the single tool fully covers that action. No additional operations are needed for this domain.
Available Tools
1 toolrobots_txt_checkerrobots_txt_checkerAInspect
Checks if a URL path is allowed by robots.txt. x402 payment required (Base mainnet USDC).
| Name | Required | Description | Default |
|---|---|---|---|
| checks | Yes | JSON-encoded strings, each an object like {"url":...,"path":...,"userAgent":...} | |
| payment | No | x402 payment payload (base64 JSON), signed EIP-712 exact/transferWithAuthorization. Omit on the first call — the tool will return payment requirements (accepts[]) as an error result. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
There are no annotations, so the description carries the behavioral burden. It discloses the charging mechanism, the required payment network, and the two-step flow where the first call returns accepts[] as an error result. This is valuable beyond the schema, though it does not detail the success response shape or robots.txt matching semantics.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two concise sentences with no filler. The core purpose is front-loaded, and the essential payment constraint follows immediately. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The input contract and payment handshake are covered by the schema and description together. However, with no output schema and no annotations, the description does not explain what a successful result looks like or how robots.txt matching rules are applied. This leaves some ambiguity for an agent planning to parse the tool's response.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents both parameters. The description adds the payment requirement context but does not need to redefine the parameters themselves. The baseline of 3 applies because the description neither helps nor hurts parameter understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('checks') and names the exact resource ('robots.txt') and the question it answers ('is a URL path allowed'). It is clear and unambiguous, though there are no sibling tools to differentiate from, so it does not quite reach the top tier.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides concrete invocation guidance: x402 payment is required on Base mainnet USDC, and the caller must omit the payment parameter on the first call to receive payment requirements as an error result. It does not describe alternative tools, but no siblings exist.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
1 tool update
- First observed
robots_txt_checker
Related MCP Connectors
Whether an AI agent may fetch a URL, on what terms, and at what price. Free, no key, no signup.
Checks current App Store version by bundle ID. x402 payment required (testnet USDC).
Validates email syntax and MX records. x402 payment required (testnet USDC).
Find paid APIs, check access policy and price, and route x402 requests. No autonomous spending.
Related MCP Servers
- FlicenseNot gradedqualityBmaintenanceEnables per-request paid AI analysis of public URLs, condition verification, and AI consultation via x402 micropayments on Base, with no account or subscription.5 npm-
- AlicenseNot gradedqualityBmaintenanceChecks token contract safety for honeypot, tax, proxy, blacklist, ownership risks, and returns a risk score, enabling rug-pull protection for agents via pay-per-call x402 micropayments.MIT
- AlicenseNot gradedqualityAmaintenanceChecks wash-traffic risk of Algorand x402 endpoints using on-chain analysis, helping agents decide whether to pay USDC.45 npmMIT
- FlicenseNot gradedqualityCmaintenanceProvides counterparty risk checks for any EVM address, returning malicious-history flags, tiered risk scoring, and plain-language summaries via x402 payment.-
Glama MCP Gateway
Add one secure layer between your agents and this server.