F
licenseNot graded
qualityB
maintenanceEnables per-request paid AI analysis of public URLs, condition verification, and AI consultation via x402 micropayments on Base, with no account or subscription.
13
-
robots-txt-checker
Server Details
Checks if a URL path is allowed by robots.txt. x402 payment required (testnet USDC).
If you are the author of this connector, you can claim ownership with GitHub, an HTTP challenge, or a DNS record. Claimed connector authors can inspect health checks, view analytics, and manage their listing.
- Status
- Healthy
- Last Tested
- Transport
- Streamable HTTP
- URL
Available Tools
1 toolrobots_txt_checkerrobots_txt_checkerAInspect
Checks if a URL path is allowed by robots.txt. x402 payment required (Base mainnet USDC).
ParametersJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| checks | Yes | JSON-encoded strings, each an object like {"url":...,"path":...,"userAgent":...} | |
| payment | No | x402 payment payload (base64 JSON), signed EIP-712 exact/transferWithAuthorization. Omit on the first call — the tool will return payment requirements (accepts[]) as an error result. |
TDQS
A3.9/5.0
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
There are no annotations, so the description carries the behavioral burden. It discloses the charging mechanism, the required payment network, and the two-step flow where the first call returns accepts[] as an error result. This is valuable beyond the schema, though it does not detail the success response shape or robots.txt matching semantics.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two concise sentences with no filler. The core purpose is front-loaded, and the essential payment constraint follows immediately. Every word earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The input contract and payment handshake are covered by the schema and description together. However, with no output schema and no annotations, the description does not explain what a successful result looks like or how robots.txt matching rules are applied. This leaves some ambiguity for an agent planning to parse the tool's response.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters3/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already documents both parameters. The description adds the payment requirement context but does not need to redefine the parameters themselves. The baseline of 3 applies because the description neither helps nor hurts parameter understanding.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('checks') and names the exact resource ('robots.txt') and the question it answers ('is a URL path allowed'). It is clear and unambiguous, though there are no sibling tools to differentiate from, so it does not quite reach the top tier.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides concrete invocation guidance: x402 payment is required on Base mainnet USDC, and the caller must omit the payment parameter on the first call to receive payment requirements as an error result. It does not describe alternative tools, but no siblings exist.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections. Dates show when Glama detected each change.
1 tool update
- First observed
robots_txt_checker
Frequently Asked Questions
Claiming proves that you control a remote MCP connector. It does not move, proxy, or interrupt the server.
Open the connector listing, choose Claim ownership, and sign in to Glama.
Complete one verification method:
GitHub identity — fastest for official registry listings. For a namespace such as
io.github.alice/server, link the matching GitHub user, then choose Claim with GitHub. An organization namespace such asio.github.acme/serveralso needs that organization to have installed the Glama AI GitHub App and approved its permissions, because GitHub discloses organization membership only to apps it has installed. Use HTTP or DNS when it has not.HTTP challenge — works when you can deploy a public file. Generate a token, publish the exact JSON Glama shows at
/.well-known/glama.jsonon the same origin as the connector, then choose Check HTTP challenge.DNS challenge — works when you control DNS but cannot change the server. Generate a token, create the exact TXT record Glama shows, wait for it to propagate, then choose Check DNS challenge.
After verification, Glama sends a confirmation email and gives you access to listing details, thumbnails, health checks, and analytics. Keep the HTTP file or DNS record in place: Glama periodically checks it and ownership remains verified while the token is discoverable.
The HTTP ownership file has this structure:
{
"$schema": "https://glama.ai/mcp/schemas/connector.json",
"claim": "glama_claim_..."
}Claim tokens are opaque, stable, and bound to the signed-in Glama account. They contain no email address or other personal information. If Glama can no longer discover a verified HTTP or DNS token, it starts a seven-day grace period before removing claim-based access. Restore the same token during that period to keep ownership verified. Never publish an email address, Glama session token, GitHub token, or connector credential as ownership proof.
If verification fails, confirm that you copied the current token exactly. The HTTP file must be public, return valid JSON with a successful HTTP response, and stay on the connector's origin. DNS changes may need more time to propagate. A claim cannot transfer to a different origin or hostname: if the connector target changes, Glama starts the grace period and the new target must be claimed separately after the previous claim is released.
For a connector linked to the official MCP Registry, registry updates continue to replace its name, description, and URL by default. After claiming, open Manage connector and enable Use Glama listing details as the source of truth if edits made on Glama should be preserved. Categories and thumbnails are always managed on Glama; registry linkage and technical connection settings continue to sync.
Control your server's listing on Glama, including description and metadata
Access analytics and receive server usage reports
Get monitoring and health status updates for your server
Feature your server to boost visibility and reach more users
To improve your MCP server's ranking:
Claim ownership of the server listing
Complete the server profile with an accurate description and thumbnail
Provide a test profile so Glama can connect to and evaluate the server
Keep tool definitions clear and complete to earn a high Tool Definition Quality Score (TDQS)
Route real usage through the Glama Gateway; more recorded successful server uses also improve the ranking
For users:
Full audit trail – every tool call is logged with inputs and outputs for compliance and debugging
Granular tool control – enable or disable individual tools per connector to limit what your AI agents can do
Centralized credential management – store and rotate API keys and OAuth tokens in one place
Change alerts – get notified when a connector changes its schema, adds or removes tools, or updates tool definitions, so nothing breaks silently
For server owners:
Proven adoption – public usage metrics on your listing show real-world traction and build trust with prospective users
Tool-level analytics – see which tools are being used most, helping you prioritize development and documentation
Direct user feedback – users can report issues and suggest improvements through the listing, giving you a channel you would not have otherwise
The connector status is unhealthy when Glama is unable to successfully connect to the server. This can happen for several reasons:
The server is experiencing an outage
The URL of the server is wrong
Credentials required to access the server are missing or invalid
If you are the owner of this MCP connector and would like to make modifications to the listing, including providing test credentials for accessing the server, please contact support@glama.ai.
Discussions
No comments yet. Be the first to start the discussion!
Related MCP Connectors
Checks current App Store version by bundle ID. x402 payment required (testnet USDC).
Validates email syntax and MX records. x402 payment required (testnet USDC).
11Find paid APIs, check access policy and price, and route x402 requests. No autonomous spending.
Converts text titles into URL-safe slugs and flags duplicates. x402 payment required (testnet USDC).
Related MCP Servers
- AlicenseNot gradedqualityBmaintenanceChecks token contract safety for honeypot, tax, proxy, blacklist, ownership risks, and returns a risk score, enabling rug-pull protection for agents via pay-per-call x402 micropayments.MIT
- AlicenseNot gradedqualityAmaintenanceChecks wash-traffic risk of Algorand x402 endpoints using on-chain analysis, helping agents decide whether to pay USDC.60MIT
- FlicenseNot gradedqualityCmaintenanceProvides counterparty risk checks for any EVM address, returning malicious-history flags, tiered risk scoring, and plain-language summaries via x402 payment.-
Try in Browser
Glama MCP Gateway
Add one secure layer between your agents and this server.
TDQS
A4.2/5.0
Disambiguation5/5
With only one tool exposed, there is no possibility of confusing it with another operation. The purpose is singular and clearly stated.
Naming Consistency5/5
There is only one tool name, so there is no mixed convention or irregular verb pattern to flag. The name is descriptive and matches the server's purpose.
Tool Count4/5
A single tool is minimal but appropriate for a narrowly scoped robots.txt checker. It is slightly below the typical 3-15 range, but the server is not over-scoped.
Completeness5/5
The server's entire purpose is to check a URL path against robots.txt, and the single tool fully covers that action. No additional operations are needed for this domain.