SiteCheck Web Intel

Reads a site's public robots.txt and security.txt, and says

security_robots_security_txt

Read-only

Reads a site's public robots.txt (crawl rules, sitemaps, AI-crawler rules) and security.txt (RFC 9116: contact, expiry, policy), and says whether security.txt is present, current or expired. Missing files are reported as absent, not as errors. Usually answers in a few seconds, up to 23 s on slower calls; allow a 45 s timeout. Related tools on this server, for a different need: security_cookie_security when you need to list the cookies a site sets on its public home page; security_security_headers when you need to check the live HTTP response headers of a site's home page. Price: $0.002 in USDC per successful call, paid with x402 (Base, Solana or Arc); failed calls are not charged.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "domain": "<domain>"
    },
    "name": "security_robots_security_txt"
  }
}

Input Schema

Name	Required	Description	Default
`domain`	Yes	Domain name such as github.com (required)

Output Schema

Name	Required	Description	Default
`domain`	No
`source`	No
`checkedAt`	No
`robotsTxt`	No
`securityTxt`	No

Schema Changelog

Changes observed during successful MCP inspections.

First observedOct 23, 2026

TDQS

A4.4/5.0

Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the readOnly/openWorld annotations: missing files are reported as absent rather than errors, typical latency is a few seconds with a 23 s worst case and a recommended 45 s timeout, and the call is billed $0.002 USDC via x402 with failed calls uncharged. These are exactly the operational facts an agent needs before invoking a paid network call.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loads what is read and what is returned, then handles errors, timing, siblings, and price in a compact sequence. The single dense paragraph packs multiple distinct concerns (semantics, latency, routing, billing) that could be split for scannability, but no sentence is filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With an output schema present, return structure need not be explained, and the description still covers error semantics, latency/timeout budget, cost and payment rails, and sibling routing. Nothing material for a correct single-parameter invocation is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Only one parameter exists and the schema already documents it at 100% coverage ("Domain name such as github.com"), so the baseline is 3. The description implies a bare site domain rather than a full URL but adds no further format or normalization guidance beyond what the schema states.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (reads) and specific resources (robots.txt crawl rules/sitemaps/AI-crawler rules, and security.txt per RFC 9116), plus the derived verdict it returns (present/current/expired). An agent can distinguish this from the sibling header/cookie tools without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicitly routes two neighboring needs elsewhere: security_cookie_security for cookies on the home page and security_security_headers for live HTTP response headers, each with a stated condition. It does not state exclusions for this tool itself (e.g., that it is not for general DNS or non-HTTPS inspection), so it stops just short of full when/when-not coverage.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All SiteCheck Web Intel tools Inspect

Try in Browser

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources

Need Help?