Skip to main content
Glama
Crawl Readiness MCP Server

Crawl Readiness MCP Server

Check Permission to Use a Page

check_permission
Read-only

Checks whether a page's published signals permit search, AI answers, training, or live agent use, returning a verdict, terms, and a signed receipt before you fetch or quote it.

Instructions

Ask whether a page may be used for a purpose, from every machine-readable signal the site publishes: robots.txt per crawler, Cloudflare Content-Signal, the IETF Content-Usage directive and header, an RSL licence, a TDM reservation (header or /.well-known/tdmrep.json), ai.txt, and noai/noindex/nosnippet on the page itself. Purposes: 'search' (index the page and show links and excerpts, no AI summary), 'answer' (input to a generated answer, including retrieval for RAG), 'train' (train or fine-tune a model), 'agent' (fetch or act on the page for a user, live). Returns a verdict: 'denied' (a signal reserves this use; any reservation wins, a block on any one crawler that does the job included, and a robots.txt answering a server error counts as a full disallow), 'allowed-under-terms' (permitted on the conditions listed in terms, such as payment or a licence server), 'unknown' (a source that could carry a reservation could not be read: robots.txt answered 401 or 403, the page refused the request, or a licence could not be fetched; unknown outranks allowed) or 'allowed' (no signal reserves the use, which is not a licence). With observe: true it also requests the page once with a representative crawler's user-agent string, from Crawl Readiness's servers, and reports the status in observed: that shows how the server treats the string, not what the real crawler gets. Every source read is listed in signals with what was found and what it decided. The receipt is a dated record of all of this, with the SHA-256 of every file read, signed with Ed25519; /api/permission-key publishes the current and retired public keys, so the receipt can be verified later. Use it before fetching pages for RAG, a dataset or an agent action, and whenever the user asks whether they may use, quote, index, train on or act on a page. It is a reading of the signals the site published on that date, not legal advice. 25 calls a day per network without an API key. Documented at https://machinereadiness.com/permission.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "url": "<url>",
      "purpose": "<purpose>"
    },
    "name": "check_permission"
  }
}

Input Schema

NameRequiredDescriptionDefault
urlYesThe page to ask about (e.g. 'https://example.com/article').
agentNoOptional: your own crawler's token (e.g. 'MyBot'), to read robots.txt for it. Another company's token (e.g. 'GPTBot') cannot loosen the reading for the crawlers that do the job. By default that whole class is read.
observeNoDefault false. true also requests the page once with a representative crawler's user-agent string, from our servers, and reports the status; it does not show what the real crawler gets.
purposeYesWhat you intend to do with the page: search, answer (including RAG), train, or agent.

Schema Changelog

Changes observed during successful MCP inspections.

  1. Added

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond readOnlyHint/openWorldHint: it discloses the 25-calls-per-day-per-network limit without a key, that observe:true makes a real outbound request from their servers (and that the result is not what the real crawler sees), the precedence rules among verdicts ('unknown outranks allowed', 'any reservation wins'), and that the receipt is Ed25519-signed with SHA-256 hashes verifiable via /api/permission-key. No contradiction with the read-only annotation, since only outbound reads occur.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with the core question and no filler sentences; every clause carries information (sources, verdicts, caveats, rate limit). It is a single very dense paragraph that would be easier to scan as bullets, so not quite a 5.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description carries the full return-shape burden and does so: it explains the four `verdict` values, `terms`, `signals`, `observed`, and `receipt` with verification path. For a tool touching eight-plus signal types, nothing an agent needs before calling is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so baseline is 3, but the description earns extra by defining what each enum value actually means in practice ('search' = index and show links and excerpts with no AI summary; 'answer' = input to a generated answer including RAG retrieval; 'agent' = live fetch or act), which the schema only gestures at.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb+resource ('ask whether a page may be used for a purpose') and enumerates the exact signal sources consulted, so an agent can distinguish it from validate_robots or check_ai_readiness, which inspect syntax/readiness rather than return a permission verdict.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives explicit triggers: 'before fetching pages for RAG, a dataset or an agent action' and 'whenever the user asks whether they may use, quote, index, train on or act on a page', plus the caveat that it is not legal advice. It does not name a sibling alternative or state when NOT to use it, which keeps it short of a 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.