toolvend - utility APIs for agents

robots

Fetch and parse robots.txt (+ llms.txt if present) into structured JSON, including which AI crawlers are blocked. Paid per call: $0.005 in USDC via x402.

Tool call template

Replace each placeholder with a value matching the input schema. Only required parameters are included; add optional parameters as needed.

{
  "id": 1,
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "arguments": {
      "url": "<url>"
    },
    "name": "robots"
  }
}

Input Schema

Name	Required	Description	Default
`url`	Yes	Public HTTP(S) site origin or URL

Schema Changelog

Changes observed during successful MCP inspections.

Changed1 schema field changedOct 26, 2026
- removedInput schema / additionalProperties
```
Removed value: -false
```
First observedAug 2, 2026

TDQS

A3.8/5.0

Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description must carry the behavioral burden, and it does disclose a critical trait: paid per call at $0.005 in USDC via x402. It also states that llms.txt is parsed only if present, but it does not describe error behavior for missing robots.txt or malformed responses.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two tight sentences with no wasted words. The core capability is front-loaded, followed immediately by the important cost detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter fetch/parse tool without an output schema, the description adequately explains the return shape: structured JSON including AI crawler blocking. It also discloses the payment model. It falls short of 5 because it does not mention behavior when robots.txt is absent or the request fails.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There is only one parameter, url, and the schema already documents it at 100% coverage as a public HTTP(S) site origin or URL. The description adds no further meaning about accepted formats, redirects, or how origins versus full URLs are treated, so the schema does the heavy lifting and baseline 3 applies.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource: fetch and parse robots.txt (plus llms.txt when present) into structured JSON. It clearly explains the unique value of identifying blocked AI crawlers. It does not explicitly distinguish itself from siblings like sitemap or url-inspect, which keeps it below a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied by the mention of robots.txt and AI crawler blocking, so an agent can infer when the tool is relevant. However, there is no explicit guidance on when to use this versus alternatives such as sitemap or url-inspect, and no exclusions or prerequisites are stated.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All toolvend - utility APIs for agents tools Inspect

Try in Browser

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources

Need Help?