Skip to main content
Glama

APIClaw

by io.apiclaw.api

openapi_v2_webtools_map

Discover URLs on a website

Discover URLs from a website via sitemap and on-page link discovery.

Returns a flat, relevance-ordered list of links (each with optional title / description). Use search to rank links by keyword relevance; sitemapMode to control sitemap usage (include / only / skip); includePaths / excludePaths to filter by URL path regex (≤50 patterns, ≤500 chars each); and limit (1-100000, default 5000) to cap results.

Subdomains of the seed host are included by default; set includeSubdomains to false to lock discovery to the exact seed host. URLs that differ only by query string are collapsed by default (ignoreQueryParameters = true); set false to keep query-string variants as distinct URLs.

Responses:

200: Successful Response (Success Response) Content-Type: application/json

Example Response:

{
  "success": true,
  "meta": {
    "requestId": "Requestid",
    "timestamp": "Timestamp"
  }
}

Output Schema:

{
  "properties": {
    "success": {
      "type": "boolean",
      "title": "Success",
      "description": "Whether the request was successful",
      "default": true
    },
    "data": {
      "description": "Response data payload"
    },
    "error": {
      "description": "Error details if request failed"
    },
    "meta": {
      "description": "Metadata for API responses.\n\nCredit fields follow the ADR-0003 parallel-fields strategy (Option 3):\n- `credits_remaining` / `credits_consumed` (int): legacy fields, rounded\n  to whole credits, kept for zero-breaking-change to existing SDK clients.\n- `credits_remaining_exact` / `credits_consumed_exact` (float): new\n  precision-aware fields for clients that opt in to decimal credits.\n\nSee ADR-0003 decision 5 and the \u00a78 deprecation timeline.\n\nTODO(2026-11, ADR-0003 \u00a78 +6mo): mark `credits_remaining` /\n`credits_consumed` as `deprecated=True` in their Field() definitions\nand announce in customer changelog.\nTODO(2027-05, ADR-0003 \u00a78 +12mo): remove the legacy int fields via a\nmajor-version bump of the OpenAPI surface.",
      "properties": {
        "requestId": {
          "type": "string",
          "title": "Requestid",
          "description": "Unique request identifier"
        },
        "timestamp": {
          "type": "string",
          "title": "Timestamp",
          "description": "Response timestamp in ISO 8601 format"
        },
        "total": {
          "title": "Total",
          "description": "Total number of records"
        },
        "page": {
          "title": "Page",
          "description": "Current page number"
        },
        "pageSize": {
          "title": "Pagesize",
          "description": "Number of records per page"
        },
        "totalPages": {
          "title": "Totalpages",
          "description": "Total number of pages"
        },
        "creditsRemaining": {
          "title": "Creditsremaining",
          "description": "Remaining API credits (rounded to whole credits; see creditsRemainingExact for precise value)"
        },
        "creditsConsumed": {
          "title": "Creditsconsumed",
          "description": "Credits consumed by this request (rounded; see creditsConsumedExact for precise value)"
        },
        "creditsRemainingExact": {
          "title": "Creditsremainingexact",
          "description": "Remaining API credits, precise to 1 decimal place"
        },
        "creditsConsumedExact": {
          "title": "Creditsconsumedexact",
          "description": "Credits consumed by this request, precise to 1 decimal place"
        },
        "tokensUsage": {
          "description": "Provider token-usage block \u2014 populated on terminal video polls only, null on every non-video endpoint. See TokensUsage for its fields."
        }
      },
      "type": "object",
      "required": [
        "requestId",
        "timestamp"
      ],
      "title": "ResponseMeta"
    }
  },
  "type": "object",
  "required": [
    "meta"
  ],
  "title": "OpenApiResponse[CrawlerMap]",
  "examples": []
}

422: Validation Error Content-Type: application/json

Example Response:

{
  "detail": [
    {
      "loc": [],
      "msg": "Message",
      "type": "Error Type",
      "ctx": {}
    }
  ]
}

Output Schema:

{
  "properties": {
    "detail": {
      "items": {
        "properties": {
          "loc": {
            "items": {},
            "type": "array",
            "title": "Location"
          },
          "msg": {
            "type": "string",
            "title": "Message"
          },
          "type": {
            "type": "string",
            "title": "Error Type"
          },
          "input": {
            "title": "Input"
          },
          "ctx": {
            "type": "object",
            "title": "Context"
          }
        },
        "type": "object",
        "required": [
          "loc",
          "msg",
          "type"
        ],
        "title": "ValidationError"
      },
      "type": "array",
      "title": "Detail"
    }
  },
  "type": "object",
  "title": "HTTPValidationError"
}

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
urlYesSeed URL to discover links from.
limitNoMaximum number of URLs to return (1-100000). Default 5000.
searchNoKeyword filter; discovered URLs are ranked by relevance to this term.
sitemapModeNoHow the site's sitemap is used for discovery. ``include`` (default) combines the sitemap with on-page link discovery — the most complete coverage; ``only`` returns sitemap URLs exclusively (fastest, but misses pages not in the sitemap); ``skip`` ignores the sitemap and discovers by link crawling only (misses pages that are in the sitemap but not linked, e.g. orphan/deep pages — useful when the sitemap is stale or inaccurate).include
excludePathsNoDrop URLs whose path matches one of these regex patterns (e.g. ``/admin/.*``). Max 50 patterns, each ≤500 chars.
includePathsNoOnly return URLs whose path matches one of these regex patterns (e.g. ``/blog/.*``). Max 50 patterns, each ≤500 chars.
includeSubdomainsNoInclude subdomains of the seed host (e.g. blog.example.com when mapping example.com). Defaults to ``true``; set ``false`` to lock discovery to the exact seed host.
ignoreQueryParametersNoCollapse URLs that differ only by query string so each page is returned once. Defaults to ``true`` (recommended for discovery — avoids near-duplicate ``?utm=``/``?page=`` URLs). Set ``false`` to keep query-string variants as distinct URLs.

TDQS

A4.2/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden and does so thoroughly. It explains sitemap modes (include/only/skip), subdomain handling, query parameter collapsing, and limit semantics. It also includes specific behavioral details like 'relevance-ordered list' and default recommendations, giving agents a clear model of expected behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The core functional description is well-structured and front-loaded, but the description field is bloated by a full response schema and example for both 200 and 422 statuses, plus extensive metadata credit field details. This goes well beyond what's needed and detracts from conciseness.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers all 8 parameters, key edge cases (subdomains, query params, sitemap modes), and includes response examples. It lacks explicit guidance on when to use this tool over sibling tools, but otherwise provides a complete operational picture for an AI agent.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Since the input schema already provides 100% parameter coverage, the baseline is 3. The description adds value beyond the schema by explaining why certain defaults are recommended (e.g., 'recommended for discovery — avoids near-duplicate ?utm=/?page= URLs') and by summarizing parameter interrelationships like 'Use search to rank links by keyword relevance.'

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: 'Discover URLs from a website via sitemap and on-page link discovery.' This is a specific verb+resource combination that immediately distinguishes it from sibling tools like scrape, search, or crawl_status.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description does not explicitly compare this tool to alternatives or provide when-to-use vs. when-not-to-use guidance. Usage is implied by the tool's obvious mapping purpose, but there are no exclusions or alternative recommendations, so it earns a mid-range score.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All APIClaw toolsInspect
Try in Browser
RayrunBuild, deploy, and operate MCP servers without becoming an infrastructure team.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

TDQS

B3.2/5.0
Disambiguation2/5

Several tools have unclear boundaries, most notably openapi_v2_competitor_product_keywords and openapi_v2_product_traffic_terms, which have identical descriptions. The deprecated openapi_v2_realtime_product duplicates openapi_v3_realtime_product, and openapi_v2_image_embedding overlaps heavily with openapi_v2_fashion_image_embedding.

Naming Consistency2/5

Naming is inconsistent: a few tools follow a clean verb_noun pattern (create_video_asset, list_video_assets, poll_video_task), while the vast majority are prefixed with openapi_v2_/openapi_v3_ followed by nouns or mixed verbs. The route-style prefix is not a meaningful verb and creates an arbitrary split across the tool set.

Tool Count2/5

49 tools is far beyond the well-scoped range and spans multiple unrelated domains (Amazon product data, TikTok commerce, fashion vision, web scraping, video generation, billing). The server appears to be an entire REST API surface exposed wholesale rather than a curated set of capabilities.

Completeness3/5

Each domain is individually fairly complete (Amazon search/reviews/keywords/VoC, TikTok search, web tools, video generation), but there are notable gaps: video assets lack update/delete, video tasks lack cancel/list, and there is no singular 'get daily product by ASIN' alongside the realtime variants. The broad scope makes it hard to verify full lifecycle coverage across all domains.

Resources