Skip to main content
Glama

site_map

Discover a site's real URLs and URL patterns from robots.txt sitemaps, root, or a polite crawl; filter by regex, refresh cached results, or use the Wayback index to avoid requests.

Instructions

A site's real urls (robots.txt sitemaps, a polite link walk, the catalogue's root) with their url patterns. filter is a regex. A listing is reused for 12 hours; fresh=true lists the site again (only when it changed: re-listing a whole site run after run is how crawlers get blocked). archive='only' lists the site from the Wayback Machine's index without a single request to it; 'auto' does so when the site is on hold or listed nothing. Use it instead of guessing paths.

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
urlYes
freshNo
limitNo
filterNo
archiveNoauto

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observedv0.4.0

TDQS

A3.8/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full behavioral burden and does well: it discloses a 12-hour listing reuse window, explains that fresh=true re-lists only when the site changed, warns that repeated full listing can get crawlers blocked, and describes Wayback Machine behavior for archive modes. It does not cover return format, pagination, or error behavior.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is dense but front-loaded with the tool's purpose and mechanisms. Every sentence adds some operational detail, though the parameter explanations are interleaved rather than clearly structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

There is no output schema and no annotations, so the description must be complete. It covers important behavioral details and hints at returned URL patterns, but omits the limit parameter, return format specifics, and error or permission behavior.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It explains filter as a regex, fresh=true semantics, and archive='only'/'auto' behavior, but never explains the url parameter or the limit parameter, leaving two of five parameters undocumented.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the resource and method: it returns a site's real URLs gathered from robots.txt sitemaps, a polite link walk, and the catalogue root, along with their URL patterns. It distinguishes itself from generic guessing rather than from named sibling tools, so it is clear but lacks explicit sibling differentiation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives clear context for use with 'Use it instead of guessing paths,' and explains when to use fresh=true and archive='only' or 'auto.' It does not name alternative sibling tools or state explicit exclusions, but the usage context is clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.