Skip to main content
Glama
Sign In
fairquery-web

fairquery-web

Official
by fairquery
OverviewSchemaSourceRelated ServersDiscussionsAdmin
TypeScript
Local

XML sitemap discovery and URL listing (free, paged)

web_sitemap_info
Read-onlyIdempotent

Find a site's XML sitemap, list child sitemaps, and page through URLs with lastmod to count indexed pages or surface recently modified URLs for SEO audits.

Instructions

Find a site's XML sitemap (robots.txt Sitemap: lines, then /sitemap.xml, /sitemap_index.xml, or an explicit sitemapUrl), report whether it is a sitemap index, list child sitemaps, and page through the contained URLs with lastmod. WHEN TO USE: count/inspect indexed pages, find recently modified URLs (filter with contains / sort), feed URLs into web_seo_audit. Fetching: GETs from this server with User-Agent "FairqueryBot ... Fairquery-MCP", 15 s timeout per request, body capped (1.5 MB HTML), max 5 redirects, no JavaScript rendering; URLs resolving to private/loopback/link-local addresses are refused (SSRF guard). Sitemap files are capped at 10 MB each (meta.truncated=true when cut). RETURNS: paged "url" rows (lastmod is null when the sitemap gives none); meta says which sitemap was used and whether it is an index. Errors: SITEMAP_NOT_FOUND plus INVALID_URL, BLOCKED_PRIVATE_ADDRESS, DNS_FAILED, FETCH_FAILED, TIMEOUT, TOO_MANY_REDIRECTS. WHEN NOT TO USE: not a crawler and does not audit pages - pass selected URLs to web_seo_audit; for AI-bot robots.txt rules use web_ai_crawler_access. COST: free - runs locally in this MCP server (direct HTTP GETs), no APIFY_TOKEN, no Apify billing. PARAMETERS: url: only its origin is used for discovery (the path is ignored); discovery tries robots.txt Sitemap: lines, /sitemap.xml and /sitemap_index.xml (max 4 candidates) and, if nothing is found, retries once on the origin the homepage redirects to (example.com -> www.example.com). sitemapUrl skips discovery entirely (no robots.txt, no retry) - use it for sitemaps at non-standard paths; it must be an absolute http(s) URL. maxChildSitemaps only matters when the sitemap is an index: only the first N children (index order) are downloaded, so total counts URLs of those children only; 0 lists child_sitemaps without URLs. contains is a case-sensitive substring test on loc; sort "lastmod_desc" orders by lastmod (ISO strings) with undated URLs last. Both are applied before paging, so total reflects the filtered list; continue with offset = nextOffset. EXAMPLE: {"url":"python.org","contains":"/blog/","sort":"lastmod_desc","limit":10} -> paged url rows with loc and lastmod.

Input Schema

NameRequiredDescriptionDefault
urlYesRequired. One public domain or http(s) URL (non-empty string), e.g. "vercel.com" or "https://www.python.org/about/". Bare domains are fetched over https; other schemes, credentials in the URL, dot-less hosts and private/loopback addresses are refused.
sortNoOptional, one of "sitemap" | "lastmod_desc", default "sitemap" (file order); "lastmod_desc" = most recently modified first, undated last
limitNoOptional page size: number of items to return, integer 1-1000 (default 100). Example: 20
offsetNoOptional zero-based offset into the full result list, integer >= 0 (default 0); use nextOffset from the previous page, e.g. 50
containsNoOptional, default no filter. Case-sensitive substring that loc must contain. Example: "/blog/"
sitemapUrlNoOptional, default = auto-discovery. Absolute http(s) URL of a sitemap or sitemap index to use instead of discovery. Example: "https://www.python.org/sitemap.xml"
maxChildSitemapsNoOptional integer 0-20, default 3. For sitemap indexes only: number of child sitemaps (in index order) to fetch and list URLs from; 0 = list children only. Example: 10

Output Schema

NameRequiredDescriptionDefault
metaNoTool-specific context (company, source_url, ...)
countYesNumber of items in this page
itemsYesResult rows for this page
totalYesTotal items matching the filters (before paging)
offsetYesOffset of the first item in this page
hasMoreYesTrue if more items exist after this page
nextOffsetYesOffset to pass for the next page, or null

Schema Changelog

Changes observed during successful MCP inspections.

  1. Changed7 schema fields changedv1.2.4
    • changedInput schema / properties / contains / description
      Previous value: -"Only URLs containing this substring, e.g. \"/blog/\""New value: +"Optional, default no filter. Case-sensitive substring that loc must contain. Example: \"/blog/\""
    • changedInput schema / properties / limit / description
      Previous value: -"Optional page size: number of items to return, integer 1-1000 (default 100)"New value: +"Optional page size: number of items to return, integer 1-1000 (default 100). Example: 20"
    • changedInput schema / properties / maxChildSitemaps / description
      Previous value: -"For sitemap indexes: how many child sitemaps to fetch and list URLs from (default 3, max 20; 0 = list children only)"New value: +"Optional integer 0-20, default 3. For sitemap indexes only: number of child sitemaps (in index order) to fetch and list URLs from; 0 = list children only. Example: 10"
    • changedInput schema / properties / offset / description
      Previous value: -"Zero-based offset into the full result list (default 0); use nextOffset from the previous page"New value: +"Optional zero-based offset into the full result list, integer >= 0 (default 0); use nextOffset from the previous page, e.g. 50"
    • changedInput schema / properties / sitemapUrl / description
      Previous value: -"Explicit sitemap URL to use instead of auto-discovery"New value: +"Optional, default = auto-discovery. Absolute http(s) URL of a sitemap or sitemap index to use instead of discovery. Example: \"https://www.python.org/sitemap.xml\""
    • changedInput schema / properties / sort / description
      Previous value: -"\"sitemap\" = file order (default); \"lastmod_desc\" = most recently modified first"New value: +"Optional, one of \"sitemap\" | \"lastmod_desc\", default \"sitemap\" (file order); \"lastmod_desc\" = most recently modified first, undated last"
    • changedInput schema / properties / url / description
      Previous value: -"Required. One public domain or http(s) URL, e.g. \"vercel.com\" or \"https://www.python.org/about/\". Bare domains are fetched over https; private/loopback hosts are refused."New value: +"Required. One public domain or http(s) URL (non-empty string), e.g. \"vercel.com\" or \"https://www.python.org/about/\". Bare domains are fetched over https; other schemes, credentials in the URL, dot-less hosts and private/loopback addresses are refused."
  2. Addedv1.2.1

TDQS

A4.9/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare read-only/idempotent/open-world, but the description adds substantial context beyond them: User-Agent string, 15 s per-request timeout, 1.5 MB HTML cap, max 5 redirects, no JS rendering, an SSRF guard refusing private/loopback addresses, a 10 MB sitemap cap with meta.truncated signaling, the free/no-billing cost model, and a full error-code list.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Heavily but usefully sectioned (WHEN TO USE, Fetching, RETURNS, WHEN NOT TO USE, COST, PARAMETERS, EXAMPLE) and front-loaded with purpose. It is on the verbose side and some RETURN/error detail is arguably redundant given an output schema exists, but nearly every sentence carries operational information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a 7-parameter, non-trivial discovery tool, the definition covers discovery behavior, fetch constraints, pagination, filtering, truncation, error taxonomy, cost, and a worked example. An output schema exists, yet no essential calling information is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters5/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Although schema coverage is 100%, the description adds semantics the schema does not carry: url uses only its origin for discovery, the candidate order (robots.txt Sitemap: lines, /sitemap.xml, /sitemap_index.xml, max 4) plus a redirect-origin retry, and that sitemapUrl skips discovery entirely. It also explains that maxChildSitemaps bounds the total count, that contains/sort apply before paging, and that pagination continues via nextOffset.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence states a specific verb and resource chain: 'Find a site's XML sitemap', report index status, list child sitemaps, and page through URLs with lastmod. It also explicitly distinguishes itself from web_seo_audit ('not a crawler and does not audit pages') and web_ai_crawler_access, so an agent can route without opening a schema.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Explicit WHEN TO USE ('count/inspect indexed pages, find recently modified URLs, feed URLs into web_seo_audit') and WHEN NOT TO USE sections name the alternative tools for the excluded cases. The condition selecting sitemapUrl vs. auto-discovery is also spelled out, leaving nothing to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Deploy Server

Other Tools