Skip to main content
Glama
Sign In

AskYourStack

by com.askyourstack
OverviewDiscussionsAdmin

Crawl and audit a whole site

site_audit

Crawl a whole site from the server it runs on, link by link from the start page as a search engine would (max_pages, default 500, up to 5000), and keep what every page answered: status, redirects, title, description, h1, canonical, noindex, word count, links, images without alt text, structured data, response time, clicks from the start page, and whether the sitemap lists it. It then names the issues: links to missing pages, server errors, redirect chains and links to redirects, missing or repeated titles and descriptions, the same text on several addresses, thin pages, slow pages, sitemap entries that redirect, fail or are noindex, pages only the sitemap knows, pages missing from the sitemap, pages robots.txt blocks. Pages are fetched straight from the server's own web server, so a CDN or firewall in front of the site does not block it, and only this site is fetched. It runs in the background and returns a job_id: call job_output until status is finished (the last lines are the summary), then site_audit_report for the pages behind each issue. Give site (see list_sites) or url, the address to start from. Addresses with parameters are sampled (50) unless query_urls says all or none.

Input Schema

NameRequiredDescriptionDefault
urlNoAddress to start from, like https://www.example.com/. Default: the site's own address.
viaNoauto (default): from the server's own web server when it serves the site, else over the internet. public: always as a visitor, through any CDN.
siteNoSite root folder, like /var/www/shop, or one of the site's domains. Optional when the server has one site (see list_sites).
serverNoServer name or id. Optional when the account has one server.
max_pagesNoPages to fetch, default 500, at most 5000. About 3 to 10 pages a second.
query_urlsNoAddresses with ?parameters: fetch 50 as a sample (default), all, or none.
concurrencyNoPages fetched at the same time, 1 to 8, default 3. Lower it on a small or busy server.

Schema Changelog

Changes observed during successful MCP inspections.

  1. First observed

TDQS

A4.6/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Goes well beyond the annotations by disclosing that pages are fetched directly from the server's own web server so a CDN/firewall will not block the crawl, that only the target site is fetched, that query URLs are sampled, and that the job runs in the background returning a job_id. The readOnlyHint=false, openWorldHint=true, idempotentHint=false annotations are consistent with a background crawl-and-store job.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Front-loaded with purpose and organized as purpose -> what is collected -> issues found -> mechanics -> workflow -> inputs. It is dense and long, with some enumerations that border on exhaustive, but nearly every clause carries information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

No output schema exists, and the description compensates by explaining the returned job_id, the polling loop, and that site_audit_report holds the per-issue pages. Combined with the issue taxonomy and parameter guidance, an agent has everything needed to invoke and follow up.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% so the baseline is 3, but the description adds real meaning on top of the schema: the query_urls sampling behaviour (50 unless all/none), the up-to-5000 cap on max_pages, and pointers to list_sites for the site parameter. It largely restates rather than extends for the rest.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

Opens with a specific verb+resource ('Crawl a whole site ... from the server it runs on') and enumerates exactly what data is retained and which issues are named. This is clearly distinguishable from per-page siblings like page_audit, crawl_report and bing_crawl.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Gives concrete operating context: supply either site (see list_sites) or url, and describes the async workflow (call job_output until finished, then site_audit_report). It does not, however, state when to prefer this over page_audit, crawl_report, or bing_crawl.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

All AskYourStack toolsInspect
Try in Browser
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Glama MCP Gateway

Add one secure layer between your agents and this server.

Resources