Skip to main content
Glama

RAG Dataset Builder

sitemap-to-knowledge
Read-only

Give it a domain. It reads the sitemap, fetches the pages, strips them to clean text and splits everything into ~1000-char chunks — one dataset row per chunk, ready to embed into a vector store. No browser, no LLM, no API key. — $0.02/call, x402 (USDC on base).

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
itemsYesList of domains or website URLs to crawl via their sitemap. One entry per site.
maxConcurrencyNoHow many SITES to process in parallel (each site already fetches up to 25 pages internally).

TDQS

A4.2/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, openWorldHint=true, and destructiveHint=false, and the description is consistent with these. It adds useful context beyond annotations: the tool does not use a browser, LLM, or API key, and it includes pricing ($0.02/call). It does not detail failure modes (e.g., missing sitemap, rate limits), but for a read-only tool the safety profile is already clear.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three sentences and front-loaded with the core action ('Give it a domain'). Every sentence earns its place: pipeline explanation, constraints (no browser/LLM/API), and pricing. No fluff or redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description explains the return value (one dataset row per chunk, ready for vector store) and the processing pipeline. It covers cost and constraints. Missing edge-case details (e.g., what happens without a sitemap) are not critical given the simple read-only nature and good annotation coverage.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% description coverage for both parameters (items and maxConcurrency), so the schema itself carries the parameter semantics. The description only reiterates 'Give it a domain' without adding new meaning about parameter formats or relationships. Baseline 3 is appropriate because the schema does the heavy lifting.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states its function: 'reads the sitemap, fetches the pages, strips them to clean text and splits everything into ~1000-char chunks — one dataset row per chunk.' This is a specific verb+resource (sitemap) that produces a RAG-ready dataset, and it distinguishes itself from siblings like url-to-markdown by targeting whole-site crawling via sitemap.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context with 'Give it a domain' and emphasizes its lightweight nature ('No browser, no LLM, no API key'), implying it is ideal for building RAG datasets quickly and cheaply. However, it does not explicitly mention exclusions or alternatives (e.g., 'for a single page, use url-to-markdown'), so it lacks explicit when-not-to-use guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

TDQS

A3.9/5.0
Disambiguation2/5

There are multiple overlapping tool pairs: tech-stack-detector vs tech-stack-change-detector (the latter is a superset that includes detection), and url-to-markdown vs structured-extract vs sitemap-to-knowledge (all fetch web pages and convert content, differing only in output format). These boundaries are unclear, and an agent could easily select the wrong one without deep reading of descriptions.

Naming Consistency2/5

Naming is a mix of hyphenated descriptors (domain-health-checker, tech-stack-detector), snake_case (pricing_info), and verb phrases (structured-extract, url-to-markdown). The pattern is inconsistent: some tools are named after the action (extract, convert), others after the target (shopify-store-intelligence). This makes it hard to predict tool names.

Tool Count5/5

With 10 tools, the count is well within the 3-15 ideal range. Each tool addresses a distinct web intelligence need (domain health, tech stack, e-commerce, content extraction, pricing), and none are purely redundant filler. The scale feels appropriate for the server's stated purpose.

Completeness4/5

The surface covers core web intelligence workflows well: domain auditing, tech stack detection (with change detection), content extraction, and e-commerce monitoring for Shopify and Zid. Minor gaps exist, such as missing generic e-commerce platform coverage or a dedicated WHOIS lookup, but these are not critical given the existing domain-health-checker.

Resources