Crawl4Agent
Server Details
crawl4ai-compatible web crawler API: POST a URL, get clean LLM-ready Markdown with fit_markdown...
- Status
- Healthy
- Last Tested
- Transport
- Streamable HTTP · MCP 2025-06-18
- URL
TDQS
Scored across 1 tool
With only one tool, there is no possibility of misselection or overlap; the tool's purpose (URL to Markdown) is unambiguous.
A single tool named 'md' is extremely terse and does not follow any verb_noun or descriptive pattern, though there is no second tool to be inconsistent with. Readability suffers for agents unfamiliar with the abbreviation.
One tool for an entire server is thin; the purpose (URL to Markdown conversion) is narrow enough that it is defensible, but there is no room for variant outputs or cache management as separate operations.
The tool covers the core Markdown conversion with filter/query/cache options, but the surface lacks related crawl outputs (raw HTML, text, screenshots, batch/multi-URL) that a crawling server would typically expose.
Available Tools
1 toolmdTurn a URL into LLM-ready Markdown with crawl4ai's /md contractAInspect
Turn a URL into LLM-ready Markdown with crawl4ai's /md contract: f raw|fit (raw or pruned fit_markdown), q optional user query, c cache flag. Answers {url, filter, query, cache, markdown, success}. Price: $0.008 a call.
| Name | Required | Description | Default |
|---|---|---|---|
| c | No | Cache flag ("0" default, "1" to bypass the cache) | |
| f | No | raw = raw_markdown, fit = pruned fit_markdown | |
| q | No | Optional user query for BM25 fit filtering | |
| url | Yes | The page to crawl: an https:// URL |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the burden. It usefully discloses the response shape ({url, filter, query, cache, markdown, success}), cache semantics, and per-call price ($0.008), but says nothing about auth requirements, rate limits, failure modes, or what happens to the page when success is false for a network-dependent crawl.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two tightly packed sentences: purpose first, then a param legend, then the answer shape and price. Nothing is wasted, though the telegraphic a/b/c style is terse rather than flowing and the title is repeated verbatim in the first clause.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 4-parameter tool with no output schema, the description compensates well by spelling out the return keys and cost. Direction on choosing raw vs fit, and any failure/retry behavior, are the only meaningful gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the baseline is 3. The description's parameter legend (f raw|fit, q optional query, c cache flag) largely restates what the schema already documents, including the raw_markdown/fit_markdown mapping, adding no format or syntax detail beyond it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a precise verb+resource: turn a URL into LLM-ready Markdown via crawl4ai's /md contract. It names the underlying contract, so an agent knows exactly what transform is performed. No sibling tools exist, so no differentiation is needed.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is only implied via the parameter legend (raw vs fit, optional BM25 query, cache bypass), with no explicit statement of when to choose this tool or when raw is preferred over fit. With no siblings there are no alternatives to route away from, but the when-to-use conditions remain unstated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
1 tool update
- First observed
md
Related MCP Connectors
URL to clean markdown for LLMs: a polite, robots.txt-respecting web reader. Free, no API key
Fetch any URL and get clean Markdown. Web scraping for AI agents.
Converts any URL to clean, LLM-ready Markdown using real Chrome browsers
Scrape any URL into clean LLM-ready Markdown, or crawl a whole site.
Related MCP Servers
- AlicenseAqualityCmaintenanceFetches any web URL and converts it to clean Markdown via the web2md service (Crawl4AI backend). Designed for ModelScope MCP Hub.1MIT
- AlicenseAqualityAmaintenanceConverts any web page URL into clean Markdown for LLM context (Claude, ChatGPT, etc.) with zero external API calls, running entirely locally.166 npmMIT
- AlicenseNot gradedqualityBmaintenanceEnables clean, LLM-ready markdown extraction from any URL with automatic anti-bot bypass.3MIT
- AlicenseNot gradedqualityCmaintenanceEnables converting any URL or website into clean, LLM-ready Markdown, supporting single-page scraping and full-site crawling for AI agents.MIT
Glama MCP Gateway
Add one secure layer between your agents and this server.