silkworm-mcp
Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
| MCP_HOST | No | Host for MCP HTTP server | 0.0.0.0 |
| MCP_PATH | No | Path for MCP HTTP server | |
| MCP_PORT | No | Port for MCP HTTP server | 8000 |
| MCP_TRANSPORT | No | Transport type for MCP server (stdio or http) | http |
| LIGHTPANDA_HOST | No | Lightpanda browser host | 127.0.0.1 |
| LIGHTPANDA_PORT | No | Lightpanda browser port | 9222 |
| LIGHTPANDA_ENABLED | No | Enable Lightpanda browser for CDP tools | 1 |
| LIGHTPANDA_LOG_LEVEL | No | Log level for Lightpanda | info |
| LIGHTPANDA_LOG_FORMAT | No | Log format for Lightpanda | pretty |
| SILKWORM_MCP_LOG_LEVEL | No | Log level for MCP server | |
| LIGHTPANDA_ADVERTISE_HOST | No | Advertised host for Lightpanda CDP endpoint | |
| SILKWORM_MCP_DOCUMENT_MAX_COUNT | No | Maximum number of documents in cache | |
| SILKWORM_MCP_DOCUMENT_STORE_PATH | No | Path to document store | |
| SILKWORM_MCP_DOCUMENT_TTL_SECONDS | No | Idle TTL for document cache in seconds | |
| SILKWORM_MCP_READINESS_REQUIRE_CDP | No | Whether readiness probe requires CDP browser | |
| SILKWORM_MCP_DOCUMENT_MAX_TOTAL_BYTES | No | Maximum total bytes for document cache | |
| SILKWORM_MCP_READINESS_CDP_WS_ENDPOINT | No | CDP WebSocket endpoint for readiness probe |
Instructions
Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.
This server publishes no instructions, or was last inspected before Glama recorded them.
Capabilities
Features and capabilities supported by this server
Protocol revision2025-11-25
| Capability | Details |
|---|---|
| tools | {
"listChanged": true
} |
| prompts | {
"listChanged": false
} |
| resources | {
"subscribe": false,
"listChanged": false
} |
| extensions | {
"io.modelcontextprotocol/ui": {}
} |
| experimental | {} |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| store_html_documentC | Store raw HTML in the server's in-memory document cache and return a scraper_rs summary. |
| list_documentsA | List cached documents that can be reused by handle in later tool calls. |
| delete_documentC | Delete a cached document handle. |
| clear_documentsA | Clear every cached document from the in-memory store. |
| server_statusB | Return runtime status, cache metrics, and optional CDP readiness information. |
| generate_regexC | Generate a regular expression from sample strings using grex. |
| inspect_documentC | Build a high-level summary for stored HTML or an inline HTML snippet. |
| prettify_documentC | Return prettified HTML for visual inspection. |
| parse_html_documentC | Parse a full HTML document into a structured node tree using scraper_rs.parse_document. |
| parse_html_fragmentC | Parse an HTML fragment into a structured node tree using scraper_rs.parse_fragment. |
| query_selectorC | Run a CSS or XPath query with scraper_rs and return structured match previews. |
| compare_selectorsB | Compare multiple selectors against the same document to see which one is the most stable. |
| analyze_css_selectorsC | Parse inline, linked, or raw CSS with tinycss2 and optionally match selectors back onto HTML. |
| find_selectors_by_textC | Find CSS/XPath selector candidates for the smallest matching elements by text. |
| extract_linksC | Extract and absolutize link-like attributes from matching elements. |
| silkworm_fetchB | Fetch a page through silkworm's HttpClient and optionally cache the HTML for later selector work. |
| silkworm_fetch_cdpB | Fetch rendered HTML through silkworm's CDP client for JavaScript-heavy pages. |
| query_selector_cdpA | Fetch a rendered page through CDP, then run a CSS or XPath query on the live DOM snapshot. |
| extract_structured_data_cdpC | Fetch a rendered page through CDP and extract structured records from the rendered DOM. |
| validate_spider_codeB | Statically validate generated spider code for syntax and common silkworm/CDP wiring. |
| run_crawl_blueprintB | Run a configurable silkworm spider without writing code, useful for validating a scraping plan. |
| generate_spider_templateC | Generate a production starter spider that mirrors the crawl blueprint. |
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
| plan_silkworm_scraper | |
| debug_selector_strategy |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
| status_resource | |
| reference_overview | |
| silkworm_cheatsheet | |
| silkworm_playbook | |
| template_variants_reference | |
| scraper_rs_cheatsheet | |
| crawl_blueprint_schema | |
| documents_resource |
TDQS
Scored across 22 tools
Most tools have clear, distinct purposes, and the _cdp suffix separates rendered from static operations. However, the cluster of selector-related tools (query_selector, compare_selectors, analyze_css_selectors, find_selectors_by_text) and overlapping inspection tools (inspect_document, prettify_document, parse_html_document) could cause some misselection without careful reading.
Tool names largely follow a verb_noun snake_case pattern (e.g., store_html_document, list_documents, parse_html_fragment). Minor inconsistencies like server_status (noun-first) and the silkworm_ prefix on some fetch tools break the strict pattern but remain readable and predictable.
At 22 tools, this is on the heavy side but justified by the broad scope of a web scraping framework covering fetching, caching, parsing, selector analysis, and code generation. The count is manageable, but agents may need to scan many options before choosing.
The toolset covers the full scraping lifecycle from fetching (normal and CDP) to caching, parsing, selector analysis, structured extraction, and spider validation/generation. Minor gaps exist, such as no explicit export/save tool for extracted data and no update operation for cached documents, but these are workable.