batch-fetch-text
Retrieve text content from multiple URLs simultaneously using HTTP methods like GET, POST, PUT, or DELETE. Supports concurrent requests for improved performance and offers output formats including markdown, clean text, or raw HTML.
Instructions
Batch fetch raw text content from multiple URLs using various HTTP methods. Executes requests concurrently for better performance.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| requests | Yes | Array of URLs (strings) or request objects |
Implementation Reference
- src/jsonrpc_mcp/server.py:234-242 (handler)Handler logic in @server.call_tool() for 'batch-fetch-text': validates input, extracts output_format, calls batch_fetch_urls helper, and returns JSON serialized result.elif tool_name == "batch-fetch-text": requests = args.get("requests", []) if not isinstance(requests, list) or not requests: result = "Failed to call tool, error: Missing or empty 'requests' array" else: output_format = args.get("output_format", "markdown") response_result = await batch_fetch_urls(requests, as_json=False, output_format=output_format) result = json.dumps(response_result)
- src/jsonrpc_mcp/server.py:151-195 (schema)Input schema for 'batch-fetch-text' tool defining 'requests' as array of strings (URLs) or objects with url, method, data, headers, output_format.inputSchema={ "type": "object", "properties": { "requests": { "type": "array", "description": "Array of URLs (strings) or request objects", "items": { "oneOf": [ {"type": "string"}, { "type": "object", "properties": { "url": { "type": "string", "description": "The URL to get text content from", }, "method": { "type": "string", "description": "HTTP method to use (GET, POST, PUT, DELETE, PATCH, etc.). Default is GET.", "default": "GET" }, "data": { "type": ["object", "string", "null"], "description": "Request body data for POST/PUT/PATCH requests. Can be a JSON object or string.", }, "headers": { "type": "object", "description": "Additional HTTP headers to include in the request", "additionalProperties": {"type": "string"} }, "output_format": { "type": "string", "description": "Output format: 'markdown' (default), 'clean_text', or 'raw_html'.", "enum": ["markdown", "clean_text", "raw_html"], "default": "markdown" } }, "required": ["url"] } ] }, }, }, "required": ["requests"], },
- src/jsonrpc_mcp/server.py:145-196 (registration)Registration of 'batch-fetch-text' tool in @server.list_tools() with name, description, and inputSchema.types.Tool( name="batch-fetch-text", description=( "Batch fetch raw text content from multiple URLs using various HTTP methods. " "Executes requests concurrently for better performance." ), inputSchema={ "type": "object", "properties": { "requests": { "type": "array", "description": "Array of URLs (strings) or request objects", "items": { "oneOf": [ {"type": "string"}, { "type": "object", "properties": { "url": { "type": "string", "description": "The URL to get text content from", }, "method": { "type": "string", "description": "HTTP method to use (GET, POST, PUT, DELETE, PATCH, etc.). Default is GET.", "default": "GET" }, "data": { "type": ["object", "string", "null"], "description": "Request body data for POST/PUT/PATCH requests. Can be a JSON object or string.", }, "headers": { "type": "object", "description": "Additional HTTP headers to include in the request", "additionalProperties": {"type": "string"} }, "output_format": { "type": "string", "description": "Output format: 'markdown' (default), 'clean_text', or 'raw_html'.", "enum": ["markdown", "clean_text", "raw_html"], "default": "markdown" } }, "required": ["url"] } ] }, }, }, "required": ["requests"], }, ),
- src/jsonrpc_mcp/utils.py:345-383 (helper)Core helper function implementing concurrent batch fetching of text contents from URLs or request objects, handling errors and returning list of results with success/content/error.async def batch_fetch_urls(requests: list[str | dict[str, Any]], as_json: bool = True, output_format: str = "markdown") -> list[dict[str, Any]]: """ Batch fetch content from multiple URLs concurrently. Args: requests: List of URLs (strings) or request objects with url, method, data, headers, output_format as_json: If True, validates content as JSON; if False, returns text content output_format: Default output format - "markdown", "clean_text", or "raw_html" (can be overridden per request) Returns: List of dictionaries with 'url', 'success', 'content', and optional 'error' keys """ async def fetch_single(request: str | dict[str, Any]) -> dict[str, Any]: try: if isinstance(request, str): # Simple URL string content = await fetch_url_content(request, as_json=as_json, output_format=output_format) return {"url": request, "success": True, "content": content} else: # Request object with additional parameters url = request.get("url", "") method = request.get("method", "GET") data = request.get("data") headers = request.get("headers") request_output_format = request.get("output_format", output_format) content = await fetch_url_content( url, as_json=as_json, method=method, data=data, headers=headers, output_format=request_output_format ) return {"url": url, "success": True, "content": content} except Exception as e: url = request if isinstance(request, str) else request.get("url", "") return {"url": url, "success": False, "error": str(e)} tasks = [fetch_single(request) for request in requests] results = await asyncio.gather(*tasks) return list(results)