fetch_as_markdown
Convert website HTML content to Markdown for easier use in documentation or workflows. Simply provide the URL and optional headers to fetch and transform the content.
Instructions
Fetch a website, convert its HTML content to Markdown, and return it.
Args:
url (str): URL of the website to fetch.
headers (Optional[dict[str, str]]): Custom headers for the request.
Returns:
FetchResponse: An object containing the Markdown content or an error message.
On success, isError is false and content contains the Markdown text.
On failure, isError is true and errorMessage contains the error details.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| headers | No | ||
| url | Yes |
Implementation Reference
- jij_mcp/mcp_setting.py:235-253 (handler)Handler function for the 'fetch_as_markdown' tool. Registered via @mcp.tool(). Wraps Fetcher.markdown with FetchRequestArgs.@mcp.tool() async def fetch_as_markdown( url: str, headers: typ.Optional[dict[str, str]] = None ) -> FetchResponse: """ Fetch a website, convert its HTML content to Markdown, and return it. Args: url (str): URL of the website to fetch. headers (Optional[dict[str, str]]): Custom headers for the request. Returns: FetchResponse: An object containing the Markdown content or an error message. On success, isError is false and content contains the Markdown text. On failure, isError is true and errorMessage contains the error details. """ args = FetchRequestArgs(url=url, headers=headers) return await Fetcher.markdown(args)
- jij_mcp/fetch/types.py:5-11 (schema)Pydantic input schema defining url (HttpUrl) and optional headers for the fetch_as_markdown tool.class FetchRequestArgs(BaseModel): """Input arguments schema for fetch tools.""" url: HttpUrl = Field(..., description="URL of the content to fetch.") headers: Optional[dict[str, str]] = Field( default=None, description="Optional headers to include in the request." )
- jij_mcp/fetch/types.py:14-19 (schema)Pydantic output schema for FetchResponse used by fetch_as_markdown.# エラーを含む可能性のあるレスポンス型 class FetchResponse(BaseModel): content: list[dict[str, str]] # MCP標準のcontent形式に合わせる isError: bool = False errorMessage: Optional[str] = None
- jij_mcp/fetch/fetcher.py:125-144 (helper)Core implementation of Markdown conversion in Fetcher.markdown: fetches HTML via httpx, decodes, converts to MD using NoImagesConverter.async def markdown(payload: FetchRequestArgs) -> FetchResponse: """Fetches content and converts it to Markdown.""" try: response = await Fetcher._fetch(payload) html_content = await response.aread() # Decode carefully before passing to markdownify try: html_text = html_content.decode("utf-8") except UnicodeDecodeError: detected_encoding = response.encoding or "iso-8859-1" html_text = html_content.decode(detected_encoding, errors="replace") # Use custom NoImagesConverter to ignore images converter = NoImagesConverter() md = converter.convert(html_text) return FetchResponse(content=[{"type": "text", "text": md}], isError=False) except Exception as e: return FetchResponse(content=[], isError=True, errorMessage=str(e))
- jij_mcp/fetch/fetcher.py:10-18 (helper)Custom MarkdownConverter subclass that removes images by returning empty string in convert_img.class NoImagesConverter(MarkdownConverter): """ Create a custom MarkdownConverter that ignores all images during conversion """ def convert_img(self, el, text, parent_tags): # Return empty string instead of converting the image return ""