webpage-to-markdown

Convert webpages to Markdown format for easier reading and sharing. Provide a URL to transform HTML content into structured Markdown text.

Instructions

Convert a webpage to markdown

Input Schema

TableJSON Schema

Name	Required	Description	Default
`url`	Yes	URL of the webpage to convert

Implementation Reference

src/server.ts:49-70 (handler)

Handler logic for webpage-to-markdown tool (shared with similar URL-based tools), validates the URL, checks protocol and private IP, then delegates to Markdownify.toMarkdown.

case tools.YouTubeToMarkdownTool.name:
case tools.BingSearchResultToMarkdownTool.name:
case tools.WebpageToMarkdownTool.name:
  if (!validatedArgs.url) {
    throw new Error("URL is required for this tool");
  }     
  
  const parsedUrl = new URL(validatedArgs.url);
  if (!["http:", "https:"].includes(parsedUrl.protocol)) {
    throw new Error("Only http: and https: schemes are allowed.");
  }
  
  if (is_ip_private(parsedUrl.hostname)) {
    throw new Error(`Fetching ${validatedArgs.url} is potentially dangerous, aborting.`);
  }

  result = await Markdownify.toMarkdown({
    url: validatedArgs.url,
    projectRoot: validatedArgs.projectRoot,
    uvPath: validatedArgs.uvPath || process.env.UV_PATH,
  });
  break;

src/tools.ts:49-62 (schema)

ToolSchema definition for the 'webpage-to-markdown' tool, defining input as a URL string.

export const WebpageToMarkdownTool = ToolSchema.parse({
  name: "webpage-to-markdown",
  description: "Convert a webpage to markdown",
  inputSchema: {
    type: "object",
    properties: {
      url: {
        type: "string",
        description: "URL of the webpage to convert",
      },
    },
    required: ["url"],
  },
});

src/server.ts:33-37 (registration)
Tool registration via the ListToolsRequestSchema handler, which returns all tools from the imported tools module including 'webpage-to-markdown'.
```
server.setRequestHandler(ListToolsRequestSchema, async () => {
  return {
    tools: Object.values(tools),
  };
});
```

src/Markdownify.ts:74-124 (helper)

Core helper function Markdownify.toMarkdown that handles URL fetching, temporary file creation, execution of markitdown tool via uv, and returns markdown result. This is invoked by the tool handler for webpage-to-markdown.

static async toMarkdown({
  filePath,
  url,
  projectRoot = path.resolve(__dirname, ".."),
  uvPath = "~/.local/bin/uv",
}: {
  filePath?: string;
  url?: string;
  projectRoot?: string;
  uvPath?: string;
}): Promise<MarkdownResult> {
  try {
    let inputPath: string;
    let isTemporary = false;

    if (url) {
      const response = await fetch(url);

      let extension = null;

      if (url.endsWith(".pdf")) {
        extension = "pdf";
      }

      const arrayBuffer = await response.arrayBuffer();
      const content = Buffer.from(arrayBuffer);

      inputPath = await this.saveToTempFile(content, extension);
      isTemporary = true;
    } else if (filePath) {
      inputPath = filePath;
    } else {
      throw new Error("Either filePath or url must be provided");
    }

    const text = await this._markitdown(inputPath, projectRoot, uvPath);
    const outputPath = await this.saveToTempFile(text);

    if (isTemporary) {
      fs.unlinkSync(inputPath);
    }

    return { path: outputPath, text };
  } catch (e: unknown) {
    if (e instanceof Error) {
      throw new Error(`Error processing to Markdown: ${e.message}`);
    } else {
      throw new Error("Error processing to Markdown: Unknown error occurred");
    }
  }
}

Markdownify MCP Server