download_website
Use wget to download an entire website, preserving its structure and converting links for local use. Specify URL, output path (optional), and depth level for recursive downloads.
Instructions
Download an entire website using wget
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
| depth | No | Maximum depth level for recursive downloading (optional, defaults to infinite) | |
| outputPath | No | Path where the website should be downloaded (optional, defaults to current directory) | |
| url | Yes | URL of the website to download |
Implementation Reference
- src/index.ts:83-152 (handler)The tool handler registered for CallToolRequestSchema. It validates arguments, checks for wget, constructs and executes a wget command to download the website recursively with specified options, and returns success/error content.this.server.setRequestHandler(CallToolRequestSchema, async (request) => { if (request.params.name !== 'download_website') { throw new McpError( ErrorCode.MethodNotFound, `Unknown tool: ${request.params.name}` ); } if (!isValidDownloadArgs(request.params.arguments)) { throw new McpError( ErrorCode.InvalidParams, 'Invalid download arguments' ); } const { url, outputPath = process.cwd(), depth } = request.params.arguments; try { // Check if wget is installed await execAsync('which wget'); } catch (error: any) { return { content: [ { type: 'text', text: `Error downloading website: ${error.message || 'Unknown error'}` }, ], isError: true, }; } try { // Create wget command with options for downloading website const wgetCommand = [ 'wget', '--recursive', // Download recursively `--level=${depth !== undefined ? depth : 'inf'}`, // Recursion depth (infinite if not specified) '--page-requisites', // Get all assets needed to display the page '--convert-links', // Convert links to work locally '--adjust-extension', // Add appropriate extensions to files '--span-hosts', // Include necessary resources from other hosts '--domains=' + new URL(url).hostname, // Restrict to same domain '--no-parent', // Don't follow links to parent directory '--directory-prefix=' + outputPath, // Output directory url ].join(' '); const { stdout, stderr } = await execAsync(wgetCommand); return { content: [ { type: 'text', text: `Website downloaded successfully to ${outputPath}\n\nOutput:\n${stdout}\n${stderr}`, }, ], }; } catch (error: any) { return { content: [ { type: 'text', text: `Error downloading website: ${error.message || 'Unknown error'}`, }, ], isError: true, }; } });
- src/index.ts:61-78 (schema)Input schema definition for the download_website tool, defining parameters url (required), outputPath, and depth.type: 'object', properties: { url: { type: 'string', description: 'URL of the website to download', }, outputPath: { type: 'string', description: 'Path where the website should be downloaded (optional, defaults to current directory)', }, depth: { type: 'number', description: 'Maximum depth level for recursive downloading (optional, defaults to infinite)', minimum: 0 } }, required: ['url'], },
- src/index.ts:55-81 (registration)Registration of the download_website tool in the ListToolsRequestSchema handler, including name, description, and schema.this.server.setRequestHandler(ListToolsRequestSchema, async () => ({ tools: [ { name: 'download_website', description: 'Download an entire website using wget', inputSchema: { type: 'object', properties: { url: { type: 'string', description: 'URL of the website to download', }, outputPath: { type: 'string', description: 'Path where the website should be downloaded (optional, defaults to current directory)', }, depth: { type: 'number', description: 'Maximum depth level for recursive downloading (optional, defaults to infinite)', minimum: 0 } }, required: ['url'], }, }, ], }));
- src/index.ts:22-27 (helper)Helper function to validate the arguments for download_website tool.const isValidDownloadArgs = (args: any): args is DownloadWebsiteArgs => typeof args === 'object' && args !== null && typeof args.url === 'string' && (args.outputPath === undefined || typeof args.outputPath === 'string') && (args.depth === undefined || (typeof args.depth === 'number' && args.depth >= 0));
- src/index.ts:16-20 (schema)TypeScript interface defining the expected arguments for the download_website tool.interface DownloadWebsiteArgs { url: string; outputPath?: string; depth?: number; }