MCP Deep Web Research Server

Instructions

Implementation Reference

• src/deep-research.ts:91-193 (handler)

  The primary handler function for the deep_research tool. It orchestrates the entire research process: generates queries, performs parallel searches, deduplicates results, processes URLs through ResearchSession, formats findings, and includes performance timing.
  public async startResearch(topic: string, options: DeepResearchOptions = {}): Promise<ResearchResult> { const startTime = Date.now(); const timings: { [key: string]: number } = {}; console.log('[Performance] Starting research for topic:', topic); console.log('[Performance] Options:', options); // Create new research session const session = new ResearchSession(topic, { maxDepth: options.maxDepth, maxBranching: options.maxBranching, timeout: options.timeout, minRelevanceScore: options.minRelevanceScore, maxParallelOperations: options.maxParallelOperations }); console.log('[Performance] Created research session:', session.id); this.activeSessions.set(session.id, session); try { console.log('[Performance] Starting parallel search...'); const parallelSearchStart = Date.now(); const queries = [ topic, `${topic} tutorial`, `${topic} guide`, `${topic} example`, `${topic} implementation`, `${topic} code`, `${topic} design pattern`, `${topic} best practice` ]; console.log('[Performance] Search queries:', queries); const searchResults = await this.parallelSearch.parallelSearch(queries); timings.parallelSearch = Date.now() - parallelSearchStart; console.log('[Performance] Parallel search complete. Duration:', timings.parallelSearch, 'ms'); const deduplicationStart = Date.now(); const allResults = searchResults.results.flatMap(result => result.results); console.log('[Performance] Total results:', allResults.length); const uniqueResults = this.deduplicateResults(allResults); console.log('[Performance] Unique results:', uniqueResults.length); const sortedResults = uniqueResults.sort((a, b) => b.relevanceScore - a.relevanceScore); timings.deduplication = Date.now() - deduplicationStart; console.log('[Performance] Deduplication complete. Duration:', timings.deduplication, 'ms'); // Process top results first console.log('[Performance] Processing top 5 results...'); const topProcessingStart = Date.now(); const topResults = sortedResults.slice(0, 5); await Promise.all(topResults.map(r => { console.log('[Performance] Processing URL:', r.url); return session.processUrl(r.url); })); timings.topResultsProcessing = Date.now() - topProcessingStart; console.log('[Performance] Top results processing complete. Duration:', timings.topResultsProcessing, 'ms'); // Process remaining results console.log('[Performance] Processing remaining results...'); const remainingProcessingStart = Date.now(); const remainingResults = sortedResults.slice(5); await Promise.all(remainingResults.map(r => { console.log('[Performance] Processing URL:', r.url); return session.processUrl(r.url); })); timings.remainingResultsProcessing = Date.now() - remainingProcessingStart; console.log('[Performance] Remaining results processing complete. Duration:', timings.remainingResultsProcessing, 'ms'); // Complete the session console.log('[Performance] Completing session...'); await session.complete(); // Format and return results console.log('[Performance] Formatting results...'); const results = this.formatResults(session); // Add timing information timings.total = Date.now() - startTime; results.timing.operations = { parallelSearch: timings.parallelSearch, deduplication: timings.deduplication, topResultsProcessing: timings.topResultsProcessing, remainingResultsProcessing: timings.remainingResultsProcessing, total: timings.total }; console.log('[Performance] Research complete. Total duration:', timings.total, 'ms'); console.log('[Performance] Operation timings:', timings); return results; } catch (error) { console.error(`[Performance] Error in research session ${session.id}:`, error); throw error; } finally { // Cleanup this.activeSessions.delete(session.id); await this.parallelSearch.cleanup(); } }
• src/index.ts:87-124 (registration)

  Registration of the 'deep_research' tool in the MCP server's listTools handler, including name, description, and detailed input schema.
  { name: 'deep_research', description: 'Perform deep research on a topic with content extraction and analysis', inputSchema: { type: 'object', properties: { topic: { type: 'string', description: 'Research topic or question' }, maxDepth: { type: 'number', description: 'Maximum depth of related content exploration', minimum: 1, maximum: 2 }, maxBranching: { type: 'number', description: 'Maximum number of related paths to explore', minimum: 1, maximum: 3 }, timeout: { type: 'number', description: 'Research timeout in milliseconds', minimum: 30000, maximum: 55000 }, minRelevanceScore: { type: 'number', description: 'Minimum relevance score for including content', minimum: 0, maximum: 1 } }, required: ['topic'] } },
• src/index.ts:15-21 (schema)

  TypeScript interface defining the input arguments for the deep_research tool, used in the tool handler.
  interface DeepResearchArgs { topic: string; maxDepth?: number; maxBranching?: number; timeout?: number; minRelevanceScore?: number; }
• src/index.ts:293-315 (handler)

  MCP tool dispatcher case for 'deep_research': validates input, calls the DeepResearch instance's startResearch method, and returns formatted results.
  case 'deep_research': { const args = request.params.arguments as unknown as DeepResearchArgs; if (!args?.topic) { throw new McpError(ErrorCode.InvalidParams, 'Topic is required'); } console.log(`Starting deep research on topic: ${args.topic}`); const result = await deepResearch.startResearch(args.topic, { maxDepth: Math.min(args.maxDepth || 2, 2), maxBranching: Math.min(args.maxBranching || 3, 3), timeout: Math.min(args.timeout || 55000, 55000), minRelevanceScore: args.minRelevanceScore || 0.7 }); return { content: [ { type: 'text', text: JSON.stringify(result, null, 2) } ] }; }
• src/deep-research.ts:14-51 (schema)

  Output type definition for the research results returned by the deep_research handler.
  export interface ResearchResult { sessionId: string; topic: string; findings: { mainTopics: Array<{ name: string; importance: number; relatedTopics: string[]; }>; keyInsights: Array<{ text: string; confidence: number; relatedTopics: string[]; }>; sources: Array<{ url: string; title: string; credibilityScore: number; }>; }; progress: { completedSteps: number; totalSteps: number; processedUrls: number; }; timing: { started: string; completed?: string; duration?: number; operations?: { parallelSearch?: number; deduplication?: number; topResultsProcessing?: number; remainingResultsProcessing?: number; total?: number; }; }; }