run_queue
Process and index queued documentation URLs to make them searchable through vector search. Handles URLs sequentially with error handling and progress updates.
Instructions
Process and index all URLs currently in the documentation queue. Each URL is processed sequentially, with proper error handling and retry logic. Progress updates are provided as processing occurs. Use this after adding new URLs to ensure all documentation is indexed and searchable. Long-running operations will process until the queue is empty or an unrecoverable error occurs.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
No arguments | |||
Implementation Reference
- src/handlers/run-queue.ts:23-93 (handler)Core handler logic for the 'run_queue' tool. Reads URLs from queue.txt sequentially, processes each with AddDocumentationHandler, tracks success/failure, removes processed URLs, and returns summary.
async handle(_args: any): Promise<McpToolResponse> { try { // Check if queue file exists try { await fs.access(QUEUE_FILE); } catch { return { content: [ { type: 'text', text: 'Queue is empty (queue file does not exist)', }, ], }; } let processedCount = 0; let failedCount = 0; const failedUrls: string[] = []; while (true) { // Read current queue const content = await fs.readFile(QUEUE_FILE, 'utf-8'); const urls = content.split('\n').filter(url => url.trim() !== ''); if (urls.length === 0) { break; // Queue is empty } const currentUrl = urls[0]; // Get first URL try { // Process the URL using add_documentation handler await this.addDocHandler.handle({ url: currentUrl }); processedCount++; } catch (error) { failedCount++; failedUrls.push(currentUrl); console.error(`Failed to process URL ${currentUrl}:`, error); } // Remove the processed URL from queue const remainingUrls = urls.slice(1); await fs.writeFile(QUEUE_FILE, remainingUrls.join('\n') + (remainingUrls.length > 0 ? '\n' : '')); } let resultText = `Queue processing complete.\nProcessed: ${processedCount} URLs\nFailed: ${failedCount} URLs`; if (failedUrls.length > 0) { resultText += `\n\nFailed URLs:\n${failedUrls.join('\n')}`; } return { content: [ { type: 'text', text: resultText, }, ], }; } catch (error) { return { content: [ { type: 'text', text: `Failed to process queue: ${error}`, }, ], isError: true, }; } } - src/handler-registry.ts:122-129 (schema)MCP tool schema definition for 'run_queue' including name, detailed description, and empty input schema (no parameters required).
{ name: 'run_queue', description: 'Process and index all URLs currently in the documentation queue. Each URL is processed sequentially, with proper error handling and retry logic. Progress updates are provided as processing occurs. Use this after adding new URLs to ensure all documentation is indexed and searchable. Long-running operations will process until the queue is empty or an unrecoverable error occurs.', inputSchema: { type: 'object', properties: {}, }, } as ToolDefinition, - src/handler-registry.ts:43-43 (registration)Registration of the RunQueueHandler instance in the HandlerRegistry's handlers Map under the key 'run_queue'.
this.handlers.set('run_queue', new RunQueueHandler(this.server, this.apiClient)); - src/handler-registry.ts:17-17 (registration)Import declaration for RunQueueHandler used in registration.
RunQueueHandler, - src/handlers/run-queue.ts:11-14 (helper)Helper constant defining the path to the queue.txt file used for storing pending URLs.
const __filename = fileURLToPath(import.meta.url); const __dirname = path.dirname(__filename); const QUEUE_FILE = path.join(__dirname, '..', '..', 'queue.txt');