run_queue
Process and index queued documentation URLs to make them searchable through vector search, enabling AI assistants to retrieve relevant context.
Instructions
Process and index all URLs currently in the documentation queue. Each URL is processed sequentially, with proper error handling and retry logic. Progress updates are provided as processing occurs. Use this after adding new URLs to ensure all documentation is indexed and searchable. Long-running operations will process until the queue is empty or an unrecoverable error occurs.
Input Schema
TableJSON Schema
| Name | Required | Description | Default |
|---|---|---|---|
No arguments | |||
Implementation Reference
- src/handlers/run-queue.ts:15-93 (handler)RunQueueHandler class that implements the core logic for the 'run_queue' tool. It processes URLs from a queue file using the AddDocumentationHandler and provides status on processed and failed items.export class RunQueueHandler extends BaseHandler { private addDocHandler: AddDocumentationHandler; constructor(server: Server, apiClient: ApiClient) { super(server, apiClient); this.addDocHandler = new AddDocumentationHandler(server, apiClient); } async handle(_args: any): Promise<McpToolResponse> { try { // Check if queue file exists try { await fs.access(QUEUE_FILE); } catch { return { content: [ { type: 'text', text: 'Queue is empty (queue file does not exist)', }, ], }; } let processedCount = 0; let failedCount = 0; const failedUrls: string[] = []; while (true) { // Read current queue const content = await fs.readFile(QUEUE_FILE, 'utf-8'); const urls = content.split('\n').filter(url => url.trim() !== ''); if (urls.length === 0) { break; // Queue is empty } const currentUrl = urls[0]; // Get first URL try { // Process the URL using add_documentation handler await this.addDocHandler.handle({ url: currentUrl }); processedCount++; } catch (error) { failedCount++; failedUrls.push(currentUrl); console.error(`Failed to process URL ${currentUrl}:`, error); } // Remove the processed URL from queue const remainingUrls = urls.slice(1); await fs.writeFile(QUEUE_FILE, remainingUrls.join('\n') + (remainingUrls.length > 0 ? '\n' : '')); } let resultText = `Queue processing complete.\nProcessed: ${processedCount} URLs\nFailed: ${failedCount} URLs`; if (failedUrls.length > 0) { resultText += `\n\nFailed URLs:\n${failedUrls.join('\n')}`; } return { content: [ { type: 'text', text: resultText, }, ], }; } catch (error) { return { content: [ { type: 'text', text: `Failed to process queue: ${error}`, }, ], isError: true, }; } }
- src/handler-registry.ts:122-129 (schema)Tool schema definition for 'run_queue' with name, detailed description, and empty input schema (no parameters required).{ name: 'run_queue', description: 'Process and index all URLs currently in the documentation queue. Each URL is processed sequentially, with proper error handling and retry logic. Progress updates are provided as processing occurs. Use this after adding new URLs to ensure all documentation is indexed and searchable. Long-running operations will process until the queue is empty or an unrecoverable error occurs.', inputSchema: { type: 'object', properties: {}, }, } as ToolDefinition,
- src/handler-registry.ts:43-43 (registration)Registration of the RunQueueHandler instance in the handlers Map under the key 'run_queue'.this.handlers.set('run_queue', new RunQueueHandler(this.server, this.apiClient));