OpenRouter Agents MCP Server

# QA – Output & Async Validation (v1.2.0) ## Async job flow - Submit: `POST /jobs {"query":"Async smoke test"}` → returns `job_id` - Stream: `/jobs/:jobId/events` shows `submitted → started → agent_* → synthesis_token → report_saved → complete` - UI: `/ui` displays lanes and tokens; `agent_usage` events present when provider returns usage - Metrics: `/metrics` includes `jobs` counts and `usageTotals` ## Tools sanity - `submit_research` → `get_job_status` shows events; `cancel_job` succeeds on queued jobs - `search` returns typed hits from docs & reports; hybrid ordering plausible - `query` blocks non-SELECT, `explain:true` returns concise summary ## Retrieval - `index_url` adds doc; `search { scope:"docs" }` finds it - BM25 k1/b configurable; vector fusion enabled when embeddings present ## Packaging - `npx openrouter-agents --stdio` works; MCP JSON configs spawn server # OpenRouterAI Research Agents MCP Server - QA Test Results This document contains the results of testing all features in the OpenRouterAI Research Agents MCP server. ## Table of Contents 1. [Test Environment](#test-environment) 2. [Tool: `conduct_research`](#tool-conduct_research) 3. [Tool: `research_follow_up`](#tool-research_follow_up) 4. [Tool: `get_past_research`](#tool-get_past_research) 5. [Tool: `rate_research_report`](#tool-rate_research_report) 6. [Tool: `list_research_history`](#tool-list_research_history) 7. [Performance Testing](#performance-testing) 8. [MECE Analysis & Remediation Plan](#mece-analysis--remediation-plan) ## Test Environment - Test Date: 3/29/2025, ~9:53 AM (America/Chicago, UTC-5:00) - MCP Server: openrouterai-research-agents ## Tool: `conduct_research` ### Test Case 1.1: Basic Query (Default Parameters) - **Arguments Used:** ```json { "query": "What is quantum computing?" } ``` - **Response:** ``` { "content": [ { "type": "text", "text": "Research complete. Results streamed. Report ID: report-1743260530247" } ] } ``` - **Time Taken:** 0ms - **Status:** Success - **Notes:** Generated report ID: report-1743260530247 ### Test Case 1.2: High-Cost Preference - **Arguments Used:** ```json { "query": "Compare supervised and unsupervised machine learning techniques", "costPreference": "high" } ``` - **Response:** ``` { "content": [ { "type": "text", "text": "Research complete. Results streamed. Report ID: report-1743260530760" } ] } ``` - **Time Taken:** 0ms - **Status:** Success - **Notes:** Generated report ID: report-1743260530760 ### Test Case 1.3: Audience Level and Output Format - **Arguments Used:** ```json { "query": "Explain blockchain technology", "audienceLevel": "beginner", "outputFormat": "bullet_points" } ``` - **Response:** ``` { "content": [ { "type": "text", "text": "Research complete. Results streamed. Report ID: report-1743260531265" } ] } ``` - **Time Taken:** 0ms - **Status:** Success - **Notes:** Generated report ID: report-1743260531265 ### Test Case 1.4: Without Sources - **Arguments Used:** ```json { "query": "History of artificial intelligence", "includeSources": false } ``` - **Response:** ``` { "content": [ { "type": "text", "text": "Research complete. Results streamed. Report ID: report-1743260531778" } ] } ``` - **Time Taken:** 1ms - **Status:** Success - **Notes:** Generated report ID: report-1743260531778 ### Test Case 1.5: Max Length Constraint - **Arguments Used:** ```json { "query": "Renewable energy technologies", "maxLength": 1000 } ``` - **Response:** ``` { "content": [ { "type": "text", "text": "Research complete. Results streamed. Report ID: report-1743260532295" } ] } ``` - **Time Taken:** 1ms - **Status:** Success - **Notes:** Generated report ID: report-1743260532295 ## Tool: `research_follow_up` ### Test Case 2.1: Related Follow-Up Question - **Arguments Used:** ```json { "originalQuery": "What is quantum computing?", "followUpQuestion": "What are the potential applications of quantum computing in cryptography?" } ``` - **Response:** ``` { "content": [ { "type": "text", "text": "Research complete. Results streamed. Report ID: followup-report-1743260532811" } ] } ``` - **Time Taken:** 0ms - **Status:** Success - **Notes:** Generated followup report ID: followup-report-1743260532811 ### Test Case 2.2: Unrelated Follow-Up Question - **Arguments Used:** ```json { "originalQuery": "What is quantum computing?", "followUpQuestion": "What is the current weather on Mars?" } ``` - **Response:** ``` { "content": [ { "type": "text", "text": "Research complete. Results streamed. Report ID: followup-report-1743260533324" } ] } ``` - **Time Taken:** 0ms - **Status:** Success - **Notes:** Generated followup report ID: followup-report-1743260533324 ### Test Case 2.3: High Cost Follow-Up - **Arguments Used:** ```json { "originalQuery": "What is quantum computing?", "followUpQuestion": "How does proof of stake differ from proof of work?", "costPreference": "high" } ``` - **Response:** ``` { "content": [ { "type": "text", "text": "Research complete. Results streamed. Report ID: followup-report-1743260533844" } ] } ``` - **Time Taken:** 0ms - **Status:** Success - **Notes:** Generated followup report ID: followup-report-1743260533844 ## Tool: `get_past_research` ### Test Case 5.1: Default Parameters - **Arguments Used:** ```json {} ``` - **Response:** ``` { "content": [ { "type": "text", "text": "[\n {\n \"_id\": \"report-1743260530247\",\n \"originalQuery\": \"What is quantum computing?\",\n \"createdAt\": \"2025-03-29T15:02:17.964Z\",\n \"parameters\": {\n \"costPreference\": \"low\",\n \"audienceLevel\": \"intermediate\"\n }\n }\n]" } ] } ``` - **Time Taken:** 1ms - **Status:** Success - **Notes:** ### Test Case 5.3: Query Filter - **Arguments Used:** ```json { "queryFilter": "quantum" } ``` - **Response:** ``` { "content": [ { "type": "text", "text": "[\n {\n \"_id\": \"report-1743260530247\",\n \"originalQuery\": \"What is quantum computing?\",\n \"createdAt\": \"2025-03-29T15:02:18.988Z\",\n \"parameters\": {\n \"costPreference\": \"low\",\n \"audienceLevel\": \"intermediate\"\n }\n }\n]" } ] } ``` - **Time Taken:** 1ms - **Status:** Success - **Notes:** ### Test Case 5.4: Non-Matching Filter - **Arguments Used:** ```json { "queryFilter": "nonexistentkeyword123456789" } ``` - **Response:** ``` { "content": [ { "type": "text", "text": "No recent research reports found matching filter \"nonexistentkeyword123456789\"." } ] } ``` - **Time Taken:** 0ms - **Status:** Success - **Notes:** ## Performance Testing ### Test Case 6.1: Complex Query with Low Cost - **Arguments Used:** ```json { "query": "Provide a comprehensive analysis of the ethical implications of artificial intelligence in healthcare, including issues related to patient privacy, algorithmic bias in diagnosis, responsibility for AI-driven medical errors, and the changing role of healthcare professionals in an increasingly automated environment.", "costPreference": "low" } ``` - **Response:** ``` { "content": [ { "type": "text", "text": "Error conducting research: Simulated error for testing." } ], "isError": true } ``` - **Time Taken:** 0ms - **Status:** Error - **Notes:** Simulated error for testing ### Test Case 6.2: Complex Query with High Cost - **Arguments Used:** ```json { "query": "Provide a comprehensive analysis of the ethical implications of artificial intelligence in healthcare, including issues related to patient privacy, algorithmic bias in diagnosis, responsibility for AI-driven medical errors, and the changing role of healthcare professionals in an increasingly automated environment.", "costPreference": "high" } ``` - **Response:** ``` { "content": [ { "type": "text", "text": "Error conducting research: Simulated error for testing." } ], "isError": true } ``` - **Time Taken:** 0ms - **Status:** Error - **Notes:** Simulated error for testing ## MECE Analysis & Remediation Plan [Analysis and Remediation Plan will be added after completing all tests]