KnowledgeMCP

# Feature Specification: MCP Knowledge Server **Feature Branch**: `001-mcp-knowledge-server` **Created**: 2025-10-25 **Status**: Draft **Input**: User description: "Build an MCP server that can help me use local knowledge with AI code assistants and with other agentic tools. Underneath local knowledge should be vector database that supports semantic search. Server should support knowledge-add, knowledge-search, knowledge-remove, knowledge-show, knowledge-clear, knowledge-status. Server should exposed through Streamable HTTP transport. Both documents: PDF, DOCX, PPTX, XLSX, HTML and images: JPG, SVG, PNG should be supported by knowledge-add. Documents with Images should also be supported. Server should be able to decided whether to use OCR or text processing before performing embedding." ## User Scenarios & Testing *(mandatory)* ### User Story 1 - Add Knowledge from Documents (Priority: P1) A developer wants to add technical documentation, code references, or project specifications to their local knowledge base so AI assistants can reference this information during coding sessions. **Why this priority**: This is the foundation of the entire system. Without the ability to add knowledge, no other features can function. This delivers immediate value by allowing users to build their knowledge base. **Independent Test**: Can be fully tested by adding a single PDF document via the knowledge-add command and verifying it's stored in the system using knowledge-status. **Acceptance Scenarios**: 1. **Given** an empty knowledge base, **When** user adds a PDF document via knowledge-add, **Then** the document is processed, embedded, and stored successfully 2. **Given** a knowledge base with existing documents, **When** user adds a DOCX file with images, **Then** both text and image content are extracted and stored 3. **Given** a scanned PDF with no text layer, **When** user adds it via knowledge-add, **Then** OCR is automatically applied and text is extracted 4. **Given** a PowerPoint presentation, **When** user adds it via knowledge-add, **Then** slide content and embedded images are extracted 5. **Given** an Excel spreadsheet, **When** user adds it via knowledge-add, **Then** cell content and structure are preserved and indexed 6. **Given** an HTML file with styling, **When** user adds it via knowledge-add, **Then** text content is extracted while preserving semantic structure 7. **Given** image files (JPG, PNG, SVG), **When** user adds them via knowledge-add, **Then** visual content is analyzed and indexed for semantic search --- ### User Story 2 - Search Knowledge Semantically (Priority: P2) A developer using an AI coding assistant wants to search their local knowledge base using natural language queries to find relevant information without knowing exact keywords. **Why this priority**: Semantic search is the core value proposition that differentiates this from simple file storage. It enables AI assistants to find contextually relevant information. **Independent Test**: Can be fully tested by adding 3-5 documents with different topics, then performing semantic searches to verify relevant results are returned based on meaning rather than exact keyword matches. **Acceptance Scenarios**: 1. **Given** a knowledge base with multiple documents, **When** user searches with a natural language query, **Then** semantically relevant results are returned ranked by relevance 2. **Given** documents containing technical jargon, **When** user searches using plain language description, **Then** system finds matches based on conceptual similarity 3. **Given** a knowledge base with mixed content types, **When** user searches, **Then** results include relevant excerpts from documents and descriptions of relevant images 4. **Given** a search query, **When** no relevant content exists, **Then** system returns empty results with clear message 5. **Given** a broad search query, **When** many documents match, **Then** top N most relevant results are returned (default: top 10) --- ### User Story 3 - Manage Knowledge Base (Priority: P3) A developer wants to view, update, and remove documents from their knowledge base to keep it current and relevant. **Why this priority**: Management capabilities are important for long-term usability but aren't needed for initial value delivery. Users can start with just adding and searching. **Independent Test**: Can be fully tested by adding documents, viewing the knowledge base status, removing specific documents, and verifying they're no longer searchable. **Acceptance Scenarios**: 1. **Given** a knowledge base with documents, **When** user runs knowledge-show, **Then** a list of all stored documents with metadata is displayed 2. **Given** a specific document ID, **When** user runs knowledge-remove with that ID, **Then** the document is removed from the knowledge base 3. **Given** a knowledge base with multiple documents, **When** user runs knowledge-status, **Then** summary statistics are shown (document count, total size, storage usage) 4. **Given** a knowledge base with content, **When** user runs knowledge-clear, **Then** all documents are removed after confirmation 5. **Given** an empty knowledge base, **When** user runs knowledge-status, **Then** system reports zero documents with appropriate message --- ### User Story 4 - Integrate with AI Tools via MCP (Priority: P4) An AI coding assistant or agentic tool connects to the knowledge server via MCP protocol over HTTP transport to access knowledge during operation. **Why this priority**: While critical for the intended use case, this is built on top of the core functionality. Once P1-P3 are working, the MCP interface exposes existing capabilities. **Independent Test**: Can be fully tested by connecting an MCP client (like Claude Desktop or another AI tool) to the server and verifying it can call all knowledge operations through the MCP protocol. **Acceptance Scenarios**: 1. **Given** the knowledge server is running, **When** an MCP client connects via HTTP, **Then** the connection is established using streaming transport 2. **Given** an established MCP connection, **When** client calls knowledge-add tool, **Then** documents are added to the knowledge base 3. **Given** an established MCP connection, **When** client calls knowledge-search tool, **Then** semantic search results are returned 4. **Given** an established MCP connection, **When** client calls any knowledge management tool, **Then** operations complete and results are returned in MCP format 5. **Given** multiple concurrent MCP clients, **When** they perform operations simultaneously, **Then** all requests are handled without conflicts --- ### Edge Cases - What happens when adding a document that's already in the knowledge base (same file name or content)? - How does the system handle corrupted or password-protected documents? - What happens when adding an extremely large document (e.g., 1000+ page PDF)? - How does the system handle documents in non-English languages? - What happens when the vector database storage is full or reaches capacity? - How does the system handle invalid file formats or unsupported file types? - What happens when OCR fails on a low-quality scanned image? - How does the system handle documents with mixed text encodings? - What happens when removing a document that's currently being searched? - How does the system handle search queries that are too vague or too specific? ## Requirements *(mandatory)* ### Functional Requirements - **FR-001**: System MUST accept documents in PDF, DOCX, PPTX, XLSX, and HTML formats via knowledge-add - **FR-002**: System MUST accept images in JPG, PNG, and SVG formats via knowledge-add - **FR-003**: System MUST extract text content from document formats preserving semantic structure - **FR-004**: System MUST extract and process images embedded within documents - **FR-005**: System MUST automatically determine whether to use OCR or direct text extraction based on document characteristics - **FR-006**: System MUST apply OCR to scanned documents and images containing text - **FR-007**: System MUST generate vector embeddings for all extracted content - **FR-008**: System MUST store embeddings in a vector database supporting semantic similarity search - **FR-009**: System MUST support semantic search via knowledge-search accepting natural language queries - **FR-010**: System MUST return ranked search results based on semantic relevance - **FR-011**: System MUST support listing all documents via knowledge-show with metadata (name, size, date added) - **FR-012**: System MUST support removing individual documents via knowledge-remove using document identifier - **FR-013**: System MUST support removing all documents via knowledge-clear with confirmation - **FR-014**: System MUST support viewing knowledge base statistics via knowledge-status (document count, storage size) - **FR-015**: System MUST expose all operations through MCP (Model Context Protocol) interface - **FR-016**: System MUST use Streamable HTTP transport for MCP communication - **FR-017**: System MUST handle multiple concurrent MCP client connections - **FR-018**: System MUST provide clear error messages when operations fail - **FR-019**: System MUST validate file formats before processing - **FR-020**: System MUST track metadata for each document (original filename, format, date added, processing method used) - **FR-021**: System MUST support idempotent operations (adding same document multiple times should be handled gracefully) - **FR-022**: System MUST handle documents with mixed content (text, images, tables) - **FR-023**: System MUST preserve document context in embeddings (e.g., which document a chunk came from) - **FR-024**: System MUST support incremental knowledge base updates without full reindexing ### Key Entities - **Document**: Represents an added file with metadata (ID, filename, format, date added, size, processing method, embedding IDs) - **Embedding**: Vector representation of document content chunk with reference back to source document and position - **SearchResult**: A ranked result from semantic search containing content excerpt, relevance score, source document reference - **KnowledgeBase**: The collection of all documents and their embeddings with aggregate statistics - **ProcessingStrategy**: Decision logic for whether to use OCR vs text extraction for a given document ## Success Criteria *(mandatory)* ### Measurable Outcomes - **SC-001**: Users can add a 10-page PDF document to the knowledge base and have it searchable in under 30 seconds - **SC-002**: Semantic search returns relevant results in under 2 seconds for knowledge bases with up to 1000 documents - **SC-003**: System correctly extracts text from at least 95% of supported document types - **SC-004**: OCR accurately extracts text from scanned documents with at least 90% accuracy - **SC-005**: Search results include relevant content 90% of the time based on semantic meaning (not just keyword matching) - **SC-006**: AI assistants can successfully integrate and use the knowledge server without manual configuration beyond connection details ### Performance Criteria (per Constitution IV) - **API Response Time**: MCP tool calls respond within 500ms for p95 (excluding document processing time) - **Search Performance**: Semantic search queries complete in under 2 seconds for knowledge bases with 1000+ documents - **Document Processing**: Text extraction completes within 3 seconds per page for standard documents - **OCR Processing**: OCR completes within 10 seconds per page for scanned documents - **Concurrent Operations**: Support at least 10 concurrent MCP client connections without degradation - **Memory Efficiency**: Server operates within 2GB memory limit for knowledge bases up to 10,000 documents ### User Experience Criteria (per Constitution III) - **Error Handling**: All errors provide clear, actionable messages (e.g., "Unsupported file format: .xyz. Please use PDF, DOCX, PPTX, XLSX, HTML, JPG, PNG, or SVG") - **Progress Feedback**: Long-running operations (document processing, OCR) provide progress indicators - **Operation Confirmation**: Destructive operations (knowledge-clear, knowledge-remove) require explicit confirmation - **Status Visibility**: knowledge-status provides clear summary of knowledge base state at any time - **Graceful Degradation**: Partial failures (e.g., OCR fails on one page) don't prevent processing remaining content ## Assumptions 1. **Vector Database**: Assuming use of a standard vector database solution (e.g., ChromaDB, Faiss, Qdrant) that supports semantic similarity search 2. **Embedding Model**: Assuming use of a standard embedding model (e.g., sentence-transformers) for generating vector representations 3. **OCR Engine**: Assuming use of Tesseract OCR or similar open-source OCR engine 4. **Document Parsing**: Assuming use of standard libraries for document parsing (PyPDF2/pdfplumber for PDF, python-docx for DOCX, etc.) 5. **Image Processing**: Assuming use of PIL/Pillow for image processing and analysis 6. **MCP Implementation**: Assuming use of official MCP SDK for server implementation 7. **Storage**: Assuming local filesystem storage for documents and vector database 8. **Language Support**: Initially focusing on English language content; multi-language support may require additional configuration 9. **Chunk Size**: Assuming documents are chunked into reasonable sizes (e.g., 500-1000 tokens) for embedding generation 10. **Deduplication Strategy**: Assuming content-based deduplication (comparing embeddings or file hashes) to handle duplicate documents