Magento 2 GraphQL Documentation MCP Server

# CLAUDE.md This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. ## Project Overview This is an MCP (Model Context Protocol) server that provides tools to search and retrieve Magento 2 GraphQL API documentation. It parses 350+ local markdown files with YAML frontmatter, extracts GraphQL schema elements, indexes content in SQLite with FTS5, and exposes 8 tools via FastMCP over STDIO. ## Documentation Source ### Required: Adobe Commerce GraphQL Documentation The MCP server requires access to the official Adobe Commerce GraphQL documentation markdown files from the [AdobeDocs/commerce-webapi](https://github.com/AdobeDocs/commerce-webapi) repository. **Clone the documentation repository:** ```bash git clone https://github.com/AdobeDocs/commerce-webapi.git ``` The GraphQL documentation is located at: `commerce-webapi/src/pages/graphql/` **This directory contains:** - 350+ markdown files with YAML frontmatter - Structured in categories: schema/, tutorials/, develop/, usage/, payment-methods/ - Total code examples: 963 blocks (GraphQL, JSON, PHP, Bash) - GraphQL schema elements: 51 (queries, mutations, types, interfaces) ### Configuration Options The server looks for documentation in this priority order (implemented in `config.py:get_docs_path()`): 1. **Environment Variable** (highest priority) ```bash export MAGENTO_GRAPHQL_DOCS_PATH="/path/to/commerce-webapi/src/pages/graphql" ``` - Validated on startup - will error if path doesn't exist - Use absolute paths for best results 2. **Symlink in Project** (`./data/` directory - recommended for development) ```bash cd magento-graphql-docs-mcp ln -s /path/to/commerce-webapi/src/pages/graphql data ``` - Checks if `data/` exists and contains `.md` files - Can be symlink or actual directory with markdown files - Resolved to absolute path automatically 3. **Sibling Directory** (automatic detection) ``` projects/ ├── magento-graphql-docs-mcp/ (this server) └── commerce-webapi/ └── src/pages/graphql/ (documentation) ``` - Automatically detected if commerce-webapi is cloned as sibling **Important**: All paths are validated on startup. If no valid documentation is found, the server will fail with a helpful error message explaining the three setup options. ### Verification Before running the server, verify documentation access: ```bash # Check environment variable (if set) echo $MAGENTO_GRAPHQL_DOCS_PATH ls -la $MAGENTO_GRAPHQL_DOCS_PATH # Check symlink (if used) ls -la data/ # Expected structure: # - index.md (GraphQL overview) # - release-notes.md # - schema/ (285 files - queries, mutations, types) # - tutorials/ (13 files - checkout, orders, etc.) # - develop/ (8 files - extending GraphQL) # - usage/ (10 files - using GraphQL API) # - payment-methods/ (11 files) ``` ### Database Location The SQLite database is stored at `~/.mcp/magento-graphql-docs/database.db` by default. Override with: ```bash export MAGENTO_GRAPHQL_DOCS_DB_PATH="/custom/path/database.db" ``` The database is automatically created on first run and only re-indexed when markdown files change (based on mtime). ## Core Architecture ### Three-Layer System 1. **Parser Layer** (`magento_graphql_docs_mcp/parser.py`) - Parses markdown files with YAML frontmatter using `python-frontmatter` - Extracts document metadata (title, description, keywords from frontmatter) - Extracts categories from file paths (e.g., "schema/products/queries" → category="schema", subcategory="products") - Extracts markdown structure (headers, code blocks) - Detects GraphQL elements in code blocks (queries, mutations, types, interfaces) - Builds searchable text combining all available text - Uses Pydantic models: `Document`, `CodeBlock`, `GraphQLElement` 2. **Ingestion Layer** (`magento_graphql_docs_mcp/ingest.py`) - File modification time tracking (only re-parse if files change) - Creates SQLite schema with 4 tables: documents, code_blocks, graphql_elements, metadata - FTS5 indexes on: documents.searchable_text and graphql_elements.searchable_text (trigram tokenization) - Bulk inserts for performance (350 documents, 963 code blocks, 51 GraphQL elements) - Clears existing data before re-ingestion (not incremental) 3. **Server Layer** (`magento_graphql_docs_mcp/server.py`) - FastMCP server with lifespan manager that triggers ingestion on startup - Error handling in lifespan: catches FileNotFoundError with helpful setup instructions - Exposes 8 tools over STDIO - Returns formatted markdown responses - Returns top K results for searches (configurable via MAGENTO_GRAPHQL_DOCS_TOP_K env var, default: 5) - Uses configurable constants for code preview lengths and field limits ### MCP Tools (8 Total) 1. **search_documentation**: FTS search on documents with category/subcategory/content_type filters 2. **get_document**: Direct lookup by file_path, returns full markdown content 3. **search_graphql_elements**: FTS search on graphql_elements with element_type filter 4. **get_element_details**: Lookup element by name with source document and code examples 5. **list_categories**: Hierarchical category tree with document counts 6. **get_tutorial**: Sequential tutorial steps from tutorials/ directory 7. **search_examples**: Search code_blocks by content and language 8. **get_related_documents**: Find related docs by category and keywords ## Development Commands ### Installation ```bash # Development install cd magento-graphql-docs-mcp pip install -e . ``` ### Running the Server ```bash # Run MCP server (parses docs on first run or if files modified) magento-graphql-docs-mcp # Or via Python module python3 -m magento_graphql_docs_mcp.server ``` ### Testing **Important**: All tests must be run from the project root directory: ```bash # Navigate to project root first cd magento-graphql-docs-mcp # Test markdown parser python3 tests/verify_parser.py # Test database ingestion python3 tests/verify_db.py # Test MCP server and all 8 tools python3 tests/verify_server.py # Run performance benchmarks python3 tests/benchmark_performance.py ``` Running tests from other directories will cause import errors. ### Configuration ```bash # Set custom documentation path export MAGENTO_GRAPHQL_DOCS_PATH=/path/to/commerce-webapi/src/pages/graphql # Set custom database path export MAGENTO_GRAPHQL_DOCS_DB_PATH=/custom/path/database.db # Run server magento-graphql-docs-mcp ``` ## Key Implementation Details ### Frontmatter Parsing Uses `python-frontmatter` library to parse YAML metadata from markdown files: ```yaml --- title: products query description: Describes how to construct and use the Catalog Service products query keywords: - GraphQL - Services --- ``` Fallback: If no frontmatter title, extracts first `#` header from markdown. ### Category Extraction from File Paths Pattern: `category/subcategory/file.md` - `schema/products/queries/products.md` → category="schema", subcategory="products" - `develop/exceptions.md` → category="develop", subcategory=None - `index.md` → category="root", subcategory=None Content type determined by category: - "schema" → "schema" - "tutorials" → "tutorial" - "develop", "usage" → "guide" ### GraphQL Element Detection Regex patterns in `parser.py:extract_graphql_elements()`: - `query\s+(\w+)` → Detects GraphQL queries - `mutation\s+(\w+)` → Detects mutations - `type\s+(\w+)` → Detects types - `interface\s+(\w+)` → Detects interfaces Extracts: - Element name - Fields (via regex on field patterns) - Parameters (via `$(\w+):` pattern) - Builds searchable text ### Database Design ```sql -- Documents (350 rows) documents (id, file_path, title, description, keywords_json, category, subcategory, content_type, searchable_text, headers_json, last_modified, content_md) + FTS5 index on searchable_text + Indexes on category, subcategory, content_type -- Code blocks (963 rows) code_blocks (id, document_id, language, code, context, line_number) + Indexes on document_id, language -- GraphQL elements (51 rows) graphql_elements (id, document_id, element_type, name, fields_json, parameters_json, return_type, description, searchable_text) + FTS5 index on searchable_text + Indexes on document_id, element_type, name -- Metadata (1 row) metadata (key, docs_directory_mtime, total_files, ingestion_time) ``` ### Performance Characteristics **Actual Benchmark Results** (run `tests/benchmark_performance.py`) *Benchmarked: November 2024, Apple Silicon M1/M2, 350 documents* - **Startup Time**: 0.87s (when data unchanged) | 3-5s (first run or files changed) - **Search Time**: 5.5ms average (FTS5 direct: 0.7ms) - **18x faster than 100ms target** - **Document Retrieval**: 8.2ms (direct lookup: 0.3ms) - **GraphQL Element Search**: 3.4ms - **Parsing Time**: ~2-3 seconds for 350 markdown files - **Ingestion Time**: ~1-2 seconds for database population - **Re-ingestion**: Only happens if any markdown file mtime changes - **Database Size**: ~30 MB for 350 documents All performance targets exceeded: <5s startup ✓, <100ms searches ✓ **Note**: Performance will vary based on hardware. Run `python3 tests/benchmark_performance.py` to measure on your system. ### Searchable Text Building Combines: 1. Title 2. Description (from frontmatter) 3. Keywords (from frontmatter) 4. All headers 5. Content (with code blocks removed, markdown links stripped) This enables comprehensive FTS search across all textual content. ### Code Block Context Extraction For each code block, extracts 1-5 lines before the block as context. This provides surrounding explanation for better search results in `search_examples` tool. ### Tutorial Sequencing Tutorial files ordered by file_path natural sorting: - `tutorials/checkout/create-cart.md` (Step 1) - `tutorials/checkout/add-product-to-cart.md` (Step 2) - etc. File path ordering provides sequential step ordering. ## Important Notes - The server MUST have access to GraphQL documentation markdown files - Database file requires write permissions in ~/.mcp/magento-graphql-docs/ - FTS5 is SQLite extension - requires SQLite 3.9.0+ (standard in Python 3.10+) - Frontmatter parsing gracefully handles files without frontmatter - GraphQL element extraction is best-effort (uses regex, not full parser) - Multiple elements can have same name (e.g., "Query" appears in multiple docs) ## Configuration All configuration in `config.py`: ### Path Configuration - `DB_PATH`: Database location (default: ~/.mcp/magento-graphql-docs/database.db) - `DOCS_PATH`: Documentation directory (auto-detected or from environment variable) - Override via environment variables: - `MAGENTO_GRAPHQL_DOCS_DB_PATH`: Custom database location - `MAGENTO_GRAPHQL_DOCS_PATH`: Custom documentation path ### Performance Tuning (Environment Variables) - `MAGENTO_GRAPHQL_DOCS_TOP_K`: Number of search results to return (default: 5) - `MAGENTO_GRAPHQL_DOCS_MAX_FIELDS`: Max fields per GraphQL element (default: 20) - `MAGENTO_GRAPHQL_DOCS_CODE_PREVIEW`: Max code preview length in chars (default: 400) ### Constants - `SEARCH_RESULT_MULTIPLIER`: Fetch 2x results before filtering (hardcoded: 2) ## Extension Points ### Adding New Tools 1. Add `@mcp.tool()` decorated function in `server.py` 2. Use type hints with `Annotated[Type, Field(description=...)]` for parameter docs 3. Query database using `Database(DB_PATH)` 4. Format output as markdown string 5. Test with `verify_server.py` ### Improving GraphQL Element Extraction To more accurately extract GraphQL elements: 1. Consider using a GraphQL parser library (e.g., `graphql-core`) 2. Parse code blocks marked as `graphql` language 3. Extract full AST for queries, mutations, types 4. Store more structured data (input types, output types, arguments) ### Supporting Multiple Documentation Versions To support multiple Magento versions: 1. Add version parameter to config 2. Store multiple doc directories: `graphql-2.4.7/`, `graphql-2.4.8/` 3. Include version in database table or use separate databases 4. Add version parameter to tools ### Adding Full-Text Search on Code To enable FTS search on code content (not just surrounding context): 1. Create FTS5 index on code_blocks.code 2. Update `search_examples` to use FTS instead of LIKE 3. Benefit: Faster, more relevant code search ## Comparison with Other MCP Implementations ### vs Gemini Docs MCP | Aspect | Gemini Docs MCP | GraphQL Docs MCP | |--------|----------------|------------------| | Data Source | Web (HTML) | Local (Markdown) | | Startup | 30-60s | 3-5s | | Offline | ❌ | ✅ | | Content | Unstructured | Structured + Narrative | | Tools | 3 | 8 | ### vs Magento REST API MCP | Aspect | REST API MCP | GraphQL Docs MCP | |--------|-------------|------------------| | Data Source | OpenAPI JSON | Markdown files | | Content Type | API specs only | Schemas + Guides + Tutorials | | Examples | None | 963 code blocks | | Tutorials | ❌ | ✅ | | Tools | 5 | 8 | ## Key Learnings from Implementation 1. **Frontmatter is Gold**: YAML frontmatter provides high-quality metadata 2. **File Path = Category**: Directory structure encodes valuable categorization 3. **Code Context Matters**: Surrounding text makes code examples searchable 4. **FTS5 is Fast**: Trigram tokenization + FTS5 = <100ms searches 5. **Regex is Good Enough**: Don't need full GraphQL parser for useful element extraction 6. **Markdown is Convenient**: Direct markdown content in DB allows full document retrieval 7. **Bulk Inserts Win**: Batch inserts much faster than row-by-row ## Project Dependencies - Core: `fastmcp`, `sqlite-utils`, `pydantic` - Parsing: `python-frontmatter`, `markdown-it-py` - Dev: `mcp>=1.16.0` (for testing with MCP client) - Python: 3.10+ (for modern type hints and SQLite FTS5) ## Common Pitfalls 1. **Missing Frontmatter**: Not all markdown files have frontmatter - use fallbacks 2. **GraphQL in Non-GraphQL Blocks**: Only parse code blocks with language="graphql" 3. **File Path Assumptions**: Don't assume all paths have subcategory level 4. **FTS Syntax**: FTS5 uses different syntax than LIKE queries 5. **JSON Serialization**: Must serialize lists to JSON for SQLite storage 6. **Context Extraction**: Index bounds checking when looking backwards for context ## Testing Strategy Four-level verification: 1. **Parser Test** (`tests/verify_parser.py`): Ensures markdown → Pydantic models works 2. **Database Test** (`tests/verify_db.py`): Ensures ingestion → SQLite works + FTS search works 3. **Server Test** (`tests/verify_server.py`): Ensures all 8 MCP tools work via STDIO 4. **Performance Benchmark** (`tests/benchmark_performance.py`): Measures actual startup and query times Run all four tests to verify complete system functionality and performance.