Daraja API Documentation Scraper & MCP Server

A comprehensive toolkit for accessing Safaricom's Daraja API documentation through web scraping and Model Context Protocol (MCP) integration. This project provides both a powerful scraper for keeping documentation up-to-date and a professional MCP server for AI assistant integration.

⚠️ Important Legal Notice & Disclaimers

Data Source and Usage Rights

• Documentation Source: This project scrapes documentation from Safaricom's Daraja API portal

• Authentication Required: The scraper requires login credentials to access Safaricom's developer portal

• Terms of Service: Users must comply with Safaricom's Terms of Service when using the scraper

• Data Ownership: All scraped documentation remains the intellectual property of Safaricom PLC

Legal Compliance

• Personal Use: This tool is intended for personal development and learning purposes

• Commercial Use: For commercial applications, ensure you have appropriate licenses from Safaricom

• Rate Limiting: The scraper includes delays to respect Safaricom's servers - do not modify these

• Account Responsibility: Users are responsible for their own Safaricom developer account credentials

Disclaimers

• No Warranty: This software is provided "as is" without any warranties

• Data Accuracy: Scraped documentation may become outdated - always verify with official sources

• Service Availability: Safaricom may change their portal structure, potentially breaking the scraper

• Legal Responsibility: Users assume all legal responsibility for their use of this tool

Ethical Usage Guidelines

• Respect Rate Limits: Do not overwhelm Safaricom's servers with excessive requests

• Valid Credentials: Only use your own legitimate Safaricom developer account

• Data Sharing: Be mindful of sharing scraped data - respect Safaricom's intellectual property

• Updates: Keep scraped documentation current and don't redistribute outdated information

By using this software, you acknowledge that you have read, understood, and agree to comply with these terms and Safaricom's Terms of Service.

Features

• Complete Documentation Scraper: Automated scraping of all 22 Daraja APIs with images

• Professional MCP Server: Standards-compliant server for AI assistant integration

• Docker Support: Containerized deployment for consistent environments

• Multi-Platform Support: Works on Windows, macOS, and Linux

• Editor Integration: Compatible with Kiro, VSCode, Cursor, Windsurf, Claude Desktop, and more

• Offline Ready: Complete documentation dataset included for immediate use

• Auto-Discovery: Smart path detection for seamless setup

Table of Contents

• Important Legal Notice & Disclaimers

• Quick Start

• Installation

• Docker Deployment

• Scraper Usage

• MCP Server Setup

• Editor Integration

• API Documentation

• Troubleshooting

• Contributing

• License

• Contributing

Quick Start

Option 1: Docker Deployment (Recommended)

Option 2: Native Installation

Option 3: Fresh Scraping + MCP Setup

Installation

Prerequisites

• Docker (recommended) OR

• Python 3.8+ (for scraper) + Node.js 18+ (for MCP server)

• Git (for cloning)

Platform-Specific Setup

Windows

macOS

Linux (Ubuntu/Debian)

Docker Deployment

Docker provides the most reliable and consistent deployment method across all platforms.

Quick Docker Setup

Docker Management Commands

Docker Configuration

The Docker setup includes:

• Multi-stage build for optimized image size

• Non-root user for security

• Volume mounting for documentation access

• Health checks for reliability

• Resource limits for production use

Docker Compose Configuration

MCP Configuration for Docker

When using Docker, update your editor's MCP configuration:

Scraper Usage

⚠️ IMPORTANT: Before using the scraper, ensure you have:

• A valid Safaricom developer account

• Accepted Safaricom's Terms of Service

• Permission to access the documentation programmatically

• Understanding that you're responsible for compliance with Safaricom's policies

The scraper automatically downloads documentation for all 22 Daraja APIs with images.

First Time Setup

1. Activate Python environment:

   source venv/bin/activate # Linux/macOS venv\Scripts\activate # Windows

2. Run the scraper:

   python scraper.py

3. Login Process:

   • Browser window opens automatically

   • Login to Daraja portal manually when prompted

   • Press ENTER in terminal after successful login

   • Scraper saves session for future use

Updating Documentation

To get the latest API documentation:

Output Structure

MCP Server Setup

The MCP server provides AI assistants with access to Daraja documentation through 6 powerful tools.

Native Build and Test

Available MCP Tools

The MCP server provides 6 comprehensive tools:

1. search_daraja_apis - Advanced search with category filtering

2. get_daraja_api_doc - Complete API documentation retrieval

3. list_apis_by_category - Organized API listing by categories

4. get_api_summary - Enhanced summaries with endpoints

5. get_server_stats - Usage statistics and monitoring

6. compare_apis - Side-by-side API comparison

Server Configuration

The server automatically detects documentation in these locations:

• ../daraja_docs_v3 (relative to server)

• ./daraja_docs_v3 (current directory)

• /app/daraja_docs_v3 (Docker container)

• Custom path via environment variable

Editor Integration

Kiro IDE

1. Create MCP configuration file:

   # Create .kiro/settings/mcp.json in your workspace mkdir -p .kiro/settings

2. Docker Configuration (Recommended):

   { "mcpServers": { "daraja-docs": { "command": "docker", "args": ["exec", "-i", "daraja-mcp-server", "node", "dist/index.js"], "disabled": false, "autoApprove": [ "search_daraja_apis", "get_daraja_api_doc", "list_apis_by_category", "get_api_summary", "get_server_stats", "compare_apis" ] } } }

3. Native Configuration:

   { "mcpServers": { "daraja-docs": { "command": "node", "args": ["mpesa-daraja-mcp/dist/index.js"], "disabled": false, "autoApprove": [ "search_daraja_apis", "get_daraja_api_doc", "list_apis_by_category", "get_api_summary", "get_server_stats", "compare_apis" ] } } }

VSCode with MCP Extensions

1. Install MCP extension (if available)

2. Configure in settings.json:

   { "mcp.servers": { "daraja-docs": { "command": "docker", "args": ["exec", "-i", "daraja-mcp-server", "node", "dist/index.js"] } } }

Cursor IDE

1. Open Cursor settings

2. Add MCP server configuration:

   { "mcp": { "servers": { "daraja-docs": { "command": "docker", "args": ["exec", "-i", "daraja-mcp-server", "node", "dist/index.js"] } } } }

Windsurf

1. Access Windsurf MCP settings

2. Add server configuration:

   { "mcpServers": { "daraja-docs": { "command": "docker", "args": ["exec", "-i", "daraja-mcp-server", "node", "dist/index.js"] } } }

Claude Desktop

1. Edit Claude config file:

   • macOS: ~/Library/Application Support/Claude/claude_desktop_config.json

   • Windows: %APPDATA%\Claude\claude_desktop_config.json

2. Add server configuration:

   { "mcpServers": { "daraja-docs": { "command": "docker", "args": ["exec", "-i", "daraja-mcp-server", "node", "dist/index.js"] } } }

Generic MCP Client

For any MCP-compatible client using Docker:

For native installation:

API Documentation

Available APIs

The project includes comprehensive documentation for 22 Daraja APIs organized into 6 categories:

Core Payment APIs

• Authorization - OAuth token generation and management

• MpesaExpressSimulate - STK Push payment initiation

• MpesaExpressQuery - STK Push transaction status queries

Payment Processing

• CustomerToBusiness - C2B payment processing

• BusinessToCustomer - B2C payment disbursements

• CustomerToBusinessRegisterURL - C2B URL registration

Transaction Management

• TransactionStatus - Transaction status verification

• AccountBalance - Account balance inquiries

• Reversal - Transaction reversal operations

• PullTransaction - Transaction detail retrieval

Business Operations

• BusinessPayBill - Bill payment processing

• BusinessBuyGoods - Goods purchase payments

• B2BExpressCheckout - Business-to-business transactions

• BusinessToPochi - Business to Pochi wallet transfers

Advanced Features

• DynamicQRCode - Dynamic QR code generation

• BillManager - Bill management and processing

• TaxRemittance - Tax payment processing

• MpesaRatiba - Scheduled payment management

Specialized Services

• B2CAccountTopUp - Account top-up services

• Swap - Currency swap operations

• IMSI - SIM card management services

• IotSimManagement - IoT SIM card management

Using the MCP Tools

Advanced API Search

Use search_daraja_apis tool with powerful filtering:

• Query: "payment" + Category: "core" → Core payment APIs

• Query: "balance" → AccountBalance and related APIs

• Category: "business" → All business operation APIs

• No parameters → All 22 APIs with categories

Complete Documentation Access

Use get_daraja_api_doc tool for full documentation:

• api_name: "Authorization" → Complete OAuth implementation guide

• api_name: "MpesaExpressSimulate" → Full STK Push documentation

• Includes code examples, parameters, and response formats

Enhanced API Summaries

Use get_api_summary tool for quick overviews:

• api_name: "BusinessToCustomer" → Summary with key endpoints

• include_endpoints: true → Includes endpoint URLs

• Perfect for quick reference and comparison

API Comparison

Use compare_apis tool for side-by-side analysis:

• api_names: ["Authorization", "MpesaExpressSimulate"] → Compare auth vs payment

• comparison_aspects: ["endpoints", "authentication"] → Focus on specific aspects

• Supports 2-4 APIs simultaneously

Usage Statistics

Use get_server_stats tool for monitoring:

• Total API count and categories

• Most accessed APIs

• Request statistics

• Server health information

Category-Based Browsing

Use list_apis_by_category tool for organized access:

• category: "payments" → All payment-related APIs

• No category → List all categories with counts

• Perfect for discovering related APIs

Troubleshooting

Docker Issues

Container Won't Start

Documentation Not Found in Container

MCP Connection Issues with Docker

Native Installation Issues

MCP Server Issues

Server won't start:

Documentation not found:

TypeScript errors:

Scraper Issues

Browser doesn't open:

Login session expired:

Permission errors:

Editor Integration Issues

MCP Server Not Connecting

1. Verify Docker container is running:

   pnpm run docker:status

2. Check MCP configuration syntax:

   • Ensure JSON is valid

   • Verify command and args are correct

   • Use absolute paths for native installation

3. Test server manually:

   # Docker docker exec -i daraja-mcp-server node dist/index.js # Native cd mpesa-daraja-mcp node dist/index.js

4. Restart editor after configuration changes

Tools Not Appearing

1. Check server logs in editor's MCP panel

2. Verify autoApprove list includes desired tools

3. Ensure server builds successfully: pnpm run build

4. Test with minimal configuration first

Platform-Specific Issues

Windows Platform Issues

• Use forward slashes in Docker paths: C:/path/to/project

• Run PowerShell as Administrator if Docker permission issues

• Check Windows Defender isn't blocking Docker or Node.js

• Use WSL2 for better Docker performance

macOS Platform Issues

• Install Xcode Command Line Tools: xcode-select --install

• Use Homebrew for dependency management: brew install docker

• Check Docker Desktop is running and configured

• Verify file permissions: chmod +x docker-scripts.sh

Linux Platform Issues

• Add user to docker group: sudo usermod -aG docker $USER

• Install build essentials: sudo apt install build-essential

• Check Docker service: sudo systemctl status docker

• Verify Node.js installation: which node

Performance Optimization

Docker Performance

MCP Server Performance

Getting Help

1. Check logs first: Both Docker and native installations provide detailed error messages

2. Test components individually: Use pnpm test to verify MCP server setup

3. Verify paths and permissions: Ensure all file paths are correct and accessible

4. Update dependencies: Keep Docker, Node.js, and Python packages current

5. Use Docker for consistency: Docker eliminates most platform-specific issues

Contributing

⚠️ Data Handling Guidelines for Contributors:

• No Scraped Data in PRs: Do not include scraped documentation in pull requests

• Respect IP Rights: Ensure contributions don't violate Safaricom's intellectual property

• Code Only: Contribute improvements to the scraper and MCP server code, not the data

• Documentation: Update README and code comments, not scraped API documentation

Development Setup

1. Fork the repository

2. Create feature branch: git checkout -b feature/amazing-feature

3. Setup development environment:

   # Docker development (recommended) cd mpesa-daraja-mcp pnpm run docker:build pnpm run docker:start # Native development # Python development pip install -r requirements-dev.txt # If exists # Node.js development cd mpesa-daraja-mcp pnpm install pnpm run dev

Adding New APIs

1. Update scraper URLs:

   URLS = [ "https://developer.safaricom.co.ke/apis/NewAPI", # ... existing URLs ]

2. Test scraping:

   python scraper.py

3. Update API categorization:

   // In mpesa-daraja-mcp/src/config.ts API_CATEGORIES: { new_category: ["NewAPI"], // ... existing categories }

4. Verify MCP server:

   cd mpesa-daraja-mcp pnpm test pnpm run docker:build

Code Style

• Python: Follow PEP 8, use black formatter

• TypeScript: Use Prettier formatting, ESLint rules

• Docker: Follow best practices for multi-stage builds

• Commits: Use conventional commit format

Testing

Documentation

• Update README for new features

• Add inline code comments

• Update API documentation

• Include Docker-specific instructions

License

This project is licensed under the MIT License - see the LICENSE file for details.

⚠️ Important License Clarification:

• Software License: The MIT License applies only to the scraper and MCP server code

• Documentation Rights: Scraped Daraja API documentation remains the property of Safaricom PLC

• Separate Terms: Use of Safaricom's documentation is subject to their Terms of Service

• No Transfer: This license does not grant rights to Safaricom's intellectual property

What this means

• ✅ Commercial use - Use this project in commercial applications

• ✅ Modification - Modify the source code to fit your needs

• ✅ Distribution - Distribute copies of the software

• ✅ Private use - Use the software for private purposes

• ❌ Liability - The authors are not liable for any damages

• ❌ Warranty - The software is provided "as is" without warranty

Attribution

If you use this project, please consider:

• ⭐ Starring the repository on GitHub

• 📝 Mentioning the project in your documentation

• 🔗 Linking back to the original repository

Acknowledgments

• Safaricom for providing the Daraja API platform

• Model Context Protocol team for the MCP standard

• Playwright team for the excellent automation framework

• Docker community for containerization best practices

• Open source contributors who make projects like this possible

Repository Information

• Repository: https://github.com/JacksCodeVault/mpesa-daraja-mcp

• Issues: Report bugs or request features

• Discussions: Community discussions

• Releases: Latest releases

Support

If you find this project helpful, please consider:

• ⭐ Starring the repository

• 🐛 Reporting issues you encounter

• 💡 Suggesting improvements

• 🤝 Contributing to the project

• 📢 Sharing with others who might benefit

Ready to integrate Daraja API documentation with your AI assistant?

Quick Start with Docker: cd mpesa-daraja-mcp && pnpm run docker:build && pnpm run docker:start

Browse the documentation: Use the MCP tools to explore 22 comprehensive Daraja APIs

Need help? Check the Troubleshooting section or open an issue

Start with the