GitHub MCP Toolkit (github-mcp-toolkit)

A production-grade, fault-tolerant, and benchmarked Enterprise Model Context Protocol (MCP) Server written in Python. Designed for real-world enterprise deployments, it provides Large Language Models (LLMs) with cryptographically governed, RBAC/ABAC-enforced, reversible, and token-budgeted access to GitHub repositories across 13 MCP Tools, 4 Native MCP Resources, and 2 Native MCP Prompts.

📌 Quick Navigation & Key Documents

Related MCP server: GitHub MCP Agent Server

📊 Impact & Performance Benchmark Metrics

🚀 Enterprise & Principal Engineer Differentiators

Unlike basic CRUD MCP wrappers, github-mcp-toolkit implements 15 rare enterprise capabilities required by production engineering organizations:

I. Security, Governance & Compliance

1. Enterprise Outbound DLP + Shannon Entropy + Luhn PII Scanner (policy_engine.py): Scans all outbound write parameters for 10 vendor credential patterns (GitHub PATs, AWS keys, OpenAI/Anthropic keys, JWTs, Slack tokens, DB URIs, PEM keys, SSNs), vendor-agnostic high-entropy secrets ($H(X) \ge 3.8\text{ bits/char}$), and ISO/IEC 7812 Luhn-validated payment cards.

2. Scoped Session RBAC / Agent Persona Least-Privilege (policy_engine.py): Enforces 4 hierarchical agent personas (read_only, triage_operator, maintainer, admin) via policy.json or MCP_AGENT_ROLE before any write tool executes.

3. Per-Tool 60s Sliding-Window Write Velocity Limiter (policy_engine.py): Thread-safe per-tool sliding-window rate limiter (max_writes_per_minute) stops runaway agent loops from exhausting GitHub quota or spamming repositories.

4. Cryptographic 2-Phase Preview Tokens for Single & Bulk Writes (policy_engine.py, tools/): Issues single-use, 5-minute TTL SHA256 tokens bound to normalized parameters (repo_name, issue_number, title, body, label, comment). Blocks parameter bait-and-switch and concurrent replay attacks.

5. 5-Signature Indirect Prompt-Injection Threat Scorer (tools/triage_issue.py): Escapes XML delimiters (<untrusted_issue_data>), computes a multi-vector threat score across 5 adversarial signatures, and automatically blocks auto-label mutations (apply_labels=True) when an issue contains indirect prompt-injection payloads.

6. SOC-2 / ISO-27001 Governance & Compliance Snapshot Exporter (policy_engine.py, server.py): Generates a canonical JSON compliance attestation signed with a 64-char snapshot_sha256 digest via PolicyEngine.export_compliance_snapshot() and the native github://compliance/snapshot MCP resource.

II. Reliability, Reversibility & Context-Window Protection

7. Tamper-Evident SHA-256 Hash-Chained Saga Journal (transaction_journal.py): Every write transaction is cryptographically linked (prev_hash + record_hash) in a blockchain-style audit log (verify_chain_integrity()), detecting any out-of-band tampering or deletion.

8. Saga Dry-Run Inspection & Targeted Out-of-Order Rollback (tools/undo_last_action.py): Supports dry_run=True to preview exact compensation steps without mutating GitHub state, and tx_id to roll back any specific historical transaction out-of-order while leaving newer work intact.

9. Pre-Flight Blast-Radius, Quota Impact & Reversibility Simulator (policy_engine.py): Every write tool's Phase-1 preview returns a deterministic blast_radius report (blast_radius_score, severity_tier, estimated_api_calls, quota_impact_pct, reversible_via_saga, rollback_operation).

10. Context-Window Token Budgeter & Adaptive Payload Compactor (schemas.py): Automatically compacts oversized issue bodies and enforces a global payload token budget (DEFAULT_MAX_PAYLOAD_TOKENS = 3000) in the server middleware so large repositories never overflow the LLM context window.

11. Deterministic Chaos & Fault-Injection Resilience Harness (circuit_breaker.py): Built-in ChaosInjector simulates 502_bad_gateway, 503_unavailable, 429_rate_limit, and timeout faults to verify circuit-breaker trip (CLOSED → OPEN → HALF_OPEN → CLOSED) and single-probe recovery.

III. Retrieval, Incident Triage & Full MCP Spec Coverage

12. Hybrid RRF (Reciprocal Rank Fusion) Search Engine (vector_engine.py): Combines dense-style Corpus TF-IDF cosine similarity with BM25-style exact lexical/error-code/issue-number matching using Reciprocal Rank Fusion: $$\text{RRF}(d) = \frac{1}{k + r_{\text{semantic}}(d)} + \frac{1}{k + r_{\text{lexical}}(d)}, \quad k=60$$

13. Automated Incident Runbook & Related-Issue Cross-Linker (tools/triage_issue.py): Enriches issue triage with priority-mapped SLA targets (SLA_TARGETS_HOURS), actionable remediation playbooks, and automatic Hybrid RRF cross-linking of related open issues.

14. Live SLO (p50/p95 Latency) & Guardrail Telemetry Diagnostics (tracer.py): Computes real-time p50/p95 latency percentiles, error rates, and guardrail block counters across all execution spans (get_telemetry_summary()).

15. Full MCP Protocol Coverage — Tools, Resources & Prompts (server.py): Implements all three MCP specification primitives: 13 Tools, 4 @mcp.resource endpoints, and 2 @mcp.prompt workflow templates.

🏗️ System Architecture

flowchart TD
    LLM[LLM / Claude Desktop] <-->|stdio / sse transport| MCP[FastMCP Server\nserver.py]
    MCP --> Logger[Structured Audit Logger\ntool_calls.log]
    MCP --> Tracer[SLO & Span Tracer\ntracer.py → traces.jsonl]
    MCP --> Compactor[Token Budgeter & Schema Validator\nschemas.py]
    MCP --> Tools[13 Tools + 4 Resources + 2 Prompts]

    subgraph CoreTools [Core Tools — 9]
        T1[get_open_issues]
        T2[search_issues]
        T3[create_issue]
        T4[add_label]
        T5[close_issue]
        T6[bulk_label_stale_issues]
        T7[triage_issue]
        T8[get_rate_limit_status]
        T13[list_repositories]
    end

    subgraph AdvancedTools [Advanced Tools — 4]
        T9[semantic_search_issues]
        T10[undo_last_action]
        T11[get_transaction_history]
        T12[get_trace_history]
    end

    T3 & T4 & T5 & T6 & T7 --> PE[PolicyEngine: RBAC + Velocity + DLP + Entropy + BlastRadius\npolicy_engine.py]
    T3 & T7 & T9 --> VE[Hybrid RRF + Corpus TF-IDF VectorEngine\nvector_engine.py]
    T3 & T4 & T5 & T6 --> TJ[SHA-256 Hash-Chained Saga Journal\ntransaction_journal.py]
    T10 & T11 --> TJ
    T12 --> Tracer
    T7 --> Sandbox[XML Sandbox + 5-Sig Injection Scorer] --> Ollama[Local Ollama\nllama3.2:1b]
    
    GHC[GitHubClient\ngithub_client.py] <-->|RLock CircuitBreaker + ChaosInjector| CB[circuit_breaker.py]
    CB --> GHAPI[GitHub REST API]
    CoreTools --> GHC

🛠️ MCP Primitives Reference (13 Tools, 4 Resources, 2 Prompts)

Core Tools (9)

Advanced Tools (4)

Native MCP Resources (4) & Prompts (2)

📂 Project Structure

github-mcp-toolkit/
├── .github/
│   └── workflows/
│       └── ci.yml               # CI/CD pipeline: pytest (120 tests) + 100-query eval + live parity check
├── api/
│   └── index.py                 # Interactive Web Console (Intent Router + Real Guardrail Simulator)
├── server.py                    # FastMCP server entrypoint (13 Tools, 4 Resources, 2 Prompts, Token Budgeter)
├── github_client.py             # PyGithub client wrapper (Auth.Token, timeout, lazy pagination, breaker)
├── circuit_breaker.py           # RLock CircuitBreaker + Deterministic ChaosInjector fault harness
├── vector_engine.py             # Pure-Python Corpus TF-IDF + Hybrid Reciprocal Rank Fusion (RRF) engine
├── transaction_journal.py       # Tamper-evident SHA-256 hash-chained Saga transaction journal
├── policy_engine.py             # RBAC + Velocity Limiter + Outbound DLP/Entropy/Luhn + Blast-Radius + SOC-2 Exporter
├── tracer.py                    # ContextVar span execution tracer + p50/p95 SLO telemetry engine
├── schemas.py                   # Strict Pydantic contracts for all 13 tools + Adaptive Token Budgeter
├── policy.json                  # Declarative RBAC/ABAC & velocity policy rules (editable without redeploy)
├── Dockerfile                   # Production container definition (SSE transport ready)
├── docker-compose.yml           # Stack orchestration service
├── vercel.json                  # Serverless web demo deployment configuration
├── tools/                       # 13 modular MCP tool implementations
├── eval/
│   ├── run_eval.py              # 100-query dual benchmark (Router + Live Ollama LLM mode) + parity verifier
│   ├── test_queries.json        # 80 standard natural-language queries
│   └── adversarial_queries.json # 20 adversarial prompt injection test cases
└── tests/
    ├── conftest.py              # Isolated PyGithub client fixtures
    ├── test_github_client.py    # Client layer tests
    ├── test_tools.py            # Tool guardrail integration tests
    └── test_advanced_features.py # 102 subsystem, concurrency, cryptographic, and chaos resilience tests

⚡ Quick Start

1. Local Setup (Claude Desktop)

# Clone repository
git clone https://github.com/kartik-012/GitHub-MCP-Toolkit.git
cd GitHub-MCP-Toolkit

# Setup virtual environment
python -m venv venv
venv\Scripts\activate       # Windows
# source venv/bin/activate  # Linux/macOS

# Install dependencies
pip install -r requirements.txt

# Configure environment
cp .env.example .env
# Edit .env and set GITHUB_TOKEN=ghp_...

Add to %APPDATA%\Claude\claude_desktop_config.json:

{
  "mcpServers": {
    "github-mcp-toolkit": {
      "command": "C:/path/to/venv/Scripts/python.exe",
      "args": ["C:/path/to/GitHub-MCP-Toolkit/server.py"]
    }
  }
}

2. Docker Setup (Containerized SSE Server)

docker compose up -d
docker compose logs -f

🧪 Testing & Evaluation Benchmark

# Run all 120 unit, integration, concurrency, and chaos resilience tests
.\venv\Scripts\python.exe -m pytest -v

# Run 100-query dual benchmark harness + live server parity check
.\venv\Scripts\python.exe eval/run_eval.py

==============================================================
  GitHub MCP Toolkit - Tool Selection Evaluation Harness (ROUTER MODE)
==============================================================

[PARITY]    Live Server & Schema Contract Parity (13 tools) [PASS]

[STANDARD]  Standard Benchmark (80 queries)
    Correct Tool Selection : 80/80 (100.0%)

[ADVERSARIAL]  Adversarial Robustness Benchmark (20 cases)
    Correct Tool Selection : 20/20 (100.0%) [PASS]

📜 License

MIT License — see LICENSE for details.