Skip to main content
Glama
razi2004

Custom Python MCP Server

by razi2004
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Hybrid

🛡️ AutoAudit: Autonomous Codebase Health Reviewer

Python 3.10+ MCP Tests Zero-Hallucination Dual-Mode

AutoAudit is an autonomous, agentic codebase health and security reviewer. It connects to any public GitHub repository or local directory, performs deep static analysis and reconnaissance across multiple ecosystems, and coordinates specialized worker agents to uncover security vulnerabilities, dead code, dependency risks, and test coverage gaps.

Unlike standard LLM code reviewers that hallucinate line numbers or speculate on non-existent code, AutoAudit enforces a zero-hallucination verification loop: every single claim is independently re-verified against physical repository files via Model Context Protocol (MCP) tools before it is accepted into the final scored report.

🏗️ Architecture Overview

AutoAudit operates across a coordinated 6-phase agentic pipeline:

                         ┌─────────────────────────────────────────┐
                         │   CLI: autoaudit run <github-url|path>  │
                         └────────────────────┬────────────────────┘
                                              │
              ┌───────────────────────────────┴───────────────────────────────┐
              ▼                                                               ▼
    [Local Directory Path]                                       [GitHub Remote Repository]
              │                                                               │
              │                                                (Shallow Clone --depth 1 into
              │                                                 managed temporary workspace)
              │                                                               │
              └───────────────────────────────┬───────────────────────────────┘
                                              │
                                              ▼
              ┌───────────────────────────────────────────────────────────────┐
              │             Phase 1: Custom Python MCP Server                 │
              │   • list_files          • get_file          • grep_repo       │
              │   • get_dependency_tree • get_test_coverage                   │
              └───────────────────────────────┬───────────────────────────────┘
                                              │
                                              ▼
              ┌───────────────────────────────────────────────────────────────┐
              │            Phase 2: Orchestrator / Planner Agent              │
              │   Reconnaissance ──► Claude API / Deterministic Heuristic     │
              │   Decomposes audit into discrete, category-specific tasks     │
              └───────────────────────────────┬───────────────────────────────┘
                                              │
                                              ▼
              ┌───────────────────────────────────────────────────────────────┐
              │             Phase 3: Specialized Worker Agents                │
              │   🔒 SecurityWorker       • Hardcoded secrets, weak crypto    │
              │   🧹 DeadCodeWorker       • Abandoned TODOs, orphan functions │
              │   📦 DependencyWorker     • Unpinned manifests, supply chain  │
              │   🧪 CoverageWorker       • Missing tests & uncovered lines   │
              └───────────────────────────────┬───────────────────────────────┘
                                              │
                                              ▼
              ┌───────────────────────────────────────────────────────────────┐
              │       Phase 4: Verifier Agent (Zero-Hallucination Guard)       │
              │   Re-reads physical files via MCP tools with max 2 retries    │
              │   • Attempt 1: Exact Match  • Attempt 2: ±10 Line Proximity   │
              │   • Attempt 3: File Grep    • Unverifiable Claims: DROPPED    │
              └───────────────────────────────┬───────────────────────────────┘
                                              │
                                              ▼
              ┌───────────────────────────────────────────────────────────────┐
              │           Phase 5: Scored Health Report Generator             │
              │   • Weighted Health Score (0-100) & Letter Grade (A - F)      │
              │   • Score Deductions: Critical (-25), High (-15), Med (-5)    │
              │   • Formats: Clean Markdown + Zero-CDN Standalone HTML        │
              └───────────────────────────────┬───────────────────────────────┘
                                              │
                                              ▼
                         ┌─────────────────────────────────────────┐
                         │   Final Scorecard & Deliverables Saved  │
                         │   reports/audit_report_<name>_<ts>.html │
                         │   reports/audit_report_<name>_<ts>.md   │
                         └─────────────────────────────────────────┘

Related MCP server: Recon

⚙️ How It Works

When you run autoaudit run <repo-url-or-path>, the system autonomously executes five operational stages:

  1. Workspace Ingestion & Isolation:

    • If given a remote GitHub URL, AutoAudit executes a shallow clone (git clone --depth 1) into an isolated temporary directory with automatic lifecycle cleanup upon completion. Local paths are validated safely with directory traversal protection.

  2. Automated Reconnaissance & Strategic Planning:

    • The Orchestrator queries Phase 1 MCP tools (list_files, get_dependency_tree, get_test_coverage) to identify project structure, languages, dependency manifests, and existing test suites.

    • It decomposes the review into targeted, discrete tasks across four key domains: Security, Dead Code, Dependency Risk, and Test Coverage Gaps.

  3. Grounded Evidence Gathering:

    • Specialized workers execute each task by invoking MCP tools:

      • grep_repo: Locates high-entropy credential assignments, weak hashing primitives (hashlib.sha256 on passwords), and active TODO:/FIXME: markers.

      • get_dependency_tree: Parses direct and dev dependencies across Python (pyproject.toml, requirements.txt), Node.js (package.json), Rust (Cargo.toml), and Go (go.mod).

      • get_test_coverage: Ingests Cobertura XML (coverage.xml), LCOV (lcov.info), or JSON summaries, extracting exact uncovered line numbers.

      • get_file: Extracts verbatim code snippets with 1-indexed line numbers.

  4. Physical Re-verification:

    • The Verifier Agent re-fetches each cited line to confirm physical existence.

  5. Scored Report Synthesis:

    • The Report Generator applies weighted severity deductions to calculate an objective health score (0–100) and saves timestamped Markdown and standalone HTML reports to ./reports/.

🛡️ Zero-Hallucination Verification

LLM-based code reviewers frequently suffer from hallucinated line numbers, misquoted code, and phantom vulnerabilities. AutoAudit eliminates this failure mode through a defensive, multi-attempt verification engine:

Finding from Worker ──► [Attempt 1: Exact Line Check] ──► Match? ──► ACCEPT (Verified)
                               │ (No)
                               ▼
                        [Attempt 2: ±10 Line Proximity Scan] ──► Match? ──► ACCEPT (Line Auto-Corrected)
                               │ (No)
                               ▼
                        [Attempt 3: Full File Search] ──► Match? ──► ACCEPT (Line Auto-Corrected)
                               │ (No)
                               ▼
                        [REJECTED & DROPPED] ──► Logged in Hallucination Audit Trail

Empirical Defense Validation

During automated testing, the verification engine was challenged with deliberately corrupted data:

  • Injected Fabricated Claim: An imagined AWS secret key (AKIA_COMPLETELY_FABRICATED_TOKEN_99999) claimed to be at src/auth.py:99. The Verifier Agent scanned the file, failed to locate the snippet across all 3 attempts, rejected the finding, and excluded it from the report.

  • Injected Shifted-Line Claim: A genuine secret key with a shifted line number (claimed at line 20 instead of line 7). The Verifier Agent detected the discrepancy during the proximity scan, auto-corrected the line number to line 7, and accepted the finding.

Every generated report includes a Verification Audit Trail displaying the exact number of worker claims evaluated, verified on 1st attempt, auto-corrected, and rejected.

📺 Live External Demonstration

Here is the real output from running AutoAudit directly against an external public repository (https://github.com/psf/requests-html):

╭──────────────────── 🛡️ AutoAudit Autonomous Reviewer ────────────────────╮
│ Target: https://github.com/psf/requests-html                            │
│ Repository Name: requests-html                                          │
│ Mode: ⚡ Offline / Heuristic (0 API calls)                              │
│ Output Directory: C:\AutoAudit repo\reports                             │
╰─────────────────────────────────────────────────────────────────────────╯

[1/4] 🧭 Stage 1: Reconnaissance & Orchestrator Planning...
      ✔ Generated Audit Plan with 6 discrete tasks across 4 categories.

[2/4] 🔍 Stage 2: Specialized Worker Agents Evidence Gathering...
      ✔ Worker agents extracted 6 candidate findings with concrete file/line citations.

[3/4] 🛡️ Stage 3: Verifier Agent Zero-Hallucination Defense...
      ✔ Verification complete: 6 exact, 0 corrected, 0 rejected (100.0% acceptance rate).

[4/4] 📊 Stage 4: Health Scoring & Report Synthesis...
      ✔ Health Score: 10/100 (Grade F)
      ✔ Markdown Report: reports/audit_report_requests_html_20260911_191547.md
      ✔ HTML Report    : reports/audit_report_requests_html_20260911_191547.html

                    🏆 FINAL CODEBASE HEALTH SCORECARD
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Metric                       ┃ Value                                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Overall Health Score         │ 10 / 100                                 │
├──────────────────────────────┼──────────────────────────────────────────┤
│ Letter Grade                 │ Grade F (Critical Deficits)              │
├──────────────────────────────┼──────────────────────────────────────────┤
│ Total Verified Findings      │ 6                                        │
├──────────────────────────────┼──────────────────────────────────────────┤
│ Critical Severity Findings   │ 0 finding(s)                             │
├──────────────────────────────┼──────────────────────────────────────────┤
│ High Severity Findings       │ 6 finding(s) (-90 pts)                   │
├──────────────────────────────┼──────────────────────────────────────────┤
│ Medium Severity Findings     │ 0 finding(s)                             │
├──────────────────────────────┼──────────────────────────────────────────┤
│ Low Severity Findings        │ 0 finding(s)                             │
├──────────────────────────────┼──────────────────────────────────────────┤
│ Zero-Hallucination Rate      │ 100.0% verified (0 rejected)             │
├──────────────────────────────┼──────────────────────────────────────────┤
│ Markdown Report File         │ reports/audit_report_requests_html.md    │
├──────────────────────────────┼──────────────────────────────────────────┤
│ HTML Report File             │ reports/audit_report_requests_html.html  │
└──────────────────────────────┴──────────────────────────────────────────┘

Audit complete! Reports saved to: reports/

🚀 Installation & Usage

1. Installation

# Clone the repository
git clone https://github.com/razi2004/autoaudit-mcp.git
cd autoaudit-mcp

# Install dependencies and CLI package
pip install -r requirements.txt
pip install -e .

2. Basic Usage

# Run full autonomous audit against any GitHub repository
autoaudit run https://github.com/psf/requests-html

# Run audit on a local directory
autoaudit run ./my-project

# Force offline/heuristic mode (zero external API calls)
autoaudit run ./my-project --offline

# Specify custom report output directory
autoaudit run ./my-project --output-dir ./audit-deliverables/

# Output structured JSON report directly to stdout
autoaudit run ./my-project --json-only

3. Optional: LLM-Assisted Planning with Claude

AutoAudit operates 100% offline out of the box using deterministic heuristic rules and MCP tools to gather concrete evidence and generate findings.

If you provide an Anthropic API key, AutoAudit enables LLM-Assisted Planning: Claude analyzes the initial codebase reconnaissance to construct a customized, multi-category audit task plan and can optionally enrich finding descriptions with deeper contextual commentary.

# Set via environment variable
export ANTHROPIC_API_KEY="sk-ant-api03-..."

# Or create a .env file in project root
echo "ANTHROPIC_API_KEY=sk-ant-api03-..." > .env

Note on Core Evidence Gathering: Even when an API key is present, all actual finding discovery, file inspections, line extractions, and verifications remain 100% heuristic and tool-grounded via MCP tools—Claude is never used to generate ungrounded raw findings from thin air.

4. Running the FastMCP Server Standalone

The underlying Python MCP server can be launched independently to serve tools to Claude Desktop, Cursor, or custom MCP client agents:

autoaudit server
# or: python -m autoaudit.mcp.server

To configure with Claude Desktop, add to claude_desktop_config.json:

{
  "mcpServers": {
    "autoaudit": {
      "command": "python",
      "args": ["-m", "autoaudit.mcp.server"],
      "cwd": "/path/to/AutoAudit"
    }
  }
}

💡 Design Decisions

Architectural Decision

Rationale

Heuristic-First by Default

Core code scanning (regex secret matching, manifest tree parsing, Cobertura XML coverage analysis) is deterministic. Running in heuristic mode costs $0.00, runs in sub-second time, operates without internet access, and is 100% reproducible in CI/CD pipelines.

LLM as an Assistance Layer (Not Core Scanner)

Claude is used specifically where language models excel (decomposing high-level audit task strategy and synthesizing clear remediation advice). Core evidence gathering and finding generation are performed strictly via MCP tool calls (grep_repo, get_file, get_dependency_tree, get_test_coverage), preventing token bloat and eliminating ungrounded claims.

Physical Re-Verification Guard

No finding is trusted based on worker confidence alone. Every finding must survive an independent physical file read (get_file) before entering the final score calculation.

Zero-CDN Standalone HTML

Reports must be viewable offline, in air-gapped environments, and archived without broken stylesheet links. All styles and layout fonts are embedded directly into the HTML document.

Shallow Git Cloning

Uses git clone --depth 1 --single-branch into temporary directories to ensure minimal network overhead and zero leftover disk clutter after audit completion.

🧪 Test Suite & Quality Assurance

AutoAudit includes a comprehensive automated test suite covering all six phases:

pytest tests/ -v
======================================================== test session starts ========================================================
collected 48 items

tests/test_coverage_tools.py::test_get_test_coverage_with_xml PASSED                                                           [  2%]
tests/test_dependency_tools.py::test_get_dependency_tree_auto PASSED                                                           [  4%]
tests/test_findings_model.py::test_valid_audit_finding_creation PASSED                                                         [  6%]
tests/test_findings_model.py::test_finding_validation_errors PASSED                                                            [  8%]
tests/test_findings_model.py::test_findings_report_groupings PASSED                                                            [ 10%]
tests/test_fs_tools.py::test_list_files_basic PASSED                                                                           [ 12%]
tests/test_fs_tools.py::test_list_files_pattern_filter PASSED                                                                  [ 14%]
tests/test_fs_tools.py::test_get_file_full_and_slice PASSED                                                                    [ 17%]
tests/test_fs_tools.py::test_get_file_path_traversal_prevention PASSED                                                         [ 19%]
tests/test_fs_tools.py::test_grep_repo_literal_and_regex PASSED                                                                [ 21%]
tests/test_git_utils.py::test_is_git_url PASSED                                                                                [ 23%]
tests/test_git_utils.py::test_normalize_git_url PASSED                                                                         [ 25%]
tests/test_git_utils.py::test_extract_repo_name PASSED                                                                         [ 27%]
tests/test_git_utils.py::test_managed_repo_local PASSED                                                                        [ 29%]
tests/test_heuristic_workers.py::test_security_worker_ground_truth_secret_key PASSED                                           [ 31%]
tests/test_heuristic_workers.py::test_security_worker_insecure_hashing PASSED                                                  [ 34%]
tests/test_heuristic_workers.py::test_dead_code_worker_todo_and_unreferenced_function PASSED                                   [ 36%]
tests/test_heuristic_workers.py::test_dependency_worker_manifest_inspection PASSED                                             [ 38%]
tests/test_heuristic_workers.py::test_coverage_worker_identifies_uncovered_lines PASSED                                        [ 40%]
tests/test_heuristic_workers.py::test_worker_runner_full_plan_execution PASSED                                                 [ 42%]
tests/test_llm_worker_enhancer.py::test_security_worker_llm_enhancement PASSED                                                 [ 44%]
tests/test_mcp_server.py::test_tool_registry_and_schemas PASSED                                                                [ 46%]
tests/test_mcp_server.py::test_call_tool_dispatch PASSED                                                                       [ 48%]
tests/test_models.py::test_audit_task_creation_and_validation PASSED                                                           [ 51%]
tests/test_models.py::test_audit_plan_serialization_and_grouping PASSED                                                        [ 53%]
tests/test_planner.py::test_reconnaissance PASSED                                                                              [ 55%]
tests/test_planner.py::test_heuristic_fallback_plan PASSED                                                                     [ 57%]
tests/test_planner.py::test_planner_with_mocked_anthropic_api PASSED                                                           [ 59%]
tests/test_planner.py::test_planner_api_error_fallback PASSED                                                                  [ 61%]
tests/test_verifier.py::test_verifier_exact_match_pass PASSED                                                                  [ 63%]
tests/test_verifier.py::test_verifier_line_correction_retry PASSED                                                             [ 65%]
tests/test_verifier.py::test_verifier_fabricated_finding_rejected PASSED                                                       [ 68%]
tests/test_verifier.py::test_verifier_nonexistent_file_rejected PASSED                                                         [ 70%]
tests/test_verifier.py::test_verifier_full_report_batch PASSED                                                                 [ 72%]
tests/test_scoring.py::test_clean_score PASSED                                                                                 [ 74%]
tests/test_scoring.py::test_critical_secret_deduction PASSED                                                                   [ 76%]
tests/test_scoring.py::test_high_finding_deduction PASSED                                                                       [ 78%]
tests/test_scoring.py::test_mixed_severity_deductions PASSED                                                                   [ 80%]
tests/test_scoring.py::test_score_clamping_to_zero PASSED                                                                      [ 82%]
tests/test_report_generator.py::test_markdown_report_content PASSED                                                             [ 85%]
tests/test_report_generator.py::test_html_report_content_and_standalone PASSED                                                 [ 87%]
tests/test_report_generator.py::test_report_generator_file_saving PASSED                                                       [ 89%]
tests/test_cli_integration.py::test_cli_version PASSED                                                                         [ 91%]
tests/test_cli_integration.py::test_cli_plan_command PASSED                                                                    [ 93%]
tests/test_cli_integration.py::test_cli_run_command_offline_json PASSED                                                         [ 95%]
tests/test_cli_integration.py::test_cli_run_error_on_nonexistent_path PASSED                                                   [ 95%]
tests/test_cli_integration.py::test_audit_pipeline_direct_execution PASSED                                                      [ 97%]
tests/test_cli_integration.py::test_cli_mode_labels PASSED                                                                     [100%]

======================================================= 48 passed in 2.20s ========================================================

📄 License

MIT License • Built with Python & Model Context Protocol.

This server cannot be deployed

One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

Maintenance

ActivityMaintained
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers

  • Recon

    cappyeo
    A
    license
    Not graded
    quality
    C
    maintenance
    Recon indexes your codebase into a knowledge graph and exposes it via 14 MCP tools. AI agents get dependency mapping, blast radius analysis, safe multi-file rename, execution flow tracing, Cypher queries, semantic search, and PR review without reading every file. Supports 13 languages, live re-index in ~50ms, and zero config setup.
    5 npm
    5
    MIT

  • gbrain-code

    spacerocket3
    A
    license
    Not graded
    quality
    B
    maintenance
    Enables coding agents to generate question-scoped repository maps, inspect symbols and SQL lineage, and audit working-tree diffs for ripple candidates, all verified against the current Git commit.
    1
    Apache 2.0

  • Gumi-MCP

    igv8101
    A
    license
    Not graded
    quality
    B
    maintenance
    Enables AI agents to safely inspect a local repository's code and metadata while blocking private data from leaving the machine, providing read-only tools for search, change tracking, and integrity verification.
    MIT