CodeGuard RAG MCP Server

A Python code defect and vulnerability diagnosis platform based on RAG + MCP

CodeGuard accepts Python errors, tracebacks, or code snippets; after static feature extraction and Dense + BM25 hybrid retrieval, it returns the issue classification, vulnerability type, CWE, risk level, supporting evidence, root cause, remediation suggestions, secure code, and verification methods. User code is only parsed, never executed.

The current version is personal project M1: it retains the original project's modular RAG, ChromaDB, BM25, RRF, optional Rerank, MCP Server, Streamlit Dashboard, and observability stack, while focusing the core scenario on code defect and security vulnerability diagnosis.

Project Positioning

The project addresses two types of input:

• Runtime errors: e.g., TypeError, KeyError, ImportError; outputs the defect root cause and remediation steps.

• Dangerous code: e.g., shell=True, eval(), insecure deserialization; outputs the vulnerability type, CWE, and secure coding practices.

M1 supports Python only. It is an assistive diagnostic tool; it does not replace manual code review, nor does it claim to have integrated Bandit or Semgrep or to be able to find all vulnerabilities.

Related MCP server: Lanalyzer MCP Server

Core Capabilities

• Static input parsing: extracts exception types, traceback file and line numbers, dangerous APIs, and key symbols.

• Structured security knowledge base: contains 30 Schema-validated Python defect, vulnerability, configuration, and dependency cases.

• Hybrid retrieval: Dense Embedding handles semantic matching; BM25 handles exact matching of exception names, APIs, CWE, and so on.

• Deterministic diagnosis: generates structured reports from retrieved evidence; when there is no direct code evidence, it lowers the confidence of security conclusions.

• MCP integration: exposes a unified diagnostic capability to MCP Clients through diagnose_code_issue.

• Dual-format output: returns both easy-to-read Chinese Markdown and program-consumable JSON.

• Offline regression: core tests use fixed Embeddings and fixed retrieval results, without relying on external model APIs.

System Architecture

报错 / traceback / Python 代码
              │
              ▼
    SecurityInputParser
   异常、位置、危险模式、符号
              │
              ▼
    SecurityQueryBuilder
 精确词 + 安全语义扩展 + CWE
              │
       ┌──────┴──────┐
       ▼             ▼
Dense Retrieval   BM25 Retrieval
ChromaDB/cosine   关键词精确召回
       └──────┬──────┘
              ▼
         RRF Fusion
              │
        Optional Rerank
              │
              ▼
      DiagnosticService
  分类、证据、置信度、修复方案
              │
              ▼
 diagnose_code_issue (MCP)
      Markdown + JSON 报告

Key code locations:

• src/security/analysis/: input parsing and retrieval query construction.

• src/security/loaders/: JSON/JSONL security case loading and validation.

• src/security/ingestion/: dual-index writes to ChromaDB and BM25.

• src/security/services/: diagnosis orchestration, classification, and degradation strategy.

• src/mcp_server/tools/diagnose_code_issue.py: MCP tool and report format.

• knowledge/security_cases.json: the M1 security knowledge base.

Security Case Data Model

Each case contains case_id, issue_kind, error_type, vulnerability_type, cwe, severity, symptoms, dangerous patterns, root cause, vulnerable code, remediation plan, secure code, verification methods, and reference sources.

The knowledge base supports JSON arrays and JSONL. During ingestion, one case generates one stable Chunk, and case_id is used as the document identifier in both ChromaDB and BM25, avoiding misalignment of the two result sets.

The 30 cases in M1 consist of:

• 8 general code defects

• 17 security vulnerabilities

• 3 configuration risks

• 2 dependency risks

PDF and JSON Processing

The CodeGuard main knowledge base prefers JSON/JSONL because CWE, risk level, and remediation suggestions require stable structured fields. The original PDF ingestion pipeline is retained, suitable for later importing security standards, vulnerability reports, or internal documents:

1. Use SHA256 to determine whether a file has already been processed.

2. Use MarkItDown to convert PDF text to Markdown.

3. Use PyMuPDF to extract images, save them to data/images/, and write [IMAGE: id] placeholders.

4. Optionally use a Vision LLM to generate descriptions for images; on failure, degrade to plain-text processing.

5. Chunk the document and enrich it with Metadata.

6. Write to both the Dense vector store and the BM25 index.

PDF is the general document retrieval entry point; knowledge/security_cases.json is the primary trusted basis for current diagnosis results.

Dense + BM25 + RRF + Rerank

Here, Dense Retrieval is not a specific algorithm name but a category of semantic vector retrieval:

• EmbeddingFactory selects DashScope, OpenAI, Azure OpenAI, or Ollama Embedding according to config/settings.yaml.

• Text vectors are written to a ChromaDB HNSW collection with cosine distance.

• Query vectors and case vectors are recalled by cosine similarity.

The other path uses BM25 for sparse retrieval over keywords such as TypeError, subprocess.run, shell=True, and CWE-78. RRF (Reciprocal Rank Fusion) merges exactly the Dense semantic retrieval ranking and the BM25 keyword retrieval ranking, with a default rrf_k=60. After fusion, a Cross-Encoder or LLM Rerank can be enabled by configuration; M1 disables Rerank by default for low-cost local operation.

Quick Start

The commands below target Windows PowerShell and require Python 3.11+.

cd <project-directory>
py -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install -U pip
python -m pip install -e ".[dev]"

The project uses DashScope's OpenAI-compatible interface by default: the LLM is qwen3.7-plus, the Embedding is qwen3.7-text-embedding (1024 dimensions), and the Base URL is https://dashscope.aliyuncs.com/compatible-mode/v1. The API Key is read only from the local environment variable DASHSCOPE_API_KEY and must never be written into the repository, settings.yaml, or logs.

$env:DASHSCOPE_API_KEY="<仅在本机设置，不要写入仓库>"
python scripts\check_dashscope_connectivity.py

The connectivity check above is executed explicitly: it sends one short LLM request and one single-text Embedding request. Normal startup and Dashboard readiness only check the local configuration and the knowledge base, and do not consume model quota.

• The default ChromaDB directory is data/db/chroma.

• The security case BM25 index directory is data/db/bm25/code_security_cases.

• The Base URL can be overridden via DASHSCOPE_BASE_URL, suitable for later migration to a dedicated business-space domain.

First run the basic checks that require no API Key:

python main.py
python -m pytest tests\unit\security tests\unit\test_diagnose_code_issue.py -v

Import Security Cases

On first use, or after switching the Embedding model/dimensions, rebuild the security case collection with the current DashScope Embedding:

python scripts\ingest_security_cases.py --rebuild

--rebuild only rebuilds code_security_cases and its security_ BM25 index; it does not delete other collections or the entire database directory. Ingestion calls the Embedding service and consumes tokens; a successful run should output non-zero counts of cases, Chunks, and vectors. The knowledge base stores the provider, model, and dimensions identifiers of the current Embedding. If they do not match an existing non-empty collection, the system will require an explicit rebuild to avoid mixing old vectors.

Start the MCP Server

python -m src.mcp_server.server

For the MCP Client startup configuration, you can use:

{
  "command": "<project-directory>\\.venv\\Scripts\\python.exe",
  "args": ["-m", "src.mcp_server.server"],
  "cwd": "<project-directory>"
}

Example core tool input:

{
  "name": "diagnose_code_issue",
  "arguments": {
    "error_message": "",
    "code_snippet": "subprocess.run(user_input, shell=True)",
    "language": "python",
    "top_k": 5
  }
}

The server also retains query_knowledge_hub, list_collections, and get_document_summary, making it convenient to inspect and reuse the original RAG capabilities.

Start the Dashboard

python -m streamlit run src\observability\dashboard\app.py

The Dashboard opens the "Vulnerability Diagnosis" page by default. It supports pasting Python errors or code snippets, or uploading a single UTF-8 .py file, and can download Markdown/JSON reports. Uploaded content is parsed only in memory; it is neither saved nor executed. After clicking "Diagnose", the input error or code and the retrieved case context are sent to DashScope to generate a Qwen-enhanced remediation explanation; diagnosis also consumes tokens. Do not submit keys, personal data, or production secrets that should not be sent to third-party services.

Embedding can be configured later: when Embedding is not configured, or code_security_cases has not been imported yet, the page still opens normally, but it will prompt you to first complete the configuration and run:

python scripts\ingest_security_cases.py --rebuild

No mock diagnosis results are generated in this state.

Diagnosis Example

Input:

subprocess.run(user_input, shell=True)

Expected core results:

• Classification: security_vulnerability

• Type: Command Injection

• CWE: CWE-78

• Risk level: critical

• Evidence: subprocess-shell

• Remediation: disable shell=True, use a parameter array and allowlist validation

• Similar case: PY-SEC-002

See the complete example in docs/examples/codeguard-diagnosis-example.md.

Testing and Evaluation

python -m pytest tests\unit\security tests\unit\test_diagnose_code_issue.py `
  tests\integration\test_security_case_ingestion.py `
  tests\e2e\test_codeguard_diagnosis.py -v

python -m ruff check src\security `
  src\mcp_server\tools\diagnose_code_issue.py `
  scripts\ingest_security_cases.py `
  tests\unit\security `
  tests\unit\test_diagnose_code_issue.py `
  tests\e2e\test_codeguard_diagnosis.py

A plain python -m pytest runs only offline tests by default and automatically removes the DashScope, OpenAI, and Azure OpenAI API Keys visible to the test process and its child processes. Test cases that call real model services are uniformly marked as llm and must be executed explicitly; for example:

python -m pytest -m llm tests\integration\test_chunk_refiner_llm.py -v

Run the above command only when you are prepared to consume real model quota. The currently supported and verified minimum client versions are chromadb>=1.5.9 and openai>=2.46.0.

Current verification covers the data model, case validation, static parsing, query expansion, dual-index writes, deterministic diagnosis, MCP registration, and offline end-to-end output. The project retains the original Ragas/Custom evaluation modules, but M1 does not provide accuracy figures that have not been validated by actual experiments.

Limitations and Future Directions

• M1 only parses Python and does not execute the code under diagnosis.

• The current dangerous patterns are an explainable rule set, not a complete SAST.

• Real Dense retrieval requires an available Embedding Provider; without an API Key, MCP initialization and tools/list still work, but actual hybrid retrieval returns a readable configuration error.

• When there are no retrieval results, it returns degraded=true with confidence 0.0, prompting for additional context.

• When there is only knowledge base similarity but no matching static code evidence, it does not directly conclude a vulnerability; it returns degraded=true and confidence 0.0.

• M1 confidence uses explainable evidence tiers and does not interpret the heterogeneous raw scores of RRF, BM25, or cosine directly as probabilities.

• Raw code is not directly concatenated into remote Embedding queries; common API Keys, Tokens, Passwords, and Bearer credentials in exceptions are masked first.

• Future work can add file/repository scanning, Bandit/Semgrep result normalization, Golden Test Set metrics, and multi-language support.

Resume Description Reference

Independently designed and implemented a Python code defect and vulnerability diagnosis platform based on RAG + MCP; built a knowledge base of 30 structured security cases and a JSON/JSONL validated ingestion pipeline; adopted Dense Embedding + BM25 dual-path retrieval, RRF fusion, and optional Rerank, combining static dangerous-pattern evidence to output CWE, risk level, root cause, and remediation plans; exposed a standardized diagnosis tool through MCP; and used Unit / Integration / E2E offline tests to verify the full ChromaDB, BM25, and MCP stdio pipeline.

In the resume, only write about features you have actually run, understood, and can explain; do not fill in unmeasured improvement percentages.