Skip to main content
Glama
vamsi-desu

Git RAG MCP Server

by vamsi-desu
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Local

Git RAG MCP Server

A practical MCP (Model Context Protocol) project that connects an LLM to multiple custom MCP servers.

The project demonstrates how an MCP client can communicate with specialized servers for:

  • GitHub operations

  • Employee document RAG

  • QA and test-case generation

The goal of this project is to understand and demonstrate MCP architecture, tool calling, resources, prompts, RAG retrieval, and multi-server communication.

Architecture

                         ┌──────────────────┐
                         │       UI         │
                         └────────┬─────────┘
                                  │
                                  ▼
                         ┌──────────────────┐
                         │    MCP Client    │
                         │                  │
                         │ Tool discovery   │
                         │ Tool calling     │
                         │ LLM interaction  │
                         └────────┬─────────┘
                                  │
                 ┌────────────────┼────────────────┐
                 │                │                │
                 ▼                ▼                ▼
        ┌────────────────┐ ┌──────────────┐ ┌──────────────┐
        │ GitHub Server  │ │ RAG Server   │ │ QA Server    │
        └───────┬────────┘ └──────┬───────┘ └──────────────┘
                │                 │
                ▼                 ▼
          GitHub REST API    Employee PDF
                              TF-IDF Search

Project Structure

git-mcp-server/
│
├── client/
│   └── client.py
│
├── servers/
│   ├── git_server.py
│   ├── rag_server.py
│   └── qa_server.py
│
├── data/
│   └── Employee_Details.pdf
│
├── .env
├── pyproject.toml
├── uv.lock
└── README.md

MCP Servers

Related MCP server: dev-mcp

1. GitHub Server

File:

servers/git_server.py

The GitHub server communicates with the GitHub REST API using an authenticated GitHub token.

It provides tools for:

  • Getting the authenticated GitHub user

  • Listing repositories

  • Getting repository information

  • Searching issues

  • Reading file content

  • Listing repository files

  • Searching repository code

Available Tools

get_github_user()
list_repositories()
get_repository(owner, repo)
search_issues(query, top_k)
get_file_content(owner, repo, path, ref)
list_files(owner, repo, path, ref)
search_code(owner, repo, query, top_k)

2. RAG Server

File:

servers/rag_server.py

The RAG server searches employee information stored in:

data/Employee_Details.pdf

The server:

  1. Extracts text from the PDF

  2. Identifies employee records

  3. Creates employee chunks

  4. Builds a TF-IDF vector index

  5. Uses cosine similarity for retrieval

  6. Combines TF-IDF similarity with keyword matching

  7. Returns the most relevant employee records

Available Tools

search_employees(query, top_k)
show_chunks(start, count)
get_chunk_count()

Example

User:
Find information about Ravi Rao

The MCP client can call:

search_employees(
    query="Ravi Rao",
    top_k=5
)

The RAG server returns the most relevant employee records with relevance scores.

3. QA Server

File:

servers/qa_server.py

The QA server provides testing capabilities for the MCP project.

It is designed to help test the GitHub and RAG servers rather than duplicate their functionality.

MCP Tools

create_test_case()

Creates a standardized test case.

Example:

create_test_case(
    name="Search code for a known term",
    tool_name="search_code",
    input_data="...",
    expected_result="..."
)

Example output:

TEST CASE

Name: Search code for a known term
Tool: search_code

Input:
...

Expected Result:
...

Status: Not Executed

validate_github_result()

Performs basic validation of results returned by GitHub tools.

Supported operations include:

get_github_user
list_repositories
get_repository
search_issues
get_file_content
list_files
search_code

The validator checks whether expected fields are present and returns:

PASS

or:

FAIL

validate_rag_result()

Performs basic validation of results returned by RAG tools.

Supported operations include:

search_employees
show_chunks
get_chunk_count

QA Resource

The QA server exposes:

qa://mcp-test-checklist

The resource contains a reusable checklist covering:

  • GitHub server testing

  • RAG server testing

  • MCP integration testing

  • Invalid inputs

  • Empty results

  • Authentication errors

  • Tool discovery

  • Tool argument handling

QA Prompt

The QA server exposes:

generate_mcp_test_plan(component)

It generates a reusable prompt for creating test plans.

The generated test plan includes:

  • Positive test cases

  • Negative test cases

  • Boundary cases

  • Invalid input cases

  • Empty-result cases

  • Error cases

  • MCP integration cases

  • Expected results

MCP Primitives Demonstrated

This project demonstrates the three major MCP server primitives.

Tool
  ↓
Performs an action

Resource
  ↓
Provides read-only context

Prompt
  ↓
Provides reusable prompt templates

The project therefore demonstrates:

GitHub Server
 └── Tools

RAG Server
 └── Tools

QA Server
 ├── Tools
 ├── Resource
 └── Prompt

Environment Variables

Create a .env file in the project root.

Example:

GITHUB_TOKEN=your_github_token

The GitHub server reads the token from the environment and uses it for authenticated GitHub API requests.

Do not commit .env to Git.

Add:

.env

to .gitignore.

Installation

This project uses uv for Python environment and dependency management.

Create/sync the environment:

uv sync

Run Python through the project environment:

uv run python ...

Running the Servers

The MCP servers use stdio communication.

For example:

uv run python servers/qa_server.py

The server will wait for an MCP client.

This is expected behavior.

The server should normally be launched by the MCP client rather than manually interacted with from the terminal.

Running the MCP Client

Run:

uv run python client/client.py

The client connects to the configured MCP servers and discovers their available capabilities.

The general flow is:

Client starts
     ↓
Connect to MCP servers
     ↓
Discover tools/resources/prompts
     ↓
Send available capabilities to LLM
     ↓
User asks a question
     ↓
LLM decides whether a tool is required
     ↓
MCP client executes the requested tool
     ↓
Tool result returned
     ↓
LLM generates final response

Example: QA Test Case

User:

Create a test case for the search_code tool.

The LLM can select:

create_test_case

with arguments such as:

{
  "name": "Search code for a known term",
  "tool_name": "search_code",
  "input_data": "...",
  "expected_result": "..."
}

The QA server returns the formatted test case.

Example: Employee Search

User:

Find Ravi Rao

The client can select:

search_employees

The RAG server performs retrieval against the employee PDF.

Question
   ↓
search_employees
   ↓
TF-IDF retrieval
   +
keyword matching
   ↓
Relevant employee records
   ↓
MCP client
   ↓
LLM response

Example: GitHub Search

User:

Search for MCPServer in my repository.

The client can call:

search_code(
    owner="...",
    repo="...",
    query="MCPServer",
    top_k=10
)

The GitHub server communicates with the GitHub API and returns matching code results.

Example: QA Validation

A GitHub operation can be followed by QA validation:

search_code
     ↓
GitHub result
     ↓
validate_github_result
     ↓
PASS / FAIL

Similarly:

search_employees
     ↓
RAG result
     ↓
validate_rag_result
     ↓
PASS / FAIL

This demonstrates the concept of using multiple MCP capabilities together.

Technologies Used

  • Python

  • Model Context Protocol (MCP)

  • MCP Python SDK

  • uv

  • GitHub REST API

  • httpx

  • python-dotenv

  • PyPDF

  • scikit-learn

  • TF-IDF

  • Cosine similarity

What This Project Demonstrates

This project is intended as a practical MCP learning project.

It demonstrates:

MCP

  • MCP client

  • MCP servers

  • Tool discovery

  • Tool calling

  • Resources

  • Prompts

  • stdio transport

  • Multiple MCP servers

GitHub Integration

  • Authentication

  • GitHub REST API requests

  • Repository operations

  • Issue search

  • File operations

  • Code search

RAG

  • PDF ingestion

  • Document chunking

  • TF-IDF indexing

  • Similarity search

  • Keyword matching

  • Retrieval results

QA

  • Test-case generation

  • Result validation

  • QA checklists

  • Reusable test-plan prompts

Current Limitation

The QA server's create_test_case() tool currently creates a test definition but does not execute the test automatically.

For example:

create_test_case()
        ↓
Test case created
        ↓
Status: Not Executed

Automatic execution would require the MCP client to orchestrate multiple tool calls:

create test
      ↓
execute target tool
      ↓
capture result
      ↓
validate result
      ↓
PASS / FAIL

This can be implemented as a future enhancement.

Future Improvements

Possible next improvements include:

  1. Automatic multi-tool QA workflows

  2. Automatic test execution

  3. PASS/FAIL test reports

  4. Test history

  5. ChatGPT-style UI

  6. More GitHub operations

  7. Better RAG retrieval

  8. Source/chunk citations

  9. Automated regression testing

  10. More MCP servers

Learning Objective

The main objective of this project is not to build a production GitHub client or production RAG system.

The objective is to understand how an LLM application can use MCP to connect to multiple specialized capabilities.

The core architecture is:

                 LLM
                  │
                  ▼
             MCP Client
                  │
       ┌──────────┼──────────┐
       ▼          ▼          ▼
    GitHub       RAG        QA
    Server      Server     Server
       │          │          │
       ▼          ▼          ▼
   GitHub API   PDF/RAG    Testing

Each server has a clear responsibility, while the MCP client acts as the central communication layer.

Available Tools

3 tools
create_test_caseC

Create a standardized MCP test case.

Args: name: Short name of the test.

tool_name:
    MCP tool being tested.

input_data:
    Input that should be passed to the tool.

expected_result:
    Expected behavior/result.
ParametersJSON Schema
NameRequiredDescriptionDefault
nameYes
tool_nameYes
input_dataYes
expected_resultYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

C2.9/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It says 'Create' (a mutation) but does not disclose where the case is persisted, whether it is durable, whether duplicates are allowed, or what permissions are needed. For a write tool with zero annotation coverage this is a significant gap.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is short and front-loaded with the core action, followed by a clean parameter list. It is slightly padded by restating obviously-named fields, but nothing is wasted at length.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be explained. Still, for a mutation tool with no annotations and 0% schema coverage, the description should cover persistence/side-effect and uniqueness behavior, which it omits.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description is the only documentation for the four parameters, and it does name and gloss each one. However the glosses are minimal and largely echo the parameter names ('name: Short name of the test'), adding little beyond the schema structure.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Create a standardized MCP test case.' It clearly distinguishes itself from the validate_* siblings by action, though it doesn't explicitly say how it differs from them beyond the verb.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use this tool versus the validate_github_result and validate_rag_result siblings, nor any prerequisites such as whether a test suite must already exist. The agent must infer usage entirely.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

validate_github_resultB

Perform basic validation of a GitHub MCP tool result.

Args: operation: GitHub operation being tested.

result:
    Result returned by the GitHub MCP server.
ParametersJSON Schema
NameRequiredDescriptionDefault
resultYes
operationYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

B3.1/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral-disclosure burden. It says only "basic validation," without explaining what is checked, what triggers failure, whether the tool has side effects, or any limitations; this is insufficient for a validation operation.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded, brief, and well-structured with an Args section. Every sentence contributes to understanding the tool and its inputs, with no redundant or filler text.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return-value explanation is not required. However, with no annotations and only a vague "basic validation" description, the tool's behavioral contract is under-specified for an agent deciding whether and how to call it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must supply parameter meaning. It does define both parameters (operation = GitHub operation being tested; result = result returned by the GitHub MCP server), but adds no format, examples, or expected value details.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ("validate") and resource ("GitHub MCP tool result"), making the core purpose clear. It is distinguishable from sibling validate_rag_result by the GitHub focus, though it does not explicitly name that alternative.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives no explicit guidance on when to use this tool versus alternatives such as validate_rag_result or create_test_case. Usage is only implied by the phrase "operation being tested," which is not enough to route an agent reliably.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

validate_rag_resultC

Perform basic validation of a RAG MCP tool result.

Args: operation: RAG operation being tested.

result:
    Result returned by the RAG MCP server.
ParametersJSON Schema
NameRequiredDescriptionDefault
resultYes
operationYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
resultYes

TDQS

C2.8/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full behavioral burden. 'Basic validation' is undefined: it does not say which checks run, whether a failed result raises an error or returns a negative verdict, or whether the tool is read-only. This is a significant gap for a validation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Short and front-loaded: one purpose sentence followed by a tidy Args block. Nothing is padded, though the Args entries are so terse they add little information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return values need not be described. However, with no annotations, no enum on 'operation', and 0% schema coverage, the definition does not give an agent enough to construct a correct call or predict validation outcomes.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It only restates the parameter names ('operation: RAG operation being tested', 'result: Result returned by the RAG MCP server') without giving accepted operation values, string formats, or whether 'result' is raw text or serialized JSON.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb and resource ('validate a RAG MCP tool result'), which distinguishes it from the sibling validate_github_result by target system. It stops short of explaining what 'basic validation' actually checks, so the boundary of its behavior is fuzzy.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to call this versus validate_github_result, nor any precondition such as having just invoked a RAG operation. The Args block documents inputs but not invocation context, leaving usage to inference.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 3 tool updatesv0.1.0
    • First observedcreate_test_case
    • First observedvalidate_github_result
    • First observedvalidate_rag_result
Deploy Server
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

TDQS

B3.2/5.0

Scored across 3 tools

Disambiguation4/5

Create_test_case is distinct in purpose from the two validate_* tools. The two validators are parallel and could be momentarily confused, but their names clearly indicate the GitHub vs RAG domain, so boundaries are mostly clear.

Naming Consistency5/5

All tool names follow a consistent snake_case verb_noun pattern: create_test_case, validate_github_result, validate_rag_result. The convention is predictable and uniform.

Tool Count4/5

Three tools is a small but reasonable set for a focused testing utility. It is slightly on the low side, but each tool has a clear role and the count does not feel excessive or arbitrarily inflated.

Completeness2/5

The domain is a testing harness for MCP tools, yet there is no tool to run a test case, list/retrieve existing test cases, or update/delete them. Validation covers only GitHub and RAG results, leaving other potential MCP results unsupported, which creates significant gaps for realistic workflows.

Maintenance

ActivityMaintained
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers

  • dev-mcp

    CodeIbrah
    A
    license
    Not graded
    quality
    B
    maintenance
    A local MCP server that gives AI agents access to developer tooling — GitHub (read-only), documentation search, and web research — via stdio transport.
    MIT

  • GitHub MCP Toolkit

    kartik-012
    A
    license
    Not graded
    quality
    B
    maintenance
    A production-grade MCP server that provides LLMs with safe, structured, tool-based access to GitHub repositories, including issue management, semantic search, and guarded write operations.
    MIT

  • github-mcp

    awernick
    F
    license
    A
    quality
    D
    maintenance
    MCP server exposing GitHub tools for issues, pull requests, and code browsing via the GitHub REST API. Designed for local LLM clients with flat arguments and streamable HTTP support.
    15
    -