👨‍💻 CodeMentor AI

An Autonomous Multi-Agent Pipeline That Solves, Critiques, Verifies, and Polishes Programming Code

CodeMentor AI is a state-of-the-art Model Context Protocol (MCP) server and Streamlit dashboard built to eradicate LLM hallucinations in competitive coding. By utilizing a linear state-machine verification pipeline, it moves beyond "single prompt solving" into rigorous adversarial peer-review.

Read the Kaggle Writeup • View Evaluation Metrics • Security Architecture

🎥 Demo

Note to Judges: The live video pitch and deployed application links will be placed here.

• 📺 Watch the 5-Minute YouTube Pitch Demo Output here

• 🌐 Try the Live Streamlit App here

Related MCP server: VSGuard MCP

🛑 The Problem Statement

Why do modern coding assistants hallucinate? Standard generic LLMs are autoregressive predictors, not engineers. When tasked with a dense LeetCode Hard problem, they frequently default to surface-level logic.

• Hidden Edge Cases: Single-shot prompts regularly fail to calculate bounds like integer overflows or $O(N^2)$ bottlenecks.

• Debugging Blindness: When AI-generated code fails, feeding the error back to the same monolithic agent often causes cyclic, oscillating hallucinations.

• Why Multi-Agent? We must segment the cognitive load. You would not deploy code without a peer review, a QA check, and a security audit. Your AI should not either.

💡 The Solution

CodeMentor AI introduces a deterministic, multi-agent swarm.

1. Multi-Agent Pipeline: Forces solutions through a sequenced pipeline.

2. Adversarial Reflection: A dedicated agent whose only job is to brutally critique the code.

3. Verification Layer: Acts as an air-gapped simulation proxy, mentally dry-running inputs.

4. Security Firewall: A strict O(1) memory limiter blocking prompt injections before API generation.

5. MCP Integration: Fully integrates all agents natively into IDEs (VS Code/Cursor).

✨ Key Features

📐 Architecture Diagrams

System Architecture

The top-level interaction between the user interface, the Security Firewall, and the LLM Pipeline.

graph TD
    A[User via Streamlit or IDE/MCP] -->|Payload| B(Security Firewall)
    B -->|Sanitized Valid Input| C{ManagerAgent Orchestrator}
    B --x|Prompt Injection Blocked| Z[Drop Connection]
    C --> D[True Pipeline]
    C --> E[Competitive Personas]
    C --> F[Classic Tools]

The True Agent Flow Pipeline

This diagram illustrates the State-Machine generator logic replacing the flawed "single LLM call".

sequenceDiagram
    participant Manager
    participant Solver
    participant Reflector
    participant Verification
    participant QA
    
    Manager->>Solver: Draft Algorithm
    Solver-->>Manager: V1 Code
    Manager->>Reflector: Try to break V1
    Reflector-->>Manager: Revised V2 Code
    Manager->>Verification: Mentally Dry-Run Inputs
    Verification-->>Manager: Verified / Passed
    Manager->>QA: Polish and Explain
    QA-->>Manager: Perfect Pydantic Data

IDE MCP Integration

graph LR
    IDE[VS Code / Cursor] <-->|JSON-RPC via stdio| MCP(FastMCP Server)
    MCP <--> Manager[ManagerAgent Router]
    Manager <--> Gemini[Google GenAI SDK]

🔌 MCP Integration Details

Model Context Protocol (MCP) allows your local IDEs to utilize CodeMentor's unique persona-driven logic natively. CodeMentor exposes the following precise tools:

🔒 Security Posture

AI security requires defense-in-depth methodologies. We do not rely on just prompting "Do not be malicious".

• Prompt Injection Firewall: Employs RegExp blacklists immediately rejecting known jailbreak inputs (ignore previous).

• Denial of Wallet (DoW) Limits: Strict string bounds mapping applied before the prompt touches the API.

• Session Abuse Detection: Rolling 60-second window limiting spam bot execution.

• Execution Proxy: We utilize semantic Agent dry-runs rather than exposing native eval or exec OS vectors.

graph LR
    Input[Payload] --> Bound[Length Check]
    Bound --> Regex[Heuristic Reject]
    Regex --> RateLimit[Abuse Track]
    RateLimit --> LLM[Execution]

📊 Quantitative Benchmarks

Metrics context: Benchmarking executed via simulated Leetcode Hard parameters comparing zero-shot execution versus the V2 Reflection Pipeline.

See EVALUATION_METRICS.md for our raw execution trace outputs and methodology.

📸 Presentation & Screenshots

The Timeline Dashboard

The Socratic Mock Interview

VS Code MCP Execution

Security Attack Mitigation

🚀 Installation & Local Setup

1. Repository Clone & Environment

git clone https://github.com/yourusername/codementor-ai.git
cd codementor-ai

# Python 3.11+ is strongly recommended
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt

2. Environment Variables

Copy the template and insert your GEMINI_API_KEY:

cp .env.example .env

3. Execution (Docker & Native)

Running the Web Interface (Native Streamlit):

streamlit run frontend/app.py

Running inside secure Docker containers:

docker-compose up --build

Running the MCP Server for your local IDE:

python -m mcp.server

📂 Project Structure

codementor-ai/
├── agents/                  # The Multi-Agent Intelligence Core
│   ├── manager_agent.py     # Pipeline State-Machine Router
│   ├── reflection_agent.py  # Adversarial Code Critique Component
│   ├── verification_agent.py# Code fact-checking proxy
│   ├── strategy_agent.py    # Competitive Programming Guide
│   └── (..other agents)
├── core/                    # System Integrations
│   ├── config.py            # Pydantic Settings validator
│   └── security.py          # Strict Firewall & Rate Limit logic
├── frontend/
│   └── app.py               # Glassmorphic Streamlit SaaS
├── mcp/
│   └── server.py            # FastMCP native IDE extension bindings
├── .env.example
├── docker-compose.yml
├── requirements.txt
└── README.md

🛣️ Roadmap

• Abstract initial AI logic into Pydantic structured schemas.

• Create a multi-stage generator state machine (run_pipeline).

• Deploy the Model Context Protocol (MCP) integrations.

• Build the Memory/Abuse Security Firewall.

• Connect a true virtualized sub-process REPL (e.g., gVisor) for compilation testing.

• Implement Session History export to Cloud Storage (AWS S3/GCP).

🤝 Contributing

We welcome competitive programmers, ML researchers, and open-source contributors to the CodeMentor ecosystem!

1. Fork the Project.

2. Create your Feature Branch (git checkout -b feature/AmazingAgent).

3. Commit your Changes (git commit -m 'Added memory constraint agent').

4. Push to the Branch (git push origin feature/AmazingAgent).

5. Open a Pull Request.

Please ensure any new Agent inherits from agents.base_agent and defines a strict Pydantic Output schema.

📄 License

Distributed under the MIT License. See LICENSE for more information.

🙏 Acknowledgements

• Google Gemini: For the powerhouse reasoning backing the multi-agent ensemble.

• Model Context Protocol (MCP): For the standard enabling our IDE extensibilities.

• Streamlit: For the rapid modern dashboard frontend pipeline.

• Kaggle: For catalyzing this Capstone design standard.