Skip to main content
Glama
KrishnaMuddala

MCP Chat

by KrishnaMuddala
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python
Local

MCP Chat

Note: This project is an extension of the MCP certification project originally sourced from the Anthropic Claude MCP course. It has been extended to support local LLMs instead of the Anthropic API, along with additional tooling and improvements.

A command-line interface application that enables interactive chat with a local LLM (via Ollama or any OpenAI-compatible server) using the MCP (Model Context Protocol) architecture for document management.

Prerequisites

  • Python 3.9+

  • Ollama or any OpenAI-compatible local LLM server

Setup

Step 1: Configure environment variables

Create a .env file in the project root:

LOCAL_LLM_MODEL=llama3.2
LOCAL_LLM_BASE_URL=http://localhost:11434/v1
USE_UV=1   # Set to 0 if not using uv

Step 2: Install dependencies

Option 1: With uv (Recommended)

uv is a fast Python package installer and resolver.

pip install uv
uv venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate
uv pip install -e .

Option 2: Without uv

python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate
pip install openai python-dotenv prompt-toolkit "mcp[cli]==1.8.0"

Step 3: Start your local LLM

# Ollama example
ollama serve
ollama pull llama3.2

Step 4: Run the project

# With uv
uv run main.py

# With additional MCP servers
uv run main.py extra_server.py another_server.py

Usage

Basic Chat

Type your message and press Enter:

> What is the state of the condenser tower?

Document Retrieval

Use @ followed by a document ID to include its contents in your query:

> Tell me about @deposition.md
> Summarize @financials.docx

Demo

MCP Chat CLI demo MCP Chat running with llama3.2 via Ollama on Windows

Commands

Use / prefix to execute MCP prompts. Press Tab to autocomplete:

> /format deposition.md
> /summarize report.pdf

Available Documents

Document

Description

deposition.md

Testimony of Angela Smith, P.E.

report.pdf

State of a 20m condenser tower

financials.docx

Project budget and expenditures

outlook.pdf

Projected future performance

plan.md

Project implementation steps

spec.txt

Technical equipment requirements

Architecture

main.py
  ├── MCPClient          # Manages stdio communication with MCP server(s)
  ├── mcp_server.py      # FastMCP server — tools, resources, prompts
  ├── core/claude.py     # Local LLM integration (OpenAI-compatible)
  ├── core/cli_chat.py   # Chat logic with @ and / command handling
  └── core/cli.py        # Terminal UI with Tab autocomplete

MCP Server Features

Feature

Name

Description

Tool

read_doc_contents

Read the contents of a document by ID

Tool

edit_document

Replace text within a document

Resource

docs://documents

List all available document IDs

Resource

docs://documents/{doc_id}

Fetch contents of a specific document

Prompt

/format

Rewrite a document in Markdown format

Development

Adding New Documents

Edit the docs dictionary in mcp_server.py:

docs = {
    "your_doc.md": "Your document content here",
}

Adding New MCP Servers

Pass additional server scripts as arguments when running:

uv run main.py your_custom_server.py

Adding New Tools / Prompts / Resources

Use the FastMCP decorators in mcp_server.py:

@mcp.tool(name="my_tool", description="Does something useful")
def my_tool(input: str) -> str:
    return f"Processed: {input}"

Known Limitations

  • Document edits are in-memory only and lost on server restart

  • No persistent storage backend

  • No authentication or multi-user support (stdio only — single client per server instance)

Troubleshooting

OneDrive hardlink error on Windows:

$env:UV_LINK_MODE = "copy"; uv pip install -e .

Local LLM not responding:

  • Ensure Ollama is running: ollama serve

  • Confirm the model is pulled: ollama pull llama3.2

  • Check LOCAL_LLM_BASE_URL in .env matches your server

Module not found errors:

uv add openai python-dotenv mcp

Available Tools

2 tools
edit_documentA

Edit a document by replacing a string in the documents content with a new string

ParametersJSON Schema
NameRequiredDescriptionDefault
doc_idYesId of the document that will be edited
old_strYesThe text to replace. Must match exactly, including whitespace
new_strYesThe new text to insert in place of the old text

TDQS

A3.5/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must fully disclose behavior. It states that a string is replaced, but does not clarify whether all occurrences are replaced or only the first, nor does it mention that the edit is permanent/mutating, any side effects, or error conditions. This ambiguity is a significant gap for a mutation tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, compact sentence with no filler or redundant restatement of the tool name. It is front-loaded with the action and resource, and every word contributes meaning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple 3-parameter tool, the description plus full schema coverage is mostly sufficient for an agent to make a call. However, the missing occurrence behavior (all vs. first match) and the absence of an output schema leave some uncertainty about the tool's actual effect and return value, making it minimally complete rather than fully contextualized.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema covers 100% of the parameters with clear descriptions. The tool description essentially paraphrases the schema (old_str replaced by new_str) without adding new semantic meaning, such as constraints or examples. Baseline 3 is appropriate because the schema carries the burden.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Edit'), names the exact resource ('a document'), and specifies the mechanism ('replacing a string ... with a new string'). This clearly distinguishes it from the sibling read_doc_contents, which is a read operation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage: when a document's content must be modified via string replacement. However, it provides no explicit guidance about when NOT to use it or when to prefer the sibling read_doc_contents instead. The alternative is only inferable from the tool name, not the description.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

read_doc_contentsA

Read the contents of a document and return it as a string.

ParametersJSON Schema
NameRequiredDescriptionDefault
doc_idYesId of the document to read

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden of behavioral disclosure. It does state the output behavior ('return it as a string'), but does not mention error handling, permissions, or what happens when the document does not exist. For a simple read operation, this is adequate but not rich.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no wasted words. It efficiently conveys both the operation and the return format.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

This is a simple tool with one parameter, full schema coverage, and no output schema. The description is largely complete for a read operation, especially since it explicitly states the return type. It could be slightly stronger with error or permission context, but nothing essential is missing for basic invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema already fully documents the only parameter, doc_id, as 'Id of the document to read', so schema coverage is 100%. The description adds no additional parameter meaning beyond the schema, matching the baseline score.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb and resource: 'Read the contents of a document' and explicitly notes the return type as a string. This clearly distinguishes it from the sibling 'edit_document', which implies modification rather than reading.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The verb 'Read' establishes a clear context for when this tool is appropriate, and its contrast with 'edit_document' implies read-only usage. However, it does not explicitly state when not to use it or name the alternative.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 2 tool updatesv0.1.0
    • First observededit_document
    • First observedread_doc_contents
Deploy Server
One endpoint for every MCP toolRayrun runs every MCP server behind one URL and holds the credentials. Each call is decided per client, tool, and argument, then recorded.ray.runAd

TDQS

A3.5/5.0

Scored across 2 tools

Disambiguation5/5

The two tools have distinct purposes: one for editing documents by replacing strings, and another for reading document contents. There is no overlap or ambiguity between them.

Naming Consistency5/5

Both tool names follow a consistent verb_noun pattern (edit_document, read_doc_contents) using snake_case, making them predictable and easy to understand.

Tool Count3/5

With only two tools, the server feels minimal for general document management, but it may be sufficient for a very narrow chat context. However, the small count is borderline and could benefit from additional operations.

Completeness2/5

The server lacks create and delete operations for documents, which are essential for a complete CRUD lifecycle. Agents would be unable to add or remove documents, leading to failures in typical workflows.

Resources

Related MCP Connectors