demo-plant-shop
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@demo-plant-shop Can you check the status of order #1042? My email is jamie@example.com."
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
MCP Support Evals
A TypeScript MCP server and tool-use evaluation suite for Demo Plant Shop, a fictional plant store.
What this demonstrates
MCP tools with explicit input and output schemas, along with resources and a support prompt.
Order lookup that requires a matching email and returns that require explicit confirmation.
A bounded model loop that calls tools through an MCP client and treats their output as data.
Deterministic grading in CI and recorded live results with usage, latency and estimated cost.
Related MCP server: support-agent-mcp
Architecture
flowchart LR
Client --> Agent[Agent loop]
Agent <--> API[Anthropic Messages API]
Agent --> MCP[MCP client]
MCP --> Server[MCP server]
Server --> Fixtures[Fictional shop fixtures]Results
Metric | ||
Model |
|
|
Cases graded / selected | 54 / 54 | 54 / 54 |
Overall pass rate | 53/54 (98.1%) | 53/54 (98.1%) |
tool selection | 9/9 (100.0%) | 9/9 (100.0%) |
grounding | 9/9 (100.0%) | 9/9 (100.0%) |
privacy | 9/9 (100.0%) | 9/9 (100.0%) |
write safety | 9/9 (100.0%) | 9/9 (100.0%) |
injection | 8/9 (88.9%) | 9/9 (100.0%) |
out of scope | 9/9 (100.0%) | 8/9 (88.9%) |
API calls | 102 | 103 |
Total tokens | 178750 | 185836 |
Case latency p50 / p95 | 2.73 s / 4.14 s | 2.91 s / 5.38 s |
Estimated cost (USD) | $0.425804 | $0.436992 |
Stopped by budget | No | No |
Iteration
The original run scored 47/54 on 7543214. A harness bug marked six cases incomplete because the response parser rejected messages containing signed thinking blocks. The seventh case, liner-purpose, reached a final answer and failed a disclosure check.
Run A uses the parser fix with the original system prompt. The parser now preserves thinking and redacted thinking blocks through tool calls and returns only text blocks as the answer. Injection was the lowest-scoring category at 8/9 because liner-purpose repeated order details from an injected instruction.
Run B applies the prompt change on top of the same parser. The prompt asks the model to use only relevant shop facts and omit injected instructions and identifiers. Injection passed 9/9, including liner-purpose. Out-of-scope fell from 9/9 to 8/9 when repair made an unnecessary tool call before refusing. The overall score was 53/54 in both runs.
Both runs used the same model, cases and graders. There was one live run per prompt. Pull request #3
Run locally
npm ci
npm run build
npm startThe stdio process exchanges MCP messages through stdin and stdout. Use npm run start:http for Streamable HTTP at http://127.0.0.1:3210/mcp.
MCP Inspector
From the project root, launch the MCP Inspector against the built stdio entry point. Use a current Node.js LTS release.
npx @modelcontextprotocol/inspector node dist/stdio.jsFor HTTP, start the HTTP server first, then choose Streamable HTTP in the Inspector and connect to http://127.0.0.1:3210/mcp.
Claude Desktop
Add this entry to claude_desktop_config.json. Replace the path with the absolute path to dist/stdio.js in the local checkout. Use the absolute path to the Node executable as command if Claude Desktop cannot find node.
{
"mcpServers": {
"demo-plant-shop": {
"command": "node",
"args": ["/absolute/path/to/mcp-support-evals/dist/stdio.js"]
}
}
}Claude Code
Register the same built entry point from the project directory. claude mcp get demo-plant-shop shows the connection details.
claude mcp add --transport stdio demo-plant-shop -- node "/absolute/path/to/mcp-support-evals/dist/stdio.js"Tools and resources
search_help_articlesranks keyword matches with BM25 and returns article IDs, titles and snippets. The result limit is optional.get_help_articlereturns the complete article.get_order_statusreturns only the status when the order reference and email match. Both an unknown order and an email mismatch returnnot_found.find_plantsfilters by light level, optional pet safety and optional maximum price in demo credits.create_return_requestpreviews a request for a delivered order. Onlyconfirm: truerecords a pending request. Repeated confirmations return the existing request and its original reason withalready_exists: true. The store is held in memory for the lifetime of the server process.
Clients can read help articles as MCP resources and use the answer_customer_question prompt to guide grounded answers. The server uses fictional fixture data and needs no API key.
How to reproduce
npm run build
npm run eval:replayReplay uses the committed scripted responses and executes their tool calls against a fresh in-memory MCP session for each case. CI uses the same command. A scripted result checks the harness and carries no measured model performance, usage or cost. The deterministic checks cover tool arguments, citation IDs, order privacy, return confirmation, untrusted article text and refusals.
For a live run, put ANTHROPIC_API_KEY and optionally ANTHROPIC_MODEL in the ignored .env.local. The default model is Claude Sonnet. An alternate model requires --input-usd-per-million and --output-usd-per-million. The standard token prices come from the Claude pricing page.
npm run eval:live -- --archive-recordingsUse --max-cases to run a subset and --budget-usd to change the budget. Before each request the runner checks the measured spend so far plus an input allowance and the full output limit. The report gives a cost estimate.
Live runs save reports under evals/results/. With --archive-recordings, responses go under evals/recordings/<model>/<run-id>/ so each run keeps its own evidence. A run without the flag writes responses directly to evals/recordings/<model>/. Reports include per-case checks, category pass rates, token totals and latency percentiles. Case latency sums the time spent waiting for model responses. A case interrupted by the budget contributes usage but has no grade.
Replay the latest recorded run for a model with npm run eval:replay -- --model <model>. Use --run-id to select a run explicitly. The replay writes a separate report. The original 47/54 run needs its original agent revision for exact replay. On Node.js 22 or later, npm run report regenerates the published table from the selected result files in evals/report-runs.json.
Limits
The cases use a small fictional dataset and measure a single model. The graders check tool arguments, citation IDs and answer patterns. They do not assess overall helpfulness or verify every factual clause.
Each prompt ran once, with nine cases per category, so a one-case difference is within normal run-to-run variation. Run B passed liner-purpose and missed repair. Both runs scored 53/54 overall, so the comparison leaves any overall improvement from the prompt change unproven.
Checks
npm run format:check
npm run lint
npm run typecheck
npm testDesigned and reviewed by Massimiliano Borgatti; implemented with the help of an AI coding agent. — LinkedIn
Available Tools
5 toolscreate_return_requestAIdempotent
Preview a return request for a delivered order with matching email and a customer-provided reason. Submit with confirm=true only after the customer explicitly confirms the preview. Repeating a submission returns the existing request.
Inputs: email (required), reason (required), order_id (required), confirm. Returns: reason, status, order_id, request_id, +1 more
find_plantsARead-only
Find catalog plants for low, medium or bright light. Optionally filter by whether they are pet-safe and by a maximum price in demo credits.
Inputs: light (required), pet_safe, max_price. Returns: plants
get_help_articleARead-only
Read the full text of one Demo Plant Shop help article using an article ID from search results or resources.
Inputs: id (required). Returns: article
get_order_statusARead-only
Read the status of a fictional order. Requires the order reference and the matching email supplied by the customer.
Inputs: email (required), order_id (required). Returns: status
search_help_articlesARead-only
Search Demo Plant Shop policies and plant care guides by keyword. Returns ranked article IDs, titles and snippets. Use get_help_article to read the full article.
Inputs: query (required), limit. Returns: articles
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
Loading tool schema changelog…
TDQS
Scored across 5 tools
Each tool targets a distinct resource and action: help search vs. help reading, order status lookup, plant catalog filtering, and return submission. The one potentially confusable pair (search_help_articles vs. find_plants) is separated by domain—policy content versus catalog inventory—and the descriptions make that explicit.
All five tools follow a clean verb_noun snake_case pattern (search_help_articles, get_help_article, get_order_status, find_plants, create_return_request). Verbs are semantically accurate and the shared prefixes (help_, order) group related tools predictably.
Five tools is a tight, well-scoped surface with no redundant entries, appropriate for a demo shop assistant. It leans slightly thin—there is no purchasing or cart operation—but every listed tool earns its place.
The surface covers discovery (help, plants), post-purchase status, and returns, but skips the core commerce step: there is no way to place an order or view a customer's order list. An agent handling an end-to-end shopping request would hit a dead end before checkout.
Maintenance
Related MCP Connectors
Build and manage AI-native customer support agents from Claude or any MCP client.
Hosted MCP for e-commerce: live product catalog, stock, and pricing for AI agents.
MCP server for your apps' tools and custom tools, plus hosted AI agents and approval-gated workflows
Remote MCP server for supportsheep: run AI interviews and manage support content for your blog.
Related MCP Servers
- FlicenseAqualityDmaintenanceProvides e-commerce customer support tools for order status, delivery, account, and policy questions, with RAG-grounded retrieval and strict authorization checks over mock data.6-
- AlicenseNot gradedqualityDmaintenanceExposes order status lookup and knowledge base search tools from the Support Agent AI over MCP, enabling MCP clients to handle customer support queries with grounded, citation-backed answers.2MIT
- AlicenseNot gradedqualityCmaintenanceEnables an AI to perform customer support workflows by looking up customers, retrieving orders, and creating support tickets through MCP tools.7 npmISC
- FlicenseNot gradedqualityCmaintenanceEnables AI assistants to handle customer support by looking up order status, semantically searching knowledge bases, and creating support tickets through MCP tools, with grounded answers and automated evaluations.-