mcp-judge
Server Details
Automatically evaluates other MCP servers (conformance, docs, latency) and returns a score.
- Status
- Healthy
- Last Tested
- Transport
- Streamable HTTP
- URL
Glama MCP Gateway
Connect through Glama MCP Gateway for full control over tool access and complete visibility into every call.
Full call logging
Every tool call is logged with complete inputs and outputs, so you can debug issues and audit what your agents are doing.
Tool access control
Enable or disable individual tools per connector, so you decide what your agents can and cannot do.
Managed credentials
Glama handles OAuth flows, token storage, and automatic rotation, so credentials never expire on your clients.
Usage analytics
See which tools your agents call, how often, and when, so you can understand usage patterns and catch anomalies.
Tool Definition Quality
Average 4.5/5 across 3 of 3 tools scored.
Each tool has a distinct purpose: evaluating a single server, comparing multiple servers, and listing criteria. No overlap or ambiguity.
All tools follow a consistent verb_noun pattern in snake_case: compare_mcps, evaluate_mcp, list_criteria.
Three tools perfectly cover the server's purpose without being too few or too many. Each tool is essential and well-scoped.
The tool set fully covers the domain: single evaluation, comparison, and understanding the criteria. No obvious gaps.
Available Tools
3 toolscompare_mcpsAInspect
FREE during beta (will become PAID $0.10 via x402). Compares 2-5 MCP servers head-to-head and tells an agent which one to integrate. Set 'urls' (required) to an array of the targets' MCP endpoints (Streamable HTTP), e.g. ["https://a/mcp","https://b/mcp"]. Each server is evaluated with the same objective checks as evaluate_mcp (handshake, tools/list, documentation coverage, latency, and a safe functional probe — paid tools are never called), then ranked by score with ties broken by lower latency and more tools. Returns a ranking, the recommended server, a plain-language comparison, and the full per-server report. Objective checks only — no human and no LLM opinion.
| Name | Required | Description | Default |
|---|---|---|---|
| urls | Yes | An array of 2-5 target MCP server endpoints (Streamable HTTP) to compare, e.g. ["https://motherwise-api.onrender.com/mcp", "https://mcp-judge.onrender.com/mcp"]. Required. |
Tool Definition Quality
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses the evaluation method (same checks as evaluate_mcp), that paid tools are never called, and what it returns (ranking, recommendation, comparison, full reports). It also mentions pricing intent, which is useful context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single paragraph but packs significant information concisely. It could benefit from structured bullet points, but it is front-loaded with the pricing note and clearly communicates the core function and details.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite no output schema, the description explains what the tool returns (ranking, recommendation, comparison, full per-server reports). It covers the evaluation process, input format, and constraints. No critical information is missing for an agent to decide when and how to invoke it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% with one parameter 'urls'. The description adds meaning beyond the schema: specifies array length (2-5), target endpoints (Streamable HTTP), and gives an example. This helps the agent correctly format the input.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool compares 2-5 MCP servers head-to-head and tells which one to integrate. It uses a specific verb ('compare') and resource ('MCP servers'), and is distinct from siblings evaluate_mcp (single server) and list_criteria (lists criteria).
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains when to use (comparing multiple servers, free during beta) and how to set the 'urls' parameter. It implicitly distinguishes from evaluate_mcp but could be more explicit about when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
evaluate_mcpAInspect
FREE during beta (will become PAID $0.05 via x402). Evaluates another MCP server and returns an objective quality report so an agent can decide whether to trust and integrate it. Set 'url' (required) to the target's MCP endpoint (Streamable HTTP), e.g. https://host/mcp. MCP Judge connects, runs the 'initialize' handshake and 'tools/list', measures latency, checks how well tools and parameters are documented, and safely probes free tools — it never pays to test paid tools, it only verifies they are cleanly declared. Returns a structured report: reachability, protocol and serverInfo, per-tool documentation coverage, latency, a functional probe, a 0-100 score with letter grade and breakdown, and a plain-language summary. Objective checks only — no human and no LLM opinion.
| Name | Required | Description | Default |
|---|---|---|---|
| url | Yes | The target MCP server endpoint (Streamable HTTP), e.g. https://motherwise-api.onrender.com/mcp. Required. |
Tool Definition Quality
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations exist, so description carries full burden. It comprehensively discloses behaviors: free beta, paid future, connects and runs handshake, never pays to test paid tools, probes safely, returns structured report, and uses no human/LLM opinion.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Single paragraph that efficiently conveys key information upfront (free beta, paid future) then details tool behavior. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite no output schema, description fully explains return values (reachability, protocol, scores, etc.) and tool's operation. No gaps remain for agent understanding.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema has 100% coverage with one parameter. Description adds significant value by providing an example URL and clarifying that the endpoint must use Streamable HTTP, which is more specific than schema's description.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it evaluates another MCP server and returns a quality report. It distinguishes from siblings (compare_mcps, list_criteria) by focusing on single-server evaluation with specific actions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides context on free beta status and future payment, and explains when to use (to decide trust/integration). Lacks explicit comparison with siblings but implies usage via single server focus.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
list_criteriaAInspect
FREE, no payment required. Returns the exact criteria MCP Judge uses to score an MCP server: each criterion, its weight out of 100, and the A-F grade scale. Call this to understand how evaluate_mcp works before paying for it. The evaluation is fully objective and automated — no human reviewer and no AI opinion. Takes no parameters.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
Tool Definition Quality
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations provided, so description bears full burden. It discloses the evaluation is fully objective and automated (no human or AI opinion), and describes return content. Adequate for a simple read-only tool with no side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Description is four sentences, front-loaded with key info (free), and efficient. Could trim 'FREE, no payment required' but overall concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no output schema, description adequately explains return format: criteria, weights, grade scale. Sufficient context for a zero-parameter tool with clear purpose.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
No parameters exist (schema has empty properties), baseline is 4 per guidelines. Description adds value by explaining what is returned, compensating for absence of output schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool returns criteria, weights, and grade scale used by MCP Judge. It distinguishes itself from siblings 'compare_mcps' and 'evaluate_mcp' by explaining it provides the underlying scoring system.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly advises to call this tool to understand how evaluate_mcp works before paying, and emphasizes it's free. While it doesn't specify when not to use, the guidance is strong and context-aware.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Claim this connector by publishing a /.well-known/glama.json file on your server's domain with the following structure:
{
"$schema": "https://glama.ai/mcp/schemas/connector.json",
"maintainers": [{ "email": "your-email@example.com" }]
}The email address must match the email associated with your Glama account. Once published, Glama will automatically detect and verify the file within a few minutes.
Control your server's listing on Glama, including description and metadata
Access analytics and receive server usage reports
Get monitoring and health status updates for your server
Feature your server to boost visibility and reach more users
For users:
Full audit trail – every tool call is logged with inputs and outputs for compliance and debugging
Granular tool control – enable or disable individual tools per connector to limit what your AI agents can do
Centralized credential management – store and rotate API keys and OAuth tokens in one place
Change alerts – get notified when a connector changes its schema, adds or removes tools, or updates tool definitions, so nothing breaks silently
For server owners:
Proven adoption – public usage metrics on your listing show real-world traction and build trust with prospective users
Tool-level analytics – see which tools are being used most, helping you prioritize development and documentation
Direct user feedback – users can report issues and suggest improvements through the listing, giving you a channel you would not have otherwise
The connector status is unhealthy when Glama is unable to successfully connect to the server. This can happen for several reasons:
The server is experiencing an outage
The URL of the server is wrong
Credentials required to access the server are missing or invalid
If you are the owner of this MCP connector and would like to make modifications to the listing, including providing test credentials for accessing the server, please contact support@glama.ai.
Discussions
No comments yet. Be the first to start the discussion!