Skip to main content
Glama

SaSame MCP Observatory + Gold Rush Town

audit_mcp

Read-only

Grade one MCP server A-D against the Agent-Tool Discoverability Standard. SaSame, operated by SASAME S.R.L., continuously observes and measures the Model Context Protocol ecosystem and publishes verifiable evidence and history; the MCP Factory is internal machinery and an optional product surface behind it; measurement only, not endorsement. Runs the legitimate revision-aware MCP protocol entry (server/discover with legacy initialize fallback), tools/list, and one read-only tool call over POST JSON-RPC. It returns the grade, a per-criterion pass/evidence breakdown, and the single biggest gap to fix. This returns the grade and analysis ONLY — if you want a signed, portable certificate of the same audit, use verify_mcp_ready instead. DIRECTORY PRE-FLIGHT: these criteria cover the MECHANICAL reject reasons of the Claude Connectors Directory and ChatGPT Apps Directory (annotations, typed schemas, description clarity, liveness, graceful errors, anti-ghost) — run it before you submit. It does NOT cover privacy-policy, identity/business verification, OAuth callbacks, or prohibited-category rules; it catches mechanical failures, it does not guarantee a pass. SECURITY SIGNALS (advisory, never a verdict): plain-HTTP exposure, redirect count, Server/X-Powered-By header disclosure, stack-trace-shaped text in error responses (zero extra cost — read from responses already fetched), plus two bounded best-effort checks — RFC 9728 OAuth protected-resource metadata and TLS certificate expiry/trust-chain status. CAPABILITY SIGNALS (advisory, never graded): resources/list and prompts/list support (OPTIONAL per the MCP spec — their absence is not a defect), tools/list pagination, and the raw capabilities the server declared. Protocol inspection only — no auth-bypass, no payment. Free. Best run against YOUR OWN server. (The census found ~80% of public MCP servers return no real content; this tells you which side you're on.)

Input Schema

TableJSON Schema
NameRequiredDescriptionDefault
urlYesThe MCP server endpoint URL (https) to audit — ideally your own

Schema Changelog

Changes observed during successful MCP inspections. Dates show when Glama detected each change.

  1. Changed1 schema field changed
    • changedInput schema / $schema
      Previous value: -"http://json-schema.org/draft-07/schema#"New value: +"https://json-schema.org/draft/2020-12/schema"
  2. Added

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations provide readOnlyHint=true and openWorldHint=true, lowering the bar, and the description aligns with them ('one read-only tool call over POST JSON-RPC', 'Protocol inspection only — no auth-bypass, no payment'). It adds genuine context beyond the annotations: the scope of advisory security signals (plain-HTTP, redirects, header disclosure, credentials expiry) and the explicit stance that 'it catches mechanical failures, it does not guarantee a pass.' No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description runs roughly 400 words, far beyond what a single-parameter tool needs. Substantial space goes to context an agent doesn't need to invoke the tool correctly: the SASAME corporate background, 'MCP Factory is internal machinery,' the census statistic ('~80% of public MCP servers return no real content'), and 'Free.' The key differentiator (verify_mcp_ready) and return-value statement are buried mid-way rather than front-loaded. Valuable information, but poorly compressed.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter tool with no output schema, the description is thorough: it states the return payload (grade, per-criterion pass/evidence breakdown, single biggest gap), defines the graded criteria, lists advisory security and capability signals, and bounds what it does not grade. The only omission is a precise return structure, but no output schema exists and the described returns are sufficient for an agent to know what it will get.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% — the lone `url` parameter is already described as 'The MCP server endpoint URL (https) to audit — ideally your own.' The description reinforces this by adding the https constraint and 'best run against your own server,' but doesn't add material new semantics beyond what the schema already states. Baseline 3 is correct since the schema carries the load.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The opening sentence states a specific verb, resource, and scope: 'Grade one MCP server A-D against the Agent-Tool Discoverability Standard.' It further enumerates the exact protocol operations (server/discover with legacy initialize fallback, tools/list, one read-only tool call) and differentiates from verify_mcp_ready, which returns the same audit as a signed certificate. The purpose is unambiguous and clearly set apart from siblings despite surrounding marketing drag.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage direction is explicit and actionable: 'run it before you submit' (directory pre-flight for mechanical reject reasons), 'Best run against YOUR OWN server,' and a named alternative — verify_mcp_ready — for when a signed certificate is wanted. It also states what the tool does NOT cover (privacy-policy, identity/business verification, OAuth callbacks, prohibited-category rules) and that a pass is not guaranteed. When/when-not guidance is fully spelled out.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Try in Browser

Glama MCP Gateway

Add one secure layer between your agents and this server.

TDQS

B3.1/5.0
Disambiguation2/5

With 94 tools spanning overlapping concepts (multiple readiness/audit/grade tools, many status checkers, deprecated aliases like trust_* vs observation_*), agents will frequently struggle to pick the right one. While each tool is individually distinct, the sheer volume and conceptual overlap (e.g., audit_mcp, readiness_report, verify_mcp_ready, lookup_readiness, recommend_mcp, subscribe_grade_changes) create high misselection risk.

Naming Consistency3/5

Most tools use snake_case with underscores, but the pattern is inconsistent: some are verb-first (audit_mcp, verify_mcp_ready, claim_start, check_engagement) while others are noun-first (receipt_issue, meter_open, work_order_open, agent_invoice_status). Deprecated aliases like trust_compare vs observation_compare further break consistency, though the majority remain readable.

Tool Count1/5

94 tools is far beyond any reasonable scope for a single server, even one with broad ambitions like 'observatory + town'. The calibration notes 50+ as extreme mismatch; this server far exceeds that. Many tools are highly specific (e.g., factory_resolve_dead_letter, visit_touch_status, start_here) and could be consolidated or split into separate servers.

Completeness3/5

The server covers a wide range of domains (auditing, claiming, receipts, meters, escrow, work orders, gold rush, town, analytics) and offers many CRUD-like operations, but several lifecycle gaps exist: no cancel/close for work orders (only open/accept/deliver/accept_delivery), escrow (only open/attest/status), or meters (only open/charge/status). Given the massive scope, important operations are missing, though core workflows are present.