Server Quality Checklist

Tool Scores

• get_artistA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description takes on the full burden. It discloses accepted input formats (display name, hex, npub) and return contents (profile + every published track). It does not describe error handling or what happens on ambiguous matches, and the read-only nature is implied rather than explicit, so a 5 is not warranted.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, efficiently worded sentence that front-loads the action and target, then conveys return value without filler. Every phrase earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter lookup with no output schema, the description adequately covers what will be returned (profile + tracks). It leaves a few edge cases unaddressed (e.g., ambiguous names, empty results), but the tool is simple enough that these are minor gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% and the schema already explains the parameter well (partial match ok, formats). The description essentially repeats that information without adding extra semantic context about the parameter, so it meets the baseline but does not exceed it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb ('Look up'), the resource ('artist'), and the accepted identifiers (display name or Nostr pubkey), and further specifies that it returns the profile plus all published tracks. However, it does not explicitly differentiate from sibling tools like search_catalog, so it stops short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The intended use is clear: when you need an artist's profile and tracks by name or pubkey, this is the tool. It does not explicitly address when not to use it or mention alternatives, leaving sibling differentiation to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_catalogA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It discloses the live-fetching behavior, the Nostr signature guarantee, the canonical nature of the data, and the default ordering. It does not mention potential latency, relay availability, or result shape, but for a search tool the disclosed traits are substantive and useful.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences with no filler. It front-loads the core purpose, then efficiently adds key contextual facts (live source, signed results, canonical scope, browse behavior). Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search/browse tool with no output schema and no annotations, the description covers purpose, usage, data source, trust properties, and default ordering. It is missing explicit return-format details and error behavior, but the schema covers parameters and the description gives enough context for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so all parameters are already documented in the schema. The description adds one meaningful behavioral hint—that leaving all filters empty browses the whole catalog—but does not add meaning for individual parameters beyond the schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Search or browse') and resource ('Lightning FM's Nostr-native music catalog'), and further distinguishes it by specifying the event kind (31337) and the live relay source. This makes it clearly distinct from the sibling tools get_artist and get_now_playing, which focus on singular or current-state data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives clear usage context: filters can be left empty to browse the entire catalog newest first, and the catalog is live and canonical rather than curated. However, it does not explicitly state when to prefer this tool over the sibling tools or provide exclusion criteria, so it stops short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_now_playingA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It clearly labels the operation as a read, exposes the live/NIP-53 protocol detail, and states what information is returned. It does not mention failure modes, latency, or auth, but for a zero-parameter public read that is a minor gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single tight sentence that front-loads the verb and resource, then adds protocol and return details. Every clause earns its place with no repetition or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter read with no output schema, the description needs to explain what the tool returns and does, and it does so concretely (track, stream URL, station status). It could slightly improve by noting what happens when no track is currently on air, but the station status detail partially covers that.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool takes zero parameters, so the schema holds no parameter information for the description to enhance. According to the baseline for zero-parameter tools, this is a Strong case: nothing is missing.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb ('Read'), a specific resource ('current live activity ... Lightning FM publishes'), and the exact data elements returned (track, stream URL, station status). It clearly distinguishes this from sibling search-oriented tools like search_catalog and get_artist by focusing on live, current state rather than catalog lookup.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description says it reads current live activity and the track currently on air, which gives clear context for when to use it. It does not explicitly name sibling alternatives or say when not to use it, but the 'current' vs. search distinction is strong enough that an agent should route correctly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.