Server Quality Checklist

Tool Scores

• stenographer_mark_milestoneC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It implies a write operation ('flag') but does not disclose behavioral traits such as whether the action is reversible, requires authorization, or has side effects. The lack of detail leaves significant ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence of 8 words, which is concise but lacks necessary detail. It front-loads the purpose, but the brevity sacrifices completeness. It could be expanded with parameter context without losing conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple schema (2 required parameters, no output schema) and no annotations, the description should explain parameters and behavioral context. It fails to do so, making the tool under-defined for an agent. The description is incomplete for effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, and the description does not mention any parameters. The agent receives no help understanding 'exchange_id' or 'label', which are required but undefined. This is a critical gap for proper tool invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action 'Flag' and the resource 'critical decision', and mentions 'tier-A priority', which gives a specific context. However, the tool name 'mark_milestone' suggests marking a point in time, not necessarily a decision, creating a slight disconnect. It distinguishes from sibling tools by implying a specific priority-level action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description does not mention prerequisites, when-not-to-use, or related tools. Only a brief purpose is given, leaving the agent to infer usage context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• stenographer_query_historyB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description carries full burden. It mentions it's a search but does not disclose whether it's read-only, side effects, rate limits, or return behavior. This is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is front-loaded and contains no extraneous words. Every word adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 2 parameters and no output schema or annotations, the description is severely lacking. It omits return values, pagination, and error handling, making it incomplete for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%. The description adds minimal meaning by implying 'query' is a keyword string for FTS5, but it does not explain the 'limit' parameter or any format constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as performing an FTS5 keyword search over all ingested exchanges. It distinguishes from siblings like compact_guard or ingest_exchange, which have different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. It simply states the function without context about prerequisites or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• stenographer_compact_guardB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Only vaguely indicates behavior ('compressed briefing'), but does not disclose if read-only, how fragments are selected, or any side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two succinct sentences: first states purpose, second gives usage context. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite 1 parameter and no output schema, description fails to explain input semantics or return structure. Essential info missing for agent to use effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Single parameter 'query' is completely undocumented in description (0% schema coverage). Agent has no guidance on what to supply as query.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get compressed briefing + top-k relevant fragments' with a specific verb and resource, distinguishing it from sibling tools like stenographer_get_brief by mentioning compression.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use guidance: 'Call when context window pressure hits.' However, lacks when-not-to-use or mention of alternatives like stenographer_get_brief.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• stenographer_get_briefB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It only says 'Get', implying a read operation, but does not specify side effects, authentication needs, rate limits, or what 'live' means operationally.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no wasted words. Every word contributes to the core purpose, achieving high conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite the tool's simplicity, the description omits details about the return format, document nature, and whether the 'live' aspect implies real-time updates. The lack of output schema makes this gap more significant.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no parameters, and schema description coverage is 100%. Per guidelines, 0 parameters yields a baseline of 4, as no additional parameter information is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action 'Get' and the resource 'live running notes document', directly conveying the tool's function and distinguishing it from sibling tools focused on compacting, ingesting, marking milestones, or querying history.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description lacks context on prerequisites, typical use cases, or when to prefer other tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• stenographer_ingest_exchangeA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must bear the full burden of behavioral disclosure. It only says 'Ingest', implying a write operation, but does not disclose consequences (e.g., whether it overwrites or appends), authentication needs, rate limits, or what happens on failure. More detail is needed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: two sentences, no wasted words. The first sentence states the core function, the second gives usage timing. It is front-loaded and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 3 parameters, no output schema, and no annotations, the description is minimal. It lacks details about session_id (optional, possibly for grouping), content length limits, and return behavior. Given that the agent needs to decide when to call and how to set parameters, more completeness is warranted.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, meaning the description adds no explanations for parameters. The schema includes role, content, and session_id, but the description does not clarify their semantics, especially session_id (optional) or content limits. Property names are self-explanatory, but without explanation of session_id's role, the agent may misuse it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Ingest a conversation turn', specifying the verb 'Ingest' and resource 'conversation turn'. It also provides timing context ('after every significant user or assistant message'). This distinguishes it from sibling tools like compact, get brief, mark milestone, query history, which are clearly different operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use guidance: 'Call after every significant user or assistant message.' It does not explicitly state when not to call or mention alternatives, but the sibling tools serve distinct purposes, so no confusion. Some nuance about non-significant messages could improve it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.