Server Quality Checklist

Tool Scores

• lograft_investigateA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions the pipeline steps, redaction, and delegation to azmcp for live mode, but does not cover destructive actions, authentication requirements, rate limits, or side effects. The behavioral traits disclosed are adequate but not exhaustive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences long, with no superfluous information. The first sentence lists the pipeline steps, the second explains modes, and the third gives a recommendation. It is front-loaded with essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  There is no output schema, so the description must explain return values. It states 'Returns a Bundle with paths to the written files,' which is basic. Given the complexity (8 parameters, nested objects, two modes), it could provide more detail about the bundle contents or output structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description must compensate. It explains the two mode parameters ('result' and 'live') and their sub-fields, making the combination clear. It also notes that KQL parsing is optional. While not every parameter is detailed, the description adds significant meaning beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs the full investigation pipeline, listing specific steps (parse KQL, normalize, gather repo context, correlate, redact, render) and two modes. It distinguishes itself from sibling tools by calling itself 'the tool most users want first.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains two usage modes: paste mode via the 'result' parameter and live mode via the 'live' parameter. It explicitly says 'Either pass result=... OR live=...' and recommends it as the first tool to try. However, it does not explicitly state when not to use it or mention alternatives beyond the sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lograft_normalizeA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses core behavior (normalization, output rowset fields, sessionId ref) and notes optimization (avoids re-shipping), but doesn't cover potential side effects or read-only nature.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences front-load the purpose, cover key details, and add sibling differentiation without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and four params including a nested oneOf, the description omits output format details (other than rowsetRef), error handling, and parameter constraints, making it incomplete for reliable agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%; description only mentions sessionId purpose. Does not explain source enum values, payload structure (inline vs path), or rowCap constraints, leaving significant gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool normalizes log exports into a specific 5-field rowset, lists supported input formats, and distinguishes itself from the primary sibling tool lograft_investigate.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises that most users want lograft_investigate and this tool is for partial pipelines, providing clear when-to-use and when-not-to-use guidance. Also explains sessionId usage for downstream correlation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lograft_correlateA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses that 'Output is redacted via the internal middleware before being returned,' which is a key behavioral trait beyond what the schema conveys. With no annotations provided, this description adds value. However, it does not explicitly state whether the tool is read-only or if it has side effects, leaving some gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences with no wasted words. The first sentence states the primary function, the second adds behavioral context, and the third provides usage guidance. Information is front-loaded and every sentence contributes.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (5 parameters, nested objects, no output schema), the description covers purpose, usage guidelines, and a behavioral detail (redaction). It does not explain return value structure or all parameters (e.g., externalAtoms, sessionId), but it is reasonably complete for an atomic pipeline tool, especially with sibling tool context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains the core parameters (rowset, repoContext, joinPolicy) by describing the join operation and listing the explicit keys. However, it does not explain the externalAtoms or sessionId parameters, nor does it provide details about the rowset's inline/ref structure. The description adds some meaning but is incomplete.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action: 'Join a NormalizedRowset against a RepoContext using explicit keys (operation_Id, configured ticket regex, service allowlist).' It specifies the verb (join), the resources (NormalizedRowset, RepoContext), and the key criteria. It also distinguishes itself from the sibling tool lograft_investigate, which is the preferred tool for most users.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly guides when to use: 'Most users want lograft_investigate; this atomic tool is for partial pipelines.' This tells the agent to prefer lograft_investigate unless working on partial pipelines, providing clear when-not-to-use and an alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lograft_gather_repo_contextA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully carries the burden. It declares the tool is pure read-only and shells out to git, which are important behavioral traits. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: the first explains the function with limits, the second provides usage guidance. No wasted words, front-loaded with key info.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with 3 parameters and no output schema, the description adequately covers purpose, limits, usage, and read-only nature. Lacks explicit output structure but is still comprehensive given low complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It mentions the default for sinceDays (14 days) and maxCommits (200), but does not fully explain repoPath or provide detailed parameter semantics beyond these defaults. Adds some but not complete value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool gathers a snapshot of recent commits, current branch, and origin URL from a git repository. It distinguishes itself from the sibling tool lograft_investigate, indicating this is for partial pipelines.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states that most users should use lograft_investigate, and this tool is atomic for partial pipelines. Also provides defaults and maximum limits (14 days, 200 commits) for when to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lograft_parse_kqlA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It states 'Pure compute, no side effects, no network' and 'without executing it', making behavioral traits clear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, each earning its place: purpose with key fact, outputs list, usage guidance, and safety. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations or output schema, the description covers purpose, behavior, and usage. It lists returned fields but does not specify output format, e.g., JSON structure. Slight gap but complete enough for its simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% description coverage, so description must compensate. It implies kqlText is the query and ticketRegex is for ticket mentions, but does not explicitly define ticketRegex or provide format. Some value added but incomplete.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool extracts structural facts from a KQL query without executing it, and distinguishes itself from the sibling lograft_investigate by noting it is for partial pipeline output.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says when to use this tool vs the alternative: 'Most users want lograft_investigate; use this atomic tool only when you need partial pipeline output.' Provides clear context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.