Server Quality Checklist

Tool Scores

• get_findingC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that the response includes affected entities and evidence references, but it doesn't state whether the operation is read-only, requires authentication, or how errors are handled. The name 'get' implies retrieval but safety isn't explicitly confirmed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single short phrase, which is concise, but it's so brief that it omits important usage details. It is front-loaded with the key point but lacks enough substance to be considered well-structured for agent consumption.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and a required parameter, the description is inadequate. It doesn't mention the required finding_id, the optional engagement, return format, or how it relates to other tools. An agent cannot fully understand the tool's behavior from this text alone.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds no parameter-specific information. Schema coverage is 50% (only 'engagement' has a description), and the description does not compensate for the undocumented 'finding_id' parameter or clarify parameter relationships. It merely restates the general purpose.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states it returns 'one finding' with 'affected entities and evidence references,' clearly indicating a singular retrieval operation distinct from list_findings. However, it doesn't explicitly differentiate from siblings like get_entity, so it falls short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. There's no mention of prerequisites, use cases, or when not to use it, leaving the agent to infer from the name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_tasksC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It mentions optional status filtering but fails to describe pagination behavior (though cursor is in schema), the meaning of 'implied by engagement state', whether it is read-only, or any ordering or default engagement behavior. The description is too vague to set accurate expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with no fluff. It is efficient, though it lacks structural elements like a summary of parameters or examples. It earns points for being brief and front-loading the core purpose, but the brevity comes at the cost of clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a list tool with no output schema, the description should clarify what the returned tasks look like and how the engagement state is determined. It does not mention return format, pagination expectations, or the meaning of 'implied by engagement state'. The agent is left with significant gaps in how to call and interpret results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 75% (limit, cursor, engagement have descriptions; status lacks one). The description adds no new meaning beyond what the schema already provides—'optionally filtered by status' merely restates the existence of the status parameter. Since coverage is high, a baseline of 3 is appropriate, but the description adds no value on top of the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly names the resource ('tasks') and the action (listing), and distinguishes it from sibling list tools (list_findings, list_engagements, list_entities) by focusing on tasks. However, the phrase 'implied by engagement state' is ambiguous—what engagement state and how tasks are implied is not explained, slightly reducing clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus alternatives. There is no mention of prerequisites, typical use cases, or why to choose list_tasks over search, list_findings, or list_engagements. The only hint is the optional status filter, but no context for decision-making is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the search is for an exact value, implying exact-match behavior, but does not disclose whether the operation is read-only, how pagination works (cursor), what the return format is, or whether it searches across all engagements or just the bound one. These are significant gaps for a tool with this complexity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short and front-loaded with the main purpose. However, the second sentence 'scope is segments, entities, or all' is redundant because the input schema already defines this enum. While the overall length is efficient, the redundant content prevents a perfect score.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 5 parameters, no output schema, and no annotations, the description is significantly incomplete. It does not explain the return format (what a search result looks like), how pagination via cursor and limit work in practice, or the role of engagement. An agent would lack crucial information to correctly use this tool, especially understanding what results to expect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 60%, which is moderate, so the description should add some meaning beyond the schema. It adds context for the query parameter by giving examples (IP, hostname, username) and clarifying it's an exact value, which the schema's plain 'string' type lacks. However, it does not add anything for limit, cursor, or engagement beyond what the schema already describes, and the scope mention simply repeats the enum.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear purpose: search captured terminal output and entities for an exact value, with concrete examples (IP, hostname, username, filename, secret). It is specific about the resource being searched. However, it does not explicitly differentiate from sibling tools like list_entities or get_recent_activity, so the distinct value proposition is not fully articulated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It never mentions conditions for choosing search over list_entities or get_recent_activity, nor does it state exclusions (e.g., 'use this when you need exact match, not list all'). The only hint is the phrase 'exact value', but no explicit usage context is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_taskC
  2.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does disclose the notable behavior that a dismissed task is not recreated by later extraction, which is useful. However, it omits other behavioral traits such as idempotency, whether fields are replaced or merged, side effects, or required permissions. The single behavior disclosed is meaningful but leaves many unknowns.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the action and resource, and wastes no words. The status enum is listed inline efficiently. It could be slightly more structured, but it is concise and readable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has six parameters, no output schema, and no annotations, the description is incomplete. It explains status but leaves the semantics of other fields, the effect of partial updates, and the return value undefined. The absence of any behavioral context beyond the dismissal note makes it insufficient for an agent to fully understand the tool's impact.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 17% (one of six parameters has a description). The description adds semantics for 'status' by listing the enum values, which is valuable, but it does not explain the meaning of 'notes', 'title', 'priority', or how 'engagement' interacts beyond the schema's own note. It does not compensate for the low schema coverage on the other parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool sets a task's status or fields, naming the specific resource and the action. It also enumerates valid status values, which adds specificity. It doesn't explicitly distinguish from siblings like update_finding, but the noun 'task' makes the target unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus alternatives, nor any exclusions or prerequisites. The description does not mention when not to use update_task or point to sibling tools like list_tasks for reads. The purpose is only implied, and the agent is left without explicit routing cues.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_provenanceB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the behavioral disclosure burden. It discloses the output structure (observations with segment, terminal, timestamp, byte range) but does not state whether the operation is read-only (likely, but not explicit), what happens when no evidence is found, or any side effects. The read-only nature is strongly implied by 'trace', but not stated. This is adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (two sentences) and front-loaded with the main purpose. The example clarifies intent without redundancy. It earns marks for efficiency, though it could be slightly more structured by explicitly listing the input options. Still, it is well within acceptable limits.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of three optional parameters and no output schema, the description is incomplete. It does not explain what happens when both entity_id and finding_id are passed, or what the response structure will be beyond the mentioned fields. It also does not state any requirements or error behavior. An agent would lack the information to call this tool correctly without guessing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is only 33% (only 'engagement' has a description). The tool description does not clarify the roles of entity_id and finding_id, nor how they interact (e.g., is one required? can both be used?). It only mentions 'an entity or finding' without specifying that you must supply one. The description fails to compensate for the low schema coverage, leaving agents uncertain about required parameters and their semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'trace' and the resource (fact/entity/finding), and describes the output (observations with segment, terminal, timestamp, byte range). It provides a concrete example question ('where did this credential come from'), which gives strong context. However, it does not explicitly differentiate from sibling tools like get_entity or get_finding, so it stops short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a usage scenario via the example question, implying an investigative need for provenance. However, it does not explicitly state when to prefer this tool over alternatives, nor does it mention prerequisites or exclusions (e.g., 'use get_entity if you just need the entity itself'). The guidance is implied but not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_entitiesB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It explains the filter semantics in detail, which is valuable, but it does not mention pagination behavior, default engagement handling, or return format. The reader must infer that the tool is read-only and that it returns a paged list based on the presence of limit and cursor parameters.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose, and then provides specific filter details. Every sentence carries substantive information, with no filler or repetition of schema content unnecessarily.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 6 parameters including a nested filter and no output schema, the description covers the main filter semantics and type enumeration, but it omits return structure, pagination mechanics beyond hinting via cursor, and engagement default behavior. While schema covers limit and engagement, the lack of output schema means the description should hint at what is returned, which it does not. This is adequate for a simple list tool but has minor gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 67% coverage, leaving 'type' and 'filter' undocumented. The description compensates by enumerating valid type values and explaining filter usage with concrete examples, adding meaning beyond what the schema provides. For the other parameters, the schema already covers them, so no redundancy.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (List) and the resource (entities), and enumerates the specific entity types, distinguishing it from sibling list tools like list_findings and list_tasks. It does not explicitly contrast with alternatives, but the scope is unambiguous enough for an agent to identify correct usage.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides specific filter usage examples but does not advise when to use this tool versus alternatives such as search or get_entity. There is no mention of exclusions or conditions that would route an agent to a sibling tool, leaving usage guidance implicit rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_findingsB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full responsibility for behavioral disclosure. It does not explicitly state that this is a read-only operation or that it returns a list of finding objects. The phrase 'Findings, optionally filtered...' is a noun phrase and could be read ambiguously; it never says 'returns' or 'lists'. Pagination behavior is implied only through the cursor/limit parameters, not described. For a simple list, transparency is minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences and gets straight to the point. The core action and the key filter are front-loaded. There is no redundancy or filler, though it could expand slightly without losing efficiency.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with four parameters and no output schema, the description covers the essential filtering behavior and the data-generation nuance (candidates). It does not explicitly state that the tool returns a paginated list or describe the response shape, which an agent might benefit from given the lack of an output schema. Overall, it is adequate but leaves the return structure to inference.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 75% (limit, cursor, engagement all have descriptions). The description adds value by enumerating valid status values, which the schema does not. However, it does not clarify the default behavior when status is omitted (presumably all statuses) or provide any additional context on limit/cursor beyond what the schema already states. The contribution is modest.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a clear verb ('list') and resource ('findings'), and includes a specific filtering dimension (status) with enumerated values. It stands apart from get_finding (single item), update_finding (modification), and other list tools that target different resources (tasks, engagements, entities).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explains when to use the status filter and notes that automated extraction only creates candidates, which is useful for selecting status values. However, it does not explicitly contrast with get_finding for a single result or update_finding for changes, nor does it state that this is the go-to for listing all findings. The guidance is adequate but not explicit about alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_recent_activityA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses ordering ('newest first') and that it returns previews of segments, which is useful. However, it does not mention read-only status, pagination behavior, rate limits, or any side effects. For a read operation, this is adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence. It states the purpose and a relevant parameter tip without any fluff. Every word earns its place, making it highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 optional parameters, no output schema, and no annotations, the description is minimal. It explains the core behavior but omits pagination details, default behaviors, and what 'segment' means. An agent might need to call the tool to learn these, but for a simple list operation it is passable. The lack of usage guidelines compared to siblings reduces completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 60% (limit, cursor, engagement have descriptions; since and minutes do not). The description adds meaning to 'minutes' by explaining it as a relative window, which is valuable. But it does not clarify 'since' or how 'cursor' works beyond the schema. It partially compensates for the coverage gap but not fully.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns recent terminal activity with previews, newest first. It identifies the resource (terminal activity) and behavior (newest first), but does not explicitly differentiate from sibling tools like get_segment or search. Still, the purpose is clear and not tautological.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a specific usage hint for the 'minutes' parameter ('Use minutes for a relative window'), which helps agents choose the right parameter. However, it does not state when to use this tool versus alternatives, nor any exclusions or conditions for selecting it over siblings like search or list_findings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_segmentA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the output content (terminal output, pane, timestamp, raw byte range) and the raw option's behavior, adding value beyond the schema. However, it does not explicitly state that the operation is read-only or non-destructive, nor does it mention any side effects, authentication requirements, or rate limits. This is a partial disclosure but lacks completeness for a tool with no structured safety hints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, stating the core purpose in the first sentence and the key option in the second. No filler words, and every sentence contributes to understanding. It is well-structured for quick parsing by an agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with four parameters (one required) and no output schema, the description covers the return value and the raw option but omits the meaning and usage of offset, which is a significant gap. The engagement default is covered in the schema, but offset remains undefined. The description also does not mention any pagination, limits, or error conditions. It is adequate for basic use but not fully complete for an agent to confidently invoke with all parameters correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 25% (only engagement is described). The description clarifies the raw parameter by explaining that raw=true returns original bytes, but it does not explain offset (which is likely a byte offset for pagination or range) or segment_id beyond its mandatory role. Given the low schema coverage, the description should compensate by explaining all parameters, especially offset, which remains cryptic. This partial compensation is insufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: retrieving recorded terminal output for a specified segment, including pane, timestamp, and raw byte range. It distinguishes itself from sibling getters like get_finding or get_engagement_summary by focusing on segment output, leaving no ambiguity about its purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving a segment's terminal output, and the raw=true option is explained, but there is no explicit guidance on when to use this tool over others, nor any exclusions or alternatives mentioned. It leaves the decision to the agent without context on when to prefer this over list or other getter tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_auth_attemptsA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full behavioral burden. It states that the tool lists attempts and outcomes and can be filtered, and provides insight into credential effectiveness. However, it does not disclose whether the operation is read-only, the ordering of results, or any rate limiting. The added filter explanations are helpful, but the description lacks explicit behavioral details beyond the basic list functionality.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with zero fluff. It front-loads the core purpose and then adds a practical hint about what the data reveals. Every word contributes to understanding, making it an exemplary concise and well-structured description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 6 parameters, no output schema, and no annotations, the description covers the primary purpose and key filters. However, it does not describe the response structure, pagination behavior beyond cursor, or potential side effects. While sufficient for basic calling, an agent might need more detail on default ordering or the exact fields returned, so completeness is moderate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With schema coverage at 50%, the description compensates by explaining the meaning of the undocumented parameters: 'result' (success/failure), 'target', and 'credential_id' as filters. This adds semantic value beyond the bare schema. The documented parameters (limit, cursor, engagement) already have schemas, and the description does not duplicate that. The contribution is meaningful for the uncovered parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as listing authentication attempts and their outcomes, and specifies the filters available (result, target, credential id). This is specific enough to distinguish it from generic list tools, though it does not explicitly name sibling alternatives, so a small deduction for lack of direct differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the usage context: it shows which credentials work and what has been tried. However, it does not explicitly state when to use this tool over alternatives like 'get_recent_activity' or 'list_findings', nor does it mention exclusion conditions. Usage is implied but not explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_engagementsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses ordering behavior but does not explicitly state that this is a read-only operation, nor does it mention any potential side effects, rate limits, or what happens when there are no engagements. The description is adequate but minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, well-structured sentence that front-loads the core action and scope. No wasted words; it earns its brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, no-output-schema tool, the description covers the 'what' and ordering but omits what an 'engagement' is, whether results are paginated or limited, and what the response shape looks like. Given no annotations or output schema, this is a moderate gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has zero parameters, and the description adds no parameter details (correctly). Per the baseline for 0 parameters, the description does not need to explain parameters, so a 4 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb (List), resource (engagements), scope (on this machine), and ordering (newest activity first). It is immediately distinguishable from siblings like list_findings and list_tasks, and the specificity is high.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus alternatives such as get_engagement_summary or search. With 14 sibling tools, the lack of any usage context or exclusions leaves the agent to infer appropriateness.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• record_observationA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must disclose behavioral traits. It states that all writes are attributed to the operator or agent, and explains that corrections separate entities on the next rebuild. However, it does not explicitly state that the operation mutates state, nor does it mention failure modes, permissions, or idempotency, leaving room for improvement.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded with the core purpose, then provides the kind list, the special correction usage, and the attribution rule. It is efficient, though the long list of kinds makes it dense; every sentence serves a purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  As a write operation with five parameters, the description covers the main use and the notable correction case, but does not explain the remaining parameters (fact_type, segment_ids, engagement beyond its schema description) or the success/failure behavior. This leaves agents without full information for correct invocation in all scenarios.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is low at 20%, so the description should compensate. It does clarify the 'kind' parameter by listing its allowed values and explains the special 'data' structure for corrections. However, it leaves 'fact_type', 'segment_ids', and the general shape of 'data' unexplained, which the schema also does not describe.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a precise statement of the tool's action: recording operator observations not present in captured terminal output. It enumerates concrete example kinds and a full list of allowed values, and it is clearly distinct from the sibling read/list/search tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description specifies when to use the tool: for capturing information that is not in captured terminal output. It also provides explicit usage for the special correction case. However, it does not directly name alternative tools for situations where the information is already captured, relying on inference from the tool name and siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_findingA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does disclose a key side effect: a rejected finding is not recreated by later extraction. However, it omits other important behaviors such as idempotency, persistence guarantees, whether updates are incremental or full replacements, or any permission/auth requirements. Coverage is partial.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no filler. The primary action is front-loaded, and the status usage is directly relevant. Every word earns its place, and the structure is immediately scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 10 parameters and no annotations or output schema, the description is too sparse. It does not describe the return value, error conditions, whether fields are updated partially or fully, or any prerequisites. Critical details for an agent to call this correctly (e.g., is finding_id always required, what happens if status is omitted) are absent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is only 10% (only 'engagement' has a description). The tool description adds semantics for the status parameter by explaining the meaning of 'confirmed' and 'rejected' values, which is beyond the schema. However, it does not clarify other parameters like evidence_refs or affected_entities, leaving them self-explanatory but not guaranteed. Given low coverage, it compensates only partially.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action ('Set a finding's status or fields') with a specific resource ('finding'), and elaborates on the status values with concrete examples (confirmed/rejected). This distinguishes it from sibling tools like get_finding and list_findings, which are read-only.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance on when to use specific statuses ('confirmed' to accept, 'rejected' to dismiss) and notes the behavioral consequence of rejection. However, it does not explicitly contrast with alternatives (e.g., creating a new finding vs updating) or mention when to use it over other tools, though the verb 'update' already implies modification.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_entityA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It does reveal what the tool returns (attributes, identity keys, relationships, authentication attempts, segments), which is the key output behavior. However, it does not disclose operational traits such as match behavior when multiple entities resolve to a plain name, any auth requirements, or response size/limits, leaving some behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with zero waste. The primary return scope is front-loaded, and the second sentence efficiently conveys identifier flexibility with concrete examples. Every clause earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool described as 'everything known about one entity,' the enumerated categories give an agent a solid expectation of return content despite the absence of an output schema. It is reasonably complete for the tool's scope, though it stops short of explaining edge behavior such as resolution when a plain name maps to multiple entities.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 50% — only 'engagement' has a schema description. The 'identifier' parameter is undocumented in the schema, but the tool description compensates by specifying it accepts an entity id, an identity key, or a plain name. This adds real meaning beyond the schema for the otherwise-uncovered required parameter, though it could go further (e.g., exact name-matching semantics).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb-resource pairing — 'Everything known about one entity' — and enumerates the full scope (attributes, identity keys, relationships, authentication attempts, segments). This clearly distinguishes it from sibling list_entities (plural, broad listing) and search, making the tool's singular focus unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states the acceptable input formats — 'entity id, an identity key, or a plain name such as FILE01 or an IP address' — which is valuable usage context for an agent deciding what to pass. It gives clear context for use but does not name alternatives or state when not to use this tool versus siblings like search or list_entities, so no exclusions are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_engagement_summaryA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It states the tool reports counts and metadata (non-mutating, read-like behavior) and details what is included, but does not mention potential side effects, errors, or limits. However, for a summary tool, this level of transparency is adequate; it clearly conveys a safe read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description opens with the core purpose ('Counts and metadata for the engagement'), then elaborates with useful strategic guidance in a second sentence. It is slightly verbose but each clause adds value, and the structure is logical—what it does, then when to use it.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a summary tool with one optional parameter and no output schema, the description fully covers what the agent needs: a comprehensive list of the reported counts and metadata, plus guidance on how to leverage the results. Nothing critical is missing for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, and the only parameter 'engagement' is already described as 'Engagement name; defaults to the bound one.' The tool description adds no extra meaning about parameters, so the baseline of 3 applies—the schema sufficiently documents semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reports counts and metadata for the engagement, with a specific verb ('reports') and resource ('engagement'). It lists the exact data points returned, distinguishing it from sibling list tools by providing an aggregated overview rather than individual records.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs to 'Call this first', establishing a clear order of operations. It explains that the output 'tells you which further queries are worth making', guiding the agent on when to use this tool versus querying specific lists, which is a direct usage guideline with a strategic rationale.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.