Server Quality Checklist

Tool Scores

• s1_activitiesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since no annotations are provided, the description carries the full burden of behavioral disclosure. It does disclose the ordering ('newest first') and gives examples of activity types, but it does not mention pagination details, the fact that it is read-only, or any authentication requirements. For a log tool, some of these are implied, but the description lacks explicit behavioral context beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the core purpose and provides an illustrative use case. No unnecessary words or repetition; every phrase adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 7 optional parameters, no output schema, and no annotations, the description adequately conveys the tool's role and typical use. It explains the content area and gives an example usage, which is sufficient for tool selection. It does not describe the response structure, but the absence of an output schema and the nature of a log list make this a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and each parameter has a clear description (e.g., 'Max records per response', 'Comma-separated site IDs'). The tool description adds no additional parameter semantics, but the schema already does the heavy lifting, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as an activity/audit log from the management console, listing specific content types (threat mitigations, agent installs/upgrades, etc.) and noting the 'newest first' ordering. This distinguishes it from sibling tools like s1_threats or s1_agents, which focus on different resources. The use case example further clarifies its purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a concrete use case ('show what was done about the threat on machine X'), indicating when the tool is appropriate. It does not explicitly discuss when not to use it or compare to alternatives, but the sibling tools are distinct enough that the intended usage is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• s1_agentsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does a good job: it discloses the default compact summary format, warns that full_details=true returns 'large' raw records, and lists the exact fields returned. It does not mention pagination behavior or rate limits, but the cursor is documented in the schema and the warning about large payloads adds useful safety context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences deliver a wealth of information without fluff. The description is front-loaded with the core purpose, then lists return fields, then covers the key behavioral switch (full_details) and the prerequisite call—every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 14 parameters, no output schema, and no annotations, the description is remarkably complete. It explains the compact summary fields, the alternative raw mode and its size caveat, and the site-resolution prerequisite. Minor gaps include no explicit mention of pagination or how the filters combine, but the schema covers parameters and the cursor field.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds high-level grouping of filters ('site, name, health flags or OS') and the crucial instruction to call s1_list_sites for site IDs, but it does not elaborate on parameter syntax or value formats beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Opens with a specific verb+resource ('List endpoint agents with filters') and immediately positions it as 'the drill-down tool for machines', clearly distinguishing it from sibling tools like s1_estate_overview. The description further specifies the output fields (computer name, site, OS, status) and the default vs full-details modes, making the tool's purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides concrete guidance to resolve site IDs using s1_list_sites before calling this tool, which is essential for using the site_ids filter correctly. It also implies when to use it (drill-down, compact summaries) versus other estate-level tools, though it does not explicitly list alternative tools or when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• s1_estate_overviewA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It discloses that the tool 'sweeps all agents server-side', lists the exact metrics returned, and references the safety cap. It doesn't cover performance or rate limits, but this is sufficient for a read-only overview.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with purpose, followed by concrete output details and usage guidance. Every sentence earns its place with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema, so the description compensates by enumerating the table columns and clarifying the server-side sweep. It doesn't mention pagination, but max_agents serves as a safety cap, making the description adequate for an overview tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with detailed parameter descriptions. The description only adds incidental context (e.g., 'not seen for N days' for stale agents) but doesn't meaningfully enhance parameter understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Estate-wide endpoint health overview grouped by site' which is a specific verb+resource+scope. It clearly differentiates from siblings by stating this is 'the primary tool for MSP health/AV reporting' and explicitly points to s1_agents for individual machine drill-down.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives a clear use case ('what is the state of every client's endpoints?') and explicitly instructs to 'Use s1_agents to drill into the individual machines afterwards.' However, it doesn't provide explicit when-not-to-use guidance for other siblings like s1_threats.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• s1_threatsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the default compact summary behavior, the full_details flag for raw records, and the sorting order (newest first). It does not mention rate limits or pagination details, but the core behavioral traits are well-covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each earning its place: purpose, return fields/behavior, and alternative tool. It is front-loaded and tightly written with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (14 params, no output schema, no annotations), the description covers the essential aspects: purpose, return field list, default vs full_details, sorting, and sibling distinction. It does not explain the pagination return structure, but the schema covers the cursor parameter, making this a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 14 parameters thoroughly. The description adds no additional parameter-level detail beyond mentioning full_details behavior, which is also in the schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'List threats/detections with filters' and identifies it as 'the drill-down tool for incidents,' which distinguishes it from the sibling s1_threat_summary. It enumerates the specific fields returned, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly directs users to 'Use s1_threat_summary for aggregated counts across the estate,' providing a clear alternative. The description also frames this as the drill-down tool for incidents, implying when detailed threat data is needed versus aggregate summaries.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• s1_list_sitesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses what the tool returns (name, state, licence counts, SKU/suite, expiration, health status) and implies read-only behavior. It doesn't mention pagination or rate limits, but the schema covers params; the return-field disclosure adds meaningful value beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise, front-loaded sentences. The first states the main action and context; the second explains additional return fields and usage. No wasted words or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema exists, but the description explicitly lists the returned fields, which fulfills that need. It also provides the MSP/MSSP context and workflow, making the tool self-contained for an agent unfamiliar with SentinelOne.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with each parameter adequately described. The description does not add extra parameter semantics, but it does mention feeding site IDs into other tools, which relates to output rather than parameters. Baseline 3 is appropriate given the schema already handles parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List sites' with a specific resource (sites) and adds context about MSP/MSSP segmentation. It distinguishes itself from sibling tools by focusing on site discovery as the client list entry point, not agents, threats, or activities.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says 'use this to discover the client list, then feed site IDs into the other tools,' giving clear workflow guidance. This tells the agent when to use this tool and how it relates to alternatives, making it a natural first step in an MSP/MSSP context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• s1_threat_summaryA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that the tool 'sweeps ALL matching threats server-side' (implying a potentially heavy query) and returns 'compact counts' rather than full data. It also mentions the safety cap indirectly by referencing the period filter. However, it does not explicitly note the read-only nature or any rate limits, though these are implied by the reporting context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with purpose and usage. Every clause adds value: scope, output details, when to use, and which sibling to use instead. No redundant words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description summarizes the output structure: 'per site by incident status, analyst verdict and classification, plus estate-wide totals, the most common threat names and the machines with the most detections.' Combined with usage guidance and safety cap context, it gives a complete picture for selecting and invoking the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the schema already documents all five parameters with meaningful descriptions. The description adds a usage hint for created_after ('pass created_after for the period') but does not introduce new parameter-level details beyond the schema. Baseline 3 applies because the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the tool's purpose: 'Aggregated threat counts grouped by site' with a specific verb ('sweeps' and 'returns compact counts'). It explicitly distinguishes itself from sibling tools by identifying as 'the reporting/dashboard tool' and directing users to 'Use s1_threats to drill into individual detections afterwards.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use guidance: 'Ideal for "threats this month across all clients" reporting' and instructs to 'pass created_after for the period.' It also names an alternative tool (s1_threats) for downstream drilling, making the decision boundary clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.