Server Quality Checklist

Tool Scores

• sumo_cancel_searchA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It notes the job must be 'running' and 'started by this MCP server process,' but does not explain behavior for already-completed jobs, failure modes, or the destructive nature (stopping a search). This is a significant gap for a cancel operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that is direct and front-loaded with the key action and scope. It contains no unnecessary words and is easily parsed.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple cancel operation, the description covers the essential scope (running jobs, server process), but it does not address edge cases like canceling a non-running job or what the result/response looks like. Since there is no output schema, a bit more detail on post-cancel state or errors would improve completeness, but the tool is inherently simple.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides 100% coverage for jobId, and the description adds no additional meaning beyond what the schema states. The description references 'search job' but does not elaborate on jobId formats or provenance beyond the schema's own field description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool cancels a running Sumo Logic search job, specifying both the resource and the action. It also adds scoping with 'started by this MCP server process,' distinguishing it from other search-related tools like sumo_get_search_status or sumo_search_logs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool by referencing 'running' jobs and the process boundary, but it lacks explicit guidance on when not to use it or mention of alternatives. Sibling tools are not referenced, leaving the usage context somewhat implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sumo_get_search_statusA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It only says 'Get the current status' which implies a read-only operation, but it does not mention failure modes, whether it can be called repeatedly, or what happens if the jobId is invalid. This is insufficient transparency for a tool with no structured annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no redundant words. It is front-loaded with the action and resource, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple status-check tool with one parameter and no output schema, the description is fairly complete. It states what the tool does and clarifies the source of the jobId. However, it does not describe possible return statuses or error behavior, which would be helpful given the absence of an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema documents jobId with 'The search job ID (returned by previous search operations)' at 100% coverage. The description adds the context 'started by this MCP server process', which reinforces that the job must come from this server, but this is largely already implied by the schema. The description does not add significant new meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Get' and clearly identifies the resource as 'current status of a Sumo Logic search job', which distinguishes it from sibling tools like sumo_cancel_search and sumo_search_logs. The phrase 'started by this MCP server process' adds scope and prevents confusion with jobs from other sources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage (checking status after starting a search job) but does not explicitly state when to use it versus alternatives. It provides no exclusions or direct references to sibling tools, so the guidance is only implied rather than stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sumo_search_logsA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It adds value by stating the default time range ('Defaults to the last 15 minutes') and the mutual exclusion of time inputs. However, it does not state that the tool is read-only, what fields the results contain, or any potential side effects. For a search tool, the behavioral disclosure is decent but incomplete.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the core action, and every word contributes. It efficiently communicates the main purpose, key constraints, and a default without extraneous information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 7 parameters, no output schema, and no annotations. The description covers only time-related behavior, leaving gaps around expected output format, how the 'query' parameter should be structured, and the effect of flags like includeRecords/includeMessages. While the schema is rich, the lack of output documentation and limited description make the tool moderately, but not fully, contextualized.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already has 100% parameter coverage, so the baseline is 3. The description enhances this by explicitly noting that startTime+endTime and lastMinutes are mutually exclusive ('but not both') and by revealing the default time window. These details go beyond the schema's individual descriptions, clarifying relationship and default behavior.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Run a Sumo Logic log query.' This is a specific verb+resource pair that is distinct from sibling tools like sumo_get_search_status, sumo_cancel_search, or sumo_find_errors. However, it does not explicitly name or contrast these alternatives, so it stops short of full sibling differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives concrete usage guidance for time parameters: 'Provide either startTime+endTime or lastMinutes, but not both' and the default window. This is useful for avoiding misuse, but it does not say when to choose this tool over siblings (e.g., when a general search is needed vs. an error-focused search). Thus the guidance is mainly about parameter handling, not tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sumo_search_correlation_idA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that the ID is safely escaped as a literal search term, which is a useful behavioral detail, but it does not mention result format, rate limits, authentication requirements, or page/limit behavior. The description adds some context but remains incomplete.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences: the first states the core function, the second notes a safety behavior. Both sentences earn their place, with no fluff or redundancy, and the primary action is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has moderate complexity (4 params, no annotations, no output schema). The description plus schema cover the search purpose and parameter details, but the missing output schema and lack of guidance on comparing with sibling tools leave some contextual gaps. Return value expectations and usage nuance are not fully addressed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds a few synonyms (transaction ID, etc.) for the correlationId parameter, but this is marginal beyond what the schema already specifies. It does not clarify the meaning of limit, lastMinutes, or sourceCategory beyond their schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches for trace/request/correlation/transaction IDs across logs, which is a specific verb, resource, and scope. It distinguishes itself from sibling tools like sumo_search_logs and sumo_find_errors by focusing on cross-reference identifiers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when an identifier is available, but it does not explicitly state when to use this tool over alternatives like sumo_search_logs, nor does it provide exclusions. The context is clear but lacks explicit guidance for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sumo_find_errorsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description provides the query terms, which is useful. However, it does not mention return format, pagination, rate limits, or other behavioral traits beyond the search itself.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that front-loads the purpose and includes specific query terms, with no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple search tool with no output schema, the description covers the core behavior and query terms. It is adequate but could mention return results or default behavior more explicitly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so parameters are fully documented. The description adds no additional parameter semantics beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches for recent errors using an opinionated query with specific error terms (ERROR, exception, failure, fatal, stack traces). This distinguishes it from the general sumo_search_logs sibling.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for finding errors but does not explicitly compare to sibling tools like sumo_search_logs or state when not to use it. The 'opinionated query' hints at specialization but lacks explicit guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.