Server Quality Checklist

Tool Scores

• pingA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description goes beyond the annotations by explaining the success output, error on invalid arguments, and lack of authentication. It also reinforces the read-only and side-effect-free behavior for the agent even if annotations were ignored.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three short, front-loaded sentences with no filler. It covers purpose, use context, output, errors, and authentication without repeating schema details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a trivial ping-style tool with rich annotations and an output schema, the description fully covers purpose, usage context, result, side-effect profile, error behavior, authentication, and read-only guarantee. Nothing material is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema already establishes that no arguments are valid. The description adds useful confirmation that invalid arguments will result in an error response, which is beneficial for correct invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb and resource ('Checks whether the Signalint MCP server is responsive') and names the literal output ('pong'). This clearly differentiates it from the sibling tools that check loop status, projects, files, and issue details.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly recommends using the tool to verify the server is connected before running diagnostics. It provides clear context for when to call it, though it does not state alternatives to avoid or mention exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_filesA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds substantial behavioral detail beyond the annotations: content-hash-based caching, dependency on config files like .oxlintrc and tsconfig.json, and the whole-program re-run behavior of TypeScript. It also confirms no files are modified, which complements the readOnlyHint without contradicting it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is longer than average, but every clause earns its place: it covers purpose, usage context, path constraints, caching behavior, and engine-specific nuances. The use guidance is appropriately placed near the beginning, and the caching details are grouped logically. It is thorough but not bloated.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of a clear output schema and robust annotations, the description covers everything an agent needs to decide whether and how to call this tool: purpose, engine behavior, input constraints, error conditions, caching semantics, and sibling distinction. No critical operational context is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema only says 'files' is an array of non-empty strings, so the description carries the burden of explaining path semantics. It does this well by specifying relative paths, excluding glob patterns, and warning about absolute/outside-root paths. This is strong but not exhaustive; it could also clarify whether directories are accepted, though the word 'files' likely implies not.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific action — running Oxlint, TypeScript, and optionally Biome diagnostics on a specific file list — and clearly distinguishes itself from check_project by framing this tool as the incremental variant. An agent can immediately understand what the tool does and how it differs from its nearest sibling.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly instructs when to use this tool ('after editing specific files') and when to use the alternative ('use check_project for a full project scan'). It also gives concrete constraints on expected inputs, such as relative paths and no glob patterns, so an agent has actionable selection and invocation guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_issue_detailA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare read-only and idempotent behavior. The description goes further by exposing the exact error case when both or neither parameter is supplied, and the stale response and recovery path. It also explicitly states 'no files are written or modified', reinforcing and not contradicting the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Each sentence adds value: purpose, safety, parameter constraint, error behavior, and recovery path are all covered without unnecessary repetition. The description is structured with front-loaded actionable information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with two parameters, oneOf constraints, and an output schema, the description covers everything needed to call it correctly: source of IDs, required exclusivity, error and stale states, resolution, and side-effect-free behavior. Nothing critical is omitted.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description compensates by explaining that clusterId and issueId come from the most recent check_project or check_files result and that exactly one must be supplied. It doesn't fully define what an issueId vs clusterId represents or how they appear, but the connection to the previous check calls provides meaningful context beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource ('Returns the full issue list') and clearly delimits the input ('for either one cluster ID or one issue ID'). It also ties the tool to the results of check_project or check_files, making it easy to distinguish from its siblings even without checking the schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly places this tool after a check_project or check_files call and gives a concrete alternative when the result is stale: 'call check_project or check_files again to refresh.' This is a clear when-to-use and when-not-to-use distinction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_loop_statusA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Although annotations already declare readOnly, idempotent, and non-destructive behavior, the description adds state-lifecycle context: loop history accumulates across all check_project and check_files calls and is restored from .signalint/session.jsonl on startup. It also confirms 'no files are written or modified,' which clarifies what the read-only hint actually guarantees.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is tight and efficient: it opens with the return value and risk guarantee, then gives state lifecycle, parameter count, and usage routing. Every sentence adds useful signal and no filler is present.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that the tool has no parameters, has an output schema, and conveys read-only behavior through annotations, the description is complete. It also clarifies how data is aggregated across sibling calls, how it is restored from session storage, and when to choose alternative tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, so the baseline is 4. The description explicitly says 'Takes no parameters' and the schema confirms an empty object with no additional properties. There is nothing further needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Returns all diagnostic issue signatures currently flagged as looping' and defines looping as repeatedly appearing and disappearing. It also distinguishes the tool from siblings like check_project and check_files by positioning it as the accumulated-history view rather than a fresh diagnostic runner.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly tells agents when to use it: 'identify which diagnostics an agent is oscillating on.' It also names the alternatives for fresh diagnostics: 'use check_project or check_files to run fresh diagnostics.' This is clear, direct usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• check_projectA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations (readOnlyHint, idempotentHint, non-destructive), the description adds materially useful behavioral context: every call 're-runs all enabled engines with no caching,' paths default to the project root when omitted, and absolute/out-of-root paths 'return an error response.' It also rescans the tool's safety profile by stating 'Read-only; no files are written or modified,' which is consistent with the annotations — no contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four concise sentences, each earning its place: purpose/engines, read-only guarantee, path constraints/default, and the sibling differentiation plus no-caching behavior. Nothing repeats the schema, no filler, and the most important information (what it runs and on what) is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With only one optional parameter, read-only/idempotent annotations, and an output schema (so return values need no description), the definition covers all informational needs: scope, defaults, constraints, error cases, alternative tool routing, and runtime cost behavior. There is nothing relevant an agent would have to guess about calling this tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has a single param 'paths' but 0% description coverage, so the description carries the entire burden. It adds crucial meaning: paths are 'one or more project paths,' default to the project root '.' when omitted, and must be relative — absolute or out-of-root paths return errors. That transforms what would be an opaque string array into a fully understandable parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb-resource pairing: 'Runs and clusters Oxlint and TypeScript (and optionally Biome) lint and type diagnostics for one or more project paths.' It names the exact diagnostics engines, explicitly differentiates from the sibling check_files, and clarifies the full-project scope, so an agent can distinguish it without opening any other tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit when-to-use guidance: 'Use this for a full project scan; use check_files instead for faster incremental checks after editing specific files.' It names the alternative sibling and the condition that selects it, which is the clearest possible routing for an agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.