Server Quality Checklist

Tool Scores

• tqs_list_buildingsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds some behavioral context beyond the readOnlyHint annotation by explaining the building detection criterion and the root folder default. However, it does not disclose the effect of the depth parameter, return format, or error behavior, which are relevant for a listing tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: three short sentences that immediately state the purpose, define key terminology, and note the default. No redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with good annotations, the description is mostly complete but misses the depth parameter semantics and does not hint at the return format or whether the listing is recursive. Given the complexity is low, a 3 is appropriate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must explain the parameters. It implicitly covers 'root' by mentioning the TQS_ROOT environment variable fallback, but it completely omits the 'depth' parameter, leaving unclear how the search depth works. This is a significant gap given only two parameters exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'List TQS buildings under a root folder.' It specifies the resource (buildings) and the action (list), and even defines what constitutes a building ('a subfolder containing EDIFICIO.DAT'). This distinguishes it from sibling tools like tqs_building_read, which reads a single building.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides useful context for when to use the tool: to enumerate buildings under a root folder, with a default root from TQS_ROOT. However, it lacks explicit exclusions or alternative tool recommendations, so it doesn't fully meet the 'when-not' criterion.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tqs_building_readA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, and the description aligns by using the verb 'Read'. It adds useful behavioral context: the tool outputs JSON, the specific data categories included, and the two acceptable path forms. It does not discuss errors or edge cases, but for a read-only tool this is reasonably transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence. It front-loads the main action, lists the output contents, and finishes with the required input. Every clause contributes meaning, with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description adequately covers the tool's purpose, the required 'building_path' parameter, and the output content outline. However, the optional 'sections' parameter is left undefined, and with no output schema, the description does not fully compensate for that missing schema documentation. It is adequate but has a clear gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It clarifies the 'building_path' parameter ('Pass a building folder or a path to EDIFICIO.DAT'), but the 'sections' parameter is completely unexplained. With an optional parameter that defaults to null, leaving it undocumented is a significant gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Read') and the resource ('TQS building'), and enumerates specific output contents (project metadata, analysis model, floors, concrete strengths, covers, load cases). This distinguishes it from sibling tools like tqs_building_edit and tqs_dwg_read.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains the input format ('Pass a building folder or a path to EDIFICIO.DAT'), which is actionable. However, it does not explicitly contrast with sibling tools like tqs_building_edit or tqs_list_buildings, so it lacks explicit exclusions or when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tqs_processA
  4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate destructiveness, but the description goes further by specifying exactly what gets overwritten (calculation results and drawings) and that it can run for many minutes. This adds valuable context about side effects and time cost that annotations alone do not convey.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary purpose and followed by critical behavioral traits. It is minimal, free of repetition, and the pointer to the tqsexec reference is a useful, non-redundant addition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is complex (5 params, destructive, long-running) and has no output schema, so the description should clarify what a caller can expect. It covers the core function and side effects but omits details about return values, progress indicators, and per-parameter behavior for folder/subfolder/timeout. The reference to the task list helps but is not a full substitute.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must compensate but only partially does. It hints at task types but does not explain the role of folder, subfolder, timeout, or building_path. Examples like 'rebar schedules' give some semantic meaning to 'tasks' but leave most parameters unexplained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool runs TQS Manager processing tasks (TQSExec) with concrete examples (global processing, drawing generation, rebar schedules), giving a verb and resource. This distinguishes it from sibling tools like tqs_building_read or tqs_status which are non-processing operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: it is for executing processing tasks against a building and notes that the task list is in the tqsexec reference. However, it does not explicitly compare with alternatives or state when not to use it, leaving some ambiguity versus tqs_run_script or other write tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tqs_referenceA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description adds behavioral details beyond the readOnlyHint annotation, such as being offline and the no-arguments behavior that lists topics. There is no contradiction with annotations, but it doesn't describe what happens with the 'search' parameter or the return format, leaving some transparency gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three short, front-loaded sentences with no filler. Each sentence adds distinct information: what it is, how to list topics, and when to read relevant topics. This is highly efficient and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple and read-only, and the description covers the core usage pattern. However, the 'search' parameter is omitted entirely, and there is no information about the content or format of the reference output. Given zero schema coverage, the description is not fully complete, though adequate for a basic reference tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must compensate for the parameters. It partially does by implying that no arguments lists topics and that a topic can be read, but it never names or explains the 'topic' or 'search' parameters explicitly. The 'search' parameter is entirely unaddressed, leaving significant ambiguity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it is a 'Bundled offline reference for the TQS Python Interface' and explains the basic invocation ('Call with no arguments to list topics'). This distinguishes it from sibling tools like tqs_run_script, which executes code, and tqs_introspect, which likely inspects state.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives a direct usage directive: 'Read the relevant topic before writing TQS code with tqs_run_script', establishing when this tool should be used. However, it does not explicitly discuss when not to use it or mention alternative tools, so it falls short of a full 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tqs_introspectA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, so the safety profile is covered. The description adds valuable context beyond that: it returns 'real' (installed) methods/properties with signatures, implying the output is from the live runtime rather than static docs. This is useful behavioral info not included in annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with front-loaded purpose. The first sentence states what the tool does; the second explains when to use it. No wasted words, fully earns its length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only introspection tool with optional parameters and no output schema, the description covers purpose, usage, and output shape (signatures). However, the lack of parameter guidance is a noticeable gap, making it not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must explain parameters. It does not mention 'filter' or 'target' at all. While defaults provide minor hints, the meaning of 'filter' (what it filters on) and 'target' (module vs object) is left undocumented, making it hard for an agent to use parameters correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states an action ('Inspect installed TQS API') and a specific deliverable ('list a module's or object's real methods and properties with signatures'). It distinguishes itself from siblings like tqs_process or tqs_dwg_read by being the introspection/metadata tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives an explicit trigger: 'Use this whenever you are unsure a name exists.' It also explains why (docs cover only part of the surface), providing clear context. It does not mention when not to use or explicitly name alternative tools, but the guidance is strong enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tqs_building_editA
  4.2/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description goes beyond the annotations (readOnlyHint=false, destructiveHint=true) by detailing the dry_run default, the reporting of old/new values, the commit behavior, and the backup as EDIFICIO.BAK. This adds substantial context about what happens during mutation, matching the destructive hint.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the primary action, and provides essential detail without redundancy. Every sentence adds value, covering the core function and the crucial dry-run/commit safety mechanism.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of an output schema, the description adequately covers the key context: what changes, how to control the write, and the backup behavior. It does not mention error handling or prerequisites, but for a focused edit tool, the essentials are present. Slight deduction for missing explicit guidance on input structure and potential failure modes.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description must compensate for parameter meanings. It explains the 'dry_run' parameter well, including default and behavior, and implies 'changes' and 'floors' are the properties to modify. However, it does not describe the structure or expected format for these parameters, leaving gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Change building properties and save.' This uses a specific verb ('Change') and resource ('building properties'), distinguishing it from sibling read/list tools like tqs_building_read and tqs_list_buildings. The mention of dry_run further clarifies its purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for editing building properties but does not explicitly state when to use this tool versus alternatives. No exclusions or alternative tool references are provided. The context suggests it is for modifications, but explicit guidance is missing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tqs_dwg_readA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, so the tool is known to be a safe read. The description adds valuable behavioral context by explaining the filtering capabilities (by entity type, level, text content) and noting that units are centimeters, which informs the interpretation of returned data without contradicting annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the verb and resource, and every sentence adds value. It avoids unnecessary detail and is well structured for quick comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the main aspects of the tool: what it reads, what it returns, and how to filter. With no output schema, it provides a brief overview of return content (metadata, entities, etc.). However, it omits details about pagination or the behavior of 'limit' and 'include', leaving slight gaps in completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It explains the purpose of entity_types, level, and text_contains filtering, and implies dwg_path from the drawing read context. However, it does not clarify the meaning or usage of 'limit' or 'include', leaving some parameters underspecified.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads a TQS drawing (.DWG) and enumerates the specific data returned (scale, metadata, extents, levels, blocks, entity counts, entities). This distinguishes it from sibling tools like tqs_building_read, which reads building data, by focusing on DWG files.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear context for the tool's use, particularly mentioning that extracting TEXT entities is the usual way to pull rebar callouts and labels. It does not explicitly name alternatives or exclusions, but the specific mention of a typical use case implies when to use this tool over others.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tqs_run_scriptA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare destructiveHint=true, but the description adds significant context: runs in a separate process, captures stdout, supports returning structured data via 'result', and explicitly warns that it can create, modify, and delete project files. These details go beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three concise sentences, front-loaded with the core action, and each sentence earns its place. No unnecessary verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a general-purpose script execution tool with no output schema, the description is largely complete: it explains how to return data, capture stdout, run in a separate process, and warns of destructive potential. However, it doesn't clarify the effects of 'cwd' or 'timeout', which are relevant for execution behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate for all three parameters. It only explains the 'code' parameter indirectly (assign to 'result' to return structured data, stdout is captured), but provides no information about 'cwd' or 'timeout'. This leaves two of three parameters semantically unexplained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it runs Python in a separate process with TQS modules importable, and positions itself as 'the escape hatch' for anything other tools don't cover, distinguishing it from the specific sibling tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly gives usage context: 'The escape hatch for anything the other tools do not cover.' This tells the agent when to use it (when other tools are insufficient), though it doesn't name specific alternatives or state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tqs_statusA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare the tool as read-only, and the description adds specific behavioral context about what the tool checks and reports, such as the interpreter in use and folder paths. This goes beyond the annotation's bare safety signal, though it omits details like error handling or timeout behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each delivering distinct value: what the tool does, what it reports, and when to run it. It is front-loaded with the action and contains zero filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple status-check tool with no parameters and read-only annotations, the description fully covers the purpose, specific outputs, and usage context. No additional information is necessary for an agent to invoke this tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema provides complete coverage by default. Per the rubric, a baseline of 4 applies when there are no parameters, and the description does not need to add parameter detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool checks whether the TQS Python Interface is reachable and lists specific information it reports (interpreter, modules, folders). This is a distinct diagnostic verb+resource that separates it from sibling tools like tqs_process or tqs_introspect.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says to 'Run this first when TQS tools misbehave', giving clear contextual guidance for when to use the tool. It does not name alternatives or describe when not to use it, but the 'first' directive implies precedence over other tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.