Server Quality Checklist

Tool Scores

• get_ifc_statisticsB
  3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden for behavioral transparency. The verb 'Get' implies a read-only operation, and the listed output suggests a non-destructive analysis. However, it does not disclose potential performance implications, error handling, or whether the operation leaves any side effects, leaving some ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that efficiently conveys the tool's core purpose without redundancy. However, it omits important parameter guidance, but for what it does include, it is well-structured and appropriately sized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Although an output schema exists, the description is incomplete in context: it fails to explain the purpose and effect of detail_level, does not differentiate the tool from siblings, and lacks usage guidance. Given the moderate complexity and multiple sibling tools, more contextual detail is needed for reliable selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, leaving the parameters ifc_path and detail_level completely unexplained. The description does not compensate by describing expected input formats, the meaning of detail_level, or how parameters affect the output. This is a critical gap for effective tool invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool retrieves comprehensive IFC model statistics and lists specific categories (project name, schema, elements, class distribution, spatial structure). This distinguishes it from sibling tools that focus on element queries, properties, spaces, validation, comparison, conversion, or quantities.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for obtaining high-level statistics but does not provide explicit guidance on when to use this tool versus alternatives like query_ifc_elements or extract_ifc_quantities. No exclusions or alternative comparisons are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• extract_ifc_quantitiesB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions output formats but does not disclose whether the tool is read-only, how it handles missing files, performance implications, or any side effects. The description adds minimal behavioral context beyond what the name implies.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the core action ('Extract quantity takeoffs') and then specifies the relevant details (types and formats). Every word earns its place, with no unnecessary filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While an output schema exists (which may cover return values), the description lacks essential context for a tool with four parameters and no schema descriptions. It does not explain how storey filtering works, what quantity types are available, or how the output format parameter affects results. The description is too sparse for a tool of this complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, and the description does not compensate by explaining any parameter meanings. It fails to mention ifc_path, storey, output_format, or quantity_type, leaving the agent to rely solely on parameter names. The only hint is the output format list, which loosely maps to output_format but is not explicit.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool extracts quantity takeoffs (lengths, areas, volumes, counts) and lists output formats (markdown, CSV, JSON). This distinguishes it from sibling tools like query_ifc_elements, get_ifc_statistics, and get_element_properties, which focus on other aspects of IFC data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when quantity takeoffs are needed, but it does not explicitly state when to use this tool versus alternatives, nor does it mention exclusions or prerequisites. The sibling tool names provide some context, but the description itself offers no direct comparison or selection guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_element_propertiesB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description alone must disclose behavioral traits. It states the tool 'gets' data, implying a read-only operation, but does not mention error handling, permissions, or potential side effects (though none are expected). The description is straightforward but lacks explicit safety guarantees, which would be useful without annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence containing no redundant or filler content. It conveys the essential function in fewer than twenty words, earning a perfect score for efficiency.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While an output schema exists (so return values are likely defined there), the description still lacks enough context for an agent to use the tool confidently. It doesn't explain the role of ifc_path, how it relates to the IFC file, or how this tool differs from the sibling extract_ifc_quantities. The tool appears simple, but the missing parameter context and lack of usage alternatives make it incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description must compensate for unclear parameters. It mentions 'GlobalId' (mapping to global_id) but entirely omits ifc_path, leaving its purpose and format unexplained. This is a significant gap since both parameters are required and not self-explanatory from names alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Get'), identifies the resource ('all property sets and quantities'), and specifies the key input ('by its GlobalId'). This clearly distinguishes it from general listing tools like query_ifc_elements or statistics tools, making the purpose immediately obvious.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives such as extract_ifc_quantities or query_ifc_elements. It does not mention prerequisites (e.g., having an IFC file loaded) or exclusions, leaving the agent to infer appropriate usage from the name and schema.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_ifc_elementsA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full behavioral burden. It does not disclose return type, pagination behavior, or whether the tool reads-only or has side effects, and limit parameter behavior is unmentioned.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no redundant words; front-loads the verb and resource, then provides a helpful example. Excellent economy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters, no annotations, and an output schema, the description should clarify what the tool returns and how filters interact. It leaves critical behavioral gaps, making the tool under-specified for autonomous selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description maps 'type, storey, or properties' to element_type, storey, and property_filter, giving some meaning. However, it does not explain the property_filter syntax or the limit parameter, and ifc_path is not elaborated.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Search/filter IFC elements by type, storey, or properties' and gives a concrete example. This distinguishes it from sibling tools like get_ifc_statistics and list_ifc_spaces.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The example 'How many fire doors on Level 2?' provides a clear use case for querying filtered data. It does not explicitly name alternatives, but the intended context is evident from the query language.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_ifc_modelA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosing behavior. It does mention the nature of the checks and the report format, which adds some transparency. However, it does not clarify whether the tool is read-only, what happens if the path is invalid, or whether the 'checks' parameter selects specific validation areas. Some key behavioral details are missing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is concise, front-loaded, and free of fluff. Every word earns its place, efficiently conveying the core action, scope, and expected output.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description provides a high-level overview of the tool's function and output, which is adequate for a simple tool with an output schema. However, it lacks important contextual details such as how to use the optional 'checks' parameter, whether validation is comprehensive by default, and any file prerequisites. More guidance would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description is the only source for parameter meaning beyond names/types. It lists validation areas (materials, properties, containment, naming) that likely correspond to valid values for the 'checks' parameter, which provides value. However, it does not explain the role of ifc_path or how the checks parameter behaves when null. The description partially compensates for the low schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the action ('Run QA/QC validation') and the resource (IFC model), and specifies the scope (materials, properties, containment, naming) and output (PASS/FAIL/WARNING report). This distinguishes it from sibling tools that focus on querying, statistics, conversion, or comparison.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not state when to use this tool versus alternatives like query_ifc_elements or get_ifc_statistics. There is no mention of prerequisites, exclusions, or scenarios where validation is appropriate. The context is implied but not explicitly guided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• convert_ifc_to_gltfA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavior on its own. It states the core transformation and the optional Draco compression, which gives some insight into output. However, it omits details about file writes, overwrite behavior, or the effect of the max_elements parameter, leaving safety-relevant behavior unclear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence with a front-loaded verb. Every word contributes to clarifying the operation and its optional compression feature, with no redundant phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there are 4 parameters including a non-obvious max_elements limit, and no annotations, the description is too sparse to be considered complete. It explains the conversion operation but leaves the parameters' semantics and behavioral side-effects unaddressed, despite having an output schema to cover return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema is entirely undocumented in the description; the phrase 'optionally with Draco compression' maps only to the 'compress' parameter. The required 'ifc_path' and the 'output_path' and 'max_elements' parameters are not explained in any way, and since schema coverage is 0%, the description fails to compensate for that gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the action ('Convert'), the input resource ('an IFC model'), the output format ('glTF/GLB'), and the intended use case ('for web visualization'). It is distinct from sibling tools that query or analyze IFC data rather than perform format conversion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when this tool is appropriate: when a glTF/GLB version of an IFC model is needed for web visualization. However, it does not explicitly name alternatives or state when not to use it, though the purpose alone differentiates it from the query/analysis siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compare_ifc_modelsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the transparency burden. It implies a read-only operation via 'report' but does not explicitly disclose side effects, permissions, or performance considerations. The behavioral detail about reporting added/removed/modified elements is helpful but limited.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that is front-loaded with the action and resource. No unnecessary words or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While the output schema exists (reducing the need to describe returns), the description lacks details on parameter ordering and usage context. The tool is simple, but the lack of parameter documentation and usage alternatives makes it minimally complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has zero description coverage (0%) and only two string parameters (ifc_path_v1, ifc_path_v2). The description hints at these being IFC versions but does not clarify their roles (e.g., old vs. new) or expected formats. It fails to compensate for the schema's lack of parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'Compare' with a clear resource ('two IFC versions') and specifies the output ('added, removed, and modified elements'). This clearly distinguishes it from sibling tools like query_ifc_elements or get_ifc_statistics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use the tool (comparing two IFC versions) but does not explicitly mention alternatives or exclusions. However, the context is sufficient to infer its niche relative to siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_ifc_spacesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. The verb 'List' implies a read-only operation, but it does not explicitly confirm the lack of side effects or mention prerequisites such as valid IFC file or error behavior. It adds the storey filter context but lacks explicit behavioral details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that states the action, the resource, and the optional filter. Every word contributes value, with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list operation with an output schema present, the description is sufficiently complete. It covers the core functionality and the storey filter, and the output schema handles return values. It does not mention alternatives, but that is more relevant to usage guidelines.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no descriptions for the two parameters. The description clarifies that 'storey' is an optional filter, adding meaning beyond the schema. 'ifc_path' is self-explanatory from its name and schema, so the description covers the key parameter semantics well.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'List' and clearly identifies the resource as 'spaces/rooms in the IFC model'. It also mentions the optional storey filter, which distinguishes it from sibling tools like query_ifc_elements or extract_ifc_quantities.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context (listing spaces/rooms) but no explicit when-to-use or alternative guidance. It is implied that this is the tool for space/room listing, but it does not say when to prefer it over other IFC query tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.