Server Quality Checklist

Tool Scores

• speckle_project_createB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only says 'Create', which implies a write operation, but does not mention authentication requirements, reversibility, side effects, or what happens on creation. The description adds little beyond the verb itself.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that clearly states the action and purpose with no wasted words. It is easy to parse and immediately conveys the tool's role.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has a simple schema and an output schema, but the description is too sparse for a create operation with no annotations. It omits parameter details, usage guidance, and behavioral context, making it insufficient for an agent to correctly select and invoke the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0% and the description provides no explanation of the parameters (name, visibility, description). The agent receives no guidance on parameter formats, allowed values, or their meanings, leaving the agent to rely solely on property names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool creates a new Speckle project, with a specific resource (project) and a purpose clause ('to organize models shared across your team'). This distinguishes it from sibling tools like speckle_model_create, which creates models, and speckle_projects_list/details which are read operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a use case but does not explicitly state when to use this tool over alternatives. It mentions 'organize models shared across your team,' which provides context, but lacks explicit guidance on when not to use it or how it differs from speckle_model_create.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• speckle_projects_listB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It mentions the authenticated user and accessibility scope, which is useful context. However, it does not explicitly state that the operation is read-only, nor does it describe pagination behavior or how the limit parameter affects results, leaving some ambiguity.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the purpose, and contains no extraneous words. Every sentence contributes to understanding the tool's function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While the description covers the core purpose and the output schema exists, it lacks guidance on the limit parameter's behavior and pagination. Given one optional parameter and no annotations, this is a notable gap for a list tool, but the simplicity of the operation prevents a lower score.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has one parameter (limit) with 0% description coverage, and the description does not mention it at all. The phrase 'all accessible projects' could even be misleading, as a default limit of 25 suggests not all projects are returned without further pagination. This is a significant gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool lists all accessible Speckle projects for the authenticated user, using a specific verb and resource. It also clarifies that projects are top-level containers, which distinguishes it from sibling tools like speckle_models_list and speckle_project_create.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool is for enumerating projects as a starting point, and the reference to projects as containers hints at a hierarchy. However, it does not explicitly state when to use this tool versus alternatives such as speckle_models_list, nor does it mention exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• speckle_project_detailsA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of disclosing behavioral traits. It states 'Get' which implies read-only, but does not explicitly confirm no side effects, mention authentication requirements, rate limits, or error behavior. It adds some context by listing what data is returned (models, team members), but the lack of explicit read-only declaration and any operational constraints is a gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence that is front-loaded with the primary action ('Get detailed information') and specifies key output aspects. Every word is useful, and there is no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read tool with one required parameter and an existing output schema, the description is largely sufficient. It identifies the purpose and the significant content of the result. It does not explain how to retrieve the project_id (e.g., via speckle_projects_list), but that is a usage guideline concern rather than a completeness issue for the tool itself.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no parameter description (0% coverage), and the description does not compensate by explaining the project_id parameter's meaning or how to obtain it. The parameter name 'project_id' is self-explanatory to some extent, but the description adds no semantic value beyond the schema itself.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: retrieving detailed information about a Speckle project. It specifically mentions the content of that detail (models and team members), which distinguishes it from sibling tools like speckle_projects_list (list of projects) and speckle_project_create (creation). The verb 'Get' and resource 'detailed information about a Speckle project' make the purpose explicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: when you need detailed project details including models and team members. However, it does not explicitly compare with alternatives (e.g., speckle_projects_list) or state when not to use it. There is no explicit exclusion, but the guidance is implied rather than spelled out.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• speckle_model_createA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosing effects. It states 'Create a new model', which implies a write operation, but does not mention permissions, idempotency, validation, or what happens on duplicate names. It does provide a hint about organization via discipline, but that's limited behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the action and resource. Every word serves a purpose, with no fluff or redundancy. It is appropriately sized for the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 4 parameters, an output schema, and no annotations. The description covers the core concept and discipline organization but omits edge cases, permission requirements, or relationship to the parent project. Given the sibling tools and output schema provide some structure, the description is minimally sufficient but has clear gaps in behavioral and contextual detail.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for parameter meaning. The description references 'discipline' as a key parameter ('organized by discipline'), adding semantic context beyond the schema's default value. However, it does not elaborate on project_id or description, though project_id is self-explanatory. This partial compensation warrants a 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (create) and resource (new model within a Speckle project), and adds the organizing dimension 'by discipline' which gives context. It does not explicitly distinguish from sibling tools, but the verb 'create' plus 'model' is distinct enough among the listed siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context (organizing by discipline) but does not explicitly state when to use this tool versus alternatives like speckle_projects_create or speckle_models_list. It lacks exclusions or explicit prerequisites, so it's adequate but not fully guiding.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• speckle_models_listA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full transparency burden. It states 'List all models' but does not disclose that results are limited by the 'limit' parameter, making the 'all' claim potentially misleading. It also does not describe pagination, sorting, or any read-only nuances beyond the verb 'list'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, immediately states the main action, and adds a useful explanatory clause about models. There is no redundant or filler content; each word contributes value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity, the description covers the main purpose but omits important behavioral details like the limit parameter's effect and any assumptions about read-only behavior. It is adequate for basic selection but not fully complete for invocation without referencing the schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has two parameters (limit and project_id) with no descriptions in the schema (0% coverage). The description only mentions 'Speckle project' which maps to project_id, but it does not explain the limit parameter at all. This leaves the agent without an understanding of how to control result size.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool lists models in a Speckle project, using the specific verb 'List' and the resource 'models'. It distinguishes from sibling tools like speckle_projects_list (lists projects) and speckle_model_details (gets details of a model) by specifying the scope as models within a project.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used to get a list of models when you have a project_id, and the context about models holding BIM/CAD data helps set the context. However, it does not explicitly mention when to use this tool over alternatives like speckle_model_details, nor does it state exclusions. The context is clear enough for basic guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• speckle_model_detailsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It communicates that the tool returns detailed info and full version history, which is useful context. However, it does not explicitly state that the operation is read-only, nor does it address potential response size or other side effects. The 'Get' verb implies read-only, but this is not explicitly reinforced.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that efficiently communicates the tool's purpose and key output details. Every phrase earns its place, with no filler or redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists, the description does not need to detail return values. The tool is simple with two required parameters, and the description clearly states what it does. It does not explicitly mention parameter relationships, but the sibling tools and schema context mitigate this gap. Overall, it is adequately complete for a 'get details' operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate by explaining the parameters. It does not mention project_id or model_id at all. The parameter names are self-explanatory, but the description adds no meaning beyond the schema, leaving the agent to infer the relationship between project and model.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb ('Get') and resource ('detailed model info'), and adds distinguishing details such as 'full version history with author, date, and commit messages.' This distinguishes it from sibling tools that list models or projects, making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you need detailed model information and full version history, but it does not explicitly state when to use this tool versus alternatives or mention exclusions. Sibling tool names suggest alternatives, but no direct guidance is provided in the description itself.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• speckle_send_instructionsA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool returns guidance only and has no data-transfer side effect, which is a key behavioral trait. It does not mention authentication or network behavior, but for a simple instruction-fetching tool, these are less critical. The 'Does NOT transfer data' statement adds transparency beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, under 40 words, and completely free of filler. It front-loads the core action ('Get connector-specific instructions') and immediately clarifies the non-transfer behavior. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity, an output schema exists, and no annotations are present, the description covers purpose, return type (guidance), and key exclusions. It is slightly incomplete regarding parameter semantics and the distinction from receive_instructions, but overall the description is sufficient for a straightforward instruction-lookup tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for explaining parameters. It does not mention project_id, model_id, or message at all. The tool name and general context suggest these identify a Speckle project/model, but the description offers no explicit mapping, leaving the agent without guidance on what values to provide.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Get connector-specific instructions for sending BIM data to Speckle.' It identifies the specific verb (get), resource (instructions), and domain (BIM data to Speckle). It also distinguishes from the sibling tool 'speckle_receive_instructions' by focusing on sending rather than receiving.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states what the tool does NOT do ('Does NOT transfer data') and points to an alternative ('use native Speckle connectors for actual data push'). This is a clear usage exclusion. However, it does not mention when to use this tool instead of the sibling 'speckle_receive_instructions', though that is contextually implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• speckle_receive_instructionsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description discloses a key non-behavior (not pulling objects) and points to correct alternatives. It also implies a read-only info-returning operation, though it doesn't detail return formats or side effects beyond what the output schema already covers.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: front-loaded with purpose, then a clear exclusion and alternative. Every sentence earns its place with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the tool's purpose and exclusions, and an output schema exists to detail return values. However, it lacks any guidance on parameter usage or prerequisites, leaving the meaning of version_id ambiguous for a less familiar agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description adds no meaning for project_id, model_id, or version_id. Parameter names are self-explanatory but the optional version_id semantics are entirely undeclared, leaving the agent to infer its purpose.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and resource 'version info and instructions for receiving Speckle data', distinguishing it from sibling tools like speckle_send_instructions. It also explicitly contrasts with data retrieval tools by saying 'Does NOT pull objects'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit when-not guidance: 'Does NOT pull objects — use native connectors or specklepy operations.receive() for data retrieval'. This names alternatives and clarifies that this tool is for instructions, not data fetching.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.