Server Quality Checklist

Tool Scores

• get_resume_summaryA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must fully disclose behavioral traits. It explains what the tool returns (self-introduction, name, title, contact info) and the length parameter's effect. However, it does not mention whether the operation is read-only, any authentication requirements, or rate limits. For a simple get operation, this is adequate but not exceptional.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded: two sentences of purpose followed by a clear parameter definition. Every sentence adds value, and there is no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema), the description is largely complete. It covers what the tool returns and how to use the length parameter. It could be slightly more explicit about the return format or structure, but it is sufficient for an agent to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the lack of parameter info. It does so excellently by explaining the 'length' parameter with three concrete options ('short', 'medium', 'long') and their meanings. This adds significant value beyond the schema's bare type and default.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get a self-introduction / resume summary, plus name, title, and contact info.' It also provides concrete use cases ('tell me about yourself' or 'give me a summary of this person's background'). This distinguishes it from sibling tools like list_projects and search_skills.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly tells when to use the tool: 'Use this to answer... style questions.' This gives clear context. However, it does not explicitly state when not to use it or point to alternative tools, which would be a minor improvement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_skillsA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that results are ranked by relevance and include projects, but does not explicitly state that the tool is read-only, mention any authentication needs, rate limits, or edge cases like no matches. It is adequate but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two paragraphs: the main purpose and the args section. Every sentence adds value, and the examples are front-loaded. There is no waste, and the structure is efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (single parameter, output schema exists), the description is complete. It explains the search behavior, relevance ranking, and inclusion of projects. Since an output schema is present, there is no need to detail return values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has a single parameter 'keyword' with 0% description coverage. The description compensates fully by providing clear examples ('e.g., "RAG", "Docker", "vector database", "iOS", "Next.js"') and explaining the expected format, which adds significant meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('search') and resource ('skills/technology taxonomy'), states it returns matches ranked by relevance with projects, and provides example questions like 'does this person know RAG / Docker / vector databases / iOS development?' This clearly distinguishes it from sibling tools (list_projects, get_project_details, get_resume_summary).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use this to answer questions like...' which gives clear context for when to use the tool. While it does not mention when not to use it or name alternatives, the sibling tools are not related to skills search, so the context is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_projectsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full transparency burden. It explains that links are included to avoid a second call, and describes the related_project field and its purpose. It does not mention any side effects (none expected), but could be more explicit about the read-only nature. Still, it provides useful behavioral context beyond a simple list.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear topic sentence, then enumeration of fields, an explanation of related_project, and usage guidance. Every sentence adds value. It is slightly long but not verbose; it could be tightened slightly (e.g., remove 'whichever apply' as it's implied).

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that the tool has no parameters and an output schema exists, the description is quite complete. It explains the output fields, the role of related_project, and when to use it. However, it does not mention ordering or limiting of results, and the portfolio size is assumed small. For most use cases, this is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so schema coverage is 100% trivially. The baseline for no parameters is 4, as the description does not need to add parameter semantics. However, it does describe the output fields, which is beneficial for understanding the tool's result but not directly about input parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it lists every item in the portfolio with specific fields, and distinguishes itself from the sibling 'get_project_details' by emphasizing that links and related_project are included for a comprehensive overview. The verb 'List' and resource 'projects' are specific, and the mention of 'not just the flagship case studies' clarifies scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit usage guidance is provided: 'Call this first for any broad question like "what has this person worked on?" or "does a system exist for X?".' This tells the agent when to use this tool and implicitly when not to (e.g., deep-dive should use get_project_details). No exclusions or alternatives needed beyond the sibling context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_project_detailsA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully bears the burden of behavioral disclosure. It clearly conveys the tool's non-destructive, read-only nature by stating it retrieves records. However, it does not disclose potential side effects like logging, or rate limits, which slightly limits transparency. The indication that matching is forgiving and alias-aware adds valuable behavioral context, justifying a 4.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured, front-loading the tool's purpose in the first sentence, then elaborating on behavioral nuance (flagship vs lighter items) in a natural flow. Every sentence adds value, and the Args section is clearly separated and self-contained. There is no wasted text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has only one parameter, no annotations, and no output schema, the description provides sufficient context for an AI agent to select and invoke the tool correctly. It covers input semantics, matching behavior, variation in returned data, and even suggests a complementary sibling tool (list_projects). The description is complete for this single-param retrieval tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage and only one parameter ('name'), so the description must fully compensate. It excels by describing acceptable inputs (project name, id, alias, or competition name), provides concrete examples, and explains matching behavior (case-insensitive, partial, alias-aware). This adds rich semantics far beyond the schema's bare type declaration.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool retrieves the full record for one portfolio item, differentiating between flagship projects (returns detailed fields like role, tech stack, outcomes) and lighter items (returns description and links). This clear verb+resource+variation makes the purpose highly specific and distinct from siblings like list_projects.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool (to get full details of a single portfolio item) and includes a clear when-not alternative: it advises calling list_projects first to select the right item when unsure about the name. This pre-emptive guidance prevents misuse and clarifies the tool's role in a workflow.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.