Server Quality Checklist

Tool Scores

• company_kb_syncB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description must disclose safety and side effects. It states the synchronization and index update but does not explain whether the operation modifies/overwrites local data, what 'force' does, what happens to unchanged docs, network/availability requirements, or failure modes. This is a mutation-like operation with no behavioral safeguards disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core action and followed by useful invocation context. No filler. Slightly too sparse to cover parameter guidance, but as a structure it is efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With three undocumented parameters, no annotations, and only a brief usage note, the description is not complete enough to invoke the tool correctly in non-default cases. It provides the 'when' but not the 'how to configure' information; the output schema does not compensate for missing parameter semantics.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the description contains no mention of force, scope, or project. The agent must infer the meaning of these parameters from their names/defaults alone, which is not sufficient for correct invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb ('Synchronize') and resource ('repositories that have already been added to the local RAG cache') and mentions updating the Markdown index. It clearly differentiates from normal search by calling itself a full sync, but does not explicitly distinguish from the similar sibling company_kb_sync_repository.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Gives explicit when-to-use guidance: 'Use this before answering when the user asks for fresh docs'. It also explains that 'normal search does not require a full sync', helping the agent avoid unnecessary calls. Does not name company_kb_sync_repository or other alternatives explicitly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• company_kb_answerB
  3.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and uses it well: it discloses the key behavioral boundary that the tool 'does not invent final policy' and returns cited evidence for Copilot synthesis. This prevents an agent from treating the output as authoritative policy. It doesn't cover edge cases like missing sources or citation failures, but the core behavioral contract is clear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no redundancy: the first states the tool's operation, the second clarifies its output boundary. The most important behavioral caveat is front-loaded and every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  At 4 parameters with no annotations and no parameter documentation, the description is too lean to be complete. It handles the core purpose well but omits practical invocation details such as what filters accept, how limit affects results, whether requireCitations can be disabled, and how this relates to company_kb_search. The output schema presence helps but does not fill these gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate, but it only weakly touches parameters: 'natural-language question' relates to the required question, and 'cited evidence' hints at requireCitations. It provides no explanation of limit, filters, or requireCitations semantics, so an agent parsing optional parameters would remain under-informed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves a focused evidence pack for a natural-language documentation question, with a specific verb and resource. It doesn't explicitly distinguish itself from the sibling company_kb_search, but the 'evidence pack for Copilot to synthesize' framing gives it a distinct identity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use it: for natural-language questions needing cited evidence and synthesis guidance rather than final policy. However, it doesn't provide explicit alternatives or when-not-to-use conditions, leaving sibling differentiation to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• company_kb_open_docA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It conveys a read-oriented retrieval action and a follow-up context, but it does not disclose details like how chunk references are resolved, whether version/commit matters, truncation behavior, or side-effect safety. Still, 'open' and 'after search' imply a non-destructive lookup.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences with no filler, and the primary operation is stated first. The usage guidance is placed immediately after, making the most important information front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 7 optional parameters, 0% schema coverage, and no annotations, the description is too sparse to fully support invocation. It explains the high-level purpose and when to use it, but leaves parameter semantics, expected inputs from search results, and behavioral nuances unexplained. The presence of an output schema helps but does not close these gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the 7 undocumented parameters. It only hints at concepts like 'chunk by reference' and 'source lines', which loosely map to chunkId and lineStart/lineEnd, but it fails to explain path, commit, project, maxChars, or how to construct a valid call from search results.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the operation: opening an exact document excerpt or indexed chunk by reference. It also distinguishes the tool from siblings by stating it is meant as a follow-up to search, rather than a search or repository-management action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says when to use the tool: after search, when surrounding context or precise source lines are needed. It does not mention when not to use it or name alternatives, so it stops short of fully routing behavior across multiple siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• company_kb_recent_changesA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral disclosure burden. It communicates that this is a read-only listing operation and that only indexed Markdown documents are included, but it does not explain ordering, how 'recently' is determined, or project scoping behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no wasted words. The core action is front-loaded, and the usage guidance follows immediately. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a low-complexity tool with an output schema and three optional parameters with defaults, the description is largely adequate. It explains what the tool returns, when to use it, and the Markdown/indexed scope. It could be more complete by tying 'time window' to the days parameter, but it is not seriously incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not mention days, limit, or project. The parameter names are somewhat self-explanatory, but the description does not clarify that 'days' controls the time window or that 'project' filters by repository, leaving the agent to infer this.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description opens with a specific verb and resource: 'List recently changed indexed Markdown documents.' This clearly distinguishes it from sibling tools like company_kb_search (search) and company_kb_open_doc (open one document).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says when to use the tool: when the user asks what changed recently or wants documentation updates within a time window. It does not explicitly name alternatives or state when not to use it, but the usage context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• company_kb_remove_repositoryA
  4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It explicitly states destructive local effects: deleting index entries, vectors, and by default the cached clone. It also clearly reassures that it never writes to GitLab. This is strong transparency for a destructive operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no filler. The first sentence states the core action and scope; the second adds essential destructive-effect detail and a safety guarantee. Every sentence contributes meaningful information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description conveys the tool's purpose, destructive scope, and safety boundary well, and an output schema exists so return values need not be described. However, the missing parameter semantics and lack of guidance on selecting between url and project leave the definition incomplete for confident invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for explaining url, project, and deleteCache. It does not explain how to identify the repository (url vs project, either or both, precedence) and only indirectly hints at deleteCache through 'by default the local cached clone'. This is insufficient for correct invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action ('Remove'), a specific resource ('one synchronized GitLab repository'), and the local scope ('local RAG system'). It also contrasts with sibling tools by emphasizing this operates locally and never writes to GitLab.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool is for removing a repository that has been synchronized, and the sibling company_kb_sync_repository suggests the inverse operation. However, it does not explicitly state when to use this tool versus alternatives or mention any conditions or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• company_kb_searchA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses scope ('synchronized') and output behavior ('ranked snippets with stable GitLab citations'), and the verb 'search' implies a read-only operation. It does not address auth, rate limits, or no-result behavior, but those are less critical for a read-oriented search tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences front-load the action, resource, and output type, then immediately give usage examples. Every word earns its place with no filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  An output schema exists, so return-value details are supplied structurally. The description covers scope, use cases, and output style, which is enough for basic usage; the main gap is unexplained optional parameters like filters and searchMode.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description only clarifies the query concept via the tool's purpose. It never explains limit, filters, or searchMode; 'hybrid' in particular is ambiguous, and filters has no structural constraints to help an agent infer valid values.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb ('Search') over a specific resource ('synchronized company Markdown documentation') and describes the output ('ranked snippets with stable GitLab citations'). It distinguishes itself from siblings like company_kb_answer and company_kb_open_doc by framing this as discovery/evidence gathering rather than direct answering or opening a single doc.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit use cases: 'discovery questions, finding runbooks, locating ADRs, or collecting evidence before answering.' It does not explicitly name alternatives or when-not-to-use conditions, but the 'before answering' phrasing implies a boundary with company_kb_answer.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• company_kb_sync_repositoryA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explains that the repository is cloned into a 'local read-only cache,' indexed, and made searchable, which conveys non-destructive intent. It also discloses the auth behavior (env default, anonymous fallback, override options) and explicitly warns not to pass token values. Some details about the 'force' parameter are not disclosed, but the core behavior is transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences with no filler. The main action is front-loaded, then the cache/indexing behavior, then auth instructions. Every sentence earns its place and the warning about token values is a valuable guardrail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a sync tool with four parameters, no annotations, but an output schema, the description is reasonably complete: it states the action, effect, and auth behavior. The main omission is the semantics of 'force' and the exact role of 'tokenEnv', but the description still provides enough for correct invocation in common cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must fill the gap. It clarifies authMode values ('anonymous, env, or ssh-agent') and warns about token values, which helps with tokenEnv. However, the 'force' parameter is not explained at all, and 'url' format is left implicit. The description adds useful meaning for auth-related parameters but does not fully compensate for the complete lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Synchronize one GitLab repository by URL.' It clearly states what happens (cloning into a read-only cache, indexing, making searchable), which differentiates it from sibling tools like list_repositories or search. This is unambiguous about scope: one repository, not a bulk sync.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool: when a specific GitLab repository needs to be synced into the local knowledge base. It also provides practical usage guidance on auth mode selection and warns against passing token values. It does not explicitly name alternative tools or exclusions, but the context is clear enough for an agent to select it over siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• company_kb_list_repositoriesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It clearly indicates a read-only listing operation ('List... that have been synchronized'), reveals the local cache scope (not a live GitLab query), and specifies the types of metadata returned. This is sufficient transparency for a simple list operation, though it could optionally mention that no parameters are needed or that it requires an existing cache.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The entire description is one focused sentence that front-loads the action ('List') and immediately specifies the resource and distinguishing scope. Every clause adds value: the local cache scope and the metadata fields are both essential and non-redundant with the tool name.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter read-only list tool with an output schema, the description covers all necessary context: what is listed, the scope filter (synchronized repositories), and the key output components. It is fully adequate for an agent to select and invoke this tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters and the schema is empty, so schema coverage is effectively 100%. The description does not need to explain parameters; the baseline of 4 for a zero-parameter tool applies, and the description adds no irrelevant parameter information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb 'List' and a clear resource ('GitLab repositories that have been synchronized into the local RAG cache'), plus explicit detail about returned data ('local index counts and last sync metadata'). This clearly distinguishes it from sibling tools like company_kb_sync or company_kb_status, which focus on synchronization actions or status rather than listing cached repositories.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description makes the intended context clear: it lists repositories already synchronized into the cache, implying this is the tool to consult when you need to know what is currently available locally. It does not explicitly name alternatives or state when not to use it, but the scope is specific enough that an agent can infer appropriate usage without confusion.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• company_kb_statusA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the behavioral burden. The description clearly frames this as a read-only diagnostic ('Check configuration...') and explains what aspects it inspects. It stops short of explicitly stating 'does not modify anything,' but 'status' and 'Check' strongly imply a non-mutating operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no filler. The first sentence states exactly what the tool does, and the second provides concrete trigger conditions. Everything included earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters, an output schema exists, and the operation is a simple health/status check, the description covers the essential context: what is checked and when to use it. No critical information appears to be missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and 100% schema description coverage, so there is no parameter information the description needs to add. The baseline for zero-parameter tools is 4, and the description appropriately does not introduce unnecessary parameter discussion.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Check') and names concrete resources: configuration, GitLab authentication, repository sync state, and index size. This makes the tool's purpose immediately clear and distinct from the sibling operation tools like company_kb_sync or company_kb_search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit use cases: 'when search looks stale or broken' and 'when the user asks whether the MCP server is healthy.' It provides clear context but does not explicitly state when not to use it or mention sibling alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.