Server Quality Checklist

Tool Scores

• opensolr_ingest_statusB
  3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the behavioral burden. It does convey an informational, non-mutating operation through 'status' and document counts, and adds a reference to the Control Panel. However, it does not explicitly state read-only behavior, handling of unknown job IDs, or whether the status can be polled.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, compact sentence with a parenthetical that efficiently enumerates the returned status fields. The Control Panel note adds minor context without padding, and there is no redundant or filler wording.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is low-complexity and has an output schema, so return-value details are covered externally. However, the description omits usage context and leaves the sole parameter underdescribed, forcing an agent to infer how to acquire a job_id. It is adequate but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema only specifies job_id as a required string with no description, and schema description coverage is 0%. The description does not define job_id, its format, or where to obtain it; it only implicitly connects it to a Data Ingestion job. This is a significant gap at this coverage level.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns the status of a Data Ingestion job and lists the specific status fields (state, processed/success/failed doc counts). This differentiates it from index-management siblings by focusing on ingestion jobs, though it does not explicitly name an alternative.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided about when to use this tool versus alternatives, such as after starting an ingestion job with opensolr_add_documents or when checking job completion. The note about the Control Panel is informational, not usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• opensolr_create_indexB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavioral traits. It does mention that the index is 'vector-enabled' and describes location options plus the ability to request additional regions, which is useful. But it does not disclose whether creation is synchronous, what happens if an index with the same name already exists, whether it is irreversible, or any permission/authorization requirements. These are significant gaps for a mutation operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences with no wasted words. It front-loads the core operation, then supplies parameter context, and ends with an actionable support note. Every sentence adds value and there is no repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Although an output schema exists, the description lacks important context for a create operation: it does not mention whether the operation is idempotent, what happens on duplicate index names, whether creation is immediate or asynchronous, or how the location affects data residency. It also does not specify any prerequisites or error scenarios. Given the tool's simplicity and absence of annotations, the description is not fully complete for safe autonomous invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the absence of parameter documentation. It explicitly explains the location parameter by listing valid regions and pointing to opensolr_vector_regions, which is helpful. However, it never explains the 'index' parameter—its format, allowed characters, uniqueness constraints, or whether it is a display name or an identifier. The description only partially compensates for the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Create a new vector-enabled Opensolr index.' This clearly states the tool's purpose and differentiates it from sibling tools such as opensolr_list_indexes, opensolr_search, and opensolr_add_documents. The 'vector-enabled' qualifier adds further specificity without ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The intended use is implied by the verb 'Create' and the resource 'index,' making it clear this is for creating an index rather than listing, deleting, or searching. However, the description provides no explicit 'use this when...' guidance, no mention of alternatives, and no conditions under which a different tool should be chosen. The location notes give context but not operational selection criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• opensolr_index_infoB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description explicitly states that credentials are intentionally not returned, which is a useful behavioral disclosure beyond the raw schema. With no annotations provided, the description carries the full burden, and this single caveat is helpful but limited. It does not disclose whether the operation requires special permissions, whether it makes network calls to Solr, or what happens if the index does not exist.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two concise sentences with no filler. The primary purpose and key detail (credentials not returned) are front-loaded, and every word contributes meaningful information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has a simple single-parameter interface and an output schema, so the description does not need to detail return values. The description conveys the tool's purpose and a notable security caveat. However, the lack of parameter format explanation and the absence of any usage guidance relative to sibling tools leaves an agent with some uncertainty about how to invoke it correctly in context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has one required parameter, 'index', but its description coverage is 0%, so the schema provides no meaning beyond the parameter name. The tool description also does not explain what format the index parameter should take (e.g., index name, ID, or URL). The description's mention of 'an index' adds minimal context, but the parameter semantics remain largely undocumented, leaving the agent to guess the parameter format.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb ('Get') and resource ('connection details for an index'), including the specific details returned: Solr URL, version, environment. It distinguishes itself from sibling tools that delete, search, create, or add documents, though it does not explicitly name an alternative for index listing or management.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving connection metadata for a single index, and the sibling list shows this is not a search or mutation tool. However, there is no explicit guidance on when to use this over opensolr_list_indexes, which likely provides index-level information while this provides connection details. The usage context is mostly implied rather than directly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• opensolr_delete_documentsA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the behavioral disclosure burden. It does reveal that deletion can be performed by IDs or a raw query, and gives concrete query examples. However, it omits important traits such as irreversibility, commit behavior, permission needs, and the effect of supplying both `ids` and `query`.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, well-structured sentence front-loads the action and immediately provides selection modes with examples. Every word earns its place; there is no redundant or filler content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The output schema likely covers return values, so the description need not detail those. For a destructive operation with no annotations, a note on permanence or at least one expected ID/query requirement would improve completeness. Otherwise the core deletion modes are adequately captured for a tool of this complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds useful meaning to `ids` by clarifying 'Solr ids or your original ids' and gives raw query examples for `query`. Yet it does not describe the required `index` parameter nor whether `ids` and `query` are mutually exclusive or can be combined.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Delete documents') and the two selection modes (IDs or raw query). This clearly distinguishes it from sibling tools like opensolr_search, opensolr_add_documentss, and opensolr_list_indexes, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to choose this tool over alternatives, nor any exclusions or prerequisites. The intended usage is implied by the name and description, but no explicit when/when-not context exists.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• opensolr_list_indexesA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral burden and does state a read-only 'List' operation, which makes side effects unlikely. However, it adds no further context such as pagination, limits, authentication expectations, or whether the list is exhaustive, though the output schema may cover return structure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single efficient sentence that front-loads the verb and resource with no filler. Every word adds meaning, including the 'connected account' scope.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity, zero parameters, and the presence of an output schema, the description is fully adequate. An agent can confidently invoke this tool to list all indexes without missing critical context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and 100% schema coverage, so there is no parameter semantic burden for the description to carry. The baseline for zero parameters is 4, and the description appropriately says nothing about parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('List') and resource ('all search indexes'), and scopes it to the connected Opensolr account. This distinguishes it from sibling tools like opensolr_index_info (specific index details) and opensolr_create_index (creation).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this is the tool for enumerating all indexes, but it does not explicitly state when to use it versus alternatives or when not to use it. For a simple list tool the purpose is mostly self-evident, but there is no direct routing guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• opensolr_add_documentsA
  4.2/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full behavioral disclosure. It reveals the async nature, server-side derived fields, ~1-minute visibility delay, wait semantics, metadata prefixing, URI-based identity with md5(uri), and that job info and Solr ids are returned. This is rich, honest context far beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three lean sentences front-load the core action and then pack high-value behavioral details without fluff. Every sentence earns its place, covering asynchronous execution, identity, metadata mapping, and return value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a moderately complex ingestion tool with no annotations and low schema detail, the description is unusually complete: async behavior, wait semantics, metadata transformation, identity strategy, and return values are all covered. The remaining gaps are minor-ish: the optional 'ids' parameter and prerequisites like the index needing to exist beforehand.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for all 5 parameters. It adds real meaning for wait (default true, blocks), metadatas (prefix meta_* fields), and uri (identity/Solr id). However, it leaves 'ids' unexplained and does not explicitly tie 'texts' and 'index' to their expected formats.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: 'Index plain-text documents via the Opensolr Data Ingestion API.' It clearly identifies the operation and distinguishes it from siblings like search, list, delete, and ingest status by action and purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The usage context is implied by the operation ('Index...documents') and the detailed wait/async behavior, but it never explicitly states when to choose this over alternatives. It also fails to mention prerequisites such as an existing index or the option of using opensolr_ingest_status to poll progress when wait=false.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• opensolr_vector_regionsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. The verb 'List' communicates a read-side effect, and 'currently available' plus the Solr/vector criteria specifies what conditions determine the result. It does not fully describe response shape, but the presence of an output schema reduces that need.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The entire description is one focused sentence that front-loads the action and resource, then adds only the essential clarifying condition. No wasted words or redundant restatements exist.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, no-side-effect lising tool with an output schema available, this description fully covers what the tool does and how it filters. Nothing essential is missing for an agent to select and invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool accepts zero parameters, so there is no parameter meaning to add. The schema already reflects this with an empty properties object, making further description unnecessary and the baseline 4 appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') with a clear resource ('vector-enabled Opensolr environments') and a concrete eligibility filter ('Solr 9.x with dense vectors and the hybrid query parser'). It separates this from general index-listing siblings like opensolr_list_indexes by emphasizing vector-enabled.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies its use case: if you need vector-capable Solr environments, call this tool. However, it never explicitly contrasts it with alternatives, such as opensolr_list_indexes, or states when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• opensolr_ai_answerA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations available, the description carries the full burden and does so thoroughly: it explains that retrieval runs through a tuned hybrid pipeline, saved Search Tuning applies automatically, the top rag_docs hits are used as context, and it details tuning knobs including the subtle distinction between freshness_boost and fresh_bias. This goes well beyond a bare function statement.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose and uses clear separators for parameter explanations. The exhaustive tuning list is justified because the tuning schema is a generic object, but the meta sentence 'That list is the whole set, not a sample...' adds unnecessary narrative and could be trimmed without losing value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity — 7 parameters, no annotations, and a generic tuning object — the description covers all essential operational context: what the answer is grounded in, how retrieval works, how parameters affect behavior, and which tuning options are supported. Since an output schema exists, omission of return-value details is acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for all 7 parameters. It explains query as a question, index as the target index, rag_docs and rag_words as retrieval context sizing, filter_query as a raw Solr fq expression, instruction as a prompt replacement, and tuning as an override mechanism with the complete list of supported knobs. This is strong compensation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action: 'Ask a question and get a grounded RAG answer generated ONLY from the content already indexed in the given Opensolr index.' It identifies the tool's resource (the index) and distinguishes it from raw search or management tools by emphasizing the RAG answer and grounded retrieval pipeline.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when to use the tool: for grounded question answering over already-indexed Opensolr content, with the same retrieval pipeline as the hosted search UI. It does not explicitly contrast this with sibling tools like opensolr_search or describe when not to use it, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• opensolr_searchA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations available, the description carries the full burden, and it is exceptionally transparent. It discloses that lexical mode uses zero AI quota, that fresh_bias re-orders but never filters, that hit counts stay unchanged, that documents without creation_date remain unboosted, and that fresh_bias defaults to off.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but every sentence earns its place, providing necessary detail for an 8-parameter tool with zero schema descriptions. It is front-loaded with the core purpose and uses line breaks to keep complex mode and bias semantics readable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  All meaningful parameters are covered in the description, and the output schema exists to define the return shape. Given the tool's complexity and absent annotations, the description gives an agent enough context to select modes, set filters, and understand ranking behavior without guessing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate, and it does. It explains search_mode values, the hybrid-mode sub-parameter 'mode', alpha's semantic-vs-lexical weighting, filter_query syntax with an example, fresh_bias behavior, and k in the opening sentence.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb and resource: search an Opensolr index and return the k most relevant documents. This clearly differentiates the tool from sibling operations like listing, deleting, or ingesting indexes, and the first sentence is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides strong guidance on when to choose lexical vs semantic/hybrid search modes, when fresh_bias should be enabled, and how filter_query works. However, it does not explicitly compare against sibling tools such as opensolr_ai_answer or explain when one should be preferred over the other.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.