Server Quality Checklist

Tool Scores

• openalex_search_institutionsA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It describes the tool as a search (read-only) returning specific fields and how to use the ID. It does not mention destructive behavior, rate limits, or authentication, but these are not critical for a search tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the purpose, and contains no unnecessary words. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers what the tool searches, what it returns, and how to use the result. Without an output schema, it explains return values well. It could mention that country_code and institution_type are filters, but overall it is fairly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain any of the four parameters (query, per_page, country_code, institution_type). It only mentions returned fields, not how to use the inputs. The description adds no value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Search' and the resource 'universities, research institutes, hospitals, and other organizations in OpenAlex.' It lists returned fields and how to use the OpenAlex ID, distinguishing it from sibling tools that search works, authors, or sources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance: 'Use the returned openalex_id in openalex_search_works filter'. It implies this tool is for finding institution IDs before filtering works, but does not explicitly state when not to use it or mention alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• openalex_search_sourcesA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears the full burden. It states it searches and returns data, implying a read-only operation, but does not disclose pagination behavior, rate limits, or sorting. The description adds moderate context beyond the name.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no redundancy: first sentence defines purpose and output, second provides a practical usage hint. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description lists key return fields and provides a usage tip. However, it is incomplete regarding parameter details and does not describe the structure of results (e.g., pagination info). Still, for a search tool, it covers the most critical context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain any of the five parameters (query, filters, is_oa, per_page, source_type). The agent must infer functionality from parameter names alone, which is insufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the verb (Search), resource (journals, conference proceedings, etc.), and lists specific return fields (ISSN, publisher, open-access status, etc.). This distinguishes it from sibling tools that search works, authors, or institutions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance on using the returned 'openalex_id' in work filters, which is a direct usage hint. However, it does not contrast with alternatives like openalex_get_source for when to search versus retrieve a specific source.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• openalex_search_authorsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, description carries burden. It mentions returns fields but omits pagination, rate limits, or safety (though read-only is implied). Adequate but not fully transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences, front-loaded with main action and purpose, no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 4 params with no schema descriptions and no output schema, description covers core purpose but fails to detail filter syntax or pagination. Adequate for basic use but not comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description must explain parameters. It names query and filters but lacks syntax/format details. Only high-level meaning added.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it searches authors by name or filters, returns specific fields, and distinguishes from openalex_get_author for full profiles.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says when to use search vs get_author, and lists returned fields for informed selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• openalex_aggregate_worksA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses the return format (list of {key, label, count} groups sorted descending) and mentions combining with filters. It lacks details on rate limits, authentication, or potential performance issues, but covers core behavior well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is only 3 sentences, front-loaded with the core action, and each sentence adds value. No unnecessary information or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 3 parameters and no output schema. The description explains return format and usage, but lacks detailed parameter constraints, especially for group_by valid values. The cross-reference to another tool's syntax helps but is not fully self-contained, leaving gaps for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Given 0% schema coverage, the description must compensate. It mentions group_by but does not explain valid values (e.g., field names). Query and filters are described only by cross-reference to another tool's syntax, which is helpful but not self-contained. The schema provides no further clues, so some ambiguity remains.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool aggregates works by a field to get distribution statistics, with specific examples (annual trends, type breakdown, top institutions, open-access percentage, topic distribution). It effectively distinguishes from sibling tools like openalex_search_works, which is for searching individual works.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit use cases and mentions combining with 'filters' referencing openalex_search_works syntax. It implies when to use aggregate vs search but does not explicitly state when not to use it or alternative tools for other tasks.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• openalex_get_sourceA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It implies a read operation but does not explicitly state idempotency, rate limits, or error behavior. Adequate but could be more explicit about safety.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence that front-loads the purpose and lists key fields. Could be slightly more structured but is efficient and informative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and 0% schema coverage, the description provides a good overview of input format and returned data fields. Covers essential information for tool invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but description adds significant meaning: specifies that source_id can be an S-number or ISSN, providing format guidance beyond the raw schema type.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states 'Get full details for a journal, conference, or repository' and lists specific fields like ISSN, publisher, open-access status. Distinguishes from sibling tools like search_sources by focusing on retrieving details by ID.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly mentions acceptable input formats: 'Accepts OpenAlex Source ID (S-number) or ISSN.' Implies usage context but does not explicitly state when not to use or compare to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• openalex_get_authorA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It details the return fields but does not disclose any behavioral aspects such as rate limits, data freshness, or read-only nature.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise, front-loaded sentences with no extra words, effectively communicating purpose and acceptance criteria.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description lists key return fields, making the tool's output clear. However, it could mention response structure or pagination if applicable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Only one parameter exists, and the description adds meaning by specifying the format (OpenAlex Author ID or ORCID), compensating for the 0% schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves a full researcher profile with specific fields listed, and it specifies the accepted identifiers (OpenAlex Author ID or ORCID), distinguishing it from sibling tools like search_authors.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool (when a full profile is needed) and contrasts with search tools by listing detailed fields, but does not explicitly state when not to use it or provide alternative tool names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• openalex_search_worksA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must bear the burden. It mentions read operations (search, filter, sort) and return fields, but lacks explicit disclosure of pagination behavior, rate limits, or non-destructive nature.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences, each adding unique value: purpose, usage hints, filter syntax, sort options, and return fields. No redundant or missing elements.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 5 parameters, no output schema, and no annotations, the description adequately covers purpose, usage, and return fields. Lacks detail on pagination limits or error handling, but sufficient for an agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description compensates by explaining the purpose of 'query', 'filters', and 'sort' parameters, including example filter syntax. It implicitly explains 'page' and 'per_page' via context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches scholarly works, listing specific types (articles, books, datasets, preprints, theses). It distinguishes from sibling tools like openalex_get_work (single work retrieval) and openalex_aggregate_works (aggregation).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly explains when to use 'query' and 'filters' parameters, provides filter syntax and sort options. While it does not mention when not to use it or directly compare to alternatives, the context from sibling names implies differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• openalex_get_institutionA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It correctly implies a read-only operation with 'Retrieve' and lists output fields, but omits any operational traits such as rate limits, authentication needs, or potential error conditions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with no wasted words. The first sentence lists key fields, the second clarifies input format. Information is front-loaded and every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity of the tool (single required parameter, no output schema), the description is complete. It covers the purpose, input format, and output content, making it sufficient for an AI agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage for the parameter, but the description explains that it accepts I-numbers or ROR IDs, adding crucial context that the schema alone does not provide.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The verb 'Retrieve' and resource 'full details for a university or research institution' clearly specify the tool's function. Listing specific fields (name, country, type, etc.) differentiates it from sibling tools like openalex_search_institutions, which are for searching rather than fetching by ID.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description states that it accepts an OpenAlex Institution ID or ROR ID, providing clear usage context. However, it does not explicitly mention when to avoid this tool or suggest alternatives (e.g., use openalex_search_institutions when you don't have an ID).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• openalex_get_workA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It lists return fields (title, abstract, authors, etc.) and accepted identifier formats, but does not mention error handling, rate limits, or authentication needs. For a simple read operation, this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single paragraph of three sentences, front-loaded with the action and purpose. Every sentence adds value, and there is no redundant or extraneous text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (one required parameter, no output schema), the description is complete. It explains what the tool does, what input is needed, and what the output contains, with no missing critical information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides no description for the 'identifier' parameter (0% coverage). The description compensates by explaining that the parameter accepts OpenAlex ID (W-number), DOI, PubMed ID, or MAG ID, and describes formats for DOIs and PubMed IDs. This adds significant meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves full metadata for a single scholarly work, and specifies accepted identifier types (OpenAlex ID, DOI, PubMed ID, MAG ID). This distinguishes it from sibling tools like 'openalex_search_works' which are for searching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implicitly provides usage context by listing identifier formats and stating the tool is for a single work. It does not explicitly mention when not to use or alternative tools, but the list of sibling tools provides that context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.