Server Quality Checklist

Tool Scores

• riven_usageA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, and the description only states that it returns usage statistics. It does not disclose any behavioral traits such as read-only nature, rate limits, or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no unnecessary words, making it highly concise and efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema and low complexity. The description explains the purpose but lacks details on what specific usage statistics are returned, leaving some ambiguity for the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters, so the description adds no additional semantic value beyond the schema. Baseline score of 4 is appropriate for zero-parameter tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (return) and the resource (current usage statistics for the authenticated Riven account), distinguishing it from sibling tools like riven_chat or riven_research.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives, nor any exclusions or prerequisites beyond the authenticated account context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• riven_chatA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states that the tool returns completion text, which is minimal. It does not disclose potential behavioral traits such as rate limits, authentication requirements, or whether it uses conversation history. For a simple generation call, this is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, with no redundant information. The first sentence clearly defines the function and output, the second provides usage guidance. Every word serves a purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (3 parameters, no output schema, no nested objects), the description covers purpose, parameters, and usage. It explains that the output is completion text. While more detail on token limits or error handling could be added, the current description is sufficient for an agent to understand and use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, baseline is 3. The description adds value by providing example model IDs (e.g., 'rvn-assistant-v2', 'glm-5.2') and noting the default. However, the prompt and system descriptions largely reiterate the schema. Overall, some added detail but not substantial.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the action (send a single prompt), the resource (Riven chat model), and the result (return completion text). It also clearly distinguishes from sibling tools by indicating it's for non-agentic, single-turn questions, contrasting with 'riven_research' which presumably handles agentic research.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear usage context: 'quick, single-turn questions that don't need agentic research.' This implicitly tells the agent when to use this tool vs. riven_research. However, it does not explicitly mention other siblings like riven_council or riven_usage, which might serve different purposes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• riven_councilA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It describes the basic behavior (creation, multiple models, synthesized verdict) but does not disclose potential side effects, authentication needs, or other behavioral traits beyond the stated flow.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, no wasted words. First sentence defines the functionality, second provides usage guidance. Highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose and usage guidelines adequately. No output schema is present, but the description mentions the output (synthesized verdict plus individual answers). The three parameters are fully described in the schema. Slightly lacking in behavioral transparency, but overall adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already explains each parameter. The description does not add extra context beyond what the schema provides, resulting in baseline 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states 'Create a Riven Computer council task' and explains that it involves multiple models producing a synthesized verdict plus individual answers, clearly distinguishing it from sibling tools like riven_chat.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes a clear usage guideline: 'Use when you want a cross-checked or debated answer rather than a single model's view.' It does not explicitly state when not to use, but the guidance is sufficiently clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• riven_researchA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must fully convey behavior. It discloses that the tool creates, runs, polls, times out, and returns a summary. However, it omits side effects (e.g., whether tasks persist), auth requirements, and error handling details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences: first sentence efficiently lists the full workflow, second sentence provides usage guidance. No redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description adequately describes the return as a 'summary of the resulting thread.' It covers the core workflow and usage context. Minor gap: does not specify what the thread contains or error behavior, but sufficient for a research tool with sibling alternatives.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for all 3 parameters. The description adds value by explaining the polling and timeout logic, contextualizing 'timeout_seconds', and clarifying the return format ('summary of the resulting thread'), which enriches the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool creates a Riven Computer research task, runs it, polls, and returns a summary. It specifies the output type (summary of resulting thread). Differentiates from siblings (riven_chat, riven_council, riven_usage) by targeting open-ended multi-step research.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit usage guidance: 'Use for open-ended research questions that benefit from multi-step agentic work.' Provides clear context but does not explicitly exclude scenarios (e.g., simple Q&A) or name alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.