Server Quality Checklist

Tool Scores

• create_omi_conversationC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states this is a creation operation, implying it's a write/mutation tool, but doesn't disclose any behavioral traits like permission requirements, rate limits, side effects, or what happens on success/failure. For a creation tool with zero annotation coverage, this leaves significant gaps in understanding how it behaves.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that gets straight to the point without unnecessary words. It's appropriately sized for a creation tool, though it could potentially benefit from slightly more context given the lack of annotations and usage guidelines.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (8 parameters, nested objects, no output schema, and no annotations), the description is insufficient. It doesn't explain what a successful creation returns, error conditions, or behavioral constraints. For a creation tool with multiple parameters and no structured safety hints, more descriptive context is needed to be complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the schema already documents all 8 parameters thoroughly. The description adds minimal value beyond the schema by mentioning 'text content and metadata', which loosely maps to the 'text' and other fields. Since the schema does the heavy lifting, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'creates' and the resource 'new Omi conversation', specifying it includes 'text content and metadata'. This distinguishes it from sibling tools like 'read_omi_conversations' (read vs. create) and 'create_omi_memories' (conversation vs. memories). However, it doesn't explicitly differentiate from 'create_omi_memories' beyond the resource name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention when to choose this over 'create_omi_memories' or whether it's for initial conversation creation versus updates. There's no context about prerequisites, dependencies, or typical use cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_omi_conversationsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'pagination and filtering options', which hints at some behavior, but fails to cover critical aspects like authentication requirements, rate limits, error handling, or what the return format looks like (e.g., JSON structure). For a retrieval tool with 5 parameters, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('retrieves user conversations from Omi') and adds relevant details ('with pagination and filtering options'). There is no wasted verbiage, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a retrieval tool with 5 parameters, no annotations, and no output schema, the description is incomplete. It lacks details on authentication, rate limits, error cases, and the structure of returned data (e.g., conversation objects). Without annotations or an output schema, the agent has insufficient information to handle this tool effectively in context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema fully documents all 5 parameters with descriptions. The description adds minimal value by mentioning 'pagination and filtering options', which loosely corresponds to parameters like 'limit', 'offset', and 'statuses', but doesn't provide additional semantics beyond what the schema already specifies. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('retrieves') and resource ('user conversations from Omi'), making the purpose evident. It also mentions 'pagination and filtering options' which adds specificity. However, it doesn't explicitly distinguish this tool from its sibling 'read_omi_memories', which might cause confusion about when to retrieve conversations versus memories.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'read_omi_memories' or 'create_omi_conversation'. It mentions filtering options but doesn't specify scenarios or prerequisites for usage, leaving the agent without context for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_omi_memoriesB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. While it mentions the creation action and two input methods, it doesn't disclose important behavioral traits like whether this is a write operation (implied but not stated), what permissions are needed, whether it's idempotent, what happens on failure, or what the return format looks like. For a creation tool with zero annotation coverage, this leaves significant gaps in understanding how the tool behaves.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that clearly states the tool's purpose and two key input methods. It's front-loaded with essential information and contains no redundant or unnecessary words, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that this is a creation tool with no annotations and no output schema, the description is insufficiently complete. It doesn't explain what 'Omi memories' are in this context, what the tool returns (e.g., success/failure, created memory IDs), or any behavioral constraints (e.g., rate limits, authentication needs). For a tool with 5 parameters and significant functionality, more context is needed to use it effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, meaning all parameters are well-documented in the input schema itself. The description adds minimal value beyond the schema by mentioning 'extracting from text' (hinting at the 'text' parameter) and 'using explicit memory objects' (hinting at the 'memories' parameter), but doesn't provide additional semantic context like examples, edge cases, or relationships between parameters. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('creates Omi memories') and specifies two methods ('extracting from text' or 'using explicit memory objects'), which gives a good sense of what the tool does. However, it doesn't differentiate itself from sibling tools like 'create_omi_conversation' or 'read_omi_memories', leaving some ambiguity about when to use this specific memory creation tool versus other memory/conversation tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by mentioning two input methods ('extracting from text' or 'using explicit memory objects'), which provides some context for when to use it. However, it doesn't explicitly state when to choose this tool over alternatives like 'create_omi_conversation' or 'read_omi_memories', nor does it mention any prerequisites or exclusions. The guidance is present but incomplete.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_omi_memoriesB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'pagination options', which adds some context about how results are handled, but it does not cover other aspects like rate limits, authentication needs, error conditions, or what the return format looks like. This leaves gaps in understanding the tool's behavior beyond basic retrieval.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function and key feature (pagination). It is front-loaded with the core purpose and avoids unnecessary words, making it highly concise and well-structured for quick comprehension.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (3 parameters, no output schema, no annotations), the description is minimally adequate. It covers the basic purpose and hints at pagination but lacks details on return values, error handling, or usage context. This leaves the agent with incomplete information for effective tool invocation, though it meets a baseline for a read operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, documenting all three parameters (limit, offset, user_id) with details like defaults and constraints. The description adds no additional meaning beyond this, as it only mentions 'pagination options' without elaborating on parameter usage. This meets the baseline for high schema coverage but does not enhance parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Retrieves') and resource ('user memories from Omi'), making the purpose specific and understandable. However, it does not explicitly differentiate from sibling tools like 'read_omi_conversations', which might retrieve a different type of data, so it lacks sibling differentiation for a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives, such as when to choose it over 'read_omi_conversations' or other siblings. It mentions pagination options but does not specify scenarios or prerequisites for usage, leaving the agent without contextual direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.