Server Quality Checklist

Tool Scores

• steady_pingB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states the tool verifies access via cookies and checks the /check-ins endpoint, but doesn't disclose behavioral traits like what happens on success/failure, whether it's read-only or has side effects, error conditions, or rate limits. The description is minimal and lacks operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function without unnecessary words. It's appropriately sized and front-loaded, with zero waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (0 parameters, no annotations, no output schema), the description is adequate but minimal. It covers the basic purpose but lacks details on behavior, usage context, or output format, which could be helpful for an agent to understand how to interpret results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so no parameter documentation is needed. The description doesn't add parameter details beyond the schema, but that's appropriate here. Baseline is 4 for zero parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Verify the stored cookies can access Steady (checks /check-ins).' It specifies the verb ('verify'), resource ('stored cookies'), and target system ('Steady'), though it doesn't explicitly differentiate from sibling tools like steady_login or steady_set_cookies.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. The description implies it's for verifying cookie-based access to Steady, but doesn't specify prerequisites (e.g., cookies must be set first) or when to choose this over steady_login for authentication checks.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• steady_submit_checkinB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions using Steady's web form (not an official API), which hints at potential reliability or rate limit issues, but doesn't specify authentication requirements, error handling, or what happens on submission (e.g., confirmation, side effects). For a mutation tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the main purpose and efficiently lists supported fields in a single sentence. It avoids redundancy and is appropriately sized for the tool's complexity. However, it could be slightly more structured by separating usage notes from field descriptions.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is moderately complete but has gaps. It covers the tool's purpose and parameters well via the schema, but lacks details on behavioral aspects like authentication, error handling, or return values. For a mutation tool with 6 parameters, it should provide more context on how the tool behaves in practice.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all 6 parameters thoroughly. The description adds minimal value by listing supported fields (previous work, next work, blockers, mood) but doesn't provide additional context beyond what's in the schema (e.g., format details for 'team' or 'date'). Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Submit today's Steady check-in') and resource ('for ONE team'), specifying it's for a single team rather than multiple. It distinguishes from siblings by mentioning it uses Steady's web form (not an official API), but doesn't explicitly differentiate from steady_login or steady_set_cookies which might be related to authentication. The purpose is specific but could better contrast with other tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for submitting daily check-ins and mentions it's for one team, but provides no explicit guidance on when to use this tool versus alternatives like steady_list_teams or steady_login. It doesn't state prerequisites (e.g., needing authentication first) or exclusions (e.g., not for historical dates beyond today). Usage is contextually implied but lacks clear directives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• steady_list_teamsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool lists teams with name and id from a specific page, which is useful context. However, it lacks details on behavioral traits like permissions needed, rate limits, or error handling, leaving gaps in transparency for a tool that likely interacts with a system.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the purpose and includes key details like the source and default behavior. There is no wasted text, making it appropriately sized and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 optional parameter, no output schema, no annotations), the description is adequate but minimal. It covers the purpose and basic usage but lacks details on output format or error scenarios, which could be helpful for an agent invoking the tool without an output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the 'date' parameter fully. The description adds minimal value by mentioning 'default: today', which is already in the schema description. No additional semantics beyond the schema are provided, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('List available teams') and the resource ('teams (name + id)'), with additional context about the source ('from the check-in edit page for a date'). It distinguishes from siblings like steady_login or steady_submit_checkin by focusing on listing rather than authentication or submission.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for usage ('from the check-in edit page for a date'), including a default behavior ('default: today'), which helps guide when to use it. However, it does not explicitly state when not to use it or name alternatives among siblings, such as steady_ping for health checks.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• steady_set_cookiesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that this stores cookies locally for authentication purposes, which implies persistence and sharing across tools. However, it doesn't mention security implications, storage duration, or whether this overwrites existing cookies. The behavioral context is adequate but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with two sentences that each earn their place: the first states the core purpose and benefit, the second specifies requirements. There's zero waste or redundancy, and information is front-loaded appropriately.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (authentication storage with 1 parameter), no annotations, and no output schema, the description is reasonably complete. It covers purpose, usage context, and parameter requirements. However, it could better address behavioral aspects like security or persistence details to be fully comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already fully documents the single parameter. The description adds minimal value beyond the schema by emphasizing the required cookie types, but doesn't provide additional syntax, format, or validation details. This meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Store Steady cookies locally') and the resource ('Cookie header value'), distinguishing it from sibling tools like steady_login (which likely obtains cookies) and steady_list_teams (which uses them). It explicitly mentions the purpose of enabling authentication for other tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use this tool: to store authentication cookies for other tools. It specifies required cookie types ('at least remember_user_token and/or _sthr_session'), but doesn't explicitly state when NOT to use it or name alternatives (like whether steady_login should be used first).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• steady_loginA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses that the tool performs authentication ('Log into Steady'), stores cookies locally, and reads credentials from multiple sources. However, it doesn't mention potential side effects (e.g., invalidating previous sessions), error behaviors, or rate limits. The description adds useful context but lacks comprehensive behavioral details.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured in two sentences: the first states the core purpose and outcome, the second explains credential sources. Every phrase adds value without redundancy, and it's appropriately front-loaded with the main functionality.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's authentication purpose with no annotations and no output schema, the description does well by explaining the login flow, cookie storage, and credential sources. However, it doesn't describe what happens on success/failure or return values, which would be helpful for a mutation tool. The coverage is good but not fully comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 100% description coverage for its single parameter, documenting that 'email' is an optional override. The description adds meaningful context by explaining the default credential sources (STEADY_EMAIL env var, etc.), which helps the agent understand parameter usage beyond the schema's technical specification. For a single parameter tool, this provides good supplemental information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Log into Steady'), the mechanism ('using the email+password flow'), and the outcome ('store fresh cookies locally for other tools to use'). It distinguishes itself from siblings like steady_ping or steady_list_teams by focusing on authentication rather than data retrieval or status checking.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use this tool ('for other tools to use') and mentions credential sources (env vars, macOS Keychain), but doesn't explicitly state when NOT to use it or name alternatives like steady_set_cookies for cookie management. The guidance is practical but lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.