Server Quality Checklist

Tool Scores

• duolingo_get_authenticated_profileC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, and the description does not disclose any behavioral traits such as authentication requirements, rate limits, error conditions, or the structure of the returned data. For a tool fetching user data, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no wasted words. It could be longer to include more context, but as a concise statement it functions adequately.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema and the description does not explain what the 'richer authenticated profile' contains. Missing information about prerequisites (e.g., authentication), error handling, and return structure. Not minimal viable for a profile fetch tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with clear parameter descriptions. The description adds no extra meaning beyond the schema, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Fetch') and the resource ('authenticated profile') for a given user id. It distinguishes from sibling tools like duolingo_get_public_profile by using 'authenticated' and 'richer', but does not explicitly differentiate from other siblings like duolingo_get_xp_summaries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives (e.g., when to use authenticated vs public profile, or when to use other tools). The user must infer context from the tool name and siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• duolingo_get_xp_summariesB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavior. It only states 'Fetch authenticated daily XP summaries' without mentioning authentication requirements, rate limits, data freshness, or any side effects. This is insufficient for safe invocation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence front-loaded with the core purpose. However, it could be slightly more informative without becoming verbose, hence 4 instead of 5.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 6 parameters and no output schema, yet the description does not mention return format, pagination, or limitations. It lacks completeness for an agent to understand the full context of usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is high (83%), so baseline is 3. The description adds no parameter-specific meaning beyond the schema; it does not explain how parameters like startDate/endDate affect results. No added value for parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool fetches 'daily XP summaries' with specific details 'sessions and practice time'. It uses a specific verb 'Fetch' and resource, distinguishing from sibling tools like duolingo_get_public_profile which focuses on public profile data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool vs alternatives. It implies authenticated access (via 'authenticated') but does not mention prerequisites, when not to use, or differentiation from siblings like duolingo_today_status which may also return XP data.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• duolingo_get_public_profileB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description is the sole source. It discloses the fetched data types (profile, streak, XP, courses) but does not mention any side effects, error behavior (e.g., on missing username), rate limits, or that the data is public and requires no authentication. Behavioral traits like 'only public data' are implied but not explicitly stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One clear sentence that states the action and what is fetched. It is concise but could be slightly more efficient by removing the word 'public' since it's in the tool name. No wasted words, but front-loading is adequate.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema exists, so description should explain return values. It lists key return items (profile, streak, XP, courses) but is vague on format or additional metadata. For a simple fetch tool with 1 parameter, it is moderately complete but leaves questions about structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% (the only parameter 'username' has a description). The tool description adds 'Defaults to DUOLINGO_USERNAME' which is already in the schema description. No additional meaning beyond what the schema provides, so baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states verb 'Fetch' and resource 'public Duolingo profile, streak, total XP, and courses by username.' It distinguishes from siblings like 'duolingo_get_authenticated_profile' (which requires auth) and 'duolingo_get_xp_summaries' (which focuses on XP). The purpose is specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for fetching public profile by username but provides no explicit guidance on when to use vs alternatives (e.g., when to choose this over duolingo_get_authenticated_profile or duolingo_today_status). No when-not-to-use conditions are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• duolingo_today_statusB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must cover behavioral traits. It does not mention if authentication is needed, rate limits, or side effects. The output fields are listed but not fully detailed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence of 15 words directly states the tool's purpose. No unnecessary words, front-loads the key action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, the description fails to disclose the return format (e.g., types, structure). Lacks details on default behavior for parameters and error handling.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 67%. The description does not add meaning beyond listing output fields; the timezone parameter is unexplained despite having no schema description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns today's Duolingo status, including specific data fields like XP, sessions, and streak. It effectively distinguishes from siblings such as profile or XP summarization tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. The use case is implied but not clarified, e.g., no mention of prerequisites or scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• duolingo_auth_helpA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It describes a non-destructive action (explaining) but does not explicitly state that it has no side effects or return format. Minimal but adequate for a help tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, clear sentence that conveys the purpose without any extraneous words. Excellent conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple help tool with no parameters and no output schema, the description is complete. It covers what the tool does adequately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, and input schema coverage is 100% (trivially). The description does not need to add parameter info. Baseline score of 4 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Explain how to configure a Duolingo browser session cookie for authenticated tools.' It uses a specific verb ('Explain') and a unique resource ('Duolingo browser session cookie'), differentiating it from sibling tools that retrieve data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context by specifying 'for authenticated tools,' suggesting it should be used before accessing other tools that require authentication. However, it lacks explicit when-not or alternative guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.