Server Quality Checklist

Tool Scores

• get_user_infoB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states this is a read operation ('Return general info'), but doesn't mention authentication requirements, rate limits, error conditions, or whether the data is real-time vs cached. For a user info tool with zero annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise with two sentences that directly address purpose and return values. The first sentence states the core function, and the second provides concrete examples. There's no wasted verbiage, though it could be slightly more structured by front-loading the most critical information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has zero parameters, 100% schema coverage, and an output schema exists, the description provides adequate basic information about what the tool returns. However, for a user info tool with no annotations, it should ideally mention authentication requirements or data freshness. The existence of an output schema means the description doesn't need to detail return format, but behavioral context is incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters (schema coverage 100%), so there are no parameters to document. The description appropriately doesn't waste space discussing nonexistent parameters. A perfect 5 would require the description to explicitly state 'no parameters required' or similar, but 4 is appropriate given the empty parameter set.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Return general info about the Context by Fulcra user' with specific examples of what's returned ('user references such as time zone, calendar ids, and other metadata'). However, it doesn't explicitly differentiate this user info tool from sibling tools that return location, metrics, sleep, or workout data, which would be needed for a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. There's no mention of prerequisites, timing considerations, or comparison to sibling tools that return different types of user data. The agent must infer usage context from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_location_at_timeA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behaviors: searching for closest samples within a window, handling time zones, and returning JSON data. It doesn't cover aspects like error handling, rate limits, or authentication needs, but it provides sufficient operational context for basic use.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded, starting with the core purpose. Each sentence adds value: explaining sample search logic, time zone handling, and parameter/return details. There's no redundant or wasted text, making it efficient and easy to parse for an AI agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (time-based location querying) and lack of annotations, the description is mostly complete. It covers purpose, key behaviors, and most parameters, and an output schema exists, so return values don't need explanation. The main gap is the undocumented 'reverse_geocode' parameter, which slightly reduces completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaning for 'time' (must include tz in ISO8601), 'window_size' (seconds to look back, with default), and 'include_after' (optional behavior). However, it doesn't mention 'reverse_geocode' from the input schema, leaving one parameter undocumented. This partial coverage slightly reduces the score from a perfect 5.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Gets the user's location at the given time.' It specifies the verb ('Gets') and resource ('user's location'), making it easy to understand. However, it doesn't explicitly differentiate from sibling tools like 'get_location_time_series' or 'get_metric_samples,' which could provide similar location-related data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by mentioning how the tool handles missing samples and time zone translation, but it doesn't explicitly state when to use this tool versus alternatives like 'get_location_time_series.' It provides some context on behavior (e.g., searching for closest samples) but lacks clear guidance on scenarios or prerequisites for choosing this tool over siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_location_time_seriesA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behaviors: result timestamps include time zones with translation guidance, and it explains optional filtering ('change_meters') and enhancement ('reverse_geocode'). However, it misses details like rate limits, error handling, or data freshness, which could be important for a location-tracking tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, args, returns) and uses bullet-like formatting for parameters. It's appropriately sized, but the first sentence could be more front-loaded with key details, and some phrasing (e.g., 'Always translate...') might be slightly verbose without adding critical info.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 5 parameters, no annotations, and an output schema, the description is mostly complete. It covers purpose, parameters, and return format adequately. However, it lacks context on permissions, data sources, or sibling tool differentiation, which could enhance usability given the server's other location/metric tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Given 0% schema description coverage, the description compensates fully by providing detailed semantics for all 5 parameters. It explains each parameter's purpose, format (e.g., ISO 8601), optionality, defaults, and effects (e.g., 'reverse_geocode' adds details). This adds significant value beyond the bare schema, making parameters well-understood.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Retrieve a time series of locations that the user was at.' This specifies the verb ('retrieve') and resource ('time series of locations'), making it understandable. However, it doesn't explicitly differentiate from sibling tools like 'get_location_at_time' or 'get_metric_time_series', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides some implicit guidance by mentioning 'Always translate timestamps to the user's local tz when this is known,' suggesting usage when timezone-aware location data is needed. However, it lacks explicit when-to-use instructions, alternatives (e.g., vs. 'get_location_at_time'), or exclusions, leaving gaps in practical application.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_workoutsA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds valuable context beyond the input schema: it specifies that 'Result timestamps will include time zones' and instructs to 'Always translate timestamps to the user's local time zone when this is known.' This clarifies output behavior and user-facing considerations, though it doesn't cover aspects like rate limits, authentication needs, or error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded, with the core purpose stated first, followed by behavioral notes and a dedicated 'Args:' section for parameters. Every sentence adds value: the first defines the tool, the second explains timestamp handling, and the parameter descriptions are essential given the schema gaps. There is no wasted text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 required parameters, no annotations, but with an output schema), the description is largely complete. It covers purpose, behavioral traits (timestamp handling), and parameter semantics. Since an output schema exists, it doesn't need to explain return values. However, it could improve by addressing potential limitations (e.g., data availability) or sibling tool differentiation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description provides significant semantic value beyond the input schema, which has 0% description coverage. It explicitly defines both parameters ('start_time' and 'end_time'), specifies they 'Must include tz (ISO8601)', and contextualizes them as 'The starting time of the period' and 'the ending time of the period.' This fully compensates for the schema's lack of descriptions, making the parameters clear and actionable.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get details about the workouts that the user has done during a period of time.' This specifies the verb ('Get details'), resource ('workouts'), and scope ('during a period of time'). However, it doesn't explicitly differentiate from sibling tools like 'get_metric_samples' or 'get_sleep_cycles', which might also retrieve user activity data, so it falls short of a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by specifying 'during a period of time' and providing time-based parameters, suggesting it's for retrieving historical workout data. However, it lacks explicit guidance on when to use this tool versus alternatives like 'get_metric_samples' or 'get_sleep_cycles', and doesn't mention prerequisites or exclusions (e.g., whether the user must have workouts logged).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_sleep_cyclesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behaviors: processing logic (finding gaps in sleep data), timestamp handling (includes time zones, translation guidance), and output format (JSON string representing pandas DataFrame). However, it doesn't mention performance characteristics, error conditions, or authentication requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with purpose statement, processing explanation, timestamp guidance, parameter documentation, and return format. The information is front-loaded and organized, though the parameter section is lengthy (necessary given 0% schema coverage). Every sentence serves a clear purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (6 parameters, processing logic) and the presence of an output schema, the description is quite complete. It explains the transformation logic, parameter semantics, and timestamp handling. The output schema handles return value details, so the description appropriately focuses on behavior and parameters. Minor gaps exist in error handling and performance context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by providing detailed parameter documentation in the Args section. Each of the 6 parameters is clearly explained with purpose, format examples, defaults, and optionality. This adds substantial value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Return sleep cycles summarized from sleep stages' and explains the processing logic: 'Processes raw sleep data samples into sleep cycles by finding gaps in the sleep sample data within a specified time interval.' This is specific (verb+resource+processing method) and distinguishes it from sibling tools like get_metric_samples or get_workouts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage context through the processing explanation and timestamp handling guidance, but doesn't explicitly state when to use this tool versus alternatives. No sibling tool comparisons or explicit when-not-to-use guidance is provided, leaving usage decisions to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_metrics_catalogA
  4.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It clearly indicates this is a read operation ('Get'), but doesn't specify whether authentication is required, rate limits, error conditions, or what format the catalog returns. While adequate for a simple catalog tool, it lacks details about behavioral traits beyond the basic operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that efficiently communicates the tool's purpose, resource, and usage context. Every word earns its place, with no wasted text or redundancy. It's front-loaded with the core purpose and provides necessary context in a compact form.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 0 parameters, 100% schema coverage, and an output schema exists, the description is reasonably complete. It explains what the tool does and when to use it, though it could benefit from mentioning what the catalog contains (metric definitions, types, etc.) or how the output is structured. The existence of an output schema reduces the need to describe return values in the description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage, so the baseline for no parameters is 4. The description appropriately doesn't discuss parameters since none exist, and instead focuses on the tool's purpose and usage context, which is the correct approach for a parameterless tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get') and resource ('catalog of available metrics'), and explicitly distinguishes it from sibling tools by mentioning which specific tools the metrics can be used with (metric_time_series and metric_samples). This provides clear differentiation from other siblings like get_location_time_series or get_workouts.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use this tool: to get metrics that can be used in time-series API calls, and specifically names the two sibling tools (metric_time_series and metric_samples) where these metrics are applicable. This provides clear guidance on usage context and distinguishes it from alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_metric_time_seriesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well by disclosing key behavioral traits: time range handling (inclusive start, exclusive end), timezone translation behavior, data format (JSON string), missing data handling (NA values), and constraints ('Not supported on cumulative metrics'). It lacks rate limit or authentication details, but provides substantial operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear purpose statement, time range explanation, and organized parameter documentation. Every sentence adds value, though the parameter explanations could be slightly more concise. The structure is logical with purpose first, then behavior, then detailed args section.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (6 parameters, time-series data), no annotations, and the presence of an output schema, the description is remarkably complete. It covers purpose, usage guidance, parameter semantics, behavioral constraints, and return format. The output schema handles return value details, so the description appropriately focuses on operational context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by providing detailed semantic explanations for all 6 parameters. It clarifies metric_name usage with sibling reference, explains time format requirements (ISO8601 with tz), provides defaults for optional parameters, enumerates calculations options, and explains parameter interactions (replace_nulls effect on NA values).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Get user's time-series data'), resource ('for a single Fulcra metric'), and scope ('Covers the time starting at start_time...'). It distinguishes from siblings like 'get_metric_samples' and 'get_metrics_catalog' by focusing on time-series retrieval rather than catalog or sample-level data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool (time-series data retrieval) and references 'get_metrics_catalog' to find available metrics. However, it doesn't explicitly state when NOT to use it or compare it to alternatives like 'get_metric_samples' or 'get_location_time_series'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_metric_samplesA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does an excellent job disclosing key behavioral traits: it explains the range intersection logic for samples covering time ranges, specifies that result timestamps include time zones, and instructs to translate to user's local time zone when known. It also clarifies the inclusive/exclusive nature of the time boundaries. The only minor gap is lack of information about pagination, rate limits, or error conditions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly structured and appropriately sized. It begins with a clear purpose statement, follows with important behavioral details, then provides organized parameter documentation in an Args/Returns format. Every sentence earns its place by adding essential information, with no redundant or verbose content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (time-based querying with specific behavioral rules), no annotations, and the presence of an output schema (which handles return value documentation), the description is remarkably complete. It covers purpose, usage context, detailed parameter semantics, and important behavioral characteristics like range intersection logic and time zone handling. The output schema will document the return structure, so the description appropriately focuses on operational aspects.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by providing detailed parameter semantics in the Args section. It explains what metric_name represents and references get_metrics_catalog for finding available metrics. It clarifies the format requirements for start_time and end_time (ISO 8601 or datetime object) and specifies their inclusive/exclusive behavior. This adds substantial value beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('retrieve raw samples'), target resource ('a given metric'), and scope ('for the user during a specified period'). It distinguishes from siblings like get_metric_time_series (which likely returns aggregated/processed data rather than raw samples) and get_metrics_catalog (which lists available metrics rather than retrieving samples).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context about when to use this tool (retrieving raw samples for a specific metric within a time range) and explicitly references get_metrics_catalog as a prerequisite for finding available metric names. However, it doesn't explicitly state when NOT to use it or compare alternatives like get_metric_time_series, which would be helpful for sibling differentiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.