Server Quality Checklist

Tool Scores

• scenef_discountsA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already establish read-only/non-destructive behavior. The description adds scope ('across all SF theaters'), the structured coverage, and the behavior of flagging today's applicable discounts, which is genuinely useful behavior beyond the annotation booleans.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One compact sentence packs scope, content dimensions, and the today-flag behavior. The phrase 'day-bound days' is slightly awkward, but there is no dead weight.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no required parameters, an output schema, and annotations covering safety, the description supplies the only missing context: the domain coverage and the today flag. Nothing essential is absent for invoking this tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter response_format is fully documented in the input schema, including allowed enums and default, so the description does not need to repeat it. It adds no extra meaning, matching the baseline for 100% schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The definition opens by identifying the exact resource ('Every structured discount across all SF theaters') and adds the dimensions covered (venue, label, detail, day-bound days) plus the today flag. The missing explicit verb and lack of explicit sibling comparison keep it from a 5, but it is unmistakably a discount-listing tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No when-to-use or alternative tool is mentioned, so the tool cannot be explicitly routed against siblings. However, the scope phrasing makes it clear that discount-related queries are the intended use, so usage is implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scenef_theater_infoA
  3.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the readOnlyHint and destructiveHint annotations, the description discloses concrete behavioral details: it returns the next 5 showtimes with ticket links, includes a calendar feed URL, and explains exactly what the 'detailed' mode adds (confidence level, source tier, reporting sources, verified_at timestamp). This gives the agent a precise model of what the tool will produce.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded: the first sentence establishes the resource and enumerates its contents, and the second precisely clarifies the mode-dependent behavior. There is no filler, redundant phrasing, or repeated schema material.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is largely complete for a read-only info tool, especially because the output schema exists and the annotations cover safety. It explains the main output groups and the detailed-mode behavior. It only lacks explicit guidance about when to invoke this tool versus the related sibling tools, which slightly weakens its standalone context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already documents both parameters at 100% coverage, so the description does not need to repeat them. It adds value by explaining what 'detailed' mode actually changes: every showtime additionally carries confidence, source tier, reporting sources, and verified_at, which is not stated in the schema. This makes the parameter semantics richer.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the resource as a single SF theater's card and enumerates its contents (address, neighorhood, website, showtimes, discounts, etc.), so an agent can tell what data it returns. However, it lacks an explicit verb and does not directly distinguish itself from sibling tools like scenef_whats_playing or scenef_discounts beyond the implied focus on one theater.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description says what the tool returns but gives no guidance about when to choose it over the listed siblings. There is no mention of conditions, exclusions, or alternative tools such as scenef_search_showtimes for broad showtime queries, so an agent must infer the appropriate use case from the title and contents.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scenef_nowA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, and the description goes beyond them by explaining response composition: counts, next curtains, freshness, and the additional fields in detailed mode. It also discloses the cheap/summary nature of the call, which is useful behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tight sentences, front-loaded with the most important identity ('cheap is-anything-on call') followed by concrete outputs and the mode difference. Every sentence adds information and there is no fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter, read-only tool with an output schema, the description covers the core functionality, the data scope, and the difference between the two response formats. It does not mention edge cases like timezone interpretation or what 'city-wide' means, but those are minor given the surrounding schema and sibling context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the response_format parameter is already well described in the schema. The description adds extra semantic value by specifying what 'detailed' mode includes (confidence level, source tier, reporting sources, verified_at timestamp), which is not in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the exact purpose with specific verbs and resources: 'cheap is-anything-on call', counts of screenings tonight, next 5 curtains city-wide with venue/time/film, and dataset freshness per source. The scope ('tonight', 'next 5 curtains') clearly distinguishes it from sibling tools like whats_playing or coming_soon.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use it ('cheap is-anything-on call', 'right now', 'tonight') and what it covers, but it never names alternative tools or states when NOT to use it. Siblings exist and some overlap is plausible, so more explicit routing would improve this dimension.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scenef_film_detailsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, so the safety profile is covered. The description adds useful behavioral detail by noting optional presence of URLs, the conditional last-night flag, and exactly what differs in 'detailed' mode, which helps the agent anticipate variable output.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tightly-packed sentences with no filler. The core purpose and field list are front-loaded, and the detailed-mode distinction is placed at the end without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the output schema, annotations, and sibling context, the description fully covers what the agent needs to know to invoke this tool correctly. It explains the one-film scope, the optional mode difference, and the conditional fields without relying on the schema alone.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%: both `film` and `response_format` already have meaningful descriptions and enums. The description reinforces the mode behavior but does not add new parameter meaning beyond the schema, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as returning a complete card for a single film, listing all included fields (title, year, runtime, genres, directors, cast, overview, rating, trailer/poster URLs, showtimes, last-night flag). This specific scope differentiates it from sibling list/search tools like scenef_whats_playing or scenef_search_showtimes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies clear usage context: use this when you need the full detail set for one specific film, including showtimes and metadata. It does not explicitly name alternatives or state when not to use it, but the 'one film' framing provides enough contextual guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scenef_plan_movie_nightA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare read-only, non-destructive behavior, and the description adds valuable behavioral context beyond that: rankings are preference-based and never pay-ranked, a wildcard pick is always included, and detailed mode surfaces confidence, source tier, and verified_at metadata. Nothing contradicts the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but every sentence earns its place: it covers output shape, ranking policy, and mode-specific behavior. The 'concierge' metaphor adds a bit of flavor without harming clarity, and the most decision-relevant behavior is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite the tool's nested preferences and hard time constraints, the description plus the fully covered schema provides enough for an agent to invoke it correctly. It lacks explicit mention of defaults and hard-bound fallback behavior, but those are captured in the schema descriptions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema already documents all parameters and nested preference fields thoroughly. The description adds a helpful high-level framing ('window and taste profile') but does not need to repeat or extend the detailed parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description identifies a specific verb ('plans') and a concrete resource (a movie night), then enumerates the exact output: 2-4 complete plans with film, showtime, theater, rationale, ticket and calendar links, plus a wildcard. This clearly differentiates it from sibling search/list tools like scenef_search_showtimes or scenef_whats_playing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It states the required inputs ('a window and a taste profile') and the expected result, making the primary use case obvious. However, it does not explicitly say when to prefer this tool over sibling tools or mention exclusions, so it stops just short of full routing guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scenef_accuracyA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the read-only annotation, the description discloses important behavior: results are recomputed on every call, failures are included, unreachable checks are excluded from the pass rate, and the exact denominator is provided. It even clarifies edge cases like bot walls and client-rendered pages. This is strong behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is longer than minimal but every sentence adds substantive information: what the record contains, how unreachable checks are treated, and how the numbers should be used. It is dense rather than padded, though the opening sentence is a bit contextual rather than an immediate verb-first statement.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only accuracy-record tool with an output schema, the description covers the main behavior, return contents, edge cases, and even the public accuracy API. The agent has enough information to call the tool correctly and interpret the result without guessing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already fully documents the single response_format parameter with an enum and a description of concise versus detailed output. The tool description adds nothing about this parameter, so the baseline score is appropriate; there is no semantic gap to compensate for.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a clear verb ('returns') and names the exact resource: the verification accuracy record with confidence mix, check counts, pass rates, denominator, per-venue details, and level definitions. This is highly specific and clearly distinct from the sibling showtime and theater tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description makes the intended use obvious: when accuracy numbers are needed, quote the returned record directly, and it explains how unreachable checks are handled so the numbers are not misused. It does not explicitly name alternatives or say 'use X instead,' but the context is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scenef_whats_playingA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark this as read-only and non-destructive, and the description adds substantial behavioral detail: the 'Notable tonight' lead section, evidence types like measured seat counts and lone prints, per-entry fields, and the extra metadata available in detailed mode. There is no contradiction with the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense and front-loaded: the first sentence states the core purpose and scope, the second covers special tonight behavior, the third lists entry contents, and the fourth explains detailed mode. Every sentence contributes distinct information with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only listing tool with complete schema coverage, an output schema, and safety annotations, the description covers all necessary calling context: what results look like, special behavior windows, entry fields, detailed mode, and general use guidance. Nothing essential is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3, but the description goes beyond the schema by explaining what detailed mode includes (confidence level, source tier, reporting sources, verified_at timestamp) and clarifying the content and behavior associated with the 'when' parameter. This adds real semantic value beyond the input schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool returns a ranked list of films playing in San Francisco theaters within a given window, with genre and format filters. It is specific about verb, resource, and scope, but it does not explicitly distinguish itself from siblings like scenef_search_showtimes or scenef_now, so differentiation is left to inference.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when to use the tool, including the supported windows (tonight, tomorrow, weekend, or a date) and the special 'Notable tonight' behavior. It even instructs the agent to lead with scarcity facts when asked what to see, but it does not mention when NOT to use this tool or name alternative sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scenef_coming_soonA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare read-only, non-destructive behavior. The description adds useful behavioral context: the 48-hour threshold, sorting by first night, a configurable horizon, and the extra metadata in detailed mode. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two tight sentences communicate the core behavior, the booking use case, configurability, and detailed-mode metadata without repetition. The most important scope information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only listing tool with two optional parameters and an output schema, this description is complete. It covers what is returned, how it is sorted, the time threshold, and optional response details, so an agent can confidently invoke it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds value by explaining what the horizon is for and what "detailed" mode includes, such as confidence level, source tier, reporting sources, and verified_at.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the resource: films whose first SF screening is more than 48 hours out, sorted by first night. It also frames the tool as an "on-sale radar" for early booking, which distinguishes it from siblings like scenef_whats_playing and scenef_now.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives a clear use case: identify upcoming runs worth booking early. It implies when to choose this tool over current-screening tools, though it does not explicitly name alternatives or state when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scenef_search_showtimesA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint=true and destructiveHint=false, matching the description. The description adds substantial behavioral detail not visible in annotations: fuzzy matching, ambiguous candidate return, grouping by theater, local times, tags, ticket links, and the extra metadata in detailed mode. No contradiction exists.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is information-dense but every sentence earns its place, starting with the core either/or scoping and ending with mode-specific details. It is front-loaded with the most important decision rule and contains no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 6-parameter read-only tool with an output schema, the description covers optionality, filters, grouping, tag semantics, ticket links, the alternate tool route, and detailed mode behavior. The output schema handles return-value specifics, so nothing operationally necessary is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although the schema already covers all parameters at 100%, the description enriches their meaning: film accepts a title or slug and is fuzzy-matched, venues accepts ids or names with examples, time filters use local HH:MM, and response_format maps to concise vs detailed output. This goes beyond the raw schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the exact resource (showtimes) and the two scoping dimensions (film or theater), which distinguishes it from the sibling scenef_whats_playing. It goes far beyond the title by explaining what the returned showtimes look like and which mode applies.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly tells the agent to pass at least one of film/venues, clarifies that venues is required when film is omitted, and routes whole-board browsing to scenef_whats_playing instead. The miss behavior for non-board film titles is also disclosed, so an agent knows when to fall back.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.