Server Quality Checklist

Tool Scores

• find_lessonsA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden but only partially satisfies it: it discloses the project-scoping behavior (excludes other projects' explicit lessons while retaining generic rules) and the workflow step of marking followed. It does not describe return limits, pagination, or the read-only nature, though 'find' implies a read operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short and front-loaded with purpose, and the follow-up instruction is direct. It earns efficiency, though the phrase 'Standalone' and the use of 'surface_id' as unexplained jargon slightly reduce clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple schema (3 optional params), an output schema, and clear sibling context, the description is reasonably complete. It covers the core behavior, project scoping nuance, and the expected follow-up action, leaving only minor gaps like parameter details that the schema already fills.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers 100% of parameters, so the baseline is 3. The description adds minimal meaning beyond the schema by restating query and project semantics, but it does not clarify the limit parameter or provide additional context beyond what the schema already explains.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool's purpose as 'what procedural rules apply to X' and specifies find_lessons(query, project?) returns lessons with surface_id. This clearly distinguishes it from sibling tools like recall or project_overview, though the phrasing is somewhat informal and the term 'standalone' is ambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool by framing it as a lookup for procedural rules and adds a follow-up instruction ('FOLLOW them, then mark_lesson_followed'). However, it does not explicitly state when not to use it or mention alternative tools that might be preferable in other contexts.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_goalsA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It indicates a read operation and gives an example with status='in_progress', but it does not clarify what 'open' means, the default behavior when status is null, or pagination/limit behavior. This is a moderate gap for a list tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: a clear statement, a practical example, and a usage phrase. Every segment adds value and nothing is redundant.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema present, return values need not be explained. However, the description leaves ambiguity about the default status behavior and the definition of 'open goals'. For a simple list tool, this is acceptable but not complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It only provides an example for 'status' but does not explain the 'limit' parameter or the meaning of status values (e.g., what other statuses are valid). The default behavior of status is also unclear.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List open goals' with a specific verb and resource, and provides an example invocation. It does not explicitly differentiate from sibling tools, but 'open goals' implies a subset, making the purpose reasonably clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'For what are my goals/what's in flight' provides clear usage context. It does not mention when not to use it or point to alternatives, but the intended use case is well conveyed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• session_briefA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full transparency burden. It implies a read-like informational tool ('Re-orient', 'what THIS session decided/built') but does not explicitly state side effects, return format, or any prerequisites. It provides some behavioral context but not full disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three concise sentences, front-loaded with the primary use case. Each sentence adds value, including a clear directive, with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with one optional parameter and an output schema exists, which reduces the need to explain return values. However, the parameter semantics are entirely unexplained, and the description lacks guidelines on how to interpret or use the brief, leaving a notable completeness gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter 'minutes' has no schema description and the description text does not mention it. Schema coverage is 0%, so the description fails to compensate. The name 'minutes' suggests a time span but its exact role remains ambiguous.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific language ('Re-orient after YOUR context compacted') and defines what it returns ('what THIS session decided/built'). It distinguishes from general recall but does not explicitly differentiate from sibling tools by name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear usage context (after context compaction) and a direct directive ('Don't re-ask the user'). However, it does not mention alternative tools or list exclusions, leaving some ambiguity about when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• recallA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the burden. It discloses that results include 'auto-attached lessons' and instructs to 'read & FOLLOW them,' which is a significant behavioral directive. It also provides a trust threshold for interpreting results. However, it does not state whether the tool is read-only or what happens with the project parameter, so it is transparent but not exhaustive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four concise sentences, front-loaded with the core action. The call format, return list, and trust threshold are all packed into a tight paragraph without waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers core usage, return composition, and the important 'follow lessons' directive. The output schema presumably handles return structure, so that is not a gap. However, the project parameter is never mentioned, and there are no error handling or limit notes, so completeness is moderate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the description only provides the call format 'recall(query, top_k=5).' It does not explain the meaning of top_k (number of results) or the project parameter, which is entirely omitted. This is a clear gap given the schema has three parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Search memory for anything about the user/past/project.' This clearly distinguishes it from sibling write tools like record_keyed_fact and from the more specific find_lessons. The call format reinforces the purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this is the general memory search tool by saying 'anything about the user/past/project.' It includes operational guidance like the trust threshold but does not explicitly state when to prefer it over find_lessons or session_brief, nor any when-not conditions. Thus, usage is implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• project_overviewA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of explaining behavior. It lists the data returned, which is helpful, but it does not disclose whether the operation is read-only, how it handles ambiguous matches, or what happens if no project is found. The parameter schema covers matching behavior, but the description itself lacks these nuances.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, compact sentence with an input→output notation. Every element is informative, and it is front-loaded with the key purpose. No redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter) and the presence of an output schema, the description sufficiently covers the core function and usage context. It could mention edge cases like no match or error handling, but for a one-call overview tool, the current level is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema provides a thorough description of the 'name' parameter (case-insensitive substring match, picks highest-mention entity), giving 100% coverage. The description only restates 'NAMED project' without adding new semantics, so it meets the baseline but does not exceed it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool provides 'full context' for a named project, listing specific content areas (lessons, decisions, open goals, recent activity, related entities). This distinguishes it from siblings that focus on narrower aspects like find_lessons or list_goals.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'at the start of work' gives a clear use case, and 'one-call' implies an efficiency advantage. However, it does not explicitly mention when not to use it or name alternatives among sibling tools, though the contrast is implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_goalA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses that it 'Records a goal_update event so the goal's history is preserved,' providing side-effect context. With no annotations, this is valuable, but it does not explain behavior when optional parameters are omitted (e.g., whether status is cleared or unchanged).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: a clear purpose, an inline signature, and a side-effect note. It is front-loaded with no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the main action and side effect, but leaves ambiguity about omitted optional parameters and the requirement of an existing goal. It also doesn't discuss error conditions. For a simple 4-parameter tool, this is a visible gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The signature adds semantics not in the schema: status enum ('in_progress'|'done') and progress range (0-100). This compensates for 0% schema coverage. However, it does not define the meaning of null defaults or whether omitted params are ignored.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Move a goal's status/progress' with a specific verb and resource. The inline function signature and allowed values further clarify its purpose, distinguishing it from siblings like list_goals and record_batch.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The usage is implied: to update a goal's status or progress. However, there is no explicit comparison to alternative tools, exclusions, or prerequisites. It lacks a clear 'use when' vs. 'use instead' statement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• record_keyed_factA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the transparency burden. It clearly discloses the supersede behavior ('A new value SUPERSEDES the old') and the canonical-key semantics, which are important for a write operation. It also communicates the mutable nature. It misses some details (e.g., auth, return value), but the core behavioral trait is covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three short sentences, each earning its place: the purpose/verb, the signature with example, and the critical supersede behavior. It is front-loaded with the action word 'Upsert' and wastes no words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple write operation, the description covers the essential: what it does, how parameters map, and the distinguishing supersede behavior. An output schema exists (per context), so return values need not be documented in the description. It does not discuss usage conditions or prerequisites, but that is not necessary for a low-risk tool with complete parameter documentation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already has 100% parameter coverage with clear descriptions. The description adds value by restating the signature in a callable form (record_keyed_fact(subject, attribute, value)) and giving a concrete example that maps the parameters together. This makes the parameter semantics clearer than schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with the specific verb 'Upsert' and a clear resource: 'a mutable personal attribute.' It goes on to give the exact function signature with example (user/city/Tampa) and explains the keyed supersede behavior, which distinguishes it from a generic 'record' or sibling tools like record_batch that might append rather than replace.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: call this when updating a mutable personal attribute that should have one canonical value. However, it does not explicitly contrast with alternatives such as record_batch or recall. The inclusion of 'instead of piling up' hints at a distinction but does not name a specific alternative or give when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• record_batchA
  4.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations mean the description carries the transparency burden. It adds useful constraints like 'use ABSOLUTE dates' and the efficiency benefit, but does not describe return behavior, error handling, or durability. It's adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense and efficient, front-loading the usage preference with '⚡ PREFERRED' and then detailing item types. It could benefit from line breaks, but every sentence contributes value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description provides complete item type specs and usage rules (e.g., one per turn, absolute dates). Since an output schema exists, return values don't need explanation. It's fully sufficient for a batch write tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema coverage, the description fully compensates by specifying the items parameter: a list of dicts, each with a type (fact, fact_tree, lesson, goal, plan, activity, milestone) and their individual fields. This goes far beyond the sparse schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'stores N atomic items in ONE call', identifying it as a batch recording operation. It differentiates from sibling record_* tools by labeling it 'PREFERRED' for messages with multiple memories.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'PREFERRED for any message with multiple memories' and 'ONE record_batch per turn', providing clear when-to-use guidance. It also contrasts with separate record_* calls, making the alternative usage clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• prepareA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It discloses that it returns multiple data components, that lessons have surface_id and should be followed and marked with mark_lesson_followed, and that it is fast (~10ms). However, it doesn't mention whether the call is read-only or has side effects, nor error handling for unknown projects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact, front-loaded with the READ-FIRST key phrase, and packs multiple pieces of information (returned items, lesson follow-up, performance) into two sentences without fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one parameter and an output schema, the description gives a good overview of the returned components and the lesson-follow-up workflow. It could add a note about behavior for unknown projects, but overall it's sufficiently complete for an agent to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has a single 'message' parameter with no description (0% coverage). The description compensates by showing prepare(message=<the user's message>) and explaining that message is the user's message, which adds semantic meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states this is a READ-FIRST bundle for the start of work on a known project, returning project_context, surfaced lessons, recent_activity, and open_goals. It also says it replaces several recall() calls, which distinguishes it from the recall sibling.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says to use it at the start of work on a known project, and notes it replaces several recall() calls. This provides clear when-to-use context and directly names an alternative (recall).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• mark_lesson_followedA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description explains the tool's role in the self-improvement loop and clarifies the semantic difference between followed and applicable. It omits details on idempotency or side effects, but for a simple feedback tool the described behavior is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with purpose, includes signature and crucial usage tip, no redundant words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple parameter set, full schema coverage, and presence of an output schema, the description covers the when, why, and how-to-use nuances. The only missing aspect is return value description, which is handled by the output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema already covers all 4 parameters at 100%, but the description adds a concrete function signature and, more importantly, resolves the ambiguity between followed and applicable through explicit usage rules, which is beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb-resource pairing: 'Report whether a surfaced lesson changed your behaviour.' It distinguishes from siblings like find_lessons (retrieval) by focusing on post-hoc feedback, and the signature example reinforces the tool's role.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly instructs 'Call after acting on a lesson' and provides a key disambiguation: 'Use applicable=False when the lesson was irrelevant, not followed=False.' This gives direct when-to-use guidance relative to alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.