Xzar MCP Server
OfficialXzar MCP lets an AI agent run patent-examination decision tasks: start a case, research prior art and rules, submit an answer, and review scores and usage.
Start a task episode and get the case, instructions, and tool budget; optionally choose a specific task or taskset (
patents-first-action,patents-response,patents-appeal).Search the episode's prior-art library and read specific publications.
Search the examination manual and read specific rule sections by number.
Submit a final JSON answer for scoring, once per episode.
View your agent's scorecard by skill and failure patterns.
Check your credit balance and recent usage/charges.
Click on "Deploy Server".
Wait a few minutes for the server to deploy. Once ready, it will show a "Started" state.
In the chat, type
@followed by the MCP server name and your instructions, e.g., "@Xzar MCP Serverstart a patent examination task and score my decision"
That's it! The server will respond to your query, and you can continue using it as needed.
Here is a step-by-step guide with screenshots.
Xzar MCP server
Crash-test your AI agent on real professional decisions. Xzar gives your agent real cases, cut at the moment of decision, and grades its answer against what the professional actually decided. Most cases were decided after current models were trained, so they can't have been memorized; fresh ones are added monthly.
Live today:
Tariff classification: real U.S. classification decisions, graded exactly at 2, 4, 6, 8 and 10 digits, with a consistency check (the same case asked twice). Testing a tariff classifier? See https://xzar.si/scorecard
Patent examination: first action, response and appeal.
More fields are on the way; ask for yours at https://xzar.si.
Setup
Get an API key (with $5 of free credit) at https://xzar.si/signup.html
Add the server to your MCP client:
{
"mcpServers": {
"xzar": {
"command": "uvx",
"args": ["xzar-mcp"],
"env": { "XZAR_API_KEY": "your-key" }
}
}
}Related MCP server: agentviz
Hosted connector (no install)
In Claude or any MCP client that supports remote servers: add https://api.xzar.si/mcp and sign in with your Xzar API key.
Claude plugin
The plugin/ folder is the Xzar plugin for Claude. It bundles the hosted connector with two skills: crash-test, which runs a task end to end and explains the score, and xzar-request, for custom tests, pilots, quotes and training access. In Claude Code:
/plugin install xzar --marketplace xzar-si/xzar-mcpTools
Tool | What it does |
| Start a task: the case, the instructions and the tool budget |
| Search the case's reference library |
| Read one library item |
| Search the professional rulebook for the task |
| Read one rulebook section |
| Submit the decision; returns the exact score and the grade receipt |
| Ask for a custom test for your field, a pilot, a quote, or training access; get a reference and next steps |
| Your agent's scorecard across tasks |
| Credit balance and usage |
Answer keys never leave Xzar's servers. This package holds no tasks; it calls the Xzar API with your key.
Pricing
Prepaid credits, charged per task. See https://xzar.si/docs.html. Pilots and ongoing testing: https://xzar.si.
Available Tools
9 toolsget_scorecardGet your scorecardARead-onlyIdempotentInspect
Get your agent's scorecard across all finished episodes: scores by skill and the failure types it hits most often. Read-only; no credit used.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnly, idempotent, non-destructive, and closed-world, so the safety profile is covered. The description still adds value beyond them by disclosing the cost signal ('no credit used') and the scope limitation to finished episodes — meaningful extras for an agent deciding whether to call it.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, front-loaded with the verb and resource, followed by the cost caveat. Every clause carries information an agent would use.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no input params and no output schema, the description must convey what comes back, and it does: per-skill scores and dominant failure types. It stops short of describing the structure or granularity of those scores, but for a read-only, zero-argument tool this is close to sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, so there is nothing for the description to disambiguate; the baseline for a parameterless tool applies. No parameter-related gaps exist.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Get') and resource ('your agent's scorecard') and enumerates the content returned: scores by skill plus most-frequent failure types across finished episodes. That scope ('all finished episodes') also separates it from get_usage and the task-oriented siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'across all finished episodes' implies the right moment to call it, but there is no explicit when-to-use guidance, no mention of prerequisites such as having completed episodes, and no named alternative (e.g. get_usage) to route against.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
get_usageGet your credit balance and usageARead-onlyIdempotentInspect
Get your credit balance and your most recent charges and top-ups. Read-only; no credit used.
| Name | Required | Description | Default |
|---|---|---|---|
No parameters | |||
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare read-only, idempotent, non-destructive, so the safety profile is covered; the description still adds a non-obvious fact beyond them — that the call consumes no credits, which matters for an agent operating under a budget. It does not describe units, currency, or how many 'recent' entries are returned.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, zero filler, with the capability stated first and the no-cost guarantee second — exactly the information an agent needs, front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter, read-only tool with no output schema, the description adequately previews the return content (balance, recent charges, top-ups). It stops short of stating response shape, currency, or recency window, which an agent might want given there is no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool takes zero parameters, which is the baseline-4 case; there is nothing for the description to disambiguate. Schema coverage is 100% and the empty object confirms no arguments are needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb+resource (get credit balance) and scopes the result set (most recent charges and top-ups), which is more precise than the title alone. No sibling tool covers billing/usage, so no differentiation is needed, but it also doesn't explicitly claim that territory against the rest of the catalog.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
'Read-only; no credit used' implicitly tells the agent this is always safe to call, but there is no explicit when-to-use or when-not guidance, nor any mention of alternatives (none of the siblings overlap, so the gap is small). Usage is inferred rather than stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
read_publicationRead a library publicationARead-onlyIdempotentInspect
Read the full text of one publication from the case's library, by the id search_library returned. Use it to check whether a reference really discloses what the claim requires. Counts against the tool budget.
| Name | Required | Description | Default |
|---|---|---|---|
| episode_id | Yes | The episode id returned by start_task, e.g. ep_3f9c2a1b0d4e5f6a7b8c9d0e | |
| publication | Yes | Publication id from search_library results, e.g. US20190012345A1 |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare the read-only, idempotent, non-destructive, closed-world profile, so the description doesn't need to repeat safety facts. It adds genuinely new behavioral information by warning that the call 'counts against the tool budget', which is not encoded anywhere in the annotations. It stops short of noting result size or truncation behavior for full texts.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three short sentences, front-loaded with what the tool returns, followed by why to use it and the budget cost. No filler and nothing buried.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple two-parameter read tool with full annotation coverage, the description covers purpose, input provenance, motivation, and cost. It leaves the return value underspecified beyond 'full text' (length, truncation, not-found behavior), which is a minor gap given there is no output schema.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, with both episode_id and publication documented with format examples, so the baseline is 3. The description only restates the publication id's provenance from search_library — which the schema already says — and adds nothing about episode_id or id format.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb (read) and resource (full text of one publication), and scopes it to 'one publication from the case's library', which cleanly separates it from the sibling search_library. An agent can tell exactly what this retrieves without opening the schema.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Gives explicit usage context: obtain the id from search_library, then use it to verify whether a reference actually discloses what a claim requires. It names the sibling that produces the input but doesn't spell out exclusions (e.g., don't call it just to preview a title), so it stops short of a full when/when-not rubric.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
read_ruleRead a rulebook sectionARead-onlyIdempotentInspect
Read one rulebook section by its number, as returned by search_rules. Returns the section text.
| Name | Required | Description | Default |
|---|---|---|---|
| section | Yes | Section number, e.g. 2143.01 | |
| episode_id | Yes | The episode id returned by start_task, e.g. ep_3f9c2a1b0d4e5f6a7b8c9d0e |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare readOnlyHint, idempotentHint, non-destructive, and closed-world, so the safety profile is fully covered. The description adds that the output is the section text (useful since there is no output schema) and where section numbers originate, but says nothing about error behavior or missing sections. Modest added value.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two short sentences, front-loaded with the core action, with zero redundant text. Every clause adds information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple two-parameter read tool with rich annotations, the definition covers the action, the identifier source, and the return value. A brief note on what happens when the section does not exist would complete it, but an agent has enough to call it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with both section and episode_id documented including an example format. The description reinforces that 'section' is a section number sourced from search_rules, but adds no format or eligibility detail beyond the schema. Baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: read one rulebook section by number, returning the section text. It also ties the identifier source to search_rules, which helps place it among siblings. It does not explicitly distinguish itself from read_publication, so differentiation is only partial.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
"As returned by search_rules" gives a clear usage context: obtain a section number via search_rules, then read it here. No explicit when-not-to-use or failure conditions are given, but the workflow routing is unambiguous enough for an agent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
request_serviceRequest a custom test, pilot, quote or training accessAInspect
Ask Xzar for something not available self-serve: a custom test for a field, a pilot, a quote, or training access. Returns a reference, next steps and pricing; Xzar replies by email within one business day. Does not use credit.
| Name | Required | Description | Default |
|---|---|---|---|
| name | No | Your name | |
| type | Yes | What you are asking for | |
| Yes | Work email for the reply | ||
| field | No | The professional field, e.g. insurance claims | |
| notes | No | Anything else Xzar should know | |
| volume | No | Rough number of cases or tasks you need | |
| company | No | Your company or team | |
| deadline | No | When you need it, e.g. end of Q1 | |
| decision | No | The decision your AI makes, e.g. approve or deny a claim |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare a non-read-only, non-destructive, non-idempotent, closed-world tool. The description adds substantial context: a human replies by email within one business day, a reference and pricing are returned, and no credit is consumed. Minor gap: it does not say whether repeated calls create duplicate requests, which matters given idempotentHint is false.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Two sentences, front-loaded with the purpose and followed by response expectations and cost. No sentence is redundant.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For an intake/form-submission tool, the definition covers purpose, response contents, turnaround time, and cost – exactly what an agent needs before calling. The parameters are fully documented in the schema and no output schema exists, so nothing material is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, so the schema already explains all nine parameters, including the enum and email reply address. The description adds only a restatement of the request types, not field-level guidance like what makes a useful `field`, `volume` or `decision` value. Baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource, 'Ask Xzar for something not available self-serve,' and enumerates the request types (custom test, pilot, quote, training access) that match the `type` enum. It is clearly distinguishable from self-serve siblings such as start_task or get_scorecard.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
'Ask Xzar for something not available self-serve' gives the condition that selects this tool and implicitly excludes anything covered by self-serve flows. It does not name which sibling to use instead for self-serve requests, leaving that inference to the agent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
search_librarySearch the case's reference libraryARead-onlyIdempotentInspect
Search the case's reference library (patent cases: prior-art publications dated before the filing date). Use it to find evidence before deciding. Returns matching publication ids with titles and snippets; read one with read_publication. Counts against the tool budget.
| Name | Required | Description | Default |
|---|---|---|---|
| query | Yes | Keywords or a short phrase describing the technology, e.g. 'battery cell cooling plate' | |
| episode_id | Yes | The episode id returned by start_task, e.g. ep_3f9c2a1b0d4e5f6a7b8c9d0e |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover read-only, idempotent, non-destructive, and closed-world behavior. The description adds the non-obvious cost signal ('Counts against the tool budget') and describes the shape of results (publication ids, titles, snippets) even though there is no output schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Compact and front-loaded: domain definition, usage intent, return shape and next step, and cost warning. 'Use it to find evidence before deciding' is slightly generic but still orients the agent.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
No output schema exists, yet the description explains return values and next steps, and covers the budget cost. It does not mention result limits or pagination, which would complete the picture for repeated searching.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so both parameters are already well documented in the schema. The description adds no extra syntax, constraints, or interpretation beyond what the schema provides, which is the baseline when the schema does the work.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource and immediately clarifies what the reference library contains for patent cases (prior-art publications dated before the filing date). It also distinguishes itself from the read_publication sibling by name.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly frames the tool as an evidence-gathering step before deciding, and routes the agent to read_publication for reading a hit. It does not clarify its boundary versus search_rules, which is the main adjacent alternative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
search_rulesSearch the rulebookARead-onlyIdempotentInspect
Search the professional rulebook for the case (patent cases: the examination manual examiners follow). Use it to find the legal test that applies. Returns section numbers and titles; read one with read_rule.
| Name | Required | Description | Default |
|---|---|---|---|
| query | Yes | The rule or test you need, e.g. 'obviousness combining references' | |
| episode_id | Yes | The episode id returned by start_task, e.g. ep_3f9c2a1b0d4e5f6a7b8c9d0e |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already cover safety (readOnlyHint, idempotentHint, closed-world), so the description's added value is in disclosing that results are shallow (section numbers and titles only), which prevents an agent from assuming it can answer directly from search results. It omits result-count limits and ranking behavior, but that is a minor gap given the annotation coverage.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Three short sentences, front-loaded with the core action, with the domain clarification and the output/next-step facts each earning their place. No redundant restatement of the name or annotations.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description compensates by stating what is returned (section numbers and titles) and how to continue (read_rule). It could be stronger by noting whether results are limited or ranked, but an agent has everything needed to call it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so both parameters (including the natural-language query example and the episode_id provenance) are already documented in the schema. The description adds only the implicit framing that the rulebook is case-scoped, which is marginal beyond what structured data provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ("Search the professional rulebook for the case") and even disambiguates the domain with the parenthetical about the patent examination manual. It also clarifies the output form (section numbers and titles, not full text) and names the sibling read_rule as the follow-up, so an agent can tell it apart from search_library/read_publication.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
"Use it to find the legal test that applies" gives a clear usage condition, and "read one with read_rule" directs the agent to the correct next step instead of expecting full text here. It does not explicitly contrast with the sibling search_library, so the alternative-selection guidance is incomplete rather than absent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
start_taskStart an evaluation taskAInspect
Start a new evaluation task (one episode) and get the case. Call this first. Returns the episode id (needed by every other episode tool), the case instructions, the answer format and the tool budget. Uses one task of credit; a start that fails on Xzar's side is refunded.
| Name | Required | Description | Default |
|---|---|---|---|
| task_id | No | A specific task id to retry; omit to get the next unserved case | |
| taskset | No | Which kind of case to work. Default patents-first-action. tariffs-classification asks for the tariff code of a real product. |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Goes well beyond the annotations by disclosing cost behavior ('Uses one task of credit') and the refund edge case on server-side failure, plus the critical dependency that every other episode tool needs the returned episode id. This is exactly the kind of non-obvious operational detail annotations cannot convey, and it is consistent with idempotentHint=false and readOnlyHint=false.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Four short sentences, front-loaded with the action and the 'call this first' directive, and no sentence is padding. The return-value list is slightly dense but each item is used by downstream calls, so it earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With no output schema, the description correctly takes on the burden of enumerating the returned fields (episode id, case instructions, answer format, tool budget) and flags the credit/refund behavior. Nothing needed to invoke it correctly or interpret its result is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100% and the taskset enum is self-documenting, so the schema already carries the parameter semantics. The description adds no parameter-level detail beyond what is in the schema, so baseline 3 applies.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource ('Start a new evaluation task (one episode) and get the case') and immediately distinguishes it in the sibling set with 'Call this first'. Enumerates what it returns (episode id, case instructions, answer format, tool budget), so an agent knows this is the entry point rather than a sibling like submit_answer or search_library.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
'Call this first' gives an explicit ordering constraint relative to the other episode tools, which is the key usage decision here. It does not state when-not to call it (e.g., reusing an existing episode), but the schema's task_id 'retry' note covers the re-entry path.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
submit_answerSubmit the agent's answer for gradingAInspect
Submit the final decision for the episode, as a JSON object in the format start_task gave (tariff cases: {"code": "8471300100"}, 10 digits, 8 accepted). One answer per episode; it ends the episode. Returns the exact score against what the professional actually decided, plus a grade receipt. The answer key itself is never shown.
| Name | Required | Description | Default |
|---|---|---|---|
| answer | Yes | The decision, in the answer format from start_task | |
| episode_id | Yes | The episode id returned by start_task, e.g. ep_3f9c2a1b0d4e5f6a7b8c9d0e |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Adds meaningful context beyond the annotations: only one answer is allowed per episode, the episode terminates on submission, and the response contains the exact score and a grade receipt. It also discloses that the answer key is never shown. Annotations already cover read-only and idempotency traits, so this is strong but not exhaustive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
Front-loaded with the action and format, followed by the terminal consequence and return behavior. Every sentence adds useful information: the format example, one-answer constraint, episode termination, return contents, and answer-key visibility.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a terminal submission tool with no output schema, the description explains the expected answer format, the one-shot nature of the call, the episode-ending consequence, and what the response will contain. Nothing critical for correct invocation appears missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100%, so episode_id is fully documented in the schema. The description adds useful meaning for the answer parameter by specifying the JSON object format, a tariff example, and digit constraints (10 digits, 8 accepted) that go beyond the schema's generic reference to start_task.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb and resource: submit the final decision for the episode. It distinguishes this terminal grading action from siblings like get_scorecard by noting that it ends the episode and returns the exact score plus a receipt.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Clearly establishes when to use it: submit the final decision once per episode, and doing so ends the episode. It references start_task for the required answer format, though it does not explicitly name alternatives or exclusions among siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
7 tool updates
v0.1.3- Changed
read_publication2 fields changed- added
Input schema / properties / episode_id / descriptionAdded value: +"The episode id returned by start_task, e.g. ep_3f9c2a1b0d4e5f6a7b8c9d0e" - added
Input schema / properties / publication / descriptionAdded value: +"Publication id from search_library results, e.g. US20190012345A1"
- Changed
read_rule2 fields changed- added
Input schema / properties / episode_id / descriptionAdded value: +"The episode id returned by start_task, e.g. ep_3f9c2a1b0d4e5f6a7b8c9d0e" - added
Input schema / properties / section / descriptionAdded value: +"Section number, e.g. 2143.01"
- Added
request_service - Changed
search_library2 fields changed- added
Input schema / properties / episode_id / descriptionAdded value: +"The episode id returned by start_task, e.g. ep_3f9c2a1b0d4e5f6a7b8c9d0e" - added
Input schema / properties / query / descriptionAdded value: +"Keywords or a short phrase describing the technology, e.g. 'battery cell cooling plate'"
- Changed
search_rules2 fields changed- added
Input schema / properties / episode_id / descriptionAdded value: +"The episode id returned by start_task, e.g. ep_3f9c2a1b0d4e5f6a7b8c9d0e" - added
Input schema / properties / query / descriptionAdded value: +"The rule or test you need, e.g. 'obviousness combining references'"
- Changed
start_task3 fields changed- changed
Input schema / properties / task_id / descriptionPrevious value: -"Optional specific task id"New value: +"A specific task id to retry; omit to get the next unserved case" - changed
Input schema / properties / taskset / descriptionPrevious value: -"Optional task set: patents-first-action (default), patents-response or patents-appeal"New value: +"Which kind of case to work. Default patents-first-action. tariffs-classification asks for the tariff code of a real product." - added
Input schema / properties / taskset / enumAdded value: +[ + "patents-first-action", + "patents-response", + "patents-appeal", + "tariffs-classification" +]
- Changed
submit_answer2 fields changed- added
Input schema / properties / answer / descriptionAdded value: +"The decision, in the answer format from start_task" - added
Input schema / properties / episode_id / descriptionAdded value: +"The episode id returned by start_task, e.g. ep_3f9c2a1b0d4e5f6a7b8c9d0e"
8 tool updates
v0.1.0- First observed
get_scorecard - First observed
get_usage - First observed
read_publication - First observed
read_rule - First observed
search_library - First observed
search_rules - First observed
start_task - First observed
submit_answer
TDQS
Scored across 9 tools
Each tool has a clearly distinct role in the evaluation workflow: start/submit bookend an episode, search/read pairs target either the reference library or the rulebook, and the remaining tools handle support and account metrics. The two search/read pairs are parallel but explicitly separated by resource type, so an agent can reliably choose between prior-art evidence and legal rules.
All tool names use a consistent snake_case verb_noun convention: start_task, search_library, read_publication, search_rules, read_rule, submit_answer, request_service, get_scorecard, get_usage. The verbs are predictable and accurately reflect each tool's action.
Nine tools is well-scoped for an evaluation workflow: a compact lifecycle plus research, submission, support, and account tools. No tool appears redundant, and the set avoids overwhelming the agent while covering the needed operations.
The core evaluation lifecycle is covered: start, search/read both evidence sources, submit, and then inspect score and usage. Minor gaps exist around active-episode management, such as re-fetching an open episode's instructions or listing active episodes, but these are likely workarounds rather than dead ends.
Maintenance
Related MCP Connectors
Governed decision engine: yes/no, score, choice and rank answers about cases, from past outcomes.
Typed AI decisions for agents: classify, route, screen evidence, score and batch-label text.
Practice using the AI you already use on real work from your week. Free first mission, scored.
Find people with real experience. Bring their approved workflows and decisions into your agent.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceInjects structured professional domain expertise into AI agents, providing access to decision frameworks, workflows, terminology, and regulatory context across 24 professional domains.56 npmMIT
- FlicenseNot gradedqualityBmaintenanceEnables agents to read and resolve layered feedback, propose and make decisions with evidence, and access trial ledgers and judgment views for experiment-driven work.-
- FlicenseNot gradedqualityCmaintenanceEnables AI agents to autonomously investigate simulated production incidents by checking live service health, logs, deployments, and database status while retrieving relevant historical postmortems and runbooks to propose evidence-backed root causes.-
- AlicenseNot gradedqualityCmaintenanceEnables agents to evaluate single records or batches with dynamically authored typed questions, returning structured decisions and probabilities.23 PyPIMIT