Server Quality Checklist

Tool Scores

• download_invoiceB
  3.2/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  ANNOTATION CONTRADICTION: The annotations declare readOnlyHint=true, implying this is a safe read-only operation, but the description and title clearly indicate it downloads/writes a file to disk ('su disco', 'outputPath' parameter, 'Scarica fattura su file'). This is a write/mutation operation (creating files locally), contradicting the readOnlyHint annotation. The description itself is actually accurate, but it contradicts the structured annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence that is dense and informative, enumerating all four formats with parenthetical explanations. front-loads the purpose and format options. Slightly dense but efficient with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema exists, so the description should hint at return values—it does not. For a 6-parameter, 3-required download tool with a local-file side effect and a conditional format (pdd), the description handles format semantics well but omits confirmation/error behavior (e.g., what happens on success, whether it returns file info, behavior when file exists). The pdd availability condition is a good touch. The contradiction with readOnlyHint also undermines completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so all 6 parameters are documented in the schema. The description adds limited extra value: it explains the pdd format availability condition (pddAvailable=true, which is a datum not in the schema). However, it adds nothing about id vs idSdi distinction, filename usage, or how id/idSdi/filename interplay for identification—the schema already covers definitions adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear specific verb+resource: 'Scarica una fattura su disco nel formato scelto' with explicit format options (xml, pdf, zip, pdd) and their meanings. Distinguishes itself from sibling tools like search_invoices and get_invoice_detail by describing a file download operation with format choices. The pdd availability note adds specificity beyond siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains what each format represents (xml=original file, pdf=readable, zip=invoice+SDI notifications, pdd=conservation package), which helps the agent choose the right format. However, it doesn't explicitly state when to use this tool over search_invoices/get_invoice_detail (which could also retrieve data), nor when NOT to use it, and mentions the required direction parameter implicitly without guiding when to pick sent vs received.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_invoicesA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint=true, so safety profile is covered. The description adds valuable context about the Premium account/delegation requirement and the 401 error for base accounts, plus the 10-day maximum date window. This goes beyond the annotations to reveal real operational constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is a single reasonably concise sentence followed by a brief NOTE about the Premium requirement. It's front-loaded with the core purpose. Slightly verbose with the enumerated filter list but acceptable. The note is important and earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search/list tool with no output schema, it provides good context: filter capabilities, pagination, account requirements, and the 10-day window constraint. Given the tool operates on paginated results and has 14 params, the description covers the essential operational constraints well. Could mention result ordering or default behavior but is largely complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so all 14 parameters are documented in the schema itself. The description adds some context about senderCountry/senderVatcode being mandatory for Premium accounts (which the individual parameter descriptions also mention). The description doesn't add much beyond what the schema already conveys, so baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches Italian electronic invoices (sent/received) on Aruba Fatturazione Elettronica with filters for period, status, document type, and counterparty. It clearly distinguishes from siblings (search vs get/detail/download/send/notifications). However, it doesn't explicitly name sibling alternatives for search functions, so slightly below 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context about pagination and the Premium account requirement/delegation note (401 without delegations on base accounts). It states what it searches but doesn't explicitly say when to use it vs. siblings like get_invoice_detail. The 10-day window constraint on creationEndDate is helpful usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_user_infoA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The readOnlyHint annotation already declares this is a read operation. The description adds value by enumerating specific data returned (partita IVA, codice fiscale, service status, storage space used), which goes slightly beyond the annotation. It doesn't disclose response format or pagination, but for a 0-parameter info tool this is acceptable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, focused sentence that front-loads the purpose and enumerates the specific data fields returned. It's compact and information-dense with no filler. Could arguably add sibling differentiation but that's not a conciseness concern.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 0-parameter, read-only info tool with a readOnlyHint annotation, the description is largely complete — it states what the tool does and what data it returns. No output schema exists, so the field enumeration compensates somewhat for for the lack of structured return info, though exact return format remains unspecified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, so the description has no parameter burden to carry. With no params at all, the baseline of 4 applies — the description correctly focuses entirely on what the read operation returns rather than param syntax.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description specifies a clear verb+resource ('Restituisce le informazioni dell'account Aruba Fatturazione Elettronica') and enumerates exactly what fields are returned (partita IVA, codice fiscale, stato del servizio, spazio di conservazione). It's distinct from sibling tools which are all invoice-centric operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this is a read-only account-info retrieval tool distinct from the invoice operations of siblings. However, it doesn't explicitly state when to use this vs alternatives, though the account-info scope naturally differentiates it from invoice-specific tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_invoice_detailA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations provide readOnlyHint=true, so the safety profile is already covered. The description adds useful context that this is a metadata retrieval operation (distinct from download of binary files) and mentions what fields are returned (sender, recipient, status, SDI notifications). It doesn't describe error behavior for ambiguous identifiers (e.g., what happens if multiple identifiers given or none found), but with the read-only annotation and clear output framing, this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, zero waste. First sentence states the purpose and return contents; second sentence identifies the tool by an identifying parameter. Every phrase earns its place, and the alternative-tool pointer is compact. Excellent front-loaded structure.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-invoice metadata lookup with read-only annotation, 100% schema coverage, and no output schema, the description is quite complete. It covers purpose, identification method, and directs binary downloads to the appropriate sibling. The only minor gap is not discussing multi-identifier scenarios or error/not-found behavior, but these are edge cases for a metadata tool with such well-documented parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all four parameters including the direction enum semantics. The description adds value by clarifying that only ONE of id/filename/idSdi is needed (mutually exclusive choices), which is not explicit in the schema. However, since the schema already comprehensively documents each parameter, the description adds modest incremental value beyond structured coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states it returns complete metadata for a single invoice (sender, recipient, status, SDI notifications) with a specific verb (Restituisce) and resource (una singola fattura). It clearly distinguishes from the sibling download_invoice by explicitly mentioning XML/PDF/ZIP download goes there, and from search_invoices by focusing on a single invoice identified by one of three identifiers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: to retrieve metadata of a single invoice. It provides clear identification guidance (use id, filename, or idSdi) and explicitly directs users to download_invoice for XML/PDF/ZIP, giving an explicit alternative. This is excellent when-to-use and when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_invoice_notificationsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The readOnlyHint=true annotation already covers the safety profile as a read operation, so the bar is lower. The description adds that it works with base accounts and retrieves specific notification types (delivery receipt, rejection, non-delivery), but doesn't detail return format, pagination, or what happens when notifications don't exist yet for a fresh invoice. Adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with zero wasted words. The first states purpose with examples; the second gives a critical usage note tying it to send_invoice and clarifying it works on a basic account. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read tool with 2 required, fully-documented params and a readOnlyHint annotation, the description is reasonably complete. It covers purpose, usage context, and the source of the key parameter. It doesn't describe output shape since there's no output schema, but for a notification-status tool the description covers the essential agent decisions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters well. The description adds the note that invoiceFilename matches what send_invoice returns as uploadFileName, which is useful cross-tool context, but both parameters are already clearly described in the schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool returns SDI notifications for a single invoice by filename (delivery receipt, rejection, non-delivery), a specific verb+resource with clear scope. It also distinguishes itself from siblings by noting it works with basic accounts and that it's the way to track an invoice sent via API, tying directly to send_invoice's uploadFileName.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says when to use it: to track the status of an invoice sent via API, noting it works even with a basic account and referencing that the filename comes from send_invoice's uploadFileName return value. This provides concrete usage context and distinguishes it from sibling tools like search_invoices and get_invoice_detail.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• send_invoiceA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It does an excellent job: explicitly warns that with dryRun=false the invoice is REALLY sent to SDI, clarifies the default dryRun=true only validates, and instructs that the XML must declare Aruba as intermediary. It also covers signed-file behavior (CAdES/XAdES endpoint). Minor gap: doesn't state what the response looks like or what happens on validation failure, but given no annotations, the disclosure is strong.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, all dense with actionable information: purpose, critical dryRun warning, default behavior, prerequisite (IdTrasmittente), and input method recommendation. Zero filler. The warning is appropriately front-loaded right after the main purpose statement.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a high-stakes mutation tool (real invoice submission) with 5 parameters and no output schema and no annotations. The description covers the critical safe-mode behavior, prerequisite validation, and input selection. Minor gap: doesn't mention expected return/response format, validation error behavior, or post-submission tracking (e.g., how to get notification later via get_invoice_notifications sibling). A bit more on outcomes would make it complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so parameters are already documented in the schema. The description adds value by recommending xmlFilePath over xmlContent and noting senderPIVA is only needed for TD26/multi-azienda cases. This is extra context beyond schema. However, much of the important semantic detail (dryRun behavior, signed endpoint) lives in the schema, so the description's marginal contribution is moderate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool loads an XML e-invoice onto Aruba for submission to the Sistema di Interscambio (SDI). It has a specific verb (carica/invia), specific resource (fattura elettronica XML, Aruba, SDI), and differentiates from siblings like search_invoices, get_invoice_detail, and download_invoice which are clearly distinct read operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use guidance with the critical dryRun=true default safety warning, and contrasts with siblings which are read-only tools. It explains the real-send side effect requires dryRun=false, and gives clear rules about the XML needing Aruba as trasmittente (IdTrasmittente 01879020517). Also addresses which input method to use (xmlFilePath recommended over xmlContent).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.