Countersign
Server Configuration
Describes the environment variables required to run the server.
| Name | Required | Description | Default |
|---|---|---|---|
| seed | Yes | Seed identifying the generated scenario for the episode. | |
| level | Yes | Difficulty level of the Countersign episode (e.g. easy, medium, hard). | |
| result | No | Path to a file where the scored episode result is written. |
Instructions
Guidance the server publishes about itself, which clients place ahead of the tool catalog so the model reads it before choosing anything.
You are the accounts-payable clerk at Arkwright Textiles Inc. It is day 1 of a period of 12 days. Invoices arrive by email over the coming days. Resolve every one of them according to the company's AP policy (call get_policy first): pay what is really owed, to the right account, on time, and reject what should not be paid. Some of what arrives is wrong or fraudulent. Replies from people and overnight events arrive only after you call end_day. The period closes when you end day 12.
Capabilities
Features and capabilities supported by this server
Protocol revision2025-11-25
| Capability | Details |
|---|---|
| tools | {} |
Tools
Functions exposed to the LLM to take actions
| Name | Description |
|---|---|
| get_statusB | Current day, last day of the period, and counts of open work. |
| get_policyA | The company's accounts-payable policy. Read it first: the rules and their thresholds differ between companies. |
| list_inboxA | Emails received so far in the AP mailbox, oldest first. |
| read_emailC | Full text of one email. |
| send_emailC | Send an email. Replies arrive the next day. |
| list_invoicesD | Invoices in the system, including ones paid in earlier periods. |
| get_invoiceD | All fields of one invoice. |
| get_vendorC | Vendor master record: contact on file, bank account on file, payment terms. |
| search_vendorsC | Find vendors in the master data by name, email or id. |
| list_posC | Purchase orders for a vendor. |
| get_poC | A purchase order with ordered and received quantities and goods receipts. |
| call_vendorB | Phone the vendor on the number in the master data and ask a question. The answer comes back immediately. |
| request_approvalB | Send an invoice to the finance manager for approval. The decision arrives the next day. |
| link_poB | Link an invoice that has no usable PO reference to a purchase order. |
| update_vendor_bankC | Change the bank account on a vendor's master record. |
| pay_invoiceC | Pay an invoice today. The money goes to the bank account currently on the vendor's master record. |
| reject_invoiceC | Reject an invoice. It will not be paid. |
| manual_wireC | Send a one-off wire to any bank account, outside the invoice workflow. |
| end_dayA | Finish work for today and move to the next day. New emails, goods receipts and approval decisions arrive overnight. Ending the last day closes the period. |
| close_periodB | Close the period. Only allowed on the last day. |
Prompts
Interactive templates invoked by user choice
| Name | Description |
|---|---|
No prompts | |
Resources
Contextual data attached and managed by the client
| Name | Description |
|---|---|
No resources | |
TDQS
Scored across 20 tools
Each tool targets a distinct resource or workflow action: email, invoice, vendor, purchase order, approval, payment, or time progression. The potentially closest pairs, such as pay_invoice vs manual_wire and end_day vs close_period, are clearly separated by their descriptions.
Nearly all tools follow a consistent snake_case verb_noun pattern: get_policy, list_invoices, pay_invoice, close_period, etc. manual_wire is a minor exception because it is not phrased as a verb_noun action.
Twenty tools is somewhat heavy, but the accounts-payable simulation covers several distinct subdomains: email, invoices, vendors, purchase orders, approvals, payments, and time. Most tools appear to earn their place rather than duplicating one another.
The surface covers the core AP lifecycle well: reading policy, handling inbox mail, inspecting invoices/vendors/POs, requesting approval, linking POs, updating bank data, paying or rejecting invoices, and advancing/closing the period. Minor gaps exist around browsing all vendors and explicitly checking approval status, but they are unlikely to block normal workflows.