Skip to main content
Glama
Renwang-Huang

TypeSafe MCP

Arbitype

Arbitype is an MCP-native typed decision layer for AI agents, powered by TypeSafe Jev. It turns probabilistic judgments into structured decision primitives that an agent or program can consume directly.

NOTE

Arbitype is an independent open-source project. It is not an official TypeSafe AI product or an official integration for any particular agent host.

Quick start

The shortest path is a local STDIO server launched by uvx:

export TYPESAFE_API_KEY="your-key"
uvx arbitype

The API key stays in the process environment. It is not an MCP argument and is never printed to standard output.

Install a pinned release with:

uvx --from 'arbitype==0.6.0' arbitype

Or run the repository checkout:

git clone https://github.com/Renwang-Huang/arbitype.git
cd arbitype
export TYPESAFE_API_KEY="your-key"
python3 server.py

Related MCP server: McpHub

Why Arbitype?

Generative models are excellent at prose, code, and open-ended generation. Agent workflows also need bounded decisions that software can branch on:

free-form state
      ↓
  TypeSafe Jev
      ↓
probabilistic judgment
      ↓
    Arbitype
      ↓
typed decision + probability
      ↓
agent / code branch

Arbitype provides that decision layer over MCP. It exposes short, host-neutral primitives for classification, scoring, verification, routing, review, and fail-closed gate signals. The result is structured data, not a paragraph that an agent must interpret again.

Architecture

flowchart LR
    host["AI Agent / MCP Host<br/>Codex · Claude · Cursor · VS Code"]
    arbitype["Arbitype<br/>Typed decision tools"]
    jev["TypeSafe Jev<br/>System One Model"]
    env["TYPESAFE_API_KEY<br/>process environment"]

    host -->|MCP| arbitype
    arbitype -->|validated HTTPS| jev
    jev -->|typed probabilistic judgment| arbitype
    arbitype -->|structured decision| host
    env -. credential .-> arbitype

The public product is Arbitype; TypeSafe Jev is the current provider. Provider configuration intentionally keeps the TYPESAFE_* names because the credential and endpoint belong to TypeSafe.

Tools

Arbitype advertises nine read-only, idempotent MCP tools:

Tool

Input shape

Output

evaluate

state + TypeSafe questions map

Raw typed Jev response

classify

state + instructions + labels

Choice and probability distribution

score

state + instructions + ordered levels

Weighted score and distribution

check

state + yes/no criteria

Noul probability

verify

state + claims map

Noul answer per claim

gate

state + checks + thresholds

pass, review, or fail signal

route

state + actions map

One suggested next action; no execution

review

state + checks + thresholds

Review decision and evidence

health

Optional live boolean

Local configuration; live request only when explicit

For raw Noul questions, provide non-empty instructions or at least one non-empty true/false criterion. Score levels must be non-null structured values. These provider-level constraints are validated locally before a paid request.

Probabilities and confidence are model signals, not proof. gate and review are advisory decision transformations, not authorization systems, security boundaries, or approval engines.

Host setup

Arbitype uses standard MCP STDIO. For a host that accepts an installed command:

[mcp_servers.arbitype]
command = "arbitype"
env_vars = ["TYPESAFE_API_KEY"]
startup_timeout_sec = 10
tool_timeout_sec = 60
default_tools_approval_mode = "prompt"

For a checkout:

[mcp_servers.arbitype]
command = "python3"
args = ["/absolute/path/to/arbitype/server.py"]
env_vars = ["TYPESAFE_API_KEY"]
startup_timeout_sec = 10
tool_timeout_sec = 60

The same process can be registered by Claude, Cursor, VS Code, Codex, or any other MCP host using its native configuration format. Keep the key out of host configuration files; use the host's environment forwarding mechanism.

CLI and Python

The canonical CLI and package are arbitype:

arbitype --version
arbitype doctor --json
cat request.json | arbitype evaluate
arbitype evaluate --input request.json

The Python API is intentionally small:

from arbitype import TypeSafeClient

client = TypeSafeClient()
result = client.evaluate({
    "state": "A payment failed twice.",
    "questions": {
        "urgent": {
            "type": "noul",
            "instructions": "Does this require urgent handling?",
        }
    },
})

Package and import compatibility

The canonical wheel contains the one implementation plus the legacy import shims. The old PyPI project is not deleted, yanked, or released under a new identity.

Surface

Name

Status

PyPI

arbitype

Canonical distribution

Python

arbitype

Canonical import

Python

typesafe_mcp

Legacy compatibility shim

Python

typesafe_codex_mcp

Legacy compatibility shim

CLI compatibility

Command

Status

arbitype

Canonical CLI

typesafe-mcp

Legacy CLI alias

typesafe-codex-mcp

Legacy CLI alias

Tool compatibility

Tool

Status

route

Canonical

review

Canonical

codex_route

Legacy alias

codex_review

Legacy alias

The historical typesafe-mcp PyPI project remains intact. A metadata-only replacement with the same distribution name was tested and rejected because pip can remove legacy console-script files while replacing the old distribution. Therefore no typesafe-mcp==0.6.0 migration package will be published.

Existing users should use this explicit, safe migration:

python -m pip uninstall typesafe-mcp
python -m pip install arbitype

New installations should use arbitype directly. This leaves one distribution owning the canonical implementation, compatibility shims, and all three CLI entry points.

Discovery and Registry

The canonical MCP Registry identity is:

io.github.Renwang-Huang/arbitype

The intended package entry is:

uvx arbitype

The PyPI, MCP Registry, and Glama links are prepared before publication, but their badges remain explicitly marked pending until the external listings are verified. Release ordering and the legacy Registry migration procedure are documented in docs/REGISTRY_MIGRATION.md.

The former identity io.github.Renwang-Huang/typesafe-mcp is a legacy identity. It must remain available for existing users and should be marked deprecated through the Registry publisher when that mutation is supported. New installations should use the Arbitype identity.

Configuration

Variable

Default

Purpose

TYPESAFE_API_KEY

Required TypeSafe bearer credential

TYPESAFE_BASE_URL

https://api.typesafe.ai

API base URL

TYPESAFE_MODEL

jev-latest

Model alias; legacy name supported

TYPESAFE_DEFAULT_MODEL

jev-latest

Official SDK-compatible model name

TYPESAFE_TIMEOUT_SECONDS

10

Per HTTP attempt timeout

TYPESAFE_MAX_RETRIES

2

Retries after the initial request

TYPESAFE_RETRY_BACKOFF_SECONDS

0.5

Initial exponential backoff

TYPESAFE_MAX_STATE_CHARS

120000

Serialized state limit

TYPESAFE_MAX_QUESTION_CHARS

60000

Serialized question limit

TYPESAFE_MAX_REQUEST_BYTES

512000

Whole request limit

TYPESAFE_MAX_RESPONSE_BYTES

4194304

Provider response limit

Custom provider endpoints must use HTTPS. Plain HTTP is accepted only for loopback hosts such as localhost, 127.0.0.1, and ::1. Redirects are disabled so a bearer credential is never forwarded to a redirect target.

Security boundaries

Arbitype is a local MCP adapter and typed decision layer. It is not:

  • a sandbox for untrusted code;

  • an authorization or identity system;

  • a prompt-injection firewall;

  • a security approval boundary; or

  • an official TypeSafe AI product.

It does not execute actions suggested by route, edit files, run shell commands, or treat model probabilities as proof. Read SECURITY.md before using live credentials.

Development

python3 -m unittest discover -s tests -v
python3 -m compileall -q .
python3 -m pip wheel --no-deps . --wheel-dir /tmp/arbitype-dist

The test suite uses local fakes and does not need an API key. The official MCP Python SDK interoperability smoke test is in scripts/official_sdk_smoke.py. A live Jev check is opt-in and paid:

TYPESAFE_API_KEY="your-key" arbitype doctor --live

See TESTING.md, CONTRIBUTING.md, and BENCHMARK.md for the full engineering checks and comparison. Maintainer migration details are in docs/REGISTRY_MIGRATION.md.

Release identity

Arbitype is currently released as 0.6.0 because it remains Beta while its canonical public identity moves to the new package, CLI, and Registry name. The old package history remains intact; the brand migration does not rewrite Git history or delete the former PyPI project.

Available Tools

9 tools
checkC
Read-onlyIdempotent

Estimate the probability that a bounded yes/no proposition is true.

ParametersJSON Schema
NameRequiredDescriptionDefault
modelNo
stateYes
instructionsYes
true_criteriaNo
false_criteriaNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
typeYes
modelYes
usageYes
answerYes
evaluationYes

TDQS

C2.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish this as read-only, idempotent, open-world, and non-destructive, so the description's safety burden is low. The description does add useful behavioral context by indicating the output is a probability estimate for a bounded proposition, but it does not explain edge cases, assumptions, or how 'bounded' is enforced.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no filler. Every word contributes to the core meaning, and it is appropriately sized for a tool whose safety profile is already captured by annotations.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite rich annotations and an output schema, the description is incomplete for a five-parameter tool with zero schema description coverage. It gives no guidance on required inputs like state and instructions or optional criteria like true_criteria and false_criteria, so an agent cannot confidently invoke it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate for the five undocumented parameters (model, state, instructions, true_criteria, false_criteria). It does not: no parameter is mentioned, explained, or mapped to the 'bounded yes/no proposition' concept, leaving an agent to guess how to construct a valid call.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific verb ('estimate') and a specific resource ('probability that a bounded yes/no proposition is true'), making the core function clear. It does not explicitly differentiate from siblings like evaluate or score, which could also produce probabilistic outputs, so it falls short of a 5.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance about when to use this tool versus the sibling tools (evaluate, classify, verify, score, etc.). The phrase 'bounded yes/no proposition' implies a constraint, but the description never states when this tool is preferred or what alternatives exist.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

classifyC
Read-onlyIdempotent

Choose one label from a closed set and return its probability distribution.

ParametersJSON Schema
NameRequiredDescriptionDefault
modelNo
stateYes
labelsYes
instructionsYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
typeYes
modelYes
usageYes
answerYes
evaluationYes

TDQS

C2.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover the safety profile (readOnly, idempotent, non-destructive), so the bar for additional disclosure is lower. The description adds useful behavioral context by mentioning the closed set and probability distribution output, but doesn't address edge cases like ambiguous labels or handling of unexpected inputs.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single compact sentence with no filler, front-loading the action and output. It is easy to parse, though the brevity sacrifices parameter-level detail.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with nested objects, four parameters, zero schema description coverage, and eight siblings, one sentence is insufficient. The output schema may document return values, but the description lacks input semantics, usage context, and guidance on when this tool is the right choice.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description needed to compensate by explaining state, instructions, labels, and model. It only hints at the labels parameter via 'closed set' and leaves the other parameters essentially unexplained, forcing the agent to rely on names alone.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific operation: choose one label from a closed set and return its probability distribution. This clearly conveys a classification behavior and distinguishes it from generic check/verify tools, though it doesn't explicitly name a sibling alternative.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no guidance on when to use classify versus evaluate, score, verify, or other siblings. The phrasing implies a classification task, but no exclusions, alternatives, or selection criteria are provided.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

evaluateA
Read-onlyIdempotent

Evaluate state with TypeSafe Jev and return the raw typed answers. Use for bounded noul, choice, or score questions; never for prose, code generation, arithmetic, or dates.

ParametersJSON Schema
NameRequiredDescriptionDefault
modelNoOptional model id or alias.
stateYesText or structured evidence.
questionsYesMap of question id to a TypeSafe question.

Output Schema

ParametersJSON Schema
NameRequiredDescription
modelYes
usageYes
answersYes

TDQS

A4/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, openWorldHint=true, idempotentHint=true, and destructiveHint=false, so the safety profile is fully covered. The description adds useful context about returning raw typed answers and the bounded scope of questions, but does not disclose any additional behavioral traits such as error handling, state size limits, or model selection effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: the first states the core purpose and output, the second states the applicable scope and exclusions. There is no redundancy, and the most important information is front-loaded.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the rich input schema, 100% schema description coverage, output schema, and safety annotations, the description provides enough orientation for correct invocation. It could be slightly more complete by noting when the optional model parameter matters, but that is already covered in the schema, so the remaining gap is minor.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents state, questions, and model thoroughly. The description reinforces that questions must be bounded noul/choice/score and that state can be text or structured evidence, but it does not materially add beyond what the schema's property descriptions already specify.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific action ('Evaluate state with TypeSafe Jev'), a defined resource type (state), and a concrete output ('return the raw typed answers'). It also narrows the applicable question types to bounded noul, choice, or score, and excludes prose, code generation, arithmetic, and dates, which makes its purpose distinct among the sibling tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit when-to-use guidance ('Use for bounded noul, choice, or score questions') and explicit when-not-to-use guidance ('never for prose, code generation, arithmetic, or dates'). However, it does not name alternative sibling tools, so the agent must infer which sibling handles those excluded cases.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

gateA
Read-onlyIdempotent

Evaluate bounded pass checks and turn their probabilities into pass/review/fail. This is not an authorization or security boundary; keep normal human and policy controls.

ParametersJSON Schema
NameRequiredDescriptionDefault
modelNo
stateYes
checksYes
pass_atNo
review_atNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
typeYes
checksYes
policyYes
decisionYes
evaluationYes

TDQS

A3.7/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false deminishing the need for safety disclosure. The description adds valuable behavioral context by clarifying that this is not a security control and that human/policy controls should remain in place. It does not contradict the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, front-loaded with the core purpose and immediately clarifying the most important caveat. There is no filler, repetition, or redundant information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

An output schema exists, so return-value documentation is less critical. However, the description does not explain how the five parameters interact, what makes a check 'bounded', or how pass_at and review_at thresholds map to the pass/review/fail outcome. The core concept is present but the operational details are incomplete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate by explaining the parameters. It does not explain what 'state', 'checks', 'pass_at', or 'review_at' mean beyond the generic phrase 'bounded pass checks'. The threshold parameters especially need semantic explanation, which is missing.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific action ('Evaluate bounded pass checks') and a concrete output transformation ('turn their probabilities into pass/review/fail'). It also distinguishes itself from an authorization or security boundary, though it does not explicitly differentiate from the sibling tools such as evaluate, check, or verify.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

It gives an explicit when-not-to-use signal: 'This is not an authorization or security boundary; keep normal human and policy controls.' This provides clear context about appropriate non-security usage, but it does not name specific alternative tools or broader when-to-use scenarios.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

healthA
Read-onlyIdempotent

Inspect local MCP configuration without making a network request. Set live=true only for an explicit paid provider check.

ParametersJSON Schema
NameRequiredDescriptionDefault
liveNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
liveYes
typeYes
modelNo
serverYes
statusNo
versionYes
base_urlNo
max_retriesNo
timeout_secondsNo
api_key_configuredYes

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover readOnly, idempotent, and non-destructive behavior. The description adds meaningful context beyond those annotations: it clarifies the tool inspects local configuration without network traffic and flags that live=true is tied to a paid provider check.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, each earning its place: the first states the core behavior, the second provides the only conditional parameter guidance. No filler or redundant restatement of the schema.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has one optional parameter, a rich annotation set (readOnly, idempotent, non-destructive, openWorld), and an output schema. The description fully covers when and how to invoke it, including the paid-provider edge case, so nothing critical is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description carries the burden for parameter meaning. It explains that live defaults to a local check and should only be set to true for an explicit paid provider check, which is essential semantic information beyond the raw boolean schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource: 'Inspect local MCP configuration'. It also adds the distinctive behavior 'without making a network request', which separates it from the generic sibling names like check, verify, and evaluate.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear context for the default mode (local, no network) and an explicit conditional for live=true: 'only for an explicit paid provider check'. It does not name sibling tools or exclusions, but for this simple one-parameter tool the usage guidance is sufficient.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

reviewB
Read-onlyIdempotent

Evaluate a diff, plan, or test report against checks and return pass/review/fail signals. It never edits files.

ParametersJSON Schema
NameRequiredDescriptionDefault
modelNo
stateYes
checksYes
pass_atNo
review_atNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
typeYes
checksYes
policyYes
decisionYes
evaluationYes

TDQS

B3.2/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint false; the description's 'never edits files' reinforces but does not substantially extend them. It adds the input scope and output signal names, but no details about auth, side effects, or failure modes.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences front-load the core purpose and the critical safety guarantee. Every clause contributes useful information with no filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The output schema exists, so return-value details are not required, but the tool has five parameters including nested objects and thresholds with defaults. The description provides no guidance on structuring 'state' or 'checks' or interpreting 'pass_at' and 'review_at', making it incomplete for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

With 0% schema description coverage, the description must carry parameter meaning, but it only glosses 'against checks' and implies that 'state' is a diff/plan/test report. The 'model', 'pass_at', and 'review_at' parameters and their meanings are left completely unexplained.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a clear verb ('Evaluate'), specific inputs ('a diff, plan, or test report'), and distinct outputs ('pass/review/fail signals'). It is unambiguous, though it does not explicitly distinguish itself from sibling evaluator-style tools like 'evaluate', 'check', or 'verify'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use it: when a diff, plan, or test report needs to be checked against evaluation criteria. It provides no explicit alternatives, when-not-to-use guidance, or comparison with the sibling tools, so the agent must infer routing.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

routeC
Read-onlyIdempotent

Choose the next action from a closed set. This suggests an action; it does not execute it.

ParametersJSON Schema
NameRequiredDescriptionDefault
modelNo
stateYes
actionsYes
instructionsNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
typeYes
modelYes
routeYes
usageYes
answerYes
evaluationYes

TDQS

C2.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint, idempotentHint, and destructiveHint=false, so the safety profile is covered. The description adds a meaningful behavioral clarification by emphasizing the tool only suggests and does not execute, but it does not go deeper into how state or actions are processed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences, front-loaded with the purpose and a key behavioral caveat. There is no fluff, though the extreme brevity contributes to the lack of parameter guidance.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With four parameters, zero schema descriptions, and no guidance on state/action/instructions shape, the description is incomplete for reliable invocation. The output schema may help with returns, but the core input semantics are left undocumented.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the description provides no meaning for the four parameters, especially required state and actions. The phrase 'closed set' hints that actions is a finite set, but this is far too thin to help an agent construct valid inputs.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Choose') and resource ('next action from a closed set'), making the tool's core function clear. It also distinguishes itself from executers by stating it 'suggests an action; it does not execute it,' though it does not explicitly differentiate from sibling decision tools like evaluate or classify.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use the tool: when a next action must be selected from a closed set. It also provides one exclusion ('does not execute it'), but it does not state when to prefer route over sibling tools or describe conditions where routing is inappropriate.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

scoreB
Read-onlyIdempotent

Rate state on an ordered rubric and return the weighted score, probabilities, and confidence.

ParametersJSON Schema
NameRequiredDescriptionDefault
modelNo
stateYes
levelsYes
instructionsYes

Output Schema

ParametersJSON Schema
NameRequiredDescription
typeYes
modelYes
usageYes
answerYes
evaluationYes

TDQS

B3.3/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare readOnlyHint=true, idempotentHint=true, and destructiveHint=false. The description adds that the tool returns a weighted score, probabilities, and confidence, which is useful beyond the annotations, but it does not explain the semantics of weighting or how probabilities relate to rubric levels. No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single dense sentence with no filler. Every phrase contributes: the action, the rating basis, and the return contents. It is front-loaded with the core purpose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Annotations and the presence of an output schema cover safety and return shape, but the tool has four parameters with zero schema descriptions. The description still leaves unclear what 'instructions' is for and how rubric levels should be represented, making the definition adequate but not complete for invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description must compensate. It only indirectly hints at 'state' and 'levels' via 'ordered rubric', and it never explains 'instructions', 'model', how levels should be structured, or what weights are used. This leaves most parameters under-specified.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Rate') and identifies the resource ('state') and method ('ordered rubric'), and it names concrete outputs: weighted score, probabilities, and confidence. It is clear enough to distinguish from generic health/check tools, though it does not explicitly contrast with overlapping siblings like evaluate or classify.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'Rate state on an ordered rubric' implies the tool is for rubric-based scoring, but the description gives no explicit when-to-use guidance, excluded cases, or alternatives. With siblings like evaluate, classify, verify, and gate present, more routing guidance would help.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

verifyB
Read-onlyIdempotent

Check several claims against the supplied evidence in one request. Results are review signals, not proof of truth.

ParametersJSON Schema
NameRequiredDescriptionDefault
modelNo
stateYes
claimsYes
true_criteriaNo
false_criteriaNo

Output Schema

ParametersJSON Schema
NameRequiredDescription
typeYes
answersYes
evaluationYes

TDQS

B3.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already cover read-only, open-world, idempotent, and non-destructive traits. The description adds meaningful behavioral context beyond those annotations by clarifying that results are 'review signals, not proof of truth,' which sets correct expectations about the epistemic weight of the output. This is useful context that the annotations do not provide.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two short sentences with no filler. The core action and primary caveat are front-loaded, and every phrase adds value. This is an example of efficient, well-structured prose.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Despite rich annotations and an output schema, the tool has five parameters including nested objects, a 64-claim limit, and ambiguous criteria fields. The description does not provide enough guidance for correct invocation, especially around parameter semantics and how the criteria are interpreted. Complex tools need more explanatory support than this brief description offers.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters2/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, so the description carries the burden of explaining parameters, but it only loosely maps 'claims' and 'evidence' to the schema. The required 'state' and 'claims' parameters are not explained in detail, and ambiguous optional parameters like 'true_criteria' and 'false_criteria' are entirely undocumented in both the schema and description.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action ('Check several claims') against a specific resource ('supplied evidence') and notes the batch nature ('in one request'). It clearly conveys what the tool does, though it does not explicitly differentiate itself from the sibling 'check' tool or other nearby tools like 'evaluate'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'in one request' implies a batching use case, and the mention of claims versus evidence suggests when it applies. However, there is no explicit guidance about when to choose this tool over the siblings (evaluate, classify, check, etc.) or when not to use it.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 6 tool updatesv0.6.0
    • Changedclassify2 fields changed
      • addedInput schema / properties / labels / maxProperties
        Added value: +255
      • addedInput schema / properties / labels / minProperties
        Added value: +1
    • Changedevaluate3 fields changed
      • changedInput schema / properties / questions / additionalProperties / oneOf
        Previous value: -[
        -  {
        -    "additionalProperties": false,
        -    "properties": {
        -      "criteria": {
        -        "additionalProperties": false,
        -        "properties": {
        -          "false": {
        -            "anyOf": [
        -              {
        -                "type": "string"
        -              },
        -              {
        -                "type": "object"
        -              },
        -              {
        -                "type": "array"
        -              },
        -              {
        -                "type": "null"
        -              }
        -            ]
        -          },
        -          "true": {
        -            "anyOf": [
        -              {
        -                "type": "string"
        -              },
        -              {
        -                "type": "object"
        -              },
        -              {
        -                "type": "array"
        -              },
        -              {
        -                "type": "null"
        -              }
        -            ]
        -          }
        -        },
        -        "type": "object"
        -      },
        -      "instructions": {
        -        "anyOf": [
        -          {
        -            "type": "string"
        -          },
        -          {
        -            "type": "object"
        -          },
        -          {
        -            "type": "array"
        -          },
        -          {
        -            "type": "null"
        -          }
        -        ]
        -      },
        -      "type": {
        -        "const": "noul"
        -      }
        -    },
        -    "required": [
        -      "type",
        -      "instructions"
        -    ],
        -    "type": "object"
        -  },
        -  {
        -    "additionalProperties": false,
        -    "properties": {
        -      "criteria": {
        -        "additionalProperties": {
        -          "anyOf": [
        -            {
        -              "type": "string"
        -            },
        -            {
        -              "type": "object"
        -            },
        -            {
        -              "type": "array"
        -            },
        -            {
        -              "type": "null"
        -            }
        -          ]
        -        },
        -        "maxProperties": 255,
        -        "minProperties": 1,
        -        "type": "object"
        -      },
        -      "instructions": {
        -        "anyOf": [
        -          {
        -            "type": "string"
        -          },
        -          {
        -            "type": "object"
        -          },
        -          {
        -            "type": "array"
        -          },
        -          {
        -            "type": "null"
        -          }
        -        ]
        -      },
        -      "type": {
        -        "const": "choice"
        -      }
        -    },
        -    "required": [
        -      "type",
        -      "instructions",
        -      "criteria"
        -    ],
        -    "type": "object"
        -  },
        -  {
        -    "additionalProperties": false,
        -    "properties": {
        -      "criteria": {
        -        "items": {
        -          "anyOf": [
        -            {
        -              "type": "string"
        -            },
        -            {
        -              "type": "object"
        -            },
        -            {
        -              "type": "array"
        -            },
        -            {
        -              "type": "null"
        -            }
        -          ]
        -        },
        -        "maxItems": 10,
        -        "minItems": 2,
        -        "type": "array"
        -      },
        -      "instructions": {
        -        "anyOf": [
        -          {
        -            "type": "string"
        -          },
        -          {
        -            "type": "object"
        -          },
        -          {
        -            "type": "array"
        -          },
        -          {
        -            "type": "null"
        -          }
        -        ]
        -      },
        -      "type": {
        -        "const": "score"
        -      }
        -    },
        -    "required": [
        -      "type",
        -      "instructions",
        -      "criteria"
        -    ],
        -    "type": "object"
        -  }
        -]New value: +[
        +  {
        +    "additionalProperties": false,
        +    "properties": {
        +      "criteria": {
        +        "additionalProperties": false,
        +        "properties": {
        +          "false": {
        +            "anyOf": [
        +              {
        +                "type": "string"
        +              },
        +              {
        +                "type": "object"
        +              },
        +              {
        +                "type": "array"
        +              },
        +              {
        +                "type": "null"
        +              }
        +            ]
        +          },
        +          "true": {
        +            "anyOf": [
        +              {
        +                "type": "string"
        +              },
        +              {
        +                "type": "object"
        +              },
        +              {
        +                "type": "array"
        +              },
        +              {
        +                "type": "null"
        +              }
        +            ]
        +          }
        +        },
        +        "type": "object"
        +      },
        +      "instructions": {
        +        "anyOf": [
        +          {
        +            "type": "string"
        +          },
        +          {
        +            "type": "object"
        +          },
        +          {
        +            "type": "array"
        +          },
        +          {
        +            "type": "null"
        +          }
        +        ]
        +      },
        +      "type": {
        +        "const": "noul"
        +      }
        +    },
        +    "required": [
        +      "type",
        +      "instructions"
        +    ],
        +    "type": "object"
        +  },
        +  {
        +    "additionalProperties": false,
        +    "properties": {
        +      "criteria": {
        +        "additionalProperties": {
        +          "anyOf": [
        +            {
        +              "type": "string"
        +            },
        +            {
        +              "type": "object"
        +            },
        +            {
        +              "type": "array"
        +            },
        +            {
        +              "type": "null"
        +            }
        +          ]
        +        },
        +        "maxProperties": 255,
        +        "minProperties": 1,
        +        "type": "object"
        +      },
        +      "instructions": {
        +        "anyOf": [
        +          {
        +            "type": "string"
        +          },
        +          {
        +            "type": "object"
        +          },
        +          {
        +            "type": "array"
        +          },
        +          {
        +            "type": "null"
        +          }
        +        ]
        +      },
        +      "type": {
        +        "const": "choice"
        +      }
        +    },
        +    "required": [
        +      "type",
        +      "instructions",
        +      "criteria"
        +    ],
        +    "type": "object"
        +  },
        +  {
        +    "additionalProperties": false,
        +    "properties": {
        +      "criteria": {
        +        "items": {
        +          "anyOf": [
        +            {
        +              "type": "string"
        +            },
        +            {
        +              "type": "object"
        +            },
        +            {
        +              "type": "array"
        +            }
        +          ]
        +        },
        +        "maxItems": 10,
        +        "minItems": 2,
        +        "type": "array"
        +      },
        +      "instructions": {
        +        "anyOf": [
        +          {
        +            "type": "string"
        +          },
        +          {
        +            "type": "object"
        +          },
        +          {
        +            "type": "array"
        +          },
        +          {
        +            "type": "null"
        +          }
        +        ]
        +      },
        +      "type": {
        +        "const": "score"
        +      }
        +    },
        +    "required": [
        +      "type",
        +      "instructions",
        +      "criteria"
        +    ],
        +    "type": "object"
        +  }
        +]
      • addedInput schema / properties / questions / maxProperties
        Added value: +64
      • addedInput schema / properties / questions / minProperties
        Added value: +1
    • Changedgate2 fields changed
      • addedInput schema / properties / checks / maxProperties
        Added value: +64
      • addedInput schema / properties / checks / minProperties
        Added value: +1
    • Changedreview1 field changed
      • addedInput schema / properties / checks / maxProperties
        Added value: +64
    • Changedscore1 field changed
      • changedInput schema / properties / levels / items / anyOf
        Previous value: -[
        -  {
        -    "type": "string"
        -  },
        -  {
        -    "type": "object"
        -  },
        -  {
        -    "type": "array"
        -  },
        -  {
        -    "type": "null"
        -  }
        -]New value: +[
        +  {
        +    "type": "string"
        +  },
        +  {
        +    "type": "object"
        +  },
        +  {
        +    "type": "array"
        +  }
        +]
    • Changedverify2 fields changed
      • addedInput schema / properties / claims / maxProperties
        Added value: +64
      • addedInput schema / properties / claims / minProperties
        Added value: +1
  2. 9 tool updatesv0.5.0
    • First observedcheck
    • First observedclassify
    • First observedevaluate
    • First observedgate
    • First observedhealth
    • First observedreview
    • First observedroute
    • First observedscore
    • First observedverify

TDQS

B3.3/5.0

Scored across 9 tools

Disambiguation2/5

Several tools occupy overlapping evaluative territory: evaluate, classify, score, and check all return probabilities or typed answers for bounded questions, and gate/review both produce pass/review/fail signals. The descriptions help but do not make the boundaries crisp enough for reliable tool selection.

Naming Consistency4/5

Tool names are almost uniformly single-word lowercase verbs (evaluate, classify, check, route, score, verify, gate, review), which is a consistent style. The noun 'health' is the one clear deviation from the verb pattern.

Tool Count4/5

Nine tools is within a reasonable scope for a server, and each tool has a plausible purpose. The count feels slightly higher than necessary because some tools could likely be consolidated without losing functionality.

Completeness4/5

The set covers a broad range of bounded evaluative tasks: raw typed answers, classification, probability checks, scoring, routing, verification, gating, and review. The main weakness is that the relationships between the overlapping evaluative tools are not structurally explicit, but no obvious critical operation is missing for the stated domain.

Maintenance

ActivityNo data
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • A
    license
    Not graded
    quality
    C
    maintenance
    Enables any Model Context Protocol-compatible host (Claude Desktop, Cherry Studio, Cline, MCP Inspector) to call 8 plug-and-play tools over stdio: hotspot ranking, TC3 signing, text chunking, date calculation, read-only SQLite queries, webpage text extraction, JWT decode/verify, and health checks.
    23 npm
    MIT
  • A
    license
    Not graded
    quality
    A
    maintenance
    Enables AI tools to uniformly discover, inspect, and call tools, prompts, and resources from multiple upstream MCP servers through a small set of fixed MCP tools, over stdio or HTTP.
    2
    MIT
  • A
    license
    Not graded
    quality
    B
    maintenance
    Enables natural-language interaction with TypeSafe's Jev decision API, supporting mixed question calls, batch evaluation, model listing, and confidence or composite-score gates over stdio.
    3
    MIT
  • A
    license
    B
    quality
    C
    maintenance
    Enables AI agents to obtain typed judgments from TypeSafe's Jev System One models, including yes/no probabilities, multiple-choice selections with distributions, and rubric-based scores, directly usable in code.
    5
    1
    AGPL 3.0